Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13053

allow refcat selection improve fitting

    XMLWordPrintable

    Details

    • Story Points:
      4
    • Epic Link:
    • Sprint:
      AP S18-6, AP F18-1, AP F18-2, AP F18-3, AP S19-2, AP S19-3
    • Team:
      Alert Production

      Description

      Dominique Boutigny's work on selecting reference sources by S/N and magnitude might help improve the refcat that jointcal uses. But Paul Price made a ReferenceSourceSelectorTask that we can configure for this purpose, instead of having code internal to jointcal.

      This ticket is to convert Dominique Boutigny's work in the u/fix_outliers branch (specifically this commit: https://github.com/lsst/jointcal/commit/e8393ac2e21117c15458156423445b19f75b503a ) to use ReferenceSourceSelectorTask, add a unittest to demonstrate that it does affect the fit, and update the other tests to reflect any changes in the fit metrics.

        Attachments

          Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-16226 jointcal constrained photometry model is not better than processCcd photometry on DC2 run1.2

          • Won't Fix

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-13054 Add colorterm config support to jointcal

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-17917 Add doUnresolved to ReferenceSourceSelector

          • Done

            Activity

            Ascending order - Click to sort in descending order
            11 older comments
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Yes, just one sourceSelector for now. I have a ticket for changing that (DM-12990), but it's rather difficult, because source selection happens while the data is being loaded and before the ccdImages are created. I'm putting it off until it turns out that we actually need it, and/or we've sorted out some of the memory pressure questions of working with tracts that have a lot of visits. Jointcal gets a pretty marked speedup because it only has to read the catalogs and metadata once for both astrometry and photometry. The refcat loading is a tiny fraction of that time, so splitting that was easy.

            Show
            Parejkoj John Parejko added a comment - Yes, just one sourceSelector for now. I have a ticket for changing that ( DM-12990 ), but it's rather difficult, because source selection happens while the data is being loaded and before the ccdImages are created. I'm putting it off until it turns out that we actually need it, and/or we've sorted out some of the memory pressure questions of working with tracts that have a lot of visits. Jointcal gets a pretty marked speedup because it only has to read the catalogs and metadata once for both astrometry and photometry. The refcat loading is a tiny fraction of that time, so splitting that was easy.
            Hide
            Permalink
            yusra Yusra AlSayyad added a comment -

            Yes, I would like two separate referenceSelection subtasks so we can apply two different cuts for photometry and astrometry. 

            Show
            yusra Yusra AlSayyad added a comment - Yes, I would like two separate referenceSelection subtasks so we can apply two different cuts for photometry and astrometry. 
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Yusra AlSayyad: can you review the update that splits it into two selectors? Two new commits: one to cleanup numpydoc strings, the other to do the splitting.

            New Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29470/pipeline

            Show
            Parejkoj John Parejko added a comment - Yusra AlSayyad : can you review the update that splits it into two selectors? Two new commits: one to cleanup numpydoc strings, the other to do the splitting. New Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29470/pipeline
            Hide
            Permalink
            yusra Yusra AlSayyad added a comment -

            Looks good. 

            Show
            yusra Yusra AlSayyad added a comment - Looks good. 
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Thanks for the reviews. I'll leave it to others to see how much this helps the DC2 fitting.

            Merged and done.

            Show
            Parejkoj John Parejko added a comment - Thanks for the reviews. I'll leave it to others to see how much this helps the DC2 fitting. Merged and done.

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              Yusra AlSayyad
              Watchers:
              Dominique Boutigny, Eli Rykoff, Jim Bosch, John Parejko, John Swinbank, Pierre Astier, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.