Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13053

allow refcat selection improve fitting

    XMLWordPrintable

Details

    • 4
    • AP S18-6, AP F18-1, AP F18-2, AP F18-3, AP S19-2, AP S19-3
    • Alert Production

    Description

      boutigny's work on selecting reference sources by S/N and magnitude might help improve the refcat that jointcal uses. But price made a ReferenceSourceSelectorTask that we can configure for this purpose, instead of having code internal to jointcal.

      This ticket is to convert boutigny's work in the u/fix_outliers branch (specifically this commit: https://github.com/lsst/jointcal/commit/e8393ac2e21117c15458156423445b19f75b503a ) to use ReferenceSourceSelectorTask, add a unittest to demonstrate that it does affect the fit, and update the other tests to reflect any changes in the fit metrics.

      Attachments

        Issue Links

          Activity

            Parejkoj John Parejko added a comment -

            Yes, just one sourceSelector for now. I have a ticket for changing that (DM-12990), but it's rather difficult, because source selection happens while the data is being loaded and before the ccdImages are created. I'm putting it off until it turns out that we actually need it, and/or we've sorted out some of the memory pressure questions of working with tracts that have a lot of visits. Jointcal gets a pretty marked speedup because it only has to read the catalogs and metadata once for both astrometry and photometry. The refcat loading is a tiny fraction of that time, so splitting that was easy.

            Parejkoj John Parejko added a comment - Yes, just one sourceSelector for now. I have a ticket for changing that ( DM-12990 ), but it's rather difficult, because source selection happens while the data is being loaded and before the ccdImages are created. I'm putting it off until it turns out that we actually need it, and/or we've sorted out some of the memory pressure questions of working with tracts that have a lot of visits. Jointcal gets a pretty marked speedup because it only has to read the catalogs and metadata once for both astrometry and photometry. The refcat loading is a tiny fraction of that time, so splitting that was easy.

            Yes, I would like two separate referenceSelection subtasks so we can apply two different cuts for photometry and astrometry. 

            yusra Yusra AlSayyad added a comment - Yes, I would like two separate referenceSelection subtasks so we can apply two different cuts for photometry and astrometry. 
            Parejkoj John Parejko added a comment -

            yusra: can you review the update that splits it into two selectors? Two new commits: one to cleanup numpydoc strings, the other to do the splitting.

            New Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29470/pipeline

            Parejkoj John Parejko added a comment - yusra : can you review the update that splits it into two selectors? Two new commits: one to cleanup numpydoc strings, the other to do the splitting. New Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29470/pipeline

            Looks good. 

            yusra Yusra AlSayyad added a comment - Looks good. 
            Parejkoj John Parejko added a comment -

            Thanks for the reviews. I'll leave it to others to see how much this helps the DC2 fitting.

            Merged and done.

            Parejkoj John Parejko added a comment - Thanks for the reviews. I'll leave it to others to see how much this helps the DC2 fitting. Merged and done.

            People

              Parejkoj John Parejko
              Parejkoj John Parejko
              Yusra AlSayyad
              Dominique Boutigny, Eli Rykoff, Jim Bosch, John Parejko, John Swinbank, Pierre Astier, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.