Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16596

Quality assessment of coadds created using jointcal calibrations

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: jointcal, meas_mosaic
    • Labels:
      None
    • Story Points:
      16
    • Epic Link:
    • Team:
      Data Release Production

      Description

      This issue is to account for and detail investigations into the quality of the coadds that were built using the calibrations from jointcal. They will be assessed independently and in comparison to the associated coadds that were built using meas_mosaic calibrations.

        Attachments

          Issue Links

            Activity

            Hide
            yusra Yusra AlSayyad added a comment -

            The post DM-16598 rerun is in: https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/w_2018_48_jointcal_noScaleVar_redo/ --> /datasets/hsc/repo/rerun/private/yusra/RC2/w_2018_48_jointcal_noScaleVar_redo/

            Yes, we can reassess after colorterms have been applied. On DM-16598, I just have to update pipe_tasks, run jenkins and that ticket should be good to merge.

            Show
            yusra Yusra AlSayyad added a comment - The post DM-16598 rerun is in: https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/w_2018_48_jointcal_noScaleVar_redo/ --> /datasets/hsc/repo/rerun/private/yusra/RC2/w_2018_48_jointcal_noScaleVar_redo/ Yes, we can reassess after colorterms have been applied. On DM-16598 , I just have to update pipe_tasks, run jenkins and that ticket should be good to merge.
            Hide
            lauren Lauren MacArthur added a comment -

            Let me know if you think the work to date on this ticket covers its scope and can be closed out.  If so, I would expect we would add further similar tickets following decisions and outcomes based on my recommendations here.

            Show
            lauren Lauren MacArthur added a comment - Let me know if you think the work to date on this ticket covers its scope and can be closed out.  If so, I would expect we would add further similar tickets following decisions and outcomes based on my recommendations here.
            Hide
            lauren Lauren MacArthur added a comment -

            My bottom line for now: the colorterms must be applied to the reference catalog mags in jointcal. I also believe the work for this has been done (but is perhaps a bit rotted...heh, heh) by Dominique Boutigny on DM-13054 in these commits: https://github.com/lsst/jointcal/commit/e1d0292c71275315a6b17b67ecaf47acd2ffbee8
            https://github.com/lsst/jointcal/commit/05f60a9a2c9b86a33487cccddd9b0640ca90faf1
            Given the significant magnitude of the effect, I believe this should block any further QA efforts (but I would, of course, be happy to pick them up along with the functionality being implemented). Hopefully the recent decision to Removing from current sprint per discussion of 2018-12-19. can be amended?

            Show
            lauren Lauren MacArthur added a comment - My bottom line for now: the colorterms must be applied to the reference catalog mags in jointcal . I also believe the work for this has been done (but is perhaps a bit rotted...heh, heh) by Dominique Boutigny on DM-13054 in these commits: https://github.com/lsst/jointcal/commit/e1d0292c71275315a6b17b67ecaf47acd2ffbee8 https://github.com/lsst/jointcal/commit/05f60a9a2c9b86a33487cccddd9b0640ca90faf1 Given the significant magnitude of the effect, I believe this should block any further QA efforts (but I would, of course, be happy to pick them up along with the functionality being implemented). Hopefully the recent decision to Removing from current sprint per discussion of 2018-12-19. can be amended?
            Hide
            lauren Lauren MacArthur added a comment - - edited

            NOTE: The following comments apply to the w_2018_48 RC2 reprocessing which includes coadds made using both the meas_mosaic and jointcal fits.

            I have just confirmed that the colorterm issue is our main contributor to the offsets noted above. The way things are currently running are actually inconsistent. The meas_mosaic and jointcal fits use the same Single Frame Processing run as input. The sfm's apply the colorterm corrections to the reference catalog magnitudes to do the initial photometric calibration. meas_mosaic also applies the colorterms when using the reference stars in its fit. jointcal does not. This puts jointcal in the position of "erasing" those corrections from the initial sfm photoCal and also, effectively, is using a mix-match of effective filters for the fitting (the reference stars are still on their own system whereas the newly added "internal" reference stars are on the observed system).

            My confirmation comes from remaking my plots but not applying the colorterm to the reference stars for the plots comparing ref to src matches (they look up the colorterms essentially how sfm would, so were being applied as they were unaware that jointcal was not applying them). This brings the offset down from 88[-59] mmag to 25[-4.5] mmag in HSC-I[HSC-G].

            When directly comparing the meas_mosaic and jointcal cModel mags (based on a matched catalog), the offset between them is also of order 88[-64] mmag, so the magnitude of the effect is in the right ballpark. The remaining 25[-4.5] mmag offset between the reference catalog could very well result from the relative weight of the effective mix-matching between filter systems in the jointcal fit.

            Taking into account the offset, there is still evidence for increased discrepancies around bright objects. This is perhaps the remaining effect that will be resolved with DM-16598. Yusra AlSayyad, let me know if you want me to point my scripts at any particular run that incorporates that change.

            Show
            lauren Lauren MacArthur added a comment - - edited NOTE: The following comments apply to the w_2018_48 RC2 reprocessing which includes coadds made using both the meas_mosaic and jointcal fits. I have just confirmed that the colorterm issue is our main contributor to the offsets noted above. The way things are currently running are actually inconsistent. The meas_mosaic and jointcal fits use the same Single Frame Processing run as input. The sfm's apply the colorterm corrections to the reference catalog magnitudes to do the initial photometric calibration. meas_mosaic also applies the colorterms when using the reference stars in its fit. jointcal does not. This puts jointcal in the position of "erasing" those corrections from the initial sfm photoCal and also, effectively, is using a mix-match of effective filters for the fitting (the reference stars are still on their own system whereas the newly added "internal" reference stars are on the observed system). My confirmation comes from remaking my plots but not applying the colorterm to the reference stars for the plots comparing ref to src matches (they look up the colorterms essentially how sfm would, so were being applied as they were unaware that jointcal was not applying them). This brings the offset down from 88 [ -59 ] mmag to 25 [ -4.5 ] mmag in HSC-I [ HSC-G ] . When directly comparing the meas_mosaic and jointcal cModel mags (based on a matched catalog), the offset between them is also of order 88 [ -64 ] mmag, so the magnitude of the effect is in the right ballpark. The remaining 25 [ -4.5 ] mmag offset between the reference catalog could very well result from the relative weight of the effective mix-matching between filter systems in the jointcal fit. Taking into account the offset, there is still evidence for increased discrepancies around bright objects. This is perhaps the remaining effect that will be resolved with DM-16598 . Yusra AlSayyad , let me know if you want me to point my scripts at any particular run that incorporates that change.
            Hide
            jbosch Jim Bosch added a comment -

            Ok, so we really are now down to "just" the ref vs. meas offset issue?

            I think so.

            Show
            jbosch Jim Bosch added a comment - Ok, so we really are now down to "just" the ref vs. meas offset issue? I think so.

              People

              • Assignee:
                lauren Lauren MacArthur
                Reporter:
                lauren Lauren MacArthur
                Reviewers:
                John Swinbank, Yusra AlSayyad
                Watchers:
                Hsin-Fang Chiang, Jim Bosch, John Parejko, John Swinbank, Lauren MacArthur, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel