Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13202

get visitAnalysis to work on jointcal's output

    XMLWordPrintable

    Details

    • Story Points:
      10
    • Epic Link:
    • Sprint:
      AP S18-2, AP S18-3, DRP F18-4, DRP F18-5, DRP F18-6, DRP S19-1, DRP S19-2, DRP S19-3, DRP S19-4
    • Team:
      Data Release Production

      Description

      To better understand why jointcal isn't producing the expected results on HSC PDR1, I need to make some plots. Fortunately, Lauren MacArthur and Tim Morton [X] have already done a lot of work making plots, so what I really need is to adapt some of the plots in pipe_analysis to work with jointcal's output, for both the plots comparing with a refcat and for the rerun vs. rerun plots (e.g., to compare singleFrame vs. jointcal).

      If done correctly, this shouldn't affect the use of meas_mosaic output, as it and jointcal output the WCS and PhotoCalib in a compatible way now.

        Attachments

          Issue Links

          is triggering

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-13231 Make photoCalib outField write to _flux instead of _calFlux

          • Done
          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-18573 Fix bug for running on DC2 data introduced by DM-13202

          • Done

          RFC - An issue whose primary purpose is to advertise an intent and allow stakeholders to comment. RFC-433 Update LoadAstrometryNetObjectsTask to use config instead of env variables

          • Withdrawn

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-11785 Run jointcal on acceptance data

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            swinbank John Swinbank added a comment - - edited

            We agreed per discussion with John Parejko, Jim Bosch, John Swinbank that this is not a high priority for now; we don't want to lose the work that's been done so far, but we're dropping it from the current sprint.

            (Although John Parejko will do a quick check and just see if what's there already works, in which case we'll merge and call this done.)

            Show
            swinbank John Swinbank added a comment - - edited We agreed per discussion with John Parejko , Jim Bosch , John Swinbank that this is not a high priority for now; we don't want to lose the work that's been done so far, but we're dropping it from the current sprint. (Although John Parejko will do a quick check and just see if what's there already works, in which case we'll merge and call this done.)
            Hide
            Permalink
            lauren Lauren MacArthur added a comment - - edited

            This ticket ballooned into a bit more than the description, in part to allow for the testing of the results. The default is now for the uber-calibration (currently either meas_mosaic or jointcal) to be applied to the catalog entries using the photoCalib object. This removes the dependency on meas_mosaic for running the scripts and allows for the application of calibrations on jointcal outputs (both meas_mosaic and jointcal persist photoCalib objects, while the former additionally persists its own fcr flux calibration dataset which is what was being used to apply calibrations to catalogs through meas_mosaic's applyMosaicResultsCatalog() function prior to this update).

            The photoCalib based calibration is applied to a catalog using the new calibrateSourceCatalogPhotoCalib() function in pipe_analysis's utils.py. The old calibrateSourceCatalogMosaic() function is still there and its use (i.e. apply calibrations through meas_mosaic's applyMosaicResultsCatalog() function) can be imposed via the config parameter: useMeasMosaic for visitAnalysis.py.

            Thus, calibrations can now be applied in 3 ways:

            • old-style meas_mosiac using the fcr dataset
            • new-style meas_mosaic using the photoCalib dataset
            • jointcal using the photoCalib dataset

            Additionally, the comparisonVisitAnalysis.py script has been updated to allow for direct comparisons between these different calibrations based on the same single frame processing. To impose the old-style meas_mosaic calibration on either the rerun or rerun2, use config useMeasMosaic1=True and/or useMeasMosaic2=True.

            Examples of such comparison plots can be found (for now...): https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-13202/jointcal/matchXy/plots/

            A label is added to all output plots to indicate which calibrations were applied.

            Plots comparing the base_CircularApertureFlux_12_0 to the reference mags have been added (i.e. in addition to the PSF flux comparisons) to have a diagnostic not influenced by PSF modeling issues.

            On two side notes:
            1) I've also added a plotMatchesOnly config parameter such that, when set to True, only the plots making comparisons to the reference catalog are created (to speed up runtimes if those are the only plots you are interested in).
            2) For all the "trace" plots, I've added a y-axis label that plots the FWHM scale, where 

            FWHM = 2*sqrt(2*ln(2))*Trace (pixels)

            and,

            Trace = sqrt(0.5*(Ixx + Iyy)) (pixels)

            Finally, as a check that the photoCalib-based corrections are being applied correctly (i.e. we get the same results as in the "old" meas_mosaic method), this table presents the differences between the two. Only very small differences are seen in some cases and they are larger for the rotated CCDs.  This is to be expected due to round-off errors and errors from rotation, as noted here:
            https://github.com/lsst/meas_mosaic/blob/master/tests/test_fluxFitBoundedField.py#L278-L281

            visit filter ccd nQuarter parameter mean(absDiff) mean(relDiff)
            29326 HSC-G 49 0 Ra 0.0000e+00 0.0000e+00
            1220 HSC-R 49 0 Ra 0.0000e+00 0.0000e+00
            1228 HSC-I 49 0 Ra 0.0000e+00 0.0000e+00
            17926 HSC-Z 49 0 Ra 0.0000e+00 0.0000e+00
            1880 HSC-Y 49 0 Ra 0.0000e+00 0.0000e+00
            25810 NB0921 49 0 Ra 0.0000e+00 0.0000e+00
            29326 HSC-G 50 2 Ra 0.0000e+00 0.0000e+00
            1220 HSC-R 50 2 Ra 4.1889e-18 2.3021e-18
            1228 HSC-I 50 2 Ra 0.0000e+00 0.0000e+00
            17926 HSC-Z 50 2 Ra 0.0000e+00 0.0000e+00
            1880 HSC-Y 50 2 Ra 0.0000e+00 0.0000e+00
            25810 NB0921 50 2 Ra 0.0000e+00 0.0000e+00
            29326 HSC-G 101 3 Ra 0.0000e+00 0.0000e+00
            1220 HSC-R 101 3 Ra 0.0000e+00 0.0000e+00
            1228 HSC-I 101 3 Ra 0.0000e+00 0.0000e+00
            17926 HSC-Z 101 3 Ra 0.0000e+00 0.0000e+00
            1880 HSC-Y 101 3 Ra 0.0000e+00 0.0000e+00
            25810 NB0921 101 3 Ra 0.0000e+00 0.0000e+00
            29326 HSC-G 102 1 Ra 1.3919e-17 4.1738e-18
            1220 HSC-R 102 1 Ra 8.3250e-18 3.2274e-18
            1228 HSC-I 102 1 Ra 0.0000e+00 0.0000e+00
            17926 HSC-Z 102 1 Ra 0.0000e+00 0.0000e+00
            1880 HSC-Y 102 1 Ra 0.0000e+00 0.0000e+00
            25810 NB0921 102 1 Ra 0.0000e+00 0.0000e+00
            29326 HSC-G 49 0 Dec 0.0000e+00 0.0000e+00
            1220 HSC-R 49 0 Dec 0.0000e+00 0.0000e+00
            1228 HSC-I 49 0 Dec 0.0000e+00 0.0000e+00
            17926 HSC-Z 49 0 Dec 0.0000e+00 0.0000e+00
            1880 HSC-Y 49 0 Dec 0.0000e+00 0.0000e+00
            25810 NB0921 49 0 Dec 0.0000e+00 0.0000e+00
            29326 HSC-G 50 2 Dec 1.1988e-18 9.8746e-18
            1220 HSC-R 50 2 Dec 9.1632e-19 9.3267e-18
            1228 HSC-I 50 2 Dec 3.9184e-19 6.0678e-18
            17926 HSC-Z 50 2 Dec 1.0107e-18 9.8174e-18
            1880 HSC-Y 50 2 Dec 2.7195e-19 5.1145e-18
            25810 NB0921 50 2 Dec 1.3186e-18 1.1286e-17
            29326 HSC-G 101 3 Dec 9.4286e-19 1.0475e-17
            1220 HSC-R 101 3 Dec 6.3183e-19 8.8226e-18
            1228 HSC-I 101 3 Dec 0.0000e+00 0.0000e+00
            17926 HSC-Z 101 3 Dec 3.9138e-19 6.6549e-18
            1880 HSC-Y 101 3 Dec 7.0602e-19 9.6614e-18
            25810 NB0921 101 3 Dec 0.0000e+00 0.0000e+00
            29326 HSC-G 102 1 Dec 7.8292e-18 1.0869e-16
            1220 HSC-R 102 1 Dec 4.8129e-18 4.7075e-17
            1228 HSC-I 102 1 Dec 2.2063e-18 1.2041e-17
            17926 HSC-Z 102 1 Dec 3.5334e-18 1.6709e-17
            1880 HSC-Y 102 1 Dec 3.0169e-18 1.4092e-17
            25810 NB0921 102 1 Dec 2.4384e-18 1.2833e-17
            29326 HSC-G 49 0 base_PsfFlux_instFlux 7.6667e-25 1.1526e-15
            1220 HSC-R 49 0 base_PsfFlux_instFlux 6.2743e-26 2.2536e-16
            1228 HSC-I 49 0 base_PsfFlux_instFlux 9.6340e-25 6.7861e-16
            17926 HSC-Z 49 0 base_PsfFlux_instFlux 4.6408e-26 8.8578e-17
            1880 HSC-Y 49 0 base_PsfFlux_instFlux 2.7764e-24 5.3892e-16
            25810 NB0921 49 0 base_PsfFlux_instFlux 4.9639e-25 1.6196e-16
            29326 HSC-G 50 2 base_PsfFlux_instFlux -9.8941e-16 1.4998e-06
            1220 HSC-R 50 2 base_PsfFlux_instFlux -4.6116e-16 1.4678e-06
            1228 HSC-I 50 2 base_PsfFlux_instFlux -2.3602e-15 1.4033e-06
            17926 HSC-Z 50 2 base_PsfFlux_instFlux -1.3516e-15 1.5280e-06
            1880 HSC-Y 50 2 base_PsfFlux_instFlux -9.2694e-15 1.4461e-06
            25810 NB0921 50 2 base_PsfFlux_instFlux -6.1264e-15 1.4838e-06
            29326 HSC-G 101 3 base_PsfFlux_instFlux -2.2049e-14 1.3561e-05
            1220 HSC-R 101 3 base_PsfFlux_instFlux -5.4671e-15 1.3684e-05
            1228 HSC-I 101 3 base_PsfFlux_instFlux -3.3679e-14 1.3540e-05
            17926 HSC-Z 101 3 base_PsfFlux_instFlux -2.2194e-14 1.3889e-05
            1880 HSC-Y 101 3 base_PsfFlux_instFlux -1.2011e-13 1.3958e-05
            25810 NB0921 101 3 base_PsfFlux_instFlux -1.6845e-13 1.3619e-05
            29326 HSC-G 102 1 base_PsfFlux_instFlux 1.9796e-14 7.8843e-06
            1220 HSC-R 102 1 base_PsfFlux_instFlux 3.3387e-15 7.8088e-06
            1228 HSC-I 102 1 base_PsfFlux_instFlux 1.6482e-14 7.8345e-06
            17926 HSC-Z 102 1 base_PsfFlux_instFlux 9.3632e-15 7.9582e-06
            1880 HSC-Y 102 1 base_PsfFlux_instFlux 6.0701e-14 7.8515e-06
            25810 NB0921 102 1 base_PsfFlux_instFlux 4.8597e-14 7.8921e-06
            Show
            lauren Lauren MacArthur added a comment - - edited This ticket ballooned into a bit more than the description, in part to allow for the testing of the results. The default is now for the uber-calibration (currently either  meas_mosaic  or jointcal ) to be applied to the catalog entries using the photoCalib object. This removes the dependency on meas_mosaic for running the scripts and allows for the application of calibrations on jointcal outputs (both meas_mosaic and jointcal persist photoCalib objects, while the former additionally persists its own fcr flux calibration dataset which is what was being used to apply calibrations to catalogs through meas_mosaic 's applyMosaicResultsCatalog() function prior to this update). The photoCalib based calibration is applied to a catalog using the new calibrateSourceCatalogPhotoCalib() function in pipe_analysis 's utils.py . The old calibrateSourceCatalogMosaic() function is still there and its use (i.e. apply calibrations through meas_mosaic 's applyMosaicResultsCatalog() function) can be imposed via the config parameter: useMeasMosaic for visitAnalysis.py. Thus, calibrations can now be applied in 3 ways: old-style meas_mosiac using the fcr dataset new-style meas_mosaic using the photoCalib dataset jointcal using the photoCalib dataset Additionally, the comparisonVisitAnalysis.py script has been updated to allow for direct comparisons between these different calibrations based on the same single frame processing. To impose the old-style meas_mosaic calibration on either the rerun or rerun2, use config useMeasMosaic1=True and/or useMeasMosaic2=True . Examples of such comparison plots can be found (for now...): https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-13202/jointcal/matchXy/plots/ A label is added to all output plots to indicate which calibrations were applied. Plots comparing the base_CircularApertureFlux_12_0 to the reference mags have been added (i.e. in addition to the PSF flux comparisons) to have a diagnostic not influenced by PSF modeling issues. On two side notes: 1) I've also added a plotMatchesOnly config parameter such that, when set to True , only the plots making comparisons to the reference catalog are created (to speed up runtimes if those are the only plots you are interested in). 2) For all the "trace" plots, I've added a y-axis label that plots the FWHM scale, where FWHM = 2*sqrt(2*ln(2))*Trace (pixels) and, Trace = sqrt(0.5*(Ixx + Iyy)) (pixels) Finally, as a check that the photoCalib-based corrections are being applied correctly (i.e. we get the same results as in the "old" meas_mosaic method), this table presents the differences between the two. Only very small differences are seen in some cases and they are larger for the rotated CCDs.  This is to be expected due to round-off errors and errors from rotation, as noted here: https://github.com/lsst/meas_mosaic/blob/master/tests/test_fluxFitBoundedField.py#L278-L281 visit filter ccd nQuarter parameter mean(absDiff) mean(relDiff) 29326 HSC-G 49 0 Ra 0.0000e+00 0.0000e+00 1220 HSC-R 49 0 Ra 0.0000e+00 0.0000e+00 1228 HSC-I 49 0 Ra 0.0000e+00 0.0000e+00 17926 HSC-Z 49 0 Ra 0.0000e+00 0.0000e+00 1880 HSC-Y 49 0 Ra 0.0000e+00 0.0000e+00 25810 NB0921 49 0 Ra 0.0000e+00 0.0000e+00 29326 HSC-G 50 2 Ra 0.0000e+00 0.0000e+00 1220 HSC-R 50 2 Ra 4.1889e-18 2.3021e-18 1228 HSC-I 50 2 Ra 0.0000e+00 0.0000e+00 17926 HSC-Z 50 2 Ra 0.0000e+00 0.0000e+00 1880 HSC-Y 50 2 Ra 0.0000e+00 0.0000e+00 25810 NB0921 50 2 Ra 0.0000e+00 0.0000e+00 29326 HSC-G 101 3 Ra 0.0000e+00 0.0000e+00 1220 HSC-R 101 3 Ra 0.0000e+00 0.0000e+00 1228 HSC-I 101 3 Ra 0.0000e+00 0.0000e+00 17926 HSC-Z 101 3 Ra 0.0000e+00 0.0000e+00 1880 HSC-Y 101 3 Ra 0.0000e+00 0.0000e+00 25810 NB0921 101 3 Ra 0.0000e+00 0.0000e+00 29326 HSC-G 102 1 Ra 1.3919e-17 4.1738e-18 1220 HSC-R 102 1 Ra 8.3250e-18 3.2274e-18 1228 HSC-I 102 1 Ra 0.0000e+00 0.0000e+00 17926 HSC-Z 102 1 Ra 0.0000e+00 0.0000e+00 1880 HSC-Y 102 1 Ra 0.0000e+00 0.0000e+00 25810 NB0921 102 1 Ra 0.0000e+00 0.0000e+00 29326 HSC-G 49 0 Dec 0.0000e+00 0.0000e+00 1220 HSC-R 49 0 Dec 0.0000e+00 0.0000e+00 1228 HSC-I 49 0 Dec 0.0000e+00 0.0000e+00 17926 HSC-Z 49 0 Dec 0.0000e+00 0.0000e+00 1880 HSC-Y 49 0 Dec 0.0000e+00 0.0000e+00 25810 NB0921 49 0 Dec 0.0000e+00 0.0000e+00 29326 HSC-G 50 2 Dec 1.1988e-18 9.8746e-18 1220 HSC-R 50 2 Dec 9.1632e-19 9.3267e-18 1228 HSC-I 50 2 Dec 3.9184e-19 6.0678e-18 17926 HSC-Z 50 2 Dec 1.0107e-18 9.8174e-18 1880 HSC-Y 50 2 Dec 2.7195e-19 5.1145e-18 25810 NB0921 50 2 Dec 1.3186e-18 1.1286e-17 29326 HSC-G 101 3 Dec 9.4286e-19 1.0475e-17 1220 HSC-R 101 3 Dec 6.3183e-19 8.8226e-18 1228 HSC-I 101 3 Dec 0.0000e+00 0.0000e+00 17926 HSC-Z 101 3 Dec 3.9138e-19 6.6549e-18 1880 HSC-Y 101 3 Dec 7.0602e-19 9.6614e-18 25810 NB0921 101 3 Dec 0.0000e+00 0.0000e+00 29326 HSC-G 102 1 Dec 7.8292e-18 1.0869e-16 1220 HSC-R 102 1 Dec 4.8129e-18 4.7075e-17 1228 HSC-I 102 1 Dec 2.2063e-18 1.2041e-17 17926 HSC-Z 102 1 Dec 3.5334e-18 1.6709e-17 1880 HSC-Y 102 1 Dec 3.0169e-18 1.4092e-17 25810 NB0921 102 1 Dec 2.4384e-18 1.2833e-17 29326 HSC-G 49 0 base_PsfFlux_instFlux 7.6667e-25 1.1526e-15 1220 HSC-R 49 0 base_PsfFlux_instFlux 6.2743e-26 2.2536e-16 1228 HSC-I 49 0 base_PsfFlux_instFlux 9.6340e-25 6.7861e-16 17926 HSC-Z 49 0 base_PsfFlux_instFlux 4.6408e-26 8.8578e-17 1880 HSC-Y 49 0 base_PsfFlux_instFlux 2.7764e-24 5.3892e-16 25810 NB0921 49 0 base_PsfFlux_instFlux 4.9639e-25 1.6196e-16 29326 HSC-G 50 2 base_PsfFlux_instFlux -9.8941e-16 1.4998e-06 1220 HSC-R 50 2 base_PsfFlux_instFlux -4.6116e-16 1.4678e-06 1228 HSC-I 50 2 base_PsfFlux_instFlux -2.3602e-15 1.4033e-06 17926 HSC-Z 50 2 base_PsfFlux_instFlux -1.3516e-15 1.5280e-06 1880 HSC-Y 50 2 base_PsfFlux_instFlux -9.2694e-15 1.4461e-06 25810 NB0921 50 2 base_PsfFlux_instFlux -6.1264e-15 1.4838e-06 29326 HSC-G 101 3 base_PsfFlux_instFlux -2.2049e-14 1.3561e-05 1220 HSC-R 101 3 base_PsfFlux_instFlux -5.4671e-15 1.3684e-05 1228 HSC-I 101 3 base_PsfFlux_instFlux -3.3679e-14 1.3540e-05 17926 HSC-Z 101 3 base_PsfFlux_instFlux -2.2194e-14 1.3889e-05 1880 HSC-Y 101 3 base_PsfFlux_instFlux -1.2011e-13 1.3958e-05 25810 NB0921 101 3 base_PsfFlux_instFlux -1.6845e-13 1.3619e-05 29326 HSC-G 102 1 base_PsfFlux_instFlux 1.9796e-14 7.8843e-06 1220 HSC-R 102 1 base_PsfFlux_instFlux 3.3387e-15 7.8088e-06 1228 HSC-I 102 1 base_PsfFlux_instFlux 1.6482e-14 7.8345e-06 17926 HSC-Z 102 1 base_PsfFlux_instFlux 9.3632e-15 7.9582e-06 1880 HSC-Y 102 1 base_PsfFlux_instFlux 6.0701e-14 7.8515e-06 25810 NB0921 102 1 base_PsfFlux_instFlux 4.8597e-14 7.8921e-06
            Hide
            Permalink
            lauren Lauren MacArthur added a comment -

            Ok, this is ready to go.  I think I've tested every permutation and combination of the scripts to make sure all is working as it should, but do let me know if you see/have any issues.

            Show
            lauren Lauren MacArthur added a comment - Ok, this is ready to go.  I think I've tested every permutation and combination of the scripts to make sure all is working as it should, but do let me know if you see/have any issues.
            Hide
            Permalink
            yusra Yusra AlSayyad added a comment -

            Look OK to merge

            Show
            yusra Yusra AlSayyad added a comment - Look OK to merge
            Hide
            Permalink
            lauren Lauren MacArthur added a comment -

            Thanks for the review. I either addressed or commented on all your points. Merged to master.

            Show
            lauren Lauren MacArthur added a comment - Thanks for the review. I either addressed or commented on all your points. Merged to master.

              People

              Assignee:
              lauren Lauren MacArthur
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              Yusra AlSayyad
              Watchers:
              Hsin-Fang Chiang, John Parejko, John Swinbank, Lauren MacArthur, Tim Morton [X] (Inactive), Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.