Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-19265

Jacobian lost in meas_mosaic photometric solution

    Details

      Description

      It looks like we lost the Jacobian from the meas_mosaic photometric solution with this deletion from DM-10156:
      https://github.com/lsst/meas_mosaic/commit/6e395ac11a625c877374f99e2bed771b427835b6#diff-ad69537790bfe1f2b36095cbbc6f80a9L709
      I noticed this in comparing meas_mosaic-calibrated magnitudes between the w_2019_10 and w_2019_14 RC2 reprocessings, which look like:

       

        Attachments

          Issue Links

            Activity

            Hide
            lauren Lauren MacArthur added a comment - - edited

            Ok, using your ticket branch, I ran:

            mosaic.py /datasets/hsc/repo --rerun RC/w_2019_14/DM-18300-sfm:private/lauren/DM-19265 --numCoresForRead=12 --diagnostics --diagDir=/scratch/lauren/DM-19265/mosaicDiag/ --id tract=9813 ccd=0..8^10..103 visit=1228..1232:2^1238..1248:2^19658..19662:2^19680..19684:2^19694..19698:2^19708..19712:2^30482..30504:2 filter=HSC-I
            

            The following is the comparison of this output to that of the w_2019_10 RC2 run for visit 1228 (meas_mosaic calibrations being applied here):

            So the good news is that the Jacobian is back Puzzling, though, is that there is still some difference between this branch and the w_2019_10 RC2 run. In fact, this difference also exists (in spirit) for the w14 vs w10 jointcal calibrations:

            Comparing w10 to w06 meas_mosaic looks like:

            which again show similar – but not identical – differences. Here is the same comparison but between the w10 vs w06 jointcal calibrations:

            So it seems there have been some subtle changes from weekly06-to-weekly10-to-weekly14 that slightly affect the uberCal results. I had a glance at the weekly changelogs and nothing in particular jumped out at me. Any ideas? If this is expected, then no further digging is required...otherwise we may need to figure out the source of these differences (however small?).

            Show
            lauren Lauren MacArthur added a comment - - edited Ok, using your ticket branch, I ran: mosaic.py / datasets / hsc / repo - - rerun RC / w_2019_14 / DM - 18300 - sfm:private / lauren / DM - 19265 - - numCoresForRead = 12 - - diagnostics - - diagDir = / scratch / lauren / DM - 19265 / mosaicDiag / - - id tract = 9813 ccd = 0. . 8 ^ 10. . 103 visit = 1228. . 1232 : 2 ^ 1238. . 1248 : 2 ^ 19658. . 19662 : 2 ^ 19680. . 19684 : 2 ^ 19694. . 19698 : 2 ^ 19708. . 19712 : 2 ^ 30482. . 30504 : 2 filter = HSC - I The following is the comparison of this output to that of the w_2019_10 RC2 run for visit 1228 ( meas_mosaic calibrations being applied here): So the good news is that the Jacobian is back Puzzling, though, is that there is still some difference between this branch and the  w_2019_10 RC2 run. In fact, this difference also exists (in spirit) for the w14 vs w10 jointcal calibrations: Comparing w10 to w06 meas_mosaic looks like: which again show similar – but not identical – differences. Here is the same comparison but between the w10 vs w06 jointcal calibrations: So it seems there have been some subtle changes from weekly06-to-weekly10-to-weekly14 that slightly affect the uberCal results. I had a glance at the weekly changelogs and nothing in particular jumped out at me. Any ideas? If this is expected, then no further digging is required...otherwise we may need to figure out the source of these differences (however small?).
            Hide
            rhl Robert Lupton added a comment -

            Are the raw Gaussian mags identical?

            Show
            rhl Robert Lupton added a comment - Are the raw Gaussian mags identical?
            Hide
            lauren Lauren MacArthur added a comment - - edited

            Yes: 

            Show
            lauren Lauren MacArthur added a comment - - edited Yes: 
            Hide
            lauren Lauren MacArthur added a comment -

            Seeing as these residual differences are likely unrelated to this issue, I think you’re good to merge this fix.  I’ll file a ticket to look into the cause of said differences.

            Show
            lauren Lauren MacArthur added a comment - Seeing as these residual differences are likely unrelated to this issue, I think you’re good to merge this fix.  I’ll file a ticket to look into the cause of said differences.
            Hide
            Parejkoj John Parejko added a comment -

            Thanks: yes, lets deal with each bit separately.

            Merged and done.

            Show
            Parejkoj John Parejko added a comment - Thanks: yes, lets deal with each bit separately. Merged and done.

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                lauren Lauren MacArthur
                Reviewers:
                Lauren MacArthur
                Watchers:
                Jim Bosch, John Parejko, Lauren MacArthur, Robert Lupton, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel