Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-19265

Jacobian lost in meas_mosaic photometric solution

    XMLWordPrintable

    Details

      Description

      It looks like we lost the Jacobian from the meas_mosaic photometric solution with this deletion from DM-10156:
      https://github.com/lsst/meas_mosaic/commit/6e395ac11a625c877374f99e2bed771b427835b6#diff-ad69537790bfe1f2b36095cbbc6f80a9L709
      I noticed this in comparing meas_mosaic-calibrated magnitudes between the w_2019_10 and w_2019_14 RC2 reprocessings, which look like:

       

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-17743 Shepherd tickets for HSC release candidate

          • Done
          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-19015 HSC warp making is broken with doApplyUberCal=True

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-10156 Replace all uses of Calib with PhotoCalib

          • Done

            Activity

            Ascending order - Click to sort in descending order
            5 older comments
            Hide
            Permalink
            lauren Lauren MacArthur added a comment - - edited

            Ok, using your ticket branch, I ran:

            mosaic.py /datasets/hsc/repo --rerun RC/w_2019_14/DM-18300-sfm:private/lauren/DM-19265 --numCoresForRead=12 --diagnostics --diagDir=/scratch/lauren/DM-19265/mosaicDiag/ --id tract=9813 ccd=0..8^10..103 visit=1228..1232:2^1238..1248:2^19658..19662:2^19680..19684:2^19694..19698:2^19708..19712:2^30482..30504:2 filter=HSC-I

            The following is the comparison of this output to that of the w_2019_10 RC2 run for visit 1228 (meas_mosaic calibrations being applied here):

            So the good news is that the Jacobian is back Puzzling, though, is that there is still some difference between this branch and the w_2019_10 RC2 run. In fact, this difference also exists (in spirit) for the w14 vs w10 jointcal calibrations:

            Comparing w10 to w06 meas_mosaic looks like:

            which again show similar – but not identical – differences. Here is the same comparison but between the w10 vs w06 jointcal calibrations:

            So it seems there have been some subtle changes from weekly06-to-weekly10-to-weekly14 that slightly affect the uberCal results. I had a glance at the weekly changelogs and nothing in particular jumped out at me. Any ideas? If this is expected, then no further digging is required...otherwise we may need to figure out the source of these differences (however small?).

            Show
            lauren Lauren MacArthur added a comment - - edited Ok, using your ticket branch, I ran: mosaic.py / datasets / hsc / repo - - rerun RC / w_2019_14 / DM - 18300 - sfm:private / lauren / DM - 19265 - - numCoresForRead = 12 - - diagnostics - - diagDir = / scratch / lauren / DM - 19265 / mosaicDiag / - - id tract = 9813 ccd = 0. . 8 ^ 10. . 103 visit = 1228. . 1232 : 2 ^ 1238. . 1248 : 2 ^ 19658. . 19662 : 2 ^ 19680. . 19684 : 2 ^ 19694. . 19698 : 2 ^ 19708. . 19712 : 2 ^ 30482. . 30504 : 2 filter = HSC - I The following is the comparison of this output to that of the w_2019_10 RC2 run for visit 1228 ( meas_mosaic calibrations being applied here): So the good news is that the Jacobian is back Puzzling, though, is that there is still some difference between this branch and the  w_2019_10 RC2 run. In fact, this difference also exists (in spirit) for the w14 vs w10 jointcal calibrations: Comparing w10 to w06 meas_mosaic looks like: which again show similar – but not identical – differences. Here is the same comparison but between the w10 vs w06 jointcal calibrations: So it seems there have been some subtle changes from weekly06-to-weekly10-to-weekly14 that slightly affect the uberCal results. I had a glance at the weekly changelogs and nothing in particular jumped out at me. Any ideas? If this is expected, then no further digging is required...otherwise we may need to figure out the source of these differences (however small?).
            Hide
            Permalink
            rhl Robert Lupton added a comment -

            Are the raw Gaussian mags identical?

            Show
            rhl Robert Lupton added a comment - Are the raw Gaussian mags identical?
            Hide
            Permalink
            lauren Lauren MacArthur added a comment - - edited

            Yes: 

            Show
            lauren Lauren MacArthur added a comment - - edited Yes: 
            Hide
            Permalink
            lauren Lauren MacArthur added a comment -

            Seeing as these residual differences are likely unrelated to this issue, I think you’re good to merge this fix.  I’ll file a ticket to look into the cause of said differences.

            Show
            lauren Lauren MacArthur added a comment - Seeing as these residual differences are likely unrelated to this issue, I think you’re good to merge this fix.  I’ll file a ticket to look into the cause of said differences.
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Thanks: yes, lets deal with each bit separately.

            Merged and done.

            Show
            Parejkoj John Parejko added a comment - Thanks: yes, lets deal with each bit separately. Merged and done.

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              lauren Lauren MacArthur
              Reviewers:
              Lauren MacArthur
              Watchers:
              Jim Bosch, John Parejko, Lauren MacArthur, Robert Lupton, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.