Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-19015

HSC warp making is broken with doApplyUberCal=True

    Details

      Description

      makeCoaddTempExp.py with doApplyUberCal=True to use meas_mosaic outputs fail to build warps despite the input data exist. A small example to reproduce on lsst-dev is:

       

      makeCoaddTempExp.py /datasets/hsc/repo/ --rerun RC/w_2019_14/DM-18300:private/user/name --id tract=9615 filter=HSC-G patch=4,8 --selectId ccd=0..8^10..103 visit=26050^26036^26060^26032 
      

      This doesn't really "fail" (and no logs in ERROR or FATAL level) but instead gives many confusing/misleading warnings and produce no outputs:

       

      WARN 2019-04-08T01:32:59.059 makeCoaddTempExp (DataId(initialdata={'tract': 9615, 'filter': 'HSC-G', 'patch': '4,8'}, tag=set()))(makeCoaddTempExp.py:341)- Calexp DataId(initialdata={'ccd': 103, 'visit': 26032, 'pointing': 1179, 'filter': 'HSC-G', 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'taiObs': '2015-03-25', 'expTime': 150.0, 'tract': 9615}
      , tag=set()) not found; skipping it: 'NoneType' object has no attribute'getInstFluxAtZeroMagnitude'
      

       

      The log is misleading; this is DM-16537 about the generic exceptions. I troubleshoot a bit and trace to this line
      https://github.com/lsst/meas_mosaic/blob/6e395ac11a625c877374f99e2bed771b427835b6/python/lsst/meas/mosaic/updateExposure.py#L103
      where a non-empty header is obtained but no photoCalib is returned (that photCalib is None there). Feels like a bug in afw or meas_mosaic files; I paused there to file this ticket. Besides the bug, it'd be nice if the None calib is caught earlier.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            Merged and done.

            Show
            Parejkoj John Parejko added a comment - Merged and done.
            Hide
            Parejkoj John Parejko added a comment -

            Thanks for the note on the failed test. I swore that passed for me, but it clearly couldn't have. I've fixed the test code: can someone please give it a quick sign off?

            Jenkins: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29671/pipeline

            Show
            Parejkoj John Parejko added a comment - Thanks for the note on the failed test. I swore that passed for me, but it clearly couldn't have. I've fixed the test code: can someone please give it a quick sign off? Jenkins: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29671/pipeline
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Thanks Yusra AlSayyad for testing this and also with an old repo.

            John Parejko Looks fine, I only tested one case and it worked fine. Please fix the test failure and remember to run Jenkins before merging.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Thanks Yusra AlSayyad for testing this and also with an old repo. John Parejko Looks fine, I only tested one case and it worked fine. Please fix the test failure and remember to run Jenkins before merging.
            Hide
            yusra Yusra AlSayyad added a comment -

            But, I can confirm that makeCoaddTempExp/coaddDriver works with your branch with both w_2019_14 AND w_2019_10 meas_mosaic outputs. 

            Show
            yusra Yusra AlSayyad added a comment - But, I can confirm that makeCoaddTempExp/coaddDriver works with your branch with both w_2019_14 AND w_2019_10 meas_mosaic outputs. 
            Hide
            yusra Yusra AlSayyad added a comment - - edited

            I tried building this branch to give it a spin on the RC2 data and test_fluxFitBoundedField.py doesn't pass. 

            Show
            yusra Yusra AlSayyad added a comment - - edited I tried building this branch to give it a spin on the RC2 data and test_fluxFitBoundedField.py doesn't pass. 
            Hide
            Parejkoj John Parejko added a comment -

            Hsin-Fang Chiang: can you please review the one line change here and confirm that it works when making warps?

            Do we have any way to test this on old repositories? I think that my fix should maintain backwards compatibility, but I'm not positive and don't know how to test it in any case.

            I still think we need to have a telecon about how to deal with DM-16537.

            Show
            Parejkoj John Parejko added a comment - Hsin-Fang Chiang : can you please review the one line change here and confirm that it works when making warps? Do we have any way to test this on old repositories? I think that my fix should maintain backwards compatibility, but I'm not positive and don't know how to test it in any case. I still think we need to have a telecon about how to deal with DM-16537 .
            Hide
            Parejkoj John Parejko added a comment -

            I think I have a fix for this. But I'm not sure how to ensure that my fix also works with old meas_mosaic files too. Do we have a way to test that?

            Show
            Parejkoj John Parejko added a comment - I think I have a fix for this. But I'm not sure how to ensure that my fix also works with old meas_mosaic files too. Do we have a way to test that?
            Hide
            Parejkoj John Parejko added a comment - - edited

            Yusra AlSayyad: I'm guessing the problem in getFluxFitParams is somehow due to meas_mosaic's fcr metadata not being readable by PhotoCalib.makePhotoCalibFromMetadata? Would you or someone in DRP be able to look into that please? I don't know anything about those files.

            Separately, can we have a telecon sometime to formally deal with DM-16537? We need to fix how we handle exceptions in the coadd code (as evidenced above).

            Show
            Parejkoj John Parejko added a comment - - edited Yusra AlSayyad : I'm guessing the problem in getFluxFitParams is somehow due to meas_mosaic's fcr metadata not being readable by PhotoCalib.makePhotoCalibFromMetadata ? Would you or someone in DRP be able to look into that please? I don't know anything about those files. Separately, can we have a telecon sometime to formally deal with DM-16537 ? We need to fix how we handle exceptions in the coadd code (as evidenced above).

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Reviewers:
                Hsin-Fang Chiang
                Watchers:
                Hsin-Fang Chiang, John Parejko, John Swinbank, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel