Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-17195

processCcd failed with 'INST-PA not found' since w_2019_01

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      2
    • Sprint:
      DRP S19-2
    • Team:
      Data Release Production

      Description

       processCcd.py with some HSC data on /datasets/hsc failed with w_2019_01 or w_2019_02. It worked with w_2018_52.  One example command is

      processCcd.py /datasets/hsc/repo/  --rerun private/user/name --id visit=318 ccd=0 
      

      and the error as follows: 
       

      Traceback (most recent call last):
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/pipe_base/16.0-25-g2c6bf4a+1/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__
          result = self.runTask(task, dataRef, kwargs)
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/pipe_base/16.0-25-g2c6bf4a+1/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
          return task.runDataRef(dataRef, **kwargs)
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/pipe_base/16.0-25-g2c6bf4a+1/python/lsst/pipe/base/timer.py", line 149, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/pipe_tasks/16.0-51-gbbe9c988/python/lsst/pipe/tasks/processCcd.py", line 183, in runDataRef
          exposure = self.isr.runDataRef(sensorRef).exposure
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/pipe_base/16.0-25-g2c6bf4a+1/python/lsst/pipe/base/timer.py", line 149, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/ip_isr/16.0-17-g5cf0468+1/python/lsst/ip/isr/isrTask.py", line 1087, in runDataRef
          result = self.run(ccdExposure, camera=camera, **isrData.getDict())
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/pipe_base/16.0-25-g2c6bf4a+1/python/lsst/pipe/base/timer.py", line 149, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/ip_isr/16.0-17-g5cf0468+1/python/lsst/ip/isr/isrTask.py", line 961, in run
          self.strayLight.run(ccdExposure)
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/obs_subaru/16.0-43-g07e5bd20/python/lsst/obs/subaru/strayLight/yStrayLight.py", line 84, in run
          angleStart, angleEnd = inrStartEnd(exposureMetadata)
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/obs_subaru/16.0-43-g07e5bd20/python/lsst/obs/subaru/strayLight/rotatorAngle.py", line 251, in inrStartEnd
          inst_pa = header.getDouble('INST-PA')
      lsst.pex.exceptions.wrappers.NotFoundError: 
        File "src/PropertySet.cc", line 189, in T lsst::daf::base::PropertySet::get(const string&) const [with T = double; std::string = std::basic_string<char>]
          INST-PA not found {0}
      lsst::pex::exceptions::NotFoundError: 'INST-PA not found'
      

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            The instrument position angle is stored in VisitInfo and calculated and stored in ObservationInfo by astro_metadata_translator so in theory it shouldn't need to be extracted again directly from the header.

            Show
            tjenness Tim Jenness added a comment - The instrument position angle is stored in VisitInfo and calculated and stored in ObservationInfo by astro_metadata_translator so in theory it shouldn't need to be extracted again directly from the header.
            Hide
            czw Christopher Waters added a comment -

            That definitely will save me a lot of time trying to figure out where to find that value.

            My conversion attempted to duplicate the code that already existed, but removing as many butler calls as possible.  I guess I didn't test this as much as it should be.

            Show
            czw Christopher Waters added a comment - That definitely will save me a lot of time trying to figure out where to find that value. My conversion attempted to duplicate the code that already existed, but removing as many butler calls as possible.  I guess I didn't test this as much as it should be.
            Hide
            tjenness Tim Jenness added a comment - - edited

            Ideally generic concepts like this will all be handled in a single place in astro_metadata_translator so that the algorithmic code using the concept can be generic and not rely on specific FITS headers being present. Adding extra information to ObservationInfo is allowed.

            Show
            tjenness Tim Jenness added a comment - - edited Ideally generic concepts like this will all be handled in a single place in astro_metadata_translator so that the algorithmic code using the concept can be generic and not rely on specific FITS headers being present. Adding extra information to ObservationInfo is allowed.
            Hide
            tjenness Tim Jenness added a comment -

            Looks okay but please also use the date from VisitInfo.

            Show
            tjenness Tim Jenness added a comment - Looks okay but please also use the date from VisitInfo.
            Hide
            czw Christopher Waters added a comment -

            Updated to use VisitInfo, and tested to run completely on the example bad exposure.

            Show
            czw Christopher Waters added a comment - Updated to use VisitInfo, and tested to run completely on the example bad exposure.

              People

              • Assignee:
                czw Christopher Waters
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Reviewers:
                Tim Jenness
                Watchers:
                Christopher Waters, Hsin-Fang Chiang, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel