Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-28046

Update PhosimEimgParseTask to use LsstCam

    XMLWordPrintable

    Details

      Description

      To allow ingest of phosim-simulated e-images, `obs_lsst/config/phosim/ingestEimg.py` was used as config for ingestImages.py , but with an update to the phosim repackager to emulate LsstCam headers,  a change is needed to use LsstCamMapper for header translation  (PhosimParseTask to LsstCamParseTask in  https://github.com/lsst/obs_lsst/blob/61239df17fc0cad6f2250682df420affea96f534/python/lsst/obs/lsst/phosim.py#L62  )

      Otherwise an incorrect set of header translators is used : 

      ingestImages.py /project/scichris/repackagerTest/input_w48/ /project/scichris/repackagerTest/intraRpkgd/*.fits  --configfile /project/scichris/dev/obs_lsst/config/phosim/ingestEimg.py
      

      yields 

      root INFO: Loading config overrride file '/project/scichris/dev/obs_lsst_dev/config/ingest.py'
      root INFO: Loading config overrride file '/project/scichris/dev/obs_lsst_dev/config/lsstCam/ingest.py'
      LsstCamMapper WARN: Unable to find valid calib root directory
      CameraMapper INFO: Loading Posix exposure registry from /project/scichris/repackagerTest/input_w48
      astro_metadata_translator.observationInfo WARN: Ignoring Error calculating property 'detector_unique_name' using translator <class 'lsst.obs.lsst.translators.phosim.LsstCamPhoSimTranslator'>: "Could not find ['SENSNAME'] in header"
      astro_metadata_translator.observationInfo WARN: Ignoring Error calculating property 'detector_group' using translator <class 'lsst.obs.lsst.translators.phosim.LsstCamPhoSimTranslator'>: "Could not find ['RAFTNAME'] in header"
      astro_metadata_translator.observationInfo WARN: Ignoring Error calculating property 'detector_num' using translator <class 'lsst.obs.lsst.translators.phosim.LsstCamPhoSimTranslator'>: "Could not find ['RAFTNAME'] in header"
      astro_metadata_translator.observationInfo WARN: Ignoring Error calculating property 'detector_name' using translator <class 'lsst.obs.lsst.translators.phosim.LsstCamPhoSimTranslator'>: "Could not find ['SENSNAME'] in header"
      astro_metadata_translator.observationInfo WARN: Ignoring Error calculating property 'boresight_rotation_angle' using translator <class 'lsst.obs.lsst.translators.phosim.LsstCamPhoSimTranslator'>: "Could not find ['ROTANGZ', 'ROTANGLE'] in header"
      astro_metadata_translator.observationInfo WARN: Ignoring Error calculating property 'detector_exposure_id' using translator <class 'lsst.obs.lsst.translators.phosim.LsstCamPhoSimTranslator'>: "Could not find ['RAFTNAME'] in header"
      ingest WARN: Failed to ingest file /project/scichris/repackagerTest/intraRpkgd/MC_H_20000217_000031_R00_SW0.fits: 'detector'
      Traceback (most recent call last):
        File "/opt/lsst/software/stack/stack/miniconda3-py37_4.8.2-cb4e2dc/Linux64/pipe_tasks/21.0.0-12-gb58d4cc3+0fdaeca7f2/python/lsst/pipe/tasks/ingest.py", line 625, in run
          self.runFile(infile, registry, args, pos)
        File "/opt/lsst/software/stack/stack/miniconda3-py37_4.8.2-cb4e2dc/Linux64/pipe_tasks/21.0.0-12-gb58d4cc3+0fdaeca7f2/python/lsst/pipe/tasks/ingest.py", line 598, in runFile
          if registry is not None and self.register.check(registry, fileInfo):
        File "/opt/lsst/software/stack/stack/miniconda3-py37_4.8.2-cb4e2dc/Linux64/pipe_tasks/21.0.0-12-gb58d4cc3+0fdaeca7f2/python/lsst/pipe/tasks/ingest.py", line 352, in check
          values = [self.typemap[self.config.columns[col]](info[col]) for col in self.config.unique]
        File "/opt/lsst/software/stack/stack/miniconda3-py37_4.8.2-cb4e2dc/Linux64/pipe_tasks/21.0.0-12-gb58d4cc3+0fdaeca7f2/python/lsst/pipe/tasks/ingest.py", line 352, in <listcomp>
          values = [self.typemap[self.config.columns[col]](info[col]) for col in self.config.unique]
      KeyError: 'detector'The above exception was the direct cause of the following exception:Traceback (most recent call last):
        File "/opt/lsst/software/stack/stack/miniconda3-py37_4.8.2-cb4e2dc/Linux64/pipe_tasks/21.0.0-12-gb58d4cc3+0fdaeca7f2/bin/ingestImages.py", line 3, in <module>
          IngestTask.parseAndRun()
        File "/opt/lsst/software/stack/stack/miniconda3-py37_4.8.2-cb4e2dc/Linux64/pipe_tasks/21.0.0-12-gb58d4cc3+0fdaeca7f2/python/lsst/pipe/tasks/ingest.py", line 431, in parseAndRun
          task.run(args)
        File "/opt/lsst/software/stack/stack/miniconda3-py37_4.8.2-cb4e2dc/Linux64/pipe_tasks/21.0.0-12-gb58d4cc3+0fdaeca7f2/python/lsst/pipe/tasks/ingest.py", line 629, in run
          raise IngestError(f"Failed to ingest file {infile}", infile, pos) from exc
      lsst.pipe.tasks.ingest.IngestError: Failed to ingest file /project/scichris/repackagerTest/intraRpkgd/MC_H_20000217_000031_R00_SW0.fits
      

        Attachments

          Issue Links

            Activity

            ksuberlak Krzysztof Suberlak created issue -
            ksuberlak Krzysztof Suberlak made changes -
            Field Original Value New Value
            Link This issue has to be done before DM-26836 [ DM-26836 ]
            ksuberlak Krzysztof Suberlak made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            ksuberlak Krzysztof Suberlak made changes -
            Reviewers Tim Jenness [ tjenness ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            tjenness Tim Jenness made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            ksuberlak Krzysztof Suberlak made changes -
            Link This issue is child task of DM-28556 [ DM-28556 ]
            ksuberlak Krzysztof Suberlak made changes -
            Epic Link DM-28335 [ 445220 ]
            ksuberlak Krzysztof Suberlak made changes -
            Link This issue has to be done before DM-26836 [ DM-26836 ]
            ksuberlak Krzysztof Suberlak made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            ajc Andrew Connolly made changes -
            Story Points 6

              People

              Assignee:
              ksuberlak Krzysztof Suberlak
              Reporter:
              ksuberlak Krzysztof Suberlak
              Reviewers:
              Tim Jenness
              Watchers:
              Kian-Tat Lim, Krzysztof Suberlak, Robert Lupton, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.