Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-32298

Issue with comcam visitInfo metadata at the summit

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_lsst
    • Labels:
      None
    • Story Points:
      1
    • Epic Link:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      While doing integration tests at the summit with the Main Telescope components and ComCam we noticed that the visitInfo metadata returns a series of nan values for fields that are apparently in the header.

      To reproduce the error you can do:

      import lsst.daf.butler as dafButler
      		 
       
      		 
      butler = dafButler.Butler("/repo/LSSTComCam/")
      		 
       
      		 
      data_id = {'instrument': 'LSSTComCam', 'detector': 0, 'exposure': 2021102100008}
      		 
       
      		 
      raw = butler.get('raw', dataId=data_id, collections=["LSSTComCam/raw/all"])
      		 
      print(raw.getMetadata())

      Then

      metadata = butler.get('raw.visitInfo', dataId=data_id, collections=["LSSTComCam/raw/all"])
      		 
      print(metadata)

      Which yields:

      VisitInfo(exposureId=2021102100008000, exposureTime=15, darkTime=15.5254, date=2021-10-21T15:44:59.077083175, UT1=nan, ERA=4.64471 rad, boresightRaDec=(nan, +nan), boresightAzAlt=(nan, +nan), boresightAirmass=nan, boresightRotAngle=0 rad, rotType=0, observatory=nanW, nanN  nan, weather=Weather(nan, nan, nan), instrumentLabel='LSSTComCam', id=2021102100008)

        Attachments

          Issue Links

          is triggering

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-33387 Update ComCam translator once CAP-807 and CAP-808 are done

          • To Do
          mentioned in

          Page Loading...

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            tribeiro Tiago Ribeiro added a comment -

            Output of astrometadata

            $ astrometadata -p lsst.obs.lsst.translators translate /data/lsstdata/base/comcam/oods/gen3butler/raw/2021-10-21/CC_O_20211021_000008-R22S00.fits
            		 
            Analyzing /data/lsstdata/base/comcam/oods/gen3butler/raw/2021-10-21/CC_O_20211021_000008-R22S00.fits...
            		 
            instrument: LSSTComCam
            		 
            telescope: Simonyi Survey Telescope
            		 
            datetime_begin: 2021-10-21T15:44:51.361
            		 
            altaz_begin: None
            		 
            boresight_airmass: None
            		 
            boresight_rotation_angle: 0.0 deg
            		 
            boresight_rotation_coord: unknown
            		 
            dark_time: 15.5253660678864 s
            		 
            datetime_end: 2021-10-21T15:45:06.793
            		 
            detector_exposure_id: 2021102100008000
            		 
            detector_group: R22
            		 
            detector_name: S00
            		 
            detector_num: 0
            		 
            detector_serial: ITL-3800C-229
            		 
            detector_unique_name: R22_S00
            		 
            exposure_group: 2021-10-21T15:44:51.123
            		 
            exposure_id: 2021102100008
            		 
            exposure_time: 15.0 s
            		 
            location: None
            		 
            object: UNKNOWN
            		 
            observation_counter: 8
            		 
            observation_id: CC_O_20211021_000008
            		 
            observation_reason: object
            		 
            observation_type: science
            		 
            observing_day: 20211021
            		 
            physical_filter: i_06
            		 
            pressure: None
            		 
            relative_humidity: None
            		 
            science_program: unknown
            		 
            temperature: None
            		 
            tracking_radec: None
            		 
            visit_id: 2147606911230000

            Show
            tribeiro Tiago Ribeiro added a comment - Output of astrometadata $ astrometadata -p lsst.obs.lsst.translators translate /data/lsstdata/base/comcam/oods/gen3butler/raw/2021-10-21/CC_O_20211021_000008-R22S00.fits Analyzing /data/lsstdata/base/comcam/oods/gen3butler/raw/2021-10-21/CC_O_20211021_000008-R22S00.fits... instrument: LSSTComCam telescope: Simonyi Survey Telescope datetime_begin: 2021-10-21T15:44:51.361 altaz_begin: None boresight_airmass: None boresight_rotation_angle: 0.0 deg boresight_rotation_coord: unknown dark_time: 15.5253660678864 s datetime_end: 2021-10-21T15:45:06.793 detector_exposure_id: 2021102100008000 detector_group: R22 detector_name: S00 detector_num: 0 detector_serial: ITL-3800C-229 detector_unique_name: R22_S00 exposure_group: 2021-10-21T15:44:51.123 exposure_id: 2021102100008 exposure_time: 15.0 s location: None object: UNKNOWN observation_counter: 8 observation_id: CC_O_20211021_000008 observation_reason: object observation_type: science observing_day: 20211021 physical_filter: i_06 pressure: None relative_humidity: None science_program: unknown temperature: None tracking_radec: None visit_id: 2147606911230000
            Hide
            Permalink
            tribeiro Tiago Ribeiro added a comment -

            We are using w_2021_39

            Show
            tribeiro Tiago Ribeiro added a comment - We are using w_2021_39
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            I believe the issue is that there is a TSTAND header and the translator assumes that any instrument on a test stand can't possibly be looking at sky. The fix would be to add a new implementation of _is_on_mountain in comcam translator but that is a bit of a sledgehammer approach. This does make clear that we have to solve the issue of simulated vs real on-sky data somehow on another ticket.

            Show
            tjenness Tim Jenness added a comment - I believe the issue is that there is a TSTAND header and the translator assumes that any instrument on a test stand can't possibly be looking at sky. The fix would be to add a new implementation of _is_on_mountain in comcam translator but that is a bit of a sledgehammer approach. This does make clear that we have to solve the issue of simulated vs real on-sky data somehow on another ticket.
            Hide
            Permalink
            tribeiro Tiago Ribeiro added a comment - - edited

            Not sure that is true though... We can simulate that we are on sky on the test stands... Which brings us to CAP-807 and CAP-808 I guess, as you mentioned...

            Show
            tribeiro Tiago Ribeiro added a comment - - edited Not sure that is true though... We can simulate that we are on sky on the test stands... Which brings us to CAP-807 and CAP-808 I guess, as you mentioned...
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            Yes, simulation would be better for this but TSTAND was the only knob I had to indicate that we weren't on sky.

            Show
            tjenness Tim Jenness added a comment - Yes, simulation would be better for this but TSTAND was the only knob I had to indicate that we weren't on sky.
            Hide
            Permalink
            mfisherlevine Merlin Fisher-Levine added a comment -

            So the workaround fix is pushed to this branch, but I'm guessing we shouldn't be merging that fix until we've sorted out CAP-808 and know how to do this in a way that doesn't make it look like all ComCam data is now on the sky, right?

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - So the workaround fix is pushed to this branch, but I'm guessing we shouldn't be merging that fix until we've sorted out CAP-808 and know how to do this in a way that doesn't make it look like all ComCam data is now on the sky, right?
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            Okay.

            Show
            tjenness Tim Jenness added a comment - Okay.

              People

              Assignee:
              mfisherlevine Merlin Fisher-Levine
              Reporter:
              tribeiro Tiago Ribeiro
              Reviewers:
              Tim Jenness
              Watchers:
              Alysha Shugart, Brian Stalder, Merlin Fisher-Levine, Tiago Ribeiro, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.