Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15937

Production-style processing of simulated imsim LSSTCam data

    Details

    • Epic Name:
      Production-style processing of simulated imsim LSSTCam data
    • Story Points:
      40
    • WBS:
      02C.07.06.01
    • Team:
      Data Facility
    • Cycle:
      Fall 2018

      Description

      Process a small set of imsim LSST simulated data, including setup, update, testing, execution, and debugging, and produce data products from a selected set of DRP pipelines of a stable Stack version. Use DESDM framework if possible but the DESDM framework is not required as the way to generate the data outputs.

        Attachments

          Activity

          Hide
          hchiang2 Hsin-Fang Chiang added a comment -

          In terms of ingesting data files into the DBB of the DESDM framework, the as-is packages were written for the HSC data specifically. Therefore we cannot ingest the imsim data (DM-16115) which blocks processing any imsim data in the DESDM framework.

          Michelle Gower was planning to use the new astro_metadata_translator for reading headers, and modify desdmfw_lsst_plugins or other packages so to allow imsim data ingestion. That work is waiting on the BG3 development. In the Batch Production Steering meeting on Nov 28 (with Margaret Gelman Michelle Butler Joel Plutchak Rob Kooper and many more) it was decided that DESDM is not required to perform this processing for this epic.  I'll update the stories here in this epic according to this decision. 

          Show
          hchiang2 Hsin-Fang Chiang added a comment - In terms of ingesting data files into the DBB of the DESDM framework, the as-is packages were written for the HSC data specifically. Therefore we cannot ingest the imsim data ( DM-16115 ) which blocks processing any imsim data in the DESDM framework. Michelle Gower was planning to use the new astro_metadata_translator for reading headers, and modify desdmfw_lsst_plugins or other packages so to allow imsim data ingestion. That work is waiting on the BG3 development. In the Batch Production Steering meeting on Nov 28 (with Margaret Gelman Michelle Butler Joel Plutchak Rob Kooper and many more) it was decided that DESDM is not required to perform this processing for this epic.  I'll update the stories here in this epic according to this decision. 
          Hide
          tjenness Tim Jenness added a comment -

          DM-16554 is the work on astro_metadata_translator. I've done Phosim and was working on TS8 today.

          Show
          tjenness Tim Jenness added a comment - DM-16554 is the work on astro_metadata_translator. I've done Phosim and was working on TS8 today.
          Hide
          hchiang2 Hsin-Fang Chiang added a comment -

          My understanding is the uncertainty of work needed on the DESDM side after astro_metadata_translator is ready can also be a concern.

          Show
          hchiang2 Hsin-Fang Chiang added a comment - My understanding is the uncertainty of work needed on the DESDM side after astro_metadata_translator is ready can also be a concern.
          Hide
          plutchak Joel Plutchak (Inactive) added a comment -

          Planned work has been completed. Datasets are in /datasets/DC2/repo/rerun/w49multiband/

          See DM-16956.

          Show
          plutchak Joel Plutchak (Inactive) added a comment - Planned work has been completed. Datasets are in /datasets/DC2/repo/rerun/w49multiband/ See DM-16956 .

            People

            • Assignee:
              hchiang2 Hsin-Fang Chiang
              Reporter:
              plutchak Joel Plutchak (Inactive)
              Watchers:
              Hsin-Fang Chiang, Joel Plutchak (Inactive), Tim Jenness
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Start date:
                End date:

                Summary Panel