Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8258

Curate and process some public DECam COSMOS data on Verification Cluster

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Gather partial COSMOS dataset including 291 public DECam raw data files from Feb 2013. Process them on Verification Cluster using pipe_drivers tasks as exercises.

        Attachments

          Issue Links

            Activity

            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Obtained and transferred those COSMOS raw files to Verification Cluster as well as a set of MasterCal bias/flat files to go with the raw. Ingested and made a Butler repo /scratch/hchiang2/COSMOS. Ran singleFrameDriver.py with them using /datasets/refcats/gaia_DR1_v1 as the reference catalog. The stack w_2016_46 was used. Needed to hack around the problem that the logic of fixing validity in ingestCalibs does not ignore filter for bias, while in reality the FITS header of some bias MasterCal from NOAO can be anything and confuse the validity ranges in the registry.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Obtained and transferred those COSMOS raw files to Verification Cluster as well as a set of MasterCal bias/flat files to go with the raw. Ingested and made a Butler repo /scratch/hchiang2/COSMOS. Ran singleFrameDriver.py with them using /datasets/refcats/gaia_DR1_v1 as the reference catalog. The stack w_2016_46 was used. Needed to hack around the problem that the logic of fixing validity in ingestCalibs does not ignore filter for bias, while in reality the FITS header of some bias MasterCal from NOAO can be anything and confuse the validity ranges in the registry.

              People

              • Assignee:
                hchiang2 Hsin-Fang Chiang
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Watchers:
                Hsin-Fang Chiang
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel