Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8258

Curate and process some public DECam COSMOS data on Verification Cluster

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Gather partial COSMOS dataset including 291 public DECam raw data files from Feb 2013. Process them on Verification Cluster using pipe_drivers tasks as exercises.

        Attachments

          Issue Links

          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-8681 Implement RFC-263: Add DECam COSMOS data to /datasets/decam/

          • Done

            Activity

            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            Obtained and transferred those COSMOS raw files to Verification Cluster as well as a set of MasterCal bias/flat files to go with the raw. Ingested and made a Butler repo /scratch/hchiang2/COSMOS. Ran singleFrameDriver.py with them using /datasets/refcats/gaia_DR1_v1 as the reference catalog. The stack w_2016_46 was used. Needed to hack around the problem that the logic of fixing validity in ingestCalibs does not ignore filter for bias, while in reality the FITS header of some bias MasterCal from NOAO can be anything and confuse the validity ranges in the registry.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Obtained and transferred those COSMOS raw files to Verification Cluster as well as a set of MasterCal bias/flat files to go with the raw. Ingested and made a Butler repo /scratch/hchiang2/COSMOS. Ran singleFrameDriver.py with them using /datasets/refcats/gaia_DR1_v1 as the reference catalog. The stack w_2016_46 was used. Needed to hack around the problem that the logic of fixing validity in ingestCalibs does not ignore filter for bias, while in reality the FITS header of some bias MasterCal from NOAO can be anything and confuse the validity ranges in the registry.

              People

              Assignee:
              hchiang2 Hsin-Fang Chiang
              Reporter:
              hchiang2 Hsin-Fang Chiang
              Watchers:
              Hsin-Fang Chiang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.