Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8681

Implement RFC-263: Add DECam COSMOS data to /datasets/decam/

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • None

    Description

      Transfer Feb2013 COSMOS data to /datasets/decam/ and butlerize it.

      Run some processing to verify the butler repo is useable.

      Attachments

        Issue Links

          Activity

            Ingest/butlerize using stack w_2016_51 with a hack in obs_decam ingestCalibs that ensures the "filter" column in the calibRegistry always be "NONE" for all bias masterCal data.

            hchiang2 Hsin-Fang Chiang added a comment - Ingest/butlerize using stack w_2016_51 with a hack in obs_decam ingestCalibs that ensures the "filter" column in the calibRegistry always be "NONE" for all bias masterCal data.

            The 291 visits include 39 Y, 34 u, 21 g, 12 r, 87 z, and 98 i -band. In total there are 291*61 = 17751 images.

            I processed these cosmos data using stack w_2016_52 and the u/hfc/DM-8681 branch in obs_decam; ran pipe_drivers singleFrameDriver task with the reference catalog at /datasets/refcats/astrometry_net_data/2mass.20130404 using the slurm cluster. This produced 12050 calexps. The most common failure was "Unable to match sources" in astrometry (4133), but also got other errors such as too few matches, no viable psf candidates, and so on (I'll leave QA to future tickets as QA is not the goal of this ticket)

            hchiang2 Hsin-Fang Chiang added a comment - The 291 visits include 39 Y, 34 u, 21 g, 12 r, 87 z, and 98 i -band. In total there are 291*61 = 17751 images. I processed these cosmos data using stack w_2016_52 and the u/hfc/ DM-8681 branch in obs_decam; ran pipe_drivers singleFrameDriver task with the reference catalog at /datasets/refcats/astrometry_net_data/2mass.20130404 using the slurm cluster. This produced 12050 calexps. The most common failure was "Unable to match sources" in astrometry (4133), but also got other errors such as too few matches, no viable psf candidates, and so on (I'll leave QA to future tickets as QA is not the goal of this ticket)

            I'm done; raw, calib, and repo in /datasets/decam can be locked down.

            Please make /datasets/decam/repo/rerun/private sticky.

            hchiang2 Hsin-Fang Chiang added a comment - I'm done; raw , calib , and repo in /datasets/decam can be locked down. Please make /datasets/decam/repo/rerun/private sticky.

            Locked down and applied sharing permissions

            jalt Jason Alt [X] (Inactive) added a comment - Locked down and applied sharing permissions

            People

              hchiang2 Hsin-Fang Chiang
              hchiang2 Hsin-Fang Chiang
              Jason Alt [X] (Inactive)
              Hsin-Fang Chiang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.