Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8681

Implement RFC-263: Add DECam COSMOS data to /datasets/decam/

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      Transfer Feb2013 COSMOS data to /datasets/decam/ and butlerize it.

      Run some processing to verify the butler repo is useable.

        Attachments

          Issue Links

            Activity

            Hide
            jalt Jason Alt [X] (Inactive) added a comment -

            Locked down and applied sharing permissions

            Show
            jalt Jason Alt [X] (Inactive) added a comment - Locked down and applied sharing permissions
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            I'm done; raw, calib, and repo in /datasets/decam can be locked down.

            Please make /datasets/decam/repo/rerun/private sticky.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - I'm done; raw , calib , and repo in /datasets/decam can be locked down. Please make /datasets/decam/repo/rerun/private sticky.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            The 291 visits include 39 Y, 34 u, 21 g, 12 r, 87 z, and 98 i -band. In total there are 291*61 = 17751 images.

            I processed these cosmos data using stack w_2016_52 and the u/hfc/DM-8681 branch in obs_decam; ran pipe_drivers singleFrameDriver task with the reference catalog at /datasets/refcats/astrometry_net_data/2mass.20130404 using the slurm cluster. This produced 12050 calexps. The most common failure was "Unable to match sources" in astrometry (4133), but also got other errors such as too few matches, no viable psf candidates, and so on (I'll leave QA to future tickets as QA is not the goal of this ticket)

            Show
            hchiang2 Hsin-Fang Chiang added a comment - The 291 visits include 39 Y, 34 u, 21 g, 12 r, 87 z, and 98 i -band. In total there are 291*61 = 17751 images. I processed these cosmos data using stack w_2016_52 and the u/hfc/ DM-8681 branch in obs_decam; ran pipe_drivers singleFrameDriver task with the reference catalog at /datasets/refcats/astrometry_net_data/2mass.20130404 using the slurm cluster. This produced 12050 calexps. The most common failure was "Unable to match sources" in astrometry (4133), but also got other errors such as too few matches, no viable psf candidates, and so on (I'll leave QA to future tickets as QA is not the goal of this ticket)
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Ingest/butlerize using stack w_2016_51 with a hack in obs_decam ingestCalibs that ensures the "filter" column in the calibRegistry always be "NONE" for all bias masterCal data.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Ingest/butlerize using stack w_2016_51 with a hack in obs_decam ingestCalibs that ensures the "filter" column in the calibRegistry always be "NONE" for all bias masterCal data.

              People

              • Assignee:
                hchiang2 Hsin-Fang Chiang
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Reviewers:
                Jason Alt [X] (Inactive)
                Watchers:
                Hsin-Fang Chiang
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel