Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11371

Put HiTS datasets in /datasets

    XMLWordPrintable

    Details

      Description

      Decide what to include (probably just the overlap fields being used in verify_ap?), issue RFC, and package according to https://developer.lsst.io/services/datasets.html

        Attachments

          Issue Links

          relates to

          RFC - An issue whose primary purpose is to advertise an intent and allow stakeholders to comment. RFC-372 Add HiTS 2014, 2015 DECam data to /datasets

          • Implemented

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-11575 Update packaging layout for ap_verify

          • Done

            Activity

            Ascending order - Click to sort in descending order
            4 older comments
            Hide
            Permalink
            reiss David Reiss added a comment - - edited

            Implementation: getting these warnings for every raw image ingested:

            ingest.parse WARN: Unable to find value for ccdnum (derived from CCDNUM)
            ingest.parse WARN: Unable to find value for ccd (derived from CCDNUM)

            and for every calib:

            ingestCalibs.parse WARN: translate_ccdnum failed to translate ccdnum: CALIB_ID not found

            Also, the tu* calib files seem to be duplicates of others which are named c4d*. I will skip ingesting all tu* files.

            Show
            reiss David Reiss added a comment - - edited Implementation: getting these warnings for every raw image ingested: ingest.parse WARN: Unable to find value for ccdnum (derived from CCDNUM) ingest.parse WARN: Unable to find value for ccd (derived from CCDNUM) and for every calib: ingestCalibs.parse WARN: translate_ccdnum failed to translate ccdnum: CALIB_ID not found Also, the tu* calib files seem to be duplicates of others which are named c4d* . I will skip ingesting all tu* files.
            Hide
            Permalink
            reiss David Reiss added a comment -

            Colin Slater, I'm not sure how much of this needs to be reviewed, but would you mind checking off on this ticket? Let me know if there are any obvious issues with the ingested HiTS data.
            Thanks a lot!

            Show
            reiss David Reiss added a comment - Colin Slater , I'm not sure how much of this needs to be reviewed, but would you mind checking off on this ticket? Let me know if there are any obvious issues with the ingested HiTS data. Thanks a lot!
            Hide
            Permalink
            reiss David Reiss added a comment -

            Thanks, Hsin-Fang Chiang for reviewing the repo – I have marked RFC-372 as implemented and am now closing this ticket.

            Show
            reiss David Reiss added a comment - Thanks, Hsin-Fang Chiang for reviewing the repo – I have marked RFC-372 as implemented and am now closing this ticket.
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment - - edited

            Greg Daues may you please apply immutability to:

            • /datasets/decam/_internal/raw/hits
            • /datasets/decam/_internal/preprocessed/masterCal/hits/
            • /datasets/decam/_internal/calib/cpHits
            • /datasets/decam/20*
            • /datasets/decam/registry.sqlite3
            Show
            hchiang2 Hsin-Fang Chiang added a comment - - edited Greg Daues may you please apply immutability to: /datasets/decam/_internal/raw/hits /datasets/decam/_internal/preprocessed/masterCal/hits/ /datasets/decam/_internal/calib/cpHits /datasets/decam/20* /datasets/decam/registry.sqlite3
            Hide
            Permalink
            gdaues Greg Daues added a comment -

            These files & directories are now immutable and ready for use.

            Show
            gdaues Greg Daues added a comment - These files & directories are now immutable and ready for use.

              People

              Assignee:
              reiss David Reiss
              Reporter:
              ebellm Eric Bellm
              Reviewers:
              Hsin-Fang Chiang
              Watchers:
              Colin Slater, David Reiss, Eric Bellm, Greg Daues, Hsin-Fang Chiang, John Swinbank
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.