Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11371

Put HiTS datasets in /datasets

    Details

      Description

      Decide what to include (probably just the overlap fields being used in verify_ap?), issue RFC, and package according to https://developer.lsst.io/services/datasets.html

        Attachments

          Issue Links

            Activity

            Hide
            reiss David Reiss added a comment - - edited

            Implementation: getting these warnings for every raw image ingested:

            ingest.parse WARN: Unable to find value for ccdnum (derived from CCDNUM)
            ingest.parse WARN: Unable to find value for ccd (derived from CCDNUM)

            and for every calib:

            ingestCalibs.parse WARN: translate_ccdnum failed to translate ccdnum: CALIB_ID not found

            Also, the tu* calib files seem to be duplicates of others which are named c4d*. I will skip ingesting all tu* files.

            Show
            reiss David Reiss added a comment - - edited Implementation: getting these warnings for every raw image ingested: ingest.parse WARN: Unable to find value for ccdnum (derived from CCDNUM) ingest.parse WARN: Unable to find value for ccd (derived from CCDNUM) and for every calib: ingestCalibs.parse WARN: translate_ccdnum failed to translate ccdnum: CALIB_ID not found Also, the tu* calib files seem to be duplicates of others which are named c4d* . I will skip ingesting all tu* files.
            Hide
            reiss David Reiss added a comment -

            Colin Slater, I'm not sure how much of this needs to be reviewed, but would you mind checking off on this ticket? Let me know if there are any obvious issues with the ingested HiTS data.
            Thanks a lot!

            Show
            reiss David Reiss added a comment - Colin Slater , I'm not sure how much of this needs to be reviewed, but would you mind checking off on this ticket? Let me know if there are any obvious issues with the ingested HiTS data. Thanks a lot!
            Hide
            reiss David Reiss added a comment -

            Thanks, Hsin-Fang Chiang for reviewing the repo – I have marked RFC-372 as implemented and am now closing this ticket.

            Show
            reiss David Reiss added a comment - Thanks, Hsin-Fang Chiang for reviewing the repo – I have marked RFC-372 as implemented and am now closing this ticket.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment - - edited

            Greg Daues may you please apply immutability to:

            • /datasets/decam/_internal/raw/hits
            • /datasets/decam/_internal/preprocessed/masterCal/hits/
            • /datasets/decam/_internal/calib/cpHits
            • /datasets/decam/20*
            • /datasets/decam/registry.sqlite3
            Show
            hchiang2 Hsin-Fang Chiang added a comment - - edited Greg Daues may you please apply immutability to: /datasets/decam/_internal/raw/hits /datasets/decam/_internal/preprocessed/masterCal/hits/ /datasets/decam/_internal/calib/cpHits /datasets/decam/20* /datasets/decam/registry.sqlite3
            Hide
            gdaues Greg Daues added a comment -

            These files & directories are now immutable and ready for use.

            Show
            gdaues Greg Daues added a comment - These files & directories are now immutable and ready for use.

              People

              • Assignee:
                reiss David Reiss
                Reporter:
                ebellm Eric Bellm
                Reviewers:
                Hsin-Fang Chiang
                Watchers:
                Colin Slater, David Reiss, Eric Bellm, Greg Daues, Hsin-Fang Chiang, John Swinbank
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel