Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5467

Distribute bad pixel masks in obs_decam

    Details

    • Story Points:
      4
    • Sprint:
      Alert Production F16 - 11, Alert Production F16 - 11b, Alert Production F16 - 11c, Alert Production S17 - 12, Alert Production S17 - 1, Alert Production S17 - 2, Alert Production F17 - 9, Alert Production F17 - 10, Alert Production F17 - 11
    • Team:
      Alert Production

      Description

      Processing Decam data from raw images currently requires downloading the bad pixel mask files from NOAO, then ingesting them into the calibration repository. It would be much more convenient if we included these with obs_decam itself, pending the ok to do so from NOAO. Our understanding is that we could create a default calibration repository in obs_decam, which would be the parent of users' calibration repos. We will also likely want to convert the provided files from mask images into tables of bounding boxes.

        Attachments

          Issue Links

            Activity

            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            A somewhat relevant thread: https://community.lsst.org/t/storing-an-instrument-model/679

            Maybe it's just me being confused; I wonder if this is something for RFC.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - A somewhat relevant thread: https://community.lsst.org/t/storing-an-instrument-model/679 Maybe it's just me being confused; I wonder if this is something for RFC.
            Hide
            mrawls Meredith Rawls added a comment - - edited

            Resurrecting this thread to remind, uh, myself that we ultimately want to be able to ingest decam defects with mode=link (or move or copy). The pipe_tasks ingestCalibs default is presently move, but it's necessary for now to ingest defects explicitly with mode=skip since they don't have the OBSTYPE header keyword that pipe_tasks' getDestination expects.

            Show
            mrawls Meredith Rawls added a comment - - edited Resurrecting this thread to remind, uh, myself that we ultimately want to be able to ingest decam defects with mode=link (or move or copy ). The pipe_tasks ingestCalibs default is presently move , but it's necessary for now to ingest defects explicitly with mode=skip since they don't have the OBSTYPE header keyword that pipe_tasks' getDestination expects.
            Hide
            ctslater Colin Slater added a comment -

            Bump. This issue is still really annoying and causes all sorts of bewildering errors if one doesn't ingest the defects just right.

            For instance, since the default mode for ingestCalibs.py is now move, defects can't be ingested unless one knows to add "mode=skip" (even though the error message is complaining about bad headers; Meredith Rawls's comment was the only way I figured this out). Next, the mapper now throws an error (during processCcd, not even on ingest) if the templated path is absolute rather than relative to the repo root. This means that one has to either copy the defects into the repo before ingestion, or ingest with a convoluted relative path to get to a fixed location outside the repo. This is all way more pain than it should be.

            Show
            ctslater Colin Slater added a comment - Bump. This issue is still really annoying and causes all sorts of bewildering errors if one doesn't ingest the defects just right. For instance, since the default mode for ingestCalibs.py is now move, defects can't be ingested unless one knows to add "mode=skip" (even though the error message is complaining about bad headers; Meredith Rawls 's comment was the only way I figured this out). Next, the mapper now throws an error (during processCcd, not even on ingest) if the templated path is absolute rather than relative to the repo root. This means that one has to either copy the defects into the repo before ingestion, or ingest with a convoluted relative path to get to a fixed location outside the repo. This is all way more pain than it should be.
            Hide
            krzys Krzysztof Findeisen added a comment - - edited

            Just read Meredith Rawls's last comment – the underlying bug with OBSTYPE is DM-13975.

            Show
            krzys Krzysztof Findeisen added a comment - - edited Just read Meredith Rawls 's last comment – the underlying bug with OBSTYPE is DM-13975 .
            Hide
            mrawls Meredith Rawls added a comment -

            This is finally being addressed by DM-19730! Rejoice!!

            Show
            mrawls Meredith Rawls added a comment - This is finally being addressed by DM-19730 ! Rejoice!!

              People

              • Assignee:
                mrawls Meredith Rawls
                Reporter:
                ctslater Colin Slater
                Watchers:
                Colin Slater, Hsin-Fang Chiang, Krzysztof Findeisen, Meredith Rawls, Simon Krughoff
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel