Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-25407

ap_verify cannot handle curated crosstalk data in Gen 2

    XMLWordPrintable

Details

    • Bug
    • Status: Done
    • Resolution: Done
    • None
    • ap_verify
    • None
    • 1
    • AP F20-1 (June)
    • Alert Production
    • No

    Description

      Following the merge of DM-23983, ap_verify crashes because it only ingests defects, and not other types of curated calibrations (in particular, crosstalk coefficients). Modify ap.verify.ingestion.DatasetIngestTask to process 0 or more curated calibrations in the format expected by IngestCuratedCalibsTask.

      In addition to updating the (obs_lsst, obs_decam, obs_subaru) package override files for DatasetIngestConfig, double-check that this field is not overridden in the four existing datasets.

      Since this will break all configs for DatasetIngestTask anyway, also rename textDefectPath and defectIngester to curatedCalibPaths and curatedCalibIngester, respectively.

      Attachments

        Issue Links

          Activity

            I ended up not needing to make any changes to obs_lsst (it doesn't have a config file yet), and DECam is the only instrument that actually has the new crosstalk files at the moment. Changes are otherwise as in the issue description.

            krzys Krzysztof Findeisen added a comment - I ended up not needing to make any changes to obs_lsst (it doesn't have a config file yet), and DECam is the only instrument that actually has the new crosstalk files at the moment. Changes are otherwise as in the issue description.

            Thanks for addressing this promptly! Just minor GitHub comments. (I think I found all the PRs, Jira was a little slow to discover them, let me know if I missed one.)

            Please do run ap_verify on at least the CI-sized datasets to ensure it works now. We should keep an eye on this for whenever DM-25348 is merged and inter-CCD crosstalk is implemented in Gen 3.

            mrawls Meredith Rawls added a comment - Thanks for addressing this promptly! Just minor GitHub comments. (I think I found all the PRs, Jira was a little slow to discover them, let me know if I missed one.) Please do run ap_verify on at least the CI-sized datasets to ensure it works now. We should keep an eye on this for whenever DM-25348 is merged and inter-CCD crosstalk is implemented in Gen 3.

            Thanks for the quick review! I've already tested both of the CI datasets and they both work.

            krzys Krzysztof Findeisen added a comment - Thanks for the quick review! I've already tested both of the CI datasets and they both work.

            People

              krzys Krzysztof Findeisen
              krzys Krzysztof Findeisen
              Meredith Rawls
              Christopher Waters, Kian-Tat Lim, Krzysztof Findeisen, Meredith Rawls
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.