Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-29535

Create gen3 cfht testdata_jointcal repos

    XMLWordPrintable

    Details

    • Story Points:
      6
    • Sprint:
      AP F21-6 (November), AP S22-1 (December)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      The decam and cfht gen2 test repos in testdata_jointcal are used in a variety of jointcal tests, as is a tiny "cfht_minimal" repo inside of `jointcal/tests`. Those need to be converted to gen3 so that jointcal can switch the existing tests to gen3.

      This ticket is for just the conversion of the data, not for the test code itself. The existing HSC gen2->gen3 conversion code in testdata_jointcal will be a useful start for this, but may require some tweaking. DECam should be ok, but I don't know if anyone has tried converting a CFHT repo yet.

      This ticket is for conversion of the CFHT data.

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            See DM-21865 – there is a lot of cfht support still missing.

            Show
            tjenness Tim Jenness added a comment - See DM-21865 – there is a lot of cfht support still missing.
            Hide
            Parejkoj John Parejko added a comment -

            I'd started this, but the difficulty of converting the decam/cfht data to gen3 is making me put it on pause. Two branches have some pushes (including updated gen2 file templates for obs_decam).

            Probably the best long-term solution would be to figure out how to run single frame processing on validation_data_[decam|cfht], so I can just use the output directly, instead of trying to convert the gen2 runs. I'm going to see about getting some help with that before I look at this again.

            Show
            Parejkoj John Parejko added a comment - I'd started this, but the difficulty of converting the decam/cfht data to gen3 is making me put it on pause. Two branches have some pushes (including updated gen2 file templates for obs_decam). Probably the best long-term solution would be to figure out how to run single frame processing on validation_data_ [decam|cfht] , so I can just use the output directly, instead of trying to convert the gen2 runs. I'm going to see about getting some help with that before I look at this again.
            Hide
            Parejkoj John Parejko added a comment -

            Eli Rykoff: are you able to review this CFHT update to testdata_jointcal, plus a tweak to fgcmcal to reflect the re-arranged HSC data? The testdata commits are probably best reviewed individually, so you can see the script and README changes separately from the new/moved files.

            This approach depends on the output of DM-32373, which gives a way to run validation_data_cfht in gen3. Suggestions on how to adjust the testdata_jointcal readme or whether you think the approach in the export/compress CFHT scripts involves too much butler path/file mangling and how it might be done better are very welcome.

            Relevant PRs:

            Show
            Parejkoj John Parejko added a comment - Eli Rykoff : are you able to review this CFHT update to testdata_jointcal, plus a tweak to fgcmcal to reflect the re-arranged HSC data? The testdata commits are probably best reviewed individually, so you can see the script and README changes separately from the new/moved files. This approach depends on the output of DM-32373 , which gives a way to run validation_data_cfht in gen3. Suggestions on how to adjust the testdata_jointcal readme or whether you think the approach in the export/compress CFHT scripts involves too much butler path/file mangling and how it might be done better are very welcome. Relevant PRs: https://github.com/lsst/testdata_jointcal/pull/35 https://github.com/lsst/fgcmcal/pull/76

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              Eli Rykoff
              Watchers:
              Eli Rykoff, Ian Sullivan, John Parejko, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins Builds

                  No builds found.