# Create gen3 cfht testdata_jointcal repos

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
6
• Sprint:
AP F21-6 (November), AP S22-1 (December)
• Team:
• Urgent?:
No

#### Description

The decam and cfht gen2 test repos in testdata_jointcal are used in a variety of jointcal tests, as is a tiny "cfht_minimal" repo inside of jointcal/tests. Those need to be converted to gen3 so that jointcal can switch the existing tests to gen3.

This ticket is for just the conversion of the data, not for the test code itself. The existing HSC gen2->gen3 conversion code in testdata_jointcal will be a useful start for this, but may require some tweaking. DECam should be ok, but I don't know if anyone has tried converting a CFHT repo yet.

This ticket is for conversion of the CFHT data.

#### Activity

Hide
Tim Jenness added a comment -

See DM-21865 – there is a lot of cfht support still missing.

Show
Tim Jenness added a comment - See DM-21865 – there is a lot of cfht support still missing.
Hide
John Parejko added a comment -

I'd started this, but the difficulty of converting the decam/cfht data to gen3 is making me put it on pause. Two branches have some pushes (including updated gen2 file templates for obs_decam).

Probably the best long-term solution would be to figure out how to run single frame processing on validation_data_[decam|cfht], so I can just use the output directly, instead of trying to convert the gen2 runs. I'm going to see about getting some help with that before I look at this again.

Show
John Parejko added a comment - I'd started this, but the difficulty of converting the decam/cfht data to gen3 is making me put it on pause. Two branches have some pushes (including updated gen2 file templates for obs_decam). Probably the best long-term solution would be to figure out how to run single frame processing on validation_data_ [decam|cfht] , so I can just use the output directly, instead of trying to convert the gen2 runs. I'm going to see about getting some help with that before I look at this again.
Hide
John Parejko added a comment -

Eli Rykoff: are you able to review this CFHT update to testdata_jointcal, plus a tweak to fgcmcal to reflect the re-arranged HSC data? The testdata commits are probably best reviewed individually, so you can see the script and README changes separately from the new/moved files.

This approach depends on the output of DM-32373, which gives a way to run validation_data_cfht in gen3. Suggestions on how to adjust the testdata_jointcal readme or whether you think the approach in the export/compress CFHT scripts involves too much butler path/file mangling and how it might be done better are very welcome.

Relevant PRs:

Show
John Parejko added a comment - Eli Rykoff : are you able to review this CFHT update to testdata_jointcal, plus a tweak to fgcmcal to reflect the re-arranged HSC data? The testdata commits are probably best reviewed individually, so you can see the script and README changes separately from the new/moved files. This approach depends on the output of DM-32373 , which gives a way to run validation_data_cfht in gen3. Suggestions on how to adjust the testdata_jointcal readme or whether you think the approach in the export/compress CFHT scripts involves too much butler path/file mangling and how it might be done better are very welcome. Relevant PRs: https://github.com/lsst/testdata_jointcal/pull/35 https://github.com/lsst/fgcmcal/pull/76

#### People

Assignee:
John Parejko
Reporter:
John Parejko
Reviewers:
Eli Rykoff
Watchers:
Eli Rykoff, Ian Sullivan, John Parejko, Tim Jenness