# Convert tests in obs_decam from gen2 to gen3 and remove gen2.

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Story Points:
3
• Team:
Data Release Production
• Urgent?:
No

#### Description

After DM-34862, the tests in obs_decam can be migrated from gen2 to gen3.

These tests should use the SimplePipelineExecutor to do calibration lookups and exercise basic ISR functionality including crosstalk. The current nopytest_test_processCcd.py is slow (it goes all the way to calexp) and mostly unnecessary and will be removed. Within that test file is testWcsPostIsr which is essential to maintain. This ticket will create a new test_isr.py which will replace test_crosstalk.py and include the necessary crosstalk and wcs (and any other tests that require decam ISR).

Other tests that touch gen2-only features (like testing mappers) will be removed, and tests will be migrated to gen3 as appropriate.

#### Activity

Hide
Tim Jenness added a comment -

You can also remove the gen2 code on this ticket if you like.

Show
Tim Jenness added a comment - You can also remove the gen2 code on this ticket if you like.
Hide
Lee Kelvin added a comment -

Thanks for going through this Eli, this ticket is in great shape. Old gen2 docs/code have been removed and unit tests have been updated to gen3. I don't have any major comments to add on the PR.

As a test, I git cloned and built obs_decam on the main branch and on this ticket branch. The main branch builds in ~6m, with an install size of 382M. This ticket branch builds in ~3m with an install size of 184M, so a significant improvement on both fronts.

As a further test, I ran my example Merian DECam data through step 1 processing. The run command is similar to this:

 pipetask --long-log run --register-dataset-types -j 12 -b /project/lskelvin/repo --instrument lsst.obs.decam.DarkEnergyCamera -i DECam/defaults/merian9813 -o u/lskelvin/scratch/tickets/DM-34863-ticket -p $DRP_PIPE_DIR/pipelines/DECam/DRP-Merian.yaml#step1 -d “instrument=‘DECam’ AND visit=971666”  I confirm that the PVI outputs are identical between the vanilla and ticket branch outputs, so no cause for concern there. Assuming Jenkins doesn't complain, this look good to merge to me. Thanks again Eli! Show Lee Kelvin added a comment - Thanks for going through this Eli, this ticket is in great shape. Old gen2 docs/code have been removed and unit tests have been updated to gen3. I don't have any major comments to add on the PR. As a test, I git cloned and built obs_decam on the main branch and on this ticket branch. The main branch builds in ~6m, with an install size of 382M. This ticket branch builds in ~3m with an install size of 184M, so a significant improvement on both fronts. As a further test, I ran my example Merian DECam data through step 1 processing. The run command is similar to this: pipetask --long-log run --register-dataset-types -j 12 -b /project/lskelvin/repo --instrument lsst.obs.decam.DarkEnergyCamera -i DECam/defaults/merian9813 -o u/lskelvin/scratch/tickets/DM-34863-ticket -p$DRP_PIPE_DIR/pipelines/DECam/DRP-Merian.yaml#step1 -d “instrument=‘DECam’ AND visit=971666” I confirm that the PVI outputs are identical between the vanilla and ticket branch outputs, so no cause for concern there. Assuming Jenkins doesn't complain, this look good to merge to me. Thanks again Eli!
Hide
Eli Rykoff added a comment -

Thanks so much! I think those sizes include the source + the .git tree. Looking at jenkins (from e.g. https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/36966/pipeline/46) the install size goes from 300 Mb (before) to 5.6 Mb (after).

Show
Eli Rykoff added a comment - Thanks so much! I think those sizes include the source + the .git tree. Looking at jenkins (from e.g. https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/36966/pipeline/46 ) the install size goes from 300 Mb (before) to 5.6 Mb (after).

#### People

Assignee:
Eli Rykoff
Reporter:
Eli Rykoff
Reviewers:
Lee Kelvin
Watchers:
Eli Rykoff, John Parejko, Lee Kelvin, Tim Jenness