# Create lsst_ci package as a continuous integration build target

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
1
• Team:
SQuaRE

#### Description

Create an lsst_ci package to be built for the continuous integration testing.

Plan:
1. Create empty package that has dependencies on obs_cfht, obs_decam, obs_subaru, testdata_cfht, testdata_decam, testdata_subaru.
2. Ensure above builds.
3. Add obs_lsstSim and ensure that it builds.

The following were moved to DM-5381:
[ Tim Jenness : How can I get strikethrough to work in the following list?]
3. Add dependencies on validation_data_cfht and validation_data_decam, and validate_drp.
4. Run CFHT, DECam quick examples in validate_drp.
5. Test for successful running of the above examples. Fail and trigger Jenkins FAILURE message if these examples fail.
6. Check performance of CFHT, DECam runs against reference numbers. Fail if there is a significant regression.
7. Decide how to include ci_hsc, which currently can take at least 30 minutes to process the image data.--

#### Activity

Hide
John Swinbank added a comment -

For the record, my understanding wasn't based on the discussion at DM-4251, but on a statement from SQuaRE about things they support.

I'm not sure I understand concerns around the size of testdata_ packages and lsst_distrib: isn't the raison d'être of testdata_ to make it possible to include the corresponding obs_ in a top level product without imposing undue storage requirements? Maybe that's what Kian-Tat Lim is getting at in the summary above, but it's not completely clear.

Show
John Swinbank added a comment - For the record, my understanding wasn't based on the discussion at DM-4251 , but on a statement from SQuaRE about things they support. I'm not sure I understand concerns around the size of testdata_ packages and lsst_distrib : isn't the raison d'être of testdata_ to make it possible to include the corresponding obs_ in a top level product without imposing undue storage requirements? Maybe that's what Kian-Tat Lim is getting at in the summary above, but it's not completely clear.
Hide
Michael Wood-Vasey [X] (Inactive) added a comment -

I'm concerned that putting, e.g., obs_cfht in lsst_distrib immediately puts testdata_cfht in there as well due to the setupOptional dependency.

I propose that the path forward for this present ticket, DM-5370, is to proceed with the plan outlined above, plus adding obs_lsstSim.

The remaining work is now split off into a new ticket, DM-5381.

Show
Michael Wood-Vasey [X] (Inactive) added a comment - I'm concerned that putting, e.g., obs_cfht in lsst_distrib immediately puts testdata_cfht in there as well due to the setupOptional dependency. I propose that the path forward for this present ticket, DM-5370 , is to proceed with the plan outlined above, plus adding obs_lsstSim . The remaining work is now split off into a new ticket, DM-5381 .
Hide
Michael Wood-Vasey [X] (Inactive) added a comment -

Quick review of new lsst_ci package.
https://github.com/lsst/lsst_ci/pull/1

Depends on obs_cfht, obs_decam, obs_subaru, obs_lsstSim
and their testdata
testdata_cfht, testdata_decam, testdata_subaru

Test of the utility and performance of the package as it currently stands:

1. Successfully fails on current master of obs_cfht, obs_decam.
https://ci.lsst.codes/job/stack-os-matrix/label=centos-6/8998/console

2. Successfully passes on tickets/DM-5372 of obs_cfht, obs_decam.
https://ci.lsst.codes/job/stack-os-matrix/9005/label=centos-6/console

Show
Michael Wood-Vasey [X] (Inactive) added a comment - Quick review of new lsst_ci package. https://github.com/lsst/lsst_ci/pull/1 Depends on obs_cfht, obs_decam, obs_subaru, obs_lsstSim and their testdata testdata_cfht, testdata_decam, testdata_subaru Test of the utility and performance of the package as it currently stands: 1. Successfully fails on current master of obs_cfht, obs_decam. https://ci.lsst.codes/job/stack-os-matrix/label=centos-6/8998/console 2. Successfully passes on tickets/ DM-5372 of obs_cfht, obs_decam. https://ci.lsst.codes/job/stack-os-matrix/9005/label=centos-6/console
Hide
Kian-Tat Lim added a comment -

I was going to say that if this is a pure meta-package, it doesn't need an SConstruct, .build, or .cfg. But our other meta-packages like lsst_distrib and lsst_apps do have those, so this looks fine.

Show
Kian-Tat Lim added a comment - I was going to say that if this is a pure meta-package, it doesn't need an SConstruct , .build , or .cfg . But our other meta-packages like lsst_distrib and lsst_apps do have those, so this looks fine.
Hide
Michael Wood-Vasey [X] (Inactive) added a comment -

Merged to master.

Show
Michael Wood-Vasey [X] (Inactive) added a comment - Merged to master.

#### People

Assignee:
Michael Wood-Vasey [X] (Inactive)
Reporter:
Michael Wood-Vasey [X] (Inactive)
Reviewers:
Kian-Tat Lim
Watchers:
Frossie Economou, John Swinbank, Kian-Tat Lim, Michael Wood-Vasey [X] (Inactive)