Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: obs_lsst
-
Labels:None
-
Story Points:1
-
Team:Architecture
-
Urgent?:No
Description
pytest as run by the tests SConscript attempts to scan all files, but that set can be in flux as the per-camera ingestCuratedCalibs.py executions can be running at the same time. There are no dependencies between the two.
I'm not sure if the correct solution is to sequence one before the other or to exclude those directories from pytest.
Attachments
Activity
I don't believe so. My only thought is that if new calibrations were added, that might have increased the execution time long enough to start this being a problem, but I don't recall anything new being added in the past month.
ingestCuratedCalibs.py is part of the build (SConscripts in the camera directories), not the tests.
This comment:
# Note the ordering here is critical. LATISS is put at the end here to ensure
|
# that the tests are run first and version.py is created, because creation of
|
# of the defect registry required the camera to be instantiated.
|
# If other cameras add defect generation they should add their build to
|
# the end of this list, along with LATISS
|
in the SConstruct file implies that these targets explicitly run after tests (they do also hard code a dependency on the python target) suggesting that the answer is to change the scanning code. Can someone point to an actual error report from this problem? Was there a Jenkins failure I can look at?
https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix-test/detail/stack-os-matrix-test/15/tests is an example of a failure.
My understanding from a quick glance at sconsUtils (https://github.com/lsst/sconsUtils/blob/master/python/lsst/sconsUtils/scripts.py#L208-L209) is that there is in fact no implied dependency ordering from the target list alone; any dependencies must be added separately.
In the end I added the directories to the ignore list when running pytest.
There is also an extra unrelated fix to header stuff that I saw whilst testing this but I can remove that if needed. It's a gen2-specific problem where the curated calibs ingest calls normal raw data ingest which now calls fix_header and that causes and extra log message because curated calibrations say they are LATISS but they aren't raw latiss and header fixup gets confused.
Looks fine. Thanks for dealing with this.
Christopher Waters Are these your tests? Thoughts?