Details
-
Type:
Bug
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: jenkins, sconsUtils
-
Labels:None
-
Team:Architecture
-
Urgent?:No
Description
From time to time (used to be less than once a month, but now we've had several failures in the last few weeks), we get errors in coverage.py like:
INTERNALERROR> coverage.misc.CoverageException: Couldn't use data file '/j/ws/scipipe/lsst_distrib/adacff179f/lsstsw/build/base/.coverage.cb1b93ad0284.83365.194071': no such table: file
The meta table is also implicated sometimes. Each such failure kills the build.
The file in question is a SQLite database. When checked after the failure, the file does contain the table, suggesting that this is a race condition.
Find and fix the problem.
Attachments
Activity
Looks like it's a race condition between xdist and pytest-cov. It mostly works but the coverage people seem to be saying it's not their problem. They suggested one fix of patching every setup.cfg. The other option we have is to disable coverage for multiprocessing testing and only turn on coverage in sconsUtils for single process testing.
I now believe this is a race between pySingles tests and the auto-discovered pytest-cov tests, since it only appears to happen in packages that have the former.
Attempting to fix by sequentially executing the pySingles tests before the auto-discovered ones.
Seen in #32386 on Mac: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/32386/pipeline