# ci_hsc intermittently fails with "skymap in packages" failed

XMLWordPrintable

## Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Team:
External

## Description

This appears to be due to a race condition in writing the packages. lsst.skymap is only import-ed before Task execution once: in makeSkyMap.py. If that runs at the same time as something else that's updating the packages, the skymap package version could be overwritten.

## Activity

Hide
Paul Price added a comment -

This has passed Jenkins. Well, once anyway. We'd need to run it many times before a race condition reveals itself, but at least we know the changes haven't broken things.

Show
Paul Price added a comment - This has passed Jenkins . Well, once anyway. We'd need to run it many times before a race condition reveals itself, but at least we know the changes haven't broken things.
Hide
Paul Price added a comment -

Nate Lust, since you're working in ci_hsc at the moment, would you mind reviewing this?

 price@pap-laptop:~/LSST/ci_hsc (tickets/DM-11375=) $git sub commit c17b3a9f3fe6716221486f6d7be0d8bde17deb7d Author: Paul Price  Date: Tue Jul 25 15:52:21 2017 -0400    protect against race conditions in package versions    We're getting rare, intermittent failures:  AssertionError: Failed test: skymap in packages  This appears to be due to a race condition in writing the package verisons.  lsst.skymap is only imported before Task execution (and therefore subject to  having its version recorded) once: in makeSkyMap.py. If that runs at the same  time as something else that's updating the package versions, the skymap  package version could be overwritten. This patch forces skymap generation to  be done early, and in serial to avoid race conditions.    SConstruct | 16 +++++++++-------  1 file changed, 9 insertions(+), 7 deletions(-)  Show Paul Price added a comment - Nate Lust , since you're working in ci_hsc at the moment, would you mind reviewing this? price@pap-laptop:~/LSST/ci_hsc (tickets/DM-11375=)$ git sub commit c17b3a9f3fe6716221486f6d7be0d8bde17deb7d Author: Paul Price <price@astro.princeton.edu> Date: Tue Jul 25 15:52:21 2017 -0400   protect against race conditions in package versions We're getting rare, intermittent failures: AssertionError: Failed test: skymap in packages This appears to be due to a race condition in writing the package verisons. lsst.skymap is only imported before Task execution (and therefore subject to having its version recorded) once: in makeSkyMap.py. If that runs at the same time as something else that's updating the package versions, the skymap package version could be overwritten. This patch forces skymap generation to be done early, and in serial to avoid race conditions.   SConstruct | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-)
Hide
Nate Lust added a comment -

logical flow looks fine to me

Show
Nate Lust added a comment - logical flow looks fine to me
Hide
Paul Price added a comment -

Thanks Nate.

Merged to master.

Show
Paul Price added a comment - Thanks Nate. Merged to master.

## People

• Assignee:
Paul Price
Reporter:
Paul Price
Reviewers:
Nate Lust
Watchers:
Nate Lust, Paul Price