Details
-
Type:
Bug
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: jenkins
-
Labels:None
-
Team:Architecture
-
Urgent?:No
Description
The release process is creating a manifest that uses eups package versions that include 22.0.1 (e.g. versiondb b5788). But when it later goes to do builds, those stacks seem to setup some packages with versions that use 22.0.0 (and do not include the build tag such as b5789). This carries over to the publish step and the exact versions in expanded table files, causing simple setup commands to fail in the resulting containers.
Attachments
Issue Links
- blocks
-
DM-29437 Do Release 23.0.0 of Science Pipelines
- Done
Activity
OK, it seems that the versions would be consistent except that the release workspace has the weekly build in it, and that has the same version of lsst_distrib but with different versions of the dependencies in its exact table file. So when setting up lsst_distrib ${LSST_DISTRIB_VERSION}, versions that are not in the manifest are showing up.
This now goes in the "how did this ever work" folder.
It seems one solution could be to have pkgautoversion return the same results for the release tag as for the weekly HEAD. Another is to build the official release in a clean workspace with no pre-existing stack package installs. But I'd still like to figure out how this used to work.
After some thought, I think the problem may be related to the patch release and not having any intermediate changes (although it's not clear that we would have always had changes between major releases before).
pkgautoversion does not seem to be doing what it claims to (using the lowest-numbered tag), so it seems like that would be the best place for the fix.
Looks like there are two potential fixes to pkgautoversion, both in line 147:
- Replace $REF with $(git rev-parse "$REF^{commit}")
- Replace --points-at with --contains
The second obviously seems simpler, but I'm not 100% sure it's correct in all cases.
Tim Jenness, can you check on this?
Is there an example package that has a REF that fails with the current code and works with the new?
I assume the code that needs to be modified is this:
git tag -l "$TAG_PATTERN" --points-at $REF |
to either be:
git tag -l "$TAG_PATTERN" --points-at $(git rev-parse "$REF^{commit}") |
or
git tag -l "$TAG_PATTERN" --contains $REF |
Almost any package should show this problem. pex_exceptions is the first that is encountered.
I'm still not sure explicitly what the different output is from the `git tag` output before and after the change.
Basically, --points-at returns nothing.
$ git tag -l "[0-9.]*" --points-at v23.0.0.rc1
|
$ git tag -l "[0-9.]*" --contains v23.0.0.rc1
|
22.0.0
|
22.0.1
|
$ git tag -l "[0-9]*" --points-at $(git rev-parse "v23.0.0.rc1^{commit}")
|
22.0.0
|
22.0.1
|
It looks like bare --points-at would return the tag specified (and only that tag) if it matched the pattern, but that's not what is desired anyway. I believe this has to do with the difference between <object> and <ref> or <commit> as a git argument.
On the other hand, I'm now convinced that --contains is incorrect as well, as it will list later tags that contain the given ref, but those are inappropriate for versioning use. So I think we're down to the last alternative.
Looks okay since it seems to work. Do you need me to hit the merge button on github?
Yes, please, and a release, too.
New release pushed.
It appears that pkgautoversion returns different results when given a git ref (like v23.0.0.rc1). It's still not clear why those different results aren't consistently used everywhere.