# Official release jobs are generating stacks with inconsistent eups versions

XMLWordPrintable

#### Details

• Type: Bug
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Team:
Architecture
• Urgent?:
No

#### Description

The release process is creating a manifest that uses eups package versions that include 22.0.1 (e.g. versiondb b5788). But when it later goes to do builds, those stacks seem to setup some packages with versions that use 22.0.0 (and do not include the build tag such as b5789). This carries over to the publish step and the exact versions in expanded table files, causing simple setup commands to fail in the resulting containers.

#### Activity

No builds found.
Kian-Tat Lim created issue -
Field Original Value New Value
Link This issue blocks DM-29437 [ DM-29437 ]
 Status To Do [ 10001 ] In Progress [ 3 ]
Hide
Kian-Tat Lim added a comment -

It appears that pkgautoversion returns different results when given a git ref (like v23.0.0.rc1). It's still not clear why those different results aren't consistently used everywhere.

Show
Kian-Tat Lim added a comment - It appears that pkgautoversion returns different results when given a git ref (like v23.0.0.rc1 ). It's still not clear why those different results aren't consistently used everywhere.
Hide
Kian-Tat Lim added a comment -

OK, it seems that the versions would be consistent except that the release workspace has the weekly build in it, and that has the same version of lsst_distrib but with different versions of the dependencies in its exact table file. So when setting up lsst_distrib ${LSST_DISTRIB_VERSION}, versions that are not in the manifest are showing up. This now goes in the "how did this ever work" folder. It seems one solution could be to have pkgautoversion return the same results for the release tag as for the weekly HEAD. Another is to build the official release in a clean workspace with no pre-existing stack package installs. But I'd still like to figure out how this used to work. Show Kian-Tat Lim added a comment - OK, it seems that the versions would be consistent except that the release workspace has the weekly build in it, and that has the same version of lsst_distrib but with different versions of the dependencies in its exact table file. So when setting up lsst_distrib${LSST_DISTRIB_VERSION }, versions that are not in the manifest are showing up. This now goes in the "how did this ever work" folder. It seems one solution could be to have pkgautoversion return the same results for the release tag as for the weekly HEAD . Another is to build the official release in a clean workspace with no pre-existing stack package installs. But I'd still like to figure out how this used to work.
Hide
Kian-Tat Lim added a comment -

After some thought, I think the problem may be related to the patch release and not having any intermediate changes (although it's not clear that we would have always had changes between major releases before).

pkgautoversion does not seem to be doing what it claims to (using the lowest-numbered tag), so it seems like that would be the best place for the fix.

Show
Kian-Tat Lim added a comment - After some thought, I think the problem may be related to the patch release and not having any intermediate changes (although it's not clear that we would have always had changes between major releases before). pkgautoversion does not seem to be doing what it claims to (using the lowest-numbered tag), so it seems like that would be the best place for the fix.
 Watchers Kian-Tat Lim, Yusra AlSayyad [ Kian-Tat Lim, Yusra AlSayyad ] Kian-Tat Lim, Tim Jenness, Yusra AlSayyad [ Kian-Tat Lim, Tim Jenness, Yusra AlSayyad ]
Hide
Kian-Tat Lim added a comment - - edited

Looks like there are two potential fixes to pkgautoversion, both in line 147:

1. Replace $REF with$(git rev-parse "$REF^{commit}") 2. Replace --points-at with --contains The second obviously seems simpler, but I'm not 100% sure it's correct in all cases. Tim Jenness, can you check on this? Show Kian-Tat Lim added a comment - - edited Looks like there are two potential fixes to pkgautoversion , both in line 147 : Replace$REF with $(git rev-parse "$REF^{commit}") Replace --points-at with --contains The second obviously seems simpler, but I'm not 100% sure it's correct in all cases. Tim Jenness , can you check on this?
Hide
Tim Jenness added a comment -

Is there an example package that has a REF that fails with the current code and works with the new?

I assume the code that needs to be modified is this:

 git tag -l "$TAG_PATTERN" --points-at$REF 

to either be:

 git tag -l "$TAG_PATTERN" --points-at$(git rev-parse "$REF^{commit}")  or  git tag -l "$TAG_PATTERN" --contains $REF  Show Tim Jenness added a comment - Is there an example package that has a REF that fails with the current code and works with the new? I assume the code that needs to be modified is this: git tag -l "$TAG_PATTERN" --points-at $REF to either be: git tag -l "$TAG_PATTERN" --points-at $(git rev-parse "$REF^{commit}" ) or git tag -l "$TAG_PATTERN" --contains$REF
Hide
Kian-Tat Lim added a comment -

Almost any package should show this problem. pex_exceptions is the first that is encountered.

Show
Kian-Tat Lim added a comment - Almost any package should show this problem. pex_exceptions is the first that is encountered.
Hide
Tim Jenness added a comment -

I'm still not sure explicitly what the different output is from the git tag output before and after the change.

Show
Tim Jenness added a comment - I'm still not sure explicitly what the different output is from the git tag output before and after the change.
Hide
Kian-Tat Lim added a comment -

Basically, --points-at returns nothing.

 $git tag -l "[0-9.]*" --points-at v23.0.0.rc1 $ git tag -l "[0-9.]*" --contains v23.0.0.rc1 22.0.0 22.0.1 $git tag -l "[0-9]*" --points-at$(git rev-parse "v23.0.0.rc1^{commit}") 22.0.0 22.0.1 

It looks like bare --points-at would return the tag specified (and only that tag) if it matched the pattern, but that's not what is desired anyway. I believe this has to do with the difference between <object> and <ref> or <commit> as a git argument.

On the other hand, I'm now convinced that --contains is incorrect as well, as it will list later tags that contain the given ref, but those are inappropriate for versioning use. So I think we're down to the last alternative.

Show
Kian-Tat Lim added a comment - Basically, --points-at returns nothing. $git tag -l "[0-9.]*" --points-at v23.0.0.rc1$ git tag -l "[0-9.]*" --contains v23.0.0.rc1 22.0.0 22.0.1 $git tag -l "[0-9]*" --points-at$(git rev-parse "v23.0.0.rc1^{commit}") 22.0.0 22.0.1 It looks like bare --points-at would return the tag specified (and only that tag) if it matched the pattern, but that's not what is desired anyway. I believe this has to do with the difference between <object> and <ref> or <commit> as a git argument. On the other hand, I'm now convinced that --contains is incorrect as well, as it will list later tags that contain the given ref, but those are inappropriate for versioning use. So I think we're down to the last alternative.
Hide
Kian-Tat Lim added a comment -
Show
Kian-Tat Lim added a comment - This is now https://github.com/RobertLuptonTheGood/eups/pull/147
 Reviewers Tim Jenness [ tjenness ] Status In Progress [ 3 ] In Review [ 10004 ]
Hide
Tim Jenness added a comment -

Looks okay since it seems to work. Do you need me to hit the merge button on github?

Show
Tim Jenness added a comment - Looks okay since it seems to work. Do you need me to hit the merge button on github?
 Status In Review [ 10004 ] Reviewed [ 10101 ]
Hide
Kian-Tat Lim added a comment -

Yes, please, and a release, too.

Show
Kian-Tat Lim added a comment - Yes, please, and a release, too.
 Resolution Done [ 10000 ] Status Reviewed [ 10101 ] Done [ 10002 ]
Hide
Tim Jenness added a comment -

New release pushed.

Show
Tim Jenness added a comment - New release pushed.

#### People

Assignee:
Kian-Tat Lim
Reporter:
Kian-Tat Lim
Reviewers:
Tim Jenness
Watchers:
Kian-Tat Lim, Tim Jenness, Yusra AlSayyad