# Attempt to build the stack with packaged third parties and pure conda-forge dependencies

XMLWordPrintable

## Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Sprint:
Arch 2019-07-29, Arch 2019-08-05, Arch 2019-08-19, Arch 2019-08-26, Arch 2019-09-09, Arch 2019-09-16
• Team:
Architecture

## Description

After the third parties are packaged and in conda-forge, make a new environment of our third party packages in conda-forge. Attempt to activate that environment and use the compilers meta package in conda-forge to build the stack, ideally with lsstsw, from scratch.

## Activity

Hide
Brian Van Klaveren added a comment -

OSX gets to the same spot in Jointcal.

So the only things not building are:
jointcal, fgmcal, and lsst_distrib, where fgmcal and lsst_distrib rely on jointcal.

https://gist.github.com/brianv0/42915d21efb9d364640a9969f447e4c5

Show
Brian Van Klaveren added a comment - OSX gets to the same spot in Jointcal. So the only things not building are: jointcal, fgmcal, and lsst_distrib, where fgmcal and lsst_distrib rely on jointcal. https://gist.github.com/brianv0/42915d21efb9d364640a9969f447e4c5
Hide
Tim Jenness added a comment -

Is the eigen patch we are using locally something that upstream has? Is it a case of waiting for a new conda or does the conda-forge version need to be built in a special way?

Show
Tim Jenness added a comment - Is the eigen patch we are using locally something that upstream has? Is it a case of waiting for a new conda or does the conda-forge version need to be built in a special way?
Hide
Paul Price added a comment -

I believe upstream has it on the mainline, but not on the release branch.

Show
Paul Price added a comment - I believe upstream has it on the mainline, but not on the release branch.
Hide
Brian Van Klaveren added a comment -

From some light research - I think this may be included in a 3.4 release but it was omitted from 3.3.7 release, which occurred after the changes were merged to master. I'm not sure what the rationale for that might be, but I'd probably feel slightly more comfortable leaving our eigen in eups until there's a 3.4 release. It sounds like that could be within a few months. The eigen mailing list mentioned a 3.4 beta was close to being put out earlier this year.

A possible solution is to create an eigen_cholmod, which has this specific fix for jointcal. Jointcal could rely on that explicitly instead of the acquired one from conda (Right now it gets eigen transitively from afw it appears). Everything else uses conda eigen, until there's a release with the fix in it.

Show
Brian Van Klaveren added a comment - From some light research - I think this may be included in a 3.4 release but it was omitted from 3.3.7 release, which occurred after the changes were merged to master. I'm not sure what the rationale for that might be, but I'd probably feel slightly more comfortable leaving our eigen in eups until there's a 3.4 release. It sounds like that could be within a few months. The eigen mailing list mentioned a 3.4 beta was close to being put out earlier this year. A possible solution is to create an eigen_cholmod, which has this specific fix for jointcal. Jointcal could rely on that explicitly instead of the acquired one from conda (Right now it gets eigen transitively from afw it appears). Everything else uses conda eigen, until there's a release with the fix in it.
Hide
Brian Van Klaveren added a comment -

I'm going to close this issue today because I think the intent is resolved for now.

Some smaller lingering issues after this is closed
1. Understand long keyword names with newer cfitsio libraries, some code may be needed to support this. Verify this fixes commented out afw test and modified pipe_task unit tests (DM-5299)
2. Discussion around resolving eigen issues (should create a new issue for this)
3. Push for a new release of esutil in conda-forge (I'll follow up with a PR)
4. Possibly add peigen to conda-forge (though it's in pip now, probably low priority)

Show
Brian Van Klaveren added a comment - I'm going to close this issue today because I think the intent is resolved for now. Some smaller lingering issues after this is closed 1. Understand long keyword names with newer cfitsio libraries, some code may be needed to support this. Verify this fixes commented out afw test and modified pipe_task unit tests ( DM-5299 ) 2. Discussion around resolving eigen issues (should create a new issue for this) 3. Push for a new release of esutil in conda-forge (I'll follow up with a PR) 4. Possibly add peigen to conda-forge (though it's in pip now, probably low priority)

## People

• Assignee:
Brian Van Klaveren
Reporter:
Brian Van Klaveren
Watchers:
Brian Van Klaveren, Chris Walter, Gabriele Comoretto, Krzysztof Findeisen, Paul Price, Tim Jenness