Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-25431

Add conda env to base.Packages

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: base
    • Labels:
      None
    • Story Points:
      2
    • Team:
      Architecture
    • Urgent?:
      No

      Description

      Examining the package versions we store in butler repository when running pipelines, it is clear that we are not including the details of our conda environment. We need to add code that will read the conda env and include all the versions (possibly overwriting version information determined from the current technique). This will have the advantage of giving us details of boost and related packages that are not visible to python.

      One wrinkle is that I do not yet see an obvious way to get this information without having base run conda list -e.

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            It looks like there is a conda API for this if we are prepared to have the conda package installed in our env: https://docs.conda.io/projects/conda/en/latest/api/python_api.html

            Show
            tjenness Tim Jenness added a comment - It looks like there is a conda API for this if we are prepared to have the conda package installed in our env: https://docs.conda.io/projects/conda/en/latest/api/python_api.html
            Hide
            tjenness Tim Jenness added a comment -

            Brian Van Klaveren this was a pretty quick fix.

            My main concern is that it slows things down by another 2 seconds. I am wondering whether we could arrange things such that getCondaPackages calls getPythonPackages if conda is not available, otherwise skipping all the python package import stuff. Conda is already including pypi packages. For now I read Conda, then overwrite with Python and then EUPS.

            Maybe Kian-Tat Lim has an opinion on this.

            I see that python 3.8 has a standard way for reading versions from python packages.

            Show
            tjenness Tim Jenness added a comment - Brian Van Klaveren this was a pretty quick fix. My main concern is that it slows things down by another 2 seconds. I am wondering whether we could arrange things such that getCondaPackages calls getPythonPackages if conda is not available, otherwise skipping all the python package import stuff. Conda is already including pypi packages. For now I read Conda, then overwrite with Python and then EUPS. Maybe Kian-Tat Lim has an opinion on this. I see that python 3.8 has a standard way for reading versions from python packages.

              People

              Assignee:
              tjenness Tim Jenness
              Reporter:
              tjenness Tim Jenness
              Reviewers:
              Brian Van Klaveren
              Watchers:
              Brian Van Klaveren, Kian-Tat Lim, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.