Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14920

Install conda packages used by LSST into a non-base environment

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Invalid
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: conda, lsstsw
    • Labels:
      None
    • Team:
      SQuaRE

      Description

      The way our installers currently use conda results in all of the conda packages we install going into the base environment.  That goes against the recommendations of the Anaconda team (https://conda.io/docs/user-guide/getting-started.html#managing-environments), as it makes it difficult to create completely independent environments using the same conda binary.

      It might of course be best to have different conda binaries for different environments, but it appears this is really not the way conda wants to work; it wants to have one conda tool for all environments, and regularly puts stuff I'd consider environment-specific into dotfiles in $HOME.  It might be possible to prevent that by being careful with command-line arguments when we install miniconda; having our conda install not do anything global would be another way to address this issue.

      I am going to try to just install a separate miniconda as a workaround for this, as I'm sure many other LSST users have done successfully before, but the presence of shared content in $HOME still worries me.

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            This might be related to DM-11903 – I know that Joshua Hoblitt has been wondering about conda environments.

            Show
            tjenness Tim Jenness added a comment - This might be related to DM-11903 – I know that Joshua Hoblitt has been wondering about conda environments.
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            The current conda environment files have some packages listed in them that are only allowed in a root env. Conda has also made some changes to the format of an environment file. I'm hoping to resolve the root only package issue the next time there is a package change (hopefully, as soon as 16.0 is shipped) and revisit lsstsw/newinstalls use of envs.

            Show
            jhoblitt Joshua Hoblitt added a comment - The current conda environment files have some packages listed in them that are only allowed in a root env. Conda has also made some changes to the format of an environment file. I'm hoping to resolve the root only package issue the next time there is a package change (hopefully, as soon as 16.0 is shipped) and revisit lsstsw/newinstalls use of envs.
            Hide
            tjenness Tim Jenness added a comment -

            I've linked this ticket to DM-11831.

            Show
            tjenness Tim Jenness added a comment - I've linked this ticket to DM-11831 .
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            This is somewhat resolved by DM-14011. Neither lsstsw or newinstall.sh is setting an CONDARC/etc. but this is done when running under jenkins.

            Show
            jhoblitt Joshua Hoblitt added a comment - This is somewhat resolved by DM-14011 . Neither lsstsw or newinstall.sh is setting an CONDARC /etc. but this is done when running under jenkins.
            Hide
            tjenness Tim Jenness added a comment -

            We now use a special conda environment so this ticket is no longer valid.

            Show
            tjenness Tim Jenness added a comment - We now use a special conda environment so this ticket is no longer valid.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              jbosch Jim Bosch
              Watchers:
              Jim Bosch, Joshua Hoblitt, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.