Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-17924

Transition loadLSST.bash conda environment setup to current anaconda recommendations.

    Details

    • Templates:
    • Team:
      Architecture

      Description

      Currently, the setup to set the conda environment in loadLSST.bash looks something like (copied from my /cvmfs setup):

      export PATH="/cvmfs/sw.lsst.eu/darwin-x86_64/lsst_distrib/w_2019_06/python/miniconda3-4.5.4/bin:${PATH}"
      export LSST_CONDA_ENV_NAME=${LSST_CONDA_ENV_NAME:-lsst-scipipe-fcd27eb}
      source activate "$LSST_CONDA_ENV_NAME"

      This pattern is how it was supposed to be done for the  pre-conda 4.4 setup. The new recommended setup is described here:

      https://github.com/conda/conda/blob/master/CHANGELOG.md#440-2017-12-20

      Also, if you were just to add the path (like in the loadLSST.bash file) and then try the new conda activate command you will get this error message:

      CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
      If your shell is Bash or a Bourne variant, enable conda for the current user with

      $ echo ". /cvmfs/sw.lsst.eu/darwin-x86_64/lsst_distrib/w_2019_06/python/miniconda3-4.5.4/etc/profile.d/conda.sh" >> ~/.bash_profile

      or, for all users, enable conda with

      $ sudo ln -s /cvmfs/sw.lsst.eu/darwin-x86_64/lsst_distrib/w_2019_06/python/miniconda3-4.5.4/etc/profile.d/conda.sh /etc/profile.d/conda.sh

      The options above will permanently enable the 'conda' command, but they do NOT
      put conda's base (root) environment on PATH. To do so, run

      $ conda activate

      in your terminal, or to put the base environment on PATH permanently, run

      $ echo "conda activate" >> ~/.bash_profile

      Previous to conda 4.4, the recommended way to activate conda was to modify PATH in
      your ~/.bash_profile file. You should manually remove the line that looks like

      export PATH="/cvmfs/sw.lsst.eu/darwin-x86_64/lsst_distrib/w_2019_06/python/miniconda3-4.5.4/bin:$PATH"

      ^^^ The above line should NO LONGER be in your ~/.bash_profile file! ^^^

      So, with the new scheme you should do something like this instead:

      # For Conda
      . ${LSST_DISTRIB}/python/current/etc/profile.d/conda.sh
      conda activate lsst-scipipe-fcd27eb

      The main difference is that conda is now defined as a function. In the setup above the base environment is always in the path (so conda can be found).  Aside from the other reasons anaconda wants you to switch, this means if there is some executable in the base environment it will be seen by the other environments too.

      I should note one other thing that I noticed in testing.  With conda 4.5.4 I often am not seeing the path completely reset when activating a new environment and often need to do "conda deactivate" first.  This didn't used to be the case. However, with conda 4.6 this problem completely goes away and everything acts as expected.

      I think the bottom line is that for the use case that only the scipipe environment is ever setup, the user doesn't use and switch to other environments, and the base environment is kept clean of packages you don't want the other environments to see, continuing to do things the old way will work as expected.

      However, for more complicated setups where environments are being switched between, we should  switch to the recommended setup and also update to conda 4.6. 

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gcomoretto Gabriele Comoretto
                Reporter:
                cwalter Chris Walter
                Watchers:
                Chris Walter, Fabio Hernandez, Gabriele Comoretto, Heather Kelly, John Swinbank, Joshua Hoblitt, Kian-Tat Lim, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Summary Panel