Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23098

rebuild to save environment information when default env is used.

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: lsstsw
    • Labels:
      None
    • Team:
      Architecture

      Description

      With the changes implemented in lsstsw documented in DM-16839, when a specific environment definition (reference) is used from scipipe-conda-env, the command run-rebuild is able to show the changes made in the environment locally since it has been created using bin/deploy.

      This functionality is not available, in case the default environment is deployed, lsst-scipipe, running bin/deploy without any reference.

      I think it is desirable that this functionality to be available also when the default environment is used. In order to permit this, the sha-1 of the environment has to be added to the environment name.

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            I asked on Slack and the immediate responses were:

            • Use lsst-scipipe-abcde because that's what newinstall has used for a year and people are already accustomed to it.
            • Use scipipe-abcde because lsst doesn't really matter and it's not like you are going to clash with other conda envs any how.
            • Use rubin-abcde and declare that science pipelines is driving the conda environment for the rest of the observatory (which in a way is exactly what is happening).

            And of course newinstall would be changed to match whatever. I'm edging towards the third option.

            Show
            tjenness Tim Jenness added a comment - I asked on Slack and the immediate responses were: Use lsst-scipipe-abcde because that's what newinstall has used for a year and people are already accustomed to it. Use scipipe-abcde because lsst doesn't really matter and it's not like you are going to clash with other conda envs any how. Use rubin-abcde and declare that science pipelines is driving the conda environment for the rest of the observatory (which in a way is exactly what is happening). And of course newinstall would be changed to match whatever. I'm edging towards the third option.
            Hide
            ktl Kian-Tat Lim added a comment -

            Talked to Leanne about it; the quickest, safest, easiest thing to do is lsst-scipipe-8hexchar. We can do a global search and replace for the prefix string, but I'd prefer to save that for when we get all the third-parties into conda. So I think this ticket, already reviewed, should be merged as-is.

            What will remain is to have the lsst-dev shared stack match this result; I'm not exactly sure why it doesn't right now if newinstall already has the hash appended.

            Show
            ktl Kian-Tat Lim added a comment - Talked to Leanne about it; the quickest, safest, easiest thing to do is lsst-scipipe-8hexchar . We can do a global search and replace for the prefix string, but I'd prefer to save that for when we get all the third-parties into conda. So I think this ticket, already reviewed, should be merged as-is. What will remain is to have the lsst-dev shared stack match this result; I'm not exactly sure why it doesn't right now if newinstall already has the hash appended.
            Hide
            swinbank John Swinbank added a comment -

            I'm not exactly sure why it doesn't right now if newinstall already has the hash appended

            That one's easy: the shared stack setup script explicitly overrides it.

            Why? Because it wants to be able to execute commands in the stack environment without having to source other scripts to set things up. It's not pretty, but it works.

            Show
            swinbank John Swinbank added a comment - I'm not exactly sure why it doesn't right now if newinstall already has the hash appended That one's easy: the shared stack setup script explicitly overrides it. Why? Because it wants to be able to execute commands in the stack environment without having to source other scripts to set things up. It's not pretty , but it works.
            Hide
            ktl Kian-Tat Lim added a comment -

            Re-reviewed the further modifications.  Generally fine, although there appear to be excess PATH entries for miniconda that should be removed, and I think csh is broken.  The fix is non-trivial, however, and could be postponed to another ticket (that might be a way to judge if csh support is still required...).

            Show
            ktl Kian-Tat Lim added a comment - Re-reviewed the further modifications.  Generally fine, although there appear to be excess PATH entries for miniconda that should be removed, and I think csh is broken.  The fix is non-trivial, however, and could be postponed to another ticket (that might be a way to judge if csh support is still required...).
            Hide
            tjenness Tim Jenness added a comment -

            I think given the comment from yesterday you can put this ticket straight into Reviewed.

            Show
            tjenness Tim Jenness added a comment - I think given the comment from yesterday you can put this ticket straight into Reviewed.

              People

              Assignee:
              gcomoretto Gabriele Comoretto [X] (Inactive)
              Reporter:
              gcomoretto Gabriele Comoretto [X] (Inactive)
              Reviewers:
              Kian-Tat Lim, Tim Jenness
              Watchers:
              Gabriele Comoretto [X] (Inactive), John Swinbank, Kian-Tat Lim, Leanne Guy, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.