Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5435

Provide a shared stack on lsst-dev & other relevant systems

    Details

    • Story Points:
      3
    • Sprint:
      DRP X16-1, DRP X16-2
    • Team:
      Data Release Production

      Description

      Following the discussion in RFC-156, ensure that a documented, fast, easy to initialize shared stack is available for developers to use on shared systems, certainly to include lsst-dev.

        Attachments

          Issue Links

            Activity

            Hide
            swinbank John Swinbank added a comment - - edited

            Colin Slater, do you have time to look at this from the perspective of a science pipelines user/developer please?

            As we've discussed in the past, this provides a shared stack on lsst-dev, but breaks the link between Buildbot and the stack (ie, you can't ask Buildbot to build and declare branches in this stack). I understand this is a regression for you, but I don't think it's possible to support it until SQuaRE provides the infrastructure. I've opened DM-5783 to request that.

            Also, we really want obs_decam and obs_cfht to be installed in this stack. That's currently blocked on DM-5661.

            Other than that, I think I've satisfied all user requests. Please let me know if I've missed anything.

            By the way, you are welcome to comment on the shared_stack.py script I wrote to manage the stack (and certainly, pointing out glaring errors in it would be useful), but, since we're not pushing it into the stack or advertising it as an end-user facing tool, I don't think that bringing it to a high level of polish or strict compliance with our usual coding standards is required.

            Jason Alt [X], I'm adding you as a reviewer on this primarily for your information. I trust you will find this uncontroversial (and in line with what we've previously discussed), but do shout if there's something you object to.

            Show
            swinbank John Swinbank added a comment - - edited Colin Slater , do you have time to look at this from the perspective of a science pipelines user/developer please? As we've discussed in the past, this provides a shared stack on lsst-dev , but breaks the link between Buildbot and the stack (ie, you can't ask Buildbot to build and declare branches in this stack). I understand this is a regression for you, but I don't think it's possible to support it until SQuaRE provides the infrastructure. I've opened DM-5783 to request that. Also, we really want obs_decam and obs_cfht to be installed in this stack. That's currently blocked on DM-5661 . Other than that, I think I've satisfied all user requests. Please let me know if I've missed anything. By the way, you are welcome to comment on the shared_stack.py script I wrote to manage the stack (and certainly, pointing out glaring errors in it would be useful), but, since we're not pushing it into the stack or advertising it as an end-user facing tool, I don't think that bringing it to a high level of polish or strict compliance with our usual coding standards is required. Jason Alt [X] , I'm adding you as a reviewer on this primarily for your information. I trust you will find this uncontroversial (and in line with what we've previously discussed), but do shout if there's something you object to.
            Hide
            ctslater Colin Slater added a comment -

            Seems to work without much of an issue, and it is indeed much faster than ~lsstsw. I don't have much more to add. I put a minor comment on the documentation in the pull request.

            Show
            ctslater Colin Slater added a comment - Seems to work without much of an issue, and it is indeed much faster than ~lsstsw. I don't have much more to add. I put a minor comment on the documentation in the pull request.
            Hide
            swinbank John Swinbank added a comment -

            Thanks Colin. Doc suggestion is good; I'll add some more words there.

            Jason Alt [X], did you want to weigh in on this, or are you happy for me to mark it as done after I've tweaked the docs?

            Show
            swinbank John Swinbank added a comment - Thanks Colin. Doc suggestion is good; I'll add some more words there. Jason Alt [X] , did you want to weigh in on this, or are you happy for me to mark it as done after I've tweaked the docs?
            Hide
            jalt Jason Alt [X] (Inactive) added a comment - - edited

            I'm all in favor of the change. It's not clear to me where the install has happened but I haven't had time to look either. I believe that with the upcoming hardware changes we (DM not just NCSA) may decide to relocate the stack but that is really a new change request and shouldn't hold up the completion of this transition.

            Can you point out where the shared-stack now resides and how users will find it? I believe there is an env script that you will hook into, perhaps that hook hasn't happened until this task is reviewed?

            Show
            jalt Jason Alt [X] (Inactive) added a comment - - edited I'm all in favor of the change. It's not clear to me where the install has happened but I haven't had time to look either. I believe that with the upcoming hardware changes we (DM not just NCSA) may decide to relocate the stack but that is really a new change request and shouldn't hold up the completion of this transition. Can you point out where the shared-stack now resides and how users will find it? I believe there is an env script that you will hook into, perhaps that hook hasn't happened until this task is reviewed?
            Hide
            swinbank John Swinbank added a comment -

            The new stack lives in /ssd/lsstsw/stack on both lsst-dev and lsst-dev7. It's automatically updated to include the latest releases every night.

            Users access it by sourcing /ssd/lsstsw/stack/loadLSST.bash (or .ksh, .zsh, etc). Users will discover this by reading the developer guide.

            Show
            swinbank John Swinbank added a comment - The new stack lives in /ssd/lsstsw/stack on both lsst-dev and lsst-dev7 . It's automatically updated to include the latest releases every night. Users access it by sourcing /ssd/lsstsw/stack/loadLSST.bash (or .ksh , .zsh , etc). Users will discover this by reading the developer guide .
            Hide
            jalt Jason Alt [X] (Inactive) added a comment -

            Thanks John. Everything looks great. Happy to see the transition complete.

            Show
            jalt Jason Alt [X] (Inactive) added a comment - Thanks John. Everything looks great. Happy to see the transition complete.

              People

              • Assignee:
                swinbank John Swinbank
                Reporter:
                swinbank John Swinbank
                Reviewers:
                Jason Alt [X] (Inactive)
                Watchers:
                Colin Slater, Jason Alt [X] (Inactive), John Swinbank
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel