Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-7398

Define policy on providing a shared software stack to developers

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Templates:
    • Team:
      System Management

      Description

      Historically (and simplified), we have provided a shared system for use by developers in the form of lsst-dev. It's generally taken as read that this should provide an up-to-date installation of the LSST stack in a "shared" form — that is, configured so that developers can declare their own special versions of packages and share them with their peers.

      In X16, Science Pipelines agreed to temporarily take over maintenance of the lsst-dev shared stack from SQuaRE in order to keep our developers moving while the latter group was overloaded. This has worked reasonably well: we've rolled out a system which automatically installs recent stack releases as they are tagged, which has been in use for several months now.

      However, there are known issues with this means of maintaining the stack, in particular as regards to long term support (how long should historical builds be operational?), upgrades to the underlying Anaconda installation (should this be kept in sync with the lsstsw package list, for example?) and stack "hygiene" (if developers are allowed to tag their own products, who is responsible for cleaning up after them?). See, for example, DM-7361 for an example of the issues faced. In short, more effort is required to support and update this system in the long term. Who is responsible?

      However, the issue goes beyond the narrow scope of lsst-dev. Moving forward, we're likely to see developers expecting to use some form of shared stack that's available on the Nebula infrastructure, the commissioning cluster, and so on. There are additional questions over what it means to "share" a stack in this context, since these systems will presumably not share the same filesystem. What will be provided here? Who is responsible for providing it?

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  mbutler Michelle Butler
                  Reporter:
                  swinbank John Swinbank
                  Watchers:
                  Donald Petravick, Frossie Economou, Gabriele Comoretto, Hsin-Fang Chiang, John Parejko, John Swinbank, Tim Jenness, Wil O'Mullane
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  8 Start watching this issue

                  Dates

                  • Created:
                    Updated:

                    Summary Panel