Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-962

Move sims packages to a separate buildbot

    Details

    • Team:
      SQuaRE

      Description

      There should be two separate buildbots for the LSST Stack and the Sims distribution. The users of the two products do not necessarily overlap, so combining them into one distribution is inappropriate. Their development schedules also do not overlap, and we should endeavor to limit the impact of changes in one on the other.

      Having them both built in the same process, as is done now, is arguably a bug.

        Attachments

          Activity

          Hide
          robyn Robyn Allsman [X] (Inactive) added a comment -

          From email btwn K-T and Robyn:
          Sure, but pulling the sims out of the LSST Stack buildbot should
          be a one-liner, no? (But this should be in the lsst_build command line,
          not the lsst_distrib table file, I would think.)

          It needs a change to the filter used to select the repositories (~lsstsw/etc/settings.cfg.sh):
          SBASE='https://stash.lsstcorp.org/scm/'
          export REPOSITORY_PATTERN="$SBASE/sim/%(product)s.git|$BASE/sims/%(product)s.git|$BASE/DMS/%(produ
          ct)s.git|$BASE/DMS/devenv/%(product)s.git|$BASE/DMS/testdata/%(product)s.git|$BASE/external/%(prod
          uct)s.git"

          > This new stack will use the primary
          > DM stack as subordinate when searching for packages during builds. The
          > build process will only 'find' the stash repos. The new stack will only
          > contain the sims eups modules.

          That sounds reasonable, unless the Sims development is going to
          require coordinated changes to the LSST Stack packages, in which case
          there may need to be tighter integration.

          There can be no dependency by the primary DM stack on the Sims stack. If they want a coordinated change, they'll need to update the primary stack and then the secondary stack.

          What we don't have at the moment is a way to correlate an eups-build tag with each stack in the EUPS_PATH so that the correct eups version of a module is always accessed. If we have that then nothing needs to be pushed to master before the entire ediface is updated. I.e.: first fix up a good ticket-based stack on the primary DM stack and then fix up a good ticket branch on the Sims stack. Repeat as necessary then merge the primary stack first, then merge the Sims stack. All can be done with Buildbot orchestrating the builds – but we need the eups-build tags to match the stacks together.

          Show
          robyn Robyn Allsman [X] (Inactive) added a comment - From email btwn K-T and Robyn: Sure, but pulling the sims out of the LSST Stack buildbot should be a one-liner, no? (But this should be in the lsst_build command line, not the lsst_distrib table file, I would think.) It needs a change to the filter used to select the repositories (~lsstsw/etc/settings.cfg.sh): SBASE='https://stash.lsstcorp.org/scm/' export REPOSITORY_PATTERN="$SBASE/sim/%(product)s.git|$BASE/sims/%(product)s.git|$BASE/DMS/%(produ ct)s.git|$BASE/DMS/devenv/%(product)s.git|$BASE/DMS/testdata/%(product)s.git|$BASE/external/%(prod uct)s.git" > This new stack will use the primary > DM stack as subordinate when searching for packages during builds. The > build process will only 'find' the stash repos. The new stack will only > contain the sims eups modules. That sounds reasonable, unless the Sims development is going to require coordinated changes to the LSST Stack packages, in which case there may need to be tighter integration. There can be no dependency by the primary DM stack on the Sims stack. If they want a coordinated change, they'll need to update the primary stack and then the secondary stack. What we don't have at the moment is a way to correlate an eups-build tag with each stack in the EUPS_PATH so that the correct eups version of a module is always accessed. If we have that then nothing needs to be pushed to master before the entire ediface is updated. I.e.: first fix up a good ticket-based stack on the primary DM stack and then fix up a good ticket branch on the Sims stack. Repeat as necessary then merge the primary stack first, then merge the Sims stack. All can be done with Buildbot orchestrating the builds – but we need the eups-build tags to match the stacks together.

            People

            • Assignee:
              frossie Frossie Economou
              Reporter:
              ktl Kian-Tat Lim
              Watchers:
              Kian-Tat Lim, Mario Juric, Robyn Allsman [X] (Inactive), Simon Krughoff
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel