Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-805

Rebuild the OS-level base container periodically

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      Fabio Hernandez asked when we update the operating system in our distributed containers (lsstsqre/centos-7-stack-lsst_distrib and therefore nublado derived from it). Nominally, according to the Dev Guide we aim to update within 6 months of CentOS minor releases. But we are currently 2 minor releases behind.

      The OS-level base container is built from the official centos:7 container as-is, without doing yum update at any point. This container is then used to create a lsst-newinstall container containing rubin-env, with updates to the latter occurring either when lsst/lsst is modified or when manually triggered. The newinstall container then becomes the base for installing Science Pipelines tarballs using eups distrib.

      Note that updating the OS base container more frequently or doing a yum update at a higher level in the chain would increase the storage required, as layers could no longer be shared between containers.

      Option 0 is to maintain the status quo, updating the OS base only when someone complains or remembers.

      Option 1 is to more carefully monitor CentOS releases and plan to manually rebuild the OS base container within 6 months, as stated in the Dev Guide.

      Option 2 is to automatically rebuild the OS base container periodically, perhaps once a quarter.

      Option 3 is to not rebuild the base container but instead do yum update in the newinstall container or even in the stack-lsst_distrib container.

      I think the DM-CCB needs to discuss the trade-offs between these.

      In any case, it would seem that we should have a "bleed build" that uses the latest OS as well as rubin-env to discover problems before they occur in the development builds and releases.

        Attachments

          Issue Links

            Activity

            No builds found.
            ktl Kian-Tat Lim created issue -
            Hide
            ktl Kian-Tat Lim added a comment -

            Requires DM-CCB discussion.

            Show
            ktl Kian-Tat Lim added a comment - Requires DM-CCB discussion.
            ktl Kian-Tat Lim made changes -
            Field Original Value New Value
            Status Proposed [ 10805 ] Flagged [ 10606 ]
            tjenness Tim Jenness made changes -
            Remote Link This issue links to "Page (Confluence)" [ 31081 ]
            Hide
            tjenness Tim Jenness added a comment -

            I'm fine with option 2.

            Show
            tjenness Tim Jenness added a comment - I'm fine with option 2.
            tjenness Tim Jenness made changes -
            Watchers Kian-Tat Lim, Tim Jenness [ Kian-Tat Lim, Tim Jenness ] Colin Slater, Jim Bosch, Kian-Tat Lim, Leanne Guy, Michelle Butler, Tim Jenness, Wil O'Mullane, Yusra AlSayyad [ Colin Slater, Jim Bosch, Kian-Tat Lim, Leanne Guy, Michelle Butler, Tim Jenness, Wil O'Mullane, Yusra AlSayyad ]
            Hide
            tjenness Tim Jenness added a comment -

            After discussion at CCB the outcome is that we would like to link updating of the OS to a pipelines release. The idea being to update a week or so after the formal release.

            Show
            tjenness Tim Jenness added a comment - After discussion at CCB the outcome is that we would like to link updating of the OS to a pipelines release. The idea being to update a week or so after the formal release.
            tjenness Tim Jenness made changes -
            Remote Link This issue links to "Page (Confluence)" [ 31149 ]
            tjenness Tim Jenness made changes -
            Status Flagged [ 10606 ] Board Recommended [ 11405 ]
            ktl Kian-Tat Lim made changes -
            Link This issue is triggering DM-31962 [ DM-31962 ]
            ktl Kian-Tat Lim made changes -
            Status Board Recommended [ 11405 ] Adopted [ 10806 ]
            ktl Kian-Tat Lim made changes -
            Resolution Done [ 10000 ]
            Status Adopted [ 10806 ] Implemented [ 11105 ]
            tjenness Tim Jenness made changes -
            Remote Link This issue links to "Page (Confluence)" [ 31377 ]

              People

              Assignee:
              ktl Kian-Tat Lim
              Reporter:
              ktl Kian-Tat Lim
              Watchers:
              Colin Slater, Jim Bosch, Kian-Tat Lim, Leanne Guy, Michelle Butler [X] (Inactive), Tim Jenness, Wil O'Mullane, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  Jenkins Builds

                  No builds found.