Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-33305

Use mamba to create environment in newinstall

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: jenkins
    • Labels:
      None
    • Team:
      Architecture
    • Urgent?:
      No

      Description

      Tarball builds are so slow to solve that more than half of the 8 hour time limit is being used on conda. State is being kept, but newinstall intentionally reinstalls the environment even if it already exists. (DM-31235 attempted to reuse an existing environment, but it failed and had to be reverted.) As an immediate fix, using lsstinstall instead of newinstall should speed things up massively. Unfortunately, using lsstinstall on top of an existing newinstall installation has some problems.

      A somewhat simpler solution is to modify newinstall to install mamba and then use that to create the environment. This doesn't provide some of the other features of lsstinstall, but it's 100% compatible and much faster.

        Attachments

          Activity

          Hide
          ktl Kian-Tat Lim added a comment -

          This should get us unstuck for nightly and weekly builds. stack-os-matrix is harder to fix as that is lsstsw and not newinstall.

          Show
          ktl Kian-Tat Lim added a comment - This should get us unstuck for nightly and weekly builds. stack-os-matrix is harder to fix as that is lsstsw and not newinstall.
          Hide
          ktl Kian-Tat Lim added a comment -

          Turns out that there are issues putting lsstinstall on top of an existing newinstall. Had to revert. Looking into a different fix.

          Show
          ktl Kian-Tat Lim added a comment - Turns out that there are issues putting lsstinstall on top of an existing newinstall. Had to revert. Looking into a different fix.
          Hide
          ktl Kian-Tat Lim added a comment -

          Updated the ticket to describe the new fix I'm trying.

          Show
          ktl Kian-Tat Lim added a comment - Updated the ticket to describe the new fix I'm trying.
          Hide
          ktl Kian-Tat Lim added a comment -

          Only +4 -1, but it looks like it will do the job for now.

          Show
          ktl Kian-Tat Lim added a comment - Only +4 -1, but it looks like it will do the job for now.

            People

            Assignee:
            ktl Kian-Tat Lim
            Reporter:
            ktl Kian-Tat Lim
            Reviewers:
            Tim Jenness
            Watchers:
            Kian-Tat Lim, Tim Jenness
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.