Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10209

Report: Nebula VM root filesystem failure (git-lfs.lsst.codes)

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Team:
      Data Facility

      Description

      The production LSST Git LFS server failed today at 5:52 AM MST. It was Nebula instance jmatt-ubuntu-git-lfs-test (e20a71e1-7885-4b27-9781-43e2442cd432 141.142.210.114 https://git-lfs.lsst.codes https://git-lfs-test.lsst.codes). It failed due to a corrupt root filesystem. Grub 2 could not identify a viable boot disk.

      This resulted in approximately five hours of downtime.

      It was rebuilt as Nebula instance (90cc6090-4b04-4aab-a956-d9d5b57c31b2 141.142.211.1 https://git-lfs.lsst.codes).

      Further investigation is possible. The bad instance (e20a71e1-7885-4b27-9781-43e2442cd432) still exists on Nebula.

        Attachments

          Issue Links

            Activity

            Hide
            gdaues Greg Daues added a comment -

            Updating watchers.

            Show
            gdaues Greg Daues added a comment - Updating watchers.
            Hide
            tjenness Tim Jenness added a comment -

            Michelle Butler [X] all the people involved in this ticket have left the project so I'm guessing we can close this.

            cc/ Joshua Hoblitt who is the person currently involved in git-lfs.

            Show
            tjenness Tim Jenness added a comment - Michelle Butler [X] all the people involved in this ticket have left the project so I'm guessing we can close this. cc/ Joshua Hoblitt who is the person currently involved in git-lfs.
            Hide
            mbutler Michelle Butler [X] (Inactive) added a comment -

            also since the project doesn't use Nebula with the K8 infrastructure...  I would agree we can close this.  

            M. 

             

            Show
            mbutler Michelle Butler [X] (Inactive) added a comment - also since the project doesn't use Nebula with the K8 infrastructure...  I would agree we can close this.   M.   
            Hide
            mbutler Michelle Butler [X] (Inactive) added a comment -

            we are no longer using Nebula.

            M. 

             

            Show
            mbutler Michelle Butler [X] (Inactive) added a comment - we are no longer using Nebula. M.   

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              jmatt J Matt Peterson [X] (Inactive)
              Watchers:
              J Matt Peterson [X] (Inactive), Michelle Butler [X] (Inactive), Paul Domagala [X] (Inactive), Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.