Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-35450

Jenkins fails to save useful artifacts when jobs timeout

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: jenkins
    • Labels:
    • Story Points:
      4
    • Team:
      Architecture
    • Urgent?:
      No

      Description

      When Jenkins jobs are killed due to exceeding their built-in timeouts, useful artifacts that would help with debugging the problem are often not collected and made available via the UI. Arrange for all artifacts that exist and that would be collected in the event of normal termination to be collected in the event of timeout (especially log files).

        Attachments

          Activity

          Ascending order - Click to sort in descending order
          Hide
          Permalink
          wittgen Matthias Wittgen added a comment -

          The jobs 22-24 in the test setup https://ci.lsst.codes/blue/organizations/jenkins/debug-stack-os-matrix/activity
          have different timeouts on the tasks: 20m, 40m and no timeout to simulate a build task running into its timeout.

          status.yaml is not present in that case, but the build artifacts are uploaded anyway without using the package information. pytest XMLs are uploaded but coverage XMLs are excluded. 

          Show
          wittgen Matthias Wittgen added a comment - The jobs 22-24 in the test setup https://ci.lsst.codes/blue/organizations/jenkins/debug-stack-os-matrix/activity have different timeouts on the tasks: 20m, 40m and no timeout to simulate a build task running into its timeout. status.yaml is not present in that case, but the build artifacts are uploaded anyway without using the package information. pytest XMLs are uploaded but coverage XMLs are excluded. 
          Hide
          Permalink
          ktl Kian-Tat Lim added a comment -

          This looks good (and extensible if we need more artifacts).

          Show
          ktl Kian-Tat Lim added a comment - This looks good (and extensible if we need more artifacts).

            People

            Assignee:
            wittgen Matthias Wittgen
            Reporter:
            ktl Kian-Tat Lim
            Reviewers:
            Kian-Tat Lim
            Watchers:
            Eli Rykoff, Kian-Tat Lim, Matthias Wittgen
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.