Details
-
Type:
Bug
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: System Integration and Test
-
Labels:None
-
Story Points:4
-
Epic Link:
-
Team:SQuaRE
Description
The TSSW Jenkins service, https://ts-ci.lsst.codes/blue/organizations/jenkins, had a 502 Bad Gateway blip again, but recovered very quickly. I know this because I saw the error in a browser cached console output. When the browser refreshed, the build was gone, just gone. This episode cost several hours of work and has delayed a critical software release. This is most likely related to DM-16180 and DM-16459.
The missing build in question, can not be found in the https://ts-ci.lsst.codes/job/ts_sal/ project. Notice there is a missing build #642.
Attachments
Issue Links
Activity
The jenkins master has restarted in the last day, at least. The suspected robot related faults were occurring days or weeks apart, so there likely isn't much more action to take on this ticket at this time other than periodic observation over the the next month or so.
The jenkins master still appears to be running normally:
● jenkins.service - Jenkins master service
|
Loaded: loaded (/etc/systemd/system/jenkins.service; enabled; vendor preset: disabled)
|
Active: active (running) since Wed 2019-01-30 21:04:30 PST; 2 weeks 1 days ago |
Note that the master process uptime was reset last night by the deployment of changes for DM-17540.
I'm going to go ahead and close this ticket as my "gut" feel is it is resolved and there's another core update coming, which will reset the uptime count again.
Thank you Joshua Hoblitt !!!