Details
-
Type:
Bug
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: Continuous Integration, Developer Infrastructure
-
Labels:None
-
Story Points:0.75
-
Epic Link:
-
Team:SQuaRE
Description
There appears to be a github credentials problem (it is possible this was a transient problem – I'm unsure of how the reporting works) when jenkins is trying to manage webhooks.
Attachments
Activity
It looks like the list of webhook problems isn't persisted to disk (other than appearing in log files). There doesn't seem to be any means of administratively clearing these errors and unless a job is configured which is able to successfully install new hooks for the listed repos, they are going to hand around.
I finally resorted to digging into the source code and it appears that the list is indeed held in memory and should go away the next time jenkins is restarted:
I filed an upstream ticket asking for a means of clearing these errors: https://issues.jenkins-ci.org/browse/JENKINS-55983
I've setup a github robot user named lssttsadmin and added a github oauth personal access token to jenkins for management of webhooks in the future. I'm actually not sure at this point if the errors being displayed were current or not... manually triggering a re-registration of webhooks resolved a few of the repos. Most of the rest were resolved by removing the webhook from each repo and allowing jenkins to recreated. However, there are still 4 repos remaining. Two are outside the lsst-ts org and were coming from a job named DockerBuildTest, which I have removed.
At this point, there are no longer any webhook registration errors appearing in the log after triggering a re-registration. I suspect this is merely failed state being kept by jenkins and the errors for repos not currently in use by a job are never going to go away on their own. I've got a recursive grep running trying to figure out where jenkins is storing this state – it isn't anywhere obvious and I'm wondering if it is actually greping the logs itself.