Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-17681

TSSW Jenkins github webhook management errors

    Details

      Description

      There appears to be a github credentials problem (it is possible this was a transient problem – I'm unsure of how the reporting works) when jenkins is trying to manage webhooks.

        Attachments

          Activity

          Hide
          jhoblitt Joshua Hoblitt added a comment -

          I've setup a github robot user named lssttsadmin and added a github oauth personal access token to jenkins for management of webhooks in the future. I'm actually not sure at this point if the errors being displayed were current or not... manually triggering a re-registration of webhooks resolved a few of the repos. Most of the rest were resolved by removing the webhook from each repo and allowing jenkins to recreated. However, there are still 4 repos remaining. Two are outside the lsst-ts org and were coming from a job named DockerBuildTest, which I have removed.

          At this point, there are no longer any webhook registration errors appearing in the log after triggering a re-registration. I suspect this is merely failed state being kept by jenkins and the errors for repos not currently in use by a job are never going to go away on their own. I've got a recursive grep running trying to figure out where jenkins is storing this state – it isn't anywhere obvious and I'm wondering if it is actually greping the logs itself.

          Show
          jhoblitt Joshua Hoblitt added a comment - I've setup a github robot user named lssttsadmin and added a github oauth personal access token to jenkins for management of webhooks in the future. I'm actually not sure at this point if the errors being displayed were current or not... manually triggering a re-registration of webhooks resolved a few of the repos. Most of the rest were resolved by removing the webhook from each repo and allowing jenkins to recreated. However, there are still 4 repos remaining. Two are outside the lsst-ts org and were coming from a job named DockerBuildTest , which I have removed. At this point, there are no longer any webhook registration errors appearing in the log after triggering a re-registration. I suspect this is merely failed state being kept by jenkins and the errors for repos not currently in use by a job are never going to go away on their own. I've got a recursive grep running trying to figure out where jenkins is storing this state – it isn't anywhere obvious and I'm wondering if it is actually greping the logs itself.
          Hide
          jhoblitt Joshua Hoblitt added a comment -

          It looks like the list of webhook problems isn't persisted to disk (other than appearing in log files). There doesn't seem to be any means of administratively clearing these errors and unless a job is configured which is able to successfully install new hooks for the listed repos, they are going to hand around.

          I finally resorted to digging into the source code and it appears that the list is indeed held in memory and should go away the next time jenkins is restarted:

          https://github.com/jenkinsci/github-plugin/blob/497f43251e3bfc130f125f75e45f73e06894361d/src/main/java/org/jenkinsci/plugins/github/admin/GitHubHookRegisterProblemMonitor.java#L49-L52

          I filed an upstream ticket asking for a means of clearing these errors: https://issues.jenkins-ci.org/browse/JENKINS-55983

          Show
          jhoblitt Joshua Hoblitt added a comment - It looks like the list of webhook problems isn't persisted to disk (other than appearing in log files). There doesn't seem to be any means of administratively clearing these errors and unless a job is configured which is able to successfully install new hooks for the listed repos, they are going to hand around. I finally resorted to digging into the source code and it appears that the list is indeed held in memory and should go away the next time jenkins is restarted: https://github.com/jenkinsci/github-plugin/blob/497f43251e3bfc130f125f75e45f73e06894361d/src/main/java/org/jenkinsci/plugins/github/admin/GitHubHookRegisterProblemMonitor.java#L49-L52 I filed an upstream ticket asking for a means of clearing these errors: https://issues.jenkins-ci.org/browse/JENKINS-55983

            People

            • Assignee:
              jhoblitt Joshua Hoblitt
              Reporter:
              jhoblitt Joshua Hoblitt
              Watchers:
              Andy Clements, Joshua Hoblitt, Rob Bovill
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel