Intermittent fault building ci_hsc through Jenkins

XMLWordPrintable

Details

• Type: Bug
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
3
• Sprint:
DRP X16-3, DRP F16-1
• Team:
Data Release Production

Description

Occasionally (see e.g. here and here) the ci_hsc job in Jenkins fails, reporting:

 RuntimeError: dictionary changed size during iteration 

The fault seems to be intermittent. Please fix it.

Activity

Hide
John Swinbank added a comment -

I've now eyeballed the code. Looks good to go. Thank you, Paul Price!

Show
John Swinbank added a comment - I've now eyeballed the code. Looks good to go. Thank you, Paul Price !
Hide
John Swinbank added a comment -

Oh, with the caveat that the PR on GitHub currently claims "All checks have failed" and has a ugly red cross next to it. I think that's just an artefact of ongoing maintenance – Joshua Hoblitt might be able to confirm – so it's probably safe to merge, but it's a bit scary.

Show
John Swinbank added a comment - Oh, with the caveat that the PR on GitHub currently claims "All checks have failed" and has a ugly red cross next to it. I think that's just an artefact of ongoing maintenance – Joshua Hoblitt might be able to confirm – so it's probably safe to merge, but it's a bit scary.
Hide
Paul Price added a comment -

I did multiple pushes of ci_hsc to github (push, rebase, push) and the automatically triggered Jenkins runs were holding up my manually-triggered Jenkins run (which I reported above passed), so I asked Joshua Hoblitt to kill the automatically triggered runs. The log for the checks that failed says "Aborted by user Joshua Hoblitt", which is consistent with my story. I therefore believe this is safe to merge, and will go ahead and do so now.

Show
Paul Price added a comment - I did multiple pushes of ci_hsc to github (push, rebase, push) and the automatically triggered Jenkins runs were holding up my manually-triggered Jenkins run (which I reported above passed), so I asked Joshua Hoblitt to kill the automatically triggered runs. The log for the checks that failed says "Aborted by user Joshua Hoblitt", which is consistent with my story. I therefore believe this is safe to merge, and will go ahead and do so now.
Hide
Paul Price added a comment -

Merged to master.

Show
Paul Price added a comment - Merged to master.
Hide
Joshua Hoblitt added a comment -

(I thought I had answered this... maybe it was a parallel inquiry on HC)

I concur with Paul's assessment. At the time of the incident, I looked at the configuration of the jenkins plugin that handles this integration with github (github pull request builder) and this behavior is not configurable. If this issue comes up again the plugin should be extended to either set the error status on the commit (https://developer.github.com/v3/repos/statuses/) or remove the commit status completely.

Show
Joshua Hoblitt added a comment - (I thought I had answered this... maybe it was a parallel inquiry on HC) I concur with Paul's assessment. At the time of the incident, I looked at the configuration of the jenkins plugin that handles this integration with github (github pull request builder) and this behavior is not configurable. If this issue comes up again the plugin should be extended to either set the error status on the commit ( https://developer.github.com/v3/repos/statuses/ ) or remove the commit status completely.

People

• Assignee:
Paul Price
Reporter:
John Swinbank
Reviewers:
John Swinbank
Watchers:
John Swinbank, Joshua Hoblitt, Paul Price, Tim Jenness