# automate jenkins workspace cleanup

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
3
• Team:
SQuaRE

#### Description

All jenkins jobs that use lsstsw consume couscous amounts of disk space and require periodic manual workspace purging. This should either be moved to a jenkins plugin or an automated script.

#### Activity

Hide
Joshua Hoblitt added a comment -
Show
Joshua Hoblitt added a comment - https://gist.github.com/rb2k/8372402
Hide
Joshua Hoblitt added a comment -
Show
Joshua Hoblitt added a comment - https://julienprog.wordpress.com/2016/04/14/clean-up-jenkins-workspaces/
Hide
Joshua Hoblitt added a comment -

This is the basic list of requirements I've formulated for a cleanup script:

 * cleanup the workspace of jobs on a node when free disk space drops below  threshold X   * must work with jobs nested in cloudbee's folders to an arbitrary depth. This  appears to be a limitation of many of the idenitied existing cleanup scripts.   * node marked as offline during cleanup to prevent new builds of a job from  being started while the workspace cleanup is in progress   * must not operate on nodes marked as offline by the operator   * must not interfer with running builds/pipelines   * must be possible to exclude certain jobs, nodes, or job+node combinations  from cleanup. Eg, run-rebuild on lsst-dev  

After looking a few existing cleanup scripts (there were too many options to do a comprehensive survey), I ended using https://gist.github.com/akomakom/481507c0dd79ec52a395 as a starting point and doing some modifications / cleanup work.

This was deployed to production yesterday after testing in a sandbox env and appears to be working as expected, thus far.

Show
Joshua Hoblitt added a comment - This is the basic list of requirements I've formulated for a cleanup script: * cleanup the workspace of jobs on a node when free disk space drops below threshold X   * must work with jobs nested in cloudbee's folders to an arbitrary depth. This appears to be a limitation of many of the idenitied existing cleanup scripts.   * node marked as offline during cleanup to prevent new builds of a job from being started while the workspace cleanup is in progress   * must not operate on nodes marked as offline by the operator   * must not interfer with running builds/pipelines   * must be possible to exclude certain jobs, nodes, or job+node combinations from cleanup. Eg, run-rebuild on lsst-dev After looking a few existing cleanup scripts (there were too many options to do a comprehensive survey), I ended using https://gist.github.com/akomakom/481507c0dd79ec52a395 as a starting point and doing some modifications / cleanup work. This was deployed to production yesterday after testing in a sandbox env and appears to be working as expected, thus far.

#### People

Assignee:
Joshua Hoblitt
Reporter:
Joshua Hoblitt
Watchers:
Joshua Hoblitt