Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: Continuous Integration
-
Labels:None
-
Story Points:3
-
Epic Link:
-
Team:SQuaRE
Description
All jenkins jobs that use lsstsw consume couscous amounts of disk space and require periodic manual workspace purging. This should either be moved to a jenkins plugin or an automated script.
Attachments
Issue Links
- relates to
-
DM-12942 add a job to flush all workspaces
- Won't Fix
This is the basic list of requirements I've formulated for a cleanup script:
* cleanup the workspace of jobs on a node when free disk space drops below
threshold X
* must work with jobs nested in cloudbee's folders to an arbitrary depth. This
appears to be a limitation of many of the idenitied existing cleanup scripts.
* node marked as offline during cleanup to prevent new builds of a job from
being started while the workspace cleanup is in progress
* must not operate on nodes marked as offline by the operator
* must not interfer with running builds/pipelines
* must be possible to exclude certain jobs, nodes, or job+node combinations
from cleanup. Eg, `run-rebuild` on lsst-dev
After looking a few existing cleanup scripts (there were too many options to do a comprehensive survey), I ended using https://gist.github.com/akomakom/481507c0dd79ec52a395 as a starting point and doing some modifications / cleanup work.
This was deployed to production yesterday after testing in a sandbox env and appears to be working as expected, thus far.