Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14996

split jenkins agent nodes into regular + "validate_drp" capable

    Details

    • Type: Improvement
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: Continuous Integration
    • Labels:
      None
    • Templates:
    • Team:
      SQuaRE

      Description

      At present, all of the 6 "regular/docker" aws jenkins agent instances have 1.5TiB EBS SSD volumes attached to ensure that there is enough local disk to clone the hsc dataset for an EBS footprint of ~9TiB. If only some of the nodes had a large volume, and the validate_drp job was restricted to run the hsc dataset on these nodes, a setup along the lines of 4 x 0.5TiB + 2 x 1.5TiB may be workable. This would provide a savings of ~4TiB SSD/month or about $400/mo.

      However, this would reduce the IOPS per volume/agent node from 4500 to 1500 with a max "burst" of 3000. The cloudwatch EBS data suggests that there are brief read busts of over 9K now – it isn't clear if this is real and EBS is allowing it or if its a metric collection artifact.

        Attachments

          Activity

            People

            • Assignee:
              jhoblitt Joshua Hoblitt
              Reporter:
              jhoblitt Joshua Hoblitt
              Watchers:
              Adam Thornton, Frossie Economou, Joshua Hoblitt
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Summary Panel