Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9259

add hsc dataset to jenkins validate_drp job

    Details

      Description

      Add the new hsc dataset / driver script (DM-6328) to the jenkins validate_drp job.

        Attachments

          Issue Links

            Activity

            Hide
            jhoblitt Joshua Hoblitt added a comment -

            My attempt to add post-qa json archiving to the validate_drp job didn't work due to required post-qa params (DM-9421). validate_drp has been fixed up and a new test build is running.

            Show
            jhoblitt Joshua Hoblitt added a comment - My attempt to add post-qa json archiving to the validate_drp job didn't work due to required post-qa params ( DM-9421 ). validate_drp has been fixed up and a new test build is running.
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            The validate_drp changes are now working in a sandbox env.

            Show
            jhoblitt Joshua Hoblitt added a comment - The validate_drp changes are now working in a sandbox env.
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            The decam dataset was failing on master and was disabled. I've rebased test branch for jenkins-dm-jobs on the current master. Assuming this builds cleanly over the weekend, I am planning to [finally]] deploy it to master on Monday. Note that the el7 build slaves will need to be re-provisioned at that time to increase the available disk space.

            Show
            jhoblitt Joshua Hoblitt added a comment - The decam dataset was failing on master and was disabled. I've rebased test branch for jenkins-dm-jobs on the current master. Assuming this builds cleanly over the weekend, I am planning to [finally] ] deploy it to master on Monday. Note that the el7 build slaves will need to be re-provisioned at that time to increase the available disk space.
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            The changes to jenkins-dm-jobs have been merged and the production el7 slaves have been re-provisioned with 1.5TiB of disk.

            This issue is being left open until it has been confirmed (tomorrow) that validate_drp has successfully run overnight on its new schedule.

            Show
            jhoblitt Joshua Hoblitt added a comment - The changes to jenkins-dm-jobs have been merged and the production el7 slaves have been re-provisioned with 1.5TiB of disk. This issue is being left open until it has been confirmed (tomorrow) that validate_drp has successfully run overnight on its new schedule.
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            For the last two nights, the valiation_data_hsc build has run ~14 hours because it landed on two different slaves (el7-3 & el7-1). This isn't unexpected, however, I'm going to move the start time up to 1700 project time. I also noticed how large the validation_data_hsc output json files are – several 100MiBs combined. I'm going update the job to compress them.

            Show
            jhoblitt Joshua Hoblitt added a comment - For the last two nights, the valiation_data_hsc build has run ~14 hours because it landed on two different slaves (el7-3 & el7-1). This isn't unexpected, however, I'm going to move the start time up to 1700 project time. I also noticed how large the validation_data_hsc output json files are – several 100MiBs combined. I'm going update the job to compress them.

              People

              • Assignee:
                jhoblitt Joshua Hoblitt
                Reporter:
                jhoblitt Joshua Hoblitt
                Reviewers:
                Joshua Hoblitt
                Watchers:
                Jonathan Sick, Joshua Hoblitt, Michael Wood-Vasey
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel