Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23106

Use Jenkins parallel stage functionality to thread the salgenerator jobs

    XMLWordPrintable

    Details

      Description

      In looking for ways to speed up the generation of the RPMs, I found that Jenkins pipelines can run stages in parallel. See if this functionality can work.

      https://jenkins.io/blog/2017/09/25/declarative-1/

        Attachments

          Issue Links

          is cloned by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-23925 Implement Jenkins parallel stage functionality to the SAL Release job

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            rbovill Rob Bovill added a comment -

            Got parallel staging to work! I updated the configuration in the job itself to get it working. You can see the results here: https://tssw-ci.lsst.org/job/RPM_Push_daily/252/

            Show
            rbovill Rob Bovill added a comment - Got parallel staging to work! I updated the configuration in the job itself to get it working. You can see the results here: https://tssw-ci.lsst.org/job/RPM_Push_daily/252/
            Hide
            Permalink
            rbovill Rob Bovill added a comment -

            I moved the pipeline script from the Jenkins UI and into the robotframework_salgenerator repo, so the configuration is under source and version control.

            https://github.com/lsst-ts/robotframework_salgenerator/commit/978d030ef58b230b9d3e605ec1716028a2f6cd59

            Show
            rbovill Rob Bovill added a comment - I moved the pipeline script from the Jenkins UI and into the robotframework_salgenerator repo, so the configuration is under source and version control. https://github.com/lsst-ts/robotframework_salgenerator/commit/978d030ef58b230b9d3e605ec1716028a2f6cd59
            Hide
            Permalink
            rbovill Rob Bovill added a comment -

            First job with the script from SCM: https://tssw-ci.lsst.org/job/RPM_Push_daily/258/

            Show
            rbovill Rob Bovill added a comment - First job with the script from SCM: https://tssw-ci.lsst.org/job/RPM_Push_daily/258/
            Hide
            Permalink
            mareuter Michael Reuter added a comment -

            I took a look at the times on the job and it seems the parallel stages cut 2 hours off the job time. Since the file was just placed under version control, I assume it was the addition of the "parallel" sections in the "Generate the RPMs" stage that was the key here yes?

            Show
            mareuter Michael Reuter added a comment - I took a look at the times on the job and it seems the parallel stages cut 2 hours off the job time. Since the file was just placed under version control, I assume it was the addition of the "parallel" sections in the "Generate the RPMs" stage that was the key here yes?
            Hide
            Permalink
            rbovill Rob Bovill added a comment -

            Michael Reuter yes, your assumption is correct.

            Show
            rbovill Rob Bovill added a comment - Michael Reuter yes, your assumption is correct.
            Hide
            Permalink
            mareuter Michael Reuter added a comment -

            Nice improvement to the build situation.

            Show
            mareuter Michael Reuter added a comment - Nice improvement to the build situation.

              People

              Assignee:
              rbovill Rob Bovill
              Reporter:
              rbovill Rob Bovill
              Reviewers:
              Michael Reuter
              Watchers:
              Michael Reuter, Rob Bovill
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.