Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23925

Implement Jenkins parallel stage functionality to the SAL Release job

    XMLWordPrintable

    Details

      Description

      As demonstrated in the work on DM-23106, the parallel stage functionality is a real time saver.  Implement this same strategy in the SAL release job, also parallelizing the messaging tests.

        Attachments

          Issue Links

          clones

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-23106 Use Jenkins parallel stage functionality to thread the salgenerator jobs

          • Done

            Activity

            Ascending order - Click to sort in descending order
            No builds found.
            rbovill Rob Bovill created issue -
            rbovill Rob Bovill made changes -
            Field Original Value New Value
            Epic Link DM-22789 [ 428321 ]
            rbovill Rob Bovill made changes -
            Link This issue clones DM-23106 [ DM-23106 ]
            rbovill Rob Bovill made changes -
            Reviewers Michael Reuter [ mareuter ]
            rbovill Rob Bovill made changes -
            Description In looking for ways to speed up the generation of the RPMs, I found that Jenkins pipelines can run stages in parallel. See if this functionality can work.

            https://jenkins.io/blog/2017/09/25/declarative-1/             		As demonstrated in the work on DM-23106, the parallel stage functionality is a real time saver.  Implement this same strategy in the SAL release job, also parallelizing the messaging tests.
            Hide
            Permalink
            rbovill Rob Bovill added a comment -

            Starting the work here: https://github.com/lsst-ts/ts_Dockerfiles/tree/tickets/DM-23925

            https://github.com/lsst-ts/ts_Dockerfiles/blob/tickets/DM-23925/robotsal/Jenkinsfile.release

            Show
            rbovill Rob Bovill added a comment - Starting the work here: https://github.com/lsst-ts/ts_Dockerfiles/tree/tickets/DM-23925 https://github.com/lsst-ts/ts_Dockerfiles/blob/tickets/DM-23925/robotsal/Jenkinsfile.release
            rbovill Rob Bovill made changes -
            Sprint TSSW Sprint - Apr 27 - May 11 [ 1015 ]
            Urgent? off
            rbovill Rob Bovill made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            Hide
            Permalink
            rbovill Rob Bovill added a comment -

            I moved Jenkinsfile.release to https://github.com/lsst-ts/robotframework_salgenerator

            Show
            rbovill Rob Bovill added a comment - I moved Jenkinsfile.release to https://github.com/lsst-ts/robotframework_salgenerator
            Hide
            Permalink
            rbovill Rob Bovill added a comment -

            I created new job to handle SAL/XML releases using parallel stages to speed up the build:
            https://tssw-ci.lsst.org/job/Parallel_Release/
            It also runs the messaging tests

            Show
            rbovill Rob Bovill added a comment - I created new job to handle SAL/XML releases using parallel stages to speed up the build: https://tssw-ci.lsst.org/job/Parallel_Release/ It also runs the messaging tests
            Hide
            Permalink
            rbovill Rob Bovill added a comment -

            With XML 5.1 work wrapping up 4/28/2020, I plan to use this job to handle the release. If successful, this task is complete.

            Show
            rbovill Rob Bovill added a comment - With XML 5.1 work wrapping up 4/28/2020, I plan to use this job to handle the release. If successful, this task is complete.
            Hide
            Permalink
            rbovill Rob Bovill added a comment -

            SAL v4.1.1 and XML v5.1.0 Release build worked great.
            https://tssw-ci.lsst.org/job/Parallel_Release/99/

            Build products on Nexus3:
            https://repo-nexus.lsst.org/nexus/#browse/search=keyword%3D%224.1.1-5.1.0.el7%22

            https://repo-nexus.lsst.org/nexus/repository/ts_yum/releases/ts_sal_utils-4.1.1-1.x86_64.rpm

            https://repo-nexus.lsst.org/nexus/repository/ocs-maven2/org/lsst/lsst-tsvm/sal_ATCamera/4.1.1_5.1.0/sal_ATCamera-4.1.1_5.1.0.jar

            And similarly for CC and MT Camera jars.

            Show
            rbovill Rob Bovill added a comment - SAL v4.1.1 and XML v5.1.0 Release build worked great. https://tssw-ci.lsst.org/job/Parallel_Release/99/ Build products on Nexus3: https://repo-nexus.lsst.org/nexus/#browse/search=keyword%3D%224.1.1-5.1.0.el7%22 https://repo-nexus.lsst.org/nexus/repository/ts_yum/releases/ts_sal_utils-4.1.1-1.x86_64.rpm https://repo-nexus.lsst.org/nexus/repository/ocs-maven2/org/lsst/lsst-tsvm/sal_ATCamera/4.1.1_5.1.0/sal_ATCamera-4.1.1_5.1.0.jar And similarly for CC and MT Camera jars.
            Hide
            Permalink
            rbovill Rob Bovill added a comment -

            Ready for review.

            Show
            rbovill Rob Bovill added a comment - Ready for review.
            rbovill Rob Bovill made changes -
            Reviewers Michael Reuter [ mareuter ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            Hide
            Permalink
            rbovill Rob Bovill added a comment -

            Created https://github.com/lsst-ts/ts_Dockerfiles/pull/75 to add the python373 RPM to robot/ to install to the docker image for testing.

            Show
            rbovill Rob Bovill added a comment - Created https://github.com/lsst-ts/ts_Dockerfiles/pull/75 to add the python373 RPM to robot/ to install to the docker image for testing.
            Hide
            Permalink
            mareuter Michael Reuter added a comment -

            Reviewed the Jenkins file for the parallel messaging tests. Looks good!

            Show
            mareuter Michael Reuter added a comment - Reviewed the Jenkins file for the parallel messaging tests. Looks good!
            mareuter Michael Reuter made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            rbovill Rob Bovill made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]

              People

              Assignee:
              rbovill Rob Bovill
              Reporter:
              rbovill Rob Bovill
              Reviewers:
              Michael Reuter
              Watchers:
              Michael Reuter, Rob Bovill
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.