Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31326

Run faro on gen3_RC2_subset and set it up in Jenkins

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: faro
    • Labels:

      Description

      To confirm that gen3_RC2_subset will meet our needs for a CI dataset, run `faro` on those data and check the resulting metrics.

      Make necessary changes to the pipeline in gen3_RC2_subset and set it up to regularly run in Jenkins.

        Attachments

          Issue Links

          mentioned in

          Page Loading...

            Activity

            Ascending order - Click to sort in descending order
            5 older comments
            Hide
            Permalink
            jcarlin Jeffrey Carlin added a comment -

            Oh, so the script (DRP.yaml) that is in gen3_RC2_subset is different from what would be needed in Jenkins? (Sorry if I'm being dense...)

            Show
            jcarlin Jeffrey Carlin added a comment - Oh, so the script (DRP.yaml) that is in gen3_RC2_subset is different from what would be needed in Jenkins? (Sorry if I'm being dense...)
            Hide
            Permalink
            jcarlin Jeffrey Carlin added a comment - - edited

            I added the `faro` steps to the DRP.yaml file in rc2_subset (PR here), and added a shell script to run all the processing in `faro` (PR here).

            I have confirmed that the script runs to completion and generates the expected output data. The next step is to set up a Jenkins job (under the name `verify_drp_metrics`).

            Show
            jcarlin Jeffrey Carlin added a comment - - edited I added the `faro` steps to the DRP.yaml file in rc2_subset (PR here ), and added a shell script to run all the processing in `faro` (PR here ). I have confirmed that the script runs to completion and generates the expected output data. The next step is to set up a Jenkins job (under the name `verify_drp_metrics`).
            Hide
            Permalink
            krughoff Simon Krughoff added a comment -

            I'd like you to take a look at some of the comments I have in the faro PR, but it all looks generally OK to me.

            Show
            krughoff Simon Krughoff added a comment - I'd like you to take a look at some of the comments I have in the faro PR, but it all looks generally OK to me.
            Hide
            Permalink
            jcarlin Jeffrey Carlin added a comment -

            Thanks for the suggestions – I implemented them, and confirmed that it runs to completion. With just patch 40, it took ~4.5 hours on lsst-devl with 8 cores.

            Successful Jenkins run here.

            Show
            jcarlin Jeffrey Carlin added a comment - Thanks for the suggestions – I implemented them, and confirmed that it runs to completion. With just patch 40, it took ~4.5 hours on lsst-devl with 8 cores. Successful Jenkins run here .
            Hide
            Permalink
            jcarlin Jeffrey Carlin added a comment -

            Both PRs have been merged.

            Show
            jcarlin Jeffrey Carlin added a comment - Both PRs have been merged.

              People

              Assignee:
              jcarlin Jeffrey Carlin
              Reporter:
              jcarlin Jeffrey Carlin
              Reviewers:
              Simon Krughoff
              Watchers:
              Jeffrey Carlin, Leanne Guy, Simon Krughoff
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.