Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31326

Run faro on gen3_RC2_subset and set it up in Jenkins

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: faro
    • Labels:

      Description

      To confirm that gen3_RC2_subset will meet our needs for a CI dataset, run `faro` on those data and check the resulting metrics.

      Make necessary changes to the pipeline in gen3_RC2_subset and set it up to regularly run in Jenkins.

        Attachments

          Issue Links

            Activity

            Hide
            jcarlin Jeffrey Carlin added a comment - - edited

            I've attached a table of `faro` metrics measured on that dataset. The best point of comparison would probably be the last Characterization Metric Report.

            As expected, the values are typically higher than the full RC2 results, but they mostly seem reasonable.

            Show
            jcarlin Jeffrey Carlin added a comment - - edited I've attached a table of `faro` metrics measured on that dataset. The best point of comparison would probably be the last Characterization Metric Report . As expected, the values are typically higher than the full RC2 results, but they mostly seem reasonable.
            Hide
            jcarlin Jeffrey Carlin added a comment -

            Let me know if you think there are other things I should check before we move forward with `gen3_RC2_subset` as our CI dataset.

            Show
            jcarlin Jeffrey Carlin added a comment - Let me know if you think there are other things I should check before we move forward with `gen3_RC2_subset` as our CI dataset.
            Hide
            krughoff Simon Krughoff added a comment -

            It would be good to add a script that does the processing you need to run faro. That will make it easier to swap out in jenkins

            Show
            krughoff Simon Krughoff added a comment - It would be good to add a script that does the processing you need to run faro. That will make it easier to swap out in jenkins
            Hide
            jcarlin Jeffrey Carlin added a comment -

            I'm not sure I understand what you mean? Do you mean to add a faro script to the `gen3_RC2_subset` Jenkins job?

            For this purpose, I ran "DRP.yaml#singleFrame", "DRP.yaml#jointcal", and "DRP.yaml#fgcm" from the repo. Then I ran "validate_drp_metrics_pipeline.yaml" from faro. (Just calculated matched-visit metrics for now.)

            Show
            jcarlin Jeffrey Carlin added a comment - I'm not sure I understand what you mean? Do you mean to add a faro script to the `gen3_RC2_subset` Jenkins job? For this purpose, I ran "DRP.yaml#singleFrame", "DRP.yaml#jointcal", and "DRP.yaml#fgcm" from the repo. Then I ran "validate_drp_metrics_pipeline.yaml" from faro. (Just calculated matched-visit metrics for now.)
            Hide
            krughoff Simon Krughoff added a comment -

            I mean we have scripts in faro that make it easy for jenkins to run processing and metric measurement. I think we need those same scripts (probably in faro) for doing the processing and measurement on this repository instead

            Show
            krughoff Simon Krughoff added a comment - I mean we have scripts in faro that make it easy for jenkins to run processing and metric measurement. I think we need those same scripts (probably in faro) for doing the processing and measurement on this repository instead
            Hide
            jcarlin Jeffrey Carlin added a comment -

            Oh, so the script (DRP.yaml) that is in gen3_RC2_subset is different from what would be needed in Jenkins? (Sorry if I'm being dense...)

            Show
            jcarlin Jeffrey Carlin added a comment - Oh, so the script (DRP.yaml) that is in gen3_RC2_subset is different from what would be needed in Jenkins? (Sorry if I'm being dense...)
            Hide
            jcarlin Jeffrey Carlin added a comment - - edited

            I added the `faro` steps to the DRP.yaml file in rc2_subset (PR here), and added a shell script to run all the processing in `faro` (PR here).

            I have confirmed that the script runs to completion and generates the expected output data. The next step is to set up a Jenkins job (under the name `verify_drp_metrics`).

            Show
            jcarlin Jeffrey Carlin added a comment - - edited I added the `faro` steps to the DRP.yaml file in rc2_subset (PR here ), and added a shell script to run all the processing in `faro` (PR here ). I have confirmed that the script runs to completion and generates the expected output data. The next step is to set up a Jenkins job (under the name `verify_drp_metrics`).
            Hide
            krughoff Simon Krughoff added a comment -

            I'd like you to take a look at some of the comments I have in the faro PR, but it all looks generally OK to me.

            Show
            krughoff Simon Krughoff added a comment - I'd like you to take a look at some of the comments I have in the faro PR, but it all looks generally OK to me.
            Hide
            jcarlin Jeffrey Carlin added a comment -

            Thanks for the suggestions – I implemented them, and confirmed that it runs to completion. With just patch 40, it took ~4.5 hours on lsst-devl with 8 cores.

            Successful Jenkins run here.

            Show
            jcarlin Jeffrey Carlin added a comment - Thanks for the suggestions – I implemented them, and confirmed that it runs to completion. With just patch 40, it took ~4.5 hours on lsst-devl with 8 cores. Successful Jenkins run here .
            Hide
            jcarlin Jeffrey Carlin added a comment -

            Both PRs have been merged.

            Show
            jcarlin Jeffrey Carlin added a comment - Both PRs have been merged.

              People

              Assignee:
              jcarlin Jeffrey Carlin
              Reporter:
              jcarlin Jeffrey Carlin
              Reviewers:
              Simon Krughoff
              Watchers:
              Jeffrey Carlin, Leanne Guy, Simon Krughoff
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.