Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31326

Run faro on gen3_RC2_subset and set it up in Jenkins

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • faro

    Description

      To confirm that gen3_RC2_subset will meet our needs for a CI dataset, run `faro` on those data and check the resulting metrics.

      Make necessary changes to the pipeline in gen3_RC2_subset and set it up to regularly run in Jenkins.

      Attachments

        Issue Links

          Activity

            jcarlin Jeffrey Carlin added a comment - - edited

            I've attached a table of `faro` metrics measured on that dataset. The best point of comparison would probably be the last Characterization Metric Report.

            As expected, the values are typically higher than the full RC2 results, but they mostly seem reasonable.

            jcarlin Jeffrey Carlin added a comment - - edited I've attached a table of `faro` metrics measured on that dataset. The best point of comparison would probably be the last Characterization Metric Report . As expected, the values are typically higher than the full RC2 results, but they mostly seem reasonable.

            Let me know if you think there are other things I should check before we move forward with `gen3_RC2_subset` as our CI dataset.

            jcarlin Jeffrey Carlin added a comment - Let me know if you think there are other things I should check before we move forward with `gen3_RC2_subset` as our CI dataset.

            It would be good to add a script that does the processing you need to run faro. That will make it easier to swap out in jenkins

            krughoff Simon Krughoff (Inactive) added a comment - It would be good to add a script that does the processing you need to run faro. That will make it easier to swap out in jenkins

            I'm not sure I understand what you mean? Do you mean to add a faro script to the `gen3_RC2_subset` Jenkins job?

            For this purpose, I ran "DRP.yaml#singleFrame", "DRP.yaml#jointcal", and "DRP.yaml#fgcm" from the repo. Then I ran "validate_drp_metrics_pipeline.yaml" from faro. (Just calculated matched-visit metrics for now.)

            jcarlin Jeffrey Carlin added a comment - I'm not sure I understand what you mean? Do you mean to add a faro script to the `gen3_RC2_subset` Jenkins job? For this purpose, I ran "DRP.yaml#singleFrame", "DRP.yaml#jointcal", and "DRP.yaml#fgcm" from the repo. Then I ran "validate_drp_metrics_pipeline.yaml" from faro. (Just calculated matched-visit metrics for now.)

            I mean we have scripts in faro that make it easy for jenkins to run processing and metric measurement. I think we need those same scripts (probably in faro) for doing the processing and measurement on this repository instead

            krughoff Simon Krughoff (Inactive) added a comment - I mean we have scripts in faro that make it easy for jenkins to run processing and metric measurement. I think we need those same scripts (probably in faro) for doing the processing and measurement on this repository instead

            Oh, so the script (DRP.yaml) that is in gen3_RC2_subset is different from what would be needed in Jenkins? (Sorry if I'm being dense...)

            jcarlin Jeffrey Carlin added a comment - Oh, so the script (DRP.yaml) that is in gen3_RC2_subset is different from what would be needed in Jenkins? (Sorry if I'm being dense...)
            jcarlin Jeffrey Carlin added a comment - - edited

            I added the `faro` steps to the DRP.yaml file in rc2_subset (PR here), and added a shell script to run all the processing in `faro` (PR here).

            I have confirmed that the script runs to completion and generates the expected output data. The next step is to set up a Jenkins job (under the name `verify_drp_metrics`).

            jcarlin Jeffrey Carlin added a comment - - edited I added the `faro` steps to the DRP.yaml file in rc2_subset (PR here ), and added a shell script to run all the processing in `faro` (PR here ). I have confirmed that the script runs to completion and generates the expected output data. The next step is to set up a Jenkins job (under the name `verify_drp_metrics`).

            I'd like you to take a look at some of the comments I have in the faro PR, but it all looks generally OK to me.

            krughoff Simon Krughoff (Inactive) added a comment - I'd like you to take a look at some of the comments I have in the faro PR, but it all looks generally OK to me.

            Thanks for the suggestions – I implemented them, and confirmed that it runs to completion. With just patch 40, it took ~4.5 hours on lsst-devl with 8 cores.

            Successful Jenkins run here.

            jcarlin Jeffrey Carlin added a comment - Thanks for the suggestions – I implemented them, and confirmed that it runs to completion. With just patch 40, it took ~4.5 hours on lsst-devl with 8 cores. Successful Jenkins run here .

            Both PRs have been merged.

            jcarlin Jeffrey Carlin added a comment - Both PRs have been merged.

            People

              jcarlin Jeffrey Carlin
              jcarlin Jeffrey Carlin
              Simon Krughoff (Inactive)
              Jeffrey Carlin, Leanne Guy, Simon Krughoff (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.