# Add faro steps to DRP.yaml in obs_subaru

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Team:
DM Science
• Urgent?:
No

#### Description

In order to integrate faro metric calculation into the RC2 workflow, we need to update the DRP.yaml file in obs_subaru. This should integrate faro steps into the various steps that produce the required inputs for them (e.g., metrics that only require singleFrame processing can be calculated before moving on to coadds).

#### Activity

Hide
Jeffrey Carlin added a comment -

DM-32059 moves the tasks that were causing failures in obs_subaru to separate pipelines in faro so they won't affect obs_subaru.

Show
Jeffrey Carlin added a comment - DM-32059 moves the tasks that were causing failures in obs_subaru to separate pipelines in faro so they won't affect obs_subaru.
Hide
Dan Taranu added a comment - - edited

FYI, if you want to get the list of tasks from a hierarchical pipeline, you can do something like:

pipetask build -p $FARO_DIR/pipelines/metrics_pipeline_jointcal_fgcm.yaml --show pipeline | awk '{if((length($0) > 2) && (substr($0, 3, 1) != " ")) print$0}'

(The awk bit is ugly and depends on the yaml formatting so you could handle that better if needed).

Show
Dan Taranu added a comment - - edited FYI, if you want to get the list of tasks from a hierarchical pipeline, you can do something like: pipetask build -p $FARO_DIR/pipelines/metrics_pipeline_jointcal_fgcm.yaml --show pipeline | awk '{if((length($0) > 2) && (substr($0, 3, 1) != " ")) print$0}'   (The awk bit is ugly and depends on the yaml formatting so you could handle that better if needed).
Hide
Jeffrey Carlin added a comment -

Sorry, Arun - I forgot to mention that there are two PRs for this ticket. The ci_hsc_gen3 PR is just to exclude faro from being run during that CI job.

Show
Jeffrey Carlin added a comment - Sorry, Arun - I forgot to mention that there are two PRs for this ticket. The ci_hsc_gen3 PR is just to exclude faro from being run during that CI job.
Hide

Looks good. Left a comment about a minor reorganization at a later point.

Show
Arun Kannawadi added a comment - Looks good. Left a comment about a minor reorganization at a later point.
Hide
Jeffrey Carlin added a comment -

Did a little reorganization of the subsets. I was going to simply call "faro_step2" in "step2" of the pipeline, but Nate pointed out that subsets of subsets don't work. Nonetheless, at least now there are faro_step2, faro_step3, and faro_step5 subsets.

Show
Jeffrey Carlin added a comment - Did a little reorganization of the subsets. I was going to simply call "faro_step2" in "step2" of the pipeline, but Nate pointed out that subsets of subsets don't work. Nonetheless, at least now there are faro_step2, faro_step3, and faro_step5 subsets. Jenkins is green:  https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/35188/pipeline

#### People

Assignee:
Jeffrey Carlin
Reporter:
Jeffrey Carlin
Reviewers:
Watchers:
Arun Kannawadi, Dan Taranu, Jeffrey Carlin