# Allow gen3.py to be called with an external butler configuration

XMLWordPrintable

## Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
2
• Sprint:
Arch 2019-04-29
• Team:
Architecture

## Description

Currently the butler configuration used for gen 3 testing in ci_hsc is hard coded into the lsst.ci.hsc.gen3 module. This makes it hard to use for oracle testing where we want to override the configuration. Update gen3.py to be a class allowing an override butler and add command line to the gen3.py script to use that functionality.

## Activity

Hide
Tim Jenness added a comment -

I've reworked the gen3 wrapper stuff in ci_hsc so in theory you can specify a different configuration. I also set the config dir so I think Michelle Gower this means that you won't need to specify the top level root yourself (or override the one in datastore).

Show
Tim Jenness added a comment - I've reworked the gen3 wrapper stuff in ci_hsc so in theory you can specify a different configuration. I also set the config dir so I think Michelle Gower this means that you won't need to specify the top level root yourself (or override the one in datastore).
Hide
Michelle Gower added a comment -

I can override butler configuration (e.g., oracle values) using the new command line option.   Has my vote for ready to merge.

My notes from running in case we ever need to look back:

• Cannot override the butler root in any config file(s).  gen3.py has hardcoded ${CI_HSC_DIR}/DATA • Problems occur when trying to copy DATA from a previous run of ci_hsc (even same version). • My override config file for gen3.py needed to use the includeConfigs to point to main butler.yaml file Show Michelle Gower added a comment - I can override butler configuration (e.g., oracle values) using the new command line option. Has my vote for ready to merge. My notes from running in case we ever need to look back: Cannot override the butler root in any config file(s). gen3.py has hardcoded${CI_HSC_DIR}/DATA Problems occur when trying to copy DATA from a previous run of ci_hsc (even same version). My override config file for gen3.py needed to use the includeConfigs to point to main butler.yaml file
Hide
Tim Jenness added a comment -

Thanks. Merged.

Show
Tim Jenness added a comment - Thanks. Merged.

## People

• Assignee:
Tim Jenness
Reporter:
Tim Jenness
Reviewers:
Jim Bosch, Michelle Gower
Watchers:
Jim Bosch, Michelle Gower, Tim Jenness