# add ability to run an obs_base command via the butler command

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Story Points:
8
• Sprint:
DB_S20_02
• Team:
Data Access and Database
• Urgent?:
No

#### Description

Make the butler command in daf_butler able to call the register command in obs_base (at python/lsst/obs/base/instrument.py).

To do this, when obs_base is setup, it should add a colon separated list of CLI python code modules or functions. For example set it to lsst.obs.base.cli.

Then in butler command itself do something like:

 for module in os.environ["DAF_BUTLER_PLUGINS"].split(":"): blah = doImport(cls)

and maybe have the imported class import the subcommands to click somehow
...it may help to be explicit and have the env var contain the actual command function (so lsst.obs.base.cli.register or somesuch).

#### Activity

Hide
Tim Jenness added a comment -

I was thinking that it would not be a path but would be a colon separated list of CLI python code modules or functions. So for example set it to lsst.obs.base.cli (since we would have the code in the python path).

Then in butler command itself you do something like:

 for module in os.environ["DAF_BUTLER_PLUGINS"].split(":"): blah = doImport(cls)

and maybe have the imported class import the subcommands to click somehow (I don't know what the equivalent of from x import * would be or how importing code registers with click.

Maybe it helps to be explicit and have the env var contain the actual command function (so lsst.obs.base.cli.register or somesuch).

Show
Tim Jenness added a comment - I was thinking that it would not be a path but would be a colon separated list of CLI python code modules or functions. So for example set it to lsst.obs.base.cli (since we would have the code in the python path). Then in butler command itself you do something like: for module in os.environ[ "DAF_BUTLER_PLUGINS" ].split( ":" ): blah = doImport( cls ) and maybe have the imported class import the subcommands to click somehow (I don't know what the equivalent of from x import * would be or how importing code registers with click. Maybe it helps to be explicit and have the env var contain the actual command function (so lsst.obs.base.cli.register or somesuch).
Hide
Nate Pease added a comment -

Tim Jenness thanks for the clarification, I updated the description.

Show
Nate Pease added a comment - Tim Jenness thanks for the clarification, I updated the description.
Hide
Nate Pease added a comment -
Show
Nate Pease added a comment - https://github.com/lsst/ci_hsc_gen3/pull/16 https://github.com/lsst/daf_butler/pull/258/ https://github.com/lsst/obs_base/pull/226
Hide
Nate Pease added a comment -

Tim, can you nominate anyone else you think should look at this?

Show
Nate Pease added a comment - Tim, can you nominate anyone else you think should look at this?
Hide
Nate Pease added a comment -

Tim Jenness is this good for a CLO post related to these changes?

Topic:

Butler and obs_base command line scripts have changed to a single command with subcommands

Tags:

hsc, butler, obspackages

Body:

The following butler scripts have been replaced with a top-level butler command, with git-style subcommands:

dumpButlerConfig is now butler dump-config
makeButlerRepo is now butler create
validateButlerConfiguration is now butler validate-config

Supported options have remained the same, but some of the names have changed with the intention of standardizing on common option names.

the ci_hsc_gen3 script registrerInstruments has been replaced with two commands for registering the instrument and writing the calibrations:
butler register
butler write-curated-calibrations

A list of all available butler subcommands is available by the standard --help option; butler --help.

The butler command is defined in daf_butler, but subcommand "plugins" are loaded dynamically so make sure you have setup the packages you intend to use before calling butler --help or attempting to use sub-commands.

Similarly, help for each of the subcommands is available by calling butler <subcommand> --help.

Show
Nate Pease added a comment - Tim Jenness is this good for a CLO post related to these changes? Topic:  Butler and obs_base command line scripts have changed to a single command with subcommands Tags: hsc, butler, obspackages Body: The following butler scripts have been replaced with a top-level butler command, with git-style subcommands: dumpButlerConfig is now butler dump-config makeButlerRepo is now butler create validateButlerConfiguration is now butler validate-config Supported options have remained the same, but some of the names have changed with the intention of standardizing on common option names. the ci_hsc_gen3 script registrerInstruments has been replaced with two commands for registering the instrument and writing the calibrations: butler register butler write-curated-calibrations A list of all available butler subcommands is available by the standard --help option; butler --help. The butler command is defined in daf_butler, but subcommand "plugins" are loaded dynamically so make sure you have setup the packages you intend to use before calling butler --help or attempting to use sub-commands. Similarly, help for each of the subcommands is available by calling butler <subcommand> --help.
Hide
Tim Jenness added a comment -

That looks okay. Not sure hsc is needed as a tag. Looking at it I do wonder if we want butler config-dump and butler config-validate as the subcommands so that they would be more directly config related and sort together in the help. Jim Bosch do you have an opinion on that?

Show
Tim Jenness added a comment - That looks okay. Not sure hsc is needed as a tag. Looking at it I do wonder if we want butler config-dump and butler config-validate as the subcommands so that they would be more directly config related and sort together in the help. Jim Bosch do you have an opinion on that?
Hide
Jim Bosch added a comment -

we want butler config-dump and butler config-validate as the subcommands so that they would be more directly config related and sort together in the help

, and I think that's a pattern we'll want to use for other commands, too. On that note, I'd recommend butler register-instrument, as we'll also have (at least) butler register-skymap.

Show
Jim Bosch added a comment - we want butler config-dump and butler config-validate as the subcommands so that they would be more directly config related and sort together in the help , and I think that's a pattern we'll want to use for other commands, too. On that note, I'd recommend butler register-instrument , as we'll also have (at least) butler register-skymap .
Hide
Tim Jenness added a comment -

Looks good. Some minor comments on GitHub.

Show
Tim Jenness added a comment - Looks good. Some minor comments on GitHub.

#### People

Assignee:
Nate Pease
Reporter:
Nate Pease
Reviewers:
Tim Jenness
Watchers:
Jim Bosch, Nate Pease, Tim Jenness