Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: pipe_base
-
Labels:
-
Story Points:6
-
Epic Link:
-
Team:Data Release Production
-
Urgent?:No
Description
As described in DMTN-177 pipe_base needs to be updated to enable the ability to create a new subset registry (sqlite) containing everything needed for the pipeline execution. This includes all output datasets and all necessary inputs (including calibrations).
Andy,
For testing I inserting the following into `CmdLineFwk.makeGraph`. It is not a full long term solution, but I wanted to highlight what I did to get things working:
```
@@ -540,6 +541,19 @@ class CmdLineFwk:
skipExisting=args.skip_existing)
qgraph = graphBuilder.makeGraph(pipeline, collections, run, args.data_query)
+ # new stuff below
+ newArgs = copy.deepcopy(args)
+ butler = _ButlerFactory.makeReadButler(args)
+ butlerLight = LightWeightButlerBuilder(butler, args.input)
+
+ def builderShim(config):
+ newArgs.butler_config = config
+ newButler = _ButlerFactory.makeWriteButler(newArgs)
+ return newButler
+
+ butlerLight.build(qgraph, '/home/nate2/lightweightButlerV1/', run, clobber=True,
+ butlerModifier=builderShim)
+
nQuanta = len(qgraph)
if nQuanta == 0:
```
I don't know if we would need this at some point, but I designed the object to take a butler to be initialized, and then the build function can be used to build multiple butlers from different `QuantumGraphs`