Hsin-Fang Chiang added a comment - 22/Aug/18 2:10 PM Running multiBandDriver.py with w_2018_32 gave the following error: File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/16.0-9-gc30114dc+2/python/lsst/pipe/tasks/multiBand.py", line 1304, in __init__ self.makeSubtask("measurement", schema=self.schema, algMetadata=self.algMetadata) File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/16.0-10-g0b41441+2/python/lsst/pipe/base/task.py", line 300, in makeSubtask subtask = taskField.apply(name=name, parentTask=self, **keyArgs) File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pex_config/16.0-2-g636dfb1/python/lsst/pex/config/configurableField.py", line 83, in apply return self.target(*args, config=self.value, **kw) File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/meas_base/16.0-11-gf9130ea+3/python/lsst/meas/base/sfm.py", line 267, in __init__ self.initializePlugins(schema=self.schema) File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/meas_base/16.0-11-gf9130ea+3/python/lsst/meas/base/baseMeasurement.py", line 242, in initializePlugins self.plugins[name] = PluginClass(config, name, metadata=self.algMetadata, **kwds) File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/meas_base/16.0-11-gf9130ea+3/python/lsst/meas/base/wrappers.py", line 20, in __init__ self.cpp = self.factory(config, name, schema, metadata) File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/meas_base/16.0-11-gf9130ea+3/python/lsst/meas/base/wrappers.py", line 246, in factory return AlgClass(config.makeControl(), name, schema, **kwargs) lsst.pex.exceptions.wrappers.NotFoundError: File "src/table/Schema.cc", line 225, in lsst::afw::table::SchemaItem<T> lsst::afw::table::detail::SchemaImpl::find(const string&) const [with T = int; std::string = std::basic_string<char>] Field or subfield withname 'deblend_nChild' not found with type 'I'. {0} lsst::pex::exceptions::NotFoundError: 'Field or subfield withname 'deblend_nChild' not found with type 'I'.' This was caused by DM-15104 . The required change in the pipe_drivers package was forgotten when ticket merged before w_2018_32. The branch was later pushed to GitHub and is now in PR https://github.com/lsst/pipe_drivers/pull/62 With the pipe_drivers branch, I got the following error: File "/home/hchiang2/stack/pipe_drivers/python/lsst/pipe/drivers/multiBandDriver.py", line 483, in runMergeMeasurements self.mergeCoaddMeasurements.runDataRef(dataRefList) File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/16.0-9-gc30114dc+2/python/lsst/pipe/tasks/multiBand.py", line 504, in runDataRef mergedCatalog = self.run(catalogs, patchRefList[0]) File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/16.0-9-gc30114dc+2/python/lsst/pipe/tasks/multiBand.py", line 1594, in run raise ValueError("Error in inputs to MergeCoaddMeasurements: source IDs do not match") ValueError: Error in inputs to MergeCoaddMeasurements: source IDs do not match

Hsin-Fang Chiang added a comment - 30/Aug/18 8:52 AM DM-15531 and the new branch of DM-15104 have been merged and solved the multiBandDriver errors. During w_2018_32 and w_2018_34 there was a breaking change of DM-14997 , so simply running multiBandDriver using w_2018_34 stack on a w_2018_32-generated repo does not work: errors as follows File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_drivers/16.0-4-g17aeff3/python/lsst/pipe/drivers/multiBandDriver.py", line 464, in runMeasurements self.measureCoaddSources.runDataRef(dataRef) File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/16.0-12-g85846a17/python/lsst/pipe/tasks/multiBand.py", line 1361, in runDataRef exposure.getWcs()) File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/16.0-12-g85846a17/python/lsst/pipe/tasks/propagateVisitFlags.py", line 191, in run matches = afwTable.matchRaDec(coaddSources, ccdSources[ccdSources.get(flag)], radius, mc) File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/afw/16.0-31-gd4f695684/python/lsst/afw/table/baseColumnView/baseColumnViewContinued.py", line 59, in __getitem__ keyobj = self.schema.find(key).key KeyError: "Field with name 'calib_psf_candidate' not found" To continue this week 32 campaign, the new master of pipe_drivers at 17aeff3 , and pipe_tasks as of w_2018_32 plus the DM-15531 patch were used. With this mix, multiBandDriver was able to finish.

Hsin-Fang Chiang added a comment - 30/Aug/18 12:02 PM coadd/multiband: the number of data products and empty patches are the same as w_2018_15/ DM-14123 ( this comment )

Hsin-Fang Chiang added a comment - 31/Aug/18 9:10 AM The output repo is at /datasets/hsc/repo/rerun/RC/w_2018_32/ DM-15184 / The logs have been copied to /datasets/hsc/repo/rerun/RC/w_2018_32/ DM-15184 /logs The pipe_analysis and validate_drp runs are at DM-15519

Hsin-Fang Chiang added a comment - 31/Aug/18 9:14 AM In this run the optional packages such as meas_extensions_convolved and meas_extensions_photometryKron were not set up when the single frame processing was run, so the Kron magnitudes were not in the src catalogs, etc. Still closing this ticket for now; a new ticket will be filed if a new re-run is necessary.