Paul Price added a comment - 14/Mar/17 10:46 PM We need srcMatchFull (denormalized matches) from single-frame processing. Rename in the mapper: deepCoadd_srcMatch --> deepCoadd_measMatch We need deepCoadd_measMatchFull (denormalized matches) from multiband processing.

Paul Price added a comment - 21/Mar/17 3:37 AM RFC-306 proposes the rename deepCoadd_srcMatch to deepCoadd_measMatch . I'm going to move that to a separate ticket so this work can be merged.

Paul Price added a comment - 21/Mar/17 3:40 AM DM-9885 will do the rename folllowing RFC-306 's acceptance.

Paul Price added a comment - 21/Mar/17 3:41 AM This has passed Jenkins .

Paul Price added a comment - 21/Mar/17 3:46 AM Lauren MacArthur , could you review this please? We're hoping to get this into the HSC release real soon, so please let me know if you can't get to it today and I'll find someone else. price@pap-laptop:~/LSST/obs_base (tickets/DM-9812=) $ git sub commit 1578275f7cd25844a318e5a29868db66dc65b06a Author: Paul Price <price@astro.princeton.edu> Date: Mon Mar 20 09:36:44 2017 -0400   datasets: add denormalized matches Denormalized matches (srcMatchFull) are the matches (srcMatch) written in a denormalized format that uses more space but is more convenient to read.   policy/datasets.yaml | 12 ++++++++++++ 1 file changed, 12 insertions(+)     price@pap-laptop:~/LSST/meas_astrom (tickets/DM-9812=) $ git sub commit cfec1e75a0ce8ac9211ef8b8ca4a309eb1faf2a7 Author: Paul Price <price@astro.princeton.edu> Date: Fri Mar 17 15:51:26 2017 -0400   add function to convert matches to denormalized catalog This produces persisted catalogs that are easier to read, even though they take more space.   python/lsst/meas/astrom/__init__.py | 1 + python/lsst/meas/astrom/denormalizedMatches.py | 58 +++++++++++++++++++++++ tests/testDenormalizedMatches.py | 65 ++++++++++++++++++++++++++ 3 files changed, 124 insertions(+)     price@pap-laptop:~/LSST/pipe_tasks (tickets/DM-9812=) $ git sub commit a178704c2e7800dc148b735cd8619589b2acb4d9 Author: Paul Price <price@astro.princeton.edu> Date: Wed Mar 15 14:43:40 2017 +0900   processCcd: disable exposure writing from charImage Writing exposures from charImage is unnecessary because we're going to write the calexp from calibrate.   python/lsst/pipe/tasks/processCcd.py | 1 + tests/testProcessCcd.py | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)   commit f2a01131439ca0f0b756f4cac20fa5fd26731a82 Author: Paul Price <price@astro.princeton.edu> Date: Mon Mar 20 10:13:24 2017 -0400   calibrate, measureCoaddSources: add option to write denormalized matches This writes the matches in a format that uses more disk space but is much more convenient to read.   python/lsst/pipe/tasks/calibrate.py | 12 +++++++++++- python/lsst/pipe/tasks/multiBand.py | 12 +++++++++++- 2 files changed, 22 insertions(+), 2 deletions(-)     price@pap-laptop:~/LSST/obs_subaru (tickets/DM-9812=) $ git sub commit e7322b7782037f401bf2c43e25e59507b297e36e Author: Paul Price <price@astro.princeton.edu> Date: Mon Mar 20 09:39:51 2017 -0400   HscMapper: add denormalized matches Denormalized matches (srcMatchFull) are the matches (srcMatch) written in a denormalized format that uses more space but is more convenient to read.   policy/HscMapper.paf | 6 ++++++ 1 file changed, 6 insertions(+)   commit 53d43a4a818b303ce288fe9f4ffbec8843dd1fb5 Author: Paul Price <price@astro.princeton.edu> Date: Mon Mar 20 10:12:22 2017 -0400   config: enable writing of denormalized matches   config/measureCoaddSources.py | 2 ++ config/processCcd.py | 2 ++ 2 files changed, 4 insertions(+)     price@pap-laptop:~/LSST/ci_hsc (tickets/DM-9812=) $ git sub commit d06bb4598a241264c075318d9567f4423e60ad8e Author: Paul Price <price@astro.princeton.edu> Date: Mon Mar 20 10:31:28 2017 -0400   validate denormalized matches We want denormalized matches to be written, so ensure they're there.   python/lsst/ci/hsc/validate.py | 11 +++++++++++ 1 file changed, 11 insertions(+)

Paul Price added a comment - 21/Mar/17 7:45 AM Lauren MacArthur is busy preparing for the same release this is aimed at, so asking Yusra AlSayyad to review instead.

Yusra AlSayyad added a comment - 21/Mar/17 1:52 PM Looks good. 2 things Why no srcMatchFull path for obs_cfht, obs_decam, obs_lsstSim, obs_sdss? I mentioned this as a comment in the PR. It was clear to me what denormalizeMatches means, but unclear to me what matchesToCatalog means. I offered some replacements there.

Paul Price added a comment - 21/Mar/17 4:27 PM I'm going to put off adding srcMatchFull until DM-9885 (planning to do that later this week after RFC-306 is accepted), where I'll be touching all the obs packages anyway. I just added a note on that ticket so I don't forget. I'm renaming the function to denormalizedMatches (matches the filename, and properly conveys the intent), and fixing the docstring.

Jim Bosch added a comment - 21/Mar/17 4:31 PM Did you mean denormalizeMatches ? Seems like it ought to start with a present-tense verb, not an adjective (and coding guidelines agree).

Paul Price added a comment - 21/Mar/17 4:49 PM denormalizeMatches it is.

Paul Price added a comment - 21/Mar/17 6:58 PM Revised changes pass Jenkins . I'm about to merge.

Yusra AlSayyad added a comment - 21/Mar/17 7:00 PM

Paul Price added a comment - 21/Mar/17 7:01 PM Thanks Yusra! Merged to master.