Simon Krughoff would you mind reviewing these changes in branch tickets/DM-4559? These are
a bunch of miscellaneous small fixes, mostly for ISR processing, with the exception of the last one for making coadd.

In response to the previous conversations about defects format, I added a script to output the defects as in the format of obs_subaru/hsc. There can be redundant files in the output files. I think only the diff is recorded for HSC, while my script outputs everything at each date, but the resultant defects should be the same.

I decided not to switch the defect format in obs_decam for now when we still rely on the Community Pipeline calibration files, but am happy to see it change in the future.
For now I would rather make the users responbile of maintaining their defects files in their data repositories, just like other calibration files (flat/bias/fringe). The script may be a first step for switching to that format.

I just caught an issue in the decam policy file, since it's only two lines Hsin-Fang Chiang maybe you could fold this into your updates? The issue is that deepDiff_config has python: "lsst.pex.config.Config", but when I try to run image differencing it complains and it should be "lsst.pipe.tasks.imageDifference.ImageDifferenceConfig". Changing the policy file takes care of the issue when I test it.

Looking ahead, I see that the same issue exists with goodSeeingDiff_config; I think it should get the same fix.

Colin Slater reported errors from here without the config type fix:

assert type(config)==lsst.pipe.tasks.imageDifference.ImageDifferenceConfig, 'config is of type %s.%s instead of lsst.pipe.tasks.imageDifference.ImageDifferenceC     onfig' % (type(config).__module__, type(config).__name__)

I added the output files of examples/genDefectText.py script run with my edited Community Pipeline BPM files. They are in the last commit of branch u/hfc/DM-4559-example just to show how the output files look like and are not meant to be merged to obs_decam.

About adding coadd data products:

• If I understand correctly, DM-4180 and subsequent refactoring will change this.
• The methods bypass_deepCoaddId, bypass_deepCoaddId_bits, and _computeCoaddExposureId are implemented similarly in obs_subaru, obs_sdss, obs_cfht, and obs_lsstSim. If https://dev.lsstcorp.org/trac/ticket/2797 or something similar is not happening, why not add them to CameraMapper?

I have a couple of comments in the pull request, but otherwise looks fine. I also had the question about whether the bypass methods should also be moved to CameraMapper. Feel free to merge after you address my comments.

Thank you for your review!

One possible reason the bypass methods are not be added to CameraMapper is that the mapper policy file lives in each obs package and is camera-specific. It can't work alone without adding those datasets to the policy file. I could understand adding them to CameraMapper would look somewhat odd, although having some default implementations in CameraMapper would be convenient for new cameras.

Probably DM-4180 will change that. I guess we can revisit this after DM-4180.
(if it's better to create a ticket now, I can do that too)

I'd say don't worry about a new ticket. I think, since we've added these to most obs packages by now anyway, we can just see how the tickets you mention work out.

Simon Krughoff I wasn't sure if I understand your note on testing the edge bit setting. Do you mean testing if meas.algorithms.detection.SourceDetectionTask.setEdgeBits correctly does what it says? If so, I am thinking maybe that test would suit better in meas_algorithms?

oops my force-push seemed to have eliminated your notes...

The note was about this part of the code:
https://github.com/lsst/obs_decam/pull/13/files#diff-c8df75c8df81724a84a227b16263525bR145

I added one line comment hopefully to make it more clear, so the previous link is now off by one line, but you get the idea.

I merged the changes to master.

obs_decam

 README.md                            |  10 +-

 examples/genDefectText.py            |  49 ++++++++

 policy/DecamMapper.paf               | 225 ++++++++++++++++++++++++++++++++++-

 python/lsst/obs/decam/decamMapper.py |  48 +++++++-

 python/lsst/obs/decam/isr.py         |  26 ++++

 tests/getRaw.py                      |   2 +-

 6 files changed, 351 insertions(+), 9 deletions(-)

testdata_decam

 calib/calibRegistry.sqlite3                          | 2 +-

 calib/fringe/DECam_Master_20131115v1-zG_ci_z_01.fits | 3 +++

 calib/fringe/DECam_Master_20131115v1-zG_ci_z_13.fits | 3 ---

 3 files changed, 4 insertions(+), 4 deletions(-)

I didn't add a test in meas.algorithms.detection.SourceDetectionTask.setEdgeBits. A new ticket may be filed. As for its use in obs.decam.isr, I added some comments in the code to describe what is going on there. Images were also visually inspected to check it does do what we intend. For example the attacked screenshots and were created with isr.numEdgeSuspect=30 and isr.numEdgeSuspect=35, respectively. Yellow is SUSPECT, red (which looks more like orange with my transparency setting) is BAD, and green is INTRP. The yellow mask covers 30 and 30 pixels wide as expected.