This looks great. Thanks for the improvements. I have a few minor things to ask about, but nothing that would stop you from merging now if you like.

pipe_tasks

You say "For defects, the products are valid from the date they are registered until the next date. For other calibration products, a midpoint is used to fix the conflicts.". On the whole this is pretty clear, but I suggest trying to use a different phrase than "fix the conflicts". Do we actually expect conflicts? I thought data products had a start date, but not an end date, so conflicts will not occur. Perhaps you could say "For most calibration products, the calibration data with whose start date is nearest the date of the observation is used. However, defects files are valid from their start date until they are superseded by subsequent defect data".

The --calibType argument help is much clearer, but I still suggest explicitly saying that the OBSTYPE header entry is used (as per my original review).

obs_decam

Would it make sense to shorten the key "ccdnum" down to simply "ccd"? (Certainly not if you already use the key "ccd" for something else).

Why in std_raw do you copy over all metadata? I'm referring to the following snippet:

    for kw in md0.paramNames():

        if kw not in md.paramNames():

            md.add(kw, md0.get(kw))

isr.py: I am a bit surprised that you require that the amount bias and flats are trimmed is the same in x and y. After all, you had separate parameters for those earlier. But if you are positive this is a safe assumption then I have no objection. If you do want to allow them to be different the code would be easy to expand to return a tuple of x, y trim amounts.

One more thing that Simon Krughoff pointed out: in the paf file you use "[%(hdu)d]" for raw images, but "[%(ccdnum)d]" for all other image types. It seems as if the keys "hdu" and "ccdnum" are being used to mean the same thing. I assume this is intentional, but it seems odd and I just wanted to be sure

Thanks a lot for your speedy reply! Sorry I didn't mean to rush you.

I see what you mean the wording about "fixing the conflicts" is not clear, I will edit that.

As far as I know at this moment pipe_tasks/ingestCalibs is only used by obs_decam. I was thinking if other obs_* packages want to use it in the future, they will likely have to override CalibsPasreTask.getCalibType and they might want to use a different header keyword or they can also use filenames. That's why I didn't want to say "OBSTYPE" header explicitly.

Would it make sense to shorten the key "ccdnum" down to simply "ccd"? (Certainly not if you already use the key "ccd" for something else).

The short answer is "ccd" is used for something else.
A bit longer answer is that there was some confusing history about the use of "ccdnum" and "ccd" in obs_decam.
With my limited knowledge I think that use of "ccd" can be eliminated altogether and then replace ccdnum by ccd (I think I know how and sometimes I really hope to do so). However I'm not completely sure about all the consequenses. Also I don't want to add more confusions.

Why in std_raw do you copy over all metadata?

There are more header keywords existing only in the zeroth header but not in others. So far I only encountered problems due to EXPTIME and MJD-OBS. I was thinking it doesn't hurt to copy all non-repeating keywords. (a bit waste of space...)

isr.py: I am a bit surprised that you require that the amount bias and flats are trimmed is the same in x and y. After all, you had separate parameters for those earlier. But if you are positive this is a safe assumption then I have no objection. If you do want to allow them to be different the code would be easy to expand to return a tuple of x, y trim amounts.

True, I guess I preferred to have a strict assumption (and force people to look at the code again when Community Pipeline changes their settings )

in the paf file you use "[%(hdu)d]" for raw images, but "[%(ccdnum)d]" for all other image types.

The keys "hdu" and "ccdnum" have different meanings. "[%(ccdnum)d]" in the instcal/dqmask/wtmap template is a known bug. "[%(ccdnum)d]" in the bias/flat template is correct (so far as I understand) and was my convenient choice. If we want to be proactive to future Community Pipeline changes, we may want to change all of them to "hdu" (changes in ingest task needed).

use a different phrase than "fix the conflicts". Do we actually expect conflicts?

I guess the confusion might have also come from the method name "resolveOverlaps". Previously it was indeed for fixing overlaps in validity ranges. The validity range is set through the file header or filename information or/and specified through user's input (validity period in days), so the ranges can indeed overlap. For example if there are daily calibration products but the user input a validity period of 10 days.

But the wording was not accurate for defects products. I renamed the method name and edited the comments. Hopefully it's more clear now.

A bit more on hdu and ccdnum...

"hdu" refers to the HDU number in the FITS files. They are not necessarily ordered by ccdnum, which corresponds to a specific and fixed CCD of DECam.
In Community Pipeline bias and flat data I've checked so far, they are ordered nicely and always the same (hdu == ccdnum). Raw data have the order shuffled.

Thanks a lot for your careful review and thoughtful comments! Changes are merged to master.

pipe_tasks

 python/lsst/pipe/tasks/ingestCalibs.py | 47 ++++++++++++++++++++++++----------

 1 file changed, 34 insertions(+), 13 deletions(-)

obs_decam

README.md                             |   2 +-

 config/ingestCalibs.py                |   2 +-

 config/processCcd.py                  |  11 ++

 decam/camGeom/N1.fits                 |   2 +-

 .... (in total 62 fits files of the updated CCD files)

 decam/segmentfile.txt                 | 248 +++++++++++++++++-----------------

 policy/DecamMapper.paf                |  36 ++++-

 python/lsst/obs/decam/decamMapper.py  |  83 ++++++++++--

 python/lsst/obs/decam/ingest.py       |  22 ++-

 python/lsst/obs/decam/ingestCalibs.py |  23 ++++

 python/lsst/obs/decam/isr.py          |  80 ++++++++++-

 tests/getRaw.py                       |   7 +

 72 files changed, 440 insertions(+), 214 deletions(-)