I made DM-27919 just now for all your physical filter discussions  

Belated thank you to Jim for responding to all my quibbles!

I have the 2015 calibs all done now too, in /project/mrawls/hits2015-3. I followed the same steps as for 2014, but for these visits and validity ranges.
 
BIASES
#2015-02-16 - valid from 2015-01-01 to 2015-02-16T23:59:59
(410714, 410715, 410716, 410717, 410718, 410719, 410720, 410721, 410722, 410723, 410724)
#2015-02-17 - valid from 2015-02-17T00:00:00 to 2015-02-17T23:59:59
(411102, 411103, 411104, 411105, 411106, 411107, 411108, 411109, 411110, 411111, 411112)
#2015-02-18 - valid from 2015-02-18T00:00:00 to 2015-02-18T23:59:59
(411502, 411503, 411504, 411505, 411506, 411507, 411508, 411509, 411510, 411511, 411512)
#2015-02-19 - valid from 2015-02-19T00:00:00 to 2015-02-19T23:59:59
(411904, 411905, 411906, 411907, 411908, 411909, 411910, 411911, 411912, 411913, 411914)
#2015-02-20 - valid from 2015-02-20T00:00:00 to 2015-02-20T23:59:59
(412096, 412097, 412098, 412099, 412100, 412101, 412102, 412103, 412104, 412105, 412106)
#2015-02-21 - valid from 2015-02-21T00:00:00 to 2015-02-22T23:59:59
(412355, 412356, 412357, 412358, 412359, 412360, 412361, 412362, 412363, 412364, 412365)
#2015-02-23 - valid from 2015-02-23T00:00:00 to 2015-02-24T23:59:59
(413448, 413449, 413450, 413451, 413452, 413453, 413454, 413455, 413456, 413457, 413458)
#2015-02-26 - valid from 2015-02-25T00:00:00 to 2015-03-05T23:59:59
(415136, 415137, 415138, 415139, 415140, 415141, 415142, 415143, 415144, 415145, 415146)
#2015-03-13 - valid from 2015-03-06T00:00:00 to 2016-01-01
(421350, 421351, 421352, 421353, 421354, 421355, 421356, 421357, 421358, 421359, 421360)

FLATS
#2015-02-16 - valid from 2015-01-01 to 2015-02-16T23:59:59
(410790, 410791, 410792, 410793, 410794, 410795, 410796, 410797, 410798, 410799, 410800)
#2015-02-17 - valid from 2015-02-17T00:00:00 to 2015-02-17T23:59:59
(411178, 411179, 411180, 411181, 411182, 411183, 411184, 411185, 411186, 411187, 411188)
#2015-02-18 - valid from 2015-02-18T00:00:00 to 2015-02-18T23:59:59
(411578, 411579, 411580, 411581, 411582, 411583, 411584, 411585, 411586, 411587, 411588)
#2015-02-19 - valid from 2015-02-19T00:00:00 to 2015-02-19T23:59:59
(411980, 411981, 411982, 411983, 411984, 411985, 411986, 411987, 411988, 411989, 411990)
#2015-02-20 - valid from 2015-02-20T00:00:00 to 2015-02-20T23:59:59
(412172, 412173, 412174, 412175, 412176, 412177, 412178, 412179, 412180, 412181, 412182)
#2015-02-21 - valid from 2015-02-21T00:00:00 to 2015-02-22T23:59:59
(412431, 412432, 412433, 412434, 412435, 412436, 412437, 412438, 412439, 412440, 412441)
#2015-02-23 - valid from 2015-02-23T00:00:00 to 2015-02-24T23:59:59
(413524, 413525, 413526, 413527, 413528, 413529, 413530, 413531, 413532, 413533, 413534)
#2015-02-26 - valid from 2015-02-25T00:00:00 to 2015-03-05T23:59:59
(415212, 415213, 415214, 415215, 415216, 415217, 415218, 415219, 415220, 415221, 415222)
#2015-03-13 - valid from 2015-03-06T00:00:00 to 2016-01-01
(421426, 421427, 421428, 421429, 421430, 421431, 421432, 421433, 421434, 421435, 421436)

I realized belatedly that I probably should have done this all in one humongous repo instead of splitting it into two repos (one for 2014 and one for 2015). I'm not too worried about this since the steps will all need to be redone in 2021 when we have linearity and crosstalk for DECam in gen3. But I would welcome ideas for how to best use coadds I make in the 2014 repo as difference imaging templates for science images in the 2015 repo.

For the review, please (1) verify the calibs exist as I claim they do and (2) comment on idiosyncrasies or things you'd recommend I do differently next time, for any reason. Thank you!

I have not exhaustively checked, but a spot check of exposures from the list above does find that bias and flat frames are found for each the raw exposures.  If I remember correctly, the bias frames have different filter names, and so only bias frames are found for those.  I've attached an example qgraph that I used to do this check to the ticket.

For the grouping/collection count, it is possible to split the pipelines to run all the ISR processing for all dates at the same time, but separate calls (with separate collections) would still be needed for the `cpCombine` step.  There's been an idea of writing a pipeline that would allow an uncertified calibration to be used during processing, but I'm not sure if that's blocked  by anything.  This would allow a single collection to be used to generate all calibrations for a date range.  The `cp_verify` code will need this functionality, as it will run on uncertified calibrations to check that they are useful.

`cp_verify` is also my proposed solution for valid date ranges: as new calibration data is taken, metrics will be used to see when the existing calibrations no longer meet a to-be-defined "useful" value.  That will trigger (manually at first) new calibrations generated from the new calibration data that should capture the time-varying changes.  Beyond that, the obs_date values for all input exposures is in the headers of FITS image calibrations (and will be required in others when I finish rewriting DMTN-148).  Pulling out all  keywords starting with `CPP_INPUT_DATE_` will at least identify when the inputs were taken.  This isn't always going to be the complete validStart/validEnd range, and that's why I've been hesitant to produce detailed "guesses".

This has also made me think more about the decertify process, and the partially related question of knowing which calibration was used.  The qgraph I produced above shows that a bias and a flat will be used, but not which one.  This information exists in the butler, but I'm not sure what the human readable form of that is.  Whatever that form is is probably also the form that the decertify command line interface should use.  This may be simply the set of (calibration collection, calibration type, filter, validStart, validEnd), but I'm also open to suggestions on this.

I'm marking this ticket reviewed, as (1) the calibs exist as you claim and (2) I don't think I have any suggestions for things to do differently, as this matches my current expected workflow for calibration production.

Thank you - I look forward to a future cp_verify product as you describe!

I agree it would be best if the qgraph somehow showed exactly which bias and flat will be used. Perhaps the "origin collection" name?

I suspect I will come bothering you (or others) shortly about how to use templates from one gen3 repo to do difference imaging in another, but one problem at a time.

I agree it would be best if the qgraph somehow showed exactly which bias and flat will be used. Perhaps the "origin collection" name?

The origin RUN-type collection is already known to the code that makes these graphs and would be easy to add here. And it would probably be worth adding to the non-calibration dataset boxes, too.