Job metadata should also give some indication of swappable or otherwise non-standard pipeline components (suggested by Ian Sullivan) used in a run. Perhaps this can be mined from the Task metadata?

I think we can give a try and specify some metadata tags for ap_verify.

There are metadata that can be added by the pipeline code it self. For the ap_verify running in CI, I currently see:

{'pipe_tasks.CalibrateTime.estimator': 'pipe.base.timeMethod', 'pipe_tasks.ImageDifferenceTime.estimator': 'pipe.base.timeMethod', 'ap_association.AssociationTime.estimator': 'pipe.base.timeMethod', 'pipe_tasks.CharacterizeImageTime.estimator': 'pipe.base.timeMethod', 'pipe_tasks.ProcessCcdTime.estimator': 'pipe.base.timeMethod', 'ip_diffim.DipoleFitTime.estimator': 'pipe.base.timeMethod', 'ip_isr.IsrTime.estimator': 'pipe.base.timeMethod', 'meas_algorithms.SourceDetectionTime.estimator': 'pipe.base.timeMethod'}

So you are already familiar with the mechanism used to add those.

There is also metadata that is captured from the execution context, and thus available at run time only. We call those "environment metadata". Since ap_verify is running in CI, we are capturing these environment metadata right after the execution is done, and before sending the verification job to SQuaSH.

For example:

{'env_name': jenkins, 'ci_id': 4, 'ci_name': ap_verify,  'ci_dataset': CI-HiTS2015, 'ci_url': "https://ci.lsst.codes/job/scipipe/job/ap_verify/4/", 'date': "08/10/2018 05:30:35", 'packages': [ ] }

Following discussions with Krzysztof Findeisen at #ap-prototype-pipeline I am adding to those visit and ccddum which allow us to store the metric values computed per CCD.

For the record, I'm still uncomfortable with visit and ccdnum/ccd being required as properties of a Job. Requiring that all metrics be calculated as such a fine-grained level will break pretty much the entire ap_verify system as currently planned. In particular, most metrics that characterize source association make no sense in the context of a specific visit.

Perhaps, if you want this information, it could be included as measurement metadata?

Rather than adding "visit" and "ccdnum" I suggest you simply add a single "dataId" entry. The content can then vary as appropriate for the metric value in question ("visit/ccdnum", "tract/patch", etc.), and it track the QA WG recommentation that "Metric values should have Butler dataIds."

Good point Eric Bellm I will add a single dataId tag, and yes I agree with Krzysztof Findeisen that this should be measurement metadata.

DM-16333 is blocked by missing metadata, namely filter_name and instrument. It's not yet clear whether these are job- or measurement-level, nor if there are any requirements on the values beyond internal consistency; Joshua Hoblitt said the lack of documentation is being discussed by SQuaRE.

FYI, I'm going to fix this next week if I can find the time.  Follow along on DM-16333.

I think this ticket is obsolete, given DM-16333 and DM-16016. If we want more metadata than is currently provided, we should open tickets for those specific changes.