Per discussion on DM-21885, the new code will need to be able to get what's currently Job metadata out of the registry. Presumably Measurement metadata will be unaffected.

See verify/tests/test_measurements.py and verify/tests/butler_utils.py for some examples of using the Butler in arbitrary scripts. I don't try to read the registry, though.

Jonathan Sick asked on #dm-square about whether the SQuaSH API needs to be updated to handle uploads of up to one metric value (measurement) per metric per data ID, rather than just per metric.

Krzysztof Findeisen this relates to the work Simon Krughoff is doing with the V&V team PR #13 has code that collects metric values from a Gen3 Butler repository, adds job metadata and create a verification job that can be upload to SQuaSH.

I assumed the script in metric-pipeline-tasks was a workaround for this issue. Or do you plan for the Job class to always be required as a sort of intermediate step?

I think reconstructing the Job as an intermediate step is a possible approach to use dispatch_verify.py to submit to SQuaSH. This code could live in the lsst.verify package and be used by dispatch_verify.py if pointed to a Butler Gen 3 registry. Haven't thought much about this yet but it seems promising.

I've been working on updating our SQuaSH stuff for DM-24262, and I noticed we don't seem to have a tag(?) that distinguishes between Gen 2 and Gen 3 runs. This would be very useful once we have more data from both.

That is a good point. I hope to have validate_drp running on HSC in nightlies using both gen2 and gen3 in the next few weeks. It would be very good to be able to differentiate between the two using tags.

I was initially thinking about starting another database, but I think separating by tags is probably better.