John Swinbank your reading of the ticket is exactly the opposite of what I intended when I wrote it. I'm sorry I was not clearer.

What I want is a clean solution for this problem. That implies not a short-term workaround, but a good, solid fix. I think it should be fixed in the new butler, and that this represents an unusual and important use case for that butler.

However, if somebody does have a better short-term solution, I'm willing to accept it.

Sounds like we're all agreed that this is a Butler issue, anyway. I think it's up to Kian-Tat Lim & Nate Pease whether they want to just adopt this issue or incorporate it into their other plans.

Nate Pease and I talked about versioning of repositories and datasets a bit yesterday with regard to DM-4168. Versioning of reference catalog repositories and selection of the appropriate version would rely on that ticket, but that does not seem to be what is requested here.

For this use case, it sounds like retrieving reference catalogs and colorterm dictionaries as Butler datasets would allow the filter and other information in the dataId to be mapped appropriately by camera-specific mapper code to the closest equivalents in the reference catalog and for validity dates to be specified for colorterms. It sounds like it might be easier to write the camera-specific code if a consistent metadata definition for reference catalog repositories were to be defined by the Science Pipelines teams. All of this can I think be done using the current Butler; some of the code would be in the daf.butlerUtils.CameraMapper class (or perhaps a subclass of the proposed new Repository class) but much would go into pipe_base and the obs_* packages. While implementing this will likely require some detailed understanding of the use case, it may be possible for Nate Pease to write the code with some assistance, but that would mean postponement of more foundational and generic Butler work. Alternatively, I could sketch out how I think this would work to someone like Russell Owen.

Please advise as to the urgency/priority of this work and who you think is best able to tackle it.

I agree that we need to think a bit harder about what is being requested.

My guess is that what the pipelines want to say is

Give me the colorterms for using PanSTARRS photometry version 2.3 with the LSST camera on 2028-02-05

Somewhere on the pipeline side could resolve "colorterms for using PanSTARRS photometry version 2.3 with the LSST camera" to a filename, but then we'd need to add mapper entries for N catalogues to each obs_camera. That would be a possibility. Another would be for the pipelines to define the template for the N*M possibilities, then pass that to the butler with appropriate dataId entries to resolve to a particular file (modulo date – I think we all agree that this is a butler issue).

While RFC-624 doesn't solve this problem, we should make sure it is compatible with potential future solutions to this problem (which in turn should likely be the subject of Community and/or RFC discussion).