Nate Lust added a comment - 02/Mar/21 4:50 PM - edited I'm cross posting this from a personal message in case it proves useful for someone else to see, or for long term records. Below is a script to look up the ref cats as required and contains comments on how this would be different as part of a lookup function. from lsst.daf.butler import Butler butlerPath = '/datasets/hsc/gen3repo/rc2w02_ssw03/' collections = 'HSC/runs/RC2/w_2021_02' # this will be provided to you butler = Butler(butlerPath) # registry is what you will be provided registry = butler.registry # You will be given a quantum dataId, but we will look up one for this example, look up tract 9813 for i band quantaId = list (registry.queryDataIds(( "tract" , "band" , "instrument" , "skymap" ), where = "tract=9813 AND skymap='hsc_rings_v1' AND instrument='HSC' AND band='i'" ))[ 0 ] # Now with the given quantumId look up the visits that correspond with that Tract visitRefs = set (butler.registry.queryDataIds(( "visit" , "instrument" ), dataId = quantaId)) # get the refcats that overlap with each visit # note the datasettypename will be self.name in the lookup function dsTypeName = 'gaia_dr2_20200414'   refCatRefs = set () for vr in visitRefs: refCats = refCats.union(butler.registry.queryDatasets(dsTypeName, collections = collections, dataId = vr)) print (refCats) # These are the DatasetRefs for refcats which correspond to the visits that overlap the tract you are working with

Jim Bosch added a comment - 09/Apr/21 7:35 PM I have a probable fix for this in place on tickets/ DM-29615 , where I was changing nearby code in similar ways and saw a TODO comment for this ticket. I've tested that it doesn't break anything, and I'm pretty convinced that means it's doing the right thing, but I haven't directly confirmed that it addresses the problem (e.g. by inspecting the QuantumGraph in an RC2 run and plotting various regions), and haven't begun to think about how to unit test it (and have no plans to on that ticket), so I think it's worth keeping this ticket around for some kind of testing, even if that makes it a lower priority.

Jim Bosch added a comment - 04/Feb/22 3:20 AM The last comment notes that this was done on DM-29615 , and this ticket was kept around just to test that it worked.  We've certainly done one-off testing by this point, and I'm pretty sure the regular jointcal unit tests have got this covered now, too.  John Parejko can reopen if he disagrees.

John Parejko added a comment - 04/Feb/22 10:28 PM Jointcal unittests don't cover this aspect: it needs to have refcat data extending past the tract boundary (or equivalent mocks), and I don't have testdata like that. The relevant method is `lookupVisitRefCats`, added by you; see discussion here . Jointcal at least outputs the relevant metrics in gen3 now, so a gen2/gen3 comparison on a full tracted+multiple dithers would probably tell us for sure that we're getting it right. I think Lauren MacArthur 's work on DM-29821 demonstrated that we were loading the same refcat area in gen2/gen3 on 9813 (assuming that's a good comparison tract?), so we're probably ok here.