Details
-
Type:
Story
-
Status: Won't Fix
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: pipe_tasks
-
Labels:None
-
Epic Link:
-
Team:Data Release Production
-
Urgent?:No
Description
It looks like the deepCoadd_obj tables are using the old pseudo-physical_filter labels like "HSC-I -> HSC-I OR HSC-I2" (at least as of recent weeklies) instead of "band". We need to fix even the Gen2 logic that creates these in order this to get the Gen3 analysis code that uses them working properly in Gen3, because we may be using the Gen2->Gen3 converted versions of these datasets for a while.
Attachments
Issue Links
- blocks
-
- Done
Activity
Watch for a Community post about this once I merge DM-27170, but I think the TLDR is that gen2 dataIds will not change at all, just what values are stored in persisted output. I think that the change you requested is even being done as part of DM-27170 (it's at least related to what Yusra AlSayyad and I worked on on Monday, if not exactly the same), but I'm not positive. One of the things I've tried to do is replace uses of dataId["filter"] with e.g. dataRef.get('something_filterLabel') and then using either bandLabel or physicalLabel as appropriate, but I doubt I caught all such uses.
Thanks for the update, John Parejko. I think we can wait on that to land before considering this ticket as "let's see if it's already fixed" ticket. Lauren MacArthur, please go ahead with DM-28389 with some TODO comments, and we'll fix that code up later as part of this ticket as well.
This ticket seems to be about improving gen2 to support the transition to gen3. Does this mean the ticket can now be closed?
John Parejko, Krzysztof Findeisen : do you know if your FilterLabel changes have involved or will involve changing the "filter" data ID values passed to CmdLineTasks that run on coadds and coadd-based datasets (from "pseudo physical_filter" to band, in particular)? I'm guessing no, because that sounds like it's deep in the bowels of Gen2, and I'm guessing it'll be easier to avoid that and instead make as-needed changes to Gen2 tasks so that any in-dataset filter values they write ( (like those relevant to this ticket)) are band, as they will naturally be in Gen3. But if you're already going down that path, it will certainly make tickets like this one easier.