# jointcal butler templates don't include filter

XMLWordPrintable

## Details

• Type: Bug
• Status: Invalid
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Story Points:
1
• Sprint:
Alert Production F17 - 11, AP S18-1, AP S18-2, AP S18-3
• Team:

## Description

It looks like many of the as-implemented butler dataset templates for the jointcal output (wcs and photoCalib) were not written to include the filter name. This is, as you can expect, a problem when running data with multiple filters.

## Activity

Hide
John Parejko added a comment -

My point is that it's much easier to check whether something has been written (when you're having other problems) by looking at the files themselves (e.g., ls some/butler/directory). Having the files separated per-filter makes that much easier.

Show
John Parejko added a comment - My point is that it's much easier to check whether something has been written (when you're having other problems) by looking at the files themselves (e.g., ls some/butler/directory ). Having the files separated per-filter makes that much easier.
Hide
Lauren MacArthur added a comment -

I'm also a +1 on John Parejko's arguments.  I was also going to point out that it also keeps the file structure more consistent with other tract level outputs (although only up to the pre-patch level), e.g. deepCoadd_calexp has:

 template: deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/calexp-%(filter)s-%(tract)d-%(patch)s.fits 

but it seems the proposal for jointcal is to swap the filter/tract order:

 template: 'jointcal-results/%(tract)d/%(filter)s/wcs-%(visit)d-%(ccd)03d.fits' 

Any reason for that?

Show
Lauren MacArthur added a comment - I'm also a +1 on John Parejko 's arguments.  I was also going to point out that it also keeps the file structure more consistent with other tract level outputs (although only up to the pre-patch level), e.g. deepCoadd_calexp has: template: deepCoadd-results/%(filter)s/%(tract)d/%(patch)s/calexp-%(filter)s-%(tract)d-%(patch)s.fits but it seems the proposal for jointcal is to swap the filter/tract order: template: 'jointcal-results/%(tract)d/%(filter)s/wcs-%(visit)d-%(ccd)03d.fits' Any reason for that?
Hide
John Parejko added a comment -

I'm not wedded to it, but I think of "process this tract" then "process this filter". Essentially, "groupby tract" is the first operation in sorting out the data.

Show
John Parejko added a comment - I'm not wedded to it, but I think of "process this tract" then "process this filter". Essentially, "groupby tract" is the first operation in sorting out the data.
Hide
Jim Bosch added a comment -

Whether tract or visit naturally comes first depends on whether you're someone doing analysis on a multi-tract survey or someone debugging a one-run-per tract algorithm, I suspect.

Anyhow, similarity with other tract-level outputs is a pretty compelling argument.  I'll make the change on DM-11138.

Show
Jim Bosch added a comment - Whether tract or visit naturally comes first depends on whether you're someone doing analysis on a multi-tract survey or someone debugging a one-run-per tract algorithm, I suspect.   Anyhow, similarity with other tract-level outputs is a pretty compelling argument.  I'll make the change on DM-11138 .
Hide
John Parejko added a comment -

Closing this as invalid, because DM-11138 is applying these templates to the new jointcal_XXX datasets. That way, we don't have to worry about backwards compatibility or anything.

Show
John Parejko added a comment - Closing this as invalid, because DM-11138 is applying these templates to the new jointcal_XXX datasets. That way, we don't have to worry about backwards compatibility or anything.

## People

• Assignee:
John Parejko
Reporter:
John Parejko
Reviewers:
Paul Price
Watchers:
Hsin-Fang Chiang, Jim Bosch, John Parejko, John Swinbank, Lauren MacArthur, Nate Pease, Paul Price, Simon Krughoff