# HSC ENG-R1 filter seems to have wrong band

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
0.5
• Team:
Architecture
• Urgent?:
No

#### Description

Looking at the physical filter -> band mapping for HSC I see:

 \$ butler query-dimension-records tmp physical_filter instrument name band  ---------- ----------------------------- -----------------------------  HSC ENG-R1 r1  HSC HSC-G g  HSC HSC-I i  HSC HSC-I2 i  HSC HSC-R r  HSC HSC-R2 r  HSC HSC-Y y  HSC HSC-Z z  HSC IB0945 I945  HSC NB0387 N387  HSC NB0400 N400  HSC NB0468 N468  HSC NB0515 N515  HSC NB0527 N527  HSC NB0656 N656  HSC NB0718 N718  HSC NB0816 N816  HSC NB0921 N921  HSC NB0926 N926  HSC NB0973 N973  HSC NB1010 N1010  HSC NONE UNRECOGNISED  HSC PH PH  HSC SH SH 

• Shouldn't ENG-R1 map to band r?
• NONE implies "no filter" aka "white" rather than "unrecognised" (why UK spelling?) See also RFC-737.

#### Activity

No builds found.
Tim Jenness created issue -
John Parejko made changes -
Field Original Value New Value
Watchers John Parejko, Krzysztof Findeisen, Robert Lupton, Tim Jenness [ John Parejko, Krzysztof Findeisen, Robert Lupton, Tim Jenness ] Jim Bosch, John Parejko, Krzysztof Findeisen, Robert Lupton, Tim Jenness [ Jim Bosch, John Parejko, Krzysztof Findeisen, Robert Lupton, Tim Jenness ]
John Parejko made changes -
 Link This issue relates to DM-20763 [ DM-20763 ]
Hide
John Parejko added a comment -

An HSC person should chime in on this, but when Jim Bosch and I created the HSC FilterDefinitions in DM-20763), we explicitly made ENG-R1 abstract=r1.

According to Paul Price's comments on this slack thread, "Unrecognised" means "we don't know what filter it was", whereas "None" means either "irrelevant" (e.g. darks and biases) or "no filter in place". So without deeper consulting by HSC people, I don't think we can use "white" for the band of either of those cases.

Show
John Parejko added a comment - An HSC person should chime in on this, but when Jim Bosch and I created the HSC FilterDefinitions in DM-20763 ), we explicitly made ENG-R1 abstract=r1 . According to Paul Price 's comments on this slack thread , "Unrecognised" means "we don't know what filter it was", whereas "None" means either "irrelevant" (e.g. darks and biases) or "no filter in place". So without deeper consulting by HSC people, I don't think we can use "white" for the band of either of those cases.
Tim Jenness made changes -
 Watchers Jim Bosch, John Parejko, Krzysztof Findeisen, Robert Lupton, Tim Jenness [ Jim Bosch, John Parejko, Krzysztof Findeisen, Robert Lupton, Tim Jenness ] Jim Bosch, John Parejko, Krzysztof Findeisen, Paul Price, Robert Lupton, Tim Jenness [ Jim Bosch, John Parejko, Krzysztof Findeisen, Paul Price, Robert Lupton, Tim Jenness ]
Hide
Tim Jenness added a comment -

Using "r1" as the band seems to go against what we are defining band to mean. We use "r" for HSC-R and HSC-R2 so why wouldn't we use "r" for ENG-R1? It's still all r-band.

"no filter in place" is exactly what we mean by "white" as the band isn't it?

Paul Price's comment is not what is currently implemented because we have NONE filter matching unrecognized band. Paul Price's comment suggests that we need a NONE and a UNRECOGNIZED filter and distinct bands.

LSST instruments seem to be using "empty""white" and "unknown""unknown" for filter+band and I think the filter used for bias and darks is whatever filter is in the beam even if we know that the filter is not relevant. It would be nice to be consistent here.

Show
Tim Jenness added a comment - Using "r1" as the band seems to go against what we are defining band to mean. We use "r" for HSC-R and HSC-R2 so why wouldn't we use "r" for ENG-R1? It's still all r-band. "no filter in place" is exactly what we mean by "white" as the band isn't it? Paul Price 's comment is not what is currently implemented because we have NONE filter matching unrecognized band. Paul Price 's comment suggests that we need a NONE and a UNRECOGNIZED filter and distinct bands. LSST instruments seem to be using "empty" "white" and "unknown" "unknown" for filter+band and I think the filter used for bias and darks is whatever filter is in the beam even if we know that the filter is not relevant. It would be nice to be consistent here.
Hide
Robert Lupton added a comment - - edited

I think that ENG-R1 should be abstract r, but it's probably not all that important as there's essentially no data taken using it.

Show
Robert Lupton added a comment - - edited I think that ENG-R1  should be abstract r , but it's probably not all that important as there's essentially no data taken using it.
Hide
John Parejko added a comment -

I think we need some more guidance from HSC to define "NONE" and "Unrecognised" based on their actual usage by the instrument team and how they appear in the camera files. Jim and I implemented it based on the previous massive overload of aliases.

Show
John Parejko added a comment - I think we need some more guidance from HSC to define "NONE" and "Unrecognised" based on their actual usage by the instrument team and how they appear in the camera files. Jim and I implemented it based on the previous massive overload of aliases .
Hide
Tim Jenness added a comment -

Given Robert Lupton's comment I'm more than happy to fix the ENG-R1 – I don't see any reason for us to confuse the gen3 registry by leaving an "r1" band hanging around in there.

John Parejko I'm more than happy for Paul Price to comment but the current situation does not match the comment from him you posted above.

Show
Tim Jenness added a comment - Given Robert Lupton 's comment I'm more than happy to fix the ENG-R1 – I don't see any reason for us to confuse the gen3 registry by leaving an "r1" band hanging around in there. John Parejko I'm more than happy for Paul Price to comment but the current situation does not match the comment from him you posted above.
Hide
Jim Bosch added a comment -

I have no recollection of intentionally special-casing "r1" or "ENG-R1", and it's plausible I just missed it (and miscommunicated with John on that). I think I would have remembered doing it if I did have a good reason, unless it was just out of a no-unnecessary-changes paranoia, which might have been a good reason to leave it "r1" back then, but would not be a good reason to leave it "r1" now.

Nobody's going to be able to come up with a solid answer to NONE vs UNRECOGNI[ZS]ED without digging in and looking at the data that actually uses them, I fear, and I suspect that's not a good use of anyone's time. I'm not sure what to do about that.

Show
Jim Bosch added a comment - I have no recollection of intentionally special-casing "r1" or "ENG-R1", and it's plausible I just missed it (and miscommunicated with John on that). I think I would have remembered doing it if I did have a good reason, unless it was just out of a no-unnecessary-changes paranoia, which might have been a good reason to leave it "r1" back then, but would not be a good reason to leave it "r1" now. Nobody's going to be able to come up with a solid answer to NONE vs UNRECOGNI [ZS] ED without digging in and looking at the data that actually uses them, I fear, and I suspect that 's not a good use of anyone's time. I'm not sure what to do about that.
Tim Jenness made changes -
 Assignee Tim Jenness [ tjenness ]
Tim Jenness made changes -
 Labels SciencePipelines
Tim Jenness made changes -
 Team Architecture [ 10304 ]
Tim Jenness made changes -
 Status To Do [ 10001 ] In Progress [ 3 ]
Hide
Tim Jenness added a comment -

Paul Price can you take a quick look?

I do feel that we should match bands so if you object to UNRECOGNISED becoming "unknown" and NONE becoming "empty" then I still think we should unify the band name of "unknown" and "white" to match the recent RFC and obs_lsst.

I can easily change the metadata translator to convert UNRECOGNISED and NONE in the headers (are all those other aliases used in headers?) to unknown and empty but wanted your feedback first.

Show
Tim Jenness added a comment - Paul Price can you take a quick look? I do feel that we should match bands so if you object to UNRECOGNISED becoming "unknown" and NONE becoming "empty" then I still think we should unify the band name of "unknown" and "white" to match the recent RFC and obs_lsst. I can easily change the metadata translator to convert UNRECOGNISED and NONE in the headers (are all those other aliases used in headers?) to unknown and empty but wanted your feedback first.
Tim Jenness made changes -
 Reviewers Paul Price [ price ] Status In Progress [ 3 ] In Review [ 10004 ]
Hide
Paul Price added a comment -

Seems reasonable to me.

It's possible "Unrecognised" is Subaru's choice of value when the true value isn't available to the header writer, but it's at least equally likely (due to the English spelling) it's something I chose for when we didn't have the filter name in our list. In any case, you should have HSC correspond to other obs packages as much as possible, and we can worry about corner cases if and when they arise.

Show
Paul Price added a comment - Seems reasonable to me. It's possible "Unrecognised" is Subaru's choice of value when the true value isn't available to the header writer, but it's at least equally likely (due to the English spelling) it's something I chose for when we didn't have the filter name in our list. In any case, you should have HSC correspond to other obs packages as much as possible, and we can worry about corner cases if and when they arise.
Paul Price made changes -
 Status In Review [ 10004 ] Reviewed [ 10101 ]
Hide
Tim Jenness added a comment -

Okay. Thanks. I will go ahead with modifying the metadata translator to catch the aliases and convert them to standard form.

Show
Tim Jenness added a comment - Okay. Thanks. I will go ahead with modifying the metadata translator to catch the aliases and convert them to standard form.
Tim Jenness made changes -
 Story Points 0.5
Tim Jenness made changes -
 Resolution Done [ 10000 ] Status Reviewed [ 10101 ] Done [ 10002 ]
Tim Jenness made changes -
 Link This issue is triggered by RFC-737 [ RFC-737 ]

#### People

Assignee:
Tim Jenness
Reporter:
Tim Jenness
Reviewers:
Paul Price
Watchers:
Jim Bosch, John Parejko, Krzysztof Findeisen, Paul Price, Robert Lupton, Tim Jenness