# LATISS UTC/TAI problems were fixed on 2021-02-12

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
0.5
• Team:
Architecture
• Urgent?:
No

#### Description

The obs_lsst LATISS metadata translator assumes that data after RASTART_IS_BAD is off by 37s, but in fact this was fixed on 2021-02-12. Please fix the metadata translator.

I'd do it myself, but I don't understand the comment

 # RASTART/DECSTART/RAEND/DECEND used wrong telescope location before this # 2020-02-01T00:00 we fixed the telescope location, but RASTART is still # in mount coordinates, so off by pointing model. RASTART_IS_BAD = Time("2020-05-01T00:00", format="isot", scale="utc") 

and would prefer you to clarify that the problem being fixed is not this one.

There is a more serious problem too, namely that the correction of c. 0.154167 is applied multiple times. The RAEND in the header is 130.211695172908, but:

 exposure.getMetadata().get("RAEND"), butler.get("raw_md", dataId).get("RAEND") (130.82836183957568, 130.36586183957567) 

#### Activity

No builds found.
Robert Lupton created issue -
Hide
Robert Lupton added a comment -

I'm going to push an end date for the UTC correction to u/rhl/DM-29187 (which means that the LATISS data headers are correct), but I'm not planning to take over this ticket.

Show
Robert Lupton added a comment - I'm going to push an end date for the UTC correction to u/rhl/ DM-29187 (which means that the LATISS data headers are correct), but I'm not planning to take over this ticket.
Hide
Robert Lupton added a comment -

Furthermore the RASTART test fails:

  def test_fix_header(self):  from astro_metadata_translator import fix_header  from astro_metadata_translator.tests import read_test_file  # Test that header fix up is working  # Not all headers are used in metadata translation  test_data = (  ("latiss-AT_O_20210212_000006.yaml",  dict(RASTART=260.1785517385836)),  ("latiss-AT_O_20210210_000011.yaml",  dict(RASTART=355.41750341182313)),  ) 

It appears to use test data taken on the day that this problem was fixed!

Show
Robert Lupton added a comment - Furthermore the RASTART test fails: def test_fix_header(self): from astro_metadata_translator import fix_header from astro_metadata_translator.tests import read_test_file # Test that header fix up is working # Not all headers are used in metadata translation test_data = ( ("latiss-AT_O_20210212_000006.yaml", dict(RASTART=260.1785517385836)), ("latiss-AT_O_20210210_000011.yaml", dict(RASTART=355.41750341182313)), ) It appears to use test data taken on the day that this problem was fixed!
Hide
Robert Lupton added a comment -

Oh, sorry. The how-to-repeat is

 repo = "/lsstdata/offline/teststand/NCSA_auxTel/gen2repo" butler = dafPersist.Butler(repo)   dataId = dict(dayObs="2021-03-11" , seqNum=4)   calexp = butler.get("raw", dataId) print(exposure.getMetadata().get("RAEND"), butler.get("raw_md", dataId).get("RAEND")) 

Show
Robert Lupton added a comment - Oh, sorry. The how-to-repeat is repo = "/lsstdata/offline/teststand/NCSA_auxTel/gen2repo" butler = dafPersist.Butler(repo)   dataId = dict(dayObs="2021-03-11" , seqNum=4)   calexp = butler.get("raw", dataId) print(exposure.getMetadata().get("RAEND"), butler.get("raw_md", dataId).get("RAEND"))
Field Original Value New Value
Link This issue relates to DM-28783 [ DM-28783 ]
Hide
Tim Jenness added a comment -

That RASTART_IS_BAD date is when the values were complete garbage and should not be used.

https://github.com/lsst/obs_lsst/blob/master/python/lsst/obs/lsst/translators/latiss.py#L63 defines the RASTART_IS_HOURS date at which that was fixed to be degrees. There is no date for fixing the 37s correction because no-one had reported it fixed (see the discussion in DM-28783)

Show
Tim Jenness added a comment - That RASTART_IS_BAD date is when the values were complete garbage and should not be used. https://github.com/lsst/obs_lsst/blob/master/python/lsst/obs/lsst/translators/latiss.py#L63 defines the RASTART_IS_HOURS date at which that was fixed to be degrees. There is no date for fixing the 37s correction because no-one had reported it fixed (see the discussion in DM-28783 )
Hide
Tim Jenness added a comment -

Do I assume that after the 12th the RASTART/DECSTART are correct in the header and do not need any fixes?

I think for the double correction I'm going to have to finally bite the bullet and add a sentinel to the header indicating that it's been fixed. I've been wary of doing that because I didn't want to surprise people when something weird appears in their header. Maybe I should use something like HIERARCH ASTROMETADATA HEADER FIXED = TRUE.

Show
Tim Jenness added a comment - Do I assume that after the 12th the RASTART/DECSTART are correct in the header and do not need any fixes? I think for the double correction I'm going to have to finally bite the bullet and add a sentinel to the header indicating that it's been fixed. I've been wary of doing that because I didn't want to surprise people when something weird appears in their header. Maybe I should use something like HIERARCH ASTROMETADATA HEADER FIXED = TRUE .
Hide
Tim Jenness added a comment -

To clarify the situation inherited from DM-28783:

• At 2021-02-11T1845Z the headers were decimal degrees but had a 37s error in RASTART.
• At 2021-02-12T0000Z the 37s was fixed.

So it was less than 6 hours of data where the 37s error was a real problem.

Show
Tim Jenness added a comment - To clarify the situation inherited from DM-28783 : At 2021-02-11T1845Z the headers were decimal degrees but had a 37s error in RASTART. At 2021-02-12T0000Z the 37s was fixed. So it was less than 6 hours of data where the 37s error was a real problem.
Hide
Tim Jenness added a comment -

Furthermore, the RASTART test will fail because even as I was fixing the translator to deal with the problem of the time offset the problem had been fixed but I had not been informed. This means that the data I was looking at was already correct. The reason I thought it needed fixing was that this did make the number closer to the target RA (260.178 [fixed] vs 260.181 [demand] vs 260.024 [header]).

Show
Tim Jenness added a comment - Furthermore, the RASTART test will fail because even as I was fixing the translator to deal with the problem of the time offset the problem had been fixed but I had not been informed. This means that the data I was looking at was already correct. The reason I thought it needed fixing was that this did make the number closer to the target RA (260.178 [fixed] vs 260.181 [demand] vs 260.024 [header] ).
 Status To Do [ 10001 ] In Progress [ 3 ]
 Component/s obs_lsst [ 16504 ]
 Story Points 0.5 Team Architecture [ 10304 ]
Hide
Tim Jenness added a comment -

I took your fix and sorted out the test.

I'm going to do the double fixup fix in a different ticket because that can be done entirely inside astro_metadata_translator.

Show
Tim Jenness added a comment - I took your fix and sorted out the test. I'm going to do the double fixup fix in a different ticket because that can be done entirely inside astro_metadata_translator.
 Reviewers Robert Lupton [ rhl ] Status In Progress [ 3 ] In Review [ 10004 ]
 Link This issue relates to DM-29218 [ DM-29218 ]
Hide
Robert Lupton added a comment -

Looks fine

Show
Robert Lupton added a comment - Looks fine
 Status In Review [ 10004 ] Reviewed [ 10101 ]
 Resolution Done [ 10000 ] Status Reviewed [ 10101 ] Done [ 10002 ]

#### People

Assignee:
Tim Jenness
Reporter:
Robert Lupton
Reviewers:
Robert Lupton
Watchers:
Michael Reuter, Robert Lupton, Tim Jenness