Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31432

Problems with EFD timestamps and/or code

    XMLWordPrintable

Details

    • Bug
    • Status: Done
    • Resolution: Done
    • None
    • efd
    • None
    • SQuaRE
    • No

    Description

      I've been looking at the EFD timebase issues, and I believe there are definitely issues that need fixing. Perhaps I am confused, if so I would appreciate it if someone could point me at a description of how this is all supposed to work. I looked at an image from the latest AuxTel observing run on 04-Aug-21. I tried to distill the notebook down to the minimum necessary to see the problems. It is at this location (https://github.com/craiglagegit/ScratchStuff/blob/master/notebooks/Plot_Tracking_3_13Aug21.ipynb), or available at NCSA at /project/cslage/AuxTel/notebooks/Plot_Tracking_3_13Aug21.ipynb . I see three problems:

      (1) Why is the DATE keyword in the header ~30 seconds before DATE-BEG and DATE-END ?
      (2) I need to lie to EfdClient.select_time_series and tell it the UTC timestamps are in TAI, otherwise I get a warning and the whole dataset is shifted by ~30 seconds.
      (3) I need to override the default for internal_time_scale in merge_packed_time_series. The default is TAI, but I need to make it UTC for it all to work.

      If I do all of those things then the plots make sense. The plot below shows an exposure where after a small AzEl adjustment there is an allAxesInPosition timestamp followed by an exposure. The DATE-BEG and DATE-END timestamps in the header line up within milliseconds with the lsst.sal.ATCamera.logevent_shutterDetailedState timestamps in the EFD.

      Attachments

        1. Plot_Tracking_UTC_20Oct21.ipynb
          6 kB
          Simon Krughoff
        2. screenshot-1.png
          155 kB
          Simon Krughoff
        3. Screen Shot 2021-10-28 at 1.36.36 PM.png
          34 kB
          Craig Lage
        4. Tracking_Timebase_2021080400011_13Aug21.pdf
          16 kB
          Craig Lage
        5. Tracking_Timebase_2021101400011_14Oct21.pdf
          16 kB
          Craig Lage
        6. Tracking_Timebase_2021101400011_TAI_Shift_14Oct21.pdf
          17 kB
          Craig Lage

        Issue Links

          Activity

            cslage Craig Lage added a comment -

            krughoff I got the same plot as you did with the new code. It bothers me because the "All Axes In Position" timestamp should not occur until the mount has stabilized, and you can see that the mount is still moving at that point. It looks as though the packed time series is shifted to the right by a small amount (1-2 seconds?) relative to the others. In the past there has been discussion of an unexplained 2 second shift in the cRIO_timestamp. Could that still be there? I'll try to dig up the ticket that mentions this.

            cslage Craig Lage added a comment - krughoff I got the same plot as you did with the new code. It bothers me because the "All Axes In Position" timestamp should not occur until the mount has stabilized, and you can see that the mount is still moving at that point. It looks as though the packed time series is shifted to the right by a small amount (1-2 seconds?) relative to the others. In the past there has been discussion of an unexplained 2 second shift in the cRIO_timestamp. Could that still be there? I'll try to dig up the ticket that mentions this.

            If there is a small shift in the cRIO_timestamp, the client would not be able to take care of that

            krughoff Simon Krughoff (Inactive) added a comment - If there is a small shift in the cRIO_timestamp, the client would not be able to take care of that

            Here is the notebook I based on yours. The two changes I made were to use astropy.Time to do the conversion between UTC and TAI (cell 5), and I modified one query to use select_time_series instead of constructing the query by hand (cell 8). I think the convenience functions should be used as much as possible because it makes sure the queries are all formatted similarly and it also populates the query_history buffer which can help in debugging. Plot_Tracking_UTC_20Oct21.ipynb

            krughoff Simon Krughoff (Inactive) added a comment - Here is the notebook I based on yours. The two changes I made were to use astropy.Time to do the conversion between UTC and TAI (cell 5), and I modified one query to use select_time_series instead of constructing the query by hand (cell 8). I think the convenience functions should be used as much as possible because it makes sure the queries are all formatted similarly and it also populates the query_history buffer which can help in debugging. Plot_Tracking_UTC_20Oct21.ipynb
            cslage Craig Lage added a comment -

            tjennessSo I think we consider this ticket Done. Any further improvements will need to come from changes to the cRIO_timestamp in the EFD. Brian has created a separate ticket (https://jira.lsstcorp.org/browse/CAP-816) for that.

            cslage Craig Lage added a comment - tjenness So I think we consider this ticket Done. Any further improvements will need to come from changes to the cRIO_timestamp in the EFD. Brian has created a separate ticket ( https://jira.lsstcorp.org/browse/CAP-816 ) for that.
            tjenness Tim Jenness added a comment -

            Okay. I've marked it done.

            tjenness Tim Jenness added a comment - Okay. I've marked it done.

            People

              krughoff Simon Krughoff (Inactive)
              cslage Craig Lage
              Craig Lage
              Angelo Fausti, Craig Lage, Felipe Menanteau, Frossie Economou, Merlin Fisher-Levine, Patrick Ingraham, Simon Krughoff (Inactive), Steve Pietrowicz, Tiago Ribeiro, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.