Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31432

Problems with EFD timestamps and/or code

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: efd
    • Labels:
      None
    • Team:
      SQuaRE
    • Urgent?:
      No

      Description

      I've been looking at the EFD timebase issues, and I believe there are definitely issues that need fixing. Perhaps I am confused, if so I would appreciate it if someone could point me at a description of how this is all supposed to work. I looked at an image from the latest AuxTel observing run on 04-Aug-21. I tried to distill the notebook down to the minimum necessary to see the problems. It is at this location (https://github.com/craiglagegit/ScratchStuff/blob/master/notebooks/Plot_Tracking_3_13Aug21.ipynb), or available at NCSA at /project/cslage/AuxTel/notebooks/Plot_Tracking_3_13Aug21.ipynb . I see three problems:

      (1) Why is the DATE keyword in the header ~30 seconds before DATE-BEG and DATE-END ?
      (2) I need to lie to EfdClient.select_time_series and tell it the UTC timestamps are in TAI, otherwise I get a warning and the whole dataset is shifted by ~30 seconds.
      (3) I need to override the default for internal_time_scale in merge_packed_time_series. The default is TAI, but I need to make it UTC for it all to work.

      If I do all of those things then the plots make sense. The plot below shows an exposure where after a small AzEl adjustment there is an allAxesInPosition timestamp followed by an exposure. The DATE-BEG and DATE-END timestamps in the header line up within milliseconds with the lsst.sal.ATCamera.logevent_shutterDetailedState timestamps in the EFD.

        Attachments

          Issue Links

            Activity

            Hide
            cslage Craig Lage added a comment -

            Simon Krughoff I got the same plot as you did with the new code. It bothers me because the "All Axes In Position" timestamp should not occur until the mount has stabilized, and you can see that the mount is still moving at that point. It looks as though the packed time series is shifted to the right by a small amount (1-2 seconds?) relative to the others. In the past there has been discussion of an unexplained 2 second shift in the cRIO_timestamp. Could that still be there? I'll try to dig up the ticket that mentions this.

            Show
            cslage Craig Lage added a comment - Simon Krughoff I got the same plot as you did with the new code. It bothers me because the "All Axes In Position" timestamp should not occur until the mount has stabilized, and you can see that the mount is still moving at that point. It looks as though the packed time series is shifted to the right by a small amount (1-2 seconds?) relative to the others. In the past there has been discussion of an unexplained 2 second shift in the cRIO_timestamp. Could that still be there? I'll try to dig up the ticket that mentions this.
            Hide
            krughoff Simon Krughoff added a comment -

            If there is a small shift in the cRIO_timestamp, the client would not be able to take care of that

            Show
            krughoff Simon Krughoff added a comment - If there is a small shift in the cRIO_timestamp, the client would not be able to take care of that
            Hide
            krughoff Simon Krughoff added a comment -

            Here is the notebook I based on yours. The two changes I made were to use astropy.Time to do the conversion between UTC and TAI (cell 5), and I modified one query to use select_time_series instead of constructing the query by hand (cell 8). I think the convenience functions should be used as much as possible because it makes sure the queries are all formatted similarly and it also populates the query_history buffer which can help in debugging. Plot_Tracking_UTC_20Oct21.ipynb

            Show
            krughoff Simon Krughoff added a comment - Here is the notebook I based on yours. The two changes I made were to use astropy.Time to do the conversion between UTC and TAI (cell 5), and I modified one query to use select_time_series instead of constructing the query by hand (cell 8). I think the convenience functions should be used as much as possible because it makes sure the queries are all formatted similarly and it also populates the query_history buffer which can help in debugging. Plot_Tracking_UTC_20Oct21.ipynb
            Hide
            cslage Craig Lage added a comment -

            Tim JennessSo I think we consider this ticket Done. Any further improvements will need to come from changes to the cRIO_timestamp in the EFD. Brian has created a separate ticket (https://jira.lsstcorp.org/browse/CAP-816) for that.

            Show
            cslage Craig Lage added a comment - Tim Jenness So I think we consider this ticket Done. Any further improvements will need to come from changes to the cRIO_timestamp in the EFD. Brian has created a separate ticket ( https://jira.lsstcorp.org/browse/CAP-816 ) for that.
            Hide
            tjenness Tim Jenness added a comment -

            Okay. I've marked it done.

            Show
            tjenness Tim Jenness added a comment - Okay. I've marked it done.

              People

              Assignee:
              krughoff Simon Krughoff
              Reporter:
              cslage Craig Lage
              Reviewers:
              Craig Lage
              Watchers:
              Angelo Fausti, Craig Lage, Felipe Menanteau, Frossie Economou, Merlin Fisher-Levine, Patrick Ingraham, Simon Krughoff, Steve Pietrowicz, Tiago Ribeiro, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.