Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-38511

ESS:301 should go into FAULT if connection breaks

    XMLWordPrintable

    Details

    • Story Points:
      2
    • Sprint:
      TSSW Sprint - Apr 10 - Apr 24
    • Team:
      Telescope and Site
    • Urgent?:
      No

      Description

      The ESS:301 CSC is having trouble staying connected to the low-level controller. If it loses connection, it should go into FAULT if it fails to reconnect.

        Attachments

          Activity

          Hide
          wvreeven Wouter van Reeven added a comment -

          The issue was that the read loop was started in an asyncio task and exceptions from those tasks don't get automatically propagated to the calling code. Instead, the calling code needs to await the task and then check if an exception has happened. Once I implemented that in the Young32400WeatherStationDataClient, the timeout mechanism started working and the ESS CSC started going to FAULT state in case of an error.

          In a video chat with Russell this was discussed and we decided to hoist the read loop plus exception monitoring into a new BaseReadLoopDataClient subclass of BaseDataClient and convert as many existing DataClients as possible to subclasses of that new subclass. The one possible execption may be the ts_ess_labjack LabJackAccelerometerDataClient, which gets data pushed via a stream. It may be possible to convert that one as well but that is now under investigation.

          Show
          wvreeven Wouter van Reeven added a comment - The issue was that the read loop was started in an asyncio task and exceptions from those tasks don't get automatically propagated to the calling code. Instead, the calling code needs to await the task and then check if an exception has happened. Once I implemented that in the Young32400WeatherStationDataClient, the timeout mechanism started working and the ESS CSC started going to FAULT state in case of an error. In a video chat with Russell this was discussed and we decided to hoist the read loop plus exception monitoring into a new BaseReadLoopDataClient subclass of BaseDataClient and convert as many existing DataClients as possible to subclasses of that new subclass. The one possible execption may be the ts_ess_labjack LabJackAccelerometerDataClient, which gets data pushed via a stream. It may be possible to convert that one as well but that is now under investigation.
          Show
          wvreeven Wouter van Reeven added a comment - ts_ess_common PR: https://github.com/lsst-ts/ts_ess_common/pull/33 ts_ess_csc PR: https://github.com/lsst-ts/ts_ess_csc/pull/92
          Hide
          rowen Russell Owen added a comment -

          Reviewed on github. Thank you for making these useful changes.

          Show
          rowen Russell Owen added a comment - Reviewed on github. Thank you for making these useful changes.

            People

            Assignee:
            wvreeven Wouter van Reeven
            Reporter:
            mareuter Michael Reuter
            Reviewers:
            Russell Owen
            Watchers:
            Michael Reuter, Russell Owen, Wouter van Reeven
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.