The following image shows that there are no zero values send from the

 low-level controller.

Russell will check the data of low-level controller to CSC. The branch is u/rowen/print_telem in ts_mthexapod .

Russell Owen: The issue seems to also appear at NCSA:

https://lsst-chronograf-nts-efd.ncsa.illinois.edu/sources/1/dashboards/66?refresh=60s&lower=now%28%29%20-%205m

Plot upper right corner showing lsst.sal.MTHexapod.application.positionX:

SELECT "position0", "position1", "position2", "position3", "position4", "position5" FROM "efd"."autogen"."lsst.sal.MTHexapod.application" WHERE time > :dashboardTime: AND time < :upperDashboardTime:

That query is showing the "application" telemetry from both hexapods. It needs a "WHERE MTHexapodID = 1 (or 2)". When I add that the data no longer jumps. So this still appears to be something about interaction with the real low-level controller.

I see, there might have been a problem with Chronograph at NCSA. I just checked and it seems to work fine now at NCSA. Can you try and see if you can confirm?

Summit to be checked when CSC comes back.

There are also a number of displays in the CamHexapod summit dashboard https://chronograf-summit-efd.lsst.codes/sources/1/dashboards/8 that were missing "WHERE MTHexapodID = ...".

I fixed the problem as follows:

1) On the summit:

• Added a new user-settable variable named :SalIndex: with allowed values 1, 2
• Made all MTHexapod queries specify MTHexapodID = :SalIndex:
• Renamed the camera hexapod dashboard to MTHexapod: 1=Camera, 2=M2
• Confirmed that you can have the same dashboard open twice (e.g. in two different browser tabs or windows), each showing data for a different hexapod.
• Deleted the other MT hexapod dashboard, so we only have one.

At NCSA:

• Imported a copy of the summit dashboard with the same name
• Tested it
• Deleted the old MT hexapod dashboards.

So now we only have one to maintain on the summit, and I hope we can occasionally remember to duplicate it to our other sites.

Please check whether my updates to the Chronograf dashboards have fixed the problem. If they have then it is entirely due to displaying data from both hexapods in one graph and we'll have to be very careful about doing that in the future.

Thanks Russell, yes all looks good.  It looks like it was "just" the display issue. The new dashboard solution will really help to prevent this in the future.