Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Labels:None
-
Story Points:1
-
Epic Link:
-
Sprint:TSSW Sprint - Aug 19 - Aug 31
-
Team:Telescope and Site
Description
This story deploys the kafka writers (which write only to the cloud, not a local database) to publish SAL data from the pLAN network. A container with the writers can be deployed on pontus (10.0.100.200). Tiago Ribeiro has volunteered to help with the container aspects if required.
Attachments
Activity
Installing and running the Kafka writers with SAL 3.9 is really straightforward. Unfortunately, the pLAN currently runs on SAL 3.8. I tested the communication between 3.9 and 3.8 and, even though a listener won't segfault as it used to be the case, no messages are received.
I think that, unless there is a quick and easy way to build Kafka writers with SAL 3.8, we wait for the SAL upgrade in the lab to proceed with this task.
Patrick Ingraham is that ok?
But ok.
PR to ts_Dockerfiles with a dockerfile to build the writer container and a docker-compose file to launch it with the proper IP in the lab.
https://github.com/lsst-ts/ts_Dockerfiles/pull/30
I did a quick fix in salobj build process as part of this task:
https://github.com/lsst-ts/ts_salobj/pull/59 (Russell Owen will review it)
EFD is running in the lab and not in the cloud as originally intended. But this is a good thing.
To access the database one can try:
test-grafana-efd.lsst.codes
test-chronograf-efd.lsst.codes
test-schema-registry-efd.lsst.codes
(need to be in the LSST-WAP network)
Code shows that ATSpectrograph should be able to publish data, but not seeing it in the (Influx) EFD. Was it just not brought online?
All others that are currently running appear to have at least published something. Seeing lack of heartbeats but CSCs may just be offline.
Yes, the tables appears on-demand. Since the ATSpectrograph has not being brought up after the salkafka producer started, the tables are not there. I can bring it up thought.
Data appears in Influx EFD and visible using cronograf for all currently running components. Note that this task does NOT include verifying that all CSCs are correctly publishing telemetry nor is it a systematic verification that all published telemetry is arriving in the EFD.
Tiago Ribeiro can we close the ts_salobj pull request (without merging it) and delete the ticket branch? Or what should we do with it?
Tiago Ribeiro to verify if the DM-EFD Kafka brokers are receiving the messages from your SAL kafka writer deployment you can monitor the number of inbound messages per second at
https://grafana-efd-kafka.lsst.codes/
Open the Confluent Open Source dashboard, expand the Confluent Kafka panel and look at the "Messages In per second" graph.
Once that part is working I can deploy a new SAL transform app for the corresponding SAL component/subsystem) and we'll see the data flowing all the way to InfluxDB.