Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ts_aos
-
Labels:
-
Story Points:3
-
Epic Link:
-
Sprint:TSSW Sprint - Apr 15 - Apr 27
-
Team:Telescope and Site
Description
This task will evaluate the integration between influxDB and influx_writer of SAL. This task will also benchmark the performance of InfluxDB to use the SSD, NVME, or hard disk.
Compare the MariaDB quary language and InfluxDB quary language.
Evaluation steps:
- Check with the AT CSC owners with the telemetry rate according to the page: https://confluence.lsstcorp.org/display/SYSENG/Auxiliary+Telescope+Control+Computers+in+Tucson
- Clone the ts_sal repository to the /home/ttsai. Install the SAL and build the AT CSC test scripts I need. Make sure the sal version is 3.9 and opensplice version is 6.9.
- Change the test telemetry script to run infinity times (iseq in C++) with the correct telemetry frequency (delay_1s in C, the first argument is sec and the second is nano-sec).
- Run the test for SSD, NVME, and Disk.
- The path variable of "LSST_EFD_HOST" gives the IP_address of influxDB.
- I can do "./some_writer >& log.1 &" to run the writer in the background. And I can use "tail -f log.1 to see the log file".
- Once I changed the cpp file, I can go to the "stand_alone" directory and do "make -f Makefile.sacpp_blahblak_pub" to build the code in the same directory.
- I can also do the above modifications by python.
Attachments
Activity
We noticed that the InfluxDB ingested the similar amounts of data point between 20 Hz and 15 Hz. All of them are smaller than the amount of theoretical data point. Dave suggested to compare the default InfluxDB setting with DM's.
However, after the comparison, there is no big difference. Dave decided to let me to check the timestamp in InfluxDB first to try to figure out why and when the data loss.
Extracted the csv data out.
To get the final value, we can do something like: "influx -database 'EFD' -execute 'SELECT * FROM PointingComponent_timeAndDate GROUP BY * ORDER BY DESC LIMIT 1' -precision ms -format 'csv' > test.csv".
https://stackoverflow.com/questions/29193898/influxdb-getting-only-last-value-in-query
It looks like the influxwriter can only write the speed of ~0.1 sec.
Excellent summary of progress thus far. Next sprint we will add tasks
to investigate the rate limiting issue.
Dave Mills
added a comment - Excellent summary of progress thus far. Next sprint we will add tasks
to investigate the rate limiting issue.
We noticed that there is the message loss for only one CSC with high frequency (10 Hz). After the discussion with Dave, he will provide a new influxwriter for the test. In this new writer, each http API by writer will send one topic value in one time.