Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: Firefly
-
Labels:None
-
Story Points:20
-
Epic Link:
-
Sprint:SUIT Sprint 2019-04, SUIT Sprint 2019-05
-
Team:Science User Interface
Description
Using multi-cast to do service discovery does not work in GKE environment, and also it requires low-level hardware configuration. DM-17601 was created to explore the alternate way to accomplish service discovery. It recommended using redis. This is the implementation ticket.
<text taken from PR>
Objective:
The objective is to implement an alternative to multicast cache peer discovery mechanism. Multicast is not supported in the cloud, making Firefly unable to scale horizontally.
The implemented alternative is to use a publish-subscribe messaging system for peer discovery. The flow is similar to multicast in which it relies on a periodic heartbeat message from each active peer to determine ‘liveliness’. Subscribing to the same channel determine the membership of the group.
Implementation Details:
A custom CachePeerProviderFactory is created to determine which discovery mechanism to use based on configuration.
Messaging is implemented using Redis server and Jedis client. The preferred deployment of Redis is via Docker. Jedis dependencies were added to Firefly.
copied from PR:
Tests:
HINTS: to see test’s debug messages, edit ./firefly/config/test/log4j-test.properties, replace log4j.logger.test=WARN with DEBUG
3 tests were created.
JsonHelperTest is the easiest to test. Simply run:
gradle :firefly:test -x jsTest --tests *JsonHelperTest
The next 2 tests require a running Redis. To start one locally..
docker run --name test-redis -p 6379:6379 -d redis
To test them:
gradle :firefly:test -x jsTest --tests *MessengerTest
gradle :firefly:test -x jsTest --tests *CachePeerProviderFactoryTest
To perf/stress test the messaging system..
gradle -Pcategory=edu.caltech.ipac.TestCategory$Perf :firefly:test -x jsTest --tests *MessengerTest
Also: cleanup prop files