Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18341

implement the multi-cast alternate recommended by DM-17601

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Firefly
    • Labels:
      None
    • Story Points:
      20
    • Epic Link:
    • Sprint:
      SUIT Sprint 2019-04, SUIT Sprint 2019-05
    • Team:
      Science User Interface

      Description

      Using multi-cast to do service discovery does not work in GKE environment, and also it requires low-level hardware configuration. DM-17601 was created to explore the alternate way to accomplish service discovery. It recommended using redis. This is the implementation ticket. 

       

      <text taken from PR>

      Objective:
      The objective is to implement an alternative to multicast cache peer discovery mechanism. Multicast is not supported in the cloud, making Firefly unable to scale horizontally.

      The implemented alternative is to use a publish-subscribe messaging system for peer discovery. The flow is similar to multicast in which it relies on a periodic heartbeat message from each active peer to determine ‘liveliness’. Subscribing to the same channel determine the membership of the group.

      Implementation Details:
      A custom CachePeerProviderFactory is created to determine which discovery mechanism to use based on configuration.
      Messaging is implemented using Redis server and Jedis client. The preferred deployment of Redis is via Docker. Jedis dependencies were added to Firefly.

       

        Attachments

          Issue Links

            Activity

            xiuqin Xiuqin Wu [X] (Inactive) created issue -
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Field Original Value New Value
            Epic Link DM-8765 [ 28604 ]
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Link This issue is triggered by DM-17601 [ DM-17601 ]
            loi Loi Ly made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Story Points 10 20
            loi Loi Ly made changes -
            Remote Link This issue links to "Pull Request (Web Link)" [ 20320 ]
            loi Loi Ly made changes -
            Description Using multi-cast to do service discovery does not work in GKE environment, and also it requires low-level hardware configuration. DM-17601 was created to explore the alternate way to accomplish service discovery. It recommended using redis. This is the implementation ticket.  Using multi-cast to do service discovery does not work in GKE environment, and also it requires low-level hardware configuration. DM-17601 was created to explore the alternate way to accomplish service discovery. It recommended using redis. This is the implementation ticket. 

             

            <text taken from PR>

            *Objective:*
            The objective is to implement an alternative to multicast cache peer discovery mechanism. Multicast is not supported in the cloud, making Firefly unable to scale horizontally.

            The implemented alternative is to use a publish-subscribe messaging system for peer discovery. The flow is similar to multicast in which it relies on a periodic heartbeat message from each active peer to determine ‘liveliness’. Subscribing to the same channel determine the membership of the group.

            *Implementation Details:*
            A custom CachePeerProviderFactory is created to determine which discovery mechanism to use based on configuration.
            Messaging is implemented using Redis server and Jedis client. The preferred deployment of Redis is via Docker. Jedis dependencies were added to Firefly.

             
            loi Loi Ly made changes -
            Reviewers Tatiana Goldina, Trey Roby [ tatianag, roby ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            Hide
            xiuqin Xiuqin Wu [X] (Inactive) added a comment -

            copied from PR:

            Tests:
            HINTS: to see test’s debug messages, edit ./firefly/config/test/log4j-test.properties, replace log4j.logger.test=WARN with DEBUG

            3 tests were created.
            JsonHelperTest is the easiest to test. Simply run:
            gradle :firefly:test -x jsTest --tests *JsonHelperTest

            The next 2 tests require a running Redis. To start one locally..
            docker run --name test-redis -p 6379:6379 -d redis

            To test them:
            gradle :firefly:test -x jsTest --tests *MessengerTest
            gradle :firefly:test -x jsTest --tests *CachePeerProviderFactoryTest

            To perf/stress test the messaging system..
            gradle -Pcategory=edu.caltech.ipac.TestCategory$Perf :firefly:test -x jsTest --tests *MessengerTest

            Also: cleanup prop files

            • move mail.prop into common.prop
            • share common.prop with other projects
            • move differences in other projects’ common.prop into .prop

            Show
            xiuqin Xiuqin Wu [X] (Inactive) added a comment - copied from PR: Tests: HINTS: to see test’s debug messages, edit ./firefly/config/test/log4j-test.properties, replace log4j.logger.test=WARN with DEBUG 3 tests were created. JsonHelperTest is the easiest to test. Simply run: gradle :firefly:test -x jsTest --tests *JsonHelperTest The next 2 tests require a running Redis. To start one locally.. docker run --name test-redis -p 6379:6379 -d redis To test them: gradle :firefly:test -x jsTest --tests *MessengerTest gradle :firefly:test -x jsTest --tests *CachePeerProviderFactoryTest To perf/stress test the messaging system.. gradle -Pcategory=edu.caltech.ipac.TestCategory$Perf :firefly:test -x jsTest --tests *MessengerTest Also: cleanup prop files move mail.prop into common.prop share common.prop with other projects move differences in other projects’ common.prop into .prop
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Sprint SUIT Sprint 2019-04 [ 861 ] SUIT Sprint 2019-04, SUIT Sprint 2019-05 [ 861, 874 ]
            Hide
            loi Loi Ly added a comment -

            Work is done.  It's currently deployed in '-int'.

            Show
            loi Loi Ly added a comment - Work is done.  It's currently deployed in '-int'.
            loi Loi Ly made changes -
            Resolution Done [ 10000 ]
            Status In Review [ 10004 ] Done [ 10002 ]
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Remote Link This issue links to "Page (Confluence)" [ 20762 ]

              People

              • Assignee:
                loi Loi Ly
                Reporter:
                xiuqin Xiuqin Wu [X] (Inactive)
                Reviewers:
                Tatiana Goldina, Trey Roby
                Watchers:
                Emmanuel Joliet, Gregory Dubois-Felsmann, Loi Ly, Tatiana Goldina, Trey Roby, Xiuqin Wu [X] (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel