Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-4763

Exploration of In-memory database packages used in time critical applications

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Templates:
    • Story Points:
      12
    • Epic Link:
    • Team:
      Data Facility

      Description

      Begin evaluation of potential in-memory data storage tools - selecting memcached and redis to start. - 4

      With the intent to gain familiarity with these tools, procurred introductory volume on redis and began writing prototype python code to prototype lists, hashes, and lists of hashes. Sketched out and implemented base python class with virtual save method, then wrote child classes for replicators, replicator health, replicator jobs etc. and tested this code and implemented the save methods. - 4

      Installed the above code on a nebula instance that acts as a job manager, then ran real job messages through the system and simulated task assignment and completion, using redis to track jobs. - 2

      Exploring how a logging and visualization harness might be included so job activity could 1) be observed in realtime, and 2) so a session could be played back as an after action review to investigate errors, bottlenecks, cold restart behavior, etc. - 2

      To finish out this story, redis replication must be included in the above prototype.

        Attachments

          Activity

            People

            • Assignee:
              jparsons James Parsons
              Reporter:
              jparsons James Parsons
              Watchers:
              James Parsons
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel