Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23731

Create microservice for LSST the Docs that sends events to Roundtable Events (Kafka)

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      So that the new search service can respond to events such as an edition on LSST the Docs being updated, or a new product being created, we want to integrate LSST the Docs with Roundtable's SQuaRE Events. Events, which is Kafka-based, is the main system for event-driven service coordination in Roundtable. The excepted outcomes are:

      Given that LTD Keeper is Flask based, I think it makes sense to have the integration with SQuaRE Events as a separate microservice. This allows LTD Keeper to focus on maintaining the DB and service its REST API. Thus LTD Events will a microservice that integrates both SQuaRE Events and LTD Keeper.

      1. Create an "ltd-indexer service based on the roundtable_aiohttp_bot template.
      2. Receive posts from ltd-keeper (i.e. a webhook) indicating that an edition is updated (via an internal in-cluster endpoint)
      3. Add this webhook functionality to LTD Keeper
      4. Send a Kafka message on SQuaRE events indicating an edition is updated.

        Attachments

          Issue Links

            Activity

            Hide
            jsick Jonathan Sick added a comment -

            Released version 0.1.0 of LTD Events and deployed it to Roundtable. Change log:

            This is the first release of LTD Events:

            • The /webhook HTTP endpoint (accessible only from within a Kubernetes cluster) accepts webhook payloads from `LTD Keeper <https://ltd-keeper.lsst.io>.
              Currently the webhook only accepts edition.updated events from LTD Keeper.
            • LTD Events forwards edition.updated events to the ltd.events Kafka topic using the ltd.edition_update_v1 Avro schema.
            Show
            jsick Jonathan Sick added a comment - Released version 0.1.0 of LTD Events and deployed it to Roundtable. Change log: This is the first release of LTD Events: The /webhook  HTTP endpoint (accessible only from within a Kubernetes cluster) accepts webhook payloads from `LTD Keeper < https://ltd-keeper.lsst.io >. Currently the webhook only accepts edition.updated events from LTD Keeper. LTD Events forwards edition.updated  events to the ltd.events  Kafka topic using the ltd.edition_update_v1 Avro schema.

              People

              Assignee:
              jsick Jonathan Sick
              Reporter:
              jsick Jonathan Sick
              Watchers:
              Jonathan Sick
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.