Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-19745

Single machine deployment of the DM-EFD using k3s ("kubes")

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      After evaluating minikube, microk8s and k3s we decided to use ks3 for a single machine deployment of the DM-EFD which will be tested at the lab.

      This story captures the work to adapt ourTerraform deployment to support k3s and test the deployment locally (on a laptop) and on the "EFD machine" at the lab.

      If necessary we can have an actual cluster adding more workers to k3s.

        Attachments

          Issue Links

            Activity

            Hide
            afausti Angelo Fausti added a comment -

            To make our terraform deployment work on kubes the following changes were required:

            • Specify storage class for kafka, influxdb and prometheus-operator, for k3 use 'local-path'
            • Specify size of influxdb disk
            • Control if telegraf client is deployed as daemonset, for single node deployment with k3s set it to false
            • Configure the number of load balancers services. Previously there was the assumption that the number of loadbalancer services was the same as the number of brokers, and it was fixed to 3 in the terraform deployment. When doing single machine deployment we still can have n>1 brokers but only 1 loadbalancer.

            https://github.com/lsst-sqre/terraform-efd/pull/24
            https://github.com/lsst-sqre/cp-helm-charts/pull/5

            Show
            afausti Angelo Fausti added a comment - To make our terraform deployment work on kubes the following changes were required: Specify storage class for kafka, influxdb and prometheus-operator, for k3 use 'local-path' Specify size of influxdb disk Control if telegraf client is deployed as daemonset, for single node deployment with k3s set it to false Configure the number of load balancers services. Previously there was the assumption that the number of loadbalancer services was the same as the number of brokers, and it was fixed to 3 in the terraform deployment. When doing single machine deployment we still can have n>1 brokers but only 1 loadbalancer. https://github.com/lsst-sqre/terraform-efd/pull/24 https://github.com/lsst-sqre/cp-helm-charts/pull/5
            Hide
            afausti Angelo Fausti added a comment -

            Create "top level" terraform module for the EFD k3s deployment.

            At this point the k3s cluster is being created manually, so effectively this module adds the configuration for the template, null, local, aws and kubernetes providers.

            https://github.com/lsst-sqre/terraform-efd-k3s/pull/1

            Show
            afausti Angelo Fausti added a comment - Create "top level" terraform module for the EFD k3s deployment. At this point the k3s cluster is being created manually, so effectively this module adds the configuration for the template, null, local, aws and kubernetes providers. https://github.com/lsst-sqre/terraform-efd-k3s/pull/1
            Hide
            afausti Angelo Fausti added a comment -

            Configuration for the EFD k3s deployment can be found at SQuaRE 1Password, and the terragrunt-live-test repo was updated to add the new dpeloyment variables:

            https://github.com/lsst-sqre/terragrunt-live-test/pull/7

            Show
            afausti Angelo Fausti added a comment - Configuration for the EFD k3s deployment can be found at SQuaRE 1Password, and the terragrunt-live-test repo was updated to add the new dpeloyment variables: https://github.com/lsst-sqre/terragrunt-live-test/pull/7
            Hide
            afausti Angelo Fausti added a comment - - edited

            In SQR-31 we demonstrate that we can deploy the DM-EFD on a single machine with Docker and k3s ("kubes"), a lightweight Kubernetes using the same Terraform modules and Helm charts that we used in our Google Could deployment. We also provide instructions on how to operate and use the DM-EFD system.

            https://github.com/lsst-sqre/sqr-031/pull/2

            Show
            afausti Angelo Fausti added a comment - - edited In SQR-31 we demonstrate that we can deploy the DM-EFD on a single machine with Docker and k3s ("kubes"), a lightweight Kubernetes using the same Terraform modules and Helm charts that we used in our Google Could deployment. We also provide instructions on how to operate and use the DM-EFD system. https://github.com/lsst-sqre/sqr-031/pull/2
            Hide
            afausti Angelo Fausti added a comment -

            This setup will be tested with the ATCam as soon as we have SAL 3.9 or later running the kafka writers at the lab.

            I am closing this ticket for now.

            Show
            afausti Angelo Fausti added a comment - This setup will be tested with the ATCam as soon as we have SAL 3.9 or later running the kafka writers at the lab. I am closing this ticket for now.

              People

              Assignee:
              afausti Angelo Fausti
              Reporter:
              afausti Angelo Fausti
              Watchers:
              Angelo Fausti, Frossie Economou, Joshua Hoblitt, Simon Krughoff
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: