Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23064

Deploy an instance of the EFD at the NCSA test stand

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: efd
    • Labels:
      None

      Description

      Another instance of the EFD needs to be deployed at the NCSA test stand (see https://sqr-031.lsst.io/)

      The machine lsst-l1-cl-efd (141.124.238.172) was reserved for that initially. It satisfies the basic requirements to run a test stand deployment on Kubes (k3s) i.e. 16 cores, 45 GB memory, 1.8TB disk.

      However, we run into firewall issues that prevented k3s to work.  

      It was decided that this deployment should be done in a new k8s cluster at the NCSA test stand.

       

        Attachments

          Issue Links

          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-23176 Update ApiVersion for the EFD Helm Charts

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-23198 Test the local-path-provisioner on a k8s cluster

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-23201 Remove nginx-ingress from the deployment managed by Argo CD for the NCSA test stand environment.

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-23229 cp-helm-charts does not work with kubernetes 1.16

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-23419 Implementation plan for the LDF EFD

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-23446 Deploy the EFD on the LSP stable cluster at NCSA

          • Done

            Activity

            Ascending order - Click to sort in descending order
            5 older comments
            Hide
            Permalink
            afausti Angelo Fausti added a comment -

            Fixed the cp-helm-charts chart to make it compatible with Kubernetes 1.16 in DM-23229

            Show
            afausti Angelo Fausti added a comment - Fixed the cp-helm-charts chart to make it compatible with Kubernetes 1.16 in DM-23229
            Hide
            Permalink
            afausti Angelo Fausti added a comment -

            All services are accessible now, the main endpoints are:

            Chronograf: https://lsst-chronograf-nts-efd.ncsa.illinois.edu 
            InfluxDB: https://lsst-influxdb-nts-efd.ncsa.illinois.edu

            Tiago Ribeiro to configure the SAL Kafka producer to send data to the NCSA test stand EFD, you'll need the Kafka Schema Registry: https://lsst-schema-registry-nts-efd.ncsa.illinois.edu and access to the Kafka broker at lsst-kafka-0-nts-efd.ncsa.illinois.edu:31090

            I've just tested the EFD (https://sqr-031.lsst.io/#testing-the-efd) and I can reach both the Kafka schema registry and the Kafka broker, example:

            docker run -it --network=host edenhill/kafkacat:1.5.0 -b lsst-kafka-0-nts-efd.ncsa.illinois.edu:31090 -L

            Show
            afausti Angelo Fausti added a comment - All services are accessible now, the main endpoints are: Chronograf: https://lsst-chronograf-nts-efd.ncsa.illinois.edu InfluxDB: https://lsst-influxdb-nts-efd.ncsa.illinois.edu Tiago Ribeiro to configure the SAL Kafka producer to send data to the NCSA test stand EFD, you'll need the Kafka Schema Registry: https://lsst-schema-registry-nts-efd.ncsa.illinois.edu and access to the Kafka broker at lsst-kafka-0-nts-efd.ncsa.illinois.edu:31090 I've just tested the EFD ( https://sqr-031.lsst.io/#testing-the-efd ) and I can reach both the Kafka schema registry and the Kafka broker, example: docker run -it --network=host edenhill/kafkacat:1.5.0 -b lsst-kafka-0-nts-efd.ncsa.illinois.edu:31090 -L
            Hide
            Permalink
            afausti Angelo Fausti added a comment -

            Kapacitor is also working fine on the NCSA test stand.

            Show
            afausti Angelo Fausti added a comment - Kapacitor is also working fine on the NCSA test stand.
            Hide
            Permalink
            afausti Angelo Fausti added a comment -

            SAL Kafka producers are sending data.

            I've fixed a small issue with the connector configuration that was pointing to the wrong InfluxDB URL.

            Show
            afausti Angelo Fausti added a comment - SAL Kafka producers are sending data. I've fixed a small issue with the connector configuration that was pointing to the wrong InfluxDB URL.
            Hide
            Permalink
            afausti Angelo Fausti added a comment -

            SQR-034 was updated with information for this instance.

            Show
            afausti Angelo Fausti added a comment - SQR-034 was updated with information for this instance.

              People

              Assignee:
              afausti Angelo Fausti
              Reporter:
              afausti Angelo Fausti
              Watchers:
              Angelo Fausti, Matthew Thomas Long [X] (Inactive), Michael Reuter, Michelle Butler [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.