Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5194

Deploy ltd-keeper as a Docker Container

    Details

      Description

      ltd-keeper should be deployed as a Docker container as a best practice for maintainable cloud microservices.

      This involves writing a Dockerfile committed to the ltd-keeper repo and demonstrating that the container can be stood up on Google Container Engine.

      I plan on use data-containers attached to the service’s container to maintain the sqlite DB. This ticket should document how to operate ltd-keeper and apply updates to both the ltd-keeper app and DB migrations..

      This ticket also involves initial overhead in researching Docker/Kubernetes.

        Attachments

          Issue Links

            Activity

            jsick Jonathan Sick created issue -
            jsick Jonathan Sick made changes -
            Field Original Value New Value
            Epic Link DM-1139 [ 13948 ]
            jsick Jonathan Sick made changes -
            Link This issue relates to DM-4950 [ DM-4950 ]
            jsick Jonathan Sick made changes -
            Link This issue relates to DM-5291 [ DM-5291 ]
            jsick Jonathan Sick made changes -
            Epic Link DM-1139 [ 13948 ]
            jsick Jonathan Sick made changes -
            Epic Link DM-5404 [ 23211 ]
            jsick Jonathan Sick made changes -
            Description ltd-keeper should be deployed as a Docker container as a best practice for maintainable cloud microservices.

            This involves writing a Dockerfile committed to the ltd-keeper repo and demonstrating that the container can be stood up on AWS.

            I plan on use data-containers attached to the service’s container to maintain the sqlite DB and the private configuration file. This ticket should document how to update the ltd-keeper container without affecting the DB data container.

            This ticket also involves initial overhead in researching Docker.
            ltd-keeper should be deployed as a Docker container as a best practice for maintainable cloud microservices.

            This involves writing a Dockerfile committed to the ltd-keeper repo and demonstrating that the container can be stood up on Google Container Engine.

            I plan on use data-containers attached to the service’s container to maintain the sqlite DB. This ticket should document how to operate ltd-keeper and apply updates to both the ltd-keeper app *and* DB migrations..

            This ticket also involves initial overhead in researching Docker/Kubernetes.
            Hide
            jsick Jonathan Sick added a comment -
            • Created a Google Cloud Platform account for SQuaRE (creds in 1Password)
            • Got the node.js tutorial to work but had issues with tutorials hosted by Google itself, such as the Wordpress with persistent storage tutorial. I think now that I was having trouble getting my pods scheduled on the cluster.
            Show
            jsick Jonathan Sick added a comment - Created a Google Cloud Platform account for SQuaRE (creds in 1Password) Got the node.js tutorial to work but had issues with tutorials hosted by Google itself, such as the Wordpress with persistent storage tutorial. I think now that I was having trouble getting my pods scheduled on the cluster.
            jsick Jonathan Sick made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            jsick Jonathan Sick made changes -
            Story Points 1.4
            Hide
            jsick Jonathan Sick added a comment - - edited

            Question of how to do TLS termination in a Kubernetes service load balancer. Options are:

            1. Use a Compute Engine load balancer with HTTPS - https://cloud.google.com/compute/docs/load-balancing/http/
            2. Add an Nginx pod that does TLS termination - e.g. https://github.com/GoogleCloudPlatform/kube-jenkins-imager
            3. Use the new Ingress Resource features of Kubernetes 1.2 - http://kubernetes.io/docs/user-guide/ingress/ Here the Ingress resource would sit in front of the load balancing service.

            If the Ingress resource living on a node dies and get’s rescheduled, will the new Ingress resource have a different external IP? How can this be made to work with AWS Route 53 DNS (e.g., for a ltd-keeper.lsst.io domain to serve the API)

            Show
            jsick Jonathan Sick added a comment - - edited Question of how to do TLS termination in a Kubernetes service load balancer. Options are: 1. Use a Compute Engine load balancer with HTTPS - https://cloud.google.com/compute/docs/load-balancing/http/ 2. Add an Nginx pod that does TLS termination - e.g. https://github.com/GoogleCloudPlatform/kube-jenkins-imager 3. Use the new Ingress Resource features of Kubernetes 1.2 - http://kubernetes.io/docs/user-guide/ingress/ Here the Ingress resource would sit in front of the load balancing service. If the Ingress resource living on a node dies and get’s rescheduled, will the new Ingress resource have a different external IP? How can this be made to work with AWS Route 53 DNS (e.g., for a ltd-keeper.lsst.io domain to serve the API)
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            The documentation is notes that the TLS ingress controller will use a "static" IP. https://github.com/kubernetes/contrib/blob/master/ingress/controllers/gce/BETA_LIMITATIONS.md#static-and-ephemeral-ips I presume that means the IP will persistence across controller restarts?

            My preference would be to go the pure kubernetes path, if feasible. On the basis that, In theory, it should be fairly painless to move between kubernetes clusters. My second choice would be to use a GCE loadbalancer, as this would likely require less fiddling than a DIY solution.

            Show
            jhoblitt Joshua Hoblitt added a comment - The documentation is notes that the TLS ingress controller will use a "static" IP. https://github.com/kubernetes/contrib/blob/master/ingress/controllers/gce/BETA_LIMITATIONS.md#static-and-ephemeral-ips I presume that means the IP will persistence across controller restarts? My preference would be to go the pure kubernetes path, if feasible. On the basis that, In theory, it should be fairly painless to move between kubernetes clusters. My second choice would be to use a GCE loadbalancer, as this would likely require less fiddling than a DIY solution.
            jsick Jonathan Sick made changes -
            Story Points 1.4 1.6
            jsick Jonathan Sick made changes -
            Story Points 1.6 2.9
            jsick Jonathan Sick made changes -
            Story Points 2.9 4.3
            jsick Jonathan Sick made changes -
            Story Points 4.3 4.6
            jsick Jonathan Sick made changes -
            Story Points 4.6 5.9
            jsick Jonathan Sick made changes -
            Story Points 5.9 6.6
            jsick Jonathan Sick made changes -
            Story Points 6.6 8.7
            jsick Jonathan Sick made changes -
            Story Points 8.7 9.4
            jsick Jonathan Sick made changes -
            Story Points 9.4 10.4
            Hide
            jsick Jonathan Sick added a comment -

            There are two branches to look at here:

            1. https://github.com/lsst-sqre/ltd-keeper/pull/5 for the main Kubernetes deployment
            2. https://github.com/lsst-sqre/nginx-python-docker/pull/1 for the uWSGI container

            We can talk about this at the Thursday meeting so I can run you through it.

            Show
            jsick Jonathan Sick added a comment - There are two branches to look at here: 1. https://github.com/lsst-sqre/ltd-keeper/pull/5 for the main Kubernetes deployment 2. https://github.com/lsst-sqre/nginx-python-docker/pull/1 for the uWSGI container We can talk about this at the Thursday meeting so I can run you through it.
            jsick Jonathan Sick made changes -
            Reviewers J Matt Peterson, Joshua Hoblitt [ jmatt, jhoblitt ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            jsick Jonathan Sick made changes -
            Story Points 10.4 11.6
            Hide
            jsick Jonathan Sick added a comment -

            This ticket was presented at the SQuaRE group meeting and signed-off. Future tickets will touch upon code improvements to the deployment practices or codebase.

            Show
            jsick Jonathan Sick added a comment - This ticket was presented at the SQuaRE group meeting and signed-off. Future tickets will touch upon code improvements to the deployment practices or codebase.
            jsick Jonathan Sick made changes -
            Resolution Done [ 10000 ]
            Status In Review [ 10004 ] Done [ 10002 ]

              People

              • Assignee:
                jsick Jonathan Sick
                Reporter:
                jsick Jonathan Sick
                Reviewers:
                J Matt Peterson [X] (Inactive), Joshua Hoblitt
                Watchers:
                J Matt Peterson [X] (Inactive), Jonathan Sick, Joshua Hoblitt
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel