Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-17507

Decide how to split physical resources between separate k8s clusters

    XMLWordPrintable

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: Science Platform
    • Labels:
    • Team:
      SQuaRE

      Description

      Fritz Mueller, Frossie Economou and Xiuqin Wu [X] we need to help inform Michelle Butler [X] on how to split up the physical resources between our two k8s clusters: production and development. The production cluster will be outside the firewall and will have NCSA exclusive kubctl. The development cluster will be inside the firewall with broader kubctl permissions. At least initially, there will be two deployments of the LSP with lsp-stable deployed on the production cluster and lsp-int deployed on the development cluster.

      Cluster resources (I will update this description with actual numbers when I have them):
      N nodes of type 1 (standard flavor)
      M nodes of type 2 (dax flavor)

      Note that the lsp-stable will need to support of order 100 users and the lsp-int will need to support of order 10 dev users.

        Attachments

          Issue Links

            Activity

            Hide
            krughoff Simon Krughoff added a comment -

            For our part, we have resource estimates listed in https://sqr-018.lsst.io/. As currently deployed, a common per user resource load is 2 cores and 6MB RAM. Some demo notebooks require 4 cores and 12MB RAM per user.

            I think that means we'd like the split to be 14 outside the firewall with two inside.

            Show
            krughoff Simon Krughoff added a comment - For our part, we have resource estimates listed in https://sqr-018.lsst.io/ . As currently deployed, a common per user resource load is 2 cores and 6MB RAM. Some demo notebooks require 4 cores and 12MB RAM per user. I think that means we'd like the split to be 14 outside the firewall with two inside.
            Hide
            xiuqin Xiuqin Wu [X] (Inactive) added a comment -

            for SUIT needs, we can do the split as following:

            lsp-int:  one node with  124 GB ram; 40 cpu each with 10 cores (from the old SUIT nodes)

            lsp-stabel: twos nodes with 124 GB ram; 40 cpu each with 10 cores (from the old SUIT nodes)

            Show
            xiuqin Xiuqin Wu [X] (Inactive) added a comment - for SUIT needs, we can do the split as following: lsp-int:  one node with  124 GB ram; 40 cpu each with 10 cores (from the old SUIT nodes) lsp-stabel: twos nodes with 124 GB ram; 40 cpu each with 10 cores (from the old SUIT nodes)
            Hide
            tjenness Tim Jenness added a comment -

            Frossie Economou is this question entirely about NCSA configuration and so can be closed?

            Show
            tjenness Tim Jenness added a comment - Frossie Economou is this question entirely about NCSA configuration and so can be closed?

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              krughoff Simon Krughoff
              Watchers:
              Christopher Clausen [X] (Inactive), Colin Slater, Jacob Rundall, Matthew Thomas Long [X] (Inactive), Simon Krughoff, Tim Jenness, Xiuqin Wu [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:

                  Jenkins

                  No builds found.