Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18304

Adjust Firefly pod resource allocations for lsst-lsp-stable and lsst-lsp-int

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Science Platform, SUIT
    • Labels:
      None
    • Story Points:
      2
    • Epic Link:
    • Sprint:
      SUIT Sprint 2019-03, SUIT Sprint 2019-04, SUIT Sprint 2019-05, SUIT Sprint 2019-06, SUIT Sprint 2019-07, SUIT Sprint 2019-08, SUIT Sprint 2019-09
    • Team:
      Science User Interface

      Description

      The task is to change the configuration of the LSP deployments lsst-lsp-stable and lsst-lsp-int to appropriately use for the Firefly/Portal pods the legacy PDAC SUIT hardware that was brought into the Kubernetes Commons.

      Per DM-17507, two of the original nodes from the set lsst-sui-tomcat01, lsst-sui-tomcat02, and lsst-sui-proxy01 are to be used for Firefly/Portal deployments on lsst-lsp-stable and the third for the same on lsst-lsp-int. I understand the initial intent to have been to have two stable pods, each using (nearly) all the memory and core resources on one of the legacy hosts, and two int pods, each using (slightly less than) half the resources on the third host.

      1. We need SUIT group input on the exact configurations to be used at pod-creation time;
      2. We need SQuaRE group assistance to apply those configurations (which might also include applying appropriate labels to control the pod affinity with the legacy hardware in Kubernetes); and
      3. We need the SUIT group to ensure that the Tomcat servers that will run in the pods are actually aware of the resources available to them (taking advantage of the improved Docker compatibility of Java 10).

      We'll also want to be sure that we retain an additional default LSP deployment configuration that is more suitable for use in environments where large-memory / many-core hosts may not be readily available, e.g., on GKE.

        Attachments

          Issue Links

            Activity

            No builds found.
            gpdf Gregory Dubois-Felsmann created issue -
            gpdf Gregory Dubois-Felsmann made changes -
            Field Original Value New Value
            Description The task is to change the configuration of the LSP deployments {{lsst-lsp-stable}} and {{lsst-lsp-int}} to appropriately use for the Firefly/Portal pods the legacy PDAC SUIT hardware that was brought into the Kubernetes Commons.

            Per DM-17507, two of the original nodes from the set {{lsst-sui-tomcat01}}, {{lsst-sui-tomcat02}}, and {{lsst-sui-proxy01}} are to be used for Firefly/Portal deployments on {{lsst-lsp-stable}} and the third for the same on {{lsst-lsp-int}}. I understand the initial intent to have been to have two {{stable}} pods, each using (nearly) all the memory and core resources on one of the legacy hosts, and two {{int}} pods, each using (slightly less than) half the resources on the third host.

            # We need SUIT group input on the exact configurations to be used at pod-creation time;
            # We need SQuaRE group assistance to apply those configurations; and
            # We need the SUIT group to ensure that the Tomcat servers that will run in the pods are actually aware of the resources available to them (taking advantage of the improved Docker compatibility of Java 10).

            We'll also want to be sure that we retain an additional default LSP deployment configuration that is more suitable for use in environments where large-memory / many-core hosts may not be readily available, e.g., on GKE.
            The task is to change the configuration of the LSP deployments {{lsst-lsp-stable}} and {{lsst-lsp-int}} to appropriately use for the Firefly/Portal pods the legacy PDAC SUIT hardware that was brought into the Kubernetes Commons.

            Per DM-17507, two of the original nodes from the set {{lsst-sui-tomcat01}}, {{lsst-sui-tomcat02}}, and {{lsst-sui-proxy01}} are to be used for Firefly/Portal deployments on {{lsst-lsp-stable}} and the third for the same on {{lsst-lsp-int}}. I understand the initial intent to have been to have two {{stable}} pods, each using (nearly) all the memory and core resources on one of the legacy hosts, and two {{int}} pods, each using (slightly less than) half the resources on the third host.

            # We need SUIT group input on the exact configurations to be used at pod-creation time;
            # We need SQuaRE group assistance to apply those configurations (which might also include applying appropriate labels to control the pod affinity with the legacy hardware in Kubernetes); and
            # We need the SUIT group to ensure that the Tomcat servers that will run in the pods are actually aware of the resources available to them (taking advantage of the improved Docker compatibility of Java 10).

            We'll also want to be sure that we retain an additional default LSP deployment configuration that is more suitable for use in environments where large-memory / many-core hosts may not be readily available, e.g., on GKE.
            tjenness Tim Jenness made changes -
            Team Science User Interface [ 10303 ]
            Hide
            gpdf Gregory Dubois-Felsmann added a comment -

            This activity should not proceed until the unified Portal web application ("suit" plus "firefly") envisioned in DM-18396 is available (which should be soon).

            Show
            gpdf Gregory Dubois-Felsmann added a comment - This activity should not proceed until the unified Portal web application ("suit" plus "firefly") envisioned in DM-18396 is available (which should be soon).
            gpdf Gregory Dubois-Felsmann made changes -
            Link This issue is blocked by DM-18396 [ DM-18396 ]
            gpdf Gregory Dubois-Felsmann made changes -
            Link This issue blocks DM-17972 [ DM-17972 ]
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Assignee Loi Ly [ loi ]
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Epic Link DM-8765 [ 28604 ]
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Sprint SUIT Sprint 2019-04 [ 861 ]
            Story Points 4
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Sprint SUIT Sprint 2019-04 [ 861 ] SUIT Sprint 2019-03 [ 860 ]
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Story Points 4 2
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Sprint SUIT Sprint 2019-03 [ 860 ] SUIT Sprint 2019-03, SUIT Sprint 2019-04 [ 860, 861 ]
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Sprint SUIT Sprint 2019-03, SUIT Sprint 2019-04 [ 860, 861 ] SUIT Sprint 2019-03, SUIT Sprint 2019-04, SUIT Sprint 2019-05 [ 860, 861, 874 ]
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Sprint SUIT Sprint 2019-03, SUIT Sprint 2019-04, SUIT Sprint 2019-05 [ 860, 861, 874 ] SUIT Sprint 2019-03, SUIT Sprint 2019-04, SUIT Sprint 2019-05, SUIT Sprint 2019-06 [ 860, 861, 874, 875 ]
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Epic Link DM-8765 [ 28604 ] DM-17262 [ 239050 ]
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Sprint SUIT Sprint 2019-03, SUIT Sprint 2019-04, SUIT Sprint 2019-05, SUIT Sprint 2019-06 [ 860, 861, 874, 875 ] SUIT Sprint 2019-03, SUIT Sprint 2019-04, SUIT Sprint 2019-05, SUIT Sprint 2019-06, SUIT Sprint 2019-07 [ 860, 861, 874, 875, 876 ]
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Sprint SUIT Sprint 2019-03, SUIT Sprint 2019-04, SUIT Sprint 2019-05, SUIT Sprint 2019-06, SUIT Sprint 2019-07 [ 860, 861, 874, 875, 876 ] SUIT Sprint 2019-03, SUIT Sprint 2019-04, SUIT Sprint 2019-05, SUIT Sprint 2019-06, SUIT Sprint 2019-07, SUIT Sprint 2019-08 [ 860, 861, 874, 875, 876, 877 ]
            xiuqin Xiuqin Wu [X] (Inactive) made changes -
            Sprint SUIT Sprint 2019-03, SUIT Sprint 2019-04, SUIT Sprint 2019-05, SUIT Sprint 2019-06, SUIT Sprint 2019-07, SUIT Sprint 2019-08 [ 860, 861, 874, 875, 876, 877 ] SUIT Sprint 2019-03, SUIT Sprint 2019-04, SUIT Sprint 2019-05, SUIT Sprint 2019-06, SUIT Sprint 2019-07, SUIT Sprint 2019-08, SUIT Sprint 2019-09 [ 860, 861, 874, 875, 876, 877, 878 ]
            Hide
            gpdf Gregory Dubois-Felsmann added a comment -

            Some version of this work was completed both for the NCSA instance(s) and the IDF instance(s). Closing this ticket.

            The same work may be needed at the USDF but will be ticketed separately.

            Show
            gpdf Gregory Dubois-Felsmann added a comment - Some version of this work was completed both for the NCSA instance(s) and the IDF instance(s). Closing this ticket. The same work may be needed at the USDF but will be ticketed separately.
            gpdf Gregory Dubois-Felsmann made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            gpdf Gregory Dubois-Felsmann made changes -
            Resolution Done [ 10000 ]
            Status In Progress [ 3 ] Done [ 10002 ]

              People

              Assignee:
              loi Loi Ly
              Reporter:
              gpdf Gregory Dubois-Felsmann
              Watchers:
              Adam Thornton, Gregory Dubois-Felsmann, Loi Ly, Trey Roby, Xiuqin Wu [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.