Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-17972

Prepare to undo diversion of Firefly traffic to lsst-demo

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Firefly, SUIT
    • Labels:
      None
    • Story Points:
      1
    • Epic Link:
    • Sprint:
      SUIT Sprint 2019-03, SUIT Sprint 2019-04, SUIT Sprint 2019-05, SUIT Sprint 2019-06, SUIT Sprint 2019-07, SUIT Sprint 2019-08, SUIT Sprint 2019-09
    • Team:
      Science User Interface

      Description

      1. This ticket asks for the following work:
      2. Without (yet) changing the current default instance configurations of the LSP that direct Firefly traffic to the large-memory server on {{lsst-demo)} instead of the instance-local Firefly:
      3. Verify on both lsst-lsp-stable and lsst-lsp-int that the instance-local Firefly instantiations in Kubernetes (which now should be using the large-memory hardware imported from the original PDAC configuration) can handle all the usual demo notebooks with comparable performance to lsst-demo. Please document comparative timings for common operations such as load, pan, zoom, mask display, etc. Please test the detection-footprint viewing capabilities as part of this.
      4. If the current Firefly-Jupyter-extension version does not allow a user to override the default Firefly URL, please discuss with the IPAC team and file tickets as needed to ensure that this is possible. It is essential for testing and maintenance that this be possible. Ensure that firefly_client and display_firefly work correctly with this override.
      5. When this work is done and reviewed by Xiuqin Wu [X] and Gregory Dubois-Felsmann, we will file a ticket asking Adam to change the LSP instance default(s) back to the instance-local Fireflys. We will ask Adam to ensure that the capability to set the default back to lsst-demo is retained, however.

      As part of this ticket, please ensure that the versions of Firefly deployed on the LSP instances and lsst-demo are up-to-date and in sync with each other.

      I've set a deadline of March 12th to allow sufficient time to get the defaults changed by Adam before the LSP review dry runs the week of March 18th.

        Attachments

          Issue Links

          is blocked by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-18304 Adjust Firefly pod resource allocations for lsst-lsp-stable and lsst-lsp-int

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            shupe David Shupe added a comment -

            Status as of 11 March 2019:

            • Inside notebooks, the default URL for display_firefly and firefly_client can be changed by manipulating the environment variables FIREFLY_URL and fireflyURLLab. An LSP user can override the defaults in their ~/notebooks/.user_setups file.
            • For full functionality of everything in the extension outside of notebook use, which mainly means Command Palette => Open Firefly and right-click to open a FITS image in Firefly, we don't yet have a user override.
            • The lsst-lsp-stable LSP points users to the last-demo server. The lsst-lsp-int LSP points users to https:://lsst-lsp-int.ncsa.illinois.edu/firefly. These currently have the same version (2019-03-01 of lsst-dev tag).
            • Using afwDisplay, displaying the image and masks of a 4k by 4k calexp to the last-demo Firefly server takes 4 to 5 seconds. The same display to the lsst-lsp-int/firefly server takes about 25 seconds.
            • On both lsst-lsp-stable and lsst-lsp-int, overlay of LSST Source Detection Footprints is not drawing the footprints. The uploaded table does appear to be recognized and parsed as a Footprints table.
            Show
            shupe David Shupe added a comment - Status as of 11 March 2019: Inside notebooks, the default URL for display_firefly and firefly_client can be changed by manipulating the environment variables FIREFLY_URL and fireflyURLLab . An LSP user can override the defaults in their ~/notebooks/.user_setups file. For full functionality of everything in the extension outside of notebook use, which mainly means Command Palette => Open Firefly and right-click to open a FITS image in Firefly, we don't yet have a user override. The lsst-lsp-stable LSP points users to the last-demo server. The lsst-lsp-int LSP points users to https:://lsst-lsp-int.ncsa.illinois.edu/firefly . These currently have the same version (2019-03-01 of lsst-dev tag). Using afwDisplay, displaying the image and masks of a 4k by 4k calexp to the last-demo Firefly server takes 4 to 5 seconds. The same display to the lsst-lsp-int/firefly server takes about 25 seconds. On both lsst-lsp-stable and lsst-lsp-int , overlay of LSST Source Detection Footprints is not drawing the footprints. The uploaded table does appear to be recognized and parsed as a Footprints table.
            Hide
            Permalink
            gpdf Gregory Dubois-Felsmann added a comment -

            Most of this work is not worth doing until the unified "suit + firefly" application is deployed on large-memory pods in at least the lsst-lsp-int instance of the LSP.

            Show
            gpdf Gregory Dubois-Felsmann added a comment - Most of this work is not worth doing until the unified "suit + firefly" application is deployed on large-memory pods in at least the lsst-lsp-int instance of the LSP.
            Hide
            Permalink
            gpdf Gregory Dubois-Felsmann added a comment -

            Due date removed because of the dependency on Loi Ly's unification work and its deployment on at least lsst-lsp-int.

            Show
            gpdf Gregory Dubois-Felsmann added a comment - Due date removed because of the dependency on Loi Ly 's unification work and its deployment on at least lsst-lsp-int .
            Hide
            Permalink
            shupe David Shupe added a comment -

            After further testing today on the lsst-lsp-int LSP: when the user overrides the environment variables FIREFLY_URL and fireflyURLLab to point to https://lsst-demo.ncsa.illinois.edu/firefly, only some afwDisplay commands work properly, and the Firefly tab does not appear automatically. I think we can't recommend this method to users right now.

            Show
            shupe David Shupe added a comment - After further testing today on the lsst-lsp-int LSP: when the user overrides the environment variables FIREFLY_URL and fireflyURLLab to point to https://lsst-demo.ncsa.illinois.edu/firefly , only some afwDisplay commands work properly, and the Firefly tab does not appear automatically. I think we can't recommend this method to users right now.
            Hide
            Permalink
            gpdf Gregory Dubois-Felsmann added a comment -

            We are not going to actually undo the diversion to lsst-demo until there is a fix for the performance problems in the intra-Kubernetes Firefly service. However, work on this ticket can proceed so that we are ready to do that as soon as that's done.

            Show
            gpdf Gregory Dubois-Felsmann added a comment - We are not going to actually undo the diversion to lsst-demo until there is a fix for the performance problems in the intra-Kubernetes Firefly service. However, work on this ticket can proceed so that we are ready to do that as soon as that's done.
            Hide
            Permalink
            swinbank John Swinbank added a comment -

            Gregory Dubois-Felsmann — this ticket has seen no action for nicely more than a year, and the assignee is no longer on-project. Is it still relevant? Can we simply close it as “won't fix”?

            Show
            swinbank John Swinbank added a comment - Gregory Dubois-Felsmann — this ticket has seen no action for nicely more than a year, and the assignee is no longer on-project. Is it still relevant? Can we simply close it as “won't fix”?
            Hide
            Permalink
            gpdf Gregory Dubois-Felsmann added a comment - - edited

            Rendered moot long ago by shift of RSP work from NCSA to IDF.  All RSP visualization is now done with the internal Firefly and lsst-demo will disappear with the shutdown of NCSA services.

            Show
            gpdf Gregory Dubois-Felsmann added a comment - - edited Rendered moot long ago by shift of RSP work from NCSA to IDF.  All RSP visualization is now done with the internal Firefly and lsst-demo will disappear with the shutdown of NCSA services.

              People

              Assignee:
              gpdf Gregory Dubois-Felsmann
              Reporter:
              gpdf Gregory Dubois-Felsmann
              Watchers:
              David Shupe, Gregory Dubois-Felsmann, John Swinbank, Loi Ly, Trey Roby, Xiuqin Wu [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.