Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14873

Clarify what the batch and other parallel processing capabilities exposed to users in the Science Platform will be

    Details

    • Templates:
    • Epic Name:
      sst-User-Batch-Parallel-LSP
    • Team:
      DM Science

      Description

      We need to decide on (and add to the Science Platform Design document, LDM-542) a description of what the batch processing and other parallel computing capabilities available to science users in the Science Platform will be, and how they will be accessed. Presumably that access would be primarily from the Notebook Aspect, but this may also be related to the final form that emerges for the "next-to-database processing" system provided by the API Aspect.

      Under "other parallel computing" we should be including capabilities like providing Dask, or something comparable, to users.

      It is important to ensure that a non-staff perspective is maintained in the design, even though for the next few years the use of these capabilities will be dominated by staff.

      I've written this as an Epic because I can imagine a series of stories emerging as this is fleshed out: perhaps initially writing a set of use cases, then a tech note with a possible design, then a formal addition to LDM-542.

      This work needs to be completed in time for the external review of the Science Platform design.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gpdf Gregory Dubois-Felsmann
                Reporter:
                gpdf Gregory Dubois-Felsmann
                Watchers:
                Colin Slater, Frossie Economou, Gregory Dubois-Felsmann, Leanne Guy, Michael Wood-Vasey, Robert Gruendl, Simon Krughoff
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated:

                  Summary Panel