Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9867

Set up development environment on NCSA OpenStack

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Qserv
    • Labels:
      None

      Description

      A development playground (multi-node, consisting of czar, several workers, and some test data) is needed for prototyping data distribution and replication. It is anticipated that this will be a non-containerized deployment on NCSA OpenStack VMs in order to facilitate rapid development.

      The general plan for this effort includes the following steps:

      1. create and configure 1 master and 10 worker nodes
      2. external access to the cluster via the Floating IP of the master node
      3. user accounts and security: create user qserv, configure ssh, etc.
      4. allocate 11 volumes (one per node) for data and databases
      5. setup an NFS server on master to export some local folder to all workers for sharing LSST Stack and other relevant LSST packages
      6. setup the minimally required *RPM*s to support the software deployment and development efforts
      7. build a local copy of the LSST stack and other relevant package on the exported volume
      8. configure and start Qserv (using Docker container) on all nodes
      9. IMPORTANT: make a snapshot of the VMs of all instances
      10. import a data set (a subset of chunks of PDAC's wise or stripe82)

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  gapon Igor Gaponenko
                  Reporter:
                  fritzm Fritz Mueller
                  Watchers:
                  Fritz Mueller, Igor Gaponenko
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Summary Panel