Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9867

Set up development environment on NCSA OpenStack


    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Qserv
    • Labels:


      A development playground (multi-node, consisting of czar, several workers, and some test data) is needed for prototyping data distribution and replication. It is anticipated that this will be a non-containerized deployment on NCSA OpenStack VMs in order to facilitate rapid development.

      The general plan for this effort includes the following steps:

      1. create and configure 1 master and 10 worker nodes
      2. external access to the cluster via the Floating IP of the master node
      3. user accounts and security: create user qserv, configure ssh, etc.
      4. allocate 11 volumes (one per node) for data and databases
      5. setup an NFS server on master to export some local folder to all workers for sharing LSST Stack and other relevant LSST packages
      6. setup the minimally required *RPM*s to support the software deployment and development efforts
      7. build a local copy of the LSST stack and other relevant package on the exported volume
      8. configure and start Qserv (using Docker container) on all nodes
      9. IMPORTANT: make a snapshot of the VMs of all instances
      10. import a data set (a subset of chunks of PDAC's wise or stripe82)


          Issue Links



              • Assignee:
                gapon Igor Gaponenko
                fritzm Fritz Mueller
                Fritz Mueller, Igor Gaponenko
              • Votes:
                0 Vote for this issue
                2 Start watching this issue


                • Created:

                  Summary Panel