Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8392

Run prototype executor-less ci_hsc Pegasus workflow on Verification Cluster

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Learn to run the prototype Pegasus workflow developed in DM-8339 on Verification Cluster.

        Attachments

          Issue Links

            Activity

            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            For running this workflow, a stack with ci_hsc needs to be set up. To allocate node time on the Verification Cluster, ctrl_orca, ctrl_exectue, and ctrl_platform_lsstvc also need to be set up (they are all included in lsst_distrib already). All software should be on GPFS as the worker nodes do not see NFS.

            Running allocateNodes.py will give a node set name, which is the user name plus a number. Set the environment variable NODESET to this name, because currently my Pegasus setup is to read that name from env. Then plan and submit the pegasus workflow as in readme. I set the data configuration to sharedfs.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - For running this workflow, a stack with ci_hsc needs to be set up. To allocate node time on the Verification Cluster, ctrl_orca , ctrl_exectue , and ctrl_platform_lsstvc also need to be set up (they are all included in lsst_distrib already). All software should be on GPFS as the worker nodes do not see NFS. Running allocateNodes.py will give a node set name, which is the user name plus a number. Set the environment variable NODESET to this name, because currently my Pegasus setup is to read that name from env. Then plan and submit the pegasus workflow as in readme. I set the data configuration to sharedfs .
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Added some configuration setups for lsstvc and merged into https://github.com/lsst-dm/pegasus_workflow

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Added some configuration setups for lsstvc and merged into https://github.com/lsst-dm/pegasus_workflow
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            A current step-by-step example is added here

            Show
            hchiang2 Hsin-Fang Chiang added a comment - A current step-by-step example is added here

              People

              • Assignee:
                hchiang2 Hsin-Fang Chiang
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Watchers:
                Hsin-Fang Chiang
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel