# Create a simplified workflow based on the ciHsc workflow

XMLWordPrintable

## Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s: None
• Labels:
None
• Story Points:
0.5
• Team:
Data Facility

## Description

Create a simplified workflow of DM-8339 for Mikolaj Kowalik

## Attachments

1. sim.dax
13 kB
2. sim.dot
11 kB
3. simp.png
736 kB

## Activity

Hsin-Fang Chiang created issue -
Field Original Value New Value
Epic Link DM-8332 [ 27861 ]
 Assignee Hsin-Fang Chiang [ hchiang2 ] John Parejko [ parejkoj ]
 Assignee John Parejko [ parejkoj ]
Hide
Hsin-Fang Chiang added a comment -

Mikolaj Kowalik, I added a workflow in branch tickets/DM-9092. This has 4 ProcessCcdTasks + 1 MakeSkyMap + 2 MakeCoaddTempExp. The dax plot is attached. I was able to run it on lsstvc with the same steps as the ciHsc workflow, but please try it first to see if you can run it on lsstvc.

 # Set up the stack . /software/lsstsw/stack/loadLSST.bash setup -v ctrl_platform_lsstvc  setup -k -v -r /scratch/hchiang2/sw/ci_hsc/    # Allocate nodes using ctrl_execute tools allocateNodes.py -n 2 -s 2 -m 01:30:00 lsstvc   # Set up Pegasus export PEGASUS_HOME=/software/daues/pegasus/pegasus-4.7.2-1/usr export PATH=${PEGASUS_HOME}/bin:$PATH export PYTHONPATH=${PEGASUS_HOME}/lib64/python2.7/site-packages/:$PYTHONPATH   # I use this env var in pegasus_workflow setup export NODESET=${USER}_cat ~/.lsst/node-set.seq   # Make the workflow xml file using Pegasus Python API python simplified/generateDax.py -i simplified/inputData.py # Plan and submit the xml workflow using Pegasus tools ./plan_dax.sh ciHsc.dax  Show Hsin-Fang Chiang added a comment - Mikolaj Kowalik , I added a workflow in branch tickets/ DM-9092 . This has 4 ProcessCcdTasks + 1 MakeSkyMap + 2 MakeCoaddTempExp. The dax plot is attached. I was able to run it on lsstvc with the same steps as the ciHsc workflow, but please try it first to see if you can run it on lsstvc. # Set up the stack . /software/lsstsw/stack/loadLSST.bash setup -v ctrl_platform_lsstvc setup -k -v -r /scratch/hchiang2/sw/ci_hsc/ # Allocate nodes using ctrl_execute tools allocateNodes.py -n 2 -s 2 -m 01 : 30 : 00 lsstvc # Set up Pegasus export PEGASUS_HOME=/software/daues/pegasus/pegasus- 4.7 . 2 - 1 /usr export PATH=${PEGASUS_HOME}/bin:$PATH export PYTHONPATH=${PEGASUS_HOME}/lib64/python2. 7 /site-packages/:$PYTHONPATH # I use this env var in pegasus_workflow setup export NODESET=${USER}_cat ~/.lsst/node-set.seq # Make the workflow xml file using Pegasus Python API python simplified/generateDax.py -i simplified/inputData.py # Plan and submit the xml workflow using Pegasus tools ./plan_dax.sh ciHsc.dax
 Attachment simp.png [ 29116 ] Story Points 0.25 0.5 Assignee Hsin-Fang Chiang [ hchiang2 ]
 Status To Do [ 10001 ] In Progress [ 3 ]
Hide
Hsin-Fang Chiang added a comment -

Do you think this workflow is worth merging into master? I was thinking if you plan to use this exact workflow and replace jobs with Executor jobs then maybe yes? If this is only a smaller workflow for testing then maybe no because it duplicates too much with the ciHsc workflow? What do you think?

Show
Hsin-Fang Chiang added a comment - Do you think this workflow is worth merging into master? I was thinking if you plan to use this exact workflow and replace jobs with Executor jobs then maybe yes? If this is only a smaller workflow for testing then maybe no because it duplicates too much with the ciHsc workflow? What do you think?
 Reviewers Mikolaj Kowalik [ mkowalik ] Status In Progress [ 3 ] In Review [ 10004 ]
Hide
Mikolaj Kowalik added a comment -

At the moment, I don't think it's worth merging into master yet (if ever). You're correct, for now I'd just like to have a simple though not completely trivial (one task) workflow I can play with. I'll just merge your ticket branch into my user branch and I should be fine for time being. Thanks a lot for preparing it for me!

Show
Mikolaj Kowalik added a comment - At the moment, I don't think it's worth merging into master yet (if ever). You're correct, for now I'd just like to have a simple though not completely trivial (one task) workflow I can play with. I'll just merge your ticket branch into my user branch and I should be fine for time being. Thanks a lot for preparing it for me!
 Status In Review [ 10004 ] Reviewed [ 10101 ]
Hide
Hsin-Fang Chiang added a comment -

Sounds good to me

Show
Hsin-Fang Chiang added a comment - Sounds good to me
Hide
Hsin-Fang Chiang added a comment -

Also uploading the dax xml and the dot file, in case they are useful

Show
Hsin-Fang Chiang added a comment - Also uploading the dax xml and the dot file, in case they are useful
 Attachment sim.dot [ 29147 ] Attachment sim.dax [ 29148 ]
 Resolution Done [ 10000 ] Status Reviewed [ 10101 ] Done [ 10002 ]
 Team Process Middleware [ 10206 ] Data Facility [ 12219 ]

## People

• Assignee:
Hsin-Fang Chiang
Reporter:
Hsin-Fang Chiang
Reviewers:
Mikolaj Kowalik
Watchers:
Hsin-Fang Chiang, Mikolaj Kowalik