Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9235

Use Pegasus to do ProcessCcd with a subset of HSC COSMOS data on /datasets/hsc

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      On the Verification Cluster, run ProcessCcd with a small set of real HSC data using Pegasus.

        Attachments

          Issue Links

            Activity

            Hide
            hchiang2 Hsin-Fang Chiang added a comment - - edited

            Added a quick dax to do the equivalent of DM-9053 for experiment. The script is at branch tickets/DM-9235 of pegasus_workflow.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - - edited Added a quick dax to do the equivalent of DM-9053 for experiment. The script is at branch tickets/ DM-9235 of pegasus_workflow .
            Hide
            hchiang2 Hsin-Fang Chiang added a comment - - edited

            Made a wide dax and ran ProcessCcd with some HSC COSMOS data on /datasets/hsc. I don't intend to merge this dax, but it can help future exercises.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - - edited Made a wide dax and ran ProcessCcd with some HSC COSMOS data on /datasets/hsc. I don't intend to merge this dax, but it can help future exercises.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            As the script cannot handle conditional inputs yet, fringe correction was turned off for this exercise.
            That dax, consisting 167 visits * 103 ccd = 17201 independent jobs, ran on lsstvc. All succeeded except one (visit=19212 ccd=101).

            Show
            hchiang2 Hsin-Fang Chiang added a comment - As the script cannot handle conditional inputs yet, fringe correction was turned off for this exercise. That dax, consisting 167 visits * 103 ccd = 17201 independent jobs, ran on lsstvc. All succeeded except one (visit=19212 ccd=101).
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            This had only 103 CCDs because ccd=9 was excluded (known from DM-9053)

            Show
            hchiang2 Hsin-Fang Chiang added a comment - This had only 103 CCDs because ccd=9 was excluded (known from DM-9053 )

              People

              • Assignee:
                hchiang2 Hsin-Fang Chiang
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Watchers:
                Hsin-Fang Chiang
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel