Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10991

Optimizing workflows

    Details

      Description

      There are always overheads when scheduling jobs, commonly related to scheduling, data transfers, state notifications, and task book keeping. These overheads can be very noticeable for short jobs. The solution recommended by Pegasus team to minimize those overhead is to use task clustering. The goal is to explore this mechanism and access if it is flexible enough to address common operational requirements.

        Attachments

          Activity

          Hide
          mkowalik Mikolaj Kowalik added a comment -

          I read available documentation regarding job clustering and tried out available clustering mechanisms on simple workflows.

          Show
          mkowalik Mikolaj Kowalik added a comment - I read available documentation regarding job clustering and tried out available clustering mechanisms on simple workflows.

            People

            • Assignee:
              mkowalik Mikolaj Kowalik
              Reporter:
              mkowalik Mikolaj Kowalik
              Watchers:
              Mikolaj Kowalik
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel