Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-414

Configure slurm to accept jobs to use only partial nodes on lsst-dev verification cluster

    Details

    • Type: RFC
    • Status: Adopted
    • Resolution: Unresolved
    • Component/s: DM
    • Labels:
      None

      Description

      As described in IHS-576, it would be useful for the productivity of the development team for the verification cluster at NCSA to accept jobs that use partial nodes. As it stands now, if I launch a job via slurm that requests only a single core, it will take up a whole node (48 cores). This is a very inefficient way to use resources when jobs are not actually using all those cores, and it often leads to an unnecessary backlog in the queue while many cores sit idle on fully allocated nodes. I have experienced this, as has Lauren MacArthur, Nate Lust, and probably others.

      I am proposing (as requested in IHS-576) that the VC be configured to allow partial nodes to be allocated. The Princeton Slurm clusters (e.g., tiger, perseus, della, etc.) are all configured such that partial nodes can be used, and this can be very useful, e.g. for launching job arrays of many serial jobs. It will make the VC much more useful to the development team.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                pdomagala Paul Domagala [X] (Inactive)
                Reporter:
                tmorton Tim Morton
                Watchers:
                Andrew Loftus, Bob Armstrong, Brian Van Klaveren, Hsin-Fang Chiang, Jim Bosch, John Parejko, John Swinbank, Kian-Tat Lim, Paul Domagala [X] (Inactive), Paul Price, Pim Schellart [X] (Inactive), Tim Jenness, Tim Morton
              • Votes:
                1 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Planned End:

                  Summary Panel