Types of Parallel Computing Jobs

Applies To: Windows Compute Cluster Server 2003

In general, there are three types of parallel computing jobs: parallel task, parametric sweep, and task flow. The three types are not mutually exclusive.

• Parallel task

• Parametric sweep

• Task flow

Parallel task

A parallel task can take a number of forms, depending on the application and the software that supports it. For a Message Passing Interface (MPI) application, a parallel task usually consists of a single executable running concurrently on multiple processors, with communication between the processes. This is shown in the following diagram:

The process numbers 0, 1 and 2represent the process rank and have greater or less significance depending on the processing paradigm. At the minimum, Process 0 handles file input/output and determines what other processes are running. In one paradigm, Process 0 also acts as a master process; in this case, the others are worker processes that communicate with the master process but not with each other. In another paradigm, the relationship among the processes is symmetrical, with all processes equal and each communicating with every other process.

From the cluster perspective, what most distinguishes a parallel task from a serial task is that the user application, and therefore the task itself, is intrinsically parallel, and parallel processing takes place independently of whether the task is running concurrently with other tasks. For parallel tasks, Windows Compute Cluster Server 2003 includes an MPI package based on Argonne National Laboratory's MPICH2 standard. The Microsoft implementation of MPI, called MS MPI, includes the launcher mpiexec, an MPI service for each node, and an SDK for user application development. Windows Compute Cluster Server 2003 also supports applications that provide their own parallel processing mechanisms. Documentation for the SDK can be found online at the MSDN Web site (https://go.microsoft.com/fwlink/?linkid=55873.)

Parametric sweep

A parametric sweep job is illustrated in the following figure:

A parametric sweep consists of multiple instances of the same program, usually serial, running concurrently, with input supplied by an input file and output directed to an output file. There is no communication or interdependency among the tasks. Typically, the parallelization is performed exclusively (or almost exclusively) by the scheduler, based on the fact that all the tasks are in the same job.

Task flow

A task flow job is one in which a set of unlike tasks are executed in a prescribed order, usually because one task depends on the result of another task.

A task flow job is illustrated in the following figure. Note that only Tasks 2 and 3 are performed in parallel, because neither is dependent on the other.