Data Distributions for Task-Parallel Programs

Thomas Rauber, Gudula Ruenger

We consider parallel programs consisting of data-parallel modules that are combined according to a given coordination specification. For an execution on a distributed memory machine, each data-parallel module has to use a data distribution for its variables. If cooperating modules are based on different data distributions for the same variable, data redistributions have to be performed between the activations of the modules. Thus, considering the data-parallel modules in the context of a coordination program may require to use sub-optimal local data distributions to obtain a global optimal solution. We describe an algorithm to determine data distributions for the different modules such that redistributions are taken into consideration. The algorithm is based on dynamic programming and uses runtime functions to describe the computation and communication behavior of the modules.