Outline
1. Current status
Splitting computation into independent tasks:
for year in {1958..2017}; do
./calculate_for_year.sh ${year}
done
Or (naively) parallelizing over these independent tasks:
msub <<EOF
#!/bin/bash
#PBS -l feature=smp2
#PBS -l nodes=1:ppn=40
#PBS -l walltime=12:00:00
echo {1958..2017} | tr ' ' '\n' | xargs -n1 -P${PBS_NP} ./calculate_for_year.sh ${year}
EOF
2. Problems with current status
- User has to explicitly care for parallelization
- choose "axis" along which do parallelize
- set up the logics and logistics of the parallelizatoin
- does not scale well beyond limits of a CPU socket / node / data center / ...
3. Towards distributed systems
4. Demo
...