Example  job-script

The job script includes the options (qsub option) to be passed to the job scheduler and the execution command.

qsub Option: Start with #PBS at the beginning of the line then the arguments to be passed to the executed command.Other parts: treated as a command to be executed by the shell.Job script example:Execute MPI program (compiled)

#!/bin/csh　　　　　　　　　　　　　　　　　　← Specify the shell to use for execution
#PBS -q SINGLE　　　　　　　　　　　　　　← Specify the destination queue
#PBS -oe 　　　　　　　　　　　　　　　　　　← Direct standard output and error to the same file
#PBS -l select=1:mpiprocs=8  　← Specify the required resources（in this example 8 parallel MPI processors） 
#PBS -N my-job　　　　　　　　　　　　　　← Specify a job name

cd ${PBS_O_WORKDIR} 　　　　　　　← Change the working directory

mpirun -r ssh  -machinefile ${PBS_NODEFILE} -np 8 ./hello_mpi　　　           　← Run the complied executable

Job-script sample

Other samples of various job-scripts are shown here．

• MPI
• OpenMP
• Hybrid(MPI+OpenMP)
• MaterialsStudio

Please have copy the sample PBS script below and use it.

kagayaki:/Samples

How to calculate the resources needed in chunks

In most queue settings, Default chunk size of this system is:

1 Chunk:
0.25 CPU (16 CPU Core/ncpus= 16 )

If PBS is set as below,  "2Chunks, x 16 CPU Core = 32CPU Core" is provided.

#PBS -l select=2:ncpus=16

If the resources are specified as below, the chunk size will change．In this example, 2 chunks, 64 CPU Core is set as one chunk, 32 CPU Core.

#PBS -l select=2:ncpus=32

1CPU (32CPU Core, 128GB Mem), 32 MPI parallelized job with 1/4 node.

#!/bin/csh
#PBS -l select=2:npus=16:mpiprocs=16
#PBS -j oe
#PBS -N MPI-job

cd $PBS_O_WORKDIR

mpirun -machinefile ${PBS_NODEFILE} -np 32 ./hello_mpi

2CPU (128CPU Core, 64GB Mem), 64 MPI parallelized proccesses with 1 node. (64 CPU will be free)

#!/bin/csh
#PBS -l select=8:ncpus=16:mpiprocs=8
#PBS -j oe
#PBS -N MPI-job

cd $PBS_O_WORKDIR

mpirun -machinefile ${PBS_NODEFILE} -np 64 ./hello_mpi

0.5CPU (32CPU Core) 32 Threads parallelized job

#!/bin/csh
#PBS -l select=2:ncpus=16
#PBS -j oe
#PBS -N OpenMP-job

cd $PBS_O_WORKDIR
setenv OMP_NUM_THREADS 32

./a.out

Using 1CPU (64CPU Core) , 8(MPI) process x 2 (OpenMP) threads per 16 Core

#!/bin/bash
#PBS -l select=4:ncpus=16:mpiprocs=8　　　<-- 8 MPI processes per 16 CPU Core
#PBS -j oe
#PBS -N hybrid-job

cd $PBS_O_WORKDIR

export OMP_NUM_THREADS=2    <-- 2 threads per process

mpirun -machinefile ${PBS_NODEFILE} -np 32 ./hello_hyb　<-- 32 Process generation

Using 2CPU (128CPU Core), 4 MPI process x 4 OpenMP threads per 16 Cores

#!/bin/bash
#PBS -l select=8:ncpus=16:mpiprocs=4　　　<-- 4 MPI processes per 16 CPU Cores ( = in total 128 CPU Cores, 16 MPI processes )
#PBS -j oe
#PBS -N hybrid-job

cd $PBS_O_WORKDIR

export OMP_NUM_THREADS=4    <--  4threads per process

mpirun -machinefile ${PBS_NODEFILE} -np 32 ./hello_hyb　<-- 32 MPI process generation