GPUs Basel 2018

Here we describe on how to use the GPUs provided for the Basel Workshop 2018. We go through each step by using a simple tutorial dataset/project as an example. You can use the same steps on your project of choice.

GPUs are on cluster SCICORE (link) and use queing system SLURM (a queuing system arranges the tasks from different users that want to use the same gpu)

We differ from 3 spaces: lcoal matlab, local shell, scicore shell

On your local Matlab session with dynamo loaded: 1) Create tutorial project: dtutorial pTutorial -p dTutorial -M 128 2) select gpu as computing environement (default id, will be changed later) 3) unfold 4) tar project: in dcp gui go to tools and then create tarball

On local shell: 5) download standalone dynamo 6) copy it to scicore: rsync -avuP dynamo-v-1.1.333_MCR-9.2.0_GLNXA64_withMCR.tar scaramuz@login.bc2.unibas.ch:/scicore/home/stahlberg/scaramuz/dynamo_standalone 7) copy project data (particles) to scicore: rsync -avuP pTutorial scaramuz@login.bc2.unibas.ch:/scicore/home/stahlberg/scaramuz/dynamo_scicore_testing 8) copy tar of project to scicore: rsync -avuP dTutorial.tar scaramuz@login.bc2.unibas.ch:/scicore/home/stahlberg/scaramuz/dynamo_scicore_testing 9) login to scicore: ssh -Y scaramuz@login.scicore.unibas.ch

On scicore: 10) untar stnadalone dynamo: tar -xf dynamo-v-1.1.333_MCR-9.2.0_GLNXA64_withMCR.tar 11) compile cuda executables:

    module load CUDA/7.5.18
    go to /cuda directory of matlab standalone folder
    which nvcc (should be correct version)
    source config.sh
  make clean
   make all

12) put mcr cache root on scratch??? 13) activate dynamo: source dynamo_activate_linux_shipped_MCR.sh 14) untar dynamo project: dynamo dvuntar dTutorial.tar 15) create slurm script "dTest_submit_k80_ID.sh":

1. !/bin/bash -l
3. SBATCH --job-name=dTest
4. SBATCH --qos=30min (for titanX: emgpu)
5. SBATCH --time=00:60:00 (adapt time)
6. SBATCH --mem=16G
7. SBATCH --nodes=1
8. SBATCH --ntasks-per-node=1
9. SBATCH --cpus-per-task=1
10. SBATCH --partition=k80 (for titanX: titanx)
11. SBATCH --gres=gpu:1

module load CUDA/7.5.18 source /scicore/home/stahlberg/scaramuz/dynamo_standalone/dynamo_activate_linux_shipped_MCR.sh cd /scicore/home/stahlberg/scaramuz/dynamo_scicore_testing/ echo "dvput dTutorial -gpu_identifier_set $CUDA_VISIBLE_DEVICES" > commands.sh echo "dvunfold dTutorial" >> commands.sh dynamo commands.sh chmod u=rxw ./dTutorial.m ./dTutorial.m

16) launch job on slurm with: sbatch dTest_submit_k80_ID.sh

17) check queue: squeue -u scaramuz

see all users in queue: squeue -q 30min (for titanX: squeue -q empgu)

18) cancel job: scancel ????? (job id given from squeue command)

19) check last output: ls -rtl tail -f slurm-45994509.out less slurm-45994509.out

20) check last average dynamo ddb dTutorial:a -v

21) interactive session for testing: run --nodes=1 --cpus-per-task=1 --mem=16G --gres=gpu:4 --partition=k80 --pty bash