NAMD
Ways to run NAMD on Hopper
You can run different versions of NAMD on the Hopper cluster and the DGX A100 GPU server that is integrated to it.
If you want to take advantage of the GPUs, you can eiither use the Singularity containers we provide at /containers/dgx/Containers
or the native application we provision using modules.
However, you can run your simulations using the CUDA-enabled NAMD 2.14 module.
If you prefer to use CPUs only, you can load up the NAMD 2.14 module as you would on Argo.
The two approaches, namely running native NAMD and containterized NAMD, are outlined below.
Using Native Application
2.14 GPU enabled
2.14 GPU-enabled (
/opt/sw/other/apps/namd/2.14/NAMD_2.14_Linux-x86_64-verbs-smp-CUDA
)module =
namd/2.14-verbs-smp-cuda
sample run -
/groups/ORC-VAST/app-tests/NAMD/native/2.14-CUDA-enabled
sample batch submission file -
/groups/ORC-VAST/app-tests/NAMD/native/2.14-CUDA-enabled/run.slurm
2.14 CPU-only
2.14 cpu-only (
/opt/sw/other/apps/namd/2.14/NAMD_2.14_Linux-x86_64-verbs-smp
)module -
namd/2.14-verbs-smp
example directory -
/groups/ORC-VAST/app-tests/NAMD/native/2.14-CPUonly
sample batch submission file -
/groups/ORC-VAST/app-tests/NAMD/native/2.14-CPU-only/run.slurm
Using Containerized Application
We currently provide the following GPU/CUDA-enabled containers from NVIDIA
3.0-alpha GPU enabled
3.0-alpha
Container -
/containers/dgx/Containers/namd/namd_3.0-alpha3-singlenode.sif
example directory -
/groups/ORC-VAST/app-tests/NAMD/containerized/3.0
sample batch submission file -
/groups/ORC-VAST/app-tests/NAMD/containerized/3.0/run*gpu*cores.slurm
2.13 GPU enabled
2.13
Container -
/containers/dgx/Containers/namd/namd_2.13-singlenode.sif
example directory -
/groups/ORC-VAST/app-tests/NAMD/containerized/2.13
sample batch submission file -
/groups/ORC-VAST/app-tests/NAMD/containerized/2.13/run*gpu*cores.slurm
Benchmarks
Timings
Containerized GPU
Native GPU
CPU-only
Conclusions
GP-GPU runs are more than 1-2 orders of magnitude faster than CPU-only runs. However, the speedup from using multiple GPUs is small, at least for the APOA1 example.
NAMD 3.0 offloads almost all computations to the GPU while NAMD 2.1[3/4] split the computations between the CPU and GPU
NAMD 3.0 is 2-3 times faster than NAMD 2.1[3/4]
If using NAMD 3.0, there is no reason to request many CPU cores.
Caveats
the NAMD 3.0-alpha container has impressive performance, but it is unclear if it is suitable for production work (http://www.ks.uiuc.edu/Research/namd/alpha/3.0alpha/)
we have not tested multi-GPU or multi-node runs using the containers or native applications. We'll add such tests and examples later
since the DGX is the only server with GPUs on Hopper, we will need to take extra steps to make sure it is used equitably. Please refrain from reserving more GPUs than you truly need.
we realize the native (2.14) and containerized apps (2.13, 3.0) are different versions. 2.13 and 3.0 are the only ones available as containers. Please let us know if you want anything other than 2.14 as a native application.
Last updated