Install prerequisites (You may contact the administrator to help you with this part)
# already satisfied
CMake >= 3.10.2
MPI >= 3.0
Python >= 3.6
automake >=1.15
# install requirements using root
sudo apt-get install openmpi-bin libopenmpi-dev libncurses5-dev libncursesw5-dev automake flex bison libncurses5-dev=5.0 libncursesw5-dev=5.0 cmake
Install Nvidia-driver, CUDA and HPC-SDK
<aside> 💬 Note: your HPC-SDK version should compatible with your CUDA version.
</aside>
Clone DeepDendrite repositoy
git clone <https://github.com/pkuzyc/DeepDendrite.git>
Compile NEURON simulator
DeepDendrite provide a modified NEURON software, which you need to compile on your own environment.
<aside>
💬 Note: Replace am__api_version='1.15'
with am__api_version='1.16'
in src/nrn_modified/configure
for machines with latest version of Linux systems.
</aside>
cd src/nrn_modify
chmod 777 configure # make it executable
./configure --prefix <abs_path/to/install/nrn_modified> --without-iv --with-paranrn --with-nrnpython=`which python` --without-memacs
make -j
make install
Compile DeepDendrite:
cd ../DeepDendrite
Modify the install.sh file to load correct modulefile of PGI or NVHPC SDK, and set the install path and mod file path (all starts with "/path/to/your") in cmake command.
<aside>
💬 On our A100 machine, you may replace install.sh
with the followings:
</aside>
# install.sh file content
rm -rf build
mkdir build
cd build
export PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/23.7/comm_libs/11.8/openmpi4/openmpi-4.1.5/bin:$PATH
export MANPATH=$MANPATH:/opt/nvidia/hpc_sdk/Linux_x86_64/23.7/comm_libs/mpi/man
export MANPATH=$MANPATH:/opt/nvidia/hpc_sdk/Linux_x86_64/23.7/compilers/man
export PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/23.7/compilers/bin:$PATH
export PATH=/usr/local/cuda-11.8/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-11.8/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
# note: you'd better specify correct cuda version, "/usr/local/cuda/bin" might lead to unexpected error if there are multiple cuda toolkit with different versions
echo $(which mpicc)
echo $(which mpicxx)
export CC=mpicc
export CXX=mpicxx
cmake .. -DCMAKE_C_FLAGS:STRING="-lrt -g -O0 -mp -mno-abm" \\
-DCMAKE_CXX_FLAGS:STRING="-lrt -std=c++11 -g -O0 -mp -mno-abm" \\
-DCOMPILE_LIBRARY_TYPE=STATIC \\
-DCMAKE_INSTALL_PREFIX=</path/to/install/DeepDendrite> \\ # example: /home/kchen/DeepDendrite/install/deepdendrite
-DADDITIONAL_MECHPATH=</path/of/folder/for/your/nrn_mod_files> \\ # example: /home/kchen/DeepDendrite/src/all_mechanisms
-DCUDA_HOST_COMPILER=`which gcc` \\
-DCUDA_PROPAGATE_HOST_FLAGS=OFF \\
-DENABLE_SELECTIVE_GPU_PROFILING=ON \\
-DENABLE_OPENACC=ON \\
-DAUTO_TEST_WITH_SLURM=OFF \\
-DAUTO_TEST_WITH_MPIEXEC=OFF \\
-DFUNCTIONAL_TESTS=OFF \\
-DUNIT_TESTS=OFF
make -j
make install
After finishing modifying install.sh, run it to complete installation.
bash install.sh
# Note that everytime incoorperating new NEURON mod files, you need to reinstall DeepDendrite.
Running neuron simulation on DeepDendrite takes three steps.
Step 1: compile neuron mod files using nrnivmodl
in modified NEURON simulator.
<path-to-install-nrn_modified>/x86_64/bin/nrnivmodl ./folder-path-of-mod-files
Step 2: generate model model using DeepDendrite
Define models in Python scripts gen_model.py
, and generate model using DeepDendrite
./x86_64/special gen_model.py
Step 3: run generated model with DeepDendrite
<path-to-install-DeepDendrite>/bin/deepdendrite -d <path/for/generated/models> -e 1000 -v -86 --cell-permute 3 --cell-nthread 32 --gpu
Modify step 2 and 3. Here, we use 4 GPU devices as an example.
Step 2:
/opt/nvidia/hpc_sdk/Linux_x86_64/23.7/comm_libs/11.8/openmpi4/openmpi-4.1.5/bin/mpicc -n4 ./x86_64/special gen_model.py
Step 3:
/opt/nvidia/hpc_sdk/Linux_x86_64/23.7/comm_libs/11.8/openmpi4/openmpi-4.1.5/bin/mpiexec -n4 <path-to-install-DeepDendrite>/bin/deepdendrite -d <path/for/generated/models> -e 1000 -v -86 --cell-permute 3 --cell-nthread 32 --gpu