Performance modeling for the neural simulator NEST
NEST is a simulator for large networks of spiking point neurons for neuroscience research. It is best suited for simulations which focus on the dynamics, size, and structure of these systems. NEST runs well both on single-core machines as well as on supercomputers with several hundred thousands of cores.
A typical NEST simulation consists of two stages: first the network is wired up and connections (synapses) between neurons are established ("build stage"), and second the dynamics of the whole network is simulated ("simulation stage").
Our contribution
Our work aims at developing performance models for the simulation stage of NEST by a semi-empirical approach. We collect measurements of the runtime performance of NEST under varying parameter settings, and subsequently fit theoretical models to this data. These models are intended to explain the computation time tSIM required by the simulation stage of NEST. Performance modeling has two main advantages: first it allows predictions for any simulation size without the need for new profiling runs, second it helps to identify spots in the code requiring algorithmic improvements and to predict the impacts of optimizations.
Example
NEST runs in a distributed way over M MPI processes and within each process over T OpenMP threads. Each unique thread in the simulation is called a virtual process'' (VP). Neurons are distributed evenly over VPs. For what follows we assume that NEST simulates a random balanced network which contains N neurons and both static and adaptive synapses (STDP) [1]. Random external input is provided to the network by Poisson generators within each process; the simulated network quickly converges to an asynchronous irregular state with a mean per-neuron spike firing rate F.
Our performance model for this type of simulation consists of six different components which all contribute to the simulation time tSIM. These components cover the phase of VP-internal spike routing and the subsequent update of neuronal states within each communication cycle during the simulation stage. The overall simulation time is estimated as weighted sum of the six components. The weights are estimated by fitting the model to experimental data which was collected on the supercomputer JUQUEEN, varying the parameters M, T, and NM (neurons stored on each MPI process) systematically.
[1] A. Morrison, A. Aertsen, and M. Diesmann: Spike-timing-dependent plasticity in balanced random networks, Neural Computation, vol. 19, no. 6, pp. 1437-1467, 2007.
Results
Comparison of measured and estimated simulation times:
Estimated contributions to the simulation times:
Conclusions
As can be seen in the above figures, the theoretical performance model was successfully fitted to the training data. The main complexities (simulation time components) derived from the NEST source code can explain the runtime behavior for the most part. Furthermore, we got new insights into the inner workings of NEST and can derive recommendations for further NEST development from them.
Future plans
We will use the newly developed dry-run mode of NEST for dense parameter sweeps on various machines to increase the number of input parameters to the model and to cover different types of simulated networks. In this way we will be able to refine the existing performance model and to compare the behavior of NEST on different system architectures.
Related publications
Contribution to a conference proceedings/Contribution to a book
; Baumeister, P. F. (Corresponding Author) ; Böttiger, H. ; Hater, T. ; Maurer, T. ; Pleiter, D. ; Schenck, W. ; Schifano, S. F.
Performance Evaluation of Scientific Applications on POWER8
High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation / ; Cham : Springer International Publishing, 2015, Chapter 2 ; ISSN: 0302-9743=1611-3349 ; ISBN: 978-3-319-17247-7=978-3-319-17248-4 ; doi:10.1007/978-3-319-17248-4
5th International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (held as part of SC14), PMBS 2014, New Orleans, LANew Orleans, LA, USA, 16 Nov 2014 - 16 Nov 2014
Cham : Springer International Publishing, Lecture Notes in Computer Science 8966, 24 - 45 (2015) [10.1007/978-3-319-17248-4_2]
Poster (After Call)
Schenck, W. (Corresponding Author) ; Adinets, A. ; Zaytsev, Y. ; Pleiter, D. ; Morrison, A.
Performance Model for Large-Scale Neural Simulations with NEST
Supercomputing 2014, SC14, New OrleansNew Orleans, USA, 16 Nov 2014 - 21 Nov 2014
Fulltext
Simlab Contact
Project Partners
Computational and Systems Neuroscience (INM-6) - Theoretical Neuroscience (IAS-6)