Validation methodology for network simulations
Summary
In this project, we are investigating methods for quantitative and qualitative validation of neuron models and neural network simulations.
Background and motivation
The reproduction and replication of scientific results is an indispensable aspect of good scientific practice, enabling previous studies to be built upon and increasing our level of confidence in them. However, reproducibility and replicability are not sufficient: an incorrect result will be accurately reproduced if the same incorrect methods are used. For the field of simulations of complex neural networks, the causes of incorrect results vary from insufficient model implementations and data analysis methods, deficiencies in workmanship (e.g., simulation planning, setup, and execution) to errors induced by hardware constraints (e.g., limitations in numerical precision). In order to build credibility, methods such as verification and validation have been developed. However, they are not yet well-established in the field of neural network modeling and simulation, partly due to ambiguity concerning the terminology, but also due to difficulties in their applicability. In addition to a common definition of the applied terminology, the methodology requires the definition of formalized workflows and standardized test cases, e.g., statistical test metrics that enable the quantitative validation of network models on the level of the population dynamics. In this project we also consider novel neuromorphic computing architectures and hardware accelerators as validation targets, which are being developed as part of the Jülich Advanced Computing Architectures project. This can help to build confidence in such systems and uncover shortcomings.
Our approach
In this research project, we are developing methods, workflows and tools for validating complex neural network simulations. We analyze, adopt and apply existing methodologies from engineering and model validation to the field of neural network modeling and simulation.
Current Work
We proposed a reasonable adaptation of the existing terminology for model verification and validation and applied it to the field of neural network modeling and simulation. We introduced the concept of model verification and substantiation for increasing the correctness of simulation results in the absence of experimental validation data and applied it to the issue of reproducibility on a worked example.
Publications
Our collaboration partners
This project is being conducted in collaboration with the following research groups from the INM-6.
