PLoS Comput BiolplosploscompPLOS Computational Biology1553-734X1553-7358Public Library of ScienceSan Francisco, CA USAPCOMPBIOL-D-22-0135910.1371/journal.pcbi.1011625Research ArticleBiology and life sciencesCell biologyCellular typesAnimal cellsNeuronsBiology and life sciencesNeuroscienceCellular neuroscienceNeuronsBiology and life sciencesPhysiologyElectrophysiologyMembrane potentialAction potentialsBiology and life sciencesPhysiologyElectrophysiologyNeurophysiologyAction potentialsBiology and life sciencesNeuroscienceNeurophysiologyAction potentialsBiology and life sciencesAnatomyNervous systemSynapsesMedicine and health sciencesAnatomyNervous systemSynapsesBiology and life sciencesPhysiologyElectrophysiologyNeurophysiologySynapsesBiology and life sciencesNeuroscienceNeurophysiologySynapsesComputer and information sciencesNeural networksBiology and life sciencesNeuroscienceNeural networksBiology and life sciencesPhysiologyElectrophysiologyMembrane potentialResearch and analysis methodsSimulation and modelingBiology and life sciencesNeuroscienceCellular neuroscienceNeuronal morphologyBiology and life sciencesPsychologyBehaviorSocial sciencesPsychologyBehaviorMetamodelling of a two-population spiking neural networkMetamodelling of a two-population spiking neural networkhttps://orcid.org/0000-0002-3610-5926SkaarJan-Eirik W.ConceptualizationFormal analysisInvestigationMethodologyValidationVisualizationWriting – original draftWriting – review & editing1*https://orcid.org/0000-0002-9945-7208HaugNicolaiFormal analysisWriting – review & editing1https://orcid.org/0000-0003-1646-2472StasikAlexander J.Conceptualization12EinevollGaute T.ConceptualizationWriting – review & editing12TøndelKristinConceptualizationWriting – review & editing1Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, NorwayDepartment of Physics, University of Oslo, Oslo, NorwayEngelTatianaEditorPrinceton University, UNITED STATES
The authors have declared that no competing interests exist.
* E-mail: jan-eirik.welle.skaar@nmbu.no112023301120231911e10116251392022231020232023Skaar et alThis is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
In computational neuroscience, hypotheses are often formulated as bottom-up mechanistic models of the systems in question, consisting of differential equations that can be numerically integrated forward in time. Candidate models can then be validated by comparison against experimental data. The model outputs of neural network models depend on both neuron parameters, connectivity parameters and other model inputs. Successful model fitting requires sufficient exploration of the model parameter space, which can be computationally demanding. Additionally, identifying degeneracy in the parameters, i.e. different combinations of parameter values that produce similar outputs, is of interest, as they define the subset of parameter values consistent with the data. In this computational study, we apply metamodels to a two-population recurrent spiking network of point-neurons, the so-called Brunel network. Metamodels are data-driven approximations to more complex models with more desirable computational properties, which can be run considerably faster than the original model. Specifically, we apply and compare two different metamodelling techniques, masked autoregressive flows (MAF) and deep Gaussian process regression (DGPR), to estimate the power spectra of two different signals; the population spiking activities and the local field potential. We find that the metamodels are able to accurately model the power spectra in the asynchronous irregular regime, and that the DGPR metamodel provides a more accurate representation of the simulator compared to the MAF metamodel. Using the metamodels, we estimate the posterior probability distributions over parameters given observed simulator outputs separately for both LFP and population spiking activities. We find that these distributions correctly identify parameter combinations that give similar model outputs, and that some parameters are significantly more constrained by observing the LFP than by observing the population spiking activities.
Author summary
In computational neuroscience, mechanistic models are used to simulate networks of neurons. These models exhibit complex dynamics, and the parameters of the neurons and connections between neurons shape the model’s behaviour. Due to the model complexity, running the simulations and fitting the model to experimental data can be computationally demanding. In this study, we train and compare different metamodelling techniques, data-driven approximations that are much faster to run, to two different signals generated by a two-population recurrent network model, the population spiking activities and the local field potential (LFP). Further, we invert the metamodels, and demonstrate that it can reliably find the different combinations of parameters that can give rise to an observed simulation output. We compare the accuracy of the metamodels on both the forward and inverse problem, and investigate to what degree the parameters are constrained by observing the two different signals.
http://dx.doi.org/10.13039/501100005416Norges ForskningsrådDigiBrain 248828EinevollGaute T.http://dx.doi.org/10.13039/501100005416Norges ForskningsrådCoBra 250128https://orcid.org/0000-0003-1646-2472StasikAlexander J.http://dx.doi.org/10.13039/100010661Horizon 2020 Framework Programme945539http://dx.doi.org/10.13039/100010661Horizon 2020 Framework Programme945539EinevollGaute T.http://dx.doi.org/10.13039/100010661Horizon 2020 Framework Programme754304GTE and AJS received funding from the Research Council of Norway (DigiBrain 248828, CoBra 250128), https://www.forskningsradet.no/. GTE and HEP received funding from European Union’s Horizon 2020 Framework Programme for Research and Innovation under Grant Agreements No. 945539 (Human Brain Project SGA3). Simulations and analysis were performed on the DEEP system at Jülich Supercomputing Centre built with funding from the European Union’s Horizon 2020 Programme under Grant Agreement 754304 (DEEP-EST). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.Data AvailabilityAll relevant data are within the manuscript.1 Introduction
Mechanistic modelling of neurons and networks of neurons based on the underlying biophysical principles is a well-established field and is an important part in bridging the scales between individual neurons and higher-level brain function [1–4]. Neuron models range in complexity from rate-based models, where spikes are not explicitly modelled, to point-neurons, where the detailed morphology of the individual neurons are collapsed into a single point, and further to multi-compartment models where the full morphology is accounted for. At the network level, even simple models exhibit complex dynamic behaviours depending on both neuron dynamics and network connectivity [5]. A question modellers face is how the network model can be parameterized in order to produce a specific behaviour, and how sensitive the behaviour is to changes in model parameter values. As mechanistic models can be expensive to run and typically have many parameters, that is, high-dimensional parameter spaces, accurately sampling the space of model behaviours is a nontrivial problem.
Mechanistic models of dynamical systems typically consist of a set of differential equations that can be integrated forward in time. The modeller may be interested in a subset of the model’s behaviours, e.g. the state variables of the model, such as membrane potentials, synaptic conductances or currents, individual neuron behaviours such as spike times, or aggregated statistics such as population firing rates, local field potentials [6], or current source densities. The user specifies a set of parameters, which will determine the model’s behaviour. If the model is deterministic (no random variables are involved), the output will be a unique function of its parameters and initial conditions. If the model is probabilistic (one or more random variables affect the model), the output will no longer be a unique function of the parameters and initial conditions, but will instead follow a probability distribution over possible outputs.
Conceptually, a mechanistic model can be thought of as a function M that takes input values and returns output values,
output=M(input).
If the model is stochastic, there is a distribution over possible outputs, which typically does not have a closed-form expression, since explicitly evaluating it would require integrating over all possible internal model states. Metamodels are data-driven statistical models that take a set of inputs and outputs generated by a mechanistic model, and learn an approximation of M over a restricted part of its domain [7]. The metamodel can provide either a point-estimate of the model output, or a distribution over possible outputs. In the case of stochastic models, the latter case has the advantage of providing an approximation of the intractable probability distribution over simulator outputs. The number of samples needed to achieve sufficient accuracy over the domain of interest will depend on the dimensionality of the inputs, on how the function M changes with respect to the input dimensions, and on whether it has a structure that the metamodel is able to capture.
The inputs can in principle be anything that changes the model behaviour, e.g. model parameters or some other specific model input. As the volume of the parameter space increases exponentially with the number of parameters, densely sampling high-dimensional parameter spaces is very costly. Metamodels can provide a compact representation of the model over the domain, where the density of sampling required can be determined by evaluating the model on a test data set. Metamodels can also provide simpler interpretations of the model, identify the importance of individual input dimensions, and give an overview of the original model’s behavioral repertoire, i.e. a statistical depiction of the possible outputs the model can produce under a range of input conditions [8]. Metamodels can be applied in either direction, from model input to model output, or vice versa, directly approximating the inverse model.
For stochastic models, the distribution over model outputs will depend on the parameter values and possibly other model inputs, giving a conditional probability p(x|θ), often called the model likelihood, where x is the model output and θ represents model parameters and other types of inputs. Since different model inputs possibly can give rise to the same model outputs, there is also a distribution over model inputs that can give rise to the same model outputs, p(θ|x), often called the posterior distribution.
Estimating this posterior distribution by running simulations, without direct access to the model likelihood, has been the focus of work such as Approximate Bayesian computation (ABC) [9, 10], Sequential Neural Posterior Estimation (SNPE) [11, 12] and Sequential Neural Likelihood Estimation (SNLE) [13]. SNPE directly models the posterior, typically as a normalizing flow parameterized by a neural network, while SNLE models the likelihood, and infers the posterior in a separate step. The present work focuses on creating an approximation to the likelihood, as in the first step in SNLE, and can thus also be used to infer posterior distributions in the same manner. These methods can also be focused on estimating the posterior probability p(θ|x0) for a specific observed output x0, and generating a local model of the posterior distribution around that point by iteratively running simulations where the posterior density is believed to be high. This approach can be viewed as a metamodel that is trained locally around a specific point of interest in the parameter space. In this case, the metamodel will not have to be as flexible as in the case where one is interested in a much larger volume of the parameter space.
Neuronal networks typically exhibit stochastic dynamics, where input currents and firing may be stochastic, and the network connectivity and initial conditions can be drawn randomly. The network activity is also nonlinearly dependent on the precise history of the activity [14]. This makes direct metamodelling of time series data challenging, as different instantiations of the same model will produce different results, which the metamodel will have to account for. Simplified mechanistic models of population rates have been developed, ranging from models based on mean-field theory [5, 14] where the number of neurons is assumed to be infinite and finite-size effects are disregarded, to ones where finite-size effects are fully accounted for [15], but where stochastic firing of neurons and full connectivity between all neurons are assumed.
In the present work, we train metamodels to approximate the distribution over the power spectra of the population spiking activities and local field potential (LFP), based on the model parameters of the Brunel network [5], comprising an excitatory and an inhibitory population of recurrently connected point-neurons. The metamodels approximate the conditional probability distribution p(x|θ), where x is the power spectrum of either the population spiking activity or the LFP, and θ is the biological parameters of the spiking network. The LFP, the low-frequency part of extracellular potentials, largely represents how synaptic inputs are processed in the dendrites [6, 16, 17], so we chose to include both signals as they in a sense represent the incoming and outgoing signals for a population, and it offers the opportunity to compare the information carried by the two different signals. Fig 1 shows an overview of the modelling scheme we employ.
10.1371/journal.pcbi.1011625.g001
Model overview: Given a set of parameters θ, a network of point-neurons are simulated, from which the population spiking activities ν(t) are computed (top). With morphological neurons spatially extended in a column, the synaptic spiking activities from the point-neuron network can be ‘replayed’ on the morphological neurons, and the resulting local field potential ϕ(r, t) at 6 locations is computed (bottom). Note that only one excitatory and one inhibitory neuron is shown, in simulation there is one for each point-neuron. The power density spectra Pν(t) and Pϕ(r, t), of the population spiking activities and local field potential respectively, are also computed. The metamodel M directly models the power spectra given the parameter set, or the parameters given observed power spectra.
We train metamodels based on Gaussian process regression (GPR) [18], and compare them to a masked autoregressive flow (MAF) [19], a model often used for density estimation tasks. Both models can be trained to approximate the conditional probability densities of the simulator. Due to advances in sparse approximations to Gaussian process (GP) models, allowing them to be used on much larger data sets [20, 21], and extensions to deep models (DGPRs) [22–24], which gives them increased flexibility, GPs provide a powerful class of nonparametric regression and classification models. MAF is a type of normalizing flow, a parametric model based on making successive invertible transformations, typically based on neural networks, to a simple base distribution, typically assumed to be Gaussian. It can be trained either as a conditional distribution, where the neural networks take an additional input representing the condition, or as a simpler distribution without any conditioning. Models representing conditional distributions are particularly useful as a metamodel for systems with stochastic dynamics, since that implies that there is a distribution over possible outputs that has to be modelled. We additionally train a Hierarchical Cluster-based Partial Least Squares Regression [25](HCPLSR), a model that has been used previously for metamodelling of biological systems but only gives a point-estimate of the simulator output [8, 26].
We show that the simulator can be accurately captured by metamodels trained on around 1000 example simulations, and that for the present situation, the DGPR metamodel is able to more accurately estimate the function described by the simulator, compared to the MAF metamodel. We find that posterior distributions estimated from the DGPR metamodel is more accurate compared to the ones estimated from the MAF metamodel, but takes longer to compute. We estimate the posterior distributions separately for DGPR metamodels trained on the population spiking activities and the LFP, for a variety of different simulation outputs, and find that some parameters are significantly more constrained by observing the LFP compared to the population spiking activities, while for others both signals are roughly equally informative. We also investigate the pairwise correlations of the posterior distributions over the domain on which the metamodels were trained.
2 Methods2.1 Simulations
The network considered in this study consists of two populations, one with excitatory neurons and one with inhibitory neurons, of size NE and NI respectively. The neurons are modelled as leaky integrate-and-fire neurons, connected with alpha-shaped current-based synapses described by Eq 2 in Table 1. The sub-threshold dynamics are described by a first-order differential equation given by Eq 1 in Table 1. When its membrane potential reaches a firing threshold Vthr, the neuron fires and and the synapses on all its postsynaptic neurons are activated after a time td. After firing, the membrane potential of a neuron is clamped to a voltage Vreset for a refractory period tref. Each neuron receives synapses from a fraction ϵ of all other local neurons in the network. Post-synaptic currents from excitatory synapses have a peak amplitude J, and post-synaptic currents from inhibitory synapses have a peak amplitude of −gJ. Both types of synapses share a synaptic time constant of τsyn. Additionally, each neuron receives independent excitatory input from an external population modelled as a Poisson process with fixed rate νext. This rate is determined by the parameter η=νextνthr, where νthr = VthrCm/(Jτmeτsyn), the minimum constant input rate that will drive the neuron to its threshold. Since the synaptic strength and synaptic time constant both contribute multiplicatively to the total charge transfer during a synaptic event, we opted to parameterize the synaptic strength in terms of the total charge transfer during a synaptic event Qs = τsynJ. A complete description of the point-network model is given in Table 1.
10.1371/journal.pcbi.1011625.t001
Description of point-neuron network following the guidelines of [<xref ref-type="bibr" rid="pcbi.1011625.ref033">33</xref>].
A
Model summary
Populations
One excitatory, one inhibitory
Network model
Fixed in-degree, random convergent connections
Neuron model
Local populations: leaky integrate-and-fire, external: Poisson generator
Synapse model
Current-based alpha-shaped, fixed strength for each population
B
Populations
Names
Excitatory: E
Inhibitory: I
C
Network model
Connectivity
Fixed number of incoming connections CE = ϵNE from excitatory population and CI = ϵNI from inhibitory population
Input
Poissonian synaptic input with fixed rate νext for each neuron
D
Neuron model
Type
Leaky integrate-and-fire neuron
Description
Dynamics of membrane potential Vi(t) (neuron i ∈ [1, N]):
- Spike emission at times tli with Vi(tli)≥Vthr
- Subthreshold dynamics:
τmdVi(t)dt=-Vi(t)+RmIi(t)∀l:t∉(tli,tli+tref](1)
where τm is the membrane time constant, V the membrane potential, Rm the membrane resistance, and I the synaptic inputs.
Exact integration with temporal resolution dtUniform distribution of membrane potentials Vi ∈ [Vreset, Vthr) at t = 0
E
Synapse model
Type
Alpha-shaped postsynaptic current
Description
RmIi(t)=τm∑jJijτs∑lH(t-tlj-td)te1-(t-tlj-td)/τsyn,(2)where the first sum is over all the presynaptic neurons j, including the external ones, and the second sum is over the spike times of those neurons. tlj is the lth spike of presynaptic neuron j, and td is the synaptic delay. H denotes the Heaviside step function.Jij={J,j∈{E,Eext}-gJ,j∈{I}Multapses and autapses are allowed.
Ten model parameters were uniformly randomly sampled, using Latin Hypercube sampling [27], which is a stratified sampling strategy suitable for sampling in high-dimensional systems. A complete list of model parameters and their values and ranges are given in Table 2. 10000 simulations were run using the NEST simulator [28, 29], comprising both the training data set and the test data set. The regions of the parameter space giving strongly synchronous firing, discussed in [5, 30], were avoided by avoiding unbalanced regimes caused by low inhibition, and high external input (η > 3.5). Additionally, a small number of simulations, which were found to be strongly synchronous were removed, making up roughly 3% of the total simulations. These were found by the following ad hoc criteria: more than 150 time bins of 1 ms in which more than 10% of the neurons fire, as well as more than 500 time bins in which fewer than 0.25% of the neurons fire. See Section 3.1 and Section 4 for further discussion. For 100 of the simulation outputs in the test data set, an additional 50 simulations were run with parameters sampled from the estimated posterior distributions (see Sections 2.4 and 3.4). For each simulation, the LFP was computed as described in the next section.
10.1371/journal.pcbi.1011625.t002
Point-neuron network parameters.
Point-neuron parameters
Symbol
Description
Value
η
relative amount of external input
[1.0, 3.5]
g
relative strength of inhibitory synapses
[4.5, 8.0]
Qs
total synaptic charge transfer
[25 − 100] fC
τm
membrane time constant
[15–30] ms
Cm
membrane capacitance
[100 − 300] pF
td
synaptic delay period
[0.1 − 3.0] ms
tref
absolute refractory period
[0.1 − 4.0] ms
τsyn
synaptic time constant
[1.0 − 8.0] ms
Vthr
firing threshold
[15 − 25] mV
Vreset
reset membrane potential
[0 − 10] mV
EL
passive leak reversal potential
0 mV
NE
number of excitatory neurons
8000
NI
number of inhibitory neurons
2000
ϵ
connection probability
0.1
CE
number of incoming excitatory synapses
800
CI
number of incoming inhibitory synapses
200
Simulation parameters
Training and test data
Tsim
simulation duration
10500 ms
Ttransient
start-up transient duration
500 ms
dt
time resolution
0.1 ms
The population spiking activities were saved at a resolution of 1 ms as the sum of all spike trains in the populations. The power spectrum, defined as [ψ](f)*[ψ](f)/T for some signal ψ(t), where [ψ](f) denotes the Fourier transform of ψ, and T is the window length, was estimated using Welch’s method [31] for both the population spiking activities and the LFP. The implementation in the Python SciPy [32] package (scipy.signal.welch) was used, with a Hann window of length 256 and an overlap of 128. The metamodels are trained on the base 10 logarithm of these power spectra.
2.2 Forward-model predictions of LFPs
In order to compute local field potentials (LFPs) from the point-neuron network, we utilized the hybrid LFP scheme [34] (github.com/INM-6/hybridLFPy), which allows for the decoupling of the simulation of spiking dynamics (here computed using point neurons) and predictions of extracellularly recorded LFPs. The latter part relies on reconstructed cell morphologies and multi-compartment modelling in combination with an electrostatic forward model. A complete description of the scheme (including the biophysics-based forward model) can be found in [34], and a concise description of the multi-compartment neuron model is given in Table 3. The parameters used are given in Table 4.
10.1371/journal.pcbi.1011625.t003
Description of multi-compartment neuron populations.
A
Model summary
Populations
Local excitatory and inhibitory populations
Neuron model
Multi-compartment neurons with passive cable formalism
Synapse model
Current-based alpha-function shaped, fixed strength for each population
Topology
Cylinder of 1 mm2 cross-section with somas of both populations positioned in single layer of thickness 0.1 mm.
B
Neuron models
Type
Reconstructed multi-compartment morphologies with passive electrical properties
Description
For each neuron, the membrane potential Vn of compartment n connected to m other compartments k, with a surface area an, length ln and diameter dn is given by:∑k=1mgakn(Vk-Vn)=CmndVndt+Imn(3)Cmn=cman(4)gakn=π(dn2+dk2)/(4ra(ln+lk))(5)Imn=gLn(Vn-EL)+∑jIjn,(6)where for compartment n, Cmn is the membrane capacitance, gakn the axial conductance from compartment k, Imn the membrane current, gLn the membrane leak conductance, EL the extracellular reversal potential, and Ijn the synaptic current from presynaptic neuron j.
C
Synapse model
Synapse type
α-function shaped postsynaptic current
Description
I(t)=H(t-ta)JCte1-t/τsyn(7)H(t)=0fort≤0,otherwise1.(8)Here, ta is the activation time of the synapse, J the synaptic strength, and τsyn is the synaptic time constant. C is a constant chosen so that JC∫0∞te1-t/τsyndt=CmJ, assuring that the same total charge is transferred as in the δ-function synapse in the point-neuron network.
D
Topology
Type
Cylinder with radius 1/π mm and height 0.5 mm containing two vertical sections
Description
- Cylinder extends from z = −500 μm to z = 0
- All somas are randomly placed with a uniform distribution within the boundaries r ≤ 564 μm and −450 μm ≤ z ≤ −350 μm
- Two regions separated by the plane z = −300 μm
- Synapses on inhibitory neurons are placed in lower region
- Inhibitory synapses on excitatory neurons are placed in lower region
- Excitatory synapses on excitatory neurons are split equally between regions
10.1371/journal.pcbi.1011625.t004
Multi-compartment neuron parameters.
Multi-compartment neuron parameters
Symbol
Description
Value
τm
membrane time constant
[15, 30] ms
cm
membrane capacitance
[0.6, 1.4] μF/cm2
rm
membrane resistivity
τm/cm
Ra
axial resistivity
150 Ωcm
τsyn
synaptic time constant
[1.0, 8.0] ms
EL
passive leak reversal potential
0 mV
Vinit
membrane potentials at t = 0 ms
0 mV
σe
extracellular conductivity
0.3 Sm−1
Current-based synapses and morphologies with passive membranes in the multi-compartment neuron models give a linear relationship between any presynaptic spike event and contributions to the LFP resulting from evoked currents in all postsynaptic multi-compartment neurons. Thus the LFP contribution ϕYj(r,t) at position r from a single presynaptic point-neuron j in population Y can, in general, be calculated by the convolution of its spike train νYj(t)≡∑kδ(t-tjk) with a unique kernel HYj(r,τ) as ϕYj(r,t)=(νYj*HYj)(r,t). This kernel encompasses effects of the postsynaptic neuron morphologies and biophysics, the electrostatic forward model, the synaptic connectivity pattern, conduction delay and post-synaptic currents. This kernel can be further decomposed as a convolution between the LFP response of a unit current for a single time step, and the post-synaptic current. The resulting LFP due to spikes in a presynaptic population Y is then given by
ϕY(r,t)=∑j∈Y(νYj*hYj*IYj)(r,t),
where IYj(t) is the post-synaptic current given by Eq 7 in Table 3, and (hYj*IYj)(r,t)=HYj(r,t).
As shown in [30, 34], a good approximation can be made by convolving the population firing rates νY(t)≡∑j∈YνYj(t) with averaged kernels H¯Y(r,τ)≡1/NY∑j∈YHYj(r,τ), that is,
ϕY(r,t)=(νY*H¯Y)(r,t).
As in [30, 34] these averaged kernels H¯Y(r,τ) were computed using the full hybrid-scheme. This was done by computing the LFP resulting from a fully synchronous activation of all the outgoing synapses from all neurons in the presynaptic population. Thus for the computation of the LFP kernel, we have νYj(t)≡δ(t-tY) where tY is the timing of the synchronous event in population Y.
After the single time-step kernel has been computed, the full kernel can easily be computed for any post-synaptic current, and hence synaptic parameters. In order to account for the membrane parameters τm and cm, where cm is the per-area capacitance, we compute the kernel for a single time step on a grid of different values of τm and Cm, and adjust this kernel by linear interpolation. We compute the kernels at the values [0.4, 0.6, 0.8, 1.0, 1.2] for cm and [15, 18, 21, 24, 27, 30] for τm. Since the membrane capacitance for morphological neurons is dependent on the area of the neuron, we choose a baseline value of cm = 1.0, and adjust it by the same proportion as Cm for the point-neuron network. The synaptic parameters can subsequently be accounted for in a separate step.
2.3 Gaussian process regression
Here we give a brief overview of Gaussian process regression models, their training procedures and the extension to deep models.
Gaussian process regression models are probabilistic non-parametric models that provide distributions over arbitrary functions, and are trained using supervised learning. A Gaussian process (GP) is a collection of random variables such that any finite number of them follow a joint Gaussian distribution, and a GP is completely specified by a mean functionm(θ) and a covariance functionk(θ, θ) [18]. Consider a data set consisting of n inputs of dimension d contained in the matrix Θ, and corresponding outputs x, which are assumed to be one-dimensional for notational simplicity in this section. A Gaussian process regression model assumes a distribution over an underlying latent function f(θ) which is modelled as a Gaussian process. Evaluating this distribution at the set of input points Θ gives a joint Gaussian distribution f(Θ)∼N(0,k(Θ,Θ)), where the covariance function k(Θ, Θ) must return a positive semi-definite covariance matrix between all the data points contained in Θ. The mean function is usually assumed to be 0 without loss of expressiveness. The covariance function encodes the properties of the function, and there are many different covariance functions available with different properties. In the present work, we use the Matérn 5/2 kernel
C5/2(θ,θ′)=σ2(1+5d+53d2)exp(-5d),d=5∑i(θi-θi′li)2,
where l and σ are learnable parameters.
The model of the observed outputs x is given by the underlying latent function f(Θ) corrupted by some noise ϵ∼N(0,σn2I) such that p(x|f)=N(f,σn2I). The marginal likelihood of the data is obtained as
p(x;Θ)=∫p(x|f;Θ)p(f;Θ)df=N(0,(K(Θ,Θ)+σnI)-1),
which can be found analytically in the case of a Gaussian likelihood p(x|f, Θ), and is used to optimize the model parameters.
Given a new set of inputs Θ* on which we would like to evaluate the model, the GP specifies a joint distribution over x and x*, and the predictions are made as the distribution over x* conditioned on the training data x [18]
p(x*|x;Θ*,Θ)=∫p(x*|f*;Θ*)p(f*|f;Θ,Θ*)p(f|x;Θ)dfdf*=N(x*|Kn*Knn-1x,K**-Kf*T(Knn+σn2I)-1Kf*+σn2I),
where we use the shorthand notation Knn = k(Θ, Θ), Kn* = k(Θ, Θ*) and K** = k(Θ*, Θ*). The semicolon indicates the input points corresponding to the distributions, on which the covariance function is evaluated. We can immediately see that the mean value of the distribution at the predicted locations Θ* are linear combinations of the training values x.
Deep, or composite, GP models are constructed by iteratively defining GPs with the previous GP as inputs. A joint prior distribution over all layers is given by
p(x,{fl,}l=1L)=∏i=1Np(xi|fiL)∏l=1Lp(fl;fl-1),
where f0 = θ. As in standard GP models, we would like to marginalize out the latent functions in all layers for optimization. For predictions, we would like to condition the latent functions on the training data and then marginalize them. This is obviously not possible analytically, and approximations have to be used. We do not go in to details, but refer the readers to [35] for an excellent tutorial.
For multi-dimensional outputs, one can either assume the outputs to be independent, or correlated. In the former case, a separate GP is indepedently trained for each output dimension. For the latter case, one simple way of introducing correlations between output channels is to make a linear combination of independent GPs. If the number of independent GPs is lower than the number of output dimensions, a low-dimensional representation of the output is achieved, with a linear mapping between the latent space and the output. The model can be written as
g(θ)={gp(θ)}p=1P,f(θ)=Wg(θ),
where gp(θ) forms a set of independent GPs, and W is an n × p matrix used to linearly construct the output f(θ) [36].
This can be viewed as an input-dependent linear latent space model, where we assume a linear mapping between a latent space and the output, but the coefficients of this mapping can change in a non-linear fashion. If we choose P to be a small number, a low-rank representation of the output is achieved.
We used a DGPR model based on the framework by [23]. We use 2 layers, and do a parameter scan in order to find the number of GPs to use in each layer. The final output is constructed by a linear transformation from the GPs in the second layer as given by Eq 16.
2.4 Posterior distribution over parameters
The metamodels provide a distribution p(x|θ), which is an approximation to the distribution over the outputs given by the simulations, where x is the simulation output, and θ is the simulation inputs. Introducing a prior distribution over the inputs p(θ), which we take to be uniform over the ranges of input values used to train the metamodels, the unnormalized posterior distribution over parameters given a specific observed model output x0 is given by p(θ|x0) ∝ p(x0|θ)p(θ). We sampled from this distribution using an adaptive Metropolis algorithm [37]. The adaptive Metropolis algorithm is a random walk Markov Chain Monte Carlo algorithm, where a multivariate Gaussian proposal distribution is used, and where its covariance matrix is iteratively estimated by the sample covariance. We only estimated the covariance during the burn-in period, and kept it constant during sampling. Additionally, we used a separate scaling factor for the covariance matrix which was tuned during the burn-in period to give an acceptance ratio of around 0.3. Five chains were run in parallel, and run for a total of 40000 evaluations of the proposals, where the samples from the first 12000 proposals were discarded and considered the burn-in period. For each example in the test data set, the posterior distribution was sampled separately for both the metamodel of the population spiking activities and the metamodel for the LFP. The Gelman-Rubin statistic [38] was computed independently for each parameter, for each distribution, in order to assess the convergence properties of the chains.
2.5 Technical details2.5.1 Reproducibility
The simulations were run using Python v3.10.5. All point-network simulations were run with the NEST simulator v3.3 [28, 29]. The forward-modeling of the LFP was done using hybridLFPy v0.2 [34] with LFPy v2.3 [39]. The DGPR metamodels were trained using GPyTorch v1.9.1 [40] with PyTorch v1.13.1. The MAF metamodels were trained using SBI v0.21.0 [41]. The metamodels were trained on NVIDIA Quadro RTX 8000 GPUs. All simulation and analysis code for this study is available on Github: https://github.com/janskaar/Skaar_bioRxiv_506616v1
3 Results
The aim of this study was to apply metamodelling techniques to simulations of neuronal networks. Specifically, we applied Deep Gaussian process regression (DGPR) and masked autoregressive flow (MAF) models to approximate the likelihood of the network simulator model. In all cases, we consider the output of the simulator to be the power spectrum of either the summed population spiking activity, or the local field potential generated by the network. We compare the accuracies of the forward predictions of all three models on the population spiking activities. The DGPR metamodel is trained on both the LFP and the population spiking activities in order to evaluate the differences in information in the two signals. For the DGPR and MAF metamodels, we also estimate the posterior distribution for 100 examples in the test data set, and evaluate the accuracy of these based on the posterior predictive distribution.
We start by showing some example simulation outputs, and discuss the accuracy of the forward predictions of the metamodels. We then evaluate the number of training samples needed for the metamodels to obtain high prediction accuracy, before we discuss the inverse modelling and the posterior distributions. A HCPLSR metamodel was also trained on the same task. We refer to S3 Supplementary Section. for the results from that model.
3.1 Simulations
A total of 10000 simulations were run, with parameters sampled as described in Section 2.1. A network consisting of one excitatory and one inhibitory population can produce a variety of activity states, characterized by whether the neurons fire in synchrony, and how regularly they fire, that is, whether the inter-spike intervals for individual neurons have a wide or narrow distribution. The behaviour of the present two-population network was analyzed in the framework of mean-field theory in [5] for integrate-and-fire neurons where the synaptic currents were modelled as delta functions. Brunel [5] found that the network spiking activity could be described by distinct regimes, classified as synchronous regular (SR), asynchronous irregular (AI) and synchronous irregular (SI, fast and slow), depending on which specific parameter values were used, and arise by different mechanisms. As described in Section 2.1, we avoided activity states characterized by strongly synchronous behaviour, which produces sharp peaks in the power spectrum. Fig 2 shows three examples of different states of network activity. Column A shows the synchronous regular regime, characterized by runaway excitation, where the synaptic strengths of inhibitory neurons are not strong enough to overcome the excitation, and the neurons fire regularly and synchronously with very high rates. Column B shows an example of a synchronous irregular state, produced in some cases when the network receives high amounts of external input and the inhibitory synaptic strengths are high. Column C shows asynchronous irregular activity. This is the regime in which most of the simulations used for training and validation were, although there is of course much variation within this regime. Columns A and B show the type of activity we avoided, with sharply peaked spectra.
10.1371/journal.pcbi.1011625.g002
Example activity states.
Each column shows, from top to bottom, a raster plot, population firing rates, the power spectrum of the population averaged firing rates, the LFP, and the power spectrum of the LFP. The raster plot and firing rates are taken from the excitatory population. A: Synchronous regular activity. B: Synchronous irregular activity. C: Asynchronous irregular activity.
We will not attempt to give a full description of all possible network states or the effects of all different parameters in this paper, but show these examples to make clear the states we have actively avoided, as described in Section 2.1.
3.2 Forward predictions
The metamodels were tested on a data set containing 1681 examples, jointly sampled with the training data using Latin Hypercube sampling. Models were trained on data sets ranging from 100 to 8000 examples in order to examine the density of samples needed to adequately represent the simulation output space. For all other experiments, the metamodels used were trained on the full 8000 example simulations. A grid search of hyperparameters was run for each metamodel in order to find a suitable model for the problem at hand, which can be found in S1 Supplementary Section. The metamodel obtained from the hyperparameter search was used for all subsequent experiments, and where we trained new metamodels, the hyperparmeters were the kept the same. Fig 3A and 3B show examples of simulated outputs and corresponding outputs from the metamodels for two different sets of parameter values, for the population spiking activities and LFPs respectively. Only the outputs for the excitatory population spiking activity and the topmost LFP channel are shown. 10 simulations run with different seeds are shown in black lines, and the colored lines show the output given by the metamodels. The shaded area shows two standard deviations above and below the mean for the DGPR and MAF metamodels. Visually inspecting the top example in panel A, both metamodels accurately capture the mean of the simulations for all frequencies, but slightly overestimate the variance, particularly for the lower frequencies. In the bottom example, there is a larger discrepancy between the MAF metamodel and the simulator for the middle frequencies, while the DGPR metamodel is accurately able to capture both the mean and variance. Note that the covariance between frequency channels is not considered at all in this figure. In panel B, the LFP for the same simulations and the DGPR prediction are shown. Generally, the LFPs are smoother and the variance is lower compared to the population spiking activity. Fig 3C and 3D show the distribution over maximum (over frequencies) absolute errors on the test data set, where the expectation of the metamodel is used as the prediction value. We show the maximum error over frequencies since the large number of output channels make the mean small even in the presence of a few large deviations. Moreover, as the simulation output itself is random, we are mainly interested in finding any larger deviations, rather than small deviations, which are expected to occur. The expectation of the distribution given by the DGPR metamodel more closesly matches the simulated values compared to the MAF metamodel. Fig 3E and 3F show the mean standard deviation of the metamodel distributions evaluated at the test data parameters. The DGPR metamodel produces narrower distributions over the simulator output compared to the MAF model.
10.1371/journal.pcbi.1011625.g003
A: For two different parameter sets, the power spectrum of the population spiking activity from 10 simulations (black lines), and metamodels (blue and orange lines) are shown. The shaded area shows two standard deviations of the distribution given by the metamodels. B: Same as A, but for the LFP. Only the DGPR metamodel was trained on the LFP. C: Distribution over maximum (over frequencies) absolute errors for both metamodels, on the power spectrum of the population spiking activity. D: Same as for C, but for the LFP. E: Mean standard deviation of the metamodel outputs, evaluated at the parameters in the test data set. F: Same as E, but for the LFP.
In terms of computational cost, evaluating the log probability of 50 example outputs takes around 4 seconds with the DGPR metamodel, and 0.014 seconds with the MAF metamodel. Generating 50 samples from the metamodels takes 1.6 seconds for the MAF metamodel, and 0.07 seconds with the DGPR metamodel. By contrast, running a single simulation took on average 4.4 CPU-minutes.
The training time for the metamodels depends on the model parameters and learning rate, as well as the random seed. A full epoch of the 8000 training examples takes around 4.4 seconds for the DGPR metamodel, and 1.4 seconds for the MAF model. Generally, the training converged after a few hundred epochs for both models, giving a training time (not counting the periodic train/test evaluation) on the scale of an hour for the DGPR model and around a third of that for the MAF model.
3.3 Number of training samples needed to represent the model
The DGPR metamodel was trained on data sets ranging from 100 to 8000 examples of the population spiking activities, and the performances on the same test data set were evaluated. Fig 4 shows the maximum (over frequencies) absolute prediction errors, averaged over the test data set for each trained metamodel. As the number of training samples increases, the prediction errors on the test data set decreases, as the metamodel starts to generalize better. The prediction errors on the training data set increases, as it becomes more heterogeneous and the model is less able to overfit to it. After a sharp decrease in errors as the number increases to a few hundreds, the benefit of increasing the number of training samples decreases significantly after on the order of 1000 is reached, indicating that the parameter space is sufficiently densely sampled.
10.1371/journal.pcbi.1011625.g004
Mean (over test data set and training data set) of the maximum (over frequencies) errors, in absolute scale, as a function of the number of examples used to train the metamodels on the population spiking activities.
3.4 Inverse modelling
Using the conditional distribution over simulation outputs given parameter values, estimated by the metamodels, posterior distributions for 100 simulations in the test data set were estimated. Separate posterior distributions for both metamodels conditioned on the population spiking activities, and for the DGPR metamodel conditioned on the LFP, were estimated using an adaptive Metropolis algorithm as described in Section 2.4.
The Gelman-Rubin statistic was computed for all the distributions, and was below 1.1 for over 90% of the posterior distributions for the population spiking activities, and over 95% of the posterior distributions for the LFP, indicating that the chains converged for most of the distributions. In the computation of average values over the test data set, all distributions were used, regardless of the value of the Gelman-Rubin statistic. For each of the estimated posterior distributions of both the population spiking activities and the LFP, 50 additional simulations were run with parameters drawn from the posterior distributions in order to evaluate the accuracy of the posterior distribution.
Fig 5A shows an example of a posterior distribution generated by the outputs of one simulation in the test data set. The 1D and 2D posterior marginal distributions of a subset of the parameters are shown. On the left-hand side, the posterior distribution generated by the population spiking activities are shown, and the corresponding posterior distribution from the LFP is shown on the right-hand side. The red bars and dots show the values of the parameters used in the simulation from which the posterior is generated. Qualitatively, the distributions from both the population spiking activities and the LFP appear similar for the first three parameters shown.
10.1371/journal.pcbi.1011625.g005
In A, C and D, the left-hand and right-hand side show equivalent plots for the population spiking activities and LFPs respectively. The DGPR metamodel is used in both cases. A: 1D and 2D marginal posterior distributions over a subset of the parameters, for an example in the test data set. The red dots and bars show the parameter values corresponding to the simulation output the posterior distribution was computed for. B: Left: mean (across test data set) of standard deviation of the 1D marginal distributions for each parameter. Right: Absolute error of the parameter predictions based on the expectation. C: Metamodel predictions for the power spectra of excitatory population spiking activity (left) and the uppermost channel of the LFP (right) in orange. Black lines show 50 simulation outputs run with parameters drawn from the posterior distribution. D: Distribution of maximum distance between the ground-truth simulation output from which the posterior distribution was computed, and the simulation output from the posterior distributions (black lines in C). Left plot shows the errors from the population spiking activities, right plot shows the errors from the LFP. The black lines show the simulations from the posterior conditioned on the population spiking activities, the gray lines show the simulations from the posterior conditioned on the LFP.
For this particular example, the 1D marginal distributions of both posteriors are wide for the membrane time constant τm, quite narrow for g, and somewhere in between for η. Wide marginal distributions entail that, lacking information about all other parameters, the value of the parameter cannot be determined with certainty, while narrow marginal distributions entail that the parameter can be accurately determined even without knowledge of other parameters. There is a positive correlation between η and τm in the posterior distribution, i.e. if the value of η is increased the value of τm must also be increased for the model to produce similar outputs. Equivalently, if you know the specific value of either η or τm, the uncertainty in the other parameter would decrease. Note that because we only see 1D and 2D marginals, there may be higher-order interactions present that cannot be easily visualized. For the synaptic strengths, the distribution is much wider for the model trained on the population spiking activities compared to the model trained on the LFP.
The left-hand side of Fig 5B shows the mean (over all test examples) standard deviations of the 1D marginal distributions. Generally, the parameters related to the synaptic inputs have the narrowest marginal distributions, while the neuron parameters have much wider marginal distributions, indicating that changes to neuron parameters can largely be compensated for by adapting other parameters, while changes to synaptic and input parameters have a larger effect that cannot be compensated for to the same degree. The right-hand side of Fig 5B shows the mean (over all examples in the test data set) absolute distance in standard deviations of the parameters of the simulation used to generate the posterior (i.e. red dots in Fig 5A) and the expectation of the posterior distribution. For all the parameters, the mean absolute distance is smaller than the standard deviation of the approximate marginal posterior distributions, indicating that the approximate posteriors generally covers the parameters of the simulation the posterior distribution is conditioned on.
One notable difference between the posterior distributions computed from the metamodel of the population spiking activities and the posterior distributions computed from the metamodel of the LFP is that the overall synaptic strength, Qs, and the membrane capacitance, Cm, are significantly more constrained by observing the LFP than by the population spiking activities, with a standard deviation more than three times as large for the population spiking activities. This can also be observed in the example distributions shown in Fig 5A. For the rest of the parameters, the standard deviations of the 1D marginal distributions are roughly the same when computed for the population spiking activities and the LFP. The full posterior distribution from Fig 5A is shown in S1 Supplementary Section.
Fig 5C shows the metamodel output (orange) corresponding to the same simulation as in A, and 50 individual simulation outputs (black) run with parameter values sampled from the posterior distribution of the example shown in A. Only the spectra from the excitatory population spiking activity and the topmost LFP channel are shown. The mean of the metamodel distribution is shown in dashed orange, and the shaded area shows ± 2 standard deviations. The simulations run with parameters from the posterior distribution fit nicely with the distribution over the outputs given by the metamodel, suggesting that the posterior distribution is reasonable. Fig 5D shows the distribution of the maximum (taken over frequencies and simulations for a given posterior) distance between the simulation outputs run with parameters drawn from the posterior distributions, and the simulation output the posterior is conditioned on, for the 100 posterior distributions for which the extra simulations were run.
The black line shows the errors for the simulations run with parameters drawn from the posterior generated by the population spiking activities, while the gray distribution shows the distances for the simulations run with parameters drawn from the posterior generated by LFPs. The left-hand side shows the distances of the population spiking activities, while the right-hand side shows the distances for the LFP. For both the population spiking activities and the LFP, the simulations from the posterior distributions are very similar to the signal they were conditioned on. Simulations from the posterior distributions conditioned on the LFP, also give population spiking activities very similar to the ground-truth. Simulations from the posterior distributions conditioned on the population spiking activities, on the other hand, do not give as similar LFPs to the ground-truth, which again shows that observing the LFP constrains the parameters to a greater degree than observing the population spiking activities.
Fig 6 shows a comparison between the posteriors obtained by the DGPR metamodel and the MAF metamodel. Panel A shows the marginal standard deviations, averaged over all posterior distributions. For both metamodels, they are similar, but the DGPR is generally slightly wider. Panel B shows the same plot, but for the posterior predictive distribution, i.e. the distribution over outputs when parameters are sampled from the posterior distribution. The posterior predictive distributions from the DGPR metamodel are significantly narrower than those from the MAF metamodel, even with wider posterior distributions, indicating that the posterior distributions are more correct compared to those from the MAF metamodel. Panels C and D show the mean absolute distance between the expectation of the posterior distribution and posterior predictive distribution respectively, and the simulations they are conditioned on. For both models, the errors are on the same scale as the standard deviations, indicating that neither systematically misses the simulation it is conditioned on.
10.1371/journal.pcbi.1011625.g006
A: mean marginal standard deviations of the posterior distributions. B: mean marginal standard deviations of the posterior predictive distributions. C: mean distance between expectation of marginal posterior distribution and the parameters from the simulation the posterior distribution is conditioned on. D: Same as C, but for the posterior predictive distribution.
The Pearson correlation coefficient was computed for all pairwise combinations of parameters for all sampled posterior distributions. Fig 7 shows the mean of the correlation coefficients over all examples in the test data set. The upper triangle shows the coefficients for the samples from the posterior distribution conditioned on the population spiking activities, and the lower triangle shows the coefficients conditioned on the LFP. The correlations between most of the parameters are fairly weak, but since only the average is shown, there could be stronger correlations in individual posterior distributions that the plot does not capture. For the LFP, there are strong average positive correlations between the synaptic parameters, i.e. the synaptic strength Qs, the synaptic time constant τsyn and relative inhibitory strength g. Interestingly, the corresponding coefficients for the population spiking activities are close to 0. This again highlights the difference in the posterior distributions for Qs between the population spiking activities and the LFP. For the distributions conditioned on both the LFP and the population spiking activities, there are positive correlations are present between the parameters η and τm, as well as negative correlations between η, Vreset and Cm. As these parameters all impact the amount of input required for a neuron to spike, there are possibly complex interactions between all these parameters yielding manifolds of similar activities and LFPs.
10.1371/journal.pcbi.1011625.g007
Pearson correlation coefficients between samples from posterior distribution over parameters, averaged over the test data set.
Upper triangle shows the correlation coefficients for the posterior distributions from the simulated power spectra from the population spiking activities, while the lower triangle shows the same correlation coefficients from the LFP.
4 Discussion
In the present work we have trained metamodels to estimate the power spectra of population spiking activities and local field potentials (LFPs) generated by a two-population neuronal network comprising an excitatory and an inhibitory population during stationary activity. Specifically, we have trained a deep Gaussian process regression (DGPR) model, and a faster masked autoregressive flow (MAF) model, and used the probabilistic nature of the metamodels to estimate posterior distributions over parameters given observed simulation outputs. DGPR models have not been much used for statistical modelling in the field of neuroscience before, and we chose to compare it to the MAF as it is a commonly used model for both forward modelling in the case of SNLE and inverse modelling in the case of SNPE.
Both the population spiking activities and the LFP were modelled, as this offers an opportunity to compare the information about the parameters that can be gained by observing different signals. The LFP was computed separately for each population using Eq 10. This is an approximation that does not require computing the LFP response from each individual neuron in each simulation, which is computationally costly, but only requires computing a small number of kernels, which can then be used to approximate the LFP for all simulations. The synaptic parameters can be fully adjusted for by a single simulation of a kernel, but the neuron membrane conductance and capacitance, cannot be adjusted for analytically. Kernels were therefore computed on a grid of these two parameters, and regression was done to approximate the true kernel for any value of the membrane conductance and capacitance. As shown in S4 Supplementary Section, the approximation works well, although there are some small deviations, particularly in channels 4 and 5.
The neuron and synapse parameters of the simulations were varied, with values described in Table 2, while the number of neurons and number of synapses were fixed. The parameter values were chosen such that the strongly synchronous regions of the parameter space were avoided, particularly avoiding relative inhibitory synaptic strengths that lead to an unbalanced regime. This will also have excluded some AI-like simulations in the vicinity of the boundary between AI and SR, but we consider the difference (starting at g = 4.5 instead of g = 4.0) to be sufficiently small as to not significantly diminish the utility of the models. Additionally, a small number of simulations in the strongly synchronous irregular regime were removed, comprising roughly 3% of the total number of simulations as described in Sections 2.1 and 3.1. The simulations in the test and training data sets consisted mostly of simulations in the asynchronous irregular (AI) regime. The simulations with strongly synchronous spiking activities were avoided since sharp peaks that shift continuously in frequency cannot be well approximated by a low-dimensional representation, and the low-rank approximation was necessary to keep the size of the DGPR metamodel small. The low-rank approximation is a limiting factor for the DGPR metamodel, and ideally one would not have to limit the input domain in such a way. Increasing the dimensionality of the latent space significantly is possible and would possibly also have resulted in better accuracy, even in the case of strongly synchronous activity, particularly with a larger training data set, but trade-offs have to be made with respect to the computational cost of training and evaluating the metamodels. The simulations in the AI regime could be well approximated by a low-dimensional representation, and since one is often particularly interested in this regime, we consider it reasonable to focus the metamodels on this.
The power spectra were modelled instead of the time series. Although modelling time series directly in many cases would be preferable, that would require modelling the complex temporal dependencies in the neural networks, which are also stochastic due to the external input to the network modelled as a Poisson process. Since most regression techniques assume a fixed target, modelling the power spectra is more straightforward. For the present situation, we found that both the DGPR and MAF metamodels were able to accurately predict the power spectrum of the population spiking activities, with 10 different model parameters as inputs, and we found that on the order of around 1000 examples is sufficient to accurately represent the model in this particular region of the parameter space. In general, the number of examples needed to train a metamodel will depend on the number of parameters included, the range of their values and how much they affect the model output. It is difficult to predict a priori, and the presently used model may not be representative for other, more complex models, which may require more training examples.
Theoretical approaches based on linear-response theory can provide analytical approximations to the PSDs [42–45] in the asynchronous irregular regime. The theoretical models necessarily make assumptions about the underlying physics of the system, which also makes it possible to interpret them in ways that are not possible with purely data-driven methods. Data-driven methods are complementary to theoretical ones in the sense that they do not make any underlying assumption about the model, but comes at the price of interpretability and needing to run simulations to train the models.
The forward modelling of the population spiking activities and the LFP is similar in terms of difficulty, where the errors of the DGPR metamodels are similar. This may not be surprising, as the LFP is computed as a convolution of the population spiking activities. The low-pass filtering effects of synaptic and dendritic filtering makes the power spectrum of the LFP smoother than the population spiking activities, however, which could potentially have made it an easier target [46, 47].
Comparing the DGPR metamodel to the MAF metamodel, we found that the DGPR metamodel provided better accuracy, but comes at the price of a higher cost of training. When it comes to running the trained models, the MAF is significantly faster than DGPR for evaluating the log probability, and therefore also when estimating posterior distributions. For purely sampling the forward model, the DGPR is faster than the MAF metamodel. This is not surprising, as the MAF model is designed to be good at evaluating log probabilities, at the cost of being slow at sampling. The time it takes both to train and evaluate the metamodels will depend strongly on the available hardware. GPU acceleration makes it significantly faster, and solely using CPUs would significantly increase the time of training/evaluating the metamodels.
The MAF metamodel suffered significantly more from overfitting compared to the DGPR metamodel. Simply increasing the number of simulations used to train the models would likely reduce the differences between the two metamodels. It is also possible that more regularization techniques could have increased the performance of the MAF, although utilizing dropout did not significantly improve the model.
For inverse modelling, the benefit of using the more computationally demanding DGPR will likely be smaller if one is doing sequential inference, and the region in which the metamodel must be accurate is smaller. Also depending on the complexity of the simulator one is approximating, in cases where one wants an accurate model over a larger domain, increased accuracy may be well worth the increased computational cost.
The HCPLSR metamodel, shown in S3 Supplementary Section, is significantly faster than both the DGPR and MAF metamodels, but at the cost of only giving a point-estimate of the simulation output, and less flexibility. For cases where one is only interested in the forward modelling, such alternatives can also be an option.
4.1 Model inversion
One key motivation for using probabilistic metamodels is that it allows inverting the model and finding posterior distributions over parameters given observed outputs. We opted to sample the posterior using an adaptive Metropolis algortihm as it is simple to implement and worked well for the present problem. The Gelman-Rubin statistic was computed, and for most of the sampled posterior distributions (> 90%), it was below 1.1, which is often considered to indicate convergence [48]. For some of the posterior distributions, the chains did not converge, although this was likely only for a minority. Particularly for the examples on which the forward predictions are poor, we would not expect the estimated posterior distributions to be accurate, as the metamodel generally will not produce outputs which are similar to the observed output used to generate the posteriors.
The degree to which the different model parameters were constrained by observing an output varied highly (see Fig 5B). Notably the distribution over most neuron parameters were wide, while the distribution over synapse and input parameters were narrower. Interestingly, the synaptic strength (Qs) and membrane capacitance (Cm) were significantly more constrained by the LFP than by the population spiking activities, illustrating the utility of using different signals for model validation. The effect is likely due to the fact that LFP is directly dependent on both the post-synaptic currents as well as the population spiking activities. The effect was not seen for the synaptic time constant, which also directly influences the post-synaptic current, something that indicates that the extra information may be in the scale of the LFP, as that is directly determined by the scale of the post-synaptic currents. The remaining parameters were similarly constrained by both signals, indicating that for the present situation, both signals may contain similar information about the parameters.
A previous study on model inversion using the LFP of a similar network using convolutional neural network [30] provided more accurate predictions of parameter values. In that study, neuron models with delta synapses were used, and it was limited to only three parameters, which likely makes it an easier task. Particularly, wide marginal posterior distributions might only be possible if there are sufficiently many parameters that are free to vary such that they can compensate for the effects of other parameters. An interesting question which we did not pursue here is to what degree keeping the different parameters fixed constrains the remaining parameters given observed simulation outputs.
Comparing the posterior distributions generated from the DGPR metamodel to the posterior distribution generated from the MAF metamodel, we find that the posterior distributions from the DGPR metamodel are wider, while simultaneously having a significantly narrower posterior predictive distribution. In other words, it is able to find a larger region of the parameter space that gives more similar outputs compared to the MAF metamodel. This shows the importance of accurate forward modelling in order to find good posterior distributions, an area where DGPR metamodels can be useful also in other settings when dealing with data from complex simulations.
Methods such as Approximate Bayesian Computation (ABC) and Simulation-based Inference (SBI) can also be used to estimate posterior distributions over parameters given specific observed model outputs. Our work is closely related to SBI, particularly SNLE [13]. The approach in the present work can be considered the same as SNLE run for a single round, where we use the DGPR metamodel as a likelihood estimator. Thus, in cases where one is interested in estimating posterior distributions, particularly in cases where one has limited data and model flexibility is a concern, DGPR metamodels may be of use in the SNLE framework. Methods such as Sequential Neural Variational Inference [49] for estimating the posterior distribution instead of MCMC sampling can also readily be used with the DGPR metamodel.
4.2 Generalization to more complex network models
The two-population model is fairly simple, and is a useful starting point since the dynamics are well known, and the simulations are cheap to run. A natural question is how well these metamodelling techniques generalize to more complex networks such as a multi-population model of cortical circuitry [50], a multi-area model of the visual system [51, 52], or a large-scale model of mouse primary visual cortex [53]. Models with more populations will have more potential model outputs to capture, and richer dynamics, and a larger number of parameters will yield a higher dimensional input space. The expressiveness of the metamodels may have to increase, and the number of samples needed to train a metamodel will likely be much higher. Of course, the number of parameters one chooses to include is arbitrary, and if one is only interested in a subset of the parameters, that can be modelled more cheaply. It is interesting to note, however, that in the present case, the power spectra mostly in the AI regime can be accurately captured by a low-dimensional latent representation.
In the present case, we model the external spiking input as a constant rate Poisson process, parameterised by the variable η. In more realistic networks, the input to the network will generally be more patterned. The parameters in the network we used were also kept constant, while in more realistic networks the parameters might follow a statistical distribution instead. Similar methods can be used to find the parameters of such distributions instead of the constant value as in the present case. Patterned input could also be represented in a metamodel, either through some continuous input to the model if the input changes in a continuous manner, or possibly discretely, in which case it may be necessary to train multiple metamodels for each input class separately.
The metamodels can easily be made more flexible, for the DGPR metamodel by increasing the number of GPs in either layer, and increasing the dimensionality in the latent space, and for the MAF metamodel by increasing the number of transformations or increasing the number of hidden features in each transformation. This will lead to higher computational costs, but more complex neural networks are of course also more costly to run, so the relative benefit may not change. In the present case, sampling was done using Latin Hypercube sampling, which ensures that each parameter is uniformly sampled. This can potentially lead to a too dense sampling in regions of the parameter space where the metamodel performs well, and too sparse sampling in regions where the metamodel performs poorly. Methods for adaptively sampling only or mostly in regions where additional sampling is needed would be an interesting direction for future research.
4.3 Use of metamodels
Once a metamodel has been trained, it can be evaluated much more rapidly than the model it is trained on. If it is well calibrated, it should be able to interpolate accurately inside the volume of the parameter space which was used to train it, and give a continuous approximation to the model behaviour. For models which are computationally demanding, trained metamodels could be distributed for use by others without requiring large computational resources.
For working with experimental data, metamodels could be useful in different ways. In the context of SBI, metamodels can be used as likelihood estimators in order to find parameters fitting network models to observed experimental data, in the same manner as in this work. Depending on which phenomena one is interested in modelling, this might require larger and more realistic network models than the one used here, but the principle would be the same. In the context of forward modelling, metamodels could be applied to experimental data to guide the choice of experimental parameters and their effects on experimental outcomes. How well this would work in practice, however, is not known. It would require training to specific experimental settings, and could be an interesting direction of future research.
4.4 Conclusion
We have shown that for the present situation, the power spectra of the population spiking activities and the local field potential can be accurately modelled by the DGPR and MAF metamodels, over a domain largely consisting of activity in the AI regime. The DGPR metamodel performed better than the MAF metamodel, but at the cost of higher computational demands for training and evaluation. The trained metamodels can be used to estimate posterior distributions over simulation parameters given observed simulation outputs. The posterior distribution estimated from the DGPR metamodel had higher marginal variances compared to the posterior distributions estimated from the MAF metamodel, while the posterior predictive distributions from the DGPR metamodel were narrower compared to the posterior predictive distributions from the MAF metamodel, indicating more accurate posterior distributions. When dealing with complex models, utilizing DGPR for forward modelling may therefore be a good alternative, particularly when the amount of data is a limiting factor, and the increased training time is not a problem.
Based on the estimated posterior distributions, the LFP was found to contain significantly more information about the synaptic strength and membrane capacitance, while for the remaining eight parameters, both signals were equally informative.
Supporting information
Hyperparameter scans for determining best metamodel configurations.
(PDF)
Full posterior distributions from which the subset in <xref ref-type="fig" rid="pcbi.1011625.g005">Fig 5</xref> is taken.
(PDF)
Hierarchical Partial Least Squares Regression.
Description and results from HCPLSR metamodel.
(PDF)
Validation of LFP computation scheme.
(PDF)
ReferencesEinevollGT, DestexheA, DiesmannM, GrünS, JirsaV, deKamps M, MiglioreM, NessTV, PlesserHE, SchürmannF. The Scientific Case for Brain Simulations Neuron. 2019;4:735–744DayanP, AbbottLF. Theoretical Neuroscience. MIT Press, Cambridge; 2001.KochC. Biophysics of Computation. Oxford Univ Press, Oxford; 1999.SterrattD, GrahamB, GilliesA, WillshawD. Principles of Computational Modelling in Neuroscience. Cambridge University Press; 2011.BrunelN. Dynamics of sparsely connected networks of excitatory and inhibitory neurons. Computational Neuroscience. 2000;8:183–208. doi: 10.1023/A:1008925309027EinevollGT, KayserC, LogothetisNK, PanzeriS. Modelling and analysis of local field potentials for studying the function of cortical circuits. Nat Rev Neurosci. 2013Nov;14(11):770–85. doi: 10.1038/nrn359924135696MartensH, TøndelK, TafintsevaV, KohlerA, PlahteE, VikJO, GjuvslandA, OmholtS. PLS-Based Multivariate Metamodeling of Dynamic Systems Springer Proceedings in Mathematics and Statistics. 2013;56TøndelK, MartensH. Analyzing complex mathematical model behavior by partial least squares regression-based multivariate metamodeling. WIREs Computational Statistics. 2014;6:440–475doi: 10.1002/wics.1325SunnåkerM, BusettoAG, NumminenE, CoranderJ, FollM, DessimozC. Approximate Bayesian Computation PLOS Computational Biology. 2013;9:1–10TavaréS, BaldingDJ, GriffithsRC, DonnellyP. Inferring coalescence times from DNA sequence data. Genetics. 1997;145:505–518doi: 10.1093/genetics/145.2.5059071603PapamakariosG, MurrayIFast ϵ-free Inference of Simulation Models with Bayesian Conditional Density Estimation NIPS. 2016. 1036–104426696337Lueckmann JM, Gonçalves PJ, Bassetto G,Öcal K, Nonnenmacher M, Macke JH Flexible Statistical Inference for Mechanistic Models of Neural Dynamics. Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017:1289-1299Papamakarios G, Sterratt DC, Murray I Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows. The 22nd International Conference on Artificial Intelligence and Statistics. 2018:837-848GerstnerW,KistlerWM, NaudR, PaninskiL. Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition. Cambridge University Press. 2014.SchwalgerT, DegerM, GerstnerW. Towards a theory of cortical columns: From spiking neurons to interacting neural populations of finite size. PLOS Computational Biology. 2017;13. doi: 10.1371/journal.pcbi.100550728422957BuzsákiG, AnastassiouC, KochC. The origin of extracellular fields and currents–EEG, ECoG, LFP and spikes. Nature Reviews Neuroscience. 2012;13(6):407–420. doi: 10.1038/nrn324122595786PesaranB, VinckM, EinevollGT, SirotaA, FriesP, SiegelM, et al. Investigating large-scale brain dynamics using field potential recordings: analysis and interpretation. Nature Neuroscience. 2018;21(7):903–919. doi: 10.1038/s41593-018-0171-829942039RasmussenCE, WilliamsCKI. Gaussian Processes for Machine LearningMIT Press. 2006.PapamakariosG, PavlakouT, MurrayI. Masked Autoregressive Flow for Density Estimation. 2017Hensman J, Fusi N, Lawrence ND. Gaussian Processes for Big Data. In Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence. 2013. 282–290Titsias M. Variational Learning of Inducing Variables in Sparse Gaussian Processes Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics. 2009;5:567–574Damianou A, Lawrence ND. Deep Gaussian Processes. Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics. 2013;31:207–215SalimbeniH, Deisenroth M Doubly Stochastic Variational Inference for Deep Gaussian Processes. Advances in Neural Information Processing Systems. 2017;30:4591–4602Bui T, Hernandez-Lobato D, Hernandez-Lobato J, Li Y, Turner R. Deep Gaussian Processes for Regression using Approximate Expectation Propagation. Proceedings of The 33rd International Conference on Machine Learning. 2016;48:1472–1481TøndelK, IndahlUG, GjuvslandABet al. Hierarchical Cluster-based Partial Least Squares Regression (HC-PLSR) is an efficient tool for metamodelling of nonlinear dynamic models. BMC Syst Biol5, 90 (2011). doi: 10.1186/1752-0509-5-9021627852TøndelK, VikJO, MartensH, IndahlUG, SmithN, OmholtSW. Hierarchical multivariate regression-based sensitivity analysis reveals complex parameter interaction patterns in dynamic models, Chemometrics and Intelligent Laboratory Systems. 2013;120:25–41doi: 10.1016/j.chemolab.2012.10.006McKayMD, BeckmanRJ, ConoverWJ. Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code. 1979;21:239–245Fardet T, Vennemo SB, Mitchell J, Mørk H, Graber S, Hahne J, et al. NEST 2.20.0. Zenodo. 2020.GewaltigM-O, DiesmannM. NEST (NEural Simulation Tool) Scholarpedia. 2007.SkaarJ-EW, StasikAJ, HagenE, NessTV, EinevollGT. Estimation of neural network model parameters from local field potentials (LFPs)PLOS Computational Biology. 2020;16doi: 10.1371/journal.pcbi.100772532155141WelchP. The use of fast Fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms. IEEE Transactions on audio and electroacoustics. 1967;15(2):70–73. doi: 10.1109/TAU.1967.1161901Jones E, Oliphant T, Peterson P, et al. SciPy: Open source scientific tools for Python; 2001–. Available from: http://www.scipy.org/.NordlieE, GewaltigMO, PlesserHE. Towards Reproducible Descriptions of Neuronal Network Models. PLoS Computational Biology. 2009;5(8):e1000456. doi: 10.1371/journal.pcbi.100045619662159HagenE, DahmenD, StavrinouML, LindénH, TetzlaffT, van AlbadaSJ, et al. Hybrid Scheme for Modeling Local Field Potentials from Point-Neuron Networks. Cerebral Cortex. 2016;26(12):4461–4496. doi: 10.1093/cercor/bhw23727797828Leibfried F, Dutordoir V, John S, Durrande N. A Tutorial on Sparse Gaussian Processes and Variational Inference. arXiv:2012.13962van der Wilk M, Dutordoir V, John ST, Artemev A, Adam V, Hensman J. A Framework for Interdomain and Multioutput Gaussian Processes. arXiv. 2020HaarioH, SaksmanE, TamminenJ. An Adaptive Metropolis Algorithm. Bernoulli, 7(2), 223–242. doi: 10.2307/3318737GelmanA, Rubin DB Inference from Iterative Simulation Using MultipleSequences. Statistical Science. 1992;7:457–472.Hagen E, Ness TV LFPy. https://github.com/LFPy/LFPyGardnerJR, PleissG, BindelD, WeinbergerKQ, WilsonAG. GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration. Advances in Neural Information Processing Systems. 2018. 7587–7597Tejero-CanteroA, BoeltsJ, DeistlerM, LueckmannJM, DurkanC, GonçalvesPJ, et al. sbi: A toolkit for simulation-based inferenceJournal of Open Source Software. 5. 52.LindnerB, DoironB, LongtinATheory of oscillatory firing induced by spatially correlated noise and delayed inhibitory feedbackPhys. Rev. E 6, 72 (2005) 16485986DegerM, SchwalgerT, NaudR, GerstnerWFluctuations and information filtering in coupled populations of spiking neurons with adaptationPhys. Rev. E 90, 6 (2014) 25615126BosH, DiesmannM, HeliasMIdentifying Anatomical Origins of Coexisting Oscillations in the Cortical Microcircuit. PLoS Comput Biol12(10): e1005132. doi: 10.1371/journal.pcbi.100513227736873TrousdaleJ, HuY, Shea-BrownE, JosićKImpact of Network Structure and Cellular Response on Spike Time Correlations. PLoS Comput Biol8(3): e1002408. doi: 10.1371/journal.pcbi.100240822457608ŁęskiS, LindénH, TetzlaffT, PettersenKH, EinevollGT. Frequency Dependence of Signal Power and Spatial Reach of the Local Field Potential. PLoS Computational Biology. 2013;9(7):e1003137. doi: 10.1371/journal.pcbi.100313723874180LindénH, PettersenKH, EinevollGT. Intrinsic dendritic filtering gives low-pass power spectra of local field potentials. Journal of computational neuroscience. 2010;29(3):423–444. doi: 10.1007/s10827-010-0245-420502952GelmanA, CarlinJB, SternHS, RubinDB. Bayesian Data Analysis. Chapman and Hall/CRC. 2004.Glöckler M, Deistler M, Macke JH. Variational methods for simulation-based inference arXiv. 2203.04176. 2022.PotjansTC, DiesmannM. The Cell-Type Specific Cortical Microcircuit: Relating Structure and Activity in a Full-Scale Spiking Network Model. Cerebral Cortex. 2014;24(3):785–806. doi: 10.1093/cercor/bhs35823203991SchmidtM, BakkerR, HilgetagCC, DiesmannM, van AlbadaSJ. Multi-scale account of the network structure of macaque visual cortex. Brain Structure and Function. 2018;223:1409–1435doi: 10.1007/s00429-017-1554-429143946SchmidtM, BakkerR, ShenK, BezginG, DiesmannM, van AlbadaSJ. A multi-scale layer-resolved spiking network model of resting-state dynamics in macaque visual cortical areas. (2018) PLoS Comput Biol14(10): e1006359. doi: 10.1371/journal.pcbi.100635930335761BillehYN, CaiB, GratiySL, DaiK, IyerR, GouwensNW, et al. Systematic Integration of Structural and Functional Data into Multi-scale Models of Mouse Primary Visual Cortex. Neuron. 2020;3:388–403doi: 10.1016/j.neuron.2020.01.0403214264810.1371/journal.pcbi.1011625.r001Decision Letter 0GrahamLyle J.Section EditorEngelTatianaGuest Editor2023Graham, EngelThis is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Submission Version0
19 Jan 2023
Dear Mr Skaar,
Thank you very much for submitting your manuscript "Metamodelling of a two-population spiking neural network" for consideration at PLOS Computational Biology.
As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. In light of the reviews (below this email), we would like to invite the resubmission of a significantly-revised version that takes into account the reviewers' comments.
We cannot make any decision about publication until we have seen the revised manuscript and your response to the reviewers' comments. Your revised manuscript is also likely to be sent to reviewers for further evaluation.
When you are ready to resubmit, please upload the following:
[1] A letter containing a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.
[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).
Important additional instructions are given below your reviewer comments.
Please prepare and submit your revised manuscript within 60 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email. Please note that revised manuscripts received after the 60-day due date may require evaluation and peer review similar to newly submitted manuscripts.
Thank you again for your submission. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.
Sincerely,
Tatiana Engel
Guest Editor
PLOS Computational Biology
Lyle Graham
Section Editor
PLOS Computational Biology
***********************
Reviewer's Responses to Questions
Comments to the Authors:
Please note here if the review is uploaded as an attachment.
Reviewer #1: Skaar et al. present a thorough application of state-of-the-art metamodeling techniques to a highly-relevant and well-studied neural circuit model in theoretical neuroscience: leaky-IF spiking E/I ("Brunel") model. They accurately point out, that metamodeling -- the practice of approximating f(x | theta) or p(x | theta) for expensive simulation-defined models and then leveraging those approximations for scientific/theoretical inquiry -- is underutilized in the field of computational neuroscience. They explain the computational efficiency benefits of Hierarchical Clustering-based Partial Least Squares over alternative metamodeling approximation methods like deep gaussian process regression (GPR) and compare their application to the metamodeling of power spectral density of the Brunel model given 10 parameters (synaptic strengths, voltage, capacitance and time constant parameters of the model). Finally, they show that when executing simulation-based inference with their metamodels (as likelihoods) using MCMC, the spiking activities and LFPs result in different manifolds of degeneracy in the neural circuit model. I commend the authors on the degree of detail and clarity in the manuscript.
The work presented is a rigorous application of metamodeling techniques to the Brunel model, and advocates for a well-reasoned position that metamodeling techniques offer a practical, methodologically sound alternative to popular approaches in simulation-based inference. However, the findings presented do not reach the publication standard at PLOS Comp Bio.
- a.) Without showing any methodological comparison to prominent techniques like those referenced in the paper for simulation-based inference (SBI) (ABC sampling, SNPE, SNLE, etc), the work offers no evidence of a computational benefit from using either metamodeling (either GPR or HCPLS). We're also not given a sense of how the method scales with parametric dimensionality (important for neuroscience). After all, the ultimate reason for approximating a high-fidelity simulation-based neural circuit model would be to use that approximation for inversion/parameter inference. Otherwise, the contribution is that simulation cost is reduced by coarsening the fidelity of the neural circuit model simulation, which is not stated as a fundamental aim of this manuscript.
- b.) If this manuscript lacks methodological comparisons to SBI, then one would expect the methodology presented to at least generate some foundational insights about the neural circuit model being analyzed. The insights generated regarding the Brunel model are minimal, and only elaborated upon in descriptive terms. It is certainly interesting that the two types of conditioning yield different manifolds of degeneracy, but what does that mean about the model -- or about the physiological neurons they model. Can you make a novel prediction from this methodologically-generated insight and confirm it with model simulations?
I would be happy to re-read a manuscript, which has been re-written around a firm conclusion regarding lines a or b (or both).
Regarding line of work a.) a good place to start would be to insert the metamodel as an approximation of the simulation process in ABC and MCMC, and demonstrate the speed/accuracy trade-off. In what regimes (e.g. parameter count, simulation senarios (dt, NE, NI) etc.) of SBI does metamodeling offer computationally efficient, high-fidelity model inversion. To push the envelope, you could insert the metamodel into the relevant places of SNLE/SNPE (as you allude to) and do a similar analysis.
Reviewer #2: In this work, Skaar and colleagues construct metamodels of the classic Brunel spiking neural network to do fast emulation (i.e., simulate with the metamodel instead of the real simulator) and parameter posterior inference. They perform several thousand simulations of the Brunel network of excitatory and inhibitory populations with 10 free parameters drawn from a Latin hypercube prior, compute the power spectrum of the population rate, as well as PSD of the LFP, which is simulated in a separate hybrid-scheme model using just the spike times. The set of 4000 pairs of {network parameter, PSD} datapoints are used as training data for two types of metamodels, GP regression and hierarchical cluster-based PLS regression, to learn a fast emulator of the forward simulation. They compare the accuracy of the metamodel output against a held-out test set of simulations and find good performance for both, while GPR outperforms HCPLSR. With GPR, the authors then do model inversion to infer network parameters given an observed PSD (again, simulation from a held-out test set). They find reasonable parameter recovery performance and discover parameter correlations in the posterior distribution, while the posterior predictive simulations match that of the original ground-truth simulations. They derive insights about parameters of the Brunel network based on the inference results, such as the scaling between synaptic strength and time constant, which was dependent on which representation of the data (i.e., PSD of the pop rate vs. LFP) one uses as the observation, which in general affected the inferred posterior distribution, e.g., variance of the 1D marginal distribution for some parameters.
In general, I personally find this work very interesting and valuable, and is a great addition to the thread of works the group produces towards modeling / inverting network parameters from observed data. As far as I can tell, the work is rigorously done, though some clarifications can be made regarding hyperparam choices, and also the high-level motivation for model choices (i.e., HCPLSR, expanded below). One of the main concern I have is regarding using the hybrid scheme for LFP simulation with fixed parameters. In addition, the manuscript is a bit dense at points, and some visual presentation can be improved. I have a lot of comments below, which are only loosely ranked from high- to low-level comments (my apologies), but they are all meant to improve the clarity of the presentation or strengthen the arguments made in the paper.
- just to clarify, for the LFP simulations, are the hybrid / biophysical network parameters fixed for all simulations, i.e., do all LFP simulations use, for example, a membrane time constant of 20 ms, as implied in Table 4? If so, isn’t this problematic that the spiking network simulations are run with variable membrane time constants drawn from the prior, while the LFP is computed with a different and fixed time constant for all simulations? This may also explain why some parameters are more constrained when conditioned on the LFP spectra. Given that the spiking simulations are run anyway, wouldn’t it be straightforward to just compute the LFPs based on another approximation scheme (e.g., Mazzoni et al. 2015), which would presumably be a much better approximation of the LFP for each individual network. I think this is an important point to discuss, and possibly warrants further experiments, at least for a subset of the simulations, to show that the “true” LFP computed from currents in the spiking network (which have variable parameter values) match the LFP from the hybrid scheme (which has fixed parameter values), or that this does not influence posterior inference. Related to this, are PSDs from all 6 LFP channels used for inference? If not, the computational overhead of computing 1 LFP should be minimal?
- one thing that was not clear to me at all is the motivation behind using HCPLSR in this work. It is limited in expressivity while introducing computational overhead in the clustering step (as opposed to just doing nonlinear regression), does not allow model inversion, which is a main feature of the GP metamodel, and in general performs worse than the GP (though I suppose the latter point was only discovered by actually running the experiment). Given the group’s previous work (i.e., Skaar, Stasik et al 2020), I would have expected this comparison to be at least against a feedforward CNN model (or some kind of nonlinear regression model in general), which is probably equally fast for emulation time on modern computing hardware, but without loss of expressivity. In general, I think the results regarding HCPLSR is good to know, but does not at all add to the overall story, and can probably be moved to supplemental material given that the methods section is already quite technically challenging (which would also save Fig 2 to highlight a more important result). Alternatively, the authors can provide more motivation for trying HCPLSR (instead of a variety of other equivalent or superior regression techniques) , especially since they do not take advantage of the low dimensional representation found in PLS, as it would be interesting to see if the loadings for different clusters would correspond to qualitatively different regimes of network dynamics, etc.
- related to the previous point, it was emphasized in several places that the low dimensional representation in GP is a feature that affords more interpretability, but this was never taken advantage of in downstream analyses. I’m not saying this has to be added to this paper (though it would be very interesting to see!), but without actually looking into the GP latents, it undermines the key motivation for using it, which opens up the question of why not use a more expressive model. Similarly, given the previous results in Skaar, Stasik et al, it would have been natural to use a probabilistic model like mixture density network instead of a CNN to directly learn the posterior over parameters, with the advantage of bypassing the MCMC step as well as eliminating the utility of the GP’s smoothness. This is a rather general comment that can be addressed in writing of the introduction or discussion, or by performing additional analyses (of GP latents) or testing additional models (a more flexible replacement for the GP, or MDN for direct posterior estimation, etc.), and I leave it to the authors to address it in how they see fit.
- L121-131 implies that the prior bounds were restricted to eliminate parameter regions that result in the SR state, but wouldn’t that also exclude parameters that could generate AI dynamics? In the original Brunel phase diagram, this would correspond to the fact that the phase transition between SR and AI state has a slant in the 2D parameter space, while restricting the prior is akin to drawing a proper rectangle around only a part of the AI regime, and I would imagine this effect (i.e., boundaries not parallel to the coordinate axes in the parameter space) would only be exacerbated in the 10-D model here. A discussion of this would be warranted, esp since the authors attempt to capture a wide range of the model’s parameter space with the metamodel. A simple alternative would be to simulate from a broader prior, and post hoc throw out simulations in unwanted regimes (which the authors already do anyway).
- regarding the references to SBI methods (in the intro and discussion, L633-640), and in particular, SNPE/SNLE: while it’s true that the sequential version of these algorithms target posteriors for a specific observation, when run just for a single round, they have the exact same properties as the metamodels proposed here in that they are accurate over the full domain of the mechanistic model. In fact, SNLE run for a single round is exactly analogous to the GPR approach here, where the learned emulator/metamodel can be sampled from to produce fast simulations, but just replacing the gaussian process with another conditional density estimator, often of the neural network variety (e.g., normalizing flow). This fact does not diminish the novelty or how interesting the current work is, but is a point that should be clarified (and perhaps practical differences between estimators can then be further discussed, e.g., computational cost of GP vs. flows). Also, on L688: the metamodels plugged into SBI or ABC still produces a posterior that has to be MCMC sampled (as the authors do here), so I’m not sure what this statement is meant to convey?
- as a reader, the methodological description of GPR was extremely dense to read, while the technical details (and the current Figure 1) don’t contribute to the overall goal of the paper, especially since there’s no comparison of performance as a function of GP hyperparameters. I think most of this can be moved to supplemental materials, to make it easier for a non-technical reader to digest, though this is just my opinion. Similarly, I think the current figure 3 serves as a much better figure 1, as it provides an overview of the study, and should be highlighted as early as possible, instead of technical details about prior samples for the GP or schematic for PLSR. On that note, given the observation that the metamodel output PSD differ in smoothness from the actual simulations, maybe it’s worth exploring different length scales of the GP, as well as discussing the specific choice of hyperparameters (e.g., 10 and 14 GPs in layer 1 and 2, why?)
- some discussions of applying this approach to real neural data (and the limitations) may be of interest to a broader neuroscience audience
- does the latin hypercube prior outperform a simple uniform prior? Is there a practical or theoretical difference? This is just for my own curiosity.
- I think the first paragraph of section 3 (L327-334) is a fantastic summary of the work, and (some version of this) can be restated earlier in the paper as it really helped me as a reader to orient myself in the paper.
- are raw PSDs or log PSDs used as input for all the regression models (and at evaluation)? I’d imagine using the log PSD would help for the regression tasks given the 1/f.
- Fig 5D (and others): when evaluating metamodel output accuracy, I think it would be useful to scale the error to the range of the simulations from the prior samples (i.e., training set), or was this already done?
- I think some figures can be labeled more informatively:
- Fig 5A&B can have subtitles, pop rate and LFP PSD, respectively, same for the two rows of 5D; the lines/shaded area in 5A&B are in general very difficult to see; 5C x-axis can be labeled test set index, and perhaps could be plotted as a cumulative distribution (so the transpose of the current graph) to make the same point but with a more standard plot;
- Fig 8A can have subtitles denoting spiking and LFP (and the two columns in Fig 8 in general), and red dot/line can be labeled as ground-truth in the figure; it’s a bit confusing that Fig8B breaks the 2-column pattern (but not sure if much can be done about it); lines in 8C are very difficult to see, and I think the black lines entirely block the shaded orange? Is it an issue that the two columns of 8B look almost identical? Is there some straightforward explanation for this? Typo in caption for 8D (“orange dashed line in C”)
- I’m having a bit of a hard time understanding Fig 8D: is this the max distance of the metamodel mean from the posterior predictive simulations? Shouldn’t this be computed as a distance of the ground-truth simulation (i.e. the observation used for conditioning) from the posterior predictive simulations?
- L460: full 10D pairplot would be nice to see in supplemental
- L515-517: I think it would be clearer to use “conditioned on”, i.e., “…the coefficients for the samples from the posterior distribution conditioned on the population spiking activity,….and coefficients conditioned on the LFP”
Richard Gao, PhD
University of Tuebingen
Reviewer #3: The authors fit two statistical models ("metamodels"), to the Brunel network of excitatory and inhibitory LIF neurons. Some of the model parameters are treated as unknown and these unknown parameters are the input to the metamodel. The output of the metamodel are estimations of the power spectra of the population activities or of the LFP's. The metamodels are trained using supervised learning for many parameter sets from some domain of parameter space. The trained metamodels are tested on randomly sampled parameter combinations that lead to mostly asynchronous irregular spiking activity (parameters corresponding to (strong) synchronous activity are not considered both in the training and testing phase). The authors find an excellent performance of the metamodels both for the forward prediction of power spectra and for statistical inference of parameters given power spectrum estimates (inverse modeling). The parameter inference yields some interesting insights regarding the type of parameters and parameter combinations that can be well inferred, and the benefits of the inference on LFP data vs. population activity data (e.g. LFP data are advantageous for inference of synaptic connectivity). I found the paper important and interesting: the metamodel is a highly efficient model for the forward prediction of power spectra of LFPs or population activities given biophysically interpretable parameters, and inversely, the metamodel provides estimations of unknown biophysical parameters given power spectrum data. Both aspects are of high importance in neuroscience. The metamodelling framework is interesting because it combines the advantages of statistical models (efficiency, parameter inference) with the advantages of mechanistic network models (interpretability). The paper is scientifically sound and largely well written. The metamodel applies to state-of-the-art neural network models and LFP models in computational neuroscience and I expect the method to be highly useful at the interface of large-scale cortical recordings and network models of the corresponding circuits. The description of the method should be improved. Overall, I recommend publication in PLoS Computational Biology after the minor issues given below have been addressed.
Minor issues
============
1) The introduction of the concept of metamodels is rather abstract/general and would benefit from a more concrete description (alongside the general concept). Figure 3 is very nice and already quite helpful in this respect, but only comes in Sec.3. For example, it should be made clear already in the introduction that, in the present paper, the input of the metamodel are some parameters of the network model and the output of the metamodel is some statistics (e.g. power spectra or distribution over power spectra) of some functions of network variables (e.g. LFP or population activities). It could be stressed that the metamodel does not generate realizations of population activities or LFPs, in fact there is no "time" in the metamodel, in contrast to the network model and other statistical models.
2) Similar to point 1, I found the Methods part on Gaussian process regression rather abstract and incomprehensible to non-machine-learning experts. In Sec. 2.3, the quantities X, x and bold x seem to relate to the input, whereas the bold x has been defined as the output in the introduction (As I understand the bold x would be an estimate of the power spectra). Please, make sure that unique notations are used and all quantities are properly defined (x (not bold) does not seem to be defined, X should be defined more precisely, e.g. what is the dimension of the matrix, how is it constructed from n inputs (bold x?) of dimension d?). What is d in this paper? What is the "underlying function f(x)"? In line 200, if bold x denotes a vector (let's call it \\vec{x}), (\\vec{x}-\\vec{x}')^2 is not clearly defined. Do you mean the norm squared (dot product)?
3) What exactly is the output of the GPR? If it is the likelihood p(\\vec{x}|\\theta), how is it represented or computed (where is the equation for it)?
4) In line 269, the unnormalized posterior distribution is given in terms of the likelihood, which has been computed by the GPR. Why is it necessary to sample from this distribution? Isn't the unnormalized posterior distribution sufficient (e.g. to get the mean and standard deviation in the plots of this paper)?
5) Fig.5A,B: The 10 samples of the GPR (orange lines) are barely visible. On the other hand, I find it is not necessary and rather confusing to plot them. The output of the GPR is the distribution of the power spectrum, and hence it is sufficient to show the mean and the orange shaded area (two standard deviations). Therefore, I suggest to omit the orange lines, which will also lead to a better vivibility of the black lines. Furthermore, the authors write in line 385: "the smoothness of the power spectra is underestimated for lower frequencies, as can be observed by comparing the individual orange and black lines". I guess the mean power spectrum, which should be a much smoother function, is more relevant than individual samples because the mean power spectrum will provide an estimate of a perfectly averaged power spectrum in the network model (corresponding the averaging over infinitely many Hann windows in the Welch periodogram of an infinitely long simulation).
6) For me the "power spectrum" is synonymous with power spectral density (PSD), which is defined by dividing by the window length T: PSD(f)=<[Psi]^*(f)[Psi](f)>/T (otherwise the power spectrum would scale with T). Why is the power spectrum defined without division by T? The scipy.signal.welch also provides the PSD that is rescaled by the window size. The PSD has also the advantage that the firing rate of neurons can be easily read-off from the high-frequency limit of the PSD of population activities (apart from a factor N). The PSD is not dimensionless, e.g. the PSD of population rates has units of 1/s, whereas without division by T it is dimensionless.
7) What is the unit of the LFP in this paper? The units of the corresponding power spectra seem to be missing in the plots unless the LFP has units of 1/time and hence its Fourier transform is unitless.
8) It is unclear how the method works outside the asynchronous irregular regime and where it fails. Could the method be used also for the synchronous regime?
9) Are the rates nu in Fig.4 normalized by N? According to the definition of nu they are not, but in this case the single neuron rate would be nu/N which would result in extremely small rates. Furthermore, what exactly is nu here, is it a weighted sum of nu_E and nu_I? If nu is normalized then simply nu=nu_E=nu_I, but this is not the case if nu is not normalized.
10) There seems to be a numerical artifact in power spectrum at the highest frequency (strange decrease). This decrease should not be there because the PSD of the population activities saturates to a non-zero value (the saturation is already visible in the plots).
11) In the asynchronous irregular regime, linear-response theory (especially the theory based on Lindner Phys Rev E 72, 061919, 2005) usually provides highly accurate analytical approximations of the PSDs (see e.g. Trousdale et al. PLoS Comput Biol. 2012, Deger et al. Phys. Rev E 2014, and the simpler but less accurate theories of Brunel J. Comput Neurosci. 2000 and Bos, Helias et al. PLoS Comput Biol. 2016). These analytical formulas should provide an efficient alternative to metamodels, which should be discussed in the paper.
Typos
=====
-caption Fig.5: set, . -> set.
-caption Fig.5: errors are shown in standard deviations
-Table 1: shaped-shaped -> alpha-shaped?
-line 433: an an
-line 451: membrane potential tau_m -> membrane time constant tau_m
-caption Fig.8, last sentence: there should be two times a reference to panel C not B
-caption Fig.5: errors are shown in standard deviations -> maybe a bit better: errors are shown in units of the standard deviations
**********
Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?
The PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.
Reviewer #1: Yes
Reviewer #2: Yes
Reviewer #3: Yes
**********
PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.
If you choose “no”, your identity will remain anonymous but your review may still be made public.
Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.
Reviewer #1: Yes: Sean Bittner
Reviewer #2: Yes: Richard Gao
Reviewer #3: No
Figure Files:
While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.
Data Requirements:
Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example in PLOS Biology see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.
Reproducibility:
To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols
10.1371/journal.pcbi.1011625.r002Author response to Decision Letter 0Submission Version1
10.1371/journal.pcbi.1011625.r003Decision Letter 1GrahamLyle J.Section EditorEngelTatianaGuest Editor2023Graham, EngelThis is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Submission Version1
27 Aug 2023
Dear Mr Skaar,
Thank you very much for submitting your manuscript "Metamodelling of a two-population spiking neural network" for consideration at PLOS Computational Biology. As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. The reviewers appreciated the attention to an important topic. Based on the reviews, we are likely to accept this manuscript for publication, providing that you address the concerns raised by the reviewer in the revised manuscript. In particular, state clearly what advances your work provides and what is the significance of your study for the computational neuroscience field.
Please prepare and submit your revised manuscript within 30 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email.
When you are ready to resubmit, please upload the following:
[1] A letter containing a detailed list of your responses to all review comments, and a description of the changes you have made in the manuscript. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out
[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).
Important additional instructions are given below your reviewer comments.
Thank you again for your submission to our journal. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.
Sincerely,
Tatiana Engel
Guest Editor
PLOS Computational Biology
Lyle Graham
Section Editor
PLOS Computational Biology
***********************
A link appears below if there are any accompanying review attachments. If you believe any reviews to be missing, please contact ploscompbiol@plos.org immediately:
Reviewer's Responses to Questions
Comments to the Authors:
Please note here if the review is uploaded as an attachment.
Reviewer #1: Recommendation
The authors have made diligent efforts to account for feedback in the previous round, which they should be commended for. This manuscript now very clearly lays out the role of metamodeling with respect to state-of-the-art techniques used in computational neuroscience for SBI - namely the normalizing flows (MAFs) used to estimate likelihoods (forward model) for inference as in SNLE. SNPE is primarily used in favor of SNLE, so I find this paper’s focus on the metamodel (estimation of the likelihood/forward model) to be a refreshing re-examination of the utility of such approaches for models with expensive simulations with intricate emergent phenomena like the Brunel model. However, I struggle to identify an aspect of the advancements/insights laid out in this paper that meet the publication criteria of PLoS Computational Biology. I recommend rejecting this paper for the reasons elaborated below.
Novelty/innovation/insight
In the first review, I stated my opinion that this manuscript could meet the publication criteria for this journal along the lines of innovation/insight if they did either one of two things: a.) demonstrate benefits of metamodeling with respect to SoTA approaches from of SBI or b.) generate foundational insights about the Brunel model using these techniques. Showing that a metamodel can be used to approximate the Brunel model to a reasonable degree (which they have definitely done) is not enough. While they have made considerable progress towards each of these aims, they don’t present enough evidence of a.) methodological advancement or enough analyses to yield b.) foundational insight generated with this methodology.
a.) methodological advancement
Since the original manuscript had a focus on using metamodels towards parameter inference, the authors followed recommendations from Reviewers 1 and 2 to explore the methodological tradeoffs between metamodeling and methods in SBI. Since metamodeling (estimation of the likelihood) is a process within SNLE, the authors accurately point out that the process undertaken in the manuscript is like running SNLE for 1 round. This underscores the fact that metamodeling isn’t novel in computational neuroscience, however it is certainly underutilized, and they will be making a case for its use in this paper.
The methodological comparisons in the latest manuscript focus on the approximating model of the Brunel model - the type of metamodel. Indeed they show some evidence that deep gaussian processes are a better approximation than MAFs. for the Brunel model, and also that HCPLSR lacks the expressivity necessary for this problem (I appreciate the inclusion of this in the manuscript knowing how great a focus this was in the original version). Even if this claim that DGPR was a more effective method of metamodeling than MAF for the Brunel model was firmly established (which I'm not entirely sure it is, see section Rigor/completeness), this relative advantage of DGPR would need to be demonstrated for at least two neural circuit models (not just one) to establish that it is in fact a better practice for the field moving forward – thus constituting an advancement worthy of publication.
After this first round of revision, I would have expected the authors to take a more focused approach on simulation count/computational efficiency – what is the trade-off space (simulation count and/or computational cost vs inference/estimation quality) of HCPLSR or DGPR metamodels vs established methods in SBI.
b.) foundational insight
The authors certainly made a more concerted effort to expand on the meaning of the posteriors inferred via the metamodel, however I don’t believe any concrete insights have been revealed in this manuscript. There are certainly paragraphs in Section 3.4 and Discussion along these lines, but they don’t rise beyond descriptions of the visualized/quantified correlations in the posterior, or 1D marginal variances, with some prose about how these parameters relate in an information theoretic context.
To meet publication criteria, we'd want to gain some interpretable insight from inferred posteriors - stipulate a hypothesis(es) from this insight - and then confirm them in simulation. That is not done here.
The fact that conditioning on LFPs vs spiking activity yields less posterior variance is nice to see, but I’m not sure this is surprising or has generated any new understanding of the Brunel model and its function.
Rigor/completeness
Some of this manuscript is well written and mature, but I found several parts unclear.
- Figure 3 - No mention or explanation is given for why MAF is missing from the right side of the figure.
- How many parameters are used in ideal DGPR model vs ideal MAF model? Can this explain the overfitting mentioned in the main text?
- Figure 4 - train and test labels are switched (otherwise something is very wrong).
- Are these training errors evaluated on the progressively increasing training set, or the complete dataset irrespective of the amount used in training?
- How many simulations were used to build the metamodels used in the following analyses? 1000 because its the elbow you designated?
- What’s the relative computational cost (training time per epoch etc.) of fitting these metamodels to simulation cost? I don’t have an idea of what the true bottleneck in this parameter inference problem is given the methods you’re exploring.
- Figure 5 - Which metamodel was used in this figure? I assume DGPR, but it’s not explicitly stated.
- Figure 5B - no explanation of what x and o are in this figure.
- Figure 5D should have x-axis labels indicating the difference of left vs right.
- Figure 6
- I agree that Figure 6D using the mean (or median) error of the posterior predictive wrt to the conditioning simulation is an appropriate evaluation method for the inference techniques shown here, but I disagree with the interpretation of Figure 6B, and the statement L466-469 about the posterior predictive variance.
- The posterior predictive distribution will reflect the uncertainty of the approximate posterior. A method that underestimates the posterior variance, will have lower posterior predictive variance, and that certainly doesn’t make it a better method than one accurately capturing the posterior variance.
Reviewer #2: Thanks to the authors for constructively engaging with my comments. The new experiments, especially the comparison with MAF, are quite interesting and now an additional contribution the paper makes. I'm happy with the manuscript as it is and very excited to see it in publication.
Reviewer #3: The authors have satisfactorily addressed all my comments.
**********
Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?
The PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.
Reviewer #1: Yes
Reviewer #2: Yes
Reviewer #3: None
**********
PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.
If you choose “no”, your identity will remain anonymous but your review may still be made public.
Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.
Reviewer #1: Yes: Sean Bittner
Reviewer #2: Yes: Richard Gao
Reviewer #3: Yes: Tilo Schwalger
Figure Files:
While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.
Data Requirements:
Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example in PLOS Biology see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.
Reproducibility:
To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols
References:
Review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript.
If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.
10.1371/journal.pcbi.1011625.r004Author response to Decision Letter 1Submission Version2
4 Oct 2023
Submitted filename: metamodel_response2.pdf
10.1371/journal.pcbi.1011625.r005Decision Letter 2GrahamLyle J.Section EditorEngelTatianaGuest Editor2023Graham, EngelThis is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Submission Version2
23 Oct 2023
Dear Mr Skaar,
We are pleased to inform you that your manuscript 'Metamodelling of a two-population spiking neural network' has been provisionally accepted for publication in PLOS Computational Biology.
Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.
Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.
IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.
Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us now if you or your institution is planning to press release the article. All press must be co-ordinated with PLOS.
Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Computational Biology.
Please note here if the review is uploaded as an attachment.
Reviewer #1: My concerns from the previous round of review have been addressed, and I'm happy to endorse the acceptance of this manuscript. This is a great paper that will add an important complementary perspective to the emerging field of methodological techniques for handling expensive simulation-based models in theoretical neuroscience.
**********
Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?
The PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.
Reviewer #1: None
**********
PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.
If you choose “no”, your identity will remain anonymous but your review may still be made public.
Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.
Reviewer #1: Yes: Sean Bittner
10.1371/journal.pcbi.1011625.r006Acceptance letterGrahamLyle J.Section EditorEngelTatianaGuest Editor2023Graham, EngelThis is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
3 Nov 2023
PCOMPBIOL-D-22-01359R2
Metamodelling of a two-population spiking neural network
Dear Dr Skaar,
I am pleased to inform you that your manuscript has been formally accepted for publication in PLOS Computational Biology. Your manuscript is now with our production department and you will be notified of the publication date in due course.
The corresponding author will soon be receiving a typeset proof for review, to ensure errors have not been introduced during production. Please review the PDF proof of your manuscript carefully, as this is the last chance to correct any errors. Please note that major changes, or those which affect the scientific understanding of the work, will likely cause delays to the publication date of your manuscript.
Soon after your final files are uploaded, unless you have opted out, the early version of your manuscript will be published online. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.
Thank you again for supporting PLOS Computational Biology and open-access publishing. We are looking forward to publishing your work!