Here, we turned to a previously devised meta learning pipeline (Fig 1): We used CMA-ES [40] to iteratively improve upon an initial plasticity rule, parameterized with plasticity parameters θ. At every meta iteration, CMA-ES generated a collection of plasticity rules which were evaluated in individual spiking networks according to their fitness (loss function). CMA-ES then updated its internal model of parameter interdependencies and its guess for the best rule, i.e., the mean and covariance matrix of a Gaussian distribution in plasticity parameter space θ (see Methods). To help with the stability of the meta optimization, the initial plasticity rule was chosen such that it elicited no or few weight changes in the network.

We began with a simple search space for plasticity rules encompassing first-order (i.e., containing no square-terms, Methods) spike-timing-dependent plasticity (STDP) rules, which we refer to as the “small polynomial search space.” For this search space, similar to previous work [31], the weight from presynaptic neuron i to postsynaptic neuron j, wij(t), evolved as

where Si(t) is the spike train of neuron i, xi(t) is a low pass filters of the spike train of the pre-synaptic neuron i with time constant τpre, and yj(t) is a low pass filters of the spike train of the post-synaptic neuron j with time constant τpost. The spike train is defined as Si(t)=∑ ⁡kδ(t−tki), where tki is the time of the k-th spike of neuron i and δ is the Dirac delta. In total, this search space comprised six tunable plasticity parameters: θ=[α,β,γ,κ,τpre,τpost].

Previous work uncovered inhibitory to excitatory (I-to-E) rules that would enforce network stability in a feedforward setting [31]. To discover such rules in recurrent networks, we meta-learned I-to-E plasticity rules that enforced a target population firing rate of 10 Hz, quantified with a loss function on the network activity (see Methods, Fig 2A, “stability task”).

The ES converged to a rule with low loss values such that network activity remained stable at 10 Hz for twice the longest possible training duration (2 min, Fig 2C). Visualizing the meta-learned rule with a classic pre-post protocol (pairs of pre-/postsynaptic spikes with various delays, Fig 2B, left) revealed that it did not correspond to any known rules [44]. The rule resembled the symmetric inhibitory rules found in theory and experiments [10,24], but shifted downward such that all pairs of pre-post spikes elicited depression. Note that since in our simulations the number of pre- and post- synaptic spikes (the pre- and postsynaptic firing rates) do not have to be the same, this graph does not mean that the meta learned rule has no potentiation regions (Fig 2B). In the network simulation, the I-to-E weights do settle to intermediate values as a result of both potentiation and depression (Fig 2C, bottom).

Next, we used the same stability constraint to discover plasticity rules in other synapse types (E-to-E, E-to-I, or I-to-I, individually, Fig 2D–2F). In two scenarios—E-to-E and E-to-I—the optimization was able to find rules that established the target firing rate (Fig 2C and 2D), albeit not as effectively as with I-to-E plasticity, as evidenced by relatively high losses at the end of training.

Since all successful rules differed from previously observed rules (Fig 2B), we wanted to understand the inter-dependencies of the learned plasticity parameters. We thus plotted the covariance matrix between plasticity parameters as the meta learned rules emerged during the optimization. The covariance matrix is updated alongside the optimal rule in CMA-ES and contains information about the loss landscape, albeit heuristically. Our analysis revealed that the main structure in the rules was strongly anti-correlated non-Hebbian plasticity parameters, i.e., when α is increased β is likely decreased and vice versa (Fig 3), consistent with mean-field theory (see Mean-field analysis). For I-to-I plasticity, no plasticity rule could be found that improved meaningfully upon the initial (no plasticity) rule (Fig 3D), although we note that the absence of proof is not proof of the absence of a solution.

Having established that rules from the small polynomial search space could stabilize recurrent spiking networks, we turned to a more complex, memory-related network function: familiarity detection. This ubiquitous form of memory [45,46] has already been the target of meta learning in rate networks [33], and has been shown to emerge in recurrent spiking networks with finely orchestrated, hand-tuned co-active synaptic plasticity rules [12,13,16].

To meta learn plasticity rules that would produce familiarity detection, we designed a protocol in which we stimulated a recurrent spiking network with the same stimulus multiple times, which we define as the “familiar” stimulus. After extensive stimulation with this familiar stimulus intended to induce strong changes in synapses, we compared network responses with a non-overlapping second stimulus that we define as “novel”. Successfully learning the familiar stimulus meant responding to it with a higher firing-rate compared to the novel stimulus after learning. To accomplish such a learning, we designed a loss function that constrained plasticity rules such that the network would produce high firing rates to familiar, and low firing rates to novel stimuli (see Methods and S1 Fig).

We started by optimizing only I-to-E plasticity while all other rules were inactive. We refer to such scenarios as single-active rules. The I-to-E plasticity rule belonged to the small polynomial search space. Our ES algorithm converged to a rule that achieved low loss values (Fig 4C) and produced networks that responded more strongly to familiar than to novel stimuli. As a control, a network undergoing the same task without any plasticity was unable to exhibit asymmetric responses for novel versus familiar stimuli, confirming that the learned plasticity rule was responsible for this acquired behavior (S5 Fig). When we probed the plasticity rule with classical pre/post protocols, we found that the rule did not closely resemble any of the experimentally reported temporal relationships [44]. Notably, familiarity detection was achieved here with a single active I-to-E plasticity rule, contrary to previous work in which memory-related functions were always achieved in tandem with E-to-E plasticity [12,13,16,47].

Next, we focused on E-to-E plasticity in isolation, using the same loss function. The ES found an E-to-E plasticity rule that was able to solve the familiarity task (Fig 4D), and its pre-post protocol resembled previously reported classical asymmetric STDP rules [21,22]. We also considered the other two synapse types (E-to-I, or I-to-I, individually active), but ES could not find satisfying solutions for either (Fig 4E and 4F). Note again, that this result does not prove that no solutions exist within the small polynomial search space for E-to-I or I-to-I rules.

The covariance matrices corresponding to the optimizations for the familiarity task were somewhat similar to the ones obtained on the stability task (α-β anti-correlations for Figs 3A, 3B and 5A, 5B; β-γ and β-κ anti-correlations as well as γ-κ correlation for Figs 3C and 5C; α-β, α-γ and α-κ anti-correlations for Figs 3D and 5D), suggesting similar structure in the relationships of the learned rules for both cases (strong anti-correlation between non-Hebbian parameters, Fig 5).

Inspired by previous work proposing co-active E-to-E and I-to-E rules for memory formation in spiking networks [12,13,16], we set out to meta learn jointly the E-to-E and the I-to-E plasticity rules for the familiarity detection task mentioned above (Fig 6). Since either rule (I-to-E or E-to-E) was shown to be able to solve this task individually, ES should succeed in finding at least one solution. As expected, ES converged on a solution that satisfied all constraints and displayed the hallmarks of cortical network dynamics. The learned E-to-E rule was similar to the above-described rule acting in isolation (Fig 4C), displaying a very similar shape as experimentally observed E-to-E rules [3,21,22]. The newly learned I-to-E rule, on the other hand, differed from previous experimental results [7,10,23,48] and also from the previous optimization above, showing an inverse, asymmetric “Bavarian Hat” with a tuft of potentiation. The covariance matrix revealed anti-correlation between non-Hebbian terms of each rule, and

some interactions between parameters of both rules, namely a inverse relationship between non-Hebbian parameters (Fig 6, bottom row).

We also tried other combinations of co-active rules on the same task (E-to-E and I-to-I, E-to-I and I-to-E, as well as E-to-E and E-to-I). In all cases, ES converged to solutions that elicited higher responses to familiar than to novel stimuli, but the network dynamics were biologically implausible (Fig 6), suggesting that E-to-E and I-to-E were the most useful synapse-type for the considered function. Alternatively, it could be that the plasticity rules we used were not flexible or broad enough to express biologically plausible solutions.

To capture more complex plasticity mechanisms we constructed two higher dimensional and more expressive plasticity rule parameterizations, i.e., (1) a polynomial with additional dependencies and (2) a neural network parameterization (“MLP”, a feedforward network that determines the synaptic changes of the recurrent spiking network, see Methods). We benchmarked these two new parameterizations on the same stability task as for the small search space (Fig 2).

First, we expanded the small polynomial search space, adding synaptic variables that contributed to weight updates, such as additional synaptic traces (triplets rules [8], bursts [17], voltage dependence [9], codependent plasticity [20] and weight dependence [5]),

where Si(t) and Sj(t) are the spike times of pre- and postsynaptic neurons, respectively, fpre and fpost are polynomial functions with the following synaptic variables: xi,τlong(t) and xi,τshort(t) are low pass filters of the spike train of the pre-synaptic neuron i with time constants τshort=10 ms and τlong=100 ms (and similarly for post-synaptic neuron j); Cj,E(t) and Cj,I(t) are co-dependent terms representing the activity of neighboring synapses, i.e., low-pass filtered with fixed time constants τCE=10 ms and τCI=100 ms, as in previous work [20]; and ⟨Vj(t)⟩ is the low-pass filtered membrane potential, with a time-constant τ⟨V⟩=100 ms [9]. We assumed separability of the synaptic variables, i.e., that the synaptic variables contributed independently to weight updates, which allowed us to incorporate additional dependencies to the weight updates without bloating the total number of plasticity parameters. Meta-learning I-to-E rules on the stability task in the larger polynomial search space resulted in solutions that achieved low losses (Fig 7A). However, when simulating the learned rule for longer than during training, we observed that the rule did not generalize as well as the rules from Fig 2A, with the excitatory activity showing large oscillations around the desired target of 10 Hz (Fig 7A). Additionally, some I-to-E weights reached the maximum weight (10, Fig 7A). The covariance matrix revealed a much sparser structure. The shape of the rule was ambiguous under the pre-post protocol, as it did not constrain the values of the additional synaptic variables.

Motivated by previous work proposing co-active rules that support memory formation and recall in spiking networks [12,13], we considered the same familiarity task and loss function as above, with E-to-E and I-to-E synapses plastic parameterized with the bigger polynomial search space (Fig 7G). Once again, we could meta learn rules that solved the task (Fig 7G). We verified that the learned co-active rules were able to elicit different population responses to the familiar and novel stimuli (Fig 7F). However, other aspects of network activity that were not constrained by the loss function were unrealistic. For example, most neurons in the network were either silent or fired at unrealistic rates with in highly regular patterns (Fig 7F and 7G). The E-to-E connections mostly converged to zero weights (Fig 7F and 7G). The I-to-E connections underwent rapid switching between 0 and the maximum allowed weight at the millisecond scale, resulting in a bimodal distribution (Fig 7F and 7G).

Finally, we considered a neural-network-based search space for plasticity rules, in which the same synaptic variables as in the big polynomial were combined using a (generalized) multilayer perceptron (MLP, Fig 7B) [49]. Plastic synapses from this search-space underwent spike-triggered updates such that:

using similar notations as for the big polynomial search-space (see More complex plasticity rules).

Similarly to the case with a single connection being plasticity, the MLP used to model synaptic changes taking place in the recurrent spiking network was a feedforward network with two hidden layers (50 and 4 hidden units, see Methods), in which only the final layer weights and bias were tunable, keeping all other layers fixed [49]. This design choice effectively decoupled the number of synaptic variables involved in the rule and the number of plasticity parameters to optimize, thus allowing for potentially highly non-linear dependencies on the synaptic variables while keeping the dimensionality of the search space as low as desired for the evolutionary strategy. This search space comprised a total of 11 parameters: 5 parameters for updates triggered by presynaptic spikes, 5 for postsynaptic updates, and a common learning rate (Fig 7E).

The evolutionary strategy was able to find plasticity rules that established the target firing rate in the MLP search space (Fig 7B). Similar to the big polynomial case, however, the learned rule was not as robust when tested on longer-than-training durations. In addition, the rule elicited biologically implausible network behaviours, for example, weights reaching the maximum allowed value and synchronous spiking patterns (Fig 7B).

Overall, all three parameterizations—small polynomial, big polynomial and MLP—led to rules that solved the task as it was quantified by the loss function. However, the meta learned rules from larger plasticity search spaces did not generalize as well as the simpler rules, and the resulting plastic networks exhibited implausible behaviors, such as synchronous regular firing patterns and bimodal distributions. In our hands, designing a loss function that constrained task performance alone was not sufficient to ensure that biologically relevant plasticity rules emerged.

Concerned by the impact of potential degeneracy on the rules proposed in this study, we set to test how reliable our rule predictions were on the familiarity task. Running two (intrinsically stochastic) evolutionary searches from the same starting point on the familiarity task with an I-to-E small polynomial rule converged to two plasticity rules with dissimilar pre-post protocols (Fig 8A). This shows, in agreement with previous work in rate networks [41], that at least two and probably many plasticity rules from the same search space can solve this task. This conclusion is not unique to I-to-E plasticity (Fig 8B). However, even though the plasticity rules differed across optimizations, the relationship between plasticity parameters appeared to be conserved. For example, we observed strong anti-correlations between non-Hebbian parameters in all simulations, as shown by the covariance matrices (Fig 8).

We considered recurrent networks of excitatory and inhibitory, conductance-based leaky-integrate-and-fire neurons. Two types of networks were implemented, emulating either the networks used by Vogels, Sprekeler et al. [10] or by Zenke et al. [13].

In this study, we considered three parameterizations for plasticity rules with various levels of complexity and expressivity.

Within the small polynomial search space, we performed mean-field analysis on the I-E connections to link the plasticity parameters and the population firing rates at steady state rexc∗,rinh∗, as done in previous work [10,15,31,49]:

with the additional conditions α < 0 and β+(κτpost+γτpre)rinh)>0 for stability.

We can perform similar derivations for the other three synapse types plastic in isolation:

The parameters θ of the plasticity rules were optimized using an evolutionary strategy (CMA-ES [40]). It is difficult to compute usable gradients in long unrolled computational graphs, such as spiking networks, due to exploding or vanishing gradients [53]. In the case of spiking networks, simulation time-steps have to be small (0 . 1 ms), and total simulation times need to be long enough to give time for plasticity to carve the weights in the network at biologically realistic timescales. Aware of the instability problems encountered in the training of learned optimizers in Machine Learning [35,53,54], we thus use evolutionary strategies instead, for their smoothing properties [53,54].

We chose CMA-ES for its robustness and low number of hyperparameters. Briefly, at meta-iteration i + 1, a set of n plasticity rules to evaluate is generated such that:

with θi∗ the current best "guess" at this stage of the optimization, and Ci a covariance matrix, both of which are updated at each meta-iteration based on the scores of the set of n rules tested at the current meta-iteration as well as their previous values [40].

Such gradient-free optimization strategies require the simulation of many plastic spiking networks. The generation size parameter of CMA-ES n was typically chosen to be twice the number of plasticity parameters, and the number of trials Ntrials over which to evaluate a single meant that every meta-iteration required the simulation of nNtrials (values chosen between 4 and 10) separate recurrent spiking networks. Plastic networks were simulated in C++ using Auryn, a fast simulation software for spiking networks [55].

Covariance matrix: The covariance matrix is updated at every meta-iteration in CMA-ES. In Figs 3, 5 and 6, we only show the covariance matrix at one meta-iteration, before the loss plateaus (typically meta-iterations 10 to 20). Once the loss plateaus, we observed that most terms in the covariance became close to 1 or -1 (S2 Fig).