Let us consider the adjacency (or connectivity) matrix of a weighted directed network [26], composed of vertices and without self-edges. The vertices represent the neurons, with possible synaptic connections among them. The synaptic efficacy between two neurons is expressed as a positive element in the adjacency matrix. is thus composed by positive elements off-diagonal, taking values in the bounded range and by zero diagonal entries. We define as a measure of the symmetry of (1)where is the number of instances where both and are zero, i.e. there is no connection between two neurons. The term is a normalisation factor that represents the total number of synaptic connection pairs that have at least one non-zero connection. A value of near 0 indicates that there are virtually no reciprocal connections in the network, while a value of near 1 indicates that virtually all connections are reciprocal. We exclude (0,0) pairs from our definition of the symmetry measure. Mathematically such pairs would introduce undefined terms to Eq. (1). In addition, conceptually, we expect that small weights will not be experimentally measurable. It is then reasonable to exclude them, expecting to effectively increase the signal to noise ratio.

Let us consider a large number of instances of a network whose connection weights are randomly distributed. Each adjacency matrix can be evaluated via our symmetry measure. We rewrite Eq. 1 as:(2)where is a linear index running over all the non-zero “connection pairs” within the network. We can then estimate the mean and variance of over all networks as:(3)(4)where the notation and implies that the expected value and variance are computed along the different representations of the network.

Eq. (3), (4) allow us to transfer the statistical analysis from to To derive theoretical formulas for mean value and variance of we use the fact that its probability density function (PDF), can be written as a joint distribution, where we have introduced the notation (5)within the range defined by and Similarly, we can calculate the variance as follows:(6)

We note that mean value and variance of can be numerically estimated either by using a large set of small networks or on a single very large network: What matters is that the total number of connection pairs, given by the product is sufficiently large to guarantee good statistics and that connection pairs are independent of each other. In the calculations below, we assume a very large adjacency matrix.

We first consider a network with randomly distributed connections without pruning, followed by the more general case where pruning is taken into account.

The procedure described above to derive the joint PDF of and is applicable to any distribution. In what follows, we consider a network with initial connections drawn by a truncated Gaussian distribution.

Below we describe the model neural network on which we will apply our symmetry measure.

Having calculated the mean and variance of the symmetry measure over random networks of a specific connectivity distribution, we are now able to directly evaluate the symmetry measure of a specific connectivity structure and conclude whether the symmetric or asymmetric structure observed is due to chance or it is indeed significant. A simple test is, for instance, to calculate how many standard deviations is away from Equivalently, we can form the hypothesis that the configuration is non-random and calculate the p-value by:(38)where we implicitly assume that the distribution of the symmetry measure over all random networks is Gaussian. We can compare this result with the significance level we fixed, typically and we can then conclude the nature of the symmetry of the network with a confidence level equal to or reject the hypothesis.

To demonstrate the validity of our analytical results, we compare them to simulation results. We generated a sample of networks with neurons with random connections with synaptic efficacies varying from to We evaluated the symmetry measure on each network by applying directly the definition of Eq. (37), and then we computed the mean value and variance of that sample. This process was repeated ten times, each one for a different value of the pruning parameter, The final results are shown in Fig. 3A,B, together with the analytical results, see Eq. (16) and Eq. (29). Since numerical and analytical results overlap, we used a thicker (black) line for the latter. The agreement between theoretical findings, listed in Table 2, and numerical evaluations is excellent.

We also considered two extreme cases, symmetric and asymmetric random networks, which respectively represent the upper and lower bound for the symmetry measure defined in Eq. (37). Symmetric random networks have been generated as follows: we filled the upper triangular part of the weights matrix with random values from the uniform/Gaussian distribution. We then mirrored the elements around the diagonal so as to have In the asymmetric case, instead, we generated a random adjacency matrix with values in for the upper triangular part and in for the lower triangular part, so as to have Then, we shuffled the adjacency matrix.

In Fig. 3A,B we contrast our results on random networks with numerical simulations of symmetric and asymmetric random networks: the dashed, light grey line (top line) shows the upper extreme case of a symmetric random network whereas the dash-dotted, light grey line (bottom line) shows the lower extreme case of a asymmetric random network for

When we introduce pruning, the lower bound of remains unchanged, whereas the more we prune the more a symmetric network appears as asymmetric.

In Fig. 3C,D, we show the adjacency matrix for a random pruned network with pruning parameter A network with uniformly distributed initial connectivity is shown in Fig. 3C and a network with Gaussian-distributed initial connectivity is shown in Fig. 3D. Black areas represent zero connection, The “Gaussian” network has most of the connections close to the mean value resulting in higher values for the symmetry measure than in the case of a uniform distribution, compare Fig. 3B with Fig. 3A.

This difference in the mean values of depending on the shape of the distribution implies that for example a weight configuration that would be classified as non-random under the hypothesis that the initial connectivity, before learning, is uniform, is classified as random under the hypothesis that the initial distribution of the connections is Gaussian. To more emphasise this point, we show in Fig. 4 the adjacency matrix of two different networks of neurons. The first network, Fig. 4A, is a non-pruned network with According with the values obtained from the statistical analysis (Table 2), if we assume that the connections of this network are randomly drawn from a uniform distribution, the p-value test (Eq. (38)) gives us p-value With the usual confidence level of this is a significant result, implying that the network configuration is unlikely to be random. On the other hand, if we assume that the initial connectivity is drawn from a Gaussian distribution, we obtain p-value meaning that the network configuration should be considered random.

In Fig. 4B, we show a pruned network with and In this case the opposite is true: under the hypothesis of uniform random initial connectivity, the network should be considered random, as p-value Under the hypothesis of Gaussian-distributed random initial connectivity, the network should be considered asymmetric, as p-value

In what follows, we demonstrate the relation between our symmetry measure and unidirectional and bidirectional motifs. From the definition Eq. 37, we can deduct that in the extreme case of a network with unidirectional motifs, i.e. pairs of the form (0, x), the symmetry measure will result in while in the case of bidirectional motifs i.e. pairs of the form (x, x), the symmetry measure will result in By inverting Eq. 3, we can derive the mean value for connection pairs We can use now this value to define connection pairs in a network as unidirectional or bidirectional: if than is a unidirectional motif, otherwise it is a bidirectional motif. In this way we relate unidirectional and bidirectional motifs to what is traditionally called single edge motif and second-order reciprocal motif, respectively. It is then expected that when increases, the fraction of bidirectional motifs increases towards whereas the percentage of unidirectional motifs decreases towards

We show this relation in simulations by generating networks of neurons each, with uniformly distributed random connections in and no pruning. In this case the mean value of the symmetry measure is Using Eq. 3, we have which is the value used to decide whether a connection pair is unidirectional or bidirectional. For each of these networks, we calculated the value of the symmetry measure and the fraction of unidirectional and bidirectional motifs and we plotted the results in Fig. 5A as a scatter plot (black circles - bidirectional motifs, grey circles - unidirectional motifs). Also, un Fig. 5B we show the analogous results obtained when we prune the connections with In both cases, a linear relation between and motifs is evident.

Note that in both figures the restricted domain on the s-axis: this is determined by the range of values that correspond to random networks. If we want to extend this range, we need to consider networks that are not random any more. We achieve this by fixing a distribution for connection pairs Once we decide on the desirable value of in our case the whole zero to one spectrum, we can use a distribution (e.g. Gaussian) with mean and a chosen variance to draw the values of all the connection pairs in the network. Following this procedure, we fill the upper triangular part of the weights matrix with random values from the uniform/Gaussian distribution, and derive the other half of the weights by inverting the definition of As a PDF() we chose a Gaussian distribution around with except for the extreme cases (near ) where With this technique of creating networks, we sampled the entire domain of in steps of For each value, we again generated networks of neurons with (half of the) weights uniformly distributed, and then we computed the mean value and standard deviation. Results are shown in Fig. 5C,D respectively for unpruned and pruned (with ) networks (black line - bidirectional motifs, grey line - unidirectional motifs). We can see that Fig. 5C,D correctly reproduce the linear regime observed in Fig. 5A,B for values of close enough to

Due to the method by which we generated networks, the shape of the distribution of half of the weights does not affect the shape of the dependence in Fig. 5C,D. Indeed, if we choose half of the connections to be Gaussian-distributed, we will observe only a shift in both curves as they have to cross at (results not shown).

In the definition of our symmetry measure we have deliberately excluded connection pairs. This was a conscious decision for mathematical and practical reasons, see Methods. As a consequence, pairs of the form do not contribute to the evaluation of the symmetry of the network. Instead, pairs of the form with very small, contribute to the asymmetry of the network according to our specific choice of symmetry measure (leading to see Methods). Here we further motivate this choice via a comparison of our measure to the evaluation of the symmetry via the matrix eigenvalues, for three types of networks: (i) symmetric, where each connection pair consists of synapses of the same value, (ii) asymmetric, where every connection pair has one connection set to a small value and (iii) random, where connections are uniformly distributed. We demonstrate that our measure has a clear advantage over the eigenvalues method, in particular when pruning is introduced. This difference in performance lays in the different ways that and are treated by our measure.

A crucial property of the real symmetric matrices is that all their eigenvalues are real. Fig. 6A depicts the fraction of complex eigenvalues vs the pruning parameter for a symmetric (dash-dotted, light grey line) asymmetric (dotted, dark grey line) and random (dashed, black line) matrix with uniformly distributed values, similar to Fig. 3A, with the same statistics ( networks of neurons). As expected, if no pruning takes place (), symmetric matrices have no complex eigenvalues and are clearly distinguishable from random and asymmetric matrices. On the contrary, both random and asymmetric matrices have a non-zero number of complex eigenvalues, which increases with a higher degree of asymmetry, leading to a considerable overlap between these two cases, differently from what happens with our measure in Fig. 3A.

As we introduce pruning, the mean of the complex eigenvalues of the three distinctive types of network moves towards the same value, an increase for the symmetric network and decrease for the random and non-symmetric networks. This is expected as pruning specific elements will make the symmetric network more asymmetric while it will increase the symmetry of the asymmetric network by introducing pairs of the form or The pairs are due to the construction of the asymmetric network, where half of the connections are stochastically set to very low values. This continues till after which further pruning reduces the number of complex eigenvalues of all networks: a high level of pruning implies the formation of more or pairs for the asymmetric network and more pairs for the symmetric network. In Fig. 6B we show the dependence of the fraction of complex eigenvalues for uniform random matrices on their size.

Comparing Fig. 6A to Fig. 3A, we observe that our symmetry measure offers excellent discrimination between the symmetric, asymmetric and random matrices for e.g. This is despite the fact that the structure of the asymmetric matrix per se has become less asymmetric and the structure of the symmetric matrix has become more asymmetric due to the pruning, as it is confirmed by the overlapping fraction of complex eigenvalues for asymmetric and random matrices (Fig. 6A). In our measure pairs are treated as asymmetric, pairs are ignored, and the bias that pruning introduces is taken into account allowing for good discrimination for all types of matrices, even beyond

We demonstrate the application of the symmetry measure to a network of neurons evolving in time according to a Spike-Timing Dependent Plasticity (STDP) “triplet rule” [32] by adopting the protocols of [18]. These protocols are designed to evolve a network with connections modified according to the “triplet rule”, to either a unidirectional configuration or bidirectional configuration, with the weights being stable under the presence of hard bounds. We have deliberately chosen a small size network as a “toy-model” that will allow for visual inspection and characterisation at the mesoscopic scale.

We simulated integrate-and-fire neurons (see Methods section for simulation details) initially connected with random weights drawn from either a uniform (Fig. 7) or a Gaussian (Fig. 8) distribution (see Table 1 for parameters). Where a pruning parameter is mentioned, the pruning took place prior to the learning procedure: with a fixed probability some connections were set to zero and were not allowed to grow during the simulation.

Our choice allows us to produce an asymmetric or a symmetric network depending on the external stimulation protocol applied to the network. Since the amplitude of the external stimulation we chose () is large enough to make a neuron fire every time it is presented with an input, the firing pattern of neurons reflects the input pattern and we can indifferently refer to one or another. The asymmetric network has been obtained by using a “sequential protocol”, in which neurons fire with the same frequency in a precise order one after the other, with delay, see also [18]. The symmetric network is produced by applying a “frequency protocol”, in which each neuron fires with a different frequency from the values In both cases, the input signals were jittered in time randomly with zero mean and standard deviation equal to of the period of the input itself for the frequency protocol, to of the delay for the sequential protocol. Depending on the protocol, we expect the neurons to form mostly unidirectional or bidirectional connections during the evolution.

The time evolution for both protocols and initial distributions is shown in Figs 7A,D (uniform) and 8A,D (Gaussian). Each panel represents the evolution of the symmetry measure averaged over different representations for both fully connected networks ( solid black line) and pruned networks (e.g. dashed grey line). The shaded area represents the standard deviation. The time course of the symmetry measure can be better understood with the help of the Fig. 3. At the beginning, the values of reflect what we expect from a random network. Afterwards, as the time passes, the learning process leads to the evolution of the connectivity. As expected, the frequency protocol induces the formation of mostly bidirectional connections, leading to the saturation of towards its maximum value, depending on the degree of pruning. On the other hand, when we apply the sequential protocol, connection pairs develop a high degree of asymmetry, the values of decreasing towards its minimum. Connections were constrained to remain inside the interval

The final connectivity pattern can be inspected by plotting the adjacency matrix In Fig. 7B,C and 8B,C we give an example of at the end of the evolution for one particular instance of the networks when the frequency protocol is applied. Similarly, in Fig. 7E,F and 8E,F we show the results for the sequential protocol. The corresponding values of for each of the examples in the figures are listed in Table 3. In the case that a careful inspection of Fig. 7B, 8B indicates that connectivity is bidirectional: all-to-all strong connections have been formed. On the other hand, In Fig. 7E, 8E, trying to determine if there is a particular connectivity emerging in the network starts to be considerably tough. However, by using our symmetry measure (see values in Table 3) we can infer that the connectivity is unidirectional. In the pruned networks, however, see Fig. 7C, 8C and Fig. 7F, 8F, the formation of bidirectional and unidirectional connection pairs is not as obvious as for We therefore refer again to the Table 3 and compare the values of with and with or depending on the case. We can then verify that the learning process has significantly changed the network and its inner connections from the initial random state.

We can rigorously verify the above conclusions via a statistical hypothesis test such as the p-value test, which in essence quantifies how far away the value of our symmetry measure of our final configuration is from the initial, random configuration (see also Methods). In Table 3 we show the p-values corresponding to the null hypothesis of random connectivity for the examples in the Fig. 7, 8. Once we set the significance level at we can verify that, except for the case of pruned network with initially Gaussian-distributed connections where a frequency protocol has been applied (i.e. GF the p-values are significant, implying the rejection of the null hypothesis. This is also justified by Fig. 3A, B: when we increase the pruning, the mean value of the symmetry measure of the fully symmetric network approaches that of the pruned random network and in particular for the case where the weight are randomly Gaussian-distributed.