Here we consider the reconstruction of optimal context-specific metabolic networks as the selection of a subset of reactions from a global genome-scale metabolic network for a particular organism, in a way that maximizes the agreement with experimental data, i.e., reactions in the model with evidence of being active in a given context should be preserved, and reactions with evidence of being inactive should be removed from the model. The selection of this subset of reactions is also subject to flux-based constraints, which constrain the space of possible ways in which those reactions can be selected. More formally, given:

The goal is to find the binary vector x (or equivalently the subset R_c) such that f(x) is maximized. Reactions included in the R_c set have to carry a non-zero flux under steady state conditions. This problem can be stated as a Mixed Integer Linear Programming (MILP) problem with the following form: max f ( x ) = c T x s . t . S · v = 0 x i * v min,i ≤ v i ≤ x i * v max,i v ∈ R n , x ∈ { 0 , 1 } n (1) where x_i ∈ {x₁, …, x_n} are the binary variables representing if reaction R_i is present or not, v_i ∈ {v₁, …, v_n} the variables representing the flux through each reaction R_i, and v_min and v_max the lower and upper bounds for the flux through each reaction. Note that what is subject to optimization is the selection of the reactions but not the fluxes. Fluxes are constrained within some bounds v_min and v_max, and forced to be in steady state (S ⋅ v = 0). Reactions can be included (x_i = 1) only if they can carry some non-zero flux, and reactions not included (x_i = 0) are forced to carry a zero flux. In the following, we shall use this notation to introduce different MILP problems for context-specific reconstruction of metabolic networks.

The objective function f(x) calculates the agreement between the experimental data and the selected reactions. One common strategy is to divide reactions in two disjoint sets based on experimental evidence, namely reactions associated with highly expressed enzymes (R_H ⊆ R) and reactions associated with lowly expressed enzymes (R_L ⊆ R), and then defining f(x) as: f ( x ) = ∑ i ∣ R i ∈ R H x i + ∑ i ∣ R i ∈ R L 1 - x i (2)

This is the strategy described in iMAT, in which the selection of one reaction in R_H or the removal of one reaction in R_L contribute in the same way to the score. Other strategies such as Fastcore, enforce the inclusion of all the reactions in R_H, and so f(x) evaluates only the number of selected reactions in R_L to minimize it.

In practice, the binary vector x is extended to account also for reversible reactions in the R_H set that can be active carrying a negative flux. Also, a tunable parameter ϵ corresponding to the minimal amount of flux a reaction has to carry to be considered active is usually included in the optimization problem. In the original iMAT formulation, a reaction R_i ∈ R_L which is not selected (which carries no flux) has a value of x_i = 1 representing a match with the experimental data, and so Eq 2 simplifies to just f(x) = ∑_i x_i. The full problem specification is described in Eq 3: max ∑ i x i s . t . S · v = 0 v i + x i + ( v min,i - ϵ ) ≥ v min,i ∀ i ∣ R i ∈ R H v i + x i - ( v max,i + ϵ ) ≤ v max,i ∀ i ∣ R i ∈ R H v i + x i o · v min,i ≥ v min,i ∀ i ∣ R i ∈ R L v i + x i o · v max,i ≤ v max,i ∀ i ∣ R i ∈ R L v ∈ R n , x + , x - , x o ∈ { 0 , 1 } | R | x = ( x + , x - , x o ) ∈ { 0 , 1 } 3 | R | (3)

It is important to remark that the methods presented here are general strategies for enumerating optimal metabolic network reconstructions, and therefore can be used with different base algorithms for the reconstruction, as long as they are implemented as MILPs. This means that the methods serve to enumerate iMAT-like solutions [7], Fastcore-like solutions [15], INIT-like solutions [14], or any other type of MILP-based reconstruction.

In the following sections, for practical reasons and without loss of generality, we use the original set of iMAT constraints and objective function as the base MILP problem for network enumeration, since: 1) it relies on a MILP formulation, which can be easily adapted to optimally solve different optimization problems and objectives; and 2) the default objective function optimizes a trade-off between the coverage of reactions associated with highly expressed genes and reactions associated with lowly expressed genes, which has been proven in practice a good general strategy that only requires gene expression data. This trade-off introduces flexibility in the optimization process, allowing us to predict that some reactions are not active even though they are associated with highly expressed genes, something important to account for post-transcriptional events.

The enumeration problem arises naturally in context-specific reconstruction of metabolic networks due to the discrete nature of the selection of reactions and the imbalance between the available constraints and the complex topology of the networks, leading to an undertermined problem.

In order to better analyze the enumeration problem from a computational point of view, we use a Directed Acyclic Graph (DAG) network model. Directed Acyclic Graphs are commonly used for the analysis of biology networks in general [28]. By representing metabolic networks as DAGs, we can calculate in advance how many optimal solutions we can expect, and thus compare the techniques with a ground truth (i.e., the full set of optimal solutions that exist in a specific DAG) in an objective manner focusing specifically on the computational problem of enumeration. This is important because although the scope of application is biological, the technique is basically computational and requires a proper computational analysis of the problem under study. Fig 2 shows the generic DAG metabolic network with L layers of N metabolites. Each metabolite m_i,k in layer L_k is connected to each metabolite m_j,k+1 in L_k+1 by single reactions R_ijk = (m_i,k, m_j,k+1) with only one substrate and product. The model includes two extra metabolites m_s as a source and m_t as a sink node to centralize the import and export reactions and simplify the model. The number of total metabolites, including m_s and m_t is 2 + N ⋅ L, and the number of total reactions is 2N + N² ⋅ (L − 1).

In this example, we want to extract the context-specific metabolic network, given the following conditions:

It can be seen that a metabolic network with optimal f(x) in this case is the one that carries flux from m_s to m_t using the minimum number of reactions (since they are all in the R_L set), which corresponds to a shortest path from m_s to m_t. Since there are no loops in the network, the shortest length for the path is L + 2 (including the path from m_s to L₁ and from L_N to m_t). This also implies that there is no single solution, but instead any path from m_s to m_t is an optimal solution, i.e., a context-specific reconstruction network with optimal f(x) given the previously defined conditions. Since there are N different paths to go from any metabolite in layer L_j to any metabolite in layer L_j+1, that makes N^L possible optimal networks in this particular example, that is, the number of possible optimal solutions in this example grows exponentially with the number of layers. Note also that since the number of reactions for a fixed number of metabolites grows linearly with the number of layers, the number of possible solutions grows also exponentially with the number of reactions.

This example illustrates that there are instances of the enumeration problem for which the number of optimal solutions grows exponentially with the size of the network. Thus, in general, enumerating the full set of optimal metabolic networks can be impractical, especially considering the size of networks such as Recon 3D [29] with 13,543 reactions, or the recent Human1 network [30] with around 13,000 reactions.

More formally, it can be shown that the enumeration of all optimal metabolic networks is a type of vertex enumeration problem. Let M_P be the general MILP problem for context specific reconstruction using a GSMN with n reactions and with objective function c^T x that we want to maximize, as defined in Eq 1. Let Ω be the set of all 0/1-vectors representing the feasible solutions for the MILP M_P that satisfy all the constraints defined in Eq 1. From a geometric point of view, the space of possible networks can be viewed as vertices of the hypercube C_n = {0, 1}ⁿ, and the set of feasible solutions Ω as a subset of vertices of C_n, where its convex hull is a 0/1-polytope P, that is, P = conv(Ω). Let z* be the optimal value of M_P, i.e., ∀x ∈ Ω, c^T x ≤ z*. We are interested in the set of all optimal feasible solutions Ω* ≔ {x* ∣ x* ∈ Ω∧c^T x* = z*}, where P* = conv(Ω*) ⊆ P is the 0/1-subpolytope of interest in H-representation (as the intersection of half spaces defined by all the constraints) from which we want to obtain the V-representation, that is, the set of vertices as vectors of 0/1 coordinates (the optimal context-specific networks), which is the definition of the vertex enumeration problem.

Vertex enumeration [31] is a classical problem in the field of combinatorial optimization for which some specific techniques were proposed [32]. For the special case of 0/1-polytopes [33], some notable approaches are Binary Decision Diagrams [34–38], tree search-based methods [39, 40] and techniques based on branch-and-bound and cutting planes, extensively exploited in academic/commercial solvers such as IBM CPLEX and Gurobi. In fact, as a general enumeration mechanism, these solvers incorporate the concept of a pool of optimal solutions, in which the tree of feasible solutions continues to be explored until a specific number of optimal feasible solutions have been found.

However, as discussed before, the number of optimal metabolic networks for a given problem can be extremely large, and so classical vertex enumeration techniques are not suitable for this task. One reason is that, given the potential vast number of possible solutions and a fixed amount of time to generate a variety of optimal solutions, there is no guarantee that these methods will generate a diverse set of solutions. In fact, the opposite is more likely: similar solutions (e.g., small variations in reactions on the same pathway) will probably be closer in the search space. Also, due to symmetries introduced by loops and other patterns in metabolic networks, chances are that the enumeration gets trapped performing enumeration in small dense regions of the search space that can be more related to artifacts than to solutions with true biological meaning.

In the following sections we present four enumeration strategies and analyze their advantages and drawbacks. It should be noted that we limited to a set of generic techniques that can be implemented on top of general MILP solvers and can be easily integrated in the existent pipelines for network reconstruction. One disadvantage of this is that each solution is obtained by constructing and solving a new MILP problem. Ad-hoc search strategies for the enumeration of MILP solutions based on custom branch-and-cut methods or more advanced tree search exploration, although they might be more efficient in some situations, are out of the scope of this work.

A simple way to generate alternative optimal metabolic networks can be achieved by directly manipulating the flux bounds of each reaction to force it to carry some positive flux, some negative flux (if reversible), or no flux, as in [20, 21]. The original method traverses all the reactions in the model testing forward (or backward flux if the reaction is reversible) or blocking flux in order to generate a new solution with a different activation for each reaction. Solutions that are still optimal after the modification are added to the set of optimal solutions. This method has however two major limitations: 1) it only explores variations in single reactions (if they can be active or inactive in an optimal solution), leaving the vast space of combinations between reactions completely unexplored; and 2) it generates many duplicated solutions, wasting computation time.

We included this basic idea in DEXOM as a simple mechanism to generate alternative solutions, under the name of Reaction-enum, with some modifications and further options that can be used to reduce some of the limitations in its basic form. One option that can be enabled to alleviate the problem of generating duplicated solutions consists in tracking the activation or inactivation of each reaction in the set of alternative optimal networks during the search process. If forcing the flux through a reaction R_ijk results in an optimal sub-network with another reaction R_i,j+1,k+1 active, then there is no need to force flux through R_i,j+1,k+1 as it is not guaranteed that this operation is going to generate a new sub-network (unless the solver is adjusted to increase randomness in the solutions returned).

One advantage of the Reaction-enum method is that it tests every reaction in the model to see if its presence or absence affects the quality of the solution. This generates alternative networks with modifications in every possible pathway of the metabolic network, which makes this technique a good starting point for more advanced enumeration methods (for example, to generate a set of initial candidate optimal solutions).

One simple way to perform a full enumeration of the set of optimal networks is by adding integer-cuts, linear constraints that can be added to the original problem to remove already visited solutions from the set of feasible solutions. This method, which has been already used to enumerate solutions in general for MILP problems [41], can be used as well as a mechanism to enumerate alternative metabolic networks. We adapted this technique for enumeration of context-specific reconstructions under the name of DEXOM Icut-enum. Starting with a default optimal solution x* to the MILP problem defined in Eq 3, a new solution is generated by adding a new constraint to the original problem to cut x* from the set of feasible optimal solutions. This process is repeated for each new solution, adding a new constraint per solution. A new solution is accepted if there is at least one different reaction in the candidate sub-network, that is: ∑i|xi−xi*|≥1 (4)

Although this constraint is not linear due to the absolute value, it can be linearized by considering separately the ones from the zeros. Two solutions are equal if they have the same set of active reactions and the same set of inactive reactions. Thus, for each x i * = 1, we expect to have x_i = 1, and for each x i * = 0, we expect x_i = 0 if both the previous solution and the candidate are equal. Under this situation, summing up all the ones from x_i for which x i * = 1 should be equal to ∑ ( x i * ) (except if there is one or more differences), and in the same way, summing up all the zeros from x_i for which x i * = 0 should be equal to zero. If this does not happen, then there is some difference between the candidate solution x_i and a previous optimal solution x*. More formally, the linearization of Eq 4 can be written as: ∑i∈Axi−∑i∈Bxi≤(∑ixi*)−1A={i∣xi*=1},B={i∣xi*=0} (5)

By adding this constraint for each new x* returned by the solver, we exclude all the previous solutions that have been found so far. The generation of new solutions stops when the problem becomes infeasible, that is, there are no more feasible optimal solutions. Note that this cut can be modified to cut feasible optimal solutions that differ in more than one reaction, i.e., to cut solutions that are at some specific Hamming distance.

The advantage of this method is that it enumerates all possible solutions since it removes one by one every optimal feasible solution. It is straightforward to see that this method enumerates all the feasible optimal solutions by observing that: 1) each cut removes one optimal solution; 2) the number of constraints that are added grows monotonically with every new optimal solution; and 3) the number of solutions is finite. Let us assume that for a given problem, the set of optimal feasible solutions Ω* contains N different solutions, i.e., for every pair {x*, y*} ⊂ Ω*, ∑ i | x i * - y i * | ≥ 1 (there is at least one different reaction between any two optimal solutions). For the sake of the proof, let us assume that after N steps of the algorithm, and after adding N integer cuts, one per optimal solution, the last MILP problem is still feasible, i.e., solving it returns a solution z*, thus: 1) z* is different to any other solutions in at least one reaction, which means that there are at least N + 1 solutions, contradicting the initial assumption; or 2) z* is a duplicated solution, that is, there exist a solution x* ∈ Ω* such that ∑ i | z i * - x i * | = 0, which contradicts the definition of the integer cut.

However, in practice, it is not possible to enumerate the entire space of solutions due to the potential number of possible optimal solutions. Although this technique can be also used to generate a sample of optimal solutions (stopping the search after a desired number of solutions was found), the method is not well suited for this task since: 1) the number of constraints grows linearly with the number of solutions, which increases the computational difficulty with each new solution; 2) the algorithm can get trapped enumerating solutions in a small region of the whole space of possible optimal solutions, and so diversity in the set of solutions is not guaranteed; 3) even if a new optimal solution exists, due to numerical instabilities or precision errors, the search process can prematurely stop at the first incorrectly detected infeasible problem.

Another strategy for the enumeration of optimal solutions is to search the most dissimilar metabolic network to a previous optimal one. This idea, already explored in the context of Integer Programming problems [25, 42], has been also proposed for metabolic networks [22]. The strategy requires to solve a bilevel optimization problem in which the inner optimization problem solves the original problem and the outer optimization maximizes dissimilarity. This particular bilevel optimization can be implemented as a standard MILP problem, by introducing a constraint that corresponds to the original objective function. First, an optimal solution x* with optimal score f(x*) = z* is calculated using the problem defined in Eq 3, and then the original objective function is replaced by the minimization of a function g(x, x*) which measures the similarity between the candidate solution x and a reference optimal solution x*. In order to guarantee that the new solution to this new problem is still optimal in the original problem, a new constraint f(x) = ∑_i(x_i) = z* has to be added to preserve optimality.

Although the idea of returning the most dissimilar optimal network is interesting, one of the limitations is that it can easily oscillate between a small set of optimal networks that are the most distant to each other, since only the previous optimal solution is discarded. Consequently, the technique also does not guarantee to obtain all the possible solutions. We have introduced a modification to the original idea to break this pattern and allow the complete enumeration of solutions by adding integer cuts. This modification prevents trivial oscillations between already visited solutions and enumerates the space of solutions starting from the most extreme differences. We refer to this technique as Maxdist-enum.

The objective function g of the Maxdist-enum method can be defined as the minimization of the overlapping of ones between x and x*. Note that the optimality constraint guarantees that the solutions must have the same number of ones (same score), and so removing one overlap (for example by not including a reaction in R_H which is present in the reference solution) has to be compensated by including another reaction in the set of R_H not present in the reference solution, or by removing one reaction in the R_L set which is present in the reference solution, in order to preserve the original optimal score. Minimization of the overlapping of ones between x and x* with this constraint is essentially the same as finding the most extreme vertices of the 0/1-polytope of feasible optimal solutions using the Hamming distance. min x g ( x , x * ) = ∑ i ∣ x i * = 1 x i s . t . S · v = 0 ∑ i ∈ A x i - ∑ i ∈ B x i ≤ ( ∑ i x i * ) - 1 ∑ i x i = z * v i + x i + ( v min,i - ϵ ) ≥ v min,i ∀ i ∣ R i ∈ R H v i + x i - ( v max,i + ϵ ) ≤ v max,i ∀ i ∣ R i ∈ R H v i + x i o · v min,i ≥ v min,i ∀ i ∣ R i ∈ R L v i + x i o · v max,i ≤ v max,i ∀ i ∣ R i ∈ R L (6)

The expected behavior of this algorithm is the following: starting from the default solution x*, the search process generates the most distant network with the same optimal score. This process is repeated, changing the x* in each iteration to the one previously found, pushing away the search to the boundaries of the space until the most distant networks in the space of optimal solutions are discovered. The integer-cut constraint prevents search loops, and so once the extremes are found, the distance of new solutions decreases progressively.

This method has also limitations that may prevent its use for generating a diverse sample of optimal solutions. Concretely, even though the integer-cut constraint prevents generating repeated solutions, the density of similar metabolic networks at the boundaries can be large enough to never explore other areas. This increases the risk of ending up oscillating between a small group of clusters of networks with a large inter-cluster distance but a very small intra-cluster distance. In addition to this, the method is computationally more expensive than the previous ones.

Based on the previously identified problems and improvements for each method, we also propose a novel algorithm to generate a set of diverse optimal metabolic networks, combining the advantages of the techniques described before. The basic steps of Diversity-enum are:

Algorithm 1 Diversity-enum algorithm

1: procedure Diversity_enum(iters, d_s, f)

2: x r ( 0 ) , … , x r ( k ) ← initialsolutionswiththereaction - enummethod

3: i ← 0

4: x ( i ) ← x r ( k )

5: z* = f(x⁽ⁱ⁾)

6: while i < iters and f(x⁽ⁱ⁾) = z* do

7:  y⁽ⁱ⁾ ← vector of 0s of same size as x⁽ⁱ⁾

8:  pick_prob ← 1 − exp(d_s, i) # where exp(a, b) = a^b

9:  for j ∣ x j ( i ) = 1 do

10:   sample u ∼ U ( 0 , 1 )

11:   if u ≤ pick_prob then

12:    y j ( i ) ← 1

13:  s ← solve maxdist MILP min s g ( s , y ( i ) ) (Eq 6)

14:  i ← i + 1

15:  x⁽ⁱ⁾ ← s

16: return x r ( 0 ) , … , x r ( n ) , x ( 1 ) , … , x ( i ).

A detailed version of the algorithm is described in Alg. 1. Diversity-enum combines the advantages of the previous techniques. It starts computing an initial set of solutions using the Reaction-enum method avoiding duplicated solutions. This guarantees that single variations of reactions across all pathways are explored, as long as this initial set of solutions is large enough (i.e., all reactions of the network are traversed to generate alternative solutions). Then, starting for any solution from this initial set, the algorithm explores solutions in the vicinity of the selected solution, using it as a template, for which only a subset of the reactions present in the selected solution are used to maximize the distance to the new solution. The more reactions that are selected to maximize the distance, the more different the new solution found will be from the selected one. The number of selected reactions from the template at each iteration (i.e., the distance of the next solution with respect the previous one) is controlled by the parameter d_s ∈ [0, 1]. For example, using a d_s value close to one (e.g. d_s = 0.99), the distance of the solution obtained at iteration 70 with respect the previous one (obtained in the iteration 69), is going to be 1 − 0.99⁷⁰ ≈ 0.5, that is, the expected distance of the next optimal solution with respect the previous one is half of the possible maximal distance. At iteration 1,000, this value is 1 − 0.99¹⁰⁰⁰ ≈ 1, and so the algorithm is now searching for the most distant solution, as Maxdist does. Using a higher value of d_s (e.g. d_s = 0.999) makes this transition from near to far solutions slower, since this time the value at iteration 70 is only 1 − 0.999⁷⁰ ≈ 0.07. It should be noted that if d_s = 0, after computing the initial set of solutions, Diversity-enum behaves exactly as Maxdist-enum. By default, the value of this parameter has been set to d_s = 0.995, and all experiments performed, unless otherwise indicated, have been done with this value.

Some preliminary experiments that we carried out suggest that it is preferable to start with the initial set of solutions using the Reaction-enum method and expand it by progressively looking for more distant solutions, rather than the other way around. The reason is that if we start with the most extreme solutions, as we progressively decrease the distance, the effect we get is to explore solutions that are closer to each other but still located in the extremes of the space, and still far from the initial solutions.

Given the unknown volume of the 0/1-polytope comprising the optimal solutions, it is not possible to directly estimate its size without sampling solutions from it. In order to measure how diverse are the set of solutions obtained with different methods, we need to rely instead on indirect measures. Since solutions are indexed by 0/1 coordinates, one reasonable metric to use is the Hamming distance: δ h ( x , y ) = 1| x | ∑ i = 1 n | x i - y i | (7)

For each pair of solution vectors x, y ∈ {0, 1}ⁿ obtained from the set of optimal solutions Ω*, we compute the Hamming distance (i.e., how many reactions are different between any two solutions) and we average across all the distances between any two solutions to obtain the average pairwise distance δ h ¯. One way to promote diversity is to maximize this measurement: between two different sets of optimal solutions (of a similar size), the set with a larger average pairwise distance contains solutions that are, on average, more diverse. However, relying only on the average pairwise distance might not be informative enough in some situations, since two groups of solutions that are very different between groups but very similar within groups, can have a large average pairwise distance driven by the distance between groups, even thought the diversity is low within groups. Under these circumstances, it is easy to have the false impression that the set of solutions is diverse, but instead it will contain only the two initial different solutions with very small variations.

To discriminate better between these situations, we use also the average nearest neighbor distance δ ¯ h n n defined as: δ ¯ h n n ( O ) = 1| S | ∑ x ∈ O min y ∈ O \ { x } δ h ( x , y ) (8)

That is, for each optimal solution in the solution set O ⊆ Ω* obtained with some enumeration method, we measure the distance to all other solutions and we take the distance to its closest solution (nearest neighbor). Then, we average all those distances to have the average nearest neighbor distance.

The average nearest neighbor distance measures how spread the solutions are. We want solutions that are spread to cover a wider range of different solutions and avoid the enumeration of clusters of very similar solutions.

Considering these two metrics, we can devise four situations when comparing the solution sets obtained by different methods:

Although simple, these metrics provide an idea of how different the solutions enumerated by the methods are.

Context-specific metabolic networks can be used to make predictions about the metabolism of a cell or tissue in a specific experimental condition. Of a particular interest is the prediction of essential genes. An essential gene is a gene whose totally or partially inactivation prevents the organism to growth or survive. Some genes are absolutely required for survival, whereas other genes are conditionally essential, meaning that they are essential depending on the environmental conditions. For example, the gene ARG2, which encodes glutamate N-acetyltransferase —a mitochondrial enzyme that catalyzes the first step in the biosynthesis of the arginine— is annotated as a essential gene in Saccharomyces cerevisiae (https://www.yeastgenome.org/locus/S000003607) only in the absence of arginine in the medium.

Many essential genes that are related to metabolism (those related to enzymes) can be predicted using metabolic networks. However, conditionally essential genes are particularly hard to predict since they cannot be predicted without integrating experimental data or knowledge related to the condition. Context-specific metabolic networks are able to predict them indirectly, by extracting first the sub-network which is most consistent with the experimental data. After removing all the reactions that are predicted to be inactive in a given context, conditionally essential genes that were not essential in the generic network might be now essential in the reconstructed network.

Predictions of essential genes using metabolic networks can be done by comparing the maximum flux through the biomass reaction —an artificial reaction that encodes the minimum requirements of the organism to sustain a basic metabolic activity— using Flux Balance Analysis (FBA) [43] before and after knocking out a gene in the metabolic network. If the flux through the biomass reactions is below a certain threshold after KO (e.g., below 1% with respect to the wild-type) then the gene is considered essential.

However, as explained before in this section, it is common to find more than one optimal context-specific metabolic network for a given condition, each one representing a different hypothesis of the metabolic state. Each network may predict different essential genes. Since all networks fit the experimental data equally well, there is no clear way to decide a priori which of these predictions may be true. In this situation, a reasonable strategy is to consider that if a network predicts a gene to be essential, then the ensemble decides that the gene is essential, in order to maximize the number of true essential genes (at expenses of increasing the false positives), similar to what has been done in [24] with Gap-Filling methods.

Fig 3 shows an example of how the procedure works. For each gene, a KO is simulated by maximizing the flux through the biomass reaction after knocking out the reaction or reactions associated to the gene (based on the Gene-Protein-Reaction rules), using the singleGeneDeletion method from the COBRA Toolbox [27]. If the ratio between the KO and the wild type is below 0.01 (flux after KO below 1%), the gene is classified as essential. This process is repeated for all genes and for all optimal networks, and then results are combined by performing a logical OR of the predictions across networks.

After obtaining the predictions for each gene, the True Positive Rate (TPR, sensitivity) and the False Positive Rate (FPR, 1-specificity) are calculated by comparing the predictions against the true essential genes for Saccharomyces cerevisiae (included in the repository of the code), and applying the following formula: T P R = T P T P + F N ( T P = True Positives , F N = False Negatives ) (9) F P R = F P F P + T N ( F P = False Positives , T N = True Negatives ) (10)

One common step prior to any metabolic reconstruction is pre-processing the experimental data to map it onto the metabolic networks. The way in which this data is pre-processed also depends on the objective of the reconstruction and the type of data used (generally gene expression data). A common approach is to use gene expression data to classify reactions into two groups: reactions for which there is experimental evidence of being active for a given condition, and reactions for which there is not enough evidence.

A simple method that is frequently used for this purpose is based on a prior classification of genes using quantile thresholds on the normalized gene expression [19]. In this way, genes whose expression levels are above or below certain quantiles are classified as highly or lowly expressed genes. For example, a thresholds specified as [0.25, 0.75] means that genes whose value are below the 25th percentile are classified as lowly expressed, and genes above the 75th percentile are classified as highly expressed. Afterwards, genes are mapped onto the metabolic network using the Gene-Protein-Reaction rules defined in the GSMN in order to get the reactions associated with highly expressed or lowly expressed genes.

Fig 4 shows an example for a threshold of [0.10, 0.90] on the normalized microarray gene expression of the Saccharomyces cerevisiae under aerobic conditions [18, 26]. In this example, genes whose expression levels fall above the upper threshold (around a normalized gene expression of 11) are considered highly expressed, whereas genes below a normalized gene expression around 6 are classified as lowly expressed genes.

As mentioned before in the section Methods, for practical reasons in this work we use the iMAT reconstruction objective as the base reconstruction problem for enumeration. This means that the results we enumerate are those that achieve the optimal trade-off between selection of reactions associated with highly expressed genes and removal of reactions associated with lowly expressed genes.

We use this threshold-based method for the classification of reactions based on gene expression levels because of its simplicity and widespreadness, but other methods could be used instead, for example StanDep [44] or Barcode [45] (for Affymetrix microarray data). Changing the method changes the set of optimal solutions to the problem, but does not eliminate the problem associated with the enumeration. Analyzing the correctness of the pre-processing technique is beyond the scope of this work, since the problem of enumeration is independent of the pre-processing method (multiple optimal solutions can still exist regardless of the method used).

We measure how well each method performs to generate diverse samples of optimal solutions. To do so, we generate samples of fixed size with each method and we measure the diversity of the sample using the average Hamming distance and the average nearest neighbor that were introduced in Section Methods. We consider two different scenarios: 1) obtaining a sample of optimal metabolic networks in a simulated scenario where the number of total optimal solutions is known; and 2) obtaining a sample of optimal solutions in realistic scenarios where the total number of optimal solutions is unknown.

Next, we evaluate the predictive capabilities of the different methods for in-silico prediction of essential genes in the model organism Saccharomyces cerevisiae. We used gene expression measured from yeast in aerobic conditions [18, 26]. Genes were classified into expressed and not expressed using different combinations of thresholds on the quantiles of the distribution as it is commonly done in context-specific network reconstruction. For instance, a threshold of [0.25, 0.75] indicates that genes whose normalized expression value are below the quantile 0.25 are classified as lowly expressed, whereas those above the quantile 0.75 are highly expressed. Reactions were splitted into R_H and R_L sets using the mapExpressionToReactions method from the COBRA Toolbox [27].

Essential genes in Yeast 6 [46] were curated using most updated information from YDPM (http://www-deletion.stanford.edu/YDPM) database and the SGD (https://www.yeastgenome.org) project [47] (S1 File). Genes that are essential due to mechanisms not directly related to metabolism were excluded from the set, as they cannot be predicted using FBA. In total, 188 genes out of the 900 in Yeast 6 are considered to be essential under aerobic conditions.

A maximum of 2,000 optimal networks were enumerated for each combination of threshold and method, using a time limit of 8h per threshold/method, and 5 min. timeout for each MILP problem. The lower bound of the biomass reaction was constrained to carry a small positive flux, to ensure that all initial sub-networks will allow biomass production and therefore could be used to simulate the effects of gene knockout on the biomass production using FBA. In-silico predictions of essential genes were carried out using COBRA Toolbox v3.0.6 [27], classifying each gene as essential if the flux through the biomass reaction was below 1% after KO.

Essential genes were predicted for each optimal network within the set of the optimal networks obtained by each method and threshold, but also for the ensemble of networks, by taking the union of the predictions as shown in Fig 3. That is, if a gene is predicted as essential by a single optimal network from a set of optimal networks enumerated using a given method and threshold, then the gene is classified as essential by the ensemble. Thus, in total, we generated 16 ensembles per method, one for each threshold.

Table 1 shows the True Positives Rate (TPR, sensitivity) and False Positive Rate (FPR, 1-specificity) of these ensembles. Diversity-enum achieves the best TPR for all thresholds, with the best overall TPR of 0.7713 for the threshold [0.25, 0.90], which corresponds to the correct classification of 145 genes out of the 188 essential genes in the dataset. These results are followed by the Reaction-enum method, which achieves the same TPR as Diversity-enum in 8 out of 16 tests, with a slightly lower FPR in 6 out of those 8 tests. In contrast, Maxdist-enum and Icut-enum ensembles are not very well positioned in terms of TPR, although both methods achieve the lowest rates of false positives for some ensembles. Concretely, the Icut-enum method obtained the lowest FPR in 9 out of the 16 tests.

Differences between ensembles can be better assessed by placing each ensemble in a ROC space (Fig 7), in which each point is an ensemble represented by its TPR and FPR. The upper part of the figure is dominated by Diversity-enum and the Reaction-enum method, whereas the Maxdist-enum and Icut-enum ensembles are characterized by a lower ratio of true and false positives.

One reason that explains these differences between the methods is the systematic generation of alternative solutions by testing every reaction in the model. If one reaction associated to a gene that is essential is not present in any of the set of optimal networks, the gene is not predicted to be essential. However, if there exist at least one optimal solution in which this reaction is present and essential, both Reaction-enum and Diversity-enum have more chances to detect it as they are going to test if there exist an optimal network with that reaction being active. Maxdist-enum and Icut-enum methods leave many of these solutions unexplored. Diversity-enum, in contrast, uses the Reaction-enum strategy to have an initial set of solutions with variations in single reactions, from which it expands the search incrementally, increasing the chances of detecting even more essential genes.

Differences in TPR (S6 Fig) and FPR (S7 Fig) for the ensembles show that the individual networks generated by the different methods achieve a similar rate of true positives and false positives, and so the higher rates scored by the ensembles using Diversity-enum and Reaction-enum are driven by a more diverse set of predicted essential genes. That is, individual networks enumerated by these methods are able to correctly predict different sets of true essential genes, and so the union of those predictions include a more diverse set of detected essential genes. Concretely, the median TPR for the ensembles generated with Diversity-enum and Reaction-enum increase 142% with respect the median TPR of their individual networks, whereas the TPR of the ensembles built with Maxdist-enum and Icut-enum increase only 54% and 51% respectively.

In order to test whether the distance parameter d_s has some strong impact on the results obtained with Diversity-enum method, we repeated the same experiment with parameter values d_s = 0.990 and d_s = 0.999 (S5 Fig). In both cases, the results obtained are very similar to these results obtained with the default parameter d_s = 0.995. This result suggest that most relevant solutions to the problem of prediction of essential genes are concentrated in the same region of the space of optimal solutions that is explored by both Reaction-enum and Diversity-enum. This space corresponds to the alternative solutions generated by modifying the constraints of single reactions in the networks (forcing the inclusion or knocking-out the reaction). Since the simulation of essential genes is based on simulating knockouts in the reactions associated with the genes, it is likely that most of the essential genes can be predicted in some of the optimal networks resulting from those variations in single reactions. However, there is still an advantage in using Diversity-enum, since it expands the initial set of solutions and is able to search for many more than the other technique is not capable of, increasing the probability of detecting more relevant reconstructions.

The computational time of each of the techniques is also different, although it depends on the size of the network and the number of variables (reactions associated with highly and lowly expressed genes). In general, Reaction-enum is the fastest method, while Maxdist-enum is the slowest technique, since the optimization problem of looking for the farthest solution at every step has a higher computational cost (S1 Appendix).

Next, we evaluate the ability to characterize the variability of predictions about which metabolic pathways are most active in different cancer cell lines. To do so, we reconstruct many optimal networks for each cancer cell line and we perform pathway enrichment on each network to see which pathways are more represented in the reconstructed networks than would be expected by chance. Given that there are multiple possible optimal reconstructions per cancer cell line, performing pathway enrichment on each optimal network will give different p-values for each pathway. This variability due to the method can have important implications. For example, by performing pathway enrichment on a single metabolic network, for a given significance level (e.g. α = 0.05) we can detect that pathway A is enriched whereas pathway B is not. However, if we enumerate the space of optimal solutions, we can find an alternative solution in which pathway B is enriched but pathway A is not. Reporting pathway enrichment p-values of a single context-specific metabolic network without characterizing the variability should be in general avoided, as these values are misleading.

In order to test the variability in pathway enrichment scores due to the alternative set of optimal solutions, we used data from [19] for melanoma cells (cell line A375) and leukemia cells (HL60, K562, and KBM7 cell lines) and the human Recon 1 model [48]. Table 2 shows the enrichment results for the reconstructions using two different gene expression thresholds. Column #Nets shows the number of optimal networks that each method was able to enumerate (in a time limit of 8 hours). Column #Enr. shows the number of different enriched pathways (adjusted p-value < 0.05) that were detected by each method. It is important to remark that here, detecting more enriched pathways is better, since all the methods explore the same optimal solutions (all methods for enumeration maximize the same objective function and use the same experimental data). Detecting less enriched pathways means that there exist some other alternative metabolic networks that are enriched for other pathways but the enumeration method missed it, reporting that no enrichment was detected in any of the enumerated metabolic networks.

Overall, the method Diversity-enum is able to discover more alternative hypotheses about the pathways that are most active in each cell line, especially for the threshold [0.10, 0.90]. One of the reasons why there are more pathways that can be enriched is that, with this threshold, far fewer genes are classified as expressed and not expressed, and therefore many reactions of the metabolic network remain unscored (they may or may not be active without affecting the optimality of the solution). This makes it much more likely to find alternative sub-networks whose flux is consistent for the selected genes. In other words, the fewer genes that are identified as expressed or not expressed (less constraints), the more possible hypotheses about the metabolic state are consistent with the data. This is especially relevant in studies using proteomic or exometabolomic data, where the number of identified proteins or metabolites is lower than in gene expression assays.

Among some of the differences, Diversity-enum detected the metabolic pathways Fatty acid activation, Fatty acid elongation, Fatty acid oxidation and Carnitine shuttle enriched in the reconstructions for the A375 cell line both for the thresholds [0.10, 0.90] (Fig 8) and [0.25, 0.75] (Fig 9), whereas Reaction-enum detected them enriched only for the threshold [0.10, 0.90]. The single case where Reaction-enum discovered more existing alternative solutions with enrichment in other pathways not detected by Diversity-enum was for the cell K562 for the threshold [0.25, 0.75], where was able to discover the alternative optimal solutions where pathways Ubiquinone Biosynthesis, Cysteine Metabolism, Aminosugar Metabolism, and Urea cycle/amino group metabolism were enriched in at least one of the optimal solutions enumerated with this method. Icut-enum was the strategy that obtained the worst results, not being able to find many of the optimal solutions with enrichment in other pathways that were discovered by the other methods. The Maxdist-enum method, although it detects in general less variation, finds in some cases enrichment in pathways that are not detected by any of the other techniques (e.g. Pentose and Glucoronate Interconversions in A375 for the threshold [0.10, 0.90], Fig 8). This could indicate that, although all solutions are equally valid, due to the topology of the network, the pattern of the distribution of highly expressed and low expressed genes across the network, and other factors such as the type of algorithms used by the solvers, there are certain types of solutions that are more frequently discovered than others, and therefore are more biased towards the discovery of this type of solutions. In these cases, the Diversity-enum d_s parameter could be decreased to make the search spend more time exploring distant solutions.

We also analyzed how often each pathway was detected as enriched by any optimal metabolic network for each enumeration method (S1 Appendix). Among some of the possible causes that may affect this variability, the number of selected highly and lowly expressed has a clear impact in the results, for the reasons discussed before. Using a threshold of [0.10, 0.90], it can be seen that this variability is greater than for the threshold [0.25, 0.75], simply due to the fact that in the first case only 20% of the genes are used, and more hypothesis are consistent with the data. The enumeration of alternative reconstructions helps to characterize this variability and reduces the risk of incorrectly discarding hypotheses that are equally valid for the same reconstruction method and the same data.