^{*}

The authors have declared that no competing interests exist.

Conceived and designed the experiments: YM ST. Performed the experiments: YM. Analyzed the data: YM. Contributed reagents/materials/analysis tools: YM. Wrote the paper: YM ST.

When exact values of model parameters in systems biology are not available from experiments, they need to be inferred so that the resulting simulation reproduces the experimentally known phenomena. For the purpose, Bayesian statistics with Markov chain Monte Carlo (MCMC) is a useful method. Biological experiments are often performed with cell population, and the results are represented by histograms. On another front, experiments sometimes indicate the existence of a specific bifurcation pattern. In this study, to deal with both type of such experimental results and information for parameter inference, we introduced functions to evaluate fitness to both type of experimental results, named quantitative and qualitative fitness measures respectively. We formulated Bayesian formula for those hybrid fitness measures (HFM), and implemented it to MCMC (MCMC-HFM). We tested MCMC-HFM first for a kinetic toy model with a positive feedback. Inferring kinetic parameters mainly related to the positive feedback, we found that MCMC-HFM reliably infer them with both qualitative and quantitative fitness measures. Then, we applied the MCMC-HFM to an apoptosis signal transduction network previously proposed. For kinetic parameters related to implicit positive feedbacks, which are important for bistability and irreversibility of the output, the MCMC-HFM reliably inferred these kinetic parameters. In particular, some kinetic parameters that have the experimental estimates were inferred without these data and the results were consistent with the experiments. Moreover, for some parameters, the mixed use of quantitative and qualitative fitness measures narrowed down the acceptable range of parameters. Taken together, our approach could reliably infer the kinetic parameters of the target systems.

In computational systems biology, mathematical models of gene regulatory networks or signal transduction networks are often represented by ordinary and partial differential equations. In these equations, there are kinetic parameters which characterize strengths of interactions or rates of biochemical reactions. However, all the values of kinetic parameters in the model are not always available from previous experiments and literatures. In these cases, unknown kinetic parameters need to be inferred so that the model simulation reproduces the known experimental phenomena. Parameter inference is very important for the mathematical modeling of biological phenomena, because it is known that network structures (network motifs) alone do not always determine the response or function of that network

In this respect, Bayesian statistics is a powerful method for parameter inference giving us the information about credibility and uncertainty of unknown parameters as a credible interval of posterior distribution. However, posterior distributions in Bayesian statistics are often difficult to obtain analytically. In these cases, Markov chain Monte Carlo methods (MCMC)

Biological experiments are often performed with cell population, and the results are represented by histograms. For example, delay time and switching time of caspase activation after TRAIL treatment in apoptosis signal transduction pathway were represented by histograms

To overcome this problem, we formulated Bayesian formula for hybrid fitness measures (HFM) and implemented it to MCMC. We named the method MCMC-HFM which can deal with the mixture of qualitative and quantitative fitness measures. We first tested the MCMC-HFM to a kinetic toy model with a positive feedback. Starting with an assumed set of parameters that satisfies qualitative condition, we generated kinetic data with some noise. Using the generated data and qualitative condition, we tried to infer the kinetic parameters mainly related to the positive feedback. As the result, MCMC-HFM could reliably infer the kinetic parameters with use of both qualitative and quantitative fitness measures. Next, we applied the MCMC-HFM to a mathematical model of apoptosis signal transduction network, which was proposed before

We explain the derivation of MCMC-HFM algorithm. In Bayesian statistics, under given a likelihood function P(x_{o}|θ) and a prior distribution π(θ), a posterior distribution π(θ| x_{o}) is represented as follows:

Here, θ is parameters and x_{o} is observed data. For computation of a posterior distribution, MCMC can be used and it generates samples from a posterior distribution. Conventional MCMC Metropolis-Hastings algorithm

MCMC algorithm

MC1. Initialize θ_{i} i = 0.

MC2. Propose a candidate value θ*∼q(θ|θ_{i}) where q is a proposal distribution.

MC3. Set θ_{i+1} = θ* with probability following α._{i+1} = θ_{i}.

MC4. If i<N, increment i = i+1 and go to MC2.

MCMC algorithm is designed as the stationary distribution is consistent with the target posterior distribution π(θ|x_{o}). As shown above, conventional MCMC needs explicit evaluation of a likelihood function P(x_{o}|θ) to judge whether a candidate value θ* is acceptable or not in step MC3. Conventional MCMC can be used in the case that the deviation between the experimental time series data and the simulated time series data is evaluated by probability distributions. For example, Eydgahi et al. used Gaussian distribution to evaluate the deviation

ABC-MCMC algorithm

ABC1. Initialize θ_{i} i = 0.

ABC2. Propose a candidate value θ*∼q(θ|θ_{i}) where q is a proposal distribution.

ABC3. Simulate a data set x*∼P(x|θ*).

ABC4. Set θ_{i+1} = θ* with probability following α._{i+1} = θ_{i}.

ABC5. If i<N, increment i = i+1 and go to ABC2.

Here, I(C) is an indicator function which I(C) = 1 if a condition C is true, and 0 otherwise. ρ is a distance function and ε is a tolerance _{o}|θ*)/P(x_{o}|θ_{i}) is approximated to I(ρ(x_{o}, x*)≤ε))/I(ρ(x_{o}, x_{i})≤ε)). Thus, likelihood ratio is coarsely approximated by 1 if simulated data and observed data are sufficiently “close”, and 0 otherwise

For this purpose, firstly, we consider the case that we have qualitatively different experiment data obtained in the same system. When we want to obtain a posterior distribution of parameters θ by n qualitatively different experiments, X_{1},…,X_{n}, the posterior distribution are represented as follows:

Here, n indicates the number of different experiments. Likelihood P(X_{1},…,X_{n}|θ) can be decomposed into a multiplication of conditional probabilities. In this manner, we can utilize a number of different experimental data. In this study, we want to use both quantitative condition i.e. experimental data represented by histogram and qualitative condition i.e. experimental data which indicate the existence of specific bifurcation pattern. Then, we utilize the above idea, a multiplication of conditional probabilities, and employ an assumption to deal with both quantitative and qualitative conditions. A posterior distribution conditioned by a multiple quantitative and qualitative conditions is assumed as follows:

Here, we assumed and changed a likelihood term to a multiplication of a number of quantitative fitness measures (f_{quant}) and qualitative fitness measures (f_{qual}). In the equation, Z indicates a quantitative condition and C indicates qualitative condition. f_{quant}(Z) is a quantitative fitness measure to a quantitative condition Z and f_{qual}(C) is a qualitative fitness measure to a qualitative condition C. Rigorously speaking, fitness measures are not conditional probabilities. They are defined functions to evaluate the fitness of simulated data to experimental data. For a concrete evaluation of fitness, above equation is changed as follows:

Here, quantitative fitness measures are changed to functions of z(θ). z is a concrete value calculated by numerical simulation under θ. Depending on a value of simulated z(θ), f_{quant} compare it to the experimental data represented by a histogram and returns a specific value. In this setting, we set f_{quant}(z (θ)) is the value of “Frequency” at the corresponding class in a histogram of experimentally observed z. Quantitative fitness measures are changed to functions of condition C. Depending on a satisfaction of a condition C under parameters θ, f_{qual} returns a specific value. As a default, we assumed f_{qual}(C) equals to I(C(θ)), indicator function. For example, when Hopf bifurcation is observed in numerical simulation under parameters θ, I(C(θ)) equals to 1, otherwise 0. In this example, a condition C equals to “existence of Hopf bifurcation”. We implemented this formula to the MCMC algorithm and then we could obtain MCMC-HFM algorithm as follows:

MCMC-HFM algorithm

HFM1. Initialize θ_{i} i = 0.

HFM2. Propose a candidate value θ*∼q(θ|θ_{i}) where q is a proposal distribution.

HFM3. Simulate whether C_{j} (j = 1∼m) are satisfied or not under θ*.

HFM4. Set θ_{i+1} = θ* with probability following α.

Otherwise set θ_{i+1} = θ_{i}.

HFM5. If i<N, increment i = i+1 and go to HFM2.

MCMC-HFM algorithm is designed as the stationary distribution is consistent with the target distribution f_{quant}(z_{n}(θ))…f_{quant}(z_{m+1}(θ))I(C_{m}(θ))…I(C_{1}(θ))π(θ), with all qualitative conditions C_{1},…,C_{m} are satisfied. This was demonstrated in

In this section, we explain the flow of parameter inference (

Flow chart of parameter inference. Dotted arrows and box with dotted line correspond to a preparation process of quantitative fitness f_{quant}(z) specifically for the kinetic toy model. Bold arrows and box with bold line correspond to a general parameter inference process by MCMC-HFM.

In the test, we set the model and qualitative condition used for parameter inference by ourselves (box with dotted line in _{quant}(z) by following procedures (dotted arrows in _{answer} satisfying qualitative condition. Then we generated distribution of observables by simulating the toy model with some noise into the model. We set the distribution of observables to f_{quant}(z). Given the toy model, assumed qualitative conditions, and a simulated quantitative fitness (box with bold line in

In the application, we employed the apoptosis model of Legewie et al. 's

As a test, we first applied MCMC-HFM to infer kinetic parameters of a simple kinetic toy model (

(A) Schematic diagram of the kinetic toy model. Y is a variable (protein). Arrows direct to Y represents production process. An arrow from Y represents degradation process. A lined circle represents a pool of Y. Equations correspond to the terms in the ordinary differential equation of the model. (B) Bifurcation diagram of the model. Red colored lines indicate stable steady states and the blue colored line indicates unstable steady state with “true” values of kinetic parameters θ_{answer}. “ks_{0}”, “ks_{0.2}”, “ks_{0.3}” and “ks_{1}” indicate the k_{s} values (k_{s} = 0, 0.2, 0.3, 1.0 respectively) used as the conditions to infer kinetic parameters. (C) Time series of [Y]. T_{e} represents “execution time of Y production”. (D) Distribution of “execution time of Y production” abbreviated as “T_{e}”. (E) Distribution of “concentration of Y at time = 100” abbreviated as “[Y]_{time = 100}”.

In the right hand side, the first term corresponds to synthesis of Y, the second term corresponds to degradation of Y, and the last term corresponds to the positive feedback. The synthesis rate constant, k_{s}, is the input to the system. When other constants are set to the values θ_{answer} = (k_{d},k_{p},K) = (1.0,1,0,0.5), and the Hill coefficient is set to n = 5, the concentration of Y (represented as [Y]) shows bistability and irreversibility (_{answer} is the three dimensional vector of parameters. We note that we did not specifically define the unit of time, [Y] and parameters for simplicity. Those two features, bistability and irreversibility, were used as qualitative conditions for the parameter inference. In the application of MCMC-HFM to this model, we inferred the three constants, k_{d}, k_{p} and K. Here, the Hill coefficient was fixed to n = 5 for simplification of the problem.

To show the efficiency of the use of hybrid fitness measures, in addition to qualitative conditions i.e. bistability and irreversibility, we needed to prepare experimental results used as quantitative fitness measures. When the input is set to k_{s} = 1.0 over a whole time-series simulation, [Y] is produced and reaches to the almost saturated level by time = 10 with θ_{answer} (_{e}” in _{answer}, we performed simulations for 10000 times by adding Gaussian noise into the model. When mean and variance of Gaussian noise was set to 0 and 1 respectively, execution time and concentration of Y at time = 100 showed variation as shown in histograms in

For parameter inference, we used four types of information, “bistability of Y” abbreviated as “B”, irreversibility of Y abbreviated as “I”, execution time of Y abbreviated as “T_{e}” and “concentration of Y at time = 100” abbreviated as “[Y]_{time = 100}”. For the application of MCMC-HFM, “B” and “I” are qualitative conditions. In contrast, “T_{e}” and “[Y]_{time = 100}” are quantitative conditions. Therefore, we assigned “B” as C_{1}, “I” as C_{2}, “T_{e}” as Z_{3}, and “[Y]_{time = 100}” as Z_{4} in MCMC-HFM algorithm (about the assignment to these symbols, see Methods section). Conditions C_{1} and C_{2} were judged by bifurcation analysis. The concrete definition of the condition C_{1} is that, there are two stable steady states and one unstable steady state when k_{s} = 0.2 (“ks_{0.2}” in _{s} = 0.3 (“ks_{0.3}” in _{s} = 1.0 (“ks_{1}” in _{qual}(C_{1}) = I(C_{1}(θ)) equals to 1, and 0 otherwise. Concrete definition of the condition C_{2} is that, in addition of the condition C_{1}, there are two stable steady states and one unstable steady state when k_{s} = 0.0 (“ks_{0}” in _{qual}(C_{2}) = I(C_{2}(θ)) equals to 1, and 0 otherwise. In this case study, we used the histogram of “execution time of Y” (_{quant}(Z_{3}) = f_{quant}(z_{3}(θ)), and the histogram of “concentration of Y at time = 100” (_{quant}(Z_{4}) = f_{quant}(z_{4}(θ)). z_{3}(θ) and z_{4}(θ) were calculated by dynamics simulation under parameters θ. In this setting, f_{quant}(z_{3}(θ)) is the value of “Frequency” at the corresponding class of calculated z_{3}(θ) in _{quant}(z_{4}(θ)) is the value of “Frequency” at the corresponding class of calculated z_{4}(θ) in _{time = 0} = 0.

The prior distributions of parameters were set to follow the uniform distributions on a common logarithmic scale. Upper bound and lower bound were set to tenfold and one-tenth of the values of “true” parameter vector θ_{answer} respectively. Uniform distribution of kinetic parameters on a logarithmic scale has been used in robustness analysis in systems biology

In the MCMC-HFM algorithm, the proposal distribution was set as the uniform distribution. Newly proposed parameter θ' is proposed by using unit random number “r” as follows.

Here, θ is a vector consisting of common logarithm of kinetic parameters, and σ_{q} was set to 0.5 in this case study. Perturbation of kinetic parameter on a logarithmic scale has also been used in robustness analysis in systems biology

In each step of MCMC-HFM, one of kinetic parameters was randomly chosen and perturbed by the proposal distribution q(θ'|θ). We performed totally 3.3×10^{6} Monte Carlo steps. The first 0.3×10^{6} steps were thrown away as the so-called burn-in period. Of the remaining 3.0×10^{6} steps, we recorded data every 3 steps, so that we collected totally 10^{6} data points for the parameter inference. From them, we can draw discrete marginal probability distributions as illustrated in

To investigate the efficiency of the use of hybrid fitness, we conducted the parameter inference tests using different combination of fitness, comparing these results. First, we used only the both two types of the qualitative conditions, i.e., “BI”. Second, we used the two types of qualitative conditions and the one quantitative condition, “BIT_{e}”. Third, we used “BI[Y]_{time = 100}”. Lastly, we used all the four conditions, “BIT_{e}[Y]_{time = 100}”.

In the present MCMC simulations, we define the representative parameter values of inference by the values at the mode, the peak of each marginal distribution (red arrows in

Probability distributions with “BI” (A), those with “BIT_{e}” (B), those with “BI[Y]_{time = 100}” (C), those with “BIT_{e}[Y]_{time = 100}” (D) and those with ”BIT_{e}[Y]_{time = 100}” with weaker noise in quantitative fitness (E). Red bars represent the “true” values or parameters. Red arrows indicate the modes.

The 95% credible intervals are represented by common logarithm of the ratio of upper bound and lower bound of 95% credible intervals. Blue bars represent the case with “BI”. Red bars represent the case with “BIT_{e}”. Green bars represent the case with “BI[Y]_{time = 100}”. Magenta bars represent the case with “BIT_{e} [Y]_{time = 100}”. Cyan bars represent the case with “BIT_{e} [Y]_{time = 100}” with weaker noise in quantitative fitness.

We first look into the parameter inference with “BI” (_{p} favored relatively larger values (

Next, we address the parameter inference with “BIT_{e}” (_{d} and K. The variability of k_{d} and K clearly decreased (_{d} and K inference get higher by the usage of quantitative condition, “T_{e}”. However, MCMC-HFM could not well infer the “true” value of k_{p} and the credibility of k_{p} inference did not change (_{d}) and the working threshold of positive feedback (K) needed to be restricted, but not the strength of positive feedback (k_{p}).

Next, we address the parameter inference with “BI[Y]_{time = 100}” (_{d} and K, and the variability of their distributions decreased, but not so much as “BIT_{e}” did (_{p} was still not so well inferred and had low credibility of inference (_{e}” and “[Y]_{time = 100}”, did not well infer k_{p}.

However, when both of quantitative conditions were used with qualitative conditions “BI”, MCMC-HFM could infer all three parameters well (red arrows in _{e}” restrict k_{d} and K as shown in _{time = 100}” could restrict k_{p}. Thus, to infer the strength of positive feedback k_{p} with high credibility, the usage of both quantitative fitness measures is necessary.

If we changed the histograms in

Taken together, MCMC-HFM estimated the “true” values of kinetic parameters θ_{answer} very well with use of hybrid fitness measures. In addition, we could confirm that kinetic parameters were inferred to reproduce the histograms of “T_{e}” and “[Y]_{time = 100}” (

As an application, we applied MCMC-HFM to infer kinetic parameters of the previously constructed Legewie et al. 's mathematical model of apoptosis signal transduction network (_{e} in _{s} (

(A) Schematic diagram of the model. Solid arrows represent mass flows. Dotted arrows represent enhancement of the processes. One-way arrows between components represent irreversible processes. Two-way arrows between components represent reversible processes. Apaf-1 “A” is an input stimulus, and activated caspase-3 “C3*” is an output. Abbreviations are as follows: A: Apaf-1, C9: caspase-9, C3: caspse-3, X: XIAP. (B) Simplified diagram of the apoptosis signal transduction network at cytoplasm. Arrows represent activations. Lines with horizontal bar represent inhibition by binding and sequestering. Red colored interactions are implicit positive feedbacks. (C) Bifurcation diagram of the model. Red colored lines indicate stable steady states and the blue colored line indicates unstable steady state with the set of kinetic parameters used in Legewie et al's study. “A_{0.001}”, “A_{2}” and “A_{20}” indicate the Apaf-1 concentrations (Apaf-1 = 0.001, 2.0, 20.0 respectively) used as the conditions to infer kinetic parameters. (D) Time series of active caspase-3 (C3*) with kinetic parameters used in Legewie et al's study. T_{s} represents “switching time of caspase-3 activation”. T_{e} represents “execution time of caspase-3 activation”. (E) Assumed function of switching time of caspase-3 activation. (F) Assumed function of execution time of caspase-3 activation.

In Legewie et al.'s model, input stimulus is Apaf-1 (represented as “A” in _{asso}) between XIAP and five caspases which are directly related to implicit positive feedback. Here, dissociation rate constants of XIAP-caspase complexes and other kinetic parameters were fixed to the values in their paper

For parameter inference, we used four types of information, “bistability of caspase-3” abbreviated as “B”, “irreversible activation of caspase-3” abbreviated as “I”, “switching time of caspase-3 activation” abbreviated as “T_{s}”, and “execution time of caspase-3 activation” abbreviated as “T_{e}”. For the application of MCMC-HFM, “B” and “I” are qualitative conditions. On the other hand, “T_{s}” and “T_{e}” are quantitative conditions. Therefore, we assigned “B” as C_{1}, “I” as C_{2}, “T_{s}” as Z_{3} and “T_{e}” as Z_{4} in MCMC-HFM algorithm.

Conditions C_{1} and C_{2} were judged by bifurcation analysis. The concrete definition of the condition C_{1} is that, there are two stable steady states and one unstable steady state when Apaf-1 = 2.0 [nM], and active caspase-3 concentration of lower stable steady state is below 1.0 [nM] and higher stable steady state is over 1.0 [nM] (“A_{2}” in _{20}” in _{0.001}” in ^{6}–10^{7} molecules of cellular substrate within several hours when a cell volume is 1 picoliter _{qual}(C_{1}) = I(C_{1}(θ)) equals to 1, and 0 otherwise. Concrete definition of the condition C_{2} is that, in addition of the condition C_{1}, there are two stable steady states and one unstable steady state when Apaf-1 = 0.001 [nM], and active caspase-3 concentration of lower stable steady state is below 1.0 [nM] and higher stable steady state is over 1.0 [nM] (“C_{0.001}” in _{qual}(C_{2}) = I(C_{2}(θ)) equals to 1, and 0 otherwise.

Quantitative conditions, “switching time of caspase-3 activation” and “execution time of caspase-3 activation” were calculated by dynamics simulation. The concrete definition of “switching time of caspase-3 activation” is that the duration from the time when active caspase-3 concentration reached the 2.5% of its maximum value to the time when active caspase-3 concentration reached the 97.5% of its maximum value. The maximum value of active caspase-3 was chosen as the maximum concentration in each dynamics simulation and bifurcation analysis. The mean values of switching time in cell population are 19 to 27 minutes and standard deviations are 7.7 to 13 minutes, slightly differ depending on the strength of apoptotic stimulus, TRAIL concentration, as experimentally shown in Albeck's study

Here μ_{s} and σ_{s} were set as the expected value of z_{3} equals to 23 minutes and the standard deviation equals to 10 minutes (_{3}(θ), is a variable of the quantitative fitness measure. The concrete definition of “execution time of caspase-3 activation” is that the time when active caspase-3 concentration reached the 90% of its maximum value in each dynamics simulation or bifurcation analysis. Experimentally, it is known that active caspase-3 level rises rapidly over a 10∼20 minutes period after MOMP

Here μ_{e} and σ_{e} were set as the expected value of z_{4} equals to 15 minutes and the standard deviation equals to 3 minutes, as most of parameter vectors reproduce 10∼20 minutes for execution time of caspase-3 (_{4}(θ), is a variable of the quantitative fitness measure.

In bifurcation analysis, steady states concentrations of all proteins were calculated by solving the simultaneous equations obtained by setting all the ordinary differential equations equals to zero with the standard Newton-Raphson method. Local stabilities of all the steady states were determined by evaluating eigenvalues of Jacobian matrices which were obtained by linearization of ordinary differential equations. In dynamics calculation, the ordinary differential equations were numerically solved by the fourth-order Runge-Kutta method with a time step of 0.01. Total calculation time was 500 (50000 steps). Initial condition of the apoptosis model in dynamics calculation was shown in

The prior distributions of parameters were set to follow the uniform distributions on a common logarithmic scale. Upper bound and lower bound were set to tenfold and one-tenth of the values used in Legewie et al.'s paper

In MCMC-HFM algorithm, the proposal distribution was set as the uniform distribution. Newly proposed parameter θ' is proposed by using unit random number “r” as follows.

Here, θ is a vector consisting of common logarithm of kinetic parameters, and σ_{q} was set to 1.0 in this case study.

In each step of MCMC-HFM, one of kinetic parameters was randomly chosen and perturbed by the proposal distribution q(θ'|θ). We performed totally 5.5×10^{6} Monte Carlo steps. The first 0.5×10^{6} steps were thrown away as the so-called burn-in period. Of the remaining 5.0×10^{6} steps, we recorded data every 5 steps, so that we collected totally 10^{6} data points for the parameter inference. From them, we can draw discrete marginal probability distributions as illustrated in

To investigate roles of each type of fitness on parameters, we conducted the parameter inference using one to four types of fitness, comparing these results. First, we used only the first type of qualitative condition, i.e., “B”. Second, we used the two types of qualitative conditions, “BI”. Third, we used the two types of qualitative conditions and the one quantitative condition, “BIT_{s}”. Fourth, we used “BIT_{e}”. Lastly, we used all the four conditions, “BIT_{s}T_{e}”.

In the present MCMC simulations, we also define the representative parameter values of inference by the values at the mode, the peak of each marginal distribution (red arrows in _{asso} (X-C3*)) and k_{asso} (X-C9) are experimentally characterized _{asso} (X-C9) in the reference

Probability distribution with “B” (A), that with “BI” (B), that with “BIT_{s}” (C), that with “BIT_{e}” (D), and that with ”BIT_{s}T_{e}” (E). Red bars represent experimentally estimated values. Red arrows indicate the modes.

The same as captions in

The same as captions in

The same as captions in

The same as captions in

Specifically, we first look into the parameter inference of k_{asso} (X-C3*). _{s}” and “BIT_{e}” (_{s}T_{e}” inferred the experimental value perfectly (

Next, we address the parameter inference of k_{asso} (X-C9). The inference with “B” or “BI” by MCMC-HFM (_{s}” also showed similar deviation (_{e}” and with “BIT_{s}T_{e}” narrowed the parameter range and the mode (the red arrow) is very close to the experimental value (

Next, as for k_{asso} (X-AC9), both of MCMC-HFM with “B”, “BI” and “BIT_{s}” provided nearly uniform distribution without providing any information (_{e}” and “BIT_{s}T_{e}” disfavored values larger than ∼0.06 although they still accept any values lower than ∼0.06. The mode was smaller than the value assumed in Legewie et al's paper (_{asso} (X-AC9) was not determined experimentally and thus it is difficult to conclude if the inference was succeeded or not. At least, MCMC-HFM with “BIT_{e}” and “BIT_{s}T_{e}” narrowed down the range of k_{asso} (X-AC9).

k_{asso} (X-C9*) were quite well inferred in all cases (_{asso} (X-C9*) was not determined experimentally, the inference by the current MCMC-HFM simulations strongly suggest that 0.06, the value used in the previous work, would be a good choice.

k_{asso} (X-AC9*) was inferred similarly by all the five simulations (

Taken together, MCMC-HFM estimated experimentally estimated kinetic parameters, k_{asso} (X-C3*) and k_{asso} (X-C9), perfectly in consistent with experimental values.

Next, we checked whether the switching time and execution time of caspse-3 activation are consistent with the assumed functions shown in

About the switching time of caspase-3 activation, T_{s}, as shown in _{s}. However, the calculated histogram is not consistent with the approximated histogram of the assumed function (red outline box histogram in _{s}” and “T_{e}”, the distribution got sharper but the positions of peaks do not change (

Histogram of switching time of caspase-3 activation calculated with “B” (A), that with “BI” (B), that with “BIT_{s}” (C), that with “BIT_{e}” (D), and that with “BIT_{s}T_{e}” (E). Blue bars represent calculated results. Red outline box bars represent the approximated histogram of the function shown in

About the execution time of caspase-3 activation, T_{e}, as shown in _{e}” was considered. Over 90% parameter sets showed 10∼20 minutes for caspase-3 activation. In the cases with “BI” and with “BIT_{s}”, only about 20% parameter sets showed 10∼20 minutes for caspase-3 activation and calculated histograms are clearly far from the approximated histogram of the assumed function (_{s}” assumed based on experimental results _{e} “, assumed based on experimental results

Histograms of execution time of caspase-3 activation calculated with “B” (A), that with “BI” (B), that with “BIT_{s}” (C), that with “BIT_{e}” (D), and that with “BIT_{s}T_{e}” (E). Blue bars represent calculated results. Red outline box bars represent the approximated histogram of the function shown in

Lastly, to quantify the acceptable range of parameters, we calculated 95% credible intervals of each inferred parameter. Upper bound and lower bound of 95% credible interval were defined as external regions of bounds contain 2.5% data respectively. Common logarithm of the ratio of upper bound and lower bound of 95% credible interval was shown in _{asso} (X-C3*), k_{asso} (X-C9) and k_{asso} (X-AC9) showed differences among the five MCMC simulations, while others did not show clear difference.

The 95% credible intervals are represented by common logarithm of the ratio of upper bound and lower bound of 95% credible intervals. Blue bars represent the case with “B”. Red bars represent the case with “BI”. Green bars represent the case with “BIT_{s}”. Magenta bars represent the case with “BIT_{e}”. Cyan bars represent the case with “BIT_{s}T_{e}”.

The 95% credible intervals of k_{asso} (X-C3*) became narrower by additional information of “I”. As seen in _{asso} (X-C3*) got unfavorable and higher values got favorable by additional information of irreversibility. This might be explained as follows. For irreversible activation of caspase-3, caspase-9 needs to activate caspase-3 constantly, and switch-on states of positive feedbacks needs to be sustained. For constant activation of caspase-3, a certain amount of caspase-9 has to be dissociated from XIAP. Thus, k_{asso} (X-C3*) tended to favor higher values to attract XIAP and 95% credible interval became narrower.

The 95% credible intervals of k_{asso} (X-AC9) and k_{asso} (X-C9) became narrower by additional information of “T_{e}”. This result indicates that, free AC9 and, more dominantly, free C9 determine the timing of caspase-3 activation after MOMP (MOMP is at time = 0 minutes in our simulation). This is actually consistent with our intuition. After Apaf-1 input, firstly, C9 and AC9 activate C3 to C3*. Then two positive feedbacks, one is the implicit positive feedback, and the other is the positive feedback that C3* activates C9 to C9* and AC9 to AC9* (

In this manner, the current parameter inference process provides us lessons on which parameters are important for specific system properties. In addition, investigation about correlation coefficients and joint probability distributions between inferred parameters also provides us the relationships between parameters and specific system properties (See

In the two experimentally determined parameters, k_{asso} (X-C9) had narrow interval but k_{asso} (X-C3*) had still wide intervals. A wide credible interval of posterior distribution indicates that the information for parameter inference was not enough only with the information of bistability, irreversibility, switching time of caspase-3 activation and execution time of caspase-3 activation. As experimental results on the target system properties increases, the credible interval will become narrower and better inference will be accomplished.

In the present study, we introduced functions to evaluate fitness to experimental results, named fitness measures. Then we formulated Bayesian formula for hybrid fitness measures. We implemented it and developed MCMC-HFM algorithm to deal with a mixture of quantitative and qualitative fitness measures. We tested the MCMC-HFM algorithm for parameter inference in the kinetic toy model and the mathematical model of apoptosis signal transduction network. In the former, we inferred the kinetic parameters mainly related to positive feedback. As a result, MCMC-HFM could reliably infer the kinetic parameters with use of hybrid fitness measures. In the apoptosis model, we inferred the kinetic parameters which are related to the implicit positive feedback. As a result, MCMC-HFM could reliably infer the kinetic parameters, especially those of which values were experimentally estimated

In the apoptosis model, by the credible intervals of inferred parameters, joint probability distributions and correlation coefficients between inferred parameters, we could also specify the important relationships between kinetic parameters and corresponding biochemical processes, especially for irreversibility and execution time of caspase-3 activation. In the process of parameter inference by Bayesian statistics with MCMC, we can usually obtain many parameter sets, which can be used to understand and specify important biochemical processes in the target system as shown in the current study.

In the apoptosis model, inferred parameter sets reproduced well the assumed function of execution time of caspase-3 (_{s}”, calculated histogram of T_{s} was not consistent with the assumed function (_{asso} (X-AC9), k_{asso} (X-C9*) and k_{asso} (X-AC9*) in our parameter inference simulations. If we tried to infer all the unknown parameters in the model, the assumed function of the switching time of caspase-3 might be reproduced. Otherwise the mathematical model might need to be improved to be able to reproduce experimental results shown by Albeck et al.

Of the conditions used for parameter inference in the apoptosis model, “T_{s}” did not largely narrow the distributions of any kinetic parameters (_{e}” has the information about “T_{s}” because of its definition (

Robustness analysis of kinetic parameters in systems biology sometimes assumes the size of the parameter space as the measure of robustness. For example, the volume of the ellipsoid containing 95% of the parameters generated by Monte Carlo method was calculated and assumed as the measure of robustness _{asso} (X-AC9) showed wide credible interval and roughly uniform distribution in the case with information “B” (_{asso} (X-C9*) (

In the same way as the MCMC-HFM algorithm, the idea to deal with mixture of quantitative and qualitative fitness measures simultaneously can be applied to SMC or so-called population Monte Carlo methods

_{p} using “BI”. (B) Trace plot and the probability distribution of k_{p} using “BIT_{e}[Y]_{time = 100}”. Dotted lines indicate the 300000th step, of which left side is the burn in period.

(PDF)

_{e}” and concentration of Y at time = 100 “[Y]_{time = 100}”._{e}”, and concentration of Y at time = 100, “[Y]_{time = 100}” calculated with “BI” (A-a) and (A-b) respectively, those with “BIT_{e}” (B-a) and (B-b) respectively, those with “BI[Y]_{time = 100}” (C-a) and (C-b) respectively, those with “BIT_{e}[Y]_{time = 100}” (D-a) and (D-b) respectively, and those with “BIT_{e}[Y]_{time = 100}” with weaker noise in quantitative fitness (E-a) and (E-b). Blue bars represent calculated results. Red outline box bars represent the histogram generated by adding Gaussian noise into the model.

(PDF)

_{asso} (X-C9) with “BI”. (B) Trace plot and the probability distribution of k_{asso} (X-C9) with “BIT_{s}T_{e}”. Dotted lines indicate the 500000th step, of which left side is the burn in period.

(PDF)

_{asso} (X-C9) and k_{asso} (X-AC9)._{s}” (C), that with “BIT_{e}” (D), that with “BIT_{s}T_{e}” (E).

(PDF)

_{asso} (X-C9*) and k_{asso} (X-C3*)._{s}” (C), that with “BIT_{e}” (D), that with “BIT_{s}T_{e}” (E).

(PDF)

(PDF)

_{e}” (B), those with “BI[Y]_{time = 100}” (C), those with “BIT_{e}[Y]_{time = 100}” (D), and those with ”BIT_{e}[Y]_{time = 100}” with weaker noise in quantitative fitness (see main text) (E). Red bars represent the “true” values of parameters. Red arrows indicate the modes.

(PDF)

_{e}”. Green bars represent the case with “BI[Y]_{time = 100}”. Magenta bars represent the case with “BIT_{e} [Y]_{time = 100}”. Cyan bars represent the case with “BIT_{e} [Y]_{time = 100}” with weaker noise in quantitative fitness.

(PDF)

_{asso} (X-C3*) (proposal distribution in MCMC set to normal distribution)._{s}” (C), that with “BIT_{e}” (D), and that with ”BIT_{s}T_{e}” (E). Red bars represent experimentally estimated values. Red arrows indicate the modes.

(PDF)

_{asso} (X-C9) (proposal distribution in MCMC set to normal distribution)._{s}” (C), that with “BIT_{e}” (D), and that with ”BIT_{s}T_{e}” (E). Red bars represent experimentally estimated values. Red arrows indicate the modes.

(PDF)

_{asso} (X-AC9) (proposal distribution in MCMC set to normal distribution)._{s}” (C), that with “BIT_{e}” (D), and that with ”BIT_{s}T_{e}” (E). Green bars represent the used value in Legewie et al's study but not experimentally estimated. Red arrows indicate the modes.

(PDF)

_{asso} (X-C9*) (proposal distribution in MCMC set to normal distribution)._{s}” (C), that with “BIT_{e}” (D), and that with ”BIT_{s}T_{e}” (E). Green bars represent the used value in Legewie et al's study but not experimentally estimated. Red arrows indicate the modes.

(PDF)

_{asso} (X-AC9*) (proposal distribution in MCMC set to normal distribution)._{s}” (C), that with “BIT_{e}” (D), and that with ”BIT_{s}T_{e}” (E). Green bars represent the used value in Legewie et al's study but not experimentally estimated. Red arrows indicate the modes.

(PDF)

_{s}”. Magenta bars represent the case with “BIT_{e}”. Cyan bars represent the case with “BIT_{s}T_{e}”.

(PDF)

_{s}”. Red outline box bars represent the approximated histogram of the function shown in

(PDF)

(DOC)

(DOC)

(DOC)

(DOC)

We thank Kaiichiro Ota for fruitful discussions.