^{1}

^{2}

^{*}

^{2}

^{2}

All authors contributed equally in all aspects of the paper.

The authors have declared that no competing interests exist.

Discrete stochastic simulations are a powerful tool for understanding the dynamics of chemical kinetics when there are small-to-moderate numbers of certain molecular species. In this paper we introduce delays into the stochastic simulation algorithm, thus mimicking delays associated with transcription and translation. We then show that this process may well explain more faithfully than continuous deterministic models the observed sustained oscillations in expression levels of

Delay processes are ubiquitous in the biological sciences but are not always well-represented in mathematical models attempting to describe these biological processes. Additional issues arise when attempting to capture the uncertainty (intrinsic noise) associated with chemical kinetics in dealing with when and in what order reactions take place. Complicating the situation further are important instances when certain key molecules occur only in small numbers, so that it is not meaningful to talk about concentrations.

In this paper Barrio et al. show how to incorporate delay, intrinsic noise, and discreteness associated with chemical kinetic systems into a very simple algorithm called the delay stochastic simulation algorithm (DSSA). This algorithm very naturally generalises the stochastic simulation algorithm that does not treat delays. The authors then apply the DSSA to a specific set of experiments performed by Hirata et al. who showed, amongst other things, that serum treatment of cultured cells induces cyclic expression of both mRNA and protein of the Notch effector Hes1 with a two-hour period. The authors show how this approach can explain additional experiments performed by Hirata et al., and, because this approach is very general, suggest that it can provide deep insights into the relationship between delayed processes, intrinsic noise, and small numbers of molecules in many biological systems.

The mathematical modelling and simulation of genetic regulatory networks can provide insights into the complicated biological and chemical processes associated with genetic regulation. However, it is important that the models are kept simple but nevertheless capture the key processes. In addition, by incorporating experimental data into such models (where available) their accuracy can be improved.

An important aspect associated with genetic regulation is that mRNA and protein expression levels can be quite low, and so continuous models, as described by ordinary differential equations, may be inappropriate. Furthermore, processes such as transcription and translation do not occur instantaneously and may have considerable delays associated with them. It is these two issues that we pursue in terms of understanding oscillatory expression levels of both mRNA and protein of the Notch effector Hes1.

There are many types of molecular clocks that regulate biological processes, but apart from circadian clocks [

Specifically, the data presented in the paper by Hirata et al. (

Hirata et al. examined the underlying mechanisms for the observed oscillations and showed that in the presence of the proteasome inhibitor MG132,

To explain the observed behaviour, Hirata et al. modify a mathematical model developed by Elowitz and Leibler [

This observation together with the experimental results of Hirata et al. led to a number of papers in which simple coupled delay differential equations (DDEs) representing

In fact, one of the first people to consider feedback differential equation models for the regulation of enzyme synthesis was Goodwin [

The ideas underpinning these works are that the processes of transcription, translation, and export are not instantaneous. Monk notes that there is an average delay of 10–20 min between the action of a transcription factor on the promoter region of a gene and the appearance of the corresponding mRNA in the cytosol. Similarly, there is a delay of typically 1–3 min for the translation of a protein from mRNA. Note that the model proposed by Lewis is for zebrafish but it does offer insights into the Hes1 mechanisms via the general nature of the model.

These papers were able to explain some of the observed experimental results quite well, but there are still some aspects that these delay continuous models do not address. These aspects relate to the fact that production numbers of mRNA and protein can be quite low and that intrinsic noise effects due to the uncertainty in knowing when a reaction and what reaction takes place in any given time interval can be very important. Thus the aim of this paper is to incorporate delay effects into the discrete stochastic simulation algorithm (SSA) of Gillespie [

Let

Here _{m}_{p}_{m}_{p}_{0} is such that _{0}) = 1/2.

Bernard et al. [_{q}

The astute reader might ask what happens if the transcriptional and translational delays, _{m}_{p},

But if we let _{m}_{p}_{m}

Monk and Jensen et al. investigate the dynamics of model 1, especially in terms of the onset of sustained oscillations, through simulations; while Bernard et al. give a mathematical investigation of the dynamics of models 1 and 2 by a scaling process and performing a linear stability analysis around the steady state values. Lewis shows that, given

Bernard et al. have given a detailed mathematical bifurcation analysis of the dynamics of the models 1 and 2. In the case of model 1 let the time period of the oscillations be T and define

It is clear that h, _{m},_{p}_{m}, α_{p},_{0} play no significant role. Using the data from the Hirata et al. experiments, we can now see how well model 1 matches the observed results.

We now give a brief analysis of model 1 and see how well it can describe the results of Hirata et al. and what sort of predictions it can make.

Using some of the values in

Model Parameters Used for DDE and DSSA

Jensen et al. show via simulations that for the case

Simulations and mathematical analysis show that there is no qualitative difference in terms of the onset of sustained oscillations and their period for a wide range of values for _{m}, α_{p},_{0}. Monk has performed a linear stability analysis and has shown that for a certain range (that is quite wide) of these parameters the oscillatory period is approximately constant.

It is interesting to note that when Hirata et al. lowered the temperature from 37 °C to 30 °C there was a change in the period of oscillation, but as the data is not given we are unable to test these effects in terms of model 1 except to reaffirm that the model is not sensitive to the production terms.

More significantly, in the continuous model, the Hes1 protein concentration rarely falls below the repression threshold _{0}, which means that Hes1 transcription is always repressed. While this does not contradict experimental data, as there is no mention of this threshold, it does mean that there is not a strong link between the continuous model and the actual mechanism of transcription.

Bernard et al. note that systems with just one nonlinear term often display large overshoot before solutions converge to an attractor. Indeed, that is one of the reasons why they introduce model 2. They attempt to estimate this overshoot, which is essentially due to the lack of repression mechanisms in the first few minutes. Defining the overshoot to be the ratio of the protein concentration at

With _{m}_{m}

However, the overshoot only becomes an issue with simulations if the initial conditions are set close to zero. Monk avoids this large overshoot in his simulations by setting the initial conditions close to their steady state values. The real issue here, however, is of course how the oscillations are set off within a cell by serum treatment. The Hirata data does not show any overshoot and so there is a need to relate the initial conditions of any model to the experiment itself. Inevitably, this treatment will change one or more of the model parameters, perhaps continuously. Thus more work needs to be done to understand how serum treatment induces oscillations before we can address this issue more appropriately from a modelling perspective.

Deterministic, continuous models do not match very well the peak-to-trough ratios observed by Hirata et al. Indeed, for model 1 this ratio is higher for mRNA than protein, and this appears to contradict the Hirata data.

We acknowledge that it is often hard to compare experimental and simulated results for the purposes of model validation. However, we note that Hirata et al. performed some experiments in terms of blocking protein degradation and translation, and while model 1 has not been tested in this regard we do attempt to mimic these experimental results in this present paper through our use of discrete models.

In summary, model 1 predicts possibly high Hill factors, overshoot (if the initial conditions are not chosen very carefully), and no obvious link between the values of _{0} and the actual physical basis of transcription. The model is also very sensitive to the degradation parameters but not sensitive to the production parameters. Thus, it can explain some (but not all) of the Hirata data.

However, there are two fundamental issues that the model does not address and which could well explain some of the discrepancies mentioned above. These are that mRNA and proteins can be expressed in quite small numbers and that there is intrinsic noise in terms of the uncertainty of knowing when a certain reaction and what reaction takes place. These points lead us into a discussion on discrete stochastic models for chemical kinetics and the SSA. This in turn will lead to the main idea of this paper, namely the incorporation of delays into discrete, stochastic models and how this approach may address the issues raised here.

Key molecules that are produced at low levels and a chemical systems' intrinsic noise led to Gillespie (1977) [

Let there be _{1},…,_{m}, and propensity functions _{1}(_{2}(_{m}(_{1}(_{N}^{T},_{i}_{j}_{1} + _{2} _{3}, it is _{1}(_{2}

The underlying idea behind the SSA is that at each time point

See Algorithm 1 for a pseudo-code description of the SSA.

The SSA has been used successfully in many settings (e.g., Arkin et al. [

Other approaches for improving the performance of SSA are based on the chemical master equation (CME) that describes the evolution of the probability density function

It is possible to cast this problem into the form

We note that an approximation to the mean behaviour

Given this overview of SSA, our intention is now to introduce delays into SSA and to investigate the dynamics of model 1 in this setting. Unlike the SSA, there is not necessarily a unique implementation of delay SSA (DSSA), and issues pertaining to this are discussed in more detail in

Briefly, DSSA implementations can differ in the way they handle (1) the waiting time for delayed reactions, (2) the time steps in the presence of delayed reaction updates, and (3) delayed consuming reactions. The DSSA version we used to produce the results presented in the following section works as follows: initially we specify which nonconsuming reactions are delayed and the delay size (constant or variable) associated with each reaction. Delayed consuming reactions are not allowed. Simulations proceed by drawing reactions and their waiting times (for delayed and nondelayed reactions). If a nondelayed reaction is selected, then the state is updated in the standard way (SSA), but if it is a delayed reaction that is selected then it is not updated until the appropriate time point would be passed by another simulation step. In this case, the last drawn reaction is ignored and instead the state is updated according to the delayed reaction. Simulation continues at the corresponding time point. Algorithm 2 shows a pseudo-code description of the DSSA implementation.

In general, delays in time evolutions are difficult to handle because of the non-Markovian character they introduce into the dynamical process. In this context we note that our DSSA implementation ignores the elapsed time between the last triggered reaction and the update of the next scheduled delayed reaction. It is unclear whether this affects the distribution of waiting times until the next reaction happens. It also ignores the selected reaction that should be updated beyond the current update point by preferentially updating the delayed reaction. However, it is an open question whether we should select for the delayed reaction and ignore the other. For further discussions we refer the reader to

Furthermore, we note that as soon as we introduce delays into SSA then the evolution of

Model 1 can be presented in DSSA form with four reactions defined by

We note that the time step we use for DSSA is self-selecting based on the assumption of exponential waiting times, as is the case for SSA. The stiffer the kinetics system becomes (due to large rate constants and/or large numbers of molecules), the smaller the time step. Thus, the algorithm intrinsically controls the stability of the evolution. However, in the case of the continuous DDE representation, an important issue is stepsize selection for any numerical method to avoid instabilities in the computed solutions.

In this section we present a selection of DDE solutions and DSSA trajectories displaying the dynamical properties of model 1. As for the DSSA, what we present are single simulations of just one particular strong solution based on a particular path generated by the random variables. Nevertheless, these individual solutions are very representative of the dynamics of the processes being modelled. In some cases, we perform a number of independent simulations to collect information about mean behaviour.

All DDE plots were generated using the _{p}_{0} = 10, _{0} = 10,

In _{0} can affect these dynamics. For values of _{0} is increased to 100 oscillations damp for values of _{0}.

We now consider the dynamics of the DSSA. If not stated otherwise, the initial molecular numbers of mRNA and protein are _{0},

In _{0} and _{0} = 100, _{0}, albeit for only small periods of time. However, for small delay (

The horizontal dotted line marks _{0}, mRNA is represented by the solid line, and protein by the dashed line.

In _{0} = 50,

The horizontal dotted line marks _{0}, mRNA is represented by the solid line, and protein by the dashed line.

Simulations in _{0} = 100) and (_{0} = 50), respectively. Since the values of _{0} might well be a significant factor in determining the dynamics of the system, we simulate also with varying _{0} (10,50,100,1000) choosing parameters (_{0} = (10,50,100). However, if _{0} is very large (_{0} = 1,000), the oscillations are very irregular and the numbers of protein are much larger than in the other cases. On the other hand, if _{0} is low (_{0} = 10), the amplitudes of mRNA and protein are not as large as in the other cases.

The horizontal dotted line marks _{0}, mRNA is represented by the solid line, and protein by the dashed line.

The data shown by Hirata et al. represents the average of the samples from a number of cells. We computed the time-dependent arithmetic mean over 1,000 independent simulations using the DSSA with _{0} = 100,

In addition, by performing numerous simulations for values (_{0},

In this subsection we compare our simulations with specific experiments performed by Hirata et al. One of the most important aspects of the Hirata data is the regularity of the oscillatory period, which is 2 h. We therefore performed a spectrum analysis (more than 300 independent simulations) that takes a signal in the time domain and transforms it into its component frequency representation (frequency analysis has been done using a software package provided by Barrio et al., see Acknowledgements).

The oscillation frequencies can be determined for different values of the parameters: _{0}, _{0} = 100,

Left, more than 12 h. Right, more than 2 h. The dotted horizontal line corresponds to _{0}. Solid lines represent mRNA, dashed lines protein populations.

By perturbing the reaction rate constants, we can attempt to get an idea of the system's sensitivity (we note that Hirata et al. observed alterations in the oscillatory period as the temperature was lowered). This approach does not replace a thorough analysis. However, it still leads to insights about the different dynamics and the sensitivity of the model. _{0} = 100, _{m}_{p}_{m}_{p}_{m} = 0_{p} =_{m} =_{p} =_{m}_{p}_{m} = α_{p} =_{m} = α_{p} =_{p} =_{m}_{p} =_{m}_{m}_{p},_{p},_{p}_{m} =_{p} =_{m} =_{p} =

(A) _{m}

(B) _{p}

(C) _{m}

(D) _{p}

Finally, we compare the results of some actual experiments by Hirata et al. with the corresponding modified DSSA simulations. The experiment in which Hes1 protein degradation is blocked by application of proteasome inhibitor MG132 is mimicked by setting the fourth stoichiometric vector to _{4} = (0,0)^{T},_{exp}_{0} = 100, and _{exp} =^{T}_{0} = 100, and _{exp} =

When we compare the dynamics of DSSA with the continuous delay case, we can make a number of important conclusions. Perhaps the most significant is that there are sustained oscillations for values of

The same remarks apply for estimates of the values of _{0},

Another feature of the dynamics of DSSA that we would like to emphasise is the role of _{0}. For continuous models, the role of _{0} appears not to be too significant as long as it is not too large. But from _{0} plays an important role. Apparently, when the numbers of _{0}, there is expression. This expression only occurs for very small time windows but seems to be crucial in driving the oscillations. This behaviour does not occur for the continuous deterministic models. If the value of _{0} is increased too much to _{0} = 1,000, say, then there are no oscillations and _{0}. On the other hand, if _{0} is too low (_{0} = 10), then the amplitudes of the mRNA and protein appear to be too small. This provides a prediction that should be able to be tested experimentally.

Furthermore, simulations in

We have also shown from the mathematical analysis in the supporting information (

Furthermore, our simulations suggest that overshoot is not an issue for model 1 in a discrete delay setting. One of the reasons that Bernard et al. [

The sensitivity analysis in

(A) Mimicking blocking of Hes1 protein degradation by a proteasome inhibitor.

(B) Mimicking inhibition of translation by cycloheximide treatment.

Putting all this information together we see that we get very good comparisons between simulation and experiment if the value of _{0} is on the order of 50 to 100, if the value of

In this paper we have compared continuous delay models and discrete, stochastic delay models to explain oscillations in numbers of

By careful comparisons of our simulations with the Hirata et al. data, we have been able to suggest quite specific ranges for _{0}, the Hill parameter

(89 KB PDF)

(45 KB PDF)

The Entrez Gene (

The authors would like to thank Nick Monk (Sheffield) for helpful discussions that substantially improved this paper. The authors would also like to thank Margherita Carletti (Urbino) who worked with the second author on a preliminary version of discrete delay code.

chemical master equation

delay differential equations

delay stochastic simulation algorithm

stochastic simulation algorithm