To illustrate some of the unique benefits of PyDSTool to the modeler, we will discuss an example that combines elements of the workflows (1) and (3) above using reduction techniques and hybrid dynamical systems (see [37] and references therein). Hybrid dynamical systems are defined later, but loosely speaking they are made up of smooth vector fields (such as those defined by differential equations) that are punctuated by discrete changes. This makes them especially useful for modeling systems that display modularity in functional state or inherent structure. We will discuss a measurement of modularity in Results.

In contrast to dimension reduction methods that aim to increase simulation efficiency (e.g., see [38]), reduction to hybrid system models has been demonstrated as a tool for inferring the key causal mechanisms underlying a high-dimensional or otherwise complex phenomenon, and for validating the resulting hypotheses with a dynamical system of lower dimension or greater simplicity (e.g., [39]–[42]). In lieu of a more developed qualitative theory of nonlinear dynamics for hybrid systems, explicitly simulated representations of the reduced dynamics are needed to ensure that the hypotheses are both logically self-consistent and consistent with experimental data. (E.g., see [43] for an example of exploring this issue in biomechanical modeling.) In addition, algorithms have recently been developed that assist with the inference of mechanistic relationships in ordinary differential equation (ODE) models, and also with the systematic and semi-automated construction of the resulting reduced descriptions in terms of hybrid systems [44]–[46].

A simple example of reducing an ODE in this way is the replacement of a smooth sigmoidal function having a steep slope with a piecewise-constant or piecewise-linear step function in contexts that are not sensitive to the details of the smooth slope (see [47]). This reduction can be understood in terms of functional modularity because, in this situation, the strong nonlinearity of the function ensures effective decoupling between the input and output when the input is sub-threshold. The validation of the reduction in an explicit context tests the hypothesis that the details of the function's transition were not mechanistically relevant. The integrate-and-fire neuron described in Supplementary Text S1 uses a similar replacement of smooth spiking dynamics with a discrete, instantaneous reset.

Multiple time scale systems such as the Van der Pol oscillator or chemical reaction systems with explicit small parameters are classic examples of systems that can be reduced to hybrid systems [48]–[50]. The models are studied as singularly perturbed systems, from which a quasi-steady state approximation and similar techniques obtain a ‘fast-slow’ reduction. This reduction is generally in the form of a set of differential-algebraic equations (DAEs) with domain consistency conditions, and can be seen as a piecewise-local dimension reduction of the model. Thus, it can be simulated and numerically analyzed using a hybrid model formalism [44].

We demonstrate hybrid model reduction in a scenario involving the space-clamped Hodgkin-Huxley (HH) formalism for neural action potentials (AP) [51]. This is a common biophysical model based on the first-order kinetics of ion transport across a cell membrane, and reflects an equation structure inherently similar to many models in systems biology (two recent examples that use PyDSTool can be found in [52], [53]). Although the HH model has been analyzed extensively by mathematical and numerical means, PyDSTool provides a novel opportunity to algorithmically derive, specify, analyze, and validate a reduced and explicit description of an AP, from which we claim that superior insight into its biophysical mechanism is possible.

PyDSTool is unique in providing a variety of high-level data types (‘classes’) and library functions that closely mimic mathematical counterparts in dynamical systems theory and provide intuitive functionality. For example, domains and numeric intervals are represented by the Interval class, for which membership, intersection, endpoint testing, etc., are simple operations. Similarly, numerical arrays are extended to become Pointsets, incorporating several features: named fields instead of indices for accessing variables, an associated independent variable that may parameterize the data, and metadata labels that can be indexed and cross-referenced. Pointsets are further abstracted to Trajectories (parameterized, smooth curves), which further add a transparent layer of domain checking and interpolation that allows numerically computed data to be treated as continuous, when appropriate. Among others, such classes provide intuitive abstractions that allow users to more efficiently express their mathematical ideas in new algorithms, or to naturally specify complex meta-model constraints.

At the lowest level, PyDSTool supports simulations of ordinary differential equations (ODEs), differential-algebraic equations (DAEs), and discrete mappings [57]. Few comparable dynamical systems packages support DAEs, which are useful in hybrid modeling, and rarely support hybrid dynamics beyond simple case-based ‘switch’ or ‘if’ statements or Heaviside functions. PyDSTool users can specify dynamics using evolution equations or explicit functions of time or state. The range of possible formalisms for specifying dynamics is supported by the Generator abstract class, which creates Trajectory objects on demand. There are several ODE solver implementations supported: the adaptive time step solvers Dopri and Radau (an implicit solver that is well-suited to stiff systems and also supports DAEs) [58], a 4th-order Runge-Kutta fixed time-step method, and a wrapping of VODE (via SciPy) [59]. All solvers support arbitrary-precision event detection with a simple Event class, which is crucial for defining hybrid systems and is missing from many application-specific simulators. PyDSTool is modular and can be extended to support other solvers.

Bundled toolboxes provide special functionality such as phase plane analysis, model reduction, optimization, data analysis, and templates and interfaces for application-specific modeling and third-party software. For instance, users who install the PySCeS [36] or SloppyCell [60] systems biology packages have the option to create models by exporting from those packages (e.g., based on SBML definitions [61]). Equally, with the NineML Python API installed [62], [63], many forms of neural models can be imported directly. Alternatively, models can be prepared directly in PyDSTool using modular constructors and symbolic expressions, or by writing raw text definitions. An export option to the ADOL-C periodic solver in Matlab is also provided [64].

We take a practical approach to implementing hybrid systems (sometimes known as composite models in other fields) in PyDSTool that is most applicable to biophysical models, where smooth dynamics are primary and are punctuated by finite numbers of discrete events. This differs from the majority of existing simulation platforms, which typically focus on physical models for engineering applications with many parallel discrete event processes mixed with smooth dynamics. Hybrid models in PyDSTool can be built from sub-models that mix discrete mappings, ODEs, DAEs, preset trajectories, or any other embedded code that can produce a Trajectory object.

There are many formalisms for hybrid systems, but the 6-tuple of Simić et al. is adequate for our purposes [65]: . We do not take a formal approach here, and it is sufficient to describe these elements informally and direct the reader to the reference for details. is a finite set of discrete states of the system, which we will refer to as regimes. The regime transition graph is given by nodes from and edges from . In each regime, a sub-model is defined from a corresponding -dimensional vector field from the set over a domain from the set . Transition events for the edges in are indicated by ‘guards’ from the set , which are dimensional sub-manifolds in each domain. We will define these guards by zero-crossing functions on those domains. Finally, there is an optional resetting map associated with each edge, taken from the set , which discretely changes the state variables on a regime transition. Consistent with some formalisms, we allow a subset (often just one) of the state variables (known as ‘indicator’ variables) to be discrete and therefore constant during each regime.

PyDSTool uses three essential code elements to define a hybrid model: (1) a hierarchy of component sub-models, (2) a mixture of zero-crossing events and global self-consistency conditions, and (3) transition rules between the sub-models that are applied on occurrence of an associated event or condition failure. During simulation of a sub-model, a terminal event may occur that stops the trajectory generation (see the Tutorial in Supplemental Text S1). In such a scenario, the transition rules for the stopped sub-model are applied to the final state to choose the next sub-model. The final state may also be mapped to a new value before becoming the initial condition for the next sub-model.

A user may define a hybrid model with only one sub-model, such that a terminal event or maximum elapsed time defined for that sub-model's regime will be associated with a transition back to the same sub-model, typically after a discrete change to the state is applied. An example of this is using a simple threshold-crossing event to signal an action potential (AP) in an integrate-and-fire neuron model [66], after which a discrete change is applied to reset the membrane potential. (This model's implementation is described further in Supplemental Texts S1 and S2.)

Figure 1 summarizes the implementation structure of hybrid models as a hierarchy of HybridModel objects: other HybridModels may reside at nodes while NonHybridModels (that are non-hierarchical and can only wrap Generators) reside at the leaves of the tree. In between sub-models and their parent are ModelInterface (MI) objects. MIs are generic wrappers that filter, transform or otherwise post-process the output of a sub-model (Figure 2). For the simplest hybrid models (see Text S1 for an example), MIs are neutral and no external conditions are necessary, but their high level of generality allows validation of hybrid model self-consistency conditions (see Results) and facilitates qualitative model optimization algorithms (see Future Directions). MIs do these tasks by encoding data- and hypothesis-driven constraints. Additionally, they can make optimization more robust using internal failure recovery. For example, recovery code can safeguard attempts by a standard optimization routine to test parameter values in the model that may be inconsistent with its definition. For instance, such code can catch a problem and return special values of the objective function rather than an error condition.

In this sub-section, we introduce a broad conceptual framework for the established analytical approach of piecewise-reduced models, and focus on the validation of the analysis by implementing the result as a hybrid systems reduction within PyDSTool.

Modularity of systems (more accurately, ‘near decomposability’) can be described as the occurrence of clusters of state variables that are highly inter-coupled but sparsely or weakly coupled to external variables [2], [67], [68]. Figure 3 schematically describes a spatial or structural form of the conceptual reduction framework used here, in which each inferred module can be analyzed, reduced and tested separately, and then further tuned in the context of the whole system. Following similar steps, Figure 4 summarizes the approach for a model that exhibits modular functional patterns that change over time. The modules form the basis of each sub-model of a hybrid model reduction. Reduction approaches are complicated when the effective decoupling between groups of variables is time- or state-dependent, as in the singularly perturbed systems mentioned earlier, and in the main example described below.

In its focus on supporting the exploration and understanding of emergent dynamics across scales, PyDSTool permits embedding of data playback, simplified model components, or even analytical tools as surrogates inside MI objects, thereby creating a hybrid model of heterogeneous component types. For instance, an embedded optimization tool could interface with regular model components to infer a low-dimensional characterization (e.g. a functional curve fit or a reduced-order model) of the dynamics of some part of a larger process [40], [41] (Figures 3 and 4). This exploratory approach can be described as ‘scaffolding,’ and permits users to focus on developing specific parts of a model in a contextually appropriate fashion. Some components may be temporary placeholders while others are refined, e.g. fitted to experimental data, such that the eventual goal may be to unify the model as an entirely ODE-based (non-hybrid) model. Similarly, scaffolding can allow model order reductions to be targeted to specific temporal or state sub-domains, increasing the degree of reduction possible. A more advanced example is replacing a dynamic variable with a surrogate time series that plays back experimental or simulated data (known as an ‘external input’ variable in PyDSTool) and ignoring feedback to it from other coupled variables.

The scaffolding idea has not been exploited in computational software previously, but is conceptually related to using code stubs and testing with surrogate data in software engineering, and to the mathematical ‘buffering’ principle for tackling complex models of biochemical systems [69].

We present a hybrid system implementation of the reduced Hodgkin-Huxley (HH) action potential (AP) derived from a ‘dominant scale’ analysis [70]. Previously, rigorous mathematical approaches to this system have yielded asymptotically-valid DAEs with only implicit dynamics for some slow variables, whereas we derive a fully explicit reduced-order model of the dynamics throughout an AP [46], [71]. Details of the ODEs for this system and their analysis can be found in these references, and for reasons of space we briefly summarize the mathematical setup. Coding details of this hybrid model specification can be found in Supplemental Texts S1 and S3.

There are four state variables in the HH model: for the membrane potential and three ionic channel gating variables, two for the fast sodium (, ) and one for delayed-rectifier potassium (). These are given by differential equations for their first order kinetics, and are only coupled with the equation for in a hub-like graph with non-symmetric coupling rules. The validity of a reduced regime determined by dominant scale analysis is determined by the truth of the defining assumptions. These include controls on the relative time scales of the variables (each available in explicit algebraic form) and the relative scales of dominance calculated as the quantities for , which includes the passive input terms for the leak conductance and an applied current ( and are dummy variables that are 0 or 1 depending on their inclusion in a regime). The are essentially sensitivities of the quasi-static resting potential of with respect to changes in each other variable, and are available in explicit algebraic form. The ‘active’ set of variables in a regime can now be defined as such that for some user-defined scale tolerance . A terminal event will indicate that a variable has left during the regime when the above inequality becomes an equality (a zero-crossing), with a similar event for variables joining . Post-processing of trajectories is required to determine exactly which variable left or joined (see Supplemental Text S1).

Dominant scale analysis of a periodic orbit indicated four reduced regimes that capture the essential dynamics of qualitatively distinct parts of the AP cycle relative to [70]. The sub-models for the regimes do not need to be decomposed further into sub-models, and so are implemented by NonHybridModels containing single Generator objects. Each sub-model contains and a different set of active inputs requiring ODEs for zero or more truly dynamic state variables, resulting in a DAE. The simplified equations and their consistency conditions form the scaffolding for this sub-model. There is no coupling to from the variables not in the active set, but we must track any changes they make because they are ‘slaved’ to . The validation method requires that we simulate the non-active ‘shadow variables’ in order to correctly determine when the sub-model's assumptions become inconsistent with the original model [44]. For this situation, PyDSTool naturally supports sub-models having differing dimension by allowing dummy variables to be included in a sub-model. Thus all sub-models must formally include all state variables in one form or other.

Taken together as a hybrid model, the piecewise-local reductions form a formal hypothesis about the key mechanistic relationships between variables during an AP. The inferred mechanistic description is formalized as a sequence of domain and transition rules called a ‘template’ (e.g., see Table 1 of [46]). Such clear and specific insights give an advantage for hybrid modeling over other forms of model reduction. For parameters corresponding to a fast inhibitory interneuron, the trajectories computed with this hybrid model are almost indistinguishable from that of the full model over a wide range of applied currents. For instance, the periods of oscillation match closely (Figure 5) as is increased, except around the onset of AP oscillations from a steady state (. Figure 6 summarizes the similarity of APs in a more sophisticated dominant scale analysis that determined hybrid models from four HH-type neurons all matching the same template, thereby validating the proposed mechanism [46]. Greater mismatches between reduced and original model trajectories indicate weakening self-consistency conditions in the reductions and provide a diagnostic focus for refinement of the mechanisms at work.