Revealed : The Underlying Flow , Masked by Predictive Risk Tools but Uncovered by Dynamic Simulation used in Joint Commissioning

Introduction: Predictive risk tools can enhance the planning of integrated services across health / social care and primary / acute by utilising real-time patient data to identify those who might benefit from different interventions and models. But commissioners / those planning services need to envisage the likely impact of interventions on whole populations over time. Data-led approaches enable analysis of the state of a population, but data alone cannot generate projections of what might happen in future under a redesigned system. A better alternative approach is a ‘whole systems’ based computer simulation. This submission is based on a case study in Stockport, England, in which health and social care commissioners initially jointly developed a new case-finding service using the risk stratification tool (combined PARR, patients at risk of readmission) to contact people at risk. Those contacted were assessed, with a view to increasing their uptake of existing community-based services, in the hope that this might have a preventive effect, especially on reducing acute admissions from this group. Post-implementation, hoped-for reductions in admissions were not realised. To explore why, the joint commissioning group used computer simulation within a ‘group model building’ context, facilitated by an expert modeller. Theory/Methods: The enquiry was based on System Dynamics, an application of computer simulation particularly suited to understanding, at population level, how ‘whole systems’ change over time. The approach is most powerfully experienced within the context of ‘group model building’, where an expert group (in this case, commissioners from health and social care) is facilitated to build a model, populate it with data, and run a series of scenarios exploring system behaviour. The group model building formed part of a consulting assignment. The group met six times at monthly intervals; between meetings the modeller built the model and engaged with group members to elicit data. 15th International Conference on Integrated Care, Edinburgh, UK, March 25-27, 2015 1 International Journal of Integrated Care – Volume 15, 27 May – URN:NBN:NL:UI:10-1-117034 – http://www.ijic.org/ Results: This enquiry pointed to the importance of understanding dynamics (change over time) not revealed by data-led approaches. The model showed that the intervention was offered to many whose risk-level would have improved anyway, meaning that projected savings from reduced acute admissions were unrealistically high. The model comprised four modules: • Population, the whole registered adult population (45,000) in a district, divided into sixteen ‘stocks’ (four age bands, with each age band divided into the four risk groups); as the model steps forward in time, the population ages, and people move up and down (‘flow’) between risk groups • Acute admissions, driven by different rates from each of the sixteen (age by risk) groups • Capacity and dynamics of the new intervention • Financial flows based on spend on acute admissions and the case finding service The main finding was that this case finding approach would be unlikely to generate significant savings. In most scenarios, the cost of the new service would be more than the savings generated by reduced acute admissions. The main reason was that secondary analysis of the risk stratification data revealed that although the number of people in each risk group does not vary much over time, this masks a high rate of underlying churn within the system, as people move both up and down the risk scale. Discussions: The predictive risk literature tends to be ‘stock’ based and pyramid-shaped (typically, how the population is distributed across groups, for example). It is implicit that these distributions result from a long-term deterioration dynamic similar to the combined effect of multiple long-term conditions. This new analysis of a local database tracking real-time PARR status suggests ‘combined PARR’ score gives undue weight to recent acute events. But long term deterioration undoubtedly exists. As an experiment, we invented a different set of flow dynamics. This produced the same ‘population by dependency’ profile as the ‘combined PARR’ data, but resulting from people moving, slowly, in the direction of greater dependency. This identical prevalence, but different underpinning churn, generated greater savings across the whole system over time. Given that the same distribution of people by risk group might result from different combinations of flow rates (incidence, deterioration, recovery and mortality), forecasting based on extrapolation of trends in stock values alone cannot capture the dynamic, non-linear consequences of interventions that might modify flow rates. Conclusions: A case finding intervention based on this risk stratification tool, in effect, targets many people who would have improved anyway. This does not mean that case finding approaches should be abandoned. Based on a risk stratification tool that actually captures long-term deterioration, they can be effective (in terms of benefit to recipients), maybe even saving money across the whole system. Of course, we do not argue that new approaches to integration are only justifiable if they reduce costs. Lessons learned: This study is one of several that point to commissioners needing to take a dynamic view of prevalence. Before intervening in a population, one needs to understand and model dynamics underpinning observed prevalence (incidence, duration of condition, untreated deterioration / improvement pattern, service effect, mortality). System Dynamics provides a methodology for integrating diverse sources of evidence (about prevalence, service capacity and cost, treatment effects, and expenditure over time). Group model building provides an accessible means for commissioners to collaborate, exploring, and learning about, complex system effects. Limitations: The approach has some resource requirements in the form of the need for expert modellers in a discipline that is not yet widely adopted within public health. It also requires a fresh look at existing datasets, which may not fully capture every relevant phenomenon. But significant insights can be gained relatively quickly. Comparable enquiries, such as demographic microsimulation, might require years of funding to reach similar conclusions. Suggestions for future research: Such approaches can potentially enhance economic evaluations of treatments / services across a whole system. Where a service generates costs and savings across different agencies, differentially over time, an approach that is grounded in complex systems theory merits further exploration. 15th International Conference on Integrated Care, Edinburgh, UK, March 25-27, 2015 2 International Journal of Integrated Care – Volume 15, 27 May – URN:NBN:NL:UI:10-1-117034 – http://www.ijic.org/

Theory/Methods: The enquiry was based on System Dynamics, an application of computer simulation particularly suited to understanding, at population level, how 'whole systems' change over time.The approach is most powerfully experienced within the context of 'group model building', where an expert group (in this case, commissioners from health and social care) is facilitated to build a model, populate it with data, and run a series of scenarios exploring system behaviour.The group model building formed part of a consulting assignment.The group met six times at monthly intervals; between meetings the modeller built the model and engaged with group members to elicit data.

Results:
This enquiry pointed to the importance of understanding dynamics (change over time) not revealed by data-led approaches.The model showed that the intervention was offered to many whose risk-level would have improved anyway, meaning that projected savings from reduced acute admissions were unrealistically high.The model comprised four modules: • Population, the whole registered adult population (45,000) in a district, divided into sixteen 'stocks' (four age bands, with each age band divided into the four risk groups); as the model steps forward in time, the population ages, and people move up and down ('flow') between risk groups • Acute admissions, driven by different rates from each of the sixteen (age by risk) groups • Capacity and dynamics of the new intervention • Financial flows based on spend on acute admissions and the case finding service The main finding was that this case finding approach would be unlikely to generate significant savings.In most scenarios, the cost of the new service would be more than the savings generated by reduced acute admissions.The main reason was that secondary analysis of the risk stratification data revealed that although the number of people in each risk group does not vary much over time, this masks a high rate of underlying churn within the system, as people move both up and down the risk scale.

Discussions:
The predictive risk literature tends to be 'stock' based and pyramid-shaped (typically, how the population is distributed across groups, for example).It is implicit that these distributions result from a long-term deterioration dynamic similar to the combined effect of multiple long-term conditions.This new analysis of a local database tracking real-time PARR status suggests 'combined PARR' score gives undue weight to recent acute events.But long term deterioration undoubtedly exists.As an experiment, we invented a different set of flow dynamics.This produced the same 'population by dependency' profile as the 'combined PARR' data, but resulting from people moving, slowly, in the direction of greater dependency.This identical prevalence, but different underpinning churn, generated greater savings across the whole system over time.
Given that the same distribution of people by risk group might result from different combinations of flow rates (incidence, deterioration, recovery and mortality), forecasting based on extrapolation of trends in stock values alone cannot capture the dynamic, non-linear consequences of interventions that might modify flow rates.

Conclusions:
A case finding intervention based on this risk stratification tool, in effect, targets many people who would have improved anyway.This does not mean that case finding approaches should be abandoned.Based on a risk stratification tool that actually captures long-term deterioration, they can be effective (in terms of benefit to recipients), maybe even saving money across the whole system.Of course, we do not argue that new approaches to integration are only justifiable if they reduce costs.
Lessons learned: This study is one of several that point to commissioners needing to take a dynamic view of prevalence.Before intervening in a population, one needs to understand and model dynamics underpinning observed prevalence (incidence, duration of condition, untreated deterioration / improvement pattern, service effect, mortality).System Dynamics provides a methodology for integrating diverse sources of evidence (about prevalence, service capacity and cost, treatment effects, and expenditure over time).Group model building provides an accessible means for commissioners to collaborate, exploring, and learning about, complex system effects.

Limitations:
The approach has some resource requirements in the form of the need for expert modellers in a discipline that is not yet widely adopted within public health.It also requires a fresh look at existing datasets, which may not fully capture every relevant phenomenon.But significant insights can be gained relatively quickly.Comparable enquiries, such as demographic microsimulation, might require years of funding to reach similar conclusions.
Suggestions for future research: Such approaches can potentially enhance economic evaluations of treatments / services across a whole system.Where a service generates costs and savings across different agencies, differentially over time, an approach that is grounded in complex systems theory merits further exploration.