^{1}

^{1}

^{2}

^{1}

^{2}

^{2}

^{3}

The authors have declared that no competing interests exist.

By early May 2020, the number of new COVID-19 infections started to increase rapidly in Chile, threatening the ability of health services to accommodate all incoming cases. Suddenly, ICU capacity planning became a first-order concern, and the health authorities were in urgent need of tools to estimate the demand for urgent care associated with the pandemic. In this article, we describe the approach we followed to provide such demand forecasts, and we show how the use of analytics can provide relevant support for decision making, even with incomplete data and without enough time to fully explore the numerical properties of all available forecasting methods. The solution combines autoregressive, machine learning and epidemiological models to provide a short-term forecast of ICU utilization at the regional level. These forecasts were made publicly available and were actively used to support capacity planning. Our predictions achieved average forecasting errors of 4% and 9% for one- and two-week horizons, respectively, outperforming several other competing forecasting models.

The first cases of the COVID-19 pandemic were detected in Chile by early March 2020. A few days later, all schools were closed, and a few counties with relatively high numbers of cases were quarantined. By the end of April, the available data showed that the outbreak was kept relatively under control, with a few hundred confirmed new cases every day. However, by early May, the infection rate started to increase rapidly, threatening the ability of health services to accommodate all incoming COVID-19 cases. In the middle of May, the Chilean Society of Intensive Medicine (SOCHIMI) reported a worrisome occupation rate of ICU beds of more than 95% in the capital city of Santiago, where most of the cases were concentrated. Suddenly, ICU capacity planning became a first-order concern. On May 12th, the Instituto Sistemas Complejos de Ingeniería (ISCI), which was already working on analytics related to mobility, was urged to prepare short-term forecasts of ICU bed occupancy rates for those regions with the highest utilization rates. Within 24 hours, we submitted our first report. From then on, we prepared forecasts every two days for several weeks, and then we reduced the frequency and began reporting every four days. These reports were sent directly to the authorities –particularly those on the coronavirus response committee– and to SOCHIMI. Additionally, we published the reports on ISCI’s website (

We developed a solution for generating predictions of the number of ICU beds that were going to be required by COVID-19 patients for every region in the country with a time horizon of 14 days ahead. Our methodology was based on an ensemble of a variety of forecasting models that capture different components of the evolution of the outbreak. The first model we built was a compartmental model that described patient flow as a stochastic progression through different clinical states. Here, we contemplated that new patients would require an ICU bed after a specific number of days with a given probability, and they would be discharged after a given number of days according to a certain distribution. Compartmental models have been some of the most popular approaches for characterizing the evolution of epidemics [

In this article, we describe in detail the methodology we used to generate forecasts for this very urgent problem, showing how the use of analytics provided relevant support for decision making in critical times, even with incomplete data and without enough time to fully explore the numerical properties of all available forecasting methods. Using this methodology, we produced predictions with small forecast errors that not only were useful for supporting decision making in critical times but could also be informative with regard to resource planning for potential new outbreaks. Most importantly, our approach may be easily replicated in other countries facing acute capacity constraints with respect to ICU beds.

The rest of the article is structured as follows. In Section 2, we describe the context and the data we had available, and we provide some institutional background that imposed some constraints on the design of the forecasting methodology. In Section 3, we review the relevant forecasting literature, and then we describe the statistical models we used and how we combined them to produce our forecast. In Section 5, we discuss some adjustments we introduced to accommodate changing conditions in the spread of COVID-19 and present our forecasting results. Section 6 contains a nontechnical summary of the methodology, its results and its advantages over other approaches, and we discuss possible implementations in other parts of the world. Thus, Sections 2 through 5 are devoted to providing a comprehensive technical documentation of the underlying methods we used, while readers more interested in results and implementations may read Sections 2, 6 and 7.

The rest of the article is structured as follows. In the next section, we describe the context and the data we had available, and we provide some institutional background that imposed some constraints on the design of the forecasting methodology. In the following three sections, we present the technical elements of the forecast: (i) we review the relevant literature, (ii) we describe the statistical models we used and how we combined them to produce our forecasts, (iii) we discuss some adjustments we introduced to accommodate changing conditions in the spread of COVID-19 and (iv) we present our forecasting results. These sections are devoted to providing a comprehensive documentation of the underlying methods we used. Readers more interested in results and implementations can go directly to the nontechnical summary of the methodology and results, where we present a summary of the methodology, its results and its advantages over other approaches and discuss possible implementations in other regions. We conclude with a discussion on the implications of our findings and avenues for future research.

The first COVID-19 cases were detected in Chile by early March 2020, and for the first two months, the number of new infections was relatively under control, with a few hundred confirmed new cases every day. However, by early May, the number of new COVID-19 cases increased rapidly, creating numerous and complex challenges for the country. A graphical illustration of the evolution of the pandemic in Chile is displayed in

The large peak of new cases registered in late June corresponds to a change in the governmental procedures to count cases, and this added a large number of cases that were not previously considered.

To increase ICU capacity, hospital management can follow a number of complementary strategies with different levels of complexity. A simple mechanism to increase hospital capacity is through the release of medical resources by rescheduling nonurgent procedures. Other strategies require more time for implementation. For example, pediatric rooms could be converted to receive adult patients, or anesthetic machines could be adapted to provide mechanical ventilation. As most of these mechanisms could be implemented within a time span of a few days, we decided to provide forecasts for a 14-day horizon. Despite generating forecasts of ICU utilization for each of those fourteen days, in the reports, we highlighted the number of beds that would be required in exactly one and two weeks.

Chile is administratively divided into sixteen regions, and in terms of geographical aggregation, forecasts were produced at the regional level. The country’s population is very unevenly distributed, and the Metropolitan Region, which includes the capital city of Santiago, contains near half of the national population. Despite this heterogeneous population distribution, our decision to produce regional demand forecasts is justified for two reasons. First, consistent with the administrative division, budgets are executed at the regional level. Second, from an operational perspective, if needed, patients can be transported from one hospital to another within the region, and therefore, the capacity at the regional level provides the most useful aggregation for decision making.

To estimate the models, we used data that were publicly available. Given the crucial importance of the consequences of the pandemic for the whole nation, the Ministry of Health provided frequent epidemiological reports starting on the day of the first infection. Later, the Ministry of Science consolidated all available information and created a public repository with an extensive list of statistics reported in a time series format (

Aggregation | |||
---|---|---|---|

Since | Geographical | Time | |

Number of PCR tests | 2020-04-09 | Regional | Daily |

Number of COVID-19 patients in the ICU | 2020-04-01 | Regional | Daily |

Number of COVID-19 patients in the ICU by age group | 2020-04-01 | National | Daily |

Number of new symptomatic cases | 2020-03-03 | Regional | Daily |

At the beginning of our study, the repository had information on the total daily number of new infections by region, but a few weeks later, the repository started reporting the number of new cases while distinguishing between symptomatic and asymptomatic cases. As the latter did not require ICU beds, from then on, we decided to only consider the series of symptomatic cases.

With these data in hand, we embarked on the challenging task of producing demand forecasts for ICU beds. Certainly, accurate predictions could assist decision makers in effectively preparing for the large number of expected hospitalizations. However, the exponential nature of the infections generated large variations in the expected numbers of patients in different scenarios. As we were urged to do, our goal was to create robust predictions and deliver them to health officials, with the aim of supporting them with information that could help them understand the rate at which they should be increasing ICU capacity.

Our work is related to two streams of research. First, our research is related to the use of analytics in health care and, in particular, to the use of forecasting methods for planning healthcare capacity. Second, our research is related to the use of pooled forecasting and the combination of multiple methods to generate robust predictions. Next, we discuss both streams of literature with a special focus on other recent works in the context of COVID-19.

Analytics have been shown to be relevant for supporting decisions in different components of healthcare systems [

Similar to our study, other works have proposed different models to forecast the number of infections. For instance [

The use of forecasting methods to aid hospital resource planning has been an active area of research. In this regard, time-series analysis has been one of the most widely used approaches for generating short-term demand forecasts because it provides a comprehensive treatment for seasonality and serial correlations. For example [

To predict medical requirements with a longer time horizon [

Since the start of the COVID-19 pandemic, there have been several attempts to estimate the demand for hospital resources. However, as most of this work is devoted to describing the aggregated evolution of such requirements, the results are useful for anticipating global policy making but not for supporting tactical decisions. For example [

Similar to these investigations, for our predictions, we developed a compartmental model, but we tailored it to the prediction of the demand for ICU beds in the short term. To do so, we limited our attention to the progression of patients after they had been diagnosed, and we considered a parametric distribution for patients requiring an ICU bed. Here, we incorporated clinical parameters that describe the clinical evolution of patients, and we derived detailed predictions for critical medical resources. An important drawback of compartment models is that they have limited ability to accommodate dynamic changes in key parameters; therefore, their predictions may fail to capture important variations in a given process, such as congestion or delays in testing. To overcome this limitation, we relied on ensemble forecasting models, where we combined compartment model predictions with those derived from autoregressive and machine learning models; these can effectively capture dynamic variations in the environment.

To integrate different forecasting models, we used an ensemble approach. Previous works offered strong evidence supporting the idea that combining forecasts can improve the accuracy of the output predictions [

Pooled forecasting has been applied in many diverse domains [

There are two main elements that differentiate our research from other works using multiple models. First, our method combines different predictions to produce robust estimations of the required number of ICU beds. These models can capture different components of the spread of COVID-19. For example, we considered machine learning models that can provide a great deal of flexibility to accommodate short-term variations in the environment, but we also included compartment models that provide a more detailed description of the clinical components of the disease. The second distinctive feature is that our approach is specifically devoted to supporting ICU capacity decisions; therefore, we tailored our predictions to estimate the number of beds that would be required at each point in time rather than only aggregated metrics, such as the number of beds at the peak or the cumulative number of beds that would be required over the whole duration of the pandemic.

The most widely used approach for describing the evolution of infectious diseases is compartmental models, where the population dynamically evolves through different stages [

The proportion of symptomatic patients requiring mechanical ventilation can change over time, and similarly, clinical criteria for releasing patients from the ICU can be adjusted dynamically depending on the actual usage of the existing capacity. This is not only because hospitals can relax nominal criteria but also because SARS-CoV-2 is a new virus that involves continuous learning by medical teams. For instance, the head of the Chilean Society of Intensive Medicine stated that “Patients initially stay in the ICU between 10 and 11 days, and now they are staying between 14 and 16 days. This is because, with everything we learned, we intubated less and selected more serious patients“(

Despite governmental efforts to provide timely access to relevant information, a large portion of the system for generating the data was under constant stress, and therefore, the information that we had available for any single day could be lagged. Among other issues, the results of lab tests exhibited important delays worldwide (

The data were not always available at the patient level, and there are important factors that were never observed. For example, every day, the government reported the number of new cases and the current occupation of ICU beds per region, but there was no information about how many patients entered or exited or on the lengths of the stays of patients in intensive care. Likewise, when the capacity was lacking in some regions, the government had the ability to move patients between regions, and this was not systematically reported.

To overcome the limitations of compartmental models and properly capture short-term dynamics, we combined these models with other time-series models that could be better suited to capture those dynamics. From a theoretical point of view, the use of combinations of forecasts is justified because they can lead to smaller forecasting errors and can even reduce the biases of individual forecasts [

Previous studies have offered several reasons to justify the empirical success of combining forecasts; these include model misspecification, changes in the underlying parameters and the heterogeneous use of different information sets [

We start with a classic autoregressive integrated moving average (ARIMA) approach [_{t}) depends on their lagged values and its lagged errors, and the series are further differentiated to estimate stationary processes. The ARIMAX variant is the result of considering an additional set of exogenous explanatory variables _{t}. In the vector _{t}, we considered the whole series of new cases and the positivity rate. By introducing the backward shift operator

The model depends on the relative weight of its own values (_{0}). The model also depends on the number of lags (

In our analysis, we considered ARIMA and its ARIMAX variants, but the forecasts of both were fairly similar; therefore, to create our ensemble forecast, we only considered one of the two. For ARIMAX, we included the number of new symptomatic infections in previous days as one of the key explanatory variables. For more flexible models, we considered the whole sequence of new symptomatic infections; in this case, we only included a few values in the [

We then looked at a trigonometric seasonality, Box-Cox transformation, ARMA errors, and trend seasonal components (TBATS) model. This model uses a combination of exponential smoothing and Box-Cox transformations to automatically accommodate multiple seasonal components. Each of these seasonalities is modeled by a trigonometric representation based on a Fourier series. Although this model considers a series of nested equations to represent a detailed decomposition of the series, using the backward shift operator, the model can also be expressed in a reduced form as:

One of the advantages of this model is that it provides a great deal of flexibility to automatically accommodate a large number of seasonal and trend components. However, unlike the previously discussed ARIMAX model, TBATS does not include exogenous variables and hence has limited ability to anticipate how variations in infections can be translated into different requirements of ICU beds.

In our approach, we included several neural network models. To accommodate the time series structure, we used a special class called time-delay neural networks (TDNNs). In this class, the inputs to any node can include outputs of earlier nodes not only during the current time step but also from previous time steps [

As is common in neural network learning, we trained the model structure by adjusting its parameters to minimize the induced error using a generalized feed-forward network. Thus, without loss of generality, the predictions are given by:

In this expression, _{i}(

A perceptron is a classifier that maps a vector of inputs to a single binary value through a threshold activation function. A multilayer perceptron is a network of individual classifiers that enables learning about complex processes, and it is one of the most commonly used perceptron-based learning algorithms [

An extreme learning machine is a special feed-forward neural network that only uses a single hidden layer. In this layer, nodes are randomly chosen, and the weights of the outputs are analytically determined [

We implemented an ELM following the general TDNN expression in (

A “group method of data handling” approach involves the successive selection of models based on external prediction criteria. Starting with a simple set of models, the method constructs new generations of increasingly complex models and combines them to maximize the forecasting performance [

The GMDH method allows for the inclusion of an arbitrary set of covariates in the polynomial, but in the context of time series, we only considered the lagged values of the series. Our motivation to include this model in the pool of forecasts was that it was conceived to learn complex relationships when lacking detailed knowledge about the fundamentals of the given process. In our case, we had epidemiological theory characterizing the evolution of the pandemic, but the observed data were mediated by a number of unobservable processes that might require additional layers of complexity. Another strength of GMDH is that recent computational implementations of the algorithm include automatic normalizations of the variables [

The goal of our compartment model is to predict the future utilization of ICU beds by critically ill patients due to cases of COVID-19. Thus, our model aims to replicate the behavior of the ICU process, balancing inbound and outbound flows of patients in different stages of the process. Our model considers three compartments through which the patients evolve. For each of them, we tracked the number of patients in each stage as follows:

I: The number of

C: The number of

D: The number of individuals who are

The number of infected, critically hospitalized and discharged patients fluctuated over time; therefore, we made the state variables dependent on time. Thus, the variables _{t}, _{t} and _{t} represent the number of new symptomatic cases, the number of critical patients and the number of discharged cases on day

_{1} − _{1} and _{1} + _{1} days and more severe cases staying between _{2} − _{2} and _{2} + _{2} days. Formally speaking, the equations describing the evolution of patients over time are given by:

In these equations, the series of ICU utilization and the number of new symptomatic cases are the data, while (_{1}, _{1}, _{2}, _{2}) are parameters to be estimated. These parameters are disease-specific, and we could retrieve their values from the medical literature on SARS-CoV-2. For example, the mean duration of symptoms before hospital admission was reported to be 10±2 days [

An extensive body of literature has shown that combining forecasts can improve prediction accuracy and that a simple average often performs better than highly complex combination schemes [

In our application, we used a trimmed mean approach [

Thus, our forecast was composed of an average of four models (including ICD). Considering that these predictions directly inform health officials about critical decisions, we visually inspected all forecasts before producing the final reports. In these inspections, in very exceptional cases, when more than one forecast dramatically deviated from the mean, we overruled our trimmed criteria and included both ARIMA and ARIMAX in the forecasting pool.

When inspecting the series, we found no evidence of seasonality for any variable; therefore, all models were estimated using no seasonal components. To determine the number of observations to use in every forecast, we considered information starting from April 1st, when the accumulated number of symptomatic patients reached three thousand cases. Later, when more data were accumulated, we only considered the previous sixty days of data to estimate the models.

All models were estimated using daily data. During the pandemic, the Ministry of Health provided an updated report on the evolution of the most critical variables, such as the number of new infections, the positivity rate and the number of fatal cases. All this information is uploaded to the public repository of the Ministry of Sciences and Knowledge, from which we downloaded the information automatically. The data presented very few missing values, and to address them, we used a Kalman smoothing approach [

To determine the optimal values of (

Models based on artificial neural networks can be estimated using standard back-propagation learning algorithms. However, given the time-series structure, the estimation process benefits from using automatic feature selection [

In terms of computational tools, data aggregation and preprocessing were conducted using

In the previous section, we described the general methodology we employed to produce daily forecasts for ICU beds. However, a key premise of this work is that the situation required urgent predictions. Moreover, the general environment was constantly changing, and therefore, we had to continuously update our methodology to accommodate the evolution of the pandemic and the information needs of health officials. The following is the list of the most relevant events that required adjustments to the methodology.

We generated our first solution only a few hours after the government realized that ICU planning was going to be a key element in mitigating the consequences of the pandemic. These early solutions only considered reduced-form models with no epidemiological considerations. However, we quickly realized that we needed to complement these models with others that could capture the medical structure of the problem. This is because a large fraction of the decision makers who were actively reading our reports were healthcare professionals who needed a medical narrative to explain the variations in the demand for ICU beds. This narrative was only provided by a compartment model, and therefore, in all public reports we generated, we always included those models. We also tried using a linear regression model that could provide intuitive results; however, we found that for our particular case, the linear regression model had low predictive power.

During the first two weeks, we used the series of newly confirmed cases regardless of whether the patients exhibited symptoms since that was the only information readily available. For the prediction of ICU beds, only patients with symptoms have a positive probability of requiring intensive care; therefore, the number of cases with symptoms should provide the most direct signal of the requirement for ICU beds. When the series of new cases was systematically reported depending on the existence of symptoms, we started to use symptomatic cases only.

The first two reports we generated only considered the Metropolitan Region because it contained the largest number of cases by far; consequently, it was the most urgent concern for local authorities. After a week, we added reports for three other regions (Tarapacá, Antofagasta and Valparaiso) that also showed an alarming rise in new cases. At this point, our model was completely automated to generate predictions for all regions in the country, but we only progressively added more regions as they became more worrisome. By early July, we started reporting forecasts for all sixteen regions of the country.

The GMDH model was not considered in the original list of models and was only introduced on June 11th. Since then, this model was been considered in the ensemble.

In early July, we identified that most models were starting to show that the rate at which additional ICU beds were going to be needed for the Metropolitan Region was somewhat slowing down. However, the ICD compartment model did not show any sign of saturation. After interviewing medical personnel, we realized that some patients were starting to be mechanically ventilated in emergency rooms (ERs), and so they were not counted in the nominal series of ICU utilization. Thus, in terms of capacity planning, we were required to report how many beds should be made available to cover both new cases and ventilated cases in emergency rooms. Therefore, we complemented the series of ICU beds with the number of patients ventilated in ERs. Notice however, that the number of patients ventilated in ERs decreased to almost zero by late July, and therefore, we did not report them in the final two reports.

As laboratories reached their testing capacities, the variation in the number of reported new cases increased significantly in mid-June. As a consequence, the forecasts were less stable. To overcome this problem, we preprocessed the series of new cases and used a five-day moving average instead of the raw series.

The results that we present in the next section are devoted to representing what we reported at each point in time, and they already include all methodological changes we introduced during the process.

Starting from May 16th, we generated standardized and frequent reports containing the two weeks ahead forecasts. The reports were made publicly available at

The main body of each report consisted of a summary of the number of beds that were going to be required for each region for a time horizon of two weeks, followed by a graphical summary of the forecast. A very important requirement for these reports was that they had to be concise and easy to read. The crisis committee had a very short time to evaluate all the information, so our reports were tailored to consider this situation. In

At the bottom of this figure, we show the forecasts provided for the models that survived the removal of the most extreme predictions, and then, in the upper part, we present the combined forecast. For all models, we presented both the predictions and the actual series of ICU occupancy. Furthermore, to facilitate the interpretation of the results, we highlighted the predicted numbers of beds that would be required in exactly seven and fourteen days. For the example presented in

For a systematic evaluation of our forecasts, we decompose the analysis into two parts. We first compare the performance of each model and the ensemble in terms of their forecasting errors, and then we discuss how our ad hoc trimmed algorithm fares against other pooling criteria.

From May 20th to July 28th, we produced 30 ICU utilization reports. In each report, we presented daily forecasts of the demand for ICU beds for the next two weeks in the regions considered in that instance. In every case, we generated predictions for different forecasting models, and we built our

To summarize all these daily forecasts, we compute the mean absolute percentage error (MAPE) for each model, as displayed in

1st week | 2nd week | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Date | Ensemble | ARIMAX | MLPR | ELM | TBATS | GMDH | ICD | Ensemble | ARIMAX | MLPR | ELM | TBATS | GMDH | ICD |

2020-05-20 | 2.34 | 4.79 | 2.37 | 6.64 | 1.38 | 4.21 | 5.59 | 14.70 | 15.45 | 4.19 | 7.36 | 22.24 | ||

2020-05-22 | 5.15 | 4.68 | 7.21 | 8.21 | 7.63 | 6.86 | 6.95 | 4.00 | 3.03 | 9.91 | 3.30 | 37.49 | ||

2020-05-24 | 4.94 | 1.86 | 0.96 | 2.60 | 2.27 | 19.32 | 25.09 | 5.65 | 5.56 | 2.46 | 10.07 | 85.79 | ||

2020-05-26 | 8.64 | 4.42 | 3.33 | 4.68 | 4.29 | 21.55 | 25.57 | 14.51 | 12.13 | 16.99 | 14.21 | 57.76 | ||

2020-05-28 | 7.22 | 2.70 | 4.00 | 2.54 | 3.35 | 19.94 | 24.28 | 11.09 | 12.48 | 10.92 | 12.47 | 62.65 | ||

2020-05-30 | 7.00 | 2.94 | 2.91 | 2.24 | 2.54 | 21.41 | 22.99 | 12.75 | 6.50 | 10.59 | 10.38 | 62.11 | ||

2020-06-01 | 8.06 | 6.91 | 6.98 | 5.83 | 5.21 | 12.52 | 24.63 | 18.97 | 23.78 | 15.78 | 14.95 | 40.01 | ||

2020-06-03 | 9.74 | 3.80 | 7.74 | 3.74 | 3.33 | 23.81 | 23.62 | 11.02 | 14.57 | 9.50 | 8.06 | 59.39 | ||

2020-06-05 | 7.26 | 6.85 | 3.65 | 2.79 | 3.80 | 17.71 | 12.41 | 11.43 | 9.10 | 3.01 | 4.15 | 29.63 | ||

2020-06-07 | 6.31 | 4.40 | 9.38 | 3.19 | 2.83 | 14.80 | 7.64 | 4.75 | 19.77 | 2.56 | 3.11 | 27.66 | ||

2020-06-09 | 1.87 | 1.19 | 3.08 | 1.19 | 1.84 | 7.75 | 1.75 | 2.94 | 1.68 | 6.97 | 10.85 | 16.39 | ||

2020-06-11 | 3.70 | 2.88 | 6.45 | 4.58 | 5.31 | 1.53 | 2.46 | 10.23 | 5.75 | 16.44 | 12.22 | 14.99 | 5.41 | 7.96 |

2020-06-13 | 6.60 | 4.97 | 8.12 | 5.33 | 7.07 | 4.40 | 5.88 | 10.02 | 5.36 | 13.78 | 7.88 | 12.87 | 7.24 | 5.65 |

2020-06-16 | 6.31 | 1.39 | 7.93 | 5.69 | 6.29 | 7.42 | 6.36 | 4.27 | 7.78 | 8.26 | 3.65 | 5.24 | 5.36 | 2.37 |

2020-06-18 | 0.94 | 2.66 | 1.99 | 1.42 | 2.92 | 1.55 | 2.02 | 2.68 | 13.38 | 10.09 | 6.55 | 14.31 | 2.61 | 7.82 |

2020-06-20 | 4.32 | 6.09 | 7.37 | 3.44 | 6.60 | 2.56 | 4.36 | 8.03 | 18.85 | 14.82 | 10.91 | 20.00 | 8.65 | 2.66 |

2020-06-22 | 2.43 | 5.23 | 3.85 | 3.56 | 4.73 | 1.47 | 2.54 | 5.55 | 18.36 | 14.94 | 12.19 | 14.54 | 6.46 | 13.91 |

2020-06-24 | 0.90 | 1.64 | 7.37 | 5.38 | 1.24 | 3.13 | 6.58 | 3.35 | 7.20 | 20.61 | 17.28 | 4.10 | 12.59 | 35.56 |

2020-06-26 | 3.18 | 2.23 | 3.28 | 4.66 | 1.17 | 3.77 | 10.29 | 7.23 | 10.52 | 2.99 | 17.21 | 1.63 | 12.91 | 41.43 |

2020-06-28 | 1.25 | 8.03 | 3.12 | 9.17 | 6.23 | 7.87 | 12.23 | 0.66 | 20.76 | 11.03 | 23.01 | 15.49 | 19.75 | 44.11 |

2020-06-30 | 2.34 | 6.14 | 2.71 | 5.83 | 1.03 | 4.75 | 19.30 | 3.49 | 19.42 | 5.20 | 19.37 | 4.39 | 13.76 | 49.93 |

2020-07-02 | 3.04 | 5.71 | 1.85 | 7.26 | 1.08 | 4.77 | 21.82 | 4.94 | 19.64 | 14.47 | 23.15 | 5.77 | 14.10 | 57.32 |

2020-07-03 | 1.58 | 3.96 | 3.04 | 8.26 | 3.18 | 5.70 | 21.35 | 3.07 | 13.49 | 17.54 | 25.09 | 10.95 | 16.14 | 56.06 |

2020-07-07 | 2.00 | 1.03 | 6.85 | 6.30 | 1.67 | 5.81 | 22.81 | 5.01 | 9.92 | 27.33 | 27.88 | 11.99 | 20.55 | 50.17 |

2020-07-10 | 3.05 | 2.99 | 4.66 | 7.71 | 3.55 | 10.20 | 24.43 | 1.44 | 12.19 | 21.93 | 29.64 | 13.59 | 25.77 | 53.79 |

2020-07-14 | 4.00 | 6.69 | 9.56 | 12.81 | 7.02 | 16.80 | 19.80 | 8.16 | 14.84 | 23.18 | 33.07 | 16.38 | 33.26 | 54.20 |

2020-07-17 | 2.69 | 6.02 | 1.77 | 4.62 | 1.31 | 12.37 | 23.20 | 4.72 | 12.84 | 5.84 | 17.07 | 1.50 | 25.17 | 53.88 |

2020-07-21 | 1.37 | 6.91 | 2.04 | 9.47 | 4.88 | 14.59 | 14.72 | 2.67 | 13.29 | 5.99 | 18.57 | 7.53 | 32.82 | 38.47 |

2020-07-24 | 2.50 | 3.70 | 3.07 | 7.89 | 4.59 | 10.97 | 7.46 | 3.73 | 4.27 | 4.07 | 10.01 | 2.72 | 25.32 | 23.36 |

2020-07-28 | 2.59 | 1.60 | 3.39 | 4.51 | 2.11 | 9.64 | 4.56 | 1.26 | 2.74 | 6.74 | 10.34 | 2.76 | 29.24 | 3.47 |

min | 0.90 | 1.03 | 0.96 | 1.19 | 1.03 | 1.47 | 2.02 | 0.66 | 2.74 | 1.68 | 2.46 | 1.50 | 2.61 | 2.37 |

max | 9.74 | 8.03 | 9.56 | 12.81 | 7.63 | 16.80 | 24.43 | 25.57 | 20.76 | 27.33 | 33.07 | 20.00 | 33.26 | 85.79 |

mean | 4.11 | 4.15 | 4.67 | 5.38 | 3.68 | 6.80 | 13.40 | 9.03 | 11.41 | 12.31 | 13.93 | 9.32 | 16.69 | 36.77 |

std | 2.52 | 2.01 | 2.54 | 2.65 | 2.04 | 4.54 | 7.75 | 8.28 | 5.45 | 6.95 | 8.34 | 5.23 | 9.73 | 22.24 |

In

It is important to evaluate the performance of our forecasting approach in the context of a pandemic characterized by phases of exponential growth that can lead to large prediction errors. For example, consider the case of the U.K., where early epidemiological models initially projected approximately 500,000 deaths, a number that was updated to under 20,000 deaths just two weeks later [

To further understand how individual models performed relative to the ensemble, we plot the series of MAPEs for all models in

To complete the analysis, we discuss how our conditional trimmed mean ensemble performed against other criteria for combining forecasts. As we forced our predictions to include the ICD compartment model regardless of the value of its predictions, it is possible that our ensemble might lead to a worse performance than those of other criteria that are not subject to this restriction. By design, we were willing to sacrifice precision to gain interpretability, but it is worth exploring whether our predictions were deteriorated by considering this interpretability constraint.

RMSEs are the upper panel and MAPE are in the bottom panel.

Metric | Tmean | Median | Mean |
---|---|---|---|

RMSE (overall) | 96.91 | 115.81 | 102.92 |

140.90 | 97.29 | 105.50 | |

52.37 | 163.65 | 106.82 | |

38.95 | 101.38 | 90.85 | |

MAPE (overall) | 6.78 | 7.53 | 6.97 |

9.87 | 6.49 | 7.94 | |

3.22 | 9.55 | 5.38 | |

3.26 | 7.60 | 6.48 |

From

In the early stages of the pandemic, our trimmed mean criteria were outperformed by the standard mean and median ensembles. However, after a few iterations, our predictions consistently exhibited the smallest errors. This result can be explained by the fact that the ICD model produced prediction errors with opposite signs that canceled out the errors induced by other models. We believe that feeding the model structural information about the clinical evolution of COVID-19 patients can provide a useful forecasting signal and provide additional support for the convenience of using combined forecasts.

The epidemiological literature has offered a variety of tools for understanding the dynamics of infectious diseases. In this study, we built upon these epidemiological models, and we tailored them with the specific goal of producing accurate forecasts of ICU utilization, as these has been critical components for mitigating the negative impacts of the COVID-19 pandemic. In this regard, there are three key conditions that differentiate our forecast from traditional epidemiological models.

As we only focused on patients who required hospital resources in the short term, instead of forecasting the evolution of the pandemic through the reproduction number R, we directly used the number of symptomatic cases. This information is readily available and easy to process. More importantly, the usage of the actual number of symptomatic cases instead of a projection of the infections had a material impact in terms of improving the forecasting accuracy.

We were specifically interested in characterizing ICU utilization; therefore, our model was tailored to capture the most relevant dynamics of this problem. These include the persistence of bed utilization and flexible distributions for the duration of the stay of each patient. These dynamics can be captured by two simple conservation law equations that indicate the new daily requirements of beds and the number of discharged patients. The rates at which customers arrive and leave the ICU can be derived from clinical sources, or they can be estimated from the data as we do in our application.

In our model, we combined the standard epidemiological approach with time series and machine learning models that bring additional flexibility to the forecast. Importantly, our results indicate that this additional flexibility is critical to obtain highly precise estimates. This is because standard epidemiological models do not properly capture dynamic variations in how patients evolve during critical care. This is particularly relevant for a new disease for which medical teams are continuously learning about improved treatments. Methodologically speaking, we show that the combination of different models can be achieved through a simple linear combination of forecasts.

Since epidemiological models do not incorporate detailed modeling of the dynamics of ICU requirements, they tend to have large forecasting errors. For the case of Chile, even the most sophisticated compartment models exhibited prediction errors that were up to three times larger than what we reported here (

Our numerical analysis indicates that the most classical epidemiological approach by itself produces large forecasting errors. In fact, the compartment model generated predictions with a mean error rate of 13.4% (sd of 7.75), while the methodology we proposed led to a mean error rate of 4.11% (sd of 2.52). Some of the time series and machine learning models performed reasonably well, but they failed to anticipate changes in trends. The combined forecast in general produces the most accurate predictions and it correctly anticipates when the number of ICU beds will decrease. Thus, our analysis demonstrates that a simple combination of different forecasts can generate much better predictions in the context of planning emergency resources than those of single models.

Our model may be implemented by health authorities rather easily. Indeed, the logic of our compartment model can be summarized into two flow conservation equations, and the time series and machine learning models can be estimated using standard statistical packages. All forecasts can be combined using a linear combination of the individual forecasts. To facilitate the implementation of the method for other countries or regions, in

In this research, we proposed a methodology to produce short-term forecasts for ICU beds in the context of the COVID-19 epidemic in Chile. Our algorithm is based on an ensemble method that combines autoregressive neural networks, artificial neural networks and a compartment model to generate our best prediction of ICU utilization for a time horizon of fourteen days. This algorithm captures the epidemiological dynamics of the disease with a compartmental model and is complemented by time-series models that capture short-term changes in the clinical parameters. This approach resulted in very accurate predictions, with a mean error rate of 4% for the first week and 9% for the second week. An analysis of the performance over time indicates that, in relative terms, the proposed model produced larger errors earlier in the process. This can be explained by the fact that in the early stages of the pandemic, each individual model had less data to learn from. However, we believe that a more fundamental reason is that after a few iterations, different models produced complementary results; therefore, the trimmed mean we used to ensemble the forecast generated a better forecast than that of any single model in isolation. Hence, every model contributed a different key signal that increased the accuracy of the ICU bed predictions in most of our reports. In this regard, the inclusion of a compartmental model helped to generate highly precise predictions, despite being the least accurate single model overall.

In terms of the application, the reports we made publicly available were a very useful tool for anticipating the availability of critical resources in hospitals. We generated consistent information to characterize the progression of the pandemic, providing health officials with a data-driven tool to make quick decisions about ICU planning. These reports enabled the Ministry of Health to implement a progressive increase in the number of beds, and this resulted in more than doubling the capacity in the most congested regions. We heard from health and science authorities and from SOCHIMI how these forecasts were useful for letting them know what was coming and so they could better focus their resources and efforts across the country. Importantly, the messages we were sending were well received because, following our interactions with authorities, we tailored the reports to ease communications.

We are confident that our model contributed to better planning during a critical situation where the lives of many were at risk. However, as the COVID-19 pandemic is still a major threat in many countries around the world, we consider it important to discuss potential ideas to further improve the methodology. In our work, we used the data that were available and that we identified as having predictive power. However, the use of additional disaggregated data is likely to further improve the forecasting accuracy. For example, more detailed information on patient demographics and medical histories could further help to identify what fraction of patients might require mechanical ventilation and thus provide more detailed guidelines about focused mitigation policies.

The proposed methodology can also be improved by adding additional forecasting methods into the pool of models. Although we used a wide variety of models, there are others that we did not try. For example, the recently developed

To produce our predictions, we treated different regions independently. Although this is a reasonable assumption for the case of Chile where commuting between regions was limited, it might not be a good assumption when replicating our work in other geographies. In such cases, a hierarchical model allowing for spatial correlation might be more appropriate [

List of libraries and the corresponding parameters used (optional).

(PDF)

Historical MAPE per Model—Valparaíso Region.

(PDF)

Selection Frequency per Model across Iterations in Chile.

(PDF)

(PDF)

In the following three figures, we display the summaries of the forecasts for all regions in the country and the detailed plots for the most populated regions of Valparaíso and Bíbio.

(TIF)

We are indebted to the Ministry of Science, Technology, Knowledge and Innovation for facilitating the interactions needed to generate these forecasts. We are also thankful to SOCHIMI for useful feedback on our results.