The starting point of the CTF workflow was the outline of disease models for each of the included hazards (as chosen by the hazard-based task forces), and the epidemiological data inputs that parameterized these disease models. To obtain this information, systematic reviews were commissioned and managed by three hazard-based task forces, i.e., the Enteric Diseases Task Force (EDTF), the Parasitic Diseases Task Force (PDTF), and the Chemicals and Toxins Task Force (CTTF). Details are therefore provided elsewhere [23–25].

The course of disease is characterized by various health states (e.g. acute or chronic phases, short-term or long-term sequelae), possibly having different severity levels [8]. A disease model, also referred to as an outcome tree, is a schematic representation of the various health states associated with the concerned hazard, and the possible transitions between these states. A disease model for each hazard was defined by the members and commissioned experts of each hazard-based task force, considering relevant health outcomes supported by evidence identified in the respective reviews. Discussions of the choices and uncertainties in deriving the disease models for specific hazards are provided elsewhere [23–25].

In the context of the CTF, disease models were defined as computational disease models, and not merely as biological disease models. While biological disease models merely reflect the natural history of disease, computational disease models also reflect the input parameters needed to calculate incidence and mortality of each of the concerned health states. As such, computational disease models are a combination of disease biology and data availability.

Computational disease models may be represented as directed acyclic graphs, defined by parent and child nodes and directed edges (arrows) defining the relationships between nodes. In the CTF framework, parent nodes were either incidence, mortality, YLD or YLL rates, while child nodes were multiplicative elements, such as proportions or ratios (reflecting e.g. the probability of developing a specific symptom following infection, or the proportion of illnesses attributable to the concerned hazard). A specific disease model "language" was developed to denote the relationship and contribution of the different nodes. Rectangles defined parent nodes, and rounded rectangles defined child nodes. Grey nodes did not contribute directly to the DALYs, green nodes contributed YLDs, and red nodes contributed YLLs. Nodes that contributed to the incidence of the index disease were identified by a thick border. S1 Fig gives the disease models for all 36 FERG hazards.

In general, three main approaches can be distinguished for estimating the burden due to a specific hazard in food, i.e., categorical attribution, counterfactual analysis, and risk assessment. S1 Table gives an overview of the modelling strategy applied for each included hazard. As the choice of the modelling strategy was mainly driven by the type of data available, no sensitivity analyses could be performed to triangulate different modelling approaches. Modelling choices were further driven by a strive for consistency with existing WHO Global Health Estimates.

Categorical attribution can be used when a foodborne hazard results in an outcome (death or a specific syndrome) that is identifiable as caused by the hazard in individual cases [26]. Following the typology of Devleesschauwer et al. [16], the burden due to a specific hazard can then be calculated using an attributional model (in which the incidence of the symptom is multiplied with the attributable proportion for a given hazard) or a transitional model (in which the incidence of infection with or exposure to the hazard is multiplied with the probability of developing a given symptom). Categorical attribution was applicable for all viral, bacterial and parasitic hazards, and for cyanide in cassava and peanut allergens, and was therefore the standard method used by FERG. Fig 2 shows the computational disease model for Mycobacterium bovis, which is characteristic for the attributional models. In this model, the overall incidence and mortality of tuberculosis is multiplied with the proportion attributable to M. bovis, resulting in the incidence and mortality of M. bovis tuberculosis. Fig 3 shows the computational disease model for Echinococcus granulosus, which is characteristic for the transitional models. In this model, the overall incidence of infection by this parasite was multiplied with child nodes reflecting the probability of developing the concerned health states, resulting in the incidences of the specific health states.

When the hazard elevates the risk of a disease or disability outcome that occurs in the population from other causes as well, causal attribution can only be made statistically, and not on an individual basis. This is the case for many chemicals, including aflatoxin and dioxin. Aflatoxin for instance may increase the risk of hepatocellular carcinoma, but it is not possible to specify that a specific liver cancer case was caused by aflatoxin. In this situation, the standard approach for calculating the burden of environmental exposures is to use a counterfactual analysis in which the current disease outcomes with current exposure are compared to the disease outcomes under an alternate exposure (a minimum risk exposure which could be zero, or some accepted background level) [27]. This allows calculation of a population attributable fraction (PAF) that can be applied to the all-cause burden estimates for the relevant disease outcome (the so-called burden envelope), leading to a special case of the attributional model [16]. In the context of FERG, counterfactual analysis was used to estimate the burden of aflatoxin-related hepatocellular carcinoma.

In addition to categorical attribution and counterfactual analysis, which can be considered top-down approaches, FBD burden can also be estimated by a risk assessment approach, which can be considered a bottom-up approach. In this approach, the incidences of the specific health states (e.g. impaired male fertility due to prenatal dioxin exposure) are estimated by combining exposure and dose-response data. The dose-response model may for instance define the probability of illness at a given exposure level, which can then be translated into an estimate of the number of incident or prevalent cases expected to occur in the exposed population [25,27]. As this approach does not involve burden attribution, it does not necessarily ensure consistency with existing health statistics. However, risk assessment may be a valid alternative when no burden envelopes exist or when it can be demonstrated that the estimated excess risk is additive to the background risk. In the context of FERG, risk assessment was used to estimate the burden of dioxin-related hypothyroidy and impaired fertility.

A database template was developed in Excel (Microsoft Corp., Redmond, Washington, USA) to collect the data resulting from the systematic reviews in a standardized way (S2 File). The structure of the database was based on the disease models, with one sheet per node. Three generic sheets were defined: (1) a "RATE" sheet, for rates by country; (2) a "PROB-local" sheet, for proportions or ratios by country; and (3) a "PROB-global" sheet, for a single proportion or ratio that applied to all countries.

Each sheet consisted of four tables for entering (1) the rate or proportion/ratio data; (2) the age distribution; (3) the sex distribution; and (4), if applicable, the duration. Using a drop-down menu, different formats could be selected for entering the input parameters, including a mean and 95% confidence interval; a minimum, most likely and maximum; different percentiles; the shape and rate of a Gamma distribution (for rates); and the shape parameters of a Beta distribution (for proportions). Gamma and Beta distributions were chosen because their domains correspond to those of rates and proportions, respectively, and because their parameters have an intuitive interpretation (i.e., number of cases and sample size or observation time, respectively, number of positives and number of negatives). Likewise, different levels of stratification could be selected for the duration parameters (i.e., none, by age only, by sex only, by age and sex). Age distribution, sex distribution and duration were allowed to vary by country, by defining different "groups" and assigning countries to "groups". Full details on the parameters used for quantifying the burden of the different hazards is available in the appendices to the EDTF, PDTF and CTTF manuscripts [23–25].

Extrapolation or imputation models may be needed when literature searches cannot provide essential epidemiological data such as incidence or mortality rates [28]. These models estimate parameters based on data of neighboring regions or other time periods. The external data used must thus be representative of the selected population, region and time. The CTF developed, tested and evaluated several possible approaches to impute missing incidence data at the country level [29]. This exercise identified several pitfalls in the use of explanatory covariates, such as the potential for overfitting and the arbitrariness in the selection of covariates. Therefore, and further motivated by a strive for parsimony and transparency, we decided to use a log-Normal random effects model as the default model for imputing missing country-level incidence data. We used the subregions as defined in Table 2 as the random effect or cluster variable. This model assumes that the log-transformed incidence rate in country j belonging to subregion i arises from a Normal distribution with subregion specific mean μ_i and a within-region (= between-country) variance σw2. Each subregion specific mean μ_i is in turn assumed to arise from a Normal distribution with mean μ₀ and a between-region variance σb2: log(θij)∼Normal(μi,σw2) μi∼Normal(μ0,σb2)

After fitting this hierarchical random effects model to the available data, incidence values for countries with no data were imputed based on the resulting posterior predictive distributions. In other words, we represented missing incidence data by log-Normal distributions based on the fitted mean and variance parameters. For countries in a subregion where none of the countries had data, the log-incidence was imputed as multiple random draws from a Normal distribution with mean equal to the fitted global intercept μ₀ and variance equal to the sum of the fitted between-region variance σb2 and the fitted within-region variance σw2 (thus imputing the log-incidence as that of a “random” country within a “random” subregion, with the uncertainty interval describing the variability between and within subregions): log(θij*)∼Normal(μ0,σb2+σw2)

For countries in a subregion where at least one of the other countries had data, the log-incidence was imputed as multiple random draws from a Normal distribution with mean equal to the fitted region-specific intercept μ_i and variance equal to the fitted within-region variance σw2 (thus imputing the log-incidence as that of a “random” country within the concerned subregion, with the uncertainty interval describing the variability within subregions): log(θij*)∼Normal(μi,σw2)

When countries were considered free from exposure through the food chain, they were excluded from the imputation model and thus did not contribute to the subregional estimates. This was the case for Brucella spp., as discussed in [23], and Echinococcus granulosus, as discussed in [24]. For countries with available incidence data, no imputation was performed. The incidence data used in the probabilistic burden assessments were thus a combination of actual data and imputed estimates. No additional step was included to correct incidence data for potential underreporting, as this was already captured by the previous steps of the framework. Indeed, for the hazards that used an attributional model, disease envelopes were used that had already been corrected for underreporting, while for other hazards we directly drew on GBD 2010 estimates (S1 Table). For the remaining hazards, either epidemiological data were used that did not need (further) correction, or the underreporting factor was included in the disease model (which was the case for Trichinella spp. and cyanide in cassava).

For aflatoxin, the same random effects model was used to extrapolate PAFs, but now using logit-transformed instead of log-transformed values.

The model was implemented in a Bayesian framework, using independent Normal(0, 1e5) priors for μ₀ and all μ_i; a Uniform(0, 10) prior for σ_w; and a Folded-t(1) prior for σ_b, as suggested by Gelman [30]. Sensitivity analyses using Gamma priors for the variance parameters did not yield meaningful differences. The model was run in JAGS [31] through the 'rjags' package in R [32]. After a burn-in of 5000 iterations, another 5000 iterations were retained for inference. Two chains were run, and convergence was ascertained through density and trace plots, and the multivariate potential scale reduction factor (or Brooks-Gelman-Rubin diagnostic). The applied JAGS code is given in S1 Code.

A crucial assumption made by this imputation model is that missing data were considered "missing at random" (MAR), meaning that missingness was independent of the unobserved data, given the observed data [33,34]. In our case, this assumption implied that, within each subregion, countries with data provided unbiased information on those without data, and that, across subregions, subregions with data provided unbiased information on those without data. For five hazards (Bacillus cereus, Clostridium perfringens, Clostridium botulinum, Staphylococcus aureus, and peanut allergens), however, only data from high-income subregions, i.e., subregions A or B, could be retrieved. In those instances, the assumption of MAR was clearly violated, and it was decided not to extrapolate those data to the rest of the world. As a result, those hazards were excluded from the global burden of disease estimates [35].

S1 Table shows which imputation strategy was used for each of the included hazards. For the four intoxications, peanut allergens, and cyanide in cassava, the default random effects model was not used because of the limited number of data points. Instead, the burden for each concerned country was imputed as draws from a Uniform distribution defined by the lowest and highest globally observed incidence or mortality rates. To ensure consistency with results of the Child Health Epidemiology Reference Group (CHERG), alternative imputation approaches were applied for estimating etiological fractions for the eleven diarrheal agents [23,36,37]. For six other hazards, no imputation had to be performed because data were used that had already been imputed. This was the case for hepatitis A virus, Salmonella Typhi, Salmonella Paratyphi, and Taenia solium, for which GBD 2010 data were used, and for Mycobacterium bovis and Trichinella spp., for which other published data were used [38,39].

DALYs incorporate the severity of health states through the DW, reflecting the corresponding relative reduction in health on a scale from zero to one. DWs for several health states have been derived for the GBD studies and for various national burden of disease studies [40]. To ensure comparability, the CTF adopted the DWs that were used for WHO's Global Health Estimates [15]. These DWs were based on those derived for the GBD 2010 study [41], but with an alternative value for primary infertility (i.e., 0.056 instead of 0.011). The latter revision was motivated by an analysis showing that the GBD 2010 weights undervalued the health states associated with infertility [15]. For dioxin-induced hypothyroidy, we adopted the GBD 2013 DW for hypothyroidy, as this health state was not included in the GBD 2010 DW study [42].

Several FBDs present with unique clinical signs for which no DWs have been derived. Acute trichinellosis, for instance, typically presents with myalgia and facial edema, for which no specific DWs are available [39]. When DWs were missing, proxy health states were selected by a medical expert and DW expert in the CTF and confirmed by disease experts in the hazard specific task forces. In other instances, DWs were available for severity levels that were not explicitly considered in the disease models. For diarrhea, for instance, DWs were available for mild, moderate and severe diarrhea, although the disease models only included diarrhea as such. In those cases, weighted averages were calculated based on published reviews of severity distributions, avoiding an over- or underestimation of YLDs that would occur if only one DW would have been selected. Table 1 lists the DWs used for the different FERG health states; S2 Table provides further details on their derivation.

For each hazard, incidence, mortality, YLD, YLL and DALY rates were calculated for 11 age groups (<1; 1–4; 5–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74; 75–84; ≥85) and both sexes. When necessary, age and sex specific rates were obtained by multiplying the overall rates with outcome specific age and sex distributions. The reference year for the calculation of absolute numbers was 2010, with population estimates obtained from the 2012 revision of the United Nations World Population Prospects [43]. All estimates were generated per country, and subsequently aggregated per subregion, per region, and globally (Table 2). The resulting estimates are presented in three hazard-specific papers [23–25] and in a summary paper [35]. The results may also be accessed and explored through an online tool (https://extranet.who.int/sree/Reports?op=vs&path=/WHO_HQ_Reports/G36/PROD/EXT/FoodborneDiseaseBurden), which shows the breadth and flexibility of our framework in terms of aggregating and reporting estimates.

The duration component of the YLDs is defined as the average observed duration until the next health state (e.g., remission or death). For calculating YLDs when duration was lifelong, we therefore used the country-specific life expectancy (LE) [43] as duration. The YLLs, on the other hand, are essentially calculated as the number of cause-specific deaths multiplied by a loss function specifying the years lost for deaths as a function of the age at which death occurs [15]. In standard DALY methodology, the time component of the YLLs is defined as the ideal residual life expectancy a person would have if the world would be free from disease and provide maximal access to health care. In accordance with the WHO Global Health Estimates, we used the highest projected LE for 2050 as normative LE for calculating YLLs [15]. This LE table has a LE at birth of 92, higher than that of the LE tables used in the GBD studies, which were based on current death rates [13,14]. Since even for the lowest observed death rates there are a proportion of deaths which are preventable or avertable, the highest projected LE for the year 2050 was deemed to better represent the maximum life span of an individual in good health, while acknowledging that it may still not represent the ultimate achievable human life span [15]. In line with current global burden of disease assessments, no age weighting or time discounting was applied [14,15]. HIV infected invasive salmonellosis cases and deaths, and HIV infected M. bovis deaths, were excluded from the burden estimates. No further corrections were made for possible co-morbidities.

Parameter uncertainty was taken into account by performing the burden assessments in a probabilistic framework. Ten thousand Monte Carlo (parametric bootstrap) simulations of the input parameters were generated to calculate 10,000 estimates of incidence, mortality, YLD, YLL and DALY rates. These 10,000 estimates were then summarized by their median and a 95% uncertainty interval defined as the 2.5^th and 97.5^th percentile of the distribution of estimates. Special care was taken to deal with correlated uncertainties, for instance when the disease model included "global" probabilities (e.g. when it was assumed that the probability of developing a certain health state following infection was the same for each country). In such cases, a vector of random probabilities was simulated only once and applied to the different countries, instead of incorrectly simulating a new, independent vector of random probabilities for each country.

The main aim of FERG was to quantify the disease burden resulting from foodborne exposure to potentially foodborne hazards. However, many of the hazards considered are not transmitted solely by food, but have several potential exposure routes (e.g. direct contact with animals, human-to-human transmission, and waterborne transmission). For certain hazards, it was therefore necessary to attribute a proportion of their overall burden to foodborne exposure (S1 Table).

Some hazards were considered 100% foodborne, i.e., Listeria monocytogenes, M. bovis, all foodborne trematodes, T. solium, Trichinella spp., aflatoxin, cyanide in cassava, dioxin and peanut allergens.

For the remaining hazards, a structured expert elicitation using Cooke's Classical Method was conducted to attribute burden to different exposure routes, providing hazards-specific estimates for each exposure route per subregion [44]. This process yielded a probabilistic estimate of the proportion foodborne, in the form of an empirical cumulative density function from which random samples could be drawn. Foodborne cases, deaths, YLDs, YLLs and DALYs were then obtained by multiplying the vectors of random values for these parameters with a vector of random values for the proportion foodborne. As before, the perfect correlation of uncertainty was dealt with by simulating only one vector of random foodborne proportions per subregion, and by applying this vector to all parameters of all countries within the concerned subregion.