Changes in land use are among the most important human alterations of the Earth´s land surface [1,2]. As a response, private actors and governments have significantly increased the number of protected areas (PAs) to conserve natural landscapes [3,4]. In the context of less developed countries, not only has the public demand for land for conservation increased, as revealed by government policies, but so has the overall demand for land for agricultural production, urban and suburban expansion, and other activities [5]. Despite the expansion of PAs in less developed countries, including Chile, not much is known both about the policy process determining the establishment of PAs, and about how successful such areas have been in contributing to their conservation objectives [6]. To inform decision makers regarding any new investments in land conservation, a better understanding is needed regarding where previous PAs have been placed and how effective these past conservation investments have been.

In most countries, governments have not randomly distributed PAs geographically [4], in part due to historical patterns of state land ownership. In situ biodiversity conservation, for example, has traditionally relied on PAs, and historically such areas often consisted of already public lands [7]. Regardless of the scientific and public choice factors underlying the establishment of PAs, if their resulting distribution is biased to favor areas of lower conservation threats, then most previously used methods will tend to overestimate the effectiveness of protection for reducing harmful land use and land cover changes (LULCC) [8,9]. The present paper addresses these selection effects.

In terms of effectiveness of PAs, the issue of the impact of PAs on LULCC is complicated because avoided changes to land are not directly measurable. Daniela Miteva et al. [6] summarize the studies that use rigorous empirical designs to quantify the impacts of PAs, finding that these studies have focused predominantly on the effectiveness of PAs in preventing deforestation [6]. Previous studies have suggested that the impact of PAs can mainly be explained by threats of land conversion and enforcement [10–12]. Most of these studies apply empirical designs whereby PAs are compared to observationally similar unprotected lands, but without distinguishing between private and government lands as comparable control areas. Therefore, this empirical strategy ignores that traditional PA systems are usually based on converted, long-held government lands, which perhaps have different land use regimes and different levels of enforcement relative to private lands. Within this context, this research addresses an important question for the analysis of the impact of PAs—what are the effects of establishing PAs in comparison with the counterfactual scenario obtained from unprotected public lands?

This paper corrects the problems with selection bias and the proper choice of controls in evaluating the impact of PAs by presenting causal estimates of their effectiveness in Chile. The paper demonstrates that evaluations can be substantially improved by controlling for biases along dimensions that are observable. We address two main research questions: What is the impact of Chilean PAs on avoided deforestation which occurred between 1986 and 2011? And, how do estimates of the impacts of PAs vary when using only public land as control units? This research effort constitutes the first impact evaluation of one of the oldest PA systems in Latin America, which protects one of the few remaining extensive temperate rainforests in the world. In terms of methodology, to adequately estimate the effects of PAs, we employed matching methods to define adequate control groups, but not as has been done in previous research, although we recognize previous attempts to stratify the sample in the evaluation of the impact of PAs (e.g. [13])

PAs are, and will remain, the cornerstone of global conservation efforts [14]. An increasing human population and standard of living, and demand for multiple ecosystem services, will intensify competition for land inside and outside PAs. In the context of PAs and avoided deforestation, protection effectiveness will depend on the conservation opportunity costs of keeping land in forest. [5] and [10], presents a framework for considering PAs’ impacts on LULCC where protection effectiveness depend on rents determined by opportunity costs. Therefore, if we consider PAs impact on avoided deforestation, PAs may remain forested due to the protection itself (i.e. de jure protection) or because the landscape characteristics of the protected lands discourage LULCC (i.e. de facto protection). In the latter case, protection may have no impact at all. Accordingly, the primary question PAs administrators should ask is whether the conservation scheme has a sufficiently large “additionality” which is the difference in conservation between the with-PAs scenario and the without-PAs baseline [15]. At the end, the impact of PAs is an empirical question that requires rigorous empirical evidence to be answered. This is the research challenge that this paper will address in the context of one of the oldest PA network in the Latin American and The Caribbean region.

For the Chilean case, in the middle of the nineteenth century, the rapid deforestation of south-central Chile, caused by land settlement and consequent agriculture and livestock activities, increased awareness about conservation [16]. Decree Law 18362 of 1984 created the national public system of PAs (Sistema Nacional de Áreas Silvestres Protegidas del Estado, SNASPE). The purpose of the law was to organize the scattered PAs around a unified conservation system with the common purpose of protecting Chilean natural resources. With the creation of SNASPE, the government tried to promote the definition and legalization of PAs boundaries and the assignment of specific management objectives for each unit in the system, so it can be considered as the true beginning of PAs in Chile [16].

Considering the different land uses represented in the Chilean system of PAs, almost 30% corresponds to forest. Chile has the largest temperate forest area in South America and more than half of the total area of temperate forests in the southern hemisphere [17]. This Chilean temperate forest has been classified as a biodiversity hotspot for conservation [18] and has also been included among the most threatened eco-regions in the world in the Global 200 initiative launched by WWF and the World Bank [19]. Most of the Chilean forests (including native and exotic forests) are distributed along the Coastal and the Andean Range of Chile from 35° to 56° totaling an area equal to 15.6 million ha [20].

In general for the case of Chile, the main driving force behind LULCC is the replacement of native forests by forest plantations and agricultural crops, a factor that contributes to soil loss, forest clearing, and the consequent loss of biodiversity [21]. In previous qualitative evaluations of SNASPE [16,21–24], authors have recognized that SNASPE geographic distribution and ecosystem representation is insufficient to achieve adequate levels of conservation and that the poor conservation status of native forests may be explained by the Chilean forest policies followed by the Law of Forest Development (Decree Law N° 701 of 1974). This law also aimed to promote forestation by small forest landowners to prevent soil loss, and to protect and restore soils. These policies have not provided economic incentives for the sustainable management and conservation of native forests, in contrast to the use of public funds to support the establishment of forest plantations.

At present, Chile has 14.1 million ha of PAs in its national public system which represents 18.7% of the Chilean territory. The system includes 31 national parks, 48 national reserves, and 15 natural monuments, with the first two categories encompassing 99.9% of the total area, and the system is administered by the Chilean Forest Service (Corporación Nacional Forestal, CONAF). Chilean protected areas rank second in Latin America and seventh worldwide in terms of percentage of national coverage [25] and according to the National Register of Protected Areas more than 90% of PAs included in SNASPE originate from government lands.

In terms of criteria employed to establish PAs, the motivations have been diverse and have evolved to attain the current use of specific criteria and standards. However, at the beginning of the Chilean history of environmental protection, the criteria did not necessarily reflect conservation objectives, but rather they reflected a motivation to regulate wood extraction and commercialization, to protect public land without productive value (i.e. land with no capacity to sustain agriculture or cattle production), or to conserve landscape beauty [26]. Previous studies of the impact of Chilean PAs have not analyzed conservation effectiveness (i.e. what would have been the level of conservation had the PA not been established). Furthermore, there is no empirical evidence of the impact of the system which is an important knowledge gap that this paper aims to fill. Fig 1 shows the study area.

Table 4 presents the DID matching estimates of avoided deforestation as a proportion of forest conserved. Estimates based on matching methods were compared with estimates based on more conventional methods used in conservation science literature. In the second column are estimates for PAs established after SNASPE creation (1984). Decree Law 18,362 of 1984 created SNASPE, however, program implementation took time after that date. For the purpose of this study, 1986 data provide the pre-SNASPE information necessary to construct baseline forest cover scenarios for both protected and unprotected areas. The first row presents the avoided deforestation estimates generated by conventional methods used in the conservation science literature where deforestation on protected plots is compared with deforestation on unprotected areas without controlling for other covariates. It implies that 16.9% of protected plots would have been deforested by 2011 in the absence of protection.

Nevertheless, as seen in Table 3, the protected and unprotected areas differ on key covariates that could affect not only protection status but also changes in forest cover. In terms of covariates associated with protection status and deforestation, Table 3 shows that there are statistically significant differences between protected and unprotected land across all the socioeconomic and biophysical drivers associated with protection status. In general, PAs are farther from cities and road networks, and are located on steeper and higher land. Similarly, PAs also tend to be placed on land less suitable for agriculture and on soil with higher erodibility. Thus one might worry that despite our pre-matching efforts and the similar baseline forest cover before protection, the mean avoided deforestation among protected land from 1986 to 2011 in the absence of protection, may not be well represented by the mean change in forest cover for unprotected lands during the same period. In fact, avoided deforestation estimates from matching approaches were much smaller. Covariate matching estimates, with and without calipers, implied that only 4.7% and 4.8% of protected plots would have been deforested by 2011 in the absence of protection respectively. Matching with calipers was used to further address concerns about potential bias where calipers define a tolerance level for judging the quality of the matches: if available controls are not good matches for a treated unit (i.e., there is no match within the caliper), the unit is eliminated from the sample. In our study, we view calipers as a robustness check. We define the caliper as 0.5 standard deviations of each matching covariate. Matching estimates could still be because even after matching, there are statistically significant differences in the observable baseline characteristics of PAs and matched controls. Table 3 shows several metrics of covariate balance before and after matching for the sample. The fifth and sixth columns of Table 3 present two measures of the differences in the covariate distributions between protected and unprotected areas, namely, the difference in means and the average distance between the two empirical quantile functions (values greater than 0 indicate deviations between the groups in some part of the empirical distribution). If matching is effective, these measures should move towards zero [35], which is what we observe. The last column of Table 3 also shows normalized differences. By using a post-matching bias correction procedure that asymptotically removes the conditional bias in finite samples, the last row of Table 4 shows that 4.7% of protected plots would have been deforested by 2011 in the absence of protection, which is not much different from the estimates with matching without the bias correction procedure. To put Table 4‘s estimates into perspective, consider that 561,920 ha of forest were protected between 1984 and 2011. Thus, conventional methods imply 95,526 ha of avoided deforestation. In contrast, the matching methods imply 26,410–26,972 ha of avoided deforestation (see Fig 3).

The third column of Table 4 presents DID matching estimates using public land never protected and forested in 1986 as control. Neither estimates from conventional conservation science approaches nor from samples selected by matching show significant estimates of PAs’ impact on circumvented deforestation between 1986 and 2011. Table 5 shows the covariate balance for this alternative causal analysis.

Table 6 presents results from sensitivity tests that assess the degree to which potential unobservable heterogeneity makes us uncertain about the inferences drawn in Table 4. A sensitivity analysis like this asks: How would inferences about the effects of PAs be altered by hidden biases of various magnitudes? How large would these differences have to be to alter the conclusions of the study? Table 6 presents the analysis for the estimates from matching with calipers. The second column in Table 6 indicates that the estimate of 4.8% avoided deforestation remains significantly difference from zero even in the presence of moderate unobserved biases. To attribute the avoided deforestation to an unobserved covariate rather that to an effect of PAs, that unobserved covariate would need to produce more than a 1.35 increase in the odds ratio of protection to differ between protected and unprotected cells, the 95% confidence interval would still exclude zero.

The sensitivity test presented in Table 6 is free of parametric assumptions and provide an easily interpretable measure of the way in which an unobservable covariate may bias a result. In this case, the assumed unobserved covariate is a strong confounder: one that not only affects PAs selection but also determines whether deforestation is more likely for the protected units or their matched unprotected areas. Rosenbaum bounds are in this sense a ‘‘worst-case” scenario [41]. Nonetheless, they convey important information about the level of uncertainty contained in matching estimators by showing just how large the influence of a confounding variable must be to undermine the conclusions of a matching analysis

Conservation practitioners and policymakers need credible information on how PA policies affect ecosystems. The primary question PA administrators should ask is whether the conservation scheme adds a difference in conservation between the with-PAs scenario and the without-PAs baseline [15]. However, the level of additionality that a PAs network can provide also depends on where protection is being placed. Most previous evaluations on PAs’ effectiveness have relied on indirect estimates based on comparisons between protected and unprotected areas. However, such methods could easily be biased when protection is not randomly assigned, but rather is determined by characteristics that also affect LULCC (e.g., land productivity, accessibility). Our results confirmed that conventional scientific approaches claim land cover impacts for protection which are actually due to PA networks’ landscape characteristics.

For Chile, the state of conservation of its natural resources is a topic of growing concern among the general public as well as national and international conservation organizations. Previous analysis of Chilean PAs have focused only on ecosystem representation, coverage of biodiversity hot-spots, budgets, and boundary issues. However, conservation additionality of PAs has not been mentioned in any previous studies. For this case, we show that selection bias exists in assigning protection, which might then result in fewer additional conservation benefits to one of the oldest systems of PAs in Latin America. Chilean PAs are farther from cities and road networks, and are located on steeper and higher land. They tend to be placed on land less suitable for agriculture and on soil with higher erodibility.

In terms of methodological lessons, results show the importance of the comparison group (i.e., selection of appropriate unprotected land to be used as controls to match PAs). In that sense, finding causal impacts of PAs in terms of forest conservation may well depend on the selection of the appropriate control group. Our analysis illustrates how substantial improvements can be made to estimates concerning the impact of PAs. However, in estimating forest conservation effectiveness, the selection of appropriate unprotected land to compare with the outcome observed in PAs is very important. By comparing PAs with observationally similar unprotected lands (after removing non-comparable areas, such as indigenous reserves or private PAs) as done in previous research, one ignores that traditional PA systems are usually based on converted, long-held government lands which perhaps have different land use regimes relative to private lands. The causal analysis, using only public lands to build a control group, showed no difference between the conservation outcome obtained with PAs and the conservation outcome that would have happened had the PAs not been established. In the Chilean context, regulatory regimes are similar between PAs and public lands. Public lands similar to PAs are well managed and so converting these lands to PAs does not necessarily provide additional conservation benefits. These results raise important questions, such as the relative costs of different types of public land management, and whether there are any particular types of public lands where creating PAs would have a greater impact in terms of avoided deforestation. These results also suggest that conversion of private land to PAs may offer more avenues in terms of additional conservation benefits when compared to the conservation scenario including only public lands. If forests in public lands are almost as well protected as they are in PAs, as our analysis shows, the Chilean government would do well strategically to commit to keeping natural forests on public lands alongside the PA network. However, the inclusion of private land into the system of PAs may offer a higher level of supplementary conservation benefits. Therefore, the promotion of incentives to include private PAs could be very important, particularly considering that the expropriation of private land for conservation purposes is less feasible given the financial constraints of a system with very limited resources to invest in conservation. In fact, a new Biodiversity and Protected Areas Law is currently under discussion in the Chilean congress. One of the new objectives of this law refers to the inclusion of a private network of PAs into a new integrated national system of PAs.

In some countries, PAs can be assigned without adequate financial resources, or without the infrastructure and networking needed to substitute consumable resources and protect the area from development or misuse. Moreover, PAs are biased towards places where they can least prevent land conversion. Often it may be financially and politically expedient to protect public land with low financial value and to avoid places with conflictive land use alternatives when considering conservation objectives [4]. In the Chilean context, PAs established in the study area came from public land set aside for conservation purposes. Although the SNASPE law includes private land expropriation for conservation objectives, the history of PAs in Chile does not show evidence of this kind of practice. As a result, the current system of PAs originates almost exclusively from public land. Therefore, counterfactual scenarios obtained from forest cover changes on matched unprotected private lands may not reflect the same results one may obtain if these counterfactual scenarios are constructed using unprotected public lands as potential control unprotected areas.

Given that not all PAs are established to avoid deforestation, results also admit the potential for positive outcomes. Therefore, impact evaluations of PAs should also work toward developing additional measures for program outcomes including forest degradation (not just deforestation), biodiversity, wildlife, primary forests (not forests in general), among others.