By: Dr. Allison Simler-Williamson
Note from the editors: This month marks ten years since we started Natural History of Ecological Restoration! During the last decade, we’ve posted 127 times on a wide variety of ecological restoration stories from around the world. At the same time, our global readership has grown from 4,000 viewers in our first full year to more than 14,000 viewers in each of the last five years, with readers coming from 150 countries. At their best, NHER stories illuminate ecological restoration’s natural history, taken in the most inclusive sense to mean stories about the people, places, organisms, institutions, and interactions involved in ecological restoration projects.
This month’s post by Allison Simler-Williamson (Boise State University) exemplifies this standard. In her post, Dr. Simler-Williamson describes how environmental conditions, land manager decisions, and restoration outcomes interact in complex and confusing ways – and she charts a path forward for better understanding the real-world impacts generated by restoration projects.
A “randomized” experiment can be a beautiful and powerful tool in restoration ecology. Randomization ensures that an experimental treatment (such as a restoration action) is unrelated to any other environmental factors that might influence the outcome we are measuring (such as plant establishment). When we confidently compare plots that received an herbicide or planting treatment to adjacent “reference” sites, our estimation of restoration effectiveness hinges on this assumption of randomization.
But, despite their elegance, randomized experiments are labor-intensive and often spatially or temporally constrained, limiting how applicable they may be to new areas or in atypical years. Thus, randomized experiments are increasingly mismatched with widespread ecological degradation and growing needs for restoration. Emerging “big data”, such as the US Geological Survey’s Land Treatment Digital Library, which contains information about more than 65,000 restoration treatments that have occurred on Bureau of Land Management land in the western United States, could help tackle the problem of understanding restoration efficacy across wide spatial and temporal scales.
When we pivot to using these “observational” datasets, which are opportunistically collected, we incur an important tradeoff. We gain generalizability but lose the power of randomization because (and this likely is not a surprise to anyone working in restoration!) real-world management treatments are almost never applied randomly across large landscapes. Restoration occurs in certain parts of landscapes more than others, due to a mix of ecological need, bureaucratic constraints, and stakeholder decision-making processes.
Why is this lack of randomization a problem when we want to leverage these kinds of large datasets? In statistics courses, I like to use some of my son’s favorite bathtime toys as an analogy for what can occur. When you pour water into these colorful pipes, the wheels spin, and my son loves to create networks between them. If the pipes are arranged as below (Figure 1A) with water flowing through them, it would be immediately obvious that there is no direct relationship between the wheels “X” and “Y” – they are simply both being spun by the water flowing out of “Z”. However, if I were to obscure the connections between the pipes (Figure 1B) and instead ask you, “Based on your observations, is there a relationship between X and Y?”, you could detect a correlation. Depending on your understanding of the system, you might assume that this link is a direct cause of X on Y, or vice versa.
This phenomenon is an example of statistical “confounding,” in which a background driver can bias our understanding of the relationship between two other variables. This potential for confounding is a big concern if we would like to estimate the efficacy of restoration treatments that were applied in non-random places or times because it can falsely inflate or shrink apparent effects in our analyses. For instance, if restoration actions (X) are disproportionately applied in dry areas (Figure 2a), and drought stress simultaneously reduces plant establishment (Y) (Figure 2b), the correlations between variables can cause a treatment effect to shrink (even if the treatment works!), if we ignore this lack of randomization in treatment applications.
In a 2022 study (Simler-Williamson and Germino 2022), we explored how the ‘non-random’ application of restoration seedings of big sagebrush (Artemisia tridentata) influenced our estimation of treatment effectiveness, using observations of post-fire seedings across the western U.S. in the Land Treatment Digital Library.
When we used statistical models that assumed these restoration treatments were applied randomly, we found a somewhat counterintuitive result: a negative relationship between sagebrush seeding and sagebrush recovery. However, this statistical illusion emerged because of background relationships in our dataset: restoration seedings (Figure 2; X) tended to occur in hotter, drier, and more degraded places (Z), where plant establishment was already more difficult (Y). In short, restoration actions were disproportionately applied in more “dire” ecological settings, creating the illusion of failure.
Next, we compared this approach to two sets of statistical methods designed to minimize the effects of confounders (“Z”) on our treatment effect. The first set of approaches required that we include pre-existing data about the hypothesized confounding variables directly into our model. When we accounted for some of these measured drivers of “non-random” seeding application using existing data about soil types, climate conditions, and fire impacts, restoration efficacy shifted from a negative number toward a neutral effect.
Finally, the last set of approaches instead used repeated observations of sagebrush stands to ‘control for’ confounding variables, by accounting for pre-existing differences between treated and untreated stands before they had been seeded, rather than requiring the direct inclusion of measured variables. Only when measured and many unmeasured differences between treated and untreated sites were accounted for in our analysis, we revealed a positive impact (of ~4-6% in sagebrush cover by 10 years post-fire) of restoration seedings in degraded sagebrush ecosystems.
The pattern we described in that paper underscores two key needs in restoration science: one social and one statistical. These results suggest that we urgently need better information about the socio-economic drivers determining where and when we apply restoration treatments, which are poorly described. The analyses that incorporated some common ecological drivers of restoration need (e.g., fire impacts, climate variables) only accounted for some of the bias in the effects of restoration seeding. The strong shift to positive impacts of restoration after “unmeasured” sources of bias were considered suggests that there are significant additional unmeasured processes that simultaneously shape where we attempt to restore and where plant populations recover. In the focal sagebrush steppe ecosystems, these may include diverse drivers such as seed availability, bureaucratic constraints, aesthetic considerations, cultural values, land use, and grazing management. Collecting and understanding these variables seems essential to advancing our understanding of restoration effectiveness broadly.
But no matter how elegant randomized experiments are as a concept, they may not be able to generate estimates of restoration effectiveness at the broad spatial and temporal scales we require to manage rapidly changing ecosystems. As a community, I think we need to be integrating big, opportunistically collected datasets with statistical approaches that recognize the “messiness” of these data and aim to minimize the risk of confounding in treatment effects. Well-estimated treatment effects can improve how we connect restoration resources (such as seeds, time, and funding) with the locations where the ecological benefits may be greatest, both in space and time.
For more information about Dr. Simler-Williamson’s work, see her lab website or her 2022 paper in Nature Communications.
Leave a Reply