Fine scale prediction of ecological community composition using a two-step sequential machine learning ensemble
Prediction is one the last frontiers in ecology. Indeed, predicting fine scale species composition in natural systems is a complex challenge as multiple abiotic and biotic processes operate simultaneously to determine local species abundances. On the one hand, species intrinsic performance and their tolerance limits to different abiotic pressures modulate species abundances. On the other hand there is growing recognition that species interactions play an equally important role in limiting or promoting such abundances within ecological communities. Here, we present a joint effort between ecologists and data scientists to use data-driven models informed by ecological deterministic processes to predict species abundances using reasonably easy to obtain data. To overcome the classical procedure in ecology of parameterizing complex population models of multiple species interactions and poor predictive power, we followed instead a sequential data-driven modeling approach. We use this framework to predict species abundances over 5 years in a highly diverse annual plant community. Our models show a surprisingly high spatial predictive accuracy using only easy to measure variables in the field, yet such predictive power is lost when temporal dynamics are taken into account. This result suggest that predicting the temporal dimension of our system requires longer time series data. Such data would likely capture additional sources of variability that determine temporal patterns of species abundances. In addition, we show that these data-driven models can also inform back mechanistic models of important missing variables that affect species performance such as particular soil conditions (e.g. carbonate availability in our case). Being able to gain predictive power at fine-scale species composition while maintaining a mechanistic understanding of the underlying processes can be a pivotal tool for conservation, specially given the human induced rapid environmental changes we are experiencing. Here, we document how this objective can be achieved by promoting the interplay between classic modelling approaches in ecology and recently developed data-driven models.