Smoothing spatio-temporal data with complex missing data patterns

We consider spatio-temporal data and functional data with spatial dependence, characterized by complicated missing data patterns. We propose a new method capable to efficiently handle these data structures, including the case where data are missing over large portions of the spatio-temporal domain. The method is based on regression with partial differential equation regularization. The proposed model can accurately deal with data scattered over domains with irregular shapes and can accurately estimate fields exhibiting complicated local features. We demonstrate the consistency and asymptotic normality of the estimators. Moreover, we illustrate the good performances of the method in simulations studies, considering different missing data scenarios, from sparse data to more challenging scenarios where the data are missing over large portions of the spatial and temporal domains and the missing data are clustered in space and/or in time. The proposed method is compared to competing techniques, considering predictive accuracy and uncertainty quantification measures. Finally, we show an application to the analysis of lake surface water temperature data, that further illustrates the ability of the method to handle data featuring complicated patterns of missingness and highlights its potentiality for environmental studies.

HydroClim Data Portal: Cyberinfrastructure for providing high-resolution GIS modeled streamflow and water temperature data to researchers

Freshwater ecosystems play a key role in sustaining aquatic biodiversity. However, human alterations to watersheds and climate change are reducing critical habitat and the viability of populations of many aquatic species. The environmental changes have also had significant adverse impacts on water temperatures and streamflow. The changes in temperature and precipitation forecast over the next century are expected to affect the freshwater ecosystems and their biodiversity to an even greater extent than in the past. The aims of the HydroClim project are to provide openly accessible data on two key measures of stream conditions in the United States (US) and Canada for use in research, to increase public understanding of issues involving water resources, and to provide training opportunities for scientists who will be responsible for the conservation of freshwater biodiversity in the future. The project has used contemporary air temperature and precipitation data and future climate data from multiple Global Climate Model scenarios to generated high-resolution, spatially explicit, monthly streamflow and water temperature data for all watersheds across the US and Canada from 1950–2099 through multiple Soil and Water Assessment Tool (SWAT) hydrologic models. This presentation describes a cyberinfrastructure we developed for hosting the HydroClim data, consisting of a relational database and a web-based data portal that allows scientists to query and download the data. We have imported almost 1.9 billion HydroClim data records into the system. At the time of this submission, 1.3 billion records of historical data and predicted streamflow and water temperature model data are available in the HydroClim data portal for 26 watersheds in the United States. The HydroClim data are also being integrated with fish occurrence data from Fishnet 2, via the Fishnet 2 API (Application Programming Interface), which provides occurence data records for over 4.1 million species lots representing over 40 million specimens in ichthyological research collections. Our plan is to extract and merge environmental data from Hydroclim API, with fish occurrences containing geospatial information from the Fishnet 2 API, displaying the integrated data on web-based interactive hydrological maps in different time-series, and providing a tool for visualizing ecosystem diversity. The combined Hydroclim and Fishnet2 data can be used for ecological niche modeling applications, such as predicting the future distribution of threatened and endangered freshwater fish species. I will describe the cyberinfrastructure of HydroClim data portal and some of the ways the data can be used in biodiverisity research in the future.

Temporal Changes in Delaware Waters Using Long-Term (1967–2019) Water Temperature Data

The present article provides long-term (1967–2019) water temperature data collected from Delaware water quality monitoring sites. In Delaware, there are approximately 140 water quality monitoring sites in Piedmont, Delaware Bay, Chesapeake Bay, and Inland Bay drainage basins. Long-term quarterly (i.e., four times a year: Q1—January–February–March; Q2—April–May–June; Q3—July–August–September; Q4—October–November–December) water temperature data were calculated from each water quality monitoring sites’ continuous monthly data. This study focuses on water quality monitoring sites with significant (p-value identifying linear regression model) increasing or decreasing trends of water temperature. Quarterly water temperature data, statistical analysis, and maps showing increasing and decreasing trend from water quality monitoring sites with significant trends are presented in this article.

SIMO v1.0: Simplified model of the vertical temperature profile in a small warm monomictic lake

A simple 1-D energy budget model (SIMO) for the prediction of the vertical temperature profiles in small, monomictic lakes forced by a reduced number of input meteorological variables is proposed. The model estimates the net heat flux and thermal diffusion using only routinely measured hourly mean meteorological variables (namely, the air temperature, relative humidity, atmospheric pressure, wind speed, and precipitation), hourly mean ultraviolet B radiation (UVB), and climatological monthly mean cloudiness data. Except for the initial vertical temperature profile, the model does not use any lake-specific variables. The model performance was evaluated against lake temperatures measured continuously during an observational campaign in two lakes belonging to the Plitvice Lakes, Croatia (Lake 1 and Lake 12). Temperatures were measured at 15 and 16 depths ranging from 0.2 to 27 in Lake 1 (maximum depth of 37.4 m) and 0.2 to 43 m in Lake 12 (maximum depth of 46 m). A sensitivity analysis of the simulation length was performed for simulation lengths from 1 to 30 days. The model performed reasonably well and it was able to satisfactorily reproduce the vertical temperature profile at the hourly scale, the deepening of the thermocline with time, and the annual variation in the vertical temperature profile. A yearlong simulation initiated with an approximately constant vertical profile of the lake temperature (≈ 4 °C) was able to reproduce the onset of stratification and convective overturn. However, the thermocline depth was underestimated while the epilimnion temperatures were overestimated. Nevertheless, the values of the model performance measures obtained for a yearlong simulation were comparable with those reported for other more complex models. Thus, the presented model can be used for the assessment of the onset and duration of lake stratification periods when water temperature data are unavailable, which can be useful for various lake studies performed in other scientific fields, such as biology, geochemistry, and sedimentology.

Evaluation of Global Surface Water Temperature Data Sets for Use in Passive Remote Sensing of Soil Moisture

Inland open water bodies often pose a systematic error source in the passive remote sensing retrievals of soil moisture. Water temperature is a necessary variable used to compute water emissions that is required to be subtracted from satellite observation to yield actual emissions from the land portion, which in turn generates accurate soil moisture retrievals. Therefore, overestimation of soil moisture can often be corrected using concurrent water temperature data in the overall mitigation procedure. In recent years, several data sets of lake water temperature have become available, but their specifications and accuracy have rarely been investigated in the context of passive soil moisture remote sensing on a global scale. For this reason, three lake temperature products were evaluated against in-situ measurements from 2007 to 2011. The data sets include the lake surface water temperature (LSWT) from Global Observatory of Lake Responses to Environmental Change (GloboLakes), the Copernicus Global Land Operations Cryosphere and Water (C-GLOPS), as well as the lake mix-layer temperature (LMLT) from the European Centers for Medium-Range Weather Forecast (ECMWF) ERA5 Land Reanalysis. GloboLakes, C-GLOPS, and ERA5 Land have overall comparable performance with Pearson correlations (R) of 0.87, 0.92 and 0.88 in comparison with in-situ measurements. LSWT products exhibit negative median biases of −0.27 K (GloboLakes) and −0.31 K (C-GLOPS), whereas the median bias of LMLT is 1.56 K. When mapped from their respective native resolutions to a common 9 km Equal-Area Scalable Earth (EASE) Grid 2.0 projection, similar relative performance was observed. LMLT and LSWT data are closer in performance over the 9 km grid cells that exhibit a small range of lake cover fractions (0.05–0.5). Despite comparable relative performance, ERA5 Land shows great advantages in spatial coverage and temporal resolution. In summary, an integrated evaluation on data accuracy, long-term availability, global coverage, temporal resolution, and regular forward processing with modest data latency led us to conclude that LMLT from the ERA5 Land Reanalysis product represents the most optimal path for use in the development of a long-term soil moisture product.

Sensitivity Analysis of OTEC-CC-MX-1 kWe Plant Prototype

The Mexican Caribbean Sea has potential zones for Ocean Thermal Energy Conversion (OTEC) implementation. Universidad del Caribe and Instituto de Ciencias del Mar y Limnologia, with the support of the Mexican Centre of Innovation in Ocean Energy, designed and constructed a prototype OTEC plant (OTEC-CC-MX-1 kWe), which is the first initiative in Mexico for exploitation of this type of renewable energy. This paper presents a sensitivity analysis whose objective was to know, before carrying out the experimental tests, the behavior of OTEC-CC-MX-1 kWe regarding temperature differences, as well as the non-possible operating conditions, which allows us to assess possible modifications in the prototype installation. An algorithm was developed to obtain the inlet and outlet temperatures of the water and working fluid in the heat exchangers using the monthly surface and deep-water temperature data from the Hybrid Coordinate Ocean Model and Geographically Weighted Regression Temperature Model for the Mexican Caribbean Sea. With these temperatures, the following were analyzed: fluctuation of thermal efficiency, mass flows of R-152a and water and power production. By analyzing the results, we verified maximum and minimum mass flows of water and R-152a to produce 1 kWe during a typical year in the Mexican Caribbean Sea and the conditions when the production of electricity is not possible for OTEC-CC-MX-1 kWe.

A Simple Solution for Refining Lake Water Temperature Profiles Data Arrayed from High-Frequency Monitoring Sensors

The revolutionized aquatic monitoring sensors are essential in capturing environmental patterns that traditional discrete samplings might not be able to. They allow scientists to further synthesize and better conclude processes in aquatic ecosystems. These sensors produce high-frequency data that provide information on a fine temporal scale, even near real-time. The massive quantities of the streamed data, however, create challenges for scientists to grasp the concrete information. Filtering data quality, on the other hand, is another problem scientists might have encountered as sensor accuracy and precision may drift along the line. Hence, quality assurance and quality control might be quite labouring owing to the size of datasets to handle. This paper proposed a semi-mechanistic algorithm to improved false water temperature data. Using “theoretical” thermal stratification as a reference, this algorithm fixed sensors error readings. A 5-month dataset of water temperature profiles of Lake Maninjau, West Sumatra, captured every 10 minutes from a set of sensors in thermistor chain was applied. We found that most data fit to the theoretical temperature profile, R² = 0.962, RMSE = 0.081^oC. A number of errors, however, were observed in the upper layer of the lake (<20 m), the most dynamic layer in terms of its thermal variation. Sensor drifts in this active upper mixed layer can be related to the generated errors. Through this simple solution, not only improving the quality of the observed water temperature data, but was also able to identify the most probable source of errors

EXPERIENCE: Algorithms and Case Study for Explaining Repairs with Uniform Profiles over IoT Data

IoT data with timestamps are often found with outliers, such as GPS trajectories or sensor readings. While existing systems mostly focus on detecting temporal outliers without explanations and repairs, a decision maker may be more interested in the cause of the outlier appearance such that subsequent actions would be taken, e.g., cleaning unreliable readings or repairing broken devices or adopting a strategy for data repairs. Such outlier detection, explanation, and repairs are expected to be performed in either offline (batch) or online modes (over streaming IoT data with timestamps). In this work, we present TsClean, a new prototype system for detecting and repairing outliers with explanations over IoT data. The framework defines uniform profiles to explain the outliers detected by various algorithms, including the outliers with variant time intervals, and take approaches to repair outliers. Both batch and streaming processing are supported in a uniform framework. In particular, by varying the block size, it provides a tradeoff between computing the accurate results and approximating with efficient incremental computation. In this article, we present several case studies of applying TsClean in industry, e.g., how this framework works in detecting and repairing outliers over excavator water temperature data, and how to get reasonable explanations and repairs for the detected outliers in tracking excavators.

Outlier Detection of Water Quality Data Using Ensemble Empirical Mode Decomposition

Objectives : This study was conducted to propose a new methodology for efficiently identifying and removing various outliers that occur in data collected through automated water quality monitoring systems. In the present study, water temperature data were collected from domestic G_water supply system, and the performance of the proposed methodology was tested for water temperature data collected from domestic G_water supply system.Methods : We applied the following analytical procedure to identify outliers in the water quality data: First, a normality test was performed on the collected data. If normality condition was satisfied, the Z-score was used. However, if the normality condition was not satisfied, outliers were identified using the quartile, and the limitations of the existing methodology were analyzed. Second, we decomposed the intrinsic mode function using empirical mode decomposition and ensemble empirical mode decomposition for the collected data, and then considered the occurrence of modal mixing. Finally, a group of intrinsic mode functions was selected using statistical characteristics to identify outliers. In addition, the performance of the method was verified after removing and interpolating outliers using regression analysis and Cook’s distance.Results and Discussion : In the case of water temperature data, as normality condition was not satisfied, outlier identification was carried out by applying the modified quartile method. It was confirmed that outliers distributed within the seasonal component could not be identified at all. In the case of empirical mode decomposition, modal mixing occurred because of the effect of outliers. However, in the case of the ensemble empirical mode decomposition, modal mixing was resolved and the distinct seasonal components were decomposed as intrinsic mode functions. The intrinsic mode functions were synthesized, which showed statistical correlation with the raw water temperature data. As a result of developing a regression model using the synthesized intrinsic mode functions and raw water temperature data and performing outlier search based on Cook’s distances, we concluded that various outliers distributed within the seasonal component could be effectively identified.Conclusions : Considering that satisfactory results could be derived from statistical analysis of the data collected from the automated water quality monitoring system, it can be concluded that outlier identification procedures are essential. However, in the case of the conventional univariate outlier search method, it is apparent that the outlier search performance is significantly poor for data with strong inherent variability, and the interpolation method for the searched outlier cannot be performed. Conversely, the outlier identification method based on ensemble empirical mode decomposition and regression analysis proposed in this study shows excellent discrimination performance for outliers distributed in data with strong inherent variability. Moreover, this method has the advantage of reducing the analyst’s dependence on subjective judgment by presenting statistical cutoff criteria. An additional advantage of the method is that data can be interpolated after removing outliers using intrinsic mode functions. Therefore, the outlier search and interpolation method proposed in this study is expected to have greater applicability as a more effective analysis tool compared to the existing univariate outlier search method.

Seasonal overturn and stratification changes drive deep-water warming in one of Earth’s largest lakes

Most of Earth’s fresh surface water is consolidated in just a few of its largest lakes, and because of their unique response to environmental conditions, lakes have been identified as climate change sentinels. While the response of lake surface water temperatures to climate change is well documented from satellite and summer in situ measurements, our understanding of how water temperatures in large lakes are responding at depth is limited, as few large lakes have detailed long-term subsurface observations. We present an analysis of three decades of high frequency (3-hourly and hourly) subsurface water temperature data from Lake Michigan. This unique data set reveals that deep water temperatures are rising in the winter and provides precise measurements of the timing of fall overturn, the point of minimum temperature, and the duration of the winter cooling period. Relationships from the data show a shortened winter season results in higher subsurface temperatures and earlier onset of summer stratification. Shifts in the thermal regimes of large lakes will have profound impacts on the ecosystems of the world’s surface freshwater.