WaterML2.0: development of an open standard for hydrological time-series data exchange

2013 ◽  
Vol 16 (2) ◽  
pp. 425-446 ◽  
Author(s):  
P. Taylor ◽  
S. Cox ◽  
G. Walker ◽  
D. Valentine ◽  
P. Sheahan
Keyword(s):  
Time Series ◽  
Working Group ◽  
Data Exchange ◽  
Time Series Data ◽  
Series Data ◽  
Observation Data ◽  
Environmental Reporting ◽  
Open Standard ◽  
Terrestrial Water ◽  
Future Work

The increasing global demand on freshwater is resulting in nations improving their terrestrial water monitoring and reporting systems to better understand the availability, and quality, of this valuable resource. A barrier to this is the inability for stakeholders to share information relating to water observations data: traditional hydrological information systems have relied on internal custom data formats to exchange data, leading to issues in data integration and exchange. Organisations are looking to information standards to assist in data exchange, integration and interpretation to lower costs in use, and re-use, of monitoring data. The WaterML2.0 Standards Working Group (SWG), working within the Open Geospatial Consortium (OGC) and in cooperation with the joint OGC-World Meteorological Organisation (WMO) Hydrology Domain Working Group (HDWG), has developed an open standard for the exchange of water observation data. The focus of the standard is time-series data, commonly used for hydrological applications such as flood forecasting, environmental reporting and hydrological infrastructure, where a lack of standards inhibits efficient re-use and automation. This paper describes the development methodology and principles of WaterML2.0, key parts of its information model, implementation scenarios, evaluation and future work. WaterML2.0 was adopted by the OGC as an official standard in September 2012.

scholarly journals AstroCatR: a mechanism and tool for efficient time series reconstruction of large-scale astronomical catalogues

2020 ◽  
Vol 496 (1) ◽  
pp. 629-637
Author(s):  
Ce Yu ◽  
Kun Li ◽  
Shanjiang Tang ◽  
Chao Sun ◽  
Bin Ma ◽  
...  
Keyword(s):  
Time Series ◽  
High Performance ◽  
Large Scale ◽  
Extrasolar Planets ◽  
Time Series Data ◽  
Series Data ◽  
Data Sets ◽  
Observation Data ◽  
Data Volume ◽  
And Performance

ABSTRACT Time series data of celestial objects are commonly used to study valuable and unexpected objects such as extrasolar planets and supernova in time domain astronomy. Due to the rapid growth of data volume, traditional manual methods are becoming extremely hard and infeasible for continuously analysing accumulated observation data. To meet such demands, we designed and implemented a special tool named AstroCatR that can efficiently and flexibly reconstruct time series data from large-scale astronomical catalogues. AstroCatR can load original catalogue data from Flexible Image Transport System (FITS) files or data bases, match each item to determine which object it belongs to, and finally produce time series data sets. To support the high-performance parallel processing of large-scale data sets, AstroCatR uses the extract-transform-load (ETL) pre-processing module to create sky zone files and balance the workload. The matching module uses the overlapped indexing method and an in-memory reference table to improve accuracy and performance. The output of AstroCatR can be stored in CSV files or be transformed other into formats as needed. Simultaneously, the module-based software architecture ensures the flexibility and scalability of AstroCatR. We evaluated AstroCatR with actual observation data from The three Antarctic Survey Telescopes (AST3). The experiments demonstrate that AstroCatR can efficiently and flexibly reconstruct all time series data by setting relevant parameters and configuration files. Furthermore, the tool is approximately 3× faster than methods using relational data base management systems at matching massive catalogues.

Implementing a new data acquisition system for the advanced integrated atmospheric observation system KITcube

2020 ◽  
Author(s):  
Martin Kohler ◽  
Mahnaz Fekri ◽  
Andreas Wieser ◽  
Jan Handwerker
Keyword(s):  
Time Series ◽  
Data Acquisition ◽  
Real Time ◽  
Binary Data ◽  
Data Exchange ◽  
Data Access ◽  
Series Data ◽  
Observation System ◽  
Observation Data ◽  
File Server

<p>KITcube (Kalthoff et al, 2013) is a mobile advanced integrated observation system for the measurement of meteorological processes within a volume of 10x10x10 km<sup>3</sup>. A large variety of different instruments from in-situ sensors to scanning remote sensing devices are deployed during campaigns. The simultaneous operation and real time instrument control needed for maximum instrument synergy requires a real-time data management designed to cover the various user needs: Save data acquisition, fast loading, compressed storage, easy data access, monitoring and data exchange. Large volumes of data such as raw and semi-processed data of various data types, from simple ASCII time series to high frequency multi-dimensional binary data provide abundant information, but makes the integration and efficient management of such data volumes to a challenge.<br>Our data processing architecture is based on open source technologies and involves the following five sections: 1) Transferring: Data and metadata collected during a campaign are stored on a file server. 2) Populating the database: A relational database is used for time series data and a hybrid database model for very large, complex, unstructured data. 3) Quality control: Automated checks for data acceptance and data consistency. 4) Monitoring: Data visualization in a web-application. 5) Data exchange: Allows the exchange of observation data and metadata in specified data formats with external users.<br>The implemented data architecture and workflow is illustrated in this presentation using data from the MOSES project (http://moses.eskp.de/home).</p><p>References:</p><p><strong>KITcube - A mobile observation platform for convection studies deployed during HyMeX </strong>.<br>Kalthoff, N.; Adler, B.; Wieser, A.; Kohler, M.; Träumner, K.; Handwerker, J.; Corsmeier, U.; Khodayar, S.; Lambert, D.; Kopmann, A.; Kunka, N.; Dick, G.; Ramatschi, M.; Wickert, J.; Kottmeier, C.<br>2013. Meteorologische Zeitschrift, 22 (6), 633–647. doi:10.1127/0941-2948/2013/0542 </p>

scholarly journals rhizoTrak: A flexible open source Fiji plugin for user-friendly manual annotation of time-series images from minirhizotrons

2019 ◽  
Author(s):  
Birgit Möller ◽  
Hongmei Chen ◽  
Tino Schmidt ◽  
Axel Zieschank ◽  
Roman Patzak ◽  
...  
Keyword(s):  
Time Series ◽  
Open Source ◽  
Data Exchange ◽  
Time Series Data ◽  
Full Range ◽  
Open Data ◽  
Series Data ◽  
Root Turnover ◽  
User Friendly ◽  
Time Series Images

AbstractBackground and aimsMinirhizotrons are commonly used to study root turnover which is essential for understanding ecosystem carbon and nutrient cycling. Yet, extracting data from minirhizotron images requires intensive annotation effort. Existing annotation tools often lack flexibility and provide only a subset of the required functionality. To facilitate efficient root annotation in minirhizotrons, we present the user-friendly open source tool rhizoTrak.Methods and resultsrhizoTrak builds on TrakEM2 and is publically available as Fiji plugin. It uses treelines to represent branching structures in roots and assigns customizable status labels per root segment. rhizoTrak offers configuration options for visualization and various functions for root annotation mostly accessible via keyboard shortcuts. rhizoTrak allows time-series data import and particularly supports easy handling and annotation of time series images. This is facilitated via explicit temporal links (connectors) between roots which are automatically generated when copying annotations from one image to the next. rhizoTrak includes automatic consistency checks and guided procedures for resolving conflicts. It facilitates easy data exchange with other software by supporting open data formats.ConclusionsrhizoTrak covers the full range of functions required for user-friendly and efficient annotation of time-series images. Its flexibility and open source nature will foster efficient data acquisition procedures in root studies using minirhizotrons.

scholarly journals Observation error covariance specification in dynamical systems for data assimilation using recurrent neural networks

2021 ◽  
Author(s):  
Sibo Cheng ◽  
Mingming Qiu
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Dynamical Systems ◽  
Data Assimilation ◽  
Recurrent Neural Networks ◽  
Time Series Data ◽  
Series Data ◽  
Observation Data ◽  
Observation Error ◽  
Error Covariance

AbstractData assimilation techniques are widely used to predict complex dynamical systems with uncertainties, based on time-series observation data. Error covariance matrices modeling is an important element in data assimilation algorithms which can considerably impact the forecasting accuracy. The estimation of these covariances, which usually relies on empirical assumptions and physical constraints, is often imprecise and computationally expensive, especially for systems of large dimensions. In this work, we propose a data-driven approach based on long short term memory (LSTM) recurrent neural networks (RNN) to improve both the accuracy and the efficiency of observation covariance specification in data assimilation for dynamical systems. Learning the covariance matrix from observed/simulated time-series data, the proposed approach does not require any knowledge or assumption about prior error distribution, unlike classical posterior tuning methods. We have compared the novel approach with two state-of-the-art covariance tuning algorithms, namely DI01 and D05, first in a Lorenz dynamical system and then in a 2D shallow water twin experiments framework with different covariance parameterization using ensemble assimilation. This novel method shows significant advantages in observation covariance specification, assimilation accuracy, and computational efficiency.

scholarly journals Inference and Validation of The Structure of Lotka-Volterra Models

2021 ◽  
Author(s):  
Eberhard Voit ◽  
Jacob Davis ◽  
Daniel Olivenca
Keyword(s):  
Time Series ◽  
Time Series Data ◽  
The Other ◽  
Microbial Populations ◽  
Series Data ◽  
Observation Data ◽  
Volterra Models ◽  
Rich Data ◽  
Over Time ◽  
Mixed Community

Abstract For close to a century, Lotka-Volterra (LV) models have been used to investigate interactions among populations of different species. For a few species, these investigations are straightforward. However, with the arrival of large and complex microbiomes, unprecedently rich data have become available and await analysis. In particular, these data require us to ask which microbial populations of a mixed community affect other populations, whether these influences are activating or inhibiting and how the interactions change over time. Here we present two new inference strategies for interaction parameters that are based on a new algebraic LV inference (ALVI) method. One strategy uses different survivor profiles of communities grown under similar conditions, while the other pertains to time series data. In addition, we address the question of whether observation data are compliant with the LV structure or require a richer modeling format.

Accessing environmental time series data in R from Sensor Observation Services with ease

2020 ◽  
Author(s):  
Daniel Nüst ◽  
Eike H. Jürrens ◽  
Benedikt Gräler ◽  
Simon Jirka
Keyword(s):  
Time Series ◽  
Time Series Data ◽  
Data Access ◽  
R Package ◽  
Series Data ◽  
Observation Data ◽  
Software Environment ◽  
Technical Standards ◽  
Seamless Integration ◽  
Environmental Time Series

<p>Time series data of in-situ measurements is the key to many environmental studies. The first challenge in any analysis typically arises when the data needs to be imported into the analysis framework. Standardisation is one way to lower this burden. Unfortunately, relevant interoperability standards might be challenging for non-IT experts as long as they are not dealt with behind the scenes of a client application. One standard to provide access to environmental time series data is the Sensor Observation Service (SOS, ) specification published by the Open Geospatial Consortium (OGC). SOS instances are currently used in a broad range of applications such as hydrology, air quality monitoring, and ocean sciences. Data sets provided via an SOS interface can be found around the globe from Europe to New Zealand.</p><p>The R package sos4R (Nüst et al., 2011) is an extension package for the R environment for statistical computing and visualization (), which has been demonstrated a a powerful tools for conducting and communicating geospatial research (cf. Pebesma et al., 2012; ). sos4R comprises a client that can connect to an SOS server. The user can use it to query data from SOS instances using simple R function calls. It provides a convenience layer for R users to integrate observation data from data access servers compliant with the SOS standard without any knowledge about the underlying technical standards. To further improve the usability for non-SOS experts, a recent update to sos4R includes a set of wrapper functions, which remove complexity and technical language specific to OGC specifications. This update also features specific consideration of the OGC SOS 2.0 Hydrology Profile and thereby opens up a new scientific domain.</p><p>In our presentation we illustrate use cases and examples building upon sos4R easing the access of time series data in an R and Shiny () context. We demonstrate how the abstraction provided in the client library makes sensor observation data for accessible and further show how sos4R allows the seamless integration of distributed observations data, i.e., across organisational boundaries, into transparent and reproducible data analysis workflows.</p><p><strong>References</strong></p><p>Nüst D., Stasch C., Pebesma E. (2011) Connecting R to the Sensor Web. In: Geertman S., Reinhardt W., Toppen F. (eds) Advancing Geoinformation Science for a Changing World. Lecture Notes in Geoinformation and Cartography, Springer. </p><p>Pebesma, E., Nüst, D., & Bivand, R. (2012). The R software environment in reproducible geoscientific research. Eos, Transactions American Geophysical Union, 93(16), 163–163. </p>

The Design and Implementation of Multiple Square Wave Pulse Sequence Recorder

2011 ◽  
Vol 121-126 ◽  
pp. 1692-1696
Author(s):  
Xin Quan Jiao ◽  
Yong Xing Yao ◽  
Qing Meng
Keyword(s):  
Time Series ◽  
Data Exchange ◽  
Time Series Data ◽  
Pulse Sequence ◽  
Pulse Signal ◽  
Series Data ◽  
Signal Timing ◽  
Wave Pulse ◽  
Square Wave ◽  
Design And Implementation

For the need to test square wave pulse sequence, designed and implemented a recorder. Recorder using FPGA for judging square wave pulse’s edge and precise timing functions, using C8051F MCU to storage the results of time-series data and printing functions, FPGA and MCU using a custom protocol for data exchange and command transfer. After experimental verification, the recorder have 1ms accuracy to record 28 roads square wave pulse signal timing, and fixed-format print the results.

scholarly journals Statistical analysis of night radiance RH using VIIRS day/night band satellite time series data

2019 ◽  
Vol 8 (3) ◽  
pp. 249
Author(s):  
Jyoti U. Devkota
Keyword(s):  
Time Series ◽  
Time Series Data ◽  
Physical Capital ◽  
Arithmetic Mean ◽  
Rate Of Change ◽  
Series Data ◽  
Observation Data ◽  
Order Difference ◽  
First Order ◽  
Night Lights

<p class="SAP-AffiliationLastline">Amount of night lights in an area is a proxy indicator of electricity consumption. This is interlinked to indicators of economic growth such as socio-economic activities, urban population size, physical capital, incidence of poverty. These night lights are generated by renewable and non renewable energy source. In this paper the behavior of night radiance RH data was minutely analyzed over a period of 28 hour; Visible Infrared Imaging Radiometer Suite Day/Night Band (VIIRS DNB) satellite earth observation data were used. These 28 hours and 8936 observations time series data is from 2 September 2018 to 4 September 2018. The behavior of night radiance RH data over 122 time intervals was analyzed using box plots. It was seen that the arithmetic mean of RH data is more sensitive than the arithmetic mean of first order difference of RH data. The first order difference of night radiance RH was regressed on night radiance over 110 intervals of time. The box plot of slope and intercept of this linear regression showed the behavior of these regression parameters over 110 intervals of time. It is seen that the data are more scattered with respect to slope than with respect to intercept. This implies that the rate of change in RH with respect to change in time has more variability that the intrinsic value of RH data at the sampled point of time.</p>

Sign in / Sign up

Export Citation Format

Share Document