scholarly journals Compressing atmospheric data into its real information content

2021 ◽  
Vol 1 (11) ◽  
pp. 713-724 ◽  
Author(s):  
Milan Klöwer ◽  
Miha Razinger ◽  
Juan J. Dominguez ◽  
Peter D. Düben ◽  
Tim N. Palmer
Keyword(s):  
Information Content ◽  
Temporal Correlation ◽  
Turing Test ◽  
Climate Forecast ◽  
Weather And Climate ◽  
Atmospheric Data ◽  
Forecast Data ◽  
Spatio Temporal ◽  
End Use ◽  
Monitoring Service

AbstractHundreds of petabytes are produced annually at weather and climate forecast centers worldwide. Compression is essential to reduce storage and to facilitate data sharing. Current techniques do not distinguish the real from the false information in data, leaving the level of meaningful precision unassessed. Here we define the bitwise real information content from information theory for the Copernicus Atmospheric Monitoring Service (CAMS). Most variables contain fewer than 7 bits of real information per value and are highly compressible due to spatio-temporal correlation. Rounding bits without real information to zero facilitates lossless compression algorithms and encodes the uncertainty within the data itself. All CAMS data are 17× compressed relative to 64-bit floats, while preserving 99% of real information. Combined with four-dimensional compression, factors beyond 60× are achieved. A data compression Turing test is proposed to optimize compressibility while minimizing information loss for the end use of weather and climate forecast data.

scholarly journals Compressing atmospheric data into its real information content

2021 ◽  
Author(s):  
Milan Kloewer ◽  
Miha Razinger ◽  
Juan Dominguez ◽  
Peter Dueben ◽  
Tim Palmer
Keyword(s):  
Information Content ◽  
Temporal Correlation ◽  
Turing Test ◽  
Forecast Errors ◽  
Climate Forecast ◽  
Weather And Climate ◽  
Spatio Temporal ◽  
End Use ◽  
Monitoring Service ◽  
Correlation Factors

Abstract Hundreds of petabytes of data are produced annually at weather and climate forecast centres worldwide. Compression is inevitable to reduce storage and to facilitate data sharing. Current techniques do not distinguish the real from the false information in data. We define the bitwise real information content from information theory for data from the Copernicus Atmospheric Monitoring Service (CAMS). Most variables contain less than 7 bits of real information per value, which are also highly compressible due to spatio-temporal correlation. Rounding bits without real information to zero facilitates lossless compression algorithms and encodes the uncertainty within the data itself. The entire CAMS data is compressed by a factor of 17x, relative to 64-bit floats, while preserving 99% of real information. Combined with 4-dimensional compression to exploit the spatio-temporal correlation, factors beyond 60x are achieved without an increase in forecast errors. A data compression Turing test is proposed to optimize compressibility while minimizing information loss for the end use of weather and climate forecast data.

Learning spatio-temporal correlation filter for visual tracking

2021 ◽  
Vol 436 ◽  
pp. 273-282
Author(s):  
Youmin Yan ◽  
Xixian Guo ◽  
Jin Tang ◽  
Chenglong Li ◽  
Xin Wang
Keyword(s):  
Visual Tracking ◽  
Temporal Correlation ◽  
Correlation Filter ◽  
Spatio Temporal

scholarly journals A Spatio-Temporal Local Association Query Algorithm for Multi-Source Remote Sensing Big Data

2021 ◽  
Vol 13 (12) ◽  
pp. 2333
Author(s):  
Lilu Zhu ◽  
Xiaolu Su ◽  
Yanfeng Hu ◽  
Xianqing Tai ◽  
Kun Fu
Keyword(s):  
Remote Sensing ◽  
Remote Sensing Data ◽  
Temporal Correlation ◽  
Local Association ◽  
Sensing Data ◽  
Multi Scale ◽  
Correlation Networks ◽  
Efficient Integration ◽  
Spatio Temporal ◽  
Query Algorithm

It is extremely important to extract valuable information and achieve efficient integration of remote sensing data. The multi-source and heterogeneous nature of remote sensing data leads to the increasing complexity of these relationships, and means that the processing mode based on data ontology cannot meet requirements any more. On the other hand, the multi-dimensional features of remote sensing data bring more difficulties in data query and analysis, especially for datasets with a lot of noise. Therefore, data quality has become the bottleneck of data value discovery, and a single batch query is not enough to support the optimal combination of global data resources. In this paper, we propose a spatio-temporal local association query algorithm for remote sensing data (STLAQ). Firstly, we design a spatio-temporal data model and a bottom-up spatio-temporal correlation network. Then, we use the method of partition-based clustering and the method of spectral clustering to measure the correlation between spatio-temporal correlation networks. Finally, we construct a spatio-temporal index to provide joint query capabilities. We carry out local association query efficiency experiments to verify the feasibility of STLAQ on multi-scale datasets. The results show that the STLAQ weakens the barriers between remote sensing data, and improves their application value effectively.

Technology Research on the Fault Diagnosis of Wireless Sensor Network System

2013 ◽  
Vol 846-847 ◽  
pp. 442-445
Author(s):  
Chun Lin He
Keyword(s):  
Fault Diagnosis ◽  
Sensor Network ◽  
Simulation Experiment ◽  
Temporal Correlation ◽  
Sensor Nodes ◽  
Wireless Sensor ◽  
Network System ◽  
Technology Research ◽  
Spatio Temporal ◽  
Network Fault

The fault diagnosis technology have emerged and developed rapidly with the development of wireless sensor networks and requirements of applications improve. This paper describes two commonly used sensor network fault modeling. What is more, in order to solve this problem that sensor nodes are vulnerable and therefore produce wrong data, the paper proposes a distributed fault detecting algorithm based on spatio-temporal correlation among data of adjacent nodes. The simulation experiment shows that the algorithm can efficiently detect errors in the network and very few errors are introduced.

The Three-Cornered Hat Method for Estimating Error Variances of Three or More Atmospheric Data Sets – Part II: Evaluating Radio Occultation and Radiosonde Observations, Global Model Forecasts, and Reanalyses

2021 ◽  
Author(s):  
Therese Rieckh ◽  
Jeremiah P. Sjoberg ◽  
Richard A. Anthes
Keyword(s):  
Radio Occultation ◽  
State Of The Art ◽  
Specific Humidity ◽  
Data Sets ◽  
Error Growth ◽  
Atmospheric Conditions ◽  
Error Statistics ◽  
Weather And Climate ◽  
Atmospheric Data ◽  
The Impact

AbstractWe apply the three-cornered hat (3CH) method to estimate refractivity, bending angle, and specific humidity error variances for a number of data sets widely used in research and/or operations: radiosondes, radio occultation (COSMIC, COSMIC-2), NCEP global forecasts, and nine reanalyses. We use a large number and combinations of data sets to obtain insights into the impact of the error correlations among different data sets that affect 3CH estimates. Error correlations may be caused by actual correlations of errors, representativeness differences, or imperfect co-location of the data sets. We show that the 3CH method discriminates among the data sets and how error statistics of observations compare to state-of-the-art reanalyses and forecasts, as well as reanalyses that do not assimilate satellite data. We explore results for October and November 2006 and 2019 over different latitudinal regions and show error growth of the NCEP forecasts with time. Because of the importance of tropospheric water vapor to weather and climate, we compare error estimates of refractivity for dry and moist atmospheric conditions.

scholarly journals Energy Production Benefits by Wind and Wave Energies for the Autonomous System of Crete

Energies ◽  
2018 ◽  
Vol 11 (10) ◽  
pp. 2741 ◽  
Author(s):  
George Lavidas ◽  
Vengatesan Venugopal
Keyword(s):  
Wind Turbine ◽  
Wave Energy ◽  
Energy Production ◽  
Large Scale ◽  
Capital Expenditure ◽  
Temporal Correlation ◽  
Clean Energy ◽  
Limiting Factor ◽  
Spatio Temporal ◽  
Visual Impacts

At autonomous electricity grids Renewable Energy (RE) contributes significantly to energy production. Offshore resources benefit from higher energy density, smaller visual impacts, and higher availability levels. Offshore locations at the West of Crete obtain wind availability ≈80%, combining this with the installation potential for large scale modern wind turbines (rated power) then expected annual benefits are immense. Temporal variability of production is a limiting factor for wider adaptation of large offshore farms. To this end multi-generation with wave energy can alleviate issues of non-generation for wind. Spatio-temporal correlation of wind and wave energy production exhibit that wind and wave hybrid stations can contribute significant amounts of clean energy, while at the same time reducing spatial constrains and public acceptance issues. Offshore technologies can be combined as co-located or not, altering contribution profiles of wave energy to non-operating wind turbine production. In this study a co-located option contributes up to 626 h per annum, while a non co-located solution is found to complement over 4000 h of a non-operative wind turbine. Findings indicate the opportunities associated not only in terms of capital expenditure reduction, but also in the ever important issue of renewable variability and grid stability.

Sign in / Sign up

Export Citation Format

Share Document