Measuring information content from observations for data assimilation: relative entropy versus shannon entropy difference

2007 ◽  
Vol 59 (2) ◽  
pp. 198-209 ◽  
Author(s):  
Qin Xu
2016 ◽  
Author(s):  
Michael Kahnert ◽  
Emma Andersson

Abstract. We theoretically and numerically investigate the problem of assimilating lidar observations of extinction and backscattering coefficients of aerosols into a chemical transport model. More specifically, we consider the inverse problem of determining the chemical composition of aerosols from these observations. The main questions are how much information the observations contain to constrain the particles' chemical composition, and how one can optimise a chemical data assimilation system to make maximum use of the available information. We first quantify the information content of the measurements by computing the singular values of the observation operator. From the singular values we can compute the number of signal degrees of freedom and the reduction in Shannon entropy. For an observation standard deviation of 10 %, it is found that simultaneous measurements of extinction and backscattering allows us to constrain twice as many model variables as extinction measurements alone. The same holds for measurements at two wavelengths compared to measurements at a single wavelength. However, when we extend the set of measurements from two to three wavelengths then we observe only a small increase in the number of signal degrees of freedom, and a minor change in the Shannon entropy. The information content is strongly sensitive to the observation error; both the number of signal degrees of freedom and the reduction in Shannon entropy steeply decrease as the observation standard deviation increases in the range between 1 and 100 %. The right singular vectors of the observation operator can be employed to transform the model variables into a new basis in which the components of the state vector can be divided into signal-related and noise-related components. We incorporate these results in a chemical data assimilation algorithm by introducing weak constraints that restrict the assimilation algorithm to acting on the signal-related model variables only. This ensures that the information contained in the measurements is fully exploited, but not over-used. Numerical experiments confirm that the constrained data assimilation algorithm solves the inverse problem in a way that automatises the choice of control variables, and that restricts the minimisation of the costfunction to the signal-related model variables.


2012 ◽  
Vol 518-523 ◽  
pp. 1586-1591
Author(s):  
Hao Zhang ◽  
Ze Meng Zhao ◽  
Ahmet Palazoglu ◽  
Wei Sun

Surface ozone in the air boundary layer is one of the most harmful air pollutants produced by photochemical reaction between nitrogen oxides and volatile hydrocarbons, which causes great damage to human beings and environment. The prediction of surface ozone levels plays an important role in the control and the reduction of air pollutants. As model-driven statistical prediction models, hidden Markov Models (HMMs) are rich in mathematical structure and work well in many important applications. Due to the complex structure of HMM, long observation sequences would increase computational load by geometric ratio. In order to reduce training time, wavelet decomposition is used to compress the original observations into shorter ones. During compression step, observation sequences compressed by different wavelet basis functions keep different information content. This may have impact on prediction results. In this paper, ozone prediction performance of HMM based on different wavelet basis functions are discussed. Shannon entropy is employed to measure how much information content is kept in the new sequence compared to the original one. Data from Houston Metropolitan Area, TX are used in this paper. Results show that wavelet basis functions used in data compression step can affect the HMM model performance significantly. The new sequence with the maximum Shannon entropy generates the best prediction result.


2016 ◽  
Vol 144 (8) ◽  
pp. 2927-2945
Author(s):  
Nedjeljka Žagar ◽  
Jeffrey Anderson ◽  
Nancy Collins ◽  
Timothy Hoar ◽  
Kevin Raeder ◽  
...  

Abstract Global data assimilation systems for numerical weather prediction (NWP) are characterized by significant uncertainties in tropical analysis fields. Furthermore, the largest spread of global ensemble forecasts in the short range on all scales is in the tropics. The presented results suggest that these properties hold even in the perfect-model framework and the ensemble Kalman filter data assimilation with a globally homogeneous network of wind and temperature profiles. The reasons for this are discussed by using the modal analysis, which provides information about the scale dependency of analysis and forecast uncertainties and information about the efficiency of data assimilation to reduce the prior uncertainties in the balanced and inertio-gravity dynamics. The scale-dependent representation of variance reduction of the prior ensemble by the data assimilation shows that the peak efficiency of data assimilation is on the synoptic scales in the midlatitudes that are associated with quasigeostrophic dynamics. In contrast, the variance associated with the inertia–gravity modes is less successfully reduced on all scales. A smaller information content of observations on planetary scales with respect to the synoptic scales is discussed in relation to the large-scale tropical uncertainties that current data assimilation methodologies do not address successfully. In addition, it is shown that a smaller reduction of the large-scale uncertainties in the prior state for NWP in the tropics than in the midlatitudes is influenced by the applied radius for the covariance localization.


2013 ◽  
Vol 1 (1) ◽  
pp. 106-138 ◽  
Author(s):  
K. Singh ◽  
A. Sandu ◽  
M. Jardak ◽  
K. W. Bowman ◽  
M. Lee

2020 ◽  
Vol 19 (04) ◽  
pp. 2050043 ◽  
Author(s):  
Hamidreza Namazi

In this paper, we employ the information theory to analyze the development of brain as the newborn ages. We compute the Shannon entropy of Electroencephalography (EEG) signal during sleep for 10 groups of newborns who are aged 36 weeks to 45 weeks (first to the last group). Based on the obtained results, EEG signals for newborns in 36 weeks have the lowest information content, whereas EEG signals for newborns in 45 weeks show the greatest information content. Therefore, we concluded that the information content of EEG signal increases as the age of newborn increases. Th result of statistical analysis demonstrated that the influence of increment of age of newborn on the variations of informant content of their EEG signals was significant.


2019 ◽  
Vol 1 ◽  
pp. 1-1 ◽  
Author(s):  
Peichao Gao ◽  
Hong Zhang ◽  
Zhilin Li

<p><strong>Abstract.</strong> Entropy is an important concept that originated in thermodynamics. It is the subject of the famous Second Law of Thermodynamics, which states that “the entropy of a closed system increases continuously and irrevocably toward a maximum” (Huettner 1976, 102) or “the disorder in the universe always increases” (Framer and Cook 2013, 21). Accordingly, it has been widely regarded as an ideal measure of disorder. Its computation can be theoretically performed according to the Boltzmann equation, which was proposed by the Austrian physicist Ludwig Boltzmann in 1872. In practice, however, the Boltzmann equation involves two problems that are difficult to solve, that is the definition of the macrostate of a system and the determination of the number of possible microstates in the microstate. As noted by the American sociologist Kenneth Bailey, “when the notion of entropy is extended beyond physics, researchers may not be certain how to specify and measure the macrostate/microstate relations” (Bailey 2009, 151). As a result, this entropy (also referred to as Boltzmann entropy and thermodynamic entropy) has remained largely at a conceptual level.</p><p> In practice, the widely used entropy is actually proposed by the American mathematician, electrical engineer, and cryptographer Claude Elwood Shannon in 1948, hence the term Shannon entropy. Shannon entropy was proposed to quantify the statistical disorder of telegraph messages in the area of communications. The quantification result was interpreted as the information content of a telegraph message, hence also the term information entropy. This entropy has served as the cornerstone of information theory and was introduced to various fields including chemistry, biology, and geography. It has been widely utilized to quantify the information content of geographic data (or spatial data) in either a vector format (i.e., vector data) or a raster format (i.e., raster data). However, only the statistical information of spatial data can be quantified by using Shannon entropy. The spatial information is ignored by Shannon entropy; for example, a grey image and its corresponding error image share the same Shannon entropy.</p><p> Therefore, considerable efforts have been made to improve the suitability of Shannon entropy for spatial data, and a number of improved Shannon entropies have been put forward. Rather than further improving Shannon entropy, this study introduces a novel strategy, namely shifting back from Shannon entropy to Boltzmann entropy. There are two advantages of employing Boltzmann entropy. First, as previously mentioned, Boltzmann entropy is the ideal, standard measure of disorder or information. It is theoretically capable of quantifying not only the statistical information but also the spatial information of a data set. Second, Boltzmann entropy can serve as the bridge between spatial patterns and thermodynamic interpretations. In this sense, the Boltzmann entropy of spatial data may have wider applications. In this study, Boltzmann entropy is employed to quantify the spatial information of raster data, such as images, raster maps, digital elevation models, landscape mosaics, and landscape gradients. To this end, the macrostate of raster data is defined, and the number of all possible microstates in the macrostate is determined. To demonstrate the usefulness of Boltzmann entropy, it is applied to satellite remote sensing image processing, and a comparison is made between its performance and that of Shannon entropy.</p>


Sign in / Sign up

Export Citation Format

Share Document