Quality Control of Meteorological Observations

Abstract. Sub-daily meteorological observations are needed for input to and assessment of high-resolution reanalysis products to improve understanding of weather and climate variability. While there are millions such weather observations that have been collected by various organizations, many are yet to be transcribed into a useable format. Under the auspices of the European Union funded Uncertainties in Ensembles of Regional ReAnalysis (UERRA) project, we describe the compilation and development of a digital dataset of 8.8 million meteorological observations rescued across the European and southern Mediterranean region, many of them Essential Climate Variables (ECVs) as defined by the Global Climate Observing System (GCOS). By presenting the entire chain of data preparation, from the identification of regions lacking in digitized sub-daily data and the locating of original sources, through the digitization of the observations to the quality control procedures applied, we provide a rescued dataset that is as traceable as possible for use by the research community. Data from 127 stations and of 15 climate variables in the northern Africa and European sectors have been prepared for the period 1877 to 2012. Quality control of the data using a two-step semi-automatic statistical approach identified 3.5 % of observations that required correction or removal, on par with previous data rescue efforts. In addition to providing a new sub-daily meteorological dataset for the research community, our experience in the development of this UERRA sub-daily dataset gives us an opportunity to share guidance on future data rescue projects. All data are available on PANGAEA: doi:10.1594/PANGAEA.886511.

Download Full-text

Quality Control of Ground-Based Radiometric Observations of Integrated Moisture Using Surface Meteorological Observations

Journal of Atmospheric and Oceanic Technology ◽

10.1175/1520-0426(1994)011<0599:qcogbr>2.0.co;2 ◽

1994 ◽

Vol 11 (2) ◽

pp. 599-607

Author(s):

Maia S. Tatarskata ◽

Viatcheslav V. Tatarskii ◽

Valerian I. Tatarskii ◽

Ed R. Westwater

Keyword(s):

Quality Control ◽

Meteorological Observations

Download Full-text

Data rescue of national and international meteorological observations at Deutscher Wetterdienst

Advances in Science and Research ◽

10.5194/asr-12-57-2015 ◽

2015 ◽

Vol 12 (1) ◽

pp. 57-61 ◽

Cited By ~ 16

Author(s):

F. Kaspar ◽

B. Tinz ◽

H. Mächel ◽

L. Gates

Keyword(s):

Quality Control ◽

Current Status ◽

Weather Observations ◽

The World ◽

Meteorological Observations

Abstract. Germany's national meteorological service (Deutscher Wetterdienst, DWD) houses in Offenbach and Hamburg huge archives of historical handwritten journals of weather observations. They comprise not only observations from Germany, but also of the oceans and land stations in many parts of the world. DWD works on the digitization and quality control of these archives. The current status is presented here.

Download Full-text

Rescue of Ukrainian early historical climatological data

10.5194/egusphere-egu2020-17698 ◽

2020 ◽

Author(s):

Dmytro Boichuk ◽

Jürg Luterbacher ◽

Rob Allan ◽

Olesya Skrynyk ◽

Vladyslav Sidenko ◽

...

Keyword(s):

Quality Control ◽

Extreme Events ◽

Meteorological Data ◽

Hard Copy ◽

Climate Services ◽

Present Information ◽

Control Procedures ◽

Temperature Records ◽

Meteorological Observations ◽

The 19Th Century

Modern climate applications and climate services are seeing the need for more data and information (including its historical part) on climate variability at high temporal and spatial resolution. Therefore, daily or even sub-daily meteorological data are required increasingly to feel this gap and provide the basis for climate research, extreme events analysis and impact studies.The main objective of our work is to present information on results of data rescue (DARE) activity conducted recently in the Ukrainian Hydrometeorological Institute (UHMI, Kyiv, Ukraine) in close collaboration with several national and international partners. Our DARE activity was concentrated mainly on the original sub-daily, pre-1850 meteorological observations conducted at eight meteorological stations located in the territory of modern Ukraine, namely Kyiv, Kharkiv, Poltava, Kamyanets-Podilsky, Lugansk, Dnipro, Kherson and Odesa. These eight stations are the only ones, whose pre-1850 data have been found in an archive of the Central Geophysical Observatory (CGO), an observation institution of the Ukrainian Weather Service.The data are contained in 38 special hard copy books. Before digitization, the book pages were photocopied to create a database of the images of all the paper sources. Its two copy versions are now stored at the UHMI and CGO, respectively. After the creation of the images database, the data were digitized manually by the authors. In total 291&#160;103 values were digitized. These include 165&#160;980 air temperature records (~57% of the total), 124&#160;376 atmospheric pressure measurements (~42.7%) and 747 precipitation totals (~0.3%).Quality control of the digitized data was conducted, including intercomparisons between the stations as well as comparisons with monthly temperature data that were digitized previously from other sources. The quality control procedures revealed a fairly good agreement among the rescued time series on the monthly time scale as well as a good accordance with the monthly data from other sources. However, several periods at some stations should be used with caution, due to relatively large discrepancies revealed. The rescued digital dataset can be used for different meteorological and climatological purposes, including the analysis of extreme events for the pre-1850 period in comparison with today&#8217;s climate, regional climatological studies, etc. The dataset is an important supplement to existing digitized archives of meteorological measurements that were performed in the first half of the 19th century.

Download Full-text

Rescue and quality control of sub-daily meteorological data collected at Montevergine Observatory (Southern Apennines), 1884–1963

Earth System Science Data ◽

10.5194/essd-12-1467-2020 ◽

2020 ◽

Vol 12 (2) ◽

pp. 1467-1487

Author(s):

Vincenzo Capozzi ◽

Yuri Cotroneo ◽

Pasquale Castagno ◽

Carmela De Vivo ◽

Giorgio Budillon

Keyword(s):

Quality Control ◽

Atmospheric Pressure ◽

Meteorological Data ◽

Tolerance Test ◽

Rate Of Change ◽

Distribution Model ◽

Water Vapour Pressure ◽

Southern Apennines ◽

Meteorological Observations ◽

Standard Practices

Abstract. Here we present the rescue of sub-daily meteorological observations collected from 1884 to 1963 at Montevergine Observatory, located in the Southern Apennines in Italy. The recovered dataset consists of 3-daily observations of the following atmospheric variables: dry-bulb temperature, wet-bulb temperature, water vapour pressure, relative humidity, atmospheric pressure, cloud type, cloud cover, rainfall, snowfall and precipitation type. The data, originally available only as paper-based records, have been digitized following the World Meteorological Organization standard practices. After a cross-check, the digitized data went through three different automatic quality control tests: the gross error test, which verifies whether the data are within acceptable range limits; the tolerance test, which flags whether values are above or below monthly climatological limits that are defined in accordance with a probability distribution model specific to each variable; and the temporal coherency test, which checks the rate of change and flags unrealistic jumps in consecutive values. The result of this process is the publication of a new historical dataset that includes, for the first time, digitized and quality-controlled sub-daily meteorological observations collected since the late 19th century in the Mediterranean region north of the 37th parallel. These data are critical to enhancing and complementing previously rescued sub-daily historical datasets – which are currently limited to atmospheric pressure observations only – in the central and northern Mediterranean regions. Furthermore, the Montevergine Observatory (MVOBS) dataset can enrich the understanding of high-altitude weather and climate variability, and it contributes to the improvement of the accuracy of reanalysis products prior the 1950s. Data are available on the NOAA's National Centers for Environmental Information (NCEI) public repository and are associated with a DOI: https://doi.org/10.25921/cx3g-rj98 (Capozzi et al., 2019).

Download Full-text

Rescue and quality control of sub-daily meteorological data collected at Montevergine Observatory (Southern Apennines), 1884–1963

10.5194/essd-2020-38 ◽

2020 ◽

Author(s):

Vincenzo Capozzi ◽

Yuri Cotroneo ◽

Pasquale Castagno ◽

Carmela De Vivo ◽

Giorgio Budillon

Keyword(s):

Quality Control ◽

Atmospheric Pressure ◽

Meteorological Data ◽

Tolerance Test ◽

Rate Of Change ◽

Distribution Model ◽

Water Vapour Pressure ◽

Southern Apennines ◽

Meteorological Observations ◽

Standard Practices

Abstract. Here we present the rescue of sub-daily meteorological observations collected from 1884 to 1963 at Montevergine Observatory, located on the Italian Southern Apennines. The recovered dataset consists of three daily observations of the following atmospheric variables: dry bulb temperature, wet bulb temperature, water vapour pressure, relative humidity, atmospheric pressure, cloud type, cloud cover, rainfall, snowfall and precipitation type. The data, originally available only as paper-based records, have been digitized following the World Meteorological Organization standard practices. After a cross-check, the digitized data went through three different automatic quality control tests: the gross error test which verifies if the data are within acceptable range limits; the tolerance test that flags if values are above or below monthly climatological limits which are defined in accordance with a probability distribution model specific for each variable; and the temporal coherency test that checks the rate of change flagging unrealistic jumps in consecutive values. The result of this process is the publication of a new historical dataset that includes, for the first time, digitized and quality-controlled sub-daily meteorological observations collected since the late 19th century in the Mediterranean region north of the 37th parallel. These data are critical to enhance and complement previously rescued sub-daily historical datasets in central and northern Mediterranean regions, currently limited to the atmospheric pressure observations only. Furthermore, MVOBS dataset can enrich the understanding of high altitude weather and climate variability and contributes to improve the accuracy of reanalysis products prior the 1950s. Data are available on the NOAA’s National Centers for Environmental Information (NCEI) public repository and are associated to a DOI (https://doi.org/10.25921/cx3g-rj98) (Capozzi et al., 2019).

Download Full-text

A rescued dataset of sub-daily meteorological observations for Europe and the southern Mediterranean region, 1877–2012

Earth System Science Data ◽

10.5194/essd-10-1613-2018 ◽

2018 ◽

Vol 10 (3) ◽

pp. 1613-1635 ◽

Cited By ~ 12

Author(s):

Linden Ashcroft ◽

Joan Ramon Coll ◽

Alba Gilabert ◽

Peter Domonkos ◽

Manola Brunet ◽

...

Keyword(s):

Quality Control ◽

Mediterranean Region ◽

Climate Variables ◽

Daily Data ◽

Weather Observations ◽

Weather And Climate ◽

Future Data ◽

Southern Mediterranean ◽

Control Procedures ◽

Meteorological Observations

Abstract. Sub-daily meteorological observations are needed for input to and assessment of high-resolution reanalysis products to improve understanding of weather and climate variability. While there are millions of such weather observations that have been collected by various organisations, many are yet to be transcribed into a useable format.Under the auspices of the Uncertainties in Ensembles of Regional ReAnalyses (UERRA) project, we describe the compilation and development of a digital dataset of 8.8 million meteorological observations of essential climate variables (ECVs) rescued across the European and southern Mediterranean region. By presenting the entire chain of data preparation, from the identification of regions lacking in digitised sub-daily data and the location of original sources, through the digitisation of the observations to the quality control procedures applied, we provide a rescued dataset that is as traceable as possible for use by the research community.Data from 127 stations and of 15 climate variables in the northern African and European sectors have been prepared for the period 1877 to 2012. Quality control of the data using a two-step semi-automatic statistical approach identified 3.5 % of observations that required correction or removal, on par with previous data rescue efforts.In addition to providing a new sub-daily meteorological dataset for the research community, our experience in the development of this sub-daily dataset gives us an opportunity to share some suggestions for future data rescue projects.All versions of the dataset, from the raw digitised data to data that have been quality controlled and converted to standard units, are available on PANGAEA: https://doi.org/10.1594/PANGAEA.886511 (Ashcroft et al., 2018).

Download Full-text