Correction to: Kanzelhöhe Observatory: Instruments, Data Processing and Data Products

The Philippines’s PHL-Microsat program aims to launch its first earth observation satellite, DIWATA, on the first quarter of 2016. DIWATA’s payload consists of a high-precision telescope (HPT), spaceborne multispectral imager (SMI) with liquid crystal tunable filter (LCTF), and a wide field camera (WFC). Once launched, it will provide information about the Philippines, both for disaster and environmental applications. Depending on the need, different remote sensing products will be generated from the microsatellite sensors. This necessitates data processing capability on the ground control segment. Rather than rely on commercial turnkey solutions, the PHL-Microsat team, specifically Project 3:DPAD, opted to design its own ground receiving station data subsystems. This paper describes the design of the data subsystems of the ground receiving station (GRS) for DIWATA. The data subsystems include: data processing subsystem for automatic calibration and georeferencing of raw images as well as the generation of higher level processed data products; data archiving subsystem for storage and backups of both raw and processed data products; and data distribution subsystem for providing a web-based interface and product download facility for the user community. The design covers the conceptual design of the abovementioned subsystems, the free and open source software (FOSS) packages used to implement them, and the challenges encountered in adapting the existing FOSS packages to DIWATA GRS requirements.

Download Full-text

From NEON Field Sites to Data Portal: A Community Resource for Surface–Atmosphere Research Comes Online

Bulletin of the American Meteorological Society ◽

10.1175/bams-d-17-0307.1 ◽

2019 ◽

Vol 100 (11) ◽

pp. 2305-2325 ◽

Cited By ~ 2

Author(s):

Stefan Metzger ◽

Edward Ayres ◽

David Durden ◽

Christopher Florian ◽

Robert Lee ◽

...

Keyword(s):

Data Processing ◽

Atmospheric Chemistry ◽

Meteorological Data ◽

The United States ◽

Observation System ◽

Ongoing Research ◽

Continental Scale ◽

Surface Atmosphere ◽

Data Products ◽

Data Portal

AbstractThe National Ecological Observatory Network (NEON) is a multidecadal and continental-scale observatory with sites across the United States. Having entered its operational phase in 2018, NEON data products, software, and services become available to facilitate research on the impacts of climate change, land-use change, and invasive species. An essential component of NEON are its 47 tower sites, where eddy-covariance (EC) sensors are operated to determine the surface–atmosphere exchange of momentum, heat, water, and CO2. EC tower networks such as AmeriFlux, the Integrated Carbon Observation System (ICOS), and NEON are vital for providing the distributed observations to address interactions at the soil–vegetation–atmosphere interface. NEON represents the largest single-provider EC network globally, with standardized observations and data processing explicitly designed for intersite comparability and analysis of feedbacks across multiple spatial and temporal scales. Furthermore, EC is tightly integrated with soil, meteorology, atmospheric chemistry, isotope, phenology, and rich contextual observations such as airborne remote sensing and in situ sampling bouts. Here, we present an overview of NEON’s observational design, field operation, and data processing that yield community resources for the study of surface–atmosphere interactions. Near-real-time data products become available from the NEON Data Portal, and EC and meteorological data are ingested into AmeriFlux and FLUXNET globally harmonized data releases. Open-source software for reproducible, extensible, and portable data analysis includes the eddy4R family of R packages underlying the EC data product generation. These resources strive to integrate with existing infrastructures and networks, to suggest novel systemic solutions, and to synergize ongoing research efforts across science communities.

Download Full-text

Experiment to quantify the solar radiative temperature error of radiosondes

10.5194/ems2021-267 ◽

2021 ◽

Author(s):

Christoph von Rohden ◽

Michael Sommer ◽

Tatjana Naebert ◽

Ruud Dirksen

Keyword(s):

Solar Radiation ◽

Data Processing ◽

Temperature Sensor ◽

Ambient Pressure ◽

Longitudinal Axis ◽

Atmospheric Temperature ◽

Temperature Error ◽

Data Products ◽

Set Up

One of the main goals of the GCOS Reference Upper Air Network (GRUAN) is to perform reference observations of profiles of atmospheric temperature and humidity for the purpose of monitoring climate change. Two essential criteria for establishing a reference observation are measurement-traceability and the availability of measurement uncertainties. Radiosoundings have proven valuable in providing in-situ profiles of temperature, humidity and pressure at unmatched vertical resolution. Data products from commercial radiosondes often rely on black-box or proprietary algorithms, which are not disclosed to the scientific user. Furthermore, long-term time-series from these products are frequently hampered by changes in the hardware and/or the data processing. GRUAN data products (GDPs) comply with the above-mentioned criteria for a reference product. Correction algorithms are open-source and well-documented and the data include vertically resolved best-estimates of the uncertainties.This presentation discusses the quantification and the correction for the temperature error due to solar radiation that is applied in the GRUAN data processing for the Vaisala RS41 radiosonde. Heating of the temperature sensor by solar radiation is the dominant source of error for daytime radiosoundings.At Lindenberg Observatory a dedicated laboratory set-up was built to quantify the solar temperature error of radiosondes. The setup allows to create conditions that are similar to those encountered during an actual radiosounding, with special emphasis on parameters such as pressure, air flow (ventilation), and illumination conditions. The radiosonde is placed inside a quartz tube that is integrated in a wind tunnel-like construction that can be operated between ambient pressure and 3 hPa. During the measurements the radiosonde is rotated along its longitudinal axis to mimic the spinning during ascent, and the large quartz window makes it possible to illuminate the temperature sensor together with a considerable part of the sensor boom, allowing to assess the contribution of the heat transfer from the sensor boom to the sensor. A parameterization of the heating of the sensor in terms of flux, pressure, ventilation and solar elevation is presented. This parameterization is the basis of the GRUAN correction algorithm, which in addition includes a radiation model and altitude information. In conclusion the GRUAN data product is compared to the manufacturer-processed data.

Download Full-text

SDS Level-2/-3 JPL

10.5194/gstm2020-75 ◽

2020 ◽

Author(s):

David Wiese

Keyword(s):

Data Processing ◽

Data Quality ◽

Solution Strategy ◽

Data Types ◽

Preliminary Results ◽

Gravity Fields ◽

Data Products ◽

Level 2

This talk will provide a status update on Level-2 data processing at JPL.&#160; Included will be an overview of data products currently delivered to the community. &#160;Assessments of data quality and error levels, along with detailed discussions of the solution strategy will be included.&#160; A comparison of LRI and KBR gravity fields will be provided, in addition to a discussion on efforts to combine both data types in the gravity retrieval.&#160; A look towards an RL07 reprocessing of GRACE and GRACE-FO will additionally be provided, along with preliminary results.

Download Full-text

Data processing and data products from 2017 to 2019 campaign of astronomical site testing at Ali, Daocheng and Muztagh-ata

Research in Astronomy and Astrophysics ◽

10.1088/1674-4527/20/6/82 ◽

2020 ◽

Vol 20 (6) ◽

pp. 082 ◽

Cited By ~ 6

Author(s):

Zi-Huang Cao ◽

Jin-Xin Hao ◽

Lu Feng ◽

Hugh R. A. Jones ◽

Jian Li ◽

...

Keyword(s):

Data Processing ◽

Site Testing ◽

Data Products

Download Full-text

Archiving and Managing Remote Sensing Data using State of the Art Storage Technologies

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsarchives-xl-8-1153-2014 ◽

2014 ◽

Vol XL-8 ◽

pp. 1153-1158

Author(s):

B. Lakshmi ◽

C. Chandrasekhara Reddy ◽

S. V. S. R. K. Kishore

Keyword(s):

Remote Sensing ◽

Data Processing ◽

Satellite Data ◽

High Speed ◽

Remote Sensing Data ◽

Multiple Systems ◽

Sensing Data ◽

Ground Segment ◽

Data Products

Integrated Multi-mission Ground Segment for Earth Observation Satellites (IMGEOS) was established with an objective to eliminate human interaction to the maximum extent. All emergency data products will be delivered within an hour of acquisition through FTP delivery. All other standard data products will be delivered through FTP within a day. The IMGEOS activity was envisaged to reengineer the entire chain of operations at the ground segment facilities of NRSC at Shadnagar and Balanagar campuses to adopt an integrated multi-mission approach. To achieve this, the Information Technology Infrastructure was consolidated by implementing virtualized tiered storage and network computing infrastructure in a newly built Data Centre at Shadnagar Campus. One important activity that influences all other activities in the integrated multi-mission approach is the design of appropriate storage and network architecture for realizing all the envisaged operations in a highly streamlined, reliable and secure environment. Storage was consolidated based on the major factors like accessibility, long term data protection, availability, manageability and scalability. The broad operational activities are reception of satellite data, quick look, generation of browse, production of standard and valueadded data products, production chain management, data quality evaluation, quality control and product dissemination. For each of these activities, there are numerous other detailed sub-activities and pre-requisite tasks that need to be implemented to support the above operations. The IMGEOS architecture has taken care of choosing the right technology for the given data sizes, their movement and long-term lossless retention policies. Operational costs of the solution are kept to the minimum possible. Scalability of the solution is also ensured. The main function of the storage is to receive and store the acquired satellite data, facilitate high speed availability of the data for further processing at Data Processing servers and help to generate data products at a rate of about 1000 products per day. It also archives all the acquired data on tape storage for long-term retention and utilization. Data sizes per satellite pass range from hundreds of megabytes to tens of gigabytes The images acquired from remote sensing satellites are valuable assets of NRSC and are used as input for further generation of different types of user data products through multiple Data Processing systems. Hence, it is required to collect and store the data within a shared, high speed repository concurrently accessible by multiple systems. After the raw imagery is stored on a high-speed repository, the images must be processed in order for them to be useful for value-added processing or for imagery analysts. The raw image file has to be copied on to data processing servers for further processing. Given the large file sizes, it is impractical to transfer these files to processing servers via a local area network. Even at gigabit Ethernet rates (up to 60 MB/s), a 5 GB file will take at least 83 seconds. For this reason, it is useful to employ a shared file system which allows every processing system to directly access the same pool where raw images were stored. Concurrent access by multiple systems is ensured for processing and generation of data products. With the above reasons, it was chosen to have high speed disk arrays for acquisition and processing purposes and tape based storage systems for long-term huge data (Peta Bytes) archival in a virtualized multitier storage architecture. This paper explains the architecture involved in a virtualized tiered storage environment being used for acquisition, processing and archiving the remote sensing data. It also explains the data management aspects involved in ensuring data availability and archiving Peta bytes sized, remote sensing data acquired over the past 40 years.

Download Full-text

Intercomparison of the Vaisala RS92 and RS41 Radiosonde GRUAN Data Products (GDP) in the Troposphere and Lower Stratosphere

10.5194/egusphere-egu21-12735 ◽

2021 ◽

Author(s):

Tzvetan Simeonov ◽

Ruud Dirksen ◽

Christoph von Rohden ◽

Michael Sommer

Keyword(s):

Data Processing ◽

Large Scale ◽

Lower Stratosphere ◽

Time Lag ◽

Measurement Data ◽

Uncertainty Estimates ◽

Data Products ◽

Humidity Measurements ◽

Special Challenge ◽

Distributed Measurement

The GCOS Reference Upper Air Network (GRUAN) consists of 30 globally distributed measurement sites that provide reference observations of essential climate variables such as temperature and water vapour for climate monitoring. At these sites, radiosondes provide in-situ profiles of temperature, humidity and pressure at high vertical resolution. However, data products from commercial radiosondes often rely on black-box or proprietary algorithms, which are not disclosed to the scientific user. Furthermore, long-term time-series from these products are frequently hampered by changes in the hardware and/or the data processing. Therefore, GRUAN data products (GDP) are developed, that employ open-source and well-documented corrections to the measured data, thereby complying with the requirements for reference data, which include measurement traceability and the availability of measurement uncertainties. The GRUAN data processing is applied to the raw measurement data of temperature, humidity, pressure, altitude, and wind, and includes corrections of errors from known sources, such as for example solar radiation error for temperature and sensor time lag for humidity measurements. The vertically resolved uncertainty estimates include the uncertainty of the applied corrections and the calibration uncertainty of the sensors.A substantial number of GRUAN sites employ the Vaisala RS41 radiosonde, and its predecessor, the RS92, before that. This large-scale change of instrumentation poses a special challenge to the network, and great care is taken to characterize the differences between these instruments in order to prevent inhomogeneities in the data records. As part of this effort, the GRUAN data products for both radiosonde types are compared. In this study we used data from approximately 1000 RS92+RS41 twin-soundings (two sondes on a rig attached to one balloon) that were performed at 11 GRUAN sites, covering the main climate zones.The first analysis shows that daytime temperature differences in the stratosphere increase steadily with altitude, with RS92-GDP up to 0.5&#160;K warmer than RS41-GDP above 25&#160;km. In addition, at daytime the RS41-GDP is 0.2&#160;K warmer than the manufacturer-processed RS41-EDT product above 15&#160;km. Analysis of the humidity profiles shows a slight moist bias of the RS41 compared to the RS92 for both GDP and manufacturer-processed data. Differences between the RS41-EDT and GDP humidity products are most pronounced in the upper troposphere - lower stratosphere region and are attributed to the time lagcorrection. The analysis of the temperature differences will be refined by investigating the influence of the solar radiation in conjunction with sonde orientation and ventilation. Furthermore, the uncertainty of the humidity data will be assessed by comparing with coincident measurements of the water vapor profile by the Cryogenic Frostpoint Hygrometer (CFH).Key words: Radiosonde, RS41, RS92, humidity, temperature, uncertainty, GRUAN, troposphere, lower stratosphere

Download Full-text

Intercomparison of the Vaisala RS92 and RS41 Radiosonde GRUAN Data Products (GDP) in the Troposphere and Lower Stratosphere

10.5194/ems2021-190 ◽

2021 ◽

Author(s):

Tzvetan Simeonov ◽

Ruud Dirksen ◽

Christoph von Rohden ◽

Michael Sommer

Keyword(s):

Data Processing ◽

Large Scale ◽

Lower Stratosphere ◽

Time Lag ◽

Measurement Data ◽

Uncertainty Estimates ◽

Data Products ◽

Humidity Measurements ◽

Special Challenge ◽

Distributed Measurement

The GCOS Reference Upper Air Network (GRUAN) consists of 30 globally distributed measurement sites that provide reference observations of essential climate variables such as temperature and water vapour for climate monitoring. At these sites, radiosondes provide in-situ profiles of temperature, humidity and pressure at high vertical resolution. However, data products from commercial radiosondes often rely on black-box or proprietary algorithms, which are not disclosed to the scientific user. Furthermore, long-term time-series from these products are frequently hampered by changes in the hardware and/or the data processing. Therefore, GRUAN data products (GDP) are developed, that employ open-source and well-documented corrections to the measured data, thereby complying with the requirements for reference data, which include measurement traceability and the availability of measurement uncertainties. The GRUAN data processing is applied to the raw measurement data of temperature, humidity, pressure, altitude, and wind, and includes corrections of errors from known sources, such as for example solar radiation error for temperature and sensor time lag for humidity measurements. The vertically resolved uncertainty estimates include the uncertainty of the applied corrections and the calibration uncertainty of the sensors.A substantial number of GRUAN sites employ the Vaisala RS41 radiosonde, and its predecessor, the RS92, before that. This large-scale change of instrumentation poses a special challenge to the network, and great care is taken to characterize the differences between these instruments in order to prevent inhomogeneities in the data records. As part of this effort, the GRUAN data products for both radiosonde types are compared. In this study we used data from approximately 1000 RS92+RS41 twin-soundings (two sondes on a rig attached to one balloon)&#160;that were performed at 11 GRUAN sites, covering the main climate zones.The first analysis shows that daytime temperature differences in the stratosphere increase steadily with altitude, with RS92-GDP up to 0.5&#160;K warmer than RS41-GDP above 25&#160;km. In addition, at daytime the RS41-GDP is 0.2&#160;K warmer than the manufacturer-processed RS41-EDT product above 15&#160;km. Analysis of the humidity profiles shows a slight moist bias of the RS41 compared to the RS92 for both GDP and manufacturer-processed data. Differences between the RS41-EDT and GDP humidity products are most pronounced in the upper troposphere - lower stratosphere region and are attributed to the time lagcorrection. The analysis of the temperature differences will be refined by investigating the influence of the solar&#160;radiation in conjunction with sonde orientation and ventilation. Furthermore, the uncertainty of the humidity data will be assessed by comparing with coincident measurements of the water vapor profile by the Cryogenic Frostpoint Hygrometer (CFH).Key words:&#160;Radiosonde, RS41, RS92, humidity, temperature, uncertainty, GRUAN, troposphere, lower stratosphere

Download Full-text

DESIGN OF A FREE AND OPEN SOURCE DATA PROCESSING, ARCHIVING, AND DISTRIBUTION SUBSYSTEM FOR THE GROUND RECEIVING STATION OF THE PHILIPPINE SCIENTIFIC EARTH OBSERVATION MICRO-SATELLITE

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xli-b7-909-2016 ◽

2016 ◽

Vol XLI-B7 ◽

pp. 909-911

Author(s):

R. K. D. Aranas ◽

B. J. D. Jiao ◽

B. J. P. Magallon ◽

M. K. F. Ramos ◽

J. A. Amado ◽

...

Keyword(s):

Data Processing ◽

Open Source ◽

The Philippines ◽

Tunable Filter ◽

Earth Observation ◽

Wide Field ◽

Data Archiving ◽

Data Products ◽

Control Segment ◽

Processing Subsystem

The Philippines’s PHL-Microsat program aims to launch its first earth observation satellite, DIWATA, on the first quarter of 2016. DIWATA’s payload consists of a high-precision telescope (HPT), spaceborne multispectral imager (SMI) with liquid crystal tunable filter (LCTF), and a wide field camera (WFC). Once launched, it will provide information about the Philippines, both for disaster and environmental applications. Depending on the need, different remote sensing products will be generated from the microsatellite sensors. This necessitates data processing capability on the ground control segment. Rather than rely on commercial turnkey solutions, the PHL-Microsat team, specifically Project 3:DPAD, opted to design its own ground receiving station data subsystems. This paper describes the design of the data subsystems of the ground receiving station (GRS) for DIWATA. The data subsystems include: data processing subsystem for automatic calibration and georeferencing of raw images as well as the generation of higher level processed data products; data archiving subsystem for storage and backups of both raw and processed data products; and data distribution subsystem for providing a web-based interface and product download facility for the user community. The design covers the conceptual design of the abovementioned subsystems, the free and open source software (FOSS) packages used to implement them, and the challenges encountered in adapting the existing FOSS packages to DIWATA GRS requirements.

Download Full-text