scholarly journals Dynamic spatio-temporal generation of large-scale synthetic gridded precipitation: with improved spatial coherence of extremes

2019 ◽  
Vol 34 (9) ◽  
pp. 1369-1383 ◽  
Author(s):  
Dirk Diederen ◽  
Ye Liu
Keyword(s):  
Large Scale ◽  
Spatial Coherence ◽  
Original Data ◽  
Return Level ◽  
Data Sets ◽  
Large Set ◽  
Precipitation Data ◽  
Data Set ◽  
Spatio Temporal ◽  
Synthetic Precipitation

Abstract With the ongoing development of distributed hydrological models, flood risk analysis calls for synthetic, gridded precipitation data sets. The availability of large, coherent, gridded re-analysis data sets in combination with the increase in computational power, accommodates the development of new methodology to generate such synthetic data. We tracked moving precipitation fields and classified them using self-organising maps. For each class, we fitted a multivariate mixture model and generated a large set of synthetic, coherent descriptors, which we used to reconstruct moving synthetic precipitation fields. We introduced randomness in the original data set by replacing the observed precipitation fields in the original data set with the synthetic precipitation fields. The output is a continuous, gridded, hourly precipitation data set of a much longer duration, containing physically plausible and spatio-temporally coherent precipitation events. The proposed methodology implicitly provides an important improvement in the spatial coherence of precipitation extremes. We investigate the issue of unrealistic, sudden changes on the grid and demonstrate how a dynamic spatio-temporal generator can provide spatial smoothness in the probability distribution parameters and hence in the return level estimates.

scholarly journals LARGE‐SCALE DATA VISUALIZATION WITH MISSING VALUES

2006 ◽  
Vol 12 (1) ◽  
pp. 44-49
Author(s):  
Sergiy Popov
Keyword(s):  
Dimensionality Reduction ◽  
Large Scale ◽  
Missing Values ◽  
Original Data ◽  
Nonlinear Dimensionality Reduction ◽  
Data Sets ◽  
Data Set ◽  
Large Scale Data ◽  
Learning Procedure ◽  
Scale Data

Visualization of large‐scale data inherently requires dimensionality reduction to 1D, 2D, or 3D space. Autoassociative neural networks with a bottleneck layer are commonly used as a nonlinear dimensionality reduction technique. However, many real‐world problems suffer from incomplete data sets, i.e. some values can be missing. Common methods dealing with missing data include the deletion of all cases with missing values from the data set or replacement with mean or “normal” values for specific variables. Such methods are appropriate when just a few values are missing. But in the case when a substantial portion of data is missing, these methods can significantly bias the results of modeling. To overcome this difficulty, we propose a modified learning procedure for the autoassociative neural network that directly takes the missing values into account. The outputs of the trained network may be used for substitution of the missing values in the original data set.

scholarly journals Spatio-temporal patterns of Pyrenean exhumation revealed by inverse thermo-kinematic modeling of a large thermochronologic data set

Geology ◽  
2021 ◽  
Author(s):  
Magdalena Ellis Curry ◽  
Peter van der Beek ◽  
Ritske S. Huismans ◽  
Sebastian G. Wolf ◽  
Charlotte Fillon ◽  
...  
Keyword(s):  
Large Scale ◽  
Temporal Patterns ◽  
Temporal Variations ◽  
Data Sets ◽  
Kinematic Modeling ◽  
Data Set ◽  
Eastern Pyrenees ◽  
Uplift Rates ◽  
Spatio Temporal ◽  
Modeling Code

Large thermochronologic data sets enable orogen-scale investigations into spatio-temporal patterns of erosion and deformation. We present the results of a thermo-kinematic modeling study that examines large-scale controls on spatio-temporal variations in exhumation as recorded by multiple low-temperature thermochronometers in the Pyrenees mountains (France/Spain). Using 264 compiled cooling ages spanning ~200 km of the orogen, a recent model for its topographic evolution, and the thermo-kinematic modeling code Pecube, we evaluated two models for Axial Zone (AZ) exhumation: (1) thrust sheet–controlled (north-south) exhumation, and (2) along-strike (east-west) variable exhumation. We also measured the degree to which spatially variable post-orogenic erosion influenced the cooling ages. We found the best fit for a model of along-strike variable exhumation. In the eastern AZ, rock uplift rates peak at ≥1 mm/yr between 40 and 30 Ma, whereas in the western AZ, they peak between 30 and 20 Ma. The amount of post-orogenic (<20 Ma) erosion increases from <1.0 km in the eastern Pyrenees to >2.5 km in the west. The data reveal a pattern of exhumation that is primarily controlled by structural inheritance, with ancillary patterns reflecting growth and erosion of the antiformal stack and post-orogenic surface processes.

scholarly journals Galaxy spin direction distribution in HST and SDSS show similar large-scale asymmetry

2020 ◽  
Vol 37 ◽  
Author(s):  
Lior Shamir
Keyword(s):  
Large Scale ◽  
Spiral Galaxies ◽  
Hubble Space Telescope ◽  
Gravitational Interaction ◽  
Large Data ◽  
Sloan Digital Sky Survey ◽  
Data Sets ◽  
Dipole Axis ◽  
Data Set ◽  
The Asymmetry

Abstract Several recent observations using large data sets of galaxies showed non-random distribution of the spin directions of spiral galaxies, even when the galaxies are too far from each other to have gravitational interaction. Here, a data set of $\sim8.7\cdot10^3$ spiral galaxies imaged by Hubble Space Telescope (HST) is used to test and profile a possible asymmetry between galaxy spin directions. The asymmetry between galaxies with opposite spin directions is compared to the asymmetry of galaxies from the Sloan Digital Sky Survey. The two data sets contain different galaxies at different redshift ranges, and each data set was annotated using a different annotation method. The results show that both data sets show a similar asymmetry in the COSMOS field, which is covered by both telescopes. Fitting the asymmetry of the galaxies to cosine dependence shows a dipole axis with probabilities of $\sim2.8\sigma$ and $\sim7.38\sigma$ in HST and SDSS, respectively. The most likely dipole axis identified in the HST galaxies is at $(\alpha=78^{\rm o},\delta=47^{\rm o})$ and is well within the $1\sigma$ error range compared to the location of the most likely dipole axis in the SDSS galaxies with $z>0.15$ , identified at $(\alpha=71^{\rm o},\delta=61^{\rm o})$ .

scholarly journals The Midlatitude Continental Convective Clouds Experiment (MC3E) sounding network: operations, processing and analysis

2015 ◽  
Vol 8 (1) ◽  
pp. 421-434 ◽  
Author(s):  
M. P. Jensen ◽  
T. Toto ◽  
D. Troyan ◽  
P. E. Ciesielski ◽  
D. Holdridge ◽  
...  
Keyword(s):  
Large Scale ◽  
Scale Model ◽  
Data Sets ◽  
Central Plains ◽  
Data Set ◽  
Convective Systems ◽  
Convective Clouds ◽  
Quality Checks ◽  
Network Operations ◽  
The Impact

Abstract. The Midlatitude Continental Convective Clouds Experiment (MC3E) took place during the spring of 2011 centered in north-central Oklahoma, USA. The main goal of this field campaign was to capture the dynamical and microphysical characteristics of precipitating convective systems in the US Central Plains. A major component of the campaign was a six-site radiosonde array designed to capture the large-scale variability of the atmospheric state with the intent of deriving model forcing data sets. Over the course of the 46-day MC3E campaign, a total of 1362 radiosondes were launched from the enhanced sonde network. This manuscript provides details on the instrumentation used as part of the sounding array, the data processing activities including quality checks and humidity bias corrections and an analysis of the impacts of bias correction and algorithm assumptions on the determination of convective levels and indices. It is found that corrections for known radiosonde humidity biases and assumptions regarding the characteristics of the surface convective parcel result in significant differences in the derived values of convective levels and indices in many soundings. In addition, the impact of including the humidity corrections and quality controls on the thermodynamic profiles that are used in the derivation of a large-scale model forcing data set are investigated. The results show a significant impact on the derived large-scale vertical velocity field illustrating the importance of addressing these humidity biases.

scholarly journals Historical drought patterns over Canada and their teleconnections with large-scale climate signals

2018 ◽  
Vol 22 (6) ◽  
pp. 3105-3124 ◽  
Author(s):  
Zilefac Elvis Asong ◽  
Howard Simon Wheater ◽  
Barrie Bonsal ◽  
Saman Razavi ◽  
Sopan Kurkute
Keyword(s):  
Large Scale ◽  
Southern Oscillation ◽  
Low Frequency ◽  
Kendall Test ◽  
Wavelet Coherence ◽  
Climate Indices ◽  
Data Sets ◽  
Canadian Prairies ◽  
Agriculture Industry ◽  
Spatio Temporal

Abstract. Drought is a recurring extreme climate event and among the most costly natural disasters in the world. This is particularly true over Canada, where drought is both a frequent and damaging phenomenon with impacts on regional water resources, agriculture, industry, aquatic ecosystems, and health. However, nationwide drought assessments are currently lacking and impacted by limited ground-based observations. This study provides a comprehensive analysis of historical droughts over the whole of Canada, including the role of large-scale teleconnections. Drought events are characterized by the Standardized Precipitation Evapotranspiration Index (SPEI) over various temporal scales (1, 3, 6, and 12 consecutive months, 6 months from April to September, and 12 months from October to September) applied to different gridded monthly data sets for the period 1950–2013. The Mann–Kendall test, rotated empirical orthogonal function, continuous wavelet transform, and wavelet coherence analyses are used, respectively, to investigate the trend, spatio-temporal patterns, periodicity, and teleconnectivity of drought events. Results indicate that southern (northern) parts of the country experienced significant trends towards drier (wetter) conditions although substantial variability exists. Two spatially well-defined regions with different temporal evolution of droughts were identified – the Canadian Prairies and northern central Canada. The analyses also revealed the presence of a dominant periodicity of between 8 and 32 months in the Prairie region and between 8 and 40 months in the northern central region. These cycles of low-frequency variability are found to be associated principally with the Pacific–North American (PNA) and Multivariate El Niño/Southern Oscillation Index (MEI) relative to other considered large-scale climate indices. This study is the first of its kind to identify dominant periodicities in drought variability over the whole of Canada in terms of when the drought events occur, their duration, and how often they occur.

Bayesian Classifier for Sparsity-Promoting Feature Selection

2015 ◽  
Vol 29 (06) ◽  
pp. 1550022 ◽  
Author(s):  
Danlei Xu ◽  
Lan Du ◽  
Hongwei Liu ◽  
Penghui Wang
Keyword(s):  
Feature Selection ◽  
Synthetic Data ◽  
Original Data ◽  
Radar Data ◽  
Bayesian Classifier ◽  
Classification Model ◽  
Data Sets ◽  
Data Set ◽  
Classification Boundary ◽  
Nonlinear Mappings

A Bayesian classifier for sparsity-promoting feature selection is developed in this paper, where a set of nonlinear mappings for the original data is performed as a pre-processing step. The linear classification model with such mappings from the original input space to a nonlinear transformation space can not only construct the nonlinear classification boundary, but also realize the feature selection for the original data. A zero-mean Gaussian prior with Gamma precision and a finite approximation of Beta process prior are used to promote sparsity in the utilization of features and nonlinear mappings in our model, respectively. We derive the Variational Bayesian (VB) inference algorithm for the proposed linear classifier. Experimental results based on the synthetic data set, measured radar data set, high-dimensional gene expression data set, and several benchmark data sets demonstrate the aggressive and robust feature selection capability and comparable classification accuracy of our method comparing with some other existing classifiers.

scholarly journals A fast methodology for large-scale focusing inversion of gravity and magnetic data using the structured model matrix and the 2-D fast Fourier transform

2020 ◽  
Vol 223 (2) ◽  
pp. 1378-1397
Author(s):  
Rosemary A Renaut ◽  
Jarom D Hogue ◽  
Saeed Vatankhah ◽  
Shuang Liu
Keyword(s):  
Fourier Transform ◽  
Fast Fourier Transform ◽  
Linear Systems ◽  
Large Scale ◽  
Surface Measurement ◽  
Magnetic Data ◽  
Uniform Grid ◽  
Data Sets ◽  
Inversion Algorithm ◽  
Data Set

SUMMARY We discuss the focusing inversion of potential field data for the recovery of sparse subsurface structures from surface measurement data on a uniform grid. For the uniform grid, the model sensitivity matrices have a block Toeplitz Toeplitz block structure for each block of columns related to a fixed depth layer of the subsurface. Then, all forward operations with the sensitivity matrix, or its transpose, are performed using the 2-D fast Fourier transform. Simulations are provided to show that the implementation of the focusing inversion algorithm using the fast Fourier transform is efficient, and that the algorithm can be realized on standard desktop computers with sufficient memory for storage of volumes up to size n ≈ 106. The linear systems of equations arising in the focusing inversion algorithm are solved using either Golub–Kahan bidiagonalization or randomized singular value decomposition algorithms. These two algorithms are contrasted for their efficiency when used to solve large-scale problems with respect to the sizes of the projected subspaces adopted for the solutions of the linear systems. The results confirm earlier studies that the randomized algorithms are to be preferred for the inversion of gravity data, and for data sets of size m it is sufficient to use projected spaces of size approximately m/8. For the inversion of magnetic data sets, we show that it is more efficient to use the Golub–Kahan bidiagonalization, and that it is again sufficient to use projected spaces of size approximately m/8. Simulations support the presented conclusions and are verified for the inversion of a magnetic data set obtained over the Wuskwatim Lake region in Manitoba, Canada.

scholarly journals INTERESTING SPATIO-TEMPORAL REGION DISCOVERY COMPUTATIONS OVER GPU AND MAPREDUCE PLATFORMS

2015 ◽  
Vol II-4/W2 ◽  
pp. 35-41 ◽  
Author(s):  
M. McDermott ◽  
S. K. Prasad ◽  
S. Shekhar ◽  
X. Zhou
Keyword(s):  
Parallel Architecture ◽  
Computation Time ◽  
Data Sets ◽  
Temporal Region ◽  
Data Set ◽  
Atmospheric Sciences ◽  
3 Dimensional ◽  
Linear Speedup ◽  
Spatio Temporal ◽  
Mapreduce Model

Discovery of interesting paths and regions in spatio-temporal data sets is important to many fields such as the earth and atmospheric sciences, GIS, public safety and public health both as a goal and as a preliminary step in a larger series of computations. This discovery is usually an exhaustive procedure that quickly becomes extremely time consuming to perform using traditional paradigms and hardware and given the rapidly growing sizes of today’s data sets is quickly outpacing the speed at which computational capacity is growing. In our previous work (Prasad et al., 2013a) we achieved a 50 times speedup over sequential using a single GPU. We were able to achieve near linear speedup over this result on interesting path discovery by using Apache Hadoop to distribute the workload across multiple GPU nodes. Leveraging the parallel architecture of GPUs we were able to drastically reduce the computation time of a 3-dimensional spatio-temporal interest region search on a single tile of normalized difference vegetative index for Saudi Arabia. We were further able to see an almost linear speedup in compute performance by distributing this workload across several GPUs with a simple MapReduce model. This increases the speed of processing 10 fold over the comparable sequential while simultaneously increasing the amount of data being processed by 384 fold. This allowed us to process the entirety of the selected data set instead of a constrained window.

scholarly journals Six years of total ozone column measurements from SCIAMACHY nadir observations

2009 ◽  
Vol 2 (1) ◽  
pp. 87-98 ◽  
Author(s):  
C. Lerot ◽  
M. Van Roozendael ◽  
J. van Geffen ◽  
J. van Gent ◽  
C. Fayt ◽  
...  
Keyword(s):  
Cross Sections ◽  
Total Ozone ◽  
Large Scale ◽  
European Space Agency ◽  
Data Sets ◽  
Data Set ◽  
Ozone Data ◽  
Space Agency ◽  
German Aerospace ◽  
The Impact

Abstract. Total O3 columns have been retrieved from six years of SCIAMACHY nadir UV radiance measurements using SDOAS, an adaptation of the GDOAS algorithm previously developed at BIRA-IASB for the GOME instrument. GDOAS and SDOAS have been implemented by the German Aerospace Center (DLR) in the version 4 of the GOME Data Processor (GDP) and in version 3 of the SCIAMACHY Ground Processor (SGP), respectively. The processors are being run at the DLR processing centre on behalf of the European Space Agency (ESA). We first focus on the description of the SDOAS algorithm with particular attention to the impact of uncertainties on the reference O3 absorption cross-sections. Second, the resulting SCIAMACHY total ozone data set is globally evaluated through large-scale comparisons with results from GOME and OMI as well as with ground-based correlative measurements. The various total ozone data sets are found to agree within 2% on average. However, a negative trend of 0.2–0.4%/year has been identified in the SCIAMACHY O3 columns; this probably originates from instrumental degradation effects that have not yet been fully characterized.

Sign in / Sign up

Export Citation Format

Share Document