scholarly journals A geostatistical approach to multisensor rain field reconstruction and downscaling

2001 ◽  
Vol 5 (2) ◽  
pp. 201-213 ◽  
Author(s):  
P. Fiorucci ◽  
P. La Barbera ◽  
L.G. Lanza ◽  
R. Minciardi
Keyword(s):  
Large Scale ◽  
A Priori ◽  
Grid Cell ◽  
Ground Truth ◽  
Rain Gauge ◽  
Rain Event ◽  
Data Set ◽  
Field Reconstruction ◽  
Geostatistical Approach ◽  
A Posteriori Estimates

Abstract. A rain field reconstruction and downscaling methodology is presented, which allows suitable integration of large scale rainfall information and rain-gauge measurements at the ground. The former data set is assumed to provide probabilistic indicators that are used to infer the parameters of the probability density function of the stochastic rain process at each pixel site. Rain-gauge measurements are assumed as the ground truth and used to constrain the reconstructed rain field to the associated point values. Downscaling is performed by assuming the a posteriori estimates of the rain figures at each grid cell as the a priori large-scale conditioning values for reconstruction of the rain field at finer scale. The case study of an intense rain event recently observed in northern Italy is presented and results are discussed with reference to the modelling capabilities of the proposed methodology. Keywords: Reconstruction, downscaling, remote sensing, geostatistics, Meteosat

scholarly journals Comparative Analysis of Supervised and Unsupervised Approaches Applied to Large-Scale “In The Wild” Face Verification

Symmetry ◽  
2020 ◽  
Vol 12 (11) ◽  
pp. 1832
Author(s):  
Tomasz Hachaj ◽  
Patryk Mazurek
Keyword(s):  
Pattern Recognition ◽  
Large Scale ◽  
Statistical Significance ◽  
Ground Truth ◽  
Classification Algorithm ◽  
Adjusted Rand Index ◽  
Face Verification ◽  
Data Set ◽  
In The Wild ◽  
Unsupervised Approaches

Deep learning-based feature extraction methods and transfer learning have become common approaches in the field of pattern recognition. Deep convolutional neural networks trained using tripled-based loss functions allow for the generation of face embeddings, which can be directly applied to face verification and clustering. Knowledge about the ground truth of face identities might improve the effectiveness of the final classification algorithm; however, it is also possible to use ground truth clusters previously discovered using an unsupervised approach. The aim of this paper is to evaluate the potential improvement of classification results of state-of-the-art supervised classification methods trained with and without ground truth knowledge. In this study, we use two sufficiently large data sets containing more than 200,000 “taken in the wild” images, each with various resolutions, visual quality, and face poses which, in our opinion, guarantee the statistical significance of the results. We examine several clustering and supervised pattern recognition algorithms and find that knowledge about the ground truth has a very small influence on the Fowlkes–Mallows score (FMS) of the classification algorithm. In the case of the classification algorithm that obtained the highest accuracy in our experiment, the FMS improved by only 5.3% (from 0.749 to 0.791) in the first data set and by 6.6% (from 0.652 to 0.718) in the second data set. Our results show that, beside highly secure systems in which face verification is a key component, face identities discovered by unsupervised approaches can be safely used for training supervised classifiers. We also found that the Silhouette Coefficient (SC) of unsupervised clustering is positively correlated with the Adjusted Rand Index, V-measure score, and Fowlkes–Mallows score and, so, we can use the SC as an indicator of clustering performance when the ground truth of face identities is not known. All of these conclusions are important findings for large-scale face verification problems. The reason for this is the fact that skipping the verification of people’s identities before supervised training saves a lot of time and resources.

SaltSeg: Automatic 3D salt segmentation using a deep convolutional neural network

2019 ◽  
Vol 7 (3) ◽  
pp. SE113-SE122 ◽  
Author(s):  
Yunzhi Shi ◽  
Xinming Wu ◽  
Sergey Fomel
Keyword(s):  
Large Scale ◽  
Model Building ◽  
Ground Truth ◽  
Velocity Model ◽  
Training Data ◽  
Data Sets ◽  
Validation Data ◽  
Data Set ◽  
Seismic Image ◽  
Data Generator

Salt boundary interpretation is important for the understanding of salt tectonics and velocity model building for seismic migration. Conventional methods consist of computing salt attributes and extracting salt boundaries. We have formulated the problem as 3D image segmentation and evaluated an efficient approach based on deep convolutional neural networks (CNNs) with an encoder-decoder architecture. To train the model, we design a data generator that extracts randomly positioned subvolumes from large-scale 3D training data set followed by data augmentation, then feed a large number of subvolumes into the network while using salt/nonsalt binary labels generated by thresholding the velocity model as ground truth labels. We test the model on validation data sets and compare the blind test predictions with the ground truth. Our results indicate that our method is capable of automatically capturing subtle salt features from the 3D seismic image with less or no need for manual input. We further test the model on a field example to indicate the generalization of this deep CNN method across different data sets.

scholarly journals Global gridded precipitation over land: a description of the new GPCC First Guess Daily product

2014 ◽  
Vol 6 (1) ◽  
pp. 49-60 ◽  
Author(s):  
K. Schamm ◽  
M. Ziese ◽  
A. Becker ◽  
P. Finger ◽  
A. Meyer-Christoffer ◽  
...  
Keyword(s):  
Quality Control ◽  
Real Time ◽  
Daily Precipitation ◽  
Grid Cell ◽  
Rain Gauge ◽  
Kriging Interpolation ◽  
Telecommunication System ◽  
Full Data ◽  
Data Set ◽  
Global Telecommunication System

Abstract. This paper describes the new First Guess Daily product of the Global Precipitation Climatology Centre (GPCC). The new product gives an estimate of the global daily precipitation gridded at a spatial resolution of 1° latitude by 1° longitude. It is based on rain gauge data reported in near-real time via the Global Telecommunication System (GTS) and available about three to five days after the end of each observation month. In addition to the gridded daily precipitation totals in mm day−1, the standard deviation in mm day−1, the kriging interpolation error in % and the number of measurements per grid cell are also encoded into the monthly netCDF product file and provided for all months since January 2009. Prior to their interpolation, the measured precipitation values undergo a preliminary automatic quality control. For the calculation of the areal mean of the grid, anomalies are interpolated with ordinary block kriging. This approach allows for a near-real-time release. Therefore, the purely GTS-based data processing lacks an intensive quality control as well as a high data density and is denoted as First Guess. The daily data set is referenced under doi:10.5676/DWD_GPCC/FG_D_100. Two further products, the Full Data Daily and a merged satellite-gauge product, are currently under development at Deutscher Wetterdienst (DWD). These additional products will not be available in near-real time, but based on significantly more and strictly quality controlled observations. All GPCC products are provided free of charge via the GPCC webpage: ftp://ftp-anon.dwd.de/pub/data/gpcc/html/download_gate.html.

scholarly journals Exploratory Only: A Tool for Large-Scale Exploratory Analyses

2021 ◽  
Author(s):  
Jin Kim
Keyword(s):  
Web Application ◽  
Large Scale ◽  
Behavioral Science ◽  
A Priori ◽  
R Package ◽  
Data Set ◽  
Mediation Analyses ◽  
Follow Up Studies ◽  
Minimal Effort

This article presents Exploratory Only: an intuitive tool for conducting large-scale exploratory analyses easily and quickly. Available in three forms (as a web application, standalone program, and R Package) and launched as a point-and-click interface, Exploratory Only allows researchers to conduct all possible correlation, moderation, and mediation analyses among selected variables in their data set with minimal effort and time. Compared to a popular alternative, SPSS, Exploratory Only is shown to be orders of magnitude easier and faster at conducting exploratory analyses. The article demonstrates how to use Exploratory Only and discusses the caveat to using it. As long as researchers use Exploratory Only as intended—to discover novel hypotheses to investigate in follow-up studies, rather than to confirm nonexistent a priori hypotheses (i.e., p-hacking)—Exploratory Only can promote progress in behavioral science by encouraging more exploratory analyses and therefore more discoveries.

scholarly journals Envisat MIPAS measurements of CFC-11: retrieval, validation, and climatology

2008 ◽  
Vol 8 (2) ◽  
pp. 4561-4602 ◽  
Author(s):  
L. Hoffmann ◽  
M. Kaufmann ◽  
R. Spang ◽  
R. Müller ◽  
J. J. Remedios ◽  
...  
Keyword(s):  
Large Scale ◽  
Lower Stratosphere ◽  
A Priori ◽  
Michelson Interferometer ◽  
Optimal Estimation ◽  
Atmospheric Conditions ◽  
Data Set ◽  
Mixing Ratios ◽  
Priori Information ◽  
The Individual

Abstract. From July 2002 to March 2004 the Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) aboard the European Space Agency's Environmental Satellite (Envisat) measured nearly continuously mid infrared limb radiance spectra. These measurements are utilised to retrieve the global distribution of the chlorofluorocarbon CFC-11 by applying a new fast forward model for Envisat MIPAS and an accompanying optimal estimation retrieval processor. A detailed analysis shows that the total retrieval errors of the individual CFC-11 volume mixing ratios are typically below 10% and that the systematic components are dominating. Contribution of a priori information to the retrieval results are less than 5 to 10%. The vertical resolution of the observations is about 3 to 4 km. The data are successfully validated by comparison with several other space experiments, an air-borne in-situ instrument, measurements from ground-based networks, and independent Envisat MIPAS analyses. The retrieval results from 425 000 Envisat MIPAS limb scans are compiled to provide a new climatological data set of CFC-11. The climatology shows significantly lower CFC-11 abundances in the lower stratosphere compared with the Reference Atmospheres for MIPAS (RAMstan V3.1) climatology. Depending on the atmospheric conditions the differences between the climatologies are up to 30 to 110 ppt (45 to 150%) at 19 to 27 km altitude. Additionally, time series of CFC-11 mean abundance and variability for five latitudinal bands are presented. The observed CFC-11 distributions can be explained by the residual mean circulation and large-scale eddy-transports in the upper troposphere and lower stratosphere. The new CFC-11 data set is well suited for further scientific studies.

scholarly journals Two hours in Hollywood: A manually annotated ground truth data set of eye movements during movie clip watching

2020 ◽  
Vol 13 (4) ◽  
Author(s):  
Ioannis Agtzidis ◽  
Mikhail Startsev ◽  
Michael Dorr
Keyword(s):  
Eye Movement ◽  
Large Scale ◽  
Ground Truth ◽  
Training Set ◽  
Event Type ◽  
Short Article ◽  
Data Set ◽  
Ground Truth Data ◽  
The Gaze ◽  
Movie Clip

In this short article we present our manual annotation of the eye movement events in a subset of the large-scale eye tracking data set Hollywood2. Our labels include fixations, saccades, and smooth pursuits, as well as a noise event type (the latter representing either blinks, loss of tracking, or physically implausible signals). In order to achieve more consistent annotations, the gaze samples were labelled by a novice rater based on rudimentary algorithmic suggestions, and subsequently corrected by an expert rater. Overall, we annotated eye movement events in the recordings corresponding to 50 randomly selected test set clips and 6 training set clips from Hollywood2, which were viewed by 16 observers and amount to a total of approximately 130 minutes of gaze data. In these labels, 62.4% of the samples were attributed to fixations, 9.1% – to saccades, and, notably, 24.2% – to pursuit (the remainder marked as noise). After evaluation of 15 published eye movement classification algorithms on our newly collected annotated data set, we found that the most recent algorithms perform very well on average, and even reach human-level labelling quality for fixations and saccades, but all have a much larger room for improvement when it comes to smooth pursuit classification. The data set is made available at https://gin.g- node.org/ioannis.agtzidis/hollywood2_em.

scholarly journals Identification and Characterization of an Anomaly in Two-Dimensional Video Disdrometer Data

Atmosphere ◽  
2018 ◽  
Vol 9 (8) ◽  
pp. 315 ◽  
Author(s):  
Michael Larsen ◽  
Michael Schönhuber
Keyword(s):  
Ground Truth ◽  
Rain Event ◽  
Two Dimensional ◽  
Data Set ◽  
Sample Area ◽  
Precipitation Gauge ◽  
Rain Rates ◽  
Quantitative Impact ◽  
Rainfall Retrieval

The two-dimensional video distrometer (2DVD) is a well known ground based point-monitoring precipitation gauge, often used as a ground truth instrument to validate radar or satellite rainfall retrieval algorithms. This instrument records a number of variables for each detected hydrometeor, including the detected position within the sample area of the instrument. Careful analyses of real 2DVD data reveal an artifact—there are time periods where hydrometeor detections within parts of the sample area are artificially enhanced or diminished. Here, we (i) illustrate this anomaly with an exemplary 2DVD data set, (ii) describe the origin of this anomaly, (iii) develop and present an algorithm to help flag data potentially partially corrupted by this anomaly, and (iv) explore the prevalence and quantitative impact of this anomaly. Although the anomaly is seen in every major rain event studied and by every 2DVD the authors have examined, the anomaly artificially induces less than 3% of all detected drops and typically alters estimates of rain rates and accumulations by less than 2%.

scholarly journals Rain event detection in commercial microwave link attenuation data using convolutional neural networks

2019 ◽  
Author(s):  
Julius Polz ◽  
Christian Chwala ◽  
Maximilian Graf ◽  
Harald Kunstmann
Keyword(s):  
Time Series ◽  
Event Detection ◽  
Reference Data ◽  
Reference Method ◽  
Rain Gauge ◽  
Training Data ◽  
Set Covering ◽  
Rain Event ◽  
Data Set ◽  
Time Periods

Abstract. Quantitative precipitation estimation with commercial microwave links (CMLs) is a technique developed to supplement weather radar and rain gauge observations. It is exploiting the relation between the attenuation of CML signal levels and the integrated rain rate along a CML path. The opportunistic nature of this method requires a sophisticated data processing using robust methods. In this study we focus on the processing step of rain event detection in the signal level time series of the CMLs, which we treat as a binary classification problem. We analyze the performance of a convolutional neural network (CNN), which is trained to detect rainfall specific attenuation patterns in CML signal levels, using data from 3904 CMLs in Germany. The CNN consists of a feature extraction and a classification part with, in total, 20 layers of neurons and 1.4 x 105 trainable parameters. With a structure, inspired by the visual cortex of mammals, CNNs use local connections of neurons to recognize patterns independent of their location in the time-series. We test the CNNs ability to generalize to CMLs and time periods outside the training data. Our CNN is trained on four months of data from 400 randomly selected CMLs and validated on two different months of data, once for all CMLs and once for the 3504 CMLs not included in the training. No CMLs are excluded from the analysis. As a reference data set we use the gauge adjusted radar product RADOLAN-RW provided by the German meteorological service (DWD). The model predictions and the reference data are compared on an hourly basis. Model performance is compared to a reference method, which uses the rolling standard deviation of the CML signal level time series as a detection criteria. Our results show that within the analyzed period of April to September 2018, the CNN generalizes well to the validation CMLs and time periods. A receiver operating characteristic (ROC) analysis shows that the CNN is outperforming the reference method, detecting on average 87 % of all rainy and 91 % of all non-rainy periods. In conclusion, we find that CNNs are a robust and promising tool to detect rainfall induced attenuation patterns in CML signal levels from a large CML data set covering entire Germany.

scholarly journals Systematic prediction of drug resistance caused by transporter genes in cancer cells

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Yao Shen ◽  
Zhipeng Yan
Keyword(s):  
Drug Resistance ◽  
Large Scale ◽  
Ground Truth ◽  
Data Sets ◽  
Drug Transporter ◽  
Data Set ◽  
Ground Truth Data ◽  
Public Data ◽  
The Difference ◽  
Novel Drug

AbstractTo study the drug resistance problem caused by transporters, we leveraged multiple large-scale public data sets of drug sensitivity, cell line genetic and transcriptional profiles, and gene silencing experiments. Through systematic integration of these data sets, we built various machine learning models to predict the difference between cell viability upon drug treatment and the silencing of its target across the same cell lines. More than 50% of the models built with the same data set or with independent data sets successfully predicted the testing set with significant correlation to the ground truth data. Features selected by our models were also significantly enriched in known drug transporters annotated in DrugBank for more than 60% of the models. Novel drug-transporter interactions were discovered, such as lapatinib and gefitinib with ABCA1, olaparib and NVPADW742 with ABCC3, and gefitinib and AZ628 with SLC4A4. Furthermore, we identified ABCC3, SLC12A7, SLCO4A1, SERPINA1, and SLC22A3 as potential transporters for erlotinib, three of which are also significantly more highly expressed in patients who were resistant to therapy in a clinical trial.

Sign in / Sign up

Export Citation Format

Share Document