scholarly journals CLUSTERING TIME SERIES OF REPEATED SCAN DATA OF SANDY BEACHES

2019 ◽  
Vol XLII-2/W13 ◽  
pp. 1039-1046
Author(s):  
R. Lindenbergh ◽  
S. van der Kleij ◽  
M. Kuschnerus ◽  
S. Vos ◽  
S. de Vries
Keyword(s):  
Time Series ◽  
Laser Scanning ◽  
Sandy Beaches ◽  
Data Sets ◽  
Data Set ◽  
Spatially Correlated ◽  
Long Time ◽  
Clustering Approach ◽  
Scan Data ◽  
Year 2000

<p><strong>Abstract.</strong> Sandy beaches are highly dynamic areas affected by different natural and anthropogenic effects. Large changes, caused by a storm for example, are in general well-understood and easy to measure. Most times, only small changes, at the centimeter scale, are occurring, but these changes accumulate significantly over periods from weeks to months. Laser scanning is a suitable technique to measure such small signals, as it is able to obtain dense 3D terrain data at centimeter level in a time span of minutes. In this work we consider two repeated laser scan data sets of two different beaches in The Netherlands. The first data set is from around the year 2000 and consists of six consecutive yearly airborne laser scan data sets of a beach on Texel. The second data set is from 2017 and consists of 30 consecutive daily terrestrial scans of a beach near The Hague. So far, little work has been done on time series analysis of repeated scan data. To obtain a first grouping of morphologic processes, we propose to use a simple un-supervised clustering approach, k-means clustering, on de-leveled, cumulative point-wise time series. The results for both regions of interest, obtained using k=5 and k=10 clusters, indicate that such clustering gives a meaningful decomposition of the morphological laser scan data into clusters that exhibit similar change patterns. At the same time, we realize that the chosen approach is just a first step in a wide open topic of clustering spatially correlated long time series of morphological laser scan data as are now obtained by permanent laser scanning.</p>

Using FMI Transaction Data in Simulations

2016 ◽  
pp. 102-123
Author(s):  
Richard Heuver ◽  
Ronald Heijmans
Keyword(s):  
Time Series ◽  
Scenario Analysis ◽  
Statistical Problem ◽  
Point Of View ◽  
Data Sets ◽  
Insufficient Data ◽  
Data Set ◽  
Statistical Point ◽  
Transaction Data ◽  
Long Time

In this chapter the authors provide a method to aggregate large value payment system transaction data for executing simulations with the Bank of Finland payment simulator. When transaction data sets get large, simulation may become too time consuming in terms of computer power. Therefore, insufficient data from a statistical point of view can be processed. The method described in this chapter provides a solution to this statistical problem. In order to work around this problem the authors provide a method to aggregate transaction data set in such a way that it does not compromise the outcome of the simulation significantly. Depending on the type of simulations only a few business days or up to a year of data is required. In case of stress scenario analysis, in which e.g. liquidity position of banks deteriorates, long time series are preferred as business days can differ substantially. As an example this chapter shows that aggregating all low value transactions in the Dutch part of TARGET2 will not lead to a significantly different simulation outcome.

scholarly journals Time Series Clustering- Introduction to Healthcare System

2019 ◽  
Vol 9 (1) ◽  
pp. 2958-2963
Keyword(s):  
Time Series ◽  
Large Data ◽  
Large Data Sets ◽  
Data Sets ◽  
Data Set ◽  
Pull Out ◽  
Clustering Approach ◽  
Class Labels ◽  
Clustering Data ◽  
Extract Information

A clustering technique is an appropriate solvable approach for classifying information while no existence of premature information pertaining to class labels, using promising techniques like cloud based computing and big data over latest years. Investigating awareness was gradually piled up with unsupervised methods such as clustering approaches to pull out useful information from the data set available. Time series based clustering data was used in most of the technical domains to extract information enriched patterns to power the data analysis which extracts useful essence from complicated as well as large data sets. It is mostly not possible for large datasets using classification approach whereas clustering approach will resolve the problem with aid of unsupervised techniques. In the proposed methodology, main spotlight on time series health care datasets, one of the kind of admired data in clustering approaches. This summary will expose 4 major components of Time series approaches.

scholarly journals Construction of merged satellite total O<sub>3</sub> and NO<sub>2</sub> time series in the tropics for trend studies and evaluation by comparison to NDACC SAOZ measurements

2014 ◽  
Vol 7 (10) ◽  
pp. 3337-3354 ◽  
Author(s):  
M. Pastel ◽  
J.-P. Pommereau ◽  
F. Goutail ◽  
A. Richter ◽  
A. Pazmiño ◽  
...  
Keyword(s):  
Time Series ◽  
Lower Stratosphere ◽  
Atmospheric Composition ◽  
Data Sets ◽  
Composition Change ◽  
Long Time ◽  
Uv Visible ◽  
The Tropics ◽  
Lower Noise ◽  
Total Column

Abstract. Long time series of ozone and NO2 total column measurements in the southern tropics are available from two ground-based SAOZ (Système d'Analyse par Observation Zénithale) UV-visible spectrometers operated within the Network for the Detection of Atmospheric Composition Change (NDACC) in Bauru (22° S, 49° W) in S-E Brazil since 1995 and Reunion Island (21° S, 55° E) in the S-W Indian Ocean since 1993. Although the stations are located at the same latitude, significant differences are observed in the columns of both species, attributed to differences in tropospheric content and equivalent latitude in the lower stratosphere. These data are used to identify which satellites operating during the same period, are capturing the same features and are thus best suited for building reliable merged time series for trend studies. For ozone, the satellites series best matching SAOZ observations are EP-TOMS (1995–2004) and OMI-TOMS (2005–2011), whereas for NO2, best results are obtained by combining GOME version GDP5 (1996–2003) and SCIAMACHY – IUP (2003–2011), displaying lower noise and seasonality in reference to SAOZ. Both merged data sets are fully consistent with the larger columns of the two species above South America and the seasonality of the differences between the two stations, reported by SAOZ, providing reliable time series for further trend analyses and identification of sources of interannual variability in the future analysis.

scholarly journals FINE REGISTRATION OF KILO-STATION NETWORKS - A MODERN PROCEDURE FOR TERRESTRIAL LASER SCANNING DATA SETS

2016 ◽  
Vol XLI-B5 ◽  
pp. 485-492
Author(s):  
J.-F. Hullo
Keyword(s):  
Laser Scanning ◽  
Laser Scanner ◽  
Data Sets ◽  
Data Set ◽  
Information Models ◽  
3D Network ◽  
Laser Scanners ◽  
Building Information ◽  
Set Up ◽  
Fine Registration

We propose a complete methodology for the fine registration and referencing of kilo-station networks of terrestrial laser scanner data currently used for many valuable purposes such as 3D as-built reconstruction of Building Information Models (BIM) or industrial asbuilt mock-ups. This comprehensive target-based process aims to achieve the global tolerance below a few centimetres across a 3D network including more than 1,000 laser stations spread over 10 floors. This procedure is particularly valuable for 3D networks of indoor congested environments. In situ, the use of terrestrial laser scanners, the layout of the targets and the set-up of a topographic control network should comply with the expert methods specific to surveyors. Using parametric and reduced Gauss-Helmert models, the network is expressed as a set of functional constraints with a related stochastic model. During the post-processing phase inspired by geodesy methods, a robust cost function is minimised. At the scale of such a data set, the complexity of the 3D network is beyond comprehension. The surveyor, even an expert, must be supported, in his analysis, by digital and visual indicators. In addition to the standard indicators used for the adjustment methods, including Baarda’s reliability, we introduce spectral analysis tools of graph theory for identifying different types of errors or a lack of robustness of the system as well as <i>in fine</i> documenting the quality of the registration.

scholarly journals Coastal Change Patterns from Time Series Clustering of Permanent Laser Scan Data

2020 ◽  
Author(s):  
Mieke Kuschnerus ◽  
Roderik Lindenbergh ◽  
Sander Vos
Keyword(s):  
Time Series ◽  
Laser Scanning ◽  
Clustering Algorithm ◽  
Coastal Areas ◽  
Small Scale ◽  
Temporal Data ◽  
Agglomerative Clustering ◽  
Data Set ◽  
Deformation Processes ◽  
Spatio Temporal

Abstract. Sandy coasts are constantly changing environments governed by complex interacting processes. Permanent laser scanning is a promising technique to monitor such coastal areas and support analysis of geomorphological deformation processes. This novel technique delivers 3D representations of a part of the coast at hourly temporal and centimetre spatial resolution and allows to observe small scale changes in elevation over extended periods of time. These observations have the potential to improve understanding and modelling of coastal deformation processes. However, to be of use to coastal researchers and coastal management, an efficient way to find and extract deformation processes from the large spatio-temporal data set is needed. In order to allow data mining in an automated way, we extract time series in elevation or range and use unsupervised learning algorithms to derive a partitioning of the observed area according to change patterns. We compare three well known clustering algorithms, k-means, agglomerative clustering and DBSCAN, and identify areas that undergo similar evolution during one month. We test if they fulfil our criteria for a suitable clustering algorithm on our exemplary data set. The three clustering methods are applied to time series of 30 epochs (during one month) extracted from a data set of daily scans covering a part of the coast at Kijkduin, the Netherlands. A small section of the beach, where a pile of sand was accumulated by a bulldozer is used to evaluate the performance of the algorithms against a ground truth. The k-means algorithm and agglomerative clustering deliver similar clusters, and both allow to identify a fixed number of dominant deformation processes in sandy coastal areas, such as sand accumulation by a bulldozer or erosion in the intertidal area. The DBSCAN algorithm finds clusters for only about 44 % of the area and turns out to be more suitable for the detection of outliers, caused for example by temporary objects on the beach. Our study provides a methodology to efficiently mine a spatio-temporal data set for predominant deformation patterns with the associated regions, where they occur.

scholarly journals Loi and Gong Low-Degree Rotational Splittings

1998 ◽  
Vol 185 ◽  
pp. 167-168
Author(s):  
T. Appourchaux ◽  
M.C. Rabello-Soares ◽  
L. Gizon
Keyword(s):  
Time Series ◽  
The Other ◽  
Data Sets ◽  
Data Set ◽  
Low Degree ◽  
Fourier Spectra

Two different data sets have been used to derive low-degree rotational splittings. One data set comes from the Luminosity Oscillations Imager of VIRGO on board SOHO; the observation starts on 27 March 96 and ends on 26 March 97, and are made of intensity time series of 12 pixels (Appourchaux et al, 1997, Sol. Phys., 170, 27). The other data set was kindly made available by the GONG project; the observation starts on 26 August 1995 and ends on 21 August 1996, and are made of complex Fourier spectra of velocity time series for l = 0 − 9. For the GONG data, the contamination of l = 1 from the spatial aliases of l = 6 and l = 9 required some cleaning. To achieve this, we applied the inverse of the leakage matrix of l = 1, 6 and 9 to the original Fourier spectra of the same degrees; cleaning of all 3 degrees was achieved simultaneously (Appourchaux and Gizon, 1997, these proceedings).

scholarly journals An assessment of Bayesian bias estimator for numerical weather prediction

2008 ◽  
Vol 15 (6) ◽  
pp. 1013-1022 ◽  
Author(s):  
J. Son ◽  
D. Hou ◽  
Z. Toth
Keyword(s):  
Time Series ◽  
Numerical Weather Prediction ◽  
Sampling Error ◽  
Weather Prediction ◽  
Training Data ◽  
Statistical Characteristics ◽  
Forecast Errors ◽  
Data Sets ◽  
Data Set ◽  
Numerical Weather

Abstract. Various statistical methods are used to process operational Numerical Weather Prediction (NWP) products with the aim of reducing forecast errors and they often require sufficiently large training data sets. Generating such a hindcast data set for this purpose can be costly and a well designed algorithm should be able to reduce the required size of these data sets. This issue is investigated with the relatively simple case of bias correction, by comparing a Bayesian algorithm of bias estimation with the conventionally used empirical method. As available forecast data sets are not large enough for a comprehensive test, synthetically generated time series representing the analysis (truth) and forecast are used to increase the sample size. Since these synthetic time series retained the statistical characteristics of the observations and operational NWP model output, the results of this study can be extended to real observation and forecasts and this is confirmed by a preliminary test with real data. By using the climatological mean and standard deviation of the meteorological variable in consideration and the statistical relationship between the forecast and the analysis, the Bayesian bias estimator outperforms the empirical approach in terms of the accuracy of the estimated bias, and it can reduce the required size of the training sample by a factor of 3. This advantage of the Bayesian approach is due to the fact that it is less liable to the sampling error in consecutive sampling. These results suggest that a carefully designed statistical procedure may reduce the need for the costly generation of large hindcast datasets.

scholarly journals Reconstruction of Axial Tomographic High Resolution Data from Confocal Fluorescence Microscopy: A Method for Improving 3D FISH Images

2000 ◽  
Vol 20 (1) ◽  
pp. 7-15 ◽  
Author(s):  
R. Heintzmann ◽  
G. Kreth ◽  
C. Cremer
Keyword(s):  
High Resolution ◽  
Laser Scanning ◽  
Axial Direction ◽  
Laser Scanning Microscopy ◽  
Three Dimensions ◽  
Confocal Laser ◽  
Data Sets ◽  
Data Set ◽  
High Resolution Data ◽  
3D Data

Fluorescent confocal laser scanning microscopy allows an improved imaging of microscopic objects in three dimensions. However, the resolution along the axial direction is three times worse than the resolution in lateral directions. A method to overcome this axial limitation is tilting the object under the microscope, in a way that the direction of the optical axis points into different directions relative to the sample. A new technique for a simultaneous reconstruction from a number of such axial tomographic confocal data sets was developed and used for high resolution reconstruction of 3D‐data both from experimental and virtual microscopic data sets. The reconstructed images have a highly improved 3D resolution, which is comparable to the lateral resolution of a single deconvolved data set. Axial tomographic imaging in combination with simultaneous data reconstruction also opens the possibility for a more precise quantification of 3D data. The color images of this publication can be accessed from http://www.esacp.org/acp/2000/20‐1/heintzmann.htm. At this web address an interactive 3D viewer is additionally provided for browsing the 3D data. This java applet displays three orthogonal slices of the data set which are dynamically updated by user mouse clicks or keystrokes.

scholarly journals Two decades of satellite observations of AOD over mainland China using ATSR-2, AATSR and MODIS/Terra: data set evaluation and large-scale patterns

2018 ◽  
Vol 18 (3) ◽  
pp. 1573-1592 ◽  
Author(s):  
Gerrit de Leeuw ◽  
Larisa Sogacheva ◽  
Edith Rodriguez ◽  
Konstantinos Kourtidis ◽  
Aristeidis K. Georgoulias ◽  
...  
Keyword(s):  
Time Series ◽  
European Space Agency ◽  
Mainland China ◽  
Satellite Observations ◽  
Data Sets ◽  
Data Set ◽  
Modis Aod ◽  
The Difference ◽  
Along Track ◽  
Aerosol Properties

Abstract. The retrieval of aerosol properties from satellite observations provides their spatial distribution over a wide area in cloud-free conditions. As such, they complement ground-based measurements by providing information over sparsely instrumented areas, albeit that significant differences may exist in both the type of information obtained and the temporal information from satellite and ground-based observations. In this paper, information from different types of satellite-based instruments is used to provide a 3-D climatology of aerosol properties over mainland China, i.e., vertical profiles of extinction coefficients from the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP), a lidar flying aboard the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) satellite and the column-integrated extinction (aerosol optical depth – AOD) available from three radiometers: the European Space Agency (ESA)'s Along-Track Scanning Radiometer version 2 (ATSR-2), Advanced Along-Track Scanning Radiometer (AATSR) (together referred to as ATSR) and NASA's Moderate Resolution Imaging Spectroradiometer (MODIS) aboard the Terra satellite, together spanning the period 1995–2015. AOD data are retrieved from ATSR using the ATSR dual view (ADV) v2.31 algorithm, while for MODIS Collection 6 (C6) the AOD data set is used that was obtained from merging the AODs obtained from the dark target (DT) and deep blue (DB) algorithms, further referred to as the DTDB merged AOD product. These data sets are validated and differences are compared using Aerosol Robotic Network (AERONET) version 2 L2.0 AOD data as reference. The results show that, over China, ATSR slightly underestimates the AOD and MODIS slightly overestimates the AOD. Consequently, ATSR AOD is overall lower than that from MODIS, and the difference increases with increasing AOD. The comparison also shows that neither of the ATSR and MODIS AOD data sets is better than the other one everywhere. However, ATSR ADV has limitations over bright surfaces which the MODIS DB was designed for. To allow for comparison of MODIS C6 results with previous analyses where MODIS Collection 5.1 (C5.1) data were used, also the difference between the C6 and C5.1 merged DTDB data sets from MODIS/Terra over China is briefly discussed. The AOD data sets show strong seasonal differences and the seasonal features vary with latitude and longitude across China. Two-decadal AOD time series, averaged over all of mainland China, are presented and briefly discussed. Using the 17 years of ATSR data as the basis and MODIS/Terra to follow the temporal evolution in recent years when the environmental satellite Envisat was lost requires a comparison of the data sets for the overlapping period to show their complementarity. ATSR precedes the MODIS time series between 1995 and 2000 and shows a distinct increase in the AOD over this period. The two data series show similar variations during the overlapping period between 2000 and 2011, with minima and maxima in the same years. MODIS extends this time series beyond the end of the Envisat period in 2012, showing decreasing AOD.

Uncertainty of Climatol adjustment algorithm for daily time series of additive climate variables

2020 ◽  
Author(s):  
Oleg Skrynyk ◽  
Enric Aguilar ◽  
José A. Guijarro ◽  
Sergiy Bubin
Keyword(s):  
Time Series ◽  
Climate Model ◽  
Data Sets ◽  
Climate Variables ◽  
The European Union ◽  
Data Set ◽  
Raw Data ◽  
Climate Signal ◽  
Daily Time Series ◽  
Daily Time

&lt;p&gt;Before using climatological time series in research studies, it is necessary to perform their quality control and homogenization in order to remove possible artefacts (inhomogeneities) usually present in the raw data sets. In the vast majority of cases, the homogenization procedure allows to improve the consistency of the data, which then can be verified by means of the statistical comparison of the raw and homogenized time series. However, a new question then arises: how far are the homogenized data from the true climate signal or, in other words, what errors could still be present in homogenized data?&lt;/p&gt;&lt;p&gt;The main objective of our work is to estimate the uncertainty produced by the adjustment algorithm of the widely used Climatol homogenization software when homogenizing daily time series of the additive climate variables. We focused our efforts on the minimum and maximum air temperature. In order to achieve our goal we used a benchmark data set created by the INDECIS&lt;sup&gt;*&lt;/sup&gt; project. The benchmark contains clean data, extracted from an output of the Royal Netherlands Meteorological Institute Regional Atmospheric Climate Model (version 2) driven by Hadley Global Environment Model 2 - Earth System, and inhomogeneous data, created by introducing realistic breaks and errors.&lt;/p&gt;&lt;p&gt;The statistical evaluation of discrepancies between the homogenized (by means of Climatol with predefined break points) and clean data sets was performed using both a set of standard parameters and a metrics introduced in our work. All metrics used clearly identifies the main features of errors (systematic and random) present in the homogenized time series. We calculated the metrics for every time series (only over adjusted segments) as well as their averaged values as measures of uncertainties in the whole data set.&lt;/p&gt;&lt;p&gt;In order to determine how the two key parameters of the raw data collection, namely the length of time series and station density, influence the calculated measures of the adjustment error we gradually decreased the length of the period and number of stations in the area under study. The total number of cases considered was 56, including 7 time periods (1950-2005, 1954-2005, &amp;#8230;, 1974-2005) and 8 different quantities of stations (100, 90, &amp;#8230;, 30). Additionally, in order to find out how stable are the calculated metrics for each of the 56 cases and determine their confidence intervals we performed 100 random permutations in the introduced inhomogeneity time series and repeated our calculations With that the total number of homogenization exercises performed was 5600 for each of two climate variables.&lt;/p&gt;&lt;p&gt;Lastly, the calculated metrics were compared with the corresponding values, obtained for raw time series. The comparison showed some substantial improvement of the metric values after homogenization in each of the 56 cases considered (for the both variables).&lt;/p&gt;&lt;p&gt;-------------------&lt;/p&gt;&lt;p&gt;&lt;sup&gt;*&lt;/sup&gt;INDECIS is a part of ERA4CS, an ERA-NET initiated by JPI Climate, and funded by FORMAS (SE), DLR (DE), BMWFW (AT), IFD (DK), MINECO (ES), ANR (FR) with co-funding by the European Union (Grant 690462). The work has been partially supported by the Ministry of Education and Science of Kazakhstan (Grant BR05236454) and Nazarbayev University (Grant 090118FD5345).&lt;/p&gt;

Sign in / Sign up

Export Citation Format

Share Document