The information and computational system for the massive parallel processing of radar data based on Apache Spark framework

2018 ◽  
Author(s):  
В.П. Потапов ◽  
С.Е. Попов ◽  
М.А. Костылев
Keyword(s):  
Parallel Processing ◽  
High Performance ◽  
Surface Displacement ◽  
Radar Data ◽  
Computational System ◽  
Radar Images ◽  
The Earth ◽  
Persistent Scatterer ◽  
Massive Parallel Processing ◽  
Spark Framework

Рассмотрена задача создания информационно-вычислительной системы обработки радарных снимков с возможностью визуализации, конфигурирования и запуска алгоритмов основных этапов процессинга интерферометрических данных методом Persistent Scatterer в интеграции с MPP-системой (Massive Parallel Processing) для высокопроизводительного мониторинга смещений земной поверхности участков аэрокосмической съемки. Приведены основные схемы маршрутизации потоков данных исполнения заданий. Представлена программная реализация в виде веб-портала на базе компонентов ReactJS, включая автоматизированную загрузку и обновление базы данных радарных снимков Sentinel-1A посредством технологии RESTful API. The aim of the presented work is the development of an information computational system for processing radar images with the ability to visualize, configure and run algorithms for the main stages of processing interferometric data by the Persistent Scatterer method integrated with the MPP system (massive parallel processing) for high-performance monitoring of the Earth surface displacement of aerospace survey sites. As a result of the analysis of the different approaches used in the processing of radar data and the review of distributed computing technologies, a distributed information system based on the architecture of massively parallel execution of the Apache Hadoop ecosystem processes the streaming post-processing of radar images and the construction of a displacement map was proposed and implemented. A software implementation is presented in the form of a web portal based on ReactJS components, including automated downloading and updating of the Sentinel-1A radar image database using RESTful API technology. The innovation of suggested solution consists of the model of the interaction between developed processing modules based on the isolated execution context with HDFS data storage during the preparing procedure and the complete cycle for the processing of the Earth surface displacement. An integrated approach to the developing scalable front-end and back-end software complex components with the use of ReactJS, Redux and Apache Spark framework was used for the first time. Supporting of WPS specification makes it possible using almost any GIS, which works with this standard. The evaluation of a scientific and technological level of research shows high performance of the developed system while maintaining the results quality. In particular, the adapted and integrated ESA SNAP Toolbox returned identical arrays of processed interferometric data in the per-pixel comparison but the speed of the procedure is several times faster.

scholarly journals Calculation of vertical displacement of the earth surface in the “Vostochny” open pit using radar data

2021 ◽  
Author(s):  
L.S. Mikov ◽  
S.E. Popov ◽  
V.P. Potapov
Keyword(s):  
Land Surface ◽  
Surface Displacement ◽  
Vertical Displacement ◽  
Radar Data ◽  
Open Pit ◽  
Earth Surface ◽  
The Earth ◽  
Small Baseline Subset ◽  
Satellite Radar ◽  
Small Baseline

The paper deals with the issues of assessment of the condition and changes in the land surface on the territory of the Vostochny open pit (Kemerovo region). The application of the multi-pass series of Sentinel-1 satellite radar data using the Small Baseline Subset (SBaS) method to determine the Earth surface displacement dynamics using constructed vertical displacement maps is demonstrated.

scholarly journals Determination of the earth's surface deformations in the "Kiyzassky" open pit according to Sentinel-1 Data

2020 ◽  
Vol 223 ◽  
pp. 03010
Author(s):  
Leonid Mikov ◽  
Semion Popov
Keyword(s):  
Land Surface ◽  
Surface Displacement ◽  
Vertical Displacement ◽  
Radar Data ◽  
Open Pit ◽  
The Earth ◽  
Small Baseline Subset ◽  
Satellite Radar ◽  
Small Baseline

The paper deals with the issues of assessment of the condition and changes in the land surface on the territory of the Kiizassky open pit (Kemerovo region) because of the landslide that occurred in June 2019. The application of the multi-pass series of Sentinel-1 satellite radar data using the Small Baseline Subset (SBaS) method to determine the Earth surface displacement dynamics using constructed vertical displacement maps is demonstrated..

The research study in the convergence of the results of processing radar images Sentinel-1A with field observations of agricultural sites

2019 ◽  
Vol 950 (8) ◽  
pp. 52-58
Author(s):  
D.V. Mozer ◽  
Е.L. Levin ◽  
A.K. Satbergenova
Keyword(s):  
Satellite Remote Sensing ◽  
Satellite Images ◽  
Research Study ◽  
High Accuracy ◽  
Agricultural Fields ◽  
Field Observations ◽  
Radar Images ◽  
The Earth ◽  
Radar Satellite ◽  
Modern Technologies

The manuscript discusses how to monitor the condition of seedlings on agricultural fields planted with winter wheat, fodder maize and areas of fir forest located in the Freudenstadt district of Baden-Wuerttemberg in Germany. To solve the range of agricultural problems , they often use modern technologies such as satellite remote sensing of the Earth. The paper displays the monitoring results of the Sentinel-1A radar satellites scenes, as well as visual spectrum imagery of field observations are presented when leaving directly to terrain segments. The processing deployed data chain, consisting of 11 Sentinel-1A scenes acquired in the timefrane from March to November 2018. Specifically, the SNAP Sentinel Toolboxes software was used to process the radar satellite images Sentinel-1А, the. Based on the the research outcomes the Committee of Agriculture of the Freudenstadt district is able to predict the yield amount with high accuracy due to good data convergence. According to the study, the following three important problems can be resolved by means of Sentinel-1A imagery

Fusion of rain radar images and wind forecasts in adeep learning model applied to rain nowcasting

2021 ◽  
Author(s):  
Anastase Charantonis ◽  
Vincent Bouget ◽  
Dominique Béréziat ◽  
Julien Brajard ◽  
Arthur Filoche
Keyword(s):  
Deep Learning ◽  
Optical Flow ◽  
Weather Forecast ◽  
Forecast Model ◽  
Radar Data ◽  
Learning Model ◽  
Rainfall Forecasting ◽  
Radar Images ◽  
Deep Learning Model ◽  
Rain Radar

<p>Short or mid-term rainfall forecasting is a major task with several environmental applications such as agricultural management or flood risks monitoring. Existing data-driven approaches, especially deep learning models, have shown significant skill at this task, using only rainfall radar images as inputs. In order to determine whether using other meteorological parameters such as wind would improve forecasts, we trained a deep learning model on a fusion of rainfall radar images and wind velocity produced by a weather forecast model. The network was compared to a similar architecture trained only on radar data, to a basic persistence model and to an approach based on optical flow. Our network outperforms by 8% the F1-score calculated for the optical flow on moderate and higher rain events for forecasts at a horizon time of 30 minutes. Furthermore, it outperforms by 7% the same architecture trained using only rainfall radar images. Merging rain and wind data has also proven to stabilize the training process and enabled significant improvement especially on the difficult-to-predict high precipitation rainfalls. These results can also be found in Bouget, V., Béréziat, D., Brajard, J., Charantonis, A., & Filoche, A. (2020). Fusion of rain radar images and wind forecasts in a deep learning model applied to rain nowcasting. arXiv preprint arXiv:2012.05015</p>

Technical photography of surface motion

1962 ◽  
Vol 52 (5) ◽  
pp. 1007-1016
Author(s):  
B. Carder ◽  
J. Hefferman ◽  
D. Barnes
Keyword(s):  
New Mexico ◽  
Long Range ◽  
Short Range ◽  
Surface Displacement ◽  
Earth Surface ◽  
Peak Displacement ◽  
Surface Motion ◽  
Target Weight ◽  
The Earth ◽  
Analysis Of Results

abstract Photographic measurements of the earth-surface displacement were made on the gnome event, an underground nuclear detonation near Carlsbad, New Mexico, November 1961. One long range and three short range photo stations were used to provide complementary coverage. Motionless inertia weights were measured against graduated targets rigidly anchored to the surface. The experiment is described in detail including target/weight arrangement, camera specifications, and photo station locations in relation to Surface Zero. Analysis of results from 6 films from close-in stations and one film from the long range station are reported. The peak displacement measured was slightly greater than six feet at a location 106 feet from surface zero.

scholarly journals Method for transforming symbolic radar marks of low-noticeable moving objects based on the Talbot effect

Radiotekhnika ◽  
2021 ◽  
pp. 129-137
Author(s):  
V. Zhyrnov ◽  
S. Solonskaya
Keyword(s):  
Moving Objects ◽  
Radar Data ◽  
Pulse Frequency ◽  
Asymptotic Equality ◽  
Talbot Effect ◽  
Radar Images ◽  
Frequency Code ◽  
Semantic Dependencies ◽  
Symbolic Images ◽  
Synchronous Pulse

In this paper a method to transform radar images of moving aerial objects with scintillating inter-period fluctuations, sometimes resulting to complete signal fading, using the Talbot effect is considered. These transformations are reduced to the establishment of a certain correspondence of the asymptotic equality of perception of visual images, arbitrarily changing in time and space, in the statement about the conditions of simple equality of perception of images of radar marks that have different frequencies of fluctuations. It is shown how this approach can be used to analyze radar data by transforming and smoothing scintillating signal fluctuations, invisible in the presence of interference, into visible symbolic images. First, to detect and recognize the aerial objects from the analysis of relations and functional (semantic) dependencies between attributes, second, to make a decision based on semantic components of symbolic radar images. The possibility of using such transformation to generate pulse-frequency code of fluctuations of the symbolic radar angel-echo images as an important characteristic for their recognition has been experimentally verified. Algorithms for generating symbolic images in asynchronous and synchronous pulse-frequency code are formulated. The symbolic image represented by such a code is considered as an additional feature for recognizing and filtering out natural interferences such as angel-echoes.

scholarly journals High performance radar images modelling and recognition of real objects

2019 ◽  
pp. 549-552
Author(s):  
D A Zherdev ◽  
V V Prokudin
Keyword(s):  
Parallel Algorithm ◽  
High Performance ◽  
3D Models ◽  
Parallel Program ◽  
Antenna Aperture ◽  
Radar Images ◽  
Scene Description ◽  
Real Objects

In the work there is a modernization of the parallel algorithm for the radar images formation of 3D models with the synthesis of the antenna aperture. In the formation of the scene description, the various structures are used in which it is possible to use more efficient and derived calculations. In addition, it is the topical task to recognize objects on radar images. Thus, on the basis of the implemented parallel program for modelling, the high performance required for simulating multiple radar images can be achieved.

Sign in / Sign up

Export Citation Format

Share Document