FluentPS: A Parameter Server Design with Low-frequency Synchronization for Distributed Deep Learning

Underwater target recognition is an important supporting technology for the development of marine resources, which is mainly limited by the purity of feature extraction and the universality of recognition schemes. The low-frequency analysis and recording (LOFAR) spectrum is one of the key features of the underwater target, which can be used for feature extraction. However, the complex underwater environment noise and the extremely low signal-to-noise ratio of the target signal lead to breakpoints in the LOFAR spectrum, which seriously hinders the underwater target recognition. To overcome this issue and to further improve the recognition performance, we adopted a deep-learning approach for underwater target recognition, and a novel LOFAR spectrum enhancement (LSE)-based underwater target-recognition scheme was proposed, which consists of preprocessing, offline training, and online testing. In preprocessing, we specifically design a LOFAR spectrum enhancement based on multi-step decision algorithm to recover the breakpoints in LOFAR spectrum. In offline training, the enhanced LOFAR spectrum is adopted as the input of convolutional neural network (CNN) and a LOFAR-based CNN (LOFAR-CNN) for online recognition is developed. Taking advantage of the powerful capability of CNN in feature extraction, the recognition accuracy can be further improved by the proposed LOFAR-CNN. Finally, extensive simulation results demonstrate that the LOFAR-CNN network can achieve a recognition accuracy of 95.22%, which outperforms the state-of-the-art methods.

Download Full-text

The Old and the New: Can Physics-Informed Deep-Learning Replace Traditional Linear Solvers?

Frontiers in Big Data ◽

10.3389/fdata.2021.669097 ◽

2021 ◽

Vol 4 ◽

Author(s):

Stefano Markidis

Keyword(s):

Neural Networks ◽

Deep Learning ◽

High Performance ◽

Critical Role ◽

Low Frequency ◽

Accurate Solution ◽

Limiting Factor ◽

Data Set ◽

Linear Solvers ◽

Computational Performance

Physics-Informed Neural Networks (PINN) are neural networks encoding the problem governing equations, such as Partial Differential Equations (PDE), as a part of the neural network. PINNs have emerged as a new essential tool to solve various challenging problems, including computing linear systems arising from PDEs, a task for which several traditional methods exist. In this work, we focus first on evaluating the potential of PINNs as linear solvers in the case of the Poisson equation, an omnipresent equation in scientific computing. We characterize PINN linear solvers in terms of accuracy and performance under different network configurations (depth, activation functions, input data set distribution). We highlight the critical role of transfer learning. Our results show that low-frequency components of the solution converge quickly as an effect of the F-principle. In contrast, an accurate solution of the high frequencies requires an exceedingly long time. To address this limitation, we propose integrating PINNs into traditional linear solvers. We show that this integration leads to the development of new solvers whose performance is on par with other high-performance solvers, such as PETSc conjugate gradient linear solvers, in terms of performance and accuracy. Overall, while the accuracy and computational performance are still a limiting factor for the direct use of PINN linear solvers, hybrid strategies combining old traditional linear solver approaches with new emerging deep-learning techniques are among the most promising methods for developing a new class of linear solvers.

Download Full-text

FWI with reconstructed low frequency data: A label-free physics-integrated deep learning approach

10.3997/2214-4609.202084006 ◽

2020 ◽

Author(s):

W. Hu ◽

Y. Jin ◽

X. Wu ◽

J. Chen

Keyword(s):

Deep Learning ◽

Low Frequency ◽

Learning Approach ◽

Label Free ◽

Frequency Data

Download Full-text

EEG microstates of dreams

Scientific Reports ◽

10.1038/s41598-020-74075-z ◽

2020 ◽

Vol 10 (1) ◽

Cited By ~ 2

Author(s):

Lucie Bréchet ◽

Denis Brunet ◽

Lampros Perogamvros ◽

Giulio Tononi ◽

Christoph M. Michel

Keyword(s):

Temporal Dynamics ◽

Conscious Experience ◽

Low Frequency ◽

Nrem Sleep ◽

Occipital Cortex ◽

Perceptual Content ◽

Frequency Synchronization ◽

Dream Recall ◽

Brain States ◽

Waking Up

Abstract Why do people sometimes report that they remember dreams, while at other times they recall no experience? Despite the interest in dreams that may happen during the night, it has remained unclear which brain states determine whether these conscious experiences will occur and what prevents us from waking up during these episodes. Here we address this issue by comparing the EEG activity preceding awakenings with recalled vs. no recall of dreams using the EEG microstate approach. This approach characterizes transiently stable brain states of sub-second duration that involve neural networks with nearly synchronous dynamics. We found that two microstates (3 and 4) dominated during NREM sleep compared to resting wake. Further, within NREM sleep, microstate 3 was more expressed during periods followed by dream recall, whereas microstate 4 was less expressed. Source localization showed that microstate 3 encompassed the medial frontal lobe, whereas microstate 4 involved the occipital cortex, as well as thalamic and brainstem structures. Since NREM sleep is characterized by low-frequency synchronization, indicative of neuronal bistability, we interpret the increased presence of the “frontal” microstate 3 as a sign of deeper local deactivation, and the reduced presence of the “occipital” microstate 4 as a sign of local activation. The latter may account for the occurrence of dreaming with rich perceptual content, while the former may account for why the dreaming brain may undergo executive disconnection and remain asleep. This study demonstrates that NREM sleep consists of alternating brain states whose temporal dynamics determine whether conscious experience arises.

Download Full-text

Deep learning for low-frequency extrapolation from multioffset seismic data

Geophysics ◽

10.1190/geo2018-0884.1 ◽

2019 ◽

Vol 84 (6) ◽

pp. R989-R1001 ◽

Cited By ~ 19

Author(s):

Oleg Ovcharenko ◽

Vladimir Kazei ◽

Mahesh Kalita ◽

Daniel Peter ◽

Tariq Alkhalifah

Keyword(s):

Deep Learning ◽

Seismic Data ◽

Signal To Noise Ratio ◽

Waveform Inversion ◽

Low Frequency ◽

Low Frequencies ◽

Seismic Surveys ◽

Full Waveform ◽

Frequency Components ◽

Marine Seismic

Low-frequency seismic data are crucial for convergence of full-waveform inversion (FWI) to reliable subsurface properties. However, it is challenging to acquire field data with an appropriate signal-to-noise ratio in the low-frequency part of the spectrum. We have extrapolated low-frequency data from the respective higher frequency components of the seismic wavefield by using deep learning. Through wavenumber analysis, we find that extrapolation per shot gather has broader applicability than per-trace extrapolation. We numerically simulate marine seismic surveys for random subsurface models and train a deep convolutional neural network to derive a mapping between high and low frequencies. The trained network is then tested on sections from the BP and SEAM Phase I benchmark models. Our results indicate that we are able to recover 0.25 Hz data from the 2 to 4.5 Hz frequencies. We also determine that the extrapolated data are accurate enough for FWI application.

Download Full-text

Intelligent Force-Measurement System Use in Shock Tunnel

Sensors ◽

10.3390/s20216179 ◽

2020 ◽

Vol 20 (21) ◽

pp. 6179

Author(s):

Yunpeng Wang ◽

Zonglin Jiang

Keyword(s):

Deep Learning ◽

Measurement System ◽

Force Measurement ◽

Aerodynamic Force ◽

Low Frequency ◽

Shock Tunnel ◽

New Method ◽

Test Results ◽

Force Measurement System ◽

The Impact

The inertial vibration of the force measurement system (FMS) has a large influence on the force measuring result of aircraft, especially on some tests carried out in high-enthalpy impulse facilities, such as in a shock tunnel. When force tests are conducted in a shock tunnel, the low-frequency vibrations of the FMS and its motion cannot be addressed through digital filtering because of the inertial forces, which are caused by the impact flow during the starting process of the shock tunnel. Therefore, this paper focuses on the dynamic characteristics of the performance of the FMS. A new method—i.e., deep-learning-based single-vector dynamic self-calibration (DL-based SV-DSC) of an impulse FMS, is proposed to increase the accuracy of aerodynamic force measurements in a shock tunnel. A deep-learning technique is used to train the dynamic model of the FMS in this study. Convolutional neural networks with a simple structure are applied to describe the dynamic modeling so that the low-frequency vibration signals are eliminated from the test results of the shock tunnel. By validation of the force test results measured in a shock tunnel, the current trained model can realize intelligent processing of the balance signals of the FMS. Based on this new method of dynamic calibration, the reliability and accuracy of force data processing are well verified.

Download Full-text

Rapid learning in visual cortical networks

eLife ◽

10.7554/elife.08417 ◽

2015 ◽

Vol 4 ◽

Cited By ~ 2

Author(s):

Ye Wang ◽

Valentin Dragoi

Keyword(s):

Brain Activity ◽

Local Population ◽

Low Frequency ◽

Spike Timing ◽

Learning Performance ◽

Cell Populations ◽

Frequency Synchronization ◽

Population Activity ◽

Area V4 ◽

Visual Cortical

Although changes in brain activity during learning have been extensively examined at the single neuron level, the coding strategies employed by cell populations remain mysterious. We examined cell populations in macaque area V4 during a rapid form of perceptual learning that emerges within tens of minutes. Multiple single units and LFP responses were recorded as monkeys improved their performance in an image discrimination task. We show that the increase in behavioral performance during learning is predicted by a tight coordination of spike timing with local population activity. More spike-LFP theta synchronization is correlated with higher learning performance, while high-frequency synchronization is unrelated with changes in performance, but these changes were absent once learning had stabilized and stimuli became familiar, or in the absence of learning. These findings reveal a novel mechanism of plasticity in visual cortex by which elevated low-frequency synchronization between individual neurons and local population activity accompanies the improvement in performance during learning.

Download Full-text

Identification of Arrhythmia by Using a Decision Tree and Gated Network Fusion Model

Computational and Mathematical Methods in Medicine ◽

10.1155/2021/6665357 ◽

2021 ◽

Vol 2021 ◽

pp. 1-13

Author(s):

Peng Lu ◽

Yabin Zhang ◽

Bing Zhou ◽

Hongpo Zhang ◽

Liwei Chen ◽

...

Keyword(s):

Deep Learning ◽

Decision Tree ◽

Frequency Domain ◽

Weight Control ◽

High Reliability ◽

Low Frequency ◽

Model Parameters ◽

Fusion Model ◽

Tree Model ◽

Time Frequency

In recent years, deep learning (DNN) based methods have made leapfrogging level breakthroughs in detecting cardiac arrhythmias as the cost effectiveness of arithmetic power, and data size has broken through the tipping point. However, the inability of these methods to provide a basis for modeling decisions limits clinicians’ confidence on such methods. In this paper, a Gate Recurrent Unit (GRU) and decision tree fusion model, referred to as (T-GRU), was designed to explore the problem of arrhythmia recognition and to improve the credibility of deep learning methods. The fusion model multipathway processing time-frequency domain featured the introduction of decision tree probability analysis of frequency domain features, the regularization of GRU model parameters and weight control to improve the decision tree model output weights. The MIT-BIH arrhythmia database was used for validation. Results showed that the low-frequency band features dominated the model prediction. The fusion model had an accuracy of 98.31%, sensitivity of 96.85%, specificity of 98.81%, and precision of 96.73%, indicating its high reliability and clinical significance.

Download Full-text

Convolutional Neural Network to Detect Deep Low-Frequency Tremors from Seismic Waveform Images

Lecture Notes in Computer Science - Trends and Applications in Knowledge Discovery and Data Mining ◽

10.1007/978-3-030-75015-2_4 ◽

2021 ◽

pp. 31-43

Author(s):

Ryosuke Kaneko ◽

Hiromichi Nagao ◽

Shin-ichi Ito ◽

Kazushige Obara ◽

Hiroshi Tsuruoka

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Seismic Data ◽

Low Frequency ◽

Seismic Waveforms ◽

Seismic Waveform ◽

Numerous Paper ◽

Large Earthquakes ◽

Activation Mapping

AbstractThe installation of dense seismometer arrays in Japan approximately 20 years ago has led to the discovery of deep low-frequency tremors, which are oscillations clearly different from ordinary earthquakes. As such tremors may be related to large earthquakes, it is an important issue in seismology to investigate tremors that occurred before establishing dense seismometer arrays. We use deep learning aiming to detect evidence of tremors from past seismic data of more than 50 years ago, when seismic waveforms were printed on paper. First, we construct a convolutional neural network (CNN) based on the ResNet architecture to extract tremors from seismic waveform images. Experiments applying the CNN to synthetic images generated according to seismograph paper records show that the trained model can correctly determine the presence of tremors in the seismic waveforms. In addition, the gradient-weighted class activation mapping clearly indicates the tremor location on each image. Thus, the proposed CNN has a strong potential for detecting tremors on numerous paper records, which can enable to deepen the understanding of the relations between tremors and earthquakes.

Download Full-text