scholarly journals Rethink the Connections among Generalization, Memorization, and the Spectral Bias of DNNs

2021 ◽  
Author(s):  
Xiao Zhang ◽  
Haoyi Xiong ◽  
Dongrui Wu
Keyword(s):  
High Frequency ◽  
Deep Neural Networks ◽  
Random Noise ◽  
Experimental Setup ◽  
Training Set ◽  
Trade Off ◽  
High Frequencies ◽  
Test Error ◽  
Frequency Components ◽  
Bias Variance

Over-parameterized deep neural networks (DNNs) with sufficient capacity to memorize random noise can achieve excellent generalization performance, challenging the bias-variance trade-off in classical learning theory. Recent studies claimed that DNNs first learn simple patterns and then memorize noise; some other works showed a phenomenon that DNNs have a spectral bias to learn target functions from low to high frequencies during training. However, we show that the monotonicity of the learning bias does not always hold: under the experimental setup of deep double descent, the high-frequency components of DNNs diminish in the late stage of training, leading to the second descent of the test error. Besides, we find that the spectrum of DNNs can be applied to indicating the second descent of the test error, even though it is calculated from the training set only.

High-frequency atrial electrical activity

1961 ◽  
Vol 16 (2) ◽  
pp. 300-304 ◽  
Author(s):  
Cesar A. Caceres ◽  
George A. Kelser ◽  
Juan Calatayud
Keyword(s):  
High Frequency ◽  
Electrical Potential ◽  
P Wave ◽  
Right Atrial ◽  
High Frequencies ◽  
Pr Interval ◽  
Frequency Components ◽  
Left And Right ◽  
Electronic Filters ◽  
High Frequency Content

Left and right atrial intracavitary and conventional surface leads were used to study electrocardiographic activity during the PR interval. Electronic filters were employed for analysis of wave frequency and harmonic content from 1.7 to 1700 cps. Amplifiers permitting standardization sensitivity to 500 mm/mv were used to obtain oscilloscopic tracings recorded at a paper speed of 75 mm/sec. Frequency analysis of the electrical potential recorded during P wave inscription demonstrated the presence of high-frequency content that is excluded by conventional electrocardiographic amplifiers. The high-frequency components are associated with the time of inscription of the electrocardiographic intrinsic deflection and have a relationship to the characteristics of the pressure-pulse curve. These relationships suggest that intracavitary high frequencies and the electrocardiographic intrinsic deflection originate from electrical discharges associated with initiation of contractile events. Submitted on June 6, 1960

Cable Fault Recognition Based on the Wavelet Energy Spectrum of Mode Components

2013 ◽  
Vol 427-429 ◽  
pp. 834-837
Author(s):  
Mei Wang ◽  
Jing Wu
Keyword(s):  
Energy Spectrum ◽  
High Frequency ◽  
Transient Current ◽  
Power Cable ◽  
Two Phase ◽  
High Frequencies ◽  
Wavelet Energy ◽  
Fault Recognition ◽  
Frequency Components ◽  
Independent Mode

When a fault appeared in a power cable transmission line, the transient current with high frequencies would be produced in the system. Three independent mode components could be obtained by applying the phase mode transformation to the transient current. For different types of the faults, the three independent mode components have different features. Based on wavelet energy spectrum of mode components, a method for cable fault recognition is developed in this paper. First, the fault current is decomposed by using Karenbaue transformation matrix. Then, wavelet transformation is uses to obtain the coefficients of the high frequency components which reflect the original signal high frequency energy. Finally, based on the wavelet energy spectrum method and the detailed coefficient manipulation, the equivalent norms of the mode components are obtained. Compared with the traditional fault recognition method, the new method depends less on zero mode component in two-phase short to ground state, and it can recognize the fault class in the cases of different fault positions, different fault path resistances and different inception angles.

Progressive denoising of seismic data via robust noise estimation in dual domains

Geophysics ◽  
2020 ◽  
Vol 85 (1) ◽  
pp. V99-V118
Author(s):  
Yi Lin ◽  
Jinhai Zhang
Keyword(s):  
Seismic Data ◽  
High Frequency ◽  
Random Noise ◽  
Denoising Method ◽  
Data Set ◽  
Edge Preserving ◽  
Space Domain ◽  
Time Space ◽  
Frequency Components ◽  
Resolution Imaging

Random noise attenuation plays an important role in seismic data processing. Most traditional methods suppress random noise either in the time-space domain or in the transformed domain, which may encounter difficulty in retaining the detailed structures. We have introduced the progressive denoising method to suppress random noise in seismic data. This method estimates random noise at each sample independently by imposing proper constraints on local windowed data in the time-space domain and then in the transformed domain, and the denoised results of the whole data set are gradually improved by many iterations. First, we apply an unnormalized bilateral kernel in time-space domain to reject large-amplitude signals; then, we apply a range kernel in the frequency-wavenumber domain to reject medium-amplitude signals; finally, we can obtain a total estimate of random noise by repeating these steps approximately 30 times. Numerical examples indicate that the progressive denoising method can achieve a better denoising result, compared with the two typical single-domain methods: the [Formula: see text]-[Formula: see text] deconvolution method and the curvelet domain thresholding method. As an edge-preserving method, the progressive denoising method can greatly reduce the random noise without harming the useful signals, especially to those high-frequency components, which would be crucial for high-resolution imaging and interpretations in the following stages.

Computer Enhancement of Electron Micrographs

1970 ◽  
Vol 28 ◽  
pp. 28-29 ◽  
Author(s):  
Robert Nathan
Keyword(s):  
High Frequency ◽  
Signal To Noise Ratio ◽  
Random Noise ◽  
Video Recording ◽  
Video Camera ◽  
Noise Removal ◽  
Computer Application ◽  
Modulation Transfer ◽  
Frequency Components ◽  
Scan Line

The digital computer with associated video camera scanner equipment can be used in conjunction with the microscope's best performance to give improved and ultimately perhaps atomic resolution. Examples of computer application emphasize space photography where the computer enhancement methods originated. The following operations evolved as a result of working with television images: geometric correction, photometric correction, random noise removal, periodic noise removal, scan-line noise removal, and modulation transfer function ,(MTF) correction (compensation for attenuation of spatial high-frequency components).Other examples include the enhancement of a micrograph of negatively stained catalase, a micrograph of some microtubules and a video recording of indathrene olive. In a crystal, because of its periodicity, improvement can be obtained in the signal to noise ratio by translating and superimposing the image upon itself.

Simulations of high-resolution images of amorphous silicon films

1986 ◽  
Vol 44 ◽  
pp. 544-545
Author(s):  
G. Y. Fan ◽  
J. M. Cowley
Keyword(s):  
Electron Microscope ◽  
High Frequency ◽  
Fourier Transformation ◽  
Amorphous Materials ◽  
Low Frequency ◽  
Chromatic Aberration ◽  
Structure Information ◽  
Frequency Components ◽  
High Resolution Images ◽  
Spatial Frequency Spectrum

It is well known that the structure information on the specimen is not always faithfully transferred through the electron microscope. Firstly, the spatial frequency spectrum is modulated by the transfer function (TF) at the focal plane. Secondly, the spectrum suffers high frequency cut-off by the aperture (or effectively damping terms such as chromatic aberration). While these do not have essential effect on imaging crystal periodicity as long as the low order Bragg spots are inside the aperture, although the contrast may be reversed, they may change the appearance of images of amorphous materials completely. Because the spectrum of amorphous materials is continuous, modulation of it emphasizes some components while weakening others. Especially the cut-off of high frequency components, which contribute to amorphous image just as strongly as low frequency components can have a fundamental effect. This can be illustrated through computer simulation. Imaging of a whitenoise object with an electron microscope without TF limitation gives Fig. 1a, which is obtained by Fourier transformation of a constant amplitude combined with random phases generated by computer.

A Novel Adaptive PET/CT Image Fusion Algorithm

2019 ◽  
Vol 14 (7) ◽  
pp. 658-666
Author(s):  
Kai-jian Xia ◽  
Jian-qiang Wang ◽  
Jian Cai
Keyword(s):  
Lung Cancer ◽  
Image Fusion ◽  
High Frequency ◽  
Malignant Tumors ◽  
Low Frequency ◽  
Combination Method ◽  
Ct Image ◽  
Fusion Algorithm ◽  
Pet Ct ◽  
Frequency Components

Background: Lung cancer is one of the common malignant tumors. The successful diagnosis of lung cancer depends on the accuracy of the image obtained from medical imaging modalities. Objective: The fusion of CT and PET is combining the complimentary and redundant information both images and can increase the ease of perception. Since the existing fusion method sare not perfect enough, and the fusion effect remains to be improved, the paper proposes a novel method called adaptive PET/CT fusion for lung cancer in Piella framework. Methods: This algorithm firstly adopted the DTCWT to decompose the PET and CT images into different components, respectively. In accordance with the characteristics of low-frequency and high-frequency components and the features of PET and CT image, 5 membership functions are used as a combination method so as to determine the fusion weight for low-frequency components. In order to fuse different high-frequency components, we select the energy difference of decomposition coefficients as the match measure, and the local energy as the activity measure; in addition, the decision factor is also determined for the high-frequency components. Results: The proposed method is compared with some of the pixel-level spatial domain image fusion algorithms. The experimental results show that our proposed algorithm is feasible and effective. Conclusion: Our proposed algorithm can better retain and protrude the lesions edge information and the texture information of lesions in the image fusion.

Despeckling of Sar Images Using Shrinkage of Two-Dimensional Discrete Orthonormal S-Transform

2020 ◽  
pp. 2150023
Author(s):  
Priya R. Kamath ◽  
Kedarnath Senapati ◽  
P. Jidesh
Keyword(s):  
High Frequency ◽  
Low Frequency ◽  
Relevant Information ◽  
Detection Algorithm ◽  
Two Dimensional ◽  
Sar Images ◽  
S Transform ◽  
Processing Step ◽  
Frequency Components ◽  
Shock Filter

Speckles are inherent to SAR. They hide and undermine several relevant information contained in the SAR images. In this paper, a despeckling algorithm using the shrinkage of two-dimensional discrete orthonormal S-transform (2D-DOST) coefficients in the transform domain along with shock filter is proposed. Also, an attempt has been made as a post-processing step to preserve the edges and other details while removing the speckle. The proposed strategy involves decomposing the SAR image into low and high-frequency components and processing them separately. A shock filter is used to smooth out the small variations in low-frequency components, and the high-frequency components are treated with a shrinkage of 2D-DOST coefficients. The edges, for enhancement, are detected using a ratio-based edge detection algorithm. The proposed method is tested, verified, and compared with some well-known models on C-band and X-band SAR images. A detailed experimental analysis is illustrated.

scholarly journals Electrophysiological aftereffects of high-frequency transcranial random noise stimulation (hf-tRNS): an EEG investigation

2021 ◽  
Author(s):  
Filippo Ghin ◽  
Louise O’Hare ◽  
Andrea Pavan
Keyword(s):  
High Frequency ◽  
Discrimination Task ◽  
Temporal Dynamics ◽  
Random Noise ◽  
Decomposition Analysis ◽  
Oscillatory Activity ◽  
Motion Direction ◽  
Time Frequency ◽  
Direction Discrimination ◽  
Transcranial Random Noise Stimulation

AbstractThere is evidence that high-frequency transcranial random noise stimulation (hf-tRNS) is effective in improving behavioural performance in several visual tasks. However, so far there has been limited research into the spatial and temporal characteristics of hf-tRNS-induced facilitatory effects. In the present study, electroencephalogram (EEG) was used to investigate the spatial and temporal dynamics of cortical activity modulated by offline hf-tRNS on performance on a motion direction discrimination task. We used EEG to measure the amplitude of motion-related VEPs over the parieto-occipital cortex, as well as oscillatory power spectral density (PSD) at rest. A time–frequency decomposition analysis was also performed to investigate the shift in event-related spectral perturbation (ERSP) in response to the motion stimuli between the pre- and post-stimulation period. The results showed that the accuracy of the motion direction discrimination task was not modulated by offline hf-tRNS. Although the motion task was able to elicit motion-dependent VEP components (P1, N2, and P2), none of them showed any significant change between pre- and post-stimulation. We also found a time-dependent increase of the PSD in alpha and beta bands regardless of the stimulation protocol. Finally, time–frequency analysis showed a modulation of ERSP power in the hf-tRNS condition for gamma activity when compared to pre-stimulation periods and Sham stimulation. Overall, these results show that offline hf-tRNS may induce moderate aftereffects in brain oscillatory activity.

scholarly journals Rheology of rounded mammalian cells over continuous high-frequencies

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Gotthold Fläschner ◽  
Cosmin I. Roman ◽  
Nico Strohmeyer ◽  
David Martinez-Martin ◽  
Daniel J. Müller
Keyword(s):  
High Frequency ◽  
Mammalian Cells ◽  
Mechanical Response ◽  
Cell Mass ◽  
Low Frequency ◽  
Mass Measurements ◽  
High Frequencies ◽  
Polymer Relaxation ◽  
Continuous Frequency ◽  
Rheological Experiments

AbstractUnderstanding the viscoelastic properties of living cells and their relation to cell state and morphology remains challenging. Low-frequency mechanical perturbations have contributed considerably to the understanding, yet higher frequencies promise to elucidate the link between cellular and molecular properties, such as polymer relaxation and monomer reaction kinetics. Here, we introduce an assay, that uses an actuated microcantilever to confine a single, rounded cell on a second microcantilever, which measures the cell mechanical response across a continuous frequency range ≈ 1–40 kHz. Cell mass measurements and optical microscopy are co-implemented. The fast, high-frequency measurements are applied to rheologically monitor cellular stiffening. We find that the rheology of rounded HeLa cells obeys a cytoskeleton-dependent power-law, similar to spread cells. Cell size and viscoelasticity are uncorrelated, which contrasts an assumption based on the Laplace law. Together with the presented theory of mechanical de-embedding, our assay is generally applicable to other rheological experiments.

Sign in / Sign up

Export Citation Format

Share Document