scholarly journals Towards Robust ResNet: A Small Step but a Giant Leap

2019 ◽  
Author(s):  
Jingfeng Zhang ◽  
Bo Han ◽  
Laura Wynter ◽  
Bryan Kian Hsiang Low ◽  
Mohan Kankanhalli
Keyword(s):  
Deep Neural Network ◽  
Empirical Evaluation ◽  
Euler Method ◽  
Factor H ◽  
Small Step ◽  
Residual Network ◽  
Systems Perspective ◽  
Giant Leap ◽  
Real World Datasets ◽  
Step Factor

This paper presents a simple yet principled approach to boosting the robustness of the residual network (ResNet) that is motivated by a dynamical systems perspective. Namely, a deep neural network can be interpreted using a partial differential equation, which naturally inspires us to characterize ResNet based on an explicit Euler method. This consequently allows us to exploit the step factor h in the Euler method to control the robustness of ResNet in both its training and generalization. In particular, we prove that a small step factor h can benefit its training and generalization robustness during backpropagation and forward propagation, respectively. Empirical evaluation on real-world datasets corroborates our analytical findings that a small h can indeed improve both its training and generalization robustness.

scholarly journals Competitive Caching with Machine Learned Advice

2021 ◽  
Vol 68 (4) ◽  
pp. 1-25
Author(s):  
Thodoris Lykouris ◽  
Sergei Vassilvitskii
Keyword(s):  
Online Algorithms ◽  
Empirical Evaluation ◽  
Optimal Solution ◽  
Poor Performance ◽  
Machine Learning Algorithms ◽  
Average Error ◽  
Generalization Error ◽  
Worst Case ◽  
Future Events ◽  
Real World Datasets

Traditional online algorithms encapsulate decision making under uncertainty, and give ways to hedge against all possible future events, while guaranteeing a nearly optimal solution, as compared to an offline optimum. On the other hand, machine learning algorithms are in the business of extrapolating patterns found in the data to predict the future, and usually come with strong guarantees on the expected generalization error. In this work, we develop a framework for augmenting online algorithms with a machine learned predictor to achieve competitive ratios that provably improve upon unconditional worst-case lower bounds when the predictor has low error. Our approach treats the predictor as a complete black box and is not dependent on its inner workings or the exact distribution of its errors. We apply this framework to the traditional caching problem—creating an eviction strategy for a cache of size k . We demonstrate that naively following the oracle’s recommendations may lead to very poor performance, even when the average error is quite low. Instead, we show how to modify the Marker algorithm to take into account the predictions and prove that this combined approach achieves a competitive ratio that both (i) decreases as the predictor’s error decreases and (ii) is always capped by O (log k ), which can be achieved without any assistance from the predictor. We complement our results with an empirical evaluation of our algorithm on real-world datasets and show that it performs well empirically even when using simple off-the-shelf predictions.

scholarly journals One Small Step for the Gram Stain, One Giant Leap for Clinical Microbiology

2016 ◽  
Vol 54 (6) ◽  
pp. 1416-1417 ◽  
Author(s):  
Richard B. Thomson
Keyword(s):  
Clinical Microbiology ◽  
Rapid Test ◽  
Error Rates ◽  
Clinical Microbiology Laboratory ◽  
Microbiology Laboratory ◽  
Gram Stain ◽  
Small Step ◽  
Link Type ◽  
Giant Leap

The Gram stain is one of the most commonly performed tests in the clinical microbiology laboratory, yet it is poorly controlled and lacks standardization. It was once the best rapid test in microbiology, but it is no longer trusted by many clinicians. The publication by Samuel et al. (J. Clin. Microbiol. 54:1442–1447, 2016,http://dx.doi.org/10.1128/JCM.03066-15) is a start for those who want to evaluate and improve Gram stain performance. In an age of emerging rapid molecular results, is the Gram stain still relevant? How should clinical microbiologists respond to the call to reduce Gram stain error rates?

scholarly journals One small step for e-voting, one giant leap for democracy

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Marina Gorbatiuc
Keyword(s):  
Human Rights ◽  
Law Enforcement ◽  
International Organizations ◽  
The Internet ◽  
Mutual Understanding ◽  
Small Step ◽  
Law Enforcement Training ◽  
Republic Of Moldova ◽  
Giant Leap ◽  
The Republic

E-voting is the next step of the Republic of Moldova evolution. The article presents advantages of e-voting, all steps of using it, and its many types. Electronic voting refers to elections using electronic means. E-voting can be managed by phones, the Internet, private computer networks or special kiosks. Reasons for accepting this kind of voting are provided. The analyzed in the article OSCE/ODIHR’s (Office for Democratic Institutions and Human Rights) activities which are related to tolerance and non-discrimination are focus on the following areas: legislation; law enforcement training; monitoring, reporting on, and following up on responses to hate-motivated crimes and incidents; as well as educational activities to promote tolerance, respect, and mutual understanding. Conclusions from all ODIHR activities which are carried out in close co-ordination and co-operation with OSCE participating States, OSCE institutions and field operations, as well as with other international organizations are given.

scholarly journals Network Embedding via a Bi-Mode and Deep Neural Network Model

2017 ◽  
Author(s):  
Yang Fang ◽  
Xiang Zhao ◽  
Zhen Tan
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Semantic Information ◽  
Dimensional Space ◽  
Relation Extraction ◽  
Network Embedding ◽  
Structure Information ◽  
Second Mode ◽  
Real World Datasets ◽  
Low Dimensional

Network Embedding (NE) is an important method to learn the representations of network via a low-dimensional space. Conventional NE models focus on capturing the structure information and semantic information of vertices while neglecting such information for edges. In this work, we propose a novel NE model named BimoNet to capture both the structure and semantic information of edges. BimoNet is composed of two parts, i.e., the bi-mode embedding part and the deep neural network part. For bi-mode embedding part, the first mode named add-mode is used to express the entity-shared features of edges and the second mode named subtract-mode is employed to represent the entity-specific features of edges. These features actually reflect the semantic information. For deep neural network part, we firstly regard the edges in a network as nodes, and the vertices as links, which will not change the overall structure of the whole network. Then we take the nodes' adjacent matrix as the input of the deep neural network as it can obtain similar representations for nodes with similar structure. Afterwards, by jointly optimizing the objective function of these two parts, BimoNet could preserve both the semantic and structure information of edges. In experiments, we evaluate BimoNet on three real-world datasets and task of relation extraction, and BimoNet is demonstrated to outperform state-of-the-art baseline models consistently and significantly.

One small step in frailty research, a giant leap in evidence based practice

2018 ◽  
Vol 16 (1) ◽  
pp. 1-3 ◽  
Author(s):  
João Apóstolo ◽  
Elzbieta Bobrowicz-Campos ◽  
Carol Holland ◽  
Antonio Cano
Keyword(s):  
Evidence Based Practice ◽  
Evidence Based ◽  
Small Step ◽  
Giant Leap

scholarly journals Deep Residual Network with Sparse Feedback for Image Restoration

2018 ◽  
Vol 8 (12) ◽  
pp. 2417 ◽  
Author(s):  
Zhenyu Guo ◽  
Yujuan Sun ◽  
Muwei Jian ◽  
Xiaofeng Zhang
Keyword(s):  
Neural Network ◽  
Image Restoration ◽  
Deep Neural Network ◽  
Convergence Speed ◽  
Fast Convergence ◽  
Experimental Results ◽  
Residual Network ◽  
Unknown Parameters ◽  
Noisy Images ◽  
Blurred Images

A deep neural network is difficult to train due to a large number of unknown parameters. To increase trainable performance, we present a moderate depth residual network for the restoration of motion blurring and noisy images. The proposed network has only 10 layers, and the sparse feedbacks are added in the middle and the last layers, which are called FbResNet. FbResNet has fast convergence speed and effective denoising performance. In addition, it can also reduce the artificial Mosaic trace at the seam of patches, and visually pleasant output results can be produced from the blurred images or noisy images. Experimental results show the effectiveness of our designed model and method.

Sign in / Sign up

Export Citation Format

Share Document