Long-term temporal averaging for stochastic optimization of deep neural networks

Much of the controversy evoked by the use of deep neural networks as models of biological neural systems amount to debates over what constitutes scientific progress in neuroscience. In order to discuss what constitutes scientific progress, one must have a goal in mind (progress towards what?). One such long term goal is to produce scientific explanations of intelligent capacities (e.g., object recognition, relational reasoning). I argue that the most pressing philosophical questions at the intersection of neuroscience and artificial intelligence are ultimately concerned with defining the phenomena to be explained and with what constitute valid explanations of such phenomena. I propose that a foundation in the philosophy of scientific explanation and understanding can scaffold future discussions about how an integrated science of intelligence might progress. Towards this vision, I review relevant theories of scientific explanation and discuss strategies for unifying the scientific goals of neuroscience and AI.

Download Full-text

Double Compressed AMR Audio Detection Using Long-Term Features and Deep Neural Networks

2019 11th International Conference on Electrical and Electronics Engineering (ELECO) ◽

10.23919/eleco47770.2019.8990372 ◽

2019 ◽

Author(s):

Aykut Buker ◽

Cemal Hanilci

Keyword(s):

Neural Networks ◽

Deep Neural Networks

Download Full-text

Forms of explanation and understanding for neuroscience and artificial intelligence

10.31234/osf.io/5g3pn ◽

2021 ◽

Author(s):

Jessica Anne Farrell Thompson

Keyword(s):

Artificial Intelligence ◽

Neural Networks ◽

Deep Neural Networks ◽

Scientific Explanation ◽

Scientific Progress ◽

Neural Systems ◽

Relational Reasoning ◽

Integrated Science ◽

Philosophical Questions

Much of the controversy evoked by the use of deep neural networks (DNNs) as models of biological neural systems amount to debates over what constitutes scientific progress in neuroscience. In order to discuss what constitutes scientific progress, one must have a goal in mind (progress towards what?). One such long term goal is to produce scientific explanations of intelligent capacities (e.g. object recognition, relational reasoning). I argue that the most pressing philosophical questions at the intersection of neuroscience and artificial intelligence are ultimately concerned with defining the phenomena to be explained and with what constitute valid explanations of such phenomena. As such, I propose that a foundation in the philosophy of scientific explanation and understanding can scaffold future discussions about how an integrated science of intelligence might progress. Towards this vision, I review several of the most relevant theories of scientific explanation and begin to outline candidate forms of explanation for neural and cognitive phenomena.

Download Full-text

Artificial intelligence in cardiology

Heart and Metabolism - Heart Failure in patients with Diabetes ◽

10.31887/hm.2020.82/ltarassenko ◽

2020 ◽

pp. 8-11

Keyword(s):

Artificial Intelligence ◽

Machine Learning ◽

Neural Networks ◽

Deep Neural Networks ◽

Automated Image Analysis ◽

Specific Domain ◽

Feature Discovery ◽

Medical Disciplines ◽

Second Wave

In the first wave of artificial intelligence (AI), rule-based expert systems were developed, with modest success, to help generalists who lacked expertise in a specific domain. The second wave of AI, originally called artificial neural networks but now described as machine learning, began to have an impact with multilayer networks in the 1980s. Deep learning, which enables automated feature discovery, has enjoyed spectacular success in several medical disciplines, including cardiology, from automated image analysis to the identification of the electrocardiographic signature of atrial fibrillation during sinus rhythm. Machine learning is now embedded within the NHS Long-Term Plan in England, but its widespread adoption may be limited by the “black-box” nature of deep neural networks.

Download Full-text

Runtime Long-Term Reliability Management Using Stochastic Computing in Deep Neural Networks

2021 22nd International Symposium on Quality Electronic Design (ISQED) ◽

10.1109/isqed51717.2021.9424285 ◽

2021 ◽

Author(s):

Yibo Liu ◽

Shuyuan Yu ◽

Shaoyi Peng ◽

Sheldon X.-D. Tan

Keyword(s):

Neural Networks ◽

Deep Neural Networks ◽

Reliability Management ◽

Stochastic Computing

Download Full-text

Adaptive Learning Rate via Covariance Matrix Based Preconditioning for Deep Neural Networks

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2017/267 ◽

2017 ◽

Cited By ~ 3

Author(s):

Yasutoshi Ida ◽

Yasuhiro Fujiwara ◽

Sotetsu Iwamura

Keyword(s):

Neural Networks ◽

Stochastic Optimization ◽

Covariance Matrix ◽

Adaptive Learning ◽

Deep Neural Networks ◽

Learning Rate ◽

First Order ◽

Adaptive Learning Rate ◽

Approximate Hessian

Adaptive learning rate algorithms such as RMSProp are widely used for training deep neural networks. RMSProp offers efficient training since it uses first order gradients to approximate Hessian-based preconditioning. However, since the first order gradients include noise caused by stochastic optimization, the approximation may be inaccurate. In this paper, we propose a novel adaptive learning rate algorithm called SDProp. Its key idea is effective handling of the noise by preconditioning based on covariance matrix. For various neural networks, our approach is more efficient and effective than RMSProp and its variant.

Download Full-text