scholarly journals THINGSvision: A Python Toolbox for Streamlining the Extraction of Activations From Deep Neural Networks

2021 ◽  
Vol 15 ◽  
Author(s):  
Lukas Muttenthaler ◽  
Martin N. Hebart

Over the past decade, deep neural network (DNN) models have received a lot of attention due to their near-human object classification performance and their excellent prediction of signals recorded from biological visual systems. To better understand the function of these networks and relate them to hypotheses about brain activity and behavior, researchers need to extract the activations to images across different DNN layers. The abundance of different DNN variants, however, can often be unwieldy, and the task of extracting DNN activations from different layers may be non-trivial and error-prone for someone without a strong computational background. Thus, researchers in the fields of cognitive science and computational neuroscience would benefit from a library or package that supports a user in the extraction task. THINGSvision is a new Python module that aims at closing this gap by providing a simple and unified tool for extracting layer activations for a wide range of pretrained and randomly-initialized neural network architectures, even for users with little to no programming experience. We demonstrate the general utility of THINGsvision by relating extracted DNN activations to a number of functional MRI and behavioral datasets using representational similarity analysis, which can be performed as an integral part of the toolbox. Together, THINGSvision enables researchers across diverse fields to extract features in a streamlined manner for their custom image dataset, thereby improving the ease of relating DNNs, brain activity, and behavior, and improving the reproducibility of findings in these research fields.

2021 ◽  
Author(s):  
Lukas Muttenthaler ◽  
Martin N. Hebart

AbstractOver the past decade, deep neural network (DNN) models have received a lot of attention due to their near-human object classification performance and their excellent prediction of signals recorded from biological visual systems. To better understand the function of these networks and relate them to hypotheses about brain activity and behavior, researchers need to extract the activations to images across different DNN layers. The abundance of different DNN variants, however, can often be unwieldy, and the task of extracting DNN activations from different layers may be non-trivial and error-prone for someone without a strong computational background. Thus, researchers in the fields of cognitive science and computational neuroscience would benefit from a library or package that supports a user in the extraction task. THINGSvision is a new Python module that aims at closing this gap by providing a simple and unified tool for extracting layer activations for a wide range of pretrained and randomly-initialized neural network architectures, even for users with little to no programming experience. We demonstrate the general utility of THINGsvision by relating extracted DNN activations to a number of functional MRI and behavioral datasets using representational similarity analysis, which can be performed as an integral part of the toolbox. Together, THINGSvision enables researchers across diverse fields to extract features in a streamlined manner for their custom image dataset, thereby improving the ease of relating DNNs, brain activity, and behavior, and improving the reproducibility of findings in these research fields.


2017 ◽  
Author(s):  
Charlie W. Zhao ◽  
Mark J. Daley ◽  
J. Andrew Pruszynski

AbstractFirst-order tactile neurons have spatially complex receptive fields. Here we use machine learning tools to show that such complexity arises for a wide range of training sets and network architectures, and benefits network performance, especially on more difficult tasks and in the presence of noise. Our work suggests that spatially complex receptive fields are normatively good given the biological constraints of the tactile periphery.


Resources ◽  
2020 ◽  
Vol 9 (2) ◽  
pp. 15
Author(s):  
Juan Uribe-Toril ◽  
José Luis Ruiz-Real ◽  
Jaime de Pablo Valenciano

Sustainability, local development, and ecology are keywords that cover a wide range of research fields in both experimental and social sciences. The transversal nature of this knowledge area creates synergies but also divergences, making a continuous review of the existing literature necessary in order to facilitate research. There has been an increasing number of articles that have analyzed trends in the literature and the state-of-the-art in many subjects. In this Special Issue of Resources, the most prestigious researchers analyzed the past and future of Social Sciences in Resources from an economic, social, and environmental perspective.


Author(s):  
Jessica Barbosa Diniz ◽  
Filipe R. Cordeiro ◽  
Pericles B. C. Miranda ◽  
Laura A. Tomaz Da Silva

Deep Learning is a research area under the spotlight in recent years due to its successful application to many domains, such as computer vision and image recognition. The most prominent technique derived from Deep Learning is Convolutional Neural Network, which allows the network to automatically learn representations needed for detection or classification tasks. However, Convolutional Neural Networks have some limitations, as designing these networks are not easy to master and require expertise and insight. In this work, we present the use of Genetic Algorithm associated to Grammar-based Genetic Programming to optimize Convolution Neural Network architectures. To evaluate our proposed approach, we adopted CIFAR-10 dataset to validate the evolution of the generated architectures, using the metric of accuracy to evaluate its classification performance in the test dataset. The results demonstrate that our method using Grammar-based Genetic Programming can easily produce optimized CNN architectures that are competitive and achieve high accuracy results.


2021 ◽  
Author(s):  
Mark Schatza ◽  
Ethan Blackwood ◽  
Sumedh Nagrale ◽  
Alik S Widge

Closing the loop between brain activity and behavior is one of the most active areas of development in neuroscience. There is particular interest in developing closed-loop control of neural oscillations. Many studies report correlations between oscillations and functional processes. Oscillation-informed closed-loop experiments might determine whether these relationships are causal and would provide important mechanistic insights which may lead to new therapeutic tools. These closed-loop perturbations require accurate estimates of oscillatory phase and amplitude, which are challenging to compute in real time. We developed an easy to implement, fast and accurate Toolkit for Oscillatory Real-time Tracking and Estimation (TORTE). TORTE operates with the open-source Open Ephys GUI (OEGUI) system, making it immediately compatible with a wide range of acquisition systems and experimental preparations. TORTE efficiently extracts oscillatory phase and amplitude from a target signal and includes a variety of options to trigger closed-loop perturbations. Implementing these tools into existing experiments is easy and adds minimal latency to existing protocols. Most labs use in-house lab-specific approaches, limiting replication and extension of their experiments by other groups. Accuracy of the extracted analytic signal and accuracy of oscillation-informed perturbations with TORTE match presented results by these groups. However, TORTE provides access to these tools in a flexible, easy to use toolkit without requiring proprietary software. We hope that the availability of a high-quality, open-source, and broadly applicable toolkit will increase the number of labs able to perform oscillatory closed-loop experiments, and will improve the replicability of protocols and data across labs.


Author(s):  
David Bricher ◽  
Andreas Müller

In manufacturing industry, one of the main targets is to increase automation and ultimately to avoid failures under all circumstances. The plugging and locking of connectors is a class of tasks which is yet hard to be automatized with sufficiently high process stability. Due to the variation of plugging positions and external disturbances, e.g. occlusion due to cables, the quality assessment of plugging processes has emerged as a challenging task for image-based systems. For this reason, the proposed approach analyzes the inherent acoustic connector locking properties in combination with different neural network architectures in order to correctly identify connector locking signals and further to distinguish them from other machining events occurring in assembly plants. For this specific task, highly sensitive optical microphones have been applied for data acquisition. The proposed experiments are carried out under laboratory conditions as well as for the more complex situation in a real manufacturing environment. In this context, the usage of multimodal neural network architectures achieved highest levels in classification performance with accuracy levels close to 90%.


2019 ◽  
Vol 28 (06) ◽  
pp. 1960008 ◽  
Author(s):  
Grega Vrbančič ◽  
Iztok Fister ◽  
Vili Podgorelec

Over the past years, the application of deep neural networks in a wide range of areas is noticeably increasing. While many state-of-the-art deep neural networks are providing the performance comparable or in some cases even superior to humans, major challenges such as parameter settings for learning deep neural networks and construction of deep learning architectures still exist. The implications of those challenges have a significant impact on how a deep neural network is going to perform on a specific task. With the proposed method, presented in this paper, we are addressing the problem of parameter setting for a deep neural network utilizing swarm intelligence algorithms. In our experiments, we applied the proposed method variants to the classification task for distinguishing between phishing and legitimate websites. The performance of the proposed method is evaluated and compared against four different phishing datasets, two of which we prepared on our own. The results, obtained from the conducted empirical experiments, have proven the proposed approach to be very promising. By utilizing the proposed swarm intelligence based methods, we were able to statistically significantly improve the predictive performance when compared to the manually tuned deep neural network. In general, the improvement of classification accuracy ranges from 2.5% to 3.8%, while the improvement of F1-score reached even 24% on one of the datasets.


Philosophies ◽  
2020 ◽  
Vol 5 (4) ◽  
pp. 41
Author(s):  
Rolf Rutishauser

Plants and animals are both important for studies in evolutionary developmental biology (EvoDevo). Plant morphology as a valuable discipline of EvoDevo is set for a paradigm shift. Process thinking and the continuum approach in plant morphology allow us to perceive and interpret growing plants as combinations of developmental processes rather than as assemblages of structural units (“organs”) such as roots, stems, leaves, and flowers. These dynamic philosophical perspectives were already favored by botanists and philosophers such as Agnes Arber (1879–1960) and Rolf Sattler (*1936). The acceptance of growing plants as dynamic continua inspires EvoDevo scientists such as developmental geneticists and evolutionary biologists to move towards a more holistic understanding of plants in time and space. This review will appeal to many young scientists in the plant development research fields. It covers a wide range of relevant publications from the past to present.


2004 ◽  
Vol 21 ◽  
pp. 63-100 ◽  
Author(s):  
K. O. Stanley ◽  
R. Miikkulainen

Two major goals in machine learning are the discovery and improvement of solutions to complex problems. In this paper, we argue that complexification, i.e. the incremental elaboration of solutions through adding new structure, achieves both these goals. We demonstrate the power of complexification through the NeuroEvolution of Augmenting Topologies (NEAT) method, which evolves increasingly complex neural network architectures. NEAT is applied to an open-ended coevolutionary robot duel domain where robot controllers compete head to head. Because the robot duel domain supports a wide range of strategies, and because coevolution benefits from an escalating arms race, it serves as a suitable testbed for studying complexification. When compared to the evolution of networks with fixed structure, complexifying evolution discovers significantly more sophisticated strategies. The results suggest that in order to discover and improve complex solutions, evolution, and search in general, should be allowed to complexify as well as optimize.


Sign in / Sign up

Export Citation Format

Share Document