Design of a Network Intrusion Detection System Using Complex Deep Neuronal Networks

Recent years have witnessed a tremendous development in various scientific and industrial fields. As a result, different types of networks are widely introduced which are vulnerable to intrusion. In view of the same, numerous studies have been devoted to detecting all types of intrusion and protect the networks from these penetrations. In this paper, a novel network intrusion detection system has been designed to detect cyber-attacks using complex deep neuronal networks. The developed system is trained and tested on the standard dataset KDDCUP99 via pycharm program. Relevant to existing intrusion detection methods with similar deep neuronal networks and traditional machine learning algorithms, the proposed detection system achieves better results in terms of detection accuracy.

Dynamical Characteristics of Recurrent Neuronal Networks Are Robust Against Low Synaptic Weight Resolution

The representation of the natural-density, heterogeneous connectivity of neuronal network models at relevant spatial scales remains a challenge for Computational Neuroscience and Neuromorphic Computing. In particular, the memory demands imposed by the vast number of synapses in brain-scale network simulations constitute a major obstacle. Limiting the number resolution of synaptic weights appears to be a natural strategy to reduce memory and compute load. In this study, we investigate the effects of a limited synaptic-weight resolution on the dynamics of recurrent spiking neuronal networks resembling local cortical circuits and develop strategies for minimizing deviations from the dynamics of networks with high-resolution synaptic weights. We mimic the effect of a limited synaptic weight resolution by replacing normally distributed synaptic weights with weights drawn from a discrete distribution, and compare the resulting statistics characterizing firing rates, spike-train irregularity, and correlation coefficients with the reference solution. We show that a naive discretization of synaptic weights generally leads to a distortion of the spike-train statistics. If the weights are discretized such that the mean and the variance of the total synaptic input currents are preserved, the firing statistics remain unaffected for the types of networks considered in this study. For networks with sufficiently heterogeneous in-degrees, the firing statistics can be preserved even if all synaptic weights are replaced by the mean of the weight distribution. We conclude that even for simple networks with non-plastic neurons and synapses, a discretization of synaptic weights can lead to substantial deviations in the firing statistics unless the discretization is performed with care and guided by a rigorous validation process. For the network model used in this study, the synaptic weights can be replaced by low-resolution weights without affecting its macroscopic dynamical characteristics, thereby saving substantial amounts of memory.

A mean-field toolbox for spiking neuronal network model analysis

Mean-field theory of spiking neuronal networks has led to numerous advances in our analytical and intuitive understanding of the dynamics of neuronal network models during the past decades. But, the elaborate nature of many of the developed methods, as well as the difficulty of implementing them, may limit the wider neuroscientific community from taking maximal advantage of these tools. In order to make them more accessible, we implemented an extensible, easy-to-use open-source Python toolbox that collects a variety of mean-field methods for the widely used leaky integrate-and-fire neuron model. The Neuronal Network Mean-field Toolbox (NNMT) in its current state allows for estimating properties of large neuronal networks, such as firing rates, power spectra, and dynamical stability in mean-field and linear response approximation, without running simulations on high performance systems. In this article we describe how the toolbox is implemented, show how it is used to calculate neuronal network properties, and discuss different use-cases, such as extraction of network mechanisms, parameter space exploration, or hybrid modeling approaches. Although the initial version of the toolbox focuses on methods that are close to our own past and present research, its structure is designed to be open and extensible. It aims to provide a platform for collecting analytical methods for neuronal network model analysis and we discuss how interested scientists can share their own methods via this platform.

ZnO nanowire microelectrode arrays for integration with neuronal networks

<p>Microelectrode arrays (MEAs) have been shown as a successful approach for neuroscientists to monitor the signal communication within the neuronal networks for understanding the functionality of the nervous system. However, using conventional planar MEAs is shown to be incapable of precise signal recording from neuronal networks at single-cell resolution due to low signal-to-ratio (SNR). This thesis looks at developing an electronic platform that comprises of zinc oxide nanowires (ZnO-NWs) on MEAs as a future device to record action potential (AP) signals with high SNR from human neuronal networks at single-cell resolution. Specifically, I studied the controlled growth of ZnO nanowires with various morphologies at exact locations across the substrate. I then investigated the biocompatibility of ZnO nanowires with different morphology and geometry for interaction with human NTera2.D1 (hNT) neurons. Finally, I examined the electrical characteristics of MEAs that were integrated with ZnO nanowires and metal encapsulated ZnO nanowires in comparison to the planar MEAs. The hydrothermal growth of ZnO nanowires is thoroughly investigated as a technique to allow synthesis of the nanowires at a low temperature (95°C) with a low cost and high scalability that can also be applied on flexible substrates. The morphology of the ZnO nanowires was varied (diameters of 20–300 nm, lengths of 0.15–6.2 µm, aspect ratios of 6–95 and densities of 10–285 NWs/µm²) by controlling the critical growth parameters such as the precursor concentration (2.5–150 mM), growth time (1–20 h) and additive polyethylenimine (PEI) concentration (0–8 mM). The diameter and length of the ZnO nanowires were increased by increasing the precursor concentration and growth time. Using the standard precursor concentration of 25 mM, growth times of up to 4 h were found effective for the active growth of the nanowires due to the consumption of the precursor ions and precipitation of ZnO. The addition of 6 mM PEI to the growth solution was shown to mediate the growth solution, allowing the extension of the nanowire growth to 20 h or longer. The PEI molecules were also attached to the lateral faces of the nanowires that confined their lateral growth and promoted their axial growth (enhanced aspect ratio from 12 ± 3 to 67 ± 21). Standard photolithography techniques were also introduced to selectively grow ZnO nanowires on exact locations across the substrates. The role of the ZnO seed layer geometry, seed layer area and gap, on the growth of ZnO nanowires was also investigated. Despite using the constant growth parameters (25 mM of precursor concentration with 4 h of growth time) changing the seed line widths (4 µm–1 mm) and the gap between the seed lines (2 µm–800 µm) resulted in the morphology of the nanowires to vary across the same substrate (diameters of 50–240 nm, lengths of 1.2–4.6 µm, aspect ratios of 9–34 and densities of 28–120 NWs/µm²). The seed area ratio of 50% was determined as a threshold to influence the nanowire morphology, where decreasing the seed area ratio below 50% (by increasing the adjacent gap or decreasing the seed layer area) increased the growth rate of the nanowires. The biocompatibility of ZnO nanowires with human hNT neurons was investigated in this work for the first time. The adhesion and growth of hNT neurons on the arrays of ZnO nanowire florets were determined to be influenced by both geometry and morphology of the nanowires. The growth of the hNT neurons was promoted by 30% compared to the control Si/SiO₂ substrate surface when ZnO nanowires with lengths shorter than 500 nm and densities higher than 350 NWs/µm² were grown. The hNT neurons on all nanowires were also demonstrated to be functionally viable as they responded to the glutamate stimulation. ZnO nanowires were shown to improve the electrical properties of the MEAs by reducing the electrochemical impedance due to the increased 3D surface area. The ZnO nanowires that were grown with 50 mM of precursor concentration for 4 h of growth time lowered the impedance from 835 ± 40 kΩ of planar Cr/Au MEAs to 540 ± 20 kΩ at a frequency of 1 kHz. In contrast, the ZnO nanowires that were grown with PEI for 35 h showed that despite the increased surface area by a factor of 45× the impedance was found to be quite high, 2.25 ± 0.2 MΩ at 1 kHz of frequency. The adsorption of PEI molecules to the lateral surfaces of the nanowires was thought to behave as a passivation layer that could have restricted the charge transfer characteristics of the ZnO-NW MEAs. Encapsulation of the pristine ZnO nanowires that were grown with standard precursor concentration of 25 mM for 4 h of growth time with different metallic layers (Cr/Au, Ti and Pt) further improved the electrical characteristics of the MEAs. The ZnO nanowires that were encapsulated with a 10 nm thin layer of Ti and Pt achieved the lowest electrochemical impedance of 400 ± 25 kΩ at 1 kHz in this work. The robustness of the Ti encapsulated ZnO nanowires were also improved in comparison to the PEI ZnO nanowires. The improved electrochemical characteristics and mechanical stability of the MEAs integrated with metal encapsulated ZnO nanowires have shown a great promise for improving the SNR of recording signals from neuronal cells for long term measurements. This work concludes that both pristine ZnO nanowire MEAs and metal encapsulated ZnO nanowire MEAs will be capable of recording AP signals from human neuronal networks at single-cell resolution. However, further optimisation and extensions of the work are required to record AP signals from human neuronal cells.</p>

ZnO nanowire microelectrode arrays for integration with neuronal networks

<p>Microelectrode arrays (MEAs) have been shown as a successful approach for neuroscientists to monitor the signal communication within the neuronal networks for understanding the functionality of the nervous system. However, using conventional planar MEAs is shown to be incapable of precise signal recording from neuronal networks at single-cell resolution due to low signal-to-ratio (SNR). This thesis looks at developing an electronic platform that comprises of zinc oxide nanowires (ZnO-NWs) on MEAs as a future device to record action potential (AP) signals with high SNR from human neuronal networks at single-cell resolution. Specifically, I studied the controlled growth of ZnO nanowires with various morphologies at exact locations across the substrate. I then investigated the biocompatibility of ZnO nanowires with different morphology and geometry for interaction with human NTera2.D1 (hNT) neurons. Finally, I examined the electrical characteristics of MEAs that were integrated with ZnO nanowires and metal encapsulated ZnO nanowires in comparison to the planar MEAs. The hydrothermal growth of ZnO nanowires is thoroughly investigated as a technique to allow synthesis of the nanowires at a low temperature (95°C) with a low cost and high scalability that can also be applied on flexible substrates. The morphology of the ZnO nanowires was varied (diameters of 20–300 nm, lengths of 0.15–6.2 µm, aspect ratios of 6–95 and densities of 10–285 NWs/µm²) by controlling the critical growth parameters such as the precursor concentration (2.5–150 mM), growth time (1–20 h) and additive polyethylenimine (PEI) concentration (0–8 mM). The diameter and length of the ZnO nanowires were increased by increasing the precursor concentration and growth time. Using the standard precursor concentration of 25 mM, growth times of up to 4 h were found effective for the active growth of the nanowires due to the consumption of the precursor ions and precipitation of ZnO. The addition of 6 mM PEI to the growth solution was shown to mediate the growth solution, allowing the extension of the nanowire growth to 20 h or longer. The PEI molecules were also attached to the lateral faces of the nanowires that confined their lateral growth and promoted their axial growth (enhanced aspect ratio from 12 ± 3 to 67 ± 21). Standard photolithography techniques were also introduced to selectively grow ZnO nanowires on exact locations across the substrates. The role of the ZnO seed layer geometry, seed layer area and gap, on the growth of ZnO nanowires was also investigated. Despite using the constant growth parameters (25 mM of precursor concentration with 4 h of growth time) changing the seed line widths (4 µm–1 mm) and the gap between the seed lines (2 µm–800 µm) resulted in the morphology of the nanowires to vary across the same substrate (diameters of 50–240 nm, lengths of 1.2–4.6 µm, aspect ratios of 9–34 and densities of 28–120 NWs/µm²). The seed area ratio of 50% was determined as a threshold to influence the nanowire morphology, where decreasing the seed area ratio below 50% (by increasing the adjacent gap or decreasing the seed layer area) increased the growth rate of the nanowires. The biocompatibility of ZnO nanowires with human hNT neurons was investigated in this work for the first time. The adhesion and growth of hNT neurons on the arrays of ZnO nanowire florets were determined to be influenced by both geometry and morphology of the nanowires. The growth of the hNT neurons was promoted by 30% compared to the control Si/SiO₂ substrate surface when ZnO nanowires with lengths shorter than 500 nm and densities higher than 350 NWs/µm² were grown. The hNT neurons on all nanowires were also demonstrated to be functionally viable as they responded to the glutamate stimulation. ZnO nanowires were shown to improve the electrical properties of the MEAs by reducing the electrochemical impedance due to the increased 3D surface area. The ZnO nanowires that were grown with 50 mM of precursor concentration for 4 h of growth time lowered the impedance from 835 ± 40 kΩ of planar Cr/Au MEAs to 540 ± 20 kΩ at a frequency of 1 kHz. In contrast, the ZnO nanowires that were grown with PEI for 35 h showed that despite the increased surface area by a factor of 45× the impedance was found to be quite high, 2.25 ± 0.2 MΩ at 1 kHz of frequency. The adsorption of PEI molecules to the lateral surfaces of the nanowires was thought to behave as a passivation layer that could have restricted the charge transfer characteristics of the ZnO-NW MEAs. Encapsulation of the pristine ZnO nanowires that were grown with standard precursor concentration of 25 mM for 4 h of growth time with different metallic layers (Cr/Au, Ti and Pt) further improved the electrical characteristics of the MEAs. The ZnO nanowires that were encapsulated with a 10 nm thin layer of Ti and Pt achieved the lowest electrochemical impedance of 400 ± 25 kΩ at 1 kHz in this work. The robustness of the Ti encapsulated ZnO nanowires were also improved in comparison to the PEI ZnO nanowires. The improved electrochemical characteristics and mechanical stability of the MEAs integrated with metal encapsulated ZnO nanowires have shown a great promise for improving the SNR of recording signals from neuronal cells for long term measurements. This work concludes that both pristine ZnO nanowire MEAs and metal encapsulated ZnO nanowire MEAs will be capable of recording AP signals from human neuronal networks at single-cell resolution. However, further optimisation and extensions of the work are required to record AP signals from human neuronal cells.</p>

Emergence and fragmentation of the alpha-band driven by neuronal network dynamics

Rhythmic neuronal network activity underlies brain oscillations. To investigate how connected neuronal networks contribute to the emergence of the α-band and to the regulation of Up and Down states, we study a model based on synaptic short-term depression-facilitation with afterhyperpolarization (AHP). We found that the α-band is generated by the network behavior near the attractor of the Up-state. Coupling inhibitory and excitatory networks by reciprocal connections leads to the emergence of a stable α-band during the Up states, as reflected in the spectrogram. To better characterize the emergence and stability of thalamocortical oscillations containing α and δ rhythms during anesthesia, we model the interaction of two excitatory networks with one inhibitory network, showing that this minimal topology underlies the generation of a persistent α-band in the neuronal voltage characterized by dominant Up over Down states. Finally, we show that the emergence of the α-band appears when external inputs are suppressed, while fragmentation occurs at small synaptic noise or with increasing inhibitory inputs. To conclude, α-oscillations could result from the synaptic dynamics of interacting excitatory neuronal networks with and without AHP, a principle that could apply to other rhythms.

Investigating Explanatory Factors of Machine Learning Models for Plant Classification

Recent progress in machine learning and deep learning has enabled the implementation of plant and crop detection using systematic inspection of the leaf shapes and other morphological characters for identification systems for precision farming. However, the models used for this approach tend to become black-box models, in the sense that it is difficult to trace characters that are the base for the classification. The interpretability is therefore limited and the explanatory factors may not be based on reasonable visible characters. We investigate the explanatory factors of recent machine learning and deep learning models for plant classification tasks. Based on a Daucus carota and a Beta vulgaris image data set, we implement plant classification models and compare those models by their predictive performance as well as explainability. For comparison we implemented a feed forward convolutional neuronal network as a default model. To evaluate the performance, we trained an unsupervised Bayesian Gaussian process latent variable model as well as a convolutional autoencoder for feature extraction and rely on a support vector machine for classification. The explanatory factors of all models were extracted and analyzed. The experiments show, that feed forward convolutional neuronal networks (98.24% and 96.10% mean accuracy) outperforms the Bayesian Gaussian process latent variable pipeline (92.08% and 94.31% mean accuracy) as well as the convolutional autoenceoder pipeline (92.38% and 93.28% mean accuracy) based approaches in terms of classification accuracy, even though not significant for Beta vulgaris images. Additionally, we found that the neuronal network used biological uninterpretable image regions for the plant classification task. In contrast to that, the unsupervised learning models rely on explainable visual characters. We conclude that supervised convolutional neuronal networks must be used carefully to ensure biological interpretability. We recommend unsupervised machine learning, careful feature investigation, and statistical feature analysis for biological applications.

Neuron particles capture network topology and behavior from single units

While networks of neurons, glia and vascular systems enable and support brain functions, to date, mathematical tools to decode network dynamics and structure from very scarce and partially observed neuronal spiking behavior remain underdeveloped. Large neuronal networks contribute to the intrinsic neuron transfer function and observed neuronal spike trains encoding complex causal information processing, yet how this emerging causal fractal memory in the spike trains relates to the network topology is not fully understood. Towards this end, we propose a novel statistical physics inspired neuron particle model that captures the causal information flow and processing features of neuronal spiking activity. Relying on synthetic comprehensive simulations and real-world neuronal spiking activity analysis, the proposed fractional order operators governing the neuronal spiking dynamics provide insights into the memory and scale of the spike trains as well as information about the topological properties of the underlying neuronal networks. Lastly, the proposed model exhibits superior predictions of animal behavior during multiple cognitive tasks.

Long-term labeling and imaging of synaptically-connected neuronal networks in vivo using nontoxic, double-deletion-mutant rabies viruses

SummaryThe highly specific and complex connectivity between neurons is the hallmark of nervous systems, but techniques for identifying, imaging, and manipulating synaptically-connected networks of neurons are limited. Monosynaptic tracing, or the gated replication and spread of a deletion-mutant rabies virus to label neurons directly connected to a targeted population of starting neurons1, is the most widely-used technique for mapping neural circuitry, but the rapid cytotoxicity of first-generation rabies viral vectors has restricted its use almost entirely to anatomical applications. We recently introduced double-deletion-mutant second-generation rabies viral vectors, showing that they have little or no detectable toxicity and are efficient means of retrogradely targeting neurons projecting to an injection site2, but they have not previously been shown to be capable of gated replication in vivo, the basis of monosynaptic tracing. Here we present a complete second-generation system for labeling direct inputs to genetically-targeted neuronal populations with minimal toxicity, using double-deletion-mutant rabies viruses. Spread of the viruses requires complementation of both of the deleted viral genes in trans in the starting postsynaptic cells; suppressing the expression of these viral genes following an initial period of viral replication, using the Tet-Off system, reduces toxicity to the starting cells without decreasing the efficiency of viral spread. Using longitudinal two- photon imaging of live monosynaptic tracing in visual cortex, we found that 94.4% of all labeled cells, and an estimated 92.3% of starting cells, survived for the full twelve-week course of imaging. Two-photon imaging of calcium responses in labeled networks of neurons in vivo over ten weeks showed that labeled neurons’ visual response properties remained stable for as long as we followed them. This nontoxic labeling of inputs to genetically-targeted neurons in vivo is a long-held goal in neuroscience, with transformative applications including nonperturbative transcriptomic and epigenomic profiling, long-term functional imaging and behavioral studies, and optogenetic and chemogenetic manipulation of synaptically-connected neuronal networks over the lifetimes of experimental animals.