Ti3C2Tx MXene Enabled All-Optical Nonlinear Activation Function for On-Chip Photonic Deep Neural Networks

Neural networks are one of the first major milestones in developing artificial intelligence systems. The utilisation of integrated photonics in neural networks offers a promising alternative approach to microelectronic and hybrid optical-electronic implementations due to improvements in computational speed and low energy consumption in machine-learning tasks. However, at present, most of the neural network hardware systems are still electronic-based due to a lack of optical realisation of the nonlinear activation function. Here, we experimentally demonstrate two novel approaches for implementing an all-optical neural nonlinear activation function based on utilising unique light-matter interactions in 2D Ti3C2Tx (MXene) in the infrared (IR) range in two configurations: 1) a saturable absorber made of MXene thin film, and 2) a silicon waveguide with MXene flakes overlayer. These configurations may serve as nonlinear units in photonic neural networks, while their nonlinear transfer function can be flexibly designed to optimise the performance of different neuromorphic tasks, depending on the operating wavelength. The proposed configurations are reconfigurable and can therefore be adjusted for various applications without the need to modify the physical structure. We confirm the capability and feasibility of the obtained results in machine-learning applications via an Modified National Institute of Standards and Technology (MNIST) handwritten digit classifications task, with near 99% accuracy. Our developed concept for an all-optical neuron is expected to constitute a major step towards the realization of all-optically implemented deep neural networks.

Random CNN Structure – Tool to Increase Generalization Ability in Deep Learning

The paper presents a novel approach to designing the CNN structure of improved generalization capability in the presence of a small population of learning data. In contrast to the classical methods for building CNN, we propose to introduce some randomness in the choice of layers with a different type of nonlinear activation function. Image processing in these layers is performed using either the ReLU or the softplus function. This choice is random. The randomness introduced into the network structure can be interpreted as a special form of regularization. Experiments performed in the recognition of images belonging to either melanoma or non-melanoma cases have shown a significant improvement in the average quality measures, such as the accuracy, sensitivity, precision, and the area under the ROC curve.

Differential Equation Units: Learning Functional Forms of Activation Functions from Data

Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential equation units (DEUs), an improvement to modern neural networks, which enables each neuron to learn a particular nonlinear activation function from a family of solutions to an ordinary differential equation. Specifically, each neuron may change its functional form during training based on the behavior of the other parts of the network. We show that using neurons with DEU activation functions results in a more compact network capable of achieving comparable, if not superior, performance when compared to much larger networks.