Accelerating DNN Training Through Selective Localized Learning

Training Deep Neural Networks (DNNs) places immense compute requirements on the underlying hardware platforms, expending large amounts of time and energy. We propose LoCal+SGD, a new algorithmic approach to accelerate DNN training by selectively combining localized or Hebbian learning within a Stochastic Gradient Descent (SGD) based training framework. Back-propagation is a computationally expensive process that requires 2 Generalized Matrix Multiply (GEMM) operations to compute the error and weight gradients for each layer. We alleviate this by selectively updating some layers' weights using localized learning rules that require only 1 GEMM operation per layer. Further, since localized weight updates are performed during the forward pass itself, the layer activations for such layers do not need to be stored until the backward pass, resulting in a reduced memory footprint. Localized updates can substantially boost training speed, but need to be used judiciously in order to preserve accuracy and convergence. We address this challenge through a Learning Mode Selection Algorithm, which gradually selects and moves layers to localized learning as training progresses. Specifically, for each epoch, the algorithm identifies a Localized→SGD transition layer that delineates the network into two regions. Layers before the transition layer use localized updates, while the transition layer and later layers use gradient-based updates. We propose both static and dynamic approaches to the design of the learning mode selection algorithm. The static algorithm utilizes a pre-defined scheduler function to identify the position of the transition layer, while the dynamic algorithm analyzes the dynamics of the weight updates made to the transition layer to determine how the boundary between SGD and localized updates is shifted in future epochs. We also propose a low-cost weak supervision mechanism that controls the learning rate of localized updates based on the overall training loss. We applied LoCal+SGD to 8 image recognition CNNs (including ResNet50 and MobileNetV2) across 3 datasets (Cifar10, Cifar100, and ImageNet). Our measurements on an Nvidia GTX 1080Ti GPU demonstrate upto 1.5× improvement in end-to-end training time with ~0.5% loss in Top-1 classification accuracy.

Investigation on the corrosion resistance of laser cladding Fe-based alloy coating against molten zinc

In this paper, laser cladding technology was used to prepare a Fe-based coating on H13 steel substrate and its corrosion behavior in molten zinc was studied. The results show that laser-cladding Fe-based coating can effectively protect the substrate from the corrosion of molten zinc, which is mainly related to its microstructure. The typical microstructure of the coating is composed of α-(Fe, Cr) solid solution matrix and CrFeB eutectic phases continuously distribute around the matrix. When molten zinc contacts with the surface of the coating, it corrodes the α phase matrix preferentially and CrFeB eutectic phases with better corrosion resistance interweave with each other to form a three-dimensional skeletal structure, which can play the role of diffusion barrier and slow down the diffusion rate of liquid zinc. The corrosion by molten zinc leads to the formation of a transition layer and an outer corrosion layer above the coatings. With the prolongation of the corrosion time, a large number of micro cracks are generated inside the transition layer and fracture gradually occurs under the action of thermal stress. The partial spalling of the transition layer and the corrosion of α phase matrix occur at the same time, making the corrosion depth of the coating increase continuously. However, the dense corrosion layer above the coating and the dispersed boride fragments can still function as a barrier to the inward diffusion of molten zinc.

Fuzzy Adaptive Type II Controller for Two-Mass System

This paper presents original concepts of control systems for an electrical drive with an elastic mechanical coupling between the motor and the driven mechanism. The synthesis procedure of the speed controller uses a proposed quality index (cost function) of system operation ensures the minimization of both tracking errors and torsional vibrations. Proper selection of the cost function focusses more on the reduction of torsional vibrations due to their negative influence on the drive’s mechanical coupling vitality. The omission of the plant identification of an adaptive fuzzy controller was proposed. Two types of fuzzy controllers were analyzed, namely with type I and type II fuzzy membership functions. The novelty of the presented approach is in the application of a Petri transition layer in a type II fuzzy controller which reduces the numerical complexity in case of a large number of complicated type II fuzzy sets. The presented simulation and experimental results prove that the best dumping of mechanical vibrations ensures the adaptive fuzzy controller with type II functions and a Petri transition layer.

Microstructure and Characterization of Ti-Al Explosive Welding Composite Plate

In order to study the interface characteristics and microstructure formation of Ti-Al composite plate, explosive welding was carried out with TA2 titanium as the fly plate and 5083 aluminums as the base plate. Optical microscope and electron microscope were used to analyze the microstructure of intermetallic compounds. SPH method was used to simulate the welding process of composite plates. The formation conditions and initial defects of intermetallic compounds were analyzed. The results show that most of the melted metal in the wave-front stays in the wave-waist region, and there was a relative velocity difference between the vortex and the titanium tissue, which led to the existence of small pieces of fragmentation. The outer layer of the vortex had higher velocity than the inner layer. The formation of Ti3Al, its antioxidant capacity wound lead to the formation of cracks. The temperature of outer vortex was higher than that of inner vortex, and the vortex has a transition layer of 5 μm, which is thinner than the transition layer of 8 μm between cladding plate and substrate. The jet was mostly composed of aluminum metal, and the interface jet velocity reaches 3000 m·s-1 and the interface temperature reaches up to 2100 K. Compared with the molten metal in the wave-back vortex, the jet temperature at the interface was higher, resulting in a thicker transition layer at the bonding surface. The residual stress at the interface wound cause the density of the material to increase.

DIAGNOSIS OF SEMICONDUCTOR HETEROSYSTEMS USING THE PHOTOVOLTAIC METHOD

The diagnostic method is as follows: the lateral photo-EMF spectral characteristics are measured, generated in the structure (or device) when illuminated by wavelength light with a near the edge of the basic semiconductor layer. For illustrations of efficiency method the given part of the measurement results for Schottky contact samples with a nitrogen concentration of 5% and a thermal annealing temperature of 900 and 950°C. It has been found that a significant character and a small amplitude of such a characteristic indicates qualitative at a homogeneity and the necessary magnitude of the potential barrier (or barriers), that it is necessary to form to make Schottky contact or other structure. A significant characteristic and a small amplitude of such a characteristic indicates a qualitative one-line and the required value of a potential barrier (or barriers) that must be formed for the manufacture of a semiconductor structure or device. If the spectral characteristic has one maximum and amplitude that is many times higher than the amplitude of a significant characteristic, then this indicates a formed transition layer between components of heterosystems with high, compared with a quasine-power region of semiconductor, conductivity. The presence of such a layer increases the probability breaks down of the microelectronic device. Investigation of the distribution of lateral photours along the metal semiconductor interface compliant interpretation of spectral characteristics features. The linear significant form of distribution of EMF confirms the presence of a transition layer with a lower doping level compared with GaAs. An important feature of the diagnostic method is its non-destructive character, as well as the possibility of applying to semiconductor or devices based on them, in which the photovoltaic effect may occur.

FEATURES STUDIES OF TRANSITION LAYERS IN SEMICONDUCTOR HETEROSYSTEMS

Studies of spectral characteristics in Schottky's contact specimens showed that photemf generated by mono­chromatic light, according to the formation mechanism, has a predominantly lateral nature, that is, in a hetero­system there are at least two areas separated by a potential barrier along the interface, with different conductivity levels. The feature of the photoemf spectral characteristics is a variations its appearance when changing the thermal annealing temperature of the studied heterosystems. A significant characteristic and a small amplitude of the characteristic indicates the formation of a transition layer, relatively homogeneous and with insignificant, compared with the volume of GaAs, the doping concentration. If the spectral characteristic has one maximum and amplitude, which several times exceeds the amplitude of a significant characteristic, which means the formation of a transition layer in the Schottky contact depletion area with high conductivity, compared with a quasine-neutral region of a semiconductor. The distribution of lateral photoemf along the sample also has a significant character. In order to obtain the correct results relative to the heterosystem transition layer, it is necessary to measure spectral acute characteristics at a distance from the point change sign of the emf that several times the diffusion length of non-equilibrium charge carriers in GaAs. The problem of the formation of a metal-semiconductor contact and other heterosystems accompanied by the occurrence of heterogeneous transition layers, always paid attention. The use of the proposed photovoltaic method allows to establish the degree of homogeneity of semiconductor layers, components of the structure and predict the redistribution of current density flowing through the physical scope of the device.

The study of molecular composition in biomimetic interface of biocomposite/dentin

The aim of the study is the problem of formation of the biomimetic interface between the dental product and dentin of the human tooth as well as the investigations of molecular-chemical features in biointerface with the use of molecular multivariate IR-visualization. The data on synchrotron IR-mapping made enabled to differentiate the regions of sound dentin tissue and biomimetic transition layer and also to determine molecular groups responsible for the process of integration.