When Massive GPU Parallelism Ain’t Enough: A Novel Hardware Architecture of 2D-LSTM Neural Network

Multidimensional Long Short-Term Memory (MD-LSTM) neural network is an extension of one-dimensional LSTM for data with more than one dimension. MD-LSTM achieves state-of-the-art results in various applications, including handwritten text recognition, medical imaging, and many more. However, its implementation suffers from the inherently sequential execution that tremendously slows down both training and inference compared to other neural networks. The main goal of the current research is to provide acceleration for inference of MD-LSTM. We advocate that Field-Programmable Gate Array (FPGA) is an alternative platform for deep learning that can offer a solution when the massive parallelism of GPUs does not provide the necessary performance required by the application. In this article, we present the first hardware architecture for MD-LSTM. We conduct a systematic exploration to analyze a tradeoff between precision and accuracy. We use a challenging dataset for semantic segmentation, namely historical document image binarization from the DIBCO 2017 contest and a well-known MNIST dataset for handwritten digit recognition. Based on our new architecture, we implement FPGA-based accelerators that outperform Nvidia Geforce RTX 2080 Ti with respect to throughput by up to 9.9 and Nvidia Jetson AGX Xavier with respect to energy efficiency by up to 48 . Our accelerators achieve higher throughput, energy efficiency, and resource efficiency than FPGA-based implementations of convolutional neural networks (CNNs) for semantic segmentation tasks. For the handwritten digit recognition task, our FPGA implementations provide higher accuracy and can be considered as a solution when accuracy is a priority. Furthermore, they outperform earlier FPGA implementations of one-dimensional LSTMs with respect to throughput, energy efficiency, and resource efficiency.

An Efficient Handwritten Digit Recognition based on Convolutional Neural Networks with Orthogonal Learning Strategies

The predictions of characters/text/digits from the handwritten images have made the research community spotlight towards recognition. There are enormous applications and ambiguity that made prediction possible with Deep Learning (DL) approaches. Primarily, there are four necessary steps to be carried out with handwriting prediction. First, consideration of a dataset that is more appropriate for DL validation an inefficient manner. Here, Special Database 1 and Special Database 2 are used, which are combined and modified by the National Institute of Standards and Technology (NIST). Next is pre-processing of input handwritten digit recognition data by data normalization, extraction of efficient features which provides better prediction accuracy. The proposed idea uses pixel values as features with the analysis of hyper-parameters to enhance near-human performance. With SVM, non-linear and linear models are built to extract the appropriate features for further processing. The features are separate and placed over the Bag of Features (BoF), which is used by the next processing stage. Finally, a novel Convolutional Neural Network (CNN) is by built modifying the network structure with Orthogonal Learning Particle Swarm Optimization (CNN-OLPSO). This modification is adopted for evolutionarily optimizing the number of hyper-parameters. This proposed optimizer predicts the optimal values from the fitness computation and shows better efficiency when compared to various other conventional approaches. The novelty which relies on CNN adoption is to endeavor a suitable path towards digitalization and preserve the handwritten structure and help automatic feature extraction using CNN by offering better computation accuracy. The optimization approach helps to avoid over-fitting and under-fitting issues. Here, metrics like accuracy, elapsed time, recall, precision, and [Formula: see text]-measure are evaluated. The results of CNN-OLPSO give better accuracy, reduced error rate and better execution time (s) compared to other existing methods. Thus, the proposed model shows better tradeoff in the recognition rate of handwritten digits.

Research on Mnist Handwritten Numbers Recognition based on CNN

In view of the increasing demand for handwritten digit recognition, a handwritten digit recognition model based on convolutional neural network is proposed. The model includes 1 input layer and 2 convolutional layers (5*5 convolution Core), 2 pooling layers (2*2 pooling core), 1 fully connected layer, 1 output layer, and use the mnist data set for model training and prediction. After a lot of training and participation, the accuracy rate of the training set was finally reached to 100%, and the accuracy rate of 99.25% was also achieved on the test set, which can meet the requirements of recognizing handwritten digits.

Handwritten Digit Recognition System

Recently, handwritten digit recognition has become impressively significant with the escalation of the Artificial Neural Networks (ANN). Apart from this, deep learning has brought a major turnaround in machine learning, which was the main reason it attracted many researchers. We can use it in many applications. The main aim of this article is to use the neural network approach for recognizing handwritten digits. The Convolution Neural Network has become the center of all deep learning strategies. Optical character recognition (OCR) is a part of image processing that leads to excerpting text from images. Recognizing handwritten digits is part of OCR. Recognizing the numbers is an important and remarkable subject. In this way, since the handwritten digits are not of same size, thickness, position, various difficulties are faced in determining the problem of recognizing handwritten digits. The unlikeness and structure of the compositional styles of many entities further influences the example and presence of the numbers. This is the strategy for perceiving and organizing the written characters. Its applications are such as programmed bank checks, health, post offices, for education, etc. In this article, to evaluate CNN's performance, we used the MNIST dataset, which contains 60,000 images of handwritten digits. Achieves 98.85% accuracy for handwritten digit. And where 10% of the total images were used to test the data set.