A Multi-attention Collaborative Deep Learning Approach for Blood Pressure Prediction

We develop a deep learning model based on Long Short-term Memory (LSTM) to predict blood pressure based on a unique data set collected from physical examination centers capturing comprehensive multi-year physical examination and lab results. In the Multi-attention Collaborative Deep Learning model (MAC-LSTM) we developed for this type of data, we incorporate three types of attention to generate more explainable and accurate results. In addition, we leverage information from similar users to enhance the predictive power of the model due to the challenges with short examination history. Our model significantly reduces predictive errors compared to several state-of-the-art baseline models. Experimental results not only demonstrate our model’s superiority but also provide us with new insights about factors influencing blood pressure. Our data is collected in a natural setting instead of a setting designed specifically to study blood pressure, and the physical examination items used to predict blood pressure are common items included in regular physical examinations for all the users. Therefore, our blood pressure prediction results can be easily used in an alert system for patients and doctors to plan prevention or intervention. The same approach can be used to predict other health-related indexes such as BMI.

Using deep learning to detecting abnormal behavior in internet of things

<span lang="EN-US">The development of the internet of things (IoT) has increased exponentially, creating a rapid pace of changes and enabling it to become more and more embedded in daily life. This is often achieved through integration: IoT is being integrated into billions of intelligent objects, commonly labeled “things,” from which the service collects various forms of data regarding both these “things” themselves as well as their environment. While IoT and IoT-powered decices can provide invaluable services in various fields, unauthorized access and inadvertent modification are potential issues of tremendous concern. In this paper, we present a process for resolving such IoT issues using adapted long short-term memory (LSTM) recurrent neural networks (RNN). With this method, we utilize specialized deep learning (DL) methods to detect abnormal and/or suspect behavior in IoT systems. LSTM RNNs are adopted in order to construct a high-accuracy model capable of detecting suspicious behavior based on a dataset of IoT sensors readings. The model is evaluated using the Intel Labs dataset as a test domain, performing four different tests, and using three criteria: F1, Accuracy, and time. The results obtained here demonstrate that the LSTM RNN model we create is capable of detecting abnormal behavior in IoT systems with high accuracy.</span>

Improving Deep Learning based Automatic Speech Recognition for Gujarati

We present a novel approach for improving the performance of an End-to-End speech recognition system for the Gujarati language. We follow a deep learning-based approach that includes Convolutional Neural Network, Bi-directional Long Short Term Memory layers, Dense layers, and Connectionist Temporal Classification as a loss function. To improve the performance of the system with the limited size of the dataset, we present a combined language model (Word-level language Model and Character-level language model)-based prefix decoding technique and Bidirectional Encoder Representations from Transformers-based post-processing technique. To gain key insights from our Automatic Speech Recognition (ASR) system, we used the inferences from the system and proposed different analysis methods. These insights help us in understanding and improving the ASR system as well as provide intuition into the language used for the ASR system. We have trained the model on the Microsoft Speech Corpus, and we observe a 5.87% decrease in Word Error Rate (WER) with respect to base-model WER.

Secure Distributed Mobile Volunteer Computing with Android

Volunteer Computing provision of seamless connectivity that enables convenient and rapid deployment of greener and cheaper computing infrastructure is extremely promising to complement next-generation distributed computing systems. Undoubtedly, without tactile Internet and secure VC ecosystems, harnessing its full potentials and making it an alternative viable and reliable computing infrastructure is next to impossible. Android-enabled smart devices, applications, and services are inevitable for Volunteer computing. Contrarily, the progressive developments of sophisticated Android malware may reduce its exponential growth. Besides, Android malwares are considered the most potential and persistent cyber threat to mobile VC systems. To secure Android-based mobile volunteer computing, the authors proposed MulDroid, an efficient and self-learning autonomous hybrid (Long-Short-Term Memory, Convolutional Neural Network, Deep Neural Network) multi-vector Android malware threat detection framework. The proposed mechanism is highly scalable with well-coordinated infrastructure and self-optimizing capabilities to proficiently tackle fast-growing dynamic variants of sophisticated malware threats and attacks with 99.01% detection accuracy. For a comprehensive evaluation, the authors employed current state-of-the-art malware datasets (Android Malware Dataset, Androzoo) with standard performance evaluation metrics. Moreover, MulDroid is compared with our constructed contemporary hybrid DL-driven architectures and benchmark algorithms. Our proposed mechanism outperforms in terms of detection accuracy with a trivial tradeoff speed efficiency. Additionally, a 10-fold cross-validation is performed to explicitly show unbiased results.

Forecasting the Market with Machine Learning Algorithms: An Application of NMC-BERT-LSTM-DQN-X Algorithm in Quantitative Trading

Although machine learning (ML) algorithms have been widely used in forecasting the trend of stock market indices, they failed to consider the following crucial aspects for market forecasting: (1) that investors’ emotions and attitudes toward future market trends have material impacts on market trend forecasting (2) the length of past market data should be dynamically adjusted according to the market status and (3) the transition of market statutes should be considered when forecasting market trends. In this study, we proposed an innovative ML method to forecast China's stock market trends by addressing the three issues above. Specifically, sentimental factors (see Appendix [1] for full trans) were first collected to measure investors’ emotions and attitudes. Then, a non-stationary Markov chain (NMC) model was used to capture dynamic transitions of market statutes. We choose the state-of-the-art (SOTA) method, namely, Bidirectional Encoder Representations from Transformers ( BERT ), to predict the state of the market at time t , and a long short-term memory ( LSTM ) model was used to estimate the varying length of past market data in market trend prediction, where the input of LSTM (the state of the market at time t ) was the output of BERT and probabilities for opening and closing of the gates in the LSTM model were based on outputs of the NMC model. Finally, the optimum parameters of the proposed algorithm were calculated using a reinforced learning-based deep Q-Network. Compared to existing forecasting methods, the proposed algorithm achieves better results with a forecasting accuracy of 61.77%, annualized return of 29.25%, and maximum losses of −8.29%. Furthermore, the proposed model achieved the lowest forecasting error: mean square error (0.095), root mean square error (0.0739), mean absolute error (0.104), and mean absolute percent error (15.1%). As a result, the proposed market forecasting model can help investors obtain more accurate market forecast information.

Investigating the Feasibility of Deep Learning Methods for Urdu Word Sense Disambiguation

Word Sense Disambiguation (WSD), the process of automatically identifying the correct meaning of a word used in a given context, is a significant challenge in Natural Language Processing. A range of approaches to the problem has been explored by the research community. The majority of these efforts has focused on a relatively small set of languages, particularly English. Research on WSD for South Asian languages, particularly Urdu, is still in its infancy. In recent years, deep learning methods have proved to be extremely successful for a range of Natural Language Processing tasks. The main aim of this study is to apply, evaluate, and compare a range of deep learning methods approaches to Urdu WSD (both Lexical Sample and All-Words) including Simple Recurrent Neural Networks, Long-Short Term Memory, Gated Recurrent Units, Bidirectional Long-Short Term Memory, and Ensemble Learning. The evaluation was carried out on two benchmark corpora: (1) the ULS-WSD-18 corpus and (2) the UAW-WSD-18 corpus. Results (Accuracy = 63.25% and F1-Measure = 0.49) show that a deep learning approach outperforms previously reported results for the Urdu All-Words WSD task, whereas performance using deep learning approaches (Accuracy = 72.63% and F1-Measure = 0.60) are low in comparison to previously reported for the Urdu Lexical Sample task.

Compression of Deep Learning Models for Text: A Survey

In recent years, the fields of natural language processing (NLP) and information retrieval (IR) have made tremendous progress thanks to deep learning models like Recurrent Neural Networks (RNNs), Gated Recurrent Units (GRUs) and Long Short-Term Memory (LSTMs) networks, and Transformer [ 121 ] based models like Bidirectional Encoder Representations from Transformers (BERT) [ 24 ], Generative Pre-training Transformer (GPT-2) [ 95 ], Multi-task Deep Neural Network (MT-DNN) [ 74 ], Extra-Long Network (XLNet) [ 135 ], Text-to-text transfer transformer (T5) [ 96 ], T-NLG [ 99 ], and GShard [ 64 ]. But these models are humongous in size. On the other hand, real-world applications demand small model size, low response times, and low computational power wattage. In this survey, we discuss six different types of methods (Pruning, Quantization, Knowledge Distillation (KD), Parameter Sharing, Tensor Decomposition, and Sub-quadratic Transformer-based methods) for compression of such models to enable their deployment in real industry NLP projects. Given the critical need of building applications with efficient and small models, and the large amount of recently published work in this area, we believe that this survey organizes the plethora of work done by the “deep learning for NLP” community in the past few years and presents it as a coherent story.

Application of Artificial Intelligence Technology Optimized by Deep Learning to Rural Financial Development and Rural Governance

The aim of this article is to promote the development of rural finance and the further informatization of rural banks. Based on DL (deep learning) and artificial intelligence technology, data pre-processing and feature selection are conducted on the customer information of rural banks in a certain region, including the historical deposit and loan, transaction record, and credit information. Besides, four DL models are proposed with a precision of more than 87% by test to improve the simulation effect and explore the application of DL. The BLSTM-CNN (Bi-directional Long Short-Term Memory-Convolutional Neural Network) model with a precision of 95.8%, which integrates RNN (Recurrent Neural Network) and CNN (Convolutional Neural Network) in parallel, solves the shortcomings of RNN and CNN separately. The research result can provide a more reasonable prediction model for rural banks, and ideas for the development of rural informatization and promoting rural governance.