Compressing RNNs to Kilobyte Budget for IoT Devices Using Kronecker Products

2021 ◽  
Vol 17 (4) ◽  
pp. 1-18
Author(s):  
Urmish Thakker ◽  
Igor Fedorov ◽  
Chu Zhou ◽  
Dibakar Gope ◽  
Matthew Mattina ◽  
...  
Keyword(s):  
Internet Of Things ◽  
Time Series Data ◽  
Machine Learning Algorithms ◽  
Series Data ◽  
Model Parameters ◽  
Resource Constrained ◽  
Fine Grained ◽  
Constrained Environments ◽  
Task Accuracy ◽  
Iot Devices

Micro-controllers (MCUs) make up most of the processors in the world with widespread applicability from automobile to medical devices. The Internet of Things promises to enable these resource-constrained MCUs with machine learning algorithms to provide always-on intelligence. Many Internet of Things applications consume time-series data that are naturally suitable for recurrent neural networks (RNNs) like LSTMs and GRUs. However, RNNs can be large and difficult to deploy on these devices, as they have few kilobytes of memory. As a result, there is a need for compression techniques that can significantly compress RNNs without negatively impacting task accuracy. This article introduces a method to compress RNNs for resource-constrained environments using the Kronecker product (KP). KPs can compress RNN layers by 16× to 38× with minimal accuracy loss. By quantizing the resulting models to 8 bits, we further push the compression factor to 50×. We compare KP with other state-of-the-art compression techniques across seven benchmarks spanning five different applications and show that KP can beat the task accuracy achieved by other techniques by a large margin while simultaneously improving the inference runtime. Sometimes the KP compression mechanism can introduce an accuracy loss. We develop a hybrid KP approach to mitigate this. Our hybrid KP algorithm provides fine-grained control over the compression ratio, enabling us to regain accuracy lost during compression by adding a small number of model parameters.

Efficient End-to-End Asynchronous Time-Series Modeling With Deep Learning to Predict Customer Attrition

2021 ◽  
pp. 268-281
Author(s):  
Victor Potapenko ◽  
Malek Adjouadi ◽  
Naphtali Rishe
Keyword(s):  
Time Series ◽  
Time Series Data ◽  
Machine Learning Algorithms ◽  
Series Data ◽  
Gradient Boosting ◽  
Feature Engineering ◽  
Resource Requirements ◽  
Constrained Environments ◽  
Multi Class Classification ◽  
Customer Attrition

Modeling time-series data with asynchronous, multi-cardinal, and uneven patterns presents several unique challenges that may impede convergence of supervised machine learning algorithms, or significantly increase resource requirements, thus rendering modeling efforts infeasible in resource-constrained environments. The authors propose two approaches to multi-class classification of asynchronous time-series data. In the first approach, they create a baseline by reducing the time-series data using a statistical approach and training a model based on gradient boosted trees. In the second approach, they implement a fully convolutional network (FCN) and train it on asynchronous data without any special feature engineering. Evaluation of results shows that FCN performs as well as the gradient boosting based on mean F1-score without computationally complex time-series feature engineering. This work has been applied in the prediction of customer attrition at a large retail automotive finance company.

scholarly journals Weather Prediction for Tourism Application using Time Series Algorithms

2020 ◽  
Vol 9 (11) ◽  
pp. 406-412
Keyword(s):  
Time Series ◽  
Time Series Data ◽  
Weather Forecasting ◽  
Human Life ◽  
Weather Prediction ◽  
Machine Learning Algorithms ◽  
Weather Data ◽  
Series Data ◽  
Observation Data ◽  
Fine Grained

Precise projections of future events are crucial in many areas, one of which is the tourism sector. Usually counter-trials and towns spend a enormous quantity of cash in planning and preparation to accommodate (and benefit) visitors. Precisely predicting the amount of visits in the days or months, that follow would benefit the economy and tourists both. Previous studies in this field investigate predictions for a nation as a whole rather than for fine-grained fields within a nation. Weather forecasting has drawn the attention of many scientists from distinct research communities due to its impact on human life globally. The developing deep learning methods coupled with the wide accessibility of huge weather observation data and the advancement of machine learning algorithms has motivated many scientists to investigate hidden hierarchical patterns for weather forecasting in large amounts of weather data over the previous century. To predict climate information accurately, heavy statistical algorithms are used on the big quantity of historical information. Time series Analysis enables us know the fundamental forces leading to a specific trend in time series data points and enables us to predict and monitor information points by fitting suitable models into them. In this study, Holt-Winter model is used for predicting time series. The forecasting algorithm for Holt-Winters enables users to construct a time series and then use that data to forecast interest areas. Exponential smoothing allocates weights and their respective values against past data to decrease exponentially, to decrease the weight value for older data.

scholarly journals State-of-the-Art Software-Based Remote Attestation: Opportunities and Open Issues for Internet of Things

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1598
Author(s):  
Sigurd Frej Joel Jørgensen Ankergård ◽  
Edlira Dushku ◽  
Nicola Dragoni
Keyword(s):  
Internet Of Things ◽  
Smart Cities ◽  
Cyber Attacks ◽  
Resource Constrained ◽  
Remote Attestation ◽  
Comprehensive Overview ◽  
Research Issues ◽  
Advantages And Disadvantages ◽  
Iot Devices ◽  
Specialized Hardware

The Internet of Things (IoT) ecosystem comprises billions of heterogeneous Internet-connected devices which are revolutionizing many domains, such as healthcare, transportation, smart cities, to mention only a few. Along with the unprecedented new opportunities, the IoT revolution is creating an enormous attack surface for potential sophisticated cyber attacks. In this context, Remote Attestation (RA) has gained wide interest as an important security technique to remotely detect adversarial presence and assure the legitimate state of an IoT device. While many RA approaches proposed in the literature make different assumptions regarding the architecture of IoT devices and adversary capabilities, most typical RA schemes rely on minimal Root of Trust by leveraging hardware that guarantees code and memory isolation. However, the presence of a specialized hardware is not always a realistic assumption, for instance, in the context of legacy IoT devices and resource-constrained IoT devices. In this paper, we survey and analyze existing software-based RA schemes (i.e., RA schemes not relying on specialized hardware components) through the lens of IoT. In particular, we provide a comprehensive overview of their design characteristics and security capabilities, analyzing their advantages and disadvantages. Finally, we discuss the opportunities that these RA schemes bring in attesting legacy and resource-constrained IoT devices, along with open research issues.

scholarly journals Implementation of IoT Framework with Data Analysis Using Deep Learning Methods for Occupancy Prediction in a Building

2021 ◽  
Vol 13 (3) ◽  
pp. 67
Author(s):  
Eric Hitimana ◽  
Gaurav Bajpai ◽  
Richard Musabe ◽  
Louis Sibomana ◽  
Jayavel Kayalvizhi
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Time Series Data ◽  
Multivariate Time Series ◽  
Machine Learning Algorithms ◽  
Series Data ◽  
Support Vector ◽  
Human Beings ◽  
Feed Forward Network

Many countries worldwide face challenges in controlling building incidence prevention measures for fire disasters. The most critical issues are the localization, identification, detection of the room occupant. Internet of Things (IoT) along with machine learning proved the increase of the smartness of the building by providing real-time data acquisition using sensors and actuators for prediction mechanisms. This paper proposes the implementation of an IoT framework to capture indoor environmental parameters for occupancy multivariate time-series data. The application of the Long Short Term Memory (LSTM) Deep Learning algorithm is used to infer the knowledge of the presence of human beings. An experiment is conducted in an office room using multivariate time-series as predictors in the regression forecasting problem. The results obtained demonstrate that with the developed system it is possible to obtain, process, and store environmental information. The information collected was applied to the LSTM algorithm and compared with other machine learning algorithms. The compared algorithms are Support Vector Machine, Naïve Bayes Network, and Multilayer Perceptron Feed-Forward Network. The outcomes based on the parametric calibrations demonstrate that LSTM performs better in the context of the proposed application.

Prediction and Analysis of Gold Prices using Ensemble Machine Learning Algorithms

2021 ◽  
Vol 9 (VI) ◽  
pp. 4367-4374
Author(s):  
Gudipally Chandrashakar
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Time Series Data ◽  
Gold Price ◽  
Machine Learning Algorithms ◽  
Series Data ◽  
Gradient Boosting ◽  
Support Vector ◽  
Average Value ◽  
Ensemble Machine Learning

In this article, we used historical time series data up to the current day gold price. In this study of predicting gold price, we consider few correlating factors like silver price, copper price, standard, and poor’s 500 value, dollar-rupee exchange rate, Dow Jones Industrial Average Value. Considering the prices of every correlating factor and gold price data where dates ranging from 2008 January to 2021 February. Few algorithms of machine learning are used to analyze the time-series data are Random Forest Regression, Support Vector Regressor, Linear Regressor, ExtraTrees Regressor and Gradient boosting Regression. While seeing the results the Extra Tree Regressor algorithm gives the predicted value of gold prices more accurately.

scholarly journals Risk Monitoring and Quantitative Results of Various Attributes of Machine Learning Algorithms with a Time Series Data

2019 ◽  
Vol 8 (11) ◽  
pp. 4018-4022
Keyword(s):  
Machine Learning ◽  
Time Series Data ◽  
Learning Algorithm ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Series Data ◽  
Machine Learning Algorithm ◽  
Risk Modelling ◽  
Risk Monitoring ◽  
Quantitative Results

The aim of this research is to do risk modelling after analysis of twitter posts based on certain sentiment analysis. In this research we analyze posts of several users or a particular user to check whether they can be cause of concern to the society or not. Every sentiment like happy, sad, anger and other emotions are going to provide scaling of severity in the conclusion of final table on which machine learning algorithm is applied. The data which is put under the machine learning algorithms are been monitored over a period of time and it is related to a particular topic in an area

scholarly journals Comparative Performance of Machine Learning Algorithms for Cryptocurrency Forecasting

2018 ◽  
Vol 11 (3) ◽  
pp. 1121 ◽  
Author(s):  
Nor Azizah Hitam ◽  
Amelia Ritahani Ismail
Keyword(s):  
Machine Learning ◽  
Time Series Data ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Series Data ◽  
Support Vector ◽  
Small Range ◽  
Accuracy Rate ◽  
Comparative Performance ◽  
Vector Machines

Machine Learning is part of Artificial Intelligence that has the ability to make future forecastings based on the previous experience. Methods has been proposed to construct models including machine learning algorithms such as Neural Networks (NN), Support Vector Machines (SVM) and Deep Learning. This paper presents a comparative performance of Machine Learning algorithms for cryptocurrency forecasting. Specifically, this paper concentrates on forecasting of time series data. SVM has several advantages over the other models in forecasting, and previous research revealed that SVM provides a result that is almost or close to actual result yet also improve the accuracy of the result itself. However, recent research has showed that due to small range of samples and data manipulation by inadequate evidence and professional analyzers, overall status and accuracy rate of the forecasting needs to be improved in further studies. Thus, advanced research on the accuracy rate of the forecasted price has to be done.

scholarly journals Early Season Mapping of Sugarcane by Applying Machine Learning Algorithms to Sentinel-1A/2 Time Series Data: A Case Study in Zhanjiang City, China

2019 ◽  
Vol 11 (7) ◽  
pp. 861 ◽  
Author(s):  
Hao Jiang ◽  
Dan Li ◽  
Wenlong Jing ◽  
Jianhui Xu ◽  
Jianxi Huang ◽  
...  
Keyword(s):  
Time Series ◽  
South China ◽  
Time Series Data ◽  
Growing Season ◽  
Machine Learning Algorithms ◽  
Series Data ◽  
Early Season ◽  
Cloudy Weather ◽  
Sar Imagery ◽  
Harvest Year

More than 90% of the sugar production in China comes from sugarcane, which is widely grown in South China. Optical image time series have proven to be efficient for sugarcane mapping. There are, however, two limitations associated with previous research: one is that the critical observations during the sugarcane growing season are limited due to frequent cloudy weather in South China; the other is that the classification method requires imagery time series covering the entire growing season, which reduces the time efficiency. The Sentinel-1A (S1A) synthetic aperture radar (SAR) data featuring relatively high spatial-temporal resolution provides an ideal data source for all-weather observations. In this study, we attempted to develop a method for the early season mapping of sugarcane. First, we proposed a framework consisting of two procedures: initial sugarcane mapping using the S1A SAR imagery time series, followed by non-vegetation removal using Sentinel-2 optical imagery. Second, we tested the framework using an incremental classification strategy based on S1A imagery covering the entire 2017–2018 sugarcane season. The study area was in Suixi and Leizhou counties of Zhanjiang city, China. Results indicated that an acceptable accuracy, in terms of Kappa coefficient, can be achieved to a level above 0.902 using time series three months before sugarcane harvest. In general, sugarcane mapping utilizing the combination of VH + VV as well as VH polarization alone outperformed mapping using VV alone. Although the XGBoost classifier with VH + VV polarization achieved a maximum accuracy that was slightly lower than the random forest (RF) classifier, the XGBoost shows promising performance in that it was more robust to overfitting with noisy VV time series and the computation speed was 7.7 times faster than RF classifier. The total sugarcane areas in Suixi and Leizhou for the 2017–2018 harvest year estimated by this study were approximately 598.95 km2 and 497.65 km2, respectively. The relative accuracy of the total sugarcane mapping area was approximately 86.3%.

Sign in / Sign up

Export Citation Format

Share Document