scholarly journals k-Nearest Neighbors by Means of Sequence to Sequence Deep Neural Networks and Memory Networks

2021 ◽  
Author(s):  
Yiming Xu ◽  
Diego Klabjan
Keyword(s):  
Neural Network ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Sampling Technique ◽  
Nearest Neighbors ◽  
Classification Models ◽  
Feed Forward Neural Network ◽  
K Nearest Neighbors ◽  
Out Of Sample ◽  
Memory Network

k-Nearest Neighbors is one of the most fundamental but effective classification models. In this paper, we propose two families of models built on a sequence to sequence model and a memory network model to mimic the k-Nearest Neighbors model, which generate a sequence of labels, a sequence of out-of-sample feature vectors and a final label for classification, and thus they could also function as oversamplers. We also propose 'out-of-core' versions of our models which assume that only a small portion of data can be loaded into memory. Computational experiments show that our models on structured datasets outperform k-Nearest Neighbors, a feed-forward neural network, XGBoost, lightGBM, random forest and a memory network, due to the fact that our models must produce additional output and not just the label. On image and text datasets, the performance of our model is close to many state-of-the-art deep models. As an oversampler on imbalanced datasets, the sequence to sequence kNN model often outperforms Synthetic Minority Over-sampling Technique and Adaptive Synthetic Sampling.

scholarly journals Part-of-Speech Tagging via Deep Neural Networks for Northern-Ethiopic Languages

2020 ◽  
Vol 49 (4) ◽  
pp. 482-494
Author(s):  
Jurgita Kapočiūtė-Dzikienė ◽  
Senait Gebremichael Tesfagergish
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Language Processing ◽  
Deep Neural Networks ◽  
Short Term Memory ◽  
Parameter Tuning ◽  
Feed Forward Neural Network ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Pos Tagger

Deep Neural Networks (DNNs) have proven to be especially successful in the area of Natural Language Processing (NLP) and Part-Of-Speech (POS) tagging—which is the process of mapping words to their corresponding POS labels depending on the context. Despite recent development of language technologies, low-resourced languages (such as an East African Tigrinya language), have received too little attention. We investigate the effectiveness of Deep Learning (DL) solutions for the low-resourced Tigrinya language of the Northern-Ethiopic branch. We have selected Tigrinya as the testbed example and have tested state-of-the-art DL approaches seeking to build the most accurate POS tagger. We have evaluated DNN classifiers (Feed Forward Neural Network – FFNN, Long Short-Term Memory method – LSTM, Bidirectional LSTM, and Convolutional Neural Network – CNN) on a top of neural word2vec word embeddings with a small training corpus known as Nagaoka Tigrinya Corpus. To determine the best DNN classifier type, its architecture and hyper-parameter set both manual and automatic hyper-parameter tuning has been performed. BiLSTM method was proved to be the most suitable for our solving task: it achieved the highest accuracy equal to 92% that is 65% above the random baseline.

The influence of knowledge governance and boundary-spanning search on innovation performance

2020 ◽  
Vol 34 (29) ◽  
pp. 2050326
Author(s):  
Ning Cao ◽  
Jianjun Wang
Keyword(s):  
Neural Network ◽  
Detrimental Effect ◽  
Nearest Neighbors ◽  
Boundary Spanning ◽  
Support Vector ◽  
K Nearest Neighbors ◽  
Machine Learning Methods ◽  
High Knowledge ◽  
Knowledge Governance ◽  
Exploratory Innovation

The realization of exploratory innovation is a complex and nonlinear evolutionary problem. Existing works point out that it is closely related with knowledge governance and boundary-spanning search. However, the intricate relationship among them still lacks exact quantitative explanations. Motivated by this, using four machine learning methods, namely, linear regression (LR), neural network (NN), support vector machine (SVM) and k-nearest neighbors (KNN), we explore how boundary-spanning search combined with knowledge governance influences innovation. Results show that SVM has the highest values of both stability and goodness of fitting. The SVM results show that the combination of low knowledge governance and high boundary-spanning search boosts innovation most efficiently, while high knowledge governance combined with low boundary-spanning search caused the most detrimental effect on innovation. Our results reveal enhancing boundary-spanning search is essential and beneficial to innovation.

scholarly journals Multiple Source Localization in a Shallow Water Waveguide Exploiting Subarray Beamforming and Deep Neural Networks

Sensors ◽  
2019 ◽  
Vol 19 (21) ◽  
pp. 4768 ◽  
Author(s):  
Zhaoqiong Huang ◽  
Ji Xu ◽  
Zaixiao Gong ◽  
Haibin Wang ◽  
Yonghong Yan
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Shallow Water ◽  
Source Localization ◽  
Deep Neural Networks ◽  
Short Term Memory ◽  
Direction Finding ◽  
Multiple Source ◽  
Feed Forward Neural Network ◽  
Subarray Beamforming

Deep neural networks (DNNs) have been shown to be effective for single sound source localization in shallow water environments. However, multiple source localization is a more challenging task because of the interactions among multiple acoustic signals. This paper proposes a framework for multiple source localization on underwater horizontal arrays using deep neural networks. The two-stage DNNs are adopted to determine both the directions and ranges of multiple sources successively. A feed-forward neural network is trained for direction finding, while the long short term memory recurrent neural network is used for source ranging. Particularly, in the source ranging stage, we perform subarray beamforming to extract features of sources that are detected by the direction finding stage, because subarray beamforming can enhance the mixed signal to the desired direction while preserving the horizontal-longitudinal correlations of the acoustic field. In this way, a universal model trained in the single-source scenario can be applied to multi-source scenarios with arbitrary numbers of sources. Both simulations and experiments in a range-independent shallow water environment of SWellEx-96 Event S5 are given to demonstrate the effectiveness of the proposed method.

scholarly journals Analisis Perbandingan Algoritma SVM, KNN, dan CNN untuk Klasifikasi Citra Cuaca

2021 ◽  
Vol 8 (2) ◽  
pp. 311
Author(s):  
Mohammad Farid Naufal
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Computer Vision ◽  
Support Vector Machine ◽  
Convolutional Neural Network ◽  
Cross Validation ◽  
Nearest Neighbors ◽  
Support Vector ◽  
Classification Algorithms ◽  
K Nearest Neighbors

<p class="Abstrak">Cuaca merupakan faktor penting yang dipertimbangkan untuk berbagai pengambilan keputusan. Klasifikasi cuaca manual oleh manusia membutuhkan waktu yang lama dan inkonsistensi. <em>Computer vision</em> adalah cabang ilmu yang digunakan komputer untuk mengenali atau melakukan klasifikasi citra. Hal ini dapat membantu pengembangan <em>self autonomous machine</em> agar tidak bergantung pada koneksi internet dan dapat melakukan kalkulasi sendiri secara <em>real time</em>. Terdapat beberapa algoritma klasifikasi citra populer yaitu K-Nearest Neighbors (KNN), Support Vector Machine (SVM), dan Convolutional Neural Network (CNN). KNN dan SVM merupakan algoritma klasifikasi dari <em>Machine Learning</em> sedangkan CNN merupakan algoritma klasifikasi dari Deep Neural Network. Penelitian ini bertujuan untuk membandingkan performa dari tiga algoritma tersebut sehingga diketahui berapa gap performa diantara ketiganya. Arsitektur uji coba yang dilakukan adalah menggunakan 5 cross validation. Beberapa parameter digunakan untuk mengkonfigurasikan algoritma KNN, SVM, dan CNN. Dari hasil uji coba yang dilakukan CNN memiliki performa terbaik dengan akurasi 0.942, precision 0.943, recall 0.942, dan F1 Score 0.942.</p><p class="Abstrak"> </p><p class="Abstrak"><em><strong>Abstract</strong></em></p><p class="Abstract"><em>Weather is an important factor that is considered for various decision making. Manual weather classification by humans is time consuming and inconsistent. Computer vision is a branch of science that computers use to recognize or classify images. This can help develop self-autonomous machines so that they are not dependent on an internet connection and can perform their own calculations in real time. There are several popular image classification algorithms, namely K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Convolutional Neural Network (CNN). KNN and SVM are Machine Learning classification algorithms, while CNN is a Deep Neural Networks classification algorithm. This study aims to compare the performance of that three algorithms so that the performance gap between the three is known. The test architecture is using 5 cross validation. Several parameters are used to configure the KNN, SVM, and CNN algorithms. From the test results conducted by CNN, it has the best performance with 0.942 accuracy, 0.943 precision, 0.942 recall, and F1 Score 0.942.</em></p><p class="Abstrak"><em><strong><br /></strong></em></p>

scholarly journals Data Mining Techniques for Identification and Classification of Various Diseases in Plants

2019 ◽  
Vol 9 (2S) ◽  
pp. 676-680
Keyword(s):  
Neural Network ◽  
Data Mining ◽  
Nearest Neighbors ◽  
Crop Productivity ◽  
Vital Role ◽  
Support Vector ◽  
Data Sets ◽  
K Nearest Neighbors ◽  
Data Mining Techniques

Data mining is currently being used in various applications; In research community it plays a vital role. This paper specify about data mining techniques for the preprocessing and classification of various disease in plants. Since various plants has different diseases based on that each of them has different data sets and different objectives for knowledge discovery. Data Mining Techniques applied on plants that it helps in segmentation and classification of diseased plants, it avoids Oral Inspection and helps to increase in crop productivity. This paper provides various classification techniques Such as K-Nearest Neighbors, Support Vector Machine, Principle component Analysis, Neural Network. Thus among various techniques neural network is effective for disease detection in plants.

scholarly journals An Efficient and Fast Model Reduced Kernel KNN for Human Activity Recognition

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Zongying Liu ◽  
Shaoxi Li ◽  
Jiangling Hao ◽  
Jingfeng Hu ◽  
Mingyang Pan
Keyword(s):  
Neural Network ◽  
Activity Recognition ◽  
Human Activity ◽  
Processing Time ◽  
Kernel Method ◽  
Nearest Neighbors ◽  
Human Activity Recognition ◽  
Support Vector ◽  
K Nearest Neighbors ◽  
Proposed Model

With accumulation of data and development of artificial intelligence, human activity recognition attracts lots of attention from researchers. Many classic machine learning algorithms, such as artificial neural network, feed forward neural network, K-nearest neighbors, and support vector machine, achieve good performance for detecting human activity. However, these algorithms have their own limitations and their prediction accuracy still has space to improve. In this study, we focus on K-nearest neighbors (KNN) and solve its limitations. Firstly, kernel method is employed in model KNN, which transforms the input features to be the high-dimensional features. The proposed model KNN with kernel (K-KNN) improves the accuracy of classification. Secondly, a novel reduced kernel method is proposed and used in model K-KNN, which is named as Reduced Kernel KNN (RK-KNN). It reduces the processing time and enhances the classification performance. Moreover, this study proposes an approach of defining number of K neighbors, which reduces the parameter dependency problem. Based on the experimental works, the proposed RK-KNN obtains the best performance in benchmarks and human activity datasets compared with other models. It has super classification ability in human activity recognition. The accuracy of human activity data is 91.60% for HAPT and 92.67% for Smartphone, respectively. Averagely, compared with the conventional KNN, the proposed model RK-KNN increases the accuracy by 1.82% and decreases standard deviation by 0.27. The small gap of processing time between KNN and RK-KNN in all datasets is only 1.26 seconds.

scholarly journals Interpolation Consistency Training for Semi-supervised Learning

2019 ◽  
Author(s):  
Vikas Verma ◽  
Alex Lamb ◽  
Juho Kannala ◽  
Yoshua Bengio ◽  
David Lopez-Paz
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Supervised Learning ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Data Distribution ◽  
Network Architectures ◽  
Low Density ◽  
Decision Boundary ◽  
Classification Problems

We introduce Interpolation Consistency Training (ICT), a simple and computation efficient algorithm for training Deep Neural Networks in the semi-supervised learning paradigm. ICT encourages the prediction at an interpolation of unlabeled points to be consistent with the interpolation of the predictions at those points. In classification problems, ICT moves the decision boundary to low-density regions of the data distribution. Our experiments show that ICT achieves state-of-the-art performance when applied to standard neural network architectures on the CIFAR-10 and SVHN benchmark dataset.

scholarly journals Analisis Perbandingan Algoritma Klasifikasi Citra Chest X-ray Untuk Deteksi Covid-19

Teknika ◽  
2021 ◽  
Vol 10 (2) ◽  
pp. 96-103
Author(s):  
Mohammad Farid Naufal ◽  
Selvia Ferdiana Kusuma ◽  
Kevin Christian Tanus ◽  
Raynaldy Valentino Sukiwun ◽  
Joseph Kristiano ◽  
...  
Keyword(s):  
Neural Network ◽  
Support Vector Machine ◽  
Cross Validation ◽  
Nearest Neighbor ◽  
Nearest Neighbors ◽  
Support Vector ◽  
K Nearest Neighbor ◽  
K Nearest Neighbors ◽  
X Ray ◽  
Chest X Ray

Kondisi pandemi global Covid-19 yang muncul diakhir tahun 2019 telah menjadi permasalahan utama seluruh negara di dunia. Covid-19 merupakan virus yang menyerang organ paru-paru dan dapat mengakibatkan kematian. Pasien Covid-19 banyak yang telah dirawat di rumah sakit sehingga terdapat data citra chest X-ray paru-paru pasien yang terjangkit Covid-19. Saat ini sudah banyak peneltian yang melakukan klasifikasi citra chest X-ray menggunakan Convolutional Neural Network (CNN) untuk membedakan paru-paru sehat, terinfeksi covid-19, dan penyakit paru-paru lainnya, namun belum ada penelitian yang mencoba membandingkan performa algoritma CNN dan machine learning klasik seperti Support Vector Machine (SVM), dan K-Nearest Neighbor (KNN) untuk mengetahui gap performa dan waktu eksekusi yang dibutuhkan. Penelitian ini bertujuan untuk membandingkan performa dan waktu eksekusi algoritma klasifikasi K-Nearest Neighbors (KNN), Support Vector Machine (SVM), dan CNN  untuk mendeteksi Covid-19 berdasarkan citra chest X-Ray. Berdasarkan hasil pengujian menggunakan 5 Cross Validation, CNN merupakan algoritma yang memiliki rata-rata performa terbaik yaitu akurasi 0,9591, precision 0,9592, recall 0,9591, dan F1 Score 0,959 dengan waktu eksekusi rata-rata sebesar 3102,562 detik.

scholarly journals Modular Dynamic Neural Network: A Continual Learning Architecture

2021 ◽  
Vol 11 (24) ◽  
pp. 12078
Author(s):  
Daniel Turner ◽  
Pedro J. S. Cardoso ◽  
João M. F. Rodrigues
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Feature Extraction ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Simple Task ◽  
Dynamic Neural Network ◽  
Main Components ◽  
Over Time ◽  
Continual Learning

Learning to recognize a new object after having learned to recognize other objects may be a simple task for a human, but not for machines. The present go-to approaches for teaching a machine to recognize a set of objects are based on the use of deep neural networks (DNN). So, intuitively, the solution for teaching new objects on the fly to a machine should be DNN. The problem is that the trained DNN weights used to classify the initial set of objects are extremely fragile, meaning that any change to those weights can severely damage the capacity to perform the initial recognitions; this phenomenon is known as catastrophic forgetting (CF). This paper presents a new (DNN) continual learning (CL) architecture that can deal with CF, the modular dynamic neural network (MDNN). The presented architecture consists of two main components: (a) the ResNet50-based feature extraction component as the backbone; and (b) the modular dynamic classification component, which consists of multiple sub-networks and progressively builds itself up in a tree-like structure that rearranges itself as it learns over time in such a way that each sub-network can function independently. The main contribution of the paper is a new architecture that is strongly based on its modular dynamic training feature. This modular structure allows for new classes to be added while only altering specific sub-networks in such a way that previously known classes are not forgotten. Tests on the CORe50 dataset showed results above the state of the art for CL architectures.

Sign in / Sign up

Export Citation Format

Share Document