Multiple-Features-Based Semisupervised Clustering DDoS Detection Method

DDoS attack stream from different agent host converged at victim host will become very large, which will lead to system halt or network congestion. Therefore, it is necessary to propose an effective method to detect the DDoS attack behavior from the massive data stream. In order to solve the problem that large numbers of labeled data are not provided in supervised learning method, and the relatively low detection accuracy and convergence speed of unsupervised k-means algorithm, this paper presents a semisupervised clustering detection method using multiple features. In this detection method, we firstly select three features according to the characteristics of DDoS attacks to form detection feature vector. Then, Multiple-Features-Based Constrained-K-Means (MF-CKM) algorithm is proposed based on semisupervised clustering. Finally, using MIT Laboratory Scenario (DDoS) 1.0 data set, we verify that the proposed method can improve the convergence speed and accuracy of the algorithm under the condition of using a small amount of labeled data sets.

Download Full-text

Entropy Based Features Distribution for Anti-DDoS Model in SDN

Sustainability ◽

10.3390/su13031522 ◽

2021 ◽

Vol 13 (3) ◽

pp. 1522

Author(s):

Raja Majid Ali Ujjan ◽

Zeeshan Pervez ◽

Keshav Dahal ◽

Wajahat Ali Khan ◽

Asad Masood Khattak ◽

...

Keyword(s):

Network Security ◽

False Positive ◽

Denial Of Service ◽

Network Services ◽

Detection Accuracy ◽

Data Sets ◽

Traffic Patterns ◽

Average Accuracy ◽

Ddos Detection ◽

Quantitative Results

In modern network infrastructure, Distributed Denial of Service (DDoS) attacks are considered as severe network security threats. For conventional network security tools it is extremely difficult to distinguish between the higher traffic volume of a DDoS attack and large number of legitimate users accessing a targeted network service or a resource. Although these attacks have been widely studied, there are few works which collect and analyse truly representative characteristics of DDoS traffic. The current research mostly focuses on DDoS detection and mitigation with predefined DDoS data-sets which are often hard to generalise for various network services and legitimate users’ traffic patterns. In order to deal with considerably large DDoS traffic flow in a Software Defined Networking (SDN), in this work we proposed a fast and an effective entropy-based DDoS detection. We deployed generalised entropy calculation by combining Shannon and Renyi entropy to identify distributed features of DDoS traffic—it also helped SDN controller to effectively deal with heavy malicious traffic. To lower down the network traffic overhead, we collected data-plane traffic with signature-based Snort detection. We then analysed the collected traffic for entropy-based features to improve the detection accuracy of deep learning models: Stacked Auto Encoder (SAE) and Convolutional Neural Network (CNN). This work also investigated the trade-off between SAE and CNN classifiers by using accuracy and false-positive results. Quantitative results demonstrated SAE achieved relatively higher detection accuracy of 94% with only 6% of false-positive alerts, whereas the CNN classifier achieved an average accuracy of 93%.

Download Full-text

Key Parts of Transmission Line Detection Using Improved YOLO v3

The International Arab Journal of Information Technology ◽

10.34028/iajit/18/6/1 ◽

2021 ◽

Vol 18 (6) ◽

Author(s):

Tu Renwei ◽

Zhu Zhongjie ◽

Bai Yongqiang ◽

Gao Ming ◽

Ge Zhifeng

Keyword(s):

Transmission Line ◽

Detection Accuracy ◽

Data Sets ◽

Feature Maps ◽

Data Set ◽

Detection Model ◽

The Neural Network ◽

Low Efficiency ◽

The One ◽

Detection Speed

Unmanned Aerial Vehicle (UAV) inspection has become one of main methods for current transmission line inspection, but there are still some shortcomings such as slow detection speed, low efficiency, and inability for low light environment. To address these issues, this paper proposes a deep learning detection model based on You Only Look Once (YOLO) v3. On the one hand, the neural network structure is simplified, that is the three feature maps of YOLO v3 are pruned into two to meet specific detection requirements. Meanwhile, the K-means++ clustering method is used to calculate the anchor value of the data set to improve the detection accuracy. On the other hand, 1000 sets of power tower and insulator data sets are collected, which are inverted and scaled to expand the data set, and are fully optimized by adding different illumination and viewing angles. The experimental results show that this model using improved YOLO v3 can effectively improve the detection accuracy by 6.0%, flops by 8.4%, and the detection speed by about 6.0%.

Download Full-text

Semi supervised inspection algorithm of automatic packaging curve based on deep learning

Journal of Computational Methods in Sciences and Engineering ◽

10.3233/jcm-215690 ◽

2021 ◽

pp. 1-10

Author(s):

Yong He

Keyword(s):

Deep Learning ◽

Supervised Learning ◽

Optimization Algorithm ◽

Posterior Probability ◽

Detection Method ◽

Detection System ◽

Experimental Results ◽

Detection Accuracy ◽

Data Set ◽

Packaging Process

The current automatic packaging process is complex, requires high professional knowledge, poor universality, and difficult to apply in multi-objective and complex background. In view of this problem, automatic packaging optimization algorithm has been widely paid attention to. However, the traditional automatic packaging detection accuracy is low, the practicability is poor. Therefore, a semi-supervised detection method of automatic packaging curve based on deep learning and semi-supervised learning is proposed. Deep learning is used to extract features and posterior probability to classify unlabeled data. KDD CUP99 data set was used to verify the accuracy of the algorithm. Experimental results show that this method can effectively improve the performance of automatic packaging curve semi-supervised detection system.

Download Full-text

Research on Network Security Application Based on Deep Learning

CONVERTER ◽

10.17762/converter.235 ◽

2021 ◽

pp. 598-605

Author(s):

Zhao Jianchao

Keyword(s):

Deep Learning ◽

Learning Algorithm ◽

Rapid Development ◽

Internet Security ◽

Detection Accuracy ◽

Data Sets ◽

Learning Technology ◽

Data Set ◽

Deep Learning Algorithm ◽

Internet Industry

Behind the rapid development of the Internet industry, Internet security has become a hidden danger. In recent years, the outstanding performance of deep learning in classification and behavior prediction based on massive data makes people begin to study how to use deep learning technology. Therefore, this paper attempts to apply deep learning to intrusion detection to learn and classify network attacks. Aiming at the nsl-kdd data set, this paper first uses the traditional classification methods and several different deep learning algorithms for learning classification. This paper deeply analyzes the correlation among data sets, algorithm characteristics and experimental classification results, and finds out the deep learning algorithm which is relatively good at. Then, a normalized coding algorithm is proposed. The experimental results show that the algorithm can improve the detection accuracy and reduce the false alarm rate.

Download Full-text

Missing Values Compensation in Duplicates Detection Using Hot Deck

10.21203/rs.3.rs-390519/v1 ◽

2021 ◽

Author(s):

Abdulrazzak Ali ◽

Nurul A. Emran ◽

Siti A. Asmai

Keyword(s):

Performance Improvement ◽

Incomplete Data ◽

Missing Values ◽

Detection Method ◽

Data Sets ◽

Data Set ◽

Removal Process ◽

Matching Process ◽

A Performance

Abstract Duplicate record is a known problem within the datasets especially within databases of huge volumes. The accuracy of duplicates detection determines the efficiency of the duplicates removal process. Unfortunately, the effort to detect duplicates becomes more challenging due to the presence of missing values within the records. This is because, during the clustering and matching process, missing values can cause records that are similar to be assigned in a wrong group, causing the duplicates left undetected. In this paper, we present how duplicates detection can be improved even though missing values are present within a data set using our Duplicates Detection within the Incomplete Data set (DDID) method. We hypothetically add the missing values to the key attributes of two datasets under study, using an arbitrary pattern to simulate both complete and incomplete data sets. We analyze the results to evaluate the performance of duplicates detection using the Hot Deck method to compensate for the missing values in the key attributes. We hypothesize that by using Hot Deck, there is a performance improvement in duplicates detection. The performance of the DDID is compared with an early duplicates detection method (called DuDe) in terms of its accuracy and speed. The findings of the experiment show that, even though the data sets are incomplete, DDID is capable to offer better accuracy and faster duplicates detection as compared to a benchmark method (called DuDe). The results of this study contribute to duplicates detection under incomplete data sets constraint.

Download Full-text

Immature Apple Detection Method Based on Improved Yolov3

ASP Transactions on Internet of Things ◽

10.52810/10.52810/tiot.2021.100028 ◽

2021 ◽

Vol 1 (1) ◽

pp. 9-13

Author(s):

Zhongqiang Huang ◽

Ping Zhang ◽

Ruigang Liu ◽

Dongxu Li

Keyword(s):

Decision Making ◽

Feasible Solution ◽

Detection Method ◽

Detection Accuracy ◽

Data Set ◽

Apple Industry ◽

Backbone Network ◽

Target Frame ◽

Detection Speed ◽

Detection Effect

The identification of immature apples is a key technical link to realize automatic real-time monitoring of orchards, expert decision-making, and realization of orchard output prediction. In the orchard scene, the reflection caused by light and the color of immature apples are highly similar to the leaves, especially the obscuration and overlap of fruits by leaves and branches, which brings great challenges to the detection of immature apples. This paper proposes an improved YOLOv3 detection method for immature apples in the orchard scene. Use CSPDarknet53 as the backbone network of the model, introduce the CIOU target frame regression mechanism, and combine with the Mosaic algorithm to improve the detection accuracy. For the data set with severely occluded fruits, the F1 and mAP of the immature apple recognition model proposed in this article are 0.652 and 0.675, respectively. The inference speed for a single 416×416 picture is 12 ms, the detection speed can reach 83 frames/s on 1080ti, and the inference speed is 8.6 ms. Therefore, for the severely occluded immature apple data set, the method proposed in this article has a significant detection effect, and provides a feasible solution for the automation and mechanization of the apple industry.

Download Full-text

Missing values compensation in duplicates detection using hot deck method

Journal Of Big Data ◽

10.1186/s40537-021-00502-1 ◽

2021 ◽

Vol 8 (1) ◽

Author(s):

Abdulrazzak Ali ◽

Nurul A. Emran ◽

Siti A. Asmai

Keyword(s):

Incomplete Data ◽

Missing Values ◽

Detection Method ◽

Detection Performance ◽

Data Sets ◽

Data Set ◽

Duplicate Detection ◽

Removal Process ◽

Matching Process ◽

Study Offer

AbstractDuplicate record is a common problem within data sets especially in huge volume databases. The accuracy of duplicate detection determines the efficiency of duplicate removal process. However, duplicate detection has become more challenging due to the presence of missing values within the records where during the clustering and matching process, missing values can cause records deemed similar to be inserted into the wrong group, hence, leading to undetected duplicates. In this paper, duplicate detection improvement was proposed despite the presence of missing values within a data set through Duplicate Detection within the Incomplete Data set (DDID) method. The missing values were hypothetically added to the key attributes of three data sets under study, using an arbitrary pattern to simulate both complete and incomplete data sets. The results were analyzed, then, the performance of duplicate detection was evaluated by using the Hot Deck method to compensate for the missing values in the key attributes. It was hypothesized that by using Hot Deck, duplicate detection performance would be improved. Furthermore, the DDID performance was compared to an early duplicate detection method namely DuDe, in terms of its accuracy and speed. The findings yielded that even though the data sets were incomplete, DDID was able to offer a better accuracy and faster duplicate detection as compared to DuDe. The results of this study offer insights into constraints of duplicate detection within incomplete data sets.

Download Full-text

Transforming Retinal Photographs to Entropy Images in Deep Learning to Improve Automated Detection for Diabetic Retinopathy

Journal of Ophthalmology ◽

10.1155/2018/2159702 ◽

2018 ◽

Vol 2018 ◽

pp. 1-6 ◽

Cited By ~ 20

Author(s):

Gen-Min Lin ◽

Mei-Juan Chen ◽

Chia-Hung Yeh ◽

Yu-Yang Lin ◽

Heng-Yu Kuo ◽

...

Keyword(s):

Diabetic Retinopathy ◽

Deep Learning ◽

Sensitivity And Specificity ◽

Block Size ◽

Detection Accuracy ◽

Data Sets ◽

Feature Maps ◽

Data Set ◽

Fundus Photographs ◽

Grade 3

Entropy images, representing the complexity of original fundus photographs, may strengthen the contrast between diabetic retinopathy (DR) lesions and unaffected areas. The aim of this study is to compare the detection performance for severe DR between original fundus photographs and entropy images by deep learning. A sample of 21,123 interpretable fundus photographs obtained from a publicly available data set was expanded to 33,000 images by rotating and flipping. All photographs were transformed into entropy images using block size 9 and downsized to a standard resolution of 100 × 100 pixels. The stages of DR are classified into 5 grades based on the International Clinical Diabetic Retinopathy Disease Severity Scale: Grade 0 (no DR), Grade 1 (mild nonproliferative DR), Grade 2 (moderate nonproliferative DR), Grade 3 (severe nonproliferative DR), and Grade 4 (proliferative DR). Of these 33,000 photographs, 30,000 images were randomly selected as the training set, and the remaining 3,000 images were used as the testing set. Both the original fundus photographs and the entropy images were used as the inputs of convolutional neural network (CNN), and the results of detecting referable DR (Grades 2–4) as the outputs from the two data sets were compared. The detection accuracy, sensitivity, and specificity of using the original fundus photographs data set were 81.80%, 68.36%, 89.87%, respectively, for the entropy images data set, and the figures significantly increased to 86.10%, 73.24%, and 93.81%, respectively (all p values <0.001). The entropy image quantifies the amount of information in the fundus photograph and efficiently accelerates the generating of feature maps in the CNN. The research results draw the conclusion that transformed entropy imaging of fundus photographs can increase the machinery detection accuracy, sensitivity, and specificity of referable DR for the deep learning-based system.

Download Full-text

Solder Joint Defect Detection in the Connectors Using Improved Faster-RCNN Algorithm

Applied Sciences ◽

10.3390/app11020576 ◽

2021 ◽

Vol 11 (2) ◽

pp. 576

Author(s):

Kaihua Zhang ◽

Haikuo Shen

Keyword(s):

Solder Joint ◽

Defect Detection ◽

Data Augmentation ◽

Detection Method ◽

Detection Algorithm ◽

Detection Methods ◽

Detection Accuracy ◽

Data Set ◽

Electronic Products ◽

Original Algorithm

The miniaturization and high integration of electronic products have higher and higher requirements for welding of internal components of electronic products. A welding quality detection method has always been one of the important research contents in the industry, among which, the research on solder joint defect detection of a connector has gradually attracted people’s attention with the development of image detection algorithm. The traditional solder joint detection method of connector adopts manual detection or automatic detection methods, which is inefficient and not safe enough. With the development of deep learning, the application of a deep convolutional neural network to target detection has become a research hotspot. In this paper, a data set of connector solder joint samples was made and the number of image samples was expanded to more than 3 times of the original by using data augmentation. Clustering generates anchor boxes and transfer learning with ResNet-101 were fused, so an improved faster region-based convolutional neural networks (Faster RCNN) algorithm was proposed. The experiment verified that the improved algorithm proposed in this paper had a great improvement in all aspects compared with the original algorithm. The average detection accuracy of this method can reach 94%, and the detection rate of some defects can even reach 100%, which can completely meet the industrial requirements.

Download Full-text

DDoS Detection Using a Cloud-Edge Collaboration Method Based on Entropy-Measuring SOM and KD-Tree in SDN

Security and Communication Networks ◽

10.1155/2021/5594468 ◽

2021 ◽

Vol 2021 ◽

pp. 1-16

Author(s):

Yuhua Xu ◽

Yunfeng Yu ◽

Hanshu Hong ◽

Zhixin Sun

Keyword(s):

Detection Method ◽

Detection Efficiency ◽

Detection Accuracy ◽

Self Organizing Maps ◽

Fine Grained ◽

Tree Detection ◽

Entropy Measurement ◽

Ddos Detection ◽

The Internet Of Things ◽

The Ideal

Software-defined networking (SDN) emerges as an innovative network paradigm, which separates the control plane from the data plane to improve the network programmability and flexibility. It is widely applied in the Internet of Things (IoT). However, SDN is vulnerable to DDoS attacks, which can cause network disasters. In order to protect SDN security, a DDoS detection method using cloud-edge collaboration based on Entropy-Measuring Self-organizing Maps and KD-tree (EMSOM-KD) is designed for SDN. Entropy measurement is utilized to select the ideal SOM map and classify SOM neurons considering the limitation of dead and suspicious neurons. EMSOM can detect most flows directly and filter out a few doubtable flows. Then these flows are fine-grained, identified by KD-tree. Due to the limited and precious resources of the controller, parameter computation is performed in the cloud. The edge controller implements DDoS detection by EMSOM-KD. The experiments are conducted to evaluate the performance of the proposed method. The results show that EMSOM-KD has better detection accuracy; moreover, it improves the KD-tree detection efficiency.

Download Full-text