scholarly journals Real Time Call Monitoring System Using Spark Streaming and Network Intrusion Detection Using Distributed WekaSpark

2017 ◽  
Vol 2 (1) ◽  
pp. 7-13
Author(s):  
Darshan V S ◽  
Ria Raphael
Keyword(s):  
Intrusion Detection ◽  
Real Time ◽  
Time Data ◽  
Real Time Processing ◽  
Network Failure ◽  
Network Intrusion ◽  
Call Usage ◽  
Call Monitoring ◽  
Made In

With the increase of calls in industries it is very difficult to identify the calls made in a huge organization. The study and developing analytics out of the call history generated in terms of real time or the information stored helps in the improvement of the quality of calls in terms of network failure analysis, analysing call usage pattern from minimal to maximum to increase server efficiency, analyse user level pattern. The capability to process, analyse and evaluate real time data in a system is a challenging task, the test of building up an adaptable, shortcoming tolerant and flexible observing framework that Can deal with information continuously and at a huge scale is nontrivial. We exhibit a novel framework for real time processing and batch processing by using spark streaming and spark, also an ensemble model is used with distributed weka-spark for intrusion detection.

scholarly journals A Survey on Data-Driven Learning for Intelligent Network Intrusion Detection Systems

Electronics ◽  
2022 ◽  
Vol 11 (2) ◽  
pp. 213
Author(s):  
Ghada Abdelmoumin ◽  
Jessica Whitaker ◽  
Danda B. Rawat ◽  
Abdul Rahman
Keyword(s):  
Intrusion Detection ◽  
Real Time ◽  
Data Augmentation ◽  
Synthetic Data ◽  
Skewed Distribution ◽  
Rapid Review ◽  
Time Data ◽  
Adversarial Learning ◽  
Network Intrusion ◽  
Real Time Data

An effective anomaly-based intelligent IDS (AN-Intel-IDS) must detect both known and unknown attacks. Hence, there is a need to train AN-Intel-IDS using dynamically generated, real-time data in an adversarial setting. Unfortunately, the public datasets available to train AN-Intel-IDS are ineluctably static, unrealistic, and prone to obsolescence. Further, the need to protect private data and conceal sensitive data features has limited data sharing, thus encouraging the use of synthetic data for training predictive and intrusion detection models. However, synthetic data can be unrealistic and potentially bias. On the other hand, real-time data are realistic and current; however, it is inherently imbalanced due to the uneven distribution of anomalous and non-anomalous examples. In general, non-anomalous or normal examples are more frequent than anomalous or attack examples, thus leading to skewed distribution. While imbalanced data are commonly predominant in intrusion detection applications, it can lead to inaccurate predictions and degraded performance. Furthermore, the lack of real-time data produces potentially biased models that are less effective in predicting unknown attacks. Therefore, training AN-Intel-IDS using imbalanced and adversarial learning is instrumental to their efficacy and high performance. This paper investigates imbalanced learning and adversarial learning for training AN-Intel-IDS using a qualitative study. It surveys and synthesizes generative-based data augmentation techniques for addressing the uneven data distribution and generative-based adversarial techniques for generating synthetic yet realistic data in an adversarial setting using rapid review, structured reporting, and subgroup analysis.

scholarly journals Deep Transfer Learning Based Intrusion Detection System for Electric Vehicular Networks

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4736
Author(s):  
Sk. Tanzir Mehedi ◽  
Adnan Anwar ◽  
Ziaur Rahman ◽  
Kawsar Ahmed
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Intrusion Detection ◽  
Real Time ◽  
Transfer Learning ◽  
Security Requirements ◽  
Detection Accuracy ◽  
Area Network ◽  
Complex Data ◽  
Network Intrusion

The Controller Area Network (CAN) bus works as an important protocol in the real-time In-Vehicle Network (IVN) systems for its simple, suitable, and robust architecture. The risk of IVN devices has still been insecure and vulnerable due to the complex data-intensive architectures which greatly increase the accessibility to unauthorized networks and the possibility of various types of cyberattacks. Therefore, the detection of cyberattacks in IVN devices has become a growing interest. With the rapid development of IVNs and evolving threat types, the traditional machine learning-based IDS has to update to cope with the security requirements of the current environment. Nowadays, the progression of deep learning, deep transfer learning, and its impactful outcome in several areas has guided as an effective solution for network intrusion detection. This manuscript proposes a deep transfer learning-based IDS model for IVN along with improved performance in comparison to several other existing models. The unique contributions include effective attribute selection which is best suited to identify malicious CAN messages and accurately detect the normal and abnormal activities, designing a deep transfer learning-based LeNet model, and evaluating considering real-world data. To this end, an extensive experimental performance evaluation has been conducted. The architecture along with empirical analyses shows that the proposed IDS greatly improves the detection accuracy over the mainstream machine learning, deep learning, and benchmark deep transfer learning models and has demonstrated better performance for real-time IVN security.

scholarly journals QoS Management for Embedded Databases in Multicore-Based Embedded Systems

2015 ◽  
Vol 2015 ◽  
pp. 1-14 ◽  
Author(s):  
Woochul Kang ◽  
Jaeyong Chung
Keyword(s):  
Embedded Systems ◽  
Real Time ◽  
Control Method ◽  
Network Connectivity ◽  
Time Data ◽  
Quality Of Data ◽  
Multiple Control ◽  
Data Intensive ◽  
Real Time Data

With ubiquitous deployment of sensors and network connectivity, amounts of real-time data for embedded systems are increasing rapidly and database capability is required for many embedded systems for systematic management of real-time data. In such embedded systems, supporting the timeliness of tasks accessing databases is an important problem. However, recent multicore-based embedded architectures pose a significant challenge for such data-intensive real-time tasks since the response time of accessing data can be significantly affected by potential intercore interferences. In this paper, we propose a novel feedback control scheme that supports the timeliness of data-intensive tasks against unpredictable intercore interferences. In particular, we use multiple inputs/multiple outputs (MIMO) control method that exploits multiple control knobs, for example, CPU frequency and the Quality-of-Data (QoD) to handle highly unpredictable workloads in multicore systems. Experimental results, using actual implementation, show that the proposed approach achieves the target Quality-of-Service (QoS) goals, such as task timeliness and Quality-of-Data (QoD) while consuming less energy compared to baseline approaches.

scholarly journals On the Quality of Real-Time Altimeter Gridded Fields: Comparison with In Situ Data

2009 ◽  
Vol 26 (3) ◽  
pp. 556-569 ◽  
Author(s):  
Ananda Pascual ◽  
Christine Boone ◽  
Gilles Larnicol ◽  
Pierre-Yves Le Traon
Keyword(s):  
Real Time ◽  
Sea Level ◽  
Tide Gauge ◽  
Deep Ocean ◽  
Velocity Estimation ◽  
Time Data ◽  
In Situ Data ◽  
Delayed Time

Abstract The timeliness of satellite altimeter measurements has a significant effect on their value for operational oceanography. In this paper, an Observing System Experiment (OSE) approach is used to assess the quality of real-time altimeter products, a key issue for robust monitoring and forecasting of the ocean state. In addition, the effect of two improved geophysical corrections and the number of missions that are combined in the altimeter products are also analyzed. The improved tidal and atmospheric corrections have a significant effect in coastal areas (0–100 km from the shore), and a comparison with tide gauge observations shows a slightly better agreement with the gridded delayed-time sea level anomalies (SLAs) with two altimeters [Jason-1 and European Remote Sensing Satellite-2 (ERS-2)/Envisat] using the new geophysical corrections (mean square differences in percent of tide gauge variance of 35.3%) than those with four missions [Jason-1, ERS/Envisat, Ocean Topography Experiment (TOPEX)/Poseidoninterlaced, and Geosat Follow-On] but using the old corrections (36.7%). In the deep ocean, however, the correction improvements have little influence. The performance of fast delivery products versus delayed-time data is compared using independent in situ data (tide gauge and drifter data). It clearly highlights the degradation of real-time SLA maps versus the delayed-time SLA maps: four altimeters are needed in real time to get the similar quality performance as two altimeters in delayed time (sea level error misfit around 36%, and zonal and meridional velocity estimation errors of 27% and 33%, respectively). This study proves that the continuous improvement of geophysical corrections is very important, and that it is essential to stay above a minimum threshold of four available altimetric missions to capture the main space and time oceanic scales in fast delivery products.

On Detecting Abnormal Access for Online Ideological and Political Education

2021 ◽  
Vol 12 (2) ◽  
pp. 35-45
Author(s):  
Yuzhu Yang
Keyword(s):  
Intrusion Detection ◽  
Online Education ◽  
Real Time ◽  
Data Stream ◽  
Detection Method ◽  
Ideological And Political Education ◽  
Network Intrusion Detection ◽  
Dynamic Changes ◽  
Network Intrusion ◽  
Education Platform

With the development and spread of networks, online education has become a new way in education. The online education platform encounters a large number of concurrent visiting, while the system must guarantee network security in the process of online education. The network visiting requests are real-time and dynamic in online education. In order to detect network intrusion and abnormal access in real time and adapt to the dynamic changes of network visiting requests, this paper adopts a data stream-based network intrusion detection method to monitor and manage online education visiting. First, a knowledge library is constructed by normal visiting mode and abnormal visiting mode. Second, the dissimilarity between data point and data cluster is used to measure the similarity between normal mode and abnormal mode. Lastly, the knowledge library is updated to reflect the changes of network in online education system by re-clustering. The proposed method is evaluated on a real dataset.

Traffic Controller for Handling Service Quality in Multimedia Network

2010 ◽  
pp. 96-112
Author(s):  
Manjunath Ramachandra ◽  
Vikas Jain
Keyword(s):  
Real Time ◽  
Long Range ◽  
Multimedia Data ◽  
Time Data ◽  
Self Similarity ◽  
Real Time Data ◽  
Data Transfers ◽  
Multimedia Network ◽  
Different Origin

The present day Internet traffic largely caters for the multimedia traffic throwing open new and unthinkable applications such as tele-surgery. The complexity of data transactions increases with a demand for in time and real time data transfers, demanding the limited resources of the network beyond their capabilities. It requires a prioritization of data transfers, controlled dumping of data over the network etc. To make the matter worse, the data from different origin combine together imparting long lasting detrimental features such as self similarity and long range dependency in to the traffic. The multimedia data fortunately is associated with redundancies that may be removed through efficient compression techniques. There exists a provision to control the compression or bitrates based on the availability of resources in the network. The traffic controller or shaper has to optimize the quality of the transferred multimedia data depending up on the state of the network. In this chapter, a novel traffic shaper is introduced considering the adverse properties of the network and counteract with the same.

scholarly journals Bifrost: A Python/C++ Framework for High-Throughput Stream Processing in Astronomy

2017 ◽  
Vol 06 (04) ◽  
pp. 1750007 ◽  
Author(s):  
Miles D. Cranmer ◽  
Benjamin R. Barsdell ◽  
Danny C. Price ◽  
Jayce Dowell ◽  
Hugh Garsden ◽  
...  
Keyword(s):  
Real Time ◽  
High Throughput ◽  
Radio Astronomy ◽  
Scientific Productivity ◽  
Data Transport ◽  
Time Data ◽  
Process Data ◽  
Real Time Processing ◽  
Real Time Data Processing ◽  
High Level

Radio astronomy observatories with high throughput back end instruments require real-time data processing. While computing hardware continues to advance rapidly, development of real-time processing pipelines remains difficult and time-consuming, which can limit scientific productivity. Motivated by this, we have developed Bifrost: an open-source software framework for rapid pipeline development. (a) Bifrost combines a high-level Python interface with highly efficient reconfigurable data transport and a library of computing blocks for CPU and GPU processing. The framework is generalizable, but initially it emphasizes the needs of high-throughput radio astronomy pipelines, such as the ability to process data buffers as if they were continuous streams, the capacity to partition processing into distinct data sequences (e.g. separate observations), and the ability to extract specific intervals from buffered data. Computing blocks in the library are designed for applications such as interferometry, pulsar dedispersion and timing, and transient search pipelines. We describe the design and implementation of the Bifrost framework and demonstrate its use as the backbone in the correlation and beamforming back end of the Long Wavelength Array (LWA) station in the Sevilleta National Wildlife Refuge, NM.

Sign in / Sign up

Export Citation Format

Share Document