Intelligent data analysis: issues and challenges

Two phenomena have probably affected modern data analysts' lives more than anything else. First, the size of real-world data sets is getting increasingly large, especially during the last decade or so. Second, modern computational methods and tools are being developed which add further capability to traditional statistical analysis tools. These two developments have created a new range of problems and challenges for analysts, as well as new opportunities for intelligent systems in data analysis.

Download Full-text

Information Exploration Shootout Project And Benchmark Data Sets: Evaluating How Visualization Does In Analyzing Real-World Data Analysis Problems

Proceedings. Visualization '97 (Cat. No. 97CB36155) ◽

10.1109/visual.1997.663933 ◽

2005 ◽

Author(s):

S. Laskowski ◽

G. Wills ◽

B. Rogowitz

Keyword(s):

Data Analysis ◽

Real World ◽

Data Sets ◽

Real World Data ◽

World Data ◽

Benchmark Data ◽

Information Exploration

Download Full-text

Telemonitoring of patients with COPD reduces mortality with neutral effect on total costs: evidence from a real world data analysis over three years

10.1055/s-0039-3403237 ◽

2020 ◽

Author(s):

F Hofer ◽

WM Franz ◽

J Schreyögg ◽

T Stargardt

Keyword(s):

Data Analysis ◽

Real World ◽

Real World Data ◽

World Data ◽

Neutral Effect

Download Full-text

Hfinger: Malware HTTP Request Fingerprinting

Entropy ◽

10.3390/e23050507 ◽

2021 ◽

Vol 23 (5) ◽

pp. 507

Author(s):

Piotr Białczak ◽

Wojciech Mazurczyk

Keyword(s):

Real World ◽

Network Traffic ◽

Experimental Evaluation ◽

Data Sets ◽

Real World Data ◽

Malicious Software ◽

Default Mode ◽

World Data ◽

Effectiveness Analysis ◽

Http Protocol

Malicious software utilizes HTTP protocol for communication purposes, creating network traffic that is hard to identify as it blends into the traffic generated by benign applications. To this aim, fingerprinting tools have been developed to help track and identify such traffic by providing a short representation of malicious HTTP requests. However, currently existing tools do not analyze all information included in the HTTP message or analyze it insufficiently. To address these issues, we propose Hfinger, a novel malware HTTP request fingerprinting tool. It extracts information from the parts of the request such as URI, protocol information, headers, and payload, providing a concise request representation that preserves the extracted information in a form interpretable by a human analyst. For the developed solution, we have performed an extensive experimental evaluation using real-world data sets and we also compared Hfinger with the most related and popular existing tools such as FATT, Mercury, and p0f. The conducted effectiveness analysis reveals that on average only 1.85% of requests fingerprinted by Hfinger collide between malware families, what is 8–34 times lower than existing tools. Moreover, unlike these tools, in default mode, Hfinger does not introduce collisions between malware and benign applications and achieves it by increasing the number of fingerprints by at most 3 times. As a result, Hfinger can effectively track and hunt malware by providing more unique fingerprints than other standard tools.

Download Full-text