Design and Performance Comparison of Docker Container Based Deep Learning Model Management System for Real-Time Analysis

Author(s):  
Mo-se Lee ◽  
Min-su Kang ◽  
In-ho Kim ◽  
Jae-hun Kim
Author(s):  
Tossaporn Santad ◽  
Piyarat Silapasupphakornwong ◽  
Worawat Choensawat ◽  
Kingkarn Sookhanaphibarn

2014 ◽  
Vol 571-572 ◽  
pp. 497-501 ◽  
Author(s):  
Qi Lv ◽  
Wei Xie

Real-time log analysis on large scale data is important for applications. Specifically, real-time refers to UI latency within 100ms. Therefore, techniques which efficiently support real-time analysis over large log data sets are desired. MongoDB provides well query performance, aggregation frameworks, and distributed architecture which is suitable for real-time data query and massive log analysis. In this paper, a novel implementation approach for an event driven file log analyzer is presented, and performance comparison of query, scan and aggregation operations over MongoDB, HBase and MySQL is analyzed. Our experimental results show that HBase performs best balanced in all operations, while MongoDB provides less than 10ms query speed in some operations which is most suitable for real-time applications.


2021 ◽  
Author(s):  
Gaurav Chachra ◽  
Qingkai Kong ◽  
Jim Huang ◽  
Srujay Korlakunta ◽  
Jennifer Grannen ◽  
...  

Abstract After significant earthquakes, we can see images posted on social media platforms by individuals and media agencies owing to the mass usage of smartphones these days. These images can be utilized to provide information about the shaking damage in the earthquake region both to the public and research community, and potentially to guide rescue work. This paper presents an automated way to extract the damaged building images after earthquakes from social media platforms such as Twitter and thus identify the particular user posts containing such images. Using transfer learning and ~6500 manually labelled images, we trained a deep learning model to recognize images with damaged buildings in the scene. The trained model achieved good performance when tested on newly acquired images of earthquakes at different locations and ran in near real-time on Twitter feed after the 2020 M7.0 earthquake in Turkey. Furthermore, to better understand how the model makes decisions, we also implemented the Grad-CAM method to visualize the important locations on the images that facilitate the decision.


2021 ◽  
Author(s):  
Jannes Münchmeyer ◽  
Dino Bindi ◽  
Ulf Leser ◽  
Frederik Tilmann

<p><span>The estimation of earthquake source parameters, in particular magnitude and location, in real time is one of the key tasks for earthquake early warning and rapid response. In recent years, several publications introduced deep learning approaches for these fast assessment tasks. Deep learning is well suited for these tasks, as it can work directly on waveforms and </span><span>can</span><span> learn features and their relation from data.</span></p><p><span>A drawback of deep learning models is their lack of interpretability, i.e., it is usually unknown what reasoning the network uses. Due to this issue, it is also hard to estimate how the model will handle new data whose properties differ in some aspects from the training set, for example earthquakes in previously seismically quite regions. The discussions of previous studies usually focused on the average performance of models and did not consider this point in any detail.</span></p><p><span>Here we analyze a deep learning model for real time magnitude and location estimation through targeted experiments and a qualitative error analysis. We conduct our analysis on three large scale regional data sets from regions with diverse seismotectonic settings and network properties: Italy and Japan with dense networks </span><span>(station spacing down to 10 km)</span><span> of strong motion sensors, and North Chile with a sparser network </span><span>(station spacing around 40 km) </span><span>of broadband stations. </span></p><p><span>We obtained several key insights. First, the deep learning model does not seem to follow the classical approaches for magnitude and location estimation. For magnitude, one would classically expect the model to estimate attenuation, but the network rather seems to focus its attention on the spectral composition of the waveforms. For location, one would expect a triangulation approach, but our experiments instead show indications of a fingerprinting approach. </span>Second, we can pinpoint the effect of training data size on model performance. For example, a four times larger training set reduces average errors for both magnitude and location prediction by more than half, and reduces the required time for real time assessment by a factor of four. <span>Third, the model fails for events with few similar training examples. For magnitude, this means that the largest event</span><span>s</span><span> are systematically underestimated. For location, events in regions with few events in the training set tend to get mislocated to regions with more training events. </span><span>These characteristics can have severe consequences in downstream tasks like early warning and need to be taken into account for future model development and evaluation.</span></p>


2021 ◽  
pp. 132-143
Author(s):  
Akihiro Sugiura ◽  
Yoshiki Itazu ◽  
Kunihiko Tanaka ◽  
Hiroki Takada

Critical Care ◽  
2019 ◽  
Vol 23 (1) ◽  
Author(s):  
Soo Yeon Kim ◽  
Saehoon Kim ◽  
Joongbum Cho ◽  
Young Suh Kim ◽  
In Suk Sol ◽  
...  

Sign in / Sign up

Export Citation Format

Share Document