scholarly journals Composing High-level Stream Processing Pipelines

2020 ◽  
Author(s):  
Tanmaya Mahapatra

Abstract The growing number of Internet of Things (IoT) devices provide a massive pool of sensing data. However, turning data into actionable insights is not a trivial task, especially in the context of IoT, where application development itself is complex. The process entails working with heterogeneous devices via various communication protocols to co-ordinate and fetch datasets, followed by a series of data transformations. Graphical mashup tools, based on the principles of flow-based programming paradigm, operating at a higher-level of abstraction are in widespread use to support rapid prototyping of IoT applications. Nevertheless, the current state-of-the-art mashup tools suffer from several architectural limitations which prevent composing in-flow data analytics pipelines. In response to this, the paper contributes by (i) designing novel flow-based programming concepts based on the actor model to support data analytics pipelines in mashup tools, prototyping the ideas in a new mashup tool called aFlux and providing a detailed comparison with the existing state-of-the-art and (ii) enabling easy prototyping of streaming applications in mashup tools by abstracting the behavioural configurations of stream processing via graphical flows and validating the ease as well as the effectiveness of composing stream processing pipelines from an end-user perspective in a traffic simulation scenario.

2020 ◽  
Vol 7 (1) ◽  
Author(s):  
Tanmaya Mahapatra

Abstract The growing number of Internet of Things (IoT) devices provide a massive pool of sensing data. However, turning data into actionable insights is not a trivial task, especially in the context of IoT, where application development itself is complex. The process entails working with heterogeneous devices via various communication protocols to co-ordinate and fetch datasets, followed by a series of data transformations. Graphical mashup tools, based on the principles of flow-based programming paradigm, operating at a higher-level of abstraction are in widespread use to support rapid prototyping of IoT applications. Nevertheless, the current state-of-the-art mashup tools suffer from several architectural limitations which prevent composing in-flow data analytics pipelines. In response to this, the paper contributes by (i) designing novel flow-based programming concepts based on the actor model to support data analytics pipelines in mashup tools, prototyping the ideas in a new mashup tool called aFlux and providing a detailed comparison with the existing state-of-the-art and (ii) enabling easy prototyping of streaming applications in mashup tools by abstracting the behavioural configurations of stream processing via graphical flows and validating the ease as well as the effectiveness of composing stream processing pipelines from an end-user perspective in a traffic simulation scenario.


2020 ◽  
Author(s):  
Tanmaya Mahapatra

Abstract The growing number of Internet of Things (IoT) devices provide a massive pool of sensing data. However, turning data into actionable insights is not a trivial task, especially in the context of IoT, where application development itself is complex. The process entails working with heterogeneous devices via various communication protocols to co-ordinate and fetch datasets, followed by a series of data transformations. Graphical mashup tools, based on the principles of flow-based programming paradigm, operating at a higher-level of abstraction are in widespread use to support rapid prototyping of IoT applications. Nevertheless, the current state-of-the-art mashup tools suffer from several architectural limitations which prevent composing in-flow data analytics pipelines. In response to this, the paper contributes by (i) designing novel flow-based programming concepts based on the actor model to support data analytics pipelines in mashup tools, prototyping the ideas in a new mashup tool called aFlux and providing a detailed comparison with the existing state-of-the-art and (ii) enabling easy prototyping of streaming applications in mashup tools by abstracting the behavioural configurations of stream processing via graphical flows and validating the ease as well as the effectiveness of composing stream processing pipelines from an end-user perspective in a traffic simulation scenario.


Author(s):  
Qianli Xu ◽  
Ana Garcia Del Molino ◽  
Jie Lin ◽  
Fen Fang ◽  
Vigneshwaran Subbaraju ◽  
...  

Lifelog analytics is an emerging research area with technologies embracing the latest advances in machine learning, wearable computing, and data analytics. However, state-of-the-art technologies are still inadequate to distill voluminous multimodal lifelog data into high quality insights. In this article, we propose a novel semantic relevance mapping ( SRM ) method to tackle the problem of lifelog information access. We formulate lifelog image retrieval as a series of mapping processes where a semantic gap exists for relating basic semantic attributes with high-level query topics. The SRM serves both as a formalism to construct a trainable model to bridge the semantic gap and an algorithm to implement the training process on real-world lifelog data. Based on the SRM, we propose a computational framework of lifelog analytics to support various applications of lifelog information access, such as image retrieval, summarization, and insight visualization. Systematic evaluations are performed on three challenging benchmarking tasks to show the effectiveness of our method.


2021 ◽  
Vol 11 (15) ◽  
pp. 6975
Author(s):  
Tao Zhang ◽  
Lun He ◽  
Xudong Li ◽  
Guoqing Feng

Lipreading aims to recognize sentences being spoken by a talking face. In recent years, the lipreading method has achieved a high level of accuracy on large datasets and made breakthrough progress. However, lipreading is still far from being solved, and existing methods tend to have high error rates on the wild data and have the defects of disappearing training gradient and slow convergence. To overcome these problems, we proposed an efficient end-to-end sentence-level lipreading model, using an encoder based on a 3D convolutional network, ResNet50, Temporal Convolutional Network (TCN), and a CTC objective function as the decoder. More importantly, the proposed architecture incorporates TCN as a feature learner to decode feature. It can partly eliminate the defects of RNN (LSTM, GRU) gradient disappearance and insufficient performance, and this yields notable performance improvement as well as faster convergence. Experiments show that the training and convergence speed are 50% faster than the state-of-the-art method, and improved accuracy by 2.4% on the GRID dataset.


Author(s):  
yifan yang ◽  
Lorenz S Cederbaum

The low-lying electronic states of neutral X@C60(X=Li, Na, K, Rb) have been computed and analyzed by employing state-of-the-art high level many-electron methods. Apart from the common charge-separated states, well known...


2021 ◽  
Vol 13 (9) ◽  
pp. 4716
Author(s):  
Moustafa M. Nasralla

To develop sustainable rehabilitation systems, these should consider common problems on IoT devices such as low battery, connection issues and hardware damages. These should be able to rapidly detect any kind of problem incorporating the capacity of warning users about failures without interrupting rehabilitation services. A novel methodology is presented to guide the design and development of sustainable rehabilitation systems focusing on communication and networking among IoT devices in rehabilitation systems with virtual smart cities by using time series analysis for identifying malfunctioning IoT devices. This work is illustrated in a realistic rehabilitation simulation scenario in a virtual smart city using machine learning on time series for identifying and anticipating failures for supporting sustainability.


Sensors ◽  
2017 ◽  
Vol 17 (6) ◽  
pp. 1377 ◽  
Author(s):  
Sylvie Delepine-Lesoille ◽  
Sylvain Girard ◽  
Marcel Landolt ◽  
Johan Bertrand ◽  
Isabelle Planes ◽  
...  

2018 ◽  
Vol 2018 ◽  
pp. 1-11 ◽  
Author(s):  
Hai Wang ◽  
Lei Dai ◽  
Yingfeng Cai ◽  
Long Chen ◽  
Yong Zhang

Traditional salient object detection models are divided into several classes based on low-level features and contrast between pixels. In this paper, we propose a model based on a multilevel deep pyramid (MLDP), which involves fusing multiple features on different levels. Firstly, the MLDP uses the original image as the input for a VGG16 model to extract high-level features and form an initial saliency map. Next, the MLDP further extracts high-level features to form a saliency map based on a deep pyramid. Then, the MLDP obtains the salient map fused with superpixels by extracting low-level features. After that, the MLDP applies background noise filtering to the saliency map fused with superpixels in order to filter out the interference of background noise and form a saliency map based on the foreground. Lastly, the MLDP combines the saliency map fused with the superpixels with the saliency map based on the foreground, which results in the final saliency map. The MLDP is not limited to low-level features while it fuses multiple features and achieves good results when extracting salient targets. As can be seen in our experiment section, the MLDP is better than the other 7 state-of-the-art models across three different public saliency datasets. Therefore, the MLDP has superiority and wide applicability in extraction of salient targets.


Electronics ◽  
2020 ◽  
Vol 9 (12) ◽  
pp. 2118
Author(s):  
Gwang Hui Choi ◽  
Taehui Na

Recently, the leakage power consumption of Internet of Things (IoT) devices has become a main issue to be tackled, due to the fact that the scaling of process technology increases the leakage current in the IoT devices having limited battery capacity, resulting in the reduction of battery lifetime. The most effective method to extend the battery lifetime is to shut-off the device during standby mode. For this reason, spin-transfer-torque magnetic-tunnel-junction (STT-MTJ) based nonvolatile flip-flop (NVFF) is being considered as a strong candidate to store the computing data. Since there is a risk that the MTJ resistance may change during the read operation (i.e., the read disturbance problem), NVFF should consider the read disturbance problem to satisfy reliable data restoration. To date, several NVFFs have been proposed. Even though they satisfy the target restore yield of 4σ, most of them do not take the read disturbance into account. Furthermore, several recently proposed NVFFs which focus on the offset-cancellation technique to improve the restore yield have obvious limitation with decreasing the supply voltage (VDD), because the offset-cancellation technique uses switch operation in the critical path that can exacerbate the restore yield in the near/sub-threshold region. In this regard, this paper analyzes state-of-the-art STT-MTJ based NVFFs with respect to the voltage region and provides insight that a simple circuit having no offset-cancellation technique could achieve a better restore yield in the near/sub-threshold voltage region. Monte–Carlo HSPICE simulation results, using industry-compatible 28 nm model parameters, show that in case of VDD of 0.6 V, complex NVFF circuits having offset tolerance characteristic have a better restore yield, whereas in case of VDD of 0.4 V with sizing up strategy, a simple NVFF circuit having no offset tolerance characteristic has a better restore yield.


Sign in / Sign up

Export Citation Format

Share Document