Distributed Triangle Approximately Counting Algorithms in Simple Graph Stream

Recently, the counting algorithm of local topology structures, such as triangles, has been widely used in social network analysis, recommendation systems, user portraits and other fields. At present, the problem of counting global and local triangles in a graph stream has been widely studied, and numerous triangle counting steaming algorithms have emerged. To improve the throughput and scalability of streaming algorithms, many researches of distributed streaming algorithms on multiple machines are studied. In this article, we first propose a framework of distributed streaming algorithm based on the Master-Worker-Aggregator architecture. The two core parts of this framework are an edge distribution strategy, which plays a key role to affect the performance, including the communication overhead and workload balance, and aggregation method, which is critical to obtain the unbiased estimations of the global and local triangle counts in a graph stream. Then, we extend the state-of-the-art centralized algorithm TRIÈST into four distributed algorithms under our framework. Compared to their competitors, experimental results show that DVHT-i is excellent in accuracy and speed, performing better than the best existing distributed streaming algorithm. DEHT-b is the fastest algorithm and has the least communication overhead. What’s more, it almost achieves absolute workload balance.

Adaptive Segmentation of Streaming Sensor Data on Edge Devices

Sensor data streams often represent signals/trajectories which are twice differentiable (e.g., to give a continuous velocity and acceleration), and this property must be reflected in their segmentation. An adaptive streaming algorithm for this problem is presented. It is based on the greedy look-ahead strategy and is built on the concept of a cubic splinelet. A characteristic feature of the proposed algorithm is the real-time simultaneous segmentation, smoothing, and compression of data streams. The segmentation quality is measured in terms of the signal approximation accuracy and the corresponding compression ratio. The numerical results show the relatively high compression ratios (from 135 to 208, i.e., compressed stream sizes up to 208 times smaller) combined with the approximation errors comparable to those obtained from the state-of-the-art global reference algorithm. The proposed algorithm can be applied to various domains, including online compression and/or smoothing of data streams coming from sensors, real-time IoT analytics, and embedded time-series databases.

A Single-pass Noise Covariance Estimation Algorithm in Adaptive Kalman Filtering for Non-stationary Systems

Estimation of unknown noise covariances in a Kalman filter is a problem of significant practical interest in a wide array of applications. This paper presents a single-pass stochastic gradient descent (SGD) algorithm for noise covariance estimation for use in adaptive Kalman filters applied to non-stationary systems where the noise covariances can occasionally jump up or down by an unknown magnitude. Unlike our previous batch method or our multi-pass decision-directed algorithm, the proposed streaming algorithm reads measurement data exactly once and has similar root mean square error (RMSE). The computational efficiency of the new algorithm stems from its one-pass nature, recursive fading memory estimation of the sample cross-correlations of the innovations, and the RMSprop accelerated SGD algorithm. The comparative evaluation of the proposed method on a number of test cases demonstrates its computational efficiency and accuracy.

A Single-pass Noise Covariance Estimation Algorithm in Adaptive Kalman Filtering for Non-stationary Systems

Estimation of unknown noise covariances in a Kalman filter is a problem of significant practical interest in a wide array of applications. This paper presents a single-pass stochastic gradient descent (SGD) algorithm for noise covariance estimation for use in adaptive Kalman filters applied to non-stationary systems where the noise covariances can occasionally jump up or down by an unknown magnitude. Unlike our previous batch method or our multi-pass decision-directed algorithm, the proposed streaming algorithm reads measurement data exactly once and has similar root mean square error (RMSE). The computational efficiency of the new algorithm stems from its one-pass nature, recursive fading memory estimation of the sample cross-correlations of the innovations, and the RMSprop accelerated SGD algorithm. The comparative evaluation of the proposed method on a number of test cases demonstrates its computational efficiency and accuracy.

CoCoS: Fast and Accurate Distributed Triangle Counting in Graph Streams

Given a graph stream, how can we estimate the number of triangles in it using multiple machines with limited storage? Specifically, how should edges be processed and sampled across the machines for rapid and accurate estimation? The count of triangles (i.e., cliques of size three) has proven useful in numerous applications, including anomaly detection, community detection, and link recommendation. For triangle counting in large and dynamic graphs, recent work has focused largely on streaming algorithms and distributed algorithms but little on their combinations for “the best of both worlds.” In this work, we propose CoCoS , a fast and accurate distributed streaming algorithm for estimating the counts of global triangles (i.e., all triangles) and local triangles incident to each node. Making one pass over the input stream, CoCoS carefully processes and stores the edges across multiple machines so that the redundant use of computational and storage resources is minimized. Compared to baselines, CoCoS is: (a) accurate: giving up to smaller estimation error; (b) fast : up to faster, scaling linearly with the size of the input stream; and (c) theoretically sound : yielding unbiased estimates.