Dynamic redirection of real-time data streams for elastic stream computing

2020 ◽  
Vol 112 ◽  
pp. 193-208
Author(s):  
Dawei Sun ◽  
Shang Gao ◽  
Xunyun Liu ◽  
Xindong You ◽  
Rajkumar Buyya
Keyword(s):  
Real Time ◽  
Data Streams ◽  
Stream Computing ◽  
Time Data ◽  
Real Time Data

Performance Improvement IoT Applications Through Multimedia Analytics Using Big Data Stream Computing Platforms

2018 ◽  
pp. 200-221
Author(s):  
Rizwan Patan ◽  
Rajasekhara Babu M ◽  
Suresh Kallam
Keyword(s):  
Big Data ◽  
Real Time ◽  
Performance Improvement ◽  
Data Stream ◽  
Real Data ◽  
Stream Computing ◽  
Time Data ◽  
Real Time Data ◽  
Computing Platforms ◽  
Time And Energy

A Big Data Stream Computing (BDSC) Platform handles real-time data from various applications such as risk management, marketing management and business intelligence. Now a days Internet of Things (IoT) deployment is increasing massively in all the areas. These IoTs engender real-time data for analysis. Existing BDSC is inefficient to handle Real-data stream from IoTs because the data stream from IoTs is unstructured and has inconstant velocity. So, it is challenging to handle such real-time data stream. This work proposes a framework that handles real-time data stream through device control techniques to improve the performance. The frame work includes three layers. First layer deals with Big Data platforms that handles real data streams based on area of importance. Second layer is performance layer which deals with performance issues such as low response time, and energy efficiency. The third layer is meant for Applying developed method on existing BDSC platform. The experimental results have been shown a performance improvement 20%-30% for real time data stream from IoT application.

scholarly journals Applying Formal Methods to Modeling and Analysis of Real-time Data Streams

2011 ◽  
Vol 5 (1) ◽  
pp. 85-110 ◽  
Author(s):  
Krasimira Kapitanova ◽  
Yuan Wei ◽  
Woo-Chul Kang ◽  
Sang-H. Son
Keyword(s):  
Real Time ◽  
Formal Methods ◽  
Data Streams ◽  
Time Data ◽  
Modeling And Analysis ◽  
Real Time Data

In the land of data streams where synopses are missing, one framework to bring them all

2021 ◽  
Vol 14 (10) ◽  
pp. 1818-1831
Author(s):  
Rudi Poepsel-Lemaitre ◽  
Martin Kiefer ◽  
Joscha von Hein ◽  
Jorge-Arnulfo Quiané-Ruiz ◽  
Volker Markl
Keyword(s):  
Data Analysis ◽  
Real Time ◽  
Data Streams ◽  
High Performance ◽  
Stream Processing ◽  
Divide And Conquer ◽  
Time Data ◽  
Real Time Data ◽  
Internal Processing ◽  
Aggregate Functions

In pursuit of real-time data analysis, approximate summarization structures, i.e., synopses, have gained importance over the years. However, existing stream processing systems, such as Flink, Spark, and Storm, do not support synopses as first class citizens, i.e., as pipeline operators. Synopses' implementation is upon users. This is mainly because of the diversity of synopses, which makes a unified implementation difficult. We present Condor, a framework that supports synopses as first class citizens. Condor facilitates the specification and processing of synopsis-based streaming jobs while hiding all internal processing details. Condor's key component is its model that represents synopses as a particular case of windowed aggregate functions. An inherent divide and conquer strategy allows Condor to efficiently distribute the computation, allowing for high-performance and linear scalability. Our evaluation shows that Condor outperforms existing approaches by up to a factor of 75x and that it scales linearly with the number of cores.

Design of Network Analysis System Based on Stream Computing

2017 ◽  
Vol 14 (1) ◽  
pp. 64-68 ◽  
Author(s):  
Peng Shi ◽  
Li Li
Keyword(s):  
Network Analysis ◽  
Load Balancing ◽  
Data Processing ◽  
Real Time ◽  
Network Data ◽  
Stream Computing ◽  
Time Data ◽  
Real Time System ◽  
Real Time Data ◽  
Analysis System

The functions of the network analysis system include detection and analysis of network data stream. According to the results of the network analysis, we monitor the network accident and avoid the security risks. This can improve the network performance and increase the network availability. As the data flow in the network is constantly produced, the biggest characteristic of network analysis system is that it is a real-time system. Because of the high requirements of the network data analysis and network fault processing, the system requires very high processing efficiency of the real time data of network. Stream computing is a technique specifically for processing real-time data streams. Its idea is that the value of the data is reduced with the lapse of time, so as long as the data appearing, it must be processed as soon as possible. So we use the technology of stream computing to design network analysis system to meet the needs of real-time capability. Moreover, the stream computing framework has been widely welcomed in the field because of its good expansibility, ease of use and flexibility. In this paper, firstly, we introduce the characteristics of the data processing based on stream computing and the traditional data processing separately. We point out their difference and introduce the technique of stream computing. Then, we introduce the architecture of network analysis system designed base on the technique of stream computing. The architecture includes two main components that are logic processing layer and communication layer. We describe the characteristics of each component and functional characteristics in detail, and we introduce the system load balancing algorithm. Finally, by experiments, we verify the effectiveness of the system’s characteristics of dynamic expansion and load balancing.

Visual Analysis of Dynamic Data Streams

2009 ◽  
Vol 8 (3) ◽  
pp. 212-229 ◽  
Author(s):  
George Chin ◽  
Mudita Singhal ◽  
Grant Nakamura ◽  
Vidhya Gurumoorthi ◽  
Natalie Freeman-Cadoret
Keyword(s):  
Real Time ◽  
Data Streams ◽  
High Volume ◽  
Scientific Data ◽  
General Strategy ◽  
Dynamic Visualization ◽  
Time Data ◽  
Dynamic Data ◽  
Intensive Training ◽  
Real Time Data

For scientific data visualizations, real-time data streams present many interesting challenges when compared to static data. Real-time data are dynamic, transient, high-volume and temporal. Effective visualizations need to be able to accommodate dynamic data behavior as well as Abstract and present the data in ways that make sense to and are usable by humans. The Visual Content Analysis of Real-Time Data Streams project at the Pacific Northwest National Laboratory is researching and prototyping dynamic visualization techniques and tools to help facilitate human understanding and comprehension of high-volume, real-time data. The general strategy of the project is to develop and evolve visual contexts that will organize and orient high-volume dynamic data in conceptual and perceptive views. The goal is to allow users to quickly grasp dynamic data in forms that are intuitive and natural without requiring intensive training in the use of specific visualization or analysis tools and methods. Thus far, the project has prototyped five different visualization prototypes that represent and convey dynamic data through human-recognizable contexts and paradigms such as hierarchies, relationships, time and geography. We describe the design considerations and unique features of these dynamic visualization prototypes as well as our findings in the exploration and evaluation of their use.

Sign in / Sign up

Export Citation Format

Share Document