Efficient Foreground Extraction From HEVC Compressed Video for Application to Real-Time Analysis of Surveillance ‘Big’ Data

2015 ◽  
Vol 24 (11) ◽  
pp. 3574-3585 ◽  
Author(s):  
Bhaskar Dey ◽  
Malay K. Kundu
Keyword(s):  
Big Data ◽  
Real Time ◽  
Compressed Video ◽  
Time Analysis ◽  
Foreground Extraction ◽  
Real Time Analysis

scholarly journals A Distributed Stream Processing Middleware Framework for Real-Time Analysis of Heterogeneous Data on Big Data Platform: Case of Environmental Monitoring

Sensors ◽  
2020 ◽  
Vol 20 (11) ◽  
pp. 3166
Author(s):  
Adeyinka Akanbi ◽  
Muthoni Masinde
Keyword(s):  
Big Data ◽  
Environmental Monitoring ◽  
Real Time ◽  
Stream Processing ◽  
Heterogeneous Data ◽  
Legacy Systems ◽  
Time Analysis ◽  
Real Time Analysis ◽  
Distributed Stream Processing ◽  
Processing Engine

In recent years, the application and wide adoption of Internet of Things (IoT)-based technologies have increased the proliferation of monitoring systems, which has consequently exponentially increased the amounts of heterogeneous data generated. Processing and analysing the massive amount of data produced is cumbersome and gradually moving from classical ‘batch’ processing—extract, transform, load (ETL) technique to real-time processing. For instance, in environmental monitoring and management domain, time-series data and historical dataset are crucial for prediction models. However, the environmental monitoring domain still utilises legacy systems, which complicates the real-time analysis of the essential data, integration with big data platforms and reliance on batch processing. Herein, as a solution, a distributed stream processing middleware framework for real-time analysis of heterogeneous environmental monitoring and management data is presented and tested on a cluster using open source technologies in a big data environment. The system ingests datasets from legacy systems and sensor data from heterogeneous automated weather systems irrespective of the data types to Apache Kafka topics using Kafka Connect APIs for processing by the Kafka streaming processing engine. The stream processing engine executes the predictive numerical models and algorithms represented in event processing (EP) languages for real-time analysis of the data streams. To prove the feasibility of the proposed framework, we implemented the system using a case study scenario of drought prediction and forecasting based on the Effective Drought Index (EDI) model. Firstly, we transform the predictive model into a form that could be executed by the streaming engine for real-time computing. Secondly, the model is applied to the ingested data streams and datasets to predict drought through persistent querying of the infinite streams to detect anomalies. As a conclusion of this study, a performance evaluation of the distributed stream processing middleware infrastructure is calculated to determine the real-time effectiveness of the framework.

scholarly journals The Metamorphosis (of RAM3S)

2021 ◽  
Vol 11 (24) ◽  
pp. 11584
Author(s):  
Ilaria Bartolini ◽  
Marco Patella
Keyword(s):  
Big Data ◽  
Real Time ◽  
Data Streams ◽  
Multimedia Data ◽  
Data Streaming ◽  
Time Analysis ◽  
Real Time Analysis ◽  
Big Data Technologies ◽  
Big Data Streams ◽  
The One

The real-time analysis of Big Data streams is a terrific resource for transforming data into value. For this, Big Data technologies for smart processing of massive data streams are available, but the facilities they offer are often too raw to be effectively exploited by analysts. RAM3S (Real-time Analysis of Massive MultiMedia Streams) is a framework that acts as a middleware software layer between multimedia stream analysis techniques and Big Data streaming platforms, so as to facilitate the implementation of the former on top of the latter. RAM3S has been proven helpful in simplifying the deployment of non-parallel techniques to streaming platforms, such as Apache Storm or Apache Flink. In this paper, we show how RAM3S has been updated to incorporate novel stream processing platforms, such as Apache Samza, and to be able to communicate with different message brokers, such as Apache Kafka. Abstracting from the message broker also provides us with the ability to pipeline several RAM3S instances that can, therefore, perform different processing tasks. This represents a richer model for stream analysis with respect to the one already available in the original RAM3S version. The generality of this new RAM3S version is demonstrated through experiments conducted on three different multimedia applications, proving that RAM3S is a formidable asset for enabling efficient and effective Data Mining and Machine Learning on multimedia data streams.

scholarly journals Real-time analysis and predictability of the health functional food market using big data

2021 ◽  
Author(s):  
Sang-Soon Kim ◽  
Seokwon Lim ◽  
Sangoh Kim
Keyword(s):  
Big Data ◽  
Real Time ◽  
Functional Food ◽  
Health Food ◽  
Online Search ◽  
Time Analysis ◽  
Food Category ◽  
Real Time Analysis ◽  
Search Data ◽  
Food Market

AbstractThis study conducted a real-time analysis of the health functional food market using big data. To assess the scope of big data in market analysis, big data of the health food category were compared and analyzed with actual market data. Data were first collected using a program to obtain data, through application programming interfaces, followed by SPSS to compare and analyze the actual market index and shopping search word data. The correlation between the online search data and the actual market was high, indicating that online search data can be used to predict the trend of the actual market. Various types of data, such as items and major functional ingredients, can be collected and analyzed through the program developed for this study, which is also used to predict the market trend. The results demonstrate how APIs can be used to predict market size in the food industry effectively.

Sign in / Sign up

Export Citation Format

Share Document