scholarly journals Automated Real-Time Classification and Decision Making in Massive Data Streams from Synoptic Sky Surveys

2014 ◽  
Author(s):  
S. George Djorgovski ◽  
Ashish Mahabal ◽  
Ciro Donalek ◽  
Matthew Graham ◽  
Andrew Drake ◽  
...  
Keyword(s):  
Decision Making ◽  
Real Time ◽  
Data Streams ◽  
Massive Data ◽  
Sky Surveys ◽  
Real Time Classification

scholarly journals Real-time data mining of massive data streams from synoptic sky surveys

2016 ◽  
Vol 59 ◽  
pp. 95-104 ◽  
Author(s):  
S.G. Djorgovski ◽  
M.J. Graham ◽  
C. Donalek ◽  
A.A. Mahabal ◽  
A.J. Drake ◽  
...  
Keyword(s):  
Data Mining ◽  
Real Time ◽  
Data Streams ◽  
Massive Data ◽  
Time Data ◽  
Sky Surveys ◽  
Real Time Data

scholarly journals Real Time Anomaly Detection in Massive Data Streams with ELK Stack

2019 ◽  
Vol 15 (6) ◽  
pp. 814-823
Author(s):  
Jakup Fondaj ◽  
Zirije Hasani
Keyword(s):  
Anomaly Detection ◽  
Real Time ◽  
Data Streams ◽  
Massive Data

scholarly journals Real-Time Classification of Transient Events in Synoptic Sky Surveys

2011 ◽  
Vol 7 (S285) ◽  
pp. 355-357 ◽  
Author(s):  
Ashish A. Mahabal ◽  
C. Donalek ◽  
S. G. Djorgovski ◽  
A. J. Drake ◽  
M. J. Graham ◽  
...  
Keyword(s):  
Real Time ◽  
Automated Classification ◽  
Next Generation ◽  
Classification Techniques ◽  
Sky Surveys ◽  
Sky Survey ◽  
Transient Events ◽  
Real Time Classification

AbstractAn automated rapid classification of the transient events detected in modern synoptic sky surveys is essential for their scientific utility and effective follow-up when resources are scarce. This problem will grow by orders of magnitude with the next generation of surveys. We are exploring a variety of novel automated classification techniques, mostly Bayesian, to respond to those challenges, using the ongoing CRTS sky survey as a testbed. We describe briefly some of the methods used.

scholarly journals Scalable real-time classification of data streams with concept drift

2017 ◽  
Vol 75 ◽  
pp. 187-199 ◽  
Author(s):  
Mark Tennant ◽  
Frederic Stahl ◽  
Omer Rana ◽  
João Bártolo Gomes
Keyword(s):  
Real Time ◽  
Data Streams ◽  
Concept Drift ◽  
Real Time Classification

scholarly journals 237 Developing a Dairy Brain: Improved Decision-Making from Continuous Integrated Data

2020 ◽  
Vol 98 (Supplement_3) ◽  
pp. 125-126
Author(s):  
Victor E Cabrera
Keyword(s):  
Decision Making ◽  
Real Time ◽  
Data Streams ◽  
Data Exchange ◽  
Ad Hoc ◽  
Big Data Analytics ◽  
Service Development ◽  
Time Data ◽  
Practical Applications ◽  
Multiple Data

Abstract Data pervades the dairy farming industry. However, specific data streams are most often ad-hoc and poorly linked to each other and to decision making processes. It is imperative to develop a system that can collect, integrate, manage, and analyze on- and off-farm data in real-time for practical and relevant actions. Hence, we are developing a real-time, data-integrated, data-driven, continuous decision-making engine: The Dairy Brain by applying Precision Farming, Big Data analytics, and the Internet of Things. This is a trans-disciplinary research and extension project that engages multi-disciplinary scientists, dairy farmers, and industry professionals. We have a four-part strategy: (1) Create a Coordinated Innovation Network (CIN) to shape data service development; (2) Create a prototype Agricultural Data Hub (AgDH) to gather/disseminate multiple data streams relevant to dairy operations; (3) Build the Dairy Brain – a suite of analytical modules that leverages the AgDH to provide insight to the management of dairy operations and serve as an exemplar of an ecosystem of connected services; and (4) Design and execute an innovative Extension program. We will illustrate our Dairy Brain concept with a few practical applications. Tomorrow’s dairy industry will be built on the effective capture and integration of more data streams, not fewer. This is a critical moment to develop the structures that can move the industry towards modernized data exchange. We are confident the Dairy Brain concept could shift the paradigm of how dairy farms operate.

Improved Macro-clusters generation using Top-k shared Micro-clusters in Data Streams

2017 ◽  
Vol 7 (10) ◽  
pp. 52
Author(s):  
LAKSHMI PRANEETHA
Keyword(s):  
Real Time ◽  
Data Streams ◽  
Bloom Filter ◽  
Scientific Applications ◽  
Pruning Algorithm ◽  
Density Data ◽  
Data Points ◽  
Short Time ◽  
Information Streams

Now-a-days data streams or information streams are gigantic and quick changing. The usage of information streams can fluctuate from basic logical, scientific applications to vital business and money related ones. The useful information is abstracted from the stream and represented in the form of micro-clusters in the online phase. In offline phase micro-clusters are merged to form the macro clusters. DBSTREAM technique captures the density between micro-clusters by means of a shared density graph in the online phase. The density data in this graph is then used in reclustering for improving the formation of clusters but DBSTREAM takes more time in handling the corrupted data points In this paper an early pruning algorithm is used before pre-processing of information and a bloom filter is used for recognizing the corrupted information. Our experiments on real time datasets shows that using this approach improves the efficiency of macro-clusters by 90% and increases the generation of more number of micro-clusters within in a short time.

Sign in / Sign up

Export Citation Format

Share Document