An Approach to Adaptive Memory Management in Data Stream Systems

2006 ◽  
Author(s):  
M. Cammert ◽  
J. Kramer ◽  
B. Seeger ◽  
S. Vaupel
Keyword(s):  
Data Stream ◽  
Memory Management ◽  
Adaptive Memory ◽  
Stream Systems

Breaking down memory walls

2020 ◽  
Vol 14 (3) ◽  
pp. 241-254
Author(s):  
Chen Luo ◽  
Michael J. Carey
Keyword(s):  
Memory Management ◽  
Storage Systems ◽  
Memory Allocation ◽  
Adaptive Memory ◽  
Component Structure ◽  
Memory Wall ◽  
Buffer Cache ◽  
Management Architecture

Log-Structured Merge-trees (LSM-trees) have been widely used in modern NoSQL systems. Due to their out-of-place update design, LSM-trees have introduced memory walls among the memory components of multiple LSM-trees and between the write memory and the buffer cache. Optimal memory allocation among these regions is non-trivial because it is highly workload-dependent. Existing LSM-tree implementations instead adopt static memory allocation schemes due to their simplicity and robustness, sacrificing performance. In this paper, we attempt to break down these memory walls in LSM-based storage systems. We first present a memory management architecture that enables adaptive memory management. We then present a partitioned memory component structure with new flush policies to better exploit the write memory to minimize the write cost. To break down the memory wall between the write memory and the buffer cache, we further introduce a memory tuner that tunes the memory allocation between these two regions. We have conducted extensive experiments in the context of Apache AsterixDB using the YCSB and TPC-C benchmarks and we present the results here.

Dynamic Plan Migration for Snapshot-Equivalent Continuous Queries in Data Stream Systems

2006 ◽  
pp. 497-516 ◽  
Author(s):  
Jürgen Krämer ◽  
Yin Yang ◽  
Michael Cammert ◽  
Bernhard Seeger ◽  
Dimitris Papadias
Keyword(s):  
Data Stream ◽  
Continuous Queries ◽  
Stream Systems

Placement-Safe Operator-Graph Changes in Distributed Heterogeneous Data Stream Systems

2015 ◽  
Vol 15 (3) ◽  
pp. 203-211 ◽  
Author(s):  
Niko Pollner ◽  
Christian Steudtner ◽  
Klaus Meyer-Wegener
Keyword(s):  
Data Stream ◽  
Heterogeneous Data ◽  
Stream Systems

scholarly journals Mining Association Rule for the Abnormal Event in Data Stream Systems

2007 ◽  
Vol 14D (5) ◽  
pp. 483-490 ◽  
Author(s):  
Dae-In Kim ◽  
Joon Park ◽  
Bu-Hyun Hwang
Keyword(s):  
Association Rule ◽  
Data Stream ◽  
Mining Association Rule ◽  
Stream Systems
Sign in / Sign up

Export Citation Format

Share Document