Synopses for Massive Data: Samples, Histograms, Wavelets, Sketches

2011 ◽  
Author(s):  
Graham Cormode
Keyword(s):  
Massive Data

Approximate Join Aggregate on Massive Data

2010 ◽  
Vol 33 (10) ◽  
pp. 1919-1933
Author(s):  
Xi-Xian HAN ◽  
Dong-Hua YANG ◽  
Jian-Zhong LI
Keyword(s):  
Massive Data

Parallel massive data oriented astronomical cross-match

2010 ◽  
Vol 30 (8) ◽  
pp. 2056-2059 ◽  
Author(s):  
Qing ZHAO ◽  
Ji-zhou SUN ◽  
Ce YU ◽  
Chen-zhou CUI ◽  
Jian XIAO
Keyword(s):  
Massive Data ◽  
Cross Match

scholarly journals High-Resolution Traffic Sensing with Probe Autonomous Vehicles: A Data-Driven Approach

Sensors ◽  
2021 ◽  
Vol 21 (2) ◽  
pp. 464
Author(s):  
Wei Ma ◽  
Sean Qian
Keyword(s):  
High Resolution ◽  
Autonomous Vehicles ◽  
Penetration Rate ◽  
Data Driven ◽  
Traffic Information ◽  
Flow Density ◽  
Massive Data ◽  
Market Penetration ◽  
High Resolution Data ◽  
Data Driven Approach

Recent decades have witnessed the breakthrough of autonomous vehicles (AVs), and the sensing capabilities of AVs have been dramatically improved. Various sensors installed on AVs will be collecting massive data and perceiving the surrounding traffic continuously. In fact, a fleet of AVs can serve as floating (or probe) sensors, which can be utilized to infer traffic information while cruising around the roadway networks. Unlike conventional traffic sensing methods relying on fixed location sensors or moving sensors that acquire only the information of their carrying vehicle, this paper leverages data from AVs carrying sensors for not only the information of the AVs, but also the characteristics of the surrounding traffic. A high-resolution data-driven traffic sensing framework is proposed, which estimates the fundamental traffic state characteristics, namely, flow, density and speed in high spatio-temporal resolutions and of each lane on a general road, and it is developed under different levels of AV perception capabilities and for any AV market penetration rate. Experimental results show that the proposed method achieves high accuracy even with a low AV market penetration rate. This study would help policymakers and private sectors (e.g., Waymo) to understand the values of massive data collected by AVs in traffic operation and management.

scholarly journals Fundamental resource trade-offs for encoded distributed optimization

2020 ◽  
Author(s):  
A Salman Avestimehr ◽  
Seyed Mohammadreza Mousavi Kalan ◽  
Mahdi Soltanolkotabi
Keyword(s):  
Computational Time ◽  
Massive Data ◽  
Data Sets ◽  
Massive Data Sets ◽  
Computational Framework ◽  
Data Set ◽  
Trade Offs ◽  
Major Bottleneck ◽  
Computing Environments ◽  
Analyze Data

Abstract Dealing with the shear size and complexity of today’s massive data sets requires computational platforms that can analyze data in a parallelized and distributed fashion. A major bottleneck that arises in such modern distributed computing environments is that some of the worker nodes may run slow. These nodes a.k.a. stragglers can significantly slow down computation as the slowest node may dictate the overall computational time. A recent computational framework, called encoded optimization, creates redundancy in the data to mitigate the effect of stragglers. In this paper, we develop novel mathematical understanding for this framework demonstrating its effectiveness in much broader settings than was previously understood. We also analyze the convergence behavior of iterative encoded optimization algorithms, allowing us to characterize fundamental trade-offs between convergence rate, size of data set, accuracy, computational load (or data redundancy) and straggler toleration in this framework.

Sign in / Sign up

Export Citation Format

Share Document