Application of asynchronous invocation architecture in massive data processing of distribution automation

2006 ◽  
Author(s):  
Yi Liao ◽  
Dong Liu
Keyword(s):  
Data Processing ◽  
Distribution Automation ◽  
Massive Data ◽  
Massive Data Processing

scholarly journals Astronomical Massive Data Processing Technology

2020 ◽  
Vol 2020 ◽  
pp. 1-2
Author(s):  
Feng Wang ◽  
Wenda Cao ◽  
Denis Bastieri ◽  
Junhui Fan ◽  
Chenzhou Cui
Keyword(s):  
Data Processing ◽  
Processing Technology ◽  
Massive Data ◽  
Massive Data Processing

scholarly journals Research and Application of Massive Data Processing in Oil Services

2012 ◽  
Vol 6-7 ◽  
pp. 1036-1040
Author(s):  
Bao An Li
Keyword(s):  
Cloud Computing ◽  
Big Data ◽  
Internet Of Things ◽  
Data Processing ◽  
Rapid Growth ◽  
Massive Data ◽  
Oil Drilling ◽  
Data Problem ◽  
Traditional Industry ◽  
Massive Data Processing

Big data problem has caused widespread concern from industry to academia in recent years. As the amount of data produced by various industries and sectors of rapid growth, increasing demands on data processing and analysis capabilities, how to face the challenges of data, discover new opportunities, the issue has received wide attention. As a traditional industry, the oil drilling or refinery enterprise is facing the operational status of the system to produce large amounts of data. This text introduced an approach to massive data processing for oil enterprise based on cloud computing and Internet of Things.

scholarly journals Upgrading a high performance computing environment for massive data processing

2019 ◽  
Vol 10 (1) ◽  
Author(s):  
Lucas M. Ponce ◽  
Walter dos Santos ◽  
Wagner Meira ◽  
Dorgival Guedes ◽  
Daniele Lezzi ◽  
...  
Keyword(s):  
Big Data ◽  
Data Processing ◽  
High Performance Computing ◽  
High Performance ◽  
Data Access ◽  
Massive Data ◽  
Analysis Tool ◽  
Data Framework ◽  
Performance Computing ◽  
Massive Data Processing

Abstract High-performance computing (HPC) and massive data processing (Big Data) are two trends that are beginning to converge. In that process, aspects of hardware architectures, systems support and programming paradigms are being revisited from both perspectives. This paper presents our experience on this path of convergence with the proposal of a framework that addresses some of the programming issues derived from such integration. Our contribution is the development of an integrated environment that integretes (i) COMPSs, a programming framework for the development and execution of parallel applications for distributed infrastructures; (ii) Lemonade, a data mining and analysis tool; and (iii) HDFS, the most widely used distributed file system for Big Data systems. To validate our framework, we used Lemonade to create COMPSs applications that access data through HDFS, and compared them with equivalent applications built with Spark, a popular Big Data framework. The results show that the HDFS integration benefits COMPSs by simplifying data access and by rearranging data transfer, reducing execution time. The integration with Lemonade facilitates COMPSs’s use and may help its popularization in the Data Science community, by providing efficient algorithm implementations for experts from the data domain that want to develop applications with a higher level abstraction.

Sign in / Sign up

Export Citation Format

Share Document