Big Data Processing Workflows Oriented Real-Time Scheduling Algorithm using Task-Duplication in Geo-Distributed Clouds

2020 ◽  
Vol 6 (1) ◽  
pp. 131-144 ◽  
Author(s):  
Huangke Chen ◽  
Jinming Wen ◽  
Witold Pedrycz ◽  
Guohua Wu
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Real Time ◽  
Scheduling Algorithm ◽  
Big Data Processing ◽  
Real Time Scheduling ◽  
Task Duplication ◽  
Time Scheduling

scholarly journals The real-time big data processing method based on LSTM or GRU for the smart job shop production process

2020 ◽  
Vol 14 ◽  
pp. 174830262096239 ◽  
Author(s):  
Chuang Wang ◽  
Wenbo Du ◽  
Zhixiang Zhu ◽  
Zhifeng Yue
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Production Process ◽  
Real Time ◽  
Job Shop ◽  
Production Data ◽  
Time Data ◽  
Big Data Processing ◽  
The Real ◽  
Data Processing Method

With the wide application of intelligent sensors and internet of things (IoT) in the smart job shop, a large number of real-time production data is collected. Accurate analysis of the collected data can help producers to make effective decisions. Compared with the traditional data processing methods, artificial intelligence, as the main big data analysis method, is more and more applied to the manufacturing industry. However, the ability of different AI models to process real-time data of smart job shop production is also different. Based on this, a real-time big data processing method for the job shop production process based on Long Short-Term Memory (LSTM) and Gate Recurrent Unit (GRU) is proposed. This method uses the historical production data extracted by the IoT job shop as the original data set, and after data preprocessing, uses the LSTM and GRU model to train and predict the real-time data of the job shop. Through the description and implementation of the model, it is compared with KNN, DT and traditional neural network model. The results show that in the real-time big data processing of production process, the performance of the LSTM and GRU models is superior to the traditional neural network, K nearest neighbor (KNN), decision tree (DT). When the performance is similar to LSTM, the training time of GRU is much lower than LSTM model.

scholarly journals Big Data Usage Patterns in the Health Care Domain: A Use Case Driven Approach Applied to the Assessment of Vaccination Benefits and Risks

2014 ◽  
Vol 23 (01) ◽  
pp. 27-35 ◽  
Author(s):  
S. de Lusignan ◽  
S-T. Liaw ◽  
C. Kuziemsky ◽  
F. Mold ◽  
P. Krause ◽  
...  
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Real Time ◽  
Predictive Analytics ◽  
Regulatory Compliance ◽  
Use Cases ◽  
Data Sources ◽  
Use Case ◽  
Crowd Sourcing ◽  
Big Data Processing

Summary Background: Generally benefits and risks of vaccines can be determined from studies carried out as part of regulatory compliance, followed by surveillance of routine data; however there are some rarer and more long term events that require new methods. Big data generated by increasingly affordable personalised computing, and from pervasive computing devices is rapidly growing and low cost, high volume, cloud computing makes the processing of these data inexpensive. Objective: To describe how big data and related analytical methods might be applied to assess the benefits and risks of vaccines. Method: We reviewed the literature on the use of big data to improve health, applied to generic vaccine use cases, that illustrate benefits and risks of vaccination. We defined a use case as the interaction between a user and an information system to achieve a goal. We used flu vaccination and pre-school childhood immunisation as exemplars. Results: We reviewed three big data use cases relevant to assessing vaccine benefits and risks: (i) Big data processing using crowd-sourcing, distributed big data processing, and predictive analytics, (ii) Data integration from heterogeneous big data sources, e.g. the increasing range of devices in the “internet of things”, and (iii) Real-time monitoring for the direct monitoring of epidemics as well as vaccine effects via social media and other data sources. Conclusions: Big data raises new ethical dilemmas, though its analysis methods can bring complementary real-time capabilities for monitoring epidemics and assessing vaccine benefit-risk balance.

Modeling Big Data Analytics with a Real-Time Executable Specification Language

2015 ◽  
pp. 289-312
Author(s):  
Amir A. Khwaja
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Real Time ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Exception Handling ◽  
Specification Language ◽  
Big Data Processing ◽  
Business Decisions ◽  
Concurrent Processes

Big data explosion has already happened and the situation is only going to exacerbate with such a high number of data sources and high-end technology prevalent everywhere, generating data at a frantic pace. One of the most important aspects of big data is being able to capture, process, and analyze data as it is happening in real-time to allow real-time business decisions. Alternate approaches must be investigated especially consisting of highly parallel and real-time computations for big data processing. The chapter presents RealSpec real-time specification language that may be used for the modeling of big data analytics due to the inherent language features needed for real-time big data processing such as concurrent processes, multi-threading, resource modeling, timing constraints, and exception handling. The chapter provides an overview of RealSpec and applies the language to a detailed big data event recognition case study to demonstrate language applicability to big data framework and analytics modeling.

Modeling Big Data Analytics with a Real-Time Executable Specification Language

Big Data ◽  
2016 ◽  
pp. 418-440
Author(s):  
Amir A. Khwaja
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Real Time ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Exception Handling ◽  
Specification Language ◽  
Big Data Processing ◽  
Business Decisions ◽  
Concurrent Processes

Big data explosion has already happened and the situation is only going to exacerbate with such a high number of data sources and high-end technology prevalent everywhere, generating data at a frantic pace. One of the most important aspects of big data is being able to capture, process, and analyze data as it is happening in real-time to allow real-time business decisions. Alternate approaches must be investigated especially consisting of highly parallel and real-time computations for big data processing. The chapter presents RealSpec real-time specification language that may be used for the modeling of big data analytics due to the inherent language features needed for real-time big data processing such as concurrent processes, multi-threading, resource modeling, timing constraints, and exception handling. The chapter provides an overview of RealSpec and applies the language to a detailed big data event recognition case study to demonstrate language applicability to big data framework and analytics modeling.

Trends and Technologies in Big Data Processing

2020 ◽  
pp. 17-42
Author(s):  
Amitava Choudhury ◽  
Kalpana Rangra
Keyword(s):  
Cloud Computing ◽  
Big Data ◽  
Internet Of Things ◽  
Data Processing ◽  
Real Time ◽  
Computing Technology ◽  
Time Data ◽  
Big Data Processing ◽  
Real Time Data ◽  
Real Time Data Processing

Data type and amount in human society is growing at an amazing speed, which is caused by emerging new services such as cloud computing, internet of things, and location-based services. The era of big data has arrived. As data has been a fundamental resource, how to manage and utilize big data better has attracted much attention. Especially with the development of the internet of things, how to process a large amount of real-time data has become a great challenge in research and applications. Recently, cloud computing technology has attracted much attention to high performance, but how to use cloud computing technology for large-scale real-time data processing has not been studied. In this chapter, various big data processing techniques are discussed.

Sign in / Sign up

Export Citation Format

Share Document