Combining Live Drilling Data Stream with a Cloud Data Analytics Pipeline to Perform Real-Time Automated Projections to the Bit

2021 ◽  
Author(s):  
Daniel Cardoso Braga ◽  
Mohammadreza Kamyab ◽  
Brian Harclerode ◽  
Deep Joshi
Keyword(s):  
Trend Analysis ◽  
Real Time ◽  
Data Stream ◽  
Data Analytics ◽  
Data Sources ◽  
Rotary Drilling ◽  
Cloud Data ◽  
West Texas ◽  
Multiple Data ◽  
Drilling Data

Abstract During drilling, surveys to determine the wellbore trajectory are performed at every drilling connection. However, due to the offset between the survey instrument and the bit (typically between 30-100 ft), this survey represents the sensor's position which is lagged compared to the bit. This paper describes a method to automatically calculate projections to the bit in real-time utilizing multiple data sources: WITSML stream, BHA components and rotary trend analysis while rotary drilling. The projection to the bit calculation routine is performed in real time every 30 seconds. This paper presents results of projections for four horizontal unconventional wells drilled in West Texas. Nearly 75,000 projections were generated on the four wells, validated with 839 survey stations, with median divergence of the projections from the nearest survey stations being less than one foot.

scholarly journals Real-time Big Data Analytics Framework with Data Blending Approach for Multiple Data sources in Smart City Applications

2020 ◽  
Vol 21 (4) ◽  
pp. 611-623
Author(s):  
Manjunatha S ◽  
Annappa B
Keyword(s):  
Real Time ◽  
Smart City ◽  
Data Analytics ◽  
Public Safety ◽  
Big Data Analytics ◽  
Data Sources ◽  
Data Driven ◽  
The Public ◽  
Multiple Data Sources ◽  
Multiple Data

Advancement in Information Communication Technology (ICT) and the Internet of Things (IoT) has to lead tothe continuous generation of a large amount of data. Smart city projects are being implemented in various parts of the world where analysis of public data helps in providing a better quality of life. Data analytics plays a vital role in many such data-driven applications. Real-time analytics for finding valuable insights at the right time using smart city data is crucial in making appropriate decisions for city administration. It is essential to use multiple data sources as input for the analysis to achieve better and more accurate data-driven solutions. It helps in finding more accurate solutions and making appropriate decisions. Public safety is one of the major concerns in any smart city project in which real-time analytics is much useful in the early detection of valuable data patterns. It is crucial to find early predictions of crime-related incidents and generating emergency alerts for making appropriate decisions to provide security to the people and safety of the city infrastructure. This paper discusses the proposed real-time big data analytics framework with data blending approach using multiple data sources for smart city applications. Analytics using multiple data sources for a specific data-driven solution helps in finding more data patterns, which in turn increases the accuracy of analytics results. The data preprocessing phase is a challenging task in data analytics when data being ingested continuously in real-time into the analytics system. The proposed system helps in the preprocessing of real-time data with data blending of multiple data sources used in the analytics. The proposed framework is beneficial when data from multiple sources are ingested in real-time as input data and is also flexible to use any additional data source of interest. The experimental work carried out with the proposed framework using multiple data sources to find the crime-related insights in real-time helps the public safety solutions in the smart city. The experimental outcome shows that there is a significant increase in the number of identified useful data patterns as the number of data sources increases. A real-time based emergency alert system to help the public safety solution is implementedusing a machine learning-based classification algorithm with the proposed framework. The experiment is carried out with different classification algorithms, and the results show that Naive Bayes classification  performs better in generating emergency alerts.

An Overview of Drought Monitoring and Prediction Systems at Regional and Global Scales

2017 ◽  
Vol 98 (9) ◽  
pp. 1879-1896 ◽  
Author(s):  
Zengchao Hao ◽  
Xing Yuan ◽  
Youlong Xia ◽  
Fanghua Hao ◽  
Vijay P. Singh
Keyword(s):  
Real Time ◽  
Land Surface ◽  
Information Dissemination ◽  
Data Sources ◽  
Drought Monitoring ◽  
Severe Drought ◽  
Drought Prediction ◽  
Multiple Data ◽  
Challenges And Opportunities ◽  
Prediction Systems

Abstract In past decades, severe drought events have struck different regions around the world, leading to huge losses to a wide array of environmental and societal sectors. Because of wide impacts of drought, it is of critical importance to monitor drought in near–real time and provide early warning. This article provides an overview of the development of drought monitoring and prediction systems (DMAPS) at regional and global scales. After introducing drought indicators, drought monitoring (based on different data sources and tools) is summarized, along with an introduction of statistical and dynamical drought prediction approaches. The current progress of the development and implementation of DMAPS with various indicators at different temporal and/or spatial resolutions, based on the land surface modeling, remote sensing, and seasonal climate forecast, at the regional and global scales is then reviewed. Advances in drought monitoring with multiple data sources and tools and prediction from multimodel ensembles are highlighted. Also highlighted are challenges and opportunities, including near-real-time and long-term data products, indicator linkage to impacts, prediction skill improvement, and information dissemination/communication. The review of different components of these systems will provide useful guidelines and insights for the future development of effective DMAPS to aid drought modeling and management.

scholarly journals Automated Modeling of Real-Time Anomaly Detection using Non-Parametric Statistical technique for Data Streams in Cloud Environments

2019 ◽  
Vol 15 (3) ◽  
Author(s):  
Smrithy G S ◽  
Ramadoss Balakrishnan
Keyword(s):  
Anomaly Detection ◽  
Real Time ◽  
Data Stream ◽  
False Negative ◽  
Statistical Technique ◽  
Cloud Data ◽  
Negative Rate ◽  
Positive Rate ◽  
Online Anomaly Detection ◽  
Non Parametric

The main objective of online anomaly detection is to identify abnormal/unusual behavior such as network intrusions, malware infections, over utilized system resources due to design defects etc from real time data stream. Terrabytes of performance data generated in cloud data centers is a well accepted example of such data stream in real time. In this paper, we propose an online anomaly detection framework using non-parametric statistical technique in cloud data center. In order to determine the accuracy of the proposed work, we experiments it to data collected from RUBis cloud testbed and Yahoo Cloud Serving Benchmark (YCSB). Our experimental results shows the greater accuracy in terms of True Positive Rate (TPR), False Positive Rate (FPR), True Negative Rate (TNR) and False Negative Rate (FNR).

scholarly journals Predicting Drug Side Effects Using Data Analytics and the Integration of Multiple Data Sources

IEEE Access ◽  
2017 ◽  
Vol 5 ◽  
pp. 20449-20462 ◽  
Author(s):  
Wei-Po Lee ◽  
Jhih-Yuan Huang ◽  
Hsuan-Hao Chang ◽  
King-Teh Lee ◽  
Chao-Ti Lai
Keyword(s):  
Side Effects ◽  
Data Analytics ◽  
Data Sources ◽  
Drug Side Effects ◽  
Multiple Data Sources ◽  
Multiple Data ◽  
Using Data

scholarly journals 434. Tracking COVID-19 in Real Time: Leveraging Public Data Sources to Inform Infection Prevention Practices

2020 ◽  
Vol 7 (Supplement_1) ◽  
pp. S285-S285
Author(s):  
Kevin P O’Callaghan ◽  
Kyle Winser ◽  
Vaidehi Mehta ◽  
Eimear Kitt ◽  
Coralee DelValle Mojica ◽  
...  
Keyword(s):  
Real Time ◽  
Data Analytics ◽  
Processing System ◽  
International Travel ◽  
Data Sources ◽  
Atlantic Region ◽  
Data Set ◽  
Period Prevalence ◽  
Increased Risk ◽  
Public Data

Abstract Background Global spread of SARS-CoV-2 led to an urgent need for data on national and regional prevalence to inform public health policy. Healthcare systems were also in need of data to develop best practices around defining patient risk. We describe a data analytics tool developed at our institution which uses public data sources to track county-level prevalence of COVID-19 so as to delineate risk for individual patients. Methods We investigated a number of data sources tracking COVID-19 case counts, assessing for (1) frequency of updates, (2) granularity of geographic detail (optimally to zip-code or county) and (3) completeness of the data. We chose the Johns Hopkins University CSSE COVID-19 data set. This contains counts of new diagnoses per day by county using Federal Information Processing System (FIPS) codes. The dataset is updated daily with adjustments made for backdated corrections. We developed a data analytics tool which allowed for direct comparison of county period prevalence. We developed a metric of 10-day rolling period prevalence calculated as a total case count from the preceding 10 days, divided by county population from 2018 American Community Survey (ACS) estimates. Results Benchmarking against local (peak of 3.12 cases per 1,000 persons) and regional prevalence, we set 6 cases/1,000 persons as the threshold for a Geographic Region with Widespread Community Transmission (GReWCoT). Counties have to reach this threshold for at least 4 out of 7 days within the period 3 to 10 days prior to the evaluation, to adjust for bulking of test results and delayed reporting. We used the analytics tool to support a semimonthly review of geographic regions, and made specific recommendations for patients from qualifying regions including use of modified enhanced precautions (including surgical mask and eye protection), as well as restricted visitation of caregivers. Figure 1. Epidemic curves for 10-day rolling period prevalence of COVID-19 in the Mid-Atlantic Region: Philadelphia County, PA Figure 2. Epidemic curves for 10-day rolling period prevalence of COVID-19 in the Mid-Atlantic Region: Westchester County, NY Figure 3. Epidemic curves for 10-day rolling period prevalence of COVID-19 in the Mid-Atlantic Region: Bergen County, NJ Conclusion This approach allowed for a nuanced investigation of COVID-19 prevalence in real-time, and provided support for risk stratification of patients throughout our large catchment area. The dashboard was shared on an inward-facing site to support staff messaging about regions of increased risk. Next steps include leveraging international data to inform a similar approach to international travel for our patients and staff. Disclosures All Authors: No reported disclosures

scholarly journals Fast and Accurate Terrain Image Classification for ASTER Remote Sensing by Data Stream Mining and Evolutionary-EAC Instance-Learning-Based Algorithm

2021 ◽  
Vol 13 (6) ◽  
pp. 1123
Author(s):  
Shimin Hu ◽  
Simon Fong ◽  
Lili Yang ◽  
Shuang-Hua Yang ◽  
Nilanjan Dey ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Real Time ◽  
Image Recognition ◽  
Data Stream ◽  
Data Analytics ◽  
Classification Model ◽  
Significant Feature ◽  
Data Stream Mining ◽  
Feature Subset ◽  
Stream Mining

Remote sensing streams continuous data feed from the satellite to ground station for data analysis. Often the data analytics involves analyzing data in real-time, such as emergency control, surveillance of military operations or scenarios that change rapidly. Traditional data mining requires all the data to be available prior to inducing a model by supervised learning, for automatic image recognition or classification. Any new update on the data prompts the model to be built again by loading in all the previous and new data. Therefore, the training time will increase indefinitely making it unsuitable for real-time application in remote sensing. As a contribution to solving this problem, a new approach of data analytics for remote sensing for data stream mining is formulated and reported in this paper. Fresh data feed collected from afar is used to approximate an image recognition model without reloading the history, which helps eliminate the latency in building the model again and again. In the past, data stream mining has a drawback in approximating a classification model with a sufficiently high level of accuracy. This is due to the one-pass incremental learning mechanism inherently exists in the design of the data stream mining algorithm. In order to solve this problem, a novel streamlined sensor data processing method is proposed called evolutionary expand-and-contract instance-based learning algorithm (EEAC-IBL). The multivariate data stream is first expanded into many subspaces, and then the subspaces, which are corresponding to the characteristics of the features are selected and condensed into a significant feature subset. The selection operates stochastically instead of deterministically by evolutionary optimization, which approximates the best subgroup. Followed by data stream mining, the model learning for image recognition is done on the fly. This stochastic approximation method is fast and accurate, offering an alternative to the traditional machine learning method for image recognition application in remote sensing. Our experimental results show computing advantages over other classical approaches, with a mean accuracy improvement at 16.62%.

Closed-Loop Data Analytics for Wells Construction Management in Real Time Centre

2021 ◽  
Author(s):  
Azlesham Rosli ◽  
Whye Jin Mak ◽  
Bobbywadi Richard ◽  
Meor M Meor Hashim ◽  
M Faris Arriffin ◽  
...  
Keyword(s):  
Real Time ◽  
Information Exchange ◽  
Data Analytics ◽  
Dynamic Monitoring ◽  
Real Time Monitoring ◽  
The Real ◽  
Current Process ◽  
Execution Phase ◽  
Drilling Data ◽  
Monitoring Platform

Abstract The execution phase of the wells technical assurance process is a critical procedure where the drilling operation commences and the well planning program is implemented. During drilling operations, the real-time drilling data are streamed to a real-time centre where it is constantly monitored by a dedicated team of monitoring specialists. If any potential issues or possible opportunities arise, the team will communicate with the operation team on rig for an intervention. This workflow is further enhanced by digital initiatives via big data analytics implementation in PETRONAS. The Digital Standing Instruction to Driller (Digital SID) is a drilling operational procedures documentation tool meant to improve the current process by digitalizing information exchange between office and rig site. Boasting multi-operation usage, it is made fit to context and despite its automated generation, this tool allows flexibility for the operation team to customize the content and more importantly, monitor the execution in real-time. Another tool used in the real-time monitoring platform is the dynamic monitoring drilling system where it allows real-time drilling data to be more intuitive and gives the benefit of foresight. The dynamic nature of the system means that it will update existing roadmaps with extensive real-time data as they come in, hence improving its accuracy as we drill further. Furthermore, an automated drilling key performance indicator (KPI) and performance benchmarking system measures drilling performance to uncover areas of improvement. This will serve as the benchmark for further optimization. On top of that, an artificial intelligence (AI) driven Wells Augmented Stuck Pipe Indicator (WASP) is deployed in the real-time monitoring platform to improve the capability of monitoring specialists to identify stuck pipe symptoms way earlier before the occurrence of the incident. This proactive approach is an improvement to the current process workflow which is less timely and possibly missing the intervention opportunity. These four tools are integrated seamlessly with the real-time monitoring platform hence improving the project management efficiency during the execution phase. The tools are envisioned to offer an agile and efficient process workflow by integrating and tapering down multiple applications in different environments into a single web-based platform which enables better collaboration and faster decision making.

Fog-Cloud Collaboration for Real-Time Streaming Applications

2019 ◽  
pp. 128-147
Author(s):  
Biji Nair ◽  
S. Mary Saira Bhanu
Keyword(s):  
Real Time ◽  
Data Analytics ◽  
High Speed ◽  
Fog Computing ◽  
Data Representation ◽  
Streaming Data ◽  
Cloud Data ◽  
Streaming Applications ◽  
Stream Management

Real-time streaming applications (RTSAs) generate huge volumes of temporally ordered, infinite, continuous, high speed data streams demanding both real-time and long-term data analytics. Fog computing is a reliable solution for processing and analyzing real-time streaming data as it offers low latency, location-aware, geographically distributed service at fog node and provides long-term services at the cloud data center (DC). This chapter addresses the challenge of coordinating the fog nodes and cloud for efficient processing of real-time streaming data in motion and at rest. The fog-cloud collaboration framework proposed in this chapter employs data stream management system (DSMS) schema at the fog node for real-time stream data processing and response generation. The data representation in micro-clusters at fog node and macro-clusters at DC facilitates accurate data analytics. The coordination between fog node and DC is through local ontology and global ontology respectively.

Sign in / Sign up

Export Citation Format

Share Document