scholarly journals Automated Configuration of NoSQL Performance and Scalability Tactics for Data-Intensive Applications

Informatics ◽  
2020 ◽  
Vol 7 (3) ◽  
pp. 29
Author(s):  
Davy Preuveneers ◽  
Wouter Joosen
Keyword(s):  
Supervised Machine Learning ◽  
Tuning Parameters ◽  
Data Intensive ◽  
Optimal Service ◽  
Challenging Tasks ◽  
Growth Changes ◽  
High Level ◽  
Storage Technologies ◽  
Data Intensive Applications ◽  
Current Monitoring

This paper presents the architecture, implementation and evaluation of a middleware support layer for NoSQL storage systems. Our middleware automatically selects performance and scalability tactics in terms of application specific workloads. Enterprises are turning to NoSQL storage technologies for their data-intensive computing and analytics applications. Comprehensive benchmarks of different Big Data platforms can help drive decisions which solutions to adopt. However, selecting the best performing technology, configuring the deployment for scalability and tuning parameters at runtime for an optimal service delivery remain challenging tasks, especially when application workloads evolve over time. Our middleware solves this problem at runtime by monitoring the data growth, changes in the read-write-query mix at run-time, as well as other system metrics that are indicative of sub-optimal performance. Our middleware employs supervised machine learning on historic and current monitoring information and corresponding configurations to select the best combinations of high-level tactics and adapt NoSQL systems to evolving workloads. This work has been driven by two real world case studies with different QoS requirements. The evaluation demonstrates that our middleware can adapt to unseen workloads of data-intensive applications, and automate the configuration of different families of NoSQL systems at runtime to optimize the performance and scalability of such applications.

Creating a portable, high-level graph analytics paradigm for compute and data-intensive applications

2017 ◽  
Vol 1 (1) ◽  
pp. 1
Author(s):  
Robert Searles ◽  
Michela Taufer ◽  
Sunita Chandrasekaran ◽  
Stephen Herbein ◽  
Travis Johnston
Keyword(s):  
Graph Analytics ◽  
Data Intensive ◽  
High Level ◽  
Data Intensive Applications

scholarly journals Building High-Resolution Sky Images Using the Cell/B.E.

2009 ◽  
Vol 17 (1-2) ◽  
pp. 113-134 ◽  
Author(s):  
Ana Lucia Varbanescu ◽  
Alexander S. van Amesfoort ◽  
Tim Cornwell ◽  
Ger van Diepen ◽  
Rob van Nieuwpoort ◽  
...  
Keyword(s):  
High Performance ◽  
Complete Solution ◽  
Performance Potential ◽  
Data Intensive ◽  
Irregular Data ◽  
Original Application ◽  
The One ◽  
High Level ◽  
Data Intensive Applications ◽  
Application Specific

The performance potential of the Cell/B.E., as well as its availability, have attracted a lot of attention from various high-performance computing (HPC) fields. While computation intensive kernels proved to be exceptionally well suited for running on the Cell, irregular data-intensive applications are usually considered as poor matches. In this paper, we present our complete solution for enabling such a data-intensive application to run efficiently on the Cell/B.E. processor. Specifically, we target radioastronomy data gridding and degridding, two resembling imaging filters based on convolutional resampling. Our solution is based on building a high-level application model, used to evaluate parallelization alternatives. Next, we choose the one with the best performance potential, and we gradually exploit this potential by applying platform-specific and application-specific optimizations. After several iterations, our target application shows a speed-up factor between 10 and 20 on a dual-Cell blade when compared with the original application running on a commodity machine. Given these results, and based on our empirical observations, we are able to pinpoint a set of ten guidelines for parallelizing similar applications on the Cell/B.E. Finally, we conclude the Cell/B.E. can provide high performance for data-intensive applications at the price of increased programming efforts and with a significant aid from aggressive application-specific optimizations.

Quality Guaranteed Media Delivery over Advanced Network

2012 ◽  
pp. 121-146
Author(s):  
Zhiming Zhao ◽  
Paola Grosso ◽  
Jeroen van der Ham ◽  
Cees Th. A.M. de Laat
Keyword(s):  
Digital Media ◽  
Network Services ◽  
User Interactions ◽  
Real Time Processing ◽  
Data Intensive ◽  
Delivery Performance ◽  
High Level ◽  
Media Delivery ◽  
Data Intensive Applications ◽  
Software Services

Moving large quantities of data between distributed parties is a frequently invoked process in data intensive applications, such as collaborative digital media development. These transfers often have high quality requirements on the network services, especially when they involve user interactions or require real time processing on large volumes of data. The best effort services provided by IP-routed networks give limited guarantee on the delivery performance. Advanced networks such as hybrid networks make it feasible for high level applications, such as workflows, to request network paths and service provisioning. However, the quality of network services has so far rarely been considered in composing and executing workflow processes; applications tune the execution quality selecting only optimal software services and computing resources, and neglecting the network components. In this chapter, the authors provide an overview on this research domain, and introduce a system called NEtWork QoS Planner (NEWQoSPlanner) to provide support for including network services in high level workflow applications.

scholarly journals Creating a portable, high-level graph analytics paradigm for compute and data-intensive applications

2019 ◽  
Vol 13 (1) ◽  
pp. 105 ◽  
Author(s):  
Robert Searles ◽  
Stephen Herbein ◽  
Travis Johnston ◽  
Michela Taufer ◽  
Sunita Chandrasekaran
Keyword(s):  
Graph Analytics ◽  
Data Intensive ◽  
High Level ◽  
Data Intensive Applications

scholarly journals 6G Enabled Smart Infrastructure for Sustainable Society: Opportunities, Challenges, and Research Roadmap

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1709
Author(s):  
Agbotiname Lucky Imoize ◽  
Oluwadara Adedeji ◽  
Nistha Tandiya ◽  
Sachin Shetty
Keyword(s):  
Wireless Communication ◽  
Psychological Health ◽  
Future Research ◽  
Agriculture Education ◽  
Social Psychological ◽  
Research Issues ◽  
Data Intensive ◽  
Wireless Communication Network ◽  
Data Intensive Applications

The 5G wireless communication network is currently faced with the challenge of limited data speed exacerbated by the proliferation of billions of data-intensive applications. To address this problem, researchers are developing cutting-edge technologies for the envisioned 6G wireless communication standards to satisfy the escalating wireless services demands. Though some of the candidate technologies in the 5G standards will apply to 6G wireless networks, key disruptive technologies that will guarantee the desired quality of physical experience to achieve ubiquitous wireless connectivity are expected in 6G. This article first provides a foundational background on the evolution of different wireless communication standards to have a proper insight into the vision and requirements of 6G. Second, we provide a panoramic view of the enabling technologies proposed to facilitate 6G and introduce emerging 6G applications such as multi-sensory–extended reality, digital replica, and more. Next, the technology-driven challenges, social, psychological, health and commercialization issues posed to actualizing 6G, and the probable solutions to tackle these challenges are discussed extensively. Additionally, we present new use cases of the 6G technology in agriculture, education, media and entertainment, logistics and transportation, and tourism. Furthermore, we discuss the multi-faceted communication capabilities of 6G that will contribute significantly to global sustainability and how 6G will bring about a dramatic change in the business arena. Finally, we highlight the research trends, open research issues, and key take-away lessons for future research exploration in 6G wireless communication.

scholarly journals EZIOTracer

2021 ◽  
Vol 55 (1) ◽  
pp. 88-98
Author(s):  
Mohammed Islam Naas ◽  
François Trahay ◽  
Alexis Colin ◽  
Pierre Olivier ◽  
Stéphane Rubini ◽  
...  
Keyword(s):  
Analysis Framework ◽  
Comprehensive Understanding ◽  
Kernel Space ◽  
Data Intensive ◽  
Storage Performance ◽  
Performance Requirements ◽  
Memory Footprint ◽  
Extreme Performance ◽  
Data Intensive Applications ◽  
Kernel Level

Tracing is a popular method for evaluating, investigating, and modeling the performance of today's storage systems. Tracing has become crucial with the increase in complexity of modern storage applications/systems, that are manipulating an ever-increasing amount of data and are subject to extreme performance requirements. There exists many tracing tools focusing either on the user-level or the kernel-level, however we observe the lack of a unified tracer targeting both levels: this prevents a comprehensive understanding of modern applications' storage performance profiles. In this paper, we present EZIOTracer, a unified I/O tracer for both (Linux) kernel and user spaces, targeting data intensive applications. EZIOTracer is composed of a userland as well as a kernel space tracer, complemented with a trace analysis framework able to merge the output of the two tracers, and in particular to relate user-level events to kernel-level ones, and vice-versa. On the kernel side, EZIOTracer relies on eBPF to offer safe, low-overhead, low memory footprint, and flexible tracing capabilities. We demonstrate using FIO benchmark the ability of EZIOTracer to track down I/O performance issues by relating events recorded at both the kernel and user levels. We show that this can be achieved with a relatively low overhead that ranges from 2% to 26% depending on the I/O intensity.

scholarly journals Differences in subcortico-cortical interactions identified from connectome and microcircuit models in autism

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Bo-yong Park ◽  
Seok-Jun Hong ◽  
Sofie L. Valk ◽  
Casey Paquola ◽  
Oualid Benkarim ◽  
...  
Keyword(s):  
Data Exchange ◽  
Computational Models ◽  
Simulation Models ◽  
Autism Diagnostic Observation Schedule ◽  
Supervised Machine Learning ◽  
Global Network ◽  
Imaging Data ◽  
Phenotypic Data ◽  
Severity Scores ◽  
High Level

AbstractThe pathophysiology of autism has been suggested to involve a combination of both macroscale connectome miswiring and microcircuit anomalies. Here, we combine connectome-wide manifold learning with biophysical simulation models to understand associations between global network perturbations and microcircuit dysfunctions in autism. We studied neuroimaging and phenotypic data in 47 individuals with autism and 37 typically developing controls obtained from the Autism Brain Imaging Data Exchange initiative. Our analysis establishes significant differences in structural connectome organization in individuals with autism relative to controls, with strong between-group effects in low-level somatosensory regions and moderate effects in high-level association cortices. Computational models reveal that the degree of macroscale anomalies is related to atypical increases of recurrent excitation/inhibition, as well as subcortical inputs into cortical microcircuits, especially in sensory and motor areas. Transcriptomic association analysis based on postmortem datasets identifies genes expressed in cortical and thalamic areas from childhood to young adulthood. Finally, supervised machine learning finds that the macroscale perturbations are associated with symptom severity scores on the Autism Diagnostic Observation Schedule. Together, our analyses suggest that atypical subcortico-cortical interactions are associated with both microcircuit and macroscale connectome differences in autism.

Sign in / Sign up

Export Citation Format

Share Document