A high-performance, ad hoc, fuzzy query processing system

1993 ◽  
Vol 2 (4) ◽  
pp. 397-419 ◽  
Author(s):  
William H. Mansfield ◽  
Robert M. Fleischman
Keyword(s):  
Query Processing ◽  
High Performance ◽  
Ad Hoc ◽  
Processing System ◽  
Fuzzy Query

Research of a MapReduce Model to Process the Traffic Big Data

2014 ◽  
Vol 548-549 ◽  
pp. 1853-1856 ◽  
Author(s):  
Wen Chuan Yang ◽  
He Chen ◽  
Qing Yi Qu
Keyword(s):  
High Performance ◽  
Large Scale ◽  
Ad Hoc ◽  
Processing System ◽  
Traffic Information ◽  
Traffic Data ◽  
Fine Grain ◽  
Large Scale Dataset ◽  
Data Processing Center ◽  
Mapreduce Model

Normally, the job of the Traffic Data Processing Center (TDPC) is to monitor and retain data. There is a tendency to put more capability into the TDPC, such as ad-hoc query for speeding car identification and feedback abnormal traffic information. Thus we definitely need to think about what can be kept in working storage and how to analysis it. Obviously, the ordinary database cannot handle the massive dataset and complex ad-hoc query. MapReduce is a popular and widely used fine grain parallel runtime, which is developed for high performance processing of large scale dataset. In this paper, we propose MRTP, a MapReduce Traffic Processing system based on Hive/Hadoop frameworks. A distributed file system HDFS is used in MRTP for fast data sharing and query. MRTP supports fast locating speeding car and also optimizes the route to catch fugitive. Our results show that the model achieves a higher efficiency.

scholarly journals An Adaptive Throughput-First Packet Scheduling Algorithm for DPDK-Based Packet Processing Systems

2021 ◽  
Vol 13 (3) ◽  
pp. 78
Author(s):  
Chuanhong Li ◽  
Lei Song ◽  
Xuewen Zeng
Keyword(s):  
Packet Loss ◽  
High Performance ◽  
Packet Scheduling ◽  
Scheduling Algorithm ◽  
Processing System ◽  
System Throughput ◽  
Packet Processing ◽  
Research Attention ◽  
Continuous Increase ◽  
Packet Scheduling Algorithm

The continuous increase in network traffic has sharply increased the demand for high-performance packet processing systems. For a high-performance packet processing system based on multi-core processors, the packet scheduling algorithm is critical because of the significant role it plays in load distribution, which is related to system throughput, attracting intensive research attention. However, it is not an easy task since the canonical flow-level packet scheduling algorithm is vulnerable to traffic locality, while the packet-level packet scheduling algorithm fails to maintain cache affinity. In this paper, we propose an adaptive throughput-first packet scheduling algorithm for DPDK-based packet processing systems. Combined with the feature of DPDK burst-oriented packet receiving and transmitting, we propose using Subflow as the scheduling unit and the adjustment unit making the proposed algorithm not only maintain the advantages of flow-level packet scheduling algorithms when the adjustment does not happen but also avoid packet loss as much as possible when the target core may be overloaded Experimental results show that the proposed method outperforms Round-Robin, HRW (High Random Weight), and CRC32 on system throughput and packet loss rate.

Beyond MPI

2021 ◽  
Vol 49 (4) ◽  
pp. 12-17
Author(s):  
Feilong Liu ◽  
Claude Barthels ◽  
Spyros Blanas ◽  
Hideaki Kimura ◽  
Garret Swart
Keyword(s):  
High Performance ◽  
Processing System ◽  
Complex Interaction ◽  
Remote Memory ◽  
Interaction Patterns ◽  
Round Trip ◽  
Data Processing System ◽  
Data Intensive ◽  
Multiple Round ◽  
Programming Interface

Networkswith Remote DirectMemoryAccess (RDMA) support are becoming increasingly common. RDMA, however, offers a limited programming interface to remote memory that consists of read, write and atomic operations. With RDMA alone, completing the most basic operations on remote data structures often requires multiple round-trips over the network. Data-intensive systems strongly desire higher-level communication abstractions that supportmore complex interaction patterns. A natural candidate to consider is MPI, the de facto standard for developing high-performance applications in the HPC community. This paper critically evaluates the communication primitives of MPI and shows that using MPI in the context of a data processing system comes with its own set of insurmountable challenges. Based on this analysis, we propose a new communication abstraction named RDMO, or Remote DirectMemory Operation, that dispatches a short sequence of reads, writes and atomic operations to remote memory and executes them in a single round-trip.

A high sensitivity wireless mass-loading surface acoustic wave DNA biosensor

2014 ◽  
Vol 28 (07) ◽  
pp. 1450056 ◽  
Author(s):  
Hua-Lin Cai ◽  
Yi Yang ◽  
Yi-Han Zhang ◽  
Chang-Jian Zhou ◽  
Cang-Ran Guo ◽  
...  
Keyword(s):  
Acoustic Wave ◽  
Surface Acoustic Wave ◽  
Dna Sequences ◽  
Biological Treatment ◽  
High Performance ◽  
Processing System ◽  
High Sensitivity ◽  
Treatment Method ◽  
Saw Sensor ◽  
Target Dna

In this paper, a surface acoustic wave (SAW) biosensor with gold delay area on LiNbO 3 substrate detecting DNA sequences is proposed. By well-designed device parameters of the SAW sensor, it achieves a high performance for highly sensitive detection of target DNA. In addition, an effective biological treatment method for DNA immobilization and abundant experimental verification of the sensing effect have made it a reliable device in DNA detection. The loading mass of the probe and target DNA sequences is obtained from the frequency shifts, which are big enough in this work due to an effective biological treatment. The experimental results show that the biosensor has a high sensitivity of 1.2 pg/ml/Hz and high selectivity characteristic is also verified by the few responses of other substances. In combination with wireless transceiver, we develop a wireless receiving and processing system that can directly display the detection results.

Massive Image Treatment System Based on Cloud Computing Platform

2014 ◽  
Vol 687-691 ◽  
pp. 3733-3737
Author(s):  
Dan Wu ◽  
Ming Quan Zhou ◽  
Rong Fang Bie
Keyword(s):  
Image Processing ◽  
Cloud Computing ◽  
High Performance ◽  
Large Scale ◽  
Processing System ◽  
Virtual Space ◽  
Image Processing System ◽  
Computing Platform ◽  
Simulation Calculation ◽  
Computer Resources

Massive image processing technology requires high requirements of processor and memory, and it needs to adopt high performance of processor and the large capacity memory. While the single or single core processing and traditional memory can’t satisfy the need of image processing. This paper introduces the cloud computing function into the massive image processing system. Through the cloud computing function it expands the virtual space of the system, saves computer resources and improves the efficiency of image processing. The system processor uses multi-core DSP parallel processor, and develops visualization parameter setting window and output results using VC software settings. Through simulation calculation we get the image processing speed curve and the system image adaptive curve. It provides the technical reference for the design of large-scale image processing system.

A High-Performance Signal Processing System for Monopulse Tracking Radar

2011 ◽  
Vol 383-390 ◽  
pp. 471-475
Author(s):  
Yong Bin Hong ◽  
Cheng Fa Xu ◽  
Mei Guo Gao ◽  
Li Zhi Zhao
Keyword(s):  
Signal Processing ◽  
High Performance ◽  
Dynamic Range ◽  
Signal To Noise Ratio ◽  
Processing System ◽  
Digital Signal ◽  
Radar Signal ◽  
Signal Processing System ◽  
Parallel Pipelined ◽  
Tracking Radar

A radar signal processing system characterizing high instantaneous dynamic range and low system latency is designed based on a specifically developed signal processing platform. Instantaneous dynamic range loss is a critical problem when digital signal processing is performed on fixed-point FPGAs. In this paper, the problem is well resolved by increasing the wordlength according to signal-to-noise ratio (SNR) gain of the algorithms through the data path. The distinctive software structure featuring parallel pipelined processing and “data flow drive” reduces the system latency to one coherent processing interval (CPI), which significantly improves the maximum tracking angular velocity of the monopulse tracking radar. Additionally, some important electronic counter-countermeasures (ECCM) are incorporated into this signal processing system.

Sign in / Sign up

Export Citation Format

Share Document