Efficient External Sorting for Memory-Constrained Embedded Devices with Flash Memory

2021 ◽  
Vol 20 (4) ◽  
pp. 1-21
Author(s):  
Riley Jackson ◽  
Jonathan Gresl ◽  
Ramon Lawrence
Keyword(s):  
Data Collection ◽  
Flash Memory ◽  
Health And Safety ◽  
Sorting Algorithm ◽  
Output Buffer ◽  
Embedded Devices ◽  
Sorting Data ◽  
Limited Application ◽  
External Sorting ◽  
Merge Sort

Embedded devices are ubiquitous in areas of industrial and environmental monitoring, health and safety, and consumer appliances. A common use case is data collection, processing, and performing actions based on data analysis. Although many Internet of Things (IoT) applications use the embedded device simply for data collection, there are benefits to having more data processing done closer to data collection to reduce network transmissions and power usage and provide faster response. This work implements and evaluates algorithms for sorting data on embedded devices with specific focus on the smallest memory devices. In devices with less than 4 KB of available RAM, the standard external merge sort algorithm has limited application as it requires a minimum of three memory buffers and is not flash-aware. The contribution is a memory-optimized external sorting algorithm called no output buffer sort (NOBsort) that reduces the minimum memory required for sorting, has excellent performance for sorted or near-sorted data, and sorts on external memory such as SD cards or raw flash chips. When sorting large datasets, no output buffer sort reduces I/O and execution time by between 20% to 35% compared to standard external merge sort.

scholarly journals Perbandingan Efisiensi Algoritma Sorting dalam Penggunaan Bandwidth

2020 ◽  
Vol 12 (2) ◽  
pp. 96-103
Author(s):  
Desi Anggreani ◽  
Aji Prasetya Wibawa ◽  
Purnawansyah Purnawansyah ◽  
Herman Herman
Keyword(s):  
Execution Time ◽  
The Other ◽  
Sorting Algorithm ◽  
Memory Usage ◽  
Data Types ◽  
Network Bandwidth ◽  
Sorting Algorithms ◽  
Sorting Data ◽  
Merge Sort ◽  
Sort Algorithm

The most used algorithm is the sorting algorithm. There have been many popping sorting algorithms that can be used, in this study researchers took three sorting algorithms namely Insertion Sort, Selection Sort, and Merge Sort. As for this study will analyze the comparison of execution time and memory usage by considering the number of enter data of each algorithm used. The data used in this study is ukhuwah NET network bandwidth usage data connected in the Faculty of Computer Science in the form of double data types. After implementing and analyzing in terms of execution time merge sort algorithm has a faster execution time in sorting data with an average execution time value of 108.593777 ms on the 3000 data count. While in the same amount of data for the most execution time is the Selection Sort algorithm with a large execution time of 144.498144 ms, in terms of memory usage with the amount of data3000 Merge Sort Algorithm has the highest memory usage compared to the other two algorithms which is 21,444 MB while the other two algorithms have a succession of memory usage of 20,837 MB and 20,325MB.

PEMANFAATAN PERALATAN KESELAMATAN DAN KESEHATAN KERJA (K3) PADA PRAKTIKUM PROSES PRODUKSI

2020 ◽  
Vol 1 (2) ◽  
pp. 64-70
Author(s):  
Rezky Fajar Ramadhan
Keyword(s):  
Data Collection ◽  
Occupational Health ◽  
Engineering Education ◽  
Production Process ◽  
Mechanical Engineering ◽  
Health And Safety ◽  
Occupational Health And Safety ◽  
Education Students ◽  
Descriptive Data

Abstrak: Tujuan penelitian ini adalah untuk mendeskripsikan penggunaan peralatan Kesehatan dan Keselamatan Kerja (K3) pada Mata Kuliah Praktikum Proses Produksi mahasiswa Pendidikan Teknik Mesin, Universitas Palangka Raya. Jenis penelitian ini adalah deskriptif. Teknik pengumpulan data menggunakan teknik angket dan dokumentasi. Objek penelitian adalah seluruh mahasiswa Pendidikan Teknik Mesin, Universitas Palangka Raya yang memprogramkan Mata Kuliah Praktikum Proses Produksi. Hasil penelitian ini menunjukkan bahwa 43% mahasiswa menyatakan sangat baik dan 57% mahasiswa menyatakan baik penggunaan peralatan K3 pada Mata Kuliah Praktikum Proses Produksi. Sehingga, disimpulkan bahwa penerapan K3 dapat mendukung pelaksanaan praktikum proses produksi.   Abstract: The purpose of this study was to describe the use of Occupational Health and Safety (OHS) equipment in the Production Process Practicum Subject of Mechanical Engineering Education students, Palangka Raya University. This type of research was descriptive. Data collection techniques used questionnaires and documentation. The object of research was all students of Mechanical Engineering Education, Palangkaraya University who programed Production Process Practicum Subjects. The results of this study indicate that 43% of students stated very well and 57% of students stated good use of OHS equipment in the Production Process Practicum Course. Thus, it was concluded that the application of OHS can support the implementation of practicum in the production process.

Analisis Kinerja Algoritma Quick Double Merge Sort Paralel Menggunakan openMP

2020 ◽  
Vol 11 (2) ◽  
pp. 95-102
Author(s):  
I Nyoman Aditya Yudiswara ◽  
Abba Suganda
Keyword(s):  
Parallel Algorithms ◽  
The Other ◽  
Sorting Algorithm ◽  
Data Parallelism ◽  
Parallel Sorting ◽  
Sequential Algorithms ◽  
Sorting Algorithms ◽  
Merge Sort ◽  
Sort Algorithm ◽  
Quick Sort

Processor technology currently tends to increase the number of cores more than increasing the clock speed. This development is very useful and becomes an opportunity to improve the performance of sequential algorithms that are only done by one core. This paper discusses the sorting algorithm that is executed in parallel by several logical CPUs or cores using the openMP library. This algorithm is named QDM Sort which is a combination of sequential quick sort algorithm and double merge algorithm. This study uses a data parallelism approach to design parallel algorithms from sequential algorithms. The data used in this study are the data that have not been sorted and also the data that has been sorted is integer type which is stored in advance in a file. The parameter measured to determine the performance of the QDM Sort algorithm is speedup. In a condition where a large amount of data is above 4096 and the number of threads in QDM Sort is the same as the number of logical CPUs, the QDM Sort algorithm has a better speedup compared to the other parallel sorting algorithms discussed in this study. For small amounts of data it is still better to use sequential sorting algorithm.

An external sorting algorithm

Cybernetics ◽  
1968 ◽  
Vol 2 (5) ◽  
pp. 75-79
Author(s):  
V. A. Litvinov
Keyword(s):  
Sorting Algorithm ◽  
External Sorting

scholarly journals BioSeqZip: a collapser of NGS redundant reads for the optimization of sequence analysis

2020 ◽  
Vol 36 (9) ◽  
pp. 2705-2711 ◽  
Author(s):  
Gianvito Urgese ◽  
Emanuele Parisi ◽  
Orazio Scicolone ◽  
Santa Di Cataldo ◽  
Elisa Ficarra
Keyword(s):  
Sequence Analysis ◽  
Supplementary Information ◽  
Sorting Algorithm ◽  
Rna Seq ◽  
Compact Sets ◽  
Analysis Pipeline ◽  
Alignment Algorithms ◽  
External Sorting ◽  
Computational Resources ◽  
Generation Sequencing

Abstract Motivation High-throughput next-generation sequencing can generate huge sequence files, whose analysis requires alignment algorithms that are typically very demanding in terms of memory and computational resources. This is a significant issue, especially for machines with limited hardware capabilities. As the redundancy of the sequences typically increases with coverage, collapsing such files into compact sets of non-redundant reads has the 2-fold advantage of reducing file size and speeding-up the alignment, avoiding to map the same sequence multiple times. Method BioSeqZip generates compact and sorted lists of alignment-ready non-redundant sequences, keeping track of their occurrences in the raw files as well as of their quality score information. By exploiting a memory-constrained external sorting algorithm, it can be executed on either single- or multi-sample datasets even on computers with medium computational capabilities. On request, it can even re-expand the compacted files to their original state. Results Our extensive experiments on RNA-Seq data show that BioSeqZip considerably brings down the computational costs of a standard sequence analysis pipeline, with particular benefits for the alignment procedures that typically have the highest requirements in terms of memory and execution time. In our tests, BioSeqZip was able to compact 2.7 billion of reads into 963 million of unique tags reducing the size of sequence files up to 70% and speeding-up the alignment by 50% at least. Availability and implementation BioSeqZip is available at https://github.com/bioinformatics-polito/BioSeqZip. Supplementary information Supplementary data are available at Bioinformatics online.

LOFFS: A Low-Overhead File System for Large Flash Memory on Embedded Devices

2020 ◽  
Author(s):  
Runyu Zhang ◽  
Duo Liu ◽  
Xianzhang Chen ◽  
Xiongxiong She ◽  
Chaoshu Yang ◽  
...  
Keyword(s):  
Flash Memory ◽  
File System ◽  
Embedded Devices

Identifying and prioritizing gaming workers' health and safety concerns using mapping for data collection

2001 ◽  
Vol 39 (1) ◽  
pp. 42-51 ◽  
Author(s):  
Margaret M. Keith ◽  
Beverley Cann ◽  
James T. Brophy ◽  
Deborah Hellyer ◽  
Margaret Day ◽  
...  
Keyword(s):  
Data Collection ◽  
Health And Safety ◽  
Safety Concerns

Safety and Health in Refurbishment Works Including Partial Demolition

2015 ◽  
Vol 735 ◽  
pp. 99-103
Author(s):  
Mansooreh Rakhshanifar ◽  
M. Hosseini ◽  
Arham Abdullah
Keyword(s):  
Data Collection ◽  
Literature Review ◽  
Health And Safety ◽  
Structured Interview ◽  
Work Site ◽  
Safety Regulations ◽  
Safety And Health ◽  
The Common ◽  
The Moment

In the context of refurbishment projects, providing a safety and health checklist could assist in reducing noncompliance with health and safety regulations and would lead to improvement in communication between different participants in refurbishment projects. The practical checklist for refurbishment project is developed by revising current demolition checklist since there is no specific checklist applicable for refurbishment work at the moment. To achieve this target, demolition techniques, methods and accident reports on the refurbishment project is critically reviewed and investigated. A semi-structured interview based on reviewed literature and existing demolition checklist is organized which addresses a panel of Malaysian refurbishment experts and workers. The final results are provided by analyzing the response of interviewees in listing the common hazards occurring in the site and their recommendations regarding to appropriate precautions to prevent accidents in work site. Finally, refurbishment checklist is developed based on different categories of work phase, activities, precautionary actions and recommendations in which contents directly extracted from data collection and literature review.

Sign in / Sign up

Export Citation Format

Share Document