scholarly journals Design and Analysis of Binary Scalar Quantizer of Laplacian Source with Applications

Information ◽  
2020 ◽  
Vol 11 (11) ◽  
pp. 501
Author(s):  
Zoran Peric ◽  
Bojan Denic ◽  
Milan Savic ◽  
Vladimir Despotovic
Keyword(s):  
Neural Network ◽  
Dynamic Range ◽  
Mean Squared Error ◽  
Theoretical Models ◽  
Detailed Comparison ◽  
Quantization Noise ◽  
Scalar Quantization ◽  
Model Compression ◽  
Processing Power ◽  
Image Quantization

A compression method based on non-uniform binary scalar quantization, designed for the memoryless Laplacian source with zero-mean and unit variance, is analyzed in this paper. Two quantizer design approaches are presented that investigate the effect of clipping with the aim of reducing the quantization noise, where the minimal mean-squared error distortion is used to determine the optimal clipping factor. A detailed comparison of both models is provided, and the performance evaluation in a wide dynamic range of input data variances is also performed. The observed binary scalar quantization models are applied in standard signal processing tasks, such as speech and image quantization, but also to quantization of neural network parameters. The motivation behind the binary quantization of neural network weights is the model compression by a factor of 32, which is crucial for implementation in mobile or embedded devices with limited memory and processing power. The experimental results follow well the theoretical models, confirming their applicability in real-world applications.

scholarly journals Quantization of Weights of Neural Networks with Negligible Decreasing of Prediction Accuracy

2021 ◽  
Vol 50 (3) ◽  
pp. 558-569
Author(s):  
Zoran Peric ◽  
Bojan Denic ◽  
Milan Savic ◽  
Milan Dincic ◽  
Darko Mihajlov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Prediction Accuracy ◽  
Network Performance ◽  
Dynamic Range ◽  
Mean Squared Error ◽  
Real Data ◽  
Quantization Noise ◽  
Bit Rate ◽  
The Impact

Quantization and compression of neural network parameters using the uniform scalar quantization is carried out in this paper. The attractiveness of the uniform scalar quantizer is reflected in a low complexity and relatively good performance, making it the most popular quantization model. We present a design approach for the memoryless Laplacian source with zero-mean and unit variance, which is based on iterative rule and uses the minimal mean-squared error distortion as a performance criterion. In addition, we derive closed-form expressions for SQNR (Signal to Quantization Noise Ratio) in a wide dynamic range of variance of input data. To show effectiveness on real data, the proposed quantizer is used to compress the weights of neural networks using bit rates from 9 to 16 bps (bits/sample) instead of standardly used 32 bps full precision bit rate. The impact of weights compression on the NN (neural network) performance is analyzed, indicating good matching with the theoretical results and showing negligible decreasing of the prediction accuracy of the NN even in the case of high variance-mismatch between the variance of NN weights and the variance used for the design of quantizer, if the value of the bit-rate is properly chosen according to the rule proposed in the paper.

scholarly journals Groundwater Level Prediction based on Neural Networks: A case study in Linze, Northwestern China

2021 ◽  
Vol 266 ◽  
pp. 09005
Author(s):  
Hui Zhang ◽  
Jixuan Zhao ◽  
Chong Chen
Keyword(s):  
Neural Network ◽  
Bp Neural Network ◽  
Groundwater Level ◽  
Mean Squared Error ◽  
Rbf Neural Network ◽  
Groundwater Resources ◽  
Back Propagation ◽  
Theoretical Models ◽  
Back Propagation Neural Network ◽  
Coefficient Of Determination

Groundwater level is an important factor in evaluating groundwater resources. Due to numerous non-linear factors, establishing theoretical models is difficult.. Therefore, this paper proposesthe BP (Back Propagation) neural network and the Radial Basis Function (RBF) neural network. The study area is divided into two zones. The R2 (coefficient of determination) and RMSE (Root Mean Squared Error) are used to evaluate the performance. The BP neural network is used to predict groundwater level in the two zones with the R2of0.57 and 0.54, with the RMSE of 0.0804 meters and 0.1864 meters respectively. The RBF neural network is implemented with R2of 0.65 and 0.61, with RMSE of 0.0720 meters and 0.1519 meters, respectively. The results show the RBF neural network performs better than the BP neural network in the accuracy of predicting groundwater level. This study shows the feasibility and superiority of groundwater simulation using neural network.

scholarly journals Auto-Colorization of Historical Images Using Deep Convolutional Neural Networks

Mathematics ◽  
2020 ◽  
Vol 8 (12) ◽  
pp. 2258
Author(s):  
Madhab Raj Joshi ◽  
Lewis Nkenyereye ◽  
Gyanendra Prasad Joshi ◽  
S. M. Riazul Islam ◽  
Mohammad Abdullah-Al-Wadud ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
User Study ◽  
Mean Squared Error ◽  
Color Image ◽  
Machine Learning Techniques ◽  
Global Features ◽  
Black And White ◽  
Historical Images ◽  
Learning Techniques

Enhancement of Cultural Heritage such as historical images is very crucial to safeguard the diversity of cultures. Automated colorization of black and white images has been subject to extensive research through computer vision and machine learning techniques. Our research addresses the problem of generating a plausible colored photograph of ancient, historically black, and white images of Nepal using deep learning techniques without direct human intervention. Motivated by the recent success of deep learning techniques in image processing, a feed-forward, deep Convolutional Neural Network (CNN) in combination with Inception- ResnetV2 is being trained by sets of sample images using back-propagation to recognize the pattern in RGB and grayscale values. The trained neural network is then used to predict two a* and b* chroma channels given grayscale, L channel of test images. CNN vividly colorizes images with the help of the fusion layer accounting for local features as well as global features. Two objective functions, namely, Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR), are employed for objective quality assessment between the estimated color image and its ground truth. The model is trained on the dataset created by ourselves with 1.2 K historical images comprised of old and ancient photographs of Nepal, each having 256 × 256 resolution. The loss i.e., MSE, PSNR, and accuracy of the model are found to be 6.08%, 34.65 dB, and 75.23%, respectively. Other than presenting the training results, the public acceptance or subjective validation of the generated images is assessed by means of a user study where the model shows 41.71% of naturalness while evaluating colorization results.

scholarly journals Automatic Modulation Classification Based on Deep Feature Fusion for High Noise Level and Large Dynamic Input

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2117
Author(s):  
Hui Han ◽  
Zhiyuan Ren ◽  
Lin Li ◽  
Zhigang Zhu
Keyword(s):  
Neural Network ◽  
Frequency Domain ◽  
Noise Level ◽  
Background Noise ◽  
Dynamic Range ◽  
Feature Fusion ◽  
Superior Performance ◽  
Modulation Classification ◽  
High Background ◽  
Automatic Modulation Classification

Automatic modulation classification (AMC) is playing an increasingly important role in spectrum monitoring and cognitive radio. As communication and electronic technologies develop, the electromagnetic environment becomes increasingly complex. The high background noise level and large dynamic input have become the key problems for AMC. This paper proposes a feature fusion scheme based on deep learning, which attempts to fuse features from different domains of the input signal to obtain a more stable and efficient representation of the signal modulation types. We consider the complementarity among features that can be used to suppress the influence of the background noise interference and large dynamic range of the received (intercepted) signals. Specifically, the time-series signals are transformed into the frequency domain by Fast Fourier transform (FFT) and Welch power spectrum analysis, followed by the convolutional neural network (CNN) and stacked auto-encoder (SAE), respectively, for detailed and stable frequency-domain feature representations. Considering the complementary information in the time domain, the instantaneous amplitude (phase) statistics and higher-order cumulants (HOC) are extracted as the statistical features for fusion. Based on the fused features, a probabilistic neural network (PNN) is designed for automatic modulation classification. The simulation results demonstrate the superior performance of the proposed method. It is worth noting that the classification accuracy can reach 99.8% in the case when signal-to-noise ratio (SNR) is 0 dB.

scholarly journals Development of a Deep Neural Network Model for Estimating Joint Location of Occupant Indoor Activities for Providing Thermal Comfort

Energies ◽  
2021 ◽  
Vol 14 (3) ◽  
pp. 696
Author(s):  
Eun Ji Choi ◽  
Jin Woo Moon ◽  
Ji-hoon Han ◽  
Yongseok Yoo
Keyword(s):  
Neural Network ◽  
Thermal Comfort ◽  
Deep Neural Network ◽  
Mean Squared Error ◽  
Thermal Environment ◽  
The Body ◽  
Estimation Accuracy ◽  
Accurate Estimation ◽  
Body Parts ◽  
Joint Location

The type of occupant activities is a significantly important factor to determine indoor thermal comfort; thus, an accurate method to estimate occupant activity needs to be developed. The purpose of this study was to develop a deep neural network (DNN) model for estimating the joint location of diverse human activities, which will be used to provide a comfortable thermal environment. The DNN model was trained with images to estimate 14 joints of a person performing 10 common indoor activities. The DNN contained numerous shortcut connections for efficient training and had two stages of sequential and parallel layers for accurate joint localization. Estimation accuracy was quantified using the mean squared error (MSE) for the estimated joints and the percentage of correct parts (PCP) for the body parts. The results show that the joint MSEs for the head and neck were lowest, and the PCP was highest for the torso. The PCP for individual activities ranged from 0.71 to 0.92, while typing and standing in a relaxed manner were the activities with the highest PCP. Estimation accuracy was higher for relatively still activities and lower for activities involving wide-ranging arm or leg motion. This study thus highlights the potential for the accurate estimation of occupant indoor activities by proposing a novel DNN model. This approach holds significant promise for finding the actual type of occupant activities and for use in target indoor applications related to thermal comfort in buildings.

Prognosis of water levels in a moor groundwater system influenced by hydrology and water extraction using an artificial neural network 

2021 ◽  
Author(s):  
Sascha Flaig ◽  
Timothy Praditia ◽  
Alexander Kissinger ◽  
Ulrich Lang ◽  
Sergey Oladyshkin ◽  
...  
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Mean Squared Error ◽  
Meteorological Data ◽  
Water Levels ◽  
Typical Structure ◽  
Water Abstraction ◽  
Physical Knowledge ◽  
Artificial Neural ◽  
The Impact

<p>In order to prevent possible negative impacts of water abstraction in an ecologically sensitive moor south of Munich (Germany), a “predictive control” scheme is in place. We design an artificial neural network (ANN) to provide predictions of moor water levels and to separate hydrological from anthropogenic effects. As the moor is a dynamic system, we adopt the „Long short-term memory“ architecture.</p><p>To find the best LSTM setup, we train, test and compare LSTMs with two different structures: (1) the non-recurrent one-to-one structure, where the series of inputs are accumulated and fed into the LSTM; and (2) the recurrent many-to-many structure, where inputs gradually enter the LSTM (including LSTM forecasts from previous forecast time steps). The outputs of our LSTMs then feed into a readout layer that converts the hidden states into water level predictions. We hypothesize that the recurrent structure is the better structure because it better resembles the typical structure of differential equations for dynamic systems, as they would usually be used for hydro(geo)logical systems. We evaluate the comparison with the mean squared error as test metric, and conclude that the recurrent many-to-many LSTM performs better for the analyzed complex situations. It also produces plausible predictions with reasonable accuracy for seven days prediction horizon.</p><p>Furthermore, we analyze the impact of preprocessing meteorological data to evapotranspiration data using typical ETA models. Inserting knowledge into the LSTM in the form of ETA models (rather than implicitly having the LSTM learn the ETA relations) leads to superior prediction results. This finding aligns well with current ideas on physically-inspired machine learning.</p><p>As an additional validation step, we investigate whether our ANN is able to correctly identify both anthropogenic and natural influences and their interaction. To this end, we investigate two comparable pumping events under different meteorological conditions. Results indicate that all individual and combined influences of input parameters on water levels can be represented well. The neural networks recognize correctly that the predominant precipitation and lower evapotranspiration during one pumping event leads to a lower decrease of the hydrograph.</p><p>To further demonstrate the capability of the trained neural network, scenarios of pumping events are created and simulated.</p><p>In conclusion, we show that more robust and accurate predictions of moor water levels can be obtained if available physical knowledge of the modeled system is used to design and train the neural network. The artificial neural network can be a useful instrument to assess the impact of water abstraction by quantifying the anthropogenic influence.</p>

scholarly journals Using Monte Carlo method to estimate the behavior of neural training between balanced and unbalanced data in classification of patterns

2018 ◽  
Vol 7 (2) ◽  
pp. 1
Author(s):  
Paulo Marcelo Tasinaffo ◽  
Gildárcio Sousa Gonçalves ◽  
Adilson Marques da Cunha ◽  
Luiz Alberto Vieira Dias
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Monte Carlo ◽  
Monte Carlo Method ◽  
Mean Squared Error ◽  
Classification Problem ◽  
Computational Environment ◽  
The Monte Carlo Method ◽  
Proposed Model ◽  
Artificial Neural

This paper proposes to develop a model-based Monte Carlo method for computationally determining the best mean squared error of training for an artificial neural network with feedforward architecture. It is applied for a particular non-linear classification problem of input/output patterns in a computational environment with abundant data. The Monte Carlo method allows computationally checking that balanced data are much better than non-balanced ones for an artificial neural network to learn by means of supervised learning. The major contribution of this investigation is that, the proposed model can be tested by analogy, considering also the fraud detection problem in credit cards, where the amount of training patterns used are high.

scholarly journals Prediksi Indeks Harga Saham Gabungan (IHSG) Menggunakan Algoritma Neural Network

2018 ◽  
Vol 4 (1) ◽  
pp. 24
Author(s):  
Imam Halimi ◽  
Wahyu Andhyka Kusuma
Keyword(s):  
Neural Network ◽  
Data Mining ◽  
Linear Regression ◽  
Mean Squared Error ◽  
Composite Index ◽  
T Test ◽  
Sliding Windows ◽  
Root Mean Squared Error ◽  
Squared Error

Investasi saham merupakan hal yang tidak asing didengar maupun dilakukan. Ada berbagai macam saham di Indonesia, salah satunya adalah Indeks Harga Saham Gabungan (IHSG) atau dalam bahasa inggris disebut Indonesia Composite Index, ICI, atau IDX Composite. IHSG merupakan parameter penting yang dipertimbangkan pada saat akan melakukan investasi mengingat IHSG adalah saham gabungan. Penelitian ini bertujuan memprediksi pergerakan IHSG dengan teknik data mining menggunakan algoritma neural network dan dibandingkan dengan algoritma linear regression, yang dapat dijadikan acuan investor saat akan melakukan investasi. Hasil dari penelitian ini berupa nilai Root Mean Squared Error (RMSE) serta label tambahan angka hasil prediksi yang didapatkan setelah dilakukan validasi menggunakan sliding windows validation dengan hasil paling baik yaitu pada pengujian yang menggunakan algoritma neural network yang menggunakan windowing yaitu sebesar 37,786 dan pada pengujian yang tidak menggunakan windowing sebesar 13,597 dan untuk pengujian algoritma linear regression yang menggunakan windowing yaitu sebesar 35,026 dan pengujian yang tidak menggunakan windowing sebesar 12,657. Setelah dilakukan pengujian T-Test menunjukan bahwa pengujian menggunakan neural network yang dibandingkan dengan linear regression memiliki hasil yang tidak signifikan dengan nilai T-Test untuk pengujian dengan windowing dan tanpa windowing hasilnya sama, yaitu sebesar 1,000.

scholarly journals Forecasting Wind Power Generation Using Artificial Neural Network: “Pawan Danawi”—A Case Study from Sri Lanka

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Amila T. Peiris ◽  
Jeevani Jayasinghe ◽  
Upaka Rathnayake
Keyword(s):  
Neural Network ◽  
Power Generation ◽  
Wind Power ◽  
Wind Farm ◽  
Mean Squared Error ◽  
Wind Power Generation ◽  
Training Algorithms ◽  
Training Algorithm ◽  
Squared Error ◽  
Ann Models

Wind power, as a renewable energy resource, has taken much attention of the energy authorities in many countries, as it is used as one of the major energy sources to satisfy the ever-increasing energy demand. However, careful attention is needed in identifying the wind power potential in a particular area due to climate changes. In this sense, forecasting both wind power generation and wind power potential is essential. This paper develops artificial neural network (ANN) models to forecast wind power generation in “Pawan Danawi”, a functioning wind farm in Sri Lanka. Wind speed, wind direction, and ambient temperature of the area were used as the independent variable matrices of the developed ANN models, while the generated wind power was used as the dependent variable. The models were tested with three training algorithms, namely, Levenberg-Marquardt (LM), Scaled Conjugate Gradient (SCG), and Bayesian Regularization (BR) training algorithms. In addition, the model was calibrated for five validation percentages (5% to 25% in 5% intervals) under each algorithm to identify the best training algorithm with the most suitable training and validation percentages. Mean squared error (MSE), coefficient of correlation (R), root mean squared error ratio (RSR), Nash number, and BIAS were used to evaluate the performance of the developed ANN models. Results revealed that all three training algorithms produce acceptable predictions for the power generation in the Pawan Danawi wind farm with R > 0.91, MSE < 0.22, and BIAS < 1. Among them, the LM training algorithm at 70% of training and 5% of validation percentages produces the best forecasting results. The developed models can be effectively used in the prediction of wind power at the Pawan Danawi wind farm. In addition, the models can be used with the projected climatic scenarios in predicting the future wind power harvest. Furthermore, the models can acceptably be used in similar environmental and climatic conditions to identify the wind power potential of the area.

scholarly journals Influence of Dextransucrase of Weissella Cibaria Nitcsk4 on Low Molecular Weight Dextran Yield: a Statistical Approach using Mixed Level Taguchi Design and Artificial Neural Network

2019 ◽  
Vol 9 (2S2) ◽  
pp. 657-664
Keyword(s):  
Neural Network ◽  
Molecular Weight ◽  
Mean Squared Error ◽  
Sucrose Concentration ◽  
Average Molecular Weight ◽  
Taguchi Design ◽  
Percentage Error ◽  
Low Molecular Weight ◽  
Weight Average Molecular Weight ◽  
Weissella Cibaria

In the present study, the influence of dextransucrase of Weissella cibaria NITCSK4 (DSWc4), sucrose concentration, and reaction temperature on the yield of low molecular weight dextran (LMWD-DexWc4) was investigated using mixed level Taguchi design and back propagation neural network (BPNN). BPNN model with three neurons in a hidden layer generated a low mean squared error (MSE). The determination coefficients (R2 -value) for ANN and Taguchi models were 0.991 and 0.998, respectively. Considering absolute average deviation (AAD) and MSE, Taguchi model is more adequate. Among three factors, the percentage yield of low molecular weight of dextran is invariably dependent on the sucrose concentration. The study suggested that a low sucrose concentration (3% w/v), DSWc4 (0.25 IU/ml) and slightly high temperature (35°C) ultimately favored the production of LMWD-DexWc4 (91.639%). LMW-DexWc4 produced by DSWc4 at optimized conditions was analyzed. The weight average molecular weight of LMW-DexWc4 was calculated using M-H expression, found to be 85775 (≈90 kDa). The relative percentage error between the number and weight average molecular weight was found to be less (4.42%). The polydispersity (PD) index of the LMW-DexWc4 was found to be 0.9576 and the value is close to 1. The PD value depicted that the molecular weight distribution of dextran was narrowly dispersed.

Sign in / Sign up

Export Citation Format

Share Document