scholarly journals Residual Super-Resolution Single Shot Network for Low-Resolution Object Detection

IEEE Access ◽  
2018 ◽  
Vol 6 ◽  
pp. 47780-47793 ◽  
Author(s):  
Xiaotong Zhao ◽  
Wei Li ◽  
Yifan Zhang ◽  
Zhiyong Feng
Author(s):  
Jakaria Rabbi ◽  
Nilanjan Ray ◽  
Matthias Schubert ◽  
Subir Chowdhury ◽  
Dennis Chao

The detection performance of small objects in remote sensing images is not satisfactory compared to large objects, especially in low-resolution and noisy images. A generative adversarial network (GAN)-based model called enhanced super-resolution GAN (ESRGAN) shows remarkable image enhancement performance, but reconstructed images miss high-frequency edge information. Therefore, object detection performance degrades for the small objects on recovered noisy and low-resolution remote sensing images. Inspired by the success of edge enhanced GAN (EEGAN) and ESRGAN, we apply a new edge-enhanced super-resolution GAN (EESRGAN) to improve the image quality of remote sensing images and used different detector networks in an end-to-end manner where detector loss is backpropagated into the EESRGAN to improve the detection performance. We propose an architecture with three components: ESRGAN, Edge Enhancement Network (EEN), and Detection network. We use residual-in-residual dense blocks (RRDB) for both the GAN and EEN, and for the detector network, we use the faster region-based convolutional network (FRCNN) (two-stage detector) and single-shot multi-box detector (SSD) (one stage detector). Extensive experiments on car overhead with context and oil and gas storage tank (created by us) data sets show superior performance of our method compared to the standalone state-of-the-art object detectors.


Author(s):  
Jakaria Rabbi ◽  
Nilanjan Ray ◽  
Matthias Schubert ◽  
Subir Chowdhury ◽  
Dennis Chao

The detection performance of small objects in remote sensing images is not satisfactory compared to large objects, especially in low-resolution and noisy images. A generative adversarial network (GAN)-based model called enhanced super-resolution GAN (ESRGAN) shows remarkable image enhancement performance, but reconstructed images miss high-frequency edge information. Therefore, object detection performance degrades for small objects on recovered noisy and low-resolution remote sensing images. Inspired by the success of edge enhanced GAN (EEGAN) and ESRGAN, we apply a new edge-enhanced super-resolution GAN (EESRGAN) to improve the image quality of remote sensing images and use different detector networks in an end-to-end manner where detector loss is backpropagated into the EESRGAN to improve the detection performance. We propose an architecture with three components: ESRGAN, Edge Enhancement Network (EEN), and Detection network. We use residual-in-residual dense blocks (RRDB) for both the ESRGAN and EEN, and for the detector network, we use the faster region-based convolutional network (FRCNN) (two-stage detector) and single-shot multi-box detector (SSD) (one stage detector). Extensive experiments on a public (car overhead with context) and a self-assembled (oil and gas storage tank) satellite dataset show superior performance of our method compared to the standalone state-of-the-art object detectors.


Author(s):  
Jakaria Rabbi ◽  
Nilanjan Ray ◽  
Matthias Schubert ◽  
Subir Chowdhury ◽  
Dennis Chao

The detection performance of small objects in remote sensing images has not been satisfactory compared to large objects, especially in low-resolution and noisy images. A generative adversarial network (GAN)-based model called enhanced super-resolution GAN (ESRGAN) showed remarkable image enhancement performance, but reconstructed images usually miss high-frequency edge information. Therefore, object detection performance showed degradation for small objects on recovered noisy and low-resolution remote sensing images. Inspired by the success of edge enhanced GAN (EEGAN) and ESRGAN, we applied a new edge-enhanced super-resolution GAN (EESRGAN) to improve the quality of remote sensing images and used different detector networks in an end-to-end manner where detector loss was backpropagated into the EESRGAN to improve the detection performance. We proposed an architecture with three components: ESRGAN, EEN, and Detection network. We used residual-in-residual dense blocks (RRDB) for both the ESRGAN and EEN, and for the detector network, we used a faster region-based convolutional network (FRCNN) (two-stage detector) and a single-shot multibox detector (SSD) (one stage detector). Extensive experiments on a public (car overhead with context) dataset and another self-assembled (oil and gas storage tank) satellite dataset showed superior performance of our method compared to the standalone state-of-the-art object detectors.


2020 ◽  
Vol 12 (9) ◽  
pp. 1432 ◽  
Author(s):  
Jakaria Rabbi ◽  
Nilanjan Ray ◽  
Matthias Schubert ◽  
Subir Chowdhury ◽  
Dennis Chao

The detection performance of small objects in remote sensing images has not been satisfactory compared to large objects, especially in low-resolution and noisy images. A generative adversarial network (GAN)-based model called enhanced super-resolution GAN (ESRGAN) showed remarkable image enhancement performance, but reconstructed images usually miss high-frequency edge information. Therefore, object detection performance showed degradation for small objects on recovered noisy and low-resolution remote sensing images. Inspired by the success of edge enhanced GAN (EEGAN) and ESRGAN, we applied a new edge-enhanced super-resolution GAN (EESRGAN) to improve the quality of remote sensing images and used different detector networks in an end-to-end manner where detector loss was backpropagated into the EESRGAN to improve the detection performance. We proposed an architecture with three components: ESRGAN, EEN, and Detection network. We used residual-in-residual dense blocks (RRDB) for both the ESRGAN and EEN, and for the detector network, we used a faster region-based convolutional network (FRCNN) (two-stage detector) and a single-shot multibox detector (SSD) (one stage detector). Extensive experiments on a public (car overhead with context) dataset and another self-assembled (oil and gas storage tank) satellite dataset showed superior performance of our method compared to the standalone state-of-the-art object detectors.


2021 ◽  
pp. 387-395
Author(s):  
Muhammad Haris ◽  
Greg Shakhnarovich ◽  
Norimichi Ukita

Symmetry ◽  
2020 ◽  
Vol 12 (10) ◽  
pp. 1718
Author(s):  
Chien-Hsing Chou ◽  
Yu-Sheng Su ◽  
Che-Ju Hsu ◽  
Kong-Chang Lee ◽  
Ping-Hsuan Han

In this study, we designed a four-dimensional (4D) audiovisual entertainment system called Sense. This system comprises a scene recognition system and hardware modules that provide haptic sensations for users when they watch movies and animations at home. In the scene recognition system, we used Google Cloud Vision to detect common scene elements in a video, such as fire, explosions, wind, and rain, and further determine whether the scene depicts hot weather, rain, or snow. Additionally, for animated videos, we applied deep learning with a single shot multibox detector to detect whether the animated video contained scenes of fire-related objects. The hardware module was designed to provide six types of haptic sensations set as line-symmetry to provide a better user experience. After the system considers the results of object detection via the scene recognition system, the system generates corresponding haptic sensations. The system integrates deep learning, auditory signals, and haptic sensations to provide an enhanced viewing experience.


2021 ◽  
Vol 13 (10) ◽  
pp. 1956
Author(s):  
Jingyu Cong ◽  
Xianpeng Wang ◽  
Xiang Lan ◽  
Mengxing Huang ◽  
Liangtian Wan

The traditional frequency-modulated continuous wave (FMCW) multiple-input multiple-output (MIMO) radar two-dimensional (2D) super-resolution (SR) estimation algorithm for target localization has high computational complexity, which runs counter to the increasing demand for real-time radar imaging. In this paper, a fast joint direction-of-arrival (DOA) and range estimation framework for target localization is proposed; it utilizes a very deep super-resolution (VDSR) neural network (NN) framework to accelerate the imaging process while ensuring estimation accuracy. Firstly, we propose a fast low-resolution imaging algorithm based on the Nystrom method. The approximate signal subspace matrix is obtained from partial data, and low-resolution imaging is performed on a low-density grid. Then, the bicubic interpolation algorithm is used to expand the low-resolution image to the desired dimensions. Next, the deep SR network is used to obtain the high-resolution image, and the final joint DOA and range estimation is achieved based on the reconstructed image. Simulations and experiments were carried out to validate the computational efficiency and effectiveness of the proposed framework.


2011 ◽  
Author(s):  
G. G. Manahan ◽  
E. Brunetti ◽  
R. P. Shanks ◽  
M. R. Islam ◽  
B. Ersfeld ◽  
...  

Sign in / Sign up

Export Citation Format

Share Document