scholarly journals Mask-Refined R-CNN: A Network for Refining Object Details in Instance Segmentation

Sensors ◽  
2020 ◽  
Vol 20 (4) ◽  
pp. 1010 ◽  
Author(s):  
Yiqing Zhang ◽  
Jun Chu ◽  
Lu Leng ◽  
Jun Miao
Keyword(s):  
Receptive Field ◽  
Spatial Information ◽  
Feature Fusion ◽  
State Of The Art ◽  
Rapid Development ◽  
Experimental Results ◽  
Small Scale ◽  
Feature Maps ◽  
Segmentation Accuracy ◽  
Instance Segmentation

With the rapid development of flexible vision sensors and visual sensor networks, computer vision tasks, such as object detection and tracking, are entering a new phase. Accordingly, the more challenging comprehensive task, including instance segmentation, can develop rapidly. Most state-of-the-art network frameworks, for instance, segmentation, are based on Mask R-CNN (mask region-convolutional neural network). However, the experimental results confirm that Mask R-CNN does not always successfully predict instance details. The scale-invariant fully convolutional network structure of Mask R-CNN ignores the difference in spatial information between receptive fields of different sizes. A large-scale receptive field focuses more on detailed information, whereas a small-scale receptive field focuses more on semantic information. So the network cannot consider the relationship between the pixels at the object edge, and these pixels will be misclassified. To overcome this problem, Mask-Refined R-CNN (MR R-CNN) is proposed, in which the stride of ROIAlign (region of interest align) is adjusted. In addition, the original fully convolutional layer is replaced with a new semantic segmentation layer that realizes feature fusion by constructing a feature pyramid network and summing the forward and backward transmissions of feature maps of the same resolution. The segmentation accuracy is substantially improved by combining the feature layers that focus on the global and detailed information. The experimental results on the COCO (Common Objects in Context) and Cityscapes datasets demonstrate that the segmentation accuracy of MR R-CNN is about 2% higher than that of Mask R-CNN using the same backbone. The average precision of large instances reaches 56.6%, which is higher than those of all state-of-the-art methods. In addition, the proposed method requires low time cost and is easily implemented. The experiments on the Cityscapes dataset also prove that the proposed method has great generalization ability.

scholarly journals Split to Be Slim: An Overlooked Redundancy in Vanilla Convolution

2020 ◽  
Author(s):  
Qiulin Zhang ◽  
Zhuqing Jiang ◽  
Qishuo Lu ◽  
Jia'nan Han ◽  
Zhengxin Zeng ◽  
...  
Keyword(s):  
Feature Fusion ◽  
State Of The Art ◽  
Experimental Results ◽  
Light Weight ◽  
Feature Maps ◽  
Plug And Play ◽  
Input Feature

Many effective solutions have been proposed to reduce the redundancy of models for inference acceleration. Nevertheless, common approaches mostly focus on eliminating less important filters or constructing efficient operations, while ignoring the pattern redundancy in feature maps. We reveal that many feature maps within a layer share similar but not identical patterns. However, it is difficult to identify if features with similar patterns are redundant or contain essential details. Therefore, instead of directly removing uncertain redundant features, we propose a split based convolutional operation, namely SPConv, to tolerate features with similar patterns but require less computation. Specifically, we split input feature maps into the representative part and the uncertain redundant part, where intrinsic information is extracted from the representative part through relatively heavy computation while tiny hidden details in the uncertain redundant part are processed with some light-weight operation. To recalibrate and fuse these two groups of processed features, we propose a parameters-free feature fusion module. Moreover, our SPConv is formulated to replace the vanilla convolution in a plug-and-play way. Without any bells and whistles, experimental results on benchmarks demonstrate SPConv-equipped networks consistently outperform state-of-the-art baselines in both accuracy and inference time on GPU, with FLOPs and parameters dropped sharply.

scholarly journals Accurate Instance-Based Segmentation for Boundary Detection in Robot Grasping Application

2021 ◽  
Vol 11 (9) ◽  
pp. 4248
Author(s):  
Hong Hai Hoang ◽  
Bao Long Tran
Keyword(s):  
Object Segmentation ◽  
State Of The Art ◽  
Rapid Development ◽  
Spatial Relationship ◽  
Learning Technologies ◽  
Average Precision ◽  
Novel Approach ◽  
3D Camera ◽  
Robot Grasping ◽  
Instance Segmentation

With the rapid development of cameras and deep learning technologies, computer vision tasks such as object detection, object segmentation and object tracking are being widely applied in many fields of life. For robot grasping tasks, object segmentation aims to classify and localize objects, which helps robots to be able to pick objects accurately. The state-of-the-art instance segmentation network framework, Mask Region-Convolution Neural Network (Mask R-CNN), does not always perform an excellent accurate segmentation at the edge or border of objects. The approach using 3D camera, however, is able to extract the entire (foreground) objects easily but can be difficult or require a large amount of computation effort to classify it. We propose a novel approach, in which we combine Mask R-CNN with 3D algorithms by adding a 3D process branch for instance segmentation. Both outcomes of two branches are contemporaneously used to classify the pixels at the edge objects by dealing with the spatial relationship between edge region and mask region. We analyze the effectiveness of the method by testing with harsh cases of object positions, for example, objects are closed, overlapped or obscured by each other to focus on edge and border segmentation. Our proposed method is about 4 to 7% higher and more stable in IoU (intersection of union). This leads to a reach of 46% of mAP (mean Average Precision), which is a higher accuracy than its counterpart. The feasibility experiment shows that our method could be a remarkable promoting for the research of the grasping robot.

scholarly journals A Multi-Branch Feature Fusion Strategy Based on an Attention Mechanism for Remote Sensing Image Scene Classification

2021 ◽  
Vol 13 (10) ◽  
pp. 1950
Author(s):  
Cuiping Shi ◽  
Xin Zhao ◽  
Liguo Wang
Keyword(s):  
Remote Sensing ◽  
Feature Extraction ◽  
Classification Accuracy ◽  
Feature Fusion ◽  
State Of The Art ◽  
Rapid Development ◽  
Remote Sensing Image ◽  
Classification Performance ◽  
Attention Mechanism ◽  
Scene Classification

In recent years, with the rapid development of computer vision, increasing attention has been paid to remote sensing image scene classification. To improve the classification performance, many studies have increased the depth of convolutional neural networks (CNNs) and expanded the width of the network to extract more deep features, thereby increasing the complexity of the model. To solve this problem, in this paper, we propose a lightweight convolutional neural network based on attention-oriented multi-branch feature fusion (AMB-CNN) for remote sensing image scene classification. Firstly, we propose two convolution combination modules for feature extraction, through which the deep features of images can be fully extracted with multi convolution cooperation. Then, the weights of the feature are calculated, and the extracted deep features are sent to the attention mechanism for further feature extraction. Next, all of the extracted features are fused by multiple branches. Finally, depth separable convolution and asymmetric convolution are implemented to greatly reduce the number of parameters. The experimental results show that, compared with some state-of-the-art methods, the proposed method still has a great advantage in classification accuracy with very few parameters.

scholarly journals RFaNet: Receptive Field-Aware Network with Finger Attention for Fingerspelling Recognition Using a Depth Sensor

Mathematics ◽  
2021 ◽  
Vol 9 (21) ◽  
pp. 2815
Author(s):  
Shih-Hung Yang ◽  
Yao-Mao Cheng ◽  
Jyun-We Huang ◽  
Yon-Ping Chen
Keyword(s):  
Receptive Field ◽  
American Sign Language ◽  
Multiple Scales ◽  
Receptive Fields ◽  
Depth Image ◽  
Small Scale ◽  
Feature Maps ◽  
Depth Sensor ◽  
Depth Images ◽  
Effective Transfer

Automatic fingerspelling recognition tackles the communication barrier between deaf and hearing individuals. However, the accuracy of fingerspelling recognition is reduced by high intra-class variability and low inter-class variability. In the existing methods, regular convolutional kernels, which have limited receptive fields (RFs) and often cannot detect subtle discriminative details, are applied to learn features. In this study, we propose a receptive field-aware network with finger attention (RFaNet) that highlights the finger regions and builds inter-finger relations. To highlight the discriminative details of these fingers, RFaNet reweights the low-level features of the hand depth image with those of the non-forearm image and improves finger localization, even when the wrist is occluded. RFaNet captures neighboring and inter-region dependencies between fingers in high-level features. An atrous convolution procedure enlarges the RFs at multiple scales and a non-local operation computes the interactions between multi-scale feature maps, thereby facilitating the building of inter-finger relations. Thus, the representation of a sign is invariant to viewpoint changes, which are primarily responsible for intra-class variability. On an American Sign Language fingerspelling dataset, RFaNet achieved 1.77% higher classification accuracy than state-of-the-art methods. RFaNet achieved effective transfer learning when the number of labeled depth images was insufficient. The fingerspelling representation of a depth image can be effectively transferred from large- to small-scale datasets via highlighting the finger regions and building inter-finger relations, thereby reducing the requirement for expensive fingerspelling annotations.

Attention U-Net with Feature Fusion Module for Robust Defect Detection

2021 ◽  
pp. 2150272
Author(s):  
Yu-Jie Xiong ◽  
Yong-Bin Gao ◽  
Hong Wu ◽  
Yao Yao
Keyword(s):  
Defect Detection ◽  
Crack Detection ◽  
Feature Fusion ◽  
Experimental Results ◽  
Feature Maps ◽  
Concrete Crack ◽  
Disaster Situation ◽  
Unpredictable Environment ◽  
Segmentation Task ◽  
Noisy Background

U-Net shows a remarkable performance and makes significant progress for segmentation task in medical images. Despite the outstanding achievements, the common case of defect detection in industrial scenes is still a challenging task, due to the noisy background, unpredictable environment, varying shapes and sizes of the defects. Traditional U-Net may not be suitable for low-quality images with low illumination and corruption, which are often presented in the practical collections in real-world scenes. In this paper, we propose an attention U-Net with feature fusion module for combining multi-scale features to detect the defects in noisy images automatically. Feature fusion module contains convolution kernels of different scales to capture shallow layer features and combine them with the high-dimensional features. Meanwhile, attention gates are used to enhance the robustness of skip connection between the feature maps. The proposed method is evaluated on two datasets. The best precision rate and MIoU of defect detection are 95.6% and 92.5%. The best F-score of concrete crack detection is 95.0%. Experimental results show that the proposed approach achieves promising results in both datasets. It demonstrates that our approach consistently outperforms other U-Net-based approaches for defect detection in low-quality images. Experimental results have shown the possibility of developing a mixture system that can be deployed in many applications, such as remote sensing image analysis, earthquake disaster situation assessment, and so on.

scholarly journals Universal Word Segmentation: Implementation and Interpretation

2018 ◽  
Vol 6 ◽  
pp. 421-435 ◽  
Author(s):  
Yan Shao ◽  
Christian Hardmeier ◽  
Joakim Nivre
Keyword(s):  
State Of The Art ◽  
Word Segmentation ◽  
Experimental Results ◽  
Word Boundary ◽  
Writing Systems ◽  
Low Level ◽  
Wide Range ◽  
Segmentation Accuracy ◽  
Small Set

Word segmentation is a low-level NLP task that is non-trivial for a considerable number of languages. In this paper, we present a sequence tagging framework and apply it to word segmentation for a wide range of languages with different writing systems and typological characteristics. Additionally, we investigate the correlations between various typological factors and word segmentation accuracy. The experimental results indicate that segmentation accuracy is positively related to word boundary markers and negatively to the number of unique non-segmental terms. Based on the analysis, we design a small set of language-specific settings and extensively evaluate the segmentation system on the Universal Dependencies datasets. Our model obtains state-of-the-art accuracies on all the UD languages. It performs substantially better on languages that are non-trivial to segment, such as Chinese, Japanese, Arabic and Hebrew, when compared to previous work.

scholarly journals Integration of Convolutional Neural Network and Error Correction for Indoor Positioning

2020 ◽  
Vol 9 (2) ◽  
pp. 74
Author(s):  
Eric Hsueh-Chan Lu ◽  
Jing-Mei Ciou
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Error Correction ◽  
Spatial Information ◽  
Rapid Development ◽  
Indoor Positioning ◽  
Experimental Results ◽  
Global Navigation Satellite Systems ◽  
Signal Interference ◽  
Position Vector

With the rapid development of surveying and spatial information technologies, more and more attention has been given to positioning. In outdoor environments, people can easily obtain positioning services through global navigation satellite systems (GNSS). In indoor environments, the GNSS signal is often lost, while other positioning problems, such as dead reckoning and wireless signals, will face accumulated errors and signal interference. Therefore, this research uses images to realize a positioning service. The main concept of this work is to establish a model for an indoor field image and its coordinate information and to judge its position by image eigenvalue matching. Based on the architecture of PoseNet, the image is input into a 23-layer convolutional neural network according to various sizes to train end-to-end location identification tasks, and the three-dimensional position vector of the camera is regressed. The experimental data are taken from the underground parking lot and the Palace Museum. The preliminary experimental results show that this new method designed by us can effectively improve the accuracy of indoor positioning by about 20% to 30%. In addition, this paper also discusses other architectures, field sizes, camera parameters, and error corrections for this neural network system. The preliminary experimental results show that the angle error correction method designed by us can effectively improve positioning by about 20%.

scholarly journals Small-Scale Face Detection Based on Improved R-FCN

2020 ◽  
Vol 10 (12) ◽  
pp. 4177
Author(s):  
Chaowei Tang ◽  
Shiyu Chen ◽  
Xu Zhou ◽  
Shuai Ruan ◽  
Haotian Wen
Keyword(s):  
Receptive Field ◽  
Face Detection ◽  
Feature Fusion ◽  
Selection Method ◽  
Local Information ◽  
Small Scale ◽  
Average Precision ◽  
Convolutional Network ◽  
High Layer ◽  
Novel Method

Face detection is an important basic technique for face-related applications, such as face analysis, recognition, and reconstruction. Images in unconstrained scenes may contain many small-scale faces. The features that the detector can extract from small-scale faces are limited, which will cause missed detection and greatly reduce the precision of face detection. Therefore, this study proposes a novel method to detect small-scale faces based on region-based fully convolutional network (R-FCN). First, we propose a novel R-FCN framework with the ability of feature fusion and receptive field adaptation. Second, a bottom-up feature fusion branch is established to enrich the local information of high-layer features. Third, a receptive field adaptation block (RFAB) is proposed to ensure that the receptive field can be adaptively selected to strengthen the expression ability of features. Finally, we improve the anchor setting method and adopt soft non-maximum suppression (SoftNMS) as the selection method of candidate boxes. Experimental results show that average precision for small-scale face detection of R-FCN with feature fusion branch and RFAB (RFAB-f-R-FCN) is improved by 0.8%, 2.9%, and 11% on three subsets of Wider Face compared with that of R-FCN.

scholarly journals SE-IYOLOV3: An Accurate Small Scale Face Detector for Outdoor Security

Mathematics ◽  
2020 ◽  
Vol 8 (1) ◽  
pp. 93 ◽  
Author(s):  
Zhenrong Deng ◽  
Rui Yang ◽  
Rushi Lan ◽  
Zhenbing Liu ◽  
Xiaonan Luo
Keyword(s):  
Receptive Field ◽  
Face Detection ◽  
Network Structure ◽  
Difficult Problem ◽  
Detection Performance ◽  
Experimental Results ◽  
Small Scale ◽  
Detection Accuracy ◽  
Novel Method ◽  
Face Detector

Small scale face detection is a very difficult problem. In order to achieve a higher detection accuracy, we propose a novel method, termed SE-IYOLOV3, for small scale face in this work. In SE-IYOLOV3, we improve the YOLOV3 first, in which the anchorage box with a higher average intersection ratio is obtained by combining niche technology on the basis of the k-means algorithm. An upsampling scale is added to form a face network structure that is suitable for detecting dense small scale faces. The number of prediction boxes is five times more than the YOLOV3 network. To further improve the detection performance, we adopt the SENet structure to enhance the global receptive field of the network. The experimental results on the WIDERFACEdataset show that the IYOLOV3 network embedded in the SENet structure can significantly improve the detection accuracy of dense small scale faces.

scholarly journals DAR-Net: Dense Attentional Residual Network for Vehicle Detection in Aerial Images

2021 ◽  
Vol 2021 ◽  
pp. 1-19
Author(s):  
Kaifeng Li ◽  
Bin Wang
Keyword(s):  
Object Detection ◽  
Information Flow ◽  
Spatial Information ◽  
Rapid Development ◽  
Vehicle Detection ◽  
Aerial Images ◽  
Feature Maps ◽  
Residual Network ◽  
General Object ◽  
The Impact

With the rapid development of deep learning and the wide usage of Unmanned Aerial Vehicles (UAVs), CNN-based algorithms of vehicle detection in aerial images have been widely studied in the past several years. As a downstream task of the general object detection, there are some differences between the vehicle detection in aerial images and the general object detection in ground view images, e.g., larger image areas, smaller target sizes, and more complex background. In this paper, to improve the performance of this task, a Dense Attentional Residual Network (DAR-Net) is proposed. The proposed network employs a novel dense waterfall residual block (DW res-block) to effectively preserve the spatial information and extract high-level semantic information at the same time. A multiscale receptive field attention (MRFA) module is also designed to select the informative feature from the feature maps and enhance the ability of multiscale perception. Based on the DW res-block and MRFA module, to protect the spatial information, the proposed framework adopts a new backbone that only downsamples the feature map 3 times; i.e., the total downsampling ratio of the proposed backbone is 8. These designs could alleviate the degradation problem, improve the information flow, and strengthen the feature reuse. In addition, deep-projection units are used to reduce the impact of information loss caused by downsampling operations, and the identity mapping is applied to each stage of the proposed backbone to further improve the information flow. The proposed DAR-Net is evaluated on VEDAI, UCAS-AOD, and DOTA datasets. The experimental results demonstrate that the proposed framework outperforms other state-of-the-art algorithms.

Sign in / Sign up

Export Citation Format

Share Document