scholarly journals Non-Local and Multi-Scale Mechanisms for Image Inpainting

Sensors ◽  
2021 ◽  
Vol 21 (9) ◽  
pp. 3281
Author(s):  
Xu He ◽  
Yong Yin
Keyword(s):  
Markov Random Fields ◽  
Receptive Fields ◽  
Image Inpainting ◽  
Long Distance ◽  
Visual Appearance ◽  
Fine Grained ◽  
Multi Scale ◽  
Non Local ◽  
Relationship Of

Recently, deep learning-based techniques have shown great power in image inpainting especially dealing with squared holes. However, they fail to generate plausible results inside the missing regions for irregular and large holes as there is a lack of understanding between missing regions and existing counterparts. To overcome this limitation, we combine two non-local mechanisms including a contextual attention module (CAM) and an implicit diversified Markov random fields (ID-MRF) loss with a multi-scale architecture which uses several dense fusion blocks (DFB) based on the dense combination of dilated convolution to guide the generative network to restore discontinuous and continuous large masked areas. To prevent color discrepancies and grid-like artifacts, we apply the ID-MRF loss to improve the visual appearance by comparing similarities of long-distance feature patches. To further capture the long-term relationship of different regions in large missing regions, we introduce the CAM. Although CAM has the ability to create plausible results via reconstructing refined features, it depends on initial predicted results. Hence, we employ the DFB to obtain larger and more effective receptive fields, which benefits to predict more precise and fine-grained information for CAM. Extensive experiments on two widely-used datasets demonstrate that our proposed framework significantly outperforms the state-of-the-art approaches both in quantity and quality.

scholarly journals Domain randomization-enhanced deep learning models for bird detection

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Xin Mao ◽  
Jun Kang Chow ◽  
Pin Siang Tan ◽  
Kuan-fu Liu ◽  
Jimmy Wu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Continuous Monitoring ◽  
Bird Species ◽  
Training Data ◽  
Learning Models ◽  
Fine Grained ◽  
Bird Detection ◽  
Relationship Of ◽  
The Relationship

AbstractAutomatic bird detection in ornithological analyses is limited by the accuracy of existing models, due to the lack of training data and the difficulties in extracting the fine-grained features required to distinguish bird species. Here we apply the domain randomization strategy to enhance the accuracy of the deep learning models in bird detection. Trained with virtual birds of sufficient variations in different environments, the model tends to focus on the fine-grained features of birds and achieves higher accuracies. Based on the 100 terabytes of 2-month continuous monitoring data of egrets, our results cover the findings using conventional manual observations, e.g., vertical stratification of egrets according to body size, and also open up opportunities of long-term bird surveys requiring intensive monitoring that is impractical using conventional methods, e.g., the weather influences on egrets, and the relationship of the migration schedules between the great egrets and little egrets.

scholarly journals Scaling property of ideal granitic sequences

2007 ◽  
Vol 14 (3) ◽  
pp. 237-246 ◽  
Author(s):  
D. Xu ◽  
Q. Cheng ◽  
F. Agterberg
Keyword(s):  
Lake Area ◽  
Fine Grained ◽  
Multi Scale ◽  
Invariance Properties ◽  
The Past ◽  
Ideal Granite ◽  
Markov Properties ◽  
The Ideal

Abstract. Quantification of granite textures and structures using a mathematical model for characterization of granites has been a long-term attempt of mathematical geologists over the past four decades. It is usually difficult to determine the influence of magma properties on mineral crystallization forming fined-grained granites due to its irregular and fine-grained textures. The ideal granite model was originally developed for modeling mineral sequences from first and second-order Markov properties. This paper proposes a new model for quantifying scale invariance properties of mineral clusters and voids observed within mineral sequences. Sequences of the minerals plagioclase, quartz and orthoclase observed under the microscope for 104 aplite samples collected from the Meech Lake area, Gatineau Park, Québec were used for validation of the model. The results show that the multi-scale approaches proposed in this paper may enable quantification of the nature of the randomness of mineral grain distributions. This, in turn, may be related to original properties of the magma.

scholarly journals Deep Multi-Modal Metric Learning with Multi-Scale Correlation for Image-Text Retrieval

Electronics ◽  
2020 ◽  
Vol 9 (3) ◽  
pp. 466 ◽  
Author(s):  
Yan Hua ◽  
Yingyun Yang ◽  
Jianhe Du
Keyword(s):  
State Of The Art ◽  
Metric Learning ◽  
Heterogeneous Data ◽  
Semantic Relationship ◽  
Modal Data ◽  
Fine Grained ◽  
Semantic Correlation ◽  
Multi Scale ◽  
Deep Model ◽  
Relationship Of

Multi-modal retrieval is a challenge due to heterogeneous gap and a complex semantic relationship between different modal data. Typical research map different modalities into a common subspace with a one-to-one correspondence or similarity/dissimilarity relationship of inter-modal data, in which the distances of heterogeneous data can be compared directly; thus, inter-modal retrieval can be achieved by the nearest neighboring search. However, most of them ignore intra-modal relations and complicated semantics between multi-modal data. In this paper, we propose a deep multi-modal metric learning method with multi-scale semantic correlation to deal with the retrieval tasks between image and text modalities. A deep model with two branches is designed to nonlinearly map raw heterogeneous data into comparable representations. In contrast to binary similarity, we formulate semantic relationship with multi-scale similarity to learn fine-grained multi-modal distances. Inter-modal and intra-modal correlations constructed on multi-scale semantic similarity are incorporated to train the deep model in an end-to-end way. Experiments validate the effectiveness of our proposed method on multi-modal retrieval tasks, and our method outperforms state-of-the-art methods on NUS-WIDE, MIR Flickr, and Wikipedia datasets.

scholarly journals Associating Multi-Scale Receptive Fields For Fine-Grained Recognition

2020 ◽  
Author(s):  
Zihan Ye ◽  
Fuyuan Hu ◽  
Yin Liu ◽  
Zhenping Xia ◽  
Fan Lyu ◽  
...  
Keyword(s):  
Receptive Fields ◽  
Fine Grained ◽  
Multi Scale

scholarly journals Lightweight Image Restoration Network for Strong Noise Removal in Nuclear Radiation Scenes

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1810
Author(s):  
Xin Sun ◽  
Hongwei Luo ◽  
Guihua Liu ◽  
Chunmei Chen ◽  
Feng Xu
Keyword(s):  
Receptive Fields ◽  
Activation Function ◽  
Noise Removal ◽  
Nuclear Radiation ◽  
Model Parameters ◽  
Multi Scale ◽  
Kernel Module ◽  
Quantitative Metrics ◽  
Non Local ◽  
High Level

In order to remove the strong noise with complex shapes and high density in nuclear radiation scenes, a lightweight network composed of a Noise Learning Unit (NLU) and Texture Learning Unit (TLU) was designed. The NLU is bilinearly composed of a Multi-scale Kernel Module (MKM) and a Residual Module (RM), which learn non-local information and high-level features, respectively. Both the MKM and RM have receptive field blocks and attention blocks to enlarge receptive fields and enhance features. The TLU is at the bottom of the NLU and learns textures through an independent loss. The entire network adopts a Mish activation function and asymmetric convolutions to improve the overall performance. Compared with 12 denoising methods on our nuclear radiation dataset, the proposed method has the fewest model parameters, the highest quantitative metrics, and the best perceptual satisfaction, indicating its high denoising efficiency and rich texture retention.

scholarly journals Coarse-to-Fine Image Inpainting via Region-wise Convolutions and Non-Local Correlation

2019 ◽  
Author(s):  
Yuqing Ma ◽  
Xianglong Liu ◽  
Shihao Bai ◽  
Lei Wang ◽  
Dailan He ◽  
...  
Keyword(s):  
Image Inpainting ◽  
Local Correlation ◽  
Long Distance ◽  
Proposed Model ◽  
Different Types ◽  
Non Local ◽  
Surrounding Areas ◽  
Convolution Filters ◽  
Realistic Images ◽  
Coarse To Fine

Recently deep neural networks have achieved promising performance for filling large missing regions in image inpainting tasks. They usually adopted the standard convolutional architecture over the corrupted image, where the same convolution filters try to restore the diverse information on both existing and missing regions, and meanwhile ignores the long-distance correlation among the regions. Only relying on the surrounding areas inevitably leads to meaningless contents and artifacts, such as color discrepancy and blur. To address these problems, we first propose region-wise convolutions to locally deal with the different types of regions, which can help exactly reconstruct existing regions and roughly infer the missing ones from existing regions at the same time. Then, a non-local operation is introduced to globally model the correlation among different regions, promising visual consistency between missing and existing regions. Finally, we integrate the region-wise convolutions and non-local correlation in a coarse-to-fine framework to restore semantically reasonable and visually realistic images. Extensive experiments on three widely-used datasets for image inpainting tasks have been conducted, and both qualitative and quantitative experimental results demonstrate that the proposed model significantly outperforms the state-of-the-art approaches, especially for the large irregular missing regions.

scholarly journals MRDA-MGFSNet: Network Based on a Multi-Rate Dilated Attention Mechanism and Multi-Granularity Feature Sharer for Image-Based Butterflies Fine-Grained Classification

Symmetry ◽  
2021 ◽  
Vol 13 (8) ◽  
pp. 1351
Author(s):  
Maopeng Li ◽  
Guoxiong Zhou ◽  
Weiwei Cai ◽  
Jiayong Li ◽  
Mingxuan Li ◽  
...  
Keyword(s):  
Receptive Fields ◽  
Attention Mechanism ◽  
Good Effect ◽  
Complex Environment ◽  
High Background ◽  
Symmetrical Structure ◽  
Fine Grained ◽  
Multi Scale ◽  
Spatial Features

Aiming at solving the problems of high background complexity of some butterfly images and the difficulty in identifying them caused by their small inter-class variance, we propose a new fine-grained butterfly classification architecture, called Network based on Multi-rate Dilated Attention Mechanism and Multi-granularity Feature Sharer (MRDA-MGFSNet). First, in this network, in order to effectively identify similar patterns between butterflies and suppress the information that is similar to the butterfly’s features in the background but is invalid, a Multi-rate Dilated Attention Mechanism (MRDA) with a symmetrical structure which assigns different weights to channel and spatial features is designed. Second, fusing the multi-scale receptive field module with the depthwise separable convolution module, a Multi-granularity Feature Sharer (MGFS), which can better solve the recognition problem of a small inter-class variance and reduce the increase in parameters caused by multi-scale receptive fields, is proposed. In order to verify the feasibility and effectiveness of the model in a complex environment, compared with the existing methods, our proposed method obtained a mAP of 96.64%, and an F1 value of 95.44%, which showed that the method proposed in this paper has a good effect on the fine-grained classification of butterflies.

Long-distance contextual attention network for skin disease segmentation

2021 ◽  
pp. 1-12
Author(s):  
Yanhan Zhang ◽  
Shengwei Tian ◽  
Long Yu ◽  
Yuan Ren ◽  
Zhongyu Gao ◽  
...  
Keyword(s):  
Image Segmentation ◽  
Skin Disease ◽  
Skin Diseases ◽  
Malignant Tumors ◽  
Two Dimensions ◽  
Jaccard Index ◽  
Long Distance ◽  
Attention Network ◽  
Non Local

In recent years, the incidence of skin diseases has increased significantly, and some malignant tumors caused by skin diseases have brought great hidden dangers to people’s health. In order to help experts perform lesion measurement and auxiliary diagnosis, automatic segmentation methods are very needed in clinical practice. Deep learning and contextual information extraction methods have been applied to many image segmentation tasks. However, their performance is limited due to insufficient training of a large number of parameters and these parameters sometimes fail to capture long-term dependencies. In addition, due to the many interfering factors of the skin disease image, the complex boundary and the uncertain size and shape of the lesion, the segmentation of the skin disease image is still a challenging problem. To solve these problems, we propose a long-distance contextual attention network(LCA-Net). By connecting the non-local module and the channel attention (CAM) in parallel to form a non-local operation, the long-term dependence is captured from the two dimensions of space and channel to enhance the network’s ability to extract features of skin diseases. Our method has an average Jaccard index of 0.771 on the ISIC2017 dataset, which represents a 0.6%improvement over the ISIC2017 Challenge Champion model. The average Jaccard index of 5-fold cross-validation on the ISIC2018 dataset is 0.8256. At the same time, we also compared with some advanced methods of image segmentation, the experimental results show our proposed method has a competitive performance.

Attachment Styles, Physical Proximity, and Relational Satisfaction

2014 ◽  
Vol 4 (2) ◽  
pp. 106-112
Author(s):  
Anita Shrivastava ◽  
Andrea Burianova
Keyword(s):  
Attachment Styles ◽  
Romantic Relationship ◽  
Distinct Type ◽  
Relational Satisfaction ◽  
Long Distance ◽  
Avoidant Attachment ◽  
Geographic Separation ◽  
Relationship Of ◽  
Attachment Dimensions

This study aimed to explore the relationships between attachment styles, proximity, and relational satisfaction. This was achieved by assessing a distinct type of long distance romantic relationship of flying crews, compared with proximal (non-flying crew) romantic relationships. The responses of 139 expatriate professionals revealed significant associations between proximity and anxious and avoidant attachment dimensions. The role of the avoidant dimension in comparison with that of the anxious dimension was found to be a significant predictor of relational satisfaction. This study contributes significantly toward addressing the role of proximity and attachment in relational satisfaction in a new context of geographic separation.

Sign in / Sign up

Export Citation Format

Share Document