scholarly journals Three-Dimensional Reconstruction with a Laser Line Based on Image In-Painting and Multi-Spectral Photometric Stereo

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2131
Author(s):  
Liang Lu ◽  
Hongbao Zhu ◽  
Junyu Dong ◽  
Yakun Ju ◽  
Huiyu Zhou
Keyword(s):  
3D Reconstruction ◽  
Real World ◽  
Three Dimensional ◽  
Photometric Stereo ◽  
Laser Line ◽  
Spectral Image ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Image Dataset ◽  
Dimensional Reconstruction

This paper presents a multi-spectral photometric stereo (MPS) method based on image in-painting, which can reconstruct the shape using a multi-spectral image with a laser line. One of the difficulties in multi-spectral photometric stereo is to extract the laser line because the required illumination for MPS, e.g., red, green, and blue light, may pollute the laser color. Unlike previous methods, through the improvement of the network proposed by Isola, a Generative Adversarial Network based on image in-painting was proposed, to separate a multi-spectral image with a laser line into a clean laser image and an uncorrupted multi-spectral image without the laser line. Then these results were substituted into the method proposed by Fan to obtain high-precision 3D reconstruction results. To make the proposed method applicable to real-world objects, a rendered image dataset obtained using the rendering models in ShapeNet has been used for training the network. Evaluation using the rendered images and real-world images shows the superiority of the proposed approach over several previous methods.

scholarly journals Semi-supervised Three-dimensional Reconstruction Framework with GAN

2019 ◽  
Author(s):  
Chong Yu
Keyword(s):  
3D Reconstruction ◽  
Three Dimensional ◽  
3D Models ◽  
Generative Adversarial Network ◽  
Weak Supervision ◽  
Adversarial Network ◽  
Current State ◽  
Reconstruction Performance ◽  
Dimensional Reconstruction ◽  
Intrinsic Complexity

Because of the intrinsic complexity in computation, three-dimensional (3D) reconstruction is an essential and challenging topic in computer vision research and applications. The existing methods for 3D reconstruction often produce holes, distortions and obscure parts in the reconstructed 3D models, or can only reconstruct voxelized 3D models for simple isolated objects. So they are not adequate for real usage. From 2014, the Generative Adversarial Network (GAN) is widely used in generating unreal dataset and semi-supervised learning. So the focus of this paper is to achieve high quality 3D reconstruction performance by adopting GAN principle. We propose a novel semi-supervised 3D reconstruction framework, namely SS-3D-GAN, which can iteratively improve any raw 3D reconstruction models by training the GAN models to converge. This new model only takes real-time 2D observation images as the weak supervision, and doesn't rely on prior knowledge of shape models or any referenced observations. Finally, through the qualitative and quantitative experiments & analysis, this new method shows compelling advantages over the current state-of-the-art methods on Tanks & Temples reconstruction benchmark dataset.

scholarly journals TomoSAR 3D Reconstruction for Buildings Using Very Few Tracks of Observation: A Conditional Generative Adversarial Network Approach

2021 ◽  
Vol 13 (24) ◽  
pp. 5055
Author(s):  
Shihong Wang ◽  
Jiayi Guo ◽  
Yueting Zhang ◽  
Yuxin Hu ◽  
Chibiao Ding ◽  
...  
Keyword(s):  
3D Reconstruction ◽  
Signal To Noise Ratio ◽  
Three Dimensional ◽  
Super Resolution ◽  
Quality Data ◽  
Sar Images ◽  
High Quality ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
High Quality Result

SAR tomography (TomoSAR) is an important technology for three-dimensional (3D) reconstruction of buildings through multiple coherent SAR images. In order to obtain sufficient signal-to-noise ratio (SNR), typical TomoSAR applications often require dozens of scenes of SAR images. However, limited by time and cost, the available SAR images are often only 3–5 scenes in practice, which makes the traditional TomoSAR technique unable to produce satisfactory SNR and elevation resolution. To tackle this problem, the conditional generative adversarial network (CGAN) is proposed to improve the TomoSAR 3D reconstruction by learning the prior information of building. Moreover, the number of tracks required can be reduced to three. Firstly, a TomoSAR 3D super-resolution dataset is constructed using high-quality data from the airborne array and low-quality data obtained from a small amount of tracks sampled from all observations. Then, the CGAN model is trained to estimate the corresponding high-quality result from the low-quality input. Airborne data experiments prove that the reconstruction results are improved in areas with and without overlap, both qualitatively and quantitatively. Furthermore, the network pretrained on the airborne dataset is directly used to process the spaceborne dataset without any tuning, and generates satisfactory results, proving the effectiveness and robustness of our method. The comparative experiment with nonlocal algorithm also shows that the proposed method has better height estimation and higher time efficiency.

Three-dimensional Reconstruction of Microscopic Image Based on Multi-ST Algorithm

2020 ◽  
Vol 64 (2) ◽  
pp. 20506-1-20506-7
Author(s):  
Min Zhu ◽  
Rongfu Zhang ◽  
Pei Ma ◽  
Xuedian Zhang ◽  
Qi Guo
Keyword(s):  
3D Reconstruction ◽  
Three Dimensional ◽  
Scale Model ◽  
Microscopic Image ◽  
Multi Scale ◽  
Gaussian Pyramid ◽  
Cost Aggregation ◽  
Dimensional Reconstruction ◽  
Image Pairs ◽  
Accurate Matching

Abstract Three-dimensional (3D) reconstruction is extensively used in microscopic applications. Reducing excessive error points and achieving accurate matching of weak texture regions have been the classical challenges for 3D microscopic vision. A Multi-ST algorithm was proposed to improve matching accuracy. The process is performed in two main stages: scaled microscopic images and regularized cost aggregation. First, microscopic image pairs with different scales were extracted according to the Gaussian pyramid criterion. Second, a novel cost aggregation approach based on the regularized multi-scale model was implemented into all scales to obtain the final cost. To evaluate the performances of the proposed Multi-ST algorithm and compare different algorithms, seven groups of images from the Middlebury dataset and four groups of experimental images obtained by a binocular microscopic system were analyzed. Disparity maps and reconstruction maps generated by the proposed approach contained more information and fewer outliers or artifacts. Furthermore, 3D reconstruction of the plug gauges using the Multi-ST algorithm showed that the error was less than 0.025 mm.

scholarly journals Online Measurement of Deposit Surface in Electron Beam Freeform Fabrication

Sensors ◽  
2019 ◽  
Vol 19 (18) ◽  
pp. 4001 ◽  
Author(s):  
Shuhe Chang ◽  
Haoyu Zhang ◽  
Haiying Xu ◽  
Xinghua Sang ◽  
Li Wang ◽  
...  
Keyword(s):  
Electron Beam ◽  
3D Reconstruction ◽  
Three Dimensional ◽  
Thermal Conditions ◽  
Online Measurement ◽  
Three Dimensional Reconstruction ◽  
Continuous Change ◽  
Position Information ◽  
Freeform Fabrication ◽  
Dimensional Reconstruction

In the process of electron beam freeform fabrication (EBF3), due to the continuous change of thermal conditions and variability in wire feeding in the deposition process, geometric deviations are generated in the deposition of each layer. In order to prevent the layer-by-layer accumulation of the deviation, it is necessary to perform online geometry measurement for each deposition layer, based on which the error compensation can be done for the previous deposition layer in the next deposition layer. However, the traditional three-dimensional reconstruction method that employs structured laser cannot meet the requirements of long-term stable operation in the manufacturing process of EBF3. Therefore, this paper proposes a method to measure the deposit surfaces based on the position information of electron beam speckle, in which an electron beam is used to bombard the surface of the deposit to generate the speckle. Based on the structured information of the electron beam in the vacuum chamber, the three-dimensional reconstruction of the surface of the deposited parts is realized without need of additional structured laser sensor. In order to improve the detection accuracy, the detection error is theoretically analyzed and compensated. The absolute error after compensation is smaller than 0.1 mm, and the precision can reach 0.1%, which satisfies the requirements of 3D reconstruction of the deposited parts. An online measurement system is built for the surface of deposited parts in the process of electron beam freeform fabrication, which realizes the online 3D reconstruction of the surface of the deposited layer. In addition, in order to improve the detection stability of the whole system, the image processing algorithm suitable for this scene is designed. The reliability and speed of the algorithm are improved by ROI extraction, threshold segmentation, and expansion corrosion. In addition, the speckle size information can also reflect the thermal conditions of the surface of the deposited parts. Hence, it can be used for online detection of defects such as infusion and voids.

Direct Unsupervised Super-Resolution using Generative Adversarial Network (DUS-GAN) for Real-World Data

2021 ◽  
pp. 1-1
Author(s):  
Kalpesh Prajapati ◽  
Vishal Chudasama ◽  
Heena Patel ◽  
Kishor Upla ◽  
Kiran Raja ◽  
...  
Keyword(s):  
Real World ◽  
Super Resolution ◽  
Real World Data ◽  
Generative Adversarial Network ◽  
World Data ◽  
Adversarial Network

scholarly journals MAGAN: Mask Attention Generative Adversarial Network for Liver Tumor CT Image Synthesis

2020 ◽  
Author(s):  
Yang Liu ◽  
Lu Meng ◽  
Jianping Zhong
Keyword(s):  
Liver Tumor ◽  
Medical Image ◽  
Ct Images ◽  
Computer Aided Diagnosis ◽  
Generative Adversarial Network ◽  
Liver Ct ◽  
Adversarial Network ◽  
Computer Aided ◽  
Image Dataset ◽  
Aided Diagnosis

Abstract Background: For deep learning, the size of the dataset greatly affects the final training effect. However, in the field of computer-aided diagnosis, medical image datasets are often limited and even scarce.Methods: We aim to synthesize medical images and enlarge the size of the medical image dataset. In the present study, we synthesized the liver CT images with a tumor based on the mask attention generative adversarial network (MAGAN). We masked the pixels of the liver tumor in the image as the attention map. And both the original image and attention map were loaded into the generator network to obtain the synthesized images. Then the original images, the attention map, and the synthesized images were all loaded into the discriminator network to determine if the synthesized images were real or fake. Finally, we can use the generator network to synthesize liver CT images with a tumor.Results: The experiments showed that our method outperformed the other state-of-the-art methods, and can achieve a mean peak signal-to-noise ratio (PSNR) as 64.72dB.Conclusions: All these results indicated that our method can synthesize liver CT images with tumor, and build large medical image dataset, which may facilitate the progress of medical image analysis and computer-aided diagnosis.

scholarly journals Using the Generative Adversarial Network to Generate Recommendations

2020 ◽  
Author(s):  
A.V. Prosvetov
Keyword(s):  
Real World ◽  
Recommendation System ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Theoretical Comparison ◽  
New Methods ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Recommendation Algorithms ◽  
Cold Start Problem

Widely used recommendation systems do not meet all industry requirements, so the search for more advanced methods for creating recommendations continues. The proposed new methods based on Generative Adversarial Networks (GAN) have a theoretical comparison with other recommendation algorithms; however, real-world comparisons are needed to introduce new methods in the industry. In our work, we compare recommendations from the Generative Adversarial Network with recommendation from the Deep Semantic Similarity Model (DSSM) on real-world case of airflight tickets. We found a way to train the GAN so that users receive appropriate recommendations, and during A/B testing, we noted that the GAN-based recommendation system can successfully compete with other neural networks in generating recommendations. One of the advantages of the proposed approach is that the GAN training process avoids a negative sampling, which causes a number of distortions in the final ratings of recommendations. Due to the ability of the GAN to generate new objects from the distribution of the training set, we assume that the Conditional GAN is able to solve the cold start problem.

scholarly journals 3D-StyleGAN: A Style-Based Generative Adversarial Network for Generative Modeling of Three-Dimensional Medical Images

2021 ◽  
pp. 24-34
Author(s):  
Sungmin Hong ◽  
Razvan Marinescu ◽  
Adrian V. Dalca ◽  
Anna K. Bonkhoff ◽  
Martin Bretzner ◽  
...  
Keyword(s):  
Medical Images ◽  
Three Dimensional ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Generative Modeling
Sign in / Sign up

Export Citation Format

Share Document