scholarly journals Single-Shot 3D Multi-Person Shape Reconstruction from a Single RGB Image

Entropy ◽  
2020 ◽  
Vol 22 (8) ◽  
pp. 806
Author(s):  
Seong Hyun Kim ◽  
Ju Yong Chang
Keyword(s):  
Coordinate System ◽  
Shape Reconstruction ◽  
Reconstruction Method ◽  
Single Shot ◽  
3D Shape ◽  
End To End ◽  
The Absolute ◽  
3D Shapes ◽  
Camera Coordinate System ◽  
Rgb Image

Although the performance of the 3D human shape reconstruction method has improved considerably in recent years, most methods focus on a single person, reconstruct a root-relative 3D shape, and rely on ground-truth information about the absolute depth to convert the reconstruction result to the camera coordinate system. In this paper, we propose an end-to-end learning-based model for single-shot, 3D, multi-person shape reconstruction in the camera coordinate system from a single RGB image. Our network produces output tensors divided into grid cells to reconstruct the 3D shapes of multiple persons in a single-shot manner, where each grid cell contains information about the subject. Moreover, our network predicts the absolute position of the root joint while reconstructing the root-relative 3D shape, which enables reconstructing the 3D shapes of multiple persons in the camera coordinate system. The proposed network can be learned in an end-to-end manner and process images at about 37 fps to perform the 3D multi-person shape reconstruction task in real time.

scholarly journals Design, Measurement and Shape Reconstruction of Soft Surgical Actuator Based on Fiber Bragg Gratings

2018 ◽  
Vol 8 (10) ◽  
pp. 1773 ◽  
Author(s):  
Yanlin He ◽  
Lianqing Zhu ◽  
Guangkai Sun ◽  
Mingxin Yu ◽  
Mingli Dong
Keyword(s):  
Optical Fiber ◽  
Shape Reconstruction ◽  
Reconstruction Method ◽  
Soft Actuator ◽  
3D Shape ◽  
3D Shape Reconstruction ◽  
Fbg Sensors ◽  
Soft Actuators ◽  
Shape Sensing ◽  
3D Shapes

Soft actuators are the components responsible for organs and tissues adsorptive fixation in some surgical operations, but the lack of shape sensing and monitoring of a soft actuator greatly limits their application potential. Consequently, this paper proposes a real-time 3D shape reconstruction method of soft surgical actuator which has an embedded optical fiber with two Fiber Bragg Grating (FBG) sensors. First, the design principle and the sensing of the soft actuator based on FBG sensors are analyzed, and the fabrication process of soft actuator which has an embedded optical fiber with two FBG sensors is described. Next, the calibration of the FBG sensors is conducted. Based on curvatures and curve fitting functions, the strategy of 3D shapes reconstruction of the soft actuator is presented. Finally, some bending experiments of the soft actuator are carried out, and the 3D shapes of the soft actuator at different bending states are reconstructed. This well reconstructed 3D shape of a soft actuator demonstrates the effectiveness of the shape reconstruction method that is proposed in this paper, as well as the potential and increased applications of these structures for real soft surgical actuators.

scholarly journals Reconstructing 3D Model from Single-View Sketch with Deep Neural Network

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Fei Wang ◽  
Yu Yang ◽  
Baoquan Zhao ◽  
Dazhi Jiang ◽  
Siwei Chen ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Shape Reconstruction ◽  
Reconstruction Method ◽  
Single View ◽  
3D Shape ◽  
Proposed Model ◽  
Reconstruction Performance ◽  
3D Shape Reconstruction ◽  
3D Shapes

In this paper, we introduce a novel 3D shape reconstruction method from a single-view sketch image based on a deep neural network. The proposed pipeline is mainly composed of three modules. The first module is sketch component segmentation based on multimodal DNN fusion and is used to segment a given sketch into a series of basic units and build a transformation template by the knots between them. The second module is a nonlinear transformation network for multifarious sketch generation with the obtained transformation template. It creates the transformation representation of a sketch by extracting the shape features of an input sketch and transformation template samples. The third module is deep 3D shape reconstruction using multifarious sketches, which takes the obtained sketches as input to reconstruct 3D shapes with a generative model. It fuses and optimizes features of multiple views and thus is more likely to generate high-quality 3D shapes. To evaluate the effectiveness of the proposed method, we conduct extensive experiments on a public 3D reconstruction dataset. The results demonstrate that our model can achieve better reconstruction performance than peer methods. Specifically, compared to the state-of-the-art method, the proposed model achieves a performance gain in terms of the five evaluation metrics by an average of 25.5% on the man-made model dataset and 23.4% on the character object dataset using synthetic sketches and by an average of 31.8% and 29.5% on the two datasets, respectively, using human drawing sketches.

Digital Shape Reconstruction of a Micro-Sized Machining Tool Using Light-Field Microscopy

2016 ◽  
Vol 10 (2) ◽  
pp. 172-178 ◽  
Author(s):  
Shin Usuki ◽  
◽  
Masaru Uno ◽  
Kenjiro T. Miura ◽  
◽  
...  
Keyword(s):  
Light Field ◽  
Three Dimensional ◽  
Microlens Array ◽  
Axial Direction ◽  
Shape Reconstruction ◽  
Reconstruction Method ◽  
Actual Length ◽  
3D Shape ◽  
Depth Images ◽  
3D Shape Reconstruction

In this paper, we propose a digital shape reconstruction method for micro-sized 3D (three-dimensional) objects based on the shape from silhouette (SFS) method that reconstructs the shape of a 3D model from silhouette images taken from multiple viewpoints. In the proposed method, images used in the SFS method are depth images acquired with a light-field microscope by digital refocusing (DR) of a stacked image along the axial direction. The DR can generate refocused images from an acquired image by an inverse ray tracing technique using a microlens array. Therefore, this technique provides fast image stacking with different focal planes. Our proposed method can reconstruct micro-sized object models including edges, convex shapes, and concave shapes on the surface of an object such as micro-sized defects so that damaged structures in the objects can be visualized. Firstly, we introduce the SFS method and the light-field microscope for 3D shape reconstruction that is required in the field of micro-sized manufacturing. Secondly, we show the developed experimental equipment for microscopic image acquisition. Depth calibration using a USAF1951 test target is carried out to convert relative value into actual length. Then 3D modeling techniques including image processing are implemented for digital shape reconstruction. Finally, 3D shape reconstruction results of micro-sized machining tools are shown and discussed.

scholarly journals Single-Shot 3D Shape Reconstruction Using Structured Light and Deep Convolutional Neural Networks

Sensors ◽  
2020 ◽  
Vol 20 (13) ◽  
pp. 3718 ◽  
Author(s):  
Hieu Nguyen ◽  
Yuzeng Wang ◽  
Zhaoyang Wang
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
3D Imaging ◽  
Structured Light ◽  
Depth Map ◽  
Shape Reconstruction ◽  
Single Shot ◽  
Deep Convolutional Neural Networks ◽  
3D Shape ◽  
3D Shape Reconstruction

Single-shot 3D imaging and shape reconstruction has seen a surge of interest due to the ever-increasing evolution in sensing technologies. In this paper, a robust single-shot 3D shape reconstruction technique integrating the structured light technique with the deep convolutional neural networks (CNNs) is proposed. The input of the technique is a single fringe-pattern image, and the output is the corresponding depth map for 3D shape reconstruction. The essential training and validation datasets with high-quality 3D ground-truth labels are prepared by using a multi-frequency fringe projection profilometry technique. Unlike the conventional 3D shape reconstruction methods which involve complex algorithms and intensive computation to determine phase distributions or pixel disparities as well as depth map, the proposed approach uses an end-to-end network architecture to directly carry out the transformation of a 2D image to its corresponding 3D depth map without extra processing. In the approach, three CNN-based models are adopted for comparison. Furthermore, an accurate structured-light-based 3D imaging dataset used in this paper is made publicly available. Experiments have been conducted to demonstrate the validity and robustness of the proposed technique. It is capable of satisfying various 3D shape reconstruction demands in scientific research and engineering applications.

scholarly journals hNet: Single-shot 3D shape reconstruction using structured light and h-shaped global guidance network

2021 ◽  
pp. 100104
Author(s):  
Hieu Nguyen ◽  
Khanh L. Ly ◽  
Tan Tran ◽  
Yuzheng Wang ◽  
Zhaoyang Wang
Keyword(s):  
Structured Light ◽  
Shape Reconstruction ◽  
Single Shot ◽  
3D Shape ◽  
3D Shape Reconstruction ◽  
Global Guidance

scholarly journals ROBUST DETECTION OF SURFACE ANOMALY USING LIDAR POINT CLOUD WITH INTENSITY

2020 ◽  
Vol XLIII-B2-2020 ◽  
pp. 1129-1136
Author(s):  
Y. Ono ◽  
A. Tsuji ◽  
J. Abe ◽  
H. Noguchi ◽  
J. Abe
Keyword(s):  
Building Material ◽  
Point Cloud ◽  
Detection Method ◽  
Automatic Detection ◽  
Detection Methods ◽  
Robust Detection ◽  
3D Shape ◽  
Metallic Corrosion ◽  
3D Shapes ◽  
Rgb Image

Abstract. We have developed an automatic detection method for metallic corrosion in facilities by using a LiDAR point cloud. While visual inspections for monitoring facilities are widely conducted, the inspection result depends on human skill, and there is currently a shortage of inspectors. While automatic detection methods using an RGB image have been developed, such methods cannot be applied to inspections at night. Therefore, we propose a robust detection method that utilizes both 3D shapes and intensities in a LiDAR point cloud instead of RGB information. The proposed method segments the point cloud into a basic building material by using the 3D shape and then recognizes a point cloud with an abnormal intensity in each material as the corrosion area. We demonstrate through experiments that the proposed method can robustly detect corrosion spots in aging facilities during detection conducted both during the day and at night.

Sign in / Sign up

Export Citation Format

Share Document