Semantic 3D indoor reconstruction with stereo camera imaging

Fast track article for IS&T International Symposium on Electronic Imaging 2021: 3D Imaging and Applications 2021 proceedings.

VOXEL-BASED INDOOR RECONSTRUCTION FROM HOLOLENS TRIANGLE MESHES

Current mobile augmented reality devices are often equipped with range sensors. The Microsoft HoloLens for instance is equipped with a Time-of-Flight (ToF) range camera providing coarse triangle meshes that can be used in custom applications. We suggest to use these triangle meshes for the automatic generation of indoor models that can serve as basis for augmenting their physical counterpart with location-dependent information. In this paper, we present a novel voxel-based approach for automated indoor reconstruction from unstructured three-dimensional geometries like triangle meshes. After an initial voxelisation of the input data, rooms are detected in the resulting voxel grid by segmenting connected voxel components of ceiling candidates and extruding them downwards to find floor candidates. Semantic class labels like ’Wall’, ’Wall Opening’, ’Interior Object’ and ’Empty Interior’ are then assigned to the room voxels in-between ceiling and floor by a rule-based voxel sweep algorithm. Finally, the geometry of the detected walls and their openings is refined in voxel representation. The proposed approach is not restricted to Manhattan World scenarios and does not rely on room surfaces being planar.

In Situ Wireless Channel Visualization Using Augmented Reality and Ray Tracing

This article presents a novel methodology for predicting wireless signal propagation using ray-tracing algorithms, and visualizing signal variations in situ by leveraging Augmented Reality (AR) tools. The proposed system performs a special type of spatial mapping, capable of converting a scanned indoor environment to a vector facet model. A ray-tracing algorithm uses the facet model for wireless signal predictions. Finally, an AR application overlays the signal strength predictions on the physical space in the form of holograms. Although some indoor reconstruction models have already been developed, this paper proposes an image to a facet algorithm for indoor reconstruction and compares its performance with existing AR algorithms, such as spatial understanding that are modified to create the required facet models. In addition, the paper orchestrates AR and ray-tracing techniques to provide an in situ network visualization interface. It is shown that the accuracy of the derived facet models is acceptable, and the overall signal predictions are not significantly affected by any potential inaccuracies of the indoor reconstruction. With the expected increase of densely deployed indoor 5G networks, it is believed that these types of AR applications for network visualization will play a key role in the successful planning of 5G networks.

Automatic Indoor Reconstruction from Point Clouds in Multi-room Environments with Curved Walls

Recent developments in laser scanning systems have inspired substantial interest in indoor modeling. Semantically rich indoor models are required in many fields. Despite the rapid development of 3D indoor reconstruction methods for building interiors from point clouds, the indoor reconstruction of multi-room environments with curved walls is still not resolved. This study proposed a novel straight and curved line tracking method followed by a straight line test. Robust parameters are used, and a novel straight line regularization method is achieved using constrained least squares. The method constructs a cell complex with both straight lines and curved lines, and the indoor reconstruction is transformed into a labeling problem that is solved based on a novel Markov Random Field formulation. The optimal labeling is found by minimizing an energy function by applying a minimum graph cut approach. Detailed experiments were conducted, and the results indicate that the proposed method is well suited for 3D indoor modeling in multi-room indoor environments with curved walls.

Indoor Building Reconstruction from Occluded Point Clouds Using Graph-Cut and Ray-Tracing

Despite the increasing demand of updated and detailed indoor models, indoor reconstruction from point clouds is still in an early stage in comparison with the reconstruction of outdoor scenes. Specific challenges are related to the complex building layouts and the high presence of elements such as pieces of furniture causing clutter and occlusions. This work proposes an automatic method for modelling Manhattan-World indoors acquired with a mobile laser scanner in the presence of highly occluded walls. The core of the methodology is the transformation of indoor reconstruction into a labelling problem of structural cells in a 2D floor plan. Assuming the prevalence of orthogonal intersections between walls, indoor completion is formulated as an energy minimization problem using graph cuts. Doors and windows are detected from occlusions by implementing a ray-tracing algorithm. The methodology is tested in a real case study. Except for one window partially covered by a curtain, all building elements were successfully reconstructed.

CHALLENGES IN FLYING QUADROTOR UNMANNED AERIAL VEHICLE FOR 3D INDOOR RECONSTRUCTION

Three-dimensional modelling plays a vital role in indoor 3D tracking, navigation, guidance and emergency evacuation. Reconstruction of indoor 3D models is still problematic, in part, because indoor spaces provide challenges less-documented than their outdoor counterparts. Challenges include obstacles curtailing image and point cloud capture, restricted accessibility and a wide array of indoor objects, each with unique semantics. Reconstruction of indoor environments can be achieved through a photogrammetric approach, e.g. by using image frames, aligned using recurring corresponding image points (CIP) to build coloured point clouds. Our experiments were conducted by flying a QUAV in three indoor environments and later reconstructing 3D models which were analysed under different conditions. Point clouds and meshes were created using Agisoft PhotoScan Professional. We concentrated on flight paths from two vantage points: 1) safety and security while flying indoors and 2) data collection needed for reconstruction of 3D models. We surmised that the main challenges in providing safe flight paths are related to the physical configuration of indoor environments, privacy issues, the presence of people and light conditions. We observed that the quality of recorded video used for 3D reconstruction has a high dependency on surface materials, wall textures and object types being reconstructed. Our results show that 3D indoor reconstruction predicated on video capture using a QUAV is indeed feasible, but close attention should be paid to flight paths and conditions ultimately influencing the quality of 3D models. Moreover, it should be decided in advance which objects need to be reconstructed, e.g. bare rooms or detailed furniture.