scholarly journals Vision for Robust Robot Manipulation

Sensors ◽  
2019 ◽  
Vol 19 (7) ◽  
pp. 1648 ◽  
Author(s):  
Ester Martinez-Martin ◽  
Angel del Pobil
Keyword(s):  
Robot Manipulator ◽  
Depth Information ◽  
Indoor Environments ◽  
Tactile Sensors ◽  
Recovery Stage ◽  
Depth Cameras ◽  
Contact Points ◽  
Vision Sensors ◽  
Proprioceptive Sensors ◽  
New Generation

Advances in Robotics are leading to a new generation of assistant robots working in ordinary, domestic settings. This evolution raises new challenges in the tasks to be accomplished by the robots. This is the case for object manipulation where the detect-approach-grasp loop requires a robust recovery stage, especially when the held object slides. Several proprioceptive sensors have been developed in the last decades, such as tactile sensors or contact switches, that can be used for that purpose; nevertheless, their implementation may considerably restrict the gripper’s flexibility and functionality, increasing their cost and complexity. Alternatively, vision can be used since it is an undoubtedly rich source of information, and in particular, depth vision sensors. We present an approach based on depth cameras to robustly evaluate the manipulation success, continuously reporting about any object loss and, consequently, allowing it to robustly recover from this situation. For that, a Lab-colour segmentation allows the robot to identify potential robot manipulators in the image. Then, the depth information is used to detect any edge resulting from two-object contact. The combination of those techniques allows the robot to accurately detect the presence or absence of contact points between the robot manipulator and a held object. An experimental evaluation in realistic indoor environments supports our approach.

scholarly journals ENHANCED RGB-D MAPPING METHOD FOR DETAILED 3D MODELING OF LARGE INDOOR ENVIRONMENTS

2016 ◽  
Vol III-1 ◽  
pp. 151-158
Author(s):  
Shengjun Tang ◽  
Qing Zhu ◽  
Wu Chen ◽  
Walid Darwish ◽  
Bo Wu ◽  
...  
Keyword(s):  
3D Modeling ◽  
Visual Information ◽  
3D Models ◽  
Measurement Range ◽  
Mapping Method ◽  
Depth Information ◽  
Indoor Environments ◽  
Recovery Method ◽  
Depth Measurement ◽  
Rgb Image

RGB-D sensors are novel sensing systems that capture RGB images along with pixel-wise depth information. Although they are widely used in various applications, RGB-D sensors have significant drawbacks with respect to 3D dense mapping of indoor environments. First, they only allow a measurement range with a limited distance (e.g., within 3 m) and a limited field of view. Second, the error of the depth measurement increases with increasing distance to the sensor. In this paper, we propose an enhanced RGB-D mapping method for detailed 3D modeling of large indoor environments by combining RGB image-based modeling and depth-based modeling. The scale ambiguity problem during the pose estimation with RGB image sequences can be resolved by integrating the information from the depth and visual information provided by the proposed system. A robust rigid-transformation recovery method is developed to register the RGB image-based and depth-based 3D models together. The proposed method is examined with two datasets collected in indoor environments for which the experimental results demonstrate the feasibility and robustness of the proposed method

scholarly journals Calibration of the Relative Orientation between Multiple Depth Cameras Based on a Three-Dimensional Target

Sensors ◽  
2019 ◽  
Vol 19 (13) ◽  
pp. 3008 ◽  
Author(s):  
Zhe Liu ◽  
Zhaozong Meng ◽  
Nan Gao ◽  
Zonghua Zhang
Keyword(s):  
Coordinate System ◽  
Camera Calibration ◽  
Three Dimensional ◽  
Field Of View ◽  
Depth Information ◽  
Depth Camera ◽  
Depth Cameras ◽  
Calibration Methods ◽  
3D Shape Reconstruction ◽  
Single Coordinate

Depth cameras play a vital role in three-dimensional (3D) shape reconstruction, machine vision, augmented/virtual reality and other visual information-related fields. However, a single depth camera cannot obtain complete information about an object by itself due to the limitation of the camera’s field of view. Multiple depth cameras can solve this problem by acquiring depth information from different viewpoints. In order to do so, they need to be calibrated to be able to accurately obtain the complete 3D information. However, traditional chessboard-based planar targets are not well suited for calibrating the relative orientations between multiple depth cameras, because the coordinates of different depth cameras need to be unified into a single coordinate system, and the multiple camera systems with a specific angle have a very small overlapping field of view. In this paper, we propose a 3D target-based multiple depth camera calibration method. Each plane of the 3D target is used to calibrate an independent depth camera. All planes of the 3D target are unified into a single coordinate system, which means the feature points on the calibration plane are also in one unified coordinate system. Using this 3D target, multiple depth cameras can be calibrated simultaneously. In this paper, a method of precise calibration using lidar is proposed. This method is not only applicable to the 3D target designed for the purposes of this paper, but it can also be applied to all 3D calibration objects consisting of planar chessboards. This method can significantly reduce the calibration error compared with traditional camera calibration methods. In addition, in order to reduce the influence of the infrared transmitter of the depth camera and improve its calibration accuracy, the calibration process of the depth camera is optimized. A series of calibration experiments were carried out, and the experimental results demonstrated the reliability and effectiveness of the proposed method.

scholarly journals Dexterous Manipulation of Unknown Objects Using Virtual Contact Points

Robotics ◽  
2019 ◽  
Vol 8 (4) ◽  
pp. 86 ◽  
Author(s):  
Andrés Montaño ◽  
Raúl Suárez
Keyword(s):  
Physical Property ◽  
Robotic Hand ◽  
Tactile Sensors ◽  
Dexterous Manipulation ◽  
Everyday Objects ◽  
Simple Strategy ◽  
Contact Points ◽  
Different Shapes ◽  
Unknown Objects ◽  
Kinematic Information

The manipulation of unknown objects is a problem of special interest in robotics since it is not always possible to have exact models of the objects with which the robot interacts. This paper presents a simple strategy to manipulate unknown objects using a robotic hand equipped with tactile sensors. The hand configurations that allow the rotation of an unknown object are computed using only tactile and kinematic information, obtained during the manipulation process and reasoning about the desired and real positions of the fingertips during the manipulation. This is done taking into account that the desired positions of the fingertips are not physically reachable since they are located in the interior of the manipulated object and therefore they are virtual positions with associated virtual contact points. The proposed approach was satisfactorily validated using three fingers of an anthropomorphic robotic hand (Allegro Hand), with the original fingertips replaced by tactile sensors (WTS-FT). In the experimental validation, several everyday objects with different shapes were successfully manipulated, rotating them without the need of knowing their shape or any other physical property.

Research on Tactile Sensors Interface - Review of Theoretical and Practical Approach

2015 ◽  
Vol 772 ◽  
pp. 299-304
Author(s):  
V. Constantin Anghel ◽  
Ion Gheorghe Gheorghe
Keyword(s):  
Reaction Force ◽  
Optimal Solution ◽  
Tactile Sensor ◽  
Electronic System ◽  
Paper Analysis ◽  
Practical Approach ◽  
Human Walking ◽  
Tactile Sensors ◽  
Contact Points ◽  
Practical Design

The research presented in this paper aims to review the theoretical and practical approach for one of method to create the interface, to condition and to gather the data from the resistive force tactile sensor, in for the measuring of the vertical reaction force to the ground during human walking. The project result is a portable electronic system which allows the gathering, storing and transmission of the data from the force sensors mounted in an “overshoe” holder [1]. The article presents the research methods for the measured parameters as vertical reaction force to the ground, measured in 10 contact points between the foot sole and the ground by the help of some tactile sensors. The paper analysis all issues related to providing an optimal solution for the support with sensors and presents the conclusions for practical design.

scholarly journals RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments

2012 ◽  
Vol 31 (5) ◽  
pp. 647-663 ◽  
Author(s):  
Peter Henry ◽  
Michael Krainin ◽  
Evan Herbst ◽  
Xiaofeng Ren ◽  
Dieter Fox
Keyword(s):  
3D Modeling ◽  
Indoor Environments ◽  
Depth Cameras

scholarly journals Recent progress on tactile object recognition

2017 ◽  
Vol 14 (4) ◽  
pp. 172988141771705 ◽  
Author(s):  
Huaping Liu ◽  
Yupei Wu ◽  
Fuchun Sun ◽  
Di Guo
Keyword(s):  
Object Recognition ◽  
Internal State ◽  
Data Sets ◽  
Sensory Receptors ◽  
Intrinsic Properties ◽  
Tactile Sensors ◽  
Vision Sensors ◽  
Important Means ◽  
Background Clutter ◽  
Exploratory Procedure

Conventional visual perception technology is subject to many restrictions, such as illumination, background clutter, and occlusion. Many intrinsic properties of objects, like stiffness, hardness, and internal state, cannot be effectively perceived by visual sensors. For robots, tactile perception is a key approach to obtain environmental and object information. Different from vision sensors, tactile sensors can directly measure some physical properties of objects and environment. At the same time, humans also utilize touch sensory receptors as an important means to perceive and interact with the environment. In this article, we present a detailed discussion on tactile object recognition problem. We divide the current studies on the tactile object recognition into three subcategories and detailed analysis has been put forward on them. In addition, we also discuss some advanced topics such as visual–tactile fusion, exploratory procedure, and data sets.

scholarly journals Exploring RGBDepth Fusion for Real-Time Object Detection

Sensors ◽  
2019 ◽  
Vol 19 (4) ◽  
pp. 866 ◽  
Author(s):  
Tanguy Ophoff ◽  
Kristof Van Beeck ◽  
Toon Goedemé
Keyword(s):  
Object Detection ◽  
Real Time ◽  
Network Architecture ◽  
Depth Information ◽  
Special Focus ◽  
Single Shot ◽  
Depth Sensing ◽  
Real Time Processing ◽  
Depth Cameras ◽  
Optimal Fusion

In this paper, we investigate whether fusing depth information on top of normal RGB data for camera-based object detection can help to increase the performance of current state-of-the-art single-shot detection networks. Indeed, depth sensing is easily acquired using depth cameras such as a Kinect or stereo setups. We investigate the optimal manner to perform this sensor fusion with a special focus on lightweight single-pass convolutional neural network (CNN) architectures, enabling real-time processing on limited hardware. For this, we implement a network architecture allowing us to parameterize at which network layer both information sources are fused together. We performed exhaustive experiments to determine the optimal fusion point in the network, from which we can conclude that fusing towards the mid to late layers provides the best results. Our best fusion models significantly outperform the baseline RGB network in both accuracy and localization of the detections.

scholarly journals Parâmetros de sensores em robótica móvel para exploração autônoma de ambientes internos

2020 ◽  
Author(s):  
◽  
Flavio Barbosa Vieira
Keyword(s):  
Mobile Robotics ◽  
Indoor Environments ◽  
Probabilistic Algorithms ◽  
Improve Performance ◽  
Depth Cameras ◽  
Advantages And Disadvantages ◽  
Intelligent Robots ◽  
Diverse Application ◽  
Time Distance ◽  
Internal Information

The area of mobile robotics has developed remarkably in recent years, many researchers are motivated by the growing demand for this technology and the variety of applications. Robotics competitions foster new challenges when considering diverse application scenarios for service robotics, such as RoboCup@Home, which sets rules for autonomous and intelligent robots to be evaluated while performing tasks in domestic or public scenarios. The present work focuses on solving the problem of exploring unknown residential indoor environments. To do this, the robot must collect external and internal information through sensors, to fuse this data and interpret it efficiently, making it possible to locate itself through probabilistic algorithms, simultaneously mapping the environment, and navigate the mapped environment avoiding collisions. The work studies and tests the configuration of distance sensors (lasers, sonar and cameras) and exploration techniques, available and shared in the ROS community, to ensure that the robot is able to fully explore the residential environment in order to optimize the necessary time, distance traveled and the rotation performed. The tested exploration packages are: explore-lite, RRT-exploration and cam-exploration. The variation of sensors was crucial to understand the advantages and disadvantages of using the Lidar laser and depth cameras in different combinations. Thus, the results show that the increase in the number of sensors does not improve performance in exploration in all conditions. The work concludes that both explore-lite and RRT-exploration perform well in all proposed conditions and indicate the best sensor assemblies for each package. Thus, a package was created for the implementation of autonomous exploration in the HERA Robot

scholarly journals Visual Map Construction Using RGB-D Sensors for Image-Based Localization in Indoor Environments

2017 ◽  
Vol 2017 ◽  
pp. 1-18 ◽  
Author(s):  
Guanyuan Feng ◽  
Lin Ma ◽  
Xuezhi Tan
Keyword(s):  
Essential Elements ◽  
Visual Features ◽  
Depth Information ◽  
Indoor Environments ◽  
Epipolar Constraint ◽  
Position Errors ◽  
Depth Images ◽  
Map Construction ◽  
Global Consistency ◽  
Indoor Scenes

RGB-D sensors capture RGB images and depth images simultaneously, which makes it possible to acquire the depth information at pixel level. This paper focuses on the use of RGB-D sensors to construct a visual map which is an extended dense 3D map containing essential elements for image-based localization, such as poses of the database camera, visual features, and 3D structures of the building. Taking advantage of matched visual features and corresponding depth values, a novel local optimization algorithm is proposed to achieve point cloud registration and database camera pose estimation. Next, graph-based optimization is used to obtain the global consistency of the map. On the basis of the visual map, the image-based localization method is investigated, making use of the epipolar constraint. The performance of the visual map construction and the image-based localization are evaluated on typical indoor scenes. The simulation results show that the average position errors of the database camera and the query camera can be limited to within 0.2 meters and 0.9 meters, respectively.

scholarly journals Visual Positioning in Indoor Environments Using RGB-D Images and Improved Vector of Local Aggregated Descriptors

2021 ◽  
Vol 10 (4) ◽  
pp. 195
Author(s):  
Longyu Zhang ◽  
Hao Xia ◽  
Qingjun Liu ◽  
Chunyang Wei ◽  
Dong Fu ◽  
...  
Keyword(s):  
Mobile Devices ◽  
Depth Information ◽  
Indoor Environments ◽  
Color Information ◽  
Training Set ◽  
Visual Positioning ◽  
Opponent Color ◽  
Positioning Method ◽  
Smart Mobile ◽  
Smart Mobile Devices

Positioning information has become one of the most important information for processing and displaying on smart mobile devices. In this paper, we propose a visual positioning method using RGB-D image on smart mobile devices. Firstly, the pose of each image in the training set is calculated through feature extraction and description, image registration, and pose map optimization. Then, in the image retrieval stage, the training set and the query set are clustered to generate the vector of local aggregated descriptors (VLAD) description vector. In order to overcome the problem that the description vector loses the image color information and improve the retrieval accuracy under different lighting conditions, the opponent color information and depth information are added to the description vector for retrieval. Finally, using the point cloud corresponding to the retrieval result image and its pose, the pose of the retrieved image is calculated by perspective-n-point (PnP) method. The results of indoor scene positioning under different illumination conditions show that the proposed method not only improves the positioning accuracy compared with the original VLAD and ORB-SLAM2, but also has high computational efficiency.

Sign in / Sign up

Export Citation Format

Share Document