Obstacle detection using Microsoft Kinect

This thesis presents the development of image processing algorithms based on a Microsoft Kinect camera system. The algorithms developed during this thesis are applied on the depth image received from Kinect and are supposed to model a three dimensional object based representation of the recorded scene. The motivation behind this thesis is to develop a system which assists visually impaired people by navigating through unknown environments. The developed system is able to detect obstacles in the recorded scene and to warn about these obstacles. Since the goal of this thesis was not to develop a complete real time system but to invent reliable algorithms solving this task, the algorithms were developed in MATLAB. Additionally a control software was developed by which depth as well as color images can be received from Kinect. The developed algorithms are a combination of already known plane fitting algorithms and novel approaches. The algorithms perform a plane segmentation of the 3D point cloud and model objects out of the received segments. Each obstacle is defined by a cuboid box and thus can be illustrated easily to the blind person. For plane segmentation different approaches were compared to each other to find the most suitable approach. The first algorithm analyzed in this thesis is a normal vector based plane fitting algorithm. This algorithm supplies very accurate results but also has a high computation effort. The second approach, which was finally implemented, is a gradient based 2D image segmentation combined with a RANSAC plane segmentation (6) in a 3D points cloud. This approach has the advantage to find very small edges within the scene but also builds planes based on global constrains. Beside the development of the algorithm results of the image processing, which are really promising, are presented. Thus the algorithm is worth to be improved by further development. The developed algorithm is able to detect very small but significant obstacles but on the other hand does not represent the scene too detailed such that the result can be illustrated accurately to a blind person.

Obstacle detection using Microsoft Kinect

This thesis presents the development of image processing algorithms based on a Microsoft Kinect camera system. The algorithms developed during this thesis are applied on the depth image received from Kinect and are supposed to model a three dimensional object based representation of the recorded scene. The motivation behind this thesis is to develop a system which assists visually impaired people by navigating through unknown environments. The developed system is able to detect obstacles in the recorded scene and to warn about these obstacles. Since the goal of this thesis was not to develop a complete real time system but to invent reliable algorithms solving this task, the algorithms were developed in MATLAB. Additionally a control software was developed by which depth as well as color images can be received from Kinect. The developed algorithms are a combination of already known plane fitting algorithms and novel approaches. The algorithms perform a plane segmentation of the 3D point cloud and model objects out of the received segments. Each obstacle is defined by a cuboid box and thus can be illustrated easily to the blind person. For plane segmentation different approaches were compared to each other to find the most suitable approach. The first algorithm analyzed in this thesis is a normal vector based plane fitting algorithm. This algorithm supplies very accurate results but also has a high computation effort. The second approach, which was finally implemented, is a gradient based 2D image segmentation combined with a RANSAC plane segmentation (6) in a 3D points cloud. This approach has the advantage to find very small edges within the scene but also builds planes based on global constrains. Beside the development of the algorithm results of the image processing, which are really promising, are presented. Thus the algorithm is worth to be improved by further development. The developed algorithm is able to detect very small but significant obstacles but on the other hand does not represent the scene too detailed such that the result can be illustrated accurately to a blind person.

Globally Optimal Point Set Registration by Joint Symmetry Plane Fitting

The present work proposes a solution to the challenging problem of registering two partial point sets of the same object with very limited overlap. We leverage the fact that most objects found in man-made environments contain a plane of symmetry. By reflecting the points of each set with respect to the plane of symmetry, we can largely increase the overlap between the sets and therefore boost the registration process. However, prior knowledge about the plane of symmetry is generally unavailable or at least very hard to find, especially with limited partial views. Finding this plane could strongly benefit from a prior alignment of the partial point sets. We solve this chicken-and-egg problem by jointly optimizing the relative pose and symmetry plane parameters. We present a globally optimal solver by employing the branch-and-bound paradigm and thereby demonstrate that joint symmetry plane fitting leads to a great improvement over the current state of the art in globally optimal point set registration for common objects. We conclude with an interesting application of our method to dense 3D reconstruction of scenes with repetitive objects.

An Improved Building Reconstruction Algorithm Based on Manhattan World Assumption and Line-Restricted Hypothetical Plane Fitting

An improved patch-based multiview stereo (PMVS) algorithm based on Manhattan world assumption and the line-restricted hypothetical plane fitting method according to buildings’ spatial characteristics is proposed. Different from the original PMVS algorithm, our approach generates seed points purely from 3D line segments instead of using those feature points. First, 3D line segments are extracted using the existing Line3D++ algorithm, and the 3D line segment clustering criterion of buildings is established based on Manhattan world assumption. Next, by using the normal direction obtained using the result of 3D line segment clustering, we propose a multihypothetical plane fitting algorithm based on the mean shift method. Then, through subdividing on the triangle mesh constructed based on the building hypothetical plane model, semidense point cloud can be quickly obtained, and it is used as seed points of the PMVS pipeline instead of the sparse and noisy seed points generated by PMVS itself. After that, dense point cloud can be obtained through the existing PMVS expansion pipeline. Finally, unit and integration experiments are designed; the test results show that the proposed algorithm is 15%∼23% faster than the original PMWS in running time, and at the same time, the reconstruction quality of buildings is improved as well by successfully removing many noise points in the buildings.