The methods used to digitize and reconstruct the shapes of complex 3D objects have evolved rapidly in recent years due to attention from many industrial as well as research groups. To capture complete shape of an object, many thousands of samples must be acquired. The resulting mass of data requires algorithms that can efficiently and reliably generate computer models from these samples. Earlier, 3D models were used primarily in robotics and computer vision applications. The models for such applications require only salient geometric features so that the objects can be recognized. Therefore, it was unnecessary in these applications to faithfully capture every detail on the surface of the object. However, more recently, there has been considerable interest in the construction of 3D models for applications where the focus is more on visualization by humans. Obviously, the 3D models constructed must capture, to the maximum extent possible, the shape and the surface-texture information of real-world objects.
3D surface reconstruction from images is common to several research domains and there have been a number of attempts to model the 3D geometry of objects and scenes from images. These attempts provide a complete geometrical 3D description from a sequence of 2D images. The demand for constructing 3D models has been steadily growing, and it will continue to grow in the future due to its wide-ranging applications in various domains like reverse engineering, collaborative design, inspection, entertainment, virtual museums, medicine, geology, and home shopping.
This book aims to provide relevant theoretical frameworks and the latest empirical research findings in this expanding field. It is important for the readers to understand this technology and its benefits. This publication aims to provide developers and scholars with an extensive collection of research articles from the expanding field of 3D reconstruction. It deals with the concepts, methodologies, applications, and recent developments in this emerging field. An outstanding collection of latest research associated with advancements in 3D surface reconstruction is presented in this book. It is written for students, researchers, academics, professionals and industry practitioners working in this area who want to improve their understanding of the inter-related topics.
The prime intended audience of the book corresponds to educators, students, practitioners, professionals, and researchers working in the field of 3D surface reconstruction in various disciplines, e.g. computer science, electrical engineering, electronics engineering, systems science, and Information Technology. Moreover, the book will be a valuable and multifaceted resource that will provide insights about where the technology is going and will give a sample of some of the most interesting applications, critical issues, and emerging trends. This publication will be invaluable to all those required to use theoretical analysis, algorithms, and practical applications of 3D surface reconstruction technologies. It is also for those who want to gain a complete understanding of all pertinent aspects of 3D surface reconstruction. Finally, this book will be a welcome addition to academic, research, governmental, and public administration libraries’ research collections.
The book presents a selection of 14 high-quality chapters, written by 27 authors from 9 different countries. The book is organized into three sections: Introductory Chapters, 3D Reconstruction, and Real-World Applications, each of which is described briefly below.
Section 1 contains three chapters that describe fundamental aspects and give an overview of different methods applied in 3D reconstruction. The first chapter contains an overview of methods for a 3D shape from both the surface and the internal structure of the objects. The second chapter surveys many fundamental aspects of projective geometry that have been used extensively in computer vision, and the third chapter describes the basic concepts of partial differential equations based image modelling.
Chapter 1, entitled “Methods of 3D Object Shape Acquisition” by Pavel Zemcik, Michal Spanel, Premysl Krsek, and Miloslav Richter, describes the methods for acquisition of 3D data from surface as well as internal structure of the existing objects. The acquisition methods of interest are optical methods based on objects surface image processing and CT/NMR sensors that explore the object volume structure. The focus is on 3D surface shape acquisition methods based on multiple views, methods using single view video sequences, and methods that use a single view with a controlled light source. A set of algorithms suitable for the acquired 3D data processing and simplification are shown to demonstrate how the models data can be processed.
Chapter 2, entitled “Projective Geometry for 3D Modeling of Objects” by Rimon Elias, discusses the basic elements of projective geometry that is needed to reconstruct objects in 3D space. In particular, it discusses the role of this branch of geometry in reconstructing basic entities (e.g. 3D points, 3D lines and planes) in 3D space from multiple images. It investigates the geometrical relationships when one or two cameras are observing the scene creating single-view and two-view geometry. Finally, different approaches to deal with the existence of noise or inaccuracy in general are presented.
Chapter 3, entitled “PDE-based Image Processing: Image Restoration” by Rajeev Srivastava, explains partial differential equation (PDE) based approaches for image modelling and processing for image restoration task. The general basic concepts of partial differential equation based image modelling and processing techniques are discussed. As a case study, the topic in consideration is oriented towards image restoration using PDEs formalism since image restoration is considered to be an important pre-processing task for 3D surface geometry and reconstruction and many other applications. An image may be subjected to various types of noises during its acquisition leading to degraded quality of the image. Here, the PDE-based models for removal of these noises are discussed.
Section 2 contains six chapters, dealing with different areas of 3D reconstruction of objects, scenes, and environments. The first chapter of this section describes a surface reconstruction method which mixes interpolating as well as approximating features and its implementation in graphics hardware. The second chapter explores 3D reconstruction of underwater natural scenes and objects based on stereo vision, whereas the next chapter provides a basic understanding of how 3D statistical visual displays aid in education. The fourth chapter proposes depth estimation for stereo pair of high dynamic range images. The fifth chapter presents fusion of 3D reconstructions generated by two seminal monocular-cue based reconstruction algorithms. and the last chapter proposes a method to extract 3D models from single view perspective images.
Chapter 4, entitled “Hybrid GPU Local Delaunay Triangulation through Points Consolidation” by Carlos Buchart, Aiert Amundarain, and Diego Borro, presents a hybrid reconstruction method by combining interpolating and approximating features together in order to be implemented efficiently in parallel architectures. Hybrid methods are useful in areas such sculpting, medicine, and cultural heritage, where details must be preserved. The proposed method makes use of a point projection operator to create a regular distributed and noise free set of points, which is reconstructed using local Delaunay triangulations. Both points projection and triangulation methods are studied in its basic serial version, but aiming to design parallel versions (more specifically GPU implementations) that increase their performance. The adaptations required for the parallel reconstruction are discussed, and several implementation details are given.
Chapter 5, entitled “3D Reconstruction of Underwater Natural Scenes and Objects using Stereo Vision” by Prabhakar C.J., Praveen Kumar P.U., and Hiremath P.S., proposes a preprocessing technique to enhance degraded underwater images as well as a stereo vision based 3D reconstruction technique to reconstruct 3D surface of underwater objects. 3D reconstruction for underwater applications is a relatively recent research area with higher complexity than the 3D reconstruction for general applications. 3D reconstruction of underwater natural scenes and objects is a challenging problem due to light propagation in underwater. In contrast to light propagation in the air, the light rays are attenuated and scattered, having a great effect on image quality. The developed reconstruction technique is expected to be robust enough to reconstruct objects or scenes in a realistic manner. The system is robust, which means that it should be able to reconstruct the object or scene which is far away and captured in turbid water.
Chapter 6, entitled “3D Reconstruction of Graph Objects, Scenes and Environments” by Suhana Chikatla and Ukaiko Bitrus-Oijambo, focuses on the theoretical background, pedagogical practice, usability, and applicability of using 3D surface charts. It seeks to discuss the importance of surface objects, scenes, and environments reconstructed to enhance the interpretation of charts. Different types of 3D charts available: bar, line, and pie charts are described. The chapter also provides enlightenment about two new concepts, i.e. “3D actual” and “3D obvious” charts. Indeed, the visual communication theory provides a relevant framework from which educators can design and develop a tool to aid learners who need visually representative data via charts, graphs, and pictures to enhance learning.
Chapter 7, entitled “Depth Estimation for HDR Images” by Swamykannu Manikandan, introduces a stereo matching algorithm that analyses grayscale or color images to estimate the disparity map for 3D scene reconstruction. The proposed algorithm consists of two major techniques, namely conversion of High Dynamic Range (HDR) images to Low Dynamic Range (LDR) images or Standard Dynamic Range (SDR) images, and estimating the depth from the converted LDR / SDR stereo images. Local based tone mapping technique is used for the conversion of the HDR images to SDR images. Depth estimation is done based on the corner features of the stereo pair images and block matching algorithm.
Chapter 8, entitled “Monocular-Cues Based 3-D Reconstruction: A Comparative Review” by Sudheer Tumu, Viswanath Avasarala, Sai Tejaswi Jonnalagadda, and Prasad Wadekar, provides a brief review on 3D reconstruction, with a particular emphasis on monocular-cues based reconstruction. In recent years, some interesting breakthroughs have been made in constructing depth maps of images using monocular cues. Two recent 3D reconstruction techniques that use machine-learning algorithms trained by monocular cues are explained here. The success of these algorithms is their ability to not only use local features of image regions but also their global context in relation to the entire image. The fusion approach improves the 3-D estimation accuracy significantly as compared to the original approaches.
Chapter 9, entitled “Image Based 3D Modeling & Rendering from Single View Perspective Images” by Mohan S., and Murali S., addresses a method to construct 3D wireframes from single view perspective image based on edge length. In computer vision, 3D modeling refers to the process of developing 3D representation of the real world objects with systematic procedure. The 3D models can be built based on geometric information about the object or scene to be modeled using CAD/CAM software. A method for rectifying the perspective distortion is discussed. An application of touring into picture is also explained.
Section 3 consists of five chapters, describing the application of 3D reconstruction in various domains. The first chapter presents the use of discrete basis functions in surface modelling and its application to real-time automatic surface inspection. The second chapter introduces two kinds of applications of red, green and blue as a carrier and their testing by measuring the shape of objects’ surface. The next chapter discusses the representation of obstacles in an environment with planar ground through wide baseline set of images in the context of teleoperation, whereas the fourth chapter presents an algorithm for identifying complementary site of objects broken into two parts, and subsequently, reassembly of 3D fragments. The last chapter explains the use of an inexpensive passive method involving 3D surface reconstruction from video images taken at multiple views and the utility of the developed methodology for prosthetic designers.
Chapter 10, entitled “Surface Modelling using Discrete Basis Functions for Real-Time Automatic Inspection” by Paul O'Leary and Matthew Harker, focuses on the applications of discrete basis functions in surface modelling and automatic inspection. Emphasis is placed on a formal and stringent mathematical background, which enables an analytical a-priori estimation of the performance of the methods for specific applications. A completely new approach to synthesizing constrained basis functions is presented. The resulting constrained basis functions form a unitary matrix, i.e. are optimal with respect to numerical error propagation and have many applications, e.g. as admissible functions in Galerkin methods for solution of boundary value and initial value problems. A number of case studies are presented that show the applicability of the methods in real applications.
Chapter 11, entitled “Application of Red, Green and Blue Color Channels in 3D Shape Measurement” by Zonghua Zhang, presents the application of red, green, and blue channels as a carrier in measuring 3D shape of objects surface. Since three fringe patterns can be simultaneously projected and captured through a single composite RGB image, the acquisition time reduces to 1/3 of the value by the gray fringe pattern projection. Two kinds of application methods of red, green and blue as a carrier are discussed. The testing results confirm that red, green and blue channels can be used as a carrier to reduce the acquisition time. Optical full-field measurement techniques have been widely studied in academia and applied to many actual fields of automated inspection, reverse engineering, cosmetic surgery, and so on.
Chapter 12, entitled “Widely-Separated Stereo Views Turn into 3D Objects: An Application” by Rimon Elias, describes different steps proposed to perform scene modelling through wide baseline set of images. The camera parameters are assumed to be known approximately within some range according to the error margins of the sensors used such as inertial devices. The proposed technique is based on detecting junctions in all images using the so-called JUDOCA operator and through homographic transformation; correlation is applied to achieve point correspondences. The match set is triangulated to obtain a set of 3D points and point clustering is then performed to achieve a bounding box for each obstacle, which may be used for localization purposes by itself. Finally, a voxelization scheme is applied to determine a volumetric representation for each obstacle.
Chapter 13, entitled “Complementary Part Detection and Reassembly of 3D Fragments” by Vandana D. Kaushik and Phalguni Gupta, has explored a problem for determining the complementary part of a fragment of an object and of reassembling them to form the object. It has proposed an efficient surface inspection algorithm, which detects the corresponding cleavage sites of fragments and registers them so that the object can be formed from the given fragments. For a given 3D scanned image of broken objects, the algorithm identifies the rough sites of the broken object, transforms the object to a suitable alignment, registers it with its complementary part which belongs to the same object, and finds the local correspondence among the fragmented parts. The algorithm is found to be very effective on objects of ceramic material and archeological artifacts.
Chapter 14, entitled “3D Surface Reconstruction from Multiviews for Prosthetic Design” by Nasrul H. B. Mahmood, suggests an alternative approach to design prosthetic devices using multiviews reconstruction method and offers a significant advance for orthotic as well as prosthetic design by using an image processing technique. Existing methods that use a fringe projection technique for prosthetic designs produce good results for the trunk and lower limbs; however, the devices used for this purpose are expensive. The design and evaluation methodology, consisting of a number of techniques suitable for prosthetic design, is developed. The 3D model is obtained by a computer program, while the 3D data uses the shape-from-silhouette technique in an approximately circular motion. The methodology developed is shown to be useful for prosthetic designers as an alternative to manual impression during the design.
This book, thus, gathers contributions from various research domains that address 3D surface geometry and reconstruction from different perspectives, including both theoretical and experimental points of view. Putting together a diverse set of contributions to constitute a coherent whole is a challenging task. But also it is an enriching and rewarding experience. As is true of most writing efforts of this nature, progress continues after work on the manuscript stops. For this reason, significant effort has been devoted to the selection of material that is fundamental and whose value is likely to remain applicable in a rapidly evolving body of knowledge. I am really grateful to the contributors, not only because of their outstanding work, but because of all the new and interesting things I learned from them. The variety of points of view is one of the key features of this book, making it a precious guide for researchers, students and practitioners. I trust that readers of this book will benefit from this effort and thus find the material timely and useful in their work.
Umesh C. Pati
National Institute of Technology, India