Depth Map and 3D Imaging Applications: Algorithms and Technologies

This book has three editors, and all of us are involved in image processing and computer vision research. We have contributed to the 3D imaging research, especially in the field of passive optical 3D shape recovery methods. Over the last decade, significant progress had been made in 3D imaging research. As a result, 3D imaging methods and techniques are being employed for various applications. The objective of this book is to present various 3D algorithms developed in the recent years and to investigate the application of 3D methods in various domains.

This book is divided into five sections. Section I presents various 3D imaging algorithms that are developed in recent years. It covers quite a variety of research fields including 3D mapping, holography, and 3D shape compression. Six chapters are included in Section I. Section II deals with 3D shape recovery methods that fall in the optical passive as well as active domains. The topics covered in this section include shape from focus, shape from heating, and shape from fluorescence. Section II includes 5 chapters.

Section III is dedicated to stereoscopic vision and autostereoscopic vision. The dedication of a whole section to stereoscopic and autostereoscopic vision emphasizes the importance of these two technologies. Seven chapters are included in this section. Section IV discusses 3D vision for robotic applications. The topics included in this section are 3D scene analysis for intelligent robotics and usage of stereo vision for various applications including fire detection and suppression in buildings. This section has three chapters.

Finally, section V includes a variety of 3D imaging applications. The applications included in this section are 3D DMB player, 3D scanner, 3D mapping, morphological study of meteorite impact rocks, 3D tracking, 3D human body posture estimation, 3D face recognition, and 3D thumbnails for 3D videos. A total of nine chapters are included on several of the above mentioned applications in this section.

There are 31 chapters in this book. Chapter 1 is not included in any of the sections as it provides an introduction to 3D imaging. Chapter 1 briefly discusses the classification for 3D imaging. It provides an overview of the 3D consumer imaging products that are available commercially. It also discusses the future of 3D consumer electronics.

Section I

Chapter 2 to chapter 7 are included in this section. Chapter 2 discusses multi-view stereo reconstruction as well as shape from silhouette method. Multiple images are used with multiple views for 3D reconstruction. This chapter can be included in both section II and section III since section II deals with methods like shape from silhouette while section III covers stereovision. However, we decided to put it as the first chapter of section I because it presents an algorithm dealing with 3D shape reconstruction and also because we want to emphasize the importance of these two topics at the very beginning of this book.

Chapter 3 deals with the iterative reconstruction method that can be used in various medical imaging methods like X-ray, Computed Tomography, Positron Emission Tomography, Single Photon Emission Computed Tomography, Dose-calculation in Radiotherapy, and 3D-display Volume-rendering. This chapter is included in the book to emphasize on the importance of 3D transmissive methods that have greatly influenced our present day life style by improving the healthcare services.

Chapter 4 provides methods for generating 3D maps of the environment surrounding us. These maps are especially useful for robot navigation. This chapter especially discusses 3D map registration in detail.

Chapter 5 emphasizes the importance of compression for data storage and transmission for large chunks of 3D data. It describes a 3D image compression method that could reduce the data storage and transmission requirements.

Chapter 6 addresses holographic images. The future of true 3D lies in the holographic imaging technology. The holographic images are marred with noise and low quality. Hence, restoration and enhancement are very important for holographic images. This chapter summarizes related issues and provides solution for the restoration and enhancement of the holographic images.

Chapter 7 is the last chapter in section I. This chapter deals with an active optical 3D shape recovery method. For active fringe patterns projection, off-the-shelf projector is used in order to reduce the cost of the system.

Section II

Chapter 8 to chapter 12 are included in section II. Chapter 8 gives a very good introduction of the 3D shape recovery approaches. It includes the geometric approaches, photometric methods, and the real aperture techniques. Details are provided for various methods and techniques falling under one of the three approaches.

Chapter 9 discusses the focus measures in detail. A total of eleven focus measures are discussed, and they are categorized under four major classes. A very detailed comparison is provided for the eleven focus measures. The performance comparison is provided with respect to several types of noise, varying illumination and various types of textures.

Chapter 10 uses S-Transform for developing a focus measure method. High frequency components in the S-transform domain are targeted by the developed focus measure. The focus measure is used as a shape from focus technique to recover the 3D shape.

Chapter 11 uses genetic programming for developing a focus measure. An optimal composite depth function is developed, which utilizes multiple focus measures to get the optimized depth map for 3D shape recovery.

Chapter 12 provides two methods for recovering 3D shape of the transparent objects. Using normal optical methods, the 3D shape of transparent objects cannot be recovered accurately and precisely. This chapter discusses shape from heating and shape from fluorescence techniques to recover the 3D shape. These are new methods and have been introduced recently.

Section III

Chapter 13 to chapter 19 are included in section III. Chapter 13 to chapter 17 are related to stereoscopic vision, while the last two chapters in this section are on autostereoscopic vision. Although these two topics can be placed under section II, they have been placed in a separate section because of their importance in terms of consumer electronics.

Chapter 13 discusses a stereoscopic algorithm which treats the stereovision as modular approach. Hence, the stereovision algorithm can be divided into various stages and each of the stage can be implemented individually.

Chapter 14 and chapter 15 discuss applications of the stereovision. Off road intelligent vehicle navigation using stereovision in the agricultural environment is dealt in chapter 14 while chapter 15 discusses visually induced motion sickness (VIMS) that is associated with stereoscopic movies.

Chapter 16 provides details of viewpoint interpolation methods that are used for synthesizing the in-between views from few views that are captured by few fixed cameras. Chapter 17 presents a reversible watermarking based algorithm to deal with the high costs of memory, transmission bandwidth and computational complexity for 3D images.

Chapter 18 and chapter 19 deal with autostereoscopic vision. Stereoscopic displays require 3D glasses to view in 3D while the autostereoscopic displays do not require any 3D glasses. Chapter 18 introduces the basic concepts of autostereoscopic displays and discusses several of its technologies. Chapter 19 addresses the very important issue of bandwidth for high resolution multi-view autostereoscopic data.

Section IV

Chapter 20 to chapter 22 are included in section IV. This is the shortest section in this book. Although, all the three chapters in this section could easily be included in section III but we decided to allocate a separate section to emphasize the topic of robotic vision.

Chapter 20 is an invited chapter. It deals with intelligent robotics by capturing and analysing a scene in 3D. Real time processing is important for robotic applications and hence this chapter discusses limitations for the analysis of 3D data in real time. This chapter provides very good description of various technologies that address the limitation issues for real time processing.

Chapter 21 and chapter 22 use the stereovision for robotic applications. Chapter 21 discusses the autonomous operation of robots in real working environments while chapter 22 deals with the specific application of fire detection and suppression in the buildings.

Section V

Chapter 23 to chapter 31 are included in this section. Nine chapters deal with nine different 3D applications. It is the last section of the book. However, some of the applications dealing with stereovision, robotics and compression are also discussed in earlier sections. We placed them in those sections because we think that they are more relevant to the topics in those sections.

Chapter 23 discusses a 3D DMB player. DMB stands for digital multimedia broadcasting, and it is used for terrestrial-DMB (T-DMB) systems. The chapter also introduces an approximation method to create auto-stereoscopic images in the 3D DMB player. Hence, this chapter is also related to section III where autostereoscopic vision is discussed.

Chapter 24 presents a detailed overview of the 3D scanning technologies. Comparison of several 3D scanning methods is provided based on accuracy, speed, and the applicability of the scanning technology.

Chapter 25 deals with 3D mapping in outdoor environments, while chapter 26 presents 3D scanning method to study morphology of a meteorite rock. For 3D mapping, examples are taken from pavement runway inspection and urban mapping. For 3D scanning, meteorite rock is selected from the Karikkoselkä impact crater (Finland).

Chapter 27 discusses 3D tracking for mixed reality. 3D tracking is one of the active research areas in 3D imaging. This chapter addresses 3D tracking in mixed reality scenario. Mixed reality deals with virtual objects in real scenes. It is a very important topic with applications in medical, teaching, and gaming professions. Multi-sensor fusion methods for mixed reality with 3D camera tracking are discussed in this chapter.

Chapter 28 uses stereovision for the reconstruction of 3D human body posture that is further utilized in human activity recognition. Human activity recognition is of vital importance for visual surveillance applications. Hence, interest in human activity recognition research has increased manifolds in the recent years.

Chapter 29 deals with 3D face recognition, while chapter 30 discusses 3D face expression recognition. In chapter 29, a method for 3D face recognition is presented based on adaptive non-uniform meshes. In chapter 30, a feature extraction method is discussed that does not require any neutral face for the test object.

Chapter 31 is the last chapter of this section, as well as the last chapter of the book. Chapter 31 introduces a thumbnail format for 3D videos with depth. A framework is presented in the chapter that generates 3D thumbnails from layered depth video (LDV) and video plus depth (V+D).

Final Words

The work on this book started in November 2009 and it has taken about one and a half years to complete it. All the chapters in this book went through multiple reviews by the professionals in the field of 3D imaging and 3D vision. All the chapters had been revised based on the comments of multiple reviewers by the respective authors of the chapters. Contributors for the book chapters come from all over the world, i.e., Japan, Republic of Korea, China, Australia, Malaysia, Taiwan, Singapore, India, Tunisia, Turkey, Greece, France, Spain, Belgium, Romania, Netherlands, Italy, and United States. This indicates that this book covers a topic of vital importance for our time, and it seems that it will remain so at least for this decade.

3D imaging is a vast field and it is not possible to cover everything in one book. 3D research is ever expanding and the 3D research work will go on with the advent of new applications. This book presents state of the art research in selected topics. We hope that the topics presented in this book attract the attention of researchers in various research domains who may be able to find solutions to their problems in 3D imaging research. We further hope that this book can serve as a motivation for students as well as researchers who may pursue and contribute to the 3D imaging research.

Aamir Saeed Malik
Tae-Sun Choi
Humaira Nisar