Open Source Cloud Computing Systems: Practices and Paradigms

In Asimov’s trilogy, “Foundation,” the master author described a rotten empire, condemned to ashes, at the edge of leaving no trace of civilization for 30,000 years. A man struggled to reduce this ominous period to merely a thousand years by creating a generation of scientists who would preserve human knowledge.

In addition to accelerating the Passover through this dark era, the pupils of Asimov’s hero, the mathematician Hari Seldon, aimed at reducing the shadow period by manipulating the future at key strategic points. To this end, two foundations were created, a “visible” one, Foundation I, and an “invisible” one, Foundation II, which worked underneath to modify the course of history to make it follow the desired path of events.

Asimov’s series offers us an analogy of how Information Technologies (IT) have become a key element for the advance of human knowledge, an essential tool for reducing the time civilization has spent in dark times in which knowledge belonged to a few privileged ones. Take for instance the approximately 1000 years in which knowledge was kept in monasteries all over Europe.

The first “Foundation” was started in the times of the Enlightenment with the Encyclopedia and boosted in the 20th century with the advent of IT technologies. These helped to improve industrial processes and to optimize benefits and margins by automation. Increased benefits and division of labor contributed to increased welfare and the appearance of the mid classes. However, the task of this first foundation was still incomplete, it was still clear that the dark times were still casting some shadow. The long-envisioned goal of offering information and computing power as utilities was still far from accomplished. Information has remained as the property of organizations. They have had the computational power and resources to keep and manage information as needed. Knowledge belonged to those who created it and should be universally and ubiquitously accessible if the future was to be changed and Passover accelerated.

We are living exciting times. The second “Foundation” has started right in front of us. 20th century's nineties placed the first stone of it with the birth of the World Wide Web. This has been THE mechanism to democratize publishing and accessing of information of the most varied sources and subjects. However, a cornerstone was still missing for this foundation to be fully built: means to access ubiquitously these data together with the power to process this enormous amount of information were still required beyond organizational boundaries.

Cloud computing seems to be the missing stone in this foundation. Its promise of delivering every available networked resource as a service implies ubiquitous access and that virtually anyone can grasp the power needed to deal with huge amounts of data.

Cloud computing is a deep, and controversial, field and there is much more to it than one could find in any one book. The available literature is huge and some doubts still remain about the differences with the computational grid or the different layers in its theoretical architecture. In addition, the Cloud is a rapidly moving target and fixing some concepts will take time and effort.

What follows is not in any way a textbook on Cloud Computing, nor do we try to offer an account of it features, applications, pros or cons, etc. The book's focus is rather on how the scaffolding sustaining this second “Foundation” can be taken everywhere for a reduced cost in order to illuminate human nature by exposing unprecedented data and processing power beyond organizational boundaries.

Like Hari Seldon's followers, this reader is aimed at practitioners and academics aiming to build their own cloud for a wide variety of reasons. The knowledge required for the second Foundations is widespread, segmented into a wide variety of products and names that can make the picture really cloudy. Gathering together a comprehensive list of some of the most prominent cloud technologies is a definitive advantage for those aimed at sailing the deep waters of the cloud.

However, the book should not be taken as a manual for newbies. It also contains a step by step installation guide that can make life of system administrators much easier. The authors tried to point to the relevant references about their system, keeping a practical view on it, which constitutes a very nice filter to get to the most accurate and up to date information.

In addition to this more technical view, the authors have tried to sum up the most relevant lessons learned and pitfalls in their systems initial design. These constitute very valuable lessons for those trying to overcome these systems' limitations and/or build systems of their own. We all can be enlightened by having a detailed architectural view that matches with a strategy selection for our research and products.

The book attempts to give any interested reader a flavor of what cloud systems are handy for them to build their own cloud by using Open Source tools created by the Community at the frontiers of knowledge. The book probably presents the most relevant Open Source Cloud technologies available today with a rather practical perspective. The final objective is presenting a practical compendium of cloud technologies, where to get them, how to install them, when is it appropriate to use one technology, etc. In other words, the book tells a story of skies exploration, separating clouds of knowledge from ignorance clouds, thus fostering a generation of Hari Seldon's pupils. We truly believe it’s a story worth telling.

Chapter 1 offers a general overview of the field, open source products and technologies, while it also justifies the need for these systems. A brief, but complete, tour around the broad view of the cloud arena at the very different levels, IaaS, PaaS and SaaS is performed. The Chapter also introduces and contextualizes the main technologies as described in the book.

Next, the Chapter covers one of the major European initiatives in open source cloud computing: OpenNebula. Mainly developed with EU funding at its early stages, OpenNebula has evolved and matured to be one of the reference implementations for IaaS clouds. Its scientific, standard, and implementation contributions led OpenNebula to be a vibrant community and, arguably, the most solid product of its level as of today.
Another of such research efforts is presented in Chapter 3; EMOTIVE is another European effort to manage infrastructure clouds. EMOTIVE is gaining momentum in the research community and deserves careful attention given its exponential evolution.
Having some of the major players in the open source IaaS level, does not mean that some young initiatives are not trying to solve some of the problems identified in the lessons learned by previous experiences. Chapter 4 is focused on a new experimental system for managing virtual resources in federated clouds. The federation capabilities are proving essential in a highly heterogeneous market with high risks for vendor lock-in. In this same line, Chapter 5 presents a mechanism for enriching user experience in the cloud. This hard to accomplish target is tackled by enriching resource allocation with semantics.

Above the mere IaaS-bones a myriad of new services have arisen that fell in the border of IaaS and PaaS clouds. One of such systems is Claudia, Chapter 6. Claudia aims at offering a holistic service management experience on top of the cloud. Being born of partially funded EU projects, Claudia presents two different faces: a commercial and enhanced version, released as a product, and an open source research counterpart where new innovations are tested.

In addition to high-level management primitives, security is one of the major concerns of users when it comes to cloud adoption. Being capable of locking your cloud in hostile federated environments is even more complex than it was doing so in traditional IT systems. Chapter 7 introduces one of such systems aimed at securing the external interfaces (those potentially more vulnerable) of your IaaS cloud.

Going well above the IaaS layer, one finds a huge amount of services that help developers to create, run and maintain their applications: that is the PaaS layer of the cloud. We have gathered together two of the most relevant initiatives in this line. This layer can be seen as the heir of the traditional “application servers”, such as Tomcat. Indeed, Google and Amazon are already offering these as a service. Oracle´s virtual Java Service Containers (Chapter 8) and AppScale (Chapter 9) are the two most relevant open source initiatives in this level of the cloud. If one wants to run these containers as a service (e.g. to support a developers community), reading these two Chapters is a definitive must for making safe first steps.

Finally, there are a handful of higher-level PaaS services that offer you coordination among your distributed systems, logging capabilities, data mining algorithms, increased data management scalability, etc. Hadoop is one of such examples for data-intensive applications. Chapter 10 covers Hadoop essentials in a very concise, but complete and illustrative, manner. The reader is recalled here that the Chapter covers the most relevant service offered under the Hadoop umbrella. Hadoop today is a flourishing community, which gathers together a variety of services beyond the essentials herein covered.

Taking into account this book's practical approach, being highly focused on the technical aspects and lessons learned in the system design and implementation, we can safely say that it is the first of its kind in the area. This puts it in contrast to most other available book, which took a more descriptive approach and kept the hands-on lessons aside for the sake of emphasizing research or pedagogical targets.

Given the quality of the Chapters we deem it to be one of the main references in the field when it comes to practical purposes and systems' design. Also, the experience of the authors in writing scientific, technical, and marketing documents is self evident as one goes through the text.

Luis M. Vaquero
HP Labs, UK

Juan Cáceres
Telefonica R&D Labs, Spain

Juanjo Hierro
Telefonica, Spain