MapReduce and Hadoop

This chapter introduces the MapReduce solution for distributed computation. It explains the fundamentals of MapReduce and describes in which scenarios it can be applied (basically, processing of massive data by easily parallelizable algorithms). Also, this chapter gives an overview of the open source project Hadoop, an implementation of MapReduce. Its architecture is depicted, and an easy step-by-step guide to install Hadoop is included, along with programming examples of how to use Hadoop.

Virtual Java Service Container

First, the authors present several cloud solutions, then they describe the architecture of the VJSC framework, including practical coding and usage examples, followed by the lessons learned as a wrap-up.

Securing the External Interfaces of a Federated Infrastructure Cloud

This chapter describes an open source solution for securing the Claudia service manager and the OpenNebula virtual execution environment manager when combined in a federated RESERVOIR architecture. The security services provide confidentiality, authentication, and integrity by securing the external API. The chapter describes how to integrate the security solution in an open source cloud computing system, how to install it, and provides an illustrative case study showing its potential for the community. The aim of the chapter is to help those who want to build their own secure infrastructure clouds. The open source security code provides mutual authentication between clients and the Claudia service manager, and secures the SMI interface with role based access control. The same security services can also secure the VMI with role based access control and X509 certificates. Finally the federation can be secured by combining an LDAP server to manage the federation and XACML security policies, and using policy matching to guarantee the respect of security policies within the federation.

An Innovative Open Source Middleware for Managing Virtual Resources in Federated Clouds

This chapter describes the CLEVER architecture, focusing on its design process and highlighting the communication mechanisms employed by its entities. The last part of the chapter discusses how CLEVER could be easily integrated with higher-level cloud middleware for managing service deployment according to the cloud users’ requests.

Resource Allocation for Cloud Computing

Cloud computing has emerged as a distributed computing paradigm where resources are requested on demand and in a very dynamic fashion and paying only for what you consume. This new paradigm created an ecosystem where several providers offer heterogeneous computing resources to satisfy the customers’ computing demand. So, the allocation and adaptation of this demand to the correct resources is a key issue in this ecosystem, because it can produce a mutual benefit for the customers and providers. However, with the wide variety of customers and providers, this allocation is not an easy task. This chapter presents a toolkit that implements a methodology for improving the resource allocation between different Cloud providers. The Semantically Enhanced Resource Allocator (SERA) toolkit introduces the semantic web and multi-agent technologies for facilitating the interoperability between the users and different resource providers. Semantic web technologies provide the required semantic interoperability between the different providers’ vocabularies; meanwhile a platform of configurable agents provides an adaptable and autonomous way of allocating and managing execution requests and resources according to the customers and providers rules.

EMOTIVE Cloud

This chapter introduces Elastic Management of Tasks in Virtualized Environments (EMOTIVE), which is the Barcelona Supercomputing Center (BSC)’s IaaS open-source solution for Cloud Computing. EMOTIVE provides users with elastic fully customized virtual environments in which to execute their applications. Further, it simplifies the development of new middleware services for managing Cloud systems by supporting resource allocation and monitoring, data management, live migration, and checkpoints. These features and its facility to be extended and configured make EMOTIVE especially appropriate to support research on Cloud Computing scenarios. Offering functionality comparable to its commercial counterparts allows EMOTIME to be used on production to set up small Cloud platforms.

The OpenNebula Cloud Toolkit

This chapter aims to describe the OpenNebula Cloud Toolkit, a framework born as a result of many years of research and development that intends to provide an efficient and scalable solution for the large-scale distributed management of Virtual Machines running on a pool of physical resources. A description of the history of the project is presented, along with a detailed explanation of the characteristics of the Toolkit, including directions on how to install and configure the software, enumeration of the interfaces exposed, and information on how to use and configure the three types of clouds (private, hybrid, and public) that can be built with the framework. The chapter ends with a practical use case that shows how to deploy a service composed of Virtual Machines on top of an OpenNebula cloud, and with the lessons learned during the project and the future work planned for the Toolkit.

Claudia Service Management Platform

Claudia is an IaaS solution created by Telefonica R&D aimed at managing services as whole entities and adding advanced features such as elasticity, service level, hardware monitoring, and virtual machine autoconfiguration. Claudia also provides an abstraction layer on top of both private and public Virtualization Providers. In this chapter, the Open Source version of Claudia will be installed, and it will be used to show how a load-balanced Web service can be deployed and managed in the system.

Open Cloud Technologies

The open source perspective offers an interesting insight about cloud computing technologies: in one hand, cloud systems belong to the category of the Ultra-Large-Scale (ULS) systems, i.e. very complex systems where conventional approach for the technological development does not work. For such as systems, Free Libre Open Source Software (FLOSS) licensing attracts innovation from the developers’ communities, reduces the risks of technology adoption and fosters the interoperability between systems and the creation of open standards. In the other hand, the current systems are far from achieving interoperability; even the FLOSS´s principles remain pending for many components in the architecture of the main cloud solutions, and for these reasons many FLOSS evangelists do not recommend using them. As a balance between the obvious drawbacks and benefits, recently a new strategy has appeared: Free/Open Services. However, it seems difficult to find short term solutions. This chapter illustrates both ideas, highlighting the pros and cons of these technologies, including a reference of main “open cloud” groups and open source technologies for the cloud. The rest of the book will include additional and deeper descriptions of some of the most interesting open cloud technologies.

AppScale

The goal with AppScale is to enable research and experimentation into cloud computing and to facilitate a “write once, run anywhere” programming model for the cloud, i.e., to expedite portable application development and deployment across disparate cloud fabrics. In this chapter, the authors describe the current AppScale APIs and the ways in which users can deploy AppScale clouds and applications in public and private settings. In addition, they describe the internals of the system to give insight into how developers can investigate and extend AppScale as part of research and development on next-generation cloud software and services.