• Preface
• Big Data Basics
• Cloud Computing
• NoSQL
• Multi-tenancy
• Database-as-a-Service
• Scalability Options
• CAP Theorem
• Parallel Databases
• Big Data Storage
• Google BigTable/HBase
• Amazon Dynamo
• Amazon SimpleDB
• Amazon S3
• Windows Azure Table and Blob Storages
• Cassandra
• HyperTable
• MongoDB
• CouchDB
• Voldermot
• Database-as-a-Service
• Amazon RDS
• Microsoft SQL Azure
• Google Data Store
• Google Cloud SQL
• Heroku Postgress
• Salesforce
• Big Data Batch Processing
• MapReduce/Hadoop
• HaLoop
• iMapReduce
• Twister
• Pig
• Hive
• Dryad/DryadLinq
• Jaql
• HadoopDB
• Spark
• Stratosphere
• SpatialHadoop
• Dremel/BigQuery
• Amazon Redshift
• Apache Mahout
• Hadoop Online
• Twitter Storm
• Pregel
• GraphLab
• Pegasus
• Trinity
• Echo System
• Apache ZooKeeper
• Apache Sqoop
• Apache Flume
• Apache Oozie
• Apache Thrift
• Cascading
• Google Fusion Tables
• InfoSphere BigInsights

Cloud Computing

Recently, there has been a great deal of hype about cloud computing. Cloud computing is associated with a new paradigm for the provision of computing infrastructure. This paradigm shifts the location of this infrastructure to the network to reduce the costs associated with the management of hardware and software resources. Hence, businesses and users become able to access application services from anywhere in the world on demand. Therefore, it represents the long-held dream of envisioning computing as a utility where the economy of scale principles help to drive the cost of computing infrastructure eectively down. Big players such as Amazon, Google, IBM, Microsoft and Sun Microsystems have established their data centers for hosting cloud computing applications in various locations around the world to provide redundancy and ensure reliability in case of site failures. The figure below illustrates an example of a server farm in one of data centers for one of the big providers of cloud computing resources.

In fact, there has been much discussion in industry and academia about what cloud computing actually means. The US National Institute of Standards and Technology (NIST) has developed a working definition that covers the commonly agreed aspects of cloud computing. The NIST definition is widely referenced in US government documents and projects. It summarizes cloud computing as: "model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction". According to this definition, cloud computing has the following essential characteristics:

1. On-demand self-service: A consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with each service's provider.
2. Broad network access: Capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).
3. Resource pooling: The provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with dierent physical and virtual resources dynamically assigned and reassigned according to consumer demand. There is a sense of location independence in that the customer generally has no control or knowledge over the exact location of the provided resources but may be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter). Examples of resources include storage, processing, memory, network bandwidth, virtual networks and virtual machines.
4. Rapid elasticity: Capabilities can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and rapidly released to quickly scale in. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be purchased in any quantity at any time.
5. Measured Service: Cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported providing transparency for both the provider and consumer of the utilized service.

The denition also specifies the following levels of abstraction which are referred to as service models:

1. Infrastructure as a Service (IaaS): Provision resources such as servers (often in the form of virtual machines), network bandwidth, storage, and related tools necessary to build an application environment from scratch (e.g. Amazon EC2, GoGrid, Eucalyptus, Rackspace, Terremark, Joyent).
2. Platform as a Service (PaaS): Provides a higher-level environment where developers can write customized applications (e.g. Microsoft Azure, Google AppEngine, Cloud Foundry, Appfog, Gigaspaces). The maintenance, load-balancing and scale-out of the platform are done by the service provider and the developer can concentrate on the main functionalities of his application.
3. Software as a Service (SaaS): Refers to special-purpose software made available through the Internet (e.g. SalesForce). Therefore, it does not require each end-user to manually download, install, congure, run or use the software applications on their own computing environments.