Wednesday, April 11, 2012

Stormy forecast: Private clouds on the horizon are growing in importance

Wrote this for the IBM Thoughts on Cloud blog and reposting here in its entirety. Please use the ToC site for discussions.


Cloud computing is primarily manifest by the availability of compute capacity (primarily servers, storage, and networks) as virtualize resources.

Using a self-serving portal, available on the web, one can provision a complete unit of computation and set it up with custom or predefined software. When it is no longer needed, the resources are discarded, and you are simply charged for the time you used the resources.

The ease of use and agility that one experiences with public clouds can be replicated in a private manner for enterprises. This is especially useful to enterprises that are worried about security or that want more control over where their data and software are setup and reside. So, private clouds can be seen as a microcosm of public clouds…

Although this view is true, it is also not complete. Private clouds have other, deeper purposes, to enterprises.

Case studies

Enterprises, no matter the industry, are about running a collection of business processes (procedures involving customers, employees, partners, and IT) to achieve some goal. At the end of the day, these collections of business processes need to execute (within the enterprise’s values) the enterprise’s mission, which results (hopefully) in maximizing profits for stakeholders.

Executing these diverse business processes requires various IT resources. Private clouds allow enterprises to be flexible and agile in how they support the execution of these business processes. However, because business processes are not simply about executing software on servers, a private cloud needs to provide more than virtualize resources… Consider the following two simplified use cases of applying private clouds to enterprise business processes.

Case study 1: Application development lifecycle

As an enterprise matures, so does its business processes. However, some of these processes need to also evolve to address new market shifts and new realities. One such example is how web and now mobile computing have affected how any enterprise communicates with its customers. The result is that new custom web or mobile development is a common undertaking inside modern businesses.

Enterprises that decide to keep such development in house, need to have application development lifecycle processes in place so that such development can happen in an orderly fashion while respecting the various legal and business procedures. Further, when released, an optimizing enterprise would also want to know the ROI of such efforts and how satisfied their customers are with the new offerings.

All of this requires setting up teams to develop, test, and push applications in production. After that, each application needs to be maintained and connected with existing customer satisfaction software and legal and business audit processes. Executing these steps in a repeatable manner is key. As an enterprise becomes larger, it loses some of its agility—that comes with the territory. However, large enterprises do not need to lose their innovative spirit.

The way to retain their leaderships, large enterprises need to have infrastructures that provides knobs for flexible resource planning and allocation, repeatable deployments of infrastructure, and software, consolidation of common services, centralized management, and rock solid fault-tolerant hardware substrate. A private cloud solution must provide all of these aspects to be useful to a large enterprise.

Case study 2: Data processing and analytics

With the explosion of data caused by the popularity of social media, enterprises are finding that an efficient and practical way to stay in touch with their customers is to have a social presence. The result of these interactions through Twitter, Facebook, and Google+ is an avalanche of unstructured data in the form of comments, tweets, likes, and so on.

Analyzing such social media data has huge potential for an enterprise to better understand how their customers are reacting to new products or service offerings introduction, and also advertising campaigns. Getting concrete feedback by a posting from a customer on your enterprise’s Facebook page is direct interaction that, not long ago, was reserved for special marketing surveys or studies. Instead of going to customers, they are coming to the enterprises.

The new challenge for an enterprise is how to collect this “real-time” data and make sense of it. This analysis can be outsourced, but large enterprises understand the potential goldmine hiding behind this social data and want to analyze it themselves.

Effective analysis of unstructured social media data requires analytics engines and algorithms that are resource hogs. Lots of storage must be allocated for the stream of data and the analysis must be performed constantly. The gems of insights are discovered by monitoring the results frequently and comparing them with background data and trends. All of these processes have only one thing in common, need for fault-tolerant high-grade storage and are compute intensive.

The utility of a private cloud is best measured if it is flexible enough to run such analytics workloads. Whether it is by quickly setting up large Hadoop clusters to support an analysis or by supporting the ETL (extract, transform, and load) of large data warehouses that support dashboards from customized OLAP queries.

Future of private clouds: IaaS and PaaS

The evolution of private clouds: Secure IaaS + extensible PaaS

So the point here is simply this: private clouds need to provide more than a microcosm of public clouds. In addition to the obvious potential in increased security, a private cloud must also offer an enterprise the following benefits:

  1. Facilities to support repeatable execution of business processes

  2. Centralized management of all resources (hardware and software)

  3. Repeatable, manageable deployment of middleware with an open architecture that allows Open Source Software (OSS) components to be added to the mix

  4. Facilities to manage to provenance and governance of software and middleware parts

  5. Fault-tolerant, swappable, modular hardware design that can grow with the enterprise needs

  6. Flexibility in defining how the hardware is divided to be able to support diverse workloads, for example, transactional, web, and analytics

  7. Integrated support and service that can guarantee that your cloud is up and running 24×7 and that can help you minimize any downtime

IBM’s leadership and upcoming offerings

IBM’s leadership in private cloud with appliances such as WebSphere Cloudburst showed that our middleware was capable to support private clouds in an appliance fashion. What comes next is the complete soup to nuts, integrated solution that provides the seven characteristics that I listed and that determine a real private cloud.

Watch out, there is stormy weather ahead? Private clouds are more than you expect. Check out

dr. max