Monday, June 18, 2012

Should you virtualize your pattern or not?

This post is replicated from the IBM Expert integrated system blog.  Please visit that article if you want to comment or share.

In this post, I compare and contrast the IBM PureSystems virtual application and virtual systems patterns.  I gently introduce you to these two options and help you decide how to select the correct pattern model for your workloads.  In two subsequent posts, I will explain how to create plug-ins to extend IBM PureSystems virtual application patterns.

Defining workloads on the cloud
In most IaaS cloud computing environments, the unit of deployment is the image that gets deployed (or runs) on a virtual machine.  Each image typically comprises operating system software, dependencies, libraries, and utilities, and also the application middleware and software.  So, deploying an image results in a complete software stack.  It is then the responsibility of the developer (or IT operations) to select the correct image and make the necessary customizations and additional setups.
The problem with this approach is that it results in an explosion of base images, each trying to cover one of the various combinations possible for various software stacks.  The other problem is that the customization and setup of images can be non-trivial, resulting in various scripts, patches, and custom tweaking that can be difficult to maintain and keep current.
Fundamentally, the issue is that workloads tend to be built on standard stacks but contain variations.  These variations can be customization information to various ways in which some parts of the stack can be configured.  The resulting software combination, including the VMs running these services constitute what is called a workload topology.
Typically, fixing a workload topology for a particular business process can be difficult because non-functional requirements — for example, security, privacy, scalability, resiliency, and so on — vary with the purpose of the workload.  For instance, deploying an expense reimbursement web-based application requires a very different topology if it is to be used by a local company with about one hundred employees than if it is to be used for a world-wide organization with thousands of employees that are geographically distributed.

Finally, to complicate matters even more, the reality of cloud computing is that failures will occur.  One cannot assume that a workload will work flawlessly once deployed.  Rather, it’s important to accept the reality that various types of failures will occur during the lifetime of the workload.  These failures range from hardware and VM failures, to IaaS management failures, to hypervisor and operating system failures, and to software stack failures.  The sad truth is that all of these mostly software systems contain bugs that will lurk at the inopportune time and cause a visible failure to your customer.  Therefore, a workload must be tunable, restartable, updatable, and malleable.
For these various reasons, the IBM PureApplication System platform provides two means to represent workloads as reusable patterns: virtual systems and virtual applications.
Why two types of patterns?
The two supported types of workload pattern definitions, virtual system pattern and virtual application pattern, both serve the same purpose: to help define an open representation of workload for the cloud.  However, they have also different strengths and capabilities.
Virtual systems allow the definition of workloads as components integrated into cloud images.  These patterns essentially help to easily create cloud images based on the software stack that it contains.  This approach is essentially analogous to providing an editor to typical cloud images.
The second approach for the definition of workloads is with virtual application patterns.  The simplest way to explain what a virtual application pattern is that it is an abstract representation of a workload.  The abstraction is over cloud components and the links between these workloads.  There is less focus on the entire stack or supporting resources, for example, virtual machines (VMs).  Instead, the workload is described by laying out its components, linking them together, and specifying optional parameters.
There are advantages and shortcomings to each approach; however, together they provide a level of flexibility that we hope covers all the bases when defining cloud workloads.
Virtual system patterns
As previously mentioned, virtual system patterns are a visual representation of an image’s content in a template format.  IBM PureSystems provides a graphical representation of the pattern that can then be visually modified.  Using the tooling, a PureSystems administrator can make changes to existing patterns to add new components and scripts for customizations.

The end result is a virtual system pattern that represents one image or a collection of images and be instantiated in a repeatable manner into an IBM PureSystems installation.  This process gives all the advantages of typical IaaS image deployment while providing metadata for the image’s content.  The resulting patterns can be shared and updated as needed.

Although virtual system patterns provide a means to quickly get a workload deployed into the cloud, it also has various drawbacks.  Principally, as we discussed before, approaches that use images for workloads assume that the topologies of these workloads are well defined and do not change much.  However, this assumption does not fit well with modern enterprises that want to be nimble and evolve with current market needs.
Virtual system patterns are thus excellent for repeatable deployment of workloads that are already well defined and do not need to change much in time.
Virtual application patterns
Contrary to virtual systems patterns, the approach of virtual application is to define workloads using higher-level concepts, such as cloud components and cloud services, and also the links between them.  There is no explicit notion of the required cloud resources required to realize the pattern.  Instead the system makes these decision on your behalf, allowing various optimizations that otherwise would be hard to achieve manually.

Although with virtual application patterns, a deployment realization is done by the system, a pattern user and developer can provide hints to the system.  For instance, a workload that contains components that can auto-scale can expose quality of service (QoS) properties that users can use to scale the component.  For instance, the WebSphere Application Server component provides auto-scaling properties to specify that the server should horizontally scale (be duplicated) when the CPU reaches some threshold or the number HTTP requests exceeds some value.

The components from a virtual application pattern are defined through an open plug-in mechanism.  This approach allows anyone to create components or services representing their software services and stack.  Because of the dynamic nature of the virtual application pattern approach, every deployment of a pattern using virtual application components can dynamically fetch the software and install and configure at deployment time.  In this manner, the patterns can evolve and be malleable.

Furthermore, because the system is managing the various components for the workloads, it can also monitor each deployment and recover from various failures.  Indeed, a deployment on an IBM Pure Application Systems that uses virtual application patterns automatically inherits monitoring features such as log viewers, and also VM and service-level monitoring.  The patterns can optionally provide various actions such as a “restart” service action.  In addition to all of this, when a component is scaling in a cluster, the components of the cluster can self-monitor and recover automatically from failures by creating new instances in the cluster when a failure is detected.
One shortcoming of this approach is that deployments can be a bit slower than with a virtual system approach; however, typically in practice, the difference is less than a few minutes of added time in the worst case.  The virtual application pattern approach provides the agility that enterprises require while also enabling the system to provide optimization that would be hard to do manually.
What next?
With the announcement of the IBM PureSystems family of platform-as-a-service and private clouds, IBM also announced hundreds of ISV partners who are committed to provide and support cloud components for the IBM PureSystems.  Most of these ISV cloud components are available today in the IBM PureSystems marketplace.  Using these ISV plug-ins, users of IBM Pure Application Systems can create complex patterns that not only include IBM middleware cloud components, but a wide variety of third-party software and open source software.
In a subsequent post, I will describe how you can create your own plug-ins to extend the IBM Pure Application Systems platform. I will provide instructions of how to create such a plug-in for a popular open source software component.  This plug-in can then be used as one more example to create your own plug-in.

Wednesday, June 6, 2012

White paper: Use the Apache Libcloud Python API to manage resources on IBM SmartCloud Enterprise

Abstract This article explains the basics of using the Apache Libcloud Python API with IBM SmartCloud Enterprise. The value of the Libcloud API is that it can be used across many different cloud providers. The article will demonstrate how to create and retrieve information about cloud resources and discusses managing virtual resources across clouds. This article is suitable for readers with basic knowledge of Python and a basic knowledge of the IBM SmartCloud Enterprise and who wish to combine the two.

Full article on IBM Techdocs White Paper site