• Software-as-a-Service (SaaS)
• Platform-as-a-Service (PaaS)
• Infrastructure-as-a-Service (IaaS)

While SaaS has been around for some time (Salesforce.com started in 1999!), we are seeing an increase in adoption of IaaS and some heavy development in the PaaS world.

Now that 2011 is coming to an end, this is also the time for lists. So here are my 3 top 3’s of cloud computing.

Software-as-a-Service

3. GitHub

Distributed version control systems are gaining track rapidly, with Git leading the way. GitHub provides Git as a service, and offers unlimited repositories for open source software. It provides an excellent interface that stimulates social coding. This means it is a great incentive for open source development.

2. Dropbox

Well, who doesn’t use Dropbox? Dropbox offers a filesharing service that is easy to use. There are two reasons for the success of Dropbox; it is easy to start with 2GB of free space, and it provides clients for almost all platforms.

1. Google Apps

Google is trying hard to get us all to work in the cloud, and their Google Apps service is their way to do so. They even have a free service for private use. Google Apps provides a full application suite including e-mail, calendar and docs. GMail is massively adopted and has been a game-changer since introduction in 2004.

Infrastructure-as-a-Service

3. OpenStack

When talking about IaaS, people immediately think about the Amazon AWS platform. But what if you don’t like their terms-of-service, or simply want to create something similar in your own data center? Enter OpenStack. If there is one DIY IaaS framework that has momentum, it is OpenStack. It is backed by no less than 144 companies, and best of all, it’s open source.

2. jclouds

If all the IaaS frameworks and providers and all their different APIs are giving you an headache, jclouds is the framework for you. The jclouds API provides an abstraction of the different cloud-specific implementations. Currently over 30 providers are supported, including all the usual suspects (Amazon AWS, OpenStack, Azure, etc).

1. Amazon AWS

The undisputed number 1 of IaaS is of course Amazon AWS. Ever since the introduction in 2002, Amazon AWS is the reference implementation for IaaS. It sets the standard and the rest of the IaaS providers are merely trying to catch up. Amazon also doesn’t sit still, it constantly adds new services to its platform (and is slowly growing into a PaaS). It is available around the globe, with data centers in almost every continent.

Platform-as-a-Service

3. Heroku

Heroku is a fully hosted PaaS platform. It supports lots of languages, and it completely hides the infrastructure (servers, instances, etc) from your applications. It has a partnership with Facebook, creating the Heroku Facebook App Package, which enables quick development of Facebook apps. I think it is one of the best examples of a public PaaS.

2. CloudFoundry

CloudFoundry is being developed by VMware. After acquiring SpringSource back in 2009, this is the next logical step for them. CloudFoundy is positioned as the Open PaaS. While most PaaS solutions limit the choice of frameworks and infrastructure services, CloudFoundry tries to be open and extensible. And best of all, you can use the micro edition for development, the private (open source) edition in your own data center and the hosted edition as a public PaaS (or even a hybrid setup).

1. ???

We are seeing lots of development in this area with all different flavors of PaaS platforms and services. We have even built custom PaaS platforms for our customers based on the traditional application servers (JBoss, WebLogic, etc). But there is still lots of work to be done, before full stack solutions will be readily available. So, I think there is no number 1… yet.

And beyond…

2012 looks like it will be a good year for the cloud. I am very curious to see what the PaaS providers are going to bring to the table. Projects like OpenShift, CloudFoundry and Stratos are looking very promising, and I can’t wait to dive into them.

What are your top cloud services? Or which ones do you think that will become the next best thing? Feel free to add them to the comments below.

In this blog post we will give a description of an open source Java Application Platform as a Service that we created for our customer, using VMware, Redhat Enterprise Linux, Apache WebServer, JBoss Enterprise Application Platform, JBoss Operations Network, Puppet, Deployit,  F5 Load Balancer and  a Layer7 SecureSpan gateway.

Data Center Quality Platform

The customer wanted a data center quality Java Application Platform with the following features:

• Standard configuration
• Standardized provisioning
• Standardized deployment
• Centralized monitoring
• Centralized access control
• Virtual environment
• Proving technology

Current situation

As the current Java application platform was based on HP-UX on Itanium, the customer was facing high cost for hardware, software licenses and fading support from software vendors.  As all applications ran on a HP Superdome, it was very difficult to add resources to individual applications. In addition, development teams spend too much time taking their software through the development, test and acceptance environments, resulting in slow delivery of software into production. Finally, it was difficult to provide 24×7 availability because all applications are running on a single machine.

Java Application Platform

The following figure illustrate the solution architecture of the java application platform.

In the following paragraphs we will describe the purpose of the most important components.

Dual Data Center – HP

Not shown in the figure, is the hardware setup of the platform. It consists of HP blades setup in two data centers on two different locations. This provides the basic infrastructure for 24×7 availability and fault tolerance.

VMware ESX

VMware ESX is deployed on top of the hardware in the dual data center. This provides us with the ability to create virtual machines and provide high availability in case of single server of single site failures. It also allows us the quickly scale up virtual machines and increase the resources assigned to individual virtual machines.

For all machines in the platform we use a single VMware template image. This image is installed with RedHat Enterprise Linux and a puppet client.

Puppet

Puppet fully automates system management. It is used for the installation of software packages, conformity tests and day to day system administration tasks.  For every type of node, we have a puppet plan. When the machines boots, the puppet agent provisions the machine with all the necessary software and configuration according to the plan for that machine.

The use of Puppet completely automates and standardizes the configuration, ensures 100% reproducibility of the configuration and is fast. Provisioning of a new machine from the template to full operational mode is done in a matter of minutes.

JBoss EAP

JBoss Enterprise Application Platform is the Enterprise Java applications server for all java applications.  The installation and configuration is done by Puppet and uses the official RedHat RPMs.  Puppet configures JBoss to ensure that :

• JBoss management applications authenticate users against Active Directory, providing a single point of authorization for operations.
• A JBoss Oracle database schema is automatically provisioned for that specific instance of JBoss, providing persistence for the JBoss server system state.
• All Business Applications can authenticate users using SAML against the Layer7 Identity provider, providing a single point of authentication and authorization for their customers.
• The JBoss instance is added to the pool in the F5 Load balancer
• The application server is added to the Deployit infrastructure inventory, providing the tenants of the platform with the ability to deploy applications to the server.

JBoss application servers are always deployed in multiples of two, where each server of a pair is assigned to a physically different data center location by VMware.

The use of puppet provides us with a fast and reproducible way of provisioning JBoss application servers, allowing for a fast and reliable scale out mechanism for the applications.

JBoss Operations Network

JBoss Operations Network (JON) is used for monitoring all the resources in the platform.   By default, Puppet installs a JON agent on every machine. This agent scans the inventory of the machine and reports it to the JON server.

JON has a very good support for high availability and fail over. By simply adding a JON server machine, agents will automatically distribute themselves across the servers and failover if necessary. Each JON server also runs a JON agent, making sure that unavailability of a JON server is also covered.

In JON we created a number of alert templates for different resource types (os, apache, jboss, jon, puppet, etc.)  that will monitor and report critical conditions on the system.  All error messages from the JBoss servers logs are reported as incidents.

All alerts and clearing conditions from JBoss Operations network are reported via SNMP to TNG Unicenter.

Through the  use of JBoss Operations Network all machines, servers and resources in the platform are automatically added to the centralize monitoring system.

Deployit

Deployit is used for the automated deployment of applications onto the platform. It automatically deploys all the application components in a stack to the appropriate containers.  Deployit :

• deploys static content and proxy configuration to the apache webservers,
• deploys enterprise java application components to all individual JBoss servers in the farm,
• executes SQL scripts to the database,
• configures the F5 loadbalancers to add or remove servers or applications to the pool,
• applies environment specific changes to the application configuration.

The deployment plan for a specific application is prepared in close cooperation between the application developer and platform management staff. When the deployment plan is finished, developers can deploy new versions of the application themselves, directly from a build tool or manually.This ensures solving any installation or configuration problem isn’t postponed until the application is installed for production use, but rather is solved at the early stage of any development.

The same deployment plan is used for all environments. Authorization can be configured per enviroment and per application. LDAP is used to authorize software developers to deploy and configure an application for development and testing purposes, while integration specialist can deploy the application in production.

The use of Deployit provides the platform with a fully automated and standardized deployment mechanism, improving the speed of deployment of applications through the development, test and acceptance environments while reducing the number of staff involved and lowering the number of configuration errors.

F5 Load Balancer

The F5 Load balancers is used to support scalability and fail over for the JBoss Application Server farm.

The pools are configured to use a sticky session protocol based upon the JSESSIONID session cookie. If the cookie is not present, round-robin load balancing of the HTTP requests is performed.

Puppet adds the JBoss servers to the  pool in the F5 Load Balancer.

When a server is scheduled for a restart, the server is taken out of the pool. This ensures that this server does not get any new request, but will still be servicing existing sessions. When the session count in JBoss drops to zero, the server is restarted and restored to the pool.

The use of the F5 Load Balancer provides us with the ability to increase and decrease the number of servers in the farm, provide load balancing, fail over and graceful decommissioning of servers in the farm.

Layer7 XML Gateway / Identity Provider

The Layer7 SecureSpan Gateway is used as centralized security policy enforcement point and SAML identity provider.

Layer7 supports multiple authentication methods, Kerberos, digital certificates, username+password and is able to use multiple identity stores.

Puppet configures all JBoss application servers with SAML support and configures Layer7 as the identity provider: JBoss receives authentication (identity) and authorization (roles) information as a SAML-token. The information contained in the token is translated to a standard JEE-principal user (using a tiny layer of custom code), so all JEE applications can access the authentication and authorization information in a standard way.  Whether the JEE application is a web application or provides webservices, from a security there’s no distinction. All application designers have to do is declare the application security roles conform the JEE standard.

The use of Layer7 standardizes the authentication and authorization for all business applications and centralizes access control.

Conclusion

The customer wanted a modern data center quality Java Application Platform to ensure that java applications could be deployed with lower cost and with high availability and easy scalability.

VMware, the dual data center, Layer7, the F5 Load balancer and JBoss provide the infrastructure for a  high availability and scalability for any java application. The combination of VMware, Puppet and Deployit are the fabric to enable continuous delivery of enterprise java applications.

Through virtualization and automated provisioning and deployment it has become possible to add a completely new, correctly configured machine to a cluster in a matter of minutes, completely secure and under full monitoring.

What early adopters have learnt is that, at the enterprise scale, automating release execution quickly introduces a new bottleneck in today’s dynamic IT environments: continuous management of the deployment plan definition. A new generation of application release automation (ARA) tooling avoids this pitfall by leveraging intelligence to automate deployment planning as well as execution.

The State of ARA

Given how strongly our IT industry is dedicated to the automation of processes, it is nothing short of amazing how much of the deployment of our own solutions depends on manual actions coordinated more or less effectively between large groups of people.
Indeed, a recent analyst report noted that the majority of large enterprises were still relying on manual application release processes or on in-house scripting understood by only a small number of specialists, operated as a black box that – hopefully – will do its job and will most likely necessitate a painful troubleshooting session if it doesn’t.

With key IT trends such as Agile, Cloud and DevOps dramatically ramping up the frequency of application releases in order to increase responsiveness to business needs and provide more and better services to customers, it’s clear that this situation cannot continue.

Thinking of today’s common release processes, it is also hardly surprising that the initial drive has been to automate the actual rollout of the application itself: copying the files to the target machines, restarting the servers, running the SQL against the DB etc. Using a defined workflow to organise these activities makes lots of sense: fewer failures, no more missing steps or steps executed in the wrong order, no more typos, better visualization etc.

Lessons from the First Generation

Ironically, using one of these first generation ARA tools at an enterprise scale quickly made it obvious to early adopters how much effort is required to maintain the substantial number of workflow definitions that quickly accumulate to support full deployments, partial upgrades, rollbacks, environment scale-outs etc. across an enterprise application portfolio.

Of course, this is not a new challenge: ask anyone who has had to update 100 build job definitions in a continuous integration tool to change a compilation parameter, or 100 test plans in an automated testing setup to accommodate a different target browser, just how time-consuming and error-prone this type of maintenance is.

It’s not as though these modifications are unique per process. They tend to be systematic changes that reflect changes to the overall deployment strategy and/or context. The issue is that these first-generation tools, where all the deployment intelligence is stored in the power user’s brain, simply do not have enough internal knowledge of the structure of deployment to assist effectively.

The Next Level of Application Release Automation

A new generation of Application Release Automation includes this intelligence. These advanced tools² no longer require hand-holding by your power users every step of the way, and encode knowledge of deployment best practices and strategies to automate the planning and execution of deployments.³

No pre-provided strategy can be a 100% fit in an enterprise environment, so the strategies must be configurable, of course. Once your power users have fine-tuned them, however, all the individual deployment plans – initial installations, full and partial upgrades, downgrades, undeployments, scale-outs etc…easily hundreds across an application portfolio – are automatically tailored to your scenario.

This becomes even more efficient the fewer deployment strategies are in play, so these tools also motivate and reward increased standardisation of deployment procedures, in itself a valuable business goal. In fact, with a suitable interface and integrations into the development pipeline you essentially have an enterprise Platform as a Service, potentially on a private or hybrid cloud.

Conclusions

With adoption of Application Release Automation rapidly on the increase, a new generation of solutions are appearing that automate deployment planning as well as execution.
Based on the challenges experienced in scaling the first generation of ARA tools to enterprise levels, these next generation solutions are designed to eliminate “continuous expert hand-holding”, promote standardisation and allow organisations to create a “software factory” that continuously delivers business value.

Workload Deployer V3 and Deployit from XebiaLabs both “deploy” things, but they deploy different things. Deployit deploys application artifacts and resources, such as EAR files and data sources, to middleware systems like IBM WebSphere Application Server (but also HTML to web servers, IBM WebSphere MQ configurations to queue managers, and so on). IBM Workload Deployer, on the other hand, deploys patterns (or topologies) of virtual images to hypervisors — but not just any kind of virtual images. IBM Workload Deployer is especially geared toward deploying middleware topologies.

IBM Workload Deployer V3 is an updated and enhanced version of the WebSphere CloudBurst Appliance, renamed to reflect the expanded scope of workloads it can deploy, which are no longer limited to only WebSphere workloads. The content for this article (including screen captures) was created using a WebSphere CloudBurst Appliance, but everything noted here is equally applicable to IBM Workload Deployer V3. However, for the sake of consistency with the images presented, “WebSphere CloudBurst” is used throughout this article to refer to both products.

In other words, IBM Workload Deployer deploys the middleware systems and Deployit deploys applications to those middleware systems — complementary functionalities that form a perfect fit.

At XebiaLabs, we have been working on two exciting new integrations for Deployit. We created a Deployit plugin that enables you to deploy EAR files directly to virtual systems created by IBM Workload Deployer V3 or its predecessor, IBM WebSphere CloudBurst Appliance V2. We also created a WebSphere CloudBurst script package to deploy application artifacts and resources on newly created virtual systems.

This article explores two integrations between WebSphere CloudBurst and Deployit as a way of showing how you can leverage the WebSphere CloudBurst command line interface and script packages to integrate cloud deployment with your application deployment automation solution.

(more…)

A key part of the test harness for the original versions of Tweetstore that run on the Google App Engine is the neat GoogleDevServer class. Basically, it’s a clever wrapper around the KickStart class used under the covers by the GAE SDK tools that allows you to specify the SDK location, address, port and WAR file (or expanded WAR directory) to run.
Better still, it can programmatically be shut down cleanly, making it ideal for integration test runs¹.

For CloudBees, I was thus looking to put together a similar RunAtCloudServer. It proved more challenging than expected…²

Just spawn a container already?

Both the maven-gae-plugin and the bees-maven-plugin allow you to spawn a local GAE or RUN@cloud server simulation, of course. These can easily be integrated into your Maven build, for instance as part of your pre-integration-test setup. So why not just stick with that?

Well, for some scenarios that can be perfectly sufficient. For Tweetstore, having an inline server that cleanly shuts down allows us to:

• run integration tests alongside unit tests from an IDE without having to remember to start an external process
• debug the application running in the server on demand, without having to mess with remote debuggers or figure out a way to sometimes get the Maven plugins to start in debug mode
• work some conditional magic with the server arguments

If your project’s requirements are similar, an inline server might be a good option. And luckily, the code’s already been written!

Hangin’ around

The RUN@cloud SDK’s version of KickStart is the StaxSdkAppServer³. Figuring out the correct arguments for the launchServer factory method wasn’t too hard, so I hooked up a StaxSdkAppServer and kicked off the test.

Started like a charm! Then the test finished and…just hung on in there. Over in the trusty debugger⁴, I could see not one, but two zombie threads still running. The design was based on a “shutdown-by-JVM-kill” scenario, I guess.

Assassins and other Ninja Tricks

The first of the two threads is a Timer that something called the WebAppEngine uses to do…well, who knows? In any case, we can get a reference to it with a decent handful of reflection and cancel it. Easy enough.

Our remaining zombie is a bit more elusive, unfortunately. It’s a thread spawned by an instance of the RequestMonitorValve class in a manner analogous to

new Thread(idleTimer, "requestMonitor").start();

So no reference, and no way of interrupting it. However, the idleTimer Runnable is referenced and repeatedly calls a handler as part of its execution. How about injecting a handler that throws an exception to end the thread?

Nice idea, but it turns out the thread really doesn’t want to die that way:

while(true) {
    try {
        // do stuff
        if(callbackClient != null)
            callbackClient.updateStatus(state);
    } catch(Exception e) {
        // increment some error counters
    }

    try {
        Thread.sleep(statusIntervalSecs * 1000);
    } catch(InterruptedException interruptedexception) { }
}

So I’ve had to go one better and throw a ThreadDeath error instead. Urgh.

Now you see it, now you don’t?

The remaining gotcha involves injecting the catchily-named KillerCallback into the idleTimer. Even using reflection, this is a elementary concurrency failure because without a synchronized-with relationship⁵ between injecting the new handler and invocation of the handler by idleTimer, there is no guarantee the new handler will actually be visible to the thread we’re trying to stop.

Such a relationship could be established if the “assassin” was injected before the thread that runs idleTimer starts. Unless we want to override library classes, that would mean setting it after creating but before starting the StaxSdkAppServer⁶…which unfortunately doesn’t work either because the instances referencing idleTimer are only created when the server starts.

So?

So, for now the potential concurrency issue is documented and we hope for the best No problems observered so far…

In the previous post, we looked at how Reusable commands, Models and Conventions++ helped turn build from a “black box” process into the “just works” experience we know today.

We then shifted back to deployment and identified Develop a common model, (Re)discover vanilla and Support a “clean build” as three key steps required to achieve a similar transition.

Develop a common model

Before we can advance to the ‘model’ stage, we first…well…need a model. Thankfully, a very simple one can suffice: Packages, Environments and Deployments.

There’s nothing particularly magical to this, and indeed the concepts are commonly found in most organisations. But giving these things explicit labels helps not just formalize the ideas and gives developers and vendors something to support. It also creates a shared vocabulary and language around deployment, which is the first step to shared understanding and reusable functionality.

Indeed, the concepts are so basic that there does not appear to be much to say about them.

Packages capture the components of the versioned item to be released, both artifacts represented by actual files as well as configuration, resource settings and metadata.

In accordance with release management best practice, packages should be stored in a DSL and should be independent of the target environment, so that you have one “gold standard” package running in Development, Test, QA and Production.

Packages also mean that we can version everything, not just the application binaries but also the related configuration and environment settings.

Development and Test just mentioned are examples of Environments, simply collections of infrastructure – physical, virtual, long-running, on-demand, whatever – that applications run in as they progress through the ALM cycle, potentially with approvals or other checkpoints governing the transition from one to the next.

Deployment, then, is perhaps the one concept not immediately widely understood. A Deployment represents not just the activity of getting a Package running in a certain Environment, with a start and stop time, executing user, status and so forth.

Rather, a Deployment also documents the way in which the Package’s components have been deployed and, if applicable, customized. For instance, a Deployment will record that a certain EAR file in the package has been deployed to the following target server(s) or cluster(s), or that the data source password for this specific environment has been customized and set to a new value.

Recording this information is critical because it is very hard to be able to intelligently and correctly modify an application’s state – when upgrading to a new version, for instance, or adding new servers to the target cluster – if you do not know where and with which settings the application is currently running.

(Re)discover vanilla

If we are going to achieve hassle-free, push-button deployments, another thing we will have to reconsider is whether we really need to tweak and customize our infrastructure in every way possible. Indeed, some companies seem to almost have a policy that any setting that might be a default should be regarded with suspicion and, preferably, changed.

Much as custom project layouts made setting up a build unnecessarily tedious and complicated in a convention- and model-based system, stubbornly refusing to go with infrastructure defaults will make it harder to get hassle-free deployments that truly cover all the steps required.

Sticking with defaults not only encourages reusability because the chances are much higher that a solution developed for a different scenario will also work in yours. It also improves maintainability and cuts down on the risk of “ripple” changes, where a custom value in the setting for the servers hosting application X requires further changes to the setup of application Y etc.

Support a “clean build”

When building a large project, we try to cut down on the time taken by recompiling only the source code that has been modifying. When deploying applications, we similarly want to save time when upgrading to a new version, especially when this time represents production downtime.
However, we also know that, eventually, some parts of any incremental build will end up going out of sync, causing strange compilation problems, or features or fixes not appearing when they should.

What do we do in such a case? Do we laboriously try to track down the files that are out of sync and rebuild piece by piece? No, we simply run a clean build to start from scratch, because in 99% of cases it’s much quicker to simply rebuild than try to track down the cause of the problem.

In deployment-land, we seldom have the ability to clean build, and this is one of the main causes for the stressful, time- and resource-consuming troubleshooting hunts that are still far too common. Of course, in order to clean build a system we need full versioning of the environment, its configuration and the applications deployed to it. Virtual appliances and virtualization solutions with snapshot capabilities will have a major role to play here.

We also need a known state for durable resources such as databases, which remains challenging but is being addressed by a growing number of products out there.

Push button deployments

Taking stock, it’s clear that there is still some way to go. We’re slowly developing a common model, but both “(Re)discovering vanilla” and “Supporting a “clean build” are visions not quite yet on the horizon of most large companies.
In fact, it’s not so much technological advances that are required – many startups are pretty close to push-button deployments and continuous delivery. Indeed, the “poster children” of this movement already have setups where every commit can pass through an entire regression, integration and performance testing suite and potentially go straight to production.

No, the important hurdles to be taken are procedural and mental, changing rusty ways of working and entrenched mindsets. For those that can make it, though, the benefits in terms of accelerated business value are already proving to be game changers.

In the previous post, we compared deployment to another key process in the application lifecycle – build – and asked which key developments turned it from a magical “black box” into the “just works” process it is today.

We identified Reusable commands, Models and Conventions++ as three key steps, which we’ll look into in more detail in this post. Then, we’ll shift back to deployment and ask which improvements will be essential to getting to “just works” here.

Reusable commands

The first step, epitomised by Apache Ant, was the recognition that most of the low-level tasks or actions are the same whatever application is being built, whether calling a compiler, copying files or replacing placeholders.
Rather than copy-pasting the same OS commands into every new build script, we encapsulated these commands as libraries of reusable components that only needed to be written once.
Further, we discovered that certain patterns of step sequences would appear in many different builds. These ‘chunks’, such as constructing a classpath, or copying and processing static resources, evidently represented some higher-level build activity with a distinct function.

Models

Whilst the realisation that all the actions carried out in a build are basically a sequence of common chunks was an important start, the next big advance was brought about by recognising that we weren’t just seeing repeated patterns of actions, but that the types of data these actions were working on were also shared.
This gave rise to the notion of a true domain model for application builds, with source sets, resource sets, modules, dependencies and so forth that were originally introduced by Maven and have featured and been reused in all build systems since.

Combining the sequence of common chunks with the new domain model that structured the data being processed gave rise to notion of distinct phases, in which parts of the build model are generated, prepared and made available to subsequent commands.
In addition, Maven also supported the idea that the domain model, and thus the build phases, would have to be able to vary slightly to accommodate different types of Java artifacts that need to be delivered, such as JARs and EARs. This has subsequently been further developed to support builds of totally different technologies such as Adobe Flex applications.

Conventions++

An additional benefit of domain models for build was the ability to make use of default values in a structured way, for instance for the names of built artifacts or the location of resource files.
However, the flip side of this convenience, certainly in combination with XML as a descriptor language for builds, meant that deviating from these standards could be quite a challenge. Certainly if the aim was to extend the domain model in some way, or to support a language or technology whose build flow was reasonably different from Java’s, such as building documentation bundles or virtual machine images, for instance.
This has led to a generation of build tools, such as Gradle, that aim to restore the developer to a position of full control in which arbitrary actions can easily be defined and organised into phases, tasks and entire builds. Of course, given how used we have become to the convenience of “it just works” in simple cases, these tools still support the full domain models and conventions of common technologies such as Java.

Who’d have thought?

Reviewing this progression from today’s perspective, a couple of facts stand out that, certainly in comparison to other evolutions in IT, are quite surprising.
Firstly, whilst it’s now hard to imagine specifying a dependency using anything other than the groupId:artifactId:version pattern, none of the models or conventions that developed were formalised in industry standards. Instead, they were either based on observations of common patterns, or simply clever or even somewhat arbitrary choices (src/main/java, for instance).
Secondly, we have seen how ease-of-use based on conventions, coupled with a moderate nuisance factor of tweaking those conventions, can dramatically change user behaviour. Initially, for instance, many of those new to Maven spent quite a significant amount of time, and produced a fair amount of XML, to change the standard settings to match their own environment, naming conventions, file paths etc.
Pretty soon, though, and especially as the ratio of green field vs. legacy projects increased, it simply became easier to stick with Maven’s standard values and be done with it. Today, these conventions have become so standardised that they are supported not just by Maven, but essentially all other build systems out there, too.
It’s not just users’ preferences that were “charmed” into adopting standard conventions, though. In many cases, company standards previously seen as cast-iron were progressively discarded or modified if they could easily not be accommodated by the emerging de facto standards. Ease-of-use was able to triumph against abstract rules.

And deployment?

So much for the build process. What about deployment, today’s critical hurdle for automation in the business value delivery chain? As previously mentioned, the current industry average is somewhere between “black box” and “step sequence”. In terms of the descriptions of the build process evolution, the most advanced deployment automation systems are somewhere just beyond “reusable commands”, in other words.
Which naturally begs the question: how do we get to a push-button state? What do we need to do to be able to reach the maturity level of build today?
Looking at what we encounter in the industry today, three critical aspects will be:

1. Develop a common model
2. (Re)discover vanilla
3. Support a “clean build”

We’ll cover the details of these three concepts in the next blog…

But a small piece of background first. Deployit and WebSphere CloudBurst Appliance both “deploy” things. But they deploy different things. Deployit deploys application artifacts and resources such as EAR files and data sources to middleware systems like WebSphere Application Server (but also HTML to web servers, MQ configuration to queue managers, etc.). The WebSphere CloudBurst Appliance on the other hand, deploys patterns (topologies) of virtual images to hypervisors. But not just any kind of virtual images. It is especially geared towards deploying middleware topologies. In other words, the software Deployit wants to deploy to! This means that the functionalities of WCA and Deployit are a perfect fit; have WCA deploy the middleware systems and have Deployit deploy applications to those middleware systems.
(more…)

In this first post, we’ll focus on some of the changes and trends across the industry that have brought such increased business attention to the area of release, deployment and management of applications.

As companies focus on tuning their IT environment on rapid delivery of business value, more and more projects and initiatives within organisations are looking at the entire value chain of software production.

Whether under the banner of Devops, via integrated project teams or as part of the introduction of new development methodologies like Agile¹ or infrastructure technologies such as cloud², there is a growing awareness of the need for automated, reliable and flexible deployment procedures.

Whilst it’s clear that the generation of customer features takes place within the development, testing and QA teams, the business value inherent in these features is only unlocked once the application is actually running in a target environment, accessible to users. And it’s not just the final release to production that requires a deployment: every UAT, performance or integration test generally needs an application running in a “real” environment, not a developer’s local machine. One test, one deployment.

Given the well-known costs and adverse effects of faulty software in production, the ability to raise quality, usability and performance through a greatly increased number of testing cycles represents significant added value in itself.

One of the consequences of this increased attention is that the importance of build, release and deployment professionals as deliverers of business value is being recognized more and more. The heightened focus also means, though, that many companies are realizing that the effectivity, simplicity and overhead of their release and deployment processes lag far behind that of build and continuous integration.

As my role at XebiaLabs involves studying, automating and improving deployment processes across the industry and helping developing our vision for deployment into the future, the question of why deployment is so different from build in today’s enterprises was bound to raise its head sooner or later. The resulting discussions and reflections formed the basis for this series.

What’s in a word?

One of the challenges surrounding the discussion of deployment in relation to build, release, provisioning and other tasks in the ALM space is the – decreasing, thankfully – lack of a clear shared definition for deployment. Without wanting to promulgate this as the correct definition, for the context of this discussion I’d like to treat deployment as the process that

takes the components that make up a release (typically a specific version of an application) and getting them correctly set up in an infrastructure environment so that the release is accessible to (end) users

This would differentiate it from build and release in that it assumes that the application components have already been created, and from provisioning and other infrastructure tasks in that the target infrastructure is already assumed to be present.

On-demand virtual or cloud environments, or virtual appliances, put a bit of a different spin on the issue, and will be a topic for a subsequent blog.

Blast from the past

Taking the above as our working definition, the sobering picture is that, with very few exceptions, deployment now is pretty much at the stage where build was in the days of the make guru: a black box put together and operated by a specialist that somehow works. With luck, this precious resource is still employed and around to fix or extend things when required, but if he or she is out to lunch you’re simply out of luck.

To be fair, there are quite a few places where there is least tooling or automation in place that tries to map out the sequence of steps or actions required to carry out a deployment, to at least bring some visibility and traceability and shine some light on the ‘magic’. But this is still a long way away from the push-button experience that build is nowadays. Laboriously visualizing and walking through a process step-by-step is still a sign ultimately of a lack of trust; a truly mature process doesn’t display its internals any more, it “just works”. Much like build today.

The road to “just works”

Of course, build didn’t start out as a “just works” process either, so what actually were the steps that made this transition possible? Looking at the evolution of Java build tooling over the past decade or so, there have been at least three main developments:

1. Reusable commands
2. Models
3. Conventions++

We’ll dive into the details of these three concepts in the next blog…

In my previous post, “future of deployment, part 2″, i talked about the new ear, which is an image, with an OS and your application.
Now before diving into part 3, which gets you going in creating your own virtual appliance aka “the image”, there is one really big thing i forgot to mention; Some of the benefits of delivering a virtual appliance and getting it from your own development to the production environment! and i’ll list the benefits for administrators/ops and developers.

Ask and thou shall receive, thy environment, immediately

So you start developing your new application “CuteAnimalPark” (yes, we at Xebialabs like animals ), and you talk to the admins and ask them for 3 environments, Test, Acceptance and Production. They of-course respond, “that is fine, we’ll have it up-and-running when you can deliver your application”, aka “virtual appliance” and give you a link where you can download the Virtual Image where you can install your application on.

So what happened here? Well i wanted 3 environments and got none!? But i did get some image which is in fact the image that is going to run in production. The image is hardened, has got security enabled, all user accounts and file-system rights are in order and is tuned for real world serving the interwebs.

The job of the operators is to make sure applications get to production and run reliably and fast. They are the ones getting called out of bed if my application dies at 3 AM. So therefore they want to give you an image which is the SAME that is going to run in production.
That means IMHO, one of the day-time jobs of these guys and gals is to prepare the images (OS + middleware) for applications and give them out to development teams so they can in turn create virtual appliances.

Action #1 Get a production ready image to install the application on.
Benefit for Devs #1 You develop and deploy to “really-really-close-to” production system
Benefit for Ops #1 You get an application that is known to run on a production system

Running the production image aka virtual appliance in other environments then production

So when you have installed the application on the image you have to make sure it can be used in other environments then development. The application might need to connect to a database, now the database will mostly differ from one environment to the other, so as soon as the application is placed into the environment, you have to get in there and change the URL and most probably, username and password of the datasource the application is using so it will connect to the e.g. database in test and not the developers own MySQL database on his local machine.

Changing the way an application uses a database should be just as easy by changing the properties on a datasource, and most of the JEE containers make this fairly easy, but i’ve personally seen lots of other properties which are very environment specific which were in property files, in JAR files that needed changing when moving the application from one environment to the other. Not so easy then ;(

Make sure that when you design and develop your application that environment specific properties are easily accessible and can be changed by operators.

Action #2 for Devs Design and Build your application so it can exist in multiple environments
Action #2 for Ops This also applies to Operators, who have to be aware that the image they deliver will need to run in different environments (e.g. use hostnames when installing middleware and don’t use IP-addresses)
Benefit for Devs #2 Your application is portable, it can be moved almost anywhere and administrators can do it without your help
Benefit for Ops #2 Easily move images to wherever you want, move stuff from your private cloud to the public, will be a lot easier! per application.

After this step, it means that your virtual appliance can now be deployed to the various environments, which of-course are running some sort of virtualization hypervisor from some known vendor. Every time the appliance is deployed, before it is started the operators configure it. Making sure the Datasource connects to the right database, the Queues connect to the right Message broker, and so on.
After configuring the application you can start up the whole bunch and bask in glory. (make sure the ops know in how to properly start the application and how they can tell it works)

The flow from development to production is that simple, but wait! there’s more, troubleshooting?

Again here is the flow from the 2 paragraphs above:
- Get a production like image
- Install your application on it
- Allow operators to move your virtual appliance from test to production (without needing your assistance)

But of-course stuff can go wrong in production, your application might break, has a race condition under extreme load, and you as a developer want to get your hands on production and find out what is wrong. But alas, the operators will not allow you to access production, most likely they’ll send you the logs, thread and heap dumps but that is it. So wouldn’t it be nice if you could have the access to production when it is not used in production?

With virtualization you can get a snapshot of the images running in production, transfer it to your local development environment and really get into finding the problem. There is always the trouble of generating real user requests but this is a very big step forward to get your hands where the real problem occurred.

Action #3 Whenever an issue pops up with an application, create a snapshot of the image and let the developer have access to it
Benefit for Devs #3 You can get to the source of the problem more easily
Benefit for Ops #3 You still don’t need to give access to developers on production if you don’t want to, just let them have a copy of the current state of production

Makes sense?

Well there you have it, this is what i think will be the biggest change in the way we deploy an application from development to production in about 3-5 years. As i mentioned before, post 3 will be about doing it yourself with the current tools available, from Xebialabs of-course and VMWare.