Here, we explain how Deployit‘s Infrastructure and new Release Overview features help you achieve this goal.

Continuous Delivery & the Release Perspective

In today’s highly competitive economic environment, the need to bring new features to market quickly, flexibly and reliably is paramount – a goal that is ultimately the aim of the main IT trends Cloud, Agile and DevOps.

Continuous Delivery – extending Continuous Integration to automatically transition applications down the Dev-Test-Acc-Prod delivery pipeline – is a key component of this strategy. In order to be able to effectively implement this, your ARA solution needs to allow your developers – or, in larger organisations, release or DevOps teams, to quickly and efficiently answer questions such as:

• How far is MyApplication down the road to Production?
• When will MyApplication take the next step down the road?
• What do I still need to do before that next step can be taken²?

Ideally, this dashboard would also allow you to plan MyApplication‘s next step and calculate the estimated go-live data, perhaps even based on an analysis of previews versions of MyApplication.

(Virtual) Environment Management & the Operations Perspective

From an Operations point of view, an individual application is only a small part of the picture. Across your Dev-Test-Acc-Prod landscape, you will need to track all applications vying for these environments, to manage potentially conflicting resource requests, plan environment maintenance activities and the like.

Since these environments are often owned and managed by different teams and certainly have varying service levels, you will also want to limit your view to one or a subset of these environments at a time.

Your Operations or DevOps teams need to know:

• Which application versions are currently deployed to my environment(s), or were deployed at a certain point in time?
• Which components do these applications consist of? On which middleware and infrastructure systems are these components deployed?
• What are the current values of any properties or settings for these components? Which environment-specific customizations have been applied?

Cloud and the on-demand environments it enables will eventually replace the rigid Dev-Test-Acc-Prod distinction³. Nevertheless, the ability to present an environment-centric view will still be required, since virtual environments will still be owned by different groups or teams. Indeed, such a perspective will be even more important if you want to effectively combat “virtual sprawl”.

While the coming generations of “true” cloud architectures will hopefully reduce the shared resource conflicts that greatly complicate much of today’s Dev-Test-Acc-Prod management, databases, legacy systems and external payment providers are not likely to disappear anytime soon.

In fact, Facebook, Twitter and other social elements of your future business services may even increase the number of shared resources you need to manage!

Incorporating ARA Data in the Service Delivery Picture

Whilst your ARA solution should be your “go-to” platform for answers about how your applications and environments relate, it is equally important to consider when this data might be more effectively embedded in a broader service delivery picture.

For example, your ARA platform is not a good candidate for providing a release calendar, since it is not aware of much of the information that is relevant in this context, such as CAB⁴ meeting schedules, business sign-off dates or operational maintenance windows.

It is thus important to ensure that your ARA solution can make its data accessible via APIs such as RSS feeds, iCal calendars and other APIs, to enable effective integrations with the rest of your service delivery tooling.

Conclusion

The right Application Release Automation platform gives your Delivery and Operations teams fast, accurate insight into your application environments and delivery pipeline.

Choosing a solution like Deployit with focused Operations and Delivery overviews as well as open APIs for easy integration into your overall Service Delivery dashboards and reports greatly enhances the accessibility and effectiveness of your application release management.

Daarnaast ervaart 83 procent van de Nederlandse bedrijven die Agile werken hebben geadopteerd, meer werkplezier en 85 procent meer teammotivatie. Dit percentage is aanzienlijk hoger dan vorig jaar, toen gaf driekwart van de respondenten aan meer werkplezier en teammotivatie te ervaren. Dus de mensen die Agile werken varen er wel bij, naar mijn mening een van de belangrijkste redenen voor het succes van Agile. Dit komt ook veelal tot uiting in een lager ziekteverzuim en grotere loyaliteit naar de werkgever toe.

Andere belangrijke effecten zijn een verkorte time-to-market volgens 79 procent van de ondervraagden en toename van de productiviteit (66 procent). Het onderzoek laat ook zien dat kostenverlaging een belangrijker effect van Agile werken is dan vorig jaar (38 procent in 2011 tegen 21 procent in 2010). Niet zo vreemd natuurlijk in deze economisch zware tijden. Maar dit betekent wel dat Agile steeds meer wordt ingezet als onderdeel van een kostenverlagingstraject. Dan is het wel heel belangrijk ervoor te waken, dat met het (meer) sturen op deze doelen de Agile gedachte niet ten onder gaat in de zoektocht naar kostenverlaging. Indien kostenverlaging wordt neergezet als primair doel met Agile als middel, is de kans groot weer te verzanden in ‘ouderwets’ contract en KPI management waarvan we in het verleden nou juist geleerd hebben dat dat niet effectief is.

In de overgang naar het Agile werken is de bedrijfscultuur net als vorig jaar de belangrijkste bottleneck voor veel organisaties (49 procent). En ook dat is geen nieuw inzicht. Maar het is wel iets dat te vaak onderschat wordt, zeker door belangrijke personen die juist onderdeel zijn van die bestaande bedrijfscultuur. De adoptie van Agile vereist veel focus en energie voor langere tijd en de daadwerkelijke borging vindt plaats juist via een inbedding in die bedrijfscultuur.

Door het verbeteren van kennis, kunde en bovenal mindset van Agile zal de uitrol een grotere kans van slagen hebben. De ondervraagde organisaties erkennen dit gegeven zelf overigens ook, want volgens 36 procent is een gebrek aan kennis en kunde de belangrijkste beperkende factor in de (verdere) implementatie en uitrol van Agile. Door juist in deze economisch uitdagende tijden in kennis, kunde en mindset van Agile te investeren zullen organisaties meer rendement kunnen behalen. Het verbeteren van bedrijfsresultaat is nu bijvoorbeeld voor veel financiële instellingen in Nederland een belangrijke reden om Agile te gaan werken: zo kunnen zij namelijk effectief hun te hoge kostenstructuur aanpakken.

Agile belooft dus veel, maar maakt dit ook waar. Zeker gezien de huidige generaties Y en Z die nu opgroeien en de toekomstige werkbevolking van Nederland gaan vormen, is er geen ontkomen meer aan (iets als) Agile. Regelmatig spreek ik diverse (jongere) sollicitanten die Agile, en de manier van werken en omgaan met elkaar die daarbij hoort, heel normaal en gewoon vinden en die gruwelen bij een beschrijving van het traditionele waterval model en bijbehorende werkaanpak. Het aantrekken van jong talent en een werkwijze die niets met Agile of elementen daarvan te maken heeft, staat bijna haaks op elkaar. Blijvende continiuteit en succes op langere termijn voor ieder bedrijf bestaat voor een belangrijk deel uit het aan kunnen blijven trekken en behouden van jong talent. Agile worden of zijn, lijkt hier een belangrijke voorwaarde voor te zijn.

Ik ben benieuwd wanneer u op de succesvolle trein stapt die Agile heet!

The desire to change

Suppose one day, after a particularly bad hangover, you decide to change your life.  You long for a trim body, a balanced spirit, lots of energy and no more headaches. In short, a happy mind in a healthy, good looking body. But how do you get there?

Well, we all know that there are many practices that will help you achieve those goals.  Exercise three times a week, stop smoking, reduce your alcohol intake, and change your eating patterns. Oh, also, get more sleep, less stress, no more pills, and drink herbal tea instead of espressos.

But you hate sports, you love chocolate and smoking is your way to reduce your stress.  So what do you do?

The daily fruit practice

In order to get healthy, you decide that from now on, every day, you will eat a piece of fruit at lunch.   It’s pretty easy to do, even though you don’t like fruit very much.  So now you are satisfied, you do a healthy thing every day, and you expect to reach your goals (more energy! greater happiness! better looks!) anytime soon.

Will you? Of course not! While the daily piece of fruit is a step in the right direction, you will never fully get the benefits you are seeking if you keep up your old habits of smoking, drinking and eating greasy food,. And after a while you might even get disappointed and frustrated, and you will decide to stop eating fruit, claiming that “a healthy lifestyle simply doesn’t work for you”.

Doing just one healthy practice, or even a couple of them, isn’t enough. What you really need is to change your mindset. You want to be a healthy person, not do some healthy stuff. Once the healthy mindset becomes your way of life, the practices will simply be the natural way to go. And they will be a lot easier to start with and maintain for a long time.

The daily Agile practice

The same applies to Agile. We see many companies and teams that ‘ do‘ Agile. They take a couple of practices from an Agile framework such as Scrum, and they figure that that will be enough to give them all the Agile benefits. Some teams even think that if they just do a daily stand-up, they do ‘ Agile’.  While daily stand-ups, just like that daily piece of fruit, are definitely a good practice, your are still far removed from true agility if the rest of your behaviour consists of old, non-Agile practices.

A company or team only gets the most out of being Agile, when its mindset is based on the Agile principles stated in the Agile manifesto, such as continuous learning at all levels in the organization, business and IT working closely together, delivering the highest business value first, welcoming change, and creating an environment where motivated individuals can be creative and self-directing.

So if you are doing ‘some’ Agile, expand your practices. But more importantly, change your mindset. Don’t do Agile, be Agile. And remember:

One practice a day does not keep the old habits away…

Of course installing this application in the production environment should have been as easy as the setup on our own laptop’s right? Wrong! On our laptops it was a easy git clone followed by a npm install and voila we have a running application. So how hard could it be to do this on a server at the client. Let me tell you….

This server is not connected to the internet, so the git clone wouldn’t work in the first place. Not really a problem because it’s a small app and we could just make a tarball and ship it to the server.
Next thing was all our dependencies. We used a few modules which were mentioned as dependencies in our package.json file so npm with the install command would do it’s magic.

The npm install magic consists, among other things, of getting the modules from the npm registry and that fails if you’re not connected to the internet. Searching for a way to do this differently I figured out that npm had a cache directory and thought I could get the stuff from there. This might have worked, but with that solution I would miss the dependencies where these modules depended on. And it would be a messy kind of script that I needed to make.

Browsing the internet didn’t provide me with the right answer but it led me on the path to the npm pack function. This is used to pack your module together with all dependencies. The only thing you need to configure it correctly, is a separate array containing the dependencies to bundle with your app.

So far so good, so I went on and added a bundleDependencies section in my package.json and ran npm pack.
The result was a nice .tgz file containing all the files needed for the application together with all the modules it depended on. At least the main modules it depended on. My fellow programmers in crime from which I got these modules hadn’t bothered to add this extra section, so npm had no notice of the modules they depended on.

This was easy to solve. Just add a correct section of bundleDependencies to all package.json files.
Sounded like a boring task to do this manually and because I love my programming job, I decided to write a program for it.

My obvious choice of programming languages was: awk, grep and sed. Why? Because I can.
Without further ado, here it is:

    awk '/dependencies/,/]|}/' $file |
    grep -o '\".*\".*:' |
    sed 's/^.*{//g' |
    sed 's/\"dependencies.*\://g' |
    grep -v -e '^$' |
    uniq |
    sed 's/\"[ ^I]\:/\",/g' |
    sed 's/\"\:/\",/g' |
    sed '$ s/,/ ]/' |
    sed '1 s/\"/\"bundleDependencies\" \: [ \"/' |
    sed 's/\"/\\"/g' |
    tr -d '\n'

What it does, you ask?
I'll explain line by line:

line1: get the dependencies part (an array or json object) from the file (the package.json file)
ex: "dependencies" : { "express": "0.2.2", "findit" : "0.0.1" }

line2: get only the stuff from that object between the double quotes before the colon (removing the version number part of the dependency)
ex: "express":
"findit":

line3: remove anything preceding the {
line4: remove the original dependencies text
line5: remove any empty lines
line6: remove duplicates

line7 and line8: replace the ": by ", so we can create an array from it
ex: "express",
"findit",

line9: replace the last , by an ] to close the array
ex: "express",
"findit"]

line10: replace the first " by "bundleDependencies" : [ "
ex: "bundleDependencies" : [ "express",
"findit"]

line11: precede all quotes by a backslash so it can be safely used in the sed command to add it to the file

line12: remove all newlines
ex: "bundleDependencies" : [ "express", "findit"]

This is added to the package.json file we are currently processing and if we have finished doing this for all package.json files we can use npm pack to create our tarball.
Works pretty well I might say. But I'm the first to admit this isn't the most readable program ever written.

Of course when building a node application you might have node around to help you do this so I also created a javascript version to do this:

var fs=require('fs')
var findit=require('findit')

findit.find('.', function(name) {
  if (endsWith(name,'package.json')) {
    handleFile(name)
  }
}).on('end', bundleApp)

function handleFile(file) {
  var data = fs.readFile(file, function(err, data) {
    if (err) {
      console.log('Not processesed '+file+' bo the following error: '+err)
    } else {
      var arr = []
      var packageFile = JSON.parse(data)
      if (packageFile.bundleDependencies) {
        console.log('Bundledeps already present. Skipping')
      } else {
        for (var d in packageFile.dependencies) {
          arr.push(d+"")
        }
        if (arr.length > 0) {
          packageFile['bundleDependencies'] = arr
          fs.writeFile(file, JSON.stringify(packageFile, null, 4))
        }
      }
    }
  })
}

function bundleApp() {
  console.log('Finished. preparing package.json files for packaging. Now run npm pack to create the fullblown tarball')
  //exercise left for the reader to require('npm') and run the pack command
}

function endsWith(str, suffix) {
  return str.indexOf(suffix, str.length - suffix.length) !== -1;
}

Hope somebody can benefit from this in the future.

To skip a day, it all sounds so easy. First of all i was curious to know if similar situations have occured in the past and what the software development pitfalls could be. A small investigation showed me that you need to be careful with assumptions regarding regional timezone issues. Let’s have look at how a programming language like JAVA handles these kinds of situations.

To demonstrate the consequences of this kind of detailed timezone changes and how JAVA handles them, i have compiled the following example (Apia is the capital of Samoa):
1. TimeZone timeZoneApia = TimeZone.getTimeZone("Pacific/Apia");
2. SimpleDateFormat sdf = new SimpleDateFormat("dd-MM-yyyy HH:mm:ss");
3. sdf.setTimeZone(timeZoneApia);
4. Calendar calendar = new GregorianCalendar(timeZoneApia);
5. calendar.set(2011, 11, 29, 23, 59, 59);
6. System.out.println(sdf.format(calendar.getTime()));
Obviously, the output of line 6 is:
29-12-2011 23:59:59
Now, let’s add the following lines:
7. calendar.add(Calendar.SECOND, 1);
8. System.out.println(sdf.format(calendar.getTime()));
The output of line 8 depends on the JAVA 7 version you use. If you use the original released JAVA 7 or JAVA 7 update 1, the output will be:
30-12-2011 00:00:00
If you run JAVA 7 update 2 (or if you are running a previous version of JAVA 7 and have updated the timezone configuration) the output will be:
31-12-2011 00:00:00
This example demonstrates how well JAVA has implemented their timezone awareness.

JAVA 7 update 1 was released on the 18th of october 2011, the “Samoa case” was not included here. However, Oracle provides a tool called Timezone Update Tool to update your JAVA timezone configuration. The changelog of the tool gives a nice overview of recent timezone changes. I was suprised by the amount of updates regarding timezones.

This made me curious about other, similar, cases. I found a number of cases and would like to point out one interesting case in the Netherlands where they did not skip a whole day but only a couple of seconds.

Consider the following example:
1. TimeZone timeZoneAmsterdam = TimeZone.getTimeZone("Europe/Amsterdam");
2. SimpleDateFormat sdf = new SimpleDateFormat("dd-MM-yyyy HH:mm:ss");
3. sdf.setTimeZone(timeZoneAmsterdam);
4. calendar = new GregorianCalendar(timeZoneAmsterdam);
5. calendar.set(1937, 5, 30, 23, 59, 59);
6. System.out.println(sdf.format(calendar.getTime()));
Obviously, the output of line 6 is:
30-06-1937 23:59:59
Now, let’s to the same trick as with the “Samoa case”. Add the following lines:
7. calendar.add(Calendar.SECOND, 1);
8. System.out.println(sdf.format(calendar.getTime()));
Question: What will be the output of line 8?

The correct answer is:
01-07-1937 00:00:28
The short story behind this: in the early 1900s the Dutch government declared that the official time in the Netherlands was “Amsterdam Time”. To be more specific, the meridian of the “Westertoren” in Amsterdam. This resulted in an offset of 19 minutes and 32 seconds with GMT.

On the 1st of July 1937 it was decided that the offset with GMT should be 20 minutes, creating a 28 second gap. So, officially, the first 28 seconds of the July 1st 1937 do not exist in the Netherlands. Probably, in 1937, not so many people were worried about the possible IT consequences of such a decision.

Conclusion

With this article i wanted to show that, especially in IT, you need to be careful about assumptions regarding calendar and timezone related issues. Hard to spot errors may be around the corner without realizing it. For example, when constructing a Calendar for a date/time/timezone combination that does not exists, the following happens:

• JAVA returns a date which is “rounded off” to the nearest valid value in the future (we have seen this in both examples in this article)
• If you don’t want JAVA to return the nearest valid value in the future, please set the value of the Lienient property to false and you will receive an exception instead of the nearest valid value in the future.
So, when you are unaware of the interesting variations in the timezone you develop software for, working with calendars and dates may result in unexpected return values or exceptions.



]]> http://blog.xebia.com/2012/01/15/beware-of-the-timezone/feed/ 0 http://blog.xebia.com/2012/01/13/product-owner-scaling-problems/ http://blog.xebia.com/2012/01/13/product-owner-scaling-problems/#comments Fri, 13 Jan 2012 10:45:19 +0000 Daniel Burm scaling po’s parallel timebox scaling po’s parallel timebox http://blog.xebia.com/?p=8472 Scaling the productowner (PO) role is tricky business. When you scale up too much within the same context, things become cumbersome. We don’t want to bring back the same centralized fear ridden ineffective decision making climate, we tried to kill off in the first place. When people spend so much time and effort to bring back entrepreneurship, they don’t want to create layer over layer of hierarchical PO/CPO relationships.
So if there is this perceived risk of fallback involved, why do we actually want to scale the PO role at all?

Here are some reasons I came up with when thinking of a project context(*)

The perceived need for scaling could follow as a reaction to the scope of the project at hand. It is quite common within projects to try and do as much as possible parallel development of different semi-detached functional areas of the product. This comes quite close to how I used to manage projects in the past. Whatever in the same timebox could be done in parallel, should. I was very efficiency driven back then, but also taught and stimulated to think this way.
Within this paradigm it would be preferable, that the PO has some sort of party around him helping to create the user stories he would be in charge of. Creating big teams gives us more capacity to do more work in our timebox. Larger teams and more work create the need for more coordination, because of limited span of control, thus creating the need to scale.

Also, I think we are still very much driven by the paradigm of the chain of command. It is simply comfortable when there is someone who can make the tough decisions for us and mediate between us whenever conflicting interests arise. The need for extra coordination between and over PO’s in the same context thus fits very naturally with our needs in a growing project context.
Come to think of it, maybe it is no coincidence that the person designated CPO is more times than not the first PO that was on the project. If we look deep within ourselves wouldn’t every PO actually want their first project steps to be so successful, that the extra means are granted to grow the team and consequently rise above to lead the way as CPO? And should we be that CPO, would we ever fully trust another colleague with the work we carefully prepared and poured our blood sweat and tears into? It would be great if we could somehow keep some sort of supervision on the others. Make sure the projects success and our hard work isn’t killed off in the next sprint or two. Scaling with a CPO construction would provide means to fulfill these needs.

Of course the above reasons to scale could be valid, but there are also downsides to them.

Although doing parallel work may sound efficient, maybe validation with your customers shows that the product increment doesn’t fulfill their needs as thought up in the first place. Since you have done a lot of parallel work, the risk of waste is also greater. Should you need to radically pivot your product in another direction, it will be much harder due to the already large scale of your operation.
I thus think that scaling towards having different teams work on the same backlog potentially inhibits us from maximizing validated business value, as you are pulling forward less important features from the backlog, before actually validating and thereby knowing whether you have actually delivered value with your top items. From this angle, it would be wise considering not to scale.

When scaling from success, we run the risk of slipping into a power monger mode and form a team beneath us to gain status and control instead of branching out sideways.
When this happens we are not working towards joint company stakeholdership any more, but we trying to build our own ivory towers. When the CPO leading the PO’s claims decisive power in certain areas, I believe you would not be creating the right entrepreneurial environment in which decisions are based on arguments and value for the customer and company. Maybe, as a PO, if you already know you need a decision from the CPO, you are inclined to use other influencing methods rather than comparable and less subjective measures than for example business cases, jeopardizing the quality of decision making and therefore the success of the product.


During my study, I was taught “there is no one best way to organize”. I think of this line as a universal truth, that in my opinion also applies to scaling the PO role. I therefor think that form is less important than the route taking you there. Knowing that there are risks involved in scaling and consciously dealing with them helps you along this route to find a form that works for you.

(*)Scaling is also common in productline contexts (for instance a CPO over a group of PO’s managing a product family) or within business units where you would have various strategic themes that the CPO would like to have implemented over the same time period.



]]> http://blog.xebia.com/2012/01/13/product-owner-scaling-problems/feed/ 1 http://blog.xebia.com/2011/12/29/on-cloud-3x3/ http://blog.xebia.com/2011/12/29/on-cloud-3x3/#comments Thu, 29 Dec 2011 13:50:06 +0000 Adé Mochtar paas cloudfoundry dropbox openstack iaas heroku jclouds openshift paas cloudfoundry dropbox openstack iaas heroku jclouds openshift http://blog.xebia.com/?p=8339 2011 has been an interesting year for cloud computing. Traditionally, cloud computing can be divided into three categories:
• Software-as-a-Service (SaaS)
• Platform-as-a-Service (PaaS)
• Infrastructure-as-a-Service (IaaS)

While SaaS has been around for some time (Salesforce.com started in 1999!), we are seeing an increase in adoption of IaaS and some heavy development in the PaaS world.

Now that 2011 is coming to an end, this is also the time for lists. So here are my 3 top 3’s of cloud computing.

Software-as-a-Service

3. GitHub

Distributed version control systems are gaining track rapidly, with Git leading the way. GitHub provides Git as a service, and offers unlimited repositories for open source software. It provides an excellent interface that stimulates social coding. This means it is a great incentive for open source development.

2. Dropbox

Well, who doesn’t use Dropbox? Dropbox offers a filesharing service that is easy to use. There are two reasons for the success of Dropbox; it is easy to start with 2GB of free space, and it provides clients for almost all platforms.

1. Google Apps

Google is trying hard to get us all to work in the cloud, and their Google Apps service is their way to do so. They even have a free service for private use. Google Apps provides a full application suite including e-mail, calendar and docs. GMail is massively adopted and has been a game-changer since introduction in 2004.

Infrastructure-as-a-Service

3. OpenStack

When talking about IaaS, people immediately think about the Amazon AWS platform. But what if you don’t like their terms-of-service, or simply want to create something similar in your own data center? Enter OpenStack. If there is one DIY IaaS framework that has momentum, it is OpenStack. It is backed by no less than 144 companies, and best of all, it’s open source.

2. jclouds

If all the IaaS frameworks and providers and all their different APIs are giving you an headache, jclouds is the framework for you. The jclouds API provides an abstraction of the different cloud-specific implementations. Currently over 30 providers are supported, including all the usual suspects (Amazon AWS, OpenStack, Azure, etc).

1. Amazon AWS

The undisputed number 1 of IaaS is of course Amazon AWS. Ever since the introduction in 2002, Amazon AWS is the reference implementation for IaaS. It sets the standard and the rest of the IaaS providers are merely trying to catch up. Amazon also doesn’t sit still, it constantly adds new services to its platform (and is slowly growing into a PaaS). It is available around the globe, with data centers in almost every continent.

Platform-as-a-Service

3. Heroku

Heroku is a fully hosted PaaS platform. It supports lots of languages, and it completely hides the infrastructure (servers, instances, etc) from your applications. It has a partnership with Facebook, creating the Heroku Facebook App Package, which enables quick development of Facebook apps. I think it is one of the best examples of a public PaaS.

2. CloudFoundry

CloudFoundry is being developed by VMware. After acquiring SpringSource back in 2009, this is the next logical step for them. CloudFoundy is positioned as the Open PaaS. While most PaaS solutions limit the choice of frameworks and infrastructure services, CloudFoundry tries to be open and extensible. And best of all, you can use the micro edition for development, the private (open source) edition in your own data center and the hosted edition as a public PaaS (or even a hybrid setup).

1. ???

We are seeing lots of development in this area with all different flavors of PaaS platforms and services. We have even built custom PaaS platforms for our customers based on the traditional application servers (JBoss, WebLogic, etc). But there is still lots of work to be done, before full stack solutions will be readily available. So, I think there is no number 1… yet.

And beyond…

2012 looks like it will be a good year for the cloud. I am very curious to see what the PaaS providers are going to bring to the table. Projects like OpenShift, CloudFoundry and Stratos are looking very promising, and I can’t wait to dive into them.

What are your top cloud services? Or which ones do you think that will become the next best thing? Feel free to add them to the comments below.



]]> http://blog.xebia.com/2011/12/29/on-cloud-3x3/feed/ 4 http://blog.xebia.com/2011/12/23/innovative-agile/ http://blog.xebia.com/2011/12/23/innovative-agile/#comments Fri, 23 Dec 2011 15:01:04 +0000 Daniel Burm innovation marks radical matrix markets accommodate portfolio additive innovation marks radical matrix markets accommodate portfolio additive http://blog.xebia.com/?p=8314 My motto regarding innovation is: being a first mover is a strategic choice, moving fast isn’t. Agile and scrum can help you move fast, so how can it accommodate innovation?

Getting a view on innovation
When a company fills in a portfolio tool like a Boston Consulting Group (BCG) matrix, it gets a view on its product market combination (PMC) portfolio. You can tell a lot about the company and its business from how a BCG matrix (*) develops over time. One of the most fun things in my eyes is the amount of question marks turning into stars. A question mark being a PMC in a high growth, low share section (e.g.; doing something relatively new), a star being a PMC in a high growth, high share section (e.g.; being successful in doing new stuff).



The amount of question marks in the portfolio illustrates the amount of newly launched PMC’s on to target markets. When you also consider the amount of ideas not becoming question marks, and turn this into a ratio, you could get some idea on how innovative the company is. Add to his the amount of question marks turned into stars and you really get a sense of outward successful innovation. I distinguish outward and inward innovation, because I believe that experiencing a commercial hypothesis to be proved wrong is at least just as valuable as seeing it proved right.
This doesn’t mean that innovation can’t be applied in other quadrants, just that it might not be the smartest thing to do when for instance handling dogs. In many cases adding features to cash cow products can be a brilliant strategy. Take for example razors, where adding more razorblades, self-adjusting blades and other sleight handling improvements constantly extends the product lifecycle.

Innovation flavours
To get an idea of how scrum could accommodate for innovation, first we have to get an idea on the various sorts of innovation like “additive innovation” out there. In general there are four forms of innovation a company can venture into:

Incremental innovation: small improvements leading to slightly better results

Additive Innovation: adding product features, customization, new products in existing business lines

Complementary innovation: creating new offerings new in current business, but adjacent to current product lines

Radical innovation: doing completely new things, unknown to business and/or target markets.

Using agile to suit innovation
In the following section I will highlight a couple of alternative ways in which you could use agile and scrum mechanics to shape and facilitate these forms of innovation:

Incremental innovation:
1. Spend a retrospective or a section on this product improvement; try to get in one small improvement each sprint. Maybe this will ease the team into providing more input for the backlog.
2. Agree with the team that every sprint every individual team member comes up with at least one idea to improve the product in some way.

Additive innovation:
1. Create spikes in sprints to prototype new features, validate these with customers;
2. Hold demo like meetings with your target audience; ask them what they think about the product.

Complementary innovation
1. Keep key options open, have stories worked out in two variations and do validate these paths in spikes and demo meetings;
2. Invest time in finding the appropriate product owner for the job. Market and customer knowledge is important here as the company is going to serve adjacent and different markets than before;
3. Take care that the entire DMU is taken into account in the story map. Also look at internal impact of providing new services and products, for instance customer service needs.

Radical innovation
1. Make sure the innovation team is freed from all organizational gravity. Pull them away from status quo and peer paradigms;
2. Reserve time for existing teams to work on a free format project. This could be a once every month time box of a day for example. It can be whatever they would like, as log as results are made transparent. Let them the same social objects as in scrum (boards, graphs, backlogs);
3. Take care that you have means to measure relevant metrics early on. Every addition should increase sales, market share and other relevant metrics. Use retrospectives to find root causes and steer through story map;
4. Keep all options open, incorporate A/B tests, multi-variant tests, prototypes, feature polls and so on. Sprint goals are hypotheses you would like to see validated.

What I love about scrum, is that it so lightweight and adaptable. On the incremental- to radical innovation scale, there is no step in which scrum can’t be adapted to accommodate for innovation while remaining to move fast.




The above list is just a brain dump of what I quickly came up with. I am convinced that there are many more creative ways in which we could adapt agile and scrum practices towards innovation. Please view this blog as an open invite to share your thoughts on this subject.

PS: Merry Christmas and a very Happy New Year!

(*)A BCG matrix says nothing about profitability of the PMC, so market share in a growth market could be labeled as a vanity metric. The matrix also builds on the premise that you know a market to put question marks in. Sometimes however you don’t know what your market is going to be. Furthermore, the BCG matrix can be filled in numerous ways depending on how you define for example the market scope.



]]> http://blog.xebia.com/2011/12/23/innovative-agile/feed/ 3 http://blog.xebia.com/2011/12/19/how-to-walk-with-spacewalk/ http://blog.xebia.com/2011/12/19/how-to-walk-with-spacewalk/#comments Mon, 19 Dec 2011 02:20:16 +0000 Maarten Kennis http://blog.xebia.com/?p=8234

Introduction

One of the most common challenges of managing the configuration of servers in your typical DTAP environment is, in my opinion, keeping all the involved hosts at the same level of configuration in terms of installed operating system packages and their configuration files. It really can be a pain to keep all the systems at the same configuration level. Faillure to do so can lead to interesting situations where software produced by the project team does not run or perform on the acceptance and/or production environment while it was running perfectly on the development and/or test servers.

Ofcourse, there is the possibility of creating one golden virtualized image and pass it around your DTAP environment. However, this can introduce serious issues. For example, when the company hosting your acceptance or production environment does not accept, for obvious reasons, an alien virtualized image to be installed on their precious server farm. By that time, the project has already been running for several months, the engineer who developed the golden virtualized images has left the project and the documentation turned out to be not sufficient to reproduce the golden image.

This is where a Linux systems management solution like Red Hat Satellite can help you out. Since you need a Red Hat subscription for Satellite, this article will discuss the open source alternative called Spacewalk. Spacewalk is an open source Linux systems management solution. It is the upstream community project from which the Red Hat Network Satellite product is derived. Spacewalk manages software content updates for Red Hat derived distributions such as Fedora, CentOS, and Scientific Linux.

With spacewalk you can deploy linux systems, over and over again and always the same way (using kickstart). Centrally manage the packages to be installed on a system and last but not least centrally manage configuration files for each deployed system.

Sounds cool, i want this too!

So, enough about the theory, how does this actually work? To demonstrate this, i have compiled the following cookbook. At the end of this cookbook you will have:

• A 64 bit CentOS 5.7 server running spacewalk 1.5
• Deployed a base 64 bit CentOS 6.1 vm using spacewalk
• Deployed packages on the deployed system using spacewalk
• Deployed configuration files managed by spacewalk to the deployed server

Prerequisites

• Create a vm with the following properties:
• 1 cpu
• 1 GB of memory
• Size of harddisk = 6 GB (excluding swap) + (number of distributions x 6 GB)
• vm must have internet access during installation
• Make sure you have the following installation binaries available:
• CentOS-5.7-x86_64-netinstall.iso
• oracle-xe-11.2.0-1.0.x86_64.rpm
• oracle-instantclient11.2-basic-11.2.0.2.0.x86_64.rpm
• oracle-instantclient11.2-sqlplus-11.2.0.2.0.x86_64.rpm

Getting the VM up and running

I prefer to keep things lean and mean. For this blogpost a minimal 64 bit CentOS 5.7 will be installed using the net-installer. The following walkthrough provides you with vm ready for spacewalk to be installed.

Start your empty vm booting from the attached CentOS net installer iso. During installation select the defaults or change it to whatever suits your environment for language and keyboard-type. The installation-method is, of course, http. tcp/ip configuration: whatever suits your local network needs for internet access.

Select a mirror service from the CentOS website.

Provide the web site name: my.fast.mirror.com
CentOS directory: path/to/5.7/os/x86_64

Click next on the welcome screen, choose to do a fresh install of CentOS.

Partition your disk to suit your needs. Important note regarding partitioning: This blog article assumes some defaults, based on those defaults you should be aware that there are two locations which need sufficient disk space, you may want to keep this in mind while partitioning:

• /var/satellite (5GB per distro)
• /u01/app/oracle/oradata/XE (1GB per distro)

Network setup: configure as needed for your vm to fit in your network and to have internet connection.

Finalize the installation by selecting your timezone, entering your root password and unselecting all installation tasks including the default selected “Desktop – Gnome”. Let the installer do it’s job, once the system is rebooted you have a fresh base 64 bit centos 5.7 vm available.

Preparing the system for Spacewalk

Spacewalk uses a database for it’s back-end administration, this can be eighter a Oracle (XE) or PostgreSQL database. In this article we are going to use the Oracle 11g Express Edition (XE) database together with the Oracle 11g instant client. Transfer the rpm’s to your vm and install them (as user root) using the following commands:
yum install --nogpgcheck oracle-xe-11.2.0-1.0.x86_64.rpm
yum install --nogpgcheck oracle-instantclient11.2-basic-11.2.0.2.0.x86_64.rpm
yum install --nogpgcheck oracle-instantclient11.2-sqlplus-11.2.0.2.0.x86_64.rpm
After installation start configuration by:
/etc/init.d/oracle-xe configure
After accepting the defaults (to avoid port conflicts later on in the article, it may be a good idea to specify an other http port then suggested by default. This article assumes you use port 8888), choosing passwords and specifing oracle-xe to start at boot you should have a running oracle XE available. This can be checked by executing the following command:
ps -ef | grep pmon
which should be returning something like this:
[root@spacewalk ~]# ps -ef | grep pmon
oracle    1763     1  0 16:21 ?        00:00:00 xe_pmon_XE
root      3739  1957  0 16:56 pts/0    00:00:00 grep pmon
[root@spacewalk ~]#
Next step is to create a tablespace for spacewalk to store it’s data. Start by loading the Oracle XE environment settings (note the space between th ‘.’ and ‘/’):
. /u01/app/oracle/product/11.2.0/xe/bin/oracle_env.sh
Next, start an sqlplus session.
sqlplus sys as sysdba
Create a tablespace as follows:
create bigfile tablespace spacewalk datafile '/u01/app/oracle/oradata/XE/spacewalk.dbf' size 1G autoextend on;
Create a spacewalk database user and grant it the required privileges:
create user spacewalk identified by spacewalk default tablespace spacewalk;
grant dba to spacewalk;
Oracle XE comes with an apex based management console which can be reached at:
http://hostnameOfYourSpacewalkServer:8888/apex/f?p=4950
Navigate your browser to the url mentioned above and check if management console shows up. For future reference: Oracle XE can be stopped or started using the following command:
service oracle-xe stop
service oracle-xe start

Install Spacewalk

Finally we have arrived at the point where Spacewalk is going to be installed. As user root, perform the following commands to acquire the required repositories:
rpm -Uvh http://spacewalk.redhat.com/yum/1.5/RHEL/5/x86_64/spacewalk-repo-1.5-1.el5.noarch.rpm
rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
rpm -Uvh http://spacewalk.redhat.com/yum/1.5-client/RHEL/5/x86_64/spacewalk-client-repo-1.5-1.el5.noarch.rpm
Next step is to actually install spacewalk (note: due to the speed of the spacewalk repo’s this step may take up to 30 minutes to complete).
yum install spacewalk-oracle
Next, configure spacewalk by issuing the following command:
spacewalk-setup -disconnected
After providing the setup program with the Oracle SID (XE), spacewalk db username en password the database is populated. Mostly the defaults can be accepted and/or obvious data can be provided during the rest of the setup program.

For future reference: Spacewalk can be started and stopped using the following commands:
/usr/sbin/spacewalk-service stop
/usr/sbin/spacewalk-service start
Check if the spacewalk server is up and running using the following url (note: you may get a certificate exception upon opening this page):
https://hostnameOfYourSpacewalkServer/
The first time this url is selected the following screen appears allowing you to create an administrative user.



Populate Spacewalk

The goal is to deploy a new machine with an os, this means the obvious next step is to populate spacewalk with a Red Hat derived Linux distribution of your choice. In this article the 64 bit version of CentOS 6.1 is used.

First step is to mount the CentOS 6.1 iso’s somewhere on your spacewalk server. Make sure to get a full distro iso, this means that required directories like, for example, images/pxeboot do exist. A minimal or netinst iso of a distribution, in general, does not contain these directories. These directories are used later on in this article, most important at this stage is the content and location of the Packages directory of your distribution’s iso.

The packages which belong to a distribution are administered in Spacewalk as software channels, so we first have to create a software channel before we can add/upload packages to it.

Create a new software channel by opening the spacewalk console and navigate to:

“Channels” -> “Manage Software Channels” -> “Create New Channel”

Enter a reasonable channel name (this is for display only, this article uses: “CentOS 6.1 – 64 Bit”), a channel label (remember this name for later use, this article uses: “centos6.1-x86_64″) and select the correct architecture (x86_64).

Next step is to populate spacewalk with the CentOS packages, this proces is started by issuing the following command:

rhnpush -v --channel=centos6.1-x86_64 --server=http://localhost –dir=/path/to/Packages

where “/path/to/Packages” is the absolute path of the Packages directory of the mounted iso.

CentOS 6.1 consists of two dvd’s, execute above step for both dvd’s.

The rhnpush process uploads all packages and registers them in spacewalk. On average, rhnpush processes packages at a rate of around 2000 packages per 30 minutes (ofcourse depending on the configuration of your host and vm). CentOS 6.1 contains almost 6200 packages so, it will take around one and a half hour to upload all packages from dvd1 and dvd2 to spacewalk.

Since we want the deployed linux system to be able to connect to the spacewalk server and use it’s package and configuration management facilities it is recommended to include the spacewalk client packages in a spacewalk channel as well. In this article we will upload the packages directly from the online repository into a child channel of the just created CentOS channel.

In the spacewalk console navigate to:

“Channels” -> “Manage Software Channels” -> “Create New Channel”

Enter a reasonable channel name (this is for display only, this article uses: “Spacewalk Client 1.5 – el6 – 64 Bit”), a channel label (remember this name for later use, this article uses: “swclnt1.5-el6-x86_64″), the correct architecture (x86_64) and the correct parent channel (this article uses: “CentOS 6.1 – 64 Bit”).

Populating spacewalk with the spacewalk client packages directly from the online repository is started by issuing the following command:

spacewalk-repo-sync -c swclnt1.5-el6-x86_64 --url http://spacewalk.redhat.com/yum/1.5-client/RHEL/6/x86_64

The spacewalk client has a dependency on the python-hwdata-1.2-1.el6.noarch.rpm package from the epel repository. Download the python-hwdata-1.2-1.el6.noarch.rpm package from the epel repository ( http://download.fedora.redhat.com/pub/epel/6/x86_64/ ) and upload it to the spacewalk client child channel using the command (assuming you downloaded the rpm to a folder named epel):

rhnpush -v --channel=swclnt1.5-el6-x86_64 --server=http://localhost -dir=epel

Create a distribution

For automating the installation of a Linux system a method called kickstart can be used. First, we have to setup a directory structure on the spacewalk server based on content of the CentOS dvd1 iso. From your CentOS 6.1 dvd1, copy the following directories:

• images
• isolinux
• repodata

to:
/var/distro-trees/centos6.1-x86_64

Next, open the spacewalk console and navigate to the following location:

systems -> kickstart -> distributions -> new distribution

Enter the following parameters for the new distribution:

• Distribution label: centos6.1-x86_64
• tree path: /var/distro-trees/centos6.1-x86_64
• Base Channel: CentOS 6.1 – 64 Bit
• Installer Generation: Red Hat Enterprise Linux 6

Next step is to create a kickstart profile for the channel and distribution. Open the spacewalk console and navigate to the following location:

systems -> kickstart -> create new kickstart profile

Enter the following parameters for the new kickstart profile:

• Label: centos61-minimal
• Base channel: CentOS 6.1 – 64 Bit
• Kickstartable tree: centos6.1-x86_64
• Virtualization type: none

To make sure the spacewalk client repository is used during kickstart, navigate to the following location:

systems -> kickstart -> profiles -> centos61-minimal -> operating system

Make sure the child channel swclnt1.5-el6-x86_64 is checked.

Also, have a look at the other tabs to have an idea of the configuration options which are available. Possible interesting area’s are:

• Software: Adding extra packages or package groups in addition to the base installation. Add the package just by adding it on a new line, package groups can be added by an @-sign followed by the group name. A package can be excluded by an hyphen (-) followed by the package name
• Kickstart details -> Details -> Kernel options: Adding and removing kernel options. You can add a kernel option, just by adding it’s key/value pair to the input field. Removal is done by just mentioning the kernel option preceded by an ! and giving it ~ as a value. For example, the value “!text=~ resolution=800×600″ in the kernel option box forces the use of the graphical installer (remove the text kernel option) and sets screen resolution to 800×600.
• Kickstart details -> Advanced options: Allows detailed configuration of the kickstarted system. For example, to add an user,during installation, named weblogic with password weblogic01, tick the “user” checkbox and add the value “–name=weblogic –password=weblogic01 –plaintext” to the input field.
• Kickstart details -> Variables: the usage of variables can be done by adding a key/value pair and refer to it in another tab. For example (might be a bad example but it is just to demonstrate the usage), to define the hostname during kickstart, add a key/value pair (hostname=appsrvr1) in the variables tab and refer to it in the Advanced options by adding “–hostname $hostname” to the network text box.

Let’s cobbler

Next step is to create an iso image to boot a new vm from. Important note: In the next couple of steps we are going to deploy a new linux virtual machine. If your virtualization network setup supports a dns where the spacewalk server can be found by it’s hostname you can skip the next step. In other words, your newly created vm must be able to find the spacewalk server using it’s hostname during boot/initial setup. If this is not the case or if you are unsure, please perform the following step to change the spacewalk hostname to it’s ip-address, if you are sure dns is in place you can skip this step:

In /etc/rhn/rhn.conf change the value of the parameter cobbler.host to the ip address of the spacewalk server.
In /etc/cobbler/settings change the value of the parameters server and redhat_management_server to the ip-address of the spacewalk server.

On the spacewalk server, run the command (this only needs to be done once):

cobbler get-loaders

Next, start building the iso using the command:

cobbler buildiso

The result of the buildiso command is a file named generated.iso in the directory from where you issued the command.

Let’s kickstart

On your host, create a new virtual machine and provide it with the generated.iso file to boot from. Upon boot you will see a menu allowing you to specify the centos61-minimal setup to be installed.



Select this entry and the setup will install a base 64 bit CentOS 6.1 Linux system. If all goes well, this will happen completely automated, without any user intervention whatsoever. If, during install, you receive messages like “Error downloading kickstart file”, this probably means you have to look into dns issues as described earlier in the article.

Verify that the system registered itself in spacewalk, it should appear in the system tab on the main screen of the spacewalk web console.

Configuring the client

Now that we have installed a fresh 64 bit CentOS 6.1 Linux vm we have to configure it as a client for spacewalk. Open an ssh session to the newly deployed CentOS 6.1 vm and install the packages rhncfg-client and rhn-check using yum.
yum install -y --nogpgcheck rhncfg-client rhn-check

Managing the configuration of this newly created vm can be done in the following two ways:

• Deploy new packages to the client
• Deploy (configuration) files to the client

Deploy new packages to the client

To install a new package from the repository to the new server, go to the spacewalk web console and navigate to the following location:

system -> “your system” -> Software -> Packages -> Install

Select the required package from the repository (for example xauth) and click on “Install Selected Packages”

Next, select “Schedule action as soon as possible” at the confirmation screen and click on “Confirm”

Now, log on to the client and verify the software channels it is subscribed to by executing:

rhn-channel --list

Check if the channel where you made the pending change is in the list. Next, verify if the selected package is not installed yet by executing, on the client:

[root@appsrvr1 ~]# rpm -qa | grep -i xauth
[root@appsrvr1 ~]#

If the package is not installed yet, apply the pending change (installation of the package) by executing:

rhn_check

The server will check for any pending actions (in this case installing the selected package) and execute (install the package) them. Now, check again to verify that the (xauth) package was installed by executing:

[root@appsrvr1 ~]# rpm -qa | grep -i xauth
xorg-x11-xauth-1.0.2-7.1.el6.x86_64
[root@appsrvr1 ~]#

Deploy (configuration) files to the client

In case of managing the configuration files of a linux system through spacewalk, this can be done through configuration channels.

First of all, create a new configuration channel. Open the spacewalk web console and navigate to the following location:

Configuration -> Configuration channels -> create new config channel

Enter information to identify the config channel:

Name: My Config Channel
Label: myConfigChannel
Description: My Config Channel

Next step is to populate this channel with files and directories by navigating to the following location:

Select the configuration channel -> add files -> create file

Now you can create files, directories and symlinks, set ownerships and file permissions. In case of creating a file it is possible to add the actual content of the file in the inline editor. Click on “Create Configuration File” to finalize this action.

To deploy this file to the managed linux system, this system must first be subscribed to the config channel. In the spacewalk web console, navigate to the following location:

systems -> “your system” -> configuration -> manage configuration channels -> subscribe to channels

Next, verify if the client is successfully subscribed to the newly created config channel by executing the following command on the client:
[root@appsrvr1 ~]# rhncfg-client channels
Using server name spacewalk
Config channels:
Label Name
----- ----
myConfigChannel My Config Channel
[root@appsrvr1 ~]#

If the channel appears in the output of the previous command you can get those files (or directories) by issuing:

[root@appsrvr1 ~]# rhncfg-client get
Using server name spacewalk
Deploying /opt/oracle
Deploying /opt/oracle/middleware
Deploying /opt/oracle/middleware/jrockit
[root@appsrvr1 ~]#


If you want to verify if there is a delta between your system and the config channel you can do so by executing
rhncfg-client diff

Conclusion

As usual with this kind of systems, it takes a lot of effort upfront to set it all up. With this article i hope i will help the reader by setting up a spacewalk system relatively easy and fast. Hopefully, the reader will realize pretty soon that managing Linux systems now really is a breeze and all effort for setting it up was worth it. In my opinion, as of version 1.5, which is current at the time of writing this article, stability and functionality has increased a lot since I started working with Spacewalk. If you’re looking for a way to manage your Red Hat derived Linux systems, i highly recommend taking a look at Spacewalk.



]]> http://blog.xebia.com/2011/12/19/how-to-walk-with-spacewalk/feed/ 3 http://blog.xebia.com/2011/12/14/continuous-delivery-for-enterprise-java-applications/ http://blog.xebia.com/2011/12/14/continuous-delivery-for-enterprise-java-applications/#comments Wed, 14 Dec 2011 12:00:41 +0000 Mark van Holsteijn puppet layer7 balancer puppet layer7 balancer http://blog.xebia.com/?p=8202 How do you setup a environment that support the continuous deliver of enterprise Java applications? How do you manage the large number of machines that are involved? How do you enable self-service, continuous delivery of applications onto the platform?

In this blog post we will give a description of an open source Java Application Platform as a Service that we created for our customer, using VMware, Redhat Enterprise Linux, Apache WebServer, JBoss Enterprise Application Platform, JBoss Operations Network, Puppet, Deployit,  F5 Load Balancer and  a Layer7 SecureSpan gateway.

Data Center Quality Platform

The customer wanted a data center quality Java Application Platform with the following features:

• Standard configuration
• Standardized provisioning
• Standardized deployment
• Centralized monitoring
• Centralized access control
• Virtual environment
• Proving technology

Current situation

As the current Java application platform was based on HP-UX on Itanium, the customer was facing high cost for hardware, software licenses and fading support from software vendors.  As all applications ran on a HP Superdome, it was very difficult to add resources to individual applications. In addition, development teams spend too much time taking their software through the development, test and acceptance environments, resulting in slow delivery of software into production. Finally, it was difficult to provide 24×7 availability because all applications are running on a single machine.

Java Application Platform

The following figure illustrate the solution architecture of the java application platform.



In the following paragraphs we will describe the purpose of the most important components.

Dual Data Center – HP

Not shown in the figure, is the hardware setup of the platform. It consists of HP blades setup in two data centers on two different locations. This provides the basic infrastructure for 24×7 availability and fault tolerance.

VMware ESX

VMware ESX is deployed on top of the hardware in the dual data center. This provides us with the ability to create virtual machines and provide high availability in case of single server of single site failures. It also allows us the quickly scale up virtual machines and increase the resources assigned to individual virtual machines.

For all machines in the platform we use a single VMware template image. This image is installed with RedHat Enterprise Linux and a puppet client.

Puppet



Puppet fully automates system management. It is used for the installation of software packages, conformity tests and day to day system administration tasks.  For every type of node, we have a puppet plan. When the machines boots, the puppet agent provisions the machine with all the necessary software and configuration according to the plan for that machine.

The use of Puppet completely automates and standardizes the configuration, ensures 100% reproducibility of the configuration and is fast. Provisioning of a new machine from the template to full operational mode is done in a matter of minutes.

JBoss EAP

JBoss Enterprise Application Platform is the Enterprise Java applications server for all java applications.  The installation and configuration is done by Puppet and uses the official RedHat RPMs.  Puppet configures JBoss to ensure that :

• JBoss management applications authenticate users against Active Directory, providing a single point of authorization for operations.
• A JBoss Oracle database schema is automatically provisioned for that specific instance of JBoss, providing persistence for the JBoss server system state.
• All Business Applications can authenticate users using SAML against the Layer7 Identity provider, providing a single point of authentication and authorization for their customers.
• The JBoss instance is added to the pool in the F5 Load balancer
• The application server is added to the Deployit infrastructure inventory, providing the tenants of the platform with the ability to deploy applications to the server.

JBoss application servers are always deployed in multiples of two, where each server of a pair is assigned to a physically different data center location by VMware.

The use of puppet provides us with a fast and reproducible way of provisioning JBoss application servers, allowing for a fast and reliable scale out mechanism for the applications.

JBoss Operations Network



JBoss Operations Network (JON) is used for monitoring all the resources in the platform.   By default, Puppet installs a JON agent on every machine. This agent scans the inventory of the machine and reports it to the JON server.

JON has a very good support for high availability and fail over. By simply adding a JON server machine, agents will automatically distribute themselves across the servers and failover if necessary. Each JON server also runs a JON agent, making sure that unavailability of a JON server is also covered.

In JON we created a number of alert templates for different resource types (os, apache, jboss, jon, puppet, etc.)  that will monitor and report critical conditions on the system.  All error messages from the JBoss servers logs are reported as incidents.

All alerts and clearing conditions from JBoss Operations network are reported via SNMP to TNG Unicenter.

Through the  use of JBoss Operations Network all machines, servers and resources in the platform are automatically added to the centralize monitoring system.

Deployit



Deployit is used for the automated deployment of applications onto the platform. It automatically deploys all the application components in a stack to the appropriate containers.  Deployit :

• deploys static content and proxy configuration to the apache webservers,
• deploys enterprise java application components to all individual JBoss servers in the farm,
• executes SQL scripts to the database,
• configures the F5 loadbalancers to add or remove servers or applications to the pool,
• applies environment specific changes to the application configuration.

The deployment plan for a specific application is prepared in close cooperation between the application developer and platform management staff. When the deployment plan is finished, developers can deploy new versions of the application themselves, directly from a build tool or manually.This ensures solving any installation or configuration problem isn’t postponed until the application is installed for production use, but rather is solved at the early stage of any development.

The same deployment plan is used for all environments. Authorization can be configured per enviroment and per application. LDAP is used to authorize software developers to deploy and configure an application for development and testing purposes, while integration specialist can deploy the application in production.

The use of Deployit provides the platform with a fully automated and standardized deployment mechanism, improving the speed of deployment of applications through the development, test and acceptance environments while reducing the number of staff involved and lowering the number of configuration errors.

F5 Load Balancer



The F5 Load balancers is used to support scalability and fail over for the JBoss Application Server farm.

The pools are configured to use a sticky session protocol based upon the JSESSIONID session cookie. If the cookie is not present, round-robin load balancing of the HTTP requests is performed.

Puppet adds the JBoss servers to the  pool in the F5 Load Balancer.

When a server is scheduled for a restart, the server is taken out of the pool. This ensures that this server does not get any new request, but will still be servicing existing sessions. When the session count in JBoss drops to zero, the server is restarted and restored to the pool.

The use of the F5 Load Balancer provides us with the ability to increase and decrease the number of servers in the farm, provide load balancing, fail over and graceful decommissioning of servers in the farm.

Layer7 XML Gateway / Identity Provider



The Layer7 SecureSpan Gateway is used as centralized security policy enforcement point and SAML identity provider.

Layer7 supports multiple authentication methods, Kerberos, digital certificates, username+password and is able to use multiple identity stores.

Puppet configures all JBoss application servers with SAML support and configures Layer7 as the identity provider: JBoss receives authentication (identity) and authorization (roles) information as a SAML-token. The information contained in the token is translated to a standard JEE-principal user (using a tiny layer of custom code), so all JEE applications can access the authentication and authorization information in a standard way.  Whether the JEE application is a web application or provides webservices, from a security there’s no distinction. All application designers have to do is declare the application security roles conform the JEE standard.

The use of Layer7 standardizes the authentication and authorization for all business applications and centralizes access control.

Conclusion

The customer wanted a modern data center quality Java Application Platform to ensure that java applications could be deployed with lower cost and with high availability and easy scalability.

VMware, the dual data center, Layer7, the F5 Load balancer and JBoss provide the infrastructure for a  high availability and scalability for any java application. The combination of VMware, Puppet and Deployit are the fabric to enable continuous delivery of enterprise java applications.

Through virtualization and automated provisioning and deployment it has become possible to add a completely new, correctly configured machine to a cluster in a matter of minutes, completely secure and under full monitoring.



]]> http://blog.xebia.com/2011/12/14/continuous-delivery-for-enterprise-java-applications/feed/ 4