Part 7: Best practices

In the previous blog posts in this series we discussed the role of technical leads (and in particular of Technical Product Owners [TPO]) in an agile software development process. Since then we have had several questions. The main question was: “I am in a classic architecture team. The development teams go agile. How can I get more involved with the teams?”. In this post I would like to focus on some best practices: how can you, as a TPO get more involved and facilitate the teams to deliver better quality? Read more

Part 5: The Senior Software Engineer

In Scrum it is best practice to have in each sprint all the team members needed to implement the user stories in that particular sprint. Most members will be either developer or tester. Some of them will play a more pronounced part in delivering a potential shippable product. A reason for this is that they take part in the process from sprint one. The effect is that these team members grow seniority over other team members because they have more in depth knowledge of the system under development. They will be consulted more than others by the Product Owner and the Scrummaster. In our process from “Concept to Cash” we call them Senior Software Engineers (SSE). How is a SSE related to the Technical Product Owner (TPO)? Read more

Part 4: The Chief Technical Product Owner

In part 3 of this series, Herbert introduced the role of Technical Product owner as a counter part of the (Functional) Product Owner. Before Product Owners and Technical Product Owners can start their work, high-level priorities need to be set by upper management. In this post the Chief Technical Product Owner is introduced. He represents the technical perspective in this high-level prioritization phase.

Getting Things Prioritized

In a typical company, there are many requests for new functionality. Usually, the number of requests exceeds the realization capacity by far. Therefore, early on in the process, choices have to be made. Functionality can be picked up in the order of priority. And priority is a result of expected business value, relative cost and feasibility.

Early on in the process, it is difficult to estimate the effort to realize it. Usually, at that stage a functional request has not been analyzed, detailed or scoped strictly. Providing a low estimate gives the business owner a blank check to later on design the user story in such a way that it is more expensive to build than initially estimated. They can refer to the initial low estimate when IT says “no”.

Providing a high estimate may block the feature from ever getting picked up. The business owner may look for external parties to realize the functionality without involving IT. As a result, there are even worse architectural incompatibilities.

In the agile organization model sketched in the first post, there is a phase designated for prioritization of features. For a large part, prioritization is done by business managers. Many business managers have no idea of technology. They cannot estimate the technical size of a story.

Some functional requirements depend on larger technical changes. For people with a technical background, it seems difficult to explain the value of such technical changes to business managers. Especially when talking to upper management. This is where the Chief Technical Product Owner (CTPO) comes into play.

The Chief Technical Product Owner

The Chief Technical Product Owner is a specialized Technical Product Owner who distinguishes himself from the others with political sensitivity and the ability to communicate and negotiate with upper management. He helps the business decide which functionalities should be picked up in what order and how to improve the non-functional qualities of the system.

In an ideal world, stories should be prioritized strictly on business value. In the real world there are practical and technical hurdles to overcome like legacy systems and technical debt. Sometimes, in order to pick up a set of user stories, in an effective way, a technical change needs to be made first.

The Chief Technical Product Owner distinguishes himself from the Technical Product Owner by being more focused on the management. The Technical Product Owner is more focused on the team.

On the other hand, the Chief Technical Product Owner needs to have an understanding of technology in the Ready, Done and Production phase. For example, when the system operators are haunted by production issues, the CTPO should convince upper management to take actions that provide a structural solution on the long run rather than loading the teams with large piles of new functionality to build.

To do this, the CTPO is a high profile function. If you draw a comparison with traditional IT organization, the manager of the architecture department may be best capable of fulfilling this position. If no manager is available for this position, you could consider one of your enterprise architects to fulfill the role of CTPO. Make sure that upper management trusts this person and that he is sensitive to the technological challenges in the IT organization.

What is next?

With the roles of CTPO and TPO, the prioritization and specification phases of a user story’s lifecycle are well-managed. In terms of classic architects, the architecture manager / enterprise architect and information architect are replaced with more agile roles. The next post will discuss the architectural responsibilities of the Senior Software Engineer in an agile organization.

Stay tuned!

The metaphor technical debt was introduced at the OOPSLA conference in 1992 by Ward Cunningham. The phrase was:

“Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite…
The danger occurs when debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.”

The following picture shows how this “stand-still” described by Ward Cunningham is achieved.

http://theagileexecutive.com/2010/09/20/how-to-break-the-vicious-cycle-of-technical-debt/

In most discussions/blogs on the internet the following causes are mentioned and most are code related:

• the business forces the development team to take shortcuts, aiming to get the requested functionality life as soon as possible.
• the development team thought they didn’t need to design.
• the development team was incompetent/didn’t bother to write maintainable/changeable code.
• the development team  recognizes later for example by incorporating a change, that they made a suboptimal decision in the past (difficult to avoid).
  

Martin Fowler summarized this very well in his blog “TechnicalDebtQuadrant“. He distinguishes between reckless vs. rational behavior and  deliberate vs. carelessly/naively choices. But this is still mainly code related.

But if you look at the forms of technical debt in real life situations, you’ll see that these are not only code related. Technical debt may manifest itself in the following form to us:

• Code form (as mentioned already), e.g. duplication, high McCabe index, high unit-length per method, high number of parameters in methods, wrong comments, unclear naming of objects, methods, variables and properties, etc.
• Architectural form, e.g. violating separation of concern, the choice of not using a third party framework but build your own, not changing/adapting your architecture to the changes (gradual or abruptly) in the world around us.
• Testing form, e.g. a decision to not do automatic testing, while testing the system by hand is to complex or takes to much time, to conclude after testing that everything is OK.
• Deployment form, e.g. when a deployment is arranged in such a way that it costs a lot of time and is error prone, thus limiting the number of releases in a period in time.
• Documentation form, e.g. it is unknown how the system should react in certain situations because it is not functionally described. This makes future functional wishes time-consuming to fulfill.
• Functional form, e.g. by business requested and by the development team invented functionality that is not really needed and makes things unnecessary complex for the upcoming change.
• Project form, e.g. distribution of experienced people vs. not experienced people in your team, same applies to skills in the team, onsite/off-shore team distribution, etc.
• Organizational form, e.g. not asking your software (application) provider what kind of quality assurance policy they apply and if they can provide any quality figures based on a ISO maintainability-standard.

All these technical debt forms result in less changeability, more costs and delay in bringing new features life. All forms can be plotted on the TechnicalDebtQuadrant of Martin Fowler. Whether technical dept is categorized as deliberate, inadvertent, reckless or careful, it is good to be aware of this and see how you want to cope with it strategically.

Conclusion

Technical debt is not only technical and has several forms other then code form.  The metaphor “technical debt” is invented to visualize the problem of decreasing productivity in the life cycle of the software product. People  can communicate with the metaphor more easily about the problem it addresses. This way the concept became known by development community, and unfortunately less known in other areas listed above.

Management familiar with the metaphor, can use it as an advantage by making strategic decisions regarding technical debt, because they understand the concept of normal debt. But for strategic decisions the management should be aware how much technical debt there currently is (like a normal debt) and the chances of a technical credit crunch leading to possible technical bankruptcy….

So is technical debt only technical. No, it is not.

Part 3: The Technical Product Owner

Retrospective

In the previous 2 posts we described the agile organization model and a couple of technical roles in this model. In this post we will dive into the role of Technical Product Owner. I will compare it with what we have started to call the “classic architect”.

The classic architect

Many architects are not involved in the process of writing software. Typically they are standing on the sideline, watching the game. Their way of communicating is through documents. Read more

Part 2: Where is the employee formerly known as Architect?

The agile organization model

In the first blog post in this series Adriaan described a reference model for agile organizations. In other words: how an idea can be transformed into cash. An idea is prioritized together with all other ideas. It then gets planned, after which the idea is broken down into epics, user stories and tasks. At the end working software is deployed to production. In this process Product Owners play a key role. They coordinate the process to get ideas prioritized and user stories ready. Their main focus is business value. But, how about all those technical features and aspects? These technical features and aspects are the subject of this blog post.
 Read more

Part 1: The Employee Formerly Known As Architect

Every system has an architecture, but it is not necessarily designed by an architect. This post sketches a reference model for agile organizations without the role of Architect. Next posts in this series will provide best practices on how to improve architecture within this architect-less agile reference model.

Classic IT Architects do not fit in agile organizations

Agile organizations seem hostile environments to classic architects. The DONE-teams are encouraged to Do The Simplest Thing That Could Possibly Work (DTSTTCPW) and avoid Big Design Up Front (BDUF). In addition, teams are encouraged to be self-organizing and make their own decisions.

With a classic architectural mind frame, it is easy to feel left out in an agile organization. If you keep writing Project Start Architecture documents, it seems like the documents are never read. Or worse: you never get the chance to write the document properly, because the DONE-team has already started developing.

Architecture remains important

This blog post is the start of a series introducing best practices for architecture in an agile organization. In this new organization, there is a good chance that nobody has the title Architect anymore. Yet, the architecture of the system remains of vital importance to your long-term success.

The process for creating and maintaining architecture in an agile organization is different! Architecture becomes a team responsibility. So how can you make sure the teams make the right decisions?

Introducing an agile organization model

This first post introduces a reference model of a typical agile organization. The following parts in this series will build on this reference model to introduce the technical roles in all phases of the software lifecycle.

Getting it DONE

Agile is a container term for a number of processes: SCRUM, eXtreme Programming, RUP, DSDM and possibly others. The most popular agile process is SCRUM. This series assumes a SCRUM process extended with principles and practices from the eXtreme Programming community.

Figure 1 provides an overview of the SCRUM process. In the To Do column, there is a backlog with User Stories that are iteratively implemented into Potentially Shippable Software (DONE) in 2-3 weeks sprints. During sprints, there are daily standup meetings facilitating team communication on progress and impediments.

SCRUM is a lightweight process. It provides a helpful structure, mind set and best practices for a single development team. However, it does not help you on how to create user stories, manage architecture and scale over multiple teams. It also does not tell you how to bake a cheese omelet, for that matter.

In a larger organization, you probably need user stories, architecture and multiple teams. And probably, at some point you need cheese omelets too. So, rather than switching to a heavyweight process that covers all of these, you should extend SCRUM with additional processes and best practices for the challenges at hand.

You can choose the additions that fit with your organization.

Getting it READY

In order to write user stories before a sprint starts, you can introduce iterations preceding the SCRUM sprints to get stories READY. Figure 2 illustrates the model of an agile organization and how the READY team relates to the DONE team.

In a READY team, product owners cooperate with stakeholders, analysts and consultants in order to write specific user stories for the next sprint.

A product owner may serve multiple DONE-teams. However, in a larger organization, there may also be multiple READY-teams. For good communication, it is important to communicate over teams. A Scrum of Scrums is one of the solutions to facilitate this kind of communication.

Getting it PRIORITIZED

It takes time to write good user stories. If the READY team is unable to deliver stories before the next sprint, a DONE-team cannot start developing on time. How do you know which stories to write first? For this, you need to have a good understanding of priorities.

Priorities depend on your company goals. In most companies, the demand for new software features is larger than the capacity of people to realize the software. So, choices need to be made.

The reference agile organization model recognizes a Chief Product Owner who decides the priorities of new initiatives before user stories are written.

In order to decide on priorities, even before writing the final user stories, you need high-level analysis and communication. For this purpose, there is a Getting it PRIORITIZED phase.

Getting it in PRODUCTION

The only valuable software is software in production. Traditionally, SCRUM aims to deliver Potentially Shippable Software. In the ideal case, the DONE team goes one step further. The definition of done should be: DONE = LIVE.

At some point, there is a transition of software from development into production. And in production, software needs to be maintained.

In an agile organization, the transition needs to be as smooth as possible. And transitions need to be made as often as possible because the more often you do it, the easier it gets. Best practices to support this are described in the Getting it in PRODUCTION phase.

So, where is the architect?

Over the four phases, you have met product owners, chief product owners, developers and operators. But none of the phases bothered to mention an architect. So, where is the architect?

Remember the introduction of this blog post? It said: “In this new organization, there is a good chance that nobody has the title Architect anymore.” The good news is that the introduction also said: “Every system has an architecture, but it is not necessarily designed by an architect.”

Stay tuned for the next episode of How to Kill the Architecture Department

The next posts will explain the new roles for The Employee Formerly Known As Architect in the agile organization. Each phase has its own architectural role. Below is a sneak preview of one of the new roles. Keep reading following parts of this series and find out what the other roles are.

What are the responsibilities of a Technical Product Owner? And how do these compare to the responsibilities of an Architect in a classic organization? Please share your opinion by commenting below!

To be continued in two weeks!

This Blog is a kick off to for many writings about architecture in an Agile World. We will explore the topic from all the views possible, in order to gain a better understanding about it. By doing so, we hope to create a community of followers, who would also like to contribute or discuss about this topic.

Xebia is helping many organizations in the Netherlands, France, the United States and India with implementing an agile way of system development. In most of the cases the Scrum method is applied and very good results are achieved. Business and IT are working much closer together, resulting in more quality and much more customer satisfaction. However, lately we also see a trend in problems that seem to occur in (almost) every organization. Software is developed in a fast way with high quality, but it takes forever to get it in production. The more teams are being formed, the more interdependencies between the teams occur Read more

In my last blog I presented an illustration which shows the two primary aspects of the architects’ role. On one side they play a role in strengthening the heartbeat. On the other side, they play a role in envisioning the future.

The focus in this blog is on the solution architect or application architect. The way the Enterprise architect deals with Scrum will be explored more in detail in a later blog. This blog combined with the previous 3 blogs can be also downloaded as a whitepaper from the Xebia website: http://www.xebia.com/architects_scrum

What is the role of the architect?
Last blog I presented the illustration as shown below. In this blog I will focus on the parts of this illustration in which the solution architect / application architect plays a role

 Read more

In my last blog I presented an illustration which shows the two primary aspects of the architects’ role. On one side they play a role in strengthening the heartbeat. On the other side, they play a role in envisioning the future.

The focus in this blog is on the solution architect or application architect. The way the Enterprise architect deals with Scrum will be explored more in detail in a later blog. This blog combined with the previous 3 blogs can be also downloaded as a whitepaper from the Xebia website: http://www.xebia.com/architects_scrum

What is the role of the architect?
Last blog I presented the illustration as shown below. In this blog I will focus on the parts of this illustration in which the solution architect / application architect plays a role

 Read more