Measuring Performance for Teams

If you've ever been in an organization that had performance reviews, you may have found yourself wondering whether the measurement process used to evaluate your (and your team's) performance made sense, and if there was a better way. Even if you have goals that can be evaluated quantitatively, rather than some metric that feels arbitrary, you may feel that your personal goals may run counter to the success of the your team. For example, you may wonder whether it makes sense to help your colleague on her high priority project and risk missing a deadline for your (lower) priority one. Sometimes the problems with measurement systems are because people just don't measure well. In other cases, it's because it's impossible to measure all of the things that matter.

Rob Austin's book Measuring and Managing Performance in Organizations gives you a model to understand why measurement systems become dysfunctional, and an approach to avoid dysfunction when you are measuring.

Austin addresses some core issues that agile (and other) teams face:

• How to evaluate and reward individual knowledge workers doing complicated things in teams
• How to motivate the individuals on the team to do what helps the team meet their goals, and
• What performance measurements are helpful, and which just add noise.

Austin explains why individual measures and goals may be counter-productive unless you can measure all of the aspects of a person's work, why most people want the team to succeed, and how that desire for team success is a key motivator, and why a staple of agile project management: visible and frequent feedback on team progress can improve your chances of success. While this book was written before agile became a popular term, anyone familiar with agile methods will find the ideas in this book in tune with agile methods.

Much of what I learned from reading this book seemed obvious in retrospect, but Austin explains the problem with clarity and precision, making observations that only seemed obvious once I read them. Early in the book, for example he points out:

Employees true output (such as value to the  organization) is intangible and difficult to measure; in its place organizations choose to measure inputs (such as the amount of effort devoted to a task...)

Which seems to be such an obvious problem with many evaluation systems that you wonder why so many organizations still do it.

Reading this book won't give you a cookbook for designing a motivation and performance evaluation system.  This is a difficult problem, especially for those working in an industry where there is a strong desire to quantify and measure. But this book will help you to understand the problem and enable to evaluate and improve your current practices.

While Austin's book will help you understand the model behind effective performance measurement, there are also more day-to-day practices you need to help your team be successful. For these  consider reading Johanna Rothman and Esther Derby's book Behind Closed Doors: Secrets of Great Management, which is an excellent guide to the day to day process of people management.

Managing people can be intuitive, but it is also more difficult than many people realize. A desire to measure is useful, but it can be counter productive when you measure without understanding. Software development is a collaborative, human activity, and as such we need to understand that management and measurement and difficult, and doing either without without an understanding of the challenges can lead to unexpected results.

Things about Release Management Every Programmer Should Know

As I mentioned earlier I was privileged to contribute to the book 97 Things Every Programmer Should Know: Collective Wisdom from the Experts. In addition to the contributions about coding and design, I was pleasantly surprised to see the number of items that relate to release management. While I've long been interested in how to build architectures and processes that make deploying and releasing software easy, I sometimes get the impression that these items were often though of necessary evils that could be done at the end, often by the someone who isn't doing "more valuable work." Much like awareness of agile software development made it obvious that testing and quality assurance activities work best when they are integrated throughout the development lifecycle, agile has also made it more obvious why build and release engineering is something to work on as you go. This makes a lot of sense, as ease of release is closely tied to the physical architecture of the system, and your build process defines the physical architecture.

Some of the posts of interest are the following, though I could have added others than related to testing as well.

1. Deploy Early and Often by Steve Berczuk
2. Install Me by Marcus Baker 
3. Keep the Build Clean by Johannes Brodwall
4. One Binary by Steve Freeman
5. Own (and Refactor) the Build by Steve Berczuk
6. Put Everything Under Version Control by Diomidis Spinellis
7. Step Back and Automate, Automate, Automate by Cay Horstmann

Aside from being excited that programming experts understand the importance of build, release, and installation mechanisms, I was also excited to see that one topic that people associate with SCM wasn't mentioned: Branching.

Branching is a useful tool when used in the right context, but more often than not, branching is used as a way to avoid issue rather than to address them. Rather than branching because there is a true divergence in the code, we branch to avoid breaking existing code. The problem is that doing so simply defers a cost. Sometimes deferring the cost makes sense. Often it's better to invest in the techniques you need to enable incremental change while keeping the codeline working.

97 Things Every Programmer Should Know is about much more than just coding, or just release management. And that's the point: programming is a multi-faceted skill and to write good code, you need to know about more than just writing code.

Planning is a Gerund

One of the things teams adopting agile struggle with is deciding how much to define a plan before you start executing. Have a plan that's too well developed and you end up risking that your team may not be responsive enough to change. Too little of a plan and you may end up changing course excessively, and have no way to measure your progress towards any sort of deliverable. 

At the core of this confusion over how much to plan is the reality that plans change, and spending too much time and energy creating a plan that ends up being wrong seems wasteful. But the lack of a plan means that you have nothing to measure progress against. One way to reconcile this is to keep in mind the following quote, attributed to Dwight Eisenhower (and a variant attributed to Churchill):

Plans are nothing; Planning is everything.

If we assume that as a project progresses, that things will change, we can still benefit from talking through the options as a team. Capturing the things that we don't know, but would like too, is useful information, and gives the team a good measure of risk.

The time you spend planning is an important consideration. Constrain the amount of planning time based on the duration of your sprint. If you can't come to an understanding of what the problem is or how to approach it, you have a clue that you're trying to do too much. But rather than throw up your hands, you can aim to have some sort of plan. It might be wrong, but even building the wrong thing can increase your understanding.

For the planning activity to be useful it is important that it not be top-down but that it involve the implementation team, as they are the ones who can speak to the the implementation risks, and can propose creative solutions given the ability to probe about real goals.

One thing that may concern people with the approach of involving the team and stakeholders at the same time is that a planning which raises more questions that providing answers can make some people uncomfortable. Senior managers may be uncomfortable with acknowledging ignorance. Team members may be put off by seeing that there are legitimate disagreements among the product ownership team about some issues. And some people are just uncomfortable when you can't just tell them what to do.

This is a cultural issue that may not be easy to overcome, but agile projects work well because the team can pull together to solve problems when given all of the information, and structure their code and their work to  mitigate risk. And if the uncertainty exists it's better to identify it up front.

Regardless of the level of uncertainty about goals and dependencies it is important to exit a planning session with a common vision for the goals and target for when you will re-evaluate the goals. A well run  planning activity can helps to focus the team towards a common goal.

Book Review: Modular Java

I recently read  Craig Walls' book Modular Java: Creating Flexible Applications with Osgi and Spring (Pragmatic Programmers). This book is a very detailed tutorial that walks you through setting up an application using OSGI and Spring with the help of Maven as a build tool. If you aren't familiar with any of these technologies, this book will get you started, and quickly have you feeling like you have a basic grasp of the concepts and technologies.

You'll finish the book with a desire to learn more about the technologies, and understand the power of modular applications.  As a tutorial, this is an excellent book. This is not an general guide to how to design with OSGI.  There is some background on the frameworks, and some explanation of technologies, and also pointers to sources for more information, but this book is all about learning by building. If you pick up the book hoping to learn any details about the how and why of OSGI and Spring, two useful technologies, you might be disappointed.  But if you like learning by doing, and working with running code, you'll enjoy and value this book.

Are You Agile Enough?

I recently read an article in SD Times about how organizations tailor agile processes to fit into their environment, rather than feeling a need to be dogmatic. Adapting a process to work in your environment is very important to being successful. It's also  important, however to understand how the variations you are making help you move toward your goals.

Being agile is a means to an end; your goal is to develop better software more effectively, not to be able to wear a "We are Agile" badge. If you're considering adopting agile, you are probably doing do because your current approach isn't getting you where you need to be so it's worth giving the 'by the book' technique a shot before you try to adapt an agile method to your circumstances. This especially applies to when you consider omitting practices.  Like many approaches,  there is a synergy between the the core agile practices; any one can help you be better but the big wins come when you do them all.

 The essential parts of an agile process in my experience are:

• Feedback, with a goal of continuous improvement.
• Honest evaluation of why you're doing and why.
• A (periodically) stable codebase so that you  deliver functionality to customers quickly
• A set of goals that lead to customer value.
• A belief that the team doing the execution can find the best solution and that management needs to step back to encourage innovation.

Agile methods define some techniques for achieving these goals. Scrum has you doing sprint planning, daily scrums and sprint reviews. You need technical practices like unit testing and continuous integration to verify that the code does what your project plan expects it to do. There are other ways to achieve these goals, but if you are trying to change how you work, a template is helpful.

When I read something like this finding reported in the article:

Of agile’s core tenets—daily standup, iteration planning and unit testing—VerisonOne found in its fourth annual “State of Agile Development” survey that 69% of the 2,570 participants adhered to these three things.

or

Even a variation of Scrum exists. Known as “ScrumBut,” it was dubbed for shops that don’t fully comply with the methodology. It is for those who say, “We are doing Scrum, but…” While some people may view this as non-agile, others argue it’s simply a customization of an agile process to work better for that particular company and its structures.

I wonder what these teams are they doing instead of the core "standard" practices to achieve the goals of agile.  The argument about whether those practicing "Scrum-But" are "agile" misses the point of the label, which is to highlight the areas where teams can work to improve their productivity.

My former colleague Bruce Eckfeldt was quoted in the article as saying:

“I see the methodologies as a continuum, and at the end of the day it’s all agile with the same principles and practices,” ... “There’s nothing set in stone on how to do something. You’re always looking to improve."

If you're not improving as fast as you like, consider what steps in your process you are skipping.  It's really hard to grasp the concepts behind agile until you are disciplined and try to apply the practices. If you have another approach that gets you to the end goal, by all means do it, but at the core, agile methods don't define a lot of detail. The basics are mechanically easy. The hard part is getting comfortable with the transparency agile methods require.

"Agile" methods define a set of goals and techniques to attain these goals.  If you are trying to adopt agile, but don't want to adopt all of the practices of a method, it's fair to say that your organization isn't ready for a given practice, but be honest about it. Without the honesty and transparency, you're missing the key difference of agile methods.

As to the relationship between textbook and practical agile when adopting agile methods, Bruce is quoted as saying:

“things may be reasonably pure,” but then people start to see what works or not and begin “to look beyond the textbook versions of agile.”

The trick is not to customize too soon. Until you give the process a fair trial, there is a great tendency to fall back on comfortable ways.

Organizations look to agile methods because their current approaches don't allow them to deliver as quickly as they can. And in the end, organizations and people need to change, whether they are adopting agile, or any new process. 

Before adapting you need to understand the core values or agile before you risk throwing the benefits away. You need to understand why you want to make the adaptation and honestly express the reason. And most important of all, you need to be clear and consistent about your goals. For example, if you want to encourage unit testing, you need to be clear that unit testing is encouraged, and accept that initially slightly slower progress is possible and OK if people are learning to write tests. Nothing can kill a change endeavor more quickly than inconsistent messages.

Being dogmatic for the sake of method purity isn't useful. Following the process until you understand the benefits of the practices can be very useful as a way to facilitate change.  I'll end with a brief story.

When I first heard of XP, at an OOPSLA conference I came back to work and suggested that our definition of "done" include unit tests. The other person on my team grumbled each time I asked "did you test that?" The grumbling continued until a couple of days later when, being presented with a problem report from our QA team, he was able to diagnose the source of the problem in minutes with the help of unit tests. "Cool," I head him mumble as he found and fixed the problem.

Had we not given the discipline of testing a good shot because we believed that unit tests didn't help, we would not have understood why the practice mattered. If you think that a method has value, try it, learn, then adapt. The opposite, adapting before you try, is less useful.

The Checklist Manifesto as Agile Primer

I recently read Atul Gawande's book The Checklist Manifesto: How to Get Things Right and found a number of of useful lessons in the book for agile developers. Agile software development methods often have very few explicit processes, but these processes are essential, and require discipline to execute well. We're often tempted to skip steps, either because:

• We think that the step doesn't apply in a particular situation or
• We forgot

The former is reasonable when we really understand why the step isn't relevant, but we are often tempted to convince ourselves that a step is unnecessary before we've mastered the basic process. One common example is the Scrum Team that is tempted to skip one or more of the standard planning or review activities.  And forgetting is just something that happens when we're busy and moving quickly.

It's when we skip steps that processes break down. Think about the time's you've had a hard time tracking down a problem. How often was this problem in code that you wrote a unit test for?  While it's often tempting to say that a change is "too trivial" to break something, how likely would you make that same decision if you had to go through a process (either on paper or enforced by tools) that asks "did you write a unit test?" to which you had to explicitly say "no?"

As another example, consider the various meetings  that are part of your agile process such as your daily scrum.  Jean Tabaka's book Collaboration Explained: Facilitation Skills for Software Project Leaders,  discusses the value of posting and following agendas for the meetings that agile teams have.   In some sense these agendas are just checklists that we follow to set up a context that allows us to meet the goal of the meeting is an efficient manner. In my experience, Daily Scrums and XP stand-ups become less valuable when they stray from the agenda, because people lose focus, and start thinking of them as less valuable. And the posted agenda (checklist) empowers those who find a side conversation distracting to move the meeting along.

Discipline is essential to an effective agile software development process. But discipline, Gawande points out, is hard:

Discipline is hard—harder than trustworthiness and skill and perhaps even than selflessness. We are by nature flawed and inconstant creatures. We can’t even keep from snacking between meals. We are not built for discipline. We are built for novelty and excitement, not for careful attention to detail. Discipline is something we have to work at.

So, if checklists enforce discipline, do they do so at the expense of judgement and creativity? Gawande says no:

...the question of when to follow one’s judgment and when to follow protocol is central to doing the job well—or to doing anything else that is hard. You want people to make sure to get the stupid stuff right. Yet you also want to leave room for craft and judgment and the ability to respond to unexpected difficulties that arise along the way. The value of checklists for simple problems seems self-evident.

Using example from aviation, structural engineering, and medicine, Gawande demonstrates that well made checklists allow you to focus on the activities that require creativity, by providing a way to get the basics right.

The checklist gets the dumb stuff out of the way, the routines your brain shouldn’t have to occupy itself with

As much as I find checklists useful, if used the wrong way, they can do bad things to productivity and creativity.  As Gawande says:

Bad checklists are vague and imprecise. They are too long; they are hard to use; they are impractical. ... They treat the people using the tools as dumb and try to spell out every single step. They turn people’s brains off rather than turn them on. ...Good checklists, on the other hand, are precise. ... They do not try to spell out everything... Good checklists are, above all, practical.

Perhaps a surgeon, using examples from aviation, medicine, and structural engineering can teach agile developers something valuable. The main lesson that appealed to me as someone who values (lightweight) process is  process can enable you to be more effective and move quickly by liberating you from thinking about the well known issues, and allowing you to focus on the hard problems.

If your team is struggling with process and not getting enough done, think about whether there are some simple things you are forgetting, and write them down. And, most importantly, iterate on the checklists.

Note: This was also the first non-fiction book that I read on a Kindle  so I'm discovering how useful the Kindle is to capture notes as I read. I'll have more to say later about other lessons from the Checklist Manifesto about teamwork and collaboration.

Agile Portfolio Management

 I've heard people criticize agile methods as being too reactive and focusing too much on the little picture and ignoring larger goals. This is a misunderstanding of a basic idea of agile. Agile methods are't about thinking small.  Agile methods are about making small steps towards a goal, applying programming an management discipline along the way. (For more, have look at an elevator pitch for agile I wrote last year.)

The basic approach of all agile methods is to

• Define a goal
• Break the goal into incremental bits so that you can iterate towards the goal
• Periodically (at the end of each iteration) pause to evaluate both your progress towards the goal, and whether the goal makes sense. 

Teams often skip the second part of the this evaluation, but it's the constant evaluation of the long term goals that makes it possible to reconcile the concepts of being agile and planning.

If you have doubts about whether long-range planning in an agile environment is even possible, read Johanna Rothman's book Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, which I  recently received a review copy of.

A project portfolio is "an organization of projects, by date, and by value, that the organization commits to or is planning to commit to." This sounds like a scaled up version of a product backlog that you might use to organize your work in an agile project, but with a longer time scale. So it's certainly aligned with agility.

In this book, Johanna motivates the importance of the project portfolio to enabling agile development, and also demonstrates how the technical and project management techniques of agile teams make it easier to define and iterate on a project portfolio.

Johanna is an expert on merging the human side and technical sides of projects.  I learned quite a bit about managing people from Behind Closed Doors: Secrets of Great Management which she co-authored with Esther Derby. In  Manage It!: Your Guide to Modern, Pragmatic Project Management Johanna discussed how to manage projects. And one of the more challenging part of managing a project portfolio is overcoming the resistance some people have to defining a goal for a project, a portfolio for a product line, and mission for an organization. In Manage Your Project Portfolio she shows how how to address common obstacles to defining a project portfolio, evolving it, and using it as a tool  to allow everyone to understand where the organization is aiming.

And the benefits of a project portfolio don't just help with "fuzzy" concepts like vision, but can also help reduce and address items such as technical debt. In addition to an overview of concepts, and concrete guidance on how to address problems, the book interleaves stories that establish that this work is based on real-world experience, and help you to relate to the issues the book addresses.

I recommend this book to anyone who has a role in defining projects.