Are You Done Yet?

Johanna Rothman recently wrote, commenting on Joshua Kerievsky's proposed definition of done. Both posts are worth a read, if for no other reason than to better understand why we have such a difficult time defining what "done" is, and why defining "done" is one of the major challenges for teams  trying to adopt agile practices.

Thinking about both Joshua's and Johanna's points I wonder if the difference isn't similar to a discussion of whether principles or practices are more important to be successful when adopting agile methods. On the one hand following  practices diligently allows you to develop good habits and even to get some good results early on. The challenge comes when it's time to reflect and improve on your practices. Without a good understanding of practices it's hard to optimize.

Similarly, defining done earlier in the process can cause problems if you are thinking about the meaning of "done" the wrong way.  If "done" means washing your hands of issues ("we met the spec..."),  evaluating done as late as possible makes sense, enforcing the idea that you are not done until the customer is happy is a useful driver.

If, on the other hand, you understand (and believe) that your goal as a developer is to deliver useful, quality software,  and if the customer understands that that they may not have understood the problem until they had a working system in hand, defining done for earlier steps means that you have more tools with which to evaluate your effectiveness, status, and progress.   Done closer to the developer means that you have more, rather than fewer, chances to evaluate, learn, and improve.  By embracing the principle that delivering a useful end product is the goal, you can benefit from having some local completion criteria.



Having the definition of done closer to the team (as Johanna recommends) allows you to measure progress and identify risk. You also need to be able to acknowledge that it is possible that completing all the stories may still mean that there is still work to do. Then you have to inspect, adjust, and adapt. Which is to say: be agile.

A Review of Drive by Dan Pink

This is one of those books that describes something extremely obvious and intuitive that at the same time goes against what you were taught was "common sense." This would be a good book just for the survey of the (long) history of the study of the theory of motivation. It also concludes with a number of things you can do to create an environment that encourages mastery (as opposed to simply meeting goals) in your work and school. 


If you're an agile software developer you'll have a few aha! moments when you understand how agile practices really encourage flow and create environments where teams and individuals can be highly productive. If you're a manager, this book will encourage you to think about how teams work and how some common practices are counter-productive. 


If you're trying to understand why self organizing teams work, but with a perspective outside of software development, this is a quick read that will get you thinking and learning. 


Some other books on related topics:

• Behind Closed Doors: Secrets of Great Management (Pragmatic Programmers)
• Pragmatic Thinking and Learning: Refactor Your Wetware (Pragmatic Programmers)
• Flow: The Psychology of Optimal Experience (P.S.)

Motivation Visibility, and Unit Testing

I've always been interested in organizational patterns (such as those in Organizational Patterns of Agile Software Development). I've recently found myself thinking a lot about motivation. I'm now reading Drive: The Surprising Truth About What Motivates Us and just finished Rob Austin's book on performance measurement. Being the parent of a three year old, I'm finding more and more that "because I said so, and I'm right" isn't too effective at home. My interests in motivation are closely related to my interest in writing software effectively. Writing software is partially a technical problem about frameworks, coding, and the like, but the harder (and perhaps more interesting) problem is how to get a group of people working together towards a common goal. Agile practices, both technical and organizational, build a framework which makes having the right amount of collaboration and feedback possible. But there's a bootstrapping process: How do you get people to start doing the practices, especially technical ones, such as unit testing?

In an ideal world, everyone will know how to write unit tests,  understand their value, and want to write them. In an organization transitioning to agile having all three parts in place is not a given.  Most people understand why unit tests are useful, in principle. The problem is the execution. In  my experience, there are two reasons people who claim that that want to write unit tests give for not writing them:

• They are too hard. The overhead of the test can make the cost of developing a feature excessive. 
• They are too easy. Some functionality appears to be trivial, so why would we want to test it.

Both of these reasons can have merit at times. Testing getters and setters, and trivial calls to a system library and other simple coding constructs really don't add value. And writing a complicated test using a hard to use framework to verify something non business critical that could be quickly validated by visual inspection ("make the background color of the home screen orange") may well be not worth the effort. The problem is that most people have bad intuitions about where the lines line until they start practicing the skill of unit testing. Most teams need experience with testing to get a true feel of what is a trivial test, and what is a seemingly trivial test that can unmask a major problem

Rather than frame the testing challenge with the default being the old way of not testing:

Write a test when it makes sense.

Change your perspective to the default being to test:

Write a test unless you can explain why you did not.

The key to this approach is to make encourage people to think through their rationale for not testing. There are a few ways to do this, but one approach is to tie the explanation mechanism into something every developer works with every day: your source code repository. Have the team agree that, in addition to the source files, every commit will have a change to a test or a rationale for why you didn't write a test.  For example if I do a commit without a test I could write:

ISSUE-23: Fixed the spelling of the company name. NO TEST: it was a typo. 

or

ISSUE-26: Fixed the rendering mechanism. NO TEST: we don't have a good framework for testing this sort of thing

or even

ISSUE-28: Fixed a serious logic issue. NO TEST: I didn't feel like writing one.

By developing a team agreement to add tests or explain why not, you are starting with a small change of behavior that paves the way for a greater change based on understanding. Even if a message like the last one is acceptable, many will be uncomfortable admitting to laziness, and think harder for a reason. By reviewing the commit messages later on you can get a sense of impediments to testing (technology, organizational,  or attitude), and use that data in a retrospective to decide how to improve.

By being creative you can help people on your team understand the value of process changes, and start a conversation about how to evolve practices to suit your team.

Measuring Performance for Teams

If you've ever been in an organization that had performance reviews, you may have found yourself wondering whether the measurement process used to evaluate your (and your team's) performance made sense, and if there was a better way. Even if you have goals that can be evaluated quantitatively, rather than some metric that feels arbitrary, you may feel that your personal goals may run counter to the success of the your team. For example, you may wonder whether it makes sense to help your colleague on her high priority project and risk missing a deadline for your (lower) priority one. Sometimes the problems with measurement systems are because people just don't measure well. In other cases, it's because it's impossible to measure all of the things that matter.

Rob Austin's book Measuring and Managing Performance in Organizations gives you a model to understand why measurement systems become dysfunctional, and an approach to avoid dysfunction when you are measuring.

Austin addresses some core issues that agile (and other) teams face:

• How to evaluate and reward individual knowledge workers doing complicated things in teams
• How to motivate the individuals on the team to do what helps the team meet their goals, and
• What performance measurements are helpful, and which just add noise.

Austin explains why individual measures and goals may be counter-productive unless you can measure all of the aspects of a person's work, why most people want the team to succeed, and how that desire for team success is a key motivator, and why a staple of agile project management: visible and frequent feedback on team progress can improve your chances of success. While this book was written before agile became a popular term, anyone familiar with agile methods will find the ideas in this book in tune with agile methods.

Much of what I learned from reading this book seemed obvious in retrospect, but Austin explains the problem with clarity and precision, making observations that only seemed obvious once I read them. Early in the book, for example he points out:

Employees true output (such as value to the  organization) is intangible and difficult to measure; in its place organizations choose to measure inputs (such as the amount of effort devoted to a task...)

Which seems to be such an obvious problem with many evaluation systems that you wonder why so many organizations still do it.

Reading this book won't give you a cookbook for designing a motivation and performance evaluation system.  This is a difficult problem, especially for those working in an industry where there is a strong desire to quantify and measure. But this book will help you to understand the problem and enable to evaluate and improve your current practices.

While Austin's book will help you understand the model behind effective performance measurement, there are also more day-to-day practices you need to help your team be successful. For these  consider reading Johanna Rothman and Esther Derby's book Behind Closed Doors: Secrets of Great Management, which is an excellent guide to the day to day process of people management.

Managing people can be intuitive, but it is also more difficult than many people realize. A desire to measure is useful, but it can be counter productive when you measure without understanding. Software development is a collaborative, human activity, and as such we need to understand that management and measurement and difficult, and doing either without without an understanding of the challenges can lead to unexpected results.

Things about Release Management Every Programmer Should Know

As I mentioned earlier I was privileged to contribute to the book 97 Things Every Programmer Should Know: Collective Wisdom from the Experts. In addition to the contributions about coding and design, I was pleasantly surprised to see the number of items that relate to release management. While I've long been interested in how to build architectures and processes that make deploying and releasing software easy, I sometimes get the impression that these items were often though of necessary evils that could be done at the end, often by the someone who isn't doing "more valuable work." Much like awareness of agile software development made it obvious that testing and quality assurance activities work best when they are integrated throughout the development lifecycle, agile has also made it more obvious why build and release engineering is something to work on as you go. This makes a lot of sense, as ease of release is closely tied to the physical architecture of the system, and your build process defines the physical architecture.

Some of the posts of interest are the following, though I could have added others than related to testing as well.

1. Deploy Early and Often by Steve Berczuk
2. Install Me by Marcus Baker 
3. Keep the Build Clean by Johannes Brodwall
4. One Binary by Steve Freeman
5. Own (and Refactor) the Build by Steve Berczuk
6. Put Everything Under Version Control by Diomidis Spinellis
7. Step Back and Automate, Automate, Automate by Cay Horstmann

Aside from being excited that programming experts understand the importance of build, release, and installation mechanisms, I was also excited to see that one topic that people associate with SCM wasn't mentioned: Branching.

Branching is a useful tool when used in the right context, but more often than not, branching is used as a way to avoid issue rather than to address them. Rather than branching because there is a true divergence in the code, we branch to avoid breaking existing code. The problem is that doing so simply defers a cost. Sometimes deferring the cost makes sense. Often it's better to invest in the techniques you need to enable incremental change while keeping the codeline working.

97 Things Every Programmer Should Know is about much more than just coding, or just release management. And that's the point: programming is a multi-faceted skill and to write good code, you need to know about more than just writing code.

Planning is a Gerund

One of the things teams adopting agile struggle with is deciding how much to define a plan before you start executing. Have a plan that's too well developed and you end up risking that your team may not be responsive enough to change. Too little of a plan and you may end up changing course excessively, and have no way to measure your progress towards any sort of deliverable. 

At the core of this confusion over how much to plan is the reality that plans change, and spending too much time and energy creating a plan that ends up being wrong seems wasteful. But the lack of a plan means that you have nothing to measure progress against. One way to reconcile this is to keep in mind the following quote, attributed to Dwight Eisenhower (and a variant attributed to Churchill):

Plans are nothing; Planning is everything.

If we assume that as a project progresses, that things will change, we can still benefit from talking through the options as a team. Capturing the things that we don't know, but would like too, is useful information, and gives the team a good measure of risk.

The time you spend planning is an important consideration. Constrain the amount of planning time based on the duration of your sprint. If you can't come to an understanding of what the problem is or how to approach it, you have a clue that you're trying to do too much. But rather than throw up your hands, you can aim to have some sort of plan. It might be wrong, but even building the wrong thing can increase your understanding.

For the planning activity to be useful it is important that it not be top-down but that it involve the implementation team, as they are the ones who can speak to the the implementation risks, and can propose creative solutions given the ability to probe about real goals.

One thing that may concern people with the approach of involving the team and stakeholders at the same time is that a planning which raises more questions that providing answers can make some people uncomfortable. Senior managers may be uncomfortable with acknowledging ignorance. Team members may be put off by seeing that there are legitimate disagreements among the product ownership team about some issues. And some people are just uncomfortable when you can't just tell them what to do.

This is a cultural issue that may not be easy to overcome, but agile projects work well because the team can pull together to solve problems when given all of the information, and structure their code and their work to  mitigate risk. And if the uncertainty exists it's better to identify it up front.

Regardless of the level of uncertainty about goals and dependencies it is important to exit a planning session with a common vision for the goals and target for when you will re-evaluate the goals. A well run  planning activity can helps to focus the team towards a common goal.

Book Review: Modular Java

I recently read  Craig Walls' book Modular Java: Creating Flexible Applications with Osgi and Spring (Pragmatic Programmers). This book is a very detailed tutorial that walks you through setting up an application using OSGI and Spring with the help of Maven as a build tool. If you aren't familiar with any of these technologies, this book will get you started, and quickly have you feeling like you have a basic grasp of the concepts and technologies.

You'll finish the book with a desire to learn more about the technologies, and understand the power of modular applications.  As a tutorial, this is an excellent book. This is not an general guide to how to design with OSGI.  There is some background on the frameworks, and some explanation of technologies, and also pointers to sources for more information, but this book is all about learning by building. If you pick up the book hoping to learn any details about the how and why of OSGI and Spring, two useful technologies, you might be disappointed.  But if you like learning by doing, and working with running code, you'll enjoy and value this book.