Estimates and Priorities: Which Comes First

When developing a release plan, product owners often want to factor cost (or estimates) into where items go into the backlog. For example, you might hear "Do that soon, unless it's really hard." If this happens once in a while it's a useful way to have a conversation about a feature. If this approach is the norm and nothing gets prioritized until the team estimates it. I'd argue that the default order of operations is that features should get prioritized before anyone spends any effort on estimation. My reasons for this are both philosophical and practical.

The philosophical reason is that the Product Owner should be the one to prioritize work. By asking for the estimate first, the product owner is deferring their authority to the engineering team. This creates a risk that the team may not end up working effectively.

The practical reasons for prioritizing before estimating are:

• Estimation takes time, and if you don't start with a prioritized list to estimate with, you spend a lot of time estimating items that may never be worked on. (And yes, you may need to re-estimate items when  they hit the top of the list, as estimates may change based on experience, staffing and architecture.)
• If you estimate as work appears, you lose some of the benefits of fixed, time-boxed sprints, and you increase the overhead cost of planning. 
• By allowing the team to estimate first, and pushing an item off the list because it is too expensive, you are missing an opportunity for a conversation about how best to meet the business need.

Often the first version of a story may seem large because it includes more functionality than needed. If the the team knows that there is a critical feature to implement in a sprint, but that there isn't time to complete it, there may be a simpler, less costly, version of the feature that meets most of the business needs. If the product owner simply let's a large estimate defer the item, then the conversation will never happen and the business needs may not be met, which would be bad for everyone. Likewise if the expensive feature is lower on the list, then you need not have the conversation until later.

This balancing act between estimates and priorities underscores a key principle of agile planning: User Stories are an invitation to a conversation.  By prioritizing first, you can understand where to focus energy on analysis and design. You also keep the agile team focused on delivering business value by placing priority first, and having the engineering team and the product owners communicating actively.

Review of Adapting Configuration Management for Agile Teams: Balancing Sustainability and Speed

Adapting Configuration Management for Agile Teams: Balancing Sustainability and Speed is a book that bridges the gap between tradition SCM and Release Engineering and Agile teams. Mario asked me to be a reviewer of the draft manuscript and I knew that Mario had great experience in establishing SCM processes in larger organizations, and that he was also a strong advocate of Agile methods. I was pleased to discover a book that, while being able to help those from a traditional release management background adapt their processes to support agile, also addresses the needs of those transitioning to agile who can benefit from using appropriate SCM processes to help them.

This book will help you to understand how SCM can be an enabler for agile, and will also help you to understand how to fulfill the SCM definition of Identification, Control, Status Accounting, and Audit and Review while still being agile.

This is what I said about the book on my books page:

This book is a good guide to both CM and Agile principles, and it demonstrates how to use software configuration management to enable your team to be more agile. This book can guilde you to understanding how to manage releases in an agile environment, and how to apply basic CM concepts like build and branching successfully. While not a replacement for a book on your agile method, this book is a primer on agile for those with a traditional release management background, and and a primer on CM for those who understand agile. After reading it you will have enough background to be productive, and a good sense of what you need to learn more about. In addition, this book covers topics such as how to leverage cloud service providers for infrastructure, how to leverage SCM to make off-shore development less painful, and how to evolve your SCM process in an agile (incremental) fashion. With a good structure that allows you to navigate the book quickly, and a good use of metaphor to describe concepts, this book will help a release managers, project managers, developers and architects use the SCM process to get the most out of their agile teams.

If you are transitioning to agile and want to know how your release management team can help, rather than hinder you, give this book a look.

Sprint Boundaries and Working Weekends

A core principle of agile methods is sustainable pace. While the precise definition of this is debatable, the basic idea is that you want your work load to allow for a life outside of work, which in turn means not planning on overtime. The reality for many projects is that there will be times that the team needs to work outside of "normal" hours to meet a goal, and this is consistent with the idea of a sustainable pace if it happens occasionally, and the team decides that to meet a goal the added hours are needed.

Another core principle of agile methods is transparency. In order to improve, you need to be honest about acknowledging changes in plans, mistakes which cause more work, and misunderstandings that cause you to get things done significantly ahead of schedule.

A decision that teams make independently of the number of hours that they need to plan for is what the boundaries of a sprint should be, particularly with a 1 week sprint. This question is especially relevant if you think that weekend work will be needed. If you are doing 1 week sprints, there are basically 3 choices:

• Have sprints be Monday thru Friday, with a review first thing Monday.
• Have sprints be Friday thru Thursday.
• Have sprints be Tuesday thru Monday

The first option sound good, since you're not working on weekends -- unless you are. In which case one might argue that the weekend is really part of Friday. I don't like that idea because not only does it  because it ignores that the sprint boundaries mean something. If your team is meeting its 5-day goals only by working the weekend, then your are hiding that fact by considering the weekend as a "buffer." I'm not saying that one should never work weekends;  I'm suggesting that you should evaluate the work done in the context of the weekend work.

The second option is better; the weekend is part of the sprint boundary, and if the team agrees to work weekends you can measure that. Early in the sprint you may be more optimistic and not expect that to work, and early if you front load the riskier work, then the you might find it more useful to plan to do this work during the week when you can count on everyone being around (assuming that weekend work is done at people's own schedule).

The third option means that the weekend at the end, so you'll know if there is a need for extra work. Typically the work at the end of a sprint involves more defined tasks; you may be wrestling with getting something to work, but you probably had the design discussions already and you've tacked all of the tricky issues at the start. At this point going into a corner on the weekend to work is less likely to adversely affect someone else's work.

While  each team should define what "sustainable pace" means, and how best to meet their goals, to get the full value of an agile process, keep any days you expect to work inside the boundaries of the sprint. And when you do your release retrospective, be sure to discuss how well the  team is managing its workload.

Estimation: Precision and Accuracy (and Economics)

I was listening to a episode of the NPR show Planet Money on the accuracy of economic forecasts and how economic forecasts often have precision, but are often inaccurate. Consider this exchange between two of the show's hosts:

Alex Blumberg: On average, the leading forecasters, like Prakken's Macroeconomic Advisors, have around a one percentage point margin of error, which might sound pretty good. But let's say you forecast a two percent growth rate for the year. That means you're saying the actual rate will be somewhere between one percent and three percent.

Adam Davisdon: Now those are two totally different economies. One percent, that doesn't even keep up with population growth. That's a bad year. Unemployment will go up. It'll feel bad in the U.S. Three percent is the opposite, a pretty good year.

In one of the interviews, Simon Johnson, (Economist, MIT, Peterson Institute for International Economics) said this about economic estimates:

I would call it a necessary evil. There's so much imprecision. There's so much, you know, lagging in terms of our updating, that in some sense, we'd be better off without forecasts. We'd better off, you know, making up our minds afresh every day. But the problem is the businesses, the institutions, all involve thinking about the future and planning for the future. And you can't do that without taking a view of the future.

Which sounds quite analogous to estimates on software projects. One of the reasons that Planning Poker estimation uses a non-linear scale (for example, a Fibonacci sequence)) is that, in general, as tasks get larger, you are less accurate. While it might be meaningful to talk about a 1 hour task as opposed to a 2 hour task,  it's less likely to be useful to estimate 6 hours as opposed to 5 hours for a task. (A Planning Poker desk might have 1,2,3,5,8, 13, etc, hours as possible values.)

Planning Poker is intuitive, and often matches experience, yet many people insist on estimating a task that takes more than 3 hours as 4 rather than 5. This is probably because everyone forgets that estimates are just that: "estimates." Estimation gets better as a team works together, but the estimates are only one part of the planning process. If the goal of the team is to make a commitment to deliver functionality, then the non-linear jump in estimates is a measure of risk. The best way to reduce risk is to work in smaller pieces. When faced with a task that is bigger than an 8, and 13 seems too big, see if you can decompose the task in to smaller tasks, thus improving your ability to estimate accurately.

Estimation is difficult, not just for software teams, but we need something to help us plan, so just be mindful to give estimates the appropriate precision. And reestimate work remaining as you go.

Continuous Integration of Python Code with Unit Tests and Maven

My main development language is Java, but I also some work in Python for deployment and related tools. Being a big fan of unit testing I write unit tests in Python using PyUnit. Being a big fan of Maven and Continuous Integration, I really want the  Python unit tests to run as part of the build. I wanted to have a solution that met the following criteria:

• Used commonly available plugins
• Keep the maven structure of test and src files in the appropriate directories.
• Have the tests run in the test phase and fail the build when the tests fail.

The simplest approach I came up with to do this was to use the Exec Maven Plugin by adding the following configuration to your (python) project's POM.

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>exec-maven-plugin</artifactId>
  <executions>
   <execution>
    <configuration>
     <executable>python</executable>
     <workingDirectory>src/test/python</workingDirectory>
     <arguments>
      <argument>unitTests.py</argument>
     </arguments>    
     <environmentVariables>
       <PYTHONPATH>../../main/python:$PYTHONPATH</PYTHONPATH>
     </environmentVariables>
    </configuration>
    <id>python-test</id>
    <phase>test</phase>
    <goals>
     <goal>exec</goal>
    </goals>
   </execution>
  </executions>
 </plugin>

This works well enough. Setting the PYTHONPATH environment variable allows your pyUnit tests to find the modules you are building in the project. What's less than ideal is that, unlike other maven plugins, the person running the build needs to have python installed and configured correctly. (You can allow for some variations between environments. And if you have a developer on your project who doesn't use python, and doesn't want to there is a property you can set on the exec plugin to skip the tests, so in the end only those who use python, and the continuous integration server, need the correct things installed.

This may be obvious to some, if not many, but in case anyone is looking for an answer to how to run unit tests as part of of your maven build, I hope that this is helpful.

Versions in Maven and Source

I'm a big fan of Maven, a build (and project) management tool. When you are working with Maven, each artifact  that you develop (jar, or war file for example) has a version that's distinct from version in your SCM system.  The Maven Book has a good discussion about how versions are managed, but some there are often questions on projects about how to use Maven versions when you also have the SCM version. "SCM Version 6453" gives more information that "Artifact version 1.1" for example. yet we have 2 version numbers to manage. Here's one approach that works well for simple projects.

I'm assuming that you know the basics of how to specify dependencies in Maven. If you need a quick intro to Maven, see the Maven Book.

If you are developing a project that has artifacts that external clients will incorporate with Maven, you need to change the artifact version with each release, as you  specify the artifact version in the dependency element in your pom. If you are just using maven to develop components that will never show up in an external repository, adding the artifact version dimension to the SCM version can appear to add little value at the expense of some overhead, unless you have a clear model of how the two interact.

Many teams using maven are using it to manage external dependencies, but their own artifacts are not published to external maven repositories. A simple webapp, for example may have one war file, and have some external dependencies, and for support and validation purposes, you can use the build number or version number. There are a couple of ways to figure out the source version that you built your webapp from.

If you use a continuous integration tool like Bamboo, you can pass in the build number on the build command line by adding -DbuildNumber=${bamboo.buildNumber} to make the property buildNumber available to your project. You can make this available the application, so that you can see the build number on a login screen or about page. From the build number you can infer the SCM version. Or you can often ask the CI system the SCM version number. For exampe, in Bamboo, you can specify -DscmRevision=${custom.svn.revision.number} if you are using subversion as your SCM.

You can also use the build-number maven plugin to embed the version in your war manifest or make it available to your flex application, for example.

Regardless of how you get the revision number the revision number still gives you enough information to identify what code you are working with. The Maven convention to keep in mind is that SNAPSHOT artifacts are for active development, and non-SNAPSHOT versions are for released artifacts, so while you are working on a new projects, you'd be building a 1.0-SNAPSHOT artifact, and when you are ready to create a release branch, you would change it to 1.0.

So when should you change the project version in maven?  Here is a simple approach: One maven major/minor version per codeline. Incremental versions stay on the same codeline.

So if you are using an active development line pattern (see Software Configuration Management Patterns) this means that each Release Line has a maven version, and your Mainline is use to create the SNAPSHOT version of the next release artifact.

For example:

• Start mainline development with 1.0-SNAPSHOT version
• When you have a release:
• create the release branch and change the artifact to 1.0.
• Change the mainline version to 1.1-SNAPSHOT (or whatever the next active version is)

To do support of the 1.0 release you have 3 choices:

1. Just work with the 1.1 artifact version and use build numbers to distinguish between the current release and the support release.
2. Branch off of the 1.1 codeline, change the version to 1.1.1-SNAPSHOT, and merge back when the 1.1.1 release goes out.
3. Change the version on the support branch to 1.1.1-SNAPSHOT and don't branch. Change the version to 1.1.1 when you are done, and create a tag for the release.

I prefer the third option as the simplest, and best option, assuming that there is only one stream of development work for the maintenance branch. It balances the need for identification, with the desire to minimize overhead.


To summarize:

• New development for M.N-SNAPSHOT versions on trunk.Use SCM version & Build numbers to identify components internally.
  
• M.N versions on their own branches.
• Use M.N.n versions for maintenance releases; SNAPSHOTS for development, remove the SNAPSHOT when done. 

I'm interested in hearing what others may have done to reconcile SCM and Maven artifact versions for internal components.

Uncertainty and Agile Requirements

The key value of Agile methods is to help you to manage uncertainty. By being incremental and iterative, you manage risks by not investing a lot of effort in specifying things that may "wrong." At the start of each iteration you can look at what you have and decide that it's the right thing, in which case you can build on it, or the wrong thing, in which case you can try something else. Since you've only invested a small amount of effort relative to the specification you do in a waterfall process, you've wasted less effort and, in the end, money if you are wrong.

This approach of small stories with only some details works really well in many cases. An agile team runs into trouble when the project team confuses "uncertainty" with "vagueness." To be successful, an agile team needs to work off of a backlog that has stories that are precise enough that the team can iterate effectively with the stakeholders at the end of each iteration, and can develop with a high velocity. It's important to add precision even if you have uncertainties. While it's important to be as accurate as possible, don't use your lack of certainty about a requirement as an excuse to accept a lack of precision. When you have a good target to aim for (and you hit it) you can iterate quickly and judge if you are hitting the right targets.

How do you tell that you have enough precision?  This varies from team to team. For a team that has been together for a time and has a clear shared vision, a very brief statement of goals might well be enough. For a project where the vision is less clear,  a longer conversation may be necessary. Three concrete tests are:

• Can the team estimate a story? (See Agile Estimating and Planning for more about estimating.) If the answer is "there is not enough information to estimate" then the story is too vague, and the team and the product owner need to meet to make sure that they understand the options. If the team estimates a story that the Product Owner thought was simple at 3 weeks, you have a raised a flag that you need more conversation to understand what the PO really wants.
  
• Can you provide three options for how to implement the the story, or 3 variation of what the user experience will be? If you find yourself developing many more that seem plausable, the story is too vague, and if you can only develop one or two, then the there is not enough information for you to think through the story. 
• Can you test the story? If you can't come up with a a reasonable high-level test plan, then  the story is too vague. (Mike Cohn has written an excellent article about the value of planning with the larger team.)

While being able to do all 3 for a story is nice, being able to feel like you can estimate with confidence is the one thing that you should do to feel confident that the stories are well developed.  If you can't estimate based on what you know about the story, the good news is that  the very act of trying to come up with an estimate, options, or a test plan will help you refine the story.

One might say that this is too much planning for an agile process, and that this level of detail sounds kind of like a "waterfall."  And at a high level it seems related to the Cone Of Uncertainty, which is a model for waterfall development. The difference is that we still don't need or want to have fully defined specifications at the start of the project; as we approach a development iteration, we want enough detail to have a development target that a stakeholder can evaluate.

At the end of an iteration when something isn't quite right, you want your stakeholders to say "that's not what I want" rather than argue over what they meant. The latter will still happen, and it's OK when it does. By being precise about what you think you want to build, you will identify the high risk areas of a project early on, so that you can take full advantage of the risk management benefits of agile.