Definitions

The other day a colleague use the phrase open interval to refer to time periods that included their endpoints. This didn't sound like the definition that I'd learned in analysis class, which I will admit was not my favorite subject in grad school, so I searched for a definition and found:

An open interval is an interval that does not include its end points.

This led to a brief, but interesting, discussion about why my mathematically-minded colleague flipped the meanings. It came down to the feeling that the definition didn't make sense in terms of common english usage (when you open a door, or a group, for example, you're being inclusive).

Definitions add precision, but when you use words that have meanings in multiple contexts, you can find yourself in confusing conversations. Consider a term like normal, which can be defined as:

a lack of significant deviation from the average

This sounds pretty benign, but people often carry around more that just definitions when they hear words. Using "normal," for example, in conversation can get tricky if the group you are in has not established a baseline context;  even though there is no value judgement inherent in the word, people might assign values to being more than a certain distance from the average. And then rather than than discussing the problem at hand you get distracted by a meta conversation.

The same is true when you're adopting new engineering practices. Consider a word like refactoring, which has a precise meaning and that people often use to mean "change." When adopting agile methods, you might have a discussion about how many of the practices that you need to follow to be able to say that you are using a method. Can you do Extreme Programming as a solo programmer when Pair Programming is a practice, for example?

The discussion is very interesting and useful, as it leads to an understanding about what is useful about XP practices. But from a definitional sense, I would err on the side of saying "No, XP is defined as the practices, but you can be doing something XP-inspired."  The question you want to ask is "why does it matter if I'm doing XP (or Scrum, or another process)?" Maybe it doesn't. Unless you want to learn the value (and limits) of a practice.

The discussions get more interesting when it comes to concepts like "agile" which are very overloaded in different contexts. You can be more agile from a business perspective, yet not follow a defined agile software development method, for example. But if you want to say that your team is agile, be sure that you understand what your definition is.

Doing an "inspired by" process can be good if you are getting the results you want. When you are learning a process, it's important not to customize the practices until you understand why they are in place. And when communicating with others, being a bit fussy on the definitions (or qualifying your description) can be helpful.

Definitions help us to communicate. It's a useful default to stick with the definition in the domain. Then apply value judgements as needed. It seems better to say that you aren't agile, yet getting better results than you were, than to morph the definition of agile so that it fits your practices. Making "Agile" a checklist item itself is counter to the principles of agile. Define what you do in terms of your practices and evaluate effectiveness in terms of your results.

Words and definitions help with communication. And it's hard to come up with an unambiguous word. But try hard to use words as they are defined in your context. That will help you to communicate, set expectations, and maybe even be more agile.

Refactoring

Frank Buschmann wrote a 2-part article in the July/August and September/October issues of IEEE Software that covered an important aspect of software development: refactoring.

Yes,  you'll hear the word "refactoring" spoken often in any development team, but most developers rarely discuss what refactoring is, and whether they are refactoring or doing something else, like "reengineering" or even re-writing.

I won't repeat what Frank said, as he said it quite well,  but as I read the article I realized that there are some points that get lost in many teams and are worth emphasizing:

• Refactoring does not mean "any kind of changing." It means changing the structure of existing code to preserve functionality. In principle you are making the code "better."
• To refactor you need a way to validate that you have preserved functionality. Automated tests are probably the best way to do this. I suggest that you can't refactor without having a test plan in place. And from a practical perspective, this means automated tests.
• Refactoring is a way to improve code quality, but like other things a development teams spends time on,  you you should refactor when it provides business value. 

Being precise about when a change is refactoring and when it's reengineering, a rewrite, or simply changing code to add a feature helps set expectations with stakeholders and helps you as a developer better understand when you'll be done. And by thinking in terms of refactoring continuously you can deliver code more quickly, and avoid the need for reengineering or re-writing, and better be able to evaluate functionality with incremental delivery.

Refactoring is good. Not all good coding is refactoring.

Agile or Not: How to Get Things Done

Agile software development always felt intuitive to me. Developing software incrementally, in close collaboration with the customer is the obvious way to deal with the uncertainty inherent in both software requirements and implementation. The technical practices of automating necessary but time consuming tests, and deploying, early and often are the obvious ways to give an team the ability to evaluate the  functionality you have and to to decide if the software works as expected. And it's also important to decide if what you built still makes sense given the current environment. Agile isn't the only way that people build software, and it may not be a perfect approach, but it's one of the best ways of dealing with a system that has unknowns.

Agile software development acknowledges uncertainty. The ability of agile methods to make it very obvious very quickly when a project, or even a process, is failing makes people uncomfortable. The visibility of the failure leads to a desire to return to more "traditional" processes. For example, a common complaint is that agile organizations don't create "good enough" specifications. Someone might blame an unforeseen problem on the lack of a spec that mentioned it. This is possible, but it's only true if the person writing the spec could have foreseen the problem. 

The desire to point back to a lack of specification also points to a lack of buy-in to a fundamental premise of agile: it's a collaboration between the business and the engineering team. Some other possible causes of the problem could be:

• The development team didn't test thoroughly enough. 
• The code was not agile enough, so the bad assumptions were embedded into the code so deeply they were difficult to address.
• Communication was bad.

More complete specifications that address all of the issues a system might encounter are one way to build software. But the best that people can do is to write specifications that address what they know. Quite often no one has complete advance knowledge of what a system needs to do.

There are projects where things are known with enough certainty that a waterfall process can work. People can write good specs, the team can implement to those specs, and the end result is as exactly what everyone wanted.  I've worked on projects where this was true, and in these cases, the specifications were reviewed and tested as much as code might. 

Even  projects that seem suited to waterfall fail. For any method to be successful:

•  Everyone involved needs to be committed to the approach and 
• There needs to be a feedback loop to correct errors in time. 

The second is point is more important than the first, since people will make mistakes. But being committed to the process is what lets people accept and offer constructive feedback. The reason that waterfall projects have such a bad reputation is that many are implemented in a way that results in problems surfacing late.

Agile methods when done well have the advantage of having built in feedback loops, so  that customers and teams have ways of identifying problems early. When agile projects fail it's often because people ignore the feedback and let things degrade for longer than necessary. (Failing fast can be considered success!)

So, Agile or not, your process will only work for you if everyone works within the framework to the best of their abilities, and if you have mechanisms in place to help people do the right things. Otherwise you can blame your process, but odds are, that's not where (most of) the fault lies.

More on Being Done

Continuing the conversation from last week, Andy Singleton followed up on my post on being done with this post. Which is good as this is one of those questions that sounds simple in theory, but in practice contains some subtlety.

While I was advocating a good definition of "Done" to enable you to measure progress along a path, Andy's point seems to be that many teams don't establish enough of a path. He says:

In my opinion, most agile teams aren't doing "Test Driven Development", and they aren't doing scrum iterations where they plan everything in advance. Instead, they are doing "Release Driven Development." They focus on assembling releases, and they do a lot of planning inside the release cycle.

This is probably true in more cases than not, though one could argue whether if you are not doing iterations or TDD whether you are, in fact, doing agile. But even if can concede (which I'm not sure if I am) that you can do agile without planning and acceptance criteria of some sort, what Andy describes above still sounds like it has an element of plan, execute, adjust, though perhaps in a more chaotic way than a "textbook" scrum process, and and perhaps at a smaller scale. So knowing having a clean definition of what you want to do is still important. In the Indivisible Task I discussed how teams get stuck with planning because it's difficult to think of tasks that are small, discrete, and which produce useful work. It is possible to do so, but it is hard.

Perhaps the disagreement here is more of a misunderstanding. I consider being able to measure progress in terms of completed work items an essential part of gathering the data you need to improve your process and understand how to be more agile. While doing it at a macro (sprint) level is good,  doing it on a day-to-day or hour-to-hour basis is essential. If you do this at a fine-grained enough level, and have a good testing infrastructure, you can release software more frequently. So, at the limit, perhaps Andy and I are talking about the same thing.

Since I only summarized, I encourage you to read what Andy has to say for yourself. And I look forward to hearing comments both from Andy and other readers.

Being Done

Agile New England (which used to be called the New England Agile Bazaar, and which was started by Ken Schwaber) , has this wonderful activity before the main event each month: they host Agile 101 sessions, where people who know something about agile lead a short (30 minutes) small (about 10 people) class on agile basics for those who want to learn more about some aspect of agile. From time to time I lead a session on Agile Execution, where the goal is to help people understand how to address the following questions:

• How can software and other project elements be designed and delivered incrementally? What set of management and technical practices would enable this?
• How do you know whether your Agile project will complete on schedule?

When I lead the sessions, I tend to focus on tracking, defining stories in terms of vertical slices and the importance of continuous integration and testing to making your estimates trackable. Since the classes are so small and since the attendees have diverse experiences, the classes are sometimes more of a conversation than a lecture, and I find that I learn a lot, and sometimes find myself rethinking what I know (or at  least exploring things that I thought I understood well enough).

During the October 2011 meeting I found myself reconsidering the value of defining "done" when writing User Stories. I always have thought that defining done is essential to tracking progress. But what done means is still a difficult question. Andy Singleton of Assembla suggested that

The only useful definition of done is that you approved it to release, in whatever form

While the goal of agile methods is releasing software, I find that this definition, while appealing in its simplicity, misses some things:

• Agile methods have a goal of continuously shippable code. Of course, "shippable" might not mean "ready to release" and cane mean something closer to "runnable," but you can get there by doing no work since the end of the last release. That isn't the goal of agile.
• With that large scale definition of "done" you have no way of tracking progress within a sprint. 
• Without an agreement on what you're going to, it's hard to know when you are changing direction. And acknowledging change is an important part of being agile.

The last point about acknowledging change isn't just about "blame" for things not working out. It's about evaluating how well you understand both the business and technical aspects of your project, and it forms the basic for improvement. 

True, having incremental definitions of done that you can use to track progress does help manage budgets. But that really is the least important aspect of having a good definition of done. Even if I were on a project with an infinite time and money budget, I'd want to have a sense of what our goals are. 

Having an agreement among all of the stakeholders on what being "done" means lets me:

• Improve communication among team members and between team members and business stakeholders.
• Evaluate my understanding of the problem and help me identify what I can improve.
• Set expectations so that it's easier to develop trust between stakeholders and the engineering team that the team can, and will, deliver.

"Ready for Release" is a key component of "done" and and essential part of being agile. But it's not enough.

---

See Andy's response, and read more in Part 2 of this conversation.

Continuous Learning, Coaching, and Learning from Others

There was an article in the Boston Globe this week by Scott Kirshner: Staying Competitive in the Workplace that emphasized the importance of keeping your skills up to date.

It's a short article and worth a read. Some of the activities Kirshner suggests are similar to those Atul Gawande makes in the appendix of his book Better: A Surgeon's Notes on Performance.

Related to this theme is a New Yorker article by Gawande, Personal Best,  on the advantages of challenges of engaging someone to coach you in your profession. I continue to be amazed at how much I'm learning from Gawande, a surgeon, about how to be a better software engineer. I suspect that I first realized this when I started learning about Patterns. (The short post, The Pattern Technology of Christopher Alexander by Michael Mehaffy and Nikos Salingaros discusses how an architect influences the software development community.)

Maybe the common theme of all these writings is that it's important to be ready to learn, and you can learn from the people you least expect to.

SSQ Article on SCM and Tools

I recently was interviewed for an article  on SCM and Tools that Crystal Bedell wrote for Search Software Quality. Updating tools and processes key to overcoming SCM challenges is brief, and makes some good points about the relative value of tools compared to understanding what you are trying to accomplish with your process.


The best SCM tool, from a day-to-day perspective is the one that is the most invisible to developers, and the best tool really can't help you much if you have a process that just makes it hard to collaborate. This article was also validation that trying to be too agnostic when Brad Appleton and I wrote the SCM Patterns book was a good idea. While some tools make some practices easier, a tool can't replace having an understanding of the reasons to use (or more important, not use) techniques.

Read the article, and if you want to learn some basic SCM practices read the book. For those who want a very comprehensive discussion of branching (which seems to be, for better or worse, the topic that people most struggle with when using SCM tools),  Streamed Lines is a good place to star.