Top 10 Reasons for Slow Velocity

The SVPG blog has a good post about the Top 10 Reasons for Slow Velocity.  It's interesting that their top two items have to do with the leadership and management of the team.  Both ideas come back to focus and motivation, which turn up again and again in conversations about productivity.

Code as an English Garden

I heard a quote recently I really like: "Computer code is more like an English garden than a stone sculpture. It must be tended to and pampered, or it will be overcome with weeds and die." This goes back to an earlier quote, "Creating an agile team begins with an agile codebase."  An agile codebase, by necessity, must be tended to.  Cruft must be re-factored, tight couplings must be loosened, and complexity must be encapsulated.  It's a constant, ongoing, and never-ending endeavor.

78 Things I Have Learned in 6 Years of Agile Coaching

This article, 78 Things I Have Learned in 6 Years of Agile Coaching, provoked some further thoughts regarding my earlier post, "Scrum: What We Were Missing."

A few things I'll add to that list:

• Releasing code at the end of each sprint.  We were following the spirit of "releasable" code, but that's too easy to wiggle out of -- lots of code is releasable, but that doesn't mean it won't take two weeks to deploy it.  We keep getting ourselves in a position where we're "done," and yet we still have to figure out how to get 140 files to production without affecting the customer experience.  That's hard.  So our new goal is to actually release code at the end of each sprint.  Even if the project isn't finished, we can deploy pieces of it to production sprint by sprint.  That way, we won't be overwhelmed by a massive code push once the time comes to get it live.
• Small, cross-functional teams.  We're trying to re-capture the spirit of a start-up, where everyone's in it together, a small group of energized people are working towards a common goal, and communication is as simple as yelling a question out to the room.  As we grow, our challenges to achieving this are figuring out how to arrange seating so that teams can sit next to each other and maintaining that feeling of a small group while the company inevitably becomes a bit more stodgy as it grows.
• Public burndown charts, updated daily.  We're still trying to reach this goal, but it's becoming clear to me that the visibility these provide to both the company and the group, and the group-wide responsibility these engender on the teams, are a key part of agile.

Cross-Functional Teams

We're trying out a new idea that I'm excited about: cross-functional teams.  Our group has expanded slowly and organically over the years.  As we've needed more programmers, we've hired them.  As we've needed more testers, we've hired them.  But now we're almost 20 people strong, and we're hitting the limits of a productive team size.

I'd been unconsciously steering us towards separation by task: the product managers pass their ideas to programmers, who write the code and pass it off to testers, who test it.  But that becomes harder and harder to coordinate as the number of projects and people grow -- without domain separation, everyone needs to know everything about every project.  And that's both unscalable and unproductive.

So we're trying out the idea of cross-functional teams: three groups comprised of a few coders, a tester, a product manager, "half" of a project manager, and one or two designers.  I see this as having a few benefits:

• It will reduce task switching.  People will be able to concentrate on one of a few different projects assigned to their group.
• It will let everyone more effectively embrace the business goals expressed by the product managers.  Instead of getting sent in a different direction with each new project, people will hopefully be able to internalize a larger set of goals and apply them automatically to each project.
• It will make communication easier by limiting the number of people involved and letting people develop deeper relationships than they could with a larger group.

I'm really looking forward to the benefits this change should bring, and will keep everyone posted on how it goes.

Lean Software

I recently finished a short, insightful book called The Art of Lean Software Development.  It connects the principles of lean manufacturing with those of agile software development.  It's a fascinating path: lean manufacturing started with Toyota decades ago, and focused on obsessively ridding systems of "seven deadly wastes:"

• Defects
• Overproduction
• Transportation
• Waiting
• Inventory
• Motion
• (Over)Processing

(These can be abbreviated to "DOTWIMP" if you're ever quizzed on them.)  Most of them are self-explanatory if you consider the additional costs they could bring to the car manufacturing process.  "Motion" refers to the idea of people or machines moving around -- if you can find a way to avoid it, you're better off.  "(Over)Processing" means doing anything that's not required by the customer.

It's interesting to apply these to software, as the book does, and find corollaries:

• Defects => Bugs
• Overproduction => Extra Features
• Transportation => Handoffs
• Waiting => Delays
• Inventory => Partially Completed Work
• Motion => Task Switching
• (Over)Processing => Unneeded Processes

As with a lot of discussions about process, it's easy to write all this off as common sense.  And yet we can all get mired in processes and miss the forest for the trees.  It's always insightful to step back and use a list like this to get a fresh perspective on what's working and what needs fixing.

I find a lot of people (including myself) trying to apply software development processes without always thinking about the overall idea: they have daily standups, planning meetings, workflow charts, etc.  It's easy to think that just going through the motions of these steps will help you.  And in some cases, it does.  But it's important to constantly challenge your assumptions.  Holding yourself accountable to eliminating these "deadly wastes" seems like a great way to do that.

Faster Codebases

Continuing with the idea of code design and productivity, a colleague recently shared some keen wisdom: "Creating an agile team begins with an agile codebase."  As the last post discussed, productivity and code design are clearly related.  What's interesting is to expand this idea to the entire spectrum of agile development.

Many of agile's mainstays -- quick iterations, releasable software, and continuous integration -- all require a codebase that makes these possible.  And that goes back to two main ideas: de-coupling and encapsulation.  Those principles, in turn, make the following goals a lot easier:

• Simple, effective unit tests (tight coupling leads to brittle, cumbersome tests that are really integration tests)
• Easily deployable software (tightly coupled code is hard to deploy without breaking other parts of the system)
• Team scalability (a team can't grow quickly if each member has to intimately understand the entire codebase)

I used to think re-factoring was a process that could happen slowly in parallel with business goals.  But now I realize it needs to be an end in itself, in order to support not only productivity, but a growing team and an agile environment.

Productivity and Code Design

One of our developers suggested an important relationship that I haven't discussed yet: that of productivity and code design.  Most of the productivity benefits of good code design come after the original code is written -- they affect the ability to improve and expand the code.

You can write code designed to increase productivity by focusing on a few simple ideas:

• It should be easy to understand, so that additions or changes can be made more quickly
• It should be easy to expand, so that future enhancements don't have to re-factor the core logic
• It should be easy to deploy, so that changes don't require a massive regression testing effort

In Rapid Development, Steve McConnell makes similar points in the chapter, "Designing for Change."  The chapter suggests identifying areas likely to change, using information hiding, and using object-oriented programming, among other ideas.

Fortunately, a lot of these ideas piggyback off of good coding practices.  But the emphasis on these practices is usually towards maintainability.  It's interesting to reflect on their strong relationship to productivity, too.

This is another good way to understand how older codebases tend to reduce productivity.  Inevitably, design boundaries will have gotten blurred, shortcuts will have been taken, and "just get it done" projects will have reduced readability.  This all affects future productivity, and is another argument for periodic refactoring.

Measurement Reprise

Looking back on my initial list of what productivity-related metrics I hoped to measure makes me smile a bit: "time spent planning, time spent writing the functionality as originally spec'ed, time writing unit tests, time spent reworking functionality based on spec changes, time spent reworking tests based on spec changes, and time spent fixing bugs from the QA process."

It turns out it has proven far harder to measure any of these steps simply because their beginning and end points are so nebulous.  Despite all the projects I've been involved with, I had the naive assumption that there were discrete steps to all of them.  In fact, there are rarely clear-cut boundaries.  Are planning and coding really that separate?  Isn't coding in many ways just a more detailed, explicit form of a plan?  And a good approach to unit tests involves writing them in parallel with the main code, not as a separate, isolated step.  And in an agile environment, even the step between development and QA can be easily blurred, with bug-fixing and continuing development happening in parallel.

On top of that, it's just a mountain of information to keep track of.  It would be a full-time job just cataloging all those numbers, let alone analyzing them.

So I had to take a step back, look at our process, and think about what minimum amount of information could be recorded alongside it in as unobtrusive way as possible (after all, if measurement itself reduces productivity, it's hardly useful).

What I came up with was a simple addition to our check-in process: in addition to asking for a task or bug ID, which is already part of our check-in template, I added two lines: "hours spent" and "lessons/roadblocks".  "Hours spent" is a rough guess at how many hours the coder had to spend on the files before getting to this point of checking in the code.  I'm not asking people to keep timers running or start work diaries.  I just want a rough guess as to how much time was involved.  And "lessons/roadblocks" records any lessons learned or roadblocks encountered along the way.  The log message is parsed and the information is put into a database for later analysis.

I figure that if I can get developers to just spend a few more seconds recording that data as they check in code, we'll have a relatively painless way of collecting lots of useful information.

There's a risk that the information will be so unspecific as to be useless.  But I'm hoping that by analyzing the numbers as a group, the team will start to focus more on ways to increase its output.

Business Rule Complexity, Edge Cases, and Productivity

Lately we've run into a few situations that have made clear the relationship between business rule complexity, edge cases, and productivity.  This also relates to an aging codebase, as almost by definition the business complexity of a codebase will increase as it grows older.

One version of the problem is simply that business rule complexity leads to lower productivity, for a number of reasons:

1. Complex business rules are harder to understand and to code, and therefore take longer to implement.
2. Complex rules introduce more and subtler bugs into code.
3. Complex rules require a deep (and consequently more time-consuming) understanding of a system by people at all levels: the business owners, the programmers, the project managers, and the testers.
4. Edge cases are often overlooked by business owners, causing a lot of back-and-forth with business owners as development proceeds.

I think the importance of this last case is often overlooked.  It's very easy for executives to create general rules that in practice map to dozens of complicated logic paths.  Most executives lack the rigorous attention to detail that the identification of these logic paths requires.  So it's left to the developers (or fastidious project managers) to discover them; and once discovered, they have to be explained back to the executive (sometimes through an intermediary, which adds a level of potential error), and then translated into code.

Sometimes this complexity is hard to identify.  It's often the case that no single project leads to anything that raises a red flag.  But over years of business experimentation on a growing system, you can easily be left with a mass of hidden and hard-to-understand rules that programmers have to tiptoe around.  We're all familiar with the idea of spaghetti code; maybe we should be as wary of spaghetti rules.

In our case, our system evolved in these steps: we created a site; later, we translated it into several languages; with a more international audience, we wanted to display prices in our users' local currencies; to further cater to local customers, we introduced differential pricing based on currency; then, we offered different products based on a customer's location; finally, we started A/B testing our purchase path, and can target the tests based on a user's language, location, or both.

These are reasonable and savvy business decisions.  And yet we're left with a system in which introducing a new product involves a plethora of questions: What will it be called in every language?  How much will it cost in each currency?  Will it be offered worldwide or just in certain markets?  Will we test its introduction, and if so, how?  This complexity is probably worth the flexibility that our system allows.  And yet it's very rare for us to step back and ask the question, "If we make this change, how will it affect how quickly we can finish future projects?"

I've resolved to make it a part of my job to make clear to business owners just how complex their small tasks are making the codebase, and how that's affecting productivity.  Instead of measuring a project on the costs and benefits of the project alone, we should take a more expanded view and look how it might affect productivity in the future.  And this should be integrated into any calculation of the costs of the project.  This is, of course, easier said than done, especially since any true calculation would have to involve the projected life of the codebase, which is impossible to know.  Nonetheless, we should acknowledge it and try to account for it somehow.

It's also interesting to return to the idea of edge cases, and look at their impact.  Our lead developer and I were musing over how programmers seem magnetically inclined to not only find but dwell on edge cases.  We're all a little like lawyers, searching for that loophole that no one has thought of.  It's a point of pride to find that corner of business logic that makes everyone scratch their heads and say, "Wow, good point."

And yet, is that always a productive use of time?  If edge cases cover one billionth of a percent of use cases, we can probably all agree that they're not worth considering in all but life-critical situations.  What about a one hundredth of a percent of cases?  A full percent?  It's hard to make a blanket decision about it.  But I told my team that when they encounter an edge case, they should do a quick back-of-the-envelope calculation and if they think it will affect less than a tenth of a percent of customers, to discuss it with me -- and I'll probably give the go ahead to ignore it.

When you've been around since the creation of the codebase, it's often hard to realize how complex the business logic has become.  It was only recently when I was talking to a developer about the case of someone purchasing a product in France in euros and then flying to New York on a business trip and getting sent an offer to renew the product, but not being able to access the euro price... when I was struck by what a complex system our simple site had become.

You, Too, Can Become a Superstar

Here's a nice satirical piece entitled 3 Simple Rules That Will Make You a ‘Superstar’ Developer.  In addition to being a gentle reminder that productivity can come at the expense of quality, it reveals a gem of insight about productivity:

You’re 10x as productive when you’re working on code you wrote as on code you didn’t write.

This is certainly something that I think I need to take to heart more frequently.  The bigger a tech team gets, the more sluggish it can feel.  It's easy to look around and think, "why isn't everyone else getting as much done as I used to?!"  The answer, of course, is that I wrote it; of course I can work with it more quickly than other people.  Writing a site from scratch is a lot easier than having to dive into it mid-stream and add a piece to the insides.  And all those shortcuts I was able to make early on are now coming back to slow down the people I've handed things off to.

What can we learn from all this?  I'd love to say, "Always code with future programmers in mind," but the reality of startups is that that will very rarely happen.  Instead, we can identify projects that require a thorough understanding of the codebase (which isn't always easy to do, or isn't obvious when the project begins) and ensure the original author pairs with the new developer to explain how things work.

There's nothing like explaining why your three-year-old hack is making someone's present day miserable to improve your code quality.

Productivity from Consolidation, Deployment, and Design

Here are a few productivity ideas proposed by one of our developers, that I think are great:

• Consolidate related systems.  As our code-base has grown, we've re-implemented the same functionality a few different ways in different places.  At the time, this was done in the name of getting things out the door as quickly as possible.  Now, it's a hindrance.  Changing the functionality means either changing it in several places or sacrificing it in the more obscure places and focusing on the obvious ones.  Instead, we should consolidate the components to allow rapid development across all systems and lessen developer confusion.
• Architect systems with deployment in mind.  All too often we save a deployment plan for the end.  On larger projects, this leaves us with a tangle of code that is cumbersome to push live without breaking the production site.  That, in turn, makes the deployment process take a lot of time.  By focusing on deployment up front, we can create a plan to lower the risk of pushing code, and so speed up the entire process.  The approach should also encourage more abstraction and isolation in our systems, which is also good for maintainability.
• More centralized control over design and development.  It's hard to balance the gains from giving developers full autonomy over their code with the reality that some people are better at creating designs that lend themselves to rapid development.  We tend to give our developers full control over the projects they're given.  I think it's time to move to a more collaborative approach, where an initial design is reviewed with speed in mind.  We also do code reviews at the end of development, but at least one mid-term review would be beneficial to keep people from straying too far down an unproductive path.  We should identify those developers who have a track record of architecting solutions that allow for quick development and give them authority over design decisions for projects they're a part of.

Scrum: What We Were Missing

Last week was an interesting one for us.  We finished our first sprint of the new year using a few new Scrum tecniques, and I discovered some interesting lessons about Scrum, and how they pertain to productivity.  We've been trying to practice Scrum for a few years now, but things never felt quite right.  Here's what I think we were missing:

• Close contact with business owners.  Only recently have we had business owners working with us to plan sprints and attend the daily standups.  This boosts productivity in a few ways: first, the close involvement of business owners gives developers someone to work for.  I can try to re-transmit the needs and pressures of a business owner, but there's nothing like having them around, in person, keeping things on track in a way that provides the most value for them.  Second, without business owners we're left on our own to make dozens of small decisions each day about the direction of a project.  Hopefully we've learned to do this well, but having someone help us make these quick, small business decisions has already proved useful in pushing things forward more quickly.
• Demos.  We had our first demo on Friday, in front of a number of department heads and other interested parties.  It was wonderful.  I can't believe we haven't been doing these from the start.  It gave the developers a chance to show off their stuff, and people clapped -- when else does that happen in a programmer's life?  It provided them with "positive pressure," as one coder put it, that motivated them to complete the tasks on time.  It gave the business folks throughout the company insight into what we'd been up to for two weeks.  And it gave me great satisfaction watching both sides learn more about each other.
• Developer involvement in implementation decisions.  Previously, our tasks would be presented to us as implementation directives: "Add a link to the order page to show the user more purchase information."  Now we're moving to a more agile approach, where we talk about user needs and stories, and let the developers have a say in the final implementation.  In talking to people on our team about productivity, micro-management has been mentioned by a few of them as a productivity killer.  The opposite, autonomy and control, are offered as productivity boosters.  I think giving them more say in projects will motivate them to accomplish more with their time.

We're also trying out a few techniques that aren't necessarily prescribed by Scrum, but which are nonetheless already proving helpful:

• Break everything into half-day to one-and-a-half day tasks.  This has a few advantages.  First, it forces us to plan out those nebulous blobs of work up front, so the group as a whole can brainstorm about the most efficient approach.  Second, it acknowledges that the bigger a task is, the harder it is to accurately estimate the time it will take to complete it.  With more small tasks, we get more accurate estimates.  Finally, it provides a greater sense of accomplishment.  It's purely a psychological trick, but accomplishing five small things is a lot more gratifying than one big thing.
• In sprints, first decide on a group of goals, not a group of tasks.  Previously, our sprints were a hodgepodge of 20 different, unrelated tasks.  It's hard to prioritize 20 separate projects in any objective way.  It's a lot easier to prioritize goals.   And then you can come up with a list of tasks to accomplish those goals, and schedule those for the sprint.  And when you reach the end of the sprint, you don't ask yourself, "Have I completed all my tasks?"; you ask yourself, "Have I reached all my goals?"  In many cases, the two answers are different.  You may have finished only 80% of your tasks, but accomplished enough of your goals that you're ready to move on.  So you're left with a system that naturally filters out low-value, time-consuming work, and consequently boosts business productivity.
• Have success metrics for every project.   As a company grows, the number of new ideas and potential projects grows with it.  Anybody can come up with 100 new ideas for a company.  But asking the simple question, "How do we know if it worked?" is a powerful way to tame the circus of possibilities.  It focuses people, and forces them to consider how their abstract idea can be tied back to something tangible.  It also gives developers insight into the real-world value of what they're doing, and so provides a sense of accomplishment and motivation.

I'm excited to continue using these new approaches, and refine them further as we go.

The Productive Programmer

When talking about programmer productivity, I think it's important to acknowledge the wide range of outcomes that can be considered "productive."  For the following scenarios, which would you consider most productive?

• A developer who completes a task quickly, but produces unreadable code without documentation
• A developer who completes a task quickly, but leaves it riddled with bugs
• A developer who takes twice as long as you'd hoped, but architects the system so that it can be expanded effortlessly in the future
• A developer who takes longer to complete a project so that he can train another developer along the way, so that in the future the two of them can work in parallel
• A developer who takes eons to get any of his assigned tasks done, but along the way creates tools and systems that make the entire team more efficient in the long-run

In all cases, the trade-off is between short-term and long-term productivity.  I think most startups begin by focusing on short-term productivity, and as well they should: with the high rate of startup failure, there's no sense planning for long-term gains when the long-term might not exist.  For large, established companies, focusing on long-term productivity is usually more important; they have the luxury of existing market share to buffer them from at least some competition, and they know they'll be able to reap the rewards of their patience.  But for companies like us, somewhere between the worlds of the startup and the enterprise, where to walk the line is a lot less clear.  And the trick becomes balancing each and every decision with the need to be nimble today and sustainable tomorrow.

And so as I try to figure out how to measure productivity, I find myself wondering if I'll even be able to define productivity in our current environment.

Why Code Takes a Long Time

In this post, which I'll update throughout the year as I'm inspired, I'll try to list all the reasons why projects take longer than expected.  This list will specifically avoid addressing why estimates are incorrect; I consider that to be a separate issue.  Some of it is inspired by the Classic Mistakes Enumerated post I mentioned below.  This list comes from the book Rapid Development, which suggests posting lists like this in tech areas and encouraging everyone to review them frequently to identify problems as quickly as possible.

In addition to posting these publicly, I'll do two things with the information: 1) after each project completes, I'll try to identify which of these (if any) were responsible for slowing it down, and by how much and 2) before each project starts, I'll try to anticipate which of these might slow down the project and try to figure out how to mitigate them, if possible.

• Scope creep.  The project grows beyond its initial spec.
• Over-architecture.  The design was too complicated for the requirements.
• Technical debt.  The project required refactoring legacy code.
• Incomplete or vague spec.  The project was too loosely spec'ed, so time was spent figuring out what to do instead of doing it.
• Performance concerns.  Simple projects can become unexpectedly complicated in the context of a high-traffic, scalable website.
• Personality conflicts.  If a team doesn't work well together, productivity suffers.
• Divergent design.  Hopefully different components of a system integrate well together, or converge.  Sometimes a lack of foresight leads to a system with a bunch of components that don't end up fitting together.
• Inadequate design.  It's easy to be lulled into complacency on simple projects, when in fact they're worthy of a thorough design.
• Unknown technical work.  Whether a project needs research before starting, or involves a third-party API you're not familiar, any unknown variables are often a source of delays.

The Downside of Going Quickly

Perhaps it's because I'm a capitalist at heart; perhaps it's because I'm from a family of economists (thanks for the condolences); but Dan Pink's TED talk about how pay-for-performance reduces productivity has been haunting me a bit.  Surely there must be some way to exploit the profit motive in creative endeavors?

The other way of looking at it, which seems glaringly apparent in a lot of our projects, is that there's no downside to going slowly.  Of course, no one's twiddling their thumbs for hours each day, delaying real work.  But if you've got a choice between one unit test and twenty, which do you choose?  I'd guess most programmers would choose to write twenty, so they can feel confident that the code they release is error-free.  After all, while there's no downside to going slowly, there's typically a huge downside to being responsible for a bug that makes it to production.  Not only is there stress involved in fixing the bug, but chances are a much wider audience has discovered the issue, and no one wants the pressure of being responsible for a public problem.

So if there's no downside to going slowly and a fairly big downside to missing bugs, of course projects will take a long time.  To counter-act that tendency, you've got to offer some upside to finishing a project quickly.

But doesn't that bring us back to pay-for-performance?  Yes, in a way.  But I think the key difference is task productivity and project productivity.  If you imagine being given a complicated math problem, and are told, "You'll get $1,000 if you solve this in a minute," would the money help?  Probably not.  Financial incentives don't help you think faster, which is essentially what you'd have to do.  If anything, as the TED talk suggests, the money would add pressure and therefore distraction to your efforts, and lead you to actually go more slowly.

But now consider a different approach: you're given 100 complicated math problems, and told that you'll get $1,000 if you solve them within a week.  To me, that's an entirely different system.  Now, you're not being rewarded to think faster, you're being rewarded to figure out a system to tackle the problems as efficiently as possible.  Maybe that means scanning all 100 problems so that some unconscious part of your brain can be working on them while you sleep.  Maybe it means categorizing them so that you can remain in the same mindset while you tackle a group of them.  Whatever your strategy, you're being rewarded for figuring out how to get the best performance out of yourself.  And surely that must be beneficial and worthy of considering for the software development process.

Classic Mistakes

This blog post, Classic Mistakes Enumerated, from Steve McConnell's book Rapid Development, provides some great ideas (by inversion) of how to increase productivity.

Interestingly, while I began this project assuming I'd discover research about the speed of different software development methodologies, a lot of it seems to be returning to psychology in general, and motivation in particular.

The first mistake mentioned (though they're not in any order) is undermined motivation:

Study after study has shown that motivation probably has a larger effect on productivity and quality than any other factor (Boehm 1981). In Case Study 3-1, management took steps that undermined morale throughout the project--from giving a hokey pep talk at the beginning to requiring overtime in the middle and going on a long vacation while the team worked through the holidays to providing bonuses that work out to less than a dollar per overtime hour at the end.

It seems that motivation will be a big part of this year's endeavor.

"Noisy, crowded offices" is another interesting one that we may be falling victim to.  As an ever-growing start-up, we just had to redesign our tech area to accommodate more cubicles.  I'm hoping my offer of free noise-cancelling headsets to anyone who wants them will ease the pain a bit, but I'll have to make sure we don't start feeling like a zoo.

And "shortchanged quality assurance" hits particularly close to home right now.  Given last year's focus on quality, we've had a pretty good streak of high-quality releases.  I thought we might ease up on QA a bit in the name of getting code out the door more quickly.  We half-tested that hypothesis recently by picking a project that could "probably skip QA", given its apparent simplicity and the small risk involved if something went wrong.

In the end, we did some quick QA on the project -- and, sure enough, turned up enough bugs that it made us grateful for not skipping the process.  Had we avoided QA, all the same issues would have had to be addressed -- except we'd have to drop everything to fix them immediately (since they'd be on production) instead of being able to fit the fixes into our overall workflow.

Productivity and Financial Rewards

In the same blog post by Bruce Eckel, he cites a study about financial rewards and productivity:

Forty years ago, a seminal study showed that pay-for-performance only produces improvements in rote assembly-line-type work. For any work that involves creativity, pay-for-performance actually decreases productivity; apparently it demeans people to think that their creative work is only evaluated in terms of money (in the programming profession it's relatively well-understood that, as long as they can get by OK, programmers don't care that much about the money -- it's the quality of experience that matters). The negative impact of pay-for-performance has been emphasized in all the important business books of the past decades, the books that all the business leaders profess to read and agree with. And yet the only reward these same business leaders can think of is money, so they do exactly what has been shown again and again to produce negative results.

I must say I'm sympathetic to those business people who find this extremely counter-intuitive.  While lots of people profess to not be motivated by money, the history of capitalism's success stands as one gigantic counter-argument.  And are we to think the entire sales industry, which is based on pay-for-performance, is either horribly under-productive or blithely uncreative?

The blog post references a TED talk which is definitely worth watching.  There, Dan Pink elaborates on the research a bit, and conclude that, "'if/then' rewards work really well for those sorts of tasks: where there's a simple set of rules, and a clear destination to go to.  Rewards by their very nature narrow our focus, concentrate the mind -- that's why they work in so many cases."  But for creative tasks they don't work because, "reward narrows our focus and restricts our possibility."

He goes on to cite further studies that show, "once the task called for 'even rudimentary cognitive skill', a larger reward 'led to poorer performance'."

He concludes by arguing that for creative work, there are three keys to productivity: autonomy, mastery, purpose.  "Autonomy" and "purpose" fit nicely with the workflow of today's "enlightened" companies: the business owners define a vision and an objective, and trust the employees to work out a way to get there.  It's a bit of a leap of faith for business owners, but it sounds like one worth taking.

Estimates and Productivity

This blog post by Bruce Eckel has a number of fascinating points, particularly regarding estimation:

[In Waltzing with Bears,] DeMarco and Lister first point out something very important. When someone asks you how long a particular subproject will take, it's usually implicit, and sometimes explicit, that they want to know the shortest, most optimistic time for this task. DeMarco and Lister note that the actual time for finishing a task is a probability curve, and if you only ever give the shortest time, you are giving the leading edge of the curve, where it touches the axis. Thus, each subtask prediction has a 0% probability of being correct. This means your project completion time estimation starts out, from day one, with a 0% probability of being correct. They suggest a relatively simple change in behavior: give, instead, the middle of your probability curve for each subtask, so you begin with a palpable completion time. It doesn't make the completion time predictable, but it does make it significantly less wrong.

Should we all start giving our estimates as probability curves?  It's an interesting idea.  I think estimates are probably hard and time-consuming enough as it is; thinking of them as probabilities, while useful, might not yield much practical benefit.  And using the middle of the curve sounds nice, but I wonder if it's just adding layers of imprecision to an already imprecise guess.  If we're already bad at guessing the start of the curve, will we be any better at guessing the middle?

The post goes on to discuss a fascinating study about estimates:

In Peopleware, DeMarco and Lister also look at estimation, but for its effect on productivity. They run a test where managers and programmers estimate the completion of a project, in various combinations: the manager alone, the programmer and manager together, the programmer alone, and no estimate at all. The rather striking result was that the programmer was most productive when there was no estimate at all. So not only are we estimating very badly, the cost of estimation itself appears to be quite significant. Of course, current business thinking will look at these issues and say "very interesting, but we must have estimates and naturally we want the most optimistic ones."

This last point seems rather condescending towards "business thinking."   I think it's important to dwell on the issue a bit.  Of course folks on the business side want estimates -- how else could they possibly run their businesses?  Not only do you need estimates for scheduling and timelines, but -- far more importantly -- you need them to figure out the cost of a project so you can decide whether to do it at all.

And isn't it natural to want optimistic estimates?  Isn't part of what we're paid for to mitigate risk by avoiding potential pitfalls?

Maybe instead of trying to mold our estimates to fit into our software development practices, we have to mold our software development practices to produce accurate estimates.  It sounds like a nice idea; if only we knew how to do it.

But I don't want to get too far away from the earth-shattering conclusion of the study: "The rather striking result was that the programmer was most productive when there was no estimate at all."  What could possibly explain that?  It certainly seems to contradict the Hawthorne Effect.  The blog post goes on to discuss pay-for-performance (discussed next) and suggests that this entire way of thinking is demeaning.

Unfortunately, this leaves us mired in an awful situation: we need estimates to function, but our programmers are most productive if they don't give estimates.

Where's the Research?

One thing that surprises me about this endeavor is how little research there seems to be about writing code quickly.  There are a smattering of studies justifying quality in the name of speed; for instance, a famous study found that the longer it took to find a bug, the longer it took to fix it (though I can't find the source).  But it's hard to find concrete data on how to write more quickly.  Surely Microsoft and IBM must have studied this for decades; after all, a mere 10% productivity increase in coding speed would have huge benefits for both companies.

And yet, if they have been conducting studies, they've kept awfully quiet about them.

The most interesting analysis I've been able to find is this post by Joel Spolsky, measuring the time it takes college students to do an identical project.  As he reports,

The most obvious thing you notice here is the huge variations. The fastest students were finishing three or four times faster than the average students and as much as ten times faster than the slowest students.

The point of his article is that companies should hire highly productive programmers, which is hard to argue with.  But don't we all have a small suspicion that increased productivity can be taught?

One thing I'd like to do this year is to simply partner with different programmers and see if I can figure out what makes some faster than others.  It may just come down to "brain speed," of course, but I suspect there are some processes at work that could benefit everyone.  For instance, some programmers tend to just dive into a project and start coding; others tend to step back, create a plan first, and then proceed.  The second approach sounds like it would pay off, but it's worth testing; what if our agile approach makes most planning a waste of time?

Testing is another thing I plan to investigate.  We tend to write unit tests for most of our code.  And yet, for many features of our site, the absolute worst-case scenario is that a few customers see an error message, we notice it in the logs, and we fix it within a few minutes.  Does that justify spending a day writing unit tests, and spending time and energy maintaining those tests as the functionality changes?  Or what about the question of when to write tests?  Test-driven development encourages you to write tests first.  But in a fast-changing business environment, that can leave you with a lot of tests to re-write if the spec changes halfway through the project.

I suspect I'll spend the first part of the year just asking questions like this, to get a handle on what to measure, before doing much measuring of it.  So far on my list: time spent planning, time spent writing the functionality as originally spec'ed, time writing unit tests, time spent reworking functionality based on spec changes, time spent reworking tests based on spec changes, and time spent fixing bugs from the QA process.

Update: I probably should have given myself a few weeks to get into this project before concluding there was no research about it.  In fact, it seems that lots of companies have done lots of studies on programmer productivity, as one would hope for and expect.  I'm still working my way through Steve McConnell's Rapid Development, but so far it has cited numerous examples of research and studies that relate to productivity.

Scrum and The Hawthorne Effect

While reading this Wired article about motivation (cleverly disguised as a story about a shoe gadget), it occurred to me what the missing link was between Scrum and productivity.  If you haven't heard, Scrum tends to promise some magical endpoint of hyper-productivity.  Teams report being idle because -- gasp -- their product owners can't give them enough work.  I'll believe this when I see it -- and I have yet to see it -- but what makes it frustrating is that it's rarely (never?) discussed why this happens.  After all, there's nothing in the Scrum process that's specifically devoted to increasing productivity.  Sure, there's a lot of talk about burndown charts and improving velocity, but at the end of the day a programmer's time estimate is taken as a sacred edict.  And what good is that if things still seem to be taking too long?

And so I was delighted to be reminded of the Hawthorne Effect:

In the mid-1920s at Western Electric's manufacturing plant in Cicero, Illinois, the management began an experiment. The lighting in an area occupied by one set of workers was increased so there was better illumination to help them see the telephone relays they were building. Perhaps not surprisingly, workers who had more light were able to assemble relays faster.

Other changes were then made: Employees were given rest breaks. Their productivity increased. They were allowed to work shorter hours. Again, they were more efficient during those hours.

But then something weird happened. The lighting was cut back to normal ... and productivity still went up. In fact, just about every change the company made had only one effect: increased worker productivity. After months of tinkering, the work conditions were returned to the original state, and workers built more relays than they did in the exact same circumstances at the start of the experiment.

What was happening? Why was it that no matter what the Hawthorne plant managers did, the workers just performed better? Researchers puzzled over the results, and some still doubt the details of the experiment's protocols. But the study gave rise to what's known in sociology as the Hawthorne effect.

The gist of the idea is that people change their behavior—often for the better—when they are being observed (which is why it's sometimes called the observer effect). Those workers at Western Electric didn't build more relays because there was more or less light or because they had more or fewer breaks. The Hawthorne effect posits that they built more relays simply because they knew someone was keeping track of how many relays they built.

And that, I suspect, is why Scrum boosts productivity: knowing they're being measured, programmers speed up.  And of course all the other bits help, too: the retrospectives, the close business feedback, firewalling, short work intervals, etc.  But will a coder working alone in a room have better productivity under Scrum than not?  Yes.  And why?  The Hawthorne Effect.

I'm planning on exploiting this as much as possible over the coming year.  Posting burndown charts, tracking the accuracy of everyone's estimates, and regularly reviewing these with the team, will hopefully all have a Hawthornian payoff.

And meanwhile this very blog will be my own personal Hawthorne motivator, as the world keeps an eye on how we're doing...

What's "Faster"?

The first hurdle that The Faster Software Project has to overcome is figuring out how to objectively measure the speed of software development. Fortunately, for us, there's an easy answer: you can't. Sure, if all our programmers worked on the exact same project, with precisely the same set of instructions, with all the same resources, and started at exactly the same time -- yes, then I suppose we could be objective about it. But because that will never happen in the real world, we have to compromise.

One idea that is often proposed at this stage is to somehow quantify software development effort. These discussions lead directly to the idea of measuring lines of code or kloc. Most programmers know instinctively that this is folly. My favorite rebuttal of it is to repeat a simple status update once given by one of our lead developers: "I fixed that bug by removing a line of code."

Nonetheless, I'd entertain the idea as a very loose way of measuring programmer output if I could do it secretly. After all, I suspect that there's a decent correlation between those programmers on our team who I consider to be the most productive and the lines of code they produce each year. However, one technique I plan to exploit in my effort to improve software development speed is the Hawthorne Effect, which states roughly that people tend to be more productive when they know they're being watched. And the second you tell a group of clever programmers that not only are they being watched, but that they're being measured by the number of lines of code they produce, well -- stand back. I can only imagine the horrors that would ensue from written-out for loops, cut-and-pasted code, and -- from the cleverest -- automatic code generators. The simple idea that huge productivity gains can be achieved from code re-use is enough to kill the idea completely.

And so, having nixed the easiest option, what are we left with? Nothing terribly useful, I'm afraid. So unless I learn of a better idea -- and I'd be most grateful to hear everyone's suggestions -- I'll settle for an unquestionably non-scientific approach: for each project our team tackles this year, I'll record my own private estimate of how long I think it should take. Then I'll compare my estimates to the actual time taken. Hopefully my standards for estimation will remain consistent throughout the year, and this will yield some useful data that can help us get a sense of whether we're improving.