You, Too, Can Become a Superstar

Here's a nice satirical piece entitled 3 Simple Rules That Will Make You a ‘Superstar’ Developer.  In addition to being a gentle reminder that productivity can come at the expense of quality, it reveals a gem of insight about productivity:

You’re 10x as productive when you’re working on code you wrote as on code you didn’t write.

This is certainly something that I think I need to take to heart more frequently.  The bigger a tech team gets, the more sluggish it can feel.  It's easy to look around and think, "why isn't everyone else getting as much done as I used to?!"  The answer, of course, is that I wrote it; of course I can work with it more quickly than other people.  Writing a site from scratch is a lot easier than having to dive into it mid-stream and add a piece to the insides.  And all those shortcuts I was able to make early on are now coming back to slow down the people I've handed things off to.

What can we learn from all this?  I'd love to say, "Always code with future programmers in mind," but the reality of startups is that that will very rarely happen.  Instead, we can identify projects that require a thorough understanding of the codebase (which isn't always easy to do, or isn't obvious when the project begins) and ensure the original author pairs with the new developer to explain how things work.

There's nothing like explaining why your three-year-old hack is making someone's present day miserable to improve your code quality.

Productivity from Consolidation, Deployment, and Design

Here are a few productivity ideas proposed by one of our developers, that I think are great:

• Consolidate related systems.  As our code-base has grown, we've re-implemented the same functionality a few different ways in different places.  At the time, this was done in the name of getting things out the door as quickly as possible.  Now, it's a hindrance.  Changing the functionality means either changing it in several places or sacrificing it in the more obscure places and focusing on the obvious ones.  Instead, we should consolidate the components to allow rapid development across all systems and lessen developer confusion.
• Architect systems with deployment in mind.  All too often we save a deployment plan for the end.  On larger projects, this leaves us with a tangle of code that is cumbersome to push live without breaking the production site.  That, in turn, makes the deployment process take a lot of time.  By focusing on deployment up front, we can create a plan to lower the risk of pushing code, and so speed up the entire process.  The approach should also encourage more abstraction and isolation in our systems, which is also good for maintainability.
• More centralized control over design and development.  It's hard to balance the gains from giving developers full autonomy over their code with the reality that some people are better at creating designs that lend themselves to rapid development.  We tend to give our developers full control over the projects they're given.  I think it's time to move to a more collaborative approach, where an initial design is reviewed with speed in mind.  We also do code reviews at the end of development, but at least one mid-term review would be beneficial to keep people from straying too far down an unproductive path.  We should identify those developers who have a track record of architecting solutions that allow for quick development and give them authority over design decisions for projects they're a part of.

Scrum: What We Were Missing

Last week was an interesting one for us.  We finished our first sprint of the new year using a few new Scrum tecniques, and I discovered some interesting lessons about Scrum, and how they pertain to productivity.  We've been trying to practice Scrum for a few years now, but things never felt quite right.  Here's what I think we were missing:

• Close contact with business owners.  Only recently have we had business owners working with us to plan sprints and attend the daily standups.  This boosts productivity in a few ways: first, the close involvement of business owners gives developers someone to work for.  I can try to re-transmit the needs and pressures of a business owner, but there's nothing like having them around, in person, keeping things on track in a way that provides the most value for them.  Second, without business owners we're left on our own to make dozens of small decisions each day about the direction of a project.  Hopefully we've learned to do this well, but having someone help us make these quick, small business decisions has already proved useful in pushing things forward more quickly.
• Demos.  We had our first demo on Friday, in front of a number of department heads and other interested parties.  It was wonderful.  I can't believe we haven't been doing these from the start.  It gave the developers a chance to show off their stuff, and people clapped -- when else does that happen in a programmer's life?  It provided them with "positive pressure," as one coder put it, that motivated them to complete the tasks on time.  It gave the business folks throughout the company insight into what we'd been up to for two weeks.  And it gave me great satisfaction watching both sides learn more about each other.
• Developer involvement in implementation decisions.  Previously, our tasks would be presented to us as implementation directives: "Add a link to the order page to show the user more purchase information."  Now we're moving to a more agile approach, where we talk about user needs and stories, and let the developers have a say in the final implementation.  In talking to people on our team about productivity, micro-management has been mentioned by a few of them as a productivity killer.  The opposite, autonomy and control, are offered as productivity boosters.  I think giving them more say in projects will motivate them to accomplish more with their time.

We're also trying out a few techniques that aren't necessarily prescribed by Scrum, but which are nonetheless already proving helpful:

• Break everything into half-day to one-and-a-half day tasks.  This has a few advantages.  First, it forces us to plan out those nebulous blobs of work up front, so the group as a whole can brainstorm about the most efficient approach.  Second, it acknowledges that the bigger a task is, the harder it is to accurately estimate the time it will take to complete it.  With more small tasks, we get more accurate estimates.  Finally, it provides a greater sense of accomplishment.  It's purely a psychological trick, but accomplishing five small things is a lot more gratifying than one big thing.
• In sprints, first decide on a group of goals, not a group of tasks.  Previously, our sprints were a hodgepodge of 20 different, unrelated tasks.  It's hard to prioritize 20 separate projects in any objective way.  It's a lot easier to prioritize goals.   And then you can come up with a list of tasks to accomplish those goals, and schedule those for the sprint.  And when you reach the end of the sprint, you don't ask yourself, "Have I completed all my tasks?"; you ask yourself, "Have I reached all my goals?"  In many cases, the two answers are different.  You may have finished only 80% of your tasks, but accomplished enough of your goals that you're ready to move on.  So you're left with a system that naturally filters out low-value, time-consuming work, and consequently boosts business productivity.
• Have success metrics for every project.   As a company grows, the number of new ideas and potential projects grows with it.  Anybody can come up with 100 new ideas for a company.  But asking the simple question, "How do we know if it worked?" is a powerful way to tame the circus of possibilities.  It focuses people, and forces them to consider how their abstract idea can be tied back to something tangible.  It also gives developers insight into the real-world value of what they're doing, and so provides a sense of accomplishment and motivation.

I'm excited to continue using these new approaches, and refine them further as we go.

The Productive Programmer

When talking about programmer productivity, I think it's important to acknowledge the wide range of outcomes that can be considered "productive."  For the following scenarios, which would you consider most productive?

• A developer who completes a task quickly, but produces unreadable code without documentation
• A developer who completes a task quickly, but leaves it riddled with bugs
• A developer who takes twice as long as you'd hoped, but architects the system so that it can be expanded effortlessly in the future
• A developer who takes longer to complete a project so that he can train another developer along the way, so that in the future the two of them can work in parallel
• A developer who takes eons to get any of his assigned tasks done, but along the way creates tools and systems that make the entire team more efficient in the long-run

In all cases, the trade-off is between short-term and long-term productivity.  I think most startups begin by focusing on short-term productivity, and as well they should: with the high rate of startup failure, there's no sense planning for long-term gains when the long-term might not exist.  For large, established companies, focusing on long-term productivity is usually more important; they have the luxury of existing market share to buffer them from at least some competition, and they know they'll be able to reap the rewards of their patience.  But for companies like us, somewhere between the worlds of the startup and the enterprise, where to walk the line is a lot less clear.  And the trick becomes balancing each and every decision with the need to be nimble today and sustainable tomorrow.

And so as I try to figure out how to measure productivity, I find myself wondering if I'll even be able to define productivity in our current environment.

Why Code Takes a Long Time

In this post, which I'll update throughout the year as I'm inspired, I'll try to list all the reasons why projects take longer than expected.  This list will specifically avoid addressing why estimates are incorrect; I consider that to be a separate issue.  Some of it is inspired by the Classic Mistakes Enumerated post I mentioned below.  This list comes from the book Rapid Development, which suggests posting lists like this in tech areas and encouraging everyone to review them frequently to identify problems as quickly as possible.

In addition to posting these publicly, I'll do two things with the information: 1) after each project completes, I'll try to identify which of these (if any) were responsible for slowing it down, and by how much and 2) before each project starts, I'll try to anticipate which of these might slow down the project and try to figure out how to mitigate them, if possible.

• Scope creep.  The project grows beyond its initial spec.
• Over-architecture.  The design was too complicated for the requirements.
• Technical debt.  The project required refactoring legacy code.
• Incomplete or vague spec.  The project was too loosely spec'ed, so time was spent figuring out what to do instead of doing it.
• Performance concerns.  Simple projects can become unexpectedly complicated in the context of a high-traffic, scalable website.
• Personality conflicts.  If a team doesn't work well together, productivity suffers.
• Divergent design.  Hopefully different components of a system integrate well together, or converge.  Sometimes a lack of foresight leads to a system with a bunch of components that don't end up fitting together.
• Inadequate design.  It's easy to be lulled into complacency on simple projects, when in fact they're worthy of a thorough design.
• Unknown technical work.  Whether a project needs research before starting, or involves a third-party API you're not familiar, any unknown variables are often a source of delays.

The Downside of Going Quickly

Perhaps it's because I'm a capitalist at heart; perhaps it's because I'm from a family of economists (thanks for the condolences); but Dan Pink's TED talk about how pay-for-performance reduces productivity has been haunting me a bit.  Surely there must be some way to exploit the profit motive in creative endeavors?

The other way of looking at it, which seems glaringly apparent in a lot of our projects, is that there's no downside to going slowly.  Of course, no one's twiddling their thumbs for hours each day, delaying real work.  But if you've got a choice between one unit test and twenty, which do you choose?  I'd guess most programmers would choose to write twenty, so they can feel confident that the code they release is error-free.  After all, while there's no downside to going slowly, there's typically a huge downside to being responsible for a bug that makes it to production.  Not only is there stress involved in fixing the bug, but chances are a much wider audience has discovered the issue, and no one wants the pressure of being responsible for a public problem.

So if there's no downside to going slowly and a fairly big downside to missing bugs, of course projects will take a long time.  To counter-act that tendency, you've got to offer some upside to finishing a project quickly.

But doesn't that bring us back to pay-for-performance?  Yes, in a way.  But I think the key difference is task productivity and project productivity.  If you imagine being given a complicated math problem, and are told, "You'll get $1,000 if you solve this in a minute," would the money help?  Probably not.  Financial incentives don't help you think faster, which is essentially what you'd have to do.  If anything, as the TED talk suggests, the money would add pressure and therefore distraction to your efforts, and lead you to actually go more slowly.

But now consider a different approach: you're given 100 complicated math problems, and told that you'll get $1,000 if you solve them within a week.  To me, that's an entirely different system.  Now, you're not being rewarded to think faster, you're being rewarded to figure out a system to tackle the problems as efficiently as possible.  Maybe that means scanning all 100 problems so that some unconscious part of your brain can be working on them while you sleep.  Maybe it means categorizing them so that you can remain in the same mindset while you tackle a group of them.  Whatever your strategy, you're being rewarded for figuring out how to get the best performance out of yourself.  And surely that must be beneficial and worthy of considering for the software development process.

Classic Mistakes

This blog post, Classic Mistakes Enumerated, from Steve McConnell's book Rapid Development, provides some great ideas (by inversion) of how to increase productivity.

Interestingly, while I began this project assuming I'd discover research about the speed of different software development methodologies, a lot of it seems to be returning to psychology in general, and motivation in particular.

The first mistake mentioned (though they're not in any order) is undermined motivation:

Study after study has shown that motivation probably has a larger effect on productivity and quality than any other factor (Boehm 1981). In Case Study 3-1, management took steps that undermined morale throughout the project--from giving a hokey pep talk at the beginning to requiring overtime in the middle and going on a long vacation while the team worked through the holidays to providing bonuses that work out to less than a dollar per overtime hour at the end.

It seems that motivation will be a big part of this year's endeavor.

"Noisy, crowded offices" is another interesting one that we may be falling victim to.  As an ever-growing start-up, we just had to redesign our tech area to accommodate more cubicles.  I'm hoping my offer of free noise-cancelling headsets to anyone who wants them will ease the pain a bit, but I'll have to make sure we don't start feeling like a zoo.

And "shortchanged quality assurance" hits particularly close to home right now.  Given last year's focus on quality, we've had a pretty good streak of high-quality releases.  I thought we might ease up on QA a bit in the name of getting code out the door more quickly.  We half-tested that hypothesis recently by picking a project that could "probably skip QA", given its apparent simplicity and the small risk involved if something went wrong.

In the end, we did some quick QA on the project -- and, sure enough, turned up enough bugs that it made us grateful for not skipping the process.  Had we avoided QA, all the same issues would have had to be addressed -- except we'd have to drop everything to fix them immediately (since they'd be on production) instead of being able to fit the fixes into our overall workflow.

Productivity and Financial Rewards

In the same blog post by Bruce Eckel, he cites a study about financial rewards and productivity:

Forty years ago, a seminal study showed that pay-for-performance only produces improvements in rote assembly-line-type work. For any work that involves creativity, pay-for-performance actually decreases productivity; apparently it demeans people to think that their creative work is only evaluated in terms of money (in the programming profession it's relatively well-understood that, as long as they can get by OK, programmers don't care that much about the money -- it's the quality of experience that matters). The negative impact of pay-for-performance has been emphasized in all the important business books of the past decades, the books that all the business leaders profess to read and agree with. And yet the only reward these same business leaders can think of is money, so they do exactly what has been shown again and again to produce negative results.

I must say I'm sympathetic to those business people who find this extremely counter-intuitive.  While lots of people profess to not be motivated by money, the history of capitalism's success stands as one gigantic counter-argument.  And are we to think the entire sales industry, which is based on pay-for-performance, is either horribly under-productive or blithely uncreative?

The blog post references a TED talk which is definitely worth watching.  There, Dan Pink elaborates on the research a bit, and conclude that, "'if/then' rewards work really well for those sorts of tasks: where there's a simple set of rules, and a clear destination to go to.  Rewards by their very nature narrow our focus, concentrate the mind -- that's why they work in so many cases."  But for creative tasks they don't work because, "reward narrows our focus and restricts our possibility."

He goes on to cite further studies that show, "once the task called for 'even rudimentary cognitive skill', a larger reward 'led to poorer performance'."

He concludes by arguing that for creative work, there are three keys to productivity: autonomy, mastery, purpose.  "Autonomy" and "purpose" fit nicely with the workflow of today's "enlightened" companies: the business owners define a vision and an objective, and trust the employees to work out a way to get there.  It's a bit of a leap of faith for business owners, but it sounds like one worth taking.

Estimates and Productivity

This blog post by Bruce Eckel has a number of fascinating points, particularly regarding estimation:

[In Waltzing with Bears,] DeMarco and Lister first point out something very important. When someone asks you how long a particular subproject will take, it's usually implicit, and sometimes explicit, that they want to know the shortest, most optimistic time for this task. DeMarco and Lister note that the actual time for finishing a task is a probability curve, and if you only ever give the shortest time, you are giving the leading edge of the curve, where it touches the axis. Thus, each subtask prediction has a 0% probability of being correct. This means your project completion time estimation starts out, from day one, with a 0% probability of being correct. They suggest a relatively simple change in behavior: give, instead, the middle of your probability curve for each subtask, so you begin with a palpable completion time. It doesn't make the completion time predictable, but it does make it significantly less wrong.

Should we all start giving our estimates as probability curves?  It's an interesting idea.  I think estimates are probably hard and time-consuming enough as it is; thinking of them as probabilities, while useful, might not yield much practical benefit.  And using the middle of the curve sounds nice, but I wonder if it's just adding layers of imprecision to an already imprecise guess.  If we're already bad at guessing the start of the curve, will we be any better at guessing the middle?

The post goes on to discuss a fascinating study about estimates:

In Peopleware, DeMarco and Lister also look at estimation, but for its effect on productivity. They run a test where managers and programmers estimate the completion of a project, in various combinations: the manager alone, the programmer and manager together, the programmer alone, and no estimate at all. The rather striking result was that the programmer was most productive when there was no estimate at all. So not only are we estimating very badly, the cost of estimation itself appears to be quite significant. Of course, current business thinking will look at these issues and say "very interesting, but we must have estimates and naturally we want the most optimistic ones."

This last point seems rather condescending towards "business thinking."   I think it's important to dwell on the issue a bit.  Of course folks on the business side want estimates -- how else could they possibly run their businesses?  Not only do you need estimates for scheduling and timelines, but -- far more importantly -- you need them to figure out the cost of a project so you can decide whether to do it at all.

And isn't it natural to want optimistic estimates?  Isn't part of what we're paid for to mitigate risk by avoiding potential pitfalls?

Maybe instead of trying to mold our estimates to fit into our software development practices, we have to mold our software development practices to produce accurate estimates.  It sounds like a nice idea; if only we knew how to do it.

But I don't want to get too far away from the earth-shattering conclusion of the study: "The rather striking result was that the programmer was most productive when there was no estimate at all."  What could possibly explain that?  It certainly seems to contradict the Hawthorne Effect.  The blog post goes on to discuss pay-for-performance (discussed next) and suggests that this entire way of thinking is demeaning.

Unfortunately, this leaves us mired in an awful situation: we need estimates to function, but our programmers are most productive if they don't give estimates.

Where's the Research?

One thing that surprises me about this endeavor is how little research there seems to be about writing code quickly.  There are a smattering of studies justifying quality in the name of speed; for instance, a famous study found that the longer it took to find a bug, the longer it took to fix it (though I can't find the source).  But it's hard to find concrete data on how to write more quickly.  Surely Microsoft and IBM must have studied this for decades; after all, a mere 10% productivity increase in coding speed would have huge benefits for both companies.

And yet, if they have been conducting studies, they've kept awfully quiet about them.

The most interesting analysis I've been able to find is this post by Joel Spolsky, measuring the time it takes college students to do an identical project.  As he reports,

The most obvious thing you notice here is the huge variations. The fastest students were finishing three or four times faster than the average students and as much as ten times faster than the slowest students.

The point of his article is that companies should hire highly productive programmers, which is hard to argue with.  But don't we all have a small suspicion that increased productivity can be taught?

One thing I'd like to do this year is to simply partner with different programmers and see if I can figure out what makes some faster than others.  It may just come down to "brain speed," of course, but I suspect there are some processes at work that could benefit everyone.  For instance, some programmers tend to just dive into a project and start coding; others tend to step back, create a plan first, and then proceed.  The second approach sounds like it would pay off, but it's worth testing; what if our agile approach makes most planning a waste of time?

Testing is another thing I plan to investigate.  We tend to write unit tests for most of our code.  And yet, for many features of our site, the absolute worst-case scenario is that a few customers see an error message, we notice it in the logs, and we fix it within a few minutes.  Does that justify spending a day writing unit tests, and spending time and energy maintaining those tests as the functionality changes?  Or what about the question of when to write tests?  Test-driven development encourages you to write tests first.  But in a fast-changing business environment, that can leave you with a lot of tests to re-write if the spec changes halfway through the project.

I suspect I'll spend the first part of the year just asking questions like this, to get a handle on what to measure, before doing much measuring of it.  So far on my list: time spent planning, time spent writing the functionality as originally spec'ed, time writing unit tests, time spent reworking functionality based on spec changes, time spent reworking tests based on spec changes, and time spent fixing bugs from the QA process.

Update: I probably should have given myself a few weeks to get into this project before concluding there was no research about it.  In fact, it seems that lots of companies have done lots of studies on programmer productivity, as one would hope for and expect.  I'm still working my way through Steve McConnell's Rapid Development, but so far it has cited numerous examples of research and studies that relate to productivity.

Scrum and The Hawthorne Effect

While reading this Wired article about motivation (cleverly disguised as a story about a shoe gadget), it occurred to me what the missing link was between Scrum and productivity.  If you haven't heard, Scrum tends to promise some magical endpoint of hyper-productivity.  Teams report being idle because -- gasp -- their product owners can't give them enough work.  I'll believe this when I see it -- and I have yet to see it -- but what makes it frustrating is that it's rarely (never?) discussed why this happens.  After all, there's nothing in the Scrum process that's specifically devoted to increasing productivity.  Sure, there's a lot of talk about burndown charts and improving velocity, but at the end of the day a programmer's time estimate is taken as a sacred edict.  And what good is that if things still seem to be taking too long?

And so I was delighted to be reminded of the Hawthorne Effect:

In the mid-1920s at Western Electric's manufacturing plant in Cicero, Illinois, the management began an experiment. The lighting in an area occupied by one set of workers was increased so there was better illumination to help them see the telephone relays they were building. Perhaps not surprisingly, workers who had more light were able to assemble relays faster.

Other changes were then made: Employees were given rest breaks. Their productivity increased. They were allowed to work shorter hours. Again, they were more efficient during those hours.

But then something weird happened. The lighting was cut back to normal ... and productivity still went up. In fact, just about every change the company made had only one effect: increased worker productivity. After months of tinkering, the work conditions were returned to the original state, and workers built more relays than they did in the exact same circumstances at the start of the experiment.

What was happening? Why was it that no matter what the Hawthorne plant managers did, the workers just performed better? Researchers puzzled over the results, and some still doubt the details of the experiment's protocols. But the study gave rise to what's known in sociology as the Hawthorne effect.

The gist of the idea is that people change their behavior—often for the better—when they are being observed (which is why it's sometimes called the observer effect). Those workers at Western Electric didn't build more relays because there was more or less light or because they had more or fewer breaks. The Hawthorne effect posits that they built more relays simply because they knew someone was keeping track of how many relays they built.

And that, I suspect, is why Scrum boosts productivity: knowing they're being measured, programmers speed up.  And of course all the other bits help, too: the retrospectives, the close business feedback, firewalling, short work intervals, etc.  But will a coder working alone in a room have better productivity under Scrum than not?  Yes.  And why?  The Hawthorne Effect.

I'm planning on exploiting this as much as possible over the coming year.  Posting burndown charts, tracking the accuracy of everyone's estimates, and regularly reviewing these with the team, will hopefully all have a Hawthornian payoff.

And meanwhile this very blog will be my own personal Hawthorne motivator, as the world keeps an eye on how we're doing...

What's "Faster"?

The first hurdle that The Faster Software Project has to overcome is figuring out how to objectively measure the speed of software development. Fortunately, for us, there's an easy answer: you can't. Sure, if all our programmers worked on the exact same project, with precisely the same set of instructions, with all the same resources, and started at exactly the same time -- yes, then I suppose we could be objective about it. But because that will never happen in the real world, we have to compromise.

One idea that is often proposed at this stage is to somehow quantify software development effort. These discussions lead directly to the idea of measuring lines of code or kloc. Most programmers know instinctively that this is folly. My favorite rebuttal of it is to repeat a simple status update once given by one of our lead developers: "I fixed that bug by removing a line of code."

Nonetheless, I'd entertain the idea as a very loose way of measuring programmer output if I could do it secretly. After all, I suspect that there's a decent correlation between those programmers on our team who I consider to be the most productive and the lines of code they produce each year. However, one technique I plan to exploit in my effort to improve software development speed is the Hawthorne Effect, which states roughly that people tend to be more productive when they know they're being watched. And the second you tell a group of clever programmers that not only are they being watched, but that they're being measured by the number of lines of code they produce, well -- stand back. I can only imagine the horrors that would ensue from written-out for loops, cut-and-pasted code, and -- from the cleverest -- automatic code generators. The simple idea that huge productivity gains can be achieved from code re-use is enough to kill the idea completely.

And so, having nixed the easiest option, what are we left with? Nothing terribly useful, I'm afraid. So unless I learn of a better idea -- and I'd be most grateful to hear everyone's suggestions -- I'll settle for an unquestionably non-scientific approach: for each project our team tackles this year, I'll record my own private estimate of how long I think it should take. Then I'll compare my estimates to the actual time taken. Hopefully my standards for estimation will remain consistent throughout the year, and this will yield some useful data that can help us get a sense of whether we're improving.