Scrum and The Hawthorne Effect

While reading this Wired article about motivation (cleverly disguised as a story about a shoe gadget), it occurred to me what the missing link was between Scrum and productivity.  If you haven't heard, Scrum tends to promise some magical endpoint of hyper-productivity.  Teams report being idle because -- gasp -- their product owners can't give them enough work.  I'll believe this when I see it -- and I have yet to see it -- but what makes it frustrating is that it's rarely (never?) discussed why this happens.  After all, there's nothing in the Scrum process that's specifically devoted to increasing productivity.  Sure, there's a lot of talk about burndown charts and improving velocity, but at the end of the day a programmer's time estimate is taken as a sacred edict.  And what good is that if things still seem to be taking too long?

And so I was delighted to be reminded of the Hawthorne Effect:

In the mid-1920s at Western Electric's manufacturing plant in Cicero, Illinois, the management began an experiment. The lighting in an area occupied by one set of workers was increased so there was better illumination to help them see the telephone relays they were building. Perhaps not surprisingly, workers who had more light were able to assemble relays faster.

Other changes were then made: Employees were given rest breaks. Their productivity increased. They were allowed to work shorter hours. Again, they were more efficient during those hours.

But then something weird happened. The lighting was cut back to normal ... and productivity still went up. In fact, just about every change the company made had only one effect: increased worker productivity. After months of tinkering, the work conditions were returned to the original state, and workers built more relays than they did in the exact same circumstances at the start of the experiment.

What was happening? Why was it that no matter what the Hawthorne plant managers did, the workers just performed better? Researchers puzzled over the results, and some still doubt the details of the experiment's protocols. But the study gave rise to what's known in sociology as the Hawthorne effect.

The gist of the idea is that people change their behavior—often for the better—when they are being observed (which is why it's sometimes called the observer effect). Those workers at Western Electric didn't build more relays because there was more or less light or because they had more or fewer breaks. The Hawthorne effect posits that they built more relays simply because they knew someone was keeping track of how many relays they built.

And that, I suspect, is why Scrum boosts productivity: knowing they're being measured, programmers speed up.  And of course all the other bits help, too: the retrospectives, the close business feedback, firewalling, short work intervals, etc.  But will a coder working alone in a room have better productivity under Scrum than not?  Yes.  And why?  The Hawthorne Effect.

I'm planning on exploiting this as much as possible over the coming year.  Posting burndown charts, tracking the accuracy of everyone's estimates, and regularly reviewing these with the team, will hopefully all have a Hawthornian payoff.

And meanwhile this very blog will be my own personal Hawthorne motivator, as the world keeps an eye on how we're doing...