Measurement Reprise

Looking back on my initial list of what productivity-related metrics I hoped to measure makes me smile a bit: "time spent planning, time spent writing the functionality as originally spec'ed, time writing unit tests, time spent reworking functionality based on spec changes, time spent reworking tests based on spec changes, and time spent fixing bugs from the QA process."

It turns out it has proven far harder to measure any of these steps simply because their beginning and end points are so nebulous.  Despite all the projects I've been involved with, I had the naive assumption that there were discrete steps to all of them.  In fact, there are rarely clear-cut boundaries.  Are planning and coding really that separate?  Isn't coding in many ways just a more detailed, explicit form of a plan?  And a good approach to unit tests involves writing them in parallel with the main code, not as a separate, isolated step.  And in an agile environment, even the step between development and QA can be easily blurred, with bug-fixing and continuing development happening in parallel.

On top of that, it's just a mountain of information to keep track of.  It would be a full-time job just cataloging all those numbers, let alone analyzing them.

So I had to take a step back, look at our process, and think about what minimum amount of information could be recorded alongside it in as unobtrusive way as possible (after all, if measurement itself reduces productivity, it's hardly useful).

What I came up with was a simple addition to our check-in process: in addition to asking for a task or bug ID, which is already part of our check-in template, I added two lines: "hours spent" and "lessons/roadblocks".  "Hours spent" is a rough guess at how many hours the coder had to spend on the files before getting to this point of checking in the code.  I'm not asking people to keep timers running or start work diaries.  I just want a rough guess as to how much time was involved.  And "lessons/roadblocks" records any lessons learned or roadblocks encountered along the way.  The log message is parsed and the information is put into a database for later analysis.

I figure that if I can get developers to just spend a few more seconds recording that data as they check in code, we'll have a relatively painless way of collecting lots of useful information.

There's a risk that the information will be so unspecific as to be useless.  But I'm hoping that by analyzing the numbers as a group, the team will start to focus more on ways to increase its output.