Wednesday, July 12, 2006

Triage

The other thing we did today was to pause and triage all of the bugs that have stacked up in the system. There were a bunch of duplicates, and lots of problems that could be downgraded in priority, and I was reminded that this is one of the rare occasions where I disagree with Joel.

Item 5 from the Joel Test talks about whether you fix bugs before writing new code: "at any given time, the highest priority is to eliminate bugs before writing any new code".

This is Simply Wrong.

On any significant codebase, there are whole classes of problems that aren't worth the effort of fixing. Pretty much every bug tracking system has a Priority field which is used to set exactly this dividing line. The triage isn't an exact science, but the division can and should be re-adjusted later depending on customer feedback.

Priority is distinct from severity, which describes the impact on the product or the customer. It's possible to have bugs that are very high-severity but which are still low-priority: for example, a comprehensive system crash that only occurs when the epoch rolls over. Conversely, a glaring typo on the front of the UI would probably be high-priority, low-severity.

I once worked on a product that shipped to customers with hundreds of bugs from our internal testing still open in the database. Three years of real-world use later, the number of bug reports that had come back from the field was in the (low) dozens—and not one was a duplicate of an existing bug from the internal test phases. (Which probably implies that our internal testing was probably more thorough (read: expensive) than it should have been).

No comments: