Defect Tracking Tools Sometimes Inhibit the Scientific Method

15 04 2010
Most defect tracking tools are not made for investigating defects.  They are essential for cataloguing the solution to a problem and  track its integration into the code base.  This helps managers review proposed changes, track defect density by various categories, monitor employee workload, and report to their keepers what all of these bugs are going to cost.  This is only the small list of all the useful statistics a well stocked defect tracker can provide to your manager.
However, I have found these tools a burden for the developer.  For example, can they record symptoms without a known cause?  Frequently, the same defect may cause multiple symptoms or multiple defects may result in the same symptom.  None of this is known at the time the symptom is observed, but defect trackers want to bind a single set of symptoms to a single cause.  Their capabilities for merging or linking multiple entries are crude at best.  When such capabilities are provided, they are again oriented more for the management viewpoint than the engineering viewpoint.
The same is true for a defect tracker’s search capabilities.  If I want statistics, then there is no end to the ways a sophisticated tracking tool will allow me to cut and sort data.  Do you want to know how many defects the code base acquire between March 11 and April 17 of last year that cost more than 40 hours to fix and resulted from requirements modifications?  I can get you that in a second.  Did you just encounter a symptom that vaguely resembles a something you saw a year ago?  Do you want me to determine whether or not the conditions under which that symptom was originally observed are the same as the conditions under which you are now operating?  You’re out of luck.  Defect trackers are, after all, mostly a webby veneer stretched over a bunch of RDB tables that are hidden from the user. All that information about the actual test case was shoved into the text of the defect description and we have no way of codifying test scenarios the way we would if both managers and tools developers treated us like real scientists.  We aren’t, of course, but that’s no reason not to aim high.



One response

23 04 2010
Nitride LED and Laser Technology and Its Impact on the Display | Best Bagless Vacuum Cleaner

[…] Defect Tracking Tools Sometimes Inhibit t&#1211&#1077 Scientific Method « Candid Folly […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: