Software Testing – The Art of Proving The Presence Of Bugs, Not the Absence

Microsoft just published a critical patch for a 19 year old bug that dates back to Windows 95 and Internet Explorer 3.0.

First the obvious – since it was still there after 19 years, all the testing that Microsoft and users have done on every version of windows back to and including Windows 95 did not detect this bug – hence the title of the post.

But you might ask WHY was this bug not detected and Network World published an item that discussed that, but here are a couple of reasons –

• The person that wrote that hunk of code is no longer with the project or company and no one else understands it, so lets leave it alone.  It ain’t broke
• Supposedly, it is a subtle bug and hard to exploit, so you might have to look real hard to find it (not any more, of course)
• Didn’t all that old code base go away with Vista/Win7/Win8?  It was 16 bit code and we moved to a 32 bit code base?  Nope, it wasn’t broke, so we just recompiled it.

The article gives some other reasons too, but this doesn’t mean that you should not test.  In fact, if anything, you need to expend more resources, automate the testing, pay bug bounties, etc.  It just means that testing is hard.

What this also means is that since this bug is now in the wild and Microsoft did not issue a patch for Windows XP, if you are still running XP, here is another reason to migrate – the bad guyss now have bug, they know what Microsoft did to fix it in newer OSes and all they need to do is figure out a way to exploit it in XP.

Mitch Tanenbaum

by