Nothing is perfect (and that is why we have QA)

Brian Kelly, of UK Web Focus, gives his thoughts about bugs in validators, in an article entitled Validators Don’t Always Work.

I was very impressed with the speed with which this problem was addressed and a solution deployed.

[...]

I was, though, also very shocked that a validator for such a widely deployed standard (RSS 1.0) had such bugs

Brian Kelly's surprise is understandable: the Feed validator, where the bug was found and fixed, is an excellent and reliable piece of software, and is thoroughly tested. It is relied upon by many people, including us at W3C, where we are hosting an instance for the Web community to check their feeds.

Finding a bug in such excellent software is, however, not a problem. It is, rather, a solution to an unavoidable problem: nothing is perfect. Nothing. Even the best products, software or others, have faults. And this is precisely why there is a Quality Assurance process. To ensure products get released with a minimum amount of bugs, and to guarantee that bugs found after release can be fixed, if possible in a timely manner and never happen again.

The way a bug was found and fixed in the Feed Validator is not disturbing, I actually think it was an inspiring proof that all the aspects of its QA process worked:

1. There is a public feedback channel (several, indeed) for a problem to be reported to: Brian sent a message to the W3C QA mailing-list, asking whether someone could make sense of the problem he was facing
2. The tool is implementing public specifications: I was able to look at the RSS taxonomy module spec, and compare its prose with the implementation in the Feed validator
3. The validator is open source: 10 minutes of browsing around clear code was enough to find the issue, and create a patch, which was promptly reviewed and applied by Sam Ruby, one of the maintainers of the validator.
4. There is a test suite, to which I submitted a revised test case: now that the bug is fixed, we know that it will never appear again without being spotted automatically.

Time between original feedback and applied patch: about 24 hours.

There is nothing shocking about this bug, but the speed at which it was processed and fixed. Maybe that was lucky, I just happened to have a bit of time to look at the issue and the bug was easy to fix. Other, more complex bugs in tools that we (W3C's QA Tools development effort) maintain are not so lucky, and indeed Brian is right in pointing out that we could use more help and resources to make our tools better. But I can not agree with his slightly provocative title that Validators Don’t Always Work. The Feed validator works, and so does its Quality Assurance process, as demonstrated in the prompt fixing of a small bug in its implementation of a faulty, not widely used specification.

Validators are extremely important tools for the adoption of technologies, and it is perfectly normal to be concerned about their quality. This is why finding bugs is good news, and the best use of one's energy is not to worry about them, but to help find them, report them, patch them and build regression test cases for them.

Nothing and noone is perfect, that's why we have QA.