Building QA Experience in the W3C
Rob Lanphier
Curtis Reynolds
April 4, 2001
Speed of the W3C
-
First formed as a reaction to vendors getting out in front of the specs
-
Very successful: W3C is seen as the authority on HTML, and not just HTML
-
Now specs have gotten out in front of implementations
-
CSS 3 Working group chugging along...fully CSS 1 conformant implementations
are still relatively small percentage of deployed browsers
Well-resourced W3C QA Activity Needed
-
Create economic incentive for companies to build conformant products
-
Actively lobby for conformance requirements in contracts
-
W3C Recommendations need to be authored in such a way that "the letter
of the law" is clear; QA Activity can police that
-
Working groups need to take seriously every new feature - they could be
held to implement them (may get quiet vendors to speak up)
-
Give recommendations "teeth"
-
Of course, vendors will need tools to get conformant....
Recommendations - QA Review Group
-
Reviews specifications in the same spirit as the WAI and i18n groups, but
focusing on testability and completeness
-
Submit guidelines for specification writers that could become part of the
mandatory process for W3C Recommendations.
-
Design watertight definition of the exit criteria for Candidate Recommendation
-
Different definitions are needed for modules intended for incorporation
in many specifications, and profiles which are intended to define document
types.
Recommendations - Certification program
-
Potential certification authorities apply to the W3C for approval to certify
implementations of W3C Recommendations (scoped to itemized Recs)
-
Appeals process to ensure that the W3C has final appellate authority
-
W3C would need ability to revoke specific certification stamps as well
as certification authorities
-
Certification stamps should never be permanent, only for fixed durations
(e.g. one year).
Recommendation - Certified Recommendation Phase
-
Higher step for specs with long-term experience and proven robustness and
applicability
-
No sooner than 18 months after the W3C Recommendation is approved
-
CertRecs could pull features from the preceding W3C Recs that don't have
two certified implementations
-
Participating vendors hoping to get certified would be more careful about
adding features
-
More vendors would be motivated to get involved to ensure that poorly designed
features don't make it into the specifications
-
Vendors motivated to get certified so that pet features have higher odds
of making CertRec
Testcase/testplan markup language
-
Would be great to standardize
-
W3C may be the right place to do it
-
Being a heavy customer makes the organization qualified
-
Can be considered a higher level activity (though could be at same level
as XSLT)
-
Cautionary note: this is a tool toward a goal - shouldn't get so busy working
on markup language that other QA needs don't get addressed
(more dead horse beating) More formal issue tracking on suites and specifications
Needs to have formal steps for disposition of comments
Issue tool may be the only manage this properly
-
Otherwise, those that submit issues have to have deep knowledge of the
process (know their rights)
-
Auditability
QA Activity members may be best qualified to design workflow
Vendors must support backwards compatibility with their old products
Imperfect products happen
-
"The perfect is the enemy of the good"
-
Invalid documents may be allowed by otherwise good user agents
Old documents must survive - should care more about substance than grammar
-
Often authored using plain text editor and non-validating user agent
-
Declaration Of Independence should still be viewable even though there
are grammatical errors by today's standards.
...but there are tools that can be used
Aesthetically unpleasant element can be shown when document is not compliant
(iCab frown face is one way; blinking frown face may be even better)
Vendors can withhold new features from non-conformant documents
-
Specification features can help with this process - strong document versioning
very handy tool
Validators can be built into user agents
W3C needs to recommend rational behavior when documents don't conform
-
QA Working Group should develop W3C Recommendation which recommends bad
document handling behavior, using these points and Common User Agent Problems
note (http://www.w3.org/TR/cuap)
as starting point