W3C creates the technical specifications regarded by the Web community at large as Web standards. In order for these standards to permit full interoperability and access to all, it is very important that the quality of implementation be given as much attention as their development.
As of the beginning of 2001, W3C has published more than 20 Recommendations. As our family of specifications gets more complex, their acceptance and deployment on the market becomes an ongoing challenge. The past experiences of HTML, CSS or more recently SMIL (all implemented with various degrees of conformance by vendors) are strong incentives to start this Activity with due diligence.
A message (W3C Member-only page) related to formalizing our work on QA at W3C was sent to w3c-ac-forum@w3.org in 1999 by Daniel Dardailler. The initial feedback from AC members on this list was very positive and led us to hire a Conformance Manager (Karl Dubost) whose role would be to organize and lead this Activity.
In April 2001, a W3C QA workshop was held at NIST which again showed the interest of the W3C community in this area.
This briefing package proposes such an Activity, with a focus on solidifying and extending current practices (existing validation tools, test suites, common test framework, etc.) and thinking ahead in terms of certification, education, funding, and tracking the quality of products and services related to W3C technologies.
We therefore propose the creation of a Working Group focused on building better specifications and test tools and of an Interest Group to share ideas on using these tools, building metrics, and communication.
There has always been and still is a strong demand from the Web community (end users, Web agencies, organizations, software developers, etc.) for better support and better implementation of W3C specifications in products (commercial or not).
All Web users have at one point or another experienced the problem of invalid pages rendered incorrectly on their platform or valid pages rendered incorrectly by their platform.
Most experienced Web authors have at one point or another been faced with the choice of either developing several versions of a page or picking up one particular solution detrimental to a portion of their audience.
The overall mission of a QA team at W3C would be to improve the quality of W3C specification implementation in the field. In order to achieve that, we will need to
Fortunately enough, we're not starting from scratch in this area.
QA is absolutely necessary in order to ensure interoperability and usability and also to have consistency between the specifications we produce. The QA Activity will benefit the Web community, industry, and Members, as a guarantee of interoperable products, which is the core mission of W3C.
QA applies to all W3C Activities. The QA Activity itself will therefore not be attached to a particular W3C domain but will exist independently as a cross domain Activity. Daniel Dardailler will act as Domain Leader.
The Activity will initially be chartered for two years.
The Quality Assurance Activity is dedicated to all aspects concerning the quality of the implementations of W3C specifications in products (commercial or not) as well as the quality of the documents we produce. W3C wants to improve the level of interoperability of software used to access (read/write) and serve the Web. The Activity should help the implementations of the W3C Recommendations with tools such as validators and test suites. The Activity will strongly encourage Working Groups and/or third parties to produce test suites. It should also focus on the quality of documents produced at W3C, the consistency of these documents, and to ensure levels of conformance are well defined in our Recommendations.
The QA Activity leader is Karl Dubost <karl@w3.org>, Conformance Manager at W3C. Karl reports to Daniel Dardailler <danield@w3.org> who acts as Technical Director.
The QA Activity will begin with two groups: a Working Group on specification and tools production and an Interest Group on certification, branding, education, and documentation.
The Activity home page will be http://www.w3.org/QA/
The mission of the QA Working Group is to organize and unify the work done in W3C groups in building and designing test suites and validation tools. It also defines the terms of QA, using a glossary, a taxonomy file, and also creates "how-to" guidelines for building test suites.
The list of activities foreseen in this Working Group are:
The main objective of the Working Group is to foster the development of usable and useful test suites endorsed by the W3C, which share a common look and feel, and ensure that the validating tools of the W3C are fully operational, useful and educational.
As a step in performing this task, the QA Working Group will need to work on conformance testing methodology, on the quality of the W3C specifications with respect to conformance and clarity, and the tracking of issues related to specification ambiguity and evolution.
The QA Interest Group's main objective is to have W3C, its Membership and the Web community involved in QA at large share their understanding of the state of affairs related to Web certification, branding, educational effort, funding model, etc.
The list of activities foreseen in this Interest Group are:
Certification and branding appears to be a key point in many organizations and governmental agencies. It helps the customers to identify the tools that produce documents of quality and it applies to browsers, as well as editors, software libraries, Web servers, etc. As of the writing of this proposal, certification is considered outside the scope of the Activity but we'll need to clarify what a certification system should do, who should run it, and how to interface with this market.
Although not a core Activity of W3C in general (except in the WAI domain), we think education and documentation are a necessity to reach the level of quality we want to see in the implementation of our standards. To start with, many Web developers, technical writers and/or software developers have no idea what W3C is, what a W3C standard is, and why it is important for them. The QA Activity will need to work in cooperation with communication team to promote the awareness of standards, tools to check them and the rationales for using them.
The following picture made at the QA workshop uses concentric circles to describe the kind of work foreseen in the Activity. The innermost circle (SPECS DEV: ASSERTIONS, REVIEWS, etc.) describes the work on improving the specifications, which serves as a basis for further test development. The next circle is about test development (TESTS DEV: GUIDE, CHANGE CONTROL, EXT COORD, COMMON HARNESS). Together they form the basis for the Working Group. The last circle gives ideas about the things we should be thinking in the context of the interest group (TESTS USE: METRICS, COMM, CERTIF).
We expect the two groups and the Activity resources in general to be publicly accessible.
All documentations, test suites, validating tools produced inside this Activity will have to be defined under a license. There are questions about the kind of license to use (for instance we have two licenses at W3C: one for document and one for software, the document license being more restrictive for the change control, which may be of interest to ensure the integrity of QA tools).
We expect the tools to be freely usable, runnable, and downloadable in any case.
Prior disclosure of intellectual property rights pertaining to QA applied to W3C technology will be required, following the IPR requirements defined in the W3C Process.
In addition to this core team, which we expect to grow over the life of this Activity, it's important to point that most of the W3C technical staff acting as staff contact for a technology will end up working "part time" for QA, when their technology is up on the QA agenda. E.g. X is not in the QA team, but when the QA team starts looking closely at a test suite for the specification he or she is dealing with, we expect X to free up some time/resource to coordinate with us, interface with people in the group, help us promote whatever W3C test guidelines we have come up with during WG meetings, if needed, etc.
We also expect staff resource allocation to the Technical Architecture Group (TAG) to be accounted as part of the QA activity.
All W3C group participants should realize that QA is to be considered a "natural overhead" of any WG, even if it's not written down in our process yet. QA will succeed only if every person inside W3C participates in it.
For participation in the QA Working Group, the requirements for meeting attendance and timely response are described in the W3C Process document. This participation (attending meetings, reviewing documents and preparing drafts) is expected to consume between one half to one day per week.
We expect and will be welcoming different communities to contribute to the Activity:
As this Activity has a clear multi-stakeholder approach, we expect to use multiple approaches to reach our goals, and not a fixed set of rules that would not be applicable to all participants.
Created: 2000/08/11 - Last modified 2001/04/23 by danield
Authors: Daniel Dardailler, Janet Daly, Karl Dubost
