Copyright ©2002 W3C ® (MIT, INRIA, Keio), All Rights Reserved. W3C liability, trademark, document use and software licensing rules apply.
This document defines a set of common guidelines for conformance test materials for W3C specifications. This document is one in a family of Framework documents of the Quality Assurance (QA) Activity, which includes the other existing or in-progress specifications QA Framework: Introduction,QA Framework: Operational Guidelines, and QA Framework: Specification Guidelines.
This section describes the status of this document at the time of its publication. Other documents may supersede this document. The latest status of this document series is maintained at the W3C.
This document is a W3C Working Draft (WD), made available by the W3C Quality Assurance (QA) Activity for discussion by W3C members and other interested parties. For more information about the QA Activity, please see the QA Activity statement.
This version is the first published Working Draft. It is expected that updated WD versions of this document will be produced regularly, along with other members of the Framework documents family. Future progression of this document beyond Working Draft is possible, but has not yet been determined.
This part of the Framework document family will eventually have an informative accompanying QA Framework: Test Examples and Techniques document. It will illustrate ways in which the guidelines and checkpoints of this document might be satisfied.
The QA Working Group Patent Disclosure page contains details on known patents related to this specification, in conformance with W3C policy requirements.
At this time, the QAWG (QA Working Group) has addressed the contents of the document at a high level, agreeing to the concepts and principles as well the coverage of the guidelines. The QAWG has not, at this time, addressed and achieved consensus of the priority levels. A future version of this document will be accompanied by a "Specification Examples & Techniques" document, which will illustrate the guidelines and checkpoints with case studies, and explain how to satisfy the checkpoints.
Please send comments to www-qa@w3.org, the publicly archived list of the QA Interest Group [QAIG]. Please note that any mail sent to this list will be publicly archived and available, do not send information you wouldn't want to see distributed, such as private data.
Publication of this document does not imply endorsement by the W3C, its membership or its staff. This is a draft document and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use W3C Working Drafts as reference material or to cite them as other than "work in progress".
A list of current W3C Recommendations and other technical documents can be found at http://www.w3.org/TR/.
This section contains the definitions for all the critical terms used in the guidelines below. This does not substitute the QA Glossary [QA-GLOSSARY], but rather focuses on the most important terms for the Testing guidelines. Some terms in this section have been borrowed or adapted from other specifications.
Extract assertions/normative langauge and tag according to catigory - Using catigories provided by patrick In otherwords rather than having explicit checkpoints for each required, optional, depreciated, discressionary, underdefined, etc asserts, have a checkpoint that has all asserts or normative language extracted and then grouped by catigory. It's the same basic idea but it takes 4 checkpoints into one.
This section defines conformance of Working Group processes and operations to the requirements of this specification. The requirements of this specification are detailed in the checkpoints of the preceding "Guidelines" chapter and apply to the Working Group QA-related documents and deliverables required by this specification.
This section defines three levels of conformance to this specification:
A Working Group conforms to the "QA Framework: Test Guidelines" at Level X (A, AA, or AAA) if the Working Group meets at least all Conformance Level X requirements.
To make an assertion about conformance to this document, specify:
Example:
The Test Suite for X module, version 2.1 of this Working Group, conforms to the W3C's 'QA Framework: Test Guidelines' version 1.0, available at http://www.w3.org/TR/2002/qaframe-test/, Level AA as determ ined on January 1, 2003.
The checkpoints of this specification present verifiable conformance requirements about the quality of the test materials developed or adopted by the Working Group. As with any verifiable test requirements, users should be aware that:
The following QA Working Group and Interest Group participants have contributed significantly to the content of this document:
Added new outline to document commented out a number of sections that need editing.
Edited, improved the Introduction(goals, motivation, document's structure
Updated the definition of the checkpoint's Priorities
corrected abstract, SOT
Changed the goal of the document and wording of the checkpoints/guidelines to focus it on testing strategy, moving all the tips on tactics to be EX-TECH
Added examples to most of the checkpoints
incorporated all the editors comments
Rewrote several checkpoints in the Gd 5 - defined results verification and reporting as part of the test framework to remove redundant checkpoints
Rewrote guideline 6 and 7 to focus on strategy of test development and testing, rather then on tactics.
Updated the conformance section
Expanded introduction, added motivation, etc...
Added examples to the checkpoints in the Gd1,2,3
[MS] Changed the text of many checkpoints to make them verifiable
[DD] First pass on Introduction, added more text to the checkpoints in the Gd 3-5
Fixed definitions of priorities
Fixed the glitch with the "Test Areas" guideline
Added clarification to Ck 1.1, 1.2, 1.5 (removed "vague"), 1.6
Added short prose to each checkpoint
First draft outline