The QA Handbook (QAH) is a non-normative handbook about the
process and operational aspects of certain quality assurance practices of
W3C's Working Groups, with particular focus on testability and test topics.
It is intended for Working Group chairs and team contacts. It aims to help
them to avoid known pitfalls and benefit from experiences gathered from the
W3C Working Groups themselves. It provides techniques, tools, and templates
that should facilitate and accelerate their work. This document is one of the
QA Framework (QAF) family of documents of the Quality Assurance (QA)
Activity. QAF includes the other in-progress specification,
Specification Guidelines, plus a handful of test- and other
QA-related notes, advanced topics, and Wiki page collections.
This section needs to be updated when minor editing is
done.
This section describes the status of this document at the time of its
publication. Other documents may supersede this document. A list of current
W3C publications and the latest revision of this technical report can be
found in the W3C technical
reports index at http://www.w3.org/TR/.
This document is a W3C Working Group Note, made available by the QA Working Group of the W3C Quality
Assurance (QA) Activity for discussion by W3C Members and other interested
parties. For more information about the QA Activity, please see the QA Activity
statement.
This is the third publication of this document, and the first as a Working
Group Note. Further development and publication is anticipated. Interested
parties are asked to submit examples of the Guidelines and Good Practices,
for inclusion in future versions. The development of the templates linked
from this document will be finished. The QA Handbook will
continue to be coordinated with the other parts of the QA Framework.
Publication as a Working Group Note does not imply endorsement by the W3C
Membership. This is a draft document and may be updated, replaced or
obsoleted by other documents at any time. It is inappropriate to cite this
document as other than work in progress.
Comments on this document are welcome, and may be emailed to www-qa@w3.org, the publicly
archived list of the QA
Interest Group. Please note that your comments will be
publicly archived and available, do not send information
that should not be distributed, such as private data.
- Introduction & roadmap
- Early planning and commitment
- Day-to-day QA operations
- Licensing & branding
- Acquiring test materials
- Acknowledgments
- References
- Change History
List of Guidelines and Good Practices:
This document has two separate external appendices, QA Process Document
template and Charter template,
which contain templates to help Working Group Chairs and Staff Contacts to
implement the document's good practice guidelines.
The following five use cases for The QA Handbook (QAH), told as stories, show when and how
QAH could be helpful. They cover
five different situations that might typically arise over the life of the
Working Group.
The QA Handbook could be useful to Chairs and Staff Contacts
at Charter time...
Story 1: Think about quality early
The Charter-drafting team of a new Working Group, led by Staff Contacts and
co-Chairs, looks through The QA Handbook. Following the good
practices in the early-commitment and
planning module, the team debates how much it can realistically commit to
at this early stage. After deciding on some test materials deliverables,
milestones, and specification synchronization that it considers to be safe
commitments, the team uses the provided Charter Template to include these in
the draft Charter.
Or it could help a Chair who is trying to jump-start a test suite
project...
Story 2: Jump-starting a testing
effort
An existing Working Group has just finished writing its Requirements and Use
Cases documents, and has begun to draft its specification. At the same time,
it is taking a first look at its test suite plans. As recommended in the
The QA Handbook, the Chair jump starts the Test Suite project by
appointing a point of contact and part-time
Test Suite team, whose first output is a quick QA Process Document (QAPD)
using the provided QAPD template.
Or maybe the Chairs and Staff Contacts are pondering the transfer of a
test suite from an external entity...
Story 3: Test-suite transfer
A Working Group has re-chartered to finish a Second Edition of its
specification, and to develop the next functional version. The group did not
develop a test suite during its first charter, but a collaboration of outside
organizations and an industry consortium has developed one. The Chair and
Staff Contacts have negotiated an agreement in principle to transfer the test
suite to a stable home in W3C. In The QA Handbook, they find a
handy checklist of preliminary steps,
requirements, and specific activities for a smooth transfer.
Or maybe they need to take preemptive action due to looming possible
license-issue hassles...
Story 4: Test suite licensing
A Working Group is almost ready to request Candidate Recommendation (CR), and has gotten a
comprehensive test suite together for the CR's trial implementation
period. As the Chair starts to arrange for publication of the test suite, she
finds the Working Group split on which test suite distribution licenses to
use. Consulting The QA Handbook, she finds discussion of the pros and cons of the W3C licenses (the Software
License and the Document
License), and advice on devising an optimal licensing strategy.
Or maybe they can borrow the experience of other W3C Working Groups for
various useful and common test suite processes...
Story 5: Test development processes
A Working Group has built and released a basic test suite for its
specification. A Staff Contact has been given the Action Item to plan its
expansion to a more comprehensive test suite, by leveraging and integrating
the large test collections of several early implementors. Rather than figure
out the issues and write a Test Contribution & Review process from
scratch, he looks at the summary advice in
The QA Handbook. QAH
points him both to some useful templates, and to more detailed stuff in
collected test-guidance Wiki pages [TEST-GUIDE], significantly shortening his effort to
complete his action item.
Chairs and Staff Contacts are overworked. They share a lot of the same
concerns and pressures as specification
editors (see 2.1, "Specification editors").
Common problems include:
- never enough motivated people to do the work;
- deliverables delivered late;
- licensing and legal hassles;
- too much to do, so little time
A lot of groups have faced these issues. As a result, there is a growing
body of experience in W3C about what works and what doesn't. QAH aims to pull this together.
The QA Handbook (QAH) is intended for: Chairs and Staff
Contacts of W3C Working Groups. These include both the process-savvy as well
as the novices.
QAH is a practical guide about
applying good practices and quality assurance techniques to WG activities,
especially developing Recommendations and test materials. It flags avoidable
pitfalls, and provides techniques, tools, and templates that will facilitate
and accelerate the work of the groups. Guidance in QAH is supported by
real-world stories and examples.
There is no normative content in QAH, therefore conformance to QAH is not an issue. QAH takes this approach because:
- the diversity amongst W3C's working groups makes it difficult to craft
simple normative rules for all;
- in diverse, people-centric processes, it is more effective to encourage
and help than to mandate and proscribe.
Advice is presented in the imperative voice as Guidelines and
Good Practices. Guidelines tend to be general or high level
encapsulations of a given topic area, and good practices tend to be more
low-level, actionable details.
These suggestions will generally be helpful and enhance the quality of the
work of a Working Group. However, each suggestion should be applied or not
depending on the specific situation of the WG.
The QA Handbook (QAH) is a non-normative handbook for
integrating quality assurance techniques and good practices into the Working
Group. It focuses especially on factors affecting development and
implementation of specifications and tests, including:
- people resources;
- deliverables and their schedules;
- interaction with W3C and the world;
- W3C process, licenses, etc.
Better specifications and effective test development are emphasized goals
in QAH and its companion documents. In support of those goals, the scope of
QAH goes beyond a narrow focus on authoring topics, and touches on many
aspects of the broader context in which Working Groups operate -- from
writing charters, to the logistics of managing the group's quality practices,
to interacting with other groups and the public, to completion and
maintenance of the Working Group's deliverables.
The QA Handbook belongs to a document family collectively
called the QA Framework. The other parts of the QA
Framework are:
- QA Framework: Specification Guidelines [QAF-SPEC]
- Variability in Specifications [VIS]:
advanced topics in support of Specification Guidelines
- QA Framework Primer & User Scenarios [QAF-PRIM]
- Test guidance: Collected Wiki pages on test-related topics
[TEST-GUIDE]
Editors, test builders, and W3C members in general, in addition to Chairs
and Staff Contacts, will find advice specific to their roles in these other
framework parts. The Primer provides a brief orientation to the
whole QA Framework.
Other valuable documents about quality assurance techniques and good
practices are indexed by the QA Activity, outside of the QA
Framework.
After this orientation section, QAH contains four topical modules:
Each presents advice and guidance about tasks that
contribute to the WG's realization of quality specifications and tests. These
tend to have a chronological correspondence to phases (rec-track stages) of
its activities. Earlier is better, but later is better than never.
Why care?It's not always easy to
anticipate Working Group needs during the rush and pressure of Chartering, or
in the busyness of early Working Group life. But the earlier the group can
accurately identify and commit to its test and other quality related
deliverables, the less likely it is that the WG will get major surprises
later on, or run into problems that will delay its progress towards
Recommendation.
Additional reason — binding decisions about participation (e.g.,
numbers per company) often are made at Charter time.
What does it mean? In practice, this
means attention to a handful of points, which we enumerate in the following
series of Good Practices.
Good Practice 1: Decide as soon as
possible — will the Working Group build test materials or acquire
them?
Why care? Clearly, it is going to make a
big difference in a Working Group's staffing requirements — building
test materials tends to be labor intensive (extremely so for some types of
specifications). Even if the group imports them, some resources will have to
be applied (see the final module). The
particular test-related activities and milestones in its program of work will
in general be completely different for build versus acquire options.
Good Practice 2: Think about and
enumerate the quality-related deliverables
that might help the Working Group through the Recommendation track.
What does it mean?Minimally, the
Working Group should commit to assuring the timely existence of test
materials. Test materials provide for the evaluation of an
implementation against the requirements of a specification and/or provide
information about the implementation. Conformance Test
Materials are test materials that are used to indicate conformance
of an implementation to a specification. Note that all test materials are not
conformance test materials. For example, some Working Groups using a
test-driven specification development process emphasize tests of new,
changed, or contentious functionality.
Test Materials include:
- test suites (tests, documentation and harness);
- validation tools;
- test assertions;
- sample code (in some cases);
- checklists;
- Implementation Conformance Statement (ICS) pro-forma.
Quality-related
deliverables include:
- test materials;
- systematic, thorough reviews of specification;
- sample code (if not considered test material);
- model specification in formal language.
Test materials are by far the most common quality-related deliverable, and likely will be a
key to quick and painless passage through Candidate Recommendation (CR).
Good Practice 3: Synchronize quality-related deliverables and their development
milestones with specification milestones.
Good Practice 4: Consider whether
the Working Group should bind any quality
criteria to Rec-track advancement.
What does it mean? For example,
finishing test materials deliverables before requesting Candidate
Recommendation (CR) is
important, in order to facilitate CR's Implementation Assessment.
(Counter-example: as in the above
Story, if the group is still building them during CR, it is likely to uncover things
that will oblige it to cycle in CR.)
This good practice takes the synchronization of deliverables of the previous Good Practice a step further
— the specification may not advance unless the committed test or other quality criteria are met.
Good Practice 5: Put some thought into how
to staff the Working Group's test and other quality
assurance plans.
Why care?The earlier this is done, the
more options will be available. Charter-time staffing provisions of W3C Process, by the way, provide
another good reason to put some thought into test suite plans and other
quality-related deliverables as early as possible.
What does it mean?Some options include:
- dedicated team from Working Group participants;
- part of everyone's time;
- Consider: asking in Call for Participation for extra people who would
specifically be tasked to work on test and any
other quality-related tasks.
The third option is not really different from the first. It's just a way
of doing it. But notice that it's an option that is only available by looking
into these questions at Charter time.
By the way, there has been confusion
about "W3C Process only allows each company to have two participants on the
Working Group". In fact, that is not from W3C Process. W3C Process gives
considerable freedom for this to be tailored to Working Group needs —
W3C
Process says it may be specified in the Charter. So, for example, the
group could decide on these rules: allow two per company in technical
discussion, issue resolution, voting, etc; and, allow additional dedicated
test suite builders.
Tips for
Getting to Recommendation Faster [REC-TIPS] (section 3) also talks some about the value of
(early) staffing decisions.
Techniques
- The Charter template
provides for a superset of the things that one might want to specify and
commit to early (at chartering time). Use it for those items that are
deemed appropriate to write into the charter.
- While often "earlier is better", Charter time may not be appropriate
for each and every one of the above Good-Practice recommendations, for
every Working Group. In some cases, it might be premature to commit to
that level of detail at that early stage of the group's life.
Why care?Over and over we hear the message
from successful test suite projects — define the work process early:
test development framework, issues list, email list, faq, contribution &
review process, etc. Conversely, there are examples of disorganized
collections of test cases with no one apparently in charge, no way to find
their status, or verify their correctness.
What does it mean? The W3C Process Document
clearly organizes the way that a Working Group's specifications are
progressed through the Rec track. But there is no similar template for the
other aspects of a Working Group's life, particularly aspects related to
test and other quality assurance processes
— logistical setup, communications with the outside, licensing and
branding policy, test development process, etc.
Examples
DOM has a thorough test suite
process document referenced from its /Test/ Web page. Starting at the Test
page, one finds complete information about communications and logistics,
licensing, submission and acceptance procedures, test materials location, and
test suite status.
Starting at the CSS /Test/
page, one quickly finds the location of CSS test materials, excellent test
suite documentation for users and contributors (includes templates)
alike, and an additional test authoring
guidelines document. (Some operational details -- e.g., communications
and licensing -- are less easily located.)
Good
Practice 6: Put all of the Working Group's important test and other quality-related information in one
place in a QA Process Document.
What does it mean? In practice, it
means producing a QA Process Document recording at least those details of the
Working Group's test and quality assurance work
processes that are outlined in the template and briefly discussed in
the following subsections.
Techniques
The technique is simple: Use the QA Process Document
template. It will guide the user through everything needed, and then
some. It is not only a template, but also a checklist of sorts, for the sort
of things that the Working Group should consider having in its QA Process
Document. The template has been assembled as the union of good practices seen
in real QA Process Documents of W3C Working Groups.
In the past, before this template, test efforts would often copy-edit an
existing process document from another effort. There is not really any reason
to do this anymore. Here are some examples from which the template has been
assembled:
Examples
The very similar
XML
TS process document reflects a copy-edit by the XML TS development team
(significant membership overlap with the DOM development team).
See the corresponding
section in the template.
Good Practice 7: Identify a Working
Group point-of-contact for test materials or other quality-related business.
What does it mean? This can be a
special appointed "QA point-of-contact". Or it might be the Chair (or a
co-chair), or Staff Contact. The important part is that there be an
identified point-of-contact to whom others can address questions, bug
reports, contributions, etc, and who will be accountable for responding.
See also the QA
Process Document subsection in the template.
Good Practice 8: Specify an archived email
list to use for quality-related
communications.
What does it mean?It can be a dedicated
"Test" list. Or it can be a public Interest Group list. Or it can be a public
comment list separate from the Working Group list, in the case that there
isn't an Interest Group. Because of the variety of public/private mixes
amongst W3C's Working Groups, "one size fits all" does not work well here.
The list should be available for all quality-related topics, including
test suite questions, bug reports, contributions, etc.
See also the corresponding
QA Process Document subsection in the template.
Examples
The
SVG test pages
identify a public test-topics mail list (svg-testsuite-comments@w3.org)
prominently at the beginning.
Good
Practice 9: Identify Web page(s) for test suites, announcements,
and other quality-related topics.
What does it mean?Obviously, this needs
to be publicly accessible. Doing it all in the public portion of the Working
Group's Web space is one way to achieve that. In particular, that provides a
good, secure repository location for test materials.
Some groups, for example SVG, have two locations — a
private CVS repository for the in-development test materials, and a
simplified public location
for periodic public releases of test materials.
Techniques
Link the test pages from the Working Group's home page, and use the common
convention of putting them in the /Test/ directory under the Working Group's
home page.
See also the corresponding
QA Process Document subsection in the template.
The Working Group's QA Process Document is also a handy central location
to record its licensing policies and (any) branding policies. These are
sufficiently important that they have their own
QAH module, which
addresses:
- license terms applicable to test materials, both submitted and
published;
- any branding policy — logos, conformance icons, etc — and
associated rules and restrictions.
Examples
SVG uses a helpful convention for its
test suite pages -- the URL
is the
SVG home page URL
(http://www.w3.org/Graphics/SVG/) with /Test/ appended. Additionally, the
test page is prominently linked from the
SVG home page.
The
XML test pages follow the same
useful convention -- their location is a /Test/ directory under the WG home
page.
If the Working Group itself is developing test materials, then there are
several topics associated with test materials development process that are
going to impact its operations and management. Useful guidance and discussion
for these may be found in QA's collection of test-related Wiki pages [TEST-GUIDE]. These topics, which might
conveniently be documented in the Working Group's QA Process Document
(QAPD), include definitions
of:
Finally, as a part of
planning about "life after Working Group", the group will need to decide what
happens to both its test materials and associated processes. Discussion of
maintenance-related topics may be found in QA's collection of test-related
Wiki pages [TEST-GUIDE]. Again the
information might conveniently be located in the group's QA Process
Document:
- what happens to the test materials themselves?
- what happens to the Working Group's contribution and review processes?
[QAPD sub-section]
- how to track errata and (for example) new specification editions? [QAPD sub-section]
- what happens with bug reports about the test materials? [QAPD sub-section]
Why care?Get it right early, or it may
stall the Working Group's Rec-track progress indefinitely. While this might
seem to be a routine piece of Day-to-day
operations, it has proved to be sufficiently troublesome within W3C that
it deserves to be a Guideline by itself.
What does it mean? There are two kinds
of licensing issues: submission license, and publication license. Both of
them can be problematic and can interrupt the Working Group's progress on
Rec-track if not early addressed and carefully handled. Test materials
license issues are the subject of ongoing debate and discussion within W3C,
but there are some tactics to minimize potential problems.
Good Practice 10: As early as
possible, get agreement about acceptable license terms for submission of test
materials.
What does it mean?W3C now has an
official set of comprehensive policies for contribution
of test cases to W3C. (Formerly, it had a more or less routine submission
license, Contribution
of Software or Test Materials to W3C [CONTRIB], but without policy.)
By early attention, it should become clear whether any potential test
sources have problems with the standard terms, before possible disputes can
impact the Working Group's deliverables and schedules. There have been cases
where potential submitters would not accept the standard terms.
Good Practice 11: As soon as the
nature of the Working Group's test materials becomes clear, get agreement
about license terms for publication of the test materials.
Why care?Any W3C-hosted materials must
have approved license and use terms. Experience has shown that there is no
single license that is appropriate for all parts of all test materials, so
the Working Group needs to address this after it has come to an understanding
of the structure, content, and intended use of its test materials.
What does it mean?Currently approved
W3C licenses that may be applied to test materials are the Document
License and the Software
License. The Document License has two characteristics that are attractive
for test materials:
- It prohibits the test materials from being modified by the user upon
download, therefore guaranteeing the integrity of the test materials;
- It requires that if the test materials are copied or redistributed,
they must contain the link to the original test materials and their
status.
On the other hand, there are situations in which the Document
License is inappropriate, because (for example) some parts of the test
materials require modification or completion in order to apply them.
The W3C Advisory
Board feels (member-only link) that the Document License is the
appropriate license for test cases.
That ruling notwithstanding, test materials might contain some mixture of
different components: test software, test documentation, and test cases. In
some cases, one license may be appropriate for some component(s) of the test
materials, but another license may be better for other parts. A careful look
at the test materials' composition and use cases should reveal what is the
best solution.
The Working Group should consult with W3C Legal if it believes that:
Good Practice 12: Decide policy about
having brands, logos, or conformance icons associated with the Working
Group's test materials.
What does it mean?There are two
questions that the Working Group will face. First, whether to have logos,
icons, or brands associated with its test materials. If that answer is yes,
then it will have to address a series issues about proper usage of the
logos.
W3C Logo
and Icon Usage contains a complete catalog of the logos in use
within W3C. Of particular interest (related to conformance) are the logos
associated with content validation (HTML, CSS, etc), Web Accessibility
conformance, etc. These logos have associated issues of veracity and
enforceability of conformance claims. A Working Group considering its own
logo or brand will face similar issues, amongst which are:
- terms for valid application of the logo;
- relationship of the logo to conformance claims;
- challenge procedures for disputed logo usage;
- policing & enforcement policy.
Techniques
Two final comments about both the topics of licenses and logos/brands:
- There are places in QA
Process Document template to document the Working Group's decisions,
about licenses as well as branding/logos.
- Don't forget to make license terms obvious and visible in all
deliverables (and all their bits and pieces)!
Additional History:For those who are
interested in more background and detail on the license issues, there is an
extensive history:
Caveat. These legal and licensing
topics have been very active within W3C. Some of these references in this
present QAH version may be
superseded by future, newer advice or policies.
What does it mean? The three main
problems are:
- license issues around test materials
- human resources
- quality of the test materials
The license and human resources issues are similar in concept to what the
Working Group faces when building test materials, but probably lesser in
degree. A pre-transfer quality assessment might seem unique to the acquire
option, but the actual steps will probably look similar to a test case
contribution/review process in a build-your-own scenario.
Good Practice 13: Do a quality assessment
of proposed test materials before going any further.
What does it mean?Basic things that a
good quality assessment might cover would for example include:
- clear declaration of scope,
- traceability,
- atomicity,
- user documentation.
A more comprehensive list of things that an assessment process could add
to that basic list:
- maintainer documentation,
- correctness,
- completeness (vis à vis declared scope),
- harnesses or interfaces for application of the test materials,
- reconfigurability,
- results assessment,
- results recording & reporting,
- automation features,
- versioning/errata support,
- declaration of publication licenses,
- integrated submission procedures.
Good Practice 14: Ensure there are enough
human resources to support the transferred test materials.
What does it mean?A test materials
manager (possibly one of the job functions of the QA point of contact) is
still needed, but total human resources ought to be considerably less than
build-your-own scenarios. With luck, the original manager of the external
test materials source might become a participant in the Working Group after
the test materials are transferred.
Good Practice 15: Sort out licensing issues
with the external party that produced the test materials.
Techniques
The following is a virtualization of an actual transfer scenario that QA helped to moderate.
It could serve as a checklist of steps to consider for Working Group's taking
the acquire route.
Legend: EG the
external group or entity; QAWG the QA Working Group;
TM the test
materials; WG the
Working Group acquiring the test materials.
- Before the transfer, WG with
the help of QAWG:
- performs an assessment of the quality of candidate test materials (by WG, QAWG)
- identifies and commits to a set of test-related deliverables from
the candidate test
materials. These could be: releases, appeal/challenge
processes, maintenance plan, submission/review process, Web site,
mail list, etc. (by WG)
- identifies sufficient staffing, including at least a test materials manager.
Recommendation: recruit the test materials manager from the
EG (if one exists) to
become a WG participant
after the transfer. (by WG)
- makes the decision to seek/accept the transfer. (by WG)
- (potentially) initiates Charter amendment (by WG, QAWG may consult), if the test materials acquisition doesn't
fit within the current Charter.
- During the transfer:
- EG and W3C reach
agreement to transfer the test
materials (by WG,
QAWG, EG)
- WG and EG perform the actual transfer of
the test materials,
WG creates an initial
repository (by WG, EG)
- After transfer, initial test development/framework process setup:
- WG appoints a test materials manager.
- The test materials
manager creates a QA Process Document for WG (by WG, test materials manager, QAWG may consult)
- set up the test materials
home page, a test materials
issues mailing list (by WG,
test materials manager)
- determine the appropriate W3C publication license (by WG, QAWG)
- First W3C public release of the new test materials:
- make any needed enhancements prior to the first public release: fix
known/reported errors, produce documentation (by WG), W3C license labelling, etc.
- announce the first public release of test materials (by test materials manager,
Communications Group)
- joint W3C/EG press
release (by WG, QAWG, Communications Group,
EG)
- After the first public release, the test materials enter the maintenance
phase.
The following QA Working Group and Interest Group participants have
contributed significantly to this document:
- Daniel Dardailler (W3C),
- Dimitris Dimitriadis (Ontologicon),
- Karl Dubost (W3C),
- Dominique Hazaël-Massieux (W3C),
- Lofton Henderson (CGM Open),
- David Marston (Lotus Development Corp),
- Patrick Curran (Sun Microsystems),
- Lynne Rosenthal (NIST),
- Mark Skall (NIST),
- Andrew Thackrah (Open Group),
- Olivier Thereaux (W3C),
- Jeremy Carroll (Hewlett-Packard)
- [PROCESS]
- World Wide Web
Consortium Process Document, I. Jacobs, Ed., 05 February
2004, available at http://www.w3.org/2004/02/Process-20040205/ .
- [CONTRIB]
- Contribution
of Software or Test Materials to W3C, which defines
W3C-approved procedures and terms for submission of Software and Test
Materials, available at
http://www.w3.org/Consortium/Legal/2002/contribution-grant-20021231
.
- [PROTOCOL-WG-TS]
- The XML Protocol WG has a TS process document, available at
http://www.w3.org/2000/xp/Group/1/10/ts-contribution, and a contribution/submission
license (example of a submission legal notice), available at
http://www.w3.org/2001/10/test-materials-license.html .
- [REC-TIPS]
- Tips
for Getting to Recommendation Faster, a public part of the
(Member-only) W3C
Guidebook.
- [QA-GLOSSARY]
- A comprehensive glossary of QA terms, maintained by the QA Working
Group. (Incomplete
version under construction at http://www.w3.org/QA/glossary .)
- [QAF-PRIM]
- QA Framework
Primer & User Scenarios, an informative supplement to
The QA Handbook that provides an orientation to whole
QA Framework, including a primer and some user scenarios.
Published as a QA note at http://www.w3.org/QA/WG/qaframe-primer .
- [QAF-SPEC]
- QA
Framework: Specification Guidelines, Karl Dubost, L.
Rosenthal, Dominique Hazaël-Massieux, Lofton Henderson, Eds., W3C
Working Draft companion to this document, 22 November 2004. A
light-weight revision of the previous
(Candidate Recommendation) Specification Guidelines
collection of documents. Latest version available at
http://www.w3.org/TR/qaframe-spec/ .
- [TEST-GUIDE]
- A collection of discussion topics and guidance on test building and
other test-related subjects is found on the test-related
Wiki pages.
- [QAWG]
- Quality Assurance
Working Group of the W3C QA Activity, which may be found at
http://www.w3.org/QA/WG/ .
- [VIS]
- Variability
in Specifications , Dominique Hazaël-Massieux, Lynne
Rosenthal, W3C Working Draft 30 August 2004,
http://www.w3.org/TR/2004/WD-spec-variability-20040830/ . Latest
version available at http://www.w3.org/TR/spec-variability/ .
- [TRANSITION]
- How to Organize a
Recommendation Track Transition, a part of the Member Guide
(Member-only), is available at http://www.w3.org/2004/02/02-transitions
.
- 2004-11-22, Third publication (WG Note)
- incorporated new W3M policies on contribution of test materials
into Chapter 4.
- editorial changes per L.Rosenthal
comments and editor's reply.
- removed chapter 1 reference to "chronology diagram" (no one
volunteered a new diagram).
- changed the 4 occurrences of "Principle" to "Guideline" (but
markup still is principle-oriented).
- numbered (1..4) and labelled Guidelines.
- numbered (1..15) Good Practices (not yet labelled).
- labelled the stories at start of chapters 2..5.
- reworked all references to Test Guidelines, in some
cases substituting test-guidance Wiki collection [TEST-GUIDE].
- Removed Test Guidelines from components of QA Framework, added
ViS, Primer, and test-guidance Wiki pages.
- Shortened some (but not all) long Guidelines and Good
Practices.
- Made markup and topic sub-headings more consistent with
SpecGL.
- Removed editorial notes to self.
- Integrated into GP2 LR/MS break-out stuff (Reading) on test
materials and quality-related deliverables.
- 2004-08-30, Second Published Draft
- edits for improved language, removal of excessive use of
abbreviations, etc (per J.Carroll
comments).
- changed vague usage of "IPR", to be more specific (per D.Connolly
comments).
- anchors on all PRs and GPs, plus at all example-placeholder
locations.
- indicate "back burner" status of TestGL (in SoTD, and in
References -- leave alone the body text for now).
- split out the "Orientation to QA Framework" appendix into
separate QA Framework Roadmap document.
- integrated K.Dubost
comments.
- integrated M.Skall
comments.
- integrated L.Rosenthal
comments.
- integrated changes to the legal/license Good Practices, per
recent AB rulings.
- sorted out JC's scope comment ("test & testability") and
implemented solution.
- first cut at sorting out class="vague" stuff (needs critical
review)
- drafted icon/logo section (but more needed next version)
- fleshed out examples in section 3.1 (General modus operandi)
- 2004-05-10, First Published Working Draft
- Combines the best features of the former QA Framework:
Introduction and QA Framework: Operational
Guidelines into a lightweight, non-normative handbook for W3C
Working Group Chairs and Staff contacts.