The QA Handbook (QAH) is a non-normative handbook about the
process and operational aspects of certain quality
practices of W3C's Working Groups, with particular focus on
testability and test topics. It is intended for Working Group chairs and team
contacts. It aims to help them to avoid known pitfalls and benefit from
experiences gathered from the W3C Working Groups themselves. It provides
techniques, tools, and templates that should facilitate and accelerate their
work. This document is one of the QA Framework (QAF) family of documents of
the Quality Assurance
(QA) Activity. QAF includes the other in-progress or
planned specifications: Specification Guidelines (in progress),
and Test Guidelines (planned).
This is a QAWG Editor's development draft. It has no status or
standing, and does not necessarily represent QAWG consensus. If you're
thinking about commenting, please see above for the latest published version,
and consider carefully which version you want to comment on. Further editor's
notes:
- some issue resolutions (from 15-17 June QAWG f2f & telecons) have
not yet been completely implemented;
- this is nevertheless a suitable basis for submitting examples, per
invitation of QAWG & QAH editors to Chairs/Team;
- phrases styled like this -- e.g., quality
practices -- are vague wording that is targetted for and in
process of refinement.
The following is the (mostly) unmodified status section of FPWD QA
Handbook...
This section describes the status of this document at the time of its
publication. Other documents may supersede this document. A list of current
W3C publications and the latest revision of this technical report can be
found in the W3C technical
reports index at http://www.w3.org/TR/.
This document is a W3C Working Draft (WD), made available by the W3C Quality
Assurance (QA) Activity for discussion by W3C Members and other interested
parties. For more information about the QA Activity, please see the QA Activity
statement.
This is the First Public Working Draft of this document. This draft
accurately reflects the redesign of the QA Framework (QAF) resolved by the QA Working Group (QAWG) at its 2004 Technical Plenary
face-to-face. The QA Handbook (QAH) replaces and incorporates the best
features of the former QA Framework:
Introduction and QA Framework:
Operational Guidelines.
This draft is accurate in overall content. However it is incomplete in
some details (generally indicated by "@@" or "TBD"), especially in the examples — lots
more examples are intended. Also, some links to planned references are yet to
be done. Because this is a significant departure from the last-published
QAF, QAWG wants to get it out for review and
comment on the overall direction as soon as possible. The just-mentioned
incomplete details will be fleshed out in the next publication.
[Ed note: Proposed rewording of FPWD language,
to accommodate TestGL delay...] The QA Handbook will be
coordinated with the other parts of the QA Framework, the in-progress
QA Framework: Specification Guidelines [QAF-SPEC] and the planned QA Framework: Test
Guidelines [QAF-TEST]. (Test
Guidelines development has been temporarily suspended, pending
resource availability). Synchronization of active parts at a uniform level of
maturity is intended before advancing beyond Last Call. The earlier
publications of the various parts will occur somewhat more
independently.
The QAWG does not expect this
document to become a Recommendation. Rather, after further development,
review and refinement, it will finally be published and maintained as a
Working Group Note.
Publication as a Working Draft does not imply endorsement by the W3C
Membership. This is a draft document and may be updated, replaced or
obsoleted by other documents at any time. It is inappropriate to cite this
document as other than work in progress.
The QA
Working Group Patent Disclosure page contains details on known patents
related to this specification, in conformance with W3C policy
requirements.
Comments on this document may be emailed to www-qa@w3.org, the publicly archived list
of the QA Interest Group.
Commenters please note that your comments will be publicly
archived and available, do not send information that should not be
distributed, such as private data. Please give any feedback by 12 June 2004,
so that it may be considered by QAWG at its next face-to-face meeting.
- Introduction & roadmap
- Early planning and commitment
- Day-to-day QA operations
- Licensing & branding
- Acquiring test materials
- Acknowledgments
- References
- Change History
This document has two separate external appendices, QA Process Document template and Charter template, which contain templates to help
Working Group Chairs and Staff Contacts to implement the document's good
practice guidelines.
The following five use cases for The QA Handbook (QAH), told as stories, show when and how
QAH could be helpful. They cover
five different situations that might typically arise over the life of the
Working Group.
The QA Handbook could be useful to Chairs and Staff Contacts
at Charter time...
Story 1: Think about quality early
The Charter-drafting team of a new Working Group, led by Staff Contacts and
co-Chairs, looks through The QA Handbook. Following the good
practices in the early-commitment and
planning module, the team debates how much it can realistically
commit to at this early stage. After deciding on some test materials
deliverables, milestones, and specification synchronization that it considers
to be safe commitments, the team uses the provided Charter Template to
include these in the draft Charter.
Or it could help a Chair who is trying to jump-start a test suite
project...
Story 2: Jump-starting a testing
effort
An existing Working Group has just finished writing its Requirements and Use
Cases documents, and has begun to draft its specification. At the same time,
it is taking a first look at its test suite plans. As recommended in the
The QA Handbook, the Chair jump starts the Test Suite project by
appointing a point of contact and part-time
Test Suite team, whose first output is a quick QA Process Document (QAPD)
using the provided QAPD template.
[Ed. Should QAPD be renamed to TSPD? (TS = Test
Suite)]
Or maybe the Chairs and Staff Contacts are pondering the steps to transfer
a test suite from an external entity...
Story 3: Test-suite transfer
A Working Group has re-chartered to finish a Second Edition of its
specification, and to develop the next functional version. The group did not
develop a test suite during its first charter, but a collaboration of outside
organizations and an industry consortium has developed one. The Chair and
Staff Contacts have negotiated an agreement in principle to transfer the test
suite to a stable home in W3C. In The QA Handbook, they find a
handy checklist of preliminary steps,
requirements, and specific activities for a smooth transfer.
Or maybe they need to take preemptive action due to looming possible
license-issue hassles...
Story 4: Test suite licensing
A Working Group is almost ready to request Candidate Recommendation (CR), and has gotten a
comprehensive test suite together for the CR's trial implementation
period. As the Chair starts to arrange for publication of the test suite, she
finds the Working Group split on which test suites distribution licenses to
use. Consulting The QA Handbook, she finds discussion of the pros and cons of the W3C licenses (the Software
License and the Document
License), and advice on devising an optimal licensing strategy.
Or maybe they can borrow the experience of other W3C Working Groups for
various useful and common test suite processes...
Story 5: Test development processes
A Working Group has built and released a basic test suite for its
specification. A Staff Contact has been given the Action Item to plan its
expansion to a more comprehensive test suite, by leveraging and integrating
the large test collections of several early implementors. Rather than figure
out the issues and write a Test Contribution & Review process from
scratch, he looks at the summary advice in
The QA Handbook. QAH
points him both to some useful templates, and to more detailed stuff in
QA Framework: Test Guidelines [QAF-TEST], that significantly reduce his work to
complete his action item.
Ed note. For now, TestGL is delayed, not
cancelled. For now, is it okay to handle this by the discussion in the
reference, [QAF-TEST]?
Chairs and Staff Contacts are overworked. They share a lot of the same
concerns and pressures as @@broken link?@@ specification
editors.
Common problems include:
- never enough motivated people to do the work;
- deliverables delivered late;
- licensing and legal hassles;
- too much to do, so little time
A lot of groups have faced these issues. As a result, there is a growing
body of experience in W3C about what works and what doesn't. QAH aims to pull this together.
The QA Handbook (QAH) is intended for: Chairs and Staff
Contacts of W3C Working Groups. These include both the process-savvy as well
as the novices.
QAH is a non-normative handbook
about applying good practices and quality assurance techniques to WG
activities, especially developing Recommendations and conformance tests. It
flags avoidable pitfalls, and provides techniques, tools, and templates that
will facilitate and accelerate the work of the groups. Guidance in QAH is
supported by real-world stories and examples.
Because there is no normative content in QAH, conformance to QAH is not an issue. QAH takes this approach for several
reasons, chiefly:
- the diversity amongst W3C's working groups makes it difficult (but not
impossible) to craft simple normative rules for all;
- in diverse, people-centric processes, it is more effective to encourage
and help than to proscribe and prohibit.
Advice is presented in the imperative voice as Principles and
Good Practices. The aim is to present helpful guidance gathered from
the experience of W3C Working Groups. These suggestions will generally be
helpful and enhance the quality of the work of a Working Group. However, each
suggestion should be applied or not depending on the specific situation of
the WG.
The QA Handbook (QAH) is a practical guide for integrating
quality assurance techniques and good practices into the Working Group. The
emphasis is things affecting the development and implementation of
specifications and tests, especially:
- people resources;
- deliverables and their schedules;
- interaction with W3C and the world;
- W3C process, licenses, etc.
Although the emphasis is on tests and specification testability, the scope
touches on the total environment in which the group operates. The goal of
QAH is to capture and share
current good practices, point out hazards along the way, and suggest new
modes of operation to help the Working Groups in the timely and effective
pursuit of their missions.
The QA Handbook belongs to a document family collectively
called the QA Framework. The other in-progress or planned parts
of the QA Framework are:
- QA Framework: Specification Guidelines [QAF-SPEC]
- QA Framework: Test Guidelines [QAF-TEST]
Ed note. For now, TestGL is delayed, not
cancelled. For now, is it okay to handle this by the discussion in the
reference, [QAF-TEST]?
Editors, test builders, and W3C members in general, in addition to Chairs
and Staff Contacts, will find advice specific to their roles in these other
framework parts.
A brief orientation to the whole QA Framework is provided in
a companion note, QA Framework Roadmap [QAF-ROAD].
Other valuable documents about quality assurance techniques and good
practices are found outside of the QA Activity and its QA
Framework. QA Framework identifies and links to these
whenever appropriate.
After this orientation section, QAH contains four topical modules:
Each presents advice and guidance about tasks that
contribute to the WG's realization of quality specifications and tests. These
tend to have a chronological correspondence to phases (rec-track stages) of
its activities. Earlier is better, but later is better than never. Figure
1 depicts when, in a Working Group's life, the various QAH topics are ideally dealt with and
applicable:
[TBD. Here
is planned to be embedded a chopped down version of the OpsGLchronology diagram]
Why care?It's not always easy to
anticipate Working Group needs during the rush and pressure of Chartering, or
in the busyness of early Working Group life. But the earlier the group can
accurately identify and commit to its test and other quality related
deliverables, the less likely it is that the WG will get major surprises
later on, or run into problems that will delay its progress towards
Recommendation.
Additional reason — binding decisions about participation (e.g.,
numbers per company) often are made at Charter time.
What does this mean? In practice, this
means attention to a handful of points, which we enumerate as Good
Practices.
Good Practice: Decide as soon as possible
— will the Working Group build test materials or acquire them?
Clearly, it is going to make a big difference in a Working Group's
staffing requirements — building test materials tends to be labor
intensive (extremely so for some types of specifications). Even if the group
imports them, some resources will have to be applied (see last module). The particular test-related activities and
milestones in its programme of work will in general be completely different
for build versus acquire options.
Good Practice: Think about and
enumerate the QA activities &
deliverables that might help the Working Group through the
Recommendation track. Minimally, commit to assuring the timely existence of
test materials.
Different kinds of QA deliverables might
include:
- basic test suites,
- validators,
- test assertions,
- one or more spec quality reviews (using Specification
Guidelines [QAF-SPEC]),
- comprehensive test materials (beyond basic coverage),
- tests of new, changed, or contentious functionality.
Test materials are by far the most common QA
deliverable, and likely will be a key to quick and painless passage
through Candidate Recommendation (CR).
Good Practice: Synchronize QA deliverables with specification milestones, and
for the bigger QA deliverables, define and schedule intermediate milestones
if possible.
This advice echoes that in Tips for Getting to Recommendation
Faster [REC-TIPS] for example
see item #6 in section 2.
Good Practice: Consider whether the
Working Group should tie any quality criteria
to Rec-track advancement.
For example, finishing test materials deliverables before requesting
Candidate Recommendation (CR) is important, in order to
facilitate CR's
Implementation Assessment. (Counter-example: as in the above Story, if the group is still building them
during CR, it is likely
to uncover things that will oblige it to cycle in CR.)
This good practice takes the synchronization of deliverables of the previous Good Practice a step further
— the specification may not advance unless the committed test or other quality criteria are met.
How
to Organize a Recommendation Track Transition [TRANSITION] talks some about the role of test
suites in advancement decisions.
Good Practice: Put some thought into how
to staff the Working Group's QA plans.
The earlier this is done, the more options will be available. Some options
include:
- dedicated team from Working Group participants;
- part of everyone's time;
- Consider: asking in Call for Participation for extra people who would
specifically be tasked to work on QA
deliverables.
The third option isn't really different from the first. It's just a way of
doing it. But notice that it's an option that is only available by looking
into these questions at Charter time.
By the way, there has been confusion [TBD: find the chairs archive reference] about
"W3C Process only allows each company to have two participants on the Working
Group". In fact, that is not from W3C Process. W3C Process gives considerable
freedom for this to be tailored to Working Group needs — W3C
Process says it may be specified in the Charter. So, for example, the
group could decide on these rules: allow two per company in technical
discussion, issue resolution, voting, etc; and, allow additional dedicated
test suite builders.
By the way, this is another good reason to put some thought into test
suite plans and other quality-related deliverables as early as possible.
Tips for
Getting to Recommendation Faster [REC-TIPS] (section 3) also talks some about the value of
(early) staffing decisions.
Examples:
- [TBD. initially get some existing examples
from OpsGL/ET and Intro]
- [TBD. during reviews, explicitly solicit good
examples and counter-examples]
How can I do it?
- The Charter template
provides for a superset of the things that one might want to specify and
commit to early (at chartering time). Use it for those items that are
deemed appropriate to write it into the charter.
- While often "earlier is better", Charter time may not be appropriate
for each and every of the above Good-Practice recommendations, for every
Working Group. In some cases, it might be premature to commit to that
level of detail at that early stage of the groups life.
Why care? Over and over we hear the
message from successful test suite projects — define the work process
early: test development framework, issues list, email list, faq, contribution
& review process, etc. [Examples TBD:
DOM, Xquery, maybe SVG...] Conversely,
there are examples of disorganized collections of test cases with no one
apparently in charge, no way to find their status, correctness, [Example
TBD: find KD's counter-example of
"can't find anyone who's responsible for amorphous undocumented heap of
"tests"?]
The W3C
Process Document clearly organizes the way that a Working Group's specs
are progressed through the Rec track. But there is no similar template for
the other aspects of a Working Group's life, particularly aspects related to
test and quality processes — logistical
setup, communications with the outside, licensing and branding policy, test
development process, etc.
Good
Practice: Put all of the Working Group's important test and QA-related information in one place in a QA Process
Document.
How can I do it? Simple: Use the QA Process Document template. It will
guide the user through everything needed, and then some. It is not only a
template, but also a checklist of sorts, for the sort of things that the
Working Group should consider having in its QA Process Document. The template
has been assembled as the union of good practices seen in real QA Process
Documents of W3C Working Groups.
In the past, before the construction of this template, test efforts would
often copy-edit an existing process document from another effort. There isn't
really any reason to do this anymore. Here are some examples from which the
template has been assembled:
- [TBD:
example1]
- [TBD:
example2]
- ...etc...
What does it mean? In practice, it means producing a QA
Process Document recording at least those details of the Working Group's
test- and QA-related work processes that are
outlined in the template and briefly discussed in the following
subsections.
See the corresponding
section in the template.
Good Practice: Identify a Working Group
point-of-contact about test materials or other QA-related business.
This can be a special appointed "QA Moderator". Or it might be the Chair
(or a co-chair), or Staff Contact. The important part is that there be an
identified point-of-contact to whom others can address questions,
contributions, etc, and who will be accountable for responding.
See QA Process Document
subsection.
Here are a couple
of TBD:
Example(s).
Good Practice: Specify an archived email list
to use for QA-related communications (test
suite questions, bug reports, contributions, etc).
It can be a dedicated "Test" list. Or it can be a public Interest Group
list. Or it can be a public comment list separate from the Working Group
list, in the case that there isn't an Interest Group. Because of the variety
of public/private mixes amongst W3C's Working Groups, "one size fits all"
does not work well here.
Here are a couple of TBD:
Example(s).
See corresponding QA
Process Document subsection.
Good
Practice: Identify Web page(s) for test suites, announcements, and
any other QA-related deliverables.
Obviously, this needs to be publicly accessible. Doing it all in the
public portion of the Working Group's Web space is one way to achieve that.
In particular, that provides a good, secure repository location for test
materials.
Some groups [Example: @@SVG] have two locations
actually — a private CVS repository for the in-development test
materials, and a simplified public repository for periodic public releases of
test materials.
Here are a couple of TBD:
Example(s).
See corresponding QA
Process Document subsection.
The Working Group's QA Process Document is also a handy central location
to record its licensing policies and (any) branding policies. These are
sufficiently important that they have their own
QAH module, which
addresses:
- license terms applicable to test materials, both submitted and
published;
- any branding policy — logos, conformance icons, etc — and
associated rules and restrictions.
Ed note. For now, TestGL is delayed, not
cancelled. For now, is it okay to handle this by the discussion in the
reference, [QAF-TEST]?
[EdTBD. The QA Process
Document subsection references in this section need to be refined more
precisely.]
If the Working Group is developing test materials itself, then there are
several topics associated with test materials development process are going
to impact its operations and management. These are described, along with
Principle and Good Practice guidance, in some detail in
QA Framework: Test Guidelines [QAF-TEST]. These topics, which might conveniently be
documented in the Working Group's QA Process Document (QAPD), include definitions of:
Finally, as a part of
planning about "life after Working Group", the group will need to decide what
happens to both its test materials and associated processes. These
maintenance-related topics are described in some detail in QA
Framework: Test Guidelines [QAF-TEST] , but again the information might conveniently
be located in the group's QA Process Document:
- what happens to the test materials themselves?
- what happens to the Working Group's contribution and review processes?
[QAPD sub-section]
- how to track errata and (for example) new specification editions? [QAPD sub-section]
- what happens with bug reports about the test materials? [QAPD sub-section]
Success criteria:
TBD:
Cartoon?
Principle: Get agreement up front about
license and any other legal issues around planned test materials.
Why care?Get it right early, or it may
stall the Working Group's Rec-track progress indefinitely. While this might
seem to be a routine piece of Day-to-day
operations, it has proved to be sufficiently troublesome within W3C that
it deserves to be a Principle by itself.
What does this mean? There are two kinds
of licensing issues: submission license, and publication license. Both of
them can be problematic and can interrupt the Working Group's progress on
Rec-track if not early addressed and carefully handled. Test materials
license issues are the subject of ongoing debate and discussion within W3C,
but there are some tactics to minimize potential problems.
Good Practice: As early as possible,
get consensus and define acceptable license terms for submission of test
materials.
W3C currently has a more or less routine submission license, Contribution
of Software or Test Materials to W3C [CONTRIB]. By early attention, it should become clear
whether any potential sources require custom terms, before possible disputes
can impact the Working Group's deliverables and schedules. There have been
cases where potential submitters would not accept the standard terms, and
custom terms had to be negotiated.
Good Practice: As soon as the nature
of the Working Group's test materials becomes clear, get consensus and define
license terms for publication of the test materials.
Any W3C-hosted materials must have approved license and use terms.
Experience has shown that there is no single license that is appropriate for
all parts of all test materials, so the Working Group needs to address this
after it has come to an understanding of the structure, content, and intended
use of its test materials.
Currently approved W3C licenses that may be applied to test materials are
the Document
License and the Software
License. The Document License has two characteristics that are attractive
for test materials:
- It prohibits the test materials from being modified by the user upon
download, therefore guaranteeing the integrity of the test materials;
- It requires that if the test materials are copied or redistributed,
they must contain the link to the original test materials and their
status.
On the other hand, there are situations in which the Document
License is inappropriate, because (for example) some parts of the test
materials require modification or completion in order to apply them.
The W3C Advisory
Board feels (member-only link) that the Document License is the
appropriate license for test cases.
That ruling notwithstanding, test materials might contain some mixture of
different components: test software, test documentation, and test cases (see
Test Guidelines[QAF-TEST]?). In some cases, one license may be
appropriate for some component(s) of the test materials, but another license
may be better for other parts. A careful look at the test materials'
composition and use cases should reveal what is the best solution.
The Working Group should consult with W3C Legal if it believes that:
Here are
TBD:
Example(s): (Especially need an example of creative multi-component license
solution.)(Ex. SVG standard Doc License all parts) .
Good Practice: Consider whether to have
brands, logos, or conformance icons associated with the Working Group's test
materials. Define associated policies.
[TBD] Points to
develop about logos policy: terms for valid application, conformance claims,
challenge procedures, policing & enforcement policy, etc. Some of these
start to border on significant legal questions.
[TBD] Reference to
and discussion about W3C Logo
and Icon Usage, and maybe also SpecLite material about Conformance
Claims.
- The decision ultimately belongs to the Working Group. [TBD. Actually, we can go through a lot of
the things about Doc vs. SW, applying different licenses to different
bits, etc]
- There are places in QA
Process Document template to document the Working Group's decisions,
about licenses as well as branding/logos.
- Don't forget to make license terms obvious and visible in all
deliverables (and all their bits and pieces)!
Examples:
[TBD. Should
be easy to come up with a handful of good examples. Especially for
brands/logos. WAI, W3C content validators, etc]
Principle: The Working Group can save
lots of time and work by acquiring a test suite, but be ready to address and
resolve many of the same issues as build-your-own scenarios.
The three main problems are:
- license issues around test materials
- human resources
- quality of the test materials
The license and human resources issues are similar in concept to what the
Working Group faces when building test materials, but probably lesser in
degree. A pre-transfer quality assessment might seem unique to the acquire
option, but the actual steps will probably look similar to a test case
contribution/review process in a build-your-own scenario (see QA
Framework: Test Guidelines [QAF-TEST]).
Good Practice: Do a quality assessment of
proposed test materials before going any further.
Basic things that a good quality assessment might cover would for example
include:
- clear declaration of scope,
- traceability,
- atomicity,
- user documentation.
A more comprehensive list of things that an assessment process could add
to that basic list:
- maintainer documentation,
- correctness,
- completeness (vis à vis declared scope),
- harnesses or interfaces for application of the test materials,
- reconfigurability,
- results assessment,
- results recording & reporting,
- automation features,
- versioning/errata support,
- declaration of publication licenses,
- integrated submission procedures.
QA Framework: Test Guidelines [QAF-TEST] deals with this topic in much more detail,
including (planned) templates and assessment aids.
Good Practice: Ensure there are enough
human resources to support the transferred test materials.
A test materials manager is still needed, but total human resources ought
to be considerably less than build-your-own scenarios. With luck, the
original manager of the external test materials source might become a
participant in the Working Group after the test materials are transferred.
Good Practice: Sort out licensing issues
with the external party that produced the test materials.
How can I do it?
This is a virtualization of an actual
transfer scenario that QA helped to moderate. It could serve as a
checklist of steps to consider for Working Group's taking the
acquire route.
Legend: EG the
external group or entity; QAWG the QA Working Group;
TM the test
materials; WG the
Working Group acquiring the test materials.
- Before the transfer, WG with
the help of QAWG:
- performs an assessment of the quality of candidate test matrials (by WG, QAWG)
- identifies and commits to a set of test-related deliverables from
the candidate test
matrials. These could be: releases, appeal/challenge
processes, maintenance plan, submission/review process, Web site,
mail list, etc. (by WG)
- identifies sufficient staffing, including at least a test matrials manager.
Recommendation: recruit the test matrials manager from the
EG (if one exists) to
become a WG participant
after the transfer. (by WG)
- makes the decision to seek/accept the transfer. (by WG)
- (potentially) initiates Charter amendment (by WG, QAWG may consult), if the test matrials acquisition doesn't
fit within the current Charter.
- During the transfer:
- EG and W3C reach
agreement to transfer the test
matrials (by WG,
QAWG, EG)
- WG and EG perform the actual transfer of
the test matrials, WG creates an initial repository (by
WG, EG)
- After transfer, initial test development/framework process setup:
- WG appoints a test matrials manager.
- The test matrials manager
creates a QA Process Document for WG (by WG, test matrials manager, QAWG may consult)
- set up the test matrials
home page, a test matrials
issues mailing list (by WG,
test matrials manager)
- determine the appropriate W3C publication license (by WG, QAWG)
- First W3C public release of the new test matrials:
- make any needed enhancements prior to the first public release: fix
known/reported errors, produce documentation (by WG), W3C license labelling, etc.
- announce the first public release of test matrials (by test matrials manager,
Communications Group)
- joint W3C/EG press
release (by WG, QAWG, Communications Group,
EG)
- After the first public release, the test matrials enter the maintenance
phase (see QA Framework: Test Guidelines [QAF-TEST].
The following QA Working Group and Interest Group participants have
contributed significantly to this document:
- Daniel Dardailler (W3C),
- Dimitris Dimitriadis (Ontologicon),
- Karl Dubost (W3C),
- Dominique Hazaël-Massieux (W3C),
- Lofton Henderson (CGM Open),
- David Marston (Lotus Development Corp),
- Patrick Curran (Sun Microsystems),
- Lynne Rosenthal (NIST),
- Mark Skall (NIST),
- Andrew Thackrah (Open Group),
- Olivier Thereaux (W3C),
- Jeremy Carroll (Hewlett-Packard)
- [PROCESS]
- World Wide Web
Consortium Process Document, I. Jacobs, Ed., 05 February
2004, available at http://www.w3.org/2004/02/Process-20040205/ .
- [CONTRIB]
- Contribution
of Software or Test Materials to W3C, which defines
W3C-approved procedures and terms for submission of Software and Test
Materials, available at
http://www.w3.org/Consortium/Legal/2002/contribution-grant-20021231
.
- [MATRIX]
- QA Matrix...blah...blah...
- [PROTOCOL-WG-TS]
- The XML Protocol WG has a TS process document, available at
http://www.w3.org/2000/xp/Group/1/10/ts-contribution, and a contribution/submission
license (example of a submission legal notice), available at
http://www.w3.org/2001/10/test-materials-license.html .
- [REC-TIPS]
- Tips
for Getting to Recommendation Faster, a public part of the
(Member-only) W3C
Guidebook.
- [QA-GLOSSARY]
- A comprehensive glossary of QA terms, maintained by the QA Working
Group. (Incomplete
version under construction at
http://www.w3.org/QA/glossary .)
- [QAF-ROAD]
- QA Framework
Roadmap, L.Henderson, Ed, Working Group Note, a supplement
to The QA Handbook that provides an orientation to whole
QA Framework, including a primer and some usage scenaios.
- [QAF-SPEC]
- QA
Framework: Specification Guidelines, L. Rosenthal, Karl
Dubost, Dominique Hazaël-Massieux, Lofton Henderson, Eds., W3C Working
Draft companion to this document, 02 June 2004. A light-weight revision
of the previous
(Candidate Recommendation) Specification Guidelines
collection of documents. Available at
http://www.w3.org/TR/2004/WD-qaframe-spec-20040602/ .
- [QAF-TEST]
- QA Framework: Test
Guidelines, a planned future W3C Working Draft companion to
this document. It will be a light-weight revision of the last-published
Test Guidelines (Working Draft). June 2004
face-to-face resolved to remove it from active publication track
for now, until QAH and SpecLite have stabilized and resources become
available. Test-related
Wiki pages will meanwhile be used to develop material.
- [QAWG]
- Quality Assurance
Working Group of the W3C QA Activity, which may be found at
http://www.w3.org/QA/WG/ .
- [SCHEMA-WG-TS]
- The XML Schema WG has a TS process document, available at
http://www.w3.org/2001/05/xmlschema-test-collection.html, and a contribution/submission
license (example of a submission legal notice), available at
http://www.w3.org/2001/05/test-materials-license.html .
- [TRANSITION]
- How to Organize a
Recommendation Track Transition, a part of the Member Guide
(Member-only), is available at http://www.w3.org/2004/02/02-transitions
.
- 2004-07-xx, Editor's draft
- edits for improved language, removal of excessive use of
abbreviations, etc (per J.Carroll
comments).
- changed vague usage of "IPR", to be more specific (per D.Connolly
comments).
- anchors on all PRs and GPs, plus at all example-placeholder
locations.
- indicate "back burner" status of TestGL (in SoTD, References, and
body of text).
- split out the "Orientation to QA Framework" appendix into
separate QA Framework Roadmap document.
- integrated K.Dubost
comments.
- integrated M.Skall
comments.
- integrated L.Rosenthal
comments.
- integrated changes to the legal/license Good Practices, per
recent AB rulings.
- tbd... finish sorting out JC's scope comment ("test &
testability")
- tbd... finish sorting out pink stuff (I'm thinking ... this might
be handled together w/ previous?)
- tbd... finish icon/logo section
- tbd... more EXAMPLES !
- ...etc...
- 2004-05-10, First Published Working Draft
- Combines the best features of the former QA Framework:
Introduction and QA Framework: Operational
Guidelines into a lightweight, non-normative handbook for W3C
Working Group Chairs and Staff contacts.