W3C

QA Framework: Specification Guidelines

W3C Working Draft 28 April 2005

This version:
http://www.w3.org/TR/2005/WD-qaframe-spec-20050428/
Latest version:
http://www.w3.org/TR/qaframe-spec/
Previous version:
http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/
Editors:
Karl Dubost, W3C
Lynne Rosenthal, NIST
Dominique Hazaël-Massieux, W3C
Lofton Henderson, CGM Open
Contributors:
See Acknowledgments.

Copyright ©2002-2005 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark and document use rules apply.

Abstract

Much effort goes into writing a good specification. It takes more than knowledge of the technology to make a specification precise, implementable and testable. It takes planning, organization, and foresight about the technology and how it will be implemented and used. The goal of this document is to help W3C editors write better specifications, by making a specification easier to interpret without ambiguity and clearer as to what is required in order to conform. It focuses on how to define and specify conformance for a specification. Additionally, it addresses how a specification might allow variation among conforming implementations. The document contains a set of guidelines or requirements, supplemented with good practices, examples, and techniques.

Status of This Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This document is the 28 April 2005 Working Draft of “QA Framework: Specification Guidelines”, made available by the QA Working Group of the W3C Quality Assurance (QA) Activity for discussion by W3C members and other interested parties. For more information about the QA Activity, please see the QA Activity statement.

This Working Draft implements the resolutions of issues raised during the Last Call period started in November 2004. Commenters of the Last Call Working Draft will be contacted shortly to seek their feedback on these resolutions. The QA Working Group has started to gather implementation feedback in a draft implementation report. Based on this, it intends to request transition to Proposed Recommendation status in the upcoming weeks. This document is mainly intended to gather feedback from Last Call commenters, but any reviewer can send comments on this document to www-qa@w3.org, the publicly archived list of the QA Interest Group, although the Working Group is likely to reject substantive comments at this stage of development.

The change log contains changes since the previous version. A table of correspondence maps the Requirements and Good Practices numbers between the former and new numbering schemes.

Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

The Working Group's Patent disclosure page, in conformance with the W3C Patent Policy of 5 February 2004, contains patent disclosures relevant to this specification. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) with respect to this specification should disclose the information in accordance with section 6 of the W3C Patent Policy.

Table of contents

List of Requirements:

List of Good Practices:

Appendix

1. Introduction

1.1 Scope

This document is a guide for W3C specification editors and authors. It provides guidelines for writing better specifications.

The term specification is used as defined in ISO Guide 2-4 [ISO-GUIDE] as meaning a document that prescribes requirements to be fulfilled by a product, process or service. Specifications can be defined in one document or as a coherent set of several documents (see Umbrella specifications in Variability in Specifications [VIS] for more discussion), and can import requirements of other specifications with normative references.

This document differs from other W3C process and publication related documents in that it looks at specifications from a conformance viewpoint. It address the most basic and critical topics with respect to conformance, including how to convey what is required for an implementation in order to conform and how to allow variation among conforming implementations.

In addition to conformance, there are several other topics that should be addressed when writing a specification, such as accessibility, internationalization, security, and device independence. These topics are not directly in the scope of this document, but are evoked in section 3.3. Specification authors and editors are encouraged to consider these topics and coordinate their efforts in these areas with the relevant W3C Working Groups.

1.2 Goals and Motivation

The goal is to enable Working Groups to produce specifications that are precise, easier to interpret without ambiguity or confusion, and clearer as to what is required in order to conform. Good specifications lead to better and more interoperable implementations and foster the development of test suites and tools.

Everyone benefits from having well-written specifications. Editors may have less rework and thus, fewer issues raised during the development of the specification, and fewer errata once it is finished. Implementers can implement sooner and have a better chance to conform to the specification. Test developers are able to derive unambiguous test assertions. The end users benefit from having interoperable solutions. W3C gains by having recommendations produced with higher quality and reduced maintenance.

1.3 Why Specification Guidelines?

It is not an easy task to write accurate, clear, complete, unbiased specifications. It requires planning, organization, and foresight about the technology, how it will be implemented and used, and how technical decisions affect conformance. This document provides a collection of requirements, good practices, examples, and techniques that lead the reader through the decisions necessary to write precise requirements and establish, define, and specify conformance for specifications.

Editors and authors are busy, under pressure to get the specification published, and already have a reading list of W3C documents. A good place to start is W3C Editor's Home Page [EDITOR]. This document can be used as a checklist of things to consider, a how-to guide with examples and techniques that can be adapted, and a reference for understanding conformance concepts and terminology.

1.4 Audience of this document

The primary audience of this document is editors and authors; however, it is applicable to a broader audience including:

This document makes no distinction between the terms editors and authors and refers to them collectively as editors.

1.5 About this document

This document is a practical guide to writing a specification, presenting editors with topics to consider. The normative content is contained in a collection of a small number of Requirements, and somewhat more Good Practices.  As explained in this specification's conformance clause, the Requirements are necessary for claiming conformance to Specification Guidelines, and the Good Practices are recommendations that will further benefit the quality of a specification.

The overall objective of these requirements and good practices is to facilitate the creation of a complete conformance clause in every specification. A conformance clause template [CONF-TEMPLATE] is provided to assist editors satisfy the requirements of this document and end up with a conformance clause. Note that for some technical reports (e.g., The QA Handbook [QA-HANDBOOK], Architecture of the World Wide Web, Volume One [WEB-ARCH]) where conformance is not an issue (e.g., no normative content), the conformance clause may be an explanation of why there is no "conformance to this document" and may be presented in another section rather than in a separate conformance section.

The topics presented herein are inclusive (self-contained) and provide information needed to understand and successfully apply the Requirement or Good Practice, although related information and advanced topics may be referenced.

If in a hurry just read the first guidelines section, Specifying conformance — this may be all you need to read in order to reach the expected outcome of adhering to this document, i.e. specifying conformance. It serves as a roadmap to other parts of this document, which help achieve specifying conformance.

1.5.1 Structure of this document

This document is organized into a series of guidelines such as Specifying Conformance and Managing Variability. Each of these guidelines present and explain Requirements and Good Practices. Techniques and Examples accompany each Requirement and Good Practice. The techniques illustrate basic (and non exhaustive) questions or methods to help realize the Requirement/Good Practice and produce specification text. The examples are explanations or extractions from existing W3C specifications that specifically illustrate the point made in the Requirement/Good Practice.

The conformance clause of this document describes the conformance requirements for claiming conformance to this Specification Guidelines. A specification editor who wishes to write a specification conformant to Specification Guidelines must ensures it satisfies the conformance requirements in the conformance section of this document.

1.5.2 Other QA Framework Documents

This document is useful as a stand-alone document or as part of a family of QA Framework documents designed to help the Working Groups improve all aspects of their quality practices.

2. Guidelines

2.1 Specifying Conformance

Conformance is the fulfillment of a product, process, or service of specified requirements. These requirements are detailed in a specification as part of a conformance clause and in the body of the specification. A conformance clause is the section of a specification that identifies all the criteria that must be satisfied in order to claim conformance to the specification.

A clear presentation of conformance is crucial to successful interoperability of implementations. The conformance model and the language used for normative information determine the testability of a specification. They also influence the overall implementation landscape, ranging from a narrow conformance with few allowable variations in implementations to multiple conformance types, resulting in numerous variations in conforming implementations. The model must be chosen carefully, to produce the intended implementation range.

2.1.1 A conformance clause is essential

A good conformance clause is the ultimate goal of these guidelines, and is sanctioned by conformance to this specification.

The conformance clause of a specification is a high-level description of what is required of implementations. It, in turn, refers to other parts of the specification for details. Ideally, readers can find any conformance-related information from its conformance clause which serves as a root source.

For some specifications, the conformance model may be straightforward and simple, and the conformance clause template [CONF-TEMPLATE] when completed, may provide most of the information needed in a conformance clause.  For others, the conformance model is complex or convoluted, and the Advanced Topics references may be invaluable in creating the conformance clause.

Requirement 01: Include a conformance clause.

What does it mean? The conformance clause provides the answers to the important questions: what may conform and how? The conformance clause defines at a high-level, what is required of implementers of the specification.  It, in turn can refer to other parts of the specification or other specifications.  The conformance clause may partition the technology into functional subsets, such as profiles, modules or other structures.  Additionally, it may specify minimal requirements for certain functions, as well as extensibility, optional features and alternative approaches and how they are to be handled.

Why care? The conformance clause defines what is required to claim conformance: as such, it provides communication between the specifications creators, implementers, and users as to what is required, and gives meaning to the phrase, “conforming implementation”.  Moreover, it facilitates the consistent application of conformance within a specification and across related specifications.  It promotes interoperability.

Related

Techniques
Examples

Ruby Annotation [RUBY] specification is an example of a short specification with a detailed conformance clause.

SVG 1.0 [SVG11] specification contains a detailed conformance clause for a complex modular technology.

Good Practice 01: Define the specification's conformance model in the conformance clause.

What does it mean? The conformance model is the conceptual framework in which conformance is defined. It consists of and is defined by addressing at least these three topics:

Why care? The key is to communicate to the reader what conformance to the specification is all about.  It provides a framework for implementers to know what they need to build in order to conform and the different ways that they could claim conformance.  It provides users/buyers a basis to express their requirements.

Related The conformance model overlaps several other topics in QA Specification Guidelines, most particularly:

Techniques
  1. List the classes of products targeted by the specification
    1. Identify the types of products that will implement the specification
    2. If not already done (above step), group the products into generic categories – these are the classes of product. (e.g., content, user agent, protocol, API, specification)
  2. List the conformance designations or conformance concepts. To help realize this, consider these questions.
    1. Does conformance mean something different for different classes of products?
    2. Is more than one type of conformance defined – e.g., different designations (well-formed, valid) or degrees of conformance (A, AA, AAA)?
    3. Is conformance tied to the class of product and similarly named (e.g., host-language conformance, document conformance)?
  3. Create a name for each way that conformance can be qualified – i.e., label it
    1. For specifications subdivided into modules, profiles and/or levels, is there a conformance designation associated with each type of subdivision?
    2. If the specification has options and extensibility, will these have an affect or be affected by the conformance designation?
  4. Draw a diagram to put it all together – sometimes it is easier to work from a picture
    1. Diagram the classes of products with associated conformance designations,
    2. Add in the subdivisions and any other variability (e.g., options, extensibility)
    3. Extract from this diagram and define the conformance model.
    4. Write the description into the specification. Bonus - if the diagram helps to understand the model, include it.

Figure 1: Conformance Model of Ruby Specification

Graph illustrating the Ruby Conformance Model

Ruby Annotation [RUBY] specification defines two types of conformance tied to the content models: Simple and Full. The specification defines six types of products that we could group in three classes of products (content, specification, user agent). For each of them the conformance section defines the requirement to fulfill Simple or Full conformance.

Examples

XML 1.0 [XML10] has two classes of products (document and processor), each of those has two conformance degrees (well-formed/valid and validating/non-validating); xml:base, XML Namespaces and XLink could also be considered as "modules" for XML even though they have not been formally defined as such/

SVG 1.1 [SVG11] has roughly four classes of product (markup fragments with various extents, generators, interpreters and viewers). Some of these classes of product have various degrees of conformance (Appendix G: Conformance Criteria [SVG11]) (e.g. static / dynamic for interpreters, static / dynamic for high-quality for viewers); SVG 1.1 also defines modules that are grouped into profiles (tiny/mobile/full).

Good Practice 02: Specify in the conformance clause how to distinguish normative from informative content.

What does it mean? Normative content is the prescriptive part of the specification, whereas informative content is for informational purposes and assists in the understanding and use of the specification. Content includes all sorts of different forms — not only descriptive prose, but also illustrations, examples, use cases, formulae and other formalisms.

Why care? Conformance of implementations is defined by and measured against normative content.  Distinguishing normative content from that which is informative helps to make sure the reader can find the normative content, knows for sure that it is normative, and does not fail to notice a normative section.  This good practice aims at the high level partitioning of information (e.g., sections) in a specification.

Related What is mandatory? (See section 2.3.2)

Techniques
  1. For each section in the specification,
    • Determine if the content is normative or informative and, explicitly label it as either “normative” or “informative”.
    • Make the label part of the section heading (e.g., Informative References), as parenthetical text with the heading (e.g., Glossary (normative)).
    • Alternatively, create a list of each section and indicate its normality, (e.g., Normative Parts in this document’s conformance section).
  2. In the conformance clause, explain what the use and meaning (if necessary) of the words used to convey the normality and informality of the content. See Requirement 07: Use a consistent style for conformance requirements and explain how to distinguish them.
  3. Try to avoid language that sounds normative in an informative section. It might lead the readers to wrong assumptions.
Examples

XHTML 1.0 [XHTML10] is using the words of RFC2119, but in an extended way (See Definitions [XHTML10]). They define the exact terminology for each word.

2.1.2 Specify how to make conformance claims

The central message of this section — “have a good conformance clause” — has many ancillary details. Because the conformance clause is the foundation for defining and measuring conformance, it is also the basis for assessing conformance claims. One detail worthy of attention is “valid conformance claims”.

Rather than live with the infinite varieties of creative conformance claims that can arise in a vacuum, the specification can be proactive.

Good Practice 03: Provide the wording for conformance claims.

What does it mean? It is inevitable that people (e.g., vendors, purchasers) will either claim conformance or demand conformance to a technology. In fact, claiming conformance to a technology may be required in certain situations. Thus, it is important to provide a consistent and unambiguous way to make these claims. Identification of the specification version, class of products, and conformance label are some of the items that could be part of such wording.

Why care? Having a framework, by which to make conformance claims for a particular usage of the technology, minimizes confusion by people who are interested in such claims. Many contexts use conformance claims, including legal as part of regulations, laws, or policies and commercial when selling or buying a product.

Related Conformance claims are closely related to issues of logos and branding. See 4. Licensing and branding of The QA Handbook [QA-HANDBOOK].

Techniques
 

Conformance Claim Template:

Form 1:

  • Specification title, date, and version:  Foo Specification, version 1.0, 29 February 2003:
  • Specification URI: http://www.example.org/TR/2003/FOO-20032902
  • Completed ICS URI: http://mycompany.example.com/ICS-20032902
  • Conformance designation satisfied: "conformant processor"
  • Product name: My_Processor, version 1.2, 1 January 2004
  • Date of claim:  1 April 2004
 

Form 2:

On 1 April 2004, My_Processor, version 1.2, 1 January 2004 claims conformance as a “conformant processor” to the FOO Specification 1.0, 29 February 2003, available at http://www.example.org/TR/2003/FOO-20032902

Examples

WCAG 1.0 [WCAG10] is a good example of a specification explaining how to make a conformance claim depending on the degree of conformance.

Specification Guidelines section 4.4 Conformance Claims provides this document's template for conformance claims.

Good Practice 04: Provide an Implementation Conformance Statement Pro Forma.

What does it mean? An Implementation Conformance Statement (ICS) provides information about an implementation to a specification, by presenting in a uniform manner the implemented capabilities (e.g., functions, features) and options as well as limitations of the implementation. An ICS pro forma typically takes the form of a blank questionnaire or checklist for an implementation.  It provides the implementer a way to indicate the features implemented. Think of it as an inventory of what has been implemented.  Note that a completed ICS does not indicate conformance of the implementation.  Hence, answering "yes" to indicate a capability is supported does not mean that the capability has been tested.

This Good Practice suggests that the specification itself include an ICS pro forma.  Providing this pro forma makes it conducive to completing and helps to ensure consistency among completed ICS. 

Why care? An ICS pro forma provides a concise summary of a specification, i.e., the capabilities and options defined in the specification as well as any defined subdivisions (e.g., profiles, modules) and conformance designations. The ICS provided with the specification is blank, waiting for the implementer to complete.  This blank ICS provides implementers and users a quick overview of features defined in the specification. A completed ICS not only provides information on what has been implemented (mandatory and optional features), but can also be used to document the presence of extensions or any specializations that have been made.  A completed ICS provides information useful to facilitate the selection of applicable tests for the particular implementation.  However, that is not all.  Although the ICS content is independent of testing, associating it with conformance tests makes it an essential piece in the reporting of conformance results (see techniques in Good Practice 05).

Related

Techniques
  1. Create a list, table or form listing all features (capabilities) and indicating which are mandatory to implement.
  2. Provide space for the implementer to check:  yes (to indicate the feature is implemented), No (to indicate it is not implemented), Not Applicable, and space for comments.
  3. Organize the features according to the subdivisions of the specification or in the order they occur in the specification or in some other logical grouping
  4. If there are dependencies, express them. (For instance, if No to this question, jump to the next section.)
  5. Provide a tool to help implementers fill out the ICS and produce a report (e.g., in EARL [EARL]).
Examples

QA Specification Guidelines provides an ICS [QA-SPEC-ICS] to help implementers to asses conformance to this document. Good practices (informative) and Requirements (normative), organized following the sections of the document, are given in a table where implementers can check yes, no or not applicable, and add comments on each of these.

Web Content Accessibility Guidelines 1.0 [WCAG10] provides a checklist of checkpoint which helps the implementer to verify the accessibility of its HTML document. (See checklist of checkpoints in [WCAG10])

Good Practice 05: Require an Implementation Conformance Statement as part of valid conformance claims.

What does it mean? This simply puts together the previous two Good Practices. Not only could the specification provide an ICS pro forma for implementers, but also the specification could require a link to the ICS pro forma from its standardized conformance claim template.

Why care? Providing a completed ICS with the conformance claim might help customers and users to determine quickly the implemented capabilities as well as easily verify the level of support for individual requirements of the specifications.  Combining the ICS with a conformance test suite, can strengthen the claim.  Specifically, the ICS augmented with links to conformance tests, provides a very nice way to indicate not only what has been implemented, but also, what has been implemented correctly (i.e., conforms to the specification).

Related

Techniques
  1. Give precise instruction how the ICS becomes part of the conformance claim. It might be an external document, a link to a precise dated document, etc.
  2. Augment the ICS by providing links to the test suite, such that each feature has associated with it a test (or set of tests).  Explain what it means to check Yes or No.  Specifically, does Yes/No indicate that the implementation has the relevant feature and passes the applicable tests or does Yes/No only indicate that the feature is implemented. In the latter case, add an additional column, to indicate the result of executing the tests.  To avoid confusion as to the role of ICS, we recommend adding an additional column.
Examples

The WebCGM specification requires an ICS as part of a valid conformance claim. The WebCGM checklist describes the conformance of the subject viewer product to the WebCGM specification, according to its performance on the WebCGM Test Suite.

2.2 Setting up ground rules

2.2.1 Scope

The path to a quality specification begins with its scope.  It is critical to convey what the specification is about by describing its intent and applicability.  As the specification develops, it is a good idea to revisit the scope to make sure it still reflects the intent of the specification or if it needs to be modified.

Requirement 02: Define the scope.

What does it mean? Describe what the specification is about.  Let the reader know the topics covered in the specification.

Why care? This is one of the first sections a reader reads, so it is important to capture their attention and make sure they understand what the specification is about. It helps to keep the specification content focused. It helps reviewers determine when the specification is over-stepping its mandate and then gives the possibility to revise the specification in development. It also helps readers know the limits or boundaries of the specification and whether it is of interest to them.

Techniques
Examples

Many W3C specifications have included scope prose in the Abstract section. We advocate making the scope a separate section in the body of the specification, making it easy to find and insuring that it is an item in the table of contents.

Good Scope section:

Could have been better:

  • Mathematical Markup Language (MathML) Version 2.0 (Second Edition) [MATHML20] – Introduction, although very extensive and complete, does not give a quick view of what MathML is about.  The Abstract is much better at providing a concise description of the scope of the Recommendation
  • WCAG 1.0 [WCAG10] also has extensive and complete Introduction, but no scope.  The Abstract provides a very direct statement on the scope of the Guidelines.
Good Practice 06: Provide examples, use cases, and graphics.

What does it mean? Illustrate concepts, behaviors, functionality, interaction, etc. through whatever means makes sense, such as examples, use cases, graphics, and sample code. This aids in the understanding of the specification, especially for areas that are innately complex or for which the Working Group has had to explain to its members or the public.  Additionally, a set of broad examples and use cases can help to clarify the specification’s scope.

Why care? It is difficult to understand some concepts, behaviors, functionality, or other aspects of a specification without informative interpretations to aid the reader. Providing the reader with additional information beyond the specification’s prose, can only help in achieving implementation and interoperability.

Techniques

It is up to the Working Group to determine the best way to illustrate the scope and other parts of the specification. Typically, the nature of the specification influences the type of examples, uses cases, graphics, etc. that make sense.

Examples

For markup specifications, provide at least one example of each markup construct; illustrate each transformation capability with an example showing input and output.

SVG 1.1 [SVG11]: For each element of the SVG specification, there is a verbose definition of the element, the DTD definition, the attribute definitions and an example. For instance, in the definition of the element rect, there are precise examples with the markup to generate a rectangle, a rendering of the markup as an image to help people visualize it and a separate file with the said markup.

HTML 4.01 [HTML401]: The HTML 4.01 specification, designed in a very educative way, has some very good examples.

Good Practice 07: Write sample code or tests.

What does it mean? For each feature, the Working Group might seek early implementation to demonstrate the feature. If early implementations are not available (e.g., due to commercial constraints, IPR, etc.), it is beneficial to write test cases to illustrate a concept or use case of the technology. This provides a way to to study the interactions between the different parts of the specification and reveal problems. Additionally, these test cases can be incorporated into a test suite.

Why care? Developing a partial implementation (sample code) or test cases can provide an understanding of a concept or feature as well as help to keep it focused. It can save the Working Group and eventually implementers time and resources by:

Related

Techniques
  1. Encourage the development of proofs of concept implementations of the technology.
  2. Provide at least one example of each feature, which may also be used as the basis of a future test case.
  3. Do not put a feature into a specification without the corresponding test cases.
  4. Create a template for new feature proposals that includes a request for associated test cases.
Examples

The OWL Working Group has synchronized the publication of their specification and the publication of the OWL Test Cases [OWL-TEST]. They even went a bit further by making the test case, the necessary step to develop a feature with its requirements.

2.2.2 What needs to conform?

Requirement 03: Identify who or/and what will implement the specification.

What does it mean? Clearly identify the class of products (i.e., type of products or services) upon which the requirements are imposed.  If multiple classes of products are targeted by the specification, make sure each is described.   Examples of classes of products include: content, producer of content, player, protocol, API, agent, and guidelines.

Why care? It helps define the scope of the specification and is needed when defining conformance. It also helps the reader know the target of the specification – that is, to discover and focus on what they have turned to the document for and avoid what they may find immaterial.

Related

Techniques
Examples

Mathematical Markup Language (MathML) Version 2.0 (Second Edition) [MATHML20] defines output-compliant authoring tools and input-compliant rendering/reading tools.

SMIL 2.0 Language Profile ([SMIL20], chapter 13) has two classes of products: documents and basic user agents.

The conformance section of Ruby [RUBY] is very explicit and detailed about classes of product. For each of these classes, Ruby conformance section defines a set of rules, the implementers must respect. It defines rules for markup, DTD, document, module, generator, and interpreter.

2.2.3 Normative (and non-normative) references

Rarely written in isolation, a specification inherits from previously defined technologies. It also might set the future of other specifications by defining their base. Thus, it is essential to clearly define what is the nature of the references to these specifications (normative or informative) and the implications of these references for the future of the technology itself.

Requirement 04: Make a list of normative references.

What does it mean? A specification is rarely developed from scratch: it usually relies on other technologies defined in different specifications. The Working Group has to identify any specifications that define the core technologies of the developed technology.

Why care? For the Working Group, it has an immediate benefit: “do not reinvent the wheel”. Using features already defined in other documents helps to minimize the size of the document that is developed and avoid ambiguities by rewriting the same concepts.

Knowing the parts of the specification based on another technology is of huge benefit for implementers as it clarifies the implications for conformance. It may help them to minimize their work by using conformant libraries already implemented elsewhere.

More generally, it might help readers understand where the technology is coming from and therefore how to use it in combination with other technologies they may already know.

Related

Techniques
Examples

Most W3C specifications contain a list of normative references, clearly identified as such, at the end of the document.

Good Practice 08: When imposing requirements by normative references, address conformance dependencies.

What does it mean? Each addition of a normative reference to the specification has deep implications on the technology. Specification editors are responsible for reviewing the consequences in terms of consistency, precision, possible future changes or obsolescence as well as use of the technology under specific conditions.

Why care? A specification defines a technology potentially with a long lifespan. The choice of precise and exact normative references is thus fundamental. Using a normative reference that evolves over time might endanger the specification or other specifications relying on it. A vague reference to the other specification as a whole may leave room for conflicting interpretations or choices among variations permitted by the other specification.

For the Working Group, reducing the degree of ambiguity or variation in the normative references minimizes or removes the possibility of misunderstanding. For implementers, it removes ambiguities and contradictions between different sets of technologies. It creates a stable environment for their development efforts.

For conformance testing to be practical, all requirements needs to be unambiguous, including those imposed by normative reference to other specifications.

Related

Techniques
Examples

For the purpose of illustration, the following cases demonstrate some of the problems and implications that may occur for a theoretical technology using the notion of URI or URI-references — this example is not an exhaustive review of all possible cases. The definition of URI and URI references technology exists in a specification called RFC2396 entitled “Uniform Resource Identifiers (URI): Generic Syntax”.

Let us create a simple definition and give examples of possible problems that arise from this definition:

The value of the attribute is a URI as in [RFC2396]

For instance:

...="http://www.example.org/#foo"
...="http://[3ffe:2a00:100:7031::1]/"
...="http://666.666.666.666/"
...="foo"
...="http://www.example.org/~anaïs-nin"

Precise designation and reference: The first example is illegal as the example uses a URI Reference as opposed to only a URI; RFC 2396 clearly distinguishes between those constructs. To make the first example a valid construct, the text should have said:

The value of the attribute is a URI Reference as defined in section 4 of [RFC2396]

Superset of the reference and interpretation: RFC 2396 does not include support for IPv6 literals; RFC 2732 introduced the syntax but it does not update RFC 2396. It is not correct to assume that it does even if it seems logical. Do not interpret the intention of the external reference.

As a separate example, any specification that defines the behavior of a class of product that creates XML should address creation of XML 1.1 and anticipate future XML versions. In XSLT, creation of XML is specified by <xsl:output method="xml" version="1.0" />, and each version of XSLT defines the allowable range of values for the version attribute. Another option is to reference the XML Infoset - for instance, XML Inclusions are compatible both with XML 1.0 and XML 1.1 since they reference normatively the XML Infoset, which is the same for the two versions of XML.

In its section Referencing the Unicode Standard and ISO/IEC 10646, the specification Character Model for the World Wide Web 1.0: Fundamentals [CHARMOD] gives a detailed indications for referencing Unicode Standard and ISO/IEC 10646. Specification editors are encouraged to follow these recommendations.

2.3 Defining and using terminology

2.3.1 Define terms

Requirement 05: Define the terms used in the normative parts of the specification.

What does it mean? The normative parts of a specification often use technical terms in a very restricted sense; write down the definitions behind these terms. Use the same phrases to convey the same meaning. Repetition, considered a stylistic error in prose, diminishes ambiguity in a technical specification and lowers the threshold of vocabulary needed to understand the specification.

Why care? English (as any other natural languages) is ambiguous, such that a term's interpretation is context dependent. Implementers can achieve interoperability between implementations only if they have the same understanding of the specification; defining the terms used in the normative parts promotes a common understanding that contributes to accurate implementation.

In addition, well-defined terms are reusable in other specifications.

Techniques
  1. Review the conformance requirements/test assertions: the most important terms to define in the specification are usually easy to identify when reviewing conformance requirements or test assertions.
  2. Use the element dfn in HTML to indicate that this is the defining instance of the enclosed term. It will be easier to create a glossary of your terms later on. For example in this document 
    <dfn>Conformance</dfn> is the fulfillment 
of a product, process, or service of specified 
requirements.
Examples

Cascading Style Sheets, level 2 (CSS2) Specification [CSS20]: The interoperability of the implementation of the CSS2 box model has been problematic, due to the lack of definition for when a property is set (see discussion on this topic on the www-style Mailing list in March 2001).

Requirement 06: Create conformance labels for each part of the conformance model.

What does it mean? Many specifications define more than one type of conformance, where each type is applicable to a different class of product. For example, a language specification may define two conformance types – one for a parser and another for documents (i.e. content). Associate a well-defined label for each different type of conformance.

Why care? Having a label associated with each type of conformance helps interoperability, testing, and branding. It gives implementers a way to identify and discuss their implementations and express the degree to which an implementation has met a specific set of requirements in the conformance clause. It gives test developers a meaningful set of requirements to develop tests for and against which to make conformance verdicts. It gives users a way to articulate their buyer requirements by having a unique way to refer to conforming implementations.

Techniques
  1. Review all the different types of conformance in the specification (ideally, they should be listed in the conformance clause)
  2. Make sure all types have an associated label and this label is clearly defined. Provide a summary of the conformance labels, along with the name and its definition.
Examples

XML 1.0 [XML10] defines a well-formed XML document, a valid XML document.

WCAG 1.0 [WCAG10] defines a level A conformant document.

SVG 1.1 [SVG11] defines “conforming SVG document fragments”, “conforming interpreters”, “conforming viewers”.

Good Practice 09: Define unfamiliar terms in-line and consolidate the definitions in a glossary section.

What does it mean? Put the definition of any new term along with its first occurrence in the text, but make sure that all the definitions are also available from a central glossary section.

Why care? Having the definition in-line makes it easier to read the whole specification from top to bottom, while having a centralized section of defined terms makes it handy to look up a term when looking at a specific section, or for others to reuse the definitions.

Techniques
  • With XMLspec, using <termdef> and <term> to markup the in-line definitions makes it possible to get automatically a glossary with <glist>.
  • With XHTML, an XSLT style sheet can extract the glossary from the in-line definitions.
Examples

Thanks to XMLspec, XML Schema Part 1 [XML-SCHEMA-1] has both the in-line definition and the glossary

Good Practice 10: Use terms already defined without changing their definition.

What does it mean? When a definition for a term already exists (e.g. in a different specification) and matches the specification needs, reuse the term and its definition without changing it, and provide a reference to the source.

Why care? Reusing existing terms reduces the cost of creating new definitions and makes it easier for readers already familiar with other specifications to get into the new one. In addition, conflicting definitions for the same term leads to reduced interoperability.

Techniques
  • Use the W3C Glossary [W3C-GLOSSARY] system to check if there are existing definitions of the given term.
  • When quoting a definition from another specification, put a link back to the specification.
Examples

QA Specification Guidelines reuses the terms defined globally in the QA Glossary [QA-GLOSSARY] and used by all documents of the QA Framework.

2.3.2 What is mandatory?

Requirement 07: Use a consistent style for conformance requirements and explain how to distinguish them.

What does it mean? Specifications use different styles to convey conformance requirements: RFC 2119 [RFC2119] keywords, imperative voice, descriptive assertions, etc. Tell the readers what styles are used, especially when the specification uses different styles for different parts of the specification.

Why care? It is important for readers to be able to differentiate requirements in the specification from non-requirements in order to either implement or review them.

Techniques

  • Using RFC 2119 [RFC2119] Keywords (MUST, SHOULD, MAY, ...) makes it easy to spot conformance requirements. According to the RFC itself, they should be used only to establish interoperation [WIKI-RFC-KEYWORDS]. They are usually written with distinctive formatting, such as upper case or bold. It is a good idea to create a specific markup for them too. It will be easier to extract conformance requirements and better for accessibility (See The Manual of Style: RFC 2119 Key Words [MANUAL-STYLE]).

    A good conformance requirement using RFC Keyword is of the form: subject rfc_keyword operation, where subject is one of the classes of product, rfc_keyword one of MUST, SHOULD, MAY, and operation a verb describing one of the operations the classes of product can do (e.g. “send a message”, “process a request”).

  • The descriptive style takes a different approach. It describes the semantics of the language, rather than how it must be handled. The benefits of this approach is that it allows a wider reuse of the said semantics, but at the cost of not defining a common behavior between implementations, which may lead to interoperability issues (see also the Wiki discussion on Meaning vs. Behavior [WIKI-MEANING-BEHAVIOR]). Whatever the descriptive style chosen, stick to it.
  • The imperative style uses the imperative form to convey the requirement; guidelines or specifications that need the reader's involvement often uses it. Its weaknesses are that it does not necessarily make clear what needs to conform (since there is no subject), and the use of the imperative voice may make it harder to translate in some languages.
  • Avoid using language that looks normative (e.g. with RFC Keywords) in informative section.

Using one of these styles does not preclude using another one in a different part of the specification, provided the reader is adequately informed. For instance, when defining a language, it is a good idea to define first its semantics using the descriptive style, and then the behavior of one (or more) type of implementations using RFC Keywords.

Examples

SVG 1.1 [SVG11] Recommendation, where the semantics are defined in descriptive style, and the implementations requirements with RFC Keywords in the SVG 1.1 Conformance section.

Requirement 08: Indicate which conformance requirements are mandatory, which are recommended, and which are optional.

What does it mean? Depending on the way conformance requirements are specified, it may or may not be clear if an implementation needs to implement all of them or only part of them. Try to make sure the readers can easily distinguish the different types of requirements.

Why care? If implementers do not have the same understanding of what is required, interoperability is likely to suffer in the end.

Techniques
Examples

For instance, HTTP 1.1 [HTTP11] defines two types of conformance, one where all the MUST are respected, and one where all the MUST and the SHOULD are implemented.

Good Practice 11: Use formal languages when possible.

What does it mean? If an existing formal language (e.g. DTD, Schemas, ...) is expressive enough to describe the technical requirements of the specification, use it and when the English prose and the formal language overlap, make it clear which one takes precedence in case of discrepancy.

Why care?When possible, there is an immediate benefit of using a formal language to describe conformance requirements. It minimizes ambiguities introduced by the interpretation of the prose. There is also the possibility of using existing tools for the given language to facilitate testing and validation.

However, prose remains necessary to allow implementers to understand the specification, as well as to express additional requirements the formal language cannot express; this means that there are possible overlaps between the prose and the formal language, in which case, it is important to define which one is the main point of reference in case of disjunction.

Related

Techniques
  • There are plenty of formal languages used across W3C specifications: DTD, XML Schema, Relax NG, EBNF, Z Notation, etc. Picking the right one depends on the kind of specifications developed (language, XML or not, protocol) and the benefit from the formal language.
  • To avoid discrepancies between the English prose and the formal language, set up a process so that a given section is bound to a given part of the formal language, and one cannot be modified without the other.
  • Use the formal language tools to validate the examples given in the specification, to ensure they match.
  • When using several formal languages in combination, generate random content according to the rules defined in one of them and try to validate it with the others, to find discrepancies.
Examples

XQuery Formal Semantics [XQUERY-SEMANTICS] section 1.1 defines where the document is normative over the grammar specs (separate for XPath and XQuery) and where the grammar specs are normative.

Good Practice 12: Write Test Assertions.

What does it mean?A test assertion is a measurable or testable statement of behavior, action, or condition. It is contained within or derived from the specification's requirements and provides a normative foundation from which one or more test cases can be built.

Why care? Test assertions facilitate the development of consistent, complete specifications and promote the early development of test cases. Developing or extracting test assertions helps uncover inconsistencies, ambiguities, gaps, and non-testable statements in the specification. It can provide early feedback to the editors regarding areas that need attention. An added benefit is that the assertions are usable as input to test development efforts.

Related

Techniques
  • Create a template for new feature proposals that includes a section for adding test assertions.
  • Identify all requirements in the specification and try to write corresponding test assertions.
  • Try writing test assertions when adding features to a specification. Not being able to write a test assertion for the feature suggests that there is a problem in the way the feature is designed or explained.
Examples

Example 1: SOAP version 1.2 Test Assertions [SOAP12-TA]

Assertion x1-conformance-part2

Location of the assertion: SOAP 1.2 Part 1, Section 1.2

Text from the specification: The implementation of an Adjunct MUST implement all the pertinent mandatory requirements expressed in the specification of the Adjunct to claim conformance with the Adjunct.

Comments: This statement applies to all assertions in part 2 and as such will not be tested separately.

Example 2: HTML 4.01 Test Suite [HTML401-TEST]

Assertion 6.14-1

Reference: Section 6.14

(must) Script data ( %Script; in the DTD) can be the content of the SCRIPT element and the value of intrinsic event attributes. User agents must not evaluate script data as HTML markup but instead must pass it on as data to a script engine.

Tests: 6_14-BF-01.html

Example 3: XML Test Suite [XML-TEST]

Section: Documents

Type: Well_Formed

Purpose: A well formed document must have one or more elements.

Level 1

2.4 Managing Variability

Specifications allow for variation between conforming implementations for different reasons, e.g., adaptation to hardware capacities and extensibility.  Variability, while it can provide for broader usage of the technology, may impede interoperability.  Watch out for excessive variability – that which goes beyond the necessary.  Look for a balance between what is needed to allow for flexibility while still achieving the desired interoperability.

This section gives advice on finding the right balance; the reader will also benefits from reading, Variability in Specifications [VIS] which goes into more details on the analysis of this variability.

2.4.1 Subdivide

Subdividing the technology should be done carefully.  Too many divisions complicates conformance and can hinder interoperability by increasing the chances of conflict with other aspects of the specification (e.g., other subdivisions).  Be smart when dividing the technology so that it is not excessive and provides a positive impact on implementation and interoperability.  The benefits of subdividing should outweigh the drawbacks.

   

Benefits: Subdividing can

  • make the technology easier to implement
  • facilitate incremental implementation
  • increase interoperability by focusing the technology on specific needs 
  • help organize the structure of the technology
  • provide better normative guidance than recommended or optional features in the "core" spec
  • provide names for feature bundles, facilitating automated negotiation between sending and receiving products

Drawbacks: Too many divisions can

  • complicate conformance – need to account for interrelationships with other subdivisions and variability (e.g., extensibility)
  • hurt interoperability – increases likelihood of incompatible implementations
  • increase misinterpretation or cause conflict of requirements due to multiple or duplicated requirements. 
Good Practice 13: Create subdivisions of the technology when warranted.

What does it mean? It may make sense to subdivide the technology into related groups of functionality to target specific constituencies, address specific capabilities or hardware considerations, provide for incremental implementation, facilitate usability, etc., but consider carefully the costs/benefits analysis before doing so.

If the technology is subdivided, indicate what subdivisions exist; if it is not, state it in the conformance section.

Figure 2: One possible organization of Profiles, Modules and Levels

Graph illustrating one possible organization of profiles modules levels

  • Module A and Module B are core modules. They are mandatory for all implementations. We could say that they define the first level of conformance which we might call Basic Conformance.
  • Implementing Module C and Module D is required to reach the second level of conformance. The Level 2 includes Level 1.
  • Module E is an optional module of the technology.
  • Profile X defines an implementation of Level 2 (and thus, implicitly, Level 1 also) for a specific class of products for example.
  • Profile Z defines an implementation of Level 1 and an optional Module E.

Why care? For some specifications (e.g., huge, multi-disciplined), bundling functionality into named or anonymous packages can

  • provide an additional level of scoping control,
  • improve readability and the ability to find areas of interest, and
  • facilitate implementation and interoperability, since implementing the entire, monolithic specification may be impractical and undesirable.

Related Variability in Specifications [VIS] contains additional information on  three methods to subdivide technologies, Profiles, Levels and Modules.

Techniques
  • Use profiles. Profiles are subsets of a technology tailored to meet specific functional requirements of application communities.  The specification may define individual profiles, and may also define rules for creating new profiles. An individual profile defines the requirements for classes of products that conform to that profile.  Rules for profiles define validity criteria for profiles themselves.
  • Use functional levels (a.k.a. levels).   Functional levels are a hierarchy of nested subsets, ranging from minimal or core functionality to full functionality.  Levels are a good way to facilitate incremental development and implementation.  
  • Use modules.  Modules are discrete collections of semantically-related units of functionality that do not necessarily fit in a simple hierarchical structure.  Use modules when functionality can be implemented independently of one another  e.g., audio vs. video module.  It is common to have implementers choose multiple modules to implement.
Examples

Profile:  Mobile SVG Profiles: SVG Tiny and SVG Basic [SVG-TINY] is a profile aimed at mobile phones.

Profile: XHTML Modularization [XHTML-MOD] in section 3 and Synchronized Multimedia Integration Language (SMIL 2.0) Specification [SMIL20] specifications define rules for profiles.

CSS and DOM technologies are examples where levels are the result of progressive historical development and technology enrichment realized in a series of specifications . 

Profile/Level combination: Mobile SVG Profile: SVG Tiny, Version 1.2 [SVG-MOBILE] define three profiles -  Tiny, Basic, Full  - which are nested, like levels, each targeted a specific hardware communities. 

Modules: XHTML Modularization [XHTML-MOD] and Synchronized Multimedia Integration Language (SMIL 2.0) Specification [SMIL20] gives good examples of modules usage for a technology.

Requirement 09: If the technology is subdivided, then indicate which subdivisions are mandatory for conformance.

What does it mean? Regardless of the subdivision technique (i.e., profile, level or module) used, state whether one or more of the subdivisions is required for conformance.

Why care? Subdividing the technology affects and can complicate conformance with all the various combination of choices it provides.  Thinking about the various possibilities helps to structure the conformance model, taking into account how the subdivision can affect various classes of products.  Implementers as well as users need to know what is mandatory, optional, prohibited or conditional with respect to choosing what to implement and still be conforming.

Techniques

In the conformance clause, list the subdivisions that are mandatory for conformance. To help build this list, consider the following questions for each subdivision:

  1. Can an implementation conform without implementing the subdivision?
  2. Does the subdivision apply to specific classes of products and not to others?
  3. Is the subdivision dependent upon other subdivisions (that is, if it is implemented must others also be implemented)?
Examples

Content can be required to conform to one of the subdivisions (e.g., profiles)  or it may be conformant to the specification independently of a subdivision. The question arises for a producer (of content): is it conforming if it generates content that is otherwise valid but does not conform to the subdivision.

WCAG 1.0 [WCAG10] defines three levels of Conformance depending on the levels.

XHTML Modularization [XHTML-MOD] defines minimal requirements for including certain basic modules when designing an XHTML Modularization-conformant document.

Requirement 10: If the technology is subdivided, then address subdivision constraints.

What does it mean? This is a corollary to the Requirement above. Beyond being mandatory or not, subdivisions usually have conformance consequences due to minimal or new requirements, restrictions, interrelationships, and variability.  As part of the conformance clause, describe the constraints associated with each subdivision.

Why care? Creating subdivisions can get complicated, not just for the specification editors but also for implementers who have to choose from the set of subdivisions. Well-designed subdivisions that convey the conditions, constraints, interrelationships, etc. can improve the clarity and understanding of the specification, conformance to the specification, and facilitate implementation and interoperability.

Techniques

In the conformance clause, describe the conditions or constraints on subdivision usage.  To help accomplish this, model or graph the subdivisions indicating the following that applies:

  1. Atomicity of the subdivisions: Represent each subdivision showing whether it can be used only as a whole or not.
  2. Mandatory subdivisions: Label the mandatory subdivisions as such.
  3. Minimal requirements: List the minimal requirements for each subdivision.
  4. Dependencies among subdivisions: Show dependencies and interrelationships of the subdivisions. For example, modules that requires and builds on functionally related modules, i.e., modules that require modules from other functional areas.
  5. Conditions and constraints on subdivisions groups: Indicate conditions and constraints for combined occurrences of subdivision pairs or groups.
  6. Conditions or constraints associated with specific classes of products: Indicate which conditions and constraints are applicable to specific classes or products.
  7. Other conditions or constraints beyond these: Indicate any other conditions or constraints.
 
Examples

Synchronized Multimedia Integration Language (SMIL 2.0) Specification [SMIL20] has a SMIL 2.0 Language Profile for user agents (See section 13 [SMIL20]) but also provides a SMIL 2.0 Basic Profile for wireless and embedded devices (See section 14.3 in [SMIL20]). The SMIL 2.0 language Profile requires that a user agent implement the BasicAnimation module but not the SplineAnimation Module.  The SMIL 2.0 Basic Profile on the other hand does not require implementation of any of the animation modules.

Dependency or intertwined relationship between profiles and modules is common – The combination of XHTML 1.0 [XHTML10], SMIL 2.0 [SMIL20] and SVG 1.1 [SVG11] is an example.

Good Practice 14: If the technology is profiled, define rules for creating new profiles.

What does it mean? If the specification defines profiles of the technology and allows other profiles to be developed outside of the specification itself, then provide the rules for creating these derived profiles.  These profile rules provide instructions for building profiles (e.g., requirements on structure, functionality, encoding, etc.). Derived profiles should not contradict predefined profiles, if there are any in the base specification.

Why care? Well-defined rules facilitate the creation of derived profiles that are well specified, implementable, testable, and can foster a better interoperability across profiles.  If these rules are defined and followed, then the derived profile would conform to the specification.

Related

Techniques
  1. Create two particular profiles.
  2. Identify the common requirements in these two profiles.
  3. Identify the rules that you have followed to create these profiles and write them down.
  4. Try to create a third profile using the defined rules.
Examples

WebCGM 1.0 [WEBCGM10] defines rules to create Profiles (See section 1.2 in [WEBCGM10])

2.4.2 Optionality and Options

Options in specifications provide implementers the freedom to make choices about

These choices, also called discretionary items, give implementers of the technology the opportunity to decide from alternatives when building their applications and tools. They describe or allow optionality of behavior, functionality, parameter values, error handling, etc. They may be considered necessary because of environmental conditions (e.g., hardware limitations or software configuration), locality differences (e.g., language or time zones), dependencies on other technologies, or the need for flexibility.

Although there are perceived benefits to providing optional features, there is also a downside: optional features increase the variations that can exist among implementations. The greatest way to undo the utility of a specification is with too many optional features. Different implementations may use different combinations of optional features. This makes comparisons between implementations difficult as well as complicates conformance testing and increases the chance of non-interoperable implementations.

Story:

A concise XSLT 1.0 [XSLT10] example: the specification grants implementers separate discretion about all of the following aspects of creating attributes in the output:

In each case, one prescribed behavior exists for an implementation that chooses not to raise an error. Thus, the six separate binary choices give rise to 64 different possible behaviors for conformant processors. Typically, an implementer would be content to make a more global choice about raising errors when there is an attempt to create non-well-formed XML results.

Good Practice 15:Use optional features as warranted.

What does it mean? Examine the reason for the optional feature - is it to address a real, existing need? Should it really be optional or made a mandatory part of the specification? Be careful not to provide optional features in anticipation for something that sounds like a good idea but whose implementation is improbable - ask the implementers if they ever plan to need this. Think about the implications of both implementing the optional feature and of not implementing it. Do not make something an option just because the Working Group cannot decide on what to do or cannot reach consensus. As the specification progresses, consider removing unimplemented features.

Why care? A concise list of optional features helps to keep the specification focused and greatly increases the likelihood of obtaining interoperable solutions. Ensuring that only necessary optional features are contained in the specification also makes the job of the implementer easier and reduces costs.

Techniques
Story:

As part of its exit criteria for Candidate Recommendation, a Working Group created a set of tests to "test the specification". The tests were able to show where there was need for optionality (e.g., diversity among implementations and flexibility justified) and where it was possible to narrow the choices (e.g., implementations used a much narrower set of values than those permitted).

Good Practice 16: Clearly identify optional features.

What does it mean? When introducing an optional feature in the specification, label it as such, so that it is easy to find all the optional features defined in the specification. If there are no optional features, state so in the conformance section.

Why care? Options can be useful, but non-judicious use of optional features increases the variability among conforming implementations. In order to minimize the unnecessary optionality, it is a good idea to provide an easy way to identify them. The use of labels for optional features also helps in constructing pro forma conformance claims, comparisons between two implementations, reports to the W3C Director about implementations, etc.

Techniques

There are many ways to tag options. Any technique that distinguishes the optional feature from the required feature is acceptable. Some possibilities include:

Good Practice 17: Indicate any limitations or constraints on optional features.

What does it mean? Provide as much information as possible to narrow the allowable choices and to increase predictability. For implementation dependent values or features, if possible, provide a range or set of permitted values rather than leaving it completely open.

Discuss the implications of either using the optional feature or not. It may be helpful to provide rationale for choosing one option over another or for not using the option at all. Consider the unintended consequence of the optional feature - on other optional features, other parts of the specification, on other specifications.

Why care? Narrowing choices and increasing predictability enhance the likelihood of interoperability since the implementer chooses from a reduced sample space. Narrowing choices, providing more information, and eliminating incorrect choices increases the chances of correct implementations. An enumerated list of values is one way to constrain the choice of optionality.

Techniques

For optional features, especially enumerated lists, make sure to indicate the number of choices/options available for implementation. Specifically, can none, exactly one, or several of the allowable choices be implemented? Does this number depend on other parts of the specification or other chosen options?

Questions to consider include:

Examples

In XPath 2.0 [XPATH20] and its associated functions, the Working Group intends to require support for collation by Unicode code-point, and allow implementations to implement as many other collations as they wish.

2.4.3 Extensibility and Extensions

To accommodate changes in technology and information on the Web, a specification can be designed for extensibility. A specification is extensible when it provides a mechanism to allow an external party to create extensions. Extensions incorporate additional features beyond those defined in the specification. However, extensions can compromise interoperability if there are too many differences between implementations. Features specifically designed to allow new functionality mitigate the impact of extensions. These features provide a "standard way to be non-standard" by including hooks, conformance rules, or other mechanisms by which new functionality may be added in a conforming way, designated as extensibility mechanism.

Requirement 11: Address Extensibility.

What does it mean? Extensions might be encouraged or discouraged by the Working Group. There is a benefit to addressing the value or danger of using extensibility for the given technology. Formalizing the position of the Working Group by a clear defined section and prose removes ambiguities for the specification users about the possibility of developing extension or not. This section should at least address whether the specification is extensible or not.

Why care? Implementers will most likely want to extend the functionalities defined in the specification, if they have specific needs not covered by it. Defining clearly how to hook these extended functionalities into conformance implementations helps ensure consistency in the definition of extensions. This leads to predictable handling of extensions and minimizes issues such as interoperability problems, minimal support, and harmonious future development.

The Working Group may consider that the technology is complete, self-sufficient and does not need to be extensible. In this case, it is necessary to write this clearly in the specification and to explain why the technology is not considered extensible. It might be just for the social benefit of the community to ensure a maximum interoperability.

Related

Techniques
  1. Create a section in the specification dedicated to extensibility.
  2. Call it Extensions.
  3. Make a table of contents entry for it.
  4. Address the following Good Practices of this section.
Examples

The specification CSS3 module: Syntax [CSS3-SYNTAX] has addressed the topic of extensions for CSS 3. The specification clearly identifies extensibility in the table of contents and there is a specific section for it.

Good Practice 18: If extensibility is allowed, define an extension mechanism.

What does it mean? Extensions increase variability between implementations. Defining a mechanism helps to ensure the definition of extensions are consistent.This leads to predictable handling of extensions, including the ability to take appropriate actions (e.g., do the extension, ignore, or take a fallback behavior).

Why care? By planning for extensions and designing a specification for extensibility, there is less chance that extensions will interfere with conformance to the base specification and a better chance at achieving interoperability. Conversely, there may be areas in a specification that would not benefit from extensibility and extensions are strictly forbidden (e.g., permissible characters in XML 1.0 names for elements and attributes [XML10], most of the built-in data types in XML Schema Part 2 [XML-SCHEMA-2]). This is an example of strict conformance. Conformance of an implementation that employs only the requirements and/or functionality defined in the specification and no more defines strict conformance .

Reasons for designing extensibility into a specification include:

  • providing a stable, useful framework to manage the rapid pace of technology change,
  • reducing the chances that extensions will interfere with conformance and increasing the chances for achieving interoperability,
  • providing the ability to “test-drive” new functionality that may migrate into future specifications,
  • enabling implementers to include functionality that they consider a customer requirement.
Techniques

Technology and application needs dictate the best strategy for enabling extensibility.

When designing for extensibility, it can get complicated. Points to consider that can affect design decisions include, but are definitely not limited to, the following topics.

  • Address the use of extension in the conformance section.
  • Give a well-defined template to create extension.
  • Verify that this mechanism prevents the creation of extension conflicting with the semantics of the specification (see the next Good Practice).
  • When needed, define the scope of extensions. (e.g., are extensions authorized only for certain parts of the technology?).
  • As an example, create a fake extension as an example for implementers what would be the right way to do it.
  • Indicate if extensions interact with other extensions.
  • Indicate if extensions are combinable or are mutually exclusive.
  • Indicate if extensions apply to a specification's profiles or modules and not the core part of the specification.
  • Avoid "untested hooks": if an extensibility mechanism is defined, make sure it is well tested during the implementation phase; experience has shown that untested extensibility just does not work.
Examples
Mechanism defined within the specification, extension indicator, error handling instructions

XSLT 1.0 [XSLT10] provides extension mechanisms that allow an XSLT style sheet to determine whether a XSLT processor by which it is being processed:

  • has implementations of particular extensions available,
  • and to specify what should happen if those extensions are not available.

It defines two Boolean functions: function-available (QName) and element-available (QName) that must be present in every implementation. These functions inform the XSLT processor that there is an extension, therefore the XSLT processor can return a value of false and provides handling instructions (e.g., signal an error, perform fallback and not signal error), if the extension is not available.

WSDL 2.0 [WSDL20] defines binding extension elements which are used to provide information specific to a particular binding. It is a two-part extensibility model based on namespace-qualified elements and attributes. It provides the syntax and semantics for signaling extensions. Extension elements can be marked as mandatory, meaning that they must be processed correctly by the WSDL processor - i.e., either agree to fully abide by all the rules and semantics signaled by the extension or immediate cease processing (fault).

Mechanism based on another specification's extension mechanism

CC/PP exchange protocol based on HTTP Extension Framework [CCPP-EXCHANGE] defines a HTTP extension to exchange CC/PP descriptions effectively. The CC/PP exchange protocol conforms to HTTP/1.1 and is a generic extension mechanism for HTTP 1.1 [HTTP11] designed to inter-operate with existing HTTP applications. The CC/PP exchange protocol uses an extension declaration to indicate that an extension has been applied to a message and possibly to reserve a part of the header namespace. It provides rules for which of the HTTP Extension Framework [HTTP-EXTENSION] extension declaration strengths and extension declaration scopes to use and defines the syntax and semantics of the header fields.

OWL Reference [OWL-REF] is a vocabulary extension of RDF Semantics (See section 6 in [RDF-MT]). OWL imposes additional semantic conditions on RDF called semantic extensions of RDF. These semantic extensions conform to the semantic conditions for simple interpretations described in the RDF Semantics Recommendation. The OWL semantics is consistent with RDF semantics, but OWL when understood as RDF is "incomplete" versus the same OWL when understood as OWL. Thus, by understanding OWL, a processor learns more and nothing learned contradicts what the processor learnt by RDF alone.

Good Practice 19: Warn extension creators to create extensions that do not interfere with conformance.

What does it mean? Include in the specification a warning to those who are creating extensions that extensions should not contradict or negate conformance to the original specification. Extensions can be created in different contexts: directly by implementers, in other specifications, etc.

Why care? Conformance should be independent of whether there is an extension or not – if it conformed without the extension, then conformance should hold true with the extension.

Techniques

Include statements in the specification such as:

Examples

In XSLT 1.0 [XSLT10], extension attributes (from other namespaces) can be present on the official XSLT elements, but they are prohibited from changing the specified behavior within the detectability of conforming behavior. Thus, an extension attribute can cause the element to perform faster but cannot change the result.

The CSS2.1 [CSS2.1] specification defines the notion of vendor-specific extension. An initial dash or underscore is guaranteed never to be used in a property or keyword by any current or future level of CSS. Thus typical CSS implementations may not recognize such properties and may ignore them according to the rules for handling parsing errors. However, because the initial dash or underscore is part of the grammar, CSS2.1 implementers should always be able to use a CSS-conforming parser, whether or not they support any vendor-specific extensions.

Good Practice 20: Define error-handling for unknown extensions.

What does it mean? For each class of product affected by an error condition, include error-handling instructions for when an extension is not available or understood.

Why care? When using a strict conforming application, users might have to deal with documents, data considered invalid because they contain errors, or extended syntactically. Developers need to know what is the expected behavior of the application in such context.

Techniques

There are typically two approaches: (see section 4.2.3 Extensibility from Architecture of the World Wide Web, Volume One [WEB-ARCH])

A good way to handle these two approaches is to have a way in the syntax to distinguish which behavior is expected (e.g., mustUnderstand/mustIgnore attributes in SOAP 1.2). Which policy to choose depends heavily on the importance of the processing of the data, the user experience of applications based on the said format, etc.

Do not forget to address all the classes of products. For example, an authoring tool and a rendering tool might behave in different ways.

Examples

Ruby Annotation [RUBY] Specification defines what parts must be ignored and displayed (See section 1.2.2 in [RUBY]) when an unknown element is met.

HTML 4.01 [HTML401] defines the behavior of user agents with regards to invalid documents (See section B.1 [HTML401])

2.4.4 Deprecation

The need for deprecation comes about when there are features (e.g., function argument, element, attribute) defined in the specification that have become outdated and are being phased out, usually in favor of a specified replacement. It may mean for instance that the feature is still there and functional, but

  1. there is a better way of achieving the same thing and the Working Group prefers this better way be used, or
  2. the feature is unused and the Working Group wants to clean up the specification and eventually remove the feature.

From a user point of view, a deprecated feature is one that should not be used anymore, since it may be removed from future versions of the specification. Deprecated features are no longer recommended for use and may become obsolete and no longer defined in future versions of the specification.

Requirement 12: Identify deprecated features.

What does it mean? If the specified technology has already been published in a previous version of the specification, indicate the features from the previous version now deprecated or state in the conformance section that no features were deprecated.

Why care? It helps implementers as well as users know which features may become obsolete in future versions of the technology. This gives them the opportunity to adjust their implementation to phase out the relevant features, and provide new ways to accomplish the same task.

Related

Techniques
  1. Create a list of all deprecated features.
  2. Create a dedicated section for it.
  3. Create an entry in the table of contents going to this list.
  4. For each deprecated feature, create a link to the appropriate definition in the specification.
Examples

HTML 4.01 [HTML401]: The specification has a full list of elements and attributes. The deprecation status appears in both lists. There is an entry in the table of contents to these two lists. Each element/attribute has a link to its definition in the specification.

Namespaces in XML 1.1 [NAMESPACES11], Section 2.2.2 Use of IRIs as Namespace Names discusses the deprecation of relative IRI references, although the information is difficult to find.

Requirement 13: Define how each class of product handles each deprecated feature.

What does it mean? By deprecating a feature, the Working Group indicates its desire that the feature disappear from a future version of the specification. The motivation may be to convert an old feature to a newer one or to remove an old, dangerous, redundant or undesirable feature. Regardless of the reason, it is important to define the affect this has on implementations that may encounter this feature (e.g., consumer products such as user agents or producer products such as authoring tools). How will the user agents or producer products tolerate use of the deprecated feature? Will it signal an error or a warning? Typically, use of a deprecated feature would not affect a consumer (e.g. user agent), while a producer (e.g. authoring tool) should issue a warning.

Why care? Defining how deprecated features are handled, provides a smoother transition for the users of the specified technology and ensures more consistency of the behavior across implementations. It is also particularly important for implementations that need to support different versions of the specification.

For instance, the specification may require that an implementation supports both the features of the new and the old specifications, or suggest a converting mechanism.

Techniques
  1. Consider the effect of deprecation on all classes of products that implement the specification (e.g., authoring tools, converters, and user agents).
  2. Define how it affects conformance.
Examples

In Mathematical Markup Language (MathML) Version 2.0 (Second Edition) [MATHML20], MathML2.0 section 7.2.1.2 describes deprecated MathML 1.x features in terms of MathML-output-conformant authoring tools, MathML-input-conformant rendering/reading tools, and MathML-roundtrip-conformant processors.

HTML 4.01 [HTML401]: In the conformance section of HTML 4.01, there is the definition of deprecation and what user agents should do. The behavior for other kind of products is undefined.

User agents should continue to support deprecated elements for reasons of backward compatibility.

Good Practice 21: Explain how to avoid using a deprecated feature.

What does it mean? Deprecating a feature implies that its use is discouraged, often because there is a better technique available to achieve the same result. For each deprecated feature, it is necessary to explain how implementers and users can avoid using it. It might be helpful to give additional information providing insight into the deprecation motivation.

Why care? Examples and information about each deprecated feature help users smoothly evolve toward the new version of the technology and understand its benefits. This enables a faster adoption of the technology.

It helps implementers understand the rationale for implementing the new technology, to implement an alternative mechanism, and to tool tips or conversion scenarios to help users with the transition.

Techniques
  1. For each deprecated feature, give one or more examples showing the old way and the new way.
Examples

Namespaces XML 1.1 [NAMESPACES11] deprecation of IRI references includes a link to the deprecation ballot results, providing background information on the proposal to deprecate, what this means with respect to conformance to XML 1.0 and Namespaces as well as the affect on other specifications (i.e., DOM, XPath).

HTML 4.01 [HTML401] gives numerous examples on how to avoid the markup that was used in previous versions for deprecated elements.

Good Practice 22: Identify obsolete features.

What does it mean? If the specified technology has been published in a previous version of the specification, indicate the features from the previous version now obsolete, or state in the conformance section that no features were made obsolete.

Why care? It gives a clear message to users and implementers that obsolete features are forbidden and not part of the specification anymore. It helps avoid the creation of documents mixing old and new techniques that would be invalid.

It also helps avoid name clashing. When creating an extension to a technology, implementers are likely to use syntax token for their extended features name. Giving the name of obsolete features helps implementers avoid using the names of previous features that are now obsolete.

Related

Techniques
  1. Create a list of all features that are obsolete.
  2. Create a dedicated section for it.
  3. Create an entry in the table of contents going to this list.
  4. For each deprecated feature, create a link to the appropriate definition in the previous specification.
Examples

HTML 4.01 [HTML401]: The specification has a full list of elements and attributes. The obsolete status appears in both lists. There is an entry in the table of contents to these two lists. Each element/attribute has a link to its definition in the specification.

HTML 4.01, Appendix A: Changes lists obsolete elements and suggests an alternative element for use. The following elements are obsolete: LISTING, PLAINTEXT, and XMP. For all of them, authors should use the PRE element instead.

2.4.5 Error Handling

Good Practice 23: Define an error handling mechanism.

What does it mean? For each class of product affected by an error condition, address error handling. For instance: for a language, address what effect an error (be it syntactic or semantic) in the input has to a processor of this language; for a protocol, address how a party to this protocol should behave when a bogus message is received; for an A.P.I., indicate what exceptions are raised.

Why care? There are many reasons a system may fail; to make a technology reliable, it is crucial to define how it should react to bogus input or conditions. Defining error handling also makes it possible for a user of the technology to react appropriately to the error condition.

Techniques

Different methodologies exist to handle errors in a technology:

Examples

In HTML 4.01 [HTML401], the specification does not define an error handling policy, although it encourages a "mustIgnore" policy.

Cascading Style Sheets, level 2 (CSS2) Specification [CSS20], the specification relies on a well-defined "mustIgnore" policy.

Document Object Model (DOM) Level 3 Core Specification [DOM-CORE-3] uses well-defined exceptions and error values.

XML 1.0 [XML10] is well-known for its strictness with error conditions, where an ill-formed XML document must not be processed.

HTTP 1.1 [HTTP11] specification defines a set of well-known error, standardized through their error codes.

The XSLT 1.0 specification [XSLT10] allows a processor to recover to some of the defined errors.

3. Beyond Conformance

As all editors know, the work is not finished when the writing is completed. In fact, various review and checks of all W3C documents are required prior to their publication and advancement. Aside from the W3C Process and publication rules and before Last Call reviews, many other techniques allow for improvement of the quality and clarity of the document. Many of these things can be an integrated part of the specification development. Ensuring a quality document prior to external reviews can save time and energy in that the Working Group may get fewer comments and issues to resolve.

3.1 Define an internal publication and review process.

Story:

In 2004, QA Working Group documents entered Candidate Recommendation prior to a thorough quality review, resulting in a huge number of issues to resolve and the eventual retreat back to Working Draft for major revisions. Many of the comments could have been prevented, such as inconsistency and incompleteness of the document (e.g., links to supplementary materials that did not exist or were not complete, overlapping and conflicting requirements), undefined terminology, and unimplementable requirements.

Define (formally or not) an internal process for reviewing and developing new parts of the specifications, and how they appear in the drafts published as Technical Reports.

The more the specification work is organized, the more chances to move smoothly across W3C Process, and to have a better final product. Setting up an internal publication/review process allows avoiding recurrent errors in the document, and allows a wider participation to the editorial work, making it possible to develop the specification faster.

Make it fun to do quality control on the specification, by providing tools and templates. For instance, at the start of the work on the specification, create guidelines to help Working Group participants progress on the technology and write submissions for the specification.

Examples

QA Specification Guidelines: For editing each Good Practice and Requirement of QA Specification Guidelines, a template and a method were developed to make the edition of each part easy to follow and discuss by the QA Working Group and easy to incorporate in the final document for the editor.

3.2 Do a systematic and thorough review.

Not only should the Working Group review each part of the technology before publication, it should also review it during the editing phase. It helps the Working Group to identify missing pieces, spelling mistakes, ambiguities, dependencies. With a well-defined review process inside the Working Group, it should not be difficult to accomplish this task.

A Working Draft published with incomplete or very raw sections will probably cause the Working Group to receive many comments.At best, the Working Group will spend much time answering them, or at worst, the incomplete text will go unnoticed and appear in the final Recommendation. So when publishing a version with incomplete sections, make it clear

Related

  1. Create a simple and light review process. For instance, establish a team, where each person focuses on a different aspect of the specification's correctness.
  2. Organize at least one review cycle before publication (more if possible)
  3. Organize the review by topic and expertise. For example, each person checks a different part of the specification or do it by expertise - the grammarian checks for consistency, grammar, spelling, readability, the test builder checks requirements for precision and implementability, the conformant checks that conformance criteria exists.
Examples

For QA Specification Guidelines, a template ([QA-SPEC-TEMPLATE], archived mail) was produced to guide authors, ensuring that each requirement and good practice was written consistently and addressed the same set of information. Upon completion of a requirement or good practice, the text was circulated to the entire Working Group for comments. At the same time, a specific participant of the Working Group was assigned to review the text. This ensured that at least someone in the Working Group had the reviewing responsibility and it would not fall through the cracks. Multiple authors produced the QA Specification Guidelines, utilizing a template.

3.3 Accessibility, Internationalization, and Device Independence Considerations

Beyond the way Working Groups define the conformance model of their technologies, they should also take into account many important considerations that will profoundly affect usage of the technologies. Among them, accessibility, internationalization and device independence are currently supported by W3C Activities and should have a particular focus from other Working Groups.

Workings Groups need to design their technologies with accessibility in mind, esp. if they define technologies used in User Interactions context (e.g. HTML, SVG). Addressing accessibility in a specification increases the likelihood that the defined technology can be accessed by everyone regardless of disability. Otherwise, it may take several revisions before software addresses accessibility features, leaving people with disabilities behind.

For accessibility of XML-based vocabularies defined in a specification, refer to the XML Accessibility Guidelines [XAG]. For other information about specification accessibility, refer to the W3C Web Accessibility Initiative (WAI).

Similarly, the Working Group should make sure the defined technology is compatible with internationalization, i.e. is not specific to a particular language or culture. The benefit of addressing internationalization aspects in a specification is to ensure that the defined formats and protocols do not create barriers for languages, writing systems, character codes, and other local conventions employed by end users of the technology.

For information on interoperable text manipulation for text defined in a specification, refer to the Character Model for the World Wide Web [CHARMOD]. For other information about specification "internationalization", refer to the W3C Internationalization (I18N) Activity.

Finally, a specification of a Working Group should support device independence to the maximum extent possible and appropriate, to provide diversity of interaction with a technology by people. The benefit of addressing device independence in a specification is the increased likelihood that a specification can be accessed from any device, in any context by anyone.

For information about specification device independence, refer to the W3C Device Independence Summary.

For each of these considerations, a Working Group should consider designating a participant to monitor their applications to the specification at the earliest point possible and to be the point of contact with the relevant W3C Working Groups to seek feedback on the adopted solutions.

4. Conformance to this document

The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY ", and "OPTIONAL" are used as defined in RFC 2119 [RFC2119] in this conformance clause. Occurrences of these words in bold lowercase have normative implications. A specific markup has been added to the text.

4.1 Conformance criteria

The conformance model of the QA Framework: Specification Guidelines is very simple. 

The conformance requirements of this document found in the Requirements and in the Good Practices.  The Requirements are written in the imperative voice and denote mandatory conformance requirements.  The are equivalent to the familiar MUST statements in the RFC2119 style.  In addition to Requirements, this document contains Good Practices.  Good Practices use the same imperative voice, but are optional.  They are equivalent to SHOULD or RECOMMENDED statements in the RFC2119 style.   Their implementation or non-implementation does not affect conformance to this Specification Guidelines document, but it is recommended to implement as many as possible, as the quality of the specification will benefit.

For a specification to be Specification Guidelines conformant, all Requirements must be implemented.

One way to satisfy the Requirements and Good Practices is to implement one or more of the suggested techniques given for each Requirement and Good Practice.  Note that this is not the only way to satisfy the Requirement or Good Practice.  An Implementation Conformance Statement (ICS) provides assistance in keeping track of implemented Requirements and Good Practices.  It takes the form of a checklist [QA-SPEC-ICS].  If all the Requirements are checked on the ICS as being satisfied, then conformance can be claimed as detailed below. 

4.2 Normative Parts

The Conformance section, the Normative References section, the Requirements, and the Good Practices are the normative parts of this specification. All the other parts are informative.  The following list indicates the normativity of the guidelines subsections.

Normative and informative status of the different parts
Part Status
Requirement Normative
required
Good Practice Normative
optional
What does this mean? Informative
Why Care? Informative
Techniques Informative
Examples Informative

4.3 Specification Guidelines Extensibility

This specification is extensible.  That is, adding conformance related information and structure to the document in ways beyond what is presented as Requirements in this specification, is allowed and encouraged.  Extensions to this specification must not not contradict nor negate the requirements in this specification.

4.4 Conformance claims

To claim conformance to the QA Framework: Specification Guidelines, Working Groups must specify:

You can claim conformance to this document by using this conformance claim and replacing the content holders delimited by square brackets by the appropriate values:

On [date of the publication], this specification [name of the specification] published at [URI of the specification], edited by [name of the publishing entity], is a Specification Guidelines conformant specification, April 28, 2005 published at http://www.w3.org/TR/2005/WD-qaframe-spec-20050428/, as detailed in the Implementation Conformance Statement published at [URI of the completed ICS].

5. Acknowledgments

The following QA Working Group and Interest Group participants have contributed significantly to the content of this document:

6. References

Normative References

RFC2119
IETF RFC 2119: Key words for use in RFCs to Indicate Requirement Levels, S. Bradner, March 1997. Available at http://www.ietf.org/rfc/rfc2119.txt .

Informative References

CCPP-EXCHANGE
CC/PP exchange protocol based on HTTP Extension Framework , Hidetaka Ohto, Johan Hjelm, Editors, W3C Note, 24 June 1999, http://www.w3.org/1999/06/NOTE-CCPPexchange-19990624 . Latest version available at http://www.w3.org/TR/NOTE-CCPPexchange .
CCPP-VOCAB
Composite Capability/Preference Profiles (CC/PP): Structure and Vocabularies 1.0 , L. Tran, F. Reynolds, M. H. Butler, C. Woodrow, J. Hjelm, H. Ohto, G. Klyne, Editors, W3C Recommendation, 15 January 2004, http://www.w3.org/TR/2004/REC-CCPP-struct-vocab-20040115/ . Latest version available at http://www.w3.org/TR/CCPP-struct-vocab/ .
CHARMOD
Character Model for the World Wide Web 1.0: Fundamentals , F. Yergeau, R. Ishida, M. Wolf, M. J. Dürst, T. Texin, Editors, W3C Recommendation, 15 February 2005, http://www.w3.org/TR/2005/REC-charmod-20050215/ . Latest version available at http://www.w3.org/TR/charmod/ .
CONF-TEMPLATE
Specification Guidelines Conformance Clause Template, Lofton Henderson, Lynne Rosenthal, 30 August 2004, available at http://www.w3.org/QA/2004/08/SpecGL-template-root.html .)
CSS20
Cascading Style Sheets, level 2 (CSS2) Specification , H. Lie, I. Jacobs, C. Lilley, B. Bos, Editors, W3C Recommendation, 12 May 1998, http://www.w3.org/TR/1998/REC-CSS2-19980512/ . Latest version available at http://www.w3.org/TR/REC-CSS2/ .
CSS21
Cascading Style Sheets, level 2 revision 1 , T. Çelik, I. Hickson, H. Lie, B. Bos, Editors, W3C Candidate Recommendation (work in progress), 25 February 2004, http://www.w3.org/TR/2004/CR-CSS21-20040225 . Latest version available at http://www.w3.org/TR/CSS21 .
CSS3-SYNTAX
CSS3 module: Syntax , L. Baron, Editor, W3C Working Draft (work in progress), 13 August 2003, http://www.w3.org/TR/2003/WD-css3-syntax-20030813/ . Latest version available at http://www.w3.org/TR/css3-syntax .
CSS-TEST-DOC
CSS Test Suite Documentation, Tantek Çelı̇k, Ian Hickson, 29 January 2003, http://www.w3.org/Style/CSS/Test/testsuitedocumentation-20030129.html Latest version available at http://www.w3.org/Style/CSS/Test/testsuitedocumentation.html .
DOM-CORE-3
Document Object Model (DOM) Level 3 Core Specification , P. Le Hégaret, G. Nicol, L. Wood, M. Champion, J. Robie, S. Byrne, A. Le Hors, Editors, W3C Recommendation, 7 April 2004, http://www.w3.org/TR/2004/REC-DOM-Level-3-Core-20040407/ . Latest version available at http://www.w3.org/TR/DOM-Level-3-Core/ .
EARL
Evaluation and Report Language (EARL) 1.0 , W. Chisholm, S. B. Palmer, Editors, W3C Working Draft (work in progress), 6 December 2002, http://www.w3.org/TR/2002/WD-EARL10-20021206/ . Latest version available at http://www.w3.org/TR/EARL10/ .
EDITOR
W3C Editors' Home Page , Communication Team, Available at http://www.w3.org/2003/Editors/ .
EXT-SPECGL
QA WIKI Extension Speclite , Available at http://esw.w3.org/topic/ExtensionSpeclite .
HTML401
HTML 4.01 Specification , A. Le Hors, D. Raggett, I. Jacobs, Editors, W3C Recommendation, 24 December 1999, http://www.w3.org/TR/1999/REC-html401-19991224/ . Latest version available at http://www.w3.org/TR/html401/ .
HTML401-TEST
HTML 4.01 Test Suite , Available at http://www.w3.org/MarkUp/Test/HTML401/ .
HTTP-EXTENSION
IETF An HTTP Extension Framework , H. Nielsen, P. Leach, S. Lawrence, February 2000. Available at http://www.ietf.org/rfc/rfc2774.txt .
HTTP11
IETF RFC 2616: Hypertext Transfer Protocol — HTTP/1.1, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, T. Berners-Lee, June 1999. Available at http://www.ietf.org/rfc/rfc2616.txt .
IETF-FORMAL
Guidelines for the use of formal languages in IETF specifications, IESG Statement (work in progress), October 2001. Available at http://www.ietf.org/IESG/STATEMENTS/pseudo-code-in-specs.txt .
ISO-GUIDE
ISO/IEC Guide 2:2004 Standardization and related activities - General vocabulary. (See http://www.iso.ch/iso/en/CatalogueDetailPage.CatalogueDetail?CSNUMBER=39976 for the latest version.)
MANUAL-STYLE
W3C Manual of Style Susan Lesch, Available at http://www.w3.org/2001/06/manual/ .
MATHML20
Mathematical Markup Language (MathML) Version 2.0 (Second Edition) , P. Ion, R. Miner, D. Carlisle, N. Poppelier, Editors, W3C Recommendation, 21 October 2003, http://www.w3.org/TR/2003/REC-MathML2-20031021/ . Latest version available at http://www.w3.org/TR/MathML2/ .
NAMESPACES11
Namespaces in XML 1.1 , R. Tobin, T. Bray, A. Layman, D. Hollander, Editors, W3C Recommendation, 4 February 2004, http://www.w3.org/TR/2004/REC-xml-names11-20040204/ . Latest version available at http://www.w3.org/TR/xml-names11/ .
OWL-REF
OWL Web Ontology Language Reference , M. Dean, G. Schreiber, Editors, W3C Recommendation, 10 February 2004, http://www.w3.org/TR/2004/REC-owl-ref-20040210/ . Latest version available at http://www.w3.org/TR/owl-ref/ .
OWL-TEST
OWL Web Ontology Language Test Cases , J. De Roo, J. J. Carroll, Editors, W3C Recommendation, 10 February 2004, http://www.w3.org/TR/2004/REC-owl-test-20040210/ . Latest version available at http://www.w3.org/TR/owl-test/ .
QA-GLOSSARY
Comprehensive glossary of QA terminology. (Under construction, at http://www.w3.org/QA/glossary.)
QA-HANDBOOK
The QA Handbook, L. Henderson, Editor, W3C Working Group Note (work in progress), 22 November 2004, http://www.w3.org/TR/2004/NOTE-qa-handbook-20041122/ . Latest version available at http://www.w3.org/TR/qa-handbook/ .
QA-SPEC-ICS
QA Specification Guidelines ICS, Karl Dubost, 22 November 2004, http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/specgl-ics.html
QA-SPEC-TEMPLATE
QA Specification Guidelines Template, Karl Dubost, 10 June 2004, http://lists.w3.org/Archives/Public/www-qa-wg/2004Jun/0023.html
RDF-MT
RDF Semantics , P. Hayes, Editor, W3C Recommendation, 10 February 2004, http://www.w3.org/TR/2004/REC-rdf-mt-20040210/ . Latest version available at http://www.w3.org/TR/rdf-mt/ .
RUBY
Ruby Annotation , T. Texin, M. Suignard, M. Ishikawa, M. Sawicki, M. J. Dürst, Editors, W3C Recommendation, 31 May 2001, http://www.w3.org/TR/2001/REC-ruby-20010531/ . Latest version available at http://www.w3.org/TR/ruby/ .
SMIL20
Synchronized Multimedia Integration Language (SMIL 2.0) - [Second Edition] , E. Hodge, P. Hoschka, K. Kubota, J. van Ossenbruggen, E. Hyche, M. Jourdan, L. Rutledge, B. Saccocio, W. ten Kate, P. Schmitz, R. Lanphier, N. Layaïda, J. Ayars, T. Michel, D. Bulterman, D. Newman, A. Cohen, K. Day, Editors, W3C Recommendation, 7 August 2001, second edition published on 7 January 2005, http://www.w3.org/TR/2005/REC-SMIL2-20050107/ . Latest version available at http://www.w3.org/TR/SMIL/ .
SOAP12-TA
SOAP version 1.2 Test Assertions, Available at http://www.w3.org/TR/2003/REC-soap12-testcollection-20030624/ .
SVG-MOBILE
Mobile SVG Profile: SVG Tiny, Version 1.2 , O. Andersson, J. Ferraiolo, V. Hardy, D. Jackson, A. Quint, Editors, W3C Working Draft (work in progress), 13 April 2005, http://www.w3.org/TR/2005/WD-SVGMobile12-20050413/ . Latest version available at http://www.w3.org/TR/SVGMobile12/ .
SVG-TINY
Mobile SVG Profiles: SVG Tiny and SVG Basic , T. Capin, Editor, W3C Recommendation, 14 January 2003, http://www.w3.org/TR/2003/REC-SVGMobile-20030114/ . Latest version available at http://www.w3.org/TR/SVGMobile/ .
SVG11
Scalable Vector Graphics (SVG) 1.1 Specification , 藤沢 (Fujisawa Jun), J. Ferraiolo, D. Jackson, Editors, W3C Recommendation, 14 January 2003, http://www.w3.org/TR/2003/REC-SVG11-20030114/ . Latest version available at http://www.w3.org/TR/SVG11/ .
VIS
Variability in Specification , Dominique Hazaël-Massieux, Lynne Rosenthal, W3C Working Draft 30 August 2004, http://www.w3.org/TR/2004/WD-spec-variability-20040830/ . Latest version available at http://www.w3.org/TR/spec-variability/ .
W3C-GLOSSARY
W3C Glossary and Dictionary , http://www.w3.org/2003/glossary/
WCAG10
Web Content Accessibility Guidelines 1.0 , G. Vanderheiden, W. Chisholm, I. Jacobs, Editors, W3C Recommendation, 5 May 1999, http://www.w3.org/TR/1999/WAI-WEBCONTENT-19990505/ . Latest version available at http://www.w3.org/TR/WAI-WEBCONTENT/ .
WEB-ARCH
Architecture of the World Wide Web, Volume One , N. Walsh, I. Jacobs, Editors, W3C Recommendation, 15 December 2004, http://www.w3.org/TR/2004/REC-webarch-20041215/ . Latest version available at http://www.w3.org/TR/webarch/ .
WEBCGM10
WebCGM 1.0 Second Release , R. Platon, J. Gebhardt, L. Henderson, D. Weidenbrueck, D. Cruikshank, Editors, W3C Recommendation, 17 December 2001, http://www.w3.org/TR/2001/REC-WebCGM-20011217/ . Latest version available at http://www.w3.org/TR/REC-WebCGM/ .
WIKI-FORMAL-LANGUAGE
QA WIKI Formal Language vs Prose , Available at http://esw.w3.org/topic/FormalLanguageVsProse .
WIKI-GOOD-BAD
QA WIKI Extensibility Good Or Bad , Available at http://esw.w3.org/topic/ExtensibilityGoodOrBad .
WIKI-NORMATIVE-REF
QA WIKI Normative References , Available at esw.w3.org/topic/NormativeReferences .
WIKI-MEANING-BEHAVIOR
QA WIKI Meaning versus Behavior , Available at http://esw.w3.org/topic/MeaningVsBehavior .
WIKI-RFC-KEYWORDS
QA WIKI RFC Keywords , Available at http://esw.w3.org/topic/RfcKeywords .
WIKI-TESTABLE-NOT
QA WIKI Testable Or Not , Available at http://esw.w3.org/topic/TestableOrNot .
WSDL20
Web Services Description Language (WSDL) Version 2.0 Part 1: Core Language , J. C. Schlimmer, R. Chinnici, M. Gudgin, S. Weerawarana, J. Moreau, Editors, W3C Working Draft (work in progress), 3 August 2004, http://www.w3.org/TR/2004/WD-wsdl20-20040803/ . Latest version available at http://www.w3.org/TR/wsdl20/ .
XAG
XML Accessibility Guidelines , C. McCathieNevile, S. B. Palmer, D. Dardailler, Editors, W3C Working Draft (work in progress), 3 October 2002, http://www.w3.org/TR/2002/WD-xag-20021003 . Latest version available at http://www.w3.org/TR/xag .
XHTML-MOD
Modularization of XHTML™ , M. Altheim, F. Boumphrey, S. McCarron, S. Schnitzenbaumer, T. Wugofski, S. Dooley, Editors, W3C Recommendation, 10 April 2001, http://www.w3.org/TR/2001/REC-xhtml-modularization-20010410/ . Latest version available at http://www.w3.org/TR/xhtml-modularization/ .
XHTML10
XHTML™ 1.0 The Extensible HyperText Markup Language (Second Edition) , S. Pemberton, Editor, W3C Recommendation, 1 August 2002, http://www.w3.org/TR/2002/REC-xhtml1-20020801/ . Latest version available at http://www.w3.org/TR/xhtml1/ .
XML-SCHEMA-1
XML Schema Part 1: Structures Second Edition , D. Beech, M. Maloney, H. S. Thompson, N. Mendelsohn, Editors, W3C Recommendation, 28 October 2004, http://www.w3.org/TR/2004/REC-xmlschema-1-20041028/ . Latest version available at http://www.w3.org/TR/xmlschema-1/ .
XML-SCHEMA-2
XML Schema Part 2: Datatypes Second Edition , P. V. Biron, A. Malhotra, Editors, W3C Recommendation, 28 October 2004, http://www.w3.org/TR/2004/REC-xmlschema-2-20041028/ . Latest version available at http://www.w3.org/TR/xmlschema-2/ .
XML-TEST
Extensible Markup Language (XML) Conformance Test Suites , Available at http://www.w3.org/XML/Test/ .
XML10
Extensible Markup Language (XML) 1.0 (Third Edition) , E. Maler, J. Paoli, F. Yergeau, T. Bray, C. M. Sperberg-McQueen, Editors, W3C Recommendation, 4 February 2004, http://www.w3.org/TR/2004/REC-xml-20040204/ . Latest version available at http://www.w3.org/TR/REC-xml/ .
XPATH20
XML Path Language (XPath) 2.0 , A. Berglund, S. Boag, D. Chamberlin, M. F. Fernández, M. Kay, J. Robie, J. Siméon, Editors, W3C Working Draft (work in progress), 4 April 2005, http://www.w3.org/TR/2005/WD-xpath20-20050404/ . Latest version available at http://www.w3.org/TR/xpath20/ .
XQUERY-SEMANTICS
XQuery 1.0 and XPath 2.0 Formal Semantics , M. F. Fernández, K. Rose, D. Draper, M. Rys, J. Siméon, P. Wadler, P. Fankhauser, A. Malhotra, Editors, W3C Working Draft (work in progress), 04 April 2005, http://www.w3.org/TR/2005/WD-xquery-semantics-20050404/ . Latest version available at http://www.w3.org/TR/xquery-semantics/ .
XSLT10
XSL Transformations (XSLT) Version 1.0 , J. Clark, Editor, W3C Recommendation, 16 November 1999, http://www.w3.org/TR/1999/REC-xslt-19991116 . Latest version available at http://www.w3.org/TR/xslt .

Further Reading

The following references have been inspirational to the ideas captured in this document.

7. Glossary

Class of Products
The generic name for the group of products or services that would implement, for the same purpose, the specification, (i.e., target of the specification). A specification may identify several classes of products.
Conformance
Fulfillment by a product, process, systems, or service of a specified set of requirements.
Conformance clause
A section of the specification that defines the requirements, criteria, or conditions to be satisfied by an implementation in order to claim conformance.
Deprecated feature
An existing feature that has become outdated and is in the process of being phased out, usually in favor of a specified replacement. Deprecated features are no longer recommended for use and may cease to exist in future versions of the specification.
Dimensions of Variability (DoV)
The ways in which different products that are conformant to a specification may vary among themselves.
Discretionary Item
Deliberate and explicit grants of discretion by the specification to the implementations that describe or allow optionality of behavior, functionality, parameter values, error handling, etc.
Extensible
The ability of a specification to accept extensions in a defined way. A specification is extensible if it provides a mechanism for any party to create extensions.
Extension
The ability to incorporate additional functionality beyond what is defined in the specification. It broadens the possibility of the technology.
Implementation
An implementation is a realization of a technology in accordance to the principles defined in the technical specifications for this technology. This implementation can be a document, product, application, process, service, system, or other entity.
Implementation Conformance Statement (ICS)
A questionnaire or checklist for providing information about an implementation to a specification, by presenting in a uniform manner the implemented capabilities (e.g., functions, features) and options as well as limitations of the implementation.
Informative
Text in a specification whose purpose is informational or assistive in the understanding or use of the specification, and which contains no conformance requirements or test assertions.
Level
A technology subset that is one of a hierarchy of nested subsets, ranging from minimal or core functionality to full or complete functionally.
Module
A collection of semantically related features that represents a unit of functionality.
Normative
Text in a specification which is prescriptive or contains conformance requirements.
Obsolete feature
An existing or deprecated feature that has ceased to exist and that is listed for historical purpose.
Profile
A subset of a technology that is tailored to meet specific functional requirements of a particular application community.
Specification
Document that prescribes requirements to be fulfilled by a product, process, or service.
Strict Conformance
Conformance of an implementation that employs only the requirements and/or functionality defined in the specification and no more (i.e., no extensions to the specification are implemented).
Test Assertion
A measurable or testable statement of behavior, action, or condition. It is derived from the specification's requirements.
Testability
A proposition is testable if there is such a procedure that assesses the truth-value of a proposition with a high confidence level.