This document records the QA Working Group's (QAWG) response to comments on the Last Call Working Draft of QA Framework: Operational Guidelines Please note that the table below is for Operational Guidelines only. We will generate separate tables for the Specification Guidelines and Introduction specifications.
The full statement of each issue and its full processing history can be found in QAWG's Last Call Issues List. The issue number in the first column of the table links to that. The Resolution column has links to specific text in a Operational Guidelines editor's draft, whose purpose is to illustrate the Disposition.
The last column indicates QAWG's disposition of the issue:
The deadline for replies to this DoC document is 12pm (noon) EDT, Monday, 18 August 2003. Your reply at your earliest convenience — whether you accept QAWG's disposition of your comment(s) or not — will help us to stay on schedule for progression of Operational Guidelines. If we do not hear from you (originator) by the deadline, your default reply is "accept QAWG's disposition of comments".
If you do not accept QAWG's disposition of your comments, please provide details, including your reasons, and also what change to the disposition would be required to satisfy you.
Last update: $Date: 2003/08/27 21:11:26 $
Issue # | Originator | Description | Resolution | Status |
---|---|---|---|---|
LC-1 |
|
require a "Security Considerations" section |
|
not accepted |
LC-78 |
|
Should provide a disclaimer template |
|
accepted |
LC-22 |
|
CP 2.3 placement of Checkpoint |
|
accepted |
LC-23 |
|
CP 1.4 vague |
|
accepted with modifications |
LC-24 |
|
Introduction, Sect 1.4 |
|
accepted |
LC-25 |
|
Introduction: scope and goals |
|
accepted |
LC-26 |
|
Multiple CPs - It is not applicable |
|
not accepted |
LC-27 |
|
Typos, grammar, etc. |
|
accepted |
LC-28 |
|
document organization suggestions |
|
accepted |
LC-29.1 |
|
questions and suggestions: In general, specifications make functional requirements. In some cases, it may be necessary to make a performance requirement. How should that be handled? |
20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: if performance requirements are within the scope of the standard, and they presented the same as any other conformance requirement -- e.g., using RFC2119 keywords -- then there would not seem to be any issue. For conformance purposes, they would not need any special handling. (No changes to SpecGL.) |
accepted with modification |
LC-29.2 |
|
questions and suggestions: In UAAG 1.0, we realized that for some of our configuration requirements, it didn't matter whether the user agent offered the desired configuration or didn't implement the functionality in the first place. E.g., we decided that it was ok for a user agent to not support blinking at all, or, if blinking is implemented, to offer a configuration to turn it off. However, in other cases, the configurability was "just as important" as the functionality to be configured. Authors should consider these when they specify configuration options. |
20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: Since SpecGL 1.0 has not gone to such a level of detail as addressing configuration options, we are unsure how this comment would affect SpecGL. Note that we (QAWG) have decided that the scope of SpecGL 1.0 should be limited to the most critical and basic topics. This could be added to requirements for SpecGL 2.0. It could also be mentioned, possibly, in SpecET (Examples & Techniques). If Originator thinks that this topic fits under one of SpecGL's existing checkpoints, then perhaps he would be interested to make a contribution, to be put into SpecET. The SpecGL scope has been clarified to indicate the "basic and critical" scope of SpecGL 1.0. See draft revised text. |
accepted with modification |
LC-29.3 |
|
questions and suggestions: It might be valuable for a specification to explain how to include its requirements in another specification. This is not the same as explaining how to reference the spec, or how to claim conformance to it. |
Discussed at 20030707 telecon. We (QAWG) have decided that the scope of SpecGL 1.0 should be limited to the most critical and basic topics. We think that this topic has merit, but is more advanced and detailed than we are dealing with in SpecGL 1.0. It could be added to requirements for SpecGL 2.0. Alternately or in addition, it could also be mentioned in SpecET (Examples & Techniques). If Originator thinks that this topic fits under one of SpecGL's existing checkpoints, then perhaps he would be interested to make a contribution, to be put into SpecET. The SpecGL scope has been clarified to indicate the "basic and critical" scope of SpecGL 1.0. See draft revised text. |
accepted with modification |
LC-29.4 |
|
questions and suggestions: It might be valuable to explain some desirable characteristics of a specified technical requirement: Mutual independence from other requirements Expresses a minimal requirement Distinguish and label: requirements, exceptions to those requirements, necessary and/or sufficient techniques for satisfying those requirements. |
Discussed and resolved at 20030623 telecon. Agreed that it is useful to discuss some attributes of well-written conformance requirements. Some verbiage has been added to the introductory text of GL7 (new number). More extensive material may be added to SpecET (contributions would be welcome!). See draft revised text. |
accepted |
LC-29.5 |
|
questions and suggestions: I think that more could be stated about useful ways of allowing extensibility. See, for example, how CSS (forward-compatible parsing), XML, and HTTP handle this. What should be avoided? Is the general practice of "ignore what you don't know" a good idea or a big mistake? |
Resolved at 20030505 telecon, as part of the extensibility issue group resolution. This should not be covered in SpecGL, but it ought to be covered in the SpecET material on extensions. SpecET should illustrate numerous kinds of extensions -- some that have serious interoperability impacts, some that mitigate the impacts, some approaches that might be considered "useful", etc. Also, QAWG discussion sees some problems with examples such as CSS "forward compatible parsing" -- it opens up the ill-defined topic of what is an unknown extension versus what is corrupted or erroneous content. This has been added to the SpecET queue. |
accepted with modification |
LC-29.6 |
|
questions and suggestions: A spec should clearly indicate which illustrations (e.g., images) and examples are normative, if any. |
20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: agreed. The wording of CP13.2 has been adjusted to include all sorts of normative content. See draft revised text. |
accepted |
LC-30 |
|
Profile, module, level |
|
accepted with modifications |
LC-31 |
|
general |
|
accepted |
LC-32 |
|
4. Definitions |
|
accepted with modifications |
LC-33 |
|
4. Definitions |
|
accepted |
LC-34 |
|
Section 3.4 Conformance definition |
|
accepted |
LC-35 |
|
Section 3.2 Extensibility |
|
not accepted |
LC-36 |
|
Section 3.1 Normative sections |
|
accepted with modifications |
LC-37 |
|
CP 9.3 |
|
not accepted |
LC-38 |
|
CP 9.1 and 9.2 combine |
|
accepted |
LC-39 |
|
CP 8.4 policies for discretionary choices |
|
accepted |
LC-40 |
|
GL7 add obsolete features |
|
accepted |
LC-41 |
|
CP 4.4 derived profile |
|
accepted |
LC-42 |
|
GL3: contradiction? regarding examples |
|
accepted |
LC-55 |
|
Accessibility |
|
not accepted |
LC-84 |
|
Group dimensions of variability |
|
accepted with modifications |
LC-85 |
|
Self-sufficient guidelines and checkpoints |
|
accepted with modifications |
LC-86 |
|
Spelling, grammar, and style |
|
accepted with modifications |
LC-87 |
|
Abbreviations |
|
accepted |
LC-88 |
|
Reformat bullet list |
|
not accepted |
LC-89 |
|
Consolidate glossary and terminology |
|
accepted with modifications |
LC-90 |
|
Restructure DoV |
|
accepted with modifications |
LC-91 |
|
Complexity of numbering |
|
not accepted |
LC-92 |
|
Examples vs. Illustrations |
|
accepted with modifications |
LC-93 |
|
Classes vs. categories |
|
accepted |
LC-94 |
|
Loophole in Classes and Categories |
|
accepted with modifications |
LC-95 |
|
Why is conformance policy a DoV? |
|
accepted |
LC-96 |
|
Priorities confusing |
|
accepted with modifications |
LC-97 |
|
Modules as extension points |
|
accepted with modifications |
LC-98 |
|
Conformance levels |
|
not accepted |
LC-99 |
|
Awkward deprecation requirements |
|
accepted with modifications |
LC-100 |
|
Don't discourage extensibility |
|
not accepted |
LC-101 |
|
Remove Checkpoint 9.6 |
|
not accepted |
LC-102 |
|
Consolidate G3 and G10. |
|
accepted with modifications |
LC-103 |
|
Distributed conformance section OK |
|
accepted with modifications |
LC-104 |
|
Consolidate G11 with G3/G10 |
|
accepted with modifications |
LC-105 |
|
G3, G10, G11, G13 |
|
accepted with modifications |
LC-106 |
|
Section 3.1 - Poor Defn of Normative |
|
accepted with modifications |
LC-107 |
|
Section 3.4 - AAA-terminology useless |
|
accepted with modifications |
LC-108 |
|
Definitions |
|
accepted with modifications |
LC-73.1 |
|
Collected substantive & editorial comments: [SG-1]: Editorial (SoTD). |
Resolved, it has been fixed as suggested. |
accepted |
LC-73.2 |
|
Collected substantive & editorial comments: [SG-2]: Remove all "not applicable if..." statements. |
Discussed and resolved at 20030602 telecon (DRAFT). QAWG agrees that there are some editorial problems in these clauses. As pointed out, "It" is undefined (should be "This checkpoint"). There is also inconsistent usage, e.g., some checkpoints in Guideline 9 have applicability exclusions, and some do not. In general, however, it is thought useful that there be an explicit applicability exclusion, rather than leaving the reader to interpret this (*only* if there is an applicability exclusion can the n/a box in the ICS be checked). SpecGL editors will improve the language, clarify the ambiguous statements, and apply consistent structuring, for all checkpoints that have applicability exclusions. See draft revised text. |
accepted with modification |
LC-73.3 |
|
Collected substantive & editorial comments: [SG-3]: "Enumerate all CoP" is unreasonable requirement. |
Discussed and resolved at 20030418 telecon. We think that perhaps originator is confusing "all classes of product" with "all products". Nevertheless, the wording will be improved by: remove "all" from the checkpoint statement; and, change 1st bullet from "MUST list the classes of product it addresses" to "MUST list the classes of product for which it defines conformance requirements". See draft revised text. |
accepted with modification |
LC-73.4 |
|
Collected substantive & editorial comments: [SG-4]: Editorial (8.5, ConfReq, change "profiles"). |
Resolved, this has been fixed as suggested. |
accepted |
LC-73.5 |
|
Collected substantive & editorial comments: [SG-5]: Editorial (Questions "NOT!" in GL9.) |
Resolved, changed "NOT!" to "not." |
accepted |
LC-73.6 |
|
Collected substantive & editorial comments: [SG-6]: Editorial (Typo in CP9.1) |
Resolved. The typo has been fixed during the merger of 9.1 and 9.2 (which was the resolution of another, substantive issue.) |
accepted |
LC-73.7 |
|
Collected substantive & editorial comments: [SG-7]: "Normative refs" requirement should be priority 1. |
20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: agreed, the priority of CP10.2 (now CP7.5) has been changed to 1. |
accepted |
LC-73.8 |
|
Collected substantive & editorial comments: [SG-8]: Add Rationale and Discussion to CP13.3 ("Use consistent terminology"). |
20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: Agreed, a rationale has been added. See draft revised text. |
accepted |
LC-73.9 |
|
Collected substantive & editorial comments: [SG-9]: Improve definition of "test assertions" (GL14). |
Discussed and resolved at 6/2003 QAWG face-to-face. QAWG will make the various definitions consistent across the GL documents and the QA Glossary, but otherwise will leave the definition more or less as it was in the initial verbiage of (Last Call) SpecGL GL14. |
accepted with modification |
LC-74 |
|
Simplify & consolidate the guidelines of SpecGL |
|
accepted with modifications |
LC-79 |
|
SpecGL fails checkpoints 1.2 and 1.3 |
|
accepted |
LC-80 |
|
SpecGL should address the topic of CP applicability. |
|
not accepted |
LC-81 |
|
CP9.6 conformance requirements and rationale may be too narrow. |
|
accepted with modifications |
LC-109 |
|
Is CP1.3 needed? |
|
accepted |
LC-5 |
|
Ck 2.2 list of classes |
|
accepted |
LC-6 |
|
Guideline 2, typo |
|
accepted |
LC-7 |
|
Checkpoint 1.2 |
|
accepted |
LC-8 |
|
checkpoint 1.1 |
|
accepted |
LC-9 |
|
INTRODUCTION section 1.1, second bullet |
|
accepted |
LC-10 |
|
Section 7 |
|
not accepted |
LC-11 |
|
CK 2.3 Category of object: clarification |
|
accepted |
LC-12 |
|
section 3.4 Conformance definition |
|
accepted |
LC-13 |
|
Section 3.3 Conformance and TAs |
|
accepted |
LC-14 |
|
CP 14.1 - clarify conformance requirement |
|
accepted |
LC-15 |
|
extensions |
|
accepted |
LC-16 |
|
CP 8.4 clarify conformance requirement |
|
accepted with modifications |
LC-17 |
|
CP 7.1 deprecated features |
|
accepted with modifications |
LC-18 |
|
GL 5 non-hierarchiacal modules |
|
accepted with modifications |
LC-19 |
|
CP 4.4 explanation clarification |
|
accepted with modifications |
LC-20 |
|
CP 3.1 Conformance Requirements - clarify |
|
accepted with modifications |
LC-21 |
|
CP 2.4 - relationships of DOV - clarify |
|
accepted |
LC-4 |
|
no definition for unconditional conformance |
|
accepted with modifications |
LC-75.1 |
|
Comments on SpecGL Guidelines: Guideline 1: Checkpoint 1.4 is priority 2, yet it's listed after 1.3 which is priority 3. I suggest listing checkpoints in priority order. |
As part of the resolution of the larger issue group, CP1.3 has been merged into CP1.2 and the single checkpoint has been reworded as "Illustrate scope", so the issue becomes moot. CP1.4 (now CP1.3) is reworded as "Illustrate technical details". See draft revised text. |
accepted with modification |
LC-75.2 |
|
Comments on SpecGL Guidelines: Guideline 3: The Conformance Policy is likely to appear towards the end of a Spec. I suggest ordering the guidelines as they are likely to appear in the spec. (Besides, it belongs with, and maybe could even be combined with, guidelines 10, 11, 12, and 13.) |
Discussed and resolved at 6/2003 QAWG face-to-face. Considering all related guideline (GL) consolidation and reordering issues together, it was resolved to: merge (old numbers) GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12; and reorder as GL13, GL10+GL3, GL11+GL12. SeeSee new GL7, GL8, GL9. See draft revised text. |
accepted with modification |
LC-75.3 |
|
Comments on SpecGL Guidelines: Guidelines 4-9: This is a large number of guidelines and checkpoints to deal with something that's important, but that we wish to discourage (DOVs). Could we combine some or all of these guidelines? |
Discussed and resolved at 20030421 telecon. It was decided to consolidate GL4-6 (profiles/modules/levels), into one guideline, while preserving the identities of profiles, modules, and levels as DoV. But otherwise, the other DoV checkpoints don't seem amenable to consolidation. See resolution summary of this and related DoV issues. See draft revised text. |
accepted with modification |
LC-75.4 |
|
Comments on SpecGL Guidelines: The "or NOT!" language in guideline 9 seems a little too informal, |
Resolved at 20030505 telecon. Agreed, "NOT!" is replaced with "not." |
accepted |
LC-75.5 |
|
Comments on SpecGL Guidelines: but more substantively, why do we wait until this guideline [GL9] to express our opinion that DOVs are undesirable. |
Discussed and resolved at 20030421 telecon. A modified version of the previous per-GL DoV "excessive variability" caveat has been restored. See also resolution summary of this and related DoV issues. See draft revised text. |
accepted |
LC-75.6 |
|
Comments on SpecGL Guidelines: Guideline 10: Checkpoint 10.2 doesn't seem to be directly related to the guideline. |
Discussed and resolved at 6/2003 QAWG face-to-face. Considering all related guideline (GL) consolidation and reordering issues together, it was resolved to: merge (old numbers) GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12; and reorder as GL13, GL10+GL3, GL11+GL12. See new GL7, GL8, GL9. See draft revised text. |
accepted |
LC-75.7 |
|
Comments on SpecGL Guidelines: Guidelines 10 - 13: As suggested above, perhaps these could be combined. |
Discussed and resolved at 6/2003 QAWG face-to-face. Considering all related guideline (GL) consolidation and reordering issues together, it was resolved to: merge (old numbers) GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12; and reorder as GL13, GL10+GL3, GL11+GL12. See new GL7, GL8, GL9. See draft revised text. |
accepted with modification |
LC-75.8 |
|
Comments on SpecGL Guidelines: Guideline 14: This is a key guideline - I'd like to see it earlier in the list. |
Discussed and resolved at 6/2003 QAWG face-to-face. Taking the whole group of related guideline (GL) consolidation and reordering issues together, it was resolved to: merge GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12. While test assertions are pivotal and early in the process in *Test* Guidelines, it is resolved that this is an appropriate placement in the *SpecGL* context. |
not accepted |
LC-75.9 |
|
Comments on SpecGL Guidelines: Section 3.3: "This... document does not enumerate a list of test assertions". Haven't we agreed that our checklist is (the closest thing to) a list of assertions? |
Discussed and closed at June QAWG face-to-face. The checklists are a test results reporting form. Recognizing that there are possibly multiple conformance requirements (CR) per checkpoint, a test material for the Guidelines documents would be something like a per-CR questionnaire, "How is this CR satisfied?". |
not accepted |
LC-77.1 |
|
Collected substantive & editorial comments: [SG-1] More discussion of several DoV and their relationships. |
Discussed and resolved at 20030421 telecon. Subsection 2.1 of the new chapter "2. Concepts" provides additional discussion about relationships amongst DoV. See draft revised text. |
accepted |
LC-77.2 |
|
Collected substantive & editorial comments: [SG-2] Standard headings for (Framework GL?) documents. |
20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: Agree that this is a good principle and we will adhere to it, as best we can, recognizing that each Framework document may have some unique chapters. All documents will have the same core headings and in the same order: Intro, Guidelines, Conformance, Definitions (if applicable), Acknowledgements, References, Change History. No changes to SpecGL needed. |
accepted with modification |
LC-77.3 |
|
Collected substantive & editorial comments: [ET-1] (Affects Intro, and all GL-ET pairs) Explain GL-ET pairing better. |
Discussed at 20030714 telecon. Agreed, this is now handled by additional clarifying text in the GL document, and/or references to "QA Framework: Introduction". See draft revised text. |
accepted |
LC-77.4 |
|
Collected substantive & editorial comments: [ET-2] (Affects SpecET and the other ETs). More E&T, in order to conform to SpecGL (GL1)! |
Resolved: it is the QAWG intention that SpecET (Examples & Techniques) will have at least one example per checkpoint. This is queued for SpecET editors, but may lag slightly behind SpecGL publication (after the structure and checkpoint collection of SpecGL stabilizes.) |
accepted |
LC-77.5 |
|
Collected substantive & editorial comments: [ET-3] (Affects SpecGL/ET, and the other GL/ET) Change GL, CP, and section numbering scheme. |
Discussed at 20030407 telecon. Closed with resolution to leave numbering scheme as is. Reasons: given that the context is usually clear (or easily made clear), the shorter GL/CP numbers are considered both more convenient and sufficient; and, the numbering scheme is familiar from WAI specifications and has been successfully used there. |
not accepted |
LC-77.6 |
|
Collected substantive & editorial comments: [ET-4] (Affects SpecET) When will SpecET be released? (!?) |
The QAWG policy is that each new publication of SpecGL will be accompanied by a new version of SpecET (possibly lagging slightly behind the SpecGL publication). In this sense, it is already "released". SpecET may be updated and improved more frequently than SpecGL publications. It is for this reason that QAWG resolved to keep SpecET (as well as OpsET and TestET) in /QA/WG/ space as a Note. When SpecGL 1.0 has finally stabilized and stopped changing in /TR/, and when active work on SpecET is substantially over, then SpecET will be published in /TR/ (as a Note, since it is purely informative.) |
accepted with modification |
LC-2 |
|
Grammatical error in Scope |
|
accepted |
LC-44 |
|
Grammatical errors |
|
accepted |
LC-45 |
|
Design goal of guidelines |
|
accepted |
LC-46 |
|
Ambiguity |
|
accepted |
LC-47 |
|
Spelling error in Example and Techniques |
|
accepted |
LC-48 |
|
Categories of object not previously clearly defined |
|
accepted |
LC-49 |
|
Complexity in explanation |
|
accepted with modifications |
LC-50 |
|
Ambiguity or error? |
|
accepted |
LC-51 |
|
GLs 4, 5 and 6 |
|
accepted with modifications |
LC-52 |
|
NOT ? |
|
accepted |
LC-53 |
|
Ambiguity |
|
accepted |
LC-54 |
|
GLs 3, 10, 11, 12 and 13 |
|
accepted with modifications |
LC-61 |
|
document product class |
|
accepted with modifications |
LC-62 |
|
typos |
|
accepted |
LC-63 |
|
Table of Contents |
|
accepted |
LC-64 |
|
conformance terms |
|
accepted |
LC-65 |
|
sentences and paragraphs (section 3.1) |
|
accepted |
LC-66 |
|
edition and version DoVs |
|
accepted with modifications |
LC-67.1 |
|
limits of RFC 2119 key words: |
Discussed again and resolved at 20030707 telecon. Resolution is: SpecGL (old CP13.1) will strongly encourage RFC2119 keywords as the most widely applicable and usually the best method, but exemptions are allowed, provided that: the spec unambiguously defines how its conformance requirements are identified (even if RFC2119 keywords are used); and, the rationale for not using RFC2119 keywords is explained in the spec. |
accepted with modification |
LC-67.2 |
|
limits of RFC 2119 key words: |
Discussed and resolved 2nd part -- guidance about RFC keyword usage -- at 20030421 telecon. Resolved: SpecGL won't try to legislate correct use of RFC2119 keywords. However, SpecET (Examples & Techniques) will mention that correct-use guidance is given in Section 6 of RFC2119, and will point to it. QAWG is concerned that quoting the particular wording of RFC2119 -- with reflects a particular perspective (communications & protocols) and orientation -- creates confusion rather than clarity, in some conformance contexts. There is also considerable disagreement (see the email dialogs about this issue) within W3C about the "correct use" assertions, and there is a significant body of current practice that is contrary to them. QAWG thinks that it would be useful to consider a joint Comm-QAWG project to draft a W3C guidance Note on this topic. |
accepted with modification |