Authoring Tools Working Group Face-to-Face Minutes

In attendance

Friday, 15 July

JT: Proposals for a statement of scope for evaluation techniques? We have a disclaimer, and had an early version on the Web site.
JR: We have conformance evaluation.
JT: Should we merge conformance reviews, test suites and evaluation techniques to make them consistent?
JR: We have a test suite that covers certain tools, but not, say, graphics tools. The evaluation techniques would allow you to create your own suite for that.
JT: Scope statement for EO on these documents?
PJ: ATAG Conformance Evaluation Techniques and Reports?
JR: Or, Evaluation Against ATAG?
JR: Not "against"...
MM: Evaluating Authoring Tools for Conformance to ATAG x.x?
JT: And under those, the previously mentioned docs (Eval techs)
PJ: We need a report template, and a suite in the same format.
JT: Maintain a registry for compliance?
DG: I don't think it's a purpose?
PJ: Do we find someone else to maintain it?
LN: If you put out metadata like that and don't maintain it, it's a disaster.
DG: If we require a metadata file in order to join the registry, and that were evaluated, it would be okay.
LN: Would still have to be maintained vs. several versions, etc.
PJ: Could recommend that W3C maintains one repository, but recommends other entities maintain this type of repository.
JR: If a company makes a claim, it's simple, but if they want to see the thinking behind the claim, here is the data to verify it.
LN: There's the potential for these to cause several companies to flood the registry with individual web site templates.
PJ: This isn't the scope of our effort. It's overall authoring tools.
DG: You'll still have several authoring tools.
PJ: IBM makes thousands of individual templates.
LN: Several tools make thousands of templates that are used by others that are then inaccessible.
JR: WebCT makes several sets of templates which are used, but that is the authoring tool, not any of the documents it produces.
MM: If WebCT fails to produce accessible content, it fails ATAG, not any produced presentations.
JT: So, safe to say we're not going to keep a registry, but we're going to make it possible to make a registry?

Harmonization with WCAG 2.0

The group watched a presentation by Ben Caldwell on the latest draft of the WCAG document.

How to maintain claims of ATAG compliance

JT: Identification of evaluator should be divulged, whether they be the tool vendor itself, a consumer, etc.
JB: Disability groups are against individual compliance checking a la VPATs.
JT: What is the general sense of what W3C should be doing?
JB: Disability groups are dissatisfied with Section 508 and don't feel they're getting enough information. In Europe, there is more demand for precise information. They want more and better.
JT: We've decided that we can't deliver that.
JB: How much work would it take as a distributed effort to go for reviewing the 10 or 12 most interesting UAs and updating them according to the template.
JR: It's not clear how the template will work, insofar as it would technically result in normative documents.
PJ: There needs to be an AERT document for ATAG 2.0.
MM: There remains the potential for product reviews to be widely different based on who is providing the review.
JT: The better we create these rules, the closer the review comes to normative. The closer we come to normative, the less likely this is to happen.
JT: The primary purpose of the reports on the site is to instruct us on how to create an evaluation methodology.
JB: And to make that methodology available to anyone who wants to use it.
JT: We're developing and testing our methodology through our reports, rather than just reporting it in itself.
PJ: How many models do we do for a given category?
JT: At least one, and as many as there are views.
PJ: The EO wants evaluations on the top 10 most popular tools.
JB: That has to do with the WAI-DA program in Europe.
PJ: Didn't EO want AU to evaluate the tools?
JB: If AU can arrive at a good, stable template, and a set of reviews (10-12) on how it should be used, that would be enticing to researchers and to governmental bodies to pay for that research because they have a road map.
JT: We know we have to do the methodology. We're doing reports to deal with methodology, creating models, and down the line we're encouraging outside interests to pursue it.

Tuesday, 16 July

Review ATAG Wombat spec, determine which are affected by WCAG

1.1 accessible content in the tool
JR: This is difficult to test if it's not a relative priority checkpoint.
Decision: WCAG-dependent
1.2 preserve accessibility information in transforms
Decision: not WCAG-dependent
1.3 auto-generated content conforms to WCAG
JR: This should be changed from "conforms to WCAG 1.0" to "conforms to accessible content"
MM: Should refer to "generating content" instead of "generating markup"
Decision: WCAG-dependent
Decision: Ensure that when the tool automatically generates content, it conforms to WCAG (no version #)
Decision: make terms for WCAG compliance consistent throughout the document
1.4 pre-authored content conforms to WCAG
Decision: WCAG-dependent
1.5 preserve markup not recognized by the tool
Decision: not WCAG-dependent
2.1 latest versions of W3C Recs
Decision: not WCAG-dependent
2.2 Valid auto-generated markup
Decision: not WCAG-dependent
3.1 Prompt for alt content
Decision: WCAG-dependent (relative priority)
3.2 structured content
Decision: WCAG-dependent
3.3 not autogenerating equivalent content
Decision: not WCAG-dependent
3.4 managing alternative equivalents for multimedia
Decision: not WCAG-dependent
4.1 check for accessibility problems
Decision: WCAG-dependent
4.2 Assist in correcting problems
Decision: WCAG-dependent
4.3 summarize document's accessibility status
Decision: not WCAG-dependent
5.1 ckpts 3.1 3.2 4.1 available to the user
MM: Perhaps use the rationale as the title to this one?
JT: Question whether this is subsumed by other checkpoints
Decision: not WCAG-dependent
5.2 easy to find accessibility hooks
Decision: WCAG-dependent
5.3 integrated functionality
Decision: not WCAG-dependent
5.4 integrated documentation
Decision: not WCAG-dependent
Decision: not a relative priority (suggest P2)
JT: Criteria for 5.0 is natural integration. Criteria for 6 is inclusion of all accessible practices in documentation. So we're taking the minimum out of 5.4 and moving it to 6.
6.1 document all accessibility features
Decision: not WCAG-dependent
Decision: change to "Document all features of the tool..."
6.2 document how to use tool for accessibility
Decision: WCAG-dependent
7
dependent on UA, not GL

Conformance assessment template

JT: Do we need to have exclusions explicit along with backgrounds and criteria? JR: Generic document as a template, with test suites for individual technologies?

Success criteria for each checkpoint

Decision: change "exclusions" section (mentioned above) to "applicability"
1.1
unresolved
1.2
Preservation of accessibility information
1.3
WCAG-compliant auto-generated markup
1.4
WCAG-compliant preauthored content

Note: "At Minimum" criteria for 5.2 and 5.3 are backwards on Wombat in TR. This should be fixed by next working draft.

Test suites

@@ test suites
PJ: Need one for tool-specific techs, and one for WCAG

Dividing up checkpoints for writing evaluation techniques

Nominations

Categories of authoring tools

Proposal: Alter the granularity of these tools such that a tool author can select one or a set of these categories to determine what needs to be satisfied. Note: A Wombat version of the ATAG techniques will need to be written.

Next steps

Next meeting

19-20 September 2002. Proposed location: Vancouver, BC, Canada