w3c/wbs-design
or
by mail to sysreq
.
The results of this questionnaire are available to anybody.
This questionnaire was open from 2023-01-12 to 2023-01-23.
10 answers have been received.
Jump to results for question:
summary | by responder | by choice
Different proposals for assertions have relied on different approaches to what a "procedure" is. This ranges from assertions requiring a reference to publicly published guidance like plainlanguage.gov or a W3C Note that WCAG has approved the stricter side to allowing an organization to write its own procedure based on some guidance in WCAG 3 on the less strict side. These are not mutually exclusive. The proposed text we reviewed last week allowed two options. The possibilities that have been proposed are below. Please indicate all which you would support.
Choice | All responders |
---|---|
Results | |
A procedure must reference publicly published guidance listed in WCAG. WCAG would maintain a list of approved procedures. | 2 |
A procedure must reference publicly published guidance that meets the criteria listed in WCAG. In this case, WCAG would not maintain a list but rather a set of criteria for "good" guidance. | 3 |
A procedure must follow the guidance listed in WCAG. See Example Outcome Evaluation Procedure. from the current editor's draft. | 2 |
A procedure can be any process that improves accessibility listed in WCAG but guidance on applying it is up to the organization | 3 |
A procedure can be any process that improves accessibility. | 2 |
Skip to view by choice.
Responder | What is a procedure | Comments |
---|---|---|
E.A. Draffan | ||
Wilco Fiers | ||
Jaunita George | ||
Mary Jo Mueller | ||
Gregg Vanderheiden |
|
I checked top two boxes (rather than just the top box) because, although #2 lets a company game it --it has some controls and we are trying to encourage a company to do more - not force them to so something specific. Also lets this grow organically. AND we will never keep #1 up - if we ever get it up to start with. I don't understand #3 - and looking at it raised lots of questions (see below) #4 an#5 are so vague that one can do anything - and count it. So Silver gold would lose meaning. and I can see no end to arguments. In item 3 Can we not use *outcome* along with *procedure* since we say that procedures do not necessarily have outcomes. Very confusing. An outcome evaluation procedure should be a way of testing an outcome - which is the ordinary interpretation of that phrase. Also following the link in item 3 we find "Users can obtain help and support". This sounds like an outcome. but procedures don't have outcomes. (Also of course this uses "user" so you can't test it unless you specify which user - all users? - a user? users with what abilities? etc. |
Jeanne F Spellman |
|
I have mixed feelings about the second option. The critieria for "good" guidance will be difficult to write and maintain and may constrict organizations from using guidance that was inadvertently excluded. I am very enthusiastic about the Example Outcome Evaluation. I'm not sure about the scoring -- I think I would need to think about that more, but I do think we should explore this example more. |
Stefan Schnabel |
|
Organizations should follow approved procedures but may use their own if needed, if so, they need to explain this. |
Gundula Niemann |
|
The first three should be recommended, but not be mandatory. |
Bruce Bailey |
|
|
Alastair Campbell |
|
Choice | Responders |
---|---|
A procedure must reference publicly published guidance listed in WCAG. WCAG would maintain a list of approved procedures. |
|
A procedure must reference publicly published guidance that meets the criteria listed in WCAG. In this case, WCAG would not maintain a list but rather a set of criteria for "good" guidance. |
|
A procedure must follow the guidance listed in WCAG. See Example Outcome Evaluation Procedure. from the current editor's draft. |
|
A procedure can be any process that improves accessibility listed in WCAG but guidance on applying it is up to the organization |
|
A procedure can be any process that improves accessibility. |
|
Last week, we agreed this proposal was moving in the correct direction. Do you approve moving this text into the WCAG 3 editor's draft as exploratory?
The next step would be to build examples and bring a pull request to the group.
Choice | All responders |
---|---|
Results | |
Yes, I approve moving this content into the editor's draft as exploratory. | 5 |
Yes, I approve moving this content into the editor's draft as exploratory with the following changes (A the notes to add or text to change in comments) | 1 |
No, I do not approve moving this content into the editor's draft. A different direction that has been proposed is a better starting point. (Indicate other prooposal in comments) | |
Something else |
(4 responses didn't contain an answer to this question)
Responder | Next Step | Comments |
---|---|---|
E.A. Draffan | ||
Wilco Fiers | ||
Jaunita George | ||
Mary Jo Mueller | ||
Gregg Vanderheiden | Yes, I approve moving this content into the editor's draft as exploratory with the following changes (A the notes to add or text to change in comments) | Including this and getting comments -- is a good thing - and we should proceedl It needs a bit of cleaning up - and we should remove things that are controversial or weak (maybe list them as "we are also considering" that come up at the meeting Some thing to change that I noticed/ suggest: 1) inclusive design and Universal design are not procedures - they are philosophies or approaches. Not a process or procedures. [remove them or take some practices from them and put them in as procedures ] 2) "and possible at the bronze level" should be changed to "Note: we are also exploring if there is a way they might be included at the Bronze level or not." I think that expresses the level of interest and concern of the group - and would better solicit input. we could also add "and the WG solicits comments on this" to the end. 3) Finish this sentence so it is clear what is being deferred. WCAG 3 recommends additional supporting documentation that an organization can use to improve to support the procedure or assertion but defers [from requiring supplemental documentation beyond the items listed as required below] -- if that is what is meant. just guessing |
Jeanne F Spellman | Yes, I approve moving this content into the editor's draft as exploratory. | |
Stefan Schnabel | Yes, I approve moving this content into the editor's draft as exploratory. | Provided that "this proposal" means the "Assertions and Procedures" document |
Gundula Niemann | Yes, I approve moving this content into the editor's draft as exploratory. | |
Bruce Bailey | Yes, I approve moving this content into the editor's draft as exploratory. | |
Alastair Campbell | Yes, I approve moving this content into the editor's draft as exploratory. |
This draft assumes that organizations won't use assertions if too much documentation is required or if too much information must be made public. A third-party tester can only test content to which they have access.
Some commenters on last week's survey felt that additional documentation which proved the assertion is needed and this would likely be internal documentation available on request.
Which do you most agree with:
Choice | All responders |
---|---|
Results | |
Limit the documentation requirements to information about the assertion itself. Third-party testers would not have access to information to support the assertion. | 3 |
Require additional public documentation to support the assertion. | 2 |
Agree with requiring limited public documentation but request organizations maintain internal documentation to support the assertion that would be available on request when testing. | 2 |
Something else | 2 |
(1 response didn't contain an answer to this question)
Responder | DONE: Documentation needed | |
---|---|---|
E.A. Draffan | Require additional public documentation to support the assertion. | |
Wilco Fiers | Something else | I feel this is a regulatory question, not something WCAG should answer. It's more a question of transparency / openness of information than it is a question of accessibility. |
Jaunita George | Require additional public documentation to support the assertion. | I still feel like we already have assertions that organizations make today that are largely public (accessibility statements and VPATs) that haven't been reliable and don't see how this would improve that situation unless we added some guardrails. |
Mary Jo Mueller | Limit the documentation requirements to information about the assertion itself. Third-party testers would not have access to information to support the assertion. | I'm all for limiting burden of doing this - requiring documentation vs. voluntarily documenting. Organizations that care will likely document it as a matter of course. Organizations that don't probably won't bother making any assertions or go beyond what is absolutely required. By reducing required documentation, more might explore protocols and put them to good use. |
Gregg Vanderheiden | Limit the documentation requirements to information about the assertion itself. Third-party testers would not have access to information to support the assertion. | My thoughts as we puzzle this through.... I think the documentation should be absolute minimum. - the assertion - the date the assertion is made - the W3C listed protocol that the assertion is being done against. Rationale - you want people to embrace these - the documentation is meaningless. Since it isnt outcome based (if it was it would be an outcome) there is no way to be meaningfully documented. Websites are changed daily. Saying that "we did this - and here are documents that you can't verify that says we did" -- is no better than saying "we did it". - and requiring them to do a lot more documentation than that - will likely reduce the number of people who will bother to make an assertion -- or look at them - which means fewer that will do them. - W3C should list the ones that count. allowing anything to be a procedure means that silver and gold can be gamed. Make up something - and assert. Do a number and done. - Also they need to be more specific. Not big documents with lots of misc things in them. a) they then need to do everything? Some one thing in the doc? If a doc is full of recommendations -- does asserting that you followed it mean you did everything in the document? If you need to do everything it would be too much to try? |
Jeanne F Spellman | Agree with requiring limited public documentation but request organizations maintain internal documentation to support the assertion that would be available on request when testing. | |
Stefan Schnabel | ||
Gundula Niemann | Something else | Agree with requiring limited public documentation but request organizations maintain internal documentation to support the assertion. |
Bruce Bailey | Limit the documentation requirements to information about the assertion itself. Third-party testers would not have access to information to support the assertion. | I am okay with other choices, and would note that government agencies probably subject to Sunshine/FOIA (Freedom of Information Act) or equivalent -- so that is close to "available upon request". |
Alastair Campbell | Agree with requiring limited public documentation but request organizations maintain internal documentation to support the assertion that would be available on request when testing. | I suspect that 3rd party testers would create a form for their clients to fill in that would help to fill in assertions for a conformance claim. |
The argument for applying assertions to Silver and Gold levels (or something similar) is that assertions do not guarantee results so should be addressed at a higher conformance level than methods.
The argument for including assertions at the Bronze level (or something similar) is that some user needs are best supported by assertions so moving them to a higher level repeats the concerns expressed in A/AA and AAA division
Which is the lowest level assertions apply to (assuming they would also apply above that level)? Please explain your reasoning in the comments.
Choice | All responders |
---|---|
Results | |
Bronze | 2 |
Silver | 2 |
Gold | 1 |
Something else | 4 |
(1 response didn't contain an answer to this question)
Responder | Level Assertions Apply | Comments |
---|---|---|
E.A. Draffan | Something else | It is so difficult to set a level when the issues at stake may have so many different ways of affecting outcomes |
Wilco Fiers | Something else | Without actual outcomes for which assertions can be used, I think it's too early to say. I suspect WCAG 3 will not be equitable if we restrict it by level, but we'll just have to wait and see how things shake out I think. |
Jaunita George | Gold | Silver may become the target standard and I feel like organizations should meet all target standards before receiving any points on a statement that could not be independently verified. |
Mary Jo Mueller | Bronze | IMO, assertions COULD apply to any level. However, to achieve silver or gold, it seems assertions would be necessary or required. |
Gregg Vanderheiden | Something else | I think it should only be applied at Silver or Gold levels of conformance. BUT - I think both assertions and recommendations should be positioned mixed in with the outcomes so that you see them all when you see any of them. Even if they don't (or their company doesnt allow them to) make any assertions = just seeing them will put them in their mind and there is higher likelihood they will practice them - if not assert them. If we put them off in a separate section - they will be much less viewed |
Jeanne F Spellman | Bronze | Assertions can have a broad use in WCAG3. Bronze level assertions can be specified clearly to meet a specific Bronze outcome and can be specified to provide specific amounts of verifiable documentation on request. An example could be using clear words. The required documentation could include the list of words that the organization is using. |
Stefan Schnabel | ||
Gundula Niemann | Something else | The terms 'assertion', 'process', 'protocol', and 'method' are defined and used in a very proprietary way throughout our discussions, so I cannot memorize the details. Unfortunately, not all definitions are repeated in the linked document. Therefore I cannot answer this question. |
Bruce Bailey | Silver | Including "web accessibility statements" is very close IMHO to an assertion, and those are not uncommon. I think that makes it a better fit for Silver. |
Alastair Campbell | Silver | Silver / something else. If we uses the "badges" approach then presumably a site could include certain badges based on assertions that aren't sufficient to reach Silver? I think we should have a level you can reach without assertions, so that would be bronze. However, we should aim for Silver to be the "national level" organisation level, which requires some assertions, then gold for more/better coverage. |
Do you agree to replace section 4.3.4 procedural test and 4.4.4 Outcome implementation procedures in the WCAG 3 editor's draft?
Choice | All responders |
---|---|
Results | |
I agree to add this content as exploratory | 4 |
I agree to add this content as exploratory with the following changes | 2 |
I do not believe this is an improvement over the current content in 4.3.4 procedural test and 4.4.4 Outcome implementation procedures so should not be added | |
Something else | 3 |
(1 response didn't contain an answer to this question)
Responder | Approval to add content | Comments |
---|---|---|
E.A. Draffan | Something else | I really am not sure as the 4.3.4 procedural test and 4.4.4 Outcome implementation procedures still have so many questions that remain unanswered and this draft document about Assertions seems to just add more complexity and imponderables. |
Wilco Fiers | I agree to add this content as exploratory with the following changes | I'm not too clear on the difference between protocols and processes yet. I'd like a note saying we need work on if / how to define the boundary between them. |
Jaunita George | I agree to add this content as exploratory | |
Mary Jo Mueller | I agree to add this content as exploratory with the following changes | Are process, procedure, and protocol synonymous? It is unclear from the excerpted content in the document. While I agree this content can be added as exploratory, I think these definitions should be made clear if the terms have a different meaning. Otherwise it's a bit confusing. |
Gregg Vanderheiden | Something else | Agree 4.3.4 with "procedure test" with "procedure assertion" Disagree with 4.4.4 talking about *Outcome implementation procedure* -- this conflates outcomes and procedures. We should keep "outcomes" with "tests" and keep "procedures" with "assertions" and never have a sentence the conflates them like "outcome procedure" or "outcome assertion" or anything that conflates them. I will confuse people a lot. |
Jeanne F Spellman | I agree to add this content as exploratory | |
Stefan Schnabel | ||
Gundula Niemann | Something else | I can't see two versions, so I cannot decide on a replacement. Options 1 and 2 talk about adding, not about replacement. |
Bruce Bailey | I agree to add this content as exploratory | |
Alastair Campbell | I agree to add this content as exploratory | Under documenting assertions, I was wondering why the assertions wouldn't have the same scope as the conformance statement that they are presumably documented within? Would different assertions have different scopes? Perhaps, but shouldn't the default (easiest way) be for the same scope across assertions? The transition to "protocol" is quite abrupt, would it help to say, instead of "Assertions may apply to protocols or processes.", something like: "An assertions can be about the use of a protocol or process." (or "is about".) I'm still concerned that some "protocols" are very wide ranging (e.g. the BBC Gel), and aren't really something you can say you've followed as so much will be BBC specific. I think the processes seem more straightforward to apply (e.g. "we did usability testing and updated the site"). |
The following persons have not answered the questionnaire:
Send an email to all the non-responders.
Compact view of the results / list of email addresses of the responders
WBS home / Questionnaires / WG questionnaires / Answer this questionnaire
w3c/wbs-design
or
by mail to sysreq
.