W3C

Site Navigation

Nearby

Results of Questionnaire Protocols Odds and Ends

The results of this questionnaire are available to anybody.

This questionnaire was open from 2022-04-13 to 2022-04-16.

7 answers have been received.

Jump to results for question:

  1. What do you consider the intent of protocols in relation to conformance?
  2. How do you think protocols should assess success?
  3. Should we alternate between 8am and noon (Eastern Time) to allow for greater participation?
  4. Would you support creating a group Slack channel in the A11y Slack to help facilitate real-time communication on issues, rather than just relying on email? We could share out a summary of the discussions weekly for the rest of the group/public record.

1. What do you consider the intent of protocols in relation to conformance?

A. Protocols are above or beyond any type of conformance, aka “extra credit”

B. Protocols are a way of meeting the guidelines, and how they fit into conformance is TBD

C. Something else (please provide an explanation in comments)

Summary

ChoiceAll responders
Results
Option A. 2
Option B. 3
Option C (please explain) 2

Details

Responder What do you consider the intent of protocols in relation to conformance?
Rachael Bradley Montgomery Option B.
John Foliot Option C (please explain) Structurally, Protocols do not sit "above" Requirements, they sit "beside" them. Protocols contribute in parallel to Requirements at content creation time. (Requirement: all non-text elements require a text alternative; Protocol: use the alt text decision tree at https://www.w3.org/WAI/tutorials/images/decision-tree/ to guide the author towards a "good" text alternative)

Protocols *MAY* provide methods or advice on how to meet a specific Guideline, but they may also NOT: the guidance found in a Protocol may not have any measurable requirements (eg. Plain Language: "Choose your words carefully" - https://www.plainlanguage.gov/guidelines/words/), but still provide important guidance in the content creation phase:

"Words matter. They are the most basic building blocks of written and spoken communication. Don’t complicate things by using jargon, technical terms, or abbreviations that people won’t understand. Choose your words carefully and be consistent in your writing style.

* Prefer the familiar word to the far-fetched.
* Prefer the concrete word to the abstraction.
* Prefer the single word to the circumlocution. (JF: even Plainlanguage.gov uses complex terms)
* Prefer the short word to the long.
* Prefer the Saxon word to the Romance word.

-- end --

My concern here is that it appears that in this context "conformance" is still envisioned as a binary pass/fail state, whereas the current proposed structure of WCAG 3 suggests that "conformance" is (will be) stratified (Bronze, Silver, Gold), and that meeting "Bronze" is a tacit admission that you are less than "perfect", and thus less than "fully conforming".

In my mental model of Protocols, adopting a Protocol accrues "points" towards a conformance score, and there will be a minimum score value for Bronze, a higher score for Silver, and a better-again score for Gold.
Jennifer Strickland Option A. Protocols would not be a way of meeting minimum level of conformance.
Michael Cooper Option B.
Jeanne F Spellman Option A. I think they should be above a minimum conformance. I don't think that Protocols should be a way of making a minimum bar because they have too much potential for "gaming' the system.
Charles Adams Option C (please explain) Both A and B can be true.

For me a protocol is a set of (to be) defined principles that allow for individuals or organizations to construct their own dynamic approaches that are not otherwise documented by W3C to achieving any goal that may or may not be documented by W3C. The W3C would define the "principles" that represent the building blocks for crafting protocols, and individuals/organizations are free to build anything they can imagine using those "principles". A protocol could be in service of "conformance" to a guideline or set of guidelines, or could be a well documented statement of advocacy that does not have any influence over conformance at all.

One company's use of a "protocol" could be to demonstrate that they use native AT testers to validate their conformance to X number of guidelines which impact functional needs of the AT testers (hypothetically the "text alternatives" and "structured content" guidelines and any number of additional guidelines yet to be created), and another company's use of a "protocol" could be to demonstrate use of a new "method" (which was not defined by W3C) that helps them achieve conformance to a specific guideline, such as the "text alternatives" guideline; and a 3rd company's use of a protocol could be to demonstrate their investment in advanced research which does not directly addresses any guidelines, but otherwise demonstrates a commitment to accessibility. The first two examples would influence "conformance", and the third example would be "extra credit".

My formal definition: A protocol is a set of (yet to be) W3C defined principles that can be used to build non-W3C documented approaches for advancing a goal related to accessibility.
JaEun Jemma Ku Option B.

2. How do you think protocols should assess success?

NOTE: “Inputs” may be documentation of steps, actions taken, date completed, conformance claims, etc. that WCAG would require as proof of doing the protocol. These are TBD. “Outcomes” are the results of a given protocol.

  • A. Protocols provide a way to evaluate whether subjective WCAG requirements have been met. (In other words, protocols evaluate the inputs and the outcomes.)

    B. Protocols test the inputs against what WCAG has required

    C. Something else (please explain in comments)

    • Summary

    ChoiceAll responders
    Results
    Option A.
    Option B. 3
    Option C (please comment). 4

    Details

    Responder How do you think protocols should assess success?Comments
    Rachael Bradley Montgomery Option B.
    John Foliot Option C (please comment). Closer to Option A but not stated as such - when you attempt to evaluate outcomes the results will always be subjective, so with Protocols you cannot do both - at best you can evaluate the 'inputs' and seek evidence of the application in the outputs.

    "Conforming" to a Protocol (accruing points) is done by declaring that a Protocol has been adopted via a formal declaration, WITHOUT attempting to assess how well the Protocol has been applied (the subjective part).

    This is because, from a "legal compliance" perspective, the declaring party will always make the subjective assessment declaration in the most glowing of terms, the litigant will always make their subjective assessment claim in the absolute worst of terms, and because the outcomes of Protocols will always be subjective, when attempting to evaluate outcomes it will be impossible to arrive at conclusions with any consistency. (Evidence: read any VPAT out there and specifically "Partially Supports")

    Even evaluating "proof points" cannot ensure that the application of the protocol has been done to a 3rd party's satisfaction. Example:

    * Declaration: This response has been written using the principles of plain language
    * Proof Point(s): I have read the requirements for plain language as outlined at plainlanguage.gov; I have taken an online course on how to write content using plain language principles; I had a third-party review this content.

    Prove me wrong. Prove that the principles of plain language HAVE NOT been applied here.

    This is why you cannot accurately or consistently evaluate outcomes.
    Jennifer Strickland Option C (please comment). "WCAG would require as proof" — afaik WCAG doesn't require proof. WCAG documents guidelines. Other entities might use the guidelines and turn them into requirements, and those entities might have a "prove" mechanism, although I have yet to see any actually evaluate it. The only evaluation I've seen are organizations that review their own work, and then user feedback and complaints.

    Protocols provide a way to evaluate subjective WCAG success criteria (we don't have WCAG requirements) and they also evaluate accessibility / inclusion / equity of an outcome, so can exceed what WCAG provides guidelines for.
    Michael Cooper Option B.
    Jeanne F Spellman Option C (please comment). Protocols can evaluate guidance where the outcomes can not be measured. We can measure subjective tests without using protocols. Where protocols are useful are in measuring and evaluating processes, such as Inclusive Design. The protocols says that to get credit for using Inclusive Design (for example) you must document that the designers are trained in Inclusive Design, follow the principles of inclusive design, and receive input on their designs from users with various (specified) types of disabilities and improve the design based on feedback. We don't measure the output, we measure the inputs (training, follow the training, take actions to get feedback, improve the design based on feedback). Each of those steps can be documented to assess success. The protocol specifies the inputs and how they are documented or measured.
    Charles Adams Option C (please comment). If we define a set of "principles" that individuals/organizations can use to build their own protocols, then a protocol can do any, all or none of the elements in A and B. An individual or organization can construct a protocol that evaluates ONLY inputs, another protocol could be built that evaluates ONLY outputs, a third can be built that does both, and a fourth does neither. As long as the creator of the protocol utilizes the (yet to be) set of pre-defined "principles" to craft their unique protocol, the individual/organization has liberty to create anything they can imagine.
    JaEun Jemma Ku Option B.

    3. Should we alternate between 8am and noon (Eastern Time) to allow for greater participation?

    We surveyed possible times several weeks ago and the two possible times were 8am and 12 noon Eastern. 8am is too early for US participants on the west coast and noon is too late for our European participants.

    Yes: We should alternate between 8am and noon (Eastern Time)

    No: We should not alternate between 8am and noon (Eastern Time)

    Other: Please provide an explanation in comments

    Summary

    ChoiceAll responders
    Results
    Yes 5
    No
    Other (Please provide an explanation in comments) 2

    Details

    Responder Should we alternate between 8am and noon (Eastern Time) to allow for greater participation?Comments
    Rachael Bradley Montgomery Yes
    John Foliot Other (Please provide an explanation in comments) Whatever the group decides. 9:00 AM was working fine for me.
    Jennifer Strickland Yes 9am was conducive for both Europe and the Pacific time zone. Could we go back to that? Alternatively, I do think we should alternate to ensure attendance from the Pacific and Asia time zones. The W3C creates guidelines for the globe, so how might we be more inclusive and welcome the globe?
    Michael Cooper Other (Please provide an explanation in comments) It might be hard for groups meeting at different times to develop overall consensus unless there is enough overlap in participation. But I wouldn't oppose a majority wish to try it.
    Jeanne F Spellman Yes We need to be able to include both groups. It would be better to find an 11 ET slot, but if we can't, then we should try alternating.
    Charles Adams Yes
    JaEun Jemma Ku Yes

    4. Would you support creating a group Slack channel in the A11y Slack to help facilitate real-time communication on issues, rather than just relying on email? We could share out a summary of the discussions weekly for the rest of the group/public record.

    Yes, I support creating a group Slack channel.

    No, I do not support creating a group Slack channel.

    Other (Please provide an explanation in comments)

    Summary

    ChoiceAll responders
    Results
    Yes 4
    No 3
    Other (Please provide an explanation in comments)

    Details

    Responder Would you support creating a group Slack channel in the A11y Slack to help facilitate real-time communication on issues, rather than just relying on email? We could share out a summary of the discussions weekly for the rest of the group/public record.
    Rachael Bradley Montgomery No My preference is not to. We have a lot of communication channels already and I am concerned with adding another. That said, I will support and engage with whatever the group decides.
    John Foliot No THE STONGEST OF OBJECTIONS!

    Splintering the discussion across multiple channels does not foster better communication, it frustrates it because now you have to monitor even more channels of information. We currently already have a mailing list and GitHub issues, which can already cause 'forking' in discussions.

    If we decide to use Slack, will we also have a Microsoft Teams channel (for those participants who do not or CAN NOT use Slack at work)?
    Jennifer Strickland Yes If the A11y Slack channel was Private and kept to WCAG members that would be of interest, although I know the WCAG leadership resisted the idea in the past due to some concerns (I don't recall what they were, as it was about a year ago — when I also suggested it). I do not want the conversation open more broadly at this time and there are people in the A11y Slack that would be problematic.
    Michael Cooper Yes
    Jeanne F Spellman No Slack is not as accessible as we would like. We have a wcag3 Slack channel that gets no use. If you want to try it out, then use that channel.
    Charles Adams Yes Yes(!!!), but I think that my company might restrict access to external channels. I've not been able to successfully connect to a non company channel. This could be user error.
    JaEun Jemma Ku Yes

    More details on responses

    • Rachael Bradley Montgomery: last responded on 14, April 2022 at 11:38 (UTC)
    • John Foliot: last responded on 14, April 2022 at 12:17 (UTC)
    • Jennifer Strickland: last responded on 14, April 2022 at 13:52 (UTC)
    • Michael Cooper: last responded on 14, April 2022 at 17:06 (UTC)
    • Jeanne F Spellman: last responded on 14, April 2022 at 22:42 (UTC)
    • Charles Adams: last responded on 14, April 2022 at 23:26 (UTC)
    • JaEun Jemma Ku: last responded on 15, April 2022 at 13:48 (UTC)

    Non-responders

    The following persons have not answered the questionnaire:

    1. Gregg Vanderheiden
    2. Chris Wilson
    3. Lisa Seeman-Horwitz
    4. Janina Sajka
    5. Shawn Lawton Henry
    6. Katie Haritos-Shea
    7. Shadi Abou-Zahra
    8. Chus Garcia
    9. Steve Faulkner
    10. Patrick Lauke
    11. David MacDonald
    12. Gez Lemon
    13. Makoto Ueki
    14. Peter Korn
    15. Preety Kumar
    16. Bruce Bailey
    17. Georgios Grigoriadis
    18. Stefan Schnabel
    19. Romain Deltour
    20. Laura Carlson
    21. Chris Blouch
    22. Jedi Lin
    23. Wilco Fiers
    24. Kimberly Patch
    25. Glenda Sims
    26. Ian Pouncey
    27. Alastair Campbell
    28. Léonie Watson
    29. David Sloan
    30. Mary Jo Mueller
    31. John Kirkwood
    32. Detlev Fischer
    33. Reinaldo Ferraz
    34. Matt Garrish
    35. Mike Gifford
    36. Loïc Martínez Normand
    37. Mike Pluke
    38. Justine Pascalides
    39. Chris Loiselle
    40. Tzviya Siegman
    41. Jan McSorley
    42. Sailesh Panchang
    43. Cristina Mussinelli
    44. Jonathan Avila
    45. John Rochford
    46. Sarah Horton
    47. Sujasree Kurapati
    48. Jatin Vaishnav
    49. Sam Ogami
    50. Kevin White
    51. E.A. Draffan
    52. Paul Bohman
    53. 骅 杨
    54. Victoria Clark
    55. Avneesh Singh
    56. Mitchell Evan
    57. Michael Gower
    58. biao liu
    59. Scott McCormack
    60. Francis Storr
    61. Rick Johnson
    62. David Swallow
    63. Aparna Pasi
    64. Gregorio Pellegrino
    65. Melanie Philipp
    66. Jake Abma
    67. Nicole Windmann
    68. Oliver Keim
    69. Gundula Niemann
    70. Ruoxi Ran
    71. Wendy Reid
    72. Scott O'Hara
    73. Muhammad Saleem
    74. Amani Ali
    75. Trevor Bostic
    76. Jamie Herrera
    77. Shinya Takami
    78. Karen Herr
    79. Kathy Eng
    80. Cybele Sack
    81. Audrey Maniez
    82. Jennifer Delisi
    83. Arthur Soroken
    84. Daniel Bjorge
    85. Kai Recke
    86. David Fazio
    87. Daniel Montalvo
    88. Mario Chacón-Rivas
    89. Michael Gilbert
    90. Caryn Pagel
    91. Achraf Othman
    92. Helen Burge
    93. Fernanda Bonnin
    94. Jared Batterman
    95. Raja Kushalnagar
    96. Jan Williams
    97. Todd Libby
    98. Isabel Holdsworth
    99. Julia Chen
    100. Marcos Franco Murillo
    101. Yutaka Suzuki
    102. Azlan Cuttilan
    103. Joe Humbert
    104. Ben Tillyer
    105. Charu Pandhi
    106. Poornima Badhan Subramanian
    107. Alain Vagner
    108. Roberto Scano
    109. Rain Breaw Michaels
    110. Kun Zhang
    111. Jaunita George
    112. Regina Sanchez
    113. Shawn Thompson
    114. Thomas Brunet
    115. Kenny Dunsin
    116. Jen Goulden
    117. Mike Beganyi
    118. Ronny Hendriks
    119. Olivia Hogan-Stark
    120. Rashmi Katakwar
    121. Julie Rawe
    122. Duff Johnson
    123. Laura Miller
    124. Will Creedle
    125. Shikha Nikhil Dwivedi
    126. Marie Csanady
    127. Meenakshi Das
    128. Perrin Anto
    129. Rachele DiTullio
    130. Jan Jaap de Groot
    131. Rebecca Monteleone
    132. Ian Kersey
    133. Peter Bossley
    134. Anastasia Lanz
    135. Michael Keane
    136. Chiara De Martin
    137. Giacomo Petri
    138. Andrew Barakat
    139. Devanshu Chandra
    140. Xiao (Helen) Zhou
    141. Joe Lamyman
    142. Bryan Trogdon
    143. Mary Ann (MJ) Jawili
    144. 禹佳 陶
    145. 锦澄 王
    146. Stephen James
    147. Jay Mullen
    148. Thorsten Katzmann
    149. Tony Holland
    150. Kent Boucher
    151. Abbey Davis
    152. Phil Day
    153. Julia Kim
    154. Michelle Lana
    155. David Williams
    156. Mikayla Thompson
    157. Catherine Droege
    158. James Edwards
    159. Eric Hind
    160. Quintin Balsdon
    161. Mario Batušić
    162. David Cox
    163. Sazzad Mahamud
    164. Katy Brickley
    165. Kimberly Sarabia
    166. Corey Hinshaw
    167. Ashley Firth
    168. Daniel Harper-Wain
    169. Kiara Stewart
    170. DJ Chase
    171. Suji Sreerama
    172. Lori Oakley
    173. David Middleton
    174. Alyssa Priddy
    175. Young Choi
    176. Nichole Bui
    177. Julie Romanowski
    178. Eloisa Guerrero
    179. Daniel Henderson-Ede
    180. George Kuan
    181. YAPING LIN
    182. Justin Wilson
    183. Leonard Beasley
    184. Tiffany Burtin
    185. Shane Dittmar
    186. Nayan Padrai
    187. Niamh Kelly
    188. Matt Argomaniz Matthew Argomaniz
    189. Frankie Wolf
    190. Kimberly McGee
    191. Ahson Rana
    192. Carolina Crespo
    193. humor927 humor927
    194. Samantha McDaniel
    195. Matthäus Rojek
    196. Phong Tony Le
    197. Bram Janssens
    198. Graham Ritchie
    199. Aleksandar Cindrikj
    200. Jeroen Hulscher
    201. Alina Vayntrub
    202. Marco Sabidussi
    203. John Toles
    204. Jeanne Erickson Cooley
    205. Theo Hale
    206. Gert-Jan Vercauteren
    207. Karla Rubiano
    208. Aashutosh K
    209. Hidde de Vries
    210. Julian Kittelson-Aldred
    211. Roland Buss
    212. Aditya Surendranath
    213. Avon Kuo
    214. Elizabeth Patrick
    215. Tj Squires
    216. Nat Tarnoff
    217. Illai Zeevi
    218. Filippo Zorzi
    219. Gleidson Ramos
    220. Mike Pedersen
    221. Rachael Yomtoob
    222. Oliver Habersetzer
    223. Irfan Mukhtar

    Send an email to all the non-responders.

    Compact view of the results / list of email addresses of the responders

    WBS home / Questionnaires / WG questionnaires / Answer this questionnaire

    Report issues on GitHub project w3c/wbs-design or by mail to sysreq.