11:55:23 <RRSAgent> RRSAgent has joined #wcag3-protocols
11:55:23 <RRSAgent> logging to https://www.w3.org/2022/04/08-wcag3-protocols-irc
11:56:50 <Jaunita_George_> Agenda+ Clarifying goals for protocols
11:57:07 <Jaunita_George_> Agenda+ Next steps for the working group
11:57:24 <Jaunita_George_> rrsagent, generate minutes
11:57:24 <RRSAgent> I have made the request to generate https://www.w3.org/2022/04/08-wcag3-protocols-minutes.html Jaunita_George_
11:57:33 <Jaunita_George_> 	chair:  Jaunita George
11:57:48 <Jaunita_George_> 	Zakim, start meeting
11:57:48 <Zakim> RRSAgent, make logs Public
11:57:49 <Zakim> please title this meeting ("meeting: ..."), Jaunita_George_
11:57:56 <Jaunita_George_> meeting:  AGWG-2022-04-08
11:59:12 <Jaunita_George_> Present+
11:59:17 <Rachael> present+
12:01:05 <Jennifer> Jennifer has joined #wcag3-protocols
12:01:09 <Jennifer> present+
12:03:52 <MichaelC> present+
12:04:07 <bruce_bailey_> bruce_bailey_ has joined #wcag3-protocols
12:04:14 <JakeAbma__> JakeAbma__ has joined #wcag3-protocols
12:04:16 <Jaunita_George_> zakim, take up the next agenda item
12:04:16 <Zakim> I don't understand 'take up the next agenda item', Jaunita_George_
12:04:18 <JF> JF has joined #wcag3-protocols
12:04:20 <JakeAbma__> present+
12:04:26 <JF> Present+
12:04:46 <Jaunita_George_> zakim, take up agenda item 1
12:04:46 <Zakim> 'item\ 1' does not match any agenda item, Jaunita_George_
12:04:49 <bruce_bailey_> present+
12:05:32 <Jaunita_George_> zakim, take up item 1
12:05:32 <Zakim> agendum 1 -- Clarifying goals for protocols -- taken up [from Jaunita_George_]
12:06:00 <Jaunita_George_> *wcag3-protocols
12:07:43 <bruce_bailey_> q+
12:07:45 <Rachael> Jaunita_George_: There are two views of protocols. The first is that protocols are extra, above and beyond standards. The other view is that protocols help define conformance at some level.
12:07:56 <Rachael> ...I'd like to start by coming to a common understanding.
12:08:27 <Rachael> Bruce: Is the protocols page wiki correct? I've been following some of this work tangently. I'm not clear what our definition is for protocols.
12:08:50 <Rachael> ...I may be overloading the term due to previous group work
12:08:55 <JF> Q+
12:09:01 <bruce_bailey_> https://github.com/w3c/silver/wiki/Protocols
12:09:03 <Rachael> Jaunita_George_: I may not be characterizing this correctly.
12:09:05 <Rachael> ack bruce_bailey_
12:09:39 <Rachael> There is a view that protocols go above and beyond conformance and another that they contribute to conformance.
12:09:41 <Rachael> ack JF
12:10:22 <Rachael> JF: My vision. When you say standards, I hear requirements. In WCAG 3, we have requirements that will meet ACT format.
12:10:24 <Rachael> q+
12:10:27 <JakeAbma__> q+
12:10:29 <Jaunita_George_> q+
12:11:33 <Rachael> ...Those will be measurable, repeatable. On the other side are protocols. Protocols can support those but the defining feature for a protocol is that its educational in nature. Gives policy makers information on the right thing to do. Hard to evaluate subjective determination. Informed opinions. Make relatively good decision due to being informed.
12:11:52 <MichaelC> q+ to differentiate protocol from guidance
12:11:58 <Rachael> ...something like plain language gives you information but doesn't say "don't use multi-syllable words"
12:12:32 <Rachael> ...point is to communicate effectively. We have a goal and a document on how to acheive it but doesn't say you must do this.
12:12:36 <Rachael> ack Rachael
12:12:41 <GN015> GN015 has joined #wcag3-protocols
12:13:28 <Jaunita_George_> Rachael: We have talked about the fact there are actually four levels of tests
12:14:01 <Jaunita_George_> ...don't want to oversimplify the types of tests in WCAG 3.0
12:14:03 <bruce_bailey_> Rachael:  Whether someone has conformed to the guidelines as not,
12:14:20 <JF> Q+
12:14:31 <Jaunita_George_> ack JakeAbma_
12:14:51 <bruce_bailey_> s/guidelines as not,/guidelines or not/
12:15:09 <Rachael> Rachael: ACT is working with Scoping on how to write tests for test scenarios.
12:15:38 <Rachael> Jake: When ACT group started, focused on purely objective tests. That has changed. There is now a lot of subjectivity in ACT that can't be tested automatedly.  New rules are way more subjective. ACT moved its direction. Maybe would be good to explain a bit more. Not what it was 2-3 years ago.
12:15:58 <Rachael> ...there will be more subjectivity in act tests. There are even more.
12:16:21 <Jaunita_George_> ack Jaunita_George
12:17:19 <Rachael> Jaunita_George_: It would be a much smaller number of requirements to talk only about protocols and what ACT was a few years ago. There are two concepts here that were expressed earlier. I'm not sure if we want to focus on both or only 1.
12:17:29 <Jaunita_George_> ack MichaelC
12:17:29 <Zakim> MichaelC, you wanted to differentiate protocol from guidance
12:18:21 <Rachael> Michael: The way I'm hearing things discussed. We know we don't have a consensus definition of protocols. The definitions in dictionaries don't match. I hear there are two different goals and we are using the same word for two different goals. What I hear John describing as protocols I define a guidance. What you are doing that is in service of the goal.
12:18:21 <bruce_bailey_> q+ to endorse what chairs see as protocols
12:18:54 <Rachael> ...What I have been using the word protocol for is the steps used to achieve the goal.  Both are needed. Steps without the goal are meaningless.
12:19:12 <JF> Operational Protocols means the administrative policies and procedures of an EMS System or that provide guidance for the day-to-day operation of the system. https://www.lawinsider.com/dictionary/operational-protocols
12:19:19 <Rachael> ...Goals without the steps are also not useful. We need to clarify terms and decide on what we are focusing on.
12:19:42 <Jaunita_George_> ack JF
12:21:07 <MichaelC> q+ to say ultimately I´m intereseted in the parts that impact conformance claims
12:21:35 <Rachael> JF: I pasted in the definition for operational protocols. [reads above]. That is as close as I can to the concept. They define the goals. They may provide steps or user stories and examples. The fact that its a step by step process. May be applicable or not. The commonality of a protocol is that it provides guidance. You have a situation similar to this. These are the things you are looking at. We can give then the tools but at the
12:21:35 <Rachael> end of the day, they have to evaluate the situation and arrive at a decision. Its a contextual decision. If we use the same tools the final destination should be relatively close and meet the goal.
12:21:42 <Jaunita_George_> q+
12:22:03 <Rachael> ...uses experts creating alt text example.
12:22:10 <bruce_bailey_> JF: avatar:  Testable Measurable Repeatable
12:22:11 <Jaunita_George_> ack bruce_bailey_
12:22:11 <Zakim> bruce_bailey_, you wanted to endorse what chairs see as protocols
12:23:20 <Rachael> bruce_bailey_: I had the wrong impression coming into the meeting. I agree with testable, measurable, repeatable I don't think that's the biggest challenge facing WCAG. In 2.0 we had a suggested way to make conformance claims that hasn't been very popular. We've tried to create objective test criteria. The 100% all the time conformance model has been a challenge.
12:23:29 <Rachael> ...I think protocols is exactly what's needed.
12:23:43 <Jaunita_George_> ack MichaelC
12:23:43 <Zakim> MichaelC, you wanted to say ultimately I´m intereseted in the parts that impact conformance claims
12:24:19 <JF> Q+
12:25:07 <JakeAbma__> q+
12:25:49 <Rachael> MichaelC: For me, what I'm interested in solving is making it possible to include guidance in WCAG within the conformance model so organizations can claim conformance against something that can't be repeatably tested. Perhaps we define the guidance and put in how to fit into that format. The guidance isnt' helpful unless we have a to evaluate it. That "test" is the steps that need to be taken so someone can determine whether the
12:25:49 <Rachael> guidance has been followed or not.
12:26:03 <Rachael> ...anything that falls outside of the conformance model is not a priority for my time.
12:26:06 <Rachael> ack Jaunita_George_
12:26:13 <Jennifer> q+
12:26:18 <bruce_bailey_> michaelC:  protocols is sequence of steps for a process to follow WCAG
12:26:57 <Rachael> Jaunita_George_: If outside guidance is used to help support standards, we will need to be careful what we choose. Just stating something was done on the website. Smaller orgs might do so. Larger orgs may treat it as a loophole.
12:27:02 <Jaunita_George_> ack JF
12:27:32 <MichaelC> q+ to clarify what we test or evaluate
12:27:50 <Rachael> JF: Michael, we don't "test" for protocols. We "evaluate". Its subtle but important. A test assume we end up with the same results. An evaluation is a subjective determination of what the results look like.
12:28:17 <Rachael> ...outcomes will be different but if we can evaluate whether they have used a protocol, then we have something we can evaluate our work against.
12:28:31 <Rachael> ...Part of the problem is that we don't have a conformance model flushed out.
12:28:54 <Rachael> ...I envision an conformance model that allows protocols to score based on points.
12:28:56 <MichaelC> q+ to note work on protocols is hopefully part of filling out the conformance model
12:29:20 <Rachael> ...evaluation is harder because we have to take it on faith. We don't have the ability to do a rigid test.
12:29:33 <Jaunita_George_> ack JakeAbma__
12:29:43 <Jaunita_George_> q+
12:29:45 <JF> Q+
12:30:45 <Rachael> JakeAbma__: I would also like to respond to the Michael's remark. He said that guidance is not part of the conformance model. It depends on how we make the conformance claim. I sent part of an ISO standard. ISO is all about processes. The first two paragraphs are [reads paragaraph]
12:30:48 <JF> +1 to this Jake - this is the idea
12:30:50 <Rachael> ...its all about documentation.
12:30:55 <Rachael> q+
12:31:04 <Rachael> ...provide evidence to support the claim.
12:31:26 <Rachael> ...need to document it. The checklist is about what you need to document and how.
12:31:43 <Rachael> q+ to say I think we are all in agreement, propose resolution
12:31:46 <JF> the "public assertion" piece in my pitch
12:32:42 <MichaelC> ack je
12:34:01 <Rachael> Jennifer: Going forward, our WCAG guidelines will have things that are objective (boolean). Then there will be subjective things. References GEL. Showing your homework. I don't think protocols should be about whether someone could pass a protocol. Shoudl meet minimium levels of conformance through something that isnt' a protocol.
12:34:12 <bruce_bailey_> Jennifer:  protocols used to show your homework
12:34:13 <Rachael> ...not baseline.
12:34:34 <JF> Q?
12:34:38 <Jaunita_George_> ack MichaelC
12:34:38 <Zakim> MichaelC, you wanted to clarify what we test or evaluate and to note work on protocols is hopefully part of filling out the conformance model
12:35:05 <bruce_bailey_> s/Shoudl meet/Should meet/
12:35:57 <Rachael> Michael: For me the difference in separating the concept of the guidance, which is what we want to make better, from the protocol, which is the steps to follow the guidance. I also use test and evaluate as you do. The reason we have protocols is to evaluate what we dont' know how to test.
12:36:29 <Rachael> q+ to ask why we can't "test" whether a protocol has meet the documentation requirement
12:37:57 <Rachael> ...We use the protocol to provide the steps that need to be done and test that the steps have been followed. Did you follow them well is another question. A protocol could have evaluation as part of it. In addition to training staff and keeping protocol front and center. The test would be whether you perform the evaluation. The test is on the steps because its on the only thing that is testable.
12:38:14 <Rachael> ...I don't think we can get tied up on the lack of the conformance model. We need to focus on this.
12:38:15 <Jaunita_George_> ack
12:38:25 <MichaelC> ack ja
12:39:23 <Rachael> Jaunita_George_: It does need to be something that can be used as an evaluation tool. It is also to address SC that can't be included in current model. We have to be careful not to create a loophole.
12:39:26 <bruce_bailey_> +1 to MC that work on Protocols can advance absent a specific 3x conformance model
12:40:01 <Jaunita_George_> ack JF
12:40:14 <Rachael> JF: In my mind it keeps getting linked to the conformance model.
12:40:23 <Rachael> +1 to MC and Bruce
12:40:44 <Rachael> ...you have 100 tests. You run 100 tests. You run 80. You lost points.
12:41:32 <Rachael> ...we have subtractive tests. Jaunita had talked about loopholes. Protocols are additive.
12:42:00 <MichaelC> q+ to go back to feeding into conformance
12:42:54 <Rachael> ...you dont' want to adopt it, ok. Note quite voluntary but why wouldn't you want to adopt. When you talk about baseline, I'm not sure it doesn't get adopted into baseline. Participating in protocols should still add.
12:43:12 <Rachael> ...carrot and stick. We have to provide mechanisms to encourage people to strive.
12:43:20 <MichaelC> q+ to say I see protocols as our way *out* of AAA
12:43:21 <Rachael> ...we just put more stuff out there.
12:43:36 <Jaunita_George_> ack Rachael
12:43:36 <Zakim> Rachael, you wanted to say I think we are all in agreement, propose resolution and to ask why we can't "test" whether a protocol has meet the documentation requirement
12:44:04 <JF> Q+
12:44:24 <Rachael> Draft RESOLUTION:  Protocols evaluate success based on whether an organization meets documentation requirements or the steps required
12:45:26 <Jaunita_George_> +1 (from Jeanne)
12:45:42 <JF> ack me
12:46:11 <Jennifer> -1
12:46:16 <JF> Q+
12:46:19 <bruce_bailey_> +1 for me
12:46:22 <Jennifer> Suggestion: An accessibility protocol is a list of considerations used to evaluate the accessibility of commonly subjective success criteria.
12:46:29 <Jaunita_George_> q+
12:46:34 <Rachael> Michael: I agree with John's point that we may want to use test for that resolution.
12:46:37 <Jaunita_George_> ack MichaelC
12:46:37 <Zakim> MichaelC, you wanted to go back to feeding into conformance and to say I see protocols as our way *out* of AAA
12:46:42 <Rachael> JF: Other way around. We want to evaluate things we can't test.
12:46:49 <Rachael> MichaelC: The protocl makes it a test.
12:47:19 <Rachael> ...the stuff leading to the outcome. The result is somethign you evaluate but the steps can be a test.
12:47:24 <Jennifer> No, the protocol does not make it a test, in my opinion. The protocol is how you show your process / your homework for how you met the subjective success criteria of WCAG.
12:47:48 <bruce_bailey_> I think this describes a protocol:  https://www.section508.gov/manage/reporting/questions/
12:48:00 <Jennifer> q+
12:48:03 <Rachael> JF: I can read your public assertion and documentation. Me as a third party. I can read the statement and see what your claim is and then look and evaluate whether you met it or not. You evaluate against the claim.
12:48:05 <MichaelC> q+ to address protocols work wrt conformance
12:48:17 <Jaunita_George_> ack JF
12:48:53 <Rachael> ...the entity claims to have guidance. I can then go and read protocol and content and then evaluate.
12:49:06 <kirkwood> kirkwood has joined #wcag3-protocols
12:49:06 <Jennifer> @bruce_bailey - I don't agree that the section508 list equates to a protocol. That is more of a list of questions to document accessibility efforts.
12:49:18 <bruce_bailey_> i am okay with "evaluate" in draft resolution.  I am not currently in favor of replacing "evaluate" with "test"
12:49:27 <Rachael> MichaelC: Perhaps use a different term
12:49:39 <JF> I am with Bruce
12:49:43 <Rachael> Draft RESOLUTION:  Protocols address success based on whether an organization meets documentation requirements or the steps required
12:49:44 <Jaunita_George_> ack Jaunita_George_
12:49:47 <bruce_bailey_> i don't think protocols should / need to use the word "success criteria"
12:50:03 <JF> =1
12:50:06 <JF> -1
12:50:12 <Jennifer> -1
12:50:21 <bruce_bailey_> i might now be thinking of protocols at too high level a meaning
12:50:57 <JakeAbma__> -1
12:51:11 <bruce_bailey_> address success is very different than address success criteria
12:51:48 <JF> how does one test documentation?
12:51:56 <Rachael> Rachael I think the core question is whether once we have documentation, we either 1) only test the documentation or 2) evaluate the results against the documentation
12:52:03 <Rachael> q?
12:52:22 <JF> Protocols are not requirements
12:52:24 <Rachael> Jaunita_George_: I am not comfortable with organizations just testing documentation based on whether it first conformance.
12:52:29 <Rachael> ...lets focus discussion on what to do next.
12:53:00 <MichaelC> q-
12:53:03 <Jennifer> -1
12:53:08 <JakeAbma__> +1
12:53:26 <Jaunita_George_> ack Jennifer
12:54:22 <ShawnT> ShawnT has joined #wcag3-protocols
12:55:27 <Rachael> Jennifer: JF is here. Can we verify the original intent. The original intent was to evaluate the subjective. You take some other document, pick that, check the work using that, document the work doing that, doesn't get reviewed unless there is legal issue. Other than that, its just did you do your homework?
12:55:45 <Jaunita_George_> zakim, next item
12:55:45 <Zakim> agendum 2 -- Next steps for the working group -- taken up [from Jaunita_George_]
12:56:30 <Rachael> Jaunita_George_: What are our next steps as a working group. Draft resolution but still two ways this could go. Do we want to focus on both ways in parrallell? We develop proposals for the larger working group? Is there a draft resolution there to get consensus?
12:56:37 <kirkwood> agreed with jennifer subjective it is also needed for processes and barriers that are two complex or even too new to be covered
12:56:41 <MichaelC> +1 to parallel work so both approaches can mature
12:56:45 <MichaelC> and be compared better
12:56:59 <Jaunita_George_> Draft resolution: Should we split the group to focus on both ways to use protocols?
12:57:00 <Rachael> +1 to both approaches
12:57:05 <MichaelC> +1
12:57:14 <Jennifer> -1
12:57:39 <JakeAbma__> -1
12:57:40 <Jennifer> * it's not clear to me what the "both ways" is, as that wasn't captured.
12:58:00 <ShawnT> present+
12:58:04 <JF> Q+
12:58:25 <Rachael> Jennifer: request for clarifying both views.
12:58:25 <Jaunita_George_> ack
12:58:37 <Jennifer> ack JF
12:58:42 <MichaelC> q+ to say we´re still early enough in WCAG 3 development, we need to give ourselves room to experiment
12:58:49 <Jaunita_George_> •	View 1: Protocols are an above or beyond any type of conformance, aka “extra credit” •	View 2: Protocols are a way of meeting the guidelines, and how they fit into conformance is TBD
12:58:53 <Jennifer> +1 to JF
12:59:42 <Rachael> JF: Again, I struggle with testing a subjective evaluation. You can't test something subjective. If people want to try taht. Lots of ideas emerging. One of the things to keep in mind is that industry wants less complexity, not more. WE are making it more complex than it has to be.
12:59:47 <kirkwood> you can test with a group poll/survey. for ‘subjective’ criteria
13:00:01 <Rachael> ..is it scalable to large industry. Going in opposite direction.
13:00:09 <Jennifer> I will definitely vote -1 for View 2. Protocols shouldn't be a way of meeting the guidelines — only a way of showing how the subjective guidelines were met — they're subjective, they can't be T/F met.
13:00:20 <kirkwood> present+
13:00:40 <Rachael> Jaunita_George_: Get folks thoughts but we may not be quite ready get.
13:01:00 <MichaelC> q-
13:01:06 <Rachael> rrsagent, generate minutes
13:01:06 <RRSAgent> I have made the request to generate https://www.w3.org/2022/04/08-wcag3-protocols-minutes.html Rachael
13:02:03 <Rachael> zakim, end meeting
13:02:03 <Zakim> As of this point the attendees have been Jaunita_George_, Rachael, Jennifer, MichaelC, JakeAbma__, JF, bruce_bailey_, ShawnT, kirkwood
13:02:05 <Zakim> RRSAgent, please draft minutes
13:02:05 <RRSAgent> I have made the request to generate https://www.w3.org/2022/04/08-wcag3-protocols-minutes.html Zakim
13:02:09 <Zakim> I am happy to have been of service, Rachael; please remember to excuse RRSAgent.  Goodbye
13:02:13 <Zakim> Zakim has left #wcag3-protocols