<Rain> scribe: julierawe

Jennie can't make today's meeting

Further discussion on the outline of the testing document

<Rain> testing document link: https://docs.google.com/document/d/1Otsl4HTCvpQj63xYVKzOVkulf6o75dWEthu6WI9Vtj8/edit#heading=h.lswn4t749mdu

Shawn and Jennie have not had a chance yet to merge all 3 of the suggested outlines together

Shawn, Jennie and Julie did meet to discuss possible ways to merge.

The more we talked about it, the more Shawn and Jennie weren't sure where this subgroup is going.

We're debating is this going to be a separate document or a section within Making Content Usable?

The three proposals from Jennie/Shawn, Julie, and Lisa seem like a big document

Julie: We debated whether we need to make a comprehensive testing document that duplicates a lot of WCAG 3 test types, or if we should focus on deep dive into how to conduct user testing with people who have cognitive and learning disabilities

Shawn: Should we focus first on user testing? Or should we focus first on figuring out which test types are needed for each pattern in Making Content Usable?

John K: As I look at this doc, I wasn't sure what the goal of this work is.

John K: Is the goal to create an index of which types of test are useful for people with cognitive and learning disabilities?

John: Are we creating a resource that is broader than Making Content Usable? Are we looking for what the gaps are?

Shawn: I agree. I'm not sure what we're trying to do with this.
... When I joined this subgroup, I thought it was to come up with tests for different patterns.
... Then there was a change and it seemed like something different
... What is the title of this document? Is it "How to test Making Content Usable"?

<Rain> https://docs.google.com/document/d/1JXa94s2lbzJ0v9FHasxxws3CsOcljHHBdlQ2VOxYqAQ/edit#

Shawn: Is it "How to do user testing with people who have cognitive and learning disabilities"?

<kirkwood> what is the need? is also my question. but Rain is addressing it

Rain: All good questions. I think for version 2 of Making Content Usable, this subgroup would fall under 3rd column, for "Test process draft."
... I believe the thought here is Making Content Usable needs a good test process within the document.
... I think Shawn's initial thinking about how to test our actual patterns is not a bad idea. That would inherently make us think about how to test the patterns.
... The "how to" would inherently come out of that work.

<kirkwood> can we put these goals in top of document?

Rain: The key performance indicators (KPIs) say by March 2023 would be to have an exploratory draft on how to test with people who have cognitive and learning disabilities."

<kirkwood> “exploratory draft of how to test for people with cognitive disabilites” -PERFECT

<ShawnT> COGA Action Items: https://docs.google.com/document/d/15HtPkkYx1CIl6bAwP2nsSZKhqTVbqcuMDRz5RmtmvXg/edit#

Shawn: The COGA Action items say our KPI is to develop tests for 10 patterns with edge case
... One group to test the patterns with binary testing and another group focusing on a guide on how to test things to make sure people with cognitive and learning disabilities are involved in the testing

<kirkwood> Rain seems to be saying what the schedule is. Maybe we need a process outling in here. So Goal at top and Process below?

Rain: What would be more useful is to try to come up with the tests for the patterns themselves rather than to come up with an overarching document
... The overarching document would be so general.
... Looking at the size of this group, splitting up the group might be difficult
... I recommend thinking about what the group wants to spend time on and in what order
... What does the group think will be the most useful output?

<ShawnT> +1 to Rain

John K: I feel like we're talking about the goals and also the process.

<ShawnT> Keeping one group and maybe pivoting the focus

John K: Can we put the goals at the top and then make sure the process lines up with the goals?

<Rain> and the KPIs may need to change

Shawn: The KPIs will need to change if we decide to pivot

Rain: The KPIs need to be rewritten. The idea of having a full working draft by this March is not realistic.
... A working draft for 2 or 3 patterns might be realistic.

Shawn: When I first started looking into this, I create a spreadsheet listing all the patterns and which type of test for each pattern
... Then the switch came that we needed to create this outline
... The outline goes into depth, but is it redoing Web Accessibility Initiative (WAI) team is doing?
... I think making the decision, we need Jennie and Lisa's input

Julie: Should the outline say 1st step is to take initial guess for test type for each pattern, then start to develop tests for 1-2 patterns at a time?
... Initial guess might not work out, so may need to change the plan for a particular pattern, but won't know until really dig into it

John K: I always like to have level of effort with manual testing and/or hours involved

John K: It might be something to think about to include

Shawn: I believe there was once talk of putting that in WCAG 2.0 about how hard it was to do a test? The level of complexity?

John K: It would be very powerful but opening a can of worms

Julie: Reminder that Silver's assertion doc will be ready for COGA to provide feedback on soon

Shawn: Yes, that ties in with exercise of going through every pattern and deciding if can be binary test or needs procedural testing

Julie: "Ease of use" is hard to do as binary

John K: I agree, but after doing this for a long time, it's up to us to come with language to make it binary

John K: Is the phone number easy to find? That's a binary assertion. You can say yes or no. But the court of public opinion will determine if easy to find.

John K: It's one of those 'you know it if you see it'

John K: It's up to us as subject matter experts to put the question down in a way that there is a binary answer

John K: We have the expertise to do it. We have to figure out what the question is.

<Rain> +1 to kirkwood -- and we may find more research questions that we need to answer in order to make this happen

John K: We need to boil down to what's the question?

<ShawnT> [4.8.1 Provide Human Help (Pattern)](https://www.w3.org/TR/coga-usable/#provide-human-help-pattern)

John K: If you're using mobile rather than wide screen, and the "Help" drops off the screen, then no, it's not easy to find

John K: I'm wary of putting in gray areas

John K: I think we can do this in a way that is accessible and inclusive to all

Rain: I agree with John K. One of the things that comes up repeatedly when we try to come up with yes/no, it often brings up the question of is there research on that.
... I think we need to come up with areas where we don't know if there's research on the specific rules we're coming up with rather than giving up.

John K: We don't have much on the need for research in this testing outline.

Shawn: I agree need to relate to research.
... Is it wrong to say our expertise we see this and that's our conclusion?

Rain: We need to acknowledge it needs to be tested.
... Otherwise, it won't get approved by larger group.

Julie: Should the testing outline require a section on research?

Rain: In order for us to come up with a Yes/No, how are we sure it's accurate?
... For example, using double negatives to express a positive: The test is that you're not using double negatives to express a positive. How can we be sure that is actually a correct result every time?
... What's our proof that universally across every language that a double negative to express a positive is not a good thing?
... In most languages, confirm there is no double negative to express a positive.
... In Spanish, you might use double negatives to express a positive in this very specific instance

Shawn: Moving forward, we discuss with Lisa and Jennie to make sure we should look at the patterns and try to see how we can test against different patterns.

Julie: What should the outline be? A timeline of how we're planning to approach this work?

John K: I agree — what is the outline?

Shawn: The outline is for best ways to test, what's needed
... To me, the answer to "But what about the outline" is before we define the outline, we should look at what are the best ways to test against the patterns

John K: I recommend putting the goal at the top of the outline so we're all on the same page of what we're doing and why

John K: Let's state what we think the outline is for and see if Lisa is OK with it.

Julie: Is the outline our plan for getting the work done?

<Rain> RRSAgent: make minutes

This is scribe.perl Revision VERSION of 2020-12-31
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Default Present: Lisa, abbey, Kiki, JustineP, david-swallow, kirkwood, julierawe, ShawnT, EA, Rain
Present: Lisa, abbey, Kiki, JustineP, david-swallow, kirkwood, julierawe, ShawnT, EA, Rain
Regrets: jennie
Found Scribe: julierawe
Inferring ScribeNick: julierawe

WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: 

WARNING: Input appears to use implicit continuation lines.
You may need the "-implicitContinuations" option.


WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)