IRC log of silver on 2020-03-10

Timestamps are in UTC.

15:13:08 [RRSAgent]
RRSAgent has joined #silver
15:13:08 [RRSAgent]
logging to https://www.w3.org/2020/03/10-silver-irc
15:13:25 [Zakim]
Zakim has joined #silver
15:13:28 [jon_avila]
present+
15:13:37 [JF]
Q+ chuck
15:13:41 [JF]
ack chu
15:13:48 [jeanne]
present:
15:13:48 [jeanne]
chair: Shawn, jeanne
15:13:48 [jeanne]
present+
15:13:48 [jeanne]
zakim, clear agenda
15:13:49 [jeanne]
rrsagent, make minutes
15:13:49 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/03/10-silver-minutes.html jeanne
15:13:49 [Zakim]
agenda cleared
15:13:50 [sajkaj]
zakim, who's here?
15:13:50 [Zakim]
Present: jeanne
15:13:51 [Zakim]
On IRC I see RRSAgent, PeterKorn, JakeAbma, KimD, jon_avila, jeanne, bruce_bailey, Makoto, CharlesHall, JF, Nicaise, david-macdonald, Jennie, stevelee, Lauriat, Chuck,
15:13:51 [Zakim]
... ChrisLoiselle, kirkwood, Shri, laura, Roy, Ralph, jcraig, sajkaj, MarcJohlic, Ryladog, MichaelC, Rachael, achraf, alastairc, yatil, AWK, trackbot
15:13:55 [bruce_bailey]
q+ to talk about conformance work
15:13:56 [sajkaj]
present+
15:13:58 [ChrisLoiselle]
present+
15:14:01 [laura]
present+ Laura
15:14:03 [Jennie]
present+
15:14:04 [ChrisLoiselle]
Scribe: ChrisLoiselle
15:14:05 [jeanne]
q?
15:14:07 [kirkwood]
present+
15:15:26 [Lauriat]
Present+
15:15:33 [Lauriat]
Present+ Lucy
15:15:35 [alastairc]
present+
15:15:37 [Makoto]
present+
15:15:45 [Chuck]
present+
15:15:46 [JF]
Present+
15:15:51 [stevelee]
present+
15:15:54 [ChrisLoiselle]
Bruce Bailey: The directions for the conformance exercise for headings or visual contrast were misunderstood. Tallying for qualitative assessement may not work. I have written up some information that I'd like to share
15:16:19 [bruce_bailey]
https://docs.google.com/spreadsheets/d/1G0KLv1Nfvy5QWN7t9jPxyE6UEcTHE5A8tKYiDOhuZRY/edit#gid=1833982643&range=A1
15:16:32 [KimD]
present+
15:16:58 [jeanne]
rrsagent, make logs public
15:17:01 [ChrisLoiselle]
Spreadsheet that Jeanne shared before, Sample Scoring example is name of the google sheet
15:17:20 [Detlev]
Detlev has joined #silver
15:17:53 [AndyS]
AndyS has joined #silver
15:18:00 [jeanne]
rrsagent, make minutes
15:18:00 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/03/10-silver-minutes.html jeanne
15:18:08 [PeterKorn]
q+ (for when Bruce is finished with his description)
15:18:17 [PeterKorn]
q+
15:18:27 [JF]
Q+
15:18:30 [AndyS]
AndyS present+
15:18:37 [ChrisLoiselle]
Bruce Bailey: Ratings / Score is Outstanding / 4 , Very Good / 3 , Acceptable / 2 , Unacceptable / 1
15:18:37 [AndyS]
present+
15:18:54 [PeterKorn]
present+
15:18:56 [Lauriat]
ack bruce_bailey
15:18:56 [Zakim]
bruce_bailey, you wanted to talk about conformance work
15:19:00 [Lauriat]
ack PeterKorn
15:19:00 [ChrisLoiselle]
Reviewing the heading , use of headings homework
15:19:05 [Chuck]
q+ lucy
15:19:15 [kirkwood]
well done Bruce!
15:20:04 [PeterKorn]
q-
15:20:12 [Lauriat]
ack JF
15:20:14 [ChrisLoiselle]
PeterKorn: Mirrors what we've been doing within Amazon. I really like the potential for this to work with scoring rubric. I.e. for this product release, these items are very good, these items are acceptable, etc. This is good.
15:21:33 [ChrisLoiselle]
JF: This is getting better in terms of granularity. Where would we integrate en301 within the subsections? Where would we get into the 7 functional requirements?
15:21:58 [Lauriat]
ack lucy
15:22:01 [ChrisLoiselle]
Jeanne: They will be in a different location. We may merge this into scoring.
15:22:33 [ChrisLoiselle]
Lucy: Can you explain the scoring a bit more (to Bruce B.)
15:23:38 [ChrisLoiselle]
Bruce: Not skipping a heading level , is outstanding / 4 or Very Good / 3.
15:24:16 [ChrisLoiselle]
Bruce Bailey: If you skip levels, you at very best are at acceptable.
15:24:31 [mattg]
mattg has joined #silver
15:24:52 [mattg]
present+
15:24:59 [ChrisLoiselle]
Lucy: Acceptable to me seems that you've met every point. Actually, Very Good means you've hit every point.
15:25:56 [ChrisLoiselle]
Bruce Bailey: I also looked at Clear Language and Visual Contrast of Text and went through the same rating
15:27:15 [ChrisLoiselle]
Step 1 is to assign this to a web page in a website, then assign the website a number as well, so the numbers, mean, mode etc. would give a person a score for wcag 2.1 .
15:27:42 [ChrisLoiselle]
3 out of 4 guideline is rated as outstanding...
15:29:00 [ChrisLoiselle]
A rubric that works for assigning silver, bronze, gold rating to websites could be used as well as the rating scores.
15:29:23 [ChrisLoiselle]
Shawn L: Opens to JF for comments on his work
15:29:27 [JF]
http://john.foliot.ca/demos/HeadingsTestOne.html
15:30:02 [jeanne]
agenda?
15:30:11 [ChrisLoiselle]
JF: Shares a Testing Headings Sample Page. Talks to heading structure being used properly. Each heading has a class.
15:30:24 [jeanne]
agenda+ Review of homework: what insights did people gain from it?
15:30:24 [jeanne]
agenda+ Conformance and Minimums
15:30:24 [jeanne]
agenda+ Should we use IETF standard RFC 2119
15:30:42 [jeanne]
zakim, take up item 1
15:30:42 [Zakim]
agendum 1. "Review of homework: what insights did people gain from it?" taken up [from jeanne]
15:30:51 [jeanne]
rrsagent, make minutes
15:30:51 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/03/10-silver-minutes.html jeanne
15:31:15 [maryjom__]
maryjom__ has joined #silver
15:31:30 [ChrisLoiselle]
The entire document is made up of headings, 18 heading 1's .
15:31:55 [ChrisLoiselle]
My question is the score going to be the same as the previous example shared?
15:32:24 [PeterKorn]
q+
15:33:19 [ChrisLoiselle]
The pages John shares are Testing and Scoring Headings - Master Page , http://john.foliot.ca/demos/HeadingsTestOne.html
15:33:32 [ChrisLoiselle]
Testing Headings - Test 1 , http://john.foliot.ca/demos/HeadingsTestTwo.html
15:33:50 [ChrisLoiselle]
Testing Headings - Test 2 , http://john.foliot.ca/demos/HeadingsTestThree.html
15:34:09 [ChrisLoiselle]
Testing Headings - Test 3, http://john.foliot.ca/demos/HeadingsTestFour.html
15:34:32 [Lauriat]
ack PeterKorn
15:34:32 [ChrisLoiselle]
At end of each page, the negative impact on functional requirements is listed in a table
15:35:06 [Chuck]
q+ lucy
15:35:14 [ChrisLoiselle]
PeterKorn: These detailed examples are fantastic. Comment: Some of the examples are easy to find with a programmatic tool.
15:35:36 [ChrisLoiselle]
If a tool could have found it, and you didn't do it, it is not acceptable.
15:35:40 [jeanne]
+1 Peter
15:35:56 [PeterKorn]
q-
15:36:59 [ChrisLoiselle]
JF: Tools aren't going to catch all examples. I wouldn't outright fail for all users. Failing for some users is valid. The Functional Requirements are key to a scoring rubric.
15:37:16 [Lauriat]
ack lucy
15:37:32 [ChrisLoiselle]
Shawn L: We can talk to this in minimums
15:38:29 [david-macdonald]
interesting that JAWS and NVDA announce the level without aria level for <div role="heading" class="level_2" ...
15:38:33 [jeanne]
Meeting: Silver Virtual F2F Tuesday
15:38:41 [jeanne]
rrsagent, make minutes
15:38:41 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/03/10-silver-minutes.html jeanne
15:38:42 [ChrisLoiselle]
Lucy G: I love the examples, John. I see Bruce's where a tally needs to add up to 100 points. Everything would be weighted and have weights within it.
15:38:58 [ChrisLoiselle]
JF: A more granual score helps content creators as well.
15:39:15 [ChrisLoiselle]
Lucy G: I think we are on the correct path.
15:39:42 [ChrisLoiselle]
Jeanne: If we look at Bruce's example and drill down into more granual approach, would that help?
15:40:06 [Rachael]
Present+
15:40:09 [ChrisLoiselle]
JF: Explicit definition of semantic headings would be useful.
15:40:41 [ChrisLoiselle]
JF: The structure vs. the visual presentation helps cognitive disability user group.
15:41:26 [Chuck]
+1 to Jeanne's idea
15:41:35 [david-macdonald]
q+
15:41:38 [ChrisLoiselle]
Jeanne: What if we took each of the 4 examples JF has and turned them into the correct thing, and turn those into methods? If we then reference the individual methods within Bruce's example to know what they need to review for HTML and scored that way?
15:41:57 [ChrisLoiselle]
JF: How many methods could be constructed using the ACT rules format?
15:42:09 [Lauriat]
+1
15:42:16 [ChrisLoiselle]
Jeanne: Exactly, using ACT in methods. Scoring could reference those.
15:42:36 [ChrisLoiselle]
Referencing ACT tests would work well in methods.
15:42:38 [Lauriat]
ack david-macdonald
15:43:11 [Chuck]
q+ lucy
15:43:17 [ChrisLoiselle]
David McD: JAWS reads those headings properly, wondering what accessibility supported score would be if semantic code is not totally correct?
15:43:56 [ChrisLoiselle]
Normative and methods comment: Looking at Bruce's examples, column B could be methods.
15:44:23 [ChrisLoiselle]
Techniques would be technology specific and non normative
15:44:51 [Lauriat]
ack lucy
15:45:02 [JF]
+1 to Lucy
15:45:12 [ChrisLoiselle]
Lucy G: Assitive Technology will change mind at the moment, same as browser. Methods must stick to is code correct or not?
15:45:13 [Chuck]
+1 to lucy
15:45:17 [jeanne]
+1 Lucy
15:45:19 [laura]
+1
15:45:23 [ChrisLoiselle]
+1 to Lucy (Chris without scribing)
15:46:06 [ChrisLoiselle]
Shawn L : To group: Let us shift gears to next agenda item
15:46:56 [ChrisLoiselle]
Jeanne: Scoring examples topic: Testing real websites is best. It changed how I was approaching things. Test against a real website.
15:46:57 [jeanne]
zakim, take up next
15:46:57 [Zakim]
agendum 2. "Conformance and Minimums" taken up [from jeanne]
15:47:07 [sajkaj]
+1 to Jeanne
15:47:55 [PeterKorn]
q+
15:48:09 [ChrisLoiselle]
Shawn L: Criticality is important. How does one express it in a score and fundamental critical issue of accessibility?
15:49:20 [bruce_bailey]
q+ to say that i like adjectival rating over tally because critical aspects could be in "average" and above
15:49:23 [ChrisLoiselle]
Very poor, acceptable , etc. Within Silver, do we want to be the ones drawing the line on critical issues?
15:49:26 [CharlesHall]
q+
15:49:31 [Lauriat]
ack PeterKorn
15:50:12 [ChrisLoiselle]
PeterKorn: Looking at JF's second example, if the page only had two headings, one a heading 1 and one a heading 2...usage without vision, hiearchy structure would be a fail, but would the impact to the user be significant?
15:50:34 [JF]
exactly
15:50:35 [ChrisLoiselle]
Is it not usable without vision?
15:51:13 [PeterKorn]
q-
15:51:14 [ChrisLoiselle]
We need to think of overall functionality and impact of the fail and how we view the site?
15:52:43 [ChrisLoiselle]
JF: Peter , I agree. I built the examples where structure created for screen readers were ok. If I did unstyled divs, structure would off. Impact of different user groups needs to be looked at. Visual users vs. non visual users / screen reader users.
15:53:21 [PeterKorn]
q+
15:53:25 [Lauriat]
ack bruce_bailey
15:53:25 [Zakim]
bruce_bailey, you wanted to say that i like adjectival rating over tally because critical aspects could be in "average" and above
15:53:25 [ChrisLoiselle]
I.e. h2 to h5 , still usable ? Or a fail? How is it rated / scored? As opposed to pass or fail.
15:53:37 [Chuck]
q+ lucy
15:54:11 [JF]
s/If I did unstyled divs, structure would off./If I did unstyled divs, structure would off visually for sighted users (COGA)
15:54:17 [ChrisLoiselle]
Bruce Bailey: Critical are acceptable or above in my example.
15:54:52 [Lauriat]
ack CharlesHall
15:56:32 [ChrisLoiselle]
Charles Hall: I wanted to add to the criticality severity comments. If I'm evaluting a rubric against a subset vs. an author of the website, I the author have the right to scope of my page through a task flow
15:56:41 [JF]
Not sure if we've arrived at consensus to Charles' point
15:56:45 [JF]
\regarding scoping
15:56:57 [ChrisLoiselle]
Chris Loiselle to Charles: If I'm writing that wrong, please add in your comments. Sorry!
15:56:59 [AndyS]
q+
15:57:26 [Lauriat]
ack PeterKorn
15:57:33 [jeanne]
+1 JOhn - The consensus on scope was a logical subset, not a task
15:57:53 [PeterKorn]
q-
15:58:04 [ChrisLoiselle]
PeterKorn: Scoring with numbers is a very easy way to get lost in what does 85 % mean? Adjectivity based approach may lead to more progress quickly on scoring.
15:58:05 [Lauriat]
ack lucy
15:58:57 [PeterKorn]
q+
15:58:59 [PeterKorn]
(to respond)
15:59:04 [ChrisLoiselle]
Lucy G: If adjectival route is the way we are going, numbers will also help in the end as well. I.e. full one point can be broken down as well. Requirement has its own scoring.
15:59:16 [JF]
Q+ to respond to Peter's comment about numbers
15:59:17 [ChrisLoiselle]
When it comes to a critical criteria, that is when we can weight it.
15:59:21 [Detlev]
q+
15:59:57 [ChrisLoiselle]
Think of it as containers. Language would be its own container. Is language more critical than headings?
16:00:09 [Lauriat]
ack AndyS
16:00:10 [PeterKorn]
q-
16:00:14 [jeanne]
q+ to talk about testing criticality and severity
16:00:39 [Chuck]
we are at time to change scribes, and someone else will need to monitor the participants list for raised hands, as I will not be able to scribe and watch that list.
16:00:55 [ChrisLoiselle]
AndyS: Page is structured. Ads are present.
16:01:09 [ChrisLoiselle]
@Chuck. I can continue to scribe after a five minute break.
16:01:13 [CharlesHall]
that was CharlesHall that mentioned the <aside> as part of the scope
16:01:27 [ChrisLoiselle]
I will return very soon.
16:01:32 [Chuck]
scribe: Chuck
16:01:49 [sajkaj]
I can
16:02:09 [Chuck]
Jeanne: Any followup on what Andy said?
16:02:15 [Lauriat]
q?
16:02:21 [Lauriat]
ack JF
16:02:21 [Zakim]
JF, you wanted to respond to Peter's comment about numbers
16:02:39 [jeanne]
q+ Lucy
16:02:40 [Chuck]
JF: responding to Peter's comments on number. Appreciating all the issues a number means, my understanding is...
16:02:56 [Chuck]
JF: This exercise is about getting a number. We currently have 100% or zero (in wcag 2.x)
16:02:57 [PeterKorn]
q+
16:03:15 [Chuck]
JF: A number can be misleading at times. If anybody used the chrome tool (lighthouse), at the end of the process...
16:03:49 [Chuck]
JF: Chrome added a score. We don't know where that score came from. If we start from premis that you never get 100, that percentile becomes incentive for doing better (72%, let's try to get to 85%).
16:04:11 [Chuck]
JF: Peter, as much as numbers can be a rat hole, it's a critical part of what we are trying to do in silver.
16:04:18 [ChrisLoiselle]
Chuck, I can scribe again.
16:04:23 [ChrisLoiselle]
Scribe: ChrisLoiselle
16:04:46 [Chuck]
PK: I wasn't here for all of silver discussions... my understanding is that the high order bit is to move away from pass-fail perfection, to "mildly or largely".. numbers isn't the goal.
16:05:19 [Chuck]
PK: The goal is to get away from pass/fail. I like the rubrik. We can evaluate the rubrik evaluation. Some things are acceptable, some things are good.. or almost everything is great but some are good...
16:05:29 [ChrisLoiselle]
PeterKorn: The goal was to move away from pass / fail. I like the rubric. If we have a way to collect up the rubric evaluations, most are very good, the product will be very good.
16:05:51 [PeterKorn]
q-
16:05:51 [Chuck]
PK: Whether or not we assign numbers, we need to think on severity, the people, and the impacts. I like adjectival approach. I don't know what 87% means. I do know what acceptable and very good means.
16:06:04 [CharlesHall]
to PeterKorn’s point, a qualitative metric can be converted to a quantitative score based on the number of adjectival categories
16:06:22 [Chuck]
JF: I agree with that statement, in regulatory environment we need to hit a bar. "Intelectually honest" means that there isn't something that is 100%.
16:06:32 [Lauriat]
ack Detlev
16:06:38 [Chuck]
Shawn: I think you are agreeing on different points. Let's return to queue.
16:07:13 [Chuck]
detlev: I was wondering... you mentioned the 9 user accessibility needs. In the rating, there would be a plan to issue different types of results for different impacts...?
16:07:20 [Rachael]
Q+
16:07:41 [Chuck]
detlev: Also, why is there a mismatch between user accessibility needs in EU implementation (there you are missing limited reach)...
16:08:12 [Chuck]
detlev: you have two categories for people with hearing problems (hard of.. and no). Is there a conscious decision to drop the difference between hard of and no hearing?
16:08:31 [Chuck]
Jeanne: It may be that I took an old version of the EU directive. Can you send me a link to the current?
16:08:58 [Chuck]
Detlev: I was wondering why users with limited reach was left out. someone explained... I'm not convinced there's a good reason to leave it out.
16:09:08 [CharlesHall]
we have discussed adding functional needs to the EN standard, like “intersectional”
16:09:24 [Chuck]
Jeanne: Great. We did have a discussion of adding things, for limited cognition, vestibular. We never discussed dropping any, so I may have had an old version.
16:09:53 [Chuck]
detlev: Is there an intention in all the scoring for differentiated by user group? At one point it was a no-no. At some point we decided we didn't want to differentiate, but not sure if that has changed.
16:10:17 [Chuck]
Shawn: We've been thinking about it more closely to what John demonstrated in that for different guidelines looking at the effects on different user needs given the task or scope of testing.
16:10:42 [Chuck]
Shawn: We don't have a fleshed out illustration of that, but we have been considering. We want to accomplish making sure that we are not inadvertently leaving user needs out.
16:11:06 [Chuck]
Shawn: For example, if it works great for limited cognition but it completely leaves out users who use screen readers, we want the ability to highlight that, and visa-versa.
16:11:19 [Lauriat]
q?
16:11:20 [Chuck]
Shawn: Like if everything is symantically accurate, but visually not.
16:11:23 [Lauriat]
ack jeanne
16:11:23 [Zakim]
jeanne, you wanted to talk about testing criticality and severity
16:11:55 [Chuck]
Jeanne: I'm glad we are having conversation about criticality. I know there's a lot of... people who feel it's very important. But one of the things we agreed on in Silver is we'll be data driven and research based.
16:12:25 [Chuck]
Jeanne: whenever we tried to score real websites against criticality we consistently didn't find a way to do that in a way that didn't penalize people with some disabilities.
16:12:34 [JF]
Q+
16:12:36 [Detlev]
q+
16:13:00 [Chuck]
Jeanne: We have to find a way to stop penalizing people with some sorts of disabilities. We could not find a way to test it that didn't structurally disadvantage people with low vision and congitive issues.
16:13:18 [KimD]
+1 to Jeanne
16:13:29 [Lauriat]
ack Lucy
16:13:33 [Chuck]
Jeanne: It's great to talk about in theory, but if you want to propose, you need to show research that demonstrates everyone is speaking equally.
16:14:04 [Chuck]
lucy: Speaking to numbers, we need to offer something that leaders can relate to. If we offer something confusing, they will blank us out and do other tasks.
16:14:08 [AndyS]
Comment: IMO the only way to treat all disabilities equally involved a customization and personalization, so that individual needs are accommodated *as needed*
16:14:21 [JF]
+1 to Lucy
16:14:27 [Chuck]
lucy: We have to have those numbers. A lawyer is not going to understand what it means to have "this or that" level. They want a number and a way to improve that number.
16:14:34 [Lauriat]
ack Rachael
16:15:09 [Chuck]
Rachael: I have 3 things to keep in mind. Been brought up before, may be not in in this call. If we don't take functionality into account, and we come up with a %, that % will mean different things to different groups.
16:15:17 [bruce_bailey]
q+ to say numbers can be used to measure progress, but i have never seen numbers that were comparable from one website to another (or one tool to another)
16:15:21 [Chuck]
Rachael: 80% may mean one thing to ceasures and another to a blind.
16:15:53 [Chuck]
Rachael: If you have a group of individuals who fall in the category of blind, vs people in coga, that hierarchy may introduce discrimination.
16:16:12 [Chuck]
Rachael: This is such a rich and fantastic discussion, but whatever we come up with needs to be understandable and simple.
16:16:16 [Lauriat]
q?
16:16:23 [Lauriat]
ack JF
16:17:07 [Chuck]
JF: Addressing a comment from Jeanne. I'm support of all user groups including coga, I do so in action and words. The reality is that if we take that user group in consideration, that is one user group.
16:17:42 [Chuck]
JF: I want to recognize severities. Those heading examples, if I remove visual structure, a person who has a cognative issue and is blind is doubly disadvantaged.
16:18:17 [Chuck]
JF: If we determine that an impact has a greater impact against a group, we can factor that in. That's what I said 6 months ago. Not all things are created equal. We have to boil this down to a score and strategies to improve the score.
16:18:26 [Chuck]
Jeanne: I only disagree with the weighting.
16:18:55 [Chuck]
Jeanne: "this is more severe than that". I don't have an issue with your example. At the guideline level I think you can do that.
16:19:06 [Chuck]
Lucy: The weighting should be by criteria.
16:19:29 [Chuck]
Lucy: So many disabilities are impacted by this or that. I won't pit one against another, but if it's 4 vs 1, we have no choice but to weight that.
16:19:57 [Chuck]
Jeanne: I disagree. When you start looking at it as a whole, there are too many things that we give guidance for that are heavily weighted to blind vs cognative.
16:20:19 [Chuck]
Jeanne: The way we are set up including Silver is that we measure things granularly, and we say this is more important than that, but when we look at it as a whole...
16:20:30 [CharlesHall]
would a priority or severity not be functional need agnostic?
16:20:49 [Chuck]
Jeanne: granularly, this is a complete blocker, but someone with a cognative issue may be able to work it out. But in total the cognative issues become a blocker, because you have to look at it in totality.
16:21:02 [mattg]
q+
16:21:03 [PeterKorn]
q?
16:21:08 [Chuck]
Jeanne: Cognative loses when we say "this individual piece" is more important than "this one".
16:21:08 [jon_avila]
I object to the notion that WCAG is heavily focused on Blind and visual impairment. There are many WCAG criteria that are aimed at a wide range of users with disabilities
16:21:09 [JF]
Q+
16:21:48 [Chuck]
Jeanne: That alt text is more important than captioning. It's the totality of the website, it's all the guidelines. If they are getting a lower weight because of each individual guideline, they get a website that they cannot use.
16:21:58 [Chuck]
Jeanne: Can someone else make the argument better?
16:22:06 [alastairc]
q+ to say that how we measure things is more important than how we weight them
16:22:18 [Chuck]
Shawn: It's a complex topic and we should keep in mind as we work through. We all have the same level goals for conformance.
16:22:21 [Lauriat]
ack Detlev
16:22:54 [Chuck]
detlev: Just to get to Jeanne's argument about critical issues, penalizing others. Why is that a problem? Why wouldn't it be possible to ask all the groups involved to basically identify show stopper issues. We know those.
16:23:24 [PeterKorn]
q+
16:23:28 [Chuck]
detlev: keyboard trap, lack of captions, so on. Maybe show stopper for cognative individuals. Basically ok to collect those issues and make them critical issues.
16:23:37 [Chuck]
detlev: Can you explain further Jeanne?
16:23:47 [Chuck]
Jeanne: Let's honor queue.
16:23:53 [Lauriat]
ack bruce_bailey
16:23:53 [Zakim]
bruce_bailey, you wanted to say numbers can be used to measure progress, but i have never seen numbers that were comparable from one website to another (or one tool to another)
16:23:54 [alastairc]
q+ jeanne
16:24:26 [Chuck]
Bruce: We've been trying (everyone!) to rate your website since days of Bobby. All of these rules, all the years, it's only ever useful for the developer to make progress. That you aren't regressing.
16:24:35 [Chuck]
Bruce: Lot's of tools will give you a percentage.
16:25:17 [Chuck]
Bruce: Those only make a difference on one domain. You can't compare an 87 on one site to an 86 on another site. Or even cross tools. I don't feel like we will make progress if we try to have granular scoring systems.
16:25:37 [Chuck]
Bruce: Unless there are 1000s of data points. Enough data points that you aren't doing manually that eventually the weaknesses of the tool averages out.
16:25:47 [PeterKorn]
+1 to Bruce
16:25:51 [Chuck]
Bruce: I won't be able to say that one outstanding site compares well to another.
16:25:58 [Lauriat]
ack mattg
16:26:27 [Chuck]
Matt: Returning to yesterday, seems like there are 2 different criteria you are trying to assess. How well has the author created the content, what score can you give them, vs how do you make this understandable to the user.
16:26:35 [david-macdonald]
+1 to bruce
16:26:38 [CharlesHall]
to Detlev’s point, the challenge is not that one functional need does or does not have a critical item. i think the challenge is where 2 or more functional needs have a critical item that is in conflict.
16:27:01 [Chuck]
Matt: There's a bucket approach, there's probably going to be something that they haven't met requirements to they met them all. There are different methods which have different impact on different groups.
16:27:23 [Chuck]
Matt: That's different from the number score that the producer gets. I think that these two different aspects need to be viewed separately.
16:27:27 [Chuck]
Shawn: Indeed.
16:27:29 [Lauriat]
ack JF
16:28:07 [Chuck]
JF: Two thoughts... Jeanne mentioned one requirement is captions. If we look at the severity of captions. If you are deaf and a video is missing captions, that's critical. If I have a cognative issue and I have captions..
16:28:29 [AndyS]
q+
16:28:32 [Chuck]
JF: That helps me. Offering captions to a cognative user offers some benefit, but to a deaf user is completely critical.
16:28:51 [Lauriat]
q+ Lucy
16:28:59 [JF]
https://docs.google.com/spreadsheets/d/1EXw5W6SuMXk7mrFN33hHVQuCn0DhKysXmUCKNML3SKg/edit#gid=108726882
16:29:20 [Chuck]
JF: How do we make this fair, realizing we can't get every individual out there? In an earlier example (6-8 months ago), I put forward a draft proposal, that had suggested that as we were calculating scores we used a weighting mechanism, and use that as a multiplier.
16:29:36 [Chuck]
JF: We would ultimately have a better... more data points. More data points will give us a better score.
16:29:43 [jeanne]
q+ to say to Detlev when each group gets a show stopper, then a bonus is given to that group. But for COGA, its the overall sum of all the guidelines. So they always get the lower weight on each indiviudal guideline. COGA doesn't get "absolute blockers", so they never get a higher weighted score. But their overall website is an "absolute blocker". So we can't get an weight that is fair to
16:29:43 [jeanne]
COGA with individual guideline weighting.
16:29:50 [Chuck]
Shawn: Logistically, I'd like to get through the queue and then get to the last agenda item.
16:29:59 [Lauriat]
ack alastairc
16:29:59 [Zakim]
alastairc, you wanted to say that how we measure things is more important than how we weight them
16:30:01 [alastairc]
ack me
16:30:04 [Chuck]
Shawn: This discussion has been great in taking in all thing things we need to consider.
16:30:16 [Detlev]
@CharlesHall: "challenge is where 2 or more functional needs have a critical item that is in conflict" - can you speak to that? I'd love to know where solving one showstopper issue creates a real problem for another group...
16:30:49 [Chuck]
Alastairc: Not sure weighting is necessarily going to be the answer. A lot of the problems in terms of coverage in wcag 2.x is how things are scoped and measured. Been clear with addressing coga issues.
16:31:24 [Chuck]
Alastairc: If there is a functional outcome that has an equal weighting per guideline, that may be a reasonable way to proceed. It's within the guideline to decide what content authors need to do.
16:31:55 [Lauriat]
ack PeterKorn
16:32:03 [Chuck]
Alastairc: Let's start off even, and once we have better coverage of guidelines. Maybe later we can weight the guidelines when we have more. Let's start with "what is a reasonable thing to ask content authors to do".
16:32:30 [Chuck]
PK: Friction comes to mind. Every place where the language is harder to puzzle out is friction. The accumulation of friction can take a site from great to struggling to impossible. The spoons model.
16:33:11 [Chuck]
pk: I think the same concept applies to other disabilities. What we thought of traditionally as pass/fail. The header example, a page with 2 headers is a little bit of friction, a formal failure. Does it prevent blind individuals from using the page? Probably not.
16:33:21 [alastairc]
+1 to thinking about friction, which again could be dealt with quite granularity with good measurement, where low scores across the board start adding up (or rather, not adding up!)
16:33:46 [Chuck]
PK: Mel Brooks silent movie has text in it, except for one individual who has one word. Is it a problem for a deaf person who wants to watch that movie? I think we can look at a friction based model.
16:34:04 [JF]
+1 to Peter's point - barriers are based on the functional requirements of the different disability types
16:34:31 [Chuck]
pk: ... come up with a notion that there is a little bit of friction for blind folks because some small pages don't have headings right, there's a lot more friction for a cognative user. We can elevate a site and say that can be no worse than "good"...
16:35:19 [JF]
+1 to contemplating A,B,C,F scoring
16:35:20 [PeterKorn]
q-
16:35:21 [Chuck]
pk: therefore a site does or doesn't make it. Back to numbers. A+, B-, I think there are mechanisms that can be fairly granular, like we are getting a C- and we want to get a C+. We can get caught up in a scale of 100% and we are arguing if 86 or 87 is good enough.
16:35:24 [Lauriat]
ack jeanne
16:35:24 [Zakim]
jeanne, you wanted to say to Detlev when each group gets a show stopper, then a bonus is given to that group. But for COGA, its the overall sum of all the guidelines. So they
16:35:27 [Zakim]
... always get the lower weight on each indiviudal guideline. COGA doesn't get "absolute blockers", so they never get a higher weighted score. But their overall website is an
16:35:27 [Zakim]
... "absolute blocker". So we can't get an weight that is fair to
16:35:55 [david-macdonald]
+1 Peter had a great concept of "friction"... at some point there is too much friction to use.
16:36:05 [Chuck]
Jeanne: To detlev to weighting, when each group gets a show stopper, then a bonus is given to that group. For coga it's the overall.
16:36:28 [Detlev]
q+
16:36:41 [Chuck]
Jeanne: Each group gets more points for show stoppers, and coga gets less. Physical or sensory issues have more points. But when coga looks at the overall score... like a website gets a 93, but for coga it's not accessible.
16:37:18 [Chuck]
Jeanne: For coga it's more of an overall issue. That's why I say that people need to test their proposals across... on real websites. We found that it wasn't reflecting the issues for coga users correctly.
16:37:25 [jon_avila]
q?
16:37:33 [alastairc]
Jeanne - couldn't that be addressed by having plenty of guidelines for COGA (based on things we can't fit in WCAG 2.x), so that without meeting enough of those, you wouldn't pass?
16:37:39 [CharlesHall]
q+
16:37:40 [Detlev]
Can I answer to that directly (briefly)?
16:37:58 [Chuck]
Jeanne: It's great to have these proposals. John's proposal of putting the impact at the guideline level and we trickle that down, that can work. But if we put weighting in, I'll ask for real examples with real websites and show it's fair.
16:38:29 [Chuck]
Jeanne: I think we can do it without weighting and go back to an author and say "here's total score, here is how it breaks down by disability", I think that can work. I think we can avoid the weighting issue.
16:39:06 [Chuck]
detlev: I think you are mixing 2 different things. Nobody argues that wcag doesn't have enough for coga. Coga issues are underrepresented. I don't see a real conflict for critical issues by groups.
16:39:44 [Chuck]
detlev: cognative folk that are impacted by many different things combined, if the new guidelines and rubriks includes them, I would rather think of subtracting points. Cognative score would show the deficiencies clearly.
16:40:00 [Chuck]
detlev: You'd still have a way of showing off critical issues by groups.
16:40:04 [jeanne]
I would be interested to see a proposal with real data. I would help with testing.
16:40:41 [Chuck]
detlev: There are absolute show stopper for some users, and we need to show that. Jeanne you said that these can drown out coga issues, but I think that it can be reflected properly and benefit coga users.
16:40:41 [Lauriat]
ack AndyS
16:40:46 [Lauriat]
ack Detlev
16:41:20 [Chuck]
Andy: The cognative issue is such a complex subject. It overlaps with neurological, senses, perceptions. There are so many different varieties (A.D.D) will be different from educational handicaps.
16:41:47 [jeanne]
alastair, that is possible -- again, I would like to see a proposal with mockup of the data with proposed guidelines.
16:41:54 [Chuck]
Andy: Becomes this big mess of how we divide up, should we divide up... becomes a broad spectrum. Points towards the ability to customize and personalize as the way to ensure that all groups have equal access.
16:42:20 [alastairc]
Jeanne - I think we need to leave weighting/criticality until we have better coverage of guidelines.
16:42:25 [Chuck]
Andy: In terms of a triangle, the user is one point, the author is another point, and the technology is the 3rd point. Those 3 points need to work together in a way for every user to be accommodated.
16:42:57 [alastairc]
we aren't sure what / how-many new guidelines are likely to come from COGA.
16:43:03 [Chuck]
andy: I don't think there's a way for a website to address everyone all the time. I understand what Jeanne is saying in terms of weighting. Weight vision perception gets the site up to a high level of acceptance, but the things that made that...
16:43:25 [Lauriat]
+1 to Alastair
16:43:39 [Chuck]
andy: site a vision number may actually hurt coga issues. high contrast can cause coga issues. There's a lot of interaction there that can... how do we really make that into a matrix where you push this up and the other comes down.
16:44:11 [Chuck]
andy: Squeezing a toothpaste tube, one end decreases, the other gets larger. I don't know that weighting is the way of solving. I think customization is the way to achieve the ultimate goal. These are thoughts in my mind.
16:44:12 [kirkwood]
+1 to customization
16:44:41 [Lauriat]
q?
16:44:47 [Lauriat]
ack Lucy
16:44:50 [Chuck]
Shawn: We are unlikely to come to complete resolution to this conversation, we need to finish queue and move on.
16:44:59 [JF]
FWIW, the Personalization TF is looking at 'customization" today - but we lack the technology to make it happen today
16:45:22 [Chuck]
Lucy: If any disability is poorly affected by any system we come up with, that's our failing. We don't have the data and don't know, and if it's still failing cognative, that's our responsibility to address that group.
16:45:33 [jeanne]
Andy, that's why I like that proposal of putting data into each guideline about each disability effected, but then giving a total score for each disability.
16:45:35 [david-macdonald]
q+
16:45:42 [Chuck]
Lucy: We have fixated for so long on what we know, we still need to do the research and determine what we don't know.
16:45:46 [jeanne]
zakim, close queue
16:45:46 [Zakim]
ok, jeanne, the speaker queue is closed
16:45:56 [Chuck]
lucy: I'll say that I don't know enough, but I do take it into effect, and I want to know more.
16:45:57 [Lauriat]
ack CharlesHall
16:46:22 [jeanne]
q- david
16:46:43 [Chuck]
Charles: Historic: we are all in general agreement that this is complex, which is why it takes a long time to get through one point of conversation, but I don't think it's impossible to account for something that is critical to one functional need in order to achieve a score.
16:46:46 [Lauriat]
@JF: I'd like to know more about the technology needed to make that happen, or at least prototype things out to figure out how to make that happen.
16:47:25 [Chuck]
Charles: If we have 9 functional need categories and one has an issue, then they all do. Historic conversations, where there's a conflict. Where there is a challenge is where something in one category that is in conflict with another issue in another category...
16:47:55 [Chuck]
Charles: headings large and in one color may conflict with someone where it could trigger anxiety. It's fine to consider this, but we need to be aware of the conflicts.
16:48:19 [jeanne]
zakim, take up next
16:48:19 [Zakim]
agendum 3. "Should we use IETF standard RFC 2119" taken up [from jeanne]
16:48:23 [Chuck]
Shawn: With that, we have a lot to think about and work through. Let's move on to whether to use rfc 2119. Must/should/must not/should not
16:48:35 [Detlev]
@Charleshall: I think research shows ALL CAPS is harder to read, cannot think who would benefit
16:49:05 [Chuck]
Jeanne: W3C and standards organization around the world rely on this particular RFC (request for comments). Very old, used in technical standards by many standards orgs.
16:49:38 [Chuck]
Jeanne: W3C in the past has said they don't recommend it for guideline use, and not used in WCAG. Designed for technical specs that require interoperability. More about... John may have lots of examples.
16:49:43 [sajkaj]
Think APIs
16:49:59 [Chuck]
Jeanne: Because it's not included, we are interested in including it in Silver. This is John's proposal. John...
16:50:20 [CharlesHall]
@Detlev. i agree. was trying to create an example on the fly since I couldn’t recall some of the specific examples we discussed in the past. particularly with insights from Cybelle.
16:50:38 [Chuck]
JF: The part of the issue is that wcag has moved from being a guideline to being a standard. That's the reality. We have govts that say "you must meet wcag 2.x AA". because of that, we need to have these bright and measurable points
16:51:02 [jon_avila]
The ARIA specification is an example that uses RFC2119
16:51:07 [Chuck]
JF: To meet that requirement. When we talk about the users, the 3 points, there's a 4th point... legal requirements. RFC 2119 calls out must should and may.
16:51:30 [Chuck]
JF: It's unambiguous and clear. Failing means you didn't meet the requirement. It's about having clearly defined requirements with explicit language.
16:51:34 [alastairc]
q+ to say that must = the requirement, and should/may wouldn't be suitable for stating the requirement.
16:51:35 [jeanne]
https://tools.ietf.org/html/rfc2119
16:51:42 [david-macdonald]
q+
16:51:56 [PeterKorn]
q+
16:52:00 [Chuck]
JF: I want to use it, when we created a guideline, that was guidance. Because we are at a point where we have measurement and scoring... as part of that requirement we should use very clear language.
16:52:15 [CharlesHall]
+1 to JF on use of an unambiguous standard
16:52:16 [Chuck]
JF: We've got clarity there.
16:52:25 [Lauriat]
ack alastairc
16:52:25 [Zakim]
alastairc, you wanted to say that must = the requirement, and should/may wouldn't be suitable for stating the requirement.
16:52:49 [Chuck]
Alastairc: I think as it stands now... silver structure... normative and informative... anything that is a normative requirement is a must. A should or may would not be normative requirement.
16:53:20 [Chuck]
Alastairc: They are very tied together. We would need to be very careful using that language in informative documents. We've had charter issues, the language has strayed into informative documents.
16:53:39 [Lauriat]
ack david-macdonald
16:53:42 [Chuck]
Alastairc: introduces unnecessary uses. I don't disagree or agree, I think our current approach takes it into account already.
16:53:53 [JF]
Q+ to note that there is a real difference (in RFC 2119) between MUST and must (case sensitive)
16:54:04 [Chuck]
DM: The sc were designed to be testable statements. If the statement is true, you meet the criteria. Every one of those statements is a must.
16:54:10 [jon_avila]
I find it ironic that folks have said no disability should be weighted over others -- yet it was said cognitive disabilities are impacted by the wholistic site and thus trump everything in terms of importance
16:54:15 [AndyS]
q+
16:54:19 [Chuck]
DM: We do have must statements, we don't have shoulds in the normative docs.
16:54:26 [Lauriat]
ack PeterKorn
16:55:13 [Chuck]
PK: I see value and intellectual rigor and honestly in being explicit. We call these guidelines, they became standards. The dirty secret is that no site can be perfect. If we take on this kind of language we need to be clear that we aren't going to say "must"
16:55:30 [alastairc]
jon_avila I think there is just recognition that there as been a gap, and there needs to be work to address the gap.
16:55:35 [PeterKorn]
q-
16:55:38 [Chuck]
PK: when the must is not achievable. I don't have strong feeling for or against. But if we use this language, we need to be clear that we are creating an impossible "must".
16:55:41 [Lauriat]
ack JF
16:55:41 [Zakim]
JF, you wanted to note that there is a real difference (in RFC 2119) between MUST and must (case sensitive)
16:55:55 [jon_avila]
For what it's worth - WCAG 2.0 and 2.1 are standards according to WCAG itself - "The WCAG 2.0 document is designed to meet the needs of those who need a stable, referenceable technical standard."
16:56:03 [Chuck]
JF: Alastair... concern about language. Must should and may are always in upper case. MUST and must don't equal the same thing.
16:56:11 [alastairc]
let's not go down that route!
16:56:15 [Chuck]
JF: "must" is conversational. That avoids some of that problem.
16:56:23 [AndyS]
Thats scary
16:56:51 [Chuck]
JF: As peter noted, our guidelines have been made standards. In terms of scoring, engineers need to have black and white decisions. Everything we do is based on binary decisions. In the must we have declared what that bright line is.
16:56:53 [JF]
Q+ Lucy
16:56:57 [sajkaj]
Methinks JF forgot about analog engineering!
16:56:58 [Lauriat]
ack Lucy
16:57:09 [david-macdonald]
There are no "Must", "should" or
16:57:16 [Jennie]
I would be concerned with upper case and lower case differences in meaning, from a cognitive standpoint.
16:57:18 [ChrisLoiselle]
Chris Loiselle comment on standard: https://www.iso.org/standard/58625.html , point to standard
16:57:19 [Chuck]
Andy: I want to mention, all the other standards organizations, they use this verbage.
16:57:19 [david-macdonald]
"may" in WCAG 2 or 2.1
16:57:37 [PeterKorn]
Regrets - I need to leave a few minutes early for my next meeting.
16:57:39 [JF]
+1 to bright lines
16:58:02 [JF]
bright lines make measuring easier
16:58:03 [Chuck]
Andy: But if we were going to go there, it's important to note that this is a very bright line. Requires additional diligence to make sure that "shall" is really understood and isn't going to create situations that cannot be absolutely achieved.
16:58:21 [jeanne]
I worry that accessibility needs are not oriented toward bright lines.
16:58:41 [Chuck]
Andy: With all of the many things we are talking about that interact with each other. WCAG has them broken down in different elements. These elements interact with eachother. Bring Coga into the mix adds a layer of complexity and conflicts.
16:58:53 [david-macdonald]
There are no instances "Must", "should", shall, or "may" in WCAG 2 or 2.1
16:58:58 [Chuck]
Andy: If one "shall" conflicts with another "shall", we'll get into trouble.
16:59:02 [jeanne]
+1 Andy
16:59:05 [jon_avila]
I agree that use of these RFC 2119 terms will only complicate things
16:59:33 [Chuck]
Andy: A bit more ambiguous from other standards from other groups. ANSI specs on displays and fonts, their language and examples are set in technology of the late 80's and early 90's.
17:00:15 [KimD]
I'm concerned about internalization and not comfortable saying we need to put ourselves in the shoes of legislators.
17:00:21 [Chuck]
Andy: We start to get into ambiguous realm when we discuss different browsers render fonts differently. I like the idea of adopting this more affirmative use of terminology, but brings a great deal of complication.
17:00:38 [jeanne]
zakim, close queue
17:00:38 [Zakim]
ok, jeanne, the speaker queue is closed
17:00:50 [alastairc]
Is it worth trying this language out in a method? That seems to be the most suitable place.
17:01:12 [Chuck]
Lucy: I want to see it applied and see how it works, and then when John responded... I say what Peter said... this is not a possible thing to accomplish and remain accessible itself.
17:01:25 [Lauriat]
@Alastair: No, that would make tech-specific methods normative.
17:01:31 [Chuck]
Lucy: I like the idea, in the terms of what we have been thinking of all along, I want to see it apply to some examples first.
17:01:32 [alastairc]
Um, I'm not sure it will help with the clear language.
17:01:44 [Chuck]
Lucy: I can't tell the difference between MUST and "must".
17:01:57 [jon_avila]
There are settings in screen readers to communicate capitalization of text.
17:01:57 [JF]
<span aria-label="RFC 2119 MUST">MUST</span>
17:02:06 [Chuck]
Shawn: My proposal is to go through the minutes and pull out the pros and cons of going with this language and keeping the current language.
17:02:22 [Chuck]
Shawn: And then we can use that as a summarization for folk who couldn't make it to this call.
17:02:30 [alastairc]
Should a guideline include should/may?
17:02:32 [Chuck]
JF: I pasted some code in RFC to address your concerns.
17:02:33 [Jennie]
Won't the ARIA label only assist those using screen readers, but not those with reading challenges with vision?
17:02:43 [KimD]
+1 to Jennie
17:02:43 [david-macdonald]
There are no instances "Must", "should", shall, or "may" in WCAG 2 or 2.1 success criteria
17:02:57 [Chuck]
JF: <discusses rfc-2119 must>
17:03:05 [Chuck]
Shawn: worth looking into annotations.
17:03:06 [jon_avila]
ARIA labels on non-interactive text doesn't work well with screen readers.
17:03:10 [jeanne]
+1 Jennie
17:03:14 [alastairc]
JF - would a guideline include should or may?
17:03:22 [Chuck]
Shawn: With that, thank you everyone, and bringing examples. Super helpful as a part of these complex topics.
17:03:30 [Chuck]
Shawn: Anything else Jeanne?
17:03:34 [JF]
@ Alastair - it could
17:03:44 [Chuck]
Jeanne: Incredibly helpful, great conversations, we'll keep working.
17:04:05 [laura]
Thanks. Bye.
17:04:17 [Detlev]
pesent+
17:04:20 [jeanne]
rrsagent, make minutes
17:04:20 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/03/10-silver-minutes.html jeanne
17:04:22 [Detlev]
present+
17:04:47 [jeanne]
rrsagent, make minutes
17:04:47 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/03/10-silver-minutes.html jeanne
17:05:04 [jeanne]
s/cognative/cognitive
17:05:12 [jeanne]
rrsagent, make minutes
17:05:12 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/03/10-silver-minutes.html jeanne
17:17:39 [KimD]
KimD has left #silver
17:38:51 [Ralph]
Ralph has left #silver
20:39:39 [RRSAgent]
RRSAgent has joined #silver
20:39:39 [RRSAgent]
logging to https://www.w3.org/2020/03/10-silver-irc
22:17:04 [AndyS]
AndyS has joined #silver
22:20:11 [Fazio]
Fazio has joined #silver
22:26:02 [jeanne]
agenda?
22:26:07 [jeanne]
close item 3
22:26:11 [jeanne]
q?
22:26:15 [jeanne]
ack Andy
22:26:25 [jeanne]
zakim, close item 3
22:26:25 [Zakim]
agendum 3, Should we use IETF standard RFC 2119, closed
22:26:26 [Zakim]
I see nothing remaining on the agenda
22:27:15 [jeanne]
agenda+ ACT Tests
22:27:15 [jeanne]
agenda+ Setting up a framework for task completion testing
22:27:15 [jeanne]
agenda+ What feedback would Silver like to ask for from publishing the Challenges document?
22:49:06 [mattg]
mattg has joined #silver
22:58:48 [JF]
JF has joined #silver
22:58:55 [JF]
agenda?
22:59:05 [JF]
Present+
23:00:10 [Lauriat]
Present+
23:00:13 [AndyS]
present+
23:00:20 [Fazio]
present+
23:00:25 [Chuck]
Chuck has joined #silver
23:00:26 [Chuck]
present+
23:00:35 [KimD]
KimD has joined #silver
23:00:52 [KimD]
Present+
23:02:04 [sajkaj]
scribe: sajkaj
23:02:57 [kirkwood]
I can
23:03:29 [mattg]
present+
23:03:36 [jeanne]
present+
23:03:48 [jeanne]
zakim, take up next
23:03:48 [Zakim]
agendum 4. "ACT Tests" taken up [from jeanne]
23:03:49 [sajkaj]
present+
23:04:24 [sajkaj]
sl: How to structure our ask of ACT
23:04:56 [sajkaj]
sl: What from the existing ACT work can we bring over to 3.0?
23:05:14 [sajkaj]
sl: What can we expand upon?
23:05:24 [JF]
Q+
23:05:40 [sajkaj]
sl: ACT has an applicability concept, as do we in Silver.
23:05:40 [Jennie_Delisi]
Jennie_Delisi has joined #silver
23:06:04 [Jennie_Delisi]
present+
23:06:20 [sajkaj]
sl: Understand that it's highly specific on a per rule basis and described for how it applies
23:06:33 [david-macdonald]
david-macdonald has joined #silver
23:06:52 [sajkaj]
sl: We will certainly need to facilitate people tunneling into guidance that pertains
23:07:05 [jcraig]
present+
23:07:06 [sajkaj]
sl: It should also facilitate us exposing tech gaps
23:07:11 [CharlesHall]
CharlesHall has joined #silver
23:07:18 [sajkaj]
sl: Restating page for apps, vr, etc
23:07:20 [CharlesHall]
present+
23:08:02 [kirkwood]
present+
23:08:04 [sajkaj]
sl: A gap might be if platform didn't support lang definition;
23:08:28 [sajkaj]
sl: Expect we have much to teach each other
23:08:41 [jeanne]
ACT Rules Format: https://www.w3.org/TR/act-rules-format/
23:08:41 [Lauriat]
ack JF
23:09:04 [sajkaj]
jf: ACT rules format might template some of our other testing
23:09:21 [sajkaj]
jf: Believe it would even apply to a cognitive walkthrough
23:09:42 [sajkaj]
jf: Because we're asking a series of yes/no questions in a walkthrough
23:09:45 [sajkaj]
sl: Agrees
23:09:57 [JennC]
JennC has joined #silver
23:10:18 [sajkaj]
sl: Some rules are automatable and others aren't, but structure supports repeatability
23:10:30 [sajkaj]
jf: Critical
23:10:54 [sajkaj]
sl: They've done lots of work since we last talked, we need to know
23:10:57 [jeanne]
q+ to say that some tests will be yes/no, but not to require all tests to be yes/no.
23:11:05 [sajkaj]
sl: Next is to learn what we can reuse and map that out
23:11:12 [Lauriat]
ack jeanne
23:11:12 [Zakim]
jeanne, you wanted to say that some tests will be yes/no, but not to require all tests to be yes/no.
23:11:36 [sajkaj]
js: We should have yes/no wherever appropriate, but not all our tests will be binary
23:12:03 [CharlesHall]
act rule design: https://act-rules.github.io/pages/design/rule-design/
23:12:10 [sajkaj]
sl: Don't believe ACT inherently requires binary; believe it guides through making judgement calls
23:12:29 [sajkaj]
sl: Ex: alt for whether it provides equivalent experience
23:12:48 [sajkaj]
sl: How well, how close to equivalent might be scaled
23:12:51 [david-macdonald_]
david-macdonald_ has joined #silver
23:12:52 [sajkaj]
jf +1
23:13:09 [sajkaj]
jf: Even complex questions can be broken down to simple binary
23:13:22 [sajkaj]
jf: So, it's how we structure and string together
23:13:33 [jeanne]
Not all comple questions can be broken down to simple binary and get valid results
23:13:46 [jeanne]
s/Not all comple /Not all complex
23:13:52 [JF]
CAn you provide an example Jeanne?
23:14:22 [sajkaj]
sl: Believe we can reuse both very granular but also more in depth exploration toward test results
23:14:52 [jeanne]
@JF, Simplified navigation comes immediately to mind
23:15:00 [CharlesHall]
+1 to using the rule design
23:15:47 [sajkaj]
lucy: Are we stopped because we have no ACT people on the call?
23:15:57 [sajkaj]
sl; If you're in ACT, please speak up
23:16:18 [sajkaj]
lucy: Notes Mary Jo has been regular participant in Silver recently
23:16:53 [sajkaj]
lucy: Walking through their questions without their assistance has been a profitable exercise for me
23:17:11 [sajkaj]
sl: We have an example, but it's quite old
23:17:27 [sajkaj]
js: Let's schedule a joint call with them.
23:18:09 [sajkaj]
js: In the past we were stopped in our joint attempts because we had no content, we do now
23:18:14 [Chuck]
q+ point of order that need not be scribed.
23:18:33 [sajkaj]
sl: Probably schedule a series of calls
23:18:59 [CharlesHall]
q+
23:19:00 [david-macdonald_]
q+
23:19:09 [Lauriat]
ack CharlesHall
23:19:38 [sajkaj]
ch: Would be worth scheduling to have someone familiar with EM reporting tool
23:19:48 [sajkaj]
js: Know whom to ask
23:19:55 [Lauriat]
ack david-macdonald_
23:20:28 [sajkaj]
dm: Working with aCT on Canadian Gov study ...
23:20:40 [sajkaj]
dm: I can talk some about that ...
23:20:53 [sajkaj]
dm: There are about 60 rules now applying to 2.x SC
23:21:02 [sajkaj]
dm: about 10 test cases, very granular
23:21:50 [sajkaj]
dm: Applicability is whether the test case is applicable to the rule
23:22:08 [Rachael]
present+
23:22:57 [sajkaj]
dm: Ex test would be about filename
23:23:27 [sajkaj]
dm: then what rule is applicable, expected outcome, whether it passes
23:23:44 [sajkaj]
dm: They are trying to build automatable tests
23:23:55 [JennC]
q+
23:24:07 [sajkaj]
dm: Mostly Wilco adding tests
23:24:11 [Lauriat]
ack JennC
23:24:24 [sajkaj]
zakim, who's here?
23:24:24 [Zakim]
Present: jeanne, sajkaj, ChrisLoiselle, Laura, Jennie, kirkwood, Lauriat, Lucy, alastairc, Makoto, Chuck, JF, stevelee, KimD, AndyS, PeterKorn, mattg, Rachael, Detlev, Fazio,
23:24:28 [Zakim]
... Jennie_Delisi, jcraig, CharlesHall
23:24:28 [Zakim]
On IRC I see david-macdonald_, JennC, CharlesHall, Jennie_Delisi, KimD, Chuck, JF, mattg, Fazio, AndyS, RRSAgent, maryjom__, Zakim, jon_avila, jeanne, Lauriat, kirkwood, jcraig,
23:24:28 [Zakim]
... sajkaj, MarcJohlic, MichaelC, Rachael, achraf, alastairc, yatil, AWK, trackbot
23:25:06 [JennC]
https://act-rules.github.io/rules/
23:25:10 [sajkaj]
jc: Have a pointer to this
23:25:27 [david-macdonald_]
https://act-rules.github.io/testcases.json
23:27:01 [sajkaj]
sl: re alt-text == filename -- what's the logic?
23:27:15 [sajkaj]
dm: If you start on JC's page ...
23:27:50 [david-macdonald_]
https://act-rules.github.io/rules/
23:28:13 [sajkaj]
dm: Contains the rules -- about 60
23:28:20 [sajkaj]
dm: Click on context
23:28:22 [Lauriat]
https://act-rules.github.io/rules/ff89c9
23:28:47 [Chuck]
q+
23:28:52 [sajkaj]
dm: Will get dail incl linkage
23:29:05 [sajkaj]
lucy: Are they filterable by SC?
23:29:07 [sajkaj]
dm: No
23:29:13 [sajkaj]
dm: They're wcag agnostic
23:29:28 [sajkaj]
dm: They borrow from multiple standards
23:30:13 [sajkaj]
js: Can you talk us through Image has Accessible-Name?
23:30:29 [jeanne]
https://act-rules.github.io/rules/23a2a8
23:30:57 [Chuck]
q?
23:31:30 [jeanne]
ack chuck
23:31:42 [sajkaj]
ch: These seem ml specific?
23:31:45 [sajkaj]
dm: For now
23:31:49 [JF]
+1 yes
23:32:12 [sajkaj]
dm: Effort is to get all the automated tools gathered on one page
23:32:16 [JennC]
+1 yes for the purpose: harmonisation
23:33:09 [sajkaj]
dm: Responding to trying to get consistent rendering despite browser's tendency to render differently
23:33:37 [sajkaj]
sj: Quips it's also very today, not just the 1990's
23:34:06 [Lauriat]
Applicability: The rule applies to HTML img elements or any HTML element with the semantic role of img that is included in the accessibility tree.
23:34:38 [Chuck]
q+ lucy
23:35:06 [Lauriat]
ack lucy
23:35:33 [sajkaj]
lucy: Do I understand we're asking whether their struct is applicable to our needs? Not necessarily the actual rules?
23:35:47 [sajkaj]
lucy: Believe we want their structuring
23:35:50 [sajkaj]
sl: Agreed
23:36:11 [sajkaj]
sl: Expect we will be able to reuse their rule content
23:36:33 [CharlesHall]
+1 to use of rule design
23:36:46 [sajkaj]
sl: Hope is that we'll be able to reuse
23:37:05 [PeterKorn]
PeterKorn has joined #silver
23:37:08 [PeterKorn]
present+
23:37:28 [jeanne]
WCAG-EM
23:37:29 [CharlesHall]
WCAG-EM Report Tool
23:37:35 [david-macdonald_]
Website Accessibility Evaluation Report Generator
23:37:52 [david-macdonald_]
https://www.w3.org/WAI/eval/report-tool/#!/#%2Fopen
23:38:33 [sajkaj]
dm: Walks through the form ... ...
23:38:48 [sajkaj]
dm: Notes it's highly 2.x based
23:39:39 [sajkaj]
dm: It helps you gather up structured samples and also random samples
23:39:55 [sajkaj]
dm: At the end a report
23:40:01 [jeanne]
This is a tool for the spec that we based representative sampling on.
23:40:20 [sajkaj]
sl: Walks one through generating a report, but not through the actual tests/
23:40:22 [sajkaj]
dm: yes
23:40:47 [sajkaj]
js: It's a tool for the spec that we based Silver conformance on
23:41:12 [sajkaj]
js: That's why it should sound familiar! We've been talking about the sampling part
23:42:38 [sajkaj]
sl: Shifting to how to create a framework for defining a task and scoring task completion
23:43:01 [sajkaj]
sl: Perhaps we might discuss how we might do it, what it means to get this task defined
23:43:19 [sajkaj]
sl: Maybe there are better terms, but this is our best to date
23:43:39 [sajkaj]
sl: We want overall test results better to reflect pwd experience based on what they're trying to do
23:44:07 [sajkaj]
sl: A tiny image in the corner that does nothing and has no alt text is not really an impediment
23:44:23 [sajkaj]
sl: On the other hand the image that is a buttn and brings up the menu is a big deal without alt
23:44:25 [CharlesHall]
useful resource on how to identify top tasks (analysis) https://alistapart.com/article/what-really-matters-focusing-on-top-tasks/
23:45:01 [sajkaj]
sl: Using we've been saying "tasks" is to cover tech other than just web
23:45:21 [sajkaj]
sl: Pizza shop: find how to call; hours; menu; etc
23:45:40 [sajkaj]
sl: For Google Docs there are tasks about formatting, sharing, etc
23:45:53 [PeterKorn]
q+
23:45:57 [Chuck]
q+
23:46:11 [sajkaj]
sl: How do we guide people to defining their tasks. How do we help NY Times do this?
23:46:19 [Lauriat]
ack PeterKorn
23:47:08 [Fazio]
Likert scale
23:47:10 [PeterKorn]
q-
23:47:15 [sajkaj]
pk: Jumps back to unlabeled icon out of the task flow -- want to make sure we agenda defining what pieces matter, and which don't; should help us realign with clear lang, COGA, etc
23:47:49 [sajkaj]
sl: Agree, but let's hold off for now until after the definition. But, yes, it also applies to defining scope for conformance and may depend on a particular path through the page, for instance
23:48:07 [Lauriat]
ack Chuck
23:48:10 [sajkaj]
sl: there may be parts of the page that interfere--things in the environment that arn't directly related
23:48:18 [Lauriat]
q+ Lucy
23:48:29 [sajkaj]
ch: Should we base on standard development life cycle?
23:48:32 [Fazio]
That's the first question we ask UX research participants: "How would you expect to perform this task"
23:48:50 [sajkaj]
ch: Would it pertain to sw dev?
23:48:58 [Fazio]
Then we walk them through our intended process flow
23:49:40 [Lauriat]
ack Lucy
23:49:44 [sajkaj]
s/ch/ca/
23:49:48 [sajkaj]
ch: maybe
23:50:06 [sajkaj]
lucy: we let the owner define how to build the task; we only care about functional outcome
23:50:33 [sajkaj]
lucy: believe our job is to define success
23:50:51 [sajkaj]
sl: But we need to define what we need in order to explain what we're asking people to do
23:51:12 [sajkaj]
sl: Person running the task is perhaps not the person who defined the task
23:51:20 [sajkaj]
sl: One may need to explain to a colleague
23:51:53 [sajkaj]
lucy: OK, just don't want us to get bogged down by what should be accomplished by one visit to http://123.456
23:52:22 [sajkaj]
sl: Need to define a list of what the user would be doing, in order to define conformance
23:52:36 [sajkaj]
lucy: prefer actions and/or activities to the term task
23:53:10 [sajkaj]
ch: Think I understand the concern incl with the vocabulary; but there's history here
23:53:30 [sajkaj]
ch: Author defines scope -- a better term is fine -- but it's a common term in the field
23:53:38 [jeanne]
q+ to clarify scope
23:54:29 [Fazio]
q+
23:54:57 [sajkaj]
sl: We need a way to explain what blocks a user doing some thing on a page -- we need to be able to explain this thing here is a big problem, but this over here is annoying and time consuming, but not a showstopper
23:55:15 [sajkaj]
sl: Really don't like to go more granular because that takes us back to element based
23:55:37 [sajkaj]
sl: e.g. there may be three ways to accomplish a particular task, which may require different action paths
23:57:01 [Jennie_Delisi]
UXPA Usability Body of Knowledge uses high-level tasks, broken down to subtasks - referenced on usability.gov
23:57:06 [Lauriat]
ack jeanne
23:57:06 [Zakim]
jeanne, you wanted to clarify scope
23:57:06 [sajkaj]
ch: If I recall you can't start at an arbitrary point ...
23:57:28 [sajkaj]
js: We had considerable discussion about scoping
23:57:50 [sajkaj]
js: It has to be a logical subset, but can have various tasks, but it's not the scope itself
23:57:51 [Lauriat]
ack Fazio
23:58:32 [sajkaj]
df: Do a lot of tasks with user testing, when I work with orgs we work through the steps involved in achieving a particular outcome -- a task
23:59:04 [CharlesHall]
+1 human task(s) are about getting to the outcome
23:59:06 [sajkaj]
df: Having the author define this puts the onus on the author to identify their functional outcomes