I can scribe
janina: we got a little into the
weeds of known issues
... particularly scoring and it kind of subverted the
agenda
... it's pretty easy when questions start coming in to dive
into details and you're trying to flesh them out in real
time
... it was more positive than not and it seems the overall
approach has agreement
kim: there are three people on
our team and we all attended; the others thought it was a
little confusing
... I told them it was a little high level; once we got clarity
to look at the ideas, not the exact content, they were on
board
fazio: it's a good framework; a great starting point for where we're at
jeanne: great, thanks
jeanne: let's get back to work. I want a points system proposal soon. it's starting to feel really close
jeanne: things are starting to come together in a way that can work really strongly
<Fazio> btw is this modeled on VPAT?
jeanne: last week we were working on the points system proposal in the sampling document
<Fazio> it seems similar
jeanne: I wouldn't say it's modeled on VPAT, I wouldn't want to be far away from VPAT, but we're not taking it as a starting point
<Fazio> agreed
jeanne: we're looking similar to
VPAT by taking a similar route
... we're looking at the needs and the problems that need to be
solved; maybe they did something similar
janina: I think saying "taking a different route" is accurate
<Fazio> thanks
jeanne: we didn't get into the point system proposal last week right?
janina: we touched on it some
jeanne: it's additive and
percentage based
... you're going to need a minimum in each disability category;
I think we're all good with that...at least for a while
<KimD> starting with "There is no weighting for importance, severity, difficulty, or other reason."
jeanne: no weighting for any reason is the next bullet point
fazio: what do you mean "no weighting?"
jeanne: part of the reason people
with cognitive disabilities are poorly represented in WCAG is
in part because they're at the AAA leve
... and success criteria for other disabilities are at the A
level
... one of the factors for determining levels was whether they
were blocking issues for someone
... this ends up saying "people with physical disabilities have
a higher priority"
... because it could just be "harder, but doable" for someone
with a cognitive disability
fazio: it kind of demeans what it means to have a cognitive disability
jeanne: we've been working on
what we can do to get extra weight for the score, but none have
really worked
... I think we have the evidence to hold firm when we get any
push back on it
kimd: when you say "difficulty" do you mean for someone to get around an issue or for something to be made accessible?
jeanne: both, actually
... difficulty in implementing wasn't very consistent over
platforms. it might be easy in one, but difficulty in another.
a simple technological advance could change that
... difficulty also when building it the first time versus when
remediating an issue
janina: it will also be difficult selling the points system...and it's hard to assign any kind of points for plain language
fazio: there might be a way if you hook someone up to an EEG machine and measure some cognitive difficulty while someone is going through a task
jeanne: if you can say "this is how many active/passive verbs" you can come up with a percentage, which leads to the next bullet point
<KimD> "For automated tests, the total instances of the condition are divided by the total passes for that condition. "
thanks!
scribe: we'll have to develop
tests that do that kind of counting
... but certainly something that can be done by automated
testing
... and then we have the next one which is really hard
<KimD> Next bullet: "For advanced Silver tests, there will be a mathematical condition that will be a part of the test. For example, a rubric test would have the total points scored divided by the number of factors. (We need examples of other tests.) "
scribe: this came up in the call today; we were looking at the rubric for clear words
scribe: if you go to the test tab
we have three columns, substantially is 100%, partially is 50%,
and limited is 0%
... you can normalize it by dividing by the number of actions
we're testing for
... so you always end up with a score between 0 and 1 and will
be a percentage
... it makes each one equal since some things are important to
some and others to other groups
... it makes a difference on the individual user need
scores
... so there might be less "actions" for some types of
disabilities and more for others. by dividing it within each
category, it makes it equal
... there was no way of making it fair, but this way does!
janina: there is at least one
more W3C specification not related to accessibility;
authentication does something similar
... they weight a bunch of factors like that...can you really
trust something in a login attempt
... it's based on similar thinking
jeanne: the scoring would have to
be by Method...and using this method, it actually works by
Guideline. This was something I had gotten exciting about,
because we're also worried about folks gaming the sytem
... they couldn't score points with a lot of methods
... if it's by guideline
kimd: comment about language...some languages are passive, so we might need some disclaimer or generalization for language consideration
jeanne: thank you. it's in the
Google Doc, but I forgot to put it in the HTML version
... the beauty of this, is because it's in a method, people can
change it in a way appropriate for their language without
having to change the normative guideline since it's all in the
non-normative side
... you just change how you normalize it; if there's a language
that doesn't use passive voice...you're just dividing by one
less action so you're not being penalized or getting an unfair
advantage
... the next bullet point
<jeanne2> Once the minimum level is achieved in each category, all of the guidelines that are met to whatever percentage they are met increase the score.
<jeanne2> The total score is divided by the number of guidelines to give a final percentage.
jeanne: then all the guidelines
that are met, whichever percentage that are met increase the
score
... there are guidelines that apply in certain
circumstances..if a site doesn't have media, how do you count
those? do you just omit them
janina: if you're doing audio,
higher bitrates should score higher. and an ability to equalize
frequencies that are heard better...that's a whole way of
writing a guideline for a group that's not been thought of
yet
... but something like that will matter with real time
communications; we're looking at use cases we want support from
WebRTC for
jeanne: maybe we could use the model of visual contrast where we're giving a range of...where you can say "here's the minimum at this level, and this level, etc."
janina: something similar is likely to be quantifiable and defensible
jeanne: do we want to
say...getting back to that bullet points..do we want to say
"guidelines that apply"
... I think it's how we do the tests and how they're scored. we
have the flexibility to be situational
janina: having it be
non-normative makes it possible to be updated more
quickly
... and experience can improve the guidance
... more quickly than if it was normative
jeanne: and then the last bullet point...
<jeanne2> Bonus points to reach higher levels can be earned by task completion testing with people with disabilities or user research with people with disabilities. (Should we have this? On one hand, it could incentivize doing more substantive testing, on the other, it gives an unfair advantage to large organizations who have a larger budget for testing. For example, LFlegal.com should always be
<jeanne2> able to achieve Gold rating.)
jeanne: jeanne maybe we say that
gold level is 90% or higher and silver is lower, etc.
... we would need to test sites to know where we should draw
those lines for the levels
... we could include it like a guideline
fazio: I think the VPAT and WCAG documentation says something about usability testing
jeanne: I don't think I've seen it since it's a problem we're trying to solve
fazio: I saw it earlier today, I'll find it for you
jeanne: great!
... I propose we take this bullet point out for the moment
since I don't think it's working out how we thought it would a
while ago
... is there anyone that can work on turning this into a clear
explanation for the conformance section of the guidelines
AngelaAccessForAll: I can work on that
jeanne: do you have time this week?
AngelaAccessForAll: no, what's our timeline otherwise? is it this week or bust?
jeanne: if we can get it in this week, we can get comments from AGWG..can you do two weeks?
AngelaAccessForAll: definitely
jeanne: and I can help you with
it as well
... okay, what's left on the agenda now
... what things do we want to ask them about what we're
thinking for conformance...this includes sampling and the
points scoring proposal
janina: I'm not sure it's mature enough to ask questions right now. in general, for now maybe stick to things that are more mature for them
jeanne: maybe "does the sampling proposal work across a range of products?" like small businesses, very large, etc. are there things this doesn't work for
kimd: maybe some things that are in the weeds...like 3 categories in the rubric...is that sufficient? if we ask high level, it helps keep them focused on high level answers
jeanne: well for some guidelines, we might have more columns; but maybe something like "we can all agree on the perfect score and the 0 score. how do we figure out what the middle scores should be?"
fazio: that's where we need to do user research
janina: how do you make sure your ".5" score is consistent across different orgs
jeanne: this is a good place to stop. if you think of any questions, send them to me, I'll be able to get the survey out tomorrow
This is scribe.perl Revision: 1.154 of Date: 2018/09/25 16:35:56 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00) Present: jeanne KimD janina joecronin Makoto LuisG Fazio AngelaAccessForAll No ScribeNick specified. Guessing ScribeNick: LuisG Inferring Scribes: LuisG WARNING: No date found! Assuming today. (Hint: Specify the W3C IRC log URL, and the date will be determined from that.) Or specify the date like this: <dbooth> Date: 12 Sep 2002 People with action items: WARNING: IRC log location not specified! (You can ignore this warning if you do not want the generated minutes to contain a link to the original IRC log.)[End of scribe.perl diagnostic output]