<jeanne> Sampling proposal
<scribe> scribe: sajkaj
js: Notes a reorganized
wiki
... Now each subgroup has their individual wiki page
... Simplified each page to make finding things easier
<Lauriat> Thank you, Jeanne!
<bruce_bailey> https://www.w3.org/WAI/GL/task-forces/silver/wiki/Main_Page
js: Jake has updated his proposal following mtg feedback
ja: Started a conformance chart
with scoping and scaling but want to focus on headings for
now
... Considering how to note for whom we've looked at
accessibility in a final number
... Notes we have a flatter structure than older wcag
... Using jf's examples still
... biggest change -- Questions now based on core
principles
... 5 ver difficult; 1 very easy
... Notes these are based on the howto guideline
... So based on why and how of each guideline
... So the evaluation questions are directly based on howto
content
... Trying to get to reliable and consistent results despite
who's auditing
... reviews a table showing how to develop one's judgement
easy/hard
[ja walks through an example]
ja: notes how the calculus
works
... suggests possibility of another appropriate where minimums
are required
pk: Finding this interesting and
lots of valuable potential
... Asking whether this means far fewer top level items
ja: wasn't the aim in this
exercise
... We weanted to look at login and search tasks, so broke out
sub tasks from that
pk: maybe I'm guetting excited about the wrong thing?
ja: perhaps -- it might come out that way
pk: Feels much more understandable and workable especially for newbies
<Fazio> +1 to Peter
pk: suggests calling it structure
rather than headings ...
... Wondering how to also eval importance to the task
... how to bring in the importance of the specific subtask --
importance to the overall task
... Concerned that too much granularity will end up being way
too voluminous
ja: believe this is close to how
audits currently work
... Notes still working on overall scoping approach; this is
not that
... Notes the value of working through examples
... Recalls past challenge of wcag conformant pages that
weren't very usable
... Trying to avoid that here
jf: likes this; concerned
calculus gets a bit complex
... worried about supporting regulatory with this
... the usable with x functional reqs -- not seeing that
here
ja: By evaluating functional
needs; these the first step in functional outcome that end up
with responses to those
... notes this isn't yet complete
jf: wants to see regulatory needs more integrated into scoring
ja: notes the analysis here are contributory to making those kinds of assessments
<JF> +1 to the second exercise
ja: Trying to avoid making the evaluation too complex
df: likes how this is coming
along and incorporating core principles
... suggests we could analyze core principles for specific
functional needs
ja: yes
... re-emphasis the questions must be based on the how and
why
... if that isn't working out, it might actually mean our
howtos need more work
df: So we're going to create the questions/ not unknown third parties?
ja: yes, it's our job
df: agrees this has lots of potential
jc: asks whether automated results will feed into this?
ja: yes, but not part of the
current exercise
... There is more not showing here still in prep
... Definitely looking at automating all we can via ACT; even
extending those some
... Most ACT rules still require some manual
jc: So, how to keep people from gaming by fixing some kinds of issues and not others
ja: looking at threshold
requirements
... still working out how to build in required minimum
thresholds so that without certain things none of the rest
matters
... need to look at what those will be
js: like that the testing is
setup and driven by the functional outcomes
... also usability comeing out of the core principles
... believes this is a very adaptable approach
<Fazio> +1 to Jeanne
js: want to hear how well this might scale for large, dynamic, complex ...
pk: biggest concern so far is how to catch the truly important parts of a site/task and not spend too much time on technical violations that don't really impede the task
<JoeCronin> +1 to Peter
pk: not clear how that fits
js: Yes, probably because this is is down in the detail weeds; but agrees the overall is important and is determined by site owner, not agwg
<Fazio> Doesn't EM Conformance doc have some info on that?
pk: Yes, I looked ahead and saw
the benchmarking -- so just wanting to see more of how all that
flows through
... not a criticism, just a concern about where this needs to
go
ja: after this highly focussed scope we need to look at that big/overall task to look at how it all fits together
pk: Second concern -- sheer volume of detail that could explode to become too voluminous
js: agree
jc: suggesting it might be helpful to have non experts try to walk through these exercies
ja: I have those resources actually
pk: believe this approach does help newbies
ja: appreciate that because i've been working toward that kind of presentatin
pk: wonders whether another
yardstick might be the web aim one million -- how does this
help the world who don't understand a11y understand what needs
attention better
... think we're out to broadly make the web more
accessible
... so would be interesting to assess whether we're helping non
expert developers/authors o a better job
s: agrees
mc: notes his hci schooling has students do these things and offers to put students on this approach to see how it works out
pk: agrees, but hard to know until we put it all together
bb: finds this fantastic
... believe this is a great implementation of conformance; but
concerned this shouldn't be the bare minimum
... was hoping verbal scoring would be more approachable
... likes the idea of seeing how well students do
mc: notes a11y integration into their hci curriculum; so would expects important info from running this with them
bb: you would give this to your students?
mc: yes
... could probably recruit over the summer
js: haven't thought about giving
people these spread sheets per se, it's for us so we can start
to see it all fitting together
... what we give real people would be simpler
<Jan> +1 to Jeanne ... it will be simplified
<Jan> It has to get complex while you're working through the detail and then you can simplify it
js: wonders whether we should try the simplification for ordinary users fairly soon? would that make sense
<Jan> I would be interested in working on this.
ja: discusses how this works with large teams, each of whom may not become all that expert in a11y
<PeterKorn> Apologies - I need to drop.
ja: if we take into account all
functional needs -- it will never be easy to be an wcag 3
expert either -- but we might walk nonexperts through the
process
... notes this all goes back to the howto doc
<bruce_bailey> Basic idea from WCAG 2x: WCAG 2.1 success criteria are written as testable statements that are not technology-specific.
bb: agrees; seeing this as a real world implementation of a11y testing
<michaelcrabb> (I have to run - reminder that XR group meeting is Monday @ 9am)
bb: making silver attractive for policy makers -- this is best practices, not what it means to be wcag conformant
<Jan> There are plenty of spreadsheets behind the scenes in WCAG 2.x right now ... there's plenty of complexity with what we are currently doing.
ja: mulls on how minimum required
thresholds -- this is critical, etc -- that will help with
objective weighting
... explaining that will help
js: also a lot of room for tooling, automation, etc
ja: yes, but this is what makes that possible
<Jan> I have to drop for another call.
<Jan> Great job, Jake!
<Zakim> bruce_bailey, you wanted to just say my concern is about what we mean by quote wcag conformant unquote
<Jan> +1 Bruce's concern - it's an important point.
js: next steps?
ja: going through jf's pages to
see impact on total scoring
... next adding a few more guidelines to see whether
similarities hold up
ja; then third into a task and evaluate what is conformance, partial conformance, etc
ja; this all takes time and good examples really help
js: looks for volunteers to help
draft a representative sample task
... points to links in today's agenda
<jeanne> https://w3c.github.io/silver/guidelines/#sampling
js: notes concern of what's still
needed to get to fpwd
... we need to do that with people who've spent time becoming
familiar with this approach
... email Jeanne if interested
jf: notes almost any eval service
ultimately comes out to a single score
... believes single score is something site owners will want
and should help regulators
<Fazio> Can't we have both a total score and score by POUR, and/or disability group with this new spreadsheet Jake made?
<Fazio> I think that would be better
ja: notes various tools from
various vendors and the scores just don't tell anything
... it just has no value
<bruce_bailey> +1 that score are pretty arbitrary
ja: sees no value
jf: agrees it's not helpful to
the devs, but thinks the ceo would like to see improvement in
the final number
... suggests it's a way to show and encourage progress --
competitive analysis
<JF> https://docs.google.com/document/d/1PgmVS0s8_klxvV2ImZS1GRXHwUgKkoXQ1_y6RBMIZQw/edit
ja: suggests the scores aren't correct; different dashboards come out with different scores
<Zakim> bruce_bailey, you wanted to say multiple currencies may not be needed with the approaches we are working on
ja: ok with having scores, just rolling into a single number doesn't tell anyone much
bb: agrees with ja
... believe we're moving that way especially tracking toward
functional needs
... don't believe we need worry about that much right now
df: see both points and think we
can support both
... notes legislation doesn't have this kind of detail; just
talks in general terms of what's required
js: asks the group to consider what's W3C's responsibility vis a vis what other players bring to the party
<bruce_bailey> +1 for focus on our role as standards org
jf: notes every vendor dashboard
supports a final score
... it's what the customers want
... wants w3c to provide what the tooling vendors need
ja: not disagreeing; notes it's
supported by the sheets; just don't think it means that
much
... don't think there's much gap here
js: asks jf whether w3c based
data; is that sufficient?
... or does w3c need to tell tool vendors how to put it all
together
jf: points to html5
<bruce_bailey> @jf wcag2x does not lend itself to percental score, and that has been okay
jf: gaps in how data is measured
or caluclated would be a problem
... notes tool vendors all using ACT
... whose ever tool should all come back with approx the same
score
... looking for sufficient guidance to do proper eval; then we
can differentiate from there
... notes html5 heavy on error handling
This is scribe.perl Revision of Date Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00) Present: jeanne sajkaj CharlesHall JakeAbma bruce_bailey MichaelC Lauriat Fazio PeterKorn JF OmarBonilla Rachael Chuck KimD kirkwood AngelaAccessForAll JoeCronin michaelcrabb Jan Found Scribe: sajkaj Inferring ScribeNick: sajkaj WARNING: No date found! Assuming today. (Hint: Specify the W3C IRC log URL, and the date will be determined from that.) Or specify the date like this: <dbooth> Date: 12 Sep 2002 People with action items: WARNING: Input appears to use implicit continuation lines. You may need the "-implicitContinuations" option. WARNING: IRC log location not specified! (You can ignore this warning if you do not want the generated minutes to contain a link to the original IRC log.)[End of scribe.perl diagnostic output]