See also: IRC log
<fesch> https://www.w3.org/dpub/docs/scribing-guidelines.xhtml
<jongund> trackbot, start meeting
<ShaneM> is the code in the topic correct?
<jongund> Scribe: cynthia
<jongund> Meeting: ARIA Testing
<cyns> scribe: cyns
<jongund> scribe: cyns
<jongund> https://www.w3.org/wiki/ARIA_1.1_Testable_Statements
jg: do we have any update on that?
fe: gave state and property
statements to John F's team
... working on script that turns wiki tables to json object
<jongund> https://www.w3.org/wiki/ARIA_1.1_Testable_Statements_JSON
fe: this may be hard to use when
there is a complex tree
... looking for feedback on how usable the json is
https://www.w3.org/wiki/ARIA_1.1_Testable_Statements_JSON
jg: is there validation?
fe: no
jg: script available?
fe: will post soon
jg: is this ok for shane?
sm: yes
jg: will find out what John F and team is doing
<jongund> https://www.w3.org/wiki/index.php?title=ARIA_1.1_Testable_Statements&action=history
jg: talked to shane about wpt
solution strategy
... also microsoft strategy
... key to both is accessibility api shims that reflect back
accessibility info to testing framework
... is the shim self contained in msft architecture?
... I have a student who is going to start working on msaa/ia2
shim
... what shane needed is pretty simple
db: it is self contained. it runs
locally. spins up pages through web driver, then evaluates
against requirements.
... requirements are currently hard-coded, but will be bringing
in from json.
... looking at output in a way wpt can consume
... we have api's in c# that allow us to talk to uia, which
then talks to the browser.
... when you talk to uia, what is it giving you back? a11y tree
info?
... yes, accessibility tree objects
jg: what change is needed to use ia2 in your driver?
db: designed with other api/browser in mind. it's factored out.
jg: shane, what is your approach
sm: wpt architecture is complicated. because it's desinged to run in places where it's hard to do anything cool, like locally running code
<jongund> https://www.w3.org/wiki/ARIA_1.1_Automated_Testing
sm: wpt has test web pages with test code embedded within them.
<jongund> https://www.w3.org/wiki/ARIA_1.1_Automated_Testing#Proposed_Architecture
sm: can use webdriver but does't
need it
... can be a lot of interaction between test server and child
window
... idea is to have child window talk to wpt server as a relay
to talk to the shim
... can use in differnt environemtns, like android
cs: not needed for aria 1.1
sm: true, but good for
futre
... is msft tool pulling expected results from child
window?
db: not yet
sm: do we want evaluative smarts
in test window, so they have to be written once, or in logic
that is part of platform specific
... is it doing the evaluation?
db: yes
sm: are you oepn to pulling the logic up into a javascript library that might live in the test?
db: I think that's possible, but
a lot of work
... client code can run anywhere c# can be compiled (which is
most platforms)
sm: in the diagram, see "magic". Is it possible for get the needed info from the client window?
db: yes, but file would be easier
sm: easier for me, maybe not for you... if the local at shim had a port, then child test window could instruct it what to do from localhost
db: that makes sense to me
sm: that would let me remove the server-side part on the right
cs: details of "magic"
sm: RESTful api, with window
title and criteria, and yes or no.
... wpt would call that subtests
<ShaneM> individual results for each assertion evaluated per test window.
sm: in wpt world, when result came back, test runner can tell system to open next test
jg: would that make the microsoft and wpt tools more compatible
db: yes. changing to take input from a port rather than file shouldn't be too hard
sm: like that
... my goal is that this effort is useful going forward for
testing user agents as they are developed, like the rest of
wpt
... the easier it is for browser developers to know when aria
is broken and fix it, the better
... stanley, can we look at integration points
sh: yes
jd: sounds interesting, but need to look at code
sm: there is some setup for full-scale automated testing. you'll have to start up the shim and the browser.
jd: that's easy
(discussion of configuration requirements for setting up wpt)
jg: I have a student who will be
working on IA2. she's looking at wpt now and accprobe.
... next step will be looking at microsoft test harness
db: already prototyped
cs: branch is public
<dbrett> https://github.com/microsoftEdge/a11y/tree/aria
sm: I'll do the work for the wpt pieces
db: it can read any json
fe: I can tweak too
This is scribe.perl Revision: 1.144 of Date: 2015/11/17 08:39:34 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: RRSAgent_Text_Format (score 1.00) Found Scribe: cynthia Found Scribe: cyns Found Scribe: cyns Inferring ScribeNick: cyns Scribes: cynthia, cyns Present: Joanmarie_Diggs fesch Rich_Schwerdtfeger jongund Found Date: 31 Aug 2016 Guessing minutes URL: http://www.w3.org/2016/08/31-aria-test-minutes.html People with action items: WARNING: Input appears to use implicit continuation lines. You may need the "-implicitContinuations" option.[End of scribe.perl diagnostic output]