IRC log of aria on 2015-12-10

Timestamps are in UTC.

17:32:29 [RRSAgent]
RRSAgent has joined #aria
17:32:29 [RRSAgent]
logging to http://www.w3.org/2015/12/10-aria-irc
17:32:31 [trackbot]
RRSAgent, make logs public
17:32:31 [Zakim]
Zakim has joined #aria
17:32:33 [trackbot]
Zakim, this will be WAI_PF
17:32:33 [Zakim]
I do not see a conference matching that name scheduled within the next hour, trackbot
17:32:34 [trackbot]
Meeting: Protocols and Formats Working Group Teleconference
17:32:34 [trackbot]
Date: 10 December 2015
17:32:36 [MichaelC]
agenda: https://lists.w3.org/Archives/Public/public-pfwg/2015Dec/0093.html
17:32:45 [joanie]
present+ Joanmarie_Diggs
17:32:47 [MichaelC]
meeting: ARIA
17:32:47 [MichaelC]
chair: MichaelC
17:32:50 [MichaelC]
regrets: Rich, Léonie
17:33:01 [MichaelC]
present+ MichaelC, Janina, Suman, Fred
17:33:12 [MichaelC]
rrsagent, make log final
17:33:27 [MichaelC]
scribeOptions: -final
17:34:05 [MichaelC]
agenda+ Publications update
17:34:05 [MichaelC]
agenda+ Working Group transition update
17:34:05 [MichaelC]
agenda+ Mailing list subscription policy update
17:34:05 [MichaelC]
agenda+ Test planning (major topic for this call)
17:34:05 [MichaelC]
agenda+ Continue issue scrubbing
17:34:07 [MichaelC]
agenda+ Upcoming meeting schedule
17:34:52 [MichaelC]
present+ Cynthia
17:35:13 [MichaelC]
present+ Michiel
17:35:29 [jamesn]
present+ JamesNurthen
17:35:35 [cyns]
cyns has joined #aria
17:36:12 [jamesn]
scribe: jamesn
17:36:15 [mck]
mck has joined #aria
17:36:19 [jamesn]
agenda?
17:37:20 [jamesn]
agenda+ aria-primaryaction discussion
17:38:34 [mck]
present+ matt_king
17:38:54 [MichaelC]
present+ Matt
17:39:01 [MichaelC]
present+ Joseph
17:39:07 [clown]
clown has joined #aria
17:39:14 [MichaelC]
agenda order 1, 2, 3, 7
17:39:15 [clown]
present+ Joseph_Scheuhammer
17:39:19 [MichaelC]
zakim, next item
17:39:19 [Zakim]
agendum 1. "Publications update" taken up [from MichaelC]
17:39:33 [jamesn]
MC: main thing is that make sure everyone knows
17:39:40 [jamesn]
everythign published recently
17:39:47 [jamesn]
FPWD of some
17:39:54 [jamesn]
updated of all the rest...
17:40:26 [jamesn]
talking timeline with Rich think would like to do an updated WD of 1.1 (pseudo last call)
17:40:31 [jamesn]
early feb for that
17:40:43 [jamesn]
close out remainder of issues between now and then
17:40:51 [jamesn]
if we stick to timeline CR early april
17:40:54 [jamesn]
just after CSUN
17:41:17 [jamesn]
some of the other specs may advance to CR at same time. Core may but others may not
17:41:28 [jamesn]
push to get 1.1 done is there. want to shift focus to 2.0
17:41:56 [jamesn]
one of the issues is the piece of the puzzle for extended descriptions. nexts weeks call will focus on thaty
17:42:16 [jamesn]
want to get this done b4 the holidays so people working over holidays can do stuff
17:42:25 [jamesn]
zakim, next item
17:42:25 [Zakim]
agendum 2. "Working Group transition update" taken up [from MichaelC]
17:43:00 [jamesn]
MC: reminders. ARIA group exists. IEs can now join. nearly everyone here as member org has joined or is talking
17:43:18 [jamesn]
PF mailing list will be for rest of this month. will be migrating tracker issues
17:43:25 [jamesn]
there will be a tracker cutoff too
17:43:58 [jamesn]
PF tracker needs to split issues and actions between apa and aria
17:44:12 [jamesn]
if issues appear in new tracker b4 migration then may make it harder
17:44:22 [jamesn]
try to avoid using them until january
17:44:29 [MichaelC]
present+ Bryan
17:44:49 [jamesn]
zakim, next item
17:44:49 [Zakim]
agendum 3. "Mailing list subscription policy update" taken up [from MichaelC]
17:45:11 [MichaelC]
https://lists.w3.org/Archives/Public/public-aria-admin/2015Dec/0000.html
17:45:27 [jamesn]
was a CFC on the aria and PF lists to change the policy for the public-aria list
17:45:47 [bgaraventa1979]
bgaraventa1979 has joined #aria
17:45:51 [jamesn]
already the case that members can post but if they want to subscribe they can do so and agree to the same process requirements that others accept
17:46:08 [bgaraventa1979]
present+ Bryan_Garaventa
17:46:11 [jamesn]
there was dicsussion on the PF list that looked like it was diverging from consensus
17:46:22 [jamesn]
looks like that has moved so that we now have consensus
17:46:40 [jamesn]
want to make it official so the chair needs to ratify the consensus on list
17:46:55 [jamesn]
zakim, next item
17:46:55 [Zakim]
agendum 7. "aria-primaryaction discussion" taken up [from jamesn]
17:47:18 [MichaelC]
scribe: MichaelC
17:47:46 [MichaelC]
JN: discussion on list, implementers seem to not support
17:47:51 [MichaelC]
want it to work differently
17:47:51 [jongund]
jongund has joined #aria
17:48:04 [MichaelC]
my concern is the proposed changes move the difficulty to frameworks and authors
17:48:10 [MichaelC]
rather than have the browser address
17:48:57 [MichaelC]
js: primary button on @@
17:49:08 [MichaelC]
jn: problem is it just points to @@
17:49:19 [MichaelC]
js: AXAPI might have problem
17:49:25 [MichaelC]
because have to search up the tree to find the button
17:49:31 [MichaelC]
the others might not have a problem
17:49:35 [MichaelC]
jn: worried this is hard for authros
17:49:45 [MichaelC]
s/authros/authors/
17:50:00 [MichaelC]
js: I´m ok either way
17:50:02 [clown]
clown has left #aria
17:50:05 [MichaelC]
others might object to non idref version
17:50:11 [clown]
clown has joined #aria
17:50:11 [MichaelC]
mb: agree shouldn´t be idref
17:50:16 [MichaelC]
developers don´t want to bother with idrefs
17:50:25 [MichaelC]
eventually this should be in browser
17:50:30 [MichaelC]
so shouldn´t bother developers now
17:50:47 [MichaelC]
jn: discussion moves too quickly
17:51:15 [MichaelC]
mk: some value for dialogs only
17:51:23 [MichaelC]
not sure as important for other contexts
17:51:54 [MichaelC]
jn: wizard isn´t always in a dialog
17:51:59 [MichaelC]
js: they don´t have roles...
17:52:19 [MichaelC]
mk: unsure of value of that for screen reader users anyway
17:52:39 [MichaelC]
I don´t trust default button, so I look around
17:52:49 [MichaelC]
jn: because it doesn´t tell you, but if it did you might trust it
17:53:04 [MichaelC]
present+ Jon
17:53:36 [MichaelC]
mk: even so, not as high value for me
17:54:06 [MichaelC]
jn: mostly agree
17:54:09 [MichaelC]
it´s just a button attribute
17:54:21 [MichaelC]
mk: could be boolean on button, forget about context
17:54:33 [MichaelC]
then would need to drop language about author error
17:54:57 [joanie]
q+ To say the value of the context in the case of the web is limiting the tree dive. That said, I don't see it as must-have.
17:55:12 [MichaelC]
mc: there was some confusion, would a rephrasing on list help?
17:55:26 [MichaelC]
mk: need clearer statement of goals for screen readers
17:55:51 [MichaelC]
js: AX has a dialog, with a default button
17:56:06 [MichaelC]
though the button itself doesn´t have anything to make it the default
17:56:24 [MichaelC]
if you put aria-default on the button, the browser has to figure out what the context is that it´s the default for
17:56:46 [MichaelC]
that´s a pattern they have opposed elsewhere, for aria-current
17:57:16 [joanie]
q-
17:57:58 [MichaelC]
mc: so now there are two things with similar problems
17:58:20 [MichaelC]
js: difference is that with default button you need more information about what´s gonna get dismissed
17:58:44 [MichaelC]
mk: @@
17:59:26 [MichaelC]
mc: for aria-current, we considered aria-currentfor, but rejected it
17:59:50 [MichaelC]
mk: is there a way of not having to go up the tree?
18:01:51 [MichaelC]
just say we care about the button, is a default, less important to know what it´s for
18:02:24 [MichaelC]
jn: sighted user doesn´t know, they infer
18:02:37 [MichaelC]
mc: they infer pretty strongly
18:03:08 [MichaelC]
jd: for end user, context isn´t that important
18:03:32 [MichaelC]
but for AT, on some platforms, there´s a ¨tell me the default button¨ command
18:03:49 [MichaelC]
having to go up the hierarchy isn´t so bad
18:03:58 [MichaelC]
but having to go *down* the hierarchy is much more expensive
18:04:44 [MichaelC]
user knows they´re in a dialog etc.
18:04:51 [MichaelC]
but the application has a much wider context
18:05:01 [jamesn]
q+
18:05:13 [MichaelC]
mb: there are only so many things the default button could be in
18:05:36 [MichaelC]
q+
18:05:53 [MichaelC]
ack j
18:06:04 [MichaelC]
jn: maybe this can be solved in ARIA 2.0 with selectors
18:06:23 [MichaelC]
q+ to say take advantage of API
18:06:37 [MichaelC]
js: author has to know what to select
18:06:40 [MichaelC]
jn: they will know
18:06:42 [joanie]
q+ To say that Matt's suggestion works *if* we say the primary button MUST be in the context of a dialog
18:06:49 [MichaelC]
ack me
18:06:49 [Zakim]
MichaelC, you wanted to say take advantage of API
18:08:16 [MichaelC]
mc: even if we´re not as concerned about what AXAPI offers, is nice to support the greater richness where available
18:08:17 [MichaelC]
ack j
18:08:17 [Zakim]
joanie, you wanted to say that Matt's suggestion works *if* we say the primary button MUST be in the context of a dialog
18:08:28 [MichaelC]
jd: Matt's suggestion works *if* we say the primary button MUST be in the context of a dialog
18:08:37 [jamesn]
q+ to say wizards dont have role=dialog and cannot
18:08:58 [MichaelC]
ack j
18:08:58 [Zakim]
jamesn, you wanted to say wizards dont have role=dialog and cannot
18:09:09 [MichaelC]
jn: can´t have wizards have role=dialog
18:09:25 [MichaelC]
they have differences
18:09:32 [MichaelC]
you can leave them etc.
18:10:35 [joanie]
+1 to getting it right in 2.0
18:11:24 [MichaelC]
mc: to summarize, sounds like support for doing well in ARIA 2.0 rather than badly in ARIA 1.1
18:11:30 [MichaelC]
so adjust issues accordingly
18:13:00 [MichaelC]
and inform the thread of this
18:13:18 [jamesn]
zakim, next item
18:13:18 [Zakim]
agendum 4. "Test planning (major topic for this call)" taken up [from MichaelC]
18:13:26 [MichaelC]
scribe: jamesn
18:14:55 [jamesn]
MC: Rich wanted in particular for MC to lead this discussion.
18:15:03 [jamesn]
due to role in aria test harness
18:15:15 [jamesn]
has been discussion in various other groups.
18:15:43 [MichaelC]
-> https://rawgit.com/w3c/aria/CR-pub/aria/aria.html#sotd Draft exit criteria
18:16:27 [jamesn]
for MUSTs in 1.0 needed a testable statement
18:16:35 [jamesn]
for most part was role, states and properties
18:16:57 [jamesn]
what we care about is that is makes its way to the API so we depended on the UAIG - now the core and other mappings
18:17:05 [jamesn]
says for the feature - how it should look in the apis
18:17:17 [jamesn]
sometimes simple, sometimes more complex
18:17:39 [Stefan]
Stefan has joined #aria
18:17:47 [jamesn]
so for ARIA testing means we need to test the mapping guides - then set some sontraints that need to find 2 implelmentations of each feature
18:18:08 [jamesn]
for the mappings there may only be 1 implementation of a mapping - sometimes there is only 1 mapping for an API
18:18:16 [jamesn]
yes - fred also test that
18:18:43 [jamesn]
so find 2 instances of some mapping. so if find 1 on windows and 1 on mac then these are the 2 mappings
18:18:57 [jamesn]
if cant find 2 on 2 platforms then look for 2 browsers on the same platform
18:19:11 [jamesn]
that is a less preferred approach but did that for about 20 of the tests
18:19:39 [jamesn]
so for a given role what you need is a testable statement which says here is a test of this role and says what you see in the AAPI
18:19:46 [MichaelC]
present+ Stefan
18:20:07 [jamesn]
and then need to test it. created minimal html files which had just enough to be valid
18:20:20 [jamesn]
then the aria feature, put an id of test on it so could find easily
18:20:23 [jamesn]
then look for the mapping
18:20:36 [jamesn]
for 1.0 was entirely manual. that is where the test harmess comes in
18:21:00 [jamesn]
the test harness gives the statement, the result and the file. would open, then use your inspector to look for the result
18:21:21 [jamesn]
we have talked about automating. we believe the majority are automateable, but dont have a platofrm for it
18:21:46 [jamesn]
we do know the different vendors have platforms but hey are not public. there are some tools that can be scripted
18:22:02 [jamesn]
Joanie has done some experimentation with automation. she is hopeful
18:22:21 [jamesn]
Q - how much do we invest in the manual testing and how much do we invest in the automation
18:22:28 [jamesn]
want to come back to that too
18:23:28 [jamesn]
mostly either roles states and properties table or MUST statements - acc name is part of that
18:23:32 [fesch]
q+
18:24:10 [jamesn]
"WAI-ARIA 1.1 builds upon WAI-ARIA 1.0, which met its implementation requirements in February 2014. Only features that are new or changed in WAI-ARIA 1.1 need be tested, as the WAI-ARIA 1.0 implementation report provides implementatibility and interoperability evidence for the remaining features. Features that will be tested for WAI-ARIA 1.1 include:"
18:24:38 [jamesn]
Parts of the name computation algorithm may need to be tested but maybe not all of it
18:25:05 [jamesn]
for if statements sometimes need a test file to test every one of them. also sometimes some negative test cases
18:25:19 [jamesn]
if A+B then sometimes need failures
18:25:43 [jamesn]
for key cases where a failure would represent an incorrect implementation then may need some failures too
18:25:47 [jamesn]
q?
18:26:05 [jamesn]
FE: how do you test something that is not supposed to be in the tree
18:26:18 [jamesn]
MC: you say the a11y tree does not have such and such
18:26:41 [jamesn]
FE: in SVG have xyz does not get into the tree
18:27:05 [jamesn]
MC: id they element with id test should not show in the tree then the expected result is that id test doesn't appear
18:27:51 [jamesn]
would suggest putting something that should not appear in the tree with a certain idref
18:27:55 [jamesn]
FE: is there an automarted way?
18:28:00 [mck]
q+
18:28:04 [jamesn]
MC: I would think it could be automateable
18:28:07 [MichaelC]
ack f
18:28:08 [MichaelC]
ack m
18:28:14 [jamesn]
MK: I missed which doc
18:28:24 [jamesn]
MC: https://rawgit.com/w3c/aria/CR-pub/aria/aria.html#sotd
18:28:53 [mck]
ack me
18:29:04 [jamesn]
rrsagent, make minutes
18:29:04 [RRSAgent]
I have made the request to generate http://www.w3.org/2015/12/10-aria-minutes.html jamesn
18:29:18 [cyns]
q+
18:29:18 [jamesn]
topic: automation
18:29:44 [jamesn]
JD: in terms of automating have talked in terms of webdriver and DOM. not going to happen.
18:29:55 [jamesn]
have a POC hackaround by which i can run my tests
18:30:07 [jamesn]
as an aside - the way the hack works is that it does a tree dive
18:31:05 [MichaelC]
NVDA has a python-based tool to poke at IA2
18:31:28 [MichaelC]
I might be able to hack that to test IA12
18:31:35 [MichaelC]
s/IA12/IA2/
18:31:53 [MichaelC]
ack c
18:32:33 [jamesn]
CS: so we are going to be automating our own implementation
18:32:47 [jamesn]
I can share the results - but not certain about the code
18:33:21 [joanie]
q+ to ask the obvious, if it's UI *Automation*, there is surely public/open code we could use to create our own testing on top of the MS testing.
18:33:23 [jamesn]
if you can tell me what format can suck data in. In the ARIA 1.0 test harness could perhaps take data from elsewhere and put it into the harness
18:33:38 [jamesn]
everything we are testing - i think the testable statements are the requirements
18:33:43 [jamesn]
q+
18:34:06 [jamesn]
MC: if we had a way of mapping tests to statements couold suck in the results
18:34:16 [jamesn]
we can work on our preferred format
18:34:32 [jamesn]
would you be using our test files or are you doing your own set
18:34:33 [fesch]
q+ for asking about test file templates
18:34:49 [jamesn]
CS: i suspect some of both
18:35:03 [jamesn]
i suspect we will be doing a superset
18:35:18 [jamesn]
MC: if your test files are different there is a potentially invalid situation
18:35:31 [jamesn]
MC: my preference would be to use the same tests
18:35:54 [jamesn]
MC: may mean among other things that joanie may want to look at other platforms than ui automation
18:36:20 [jamesn]
in 1.0 decided needed to be HTML4 - likley need HTML5 and SVG files for 1.1
18:36:32 [jamesn]
went to great pains than 1.1 tests were valid html4
18:36:59 [jamesn]
still do need them to be well formed - minimize parsing artifacts. and that includes inline svg if needed
18:37:04 [MichaelC]
ack next
18:37:05 [Zakim]
joanie, you wanted to ask the obvious, if it's UI *Automation*, there is surely public/open code we could use to create our own testing on top of the MS testing.
18:37:38 [jamesn]
JD: could use for edge perhaps
18:37:42 [jamesn]
CS: yes that is true
18:37:59 [jamesn]
CS: wasnt designed for web content
18:38:06 [jamesn]
we will probably do that work
18:38:34 [jamesn]
JD: if there is public stuff there i could use to hack on your platform using open stuff
18:38:52 [jamesn]
JD: the ID attribute - on most platforms that is the case
18:39:05 [jamesn]
CS: it is exposed if there is an implicit id on the platform
18:39:27 [MichaelC]
ack next
18:40:04 [MichaelC]
jn: in firefox you could build an extension that looks at the AAPI
18:40:09 [MichaelC]
and should work on all its platforms
18:40:14 [MichaelC]
and script that with selenium
18:40:38 [MichaelC]
jd: we want to test what the AT sees
18:40:50 [MichaelC]
not what the browser says it´s sending to the AAPI
18:41:11 [MichaelC]
jn: it´s got an interface to the COM tree to show exactly what´s winding up in the AAPI
18:41:34 [MichaelC]
jd: it´s talking to its own AAPI layer
18:41:43 [MichaelC]
js: don´t know which it does
18:42:00 [MichaelC]
jn: if I ask the role, it reports something different on different platforms
18:42:19 [MichaelC]
jd: it says what it thinks it´s exposing, which varies by platform
18:43:18 [MichaelC]
mc: should explore this separately...
18:43:25 [clown]
s/an interface to the COM tree/an interface to the XPCOM information/
18:43:54 [MichaelC]
jg: if you have sample extension, we could explore making something work
18:44:29 [MichaelC]
jn: looks like it comes from accessibility retrieval interface
18:44:36 [MichaelC]
ack next
18:44:37 [Zakim]
fesch, you wanted to discuss asking about test file templates
18:44:51 [jamesn]
FE: couple of questions about the template
18:45:03 [jamesn]
can it be svg files or can it be html with svg inline
18:45:21 [jamesn]
want to make sure that we have consistent test files which could introduce artifcats
18:45:41 [jamesn]
s/want to make sure/MC: want to make sure/
18:45:51 [jamesn]
MC: best to pick 1 approach or another
18:46:01 [clown]
q+ to ask if a difference is expected between svg file and svg-in-html file?
18:46:05 [jamesn]
MC: dont want to go into too much detail about structure of the test files
18:46:12 [MichaelC]
-> https://github.com/w3c/aria/tree/master/testfiles/1.0 ARIA 1.0 test files
18:46:18 [jamesn]
the test files for 1.0 are in above
18:46:29 [jamesn]
grouped into folders as 800 or so
18:46:36 [jamesn]
lots are auto generated
18:46:59 [jamesn]
testing a label when there is a menu role
18:47:44 [jamesn]
the label should come out based on the combo box
18:47:52 [jamesn]
in the test harness should see as correct
18:48:05 [jamesn]
q+
18:48:15 [jamesn]
would suggest the same with the svg
18:49:03 [jamesn]
JN: shouldn't there be something in the file for what the result should be?
18:49:13 [jamesn]
MK: think it should all be in 1 file
18:49:25 [jamesn]
MC: there are 2 reasons they are not in the file
18:49:40 [jamesn]
even in comments could perhaps influence the results - could make it hard to debug
18:49:58 [jamesn]
another reason is that there are 5 or 6 different results according to the platform
18:50:11 [jamesn]
JD proposal does have them in the test file - we may be exploring that
18:50:32 [jamesn]
i am inclined to say we might want versions without and then with results in them for the automation tool
18:50:34 [jamesn]
ack me
18:50:57 [jamesn]
MK: certainly wouldn't hurt to have the textual description of the result in the file
18:51:11 [jamesn]
clown: in some cases use the same test file for different cases
18:51:22 [jamesn]
MK: would rather duplicate the file
18:51:27 [jamesn]
MC: link to test harness
18:53:41 [jamesn]
MC: the test harness where there is a ref to what are testing
18:53:52 [jamesn]
MC: there is a single expected result on all platforms
18:54:02 [jamesn]
sometimes different for different platforms
18:54:30 [jamesn]
i think probably need to organize a training on the test harness - if havent used it before then lets do a training
18:54:40 [jamesn]
should schedule supplementary training
18:56:34 [jamesn]
MC: will do a poll will send to aria mailing list i think
18:57:09 [jamesn]
will need to test dpub roles and graphics roles too
18:57:29 [jamesn]
i think so - will discuss in the svg a11y group
18:57:43 [jamesn]
s/i think so/FE: i think so/
18:58:13 [jamesn]
MC: i know the a11y name computation has some host language stuff. some you might rely on existsing stuff - but others are svg specific
18:58:30 [jamesn]
dont feel that have got enough on testing to add action items for over the holidays
18:58:45 [jamesn]
rrsagent, make minutes
18:58:45 [RRSAgent]
I have made the request to generate http://www.w3.org/2015/12/10-aria-minutes.html jamesn
18:59:05 [jamesn]
FE: we plan on using these files and this format of test harness and test files
18:59:32 [jamesn]
MC: so want test statements and expected results. if we dont build the test cases etc we wont be wasting time
18:59:44 [jamesn]
when to use the exact test harness is an open question
18:59:58 [jamesn]
no one loves the harness but it has the advantage that it exists ;)
19:00:10 [jamesn]
will be using for reporting
19:00:16 [MichaelC]
-> https://www.w3.org/WAI/PF/testharness/testreport?testsuite_id=1 ARIA 1.0 test report
19:00:38 [jamesn]
will be a group for svg and others for 1.1
19:00:40 [jamesn]
etc.
19:00:55 [jamesn]
if import tests from other locations need the report
19:01:20 [clown]
ack me
19:01:20 [Zakim]
clown, you wanted to ask if a difference is expected between svg file and svg-in-html file?
19:01:57 [MichaelC]
rrsagent, make minutes
19:01:57 [RRSAgent]
I have made the request to generate http://www.w3.org/2015/12/10-aria-minutes.html MichaelC
20:24:43 [clown]
clown has joined #aria
20:50:47 [MichaelC]
rrsagent, bye
20:50:47 [RRSAgent]
I see no action items
20:50:47 [MichaelC]
zakim, bye
20:50:47 [Zakim]
leaving. As of this point the attendees have been Joanmarie_Diggs, MichaelC, Janina, Suman, Fred, Cynthia, Michiel, JamesNurthen, matt_king, Joseph, Joseph_Scheuhammer, Bryan,
20:50:47 [Zakim]
Zakim has left #aria