17:32:29 <RRSAgent> RRSAgent has joined #aria
17:32:29 <RRSAgent> logging to http://www.w3.org/2015/12/10-aria-irc
17:32:31 <trackbot> RRSAgent, make logs public
17:32:31 <Zakim> Zakim has joined #aria
17:32:33 <trackbot> Zakim, this will be WAI_PF
17:32:33 <Zakim> I do not see a conference matching that name scheduled within the next hour, trackbot
17:32:34 <trackbot> Meeting: Protocols and Formats Working Group Teleconference
17:32:34 <trackbot> Date: 10 December 2015
17:32:36 <MichaelC> agenda: https://lists.w3.org/Archives/Public/public-pfwg/2015Dec/0093.html
17:32:45 <joanie> present+ Joanmarie_Diggs
17:32:47 <MichaelC> meeting: ARIA
17:32:47 <MichaelC> chair: MichaelC
17:32:50 <MichaelC> regrets: Rich, Léonie
17:33:01 <MichaelC> present+ MichaelC, Janina, Suman, Fred
17:33:12 <MichaelC> rrsagent, make log final
17:33:27 <MichaelC> scribeOptions: -final
17:34:05 <MichaelC> agenda+ Publications update
17:34:05 <MichaelC> agenda+ Working Group transition update
17:34:05 <MichaelC> agenda+ Mailing list subscription policy update
17:34:05 <MichaelC> agenda+ Test planning (major topic for this call)
17:34:05 <MichaelC> agenda+ Continue issue scrubbing
17:34:07 <MichaelC> agenda+ Upcoming meeting schedule
17:34:52 <MichaelC> present+ Cynthia
17:35:13 <MichaelC> present+ Michiel
17:35:29 <jamesn> present+ JamesNurthen
17:35:35 <cyns> cyns has joined #aria
17:36:12 <jamesn> scribe: jamesn
17:36:15 <mck> mck has joined #aria
17:36:19 <jamesn> agenda?
17:37:20 <jamesn> agenda+ aria-primaryaction discussion
17:38:34 <mck> present+ matt_king
17:38:54 <MichaelC> present+ Matt
17:39:01 <MichaelC> present+ Joseph
17:39:07 <clown> clown has joined #aria
17:39:14 <MichaelC> agenda order 1, 2, 3, 7
17:39:15 <clown> present+ Joseph_Scheuhammer
17:39:19 <MichaelC> zakim, next item
17:39:19 <Zakim> agendum 1. "Publications update" taken up [from MichaelC]
17:39:33 <jamesn> MC: main thing is that make sure everyone knows
17:39:40 <jamesn> everythign published recently
17:39:47 <jamesn> FPWD of some
17:39:54 <jamesn> updated of all the rest...
17:40:26 <jamesn> talking timeline with Rich think would like to do an updated WD of 1.1 (pseudo last call)
17:40:31 <jamesn> early feb for that
17:40:43 <jamesn> close out remainder of issues between now and then
17:40:51 <jamesn> if we stick to timeline CR early april
17:40:54 <jamesn> just after CSUN
17:41:17 <jamesn> some of the other specs may advance to CR at same time. Core may but others may not
17:41:28 <jamesn> push to get 1.1 done is there. want to shift focus to 2.0
17:41:56 <jamesn> one of the issues is the piece of the puzzle for extended descriptions. nexts weeks call will focus on thaty
17:42:16 <jamesn> want to get this done b4 the holidays so people working over holidays can do stuff
17:42:25 <jamesn> zakim, next item
17:42:25 <Zakim> agendum 2. "Working Group transition update" taken up [from MichaelC]
17:43:00 <jamesn> MC: reminders. ARIA group exists. IEs can now join. nearly everyone here as member org has joined or is talking
17:43:18 <jamesn> PF mailing list will be for rest of this month. will be migrating tracker issues
17:43:25 <jamesn> there will be a tracker cutoff too
17:43:58 <jamesn> PF tracker needs to split issues and actions between apa and aria
17:44:12 <jamesn> if issues appear in new tracker b4 migration then may make it harder
17:44:22 <jamesn> try to avoid using them until january
17:44:29 <MichaelC> present+ Bryan
17:44:49 <jamesn> zakim, next item
17:44:49 <Zakim> agendum 3. "Mailing list subscription policy update" taken up [from MichaelC]
17:45:11 <MichaelC> https://lists.w3.org/Archives/Public/public-aria-admin/2015Dec/0000.html
17:45:27 <jamesn> was a CFC on the aria and PF lists to change the policy for the public-aria list
17:45:47 <bgaraventa1979> bgaraventa1979 has joined #aria
17:45:51 <jamesn> already the case that members can post but if they want to subscribe they can do so and agree to the same process requirements that others accept
17:46:08 <bgaraventa1979> present+ Bryan_Garaventa
17:46:11 <jamesn> there was dicsussion on the PF list that looked like it was diverging from consensus
17:46:22 <jamesn> looks like that has moved so that we now have consensus
17:46:40 <jamesn> want to make it official so the chair needs to ratify the consensus on list
17:46:55 <jamesn> zakim, next item
17:46:55 <Zakim> agendum 7. "aria-primaryaction discussion" taken up [from jamesn]
17:47:18 <MichaelC> scribe: MichaelC
17:47:46 <MichaelC> JN: discussion on list, implementers seem to not support
17:47:51 <MichaelC> want it to work differently
17:47:51 <jongund> jongund has joined #aria
17:48:04 <MichaelC> my concern is the proposed changes move the difficulty to frameworks and authors
17:48:10 <MichaelC> rather than have the browser address
17:48:57 <MichaelC> js: primary button on @@
17:49:08 <MichaelC> jn: problem is it just points to @@
17:49:19 <MichaelC> js: AXAPI might have problem
17:49:25 <MichaelC> because have to search up the tree to find the button
17:49:31 <MichaelC> the others might not have a problem
17:49:35 <MichaelC> jn: worried this is hard for authros
17:49:45 <MichaelC> s/authros/authors/
17:50:00 <MichaelC> js: I´m ok either way
17:50:02 <clown> clown has left #aria
17:50:05 <MichaelC> others might object to non idref version
17:50:11 <clown> clown has joined #aria
17:50:11 <MichaelC> mb: agree shouldn´t be idref
17:50:16 <MichaelC> developers don´t want to bother with idrefs
17:50:25 <MichaelC> eventually this should be in browser
17:50:30 <MichaelC> so shouldn´t bother developers now
17:50:47 <MichaelC> jn: discussion moves too quickly
17:51:15 <MichaelC> mk: some value for dialogs only
17:51:23 <MichaelC> not sure as important for other contexts
17:51:54 <MichaelC> jn: wizard isn´t always in a dialog
17:51:59 <MichaelC> js: they don´t have roles...
17:52:19 <MichaelC> mk: unsure of value of that for screen reader users anyway
17:52:39 <MichaelC> I don´t trust default button, so I look around
17:52:49 <MichaelC> jn: because it doesn´t tell you, but if it did you might trust it
17:53:04 <MichaelC> present+ Jon
17:53:36 <MichaelC> mk: even so, not as high value for me
17:54:06 <MichaelC> jn: mostly agree
17:54:09 <MichaelC> it´s just a button attribute
17:54:21 <MichaelC> mk: could be boolean on button, forget about context
17:54:33 <MichaelC> then would need to drop language about author error
17:54:57 <joanie> q+ To say the value of the context in the case of the web is limiting the tree dive. That said, I don't see it as must-have.
17:55:12 <MichaelC> mc: there was some confusion, would a rephrasing on list help?
17:55:26 <MichaelC> mk: need clearer statement of goals for screen readers
17:55:51 <MichaelC> js: AX has a dialog, with a default button
17:56:06 <MichaelC> though the button itself doesn´t have anything to make it the default
17:56:24 <MichaelC> if you put aria-default on the button, the browser has to figure out what the context is that it´s the default for
17:56:46 <MichaelC> that´s a pattern they have opposed elsewhere, for aria-current
17:57:16 <joanie> q-
17:57:58 <MichaelC> mc: so now there are two things with similar problems
17:58:20 <MichaelC> js: difference is that with default button you need more information about what´s gonna get dismissed
17:58:44 <MichaelC> mk: @@
17:59:26 <MichaelC> mc: for aria-current, we considered aria-currentfor, but rejected it
17:59:50 <MichaelC> mk: is there a way of not having to go up the tree?
18:01:51 <MichaelC> just say we care about the button, is a default, less important to know what it´s for
18:02:24 <MichaelC> jn: sighted user doesn´t know, they infer
18:02:37 <MichaelC> mc: they infer pretty strongly
18:03:08 <MichaelC> jd: for end user, context isn´t that important
18:03:32 <MichaelC> but for AT, on some platforms, there´s a ¨tell me the default button¨ command
18:03:49 <MichaelC> having to go up the hierarchy isn´t so bad
18:03:58 <MichaelC> but having to go *down* the hierarchy is much more expensive
18:04:44 <MichaelC> user knows they´re in a dialog etc.
18:04:51 <MichaelC> but the application has a much wider context
18:05:01 <jamesn> q+
18:05:13 <MichaelC> mb: there are only so many things the default button could be in
18:05:36 <MichaelC> q+
18:05:53 <MichaelC> ack j
18:06:04 <MichaelC> jn: maybe this can be solved in ARIA 2.0 with selectors
18:06:23 <MichaelC> q+ to say take advantage of API
18:06:37 <MichaelC> js: author has to know what to select
18:06:40 <MichaelC> jn: they will know
18:06:42 <joanie> q+ To say that Matt's suggestion works *if* we say the primary button MUST be in the context of a dialog
18:06:49 <MichaelC> ack me
18:06:49 <Zakim> MichaelC, you wanted to say take advantage of API
18:08:16 <MichaelC> mc: even if we´re not as concerned about what AXAPI offers, is nice to support the greater richness where available
18:08:17 <MichaelC> ack j
18:08:17 <Zakim> joanie, you wanted to say that Matt's suggestion works *if* we say the primary button MUST be in the context of a dialog
18:08:28 <MichaelC> jd: Matt's suggestion works *if* we say the primary button MUST be in the context of a dialog
18:08:37 <jamesn> q+ to say wizards dont have role=dialog and cannot
18:08:58 <MichaelC> ack j
18:08:58 <Zakim> jamesn, you wanted to say wizards dont have role=dialog and cannot
18:09:09 <MichaelC> jn: can´t have wizards have role=dialog
18:09:25 <MichaelC> they have differences
18:09:32 <MichaelC> you can leave them etc.
18:10:35 <joanie> +1 to getting it right in 2.0
18:11:24 <MichaelC> mc: to summarize, sounds like support for doing well in ARIA 2.0 rather than badly in ARIA 1.1
18:11:30 <MichaelC> so adjust issues accordingly
18:13:00 <MichaelC> and inform the thread of this
18:13:18 <jamesn> zakim, next item
18:13:18 <Zakim> agendum 4. "Test planning (major topic for this call)" taken up [from MichaelC]
18:13:26 <MichaelC> scribe: jamesn
18:14:55 <jamesn> MC: Rich wanted in particular for MC to lead this discussion.
18:15:03 <jamesn> due to role in aria test harness
18:15:15 <jamesn> has been discussion in various other groups.
18:15:43 <MichaelC> -> https://rawgit.com/w3c/aria/CR-pub/aria/aria.html#sotd Draft exit criteria
18:16:27 <jamesn> for MUSTs in 1.0 needed a testable statement
18:16:35 <jamesn> for most part was role, states and properties
18:16:57 <jamesn> what we care about is that is makes its way to the API so we depended on the UAIG - now the core and other mappings
18:17:05 <jamesn> says for the feature - how it should look in the apis
18:17:17 <jamesn> sometimes simple, sometimes more complex
18:17:39 <Stefan> Stefan has joined #aria
18:17:47 <jamesn> so for ARIA testing means we need to test the mapping guides - then set some sontraints that need to find 2 implelmentations of each feature
18:18:08 <jamesn> for the mappings there may only be 1 implementation of a mapping - sometimes there is only 1 mapping for an API
18:18:16 <jamesn> yes - fred also test that
18:18:43 <jamesn> so find 2 instances of some mapping. so if find 1 on windows and 1 on mac then these are the 2 mappings
18:18:57 <jamesn> if cant find 2 on 2 platforms then look for 2 browsers on the same platform
18:19:11 <jamesn> that is a less preferred approach but did that for about 20 of the tests
18:19:39 <jamesn> so for a given role what you need is a testable statement which says here is a test of this role and says what you see in the AAPI
18:19:46 <MichaelC> present+ Stefan
18:20:07 <jamesn> and then need to test it. created minimal html files which had just enough to be valid
18:20:20 <jamesn> then the aria feature, put an id of test on it so could find easily
18:20:23 <jamesn> then look for the mapping
18:20:36 <jamesn> for 1.0 was entirely manual. that is where the test harmess comes in
18:21:00 <jamesn> the test harness gives the statement, the result and the file. would open, then use your inspector to look for the result
18:21:21 <jamesn> we have talked about automating. we believe the majority are automateable, but dont have a platofrm for it
18:21:46 <jamesn> we do know the different vendors have platforms but hey are not public. there are some tools that can be scripted
18:22:02 <jamesn> Joanie has done some experimentation with automation. she is hopeful
18:22:21 <jamesn> Q - how much do we invest in the manual testing and how much do we invest in the automation
18:22:28 <jamesn> want to come back to that too
18:23:28 <jamesn> mostly either roles states and properties table or MUST statements - acc name is part of that
18:23:32 <fesch> q+
18:24:10 <jamesn> "WAI-ARIA 1.1 builds upon WAI-ARIA 1.0, which met its implementation requirements in February 2014. Only features that are new or changed in WAI-ARIA 1.1 need be tested, as the WAI-ARIA 1.0 implementation report provides implementatibility and interoperability evidence for the remaining features. Features that will be tested for WAI-ARIA 1.1 include:"
18:24:38 <jamesn> Parts of the name computation algorithm may need to be tested but maybe not all of it
18:25:05 <jamesn> for if statements sometimes need a test file to test every one of them. also sometimes some negative test cases
18:25:19 <jamesn> if A+B then sometimes need failures
18:25:43 <jamesn> for key cases where a failure would represent an incorrect implementation then may need some failures too
18:25:47 <jamesn> q?
18:26:05 <jamesn> FE: how do you test something that is not supposed to be in the tree
18:26:18 <jamesn> MC: you say the a11y tree does not have such and such
18:26:41 <jamesn> FE: in SVG have xyz does not get into the tree
18:27:05 <jamesn> MC: id they element with id test should not show in the tree then the expected result is that id test doesn't appear
18:27:51 <jamesn> would suggest putting something that should not appear in the tree with a certain idref
18:27:55 <jamesn> FE: is there an automarted way?
18:28:00 <mck> q+
18:28:04 <jamesn> MC: I would think it could be automateable
18:28:07 <MichaelC> ack f
18:28:08 <MichaelC> ack m
18:28:14 <jamesn> MK: I missed which doc
18:28:24 <jamesn> MC: https://rawgit.com/w3c/aria/CR-pub/aria/aria.html#sotd
18:28:53 <mck> ack me
18:29:04 <jamesn> rrsagent, make minutes
18:29:04 <RRSAgent> I have made the request to generate http://www.w3.org/2015/12/10-aria-minutes.html jamesn
18:29:18 <cyns> q+
18:29:18 <jamesn> topic: automation
18:29:44 <jamesn> JD: in terms of automating have talked in terms of webdriver and DOM. not going to happen.
18:29:55 <jamesn> have a POC hackaround by which i can run my tests
18:30:07 <jamesn> as an aside - the way the hack works is that it does a tree dive
18:31:05 <MichaelC> NVDA has a python-based tool to poke at IA2
18:31:28 <MichaelC> I might be able to hack that to test IA12
18:31:35 <MichaelC> s/IA12/IA2/
18:31:53 <MichaelC> ack c
18:32:33 <jamesn> CS: so we are going to be automating our own implementation
18:32:47 <jamesn> I can share the results - but not certain about the code
18:33:21 <joanie> q+ to ask the obvious, if it's UI *Automation*, there is surely public/open code we could use to create our own testing on top of the MS testing.
18:33:23 <jamesn> if you can tell me what format can suck data in. In the ARIA 1.0 test harness could perhaps take data from elsewhere and put it into the harness
18:33:38 <jamesn> everything we are testing - i think the testable statements are the requirements
18:33:43 <jamesn> q+
18:34:06 <jamesn> MC: if we had a way of mapping tests to statements couold suck in the results
18:34:16 <jamesn> we can work on our preferred format
18:34:32 <jamesn> would you be using our test files or are you doing your own set
18:34:33 <fesch> q+ for asking about test file templates
18:34:49 <jamesn> CS: i suspect some of both
18:35:03 <jamesn> i suspect we will be doing a superset
18:35:18 <jamesn> MC: if your test files are different there is a potentially invalid situation
18:35:31 <jamesn> MC: my preference would be to use the same tests
18:35:54 <jamesn> MC: may mean among other things that joanie may want to look at other platforms than ui automation
18:36:20 <jamesn> in 1.0 decided needed to be HTML4 - likley need HTML5 and SVG files for 1.1
18:36:32 <jamesn> went to great pains than 1.1 tests were valid html4
18:36:59 <jamesn> still do need them to be well formed - minimize parsing artifacts. and that includes inline svg if needed
18:37:04 <MichaelC> ack next
18:37:05 <Zakim> joanie, you wanted to ask the obvious, if it's UI *Automation*, there is surely public/open code we could use to create our own testing on top of the MS testing.
18:37:38 <jamesn> JD: could use for edge perhaps
18:37:42 <jamesn> CS: yes that is true
18:37:59 <jamesn> CS: wasnt designed for web content
18:38:06 <jamesn> we will probably do that work
18:38:34 <jamesn> JD: if there is public stuff there i could use to hack on your platform using open stuff
18:38:52 <jamesn> JD: the ID attribute - on most platforms that is the case
18:39:05 <jamesn> CS: it is exposed if there is an implicit id on the platform
18:39:27 <MichaelC> ack next
18:40:04 <MichaelC> jn: in firefox you could build an extension that looks at the AAPI
18:40:09 <MichaelC> and should work on all its platforms
18:40:14 <MichaelC> and script that with selenium
18:40:38 <MichaelC> jd: we want to test what the AT sees
18:40:50 <MichaelC> not what the browser says it´s sending to the AAPI
18:41:11 <MichaelC> jn: it´s got an interface to the COM tree to show exactly what´s winding up in the AAPI
18:41:34 <MichaelC> jd: it´s talking to its own AAPI layer
18:41:43 <MichaelC> js: don´t know which it does
18:42:00 <MichaelC> jn: if I ask the role, it reports something different on different platforms
18:42:19 <MichaelC> jd: it says what it thinks it´s exposing, which varies by platform
18:43:18 <MichaelC> mc: should explore this separately...
18:43:25 <clown> s/an interface to the COM tree/an interface to the XPCOM information/
18:43:54 <MichaelC> jg: if you have sample extension, we could explore making something work
18:44:29 <MichaelC> jn: looks like it comes from accessibility retrieval interface
18:44:36 <MichaelC> ack next
18:44:37 <Zakim> fesch, you wanted to discuss asking about test file templates
18:44:51 <jamesn> FE: couple of questions about the template
18:45:03 <jamesn> can it be svg files or can it be html with svg inline
18:45:21 <jamesn> want to make sure that we have consistent test files which could introduce artifcats
18:45:41 <jamesn> s/want to make sure/MC: want to make sure/
18:45:51 <jamesn> MC: best to pick 1 approach or another
18:46:01 <clown> q+ to ask if a difference is expected between svg file and svg-in-html file?
18:46:05 <jamesn> MC: dont want to go into too much detail about structure of the test files
18:46:12 <MichaelC> -> https://github.com/w3c/aria/tree/master/testfiles/1.0 ARIA 1.0 test files
18:46:18 <jamesn> the test files for 1.0 are in above
18:46:29 <jamesn> grouped into folders as 800 or so
18:46:36 <jamesn> lots are auto generated
18:46:59 <jamesn> testing a label when there is a menu role
18:47:44 <jamesn> the label should come out based on the combo box
18:47:52 <jamesn> in the test harness should see as correct
18:48:05 <jamesn> q+
18:48:15 <jamesn> would suggest the same with the svg
18:49:03 <jamesn> JN: shouldn't there be something in the file for what the result should be?
18:49:13 <jamesn> MK: think it should all be in 1 file
18:49:25 <jamesn> MC: there are 2 reasons they are not in the file
18:49:40 <jamesn> even in comments could perhaps influence the results - could make it hard to debug
18:49:58 <jamesn> another reason is that there are 5 or 6 different results according to the platform
18:50:11 <jamesn> JD proposal does have them in the test file - we may be exploring that
18:50:32 <jamesn> i am inclined to say we might want versions without and then with results in them for the automation tool
18:50:34 <jamesn> ack me
18:50:57 <jamesn> MK: certainly wouldn't hurt to have the textual description of the result in the file
18:51:11 <jamesn> clown: in some cases use the same test file for different cases
18:51:22 <jamesn> MK: would rather duplicate the file
18:51:27 <jamesn> MC: link to test harness
18:53:41 <jamesn> MC: the test harness where there is a ref to what are testing
18:53:52 <jamesn> MC: there is a single expected result on all platforms
18:54:02 <jamesn> sometimes different for different platforms
18:54:30 <jamesn> i think probably need to organize a training on the test harness - if havent used it before then lets do a training
18:54:40 <jamesn> should schedule supplementary training
18:56:34 <jamesn> MC: will do a poll will send to aria mailing list i think
18:57:09 <jamesn> will need to test dpub roles and graphics roles too
18:57:29 <jamesn> i think so - will discuss in the svg a11y group
18:57:43 <jamesn> s/i think so/FE: i think so/
18:58:13 <jamesn> MC: i know the a11y name computation has some host language stuff. some you might rely on existsing stuff - but others are svg specific
18:58:30 <jamesn> dont feel that have got enough on testing to add action items for over the holidays
18:58:45 <jamesn> rrsagent, make minutes
18:58:45 <RRSAgent> I have made the request to generate http://www.w3.org/2015/12/10-aria-minutes.html jamesn
18:59:05 <jamesn> FE: we plan on using these files and this format of test harness and test files
18:59:32 <jamesn> MC: so want test statements and expected results. if we dont build the test cases etc we wont be wasting time
18:59:44 <jamesn> when to use the exact test harness is an open question
18:59:58 <jamesn> no one loves the harness but it has the advantage that it exists ;)
19:00:10 <jamesn> will be using for reporting
19:00:16 <MichaelC> -> https://www.w3.org/WAI/PF/testharness/testreport?testsuite_id=1 ARIA 1.0 test report
19:00:38 <jamesn> will be a group for svg and others for 1.1
19:00:40 <jamesn> etc.
19:00:55 <jamesn> if import tests from other locations need the report
19:01:20 <clown> ack me
19:01:20 <Zakim> clown, you wanted to ask if a difference is expected between svg file and svg-in-html file?
19:01:57 <MichaelC> rrsagent, make minutes
19:01:57 <RRSAgent> I have made the request to generate http://www.w3.org/2015/12/10-aria-minutes.html MichaelC
20:24:43 <clown> clown has joined #aria
20:50:47 <MichaelC> rrsagent, bye
20:50:47 <RRSAgent> I see no action items
20:50:47 <MichaelC> zakim, bye
20:50:47 <Zakim> leaving.  As of this point the attendees have been Joanmarie_Diggs, MichaelC, Janina, Suman, Fred, Cynthia, Michiel, JamesNurthen, matt_king, Joseph, Joseph_Scheuhammer, Bryan,
20:50:47 <Zakim> Zakim has left #aria