15:03:26 RRSAgent has joined #aria 15:03:26 logging to http://www.w3.org/2016/02/29-aria-irc 15:03:28 RRSAgent, make logs world 15:03:28 Zakim has joined #aria 15:03:30 Zakim, this will be 15:03:30 I don't understand 'this will be', trackbot 15:03:31 Meeting: Accessible Rich Internet Applications Working Group Teleconference 15:03:31 Date: 29 February 2016 15:03:46 Present+ JF 15:04:05 Meeting: W3C ARIA Working Group - Test Coordination call 15:04:11 present+ Joanmarie_Diggs 15:04:33 clown has joined #aria 15:04:40 what's the pass word 15:04:47 present+ Rich_Schwerdtfeger 15:05:29 its not letting me in 15:05:40 present+ fesch 15:05:55 scribe: JF 15:05:55 present+ Jossph)Scheuhammer 15:06:05 I tried the "aria" password 15:06:08 https://lists.w3.org/Archives/Public/public-aria-test/2016Feb/0002.html 15:06:12 present- Jossph)Scheuhammer 15:06:17 present+ Joseph_Scheuhammer 15:06:18 jemmaku has joined #aria 15:06:35 RS: Goals is to review what is happening due to new members coming into this effort 15:06:44 Need to discuss exit Criteria 15:06:53 all we test is what has changed since ARIA 1.- 15:06:55 https://rawgit.com/w3c/aria/CR-pub/aria/aria.html#sotd 15:06:57 present+ AmeliaBR 15:07:02 TOPIC: Exit Criteria 15:07:21 s/1.-/1.0 15:07:33 present+ JaeunJemmaku 15:07:40 MC: Exit CRiteria is what we propose so that we can move forward 15:07:55 requires 2 implementations of each to prove interop 15:08:14 chose to take a slightly different path with 1.1, to avoid having to test all of 1.0 15:08:40 also, success is proven via the mappings to Accessibility API, so often only one 15:08:56 so, we need to list what is new, and what needs to be tested 15:09:32 RS: we also need to talk about the core mapping spec 15:10:01 MC: what we've done is to test to see if the feature can be exposed in the AAPI mapping guide 15:10:56 The wording in the exit criteria is inteneded to be flexible 15:11:18 MC: mappings themselves may be more of a challenge to get past CR 15:11:56 q? 15:12:32 CS: questions about the mappings - they have changed a fair bit since 1.0 - will we be testing those as well? 15:12:46 q+ to note we didn't change UIA last time 15:12:56 MC: for ARIA we need to test changes to the mappings 15:13:10 CS: needs to test as part of the products as well 15:13:17 present+ MichaelC 15:13:20 present+ Cynthia 15:13:31 CS: also, there is a request to keep UIExpress - that may or may not have an impact 15:14:03 RS: two issues: only FF is still using UIExpress, even MS has moved on 15:14:29 other issues is the request has com e quite late, after WG removed previously 15:14:51 Q? 15:15:26 q? 15:16:44 RS: other issue for those who are new, need to explain the importance of mappingts 15:17:07 Browsers are a software application in terms of a11y 15:17:25 just like other native a11y tools, browsers support the AAPIs 15:17:51 the reason we test for the mappings, so that we can be sure that what we put in our content is "talking" to the AAPI as well 15:18:06 so, we need to edetermine skillsets 15:18:29 if you cannot test the mappings, you will need to learn to use the tools 15:18:46 else you will need to test against the samples, or wrtie the samples 15:18:55 https://www.w3.org/WAI/PF/testharness/ 15:19:01 RS: other important thing is the test harness 15:19:15 need to differentiate between 1.0 and 1.1 15:19:59 https://www.w3.org/WAI/PF/testharness/testcases?testsuite_id=4 15:19:59 ARIA 1.1 is pointing to core AAPI mappings 15:20:02 JohnJansen has joined #aria 15:20:25 test cases are testable statements 15:20:40 for each testable statement, we need a simple test case 15:21:28 review test suite, and pick a test, click edit 15:21:38 https://github.com/w3c/aria/tree/master/testfiles 15:21:41 you will see the anatomy of a test 15:22:04 requires access to GitHub, as that is where everything is stored - see Michael Cooper for access (etc.) 15:22:07 http://rawgit.com/w3c/aria/master/accname-aam/accname-aam.html 15:22:17 To get push access to the repository, send me your github username (and description of your role if I don´t know you) 15:22:48 http://rawgit.com/w3c/aria/master/core-aam/core-aam.html 15:23:11 http://rawgit.com/w3c/aria/master/core-aam/core-aam.html#mapping_role_table 15:23:15 RS: in the mapping spec, scrolling down you will see a key part - role and state mapping tables 15:23:18 http://rawgit.com/w3c/aria/master/core-aam/core-aam.html#mapping_role_table 15:23:39 click on "View as a single table" you will see the mappings for each platvorm 15:23:56 s/platvorm/platform 15:24:49 RS: subtle differences between 1.0 and 1.1 15:25:50 big change in 1.1 is that if you are using a native host language... say in 1.0 if you were going to build a grid, you had to declare all of the semantics because HTML4.1 assumed no native semantics 15:26:06 but we are testing agaisnt HTML5, which has native host semantics 15:26:25 we also have a full automation test 15:27:08 CS: Edge is using UIAutomation instead of UIExpress 15:27:30 RS: this is part of automated testing efforts of the softwre 15:27:30 For comparison: https://www.w3.org/TR/wai-aria-implementation/#mapping_role_table 15:28:17 RS: This time we will be focusing on testing Edge (and not IE) 15:28:30 RS: we also need to determine what can be automated 15:28:50 FE: name and description calculations are fairly easy to automated 15:29:01 but roles will be more complex 15:29:21 FE: in SVG, automating everything, but it is ugly 15:29:37 RS: ARIA applies to both HTML and SVG, and both have integrated ARIA 15:29:53 so one question is how we divvy up the work between the HTML effort and SVG 15:30:05 this is not an exit criteria for ARIA, but it will be for SVG 15:30:32 JJ: will be automating testing in Edge 15:30:47 likely build a new UI client and use web driver to run the test 15:31:18 will inject between the browser and a regular accessibility tree using UIA 15:31:51 tools can also point at a git repo to get the platform tests, then navigate through them using web driver to navigate against the cases 15:32:25 RS: one goal for ARIA 2.0 is to merge towards a common APIs 15:32:38 so that it doesn't matter if it is native or web, common infrastructure 15:32:53 this is what we'd like to see, instead of platform specific testing 15:33:53 [discussion on automated versus manual tests for ORCA [sic]) 15:34:12 mc: A TEST CASE HAS MULTIPLE COMPONENTS 15:34:28 looking at first test case in repository 15:34:33 q? 15:34:51 ack me 15:34:51 clown, you wanted to note we didn't change UIA last time 15:35:04 there is meta data, then testable statements (minimal and generated, focused on a very narror issue) 15:35:30 q+ to ask how reverse relationships are tested 15:35:35 then test file, is the 'executable' code that you can run the test on, in the case of the test harness it is manual, but it can be automated 15:36:04 then there is the expected results, which is done by platform -often the same, but also possible to be different by platform 15:36:24 each test, or groups of tests, can have different requirements 15:36:53 when running the tests in the test harness, if there is an expected result for a specific platform it will show that, else the general requirement 15:37:24 RS: one question, where do folks feel they have gaps (need education)? 15:37:38 do you want to limit your testss to a certain area? 15:37:57 we likely need to write the testable statements first, then write the test case(s) 15:38:29 does everyone know how to use the accessiblity test tools on the different platforms 15:38:53 q+ To ask if it makes sense to divide according to platform 15:39:10 JG: have recruited some folks, it would be good to have a training session on using the tools 15:39:21 also, how many folks do you need, and where? 15:39:49 +1 to training sessions on the tools 15:40:46 RS: we will need some help with testing WVG as well 15:40:53 s/WVG/SVG 15:41:23 ?me + 1 to joanies thinking ;) 15:41:26 https://www.w3.org/wiki/SVG_Accessibility/Testing/Test_Assertions 15:41:41 s/?me + 1 to joanies thinking ;)//me + 1 to joanies thinking ;) 15:43:02 [discussion around reverse mappings] 15:44:01 http://rawgit.com/w3c/aria/master/core-aam/core-aam.html#mapping_additional_relations_reverse_relations 15:48:31 JF: Deque folks could use some tools traingin as well. Skills in writing testable statements, and code samples, etc. 15:48:41 q? 15:48:56 ack fesh 15:49:02 acl fesch 15:49:03 ack fes 15:49:03 fesch, you wanted to ask how reverse relationships are tested 15:49:12 ack joa 15:49:12 joanie, you wanted to ask if it makes sense to divide according to platform 15:49:16 ack joanie 15:49:32 Should we be dividing things up by platform? 15:49:50 assume everyone gets a piece to check, and then we run automated tests, etc. 15:50:10 so perhaps we need to divide things up based on expertese 15:50:34 Joanie: but writing HTML is not a strong suite 15:50:43 so perhaps divide based on skill sets 15:51:12 RS: looking for people to coordinate this effort 15:51:18 perhaps Jon G and Michael? 15:52:54 Jon, Jemma and Michael to coordinate 15:54:36 CS: I can assist with the more esoteric test cases 15:54:47 JG: wonders if the test harness is fully functional today 15:55:00 https://www.w3.org/WAI/PF/testharness/testcases/edit?testsuite_id=4&testcase_id=74 15:55:03 JG: seems that some of the URLs etc are not hooked up correctly 15:55:48 MC: you found a bug, this should be easy to fix 15:56:51 what is the time frame to finish the testing? 15:57:07 RS: Send a note to Jon Jemma and Michael on team s to write testable statements, etc. 15:57:32 RS: heard a need for a refresher on test tools 15:57:43 JF: Deque team would probably need 15:57:44 +1 on getting update on test tools 15:57:54 +q 15:58:55 [discussion on that training - use a webex or similar] 15:59:07 JG we can record this for future use 15:59:24 JG can also use Blackboard to coolaborate 15:59:55 mgylling has joined #aria 16:01:40 CS: if you are at CSUN, we can meet up then for some in-person 16:02:28 next meeting scheduled for March 28 (post CSUN) 16:03:05 CS: next call would like to discuss more on automation 16:04:28 RRSAgent, make logs public 16:04:41 trackbot, end meeting 16:04:41 Zakim, list attendees 16:04:41 As of this point the attendees have been JF, Joanmarie_Diggs, Rich_Schwerdtfeger, fesch, Jossph)Scheuhammer, Joseph_Scheuhammer, AmeliaBR, JaeunJemmaku, MichaelC, Cynthia 16:04:49 RRSAgent, please draft minutes 16:04:49 I have made the request to generate http://www.w3.org/2016/02/29-aria-minutes.html trackbot 16:04:50 RRSAgent, bye 16:04:50 I see no action items