21:04:00 <RRSAgent> RRSAgent has joined #aria-test
21:04:00 <RRSAgent> logging to http://www.w3.org/2016/07/13-aria-test-irc
21:04:02 <trackbot> RRSAgent, make logs world
21:04:02 <Zakim> Zakim has joined #aria-test
21:04:04 <trackbot> Zakim, this will be
21:04:04 <Zakim> I don't understand 'this will be', trackbot
21:04:05 <trackbot> Meeting: Accessible Rich Internet Applications Working Group Teleconference
21:04:05 <trackbot> Date: 13 July 2016
21:04:06 <jongund_> jongund_ has joined #aria-test
21:04:21 <jongund_> Hi Joanie
21:04:28 <ShaneM> Chair: Jon Gunderson
21:05:10 <ShaneM> present+ ShaneM
21:05:17 <ShaneM> present+ Cyns
21:05:22 <jongund_> present+ jongund_
21:06:06 <ShaneM> ShaneM has changed the topic to: ARIA Testing meeting  647 543 114
21:06:12 <jongund_> aria
21:06:15 <jongund_> is the code
21:06:37 <ShaneM> s/aria//
21:07:18 <joanie> present+ Joanmarie_Diggs
21:08:18 <ShaneM> TOPIC: What we are doing with test automation for ARIA
21:08:25 <ShaneM> Some notes at wiki: https://www.w3.org/wiki/ARIA_1.1_Automated_Testing
21:10:10 <Rich> Rich has joined #aria-test
21:10:13 <ShaneM> present+ Rich
21:10:40 <dbrett> https://github.com/MicrosoftEdge/A11y
21:15:10 <jongund_> Shane: The WPT can hold the tests
21:15:40 <jongund_> Shane: How it interfaces with AT, my assumption was acting as an AT
21:15:48 <jongund_> David: yes
21:16:31 <jongund_> Shane: The concept is to make a channel communication between AT and the test framework
21:16:51 <jongund_> Shane: The WPT has a framework for reporting to W3C
21:17:12 <jongund_> CS: The W3C has a report framework
21:17:40 <jongund_> CS: A way to communicate the AT and the test page with some Javascript
21:18:02 <jongund_> CS: The information can then be passed/fail
21:18:18 <jongund_> CS: We output CSV
21:18:42 <jongund_> Shane: There are security issues with local file system, browser have work arounds
21:19:11 <jongund_> Shane: Providing the fake AT a URL handler with the results
21:19:27 <jongund_> Shane: Pass the data in any useful format
21:19:47 <jongund_> Shane: Goal is to inspect the result automatically
21:20:11 <jongund_> Shane: We want the tests to be run automatically during normal browser development testing
21:20:22 <jongund_> CS: We care about more than CR
21:20:37 <jongund_> CS: We don't need W3C CR reports everytime
21:20:55 <jongund_> Shane: The webkit team uses this, the data is more than just reports
21:21:17 <jongund_> Shane: Edge maybe using the WPT already, they are in the IRC channel
21:21:22 <jongund_> CS: Somebody
21:21:43 <jongund_> Shane: Joanie has been thinking of this for other APIs
21:21:51 <jongund_> JD: Narrow it down for me
21:22:00 <jongund_> Shane: Linux
21:22:09 <jongund_> JD: Yes, accessibility APIs
21:22:37 <jongund_> JD: Each platform would provide their own API
21:22:45 <jongund_> CS: Can you try to do this
21:23:07 <jongund_> Shane: David how do you feel about the code, can it send code
21:23:18 <jongund_> David: Yes I think it could be extended
21:24:13 <jongund_> Shane: It is strictly browser testing, no capability in turning on the AT connector
21:24:49 <jongund_> Shane: You would need to start the AT emulator, then the browser
21:25:05 <jongund_> Shane: Does AT attach to a window or how does it work
21:25:34 <jongund_> David: The AT looks at the Accessibility API to find edge
21:25:58 <jongund_> Shane: If there is a named window the AT could find the right window
21:26:26 <jongund_> David: You can just start with the top node and then start looking for the right window
21:26:47 <jongund_> JD: One event we get is a document load event, you can key off that event
21:27:06 <jongund_> Shane: In WPT test is a load, so there would be a load event
21:27:42 <jongund_> JD: That is like ideal, the first thing you need is the document and the event tells you it ready
21:27:59 <jongund_> RS: One question is there an MSAA?
21:28:16 <ShaneM> rrsagent, make minutes
21:28:16 <RRSAgent> I have made the request to generate http://www.w3.org/2016/07/13-aria-test-minutes.html ShaneM
21:28:27 <jongund_> David: There was initially, but we didn't create all the tests for MSAA
21:28:38 <jongund_> David: There are old commits
21:29:05 <jongund_> JG: The WPT is feasible?
21:29:41 <jongund_> Shane: Let's talk about ARIA 1.0, how many tests?
21:29:51 <jongund_> RS: ~900
21:30:07 <jongund_> Shane: They were generally automatically generated?
21:30:30 <jongund_> CS: The expected results though where human readible
21:30:53 <jongund_> CS: I was planning on working on a format
21:31:11 <jongund_> RS: Some tests required a time delay, did you allow for that
21:31:19 <jongund_> David: Yes, we can handle waiting
21:31:39 <jongund_> CS: We will start with the simpler tests
21:32:21 <jongund_> Shane: WPT and its test runner framework, has an interesting async test module, based on events, so it can handle delays
21:32:52 <jongund_> Shane: The battery test API is an example, so you have to wait for the battery percentage changes
21:33:13 <jongund_> Shane: Let's not reinvent
21:33:30 <jongund_> CS: The format on the wiki looks OK, but is too simple
21:34:02 <jongund_> RS: You doing a comparison with a predicted states and properties, can you test events
21:34:27 <jongund_> CS: Is there anything else other than live regions
21:34:34 <jongund_> CS: Can we test events?
21:34:43 <jongund_> David: It is possible
21:34:57 <jongund_> Shane: How do events get exposed?
21:35:16 <jongund_> CS: There are accessibility API event and then start plateform events
21:35:24 <Rich> http://rawgit.com/w3c/aria/master/core-aam/core-aam.html#mapping_events
21:35:27 <jongund_> RS: State change events...
21:35:58 <jongund_> RS: You have text remove..., live regions...., there are special ones for UIA
21:36:20 <jongund_> RS: MSAA/IA2 they are pretty standard
21:36:44 <jongund_> CS: It is a feature, but there are many other things more important
21:36:47 <jongund_> RS: Agree
21:36:59 <jongund_> CS: We can work on that in the future
21:37:13 <jongund_> RS: They are pretty simple tests
21:38:08 <jongund_> JD: Event testing should be easy, if we are listening for events for knowing when to check, then we are already testing the events
21:38:56 <jongund_> Shane: There are 2 kinds of test, declarative states and properties, there are more complex tests that evaluates ....
21:39:16 <jongund_> Shane: It sounds like the current test only sends back one file
21:39:48 <jongund_> CS: Event testing will be a different kind of test
21:40:06 <ShaneM> David: There is a way to get the data back iteratively.
21:40:14 <jongund_> JD: I will be automating the even test, I think this will be easy
21:40:42 <jongund_> JD: If this is going to change the harness, then that would be a priority
21:41:07 <jongund_> David: DIfferent branch for each browser and platform
21:41:39 <jongund_> JD: We need to figure out exactly needs to be done
21:42:28 <jongund_> Shane: SpecOps has some resources for this, what I can volunteer to do is get the infrastructure in place
21:43:04 <jongund_> Shane: If we set this up so there is one click to start, if the test requires a change .....
21:43:30 <jongund_> Shane: The low hanging fruit is where we should start
21:43:45 <jongund_> CS: We can do some interaction testing, but that is more work
21:44:06 <jongund_> Shane: If you are using web drive it works across browsers
21:44:19 <jongund_> David: It is just navigate and open
21:44:44 <jongund_> CS: We tested the date controls, but are more work to write
21:48:25 <jongund_> Shane: WPT is more than reporting, it will also a runner framework
21:58:53 <jongund_> CS: I will be looking at test cases, and port them to the fields
21:59:24 <jongund_> Shane: RS put a pointer in to core AAM
21:59:50 <jongund_> Shane: Aren't the assertions in the document
22:00:23 <jongund_> JD: We have a chicken and egg issue
22:00:39 <jongund_> RS: ARIA 1.1 process to CR ....
22:00:59 <jongund_> JG: Next call in two weeks
22:01:21 <jongund_> Shane: We should be looking at next time having this machine extractable
22:02:06 <jongund_> JD: Is what the core AAM does is ... the checkbox is checked is is different on each platform
22:03:55 <ShaneM> ACTION: ShaneM to create sample wrapper to talk with a fake AT under WPT by 7/27
22:03:56 <trackbot> Created ACTION-2095 - Create sample wrapper to talk with a fake at under wpt by 7/27 [on Shane McCarron - due 2016-07-20].
22:04:22 <cyns> ATK connector (Joanie) IA2 connector AX Connector Event testing (Joanie) Reporting with WPT (Shane) Document test case format (Cyns) Parsing JSON test case format (Brett) Create tests (working group)
22:05:44 <ShaneM> rrsagent, make minutes
22:05:44 <RRSAgent> I have made the request to generate http://www.w3.org/2016/07/13-aria-test-minutes.html ShaneM
22:06:15 <joanie> joanie has left #aria-test
22:06:26 <jongund_> ACTON: joanie ATK connector
22:07:04 <jongund_> ACTION: joanie ATK connector
22:07:04 <trackbot> Created ACTION-2096 - Atk connector [on Joanmarie Diggs - due 2016-07-20].
22:07:22 <jongund_> ACTION: joanie IA2 connector AX Connector Event testing
22:07:22 <trackbot> Created ACTION-2097 - Ia2 connector ax connector event testing [on Joanmarie Diggs - due 2016-07-20].
22:07:47 <jongund_> ACTION: cyns Document test case format
22:07:48 <trackbot> Created ACTION-2098 - Document test case format [on Cynthia Shelly - due 2016-07-20].
22:08:37 <jongund_> ACTION: cyns David Brett will work on parsing JSON test case format
22:08:37 <trackbot> Created ACTION-2099 - David brett will work on parsing json test case format [on Cynthia Shelly - due 2016-07-20].
22:08:49 <jongund_> rrsagent, draft minutes
22:08:49 <RRSAgent> I have made the request to generate http://www.w3.org/2016/07/13-aria-test-minutes.html jongund_
22:24:11 <jongund_> jongund_ has joined #aria-test
22:28:38 <Rich> Rich has joined #aria-test
23:51:28 <Zakim> Zakim has left #aria-test