IRC log of aria on 2017-05-18
Timestamps are in UTC.
- 16:48:26 [RRSAgent]
- RRSAgent has joined #aria
- 16:48:26 [RRSAgent]
- logging to http://www.w3.org/2017/05/18-aria-irc
- 16:48:28 [trackbot]
- RRSAgent, make logs world
- 16:48:31 [trackbot]
- Zakim, this will be
- 16:48:31 [Zakim]
- I don't understand 'this will be', trackbot
- 16:48:31 [trackbot]
- Meeting: Accessible Rich Internet Applications Working Group Teleconference
- 16:48:31 [trackbot]
- Date: 18 May 2017
- 16:48:35 [joanie]
- chair: Joanmarie_Diggs
- 16:48:35 [joanie]
- RRSAgent, make log public
- 16:48:37 [joanie]
- agenda?
- 16:48:48 [joanie]
- clear agenda
- 16:48:53 [joanie]
- agenda: this
- 16:49:07 [joanie]
- agenda+ Instructions on how to get started with manual testing
- 16:49:07 [joanie]
- agenda+ ARIA 1.1 "actions / status" check in
- 16:49:10 [joanie]
- agenda: be done
- 16:49:16 [joanie]
- agenda?
- 16:49:29 [joanie]
- Regrets: Michiel_Bijl, Rich_Schwerdtfeger, Jon_Gunderson
- 16:49:41 [joanie]
- scribeOptions: -final
- 16:57:16 [JasonDuan]
- JasonDuan has joined #aria
- 16:57:51 [joanie]
- present+ Joanmarie_Diggs
- 17:00:04 [tzviya]
- present+
- 17:01:38 [joanie]
- scribe: joanie
- 17:01:58 [joanie]
- Zakim, take up item 1
- 17:01:58 [Zakim]
- agendum 1. "Instructions on how to get started with manual testing" taken up [from joanie]
- 17:02:26 [joanie]
- Link to documentation: https://www.w3.org/wiki/ARIA_1.1_Manual_Testing
- 17:02:41 [MichaelC]
- present+
- 17:03:03 [JasonDuan]
- JasonDuan has joined #aria
- 17:03:41 [Stefan]
- Stefan has joined #aria
- 17:04:27 [jcraig]
- jcraig has joined #aria
- 17:05:01 [joanie]
- https://www.w3.org/wiki/ARIA_1.1_Manual_Testing
- 17:05:24 [joanie]
- ARIA 1.1 implementations in report: https://w3c.github.io/test-results/wai-aria/#index-of-implementations-in-reports
- 17:05:33 [tzviya]
- https://w3c.github.io/test-results/dpub-aam/#index-of-implementations-in-reports
- 17:05:46 [MichaelC]
- scribe: MichaelC
- 17:06:01 [mck]
- mck has joined #aria
- 17:06:10 [MichaelC]
- jd: above applies to ARIA and DPub
- 17:06:23 [MichaelC]
- swapping dpub-aam for wai-aria in URIs
- 17:06:43 [MichaelC]
- includes contact info of people working on platforms
- 17:06:55 [MichaelC]
- I´ve done on some non-Windows systems
- 17:07:14 [MichaelC]
- Jon working on IA2 testing, hopes to be ready by next week
- 17:07:21 [MichaelC]
- but we still don´t have a plan for UIA testing
- 17:07:28 [MichaelC]
- so at the moment, manual testing needed on that platform
- 17:07:36 [mck]
- present+ matt_king
- 17:07:39 [MichaelC]
- MS has said they plan to do it; I´ve sent the instructions
- 17:08:19 [MichaelC]
- if you get to the point you don´t have implementation data, may need to do manual testing, since the automated is happening
- 17:08:39 [joanie]
- https://w3c.github.io/test-results/wai-aria/all.html
- 17:08:48 [MichaelC]
- I´m running automated tests for ATK and AXAPI, 2 UAs each
- 17:08:55 [joanie]
- https://w3c.github.io/test-results/dpub-aam/all.html
- 17:09:03 [MichaelC]
- above links are test reports for ARIA and DPub
- 17:09:48 [MichaelC]
- The headers in the table are explained in the index of implementations page
- 17:10:20 [MichaelC]
- the two WebKit implementations show mostly passing results
- 17:10:29 [MichaelC]
- the other two mostly failures
- 17:10:45 [MichaelC]
- that doesn´t indicate manual testing needed - it´s valid data about not passing at the moment
- 17:10:53 [MichaelC]
- it´s lack of data that needs manual testing
- 17:10:56 [joanie]
- https://w3c.github.io/test-results/wai-aria/all.html
- 17:11:22 [MichaelC]
- in above link, fields with orangey-yellow lack data
- 17:11:36 [MichaelC]
- there are a few reasons this can happen
- 17:12:34 [MichaelC]
- we added tests but didn´t update report, or the test isn´t applicable to that platform, or we simply lack a test
- 17:14:03 [MichaelC]
- mc: can we distinguish don´t need data from don´t have but need?
- 17:14:23 [MichaelC]
- jd: documentation page gives pointers to figure that out
- 17:14:31 [MichaelC]
- you can usually figure it out
- 17:14:32 [MichaelC]
- q+
- 17:15:11 [MichaelC]
- q+ to talk about what we tell Director, and if we can get additional info here
- 17:15:48 [MichaelC]
- often, when there´s an additional test file under a feature, it´s to test additional platform-specific requirements
- 17:16:12 [tzviya]
- q+ to ask why I get permission denied messages when I click through
- 17:16:50 [MichaelC]
- ack me
- 17:16:51 [Zakim]
- MichaelC, you wanted to talk about what we tell Director, and if we can get additional info here
- 17:17:12 [MichaelC]
- mc: Director will want to easily see difference
- 17:17:30 [MichaelC]
- also would be good to add more clear ¨key¨ information
- 17:17:47 [MichaelC]
- jd: requires test runner upgrade, I can approach Shane about how to get that made
- 17:18:44 [MichaelC]
- jd: for ARIA 1.1 we added some things to existing roles that weren´t requirements in 1.1
- 17:19:24 [MichaelC]
- for new roles we just test the whole role
- 17:19:35 [MichaelC]
- for changes, we kinda need to test the new and old behavior
- 17:19:52 [MichaelC]
- not sure if we´ll be able to do that automatically, so that could add a small set of new manual tests needed
- 17:20:02 [MichaelC]
- though my goal is still to automated it
- 17:20:13 [MichaelC]
- there are only 10 anyways
- 17:20:15 [MichaelC]
- q?
- 17:20:24 [MichaelC]
- ack t
- 17:20:24 [Zakim]
- tzviya, you wanted to ask why I get permission denied messages when I click through
- 17:21:00 [MichaelC]
- ts: when I follow links, i get ¨permission denied¨ errors
- 17:21:30 [joanie]
- https://www.w3.org/wiki/ARIA_1.1_Manual_Testing
- 17:21:58 [MichaelC]
- jd: when you´re doing manual testing you shouldn´t encounter this
- 17:22:03 [MichaelC]
- but it is something I plan to fix
- 17:22:03 [joanie]
- https://w3c-test.org/tools/runner/index.html
- 17:22:21 [MichaelC]
- this is the test runner
- 17:22:43 [MichaelC]
- put the shortname of the spec you want to test in the field (wai-aria or dpub-aam), with a starting slash
- 17:23:46 [MichaelC]
- can take a while to load after that, then shows test case with a table
- 17:24:33 [MichaelC]
- each test case is pretty minimalist
- 17:24:49 [MichaelC]
- the table essentially says what you´re looking for, in the various APIs
- 17:25:11 [MichaelC]
- you need to use an AAPI inspector for the platform
- 17:25:19 [MichaelC]
- AccProbe for IA2?
- 17:25:46 [MichaelC]
- look through the a11y tree to find the element called out in the test, and check its properties
- 17:26:15 [MichaelC]
- see if it´s got the thing, or property has the value, the table says it should have
- 17:26:27 [MichaelC]
- then go back to main window and choose ¨pass¨ or ¨fail¨
- 17:27:01 [MichaelC]
- then it will show the next test
- 17:27:11 [MichaelC]
- so you rinse and repeat
- 17:27:11 [MichaelC]
- q+
- 17:27:23 [MichaelC]
- q+ to ask how you pause and resume a session
- 17:27:45 [MichaelC]
- be sure not to close the test pop-up window, the test runner really wants it
- 17:28:44 [MichaelC]
- ack me
- 17:28:44 [Zakim]
- MichaelC, you wanted to ask how you pause and resume a session
- 17:29:10 [MichaelC]
- mc: can I pause a session and pick up where I left off later? e.g., after my browser has restarted so I didn´t just leave it open
- 17:29:25 [MichaelC]
- jd: you can walk away from tests and come back, the harness will just sit there waiting on you
- 17:30:21 [MichaelC]
- you can also select a subset of the tests at the start of the process
- 17:31:27 [MichaelC]
- you can use regex, but more simply use prefix
- 17:33:10 [MichaelC]
- you can get a JSON file and send it to me to incorporate into the test results repo
- 17:33:26 [MichaelC]
- ts: for DPub-AAM, what´s still to test?
- 17:33:29 [MichaelC]
- jd: Windows
- 17:33:36 [MichaelC]
- ts: so there´s not a column for it yet?
- 17:34:57 [MichaelC]
- jd: right, only the UAs listed are known to the system yet
- 17:35:03 [MichaelC]
- so Windows isn´t included yet
- 17:35:19 [MichaelC]
- mc: can we have placeholder columns?
- 17:35:29 [MichaelC]
- jd: yes, after a code review
- 17:35:54 [MichaelC]
- I haven´t because I don´t like fully empty info
- 17:36:22 [MichaelC]
- mc: think an empty column is more clear for non-testing reviewers
- 17:37:43 [MichaelC]
- jd: we´re still not fully sure of MS plans
- 17:37:51 [MichaelC]
- tz: have access to various systems
- 17:38:10 [MichaelC]
- jd: Firefox on Windows might be most useful place for you to start
- 17:39:13 [MichaelC]
- suggest you test a nightly build, so you have the latest implementation
- 17:39:35 [MichaelC]
- since features are landing as we speak
- 17:39:48 [MichaelC]
- we are hoping MS will take care of Edge
- 17:39:56 [MichaelC]
- we can more easily get nightlies of FF
- 17:41:10 [MichaelC]
- also think you´ll get similar results to FF on Linux
- 17:41:14 [MichaelC]
- Linux is the redheaded stepchild of the accessibility world
- 17:41:56 [MichaelC]
- that´ll help you sanity check your process
- 17:43:13 [MichaelC]
- tz: will look through this and come back with questions
- 17:44:19 [MichaelC]
- jd: I don´t have Windows, except on a virtual machine in another country, so will appreciate additional guidance in the wiki about that platform if you have it
- 17:51:00 [MichaelC]
- ts: will look at this, and ask dpub people to as well
- 17:51:12 [MichaelC]
- jd: people can come to me with questions if needed
- 17:52:23 [MichaelC]
- jd: other comments, questions, concerns, volunteers?
- 17:53:03 [MichaelC]
- jva: also mid-stream on figuring this all out
- 17:54:13 [joanie]
- https://www.w3.org/wiki/ARIA_1.1_Manual_Testing
- 17:54:16 [MichaelC]
- jd: in https://lists.w3.org/Archives/Public/public-aria/2017May/0041.html JG says he hopes to have IA2 ATTA next week
- 17:54:27 [MichaelC]
- in which case all the magic happens
- 17:54:40 [MichaelC]
- otherwise manual testing
- 17:54:49 [MichaelC]
- zakim, next item
- 17:54:49 [Zakim]
- agendum 2. "ARIA 1.1 "actions / status" check in" taken up [from joanie]
- 17:55:17 [MichaelC]
- jd: progress on IA2 ATTA
- 17:56:01 [MichaelC]
- note an ATTA is an ATTA, it works for any version of specs
- 17:57:05 [MichaelC]
- jd: I´m focusing on the 2 right columns in the ARIA test report, turning any failures there to passes
- 17:57:37 [MichaelC]
- mc: who´s doing the other two?
- 17:58:10 [MichaelC]
- jd: bugs are filed with Mozilla
- 18:02:27 [MichaelC]
- rrsagent, make minutes
- 18:02:27 [RRSAgent]
- I have made the request to generate http://www.w3.org/2017/05/18-aria-minutes.html MichaelC
- 18:02:40 [jcraig]
- jcraig has joined #aria
- 18:03:13 [MichaelC]
- present+ Job
- 18:03:16 [MichaelC]
- rrsagent, make minutes
- 18:03:16 [RRSAgent]
- I have made the request to generate http://www.w3.org/2017/05/18-aria-minutes.html MichaelC
- 18:03:19 [MichaelC]
- rrsagent, bye
- 18:03:19 [RRSAgent]
- I see no action items