IRC log of aria on 2017-05-18

Timestamps are in UTC.

16:48:26 [RRSAgent]
RRSAgent has joined #aria
16:48:26 [RRSAgent]
logging to http://www.w3.org/2017/05/18-aria-irc
16:48:28 [trackbot]
RRSAgent, make logs world
16:48:31 [trackbot]
Zakim, this will be
16:48:31 [Zakim]
I don't understand 'this will be', trackbot
16:48:31 [trackbot]
Meeting: Accessible Rich Internet Applications Working Group Teleconference
16:48:31 [trackbot]
Date: 18 May 2017
16:48:35 [joanie]
chair: Joanmarie_Diggs
16:48:35 [joanie]
RRSAgent, make log public
16:48:37 [joanie]
agenda?
16:48:48 [joanie]
clear agenda
16:48:53 [joanie]
agenda: this
16:49:07 [joanie]
agenda+ Instructions on how to get started with manual testing
16:49:07 [joanie]
agenda+ ARIA 1.1 "actions / status" check in
16:49:10 [joanie]
agenda: be done
16:49:16 [joanie]
agenda?
16:49:29 [joanie]
Regrets: Michiel_Bijl, Rich_Schwerdtfeger, Jon_Gunderson
16:49:41 [joanie]
scribeOptions: -final
16:57:16 [JasonDuan]
JasonDuan has joined #aria
16:57:51 [joanie]
present+ Joanmarie_Diggs
17:00:04 [tzviya]
present+
17:01:38 [joanie]
scribe: joanie
17:01:58 [joanie]
Zakim, take up item 1
17:01:58 [Zakim]
agendum 1. "Instructions on how to get started with manual testing" taken up [from joanie]
17:02:26 [joanie]
Link to documentation: https://www.w3.org/wiki/ARIA_1.1_Manual_Testing
17:02:41 [MichaelC]
present+
17:03:03 [JasonDuan]
JasonDuan has joined #aria
17:03:41 [Stefan]
Stefan has joined #aria
17:04:27 [jcraig]
jcraig has joined #aria
17:05:01 [joanie]
https://www.w3.org/wiki/ARIA_1.1_Manual_Testing
17:05:24 [joanie]
ARIA 1.1 implementations in report: https://w3c.github.io/test-results/wai-aria/#index-of-implementations-in-reports
17:05:33 [tzviya]
https://w3c.github.io/test-results/dpub-aam/#index-of-implementations-in-reports
17:05:46 [MichaelC]
scribe: MichaelC
17:06:01 [mck]
mck has joined #aria
17:06:10 [MichaelC]
jd: above applies to ARIA and DPub
17:06:23 [MichaelC]
swapping dpub-aam for wai-aria in URIs
17:06:43 [MichaelC]
includes contact info of people working on platforms
17:06:55 [MichaelC]
I´ve done on some non-Windows systems
17:07:14 [MichaelC]
Jon working on IA2 testing, hopes to be ready by next week
17:07:21 [MichaelC]
but we still don´t have a plan for UIA testing
17:07:28 [MichaelC]
so at the moment, manual testing needed on that platform
17:07:36 [mck]
present+ matt_king
17:07:39 [MichaelC]
MS has said they plan to do it; I´ve sent the instructions
17:08:19 [MichaelC]
if you get to the point you don´t have implementation data, may need to do manual testing, since the automated is happening
17:08:39 [joanie]
https://w3c.github.io/test-results/wai-aria/all.html
17:08:48 [MichaelC]
I´m running automated tests for ATK and AXAPI, 2 UAs each
17:08:55 [joanie]
https://w3c.github.io/test-results/dpub-aam/all.html
17:09:03 [MichaelC]
above links are test reports for ARIA and DPub
17:09:48 [MichaelC]
The headers in the table are explained in the index of implementations page
17:10:20 [MichaelC]
the two WebKit implementations show mostly passing results
17:10:29 [MichaelC]
the other two mostly failures
17:10:45 [MichaelC]
that doesn´t indicate manual testing needed - it´s valid data about not passing at the moment
17:10:53 [MichaelC]
it´s lack of data that needs manual testing
17:10:56 [joanie]
https://w3c.github.io/test-results/wai-aria/all.html
17:11:22 [MichaelC]
in above link, fields with orangey-yellow lack data
17:11:36 [MichaelC]
there are a few reasons this can happen
17:12:34 [MichaelC]
we added tests but didn´t update report, or the test isn´t applicable to that platform, or we simply lack a test
17:14:03 [MichaelC]
mc: can we distinguish don´t need data from don´t have but need?
17:14:23 [MichaelC]
jd: documentation page gives pointers to figure that out
17:14:31 [MichaelC]
you can usually figure it out
17:14:32 [MichaelC]
q+
17:15:11 [MichaelC]
q+ to talk about what we tell Director, and if we can get additional info here
17:15:48 [MichaelC]
often, when there´s an additional test file under a feature, it´s to test additional platform-specific requirements
17:16:12 [tzviya]
q+ to ask why I get permission denied messages when I click through
17:16:50 [MichaelC]
ack me
17:16:51 [Zakim]
MichaelC, you wanted to talk about what we tell Director, and if we can get additional info here
17:17:12 [MichaelC]
mc: Director will want to easily see difference
17:17:30 [MichaelC]
also would be good to add more clear ¨key¨ information
17:17:47 [MichaelC]
jd: requires test runner upgrade, I can approach Shane about how to get that made
17:18:44 [MichaelC]
jd: for ARIA 1.1 we added some things to existing roles that weren´t requirements in 1.1
17:19:24 [MichaelC]
for new roles we just test the whole role
17:19:35 [MichaelC]
for changes, we kinda need to test the new and old behavior
17:19:52 [MichaelC]
not sure if we´ll be able to do that automatically, so that could add a small set of new manual tests needed
17:20:02 [MichaelC]
though my goal is still to automated it
17:20:13 [MichaelC]
there are only 10 anyways
17:20:15 [MichaelC]
q?
17:20:24 [MichaelC]
ack t
17:20:24 [Zakim]
tzviya, you wanted to ask why I get permission denied messages when I click through
17:21:00 [MichaelC]
ts: when I follow links, i get ¨permission denied¨ errors
17:21:30 [joanie]
https://www.w3.org/wiki/ARIA_1.1_Manual_Testing
17:21:58 [MichaelC]
jd: when you´re doing manual testing you shouldn´t encounter this
17:22:03 [MichaelC]
but it is something I plan to fix
17:22:03 [joanie]
https://w3c-test.org/tools/runner/index.html
17:22:21 [MichaelC]
this is the test runner
17:22:43 [MichaelC]
put the shortname of the spec you want to test in the field (wai-aria or dpub-aam), with a starting slash
17:23:46 [MichaelC]
can take a while to load after that, then shows test case with a table
17:24:33 [MichaelC]
each test case is pretty minimalist
17:24:49 [MichaelC]
the table essentially says what you´re looking for, in the various APIs
17:25:11 [MichaelC]
you need to use an AAPI inspector for the platform
17:25:19 [MichaelC]
AccProbe for IA2?
17:25:46 [MichaelC]
look through the a11y tree to find the element called out in the test, and check its properties
17:26:15 [MichaelC]
see if it´s got the thing, or property has the value, the table says it should have
17:26:27 [MichaelC]
then go back to main window and choose ¨pass¨ or ¨fail¨
17:27:01 [MichaelC]
then it will show the next test
17:27:11 [MichaelC]
so you rinse and repeat
17:27:11 [MichaelC]
q+
17:27:23 [MichaelC]
q+ to ask how you pause and resume a session
17:27:45 [MichaelC]
be sure not to close the test pop-up window, the test runner really wants it
17:28:44 [MichaelC]
ack me
17:28:44 [Zakim]
MichaelC, you wanted to ask how you pause and resume a session
17:29:10 [MichaelC]
mc: can I pause a session and pick up where I left off later? e.g., after my browser has restarted so I didn´t just leave it open
17:29:25 [MichaelC]
jd: you can walk away from tests and come back, the harness will just sit there waiting on you
17:30:21 [MichaelC]
you can also select a subset of the tests at the start of the process
17:31:27 [MichaelC]
you can use regex, but more simply use prefix
17:33:10 [MichaelC]
you can get a JSON file and send it to me to incorporate into the test results repo
17:33:26 [MichaelC]
ts: for DPub-AAM, what´s still to test?
17:33:29 [MichaelC]
jd: Windows
17:33:36 [MichaelC]
ts: so there´s not a column for it yet?
17:34:57 [MichaelC]
jd: right, only the UAs listed are known to the system yet
17:35:03 [MichaelC]
so Windows isn´t included yet
17:35:19 [MichaelC]
mc: can we have placeholder columns?
17:35:29 [MichaelC]
jd: yes, after a code review
17:35:54 [MichaelC]
I haven´t because I don´t like fully empty info
17:36:22 [MichaelC]
mc: think an empty column is more clear for non-testing reviewers
17:37:43 [MichaelC]
jd: we´re still not fully sure of MS plans
17:37:51 [MichaelC]
tz: have access to various systems
17:38:10 [MichaelC]
jd: Firefox on Windows might be most useful place for you to start
17:39:13 [MichaelC]
suggest you test a nightly build, so you have the latest implementation
17:39:35 [MichaelC]
since features are landing as we speak
17:39:48 [MichaelC]
we are hoping MS will take care of Edge
17:39:56 [MichaelC]
we can more easily get nightlies of FF
17:41:10 [MichaelC]
also think you´ll get similar results to FF on Linux
17:41:14 [MichaelC]
Linux is the redheaded stepchild of the accessibility world
17:41:56 [MichaelC]
that´ll help you sanity check your process
17:43:13 [MichaelC]
tz: will look through this and come back with questions
17:44:19 [MichaelC]
jd: I don´t have Windows, except on a virtual machine in another country, so will appreciate additional guidance in the wiki about that platform if you have it
17:51:00 [MichaelC]
ts: will look at this, and ask dpub people to as well
17:51:12 [MichaelC]
jd: people can come to me with questions if needed
17:52:23 [MichaelC]
jd: other comments, questions, concerns, volunteers?
17:53:03 [MichaelC]
jva: also mid-stream on figuring this all out
17:54:13 [joanie]
https://www.w3.org/wiki/ARIA_1.1_Manual_Testing
17:54:16 [MichaelC]
jd: in https://lists.w3.org/Archives/Public/public-aria/2017May/0041.html JG says he hopes to have IA2 ATTA next week
17:54:27 [MichaelC]
in which case all the magic happens
17:54:40 [MichaelC]
otherwise manual testing
17:54:49 [MichaelC]
zakim, next item
17:54:49 [Zakim]
agendum 2. "ARIA 1.1 "actions / status" check in" taken up [from joanie]
17:55:17 [MichaelC]
jd: progress on IA2 ATTA
17:56:01 [MichaelC]
note an ATTA is an ATTA, it works for any version of specs
17:57:05 [MichaelC]
jd: I´m focusing on the 2 right columns in the ARIA test report, turning any failures there to passes
17:57:37 [MichaelC]
mc: who´s doing the other two?
17:58:10 [MichaelC]
jd: bugs are filed with Mozilla
18:02:27 [MichaelC]
rrsagent, make minutes
18:02:27 [RRSAgent]
I have made the request to generate http://www.w3.org/2017/05/18-aria-minutes.html MichaelC
18:02:40 [jcraig]
jcraig has joined #aria
18:03:13 [MichaelC]
present+ Job
18:03:16 [MichaelC]
rrsagent, make minutes
18:03:16 [RRSAgent]
I have made the request to generate http://www.w3.org/2017/05/18-aria-minutes.html MichaelC
18:03:19 [MichaelC]
rrsagent, bye
18:03:19 [RRSAgent]
I see no action items