IRC log of dpub-aria on 2015-12-03

Timestamps are in UTC.

13:56:18 [RRSAgent]

RRSAgent has joined #dpub-aria

13:56:18 [RRSAgent]

logging to http://www.w3.org/2015/12/03-dpub-aria-irc

13:56:40 [tzviya]

rrsagent, make logs public

14:00:00 [ShaneM]

Present+ ShaneM

14:00:21 [tzviya]

present+ Tzviya_Siegman

14:01:02 [tzviya]

rrsagent, this is DPUB-ARIA

14:01:02 [RRSAgent]

I'm logging. I don't understand 'this is DPUB-ARIA', tzviya. Try /msg RRSAgent help

14:01:20 [tzviya]

this is DPUB-ARIA

14:01:35 [ShaneM]

zakim, this is DPUB-ARIA

14:01:35 [Zakim]

got it, ShaneM

14:02:03 [janina]

present+ janina

14:05:03 [tzviya]

agenda: https://lists.w3.org/Archives/Public/public-dpub-aria/2015Dec/0001.html

14:05:54 [tzviya]

present+ Michael

14:06:01 [tzviya]

Present+ ivan

14:06:10 [tzviya]

topic: testing DPUB-AAm

14:06:31 [tzviya]

s/AAm/AAM

14:08:47 [tzviya]

DPUB-AAM published this morning at http://www.w3.org/TR/2015/WD-dpub-aam-1.0-20151203/

14:08:55 [MichaelC]

-> https://www.w3.org/WAI/PF/testharness/ PF test harness

14:09:33 [richardschwerdtfeger]

richardschwerdtfeger has joined #dpub-aria

14:09:35 [tzviya]

michael: we have DPUB roles and AAM

14:09:53 [tzviya]

...we care more about getting roles to CR, but they are dependant on AAM

14:10:04 [tzviya]

present+ rich

14:10:37 [tzviya]

michael: expected result of roles is that it is exposed in accessibilty mapping X as X

14:11:13 [ivan]

q+

14:11:24 [tzviya]

...when testing a specific role, using a browser with Accessibility API X

14:12:58 [tzviya]

... and seeing the roles implemented in at least 2 APIs should get us to CR

14:13:19 [ivan]

present+ ivan

14:15:05 [tzviya]

tzviya: summarizes michael's comments: proof of implementation for DPUB roles is that any role is implemented in any 2 browser/API combo

14:15:10 [tzviya]

ack ivan

14:15:46 [tzviya]

ivan: the goal of CR is to prove implementability, but there is also goal to show usage

14:16:16 [tzviya]

...are we also under obligation to show that the roles are in use?

14:16:42 [ShaneM]

q+ to discuss other ways to test a vocabulary

14:16:44 [tzviya]

...similar to semantic web. How do you check a vocab?

14:17:29 [tzviya]

michael: we are first concerned with exiting CR, partly by pre-negotiating

14:18:20 [tzviya]

...we propose our criteria to the director, and then we meet the goals

14:19:29 [tzviya]

...We should care about implementation as well. For ARIA 1.0 we didn't have the resources, and let's look at the testing

14:20:02 [tzviya]

tzviya: i know of organizations IRL that will use this vocab, so we can provide real examples in the wild

14:20:15 [MichaelC]

agenda+ Test Automation

14:21:01 [tzviya]

shane: we don't need to have examples to exit CR, but if we do, best method is to demo with schemas in validator.nu

14:21:34 [tzviya]

michael: showing manual test procedure for Aria 1

14:21:38 [MichaelC]

https://www.w3.org/WAI/PF/testharness/testcases/edit?testsuite_id=1&testcase_id=183

14:21:42 [tzviya]

scribenick: tzviya

14:22:54 [tzviya]

michael: this is manual testing, but it will likely be in use for 1.1 as well (at least in part)

14:23:24 [tzviya]

...must indicate exactly what API, browser, etc testing

14:23:44 [tzviya]

q?

14:23:59 [ShaneM]

ack ShaneM

14:23:59 [Zakim]

ShaneM, you wanted to discuss other ways to test a vocabulary

14:25:14 [tzviya]

michael: having 2 passes is enough to pass CR

14:25:26 [tzviya]

... this can also be used for bug fixes

14:26:14 [tzviya]

tzviya: who did the testing for ARIA 1.0?

14:26:44 [tzviya]

michael: memebers of PF, including Jon Gunderson and others from U of Illinois

14:26:54 [richardschwerdtfeger]

present+ Rich_Schwerdtfeger

14:27:52 [tzviya]

...Rich and Cynthia did a lot of testing as well

14:27:54 [ivan]

q+

14:28:22 [tzviya]

...there are 500 - 800

14:28:26 [tzviya]

ack iv

14:29:36 [tzviya]

ivan: Other WGs had around 1000 tests as well, but the tests had reports, so the tests pushed information to the implementors

14:29:43 [tzviya]

...this may be different though

14:30:03 [tzviya]

...At the end of CR period, we need to put togethor a report

14:30:11 [MichaelC]

-> https://www.w3.org/WAI/PF/testharness/testreport?testsuite_id=1 ARIA 1.0 test report

14:31:57 [tzviya]

michael: this is php pulled from mySQL db

14:32:09 [tzviya]

agenda next

14:32:17 [tzviya]

zakim, next item

14:32:17 [Zakim]

agendum 1. "Test Automation" taken up [from MichaelC]

14:32:18 [ShaneM]

It should be trivial to push results from automation into a database - set up a web service

14:32:41 [tzviya]

michael: this stuff should be automatable

14:32:56 [tzviya]

...looking for token, varies by API

14:33:40 [ShaneM]

q+ to suggest that we not make automation dpub's problem

14:33:44 [tzviya]

...the a11y API inspector tools are scriptable, so can automated by platform, but it would be custom by platform

14:34:30 [tzviya]

...ARIA WG will talk about this further next week

14:35:16 [MichaelC]

-> http://testthewebforward.org/docs/test-format-guidelines.html Test the Web Forward

14:35:18 [tzviya]

We don't know how much automation we'll be able to do in next few months bc this is ongoing

14:36:01 [tzviya]

ack shane

14:36:01 [Zakim]

ShaneM, you wanted to suggest that we not make automation dpub's problem

14:36:34 [tzviya]

shane: it's not DPUB's problem to figure out autotmation. We need to solve this for everyone.

14:39:11 [tzviya]

tzviya: next steps - it looks like automation is a bit off

14:39:14 [ShaneM]

q+ to ask if there is a tool to extract the mappings automatically? (assuming the answer is yes)

14:39:42 [tzviya]

...how do we move this forward?

14:40:41 [tzviya]

tzviya: it would be helpful to test all the AAMs together and tap into eager student bodies

14:40:45 [tzviya]

ack shan

14:40:45 [Zakim]

ShaneM, you wanted to ask if there is a tool to extract the mappings automatically? (assuming the answer is yes)

14:41:04 [tzviya]

shane: is there a way to extract mappings auto?

14:41:12 [tzviya]

michael: we use xslt

14:41:49 [tzviya]

shane: does respec produce xhtml? does respec spit out something reasonable?

14:42:45 [ShaneM]

ACTION: Shane to script output of assertions from ReSpec into JSON

14:42:45 [trackbot]

Created ACTION-1756 - Script output of assertions from respec into json [on Shane McCarron - due 2015-12-10].

14:44:02 [tzviya]

shane: testing of DPUB is much easier than core testing, because it's just role testing

14:45:11 [tzviya]

tzviya: we have a note in current draft of DPUB that we can extend the vocab. At what point do we freeze the vocab?

14:45:32 [ivan]

+1 to Michael

14:45:45 [tzviya]

michael: freeze when you enter CR, but also goes to question of vocab extensibility

14:46:25 [tzviya]

...do something of a virtual last call

14:47:14 [ivan]

q+

14:47:47 [ShaneM]

Also need to know when the AAM is stable...

14:48:01 [ShaneM]

(in other words, do you trust the mappings?)

14:48:46 [tzviya]

michael: we need to know when the DPUB vocab is set, when we are ready to test

14:48:52 [tzviya]

ack iv

14:49:43 [tzviya]

ivan: for each of our statements, we must have a test file, using the role

14:50:10 [ShaneM]

I envision a tool chain

14:50:31 [tzviya]

...shane's script should create HTML files from the code samples in the document

14:52:42 [tzviya]

tzviya: we plan to provide more complicated, nested samples

14:53:07 [MichaelC]

q+ to talk about test file format

14:53:29 [tzviya]

ack mi

14:53:29 [Zakim]

MichaelC, you wanted to talk about test file format

14:53:54 [tzviya]

michael: test files are the smallest granular chunks possible

14:56:22 [tzviya]

tzviya: i'm thinking that we should leave existing examples, and add appendix to demo comprehensive examples

14:56:36 [ShaneM]

q+ to mention something about term references

14:57:26 [ShaneM]

<a>user agents</a>

14:58:18 [tzviya]

...timeline: 2PWD just published - we should be able to comment on state of the vocab in January after feedback

14:58:24 [tzviya]

ack shane

14:58:24 [Zakim]

ShaneM, you wanted to mention something about term references

14:58:45 [tzviya]

rrsagent, make minnutes

14:58:46 [RRSAgent]

I'm logging. I don't understand 'make minnutes', tzviya. Try /msg RRSAgent help

14:59:01 [tzviya]

rrsagent, make miutes

14:59:01 [RRSAgent]

I'm logging. I don't understand 'make miutes', tzviya. Try /msg RRSAgent help

14:59:09 [tzviya]

rrsagent, make minutes

14:59:09 [RRSAgent]

I have made the request to generate http://www.w3.org/2015/12/03-dpub-aria-minutes.html tzviya

15:02:47 [pkra]

pkra has joined #dpub-aria

15:07:46 [tzviya]

tzviya has joined #dpub-aria

15:13:27 [richardschwerdtfeger]

richardschwerdtfeger has joined #dpub-aria

15:50:37 [ShaneM]

ShaneM has joined #dpub-aria

16:01:23 [richardschwerdtfeger]

richardschwerdtfeger has joined #dpub-aria

16:30:30 [ShaneM]

ShaneM has joined #dpub-aria

16:44:02 [richardschwerdtfeger]

richardschwerdtfeger has joined #dpub-aria

17:23:17 [richardschwerdtfeger]

richardschwerdtfeger has joined #dpub-aria