IRC log of er on 2015-04-15

Timestamps are in UTC.

12:25:31 [RRSAgent]
RRSAgent has joined #er
12:25:31 [RRSAgent]
logging to http://www.w3.org/2015/04/15-er-irc
12:25:33 [trackbot]
RRSAgent, make logs public
12:25:35 [trackbot]
Zakim, this will be 3794
12:25:36 [trackbot]
Meeting: Evaluation and Repair Tools Working Group Teleconference
12:25:36 [trackbot]
Date: 15 April 2015
12:29:10 [Wilco]
Wilco has joined #er
12:29:18 [shadi]
zakim, call shadi-617
12:31:06 [Wilco]
Hi Shadi
12:32:25 [carlos]
carlos has joined #er
12:34:19 [shadi]
scribe: shadi
12:35:07 [shadi]
http://www.w3.org/WAI/ER/charter5
12:35:20 [samuelm]
samuelm has joined #er
12:38:03 [shadi]
SAZ: looking for ways to make our current work more relevant
12:38:23 [shadi]
...also explore the possibility of rule-set for conformance evaluation
12:38:38 [shadi]
...relates to auto-wcag community group
12:39:00 [shadi]
WF: came together to develop common set of tests
12:39:15 [shadi]
...focusing on automated tests
12:39:22 [shadi]
...spent some time on format and approach
12:39:30 [shadi]
...progressing fairly slow
12:39:38 [shadi]
...but reviewing them thoroughly
12:39:49 [shadi]
...about 12 tests available
12:40:00 [shadi]
...and another 12 in progress
12:40:34 [shadi]
...core group of about 8 people
12:40:50 [shadi]
...most of them from EIII project
12:41:01 [shadi]
...EC-funded project that initiated the work
12:41:35 [shadi]
...some may be able to participate beyond the project duration
12:42:13 [shadi]
CV: are there more test cases being developed within the project?
12:43:44 [shadi]
WF: EIII develops an automated and a non-automated tool
12:44:17 [shadi]
...the second tool is for non-technical people
12:44:32 [shadi]
...all tests development within the community group
12:46:41 [shadi]
SAZ: tests in auto-wcag is also called rule-sets
12:46:50 [shadi]
...but there are also test samples
12:47:03 [shadi]
...used to benchmark evaluation tools
12:47:16 [shadi]
...how interesting are these in general?
12:47:29 [shadi]
CV: unfortunately less interest in general
12:47:42 [shadi]
...we are interested in transparency but other less
12:49:30 [shadi]
SAZ: think two sides of the same coin
12:49:49 [shadi]
...need test samples to validate correct implementation of rule-sets
12:50:14 [shadi]
WF: agree ... there is much confusion about tool results
12:50:25 [shadi]
...need a common set of rules
12:51:04 [shadi]
CV: often asked how our tool compares to others
12:51:16 [shadi]
...who verifies that?
12:51:16 [shadi]
...big test suite?
12:55:02 [shadi]
...would help demonstrate quality
12:56:51 [shadi]
SAZ: is this amount of work manageable?
12:57:08 [shadi]
WF: need more people with dedicated time allocation
12:57:26 [shadi]
...work can be done reasonably fast if people invest time
12:57:39 [shadi]
...think lots of organizations interested
12:57:51 [shadi]
...have been talking with some people
12:58:41 [shadi]
SAZ: maybe chunk-down
12:58:53 [shadi]
...like focus on automated and on HTML first
12:59:05 [shadi]
WF: lots of low-hanging fruit
12:59:16 [shadi]
...like syntactic ones
12:59:25 [shadi]
...things like images was tough, for example
12:59:51 [shadi]
CV: we face the same challenge
13:00:02 [shadi]
...but it needs a coordinated effort
13:00:16 [shadi]
...not manageable by a few alone
13:00:55 [shadi]
SM: should be done little by little
13:02:13 [shadi]
...huge task
13:02:33 [shadi]
...should tackle specific techniques and aspects
13:02:48 [shadi]
...automated rules can be validated more easily
13:03:02 [shadi]
...test definitions can be benchmarked
13:03:37 [shadi]
...will fail unless we focus on specific parts
13:04:20 [shadi]
...think should be done within ERT WG
13:04:30 [shadi]
...but needs adequate participation
13:05:31 [shadi]
http://www.w3.org/TR/AERT
13:07:35 [shadi]
WF: would support move of auto-wcag work into ERT WG
13:07:46 [shadi]
...if sufficient organizations are interested
13:08:02 [shadi]
...EARL has been an important aspect in that work, though
13:08:26 [shadi]
...using it as the output format
13:08:44 [shadi]
http://www.w3.org/WAI/ER/charter5
13:10:26 [shadi]
SAZ: how would we document and present the test rules?
13:10:39 [shadi]
WF: like the Techniques
13:10:55 [shadi]
...individual documents that are aggregated into one main document
13:11:16 [shadi]
...can use Github to develop collaboratively
13:11:43 [shadi]
SAZ: what about the test samples?
13:13:07 [shadi]
CV: that's the work we did in BenToWeb
13:13:20 [shadi]
...needs a test definition language
13:13:26 [shadi]
...have an XML schema for that
13:13:36 [shadi]
...could turn it into RDF
13:13:54 [shadi]
...associate every test description with a test rule
13:14:26 [Wilco_]
Wilco_ has joined #er
13:14:43 [shadi]
SAZ: think there may be similar development by other organizations
13:15:08 [carlos]
https://github.com/webcc/bentoweb-wcag20-test-suite-v3
13:17:00 [shadi]
SAZ: so could be like 4 main deliverables
13:17:29 [shadi]
...test rules, test samples, test case description, and test results format
13:17:50 [shadi]
...could be done in task forces too
13:20:00 [shadi]
...will try to develop a new draft charter for next week
13:22:42 [shadi]
present: Carlos, Samuel, Wilco, Shadi
13:22:49 [shadi]
chair: Shadi
13:22:55 [shadi]
trackbot, end meeting
13:22:55 [trackbot]
Zakim, list attendees
13:23:03 [trackbot]
RRSAgent, please draft minutes
13:23:03 [RRSAgent]
I have made the request to generate http://www.w3.org/2015/04/15-er-minutes.html trackbot
13:23:04 [trackbot]
RRSAgent, bye
13:23:04 [RRSAgent]
I see no action items