13:56:58 <RRSAgent> RRSAgent has joined #wcag-act
13:56:58 <RRSAgent> logging to http://www.w3.org/2017/10/02-wcag-act-irc
13:57:00 <trackbot> RRSAgent, make logs public
13:57:00 <Zakim> Zakim has joined #wcag-act
13:57:02 <trackbot> Meeting: Accessibility Conformance Testing Teleconference
13:57:02 <trackbot> Date: 02 October 2017
13:57:36 <Wilco> agenda+ ACT TF at CSUN 2017
13:57:54 <Wilco> agenda+ Reviewing ACT Rules Format https://www.w3.org/2002/09/wbs/93339/ACT_Sep2017/
13:58:06 <Wilco> agenda+ Test driving ACT-RF https://github.com/dequelabs/axe-core/blob/develop/doc/rule-proposal.md
13:58:18 <Wilco> agenda+ ACT Test case suite https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/Testing_Resources
14:03:10 <Skotkjerra> present+
14:05:23 <Kathy> Kathy has joined #wcag-act
14:05:30 <Kathy> present+ Kathy
14:05:40 <shadi> present+
14:06:03 <Wilco> present+
14:06:07 <anne_thyme> anne_thyme has joined #wcag-act
14:06:15 <anne_thyme> present+
14:06:22 <anne_thyme> scribe:anne_thyme
14:06:47 <anne_thyme> zakim, take up item 1
14:06:47 <Zakim> agendum 1. "ACT TF at CSUN 2017" taken up [from Wilco]
14:07:19 <agarrison> agarrison has joined #wcag-act
14:07:35 <maryjom> maryjom has joined #wcag-act
14:07:53 <anne_thyme> Wilco: you guys wanted to talk about CSUN. From our side we are going to send in a proposal to CSUN about the deliverables we have been working on. Deadline is tomorrow
14:08:29 <maryjom> present+ MaryJoMueller
14:08:34 <anne_thyme> Stein Erik: we just wanted it on the agenda to make sure that we promote the work we are doing here. As long as there is something going on, that is great
14:08:49 <anne_thyme> Wilco: we will send out the proposal to the mailing list
14:09:21 <anne_thyme> Wilco: Mary Jo, can you send it out to the mailing list? With the changed title...
14:09:35 <Kathy> yes
14:09:36 <anne_thyme> Mary Jo: I will send it
14:09:41 <cpandhi> cpandhi has joined #wcag-act
14:09:58 <anne_thyme> Wilco: Are everyone going to be at CSUN?
14:10:08 <anne_thyme> yes...
14:10:13 <anne_thyme> zakim, take up item 2
14:10:13 <Zakim> agendum 2. "Reviewing ACT Rules Format https://www.w3.org/2002/09/wbs/93339/ACT_Sep2017/" taken up [from Wilco]
14:10:44 <Wilco> https://www.w3.org/2002/09/wbs/93339/ACT_Sep2017/results
14:11:00 <anne_thyme> Wilco: We put up a survey to get the conversation going on the Rules Format. Got a couple of responses so far
14:12:06 <Wilco> https://github.com/w3c/wcag-act/issues/109
14:12:07 <anne_thyme> Tobias responded. First comment is on an open issue regarding the input types
14:14:15 <anne_thyme> Wilco: Did you have a chance to talk about how to get the ACT Rules Format into work in your organizations?
14:15:33 <anne_thyme> Stein Erik: We are just emerging from a big release. We are starting up the work tomorrow on how to implement the ACT Rules into our own tool and starting up work on writing some of our own rules in ACT Format. So within the next couple of months
14:16:17 <anne_thyme> Charu: October is going to be hectic. My personal view is that we are going to be able to focus on this after mid-November at least
14:16:52 <anne_thyme> Charu: I'm working on getting our actual rule developers to participate in the rule creation group
14:17:24 <anne_thyme> Kathy: Had not had a chance to look at where we are in relation to test driving rules
14:17:37 <anne_thyme> Alistair: Had not had a chance to look at it either
14:17:59 <anne_thyme> Wilco: Are you still interested in doing this? And in which time frame do expect to see it done?
14:18:25 <anne_thyme> Alistair: Any large chance, and this would be a large chance, requires a business case first
14:18:32 <Skotkjerra> q+
14:18:32 <shadi> q+
14:18:48 <anne_thyme> Wilco: The purpose would be to see whether it is feasible and then use that as feedback
14:18:49 <Wilco> ask sk
14:20:21 <Wilco> ack sh
14:20:30 <Wilco> ack sk
14:20:36 <anne_thyme> Stein Erik: Are there any hurdles that are stopping us all? It seems that we are pretty much in the same situation. Are there any commen blockers that could be identified and removed? In the document or otherwise?
14:21:09 <anne_thyme> Shadi: I don't think so, the document has been worked on for a long time, we need to put it into to practice to refine it further
14:21:30 <cpandhi> presant+
14:21:38 <anne_thyme> Shadi: We need to find the real issues, so we are not wasting time
14:22:07 <anne_thyme> Shadi: We do not require any real change from the organisations, it's the assessment that we need at this point
14:22:51 <anne_thyme> Shadi: Wilco, you said you had a script for transforming the rules. Maybe this is something other should have a look at, just exporting the rules
14:24:57 <anne_thyme> Stein Erik: It might help to know what the minimum requirements are, and yes, sharing resources will also help
14:26:30 <shadi> https://www.w3.org/TR/act-rules-format/
14:26:38 <anne_thyme> Alistair: Is there a place where I can just see a test in this format? That would help me figure out how much is required
14:26:40 <Wilco> https://w3c.github.io/wcag-act-rules/rules/ACT-R1.html
14:26:55 <shadi> https://w3c.github.io/wcag-act-rules/
14:26:56 <anne_thyme> Wilco: yes, we created example rules to go with the Rules Format
14:27:15 <anne_thyme> Shadi: There are three sample rules
14:27:26 <anne_thyme> Wilco: Yes, as well as test cases that go with each
14:27:42 <anne_thyme> Shadi: Is it linked from the Wiki?
14:27:58 <anne_thyme> Shadi: I will link to it from the Wiki after the call
14:29:30 <anne_thyme> Alistair: What level of complexity can this thing do before it falls over? e.g. a function that loops through headings and put them into a function...
14:29:43 <anne_thyme> Wilco: This is like documentation
14:29:55 <Wilco> https://github.com/dequelabs/axe-core/blob/develop/doc/act-rules-format.md
14:30:09 <anne_thyme> Alistair: Shadi said you had some kind of parser to do it?
14:30:17 <anne_thyme> Wilco: No, I wrote out how to do it
14:31:28 <anne_thyme> Wilco: It's fairly straightforward, naming and replacing a few things. There are some things you can do in aXe rules that you can't do in ACT and vice versa, and I tried to write this out
14:32:06 <anne_thyme> Wilco: you don't actually have to describe how you got the foreground color and background color
14:33:03 <shadi> q+
14:33:12 <anne_thyme> Alistair: I want to try to make people come up with the fringe cases. We don't need these kind of descriptions, people can just look at the code
14:33:25 <anne_thyme> Wilco: If anyone has access to the code...
14:33:39 <anne_thyme> Alistair: At what point do you tell me how to write the code?
14:34:17 <anne_thyme> Wilco: They don't tell you how to write the code, they tell in fairly high level plain text how to perform the test
14:34:27 <anne_thyme> Alistair: Step 1 etc. that is fairly low level
14:34:38 <anne_thyme> Wilco: yes, we are trying to be unambigious
14:35:33 <Wilco> ack sh
14:35:35 <anne_thyme> Alistair: I'm worried about spending too much time writing step 1, step 2 processes. The important thing is the quick summary and the expected outcomes
14:36:52 <anne_thyme> Shadi: yes, and we have decided that each ACT rule will be accompanied by test cases, that will be showing the fringes of this test. Are these the things you are looking for, Alistair?
14:37:55 <anne_thyme> Alistair: Yes, my developers say: just give me a piece of code that should fail this test. We don't go back and explain how to write the code that is already written. It will be a major job to go back and align
14:38:55 <Wilco> https://w3c.github.io/wcag-act-rules/rule-tests/ACT-R2-tests.html
14:38:58 <anne_thyme> Shadi: We have decided that we want both audiences. Also manual testers or people who want to know what this test is doing without having access to your compiled code. But then also have the test cases for the situation you describe. Yes, it should all be test case driven
14:39:31 <anne_thyme> Wilco: We have test cases. There are test cases linked in the bottom from all rules. It's part of the Rules Format
14:39:59 <Wilco> https://github.com/dequelabs/axe-core/issues/531
14:40:01 <anne_thyme> Alistair: In my opinion we would be able to implement everything pretty fast, except rewriting test procedures
14:41:17 <anne_thyme> Wilco: I have started using them, this is a rule proposed to aXe core a few weeks ago. First describing on a high level what the rule is going to do, so we can find out if there are false positives etc. before we actually implement the rule
14:41:49 <anne_thyme> Wilco: and if people ask about what a rule does, giving them the code is rarely helpful, it's better to give them the description
14:42:34 <anne_thyme> Alistair: This is pretty simple however, what about a function that calculates if there is an accessible name? That could be quite a lengthy process
14:43:09 <anne_thyme> Wilco: In auto-WCAG we have referred to the accessible name calculation, which is publicly available.
14:43:35 <anne_thyme> Wilco: Does the rest of you have questions about the level of detail required here?
14:44:40 <anne_thyme> Stein Erik: From our side the point of the ACT rules as a whole is to be able to present documentation on our rules to help people to decide on the reliability of rules. Transparency and accuracy is key here
14:45:00 <anne_thyme> Alistair: Couldn't this not just be done by running my rules on my test suite?
14:45:47 <anne_thyme> Stein Erik: Yes, but if it's the same author of the rules and the test suite, it could go in one direction or the other
14:46:28 <anne_thyme> Shadi: You don't have to write out everything in steps unless you want to contribute rules
14:47:02 <anne_thyme> Shadi: Or you could just implement existing test rules
14:48:03 <anne_thyme> Alistair: There are a million ways to build a function, do I need to follow the steps to say that I confirm or is it enough to match the test cases?
14:48:06 <Skotkjerra> Have to run. thanks, all!
14:48:33 <anne_thyme> Wilco: If you find a more performant way of implementing it, you are free to do it
14:49:07 <anne_thyme> Alistair: Should there be a "my stuff passes, but I did it in my own way" declaration?
14:49:47 <anne_thyme> Alistair: I'm saying, we don't need to write out the steps, as long as it passes
14:50:04 <anne_thyme> Wilco: I believe we need it, it's documentation, it's about transparency
14:50:55 <anne_thyme> Alistair: I value time, we need things to go quickly, we don't want to spend time on documentation, if we can just run it on a test suite. Especially when we have more than 400 rules
14:53:48 <cpandhi> q+
14:54:03 <anne_thyme> Shadi: Can we write somewhere in the format that the programming code in the tool might deviate from the description, but that the description is created to be human readable. But as long as the input and test cases matches, it follows the standard
14:54:12 <anne_thyme> Alistair: Excactly!
14:54:37 <anne_thyme> Shadi: I don't see why we shouldn't add this
14:54:48 <anne_thyme> Wilco: Yeah, I have no problem with that
14:55:22 <anne_thyme> Wilco: Does this alleviate your concern about the amount of documentation?
14:55:26 <anne_thyme> Alistair: Yes
14:55:48 <anne_thyme> Wilco: I think the most valuable thing people can contribute are test cases
14:56:19 <anne_thyme> Wilco: Then we can always find out who should describe it. Test cases are probably more important than the descriptions here
14:56:27 <Wilco> agenda?
14:56:28 <anne_thyme> Wilco: I will add an issue for this
14:57:08 <anne_thyme> Wilco: Where are we on the agenda... We sort of did test run and test case suite
14:57:33 <anne_thyme> Kathy: I agree with a lot f what Alistair said
14:58:13 <anne_thyme> Charu: I think it was good that we had this discussion, thank you Alistair for asking so many questions, it also answered questions that I had
14:59:22 <anne_thyme> Charu: was I was going to do was propose this format to my developer and ask how much extra work this was going to create for him
14:59:56 <anne_thyme> Charu: I personally feel that these steps might help us find fringe cases that we have missed
15:00:38 <anne_thyme> Charu: I think it's a good idea going forward to compare the steps and see how we differ and if there is fringe cases we might have missed
15:00:53 <anne_thyme> Wilco: Thank you for bringing this up, Alistair
15:03:15 <shadi> trackbot, end meeting
15:03:15 <trackbot> Zakim, list attendees
15:03:15 <Zakim> As of this point the attendees have been Skotkjerra, Kathy, shadi, Wilco, anne_thyme, MaryJoMueller
15:03:23 <trackbot> RRSAgent, please draft minutes
15:03:23 <RRSAgent> I have made the request to generate http://www.w3.org/2017/10/02-wcag-act-minutes.html trackbot
15:03:24 <trackbot> RRSAgent, bye
15:03:24 <RRSAgent> I see no action items