19:05:50 <RRSAgent> RRSAgent has joined #aria-at
19:05:50 <RRSAgent> logging to https://www.w3.org/2021/07/19-aria-at-irc
19:05:54 <Zakim> Zakim has joined #aria-at
19:06:12 <s3ththompson> MEETING: ARIA and Assistive Technologies Community Group
19:06:16 <s3ththompson> CHAIR: Michael Fairchild
19:06:19 <s3ththompson> present+
19:06:41 <jugglinmike> jugglinmike has joined #aria-at
19:06:49 <Rich_Noah> present+
19:07:59 <s3ththompson> present+ Mike Pennisi
19:08:40 <s3ththompson> TOPIC: Update about Windows Speech driver accessibility
19:08:55 <s3ththompson> scribe: s3ththompson
19:09:46 <s3ththompson> Mike: current goal is not just to make Speech driver vocalize, but to make it fully accessible: namely that it remembers the previously enabled voice and reenables the new voice after the fact
19:10:09 <s3ththompson> present+ Weston Thayer
19:10:14 <s3ththompson> present+ jesdaigle
19:11:17 <s3ththompson> Mike: practical reasons for current approach: keep Speech driver minimal, core needs to be written in C++, we also want option to run without vocalization for speed
19:11:29 <s3ththompson> present+ Michael Fairchild
19:11:53 <s3ththompson> Michael: from practical sense, both approaches could work. however i have some concerns about what happens while test is executing...
19:12:29 <s3ththompson> Michael: if there's an exception or failure, how would screen reader recover?
19:12:47 <s3ththompson> Michael: also, what happens if someone tries to interact with screen reader while tests are running
19:14:09 <s3ththompson> Weston: what about a virtual machine?
19:14:35 <s3ththompson> Michael: interesting, this is testing the limits of my familiarity with this tooling. what kind of ergonomics could we provide for developers around this?
19:15:13 <s3ththompson> Weston: what if you could connect to a remote virtual machine? run remotely execute locally
19:18:18 <s3ththompson> Seth: how does debuggability work with a virtual machine? what kind of guarantees do you provide?
19:19:51 <s3ththompson> Weston: very interested in debugging... you could do remote recording, you could read things as it's running etc...
19:20:09 <s3ththompson> Weston: you could definitely provide that via a CLI or webapp
19:20:48 <s3ththompson> Michael: the way i envision is you run a single command... behind the scenes it spins up a virtual machine test remotely with mike's voice and then send us the results
19:27:14 <s3ththompson> Mike: I wonder what the expectation is for previous versions of browsers / ATs
19:28:42 <s3ththompson> Weston: i've tried to build images with portable versions of different ATs
19:30:14 <s3ththompson> Michael: I think this is probably a good way forward. i think i still have concerns about AT vocalization that we should verify with other community group members. I agree with others that Weston's cloud service vision is provocative, esp. given that we eventually aspire to run these tests on a daily basis. starting to think about that infrastructure now is really vital
19:32:03 <s3ththompson> TOPIC: Goals / success criteria for the AT Driver format
19:33:51 <s3ththompson> Mike: at a high level, my understanding is that we want to build a system that can validate AT/browser behavior automatically. we want to be able to simulate user input and then validate output. i've been thinking about how we format those instructions in a machine-readable way
19:34:26 <s3ththompson> Mike: i've been thinking we need some sort of Domain Specific Langauge (DSL) to do this. last year, Simon used a JSON-formatted list of instructions, but even since that i think we've pinned it down even further
19:35:31 <s3ththompson> Mike: for example, we've since gotten feedback that we don't want ambiguous commands like "press until..." we want the commands to be precise and specific about what's going on so we would have "press 3 times" instead
19:37:42 <s3ththompson> Mike: it seems like a necessary takeaway from the above is the following non-goal: that is that any given test should be shared by multiple ATs.
19:40:43 <s3ththompson> Mike: also a non-goal: make tests overly concise. right now i think we should focus on getting the right primitives in place... on top of which we could later build abstractions.
19:49:56 <s3ththompson> Seth: so for Checkbox Test "Navigate to an unchecked checkbox in interaction mode" assertion 1: "Role 'checkbox' is conveyed" the assertion is equivalent for all ATs, but the actual string that is asserted would be different for each AT
19:55:04 <s3ththompson> Seth: so do individual assertions have meaning in this world or are we just matching the entire output string?
19:55:35 <s3ththompson> Michael: well, we would lose a lot there because the tests would be more brittle right?
19:56:43 <s3ththompson> Weston: NVDA at least has an internal object representation before it's rendered to text...
19:56:59 <s3ththompson> Seth: i think when that's come up in the past, some CG members have felt they couldn't trust an internal object representation
19:57:12 <s3ththompson> Michael: i think it was more about the fact that it wasn't very interoperable...
20:00:15 <s3ththompson> Seth: Mike, what do you think about substring matching (per assertion) vs. entire-output string matching?
20:00:38 <s3ththompson> Mike: I think there's still value in starting with entire-output string matching in order to gather data that might help us design a higher-level abstraction
20:01:52 <s3ththompson> Mike: obviously there is a downside in the brittleness to the tests, but we can take cues from web development where people use snapshotting... might be a better model for how our CG group should respond to what happens when a vendor totally changes their output
20:02:25 <s3ththompson> Michael: great discussion. we should get James and Matts' thoughts on this.
20:03:10 <s3ththompson> rrsagent, make log public
20:03:39 <s3ththompson> rrsagent, make minutes
20:03:39 <RRSAgent> I have made the request to generate https://www.w3.org/2021/07/19-aria-at-minutes.html s3ththompson