00:59:20 <jongund> jongund has joined #aria-at
01:45:50 <jongund> jongund has joined #aria-at
02:29:11 <jongund> jongund has joined #aria-at
03:21:29 <jongund> jongund has joined #aria-at
05:20:37 <jongund> jongund has joined #aria-at
09:40:19 <jongund> jongund has joined #aria-at
19:10:56 <RRSAgent> RRSAgent has joined #aria-at
19:10:56 <RRSAgent> logging to https://www.w3.org/2022/09/26-aria-at-irc
19:11:05 <zcorpan_> present+
19:11:57 <s3ththompson> jcraig: at the time, AppleScript interface is the most supported mechanism for automating... that has a hardened security profile already
19:15:16 <s3ththompson> James Scholes: just to clarify, it wouldn't be a webpage... it would be something like webdriver
19:15:30 <s3ththompson> jcraig: right, that's still not possible at this time with our security model
19:16:22 <s3ththompson> s3ththompson: when we discussed last week, we discussed the difference between sending raw keypress input vs. higher-level semantic commands... does this escape the concern about keypresses and OS security layer?
19:19:01 <s3ththompson> jcraig: right, there is definitely functionality in AppleScript today that could be used to implement some sort of normalized commands across multiple screenreaders
19:20:15 <s3ththompson> s3ththompson: we had previously considered having 2 phases in the spec draft: raw keypresses (easier to spec) and then higher-level commands (harder to spec), but this conversation suggests maybe we should reprioritize those sections to be able to accomodate running a prototype on Apple as you mentioned
19:21:09 <s3ththompson> zcorpan_: right, we talked about prioritizing high-level commands in the Automation API... the question now is whether we should work towards agreeing on a set of normalized commands, vs. just implementing vendor specific commands
19:21:59 <s3ththompson> Matt_King: i'm slightly hesitant about the normalized commands... we want to make sure the tests can correlate 1:1 with how the user would experience it. it's one thing to say: the user would press these keys. it's another to say: we have to figure out what those commands are that are triggered by those keys
19:25:12 <s3ththompson> James Scholes: yeah, i'm wondering about how we can make sure we're exercising the full suite of commands that users might use to trigger an action... i would worry that on VoiceOver we might only be using one AppleScript method, where on window, we're using multiple different keypresses
19:28:43 <s3ththompson> jcraig: i'm definitely not trying to convince you the applescript way is a preferred way to do it... just that it's available today. i can't promise a timeline to implement anything different there.
19:29:15 <s3ththompson> Matt_King: so if we needed extra commands that aren't in the AppleScript library today, is that something we could potentially add?
19:29:32 <s3ththompson> jcraig: possibly. there is an escape hatch right now by navigating through the web menu
19:30:29 <s3ththompson> Matt_King: are there options for control-option-F3, control-option-F4 (announce current VoiceOver cursor, announce current keyboard cursor)
19:31:47 <s3ththompson> jcraig: there's `keyboard cursor object` with `bounds` and `text under cursor`
19:32:10 <s3ththompson> Matt_King: those provide the text programmatically, but can applescript actually trigger an utterance?
19:34:15 <s3ththompson> jcraig: when VO is enabled, if you're driving a VO cursor... if you're using the query interfaces through AppleScript, it will return the string
19:34:55 <s3ththompson> Matt_King: i think a next step here on our side would be to do an analysis to determine what we can map from AppleScript
19:37:34 <s3ththompson> Glen Gordon: so we're reworking the behavior in JAWS right now... in the near future, we will be able to interpret user-injected keystrokes as if they were real keystrokes... this will open things up to programatic commands
19:40:20 <s3ththompson> Matt_King: does anyone know windows' approach to the security issues at hand here?
19:40:58 <s3ththompson> Glen Gordon: definitely something i'm thinking about... right now, there are definitely limitations, but ultimately when you install a low-level keyboard hook, things are bad
19:42:03 <s3ththompson> Glen Gorder: but i think we're confusing issues, the issue is not: can an app intercept keys... it's can it inject keys?
19:46:02 <jongund> jongund has joined #aria-at
19:47:31 <s3ththompson> jcraig: the issue is if VO could be made to perform arbitrary system actions (e.g. creating new users, etc.) by injected keyboard commands... right now we know the difference between HID-level keypresses and injected HID-level keypresses and we ignore the latter
19:52:12 <s3ththompson> Glen Gorder: the difference is that it's harder to get UI Access... if screenreaders help an attacker reach that, it's a problem
19:52:29 <s3ththompson> Glen Gorder: you have to be code-signed and you have to be in a secure file location like program files
19:52:55 <s3ththompson> James Scholes: that makes sense and that's why the limitations of the portable version NVDA are what they are
20:04:21 <s3ththompson> https://www.irccloud.com/pastebin/hDCF2osD/zoom_log
20:05:59 <s3ththompson> Matt_King: so it sounds like on NVDA there are no keypress issues, on JAWS it sounds like we should proceed with the understanding that we will be able to trigger keypresses in a forthcoming version of JAWS
20:06:26 <s3ththompson> Glen Gordon: correct, with the caveat that it might be behind a setting depending on how we reflect on this convo and make a decision
20:06:35 <s3ththompson> Matt_King: that's great, that's enough for our use cases
20:07:07 <s3ththompson> s3ththompson: and James Craig, it sounds like your recommendation at least in the short term is to consider AppleScript to be the primary bridge with which we'll be able to automate VoiceOver?
20:07:14 <s3ththompson> jcraig: that's correct, yes
20:07:30 <s3ththompson> Matt_King: okay, so the next step there will be to do a coverage analysis and come back with our findings
20:07:49 <s3ththompson> James Scholes: we've been developing some AppleScript shortcuts for human users so we can help push that forward too
20:08:07 <s3ththompson> jcraig: by the way, James Scholes thanks for your comments on toggle https://github.com/w3c/aria-at/issues/784
20:08:37 <s3ththompson> rrsagent, make log public
20:08:42 <s3ththompson> rrsagent, make log public
20:08:49 <s3ththompson> rrsagent, make minutes
20:08:49 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/26-aria-at-minutes.html s3ththompson
20:09:58 <s3ththompson> MEETING: ARIA-AT Automation and Community Group Meetings
20:10:03 <s3ththompson> TOPIC: Feedback about API section for keypresses
20:10:11 <s3ththompson> rrsagent, make minutes
20:10:11 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/26-aria-at-minutes.html s3ththompson
20:11:19 <s3ththompson> present+ jcraig
20:11:23 <s3ththompson> present+ James Scholes
20:11:27 <s3ththompson> present+ Matt_King
20:11:34 <s3ththompson> present+ Sam Shaw
20:11:41 <s3ththompson> present+ Glen Gordon
20:12:09 <s3ththompson> CHAIR: Matt_King, michael_fairchild
20:12:16 <s3ththompson> present+ michael_fairchild
20:12:32 <s3ththompson> rrsagent, make minutes
20:12:32 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/26-aria-at-minutes.html s3ththompson
21:37:46 <jongund> jongund has joined #aria-at
22:50:37 <jongund> jongund has joined #aria-at
23:59:39 <jongund> jongund has joined #aria-at