20:03:04 <RRSAgent> RRSAgent has joined #aria-at
20:03:04 <RRSAgent> logging to https://www.w3.org/2021/02/25-aria-at-irc
20:03:47 <weston> weston has joined #aria-at
20:03:51 <weston> present+
20:05:16 <Matt_King> MEETING: ARIA and Assistive Technology Community Group
20:05:33 <Matt_King> CHAIR: James
20:05:43 <Matt_King> rrsagent, make log public
20:05:48 <Matt_King> present+
20:05:53 <s3ththompson> present+
20:05:53 <Matt_King> present+ Sina
20:05:59 <michael_fairchild> michael_fairchild has joined #aria-at
20:06:07 <Matt_King> rrsagent, make minutes
20:06:07 <RRSAgent> I have made the request to generate https://www.w3.org/2021/02/25-aria-at-minutes.html Matt_King
20:06:16 <michael_fairchild> present+
20:06:30 <jscholes> jscholes has joined #aria-at
20:06:35 <jongund> present+ jongund
20:06:37 <jscholes> present+
20:06:42 <jscholes> present+ sina
20:07:14 <jongund> scribe: jongund
20:07:51 <jongund> JS: First thing is issue 363
20:08:19 <jongund> https://github.com/w3c/aria-at/issues/363
20:09:01 <jongund> JS: We have a need for tests with a sequence of commands, commands followed by other commands
20:09:24 <jongund> JS: Open ended, repeating a keystroke, but may not know how many times
20:10:03 <jongund> JS: When screen readers change behaviors the number of keystrokes may change
20:10:34 <jongund> JS: For example between JAWS 2021 and 2022 the number of keystrokes may change
20:11:02 <jongund> JS: Closed sequences, for example in the menubar example
20:11:27 <jongund> JS: We need a flag to indemnify the type of sequence
20:12:08 <jongund> JS: As the person writing the test you would need to open the keys file, and then use some quoted string
20:12:23 <jongund> MK: I would like to separate isues
20:13:18 <jongund> MK: The closed sequence, they seem really ligament to me, specifying this specific set of commands
20:13:38 <jongund> MK: I am pushing back on the open ended, we really need to make sure we need it
20:13:58 <jongund> MK: On the multiple command sequences, we have this list of steps
20:14:47 <jongund> MK: We have things like the right mode, right starting point, the last set could be a sequence
20:15:30 <jongund> MK: We want to capture output for each key pressed, for example "T" and "Down Arrow", so both are command sequences
20:15:53 <jongund> MK: Then there are assertions to the combined output
20:16:19 <jongund> SB: Assertions apply to the total concatenation of the key sequence
20:16:52 <jongund> MK: You need to define something in keys file to identify a key sequence
20:17:00 <jongund> SB: I am over simplifying this
20:17:51 <jongund> JS: There is only one column in the CVS, we want to specify more than one key in the cell
20:18:04 <jongund> MK: You could to left brace and then...
20:18:14 <jongund> SB: What is good about that it is easy to check
20:20:09 <jongund> MK: Your assertions are for the whole sequence
20:21:10 <jongund> Chnages would be in: commands.csv, example: https://github.com/w3c/aria-at/blob/master/tests/modal-dialog/data/commands.csv
20:22:04 <jongund> JS: They are linked through IDs
20:22:54 <jongund> MK: The commands.csv will have an array, each item would be one of the keys
20:23:07 <jongund> MK: It's a boon for automation?
20:23:18 <jongund> SB: Because it is easy enough to do
20:23:49 <jongund> SB: For a computer it is easy
20:24:26 <jongund> MK: What you are saying, if we are gathering from humans we want to make it easy
20:24:40 <jongund> SB: The computer can join it ...
20:25:14 <jongund> MK: In the data model, if from automated tests the response will be an array of strings
20:25:40 <jongund> JS: We want two types, string from humans and arrary from automation
20:26:06 <jongund> MK: Hopefully we can get automated output toggle
20:26:23 <jongund> SB: It is easy to add to the automated output
20:26:37 <jongund> JS: If someone could write a JAWS script to do it
20:26:51 <jongund> SB: Then you are done, its just output
20:27:18 <jongund> MK: That would be awesome, ideally the same from every screen reader
20:27:43 <jongund> MK: We can just get handshake agreements, like an apple script and commander
20:27:53 <jongund> MK: JC said it was possible
20:28:02 <jongund> MK: Just make it a command
20:28:30 <jongund> MK: The thing is it has to have default configuration
20:28:44 <jongund> SB: Once we have the JS file, it is an easier conversation
20:29:15 <jongund> Issue is getting the script to be shipped with each screen reader
20:29:35 <jongund> ST: This is great, and removes some indirection
20:29:56 <jongund> MK: This is a super high priority
20:30:13 <jongund> MK: You can write it up in 363
20:30:26 <jongund> JS: Some of the commands can go in setup
20:30:34 <jongund> MK: Only it they are part of setup
20:31:54 <jongund> MK: Use the setup to get people in to the position to execute the command
20:32:16 <jongund> JS: We do not have a way to identify keys in the setup, just strings
20:32:35 <jongund> JS: How necessary is it, you could say it could be scripted
20:32:55 <jongund> JS: We really need a reason to have setup command sequence
20:33:31 <jongund> MK: Adding support for a setup sequence that pulls from the keys.mjs
20:34:06 <jongund> MK: We are looking at moving the setup script from onload to a button
20:35:16 <jongund> JG: Examples have to be designed to be scripted
20:35:51 <jongund> MK: I would like you to raise issues with APG when their are setup script issues
20:36:10 <jongund> MK: Open ended really needed?
20:36:23 <jongund> MK: For example with combobox label
20:36:37 <jongund> MK: We are looking for deterministic output from screen readers
20:37:20 <jongund> MK: I think the combobox case is a failure, there is unexpected output, using the closed sequence
20:37:42 <jongund> MK: If you used a difference labeling technique we do not want the experience to be difference
20:38:07 <jongund> JS: Are we getting too opinionated about SR behavior
20:38:19 <jongund> MK: It is being prescriptive about a bug
20:38:32 <jongund> SB: You are saying the label twice
20:39:18 <jongund> JS: I would like to press the down arrow twice and they don't get to the combobox and then it is a failure
20:39:38 <jongund> MK: It would be a separate test to press "F" to get to combobox
20:39:49 <jongund> MK: They only fail for some of the test
20:40:03 <jongund> JS: Itf they press the down arrow one more time, then they pass
20:40:45 <jongund> MK: That's were having multiple testers, we want to have testers follow precisely, so there will be conflicts if people do not follow the directions
20:41:26 <jongund> JS: Do we need any additional instructions on following example
20:41:47 <jongund> MK: I think we need to do a good job of on boarding people
20:42:24 <jongund> SB: We can use experienced testers to train new testers
20:43:22 <jongund> HR: We should observe behavior the starting point, the navigate forward from here, NVDA handles that properly, for JAWS you are asking to switch, it starts reading the page
20:43:48 <jongund> MK: That's because we are doing the test after page load
20:44:15 <jongund> MK: The next issue is starting using a command, instead of load
20:44:40 <jongund> HR: I don't see a way in JAWS to stop reading on load
20:45:02 <jongund> MK: We are using default configuration
20:45:34 <jongund> JS: I am not changing the speech rate
20:45:46 <jongund> JS: The tests are not short
20:46:04 <jongund> SB: I don't want it to be painful for the testers
20:47:09 <jongund> SB: If we do this with an add on, we can let them have their speech settings, but we can configure other features
20:47:20 <jongund> MK: There should be a command we can drive all of that
20:47:30 <jongund> SB: That gets rolled into automation
20:47:46 <jongund> JS: Issue 300
20:47:48 <jongund> https://github.com/w3c/aria-at/issues/300
20:48:01 <jongund> prsent+ hadi
20:48:13 <jongund> present+ hadi
20:48:51 <jongund> MK: This not about test writing, but it doesn't effect you as a test writer
20:49:15 <jongund> MK: There are some basic quality assurance tests, then we merge to the master branch
20:49:31 <jongund> MK: Now it can be pulled into app as a draft test
20:50:09 <jongund> MK: We have to know, test that are updated..., it's draft by default, we want human testers to see in their list of tests
20:50:22 <jongund> MK: They can review it while running it and file issues
20:51:08 <jongund> MK: Then the admin sees complete test runs with no conflicts, then it can be considered for the next step after draft
20:52:09 <jongund> JS: Do we want people to add github issues, it will be a lot of issues and the issue has to be consistently referenced
20:52:37 <jongund> JS: I am concerns about a huge number of open issues
20:53:11 <jongund> JS: There is manual overhead with have one issue per test, this doesn't allow an issue doesn't close
20:53:41 <jongund> JS: Need to be concerned about the signal to noise ratio on the issues list
20:53:53 <jongund> MK: can we move something to a DONE column
20:54:18 <jongund> MK: We can have a project we can have an issue list
20:54:24 <jongund> JS: That would work
20:55:08 <jongund> MK: For single issue we can have checkboxes, and only once person can change the checkbox
20:55:31 <jongund> JS: I like the idea of keeping everything in one place
20:55:56 <jongund> MK: I use projects in the APG, I need to manually add issues
20:56:25 <jongund> MK: Right now we are workignon tree, so I am going down looking at tree features, it has been very useful
20:56:45 <jongund> JS: If we go that route I want it to be automated
20:57:05 <jongund> MK: Github is automated, so you can move to the done column
20:57:26 <jongund> MK: If something is no longer draft. there are three stages
20:57:41 <jongund> MK: First step is testing in the community group results
20:58:19 <jongund> MK: Second, is results that re reveiwed by screen readers, and they many look at where there are problems
20:58:37 <jongund> MK: This second step is needed to move a test plan out of draft mode
20:59:02 <jongund> MK: Three steps: "draft", "something" and "final"
20:59:36 <jongund> MK: If screen reader developers have issues it could also be something about the plan
20:59:53 <jongund> JS: Need modifications to the setup scripts
21:00:04 <jongund> JS: Probably first thing for next week
21:00:20 <jongund> rrsagent, draft minutes
21:00:20 <RRSAgent> I have made the request to generate https://www.w3.org/2021/02/25-aria-at-minutes.html jongund