IRC log of aria-at on 2024-04-24

Timestamps are in UTC.

17:01:19 [RRSAgent]: RRSAgent has joined #aria-at
17:01:23 [RRSAgent]: logging to https://www.w3.org/2024/04/24-aria-at-irc
17:01:23 [Zakim]: RRSAgent, make logs Public
17:01:24 [Zakim]: please title this meeting ("meeting: ..."), Matt_King
17:01:41 [Matt_King]: MEETING: ARIA and Assistive Technologies Community Group
17:01:52 [Matt_King]: CHAIR: Matt King
17:01:56 [Matt_King]: present+
17:03:17 [jugglinmike]: present+ jugglinmike
17:03:40 [howard-e]: howard-e has joined #aria-at
17:03:44 [howard-e]: present+
17:03:50 [jongund]: jongund has joined #aria-at
17:04:11 [jugglinmike]: scribe+ jugglinmike
17:04:35 [jugglinmike]: Topic: Review agenda and next meeting dates
17:04:40 [SamShaw]: SamShaw has joined #aria-at
17:04:58 [jugglinmike]: Matt_King: The agenda is available at https://github.com/w3c/aria-at/wiki/April-24%2C-2024-Agenda
17:05:06 [jugglinmike]: Matt_King: Anyone want to modify it?
17:05:31 [jugglinmike]: present+ lola
17:06:24 [jugglinmike]: lola: Could we talk about the ARIA projects and how we want to sequence them--support tables included?
17:07:16 [jugglinmike]: Matt_King: This is what I had in mind with the "support tables" item on the agenda. We can expand on that a bit during that discussion
17:07:29 [jugglinmike]: present+ James_Scholes
17:07:31 [Joe_Humbert]: Joe_Humbert has joined #aria-at
17:08:38 [jugglinmike]: James_Scholes: I seem to remember that we wanted to debrief following our attendance at the Browser Testing and Tools ("BTT") meeting. This may also be related to that
17:12:25 [jugglinmike]: Matt_King: jugglinmike is awesome
17:12:37 [jugglinmike]: Topic: Current status
17:12:45 [jugglinmike]: Matt_King: We now have 5 plans in Candidate Review
17:12:52 [jugglinmike]: Matt_King: two of them are approved by Vispero
17:13:02 [jugglinmike]: Matt_King: Thanks to everybody for all the hard work!
17:13:17 [jugglinmike]: Matt_King: The test queue is now empty, which is an interesting state of affairs. We'll address that soon
17:13:45 [jugglinmike]: Matt_King: The radio test plan had one open issue which was feedback from Vispero. I've added a comment for them asking them to revisit now that we have a new plan
17:14:21 [jugglinmike]: Matt_King: I've created a pull request, linked in the agenda. It's a one-character change needing reviewers. The goal is to fix the "alert" plan so we can close issue 1032
17:14:41 [Joe_Humbert]: present+
17:14:43 [jugglinmike]: Matt_King: With that, we'll bereally well-positioned to meet with Apple
17:14:49 [jugglinmike]: present+ IsaDC
17:14:56 [jugglinmike]: IsaDC: I can review that; please assign me
17:15:01 [jugglinmike]: Matt_King: Thank you!
17:15:53 [jugglinmike]: Matt_King: I don't know how the app will treat that change because it doesn't impact any test results. I think it should be one of those changes that "just happens", i.e. without creating any new plan version, because it is essentially editorial
17:15:59 [jugglinmike]: Matt_King: At least, that's what I'm hoping
17:16:09 [jugglinmike]: howard-e: It will indeed create a new version
17:17:02 [jugglinmike]: howard-e: It will be named according to the date of the merge. The current version is already in candidate. When you press "advance" to the new version from "R&D", it will copy the old version's test results
17:17:26 [jugglinmike]: howard-e: From there, you can immediately finalize and advance to candidate
17:17:47 [jugglinmike]: howard-e: I was anticipating this specific question, so I tested it locally to verify that this will work as I've just described
17:18:06 [jugglinmike]: Matt_King: Our next meeting with Vispero is on Wednesday of next week
17:18:24 [jugglinmike]: howard-e: In that case, I'd be more comfortable planning a deployment for Monday
17:19:15 [jugglinmike]: howard-e: Also, there's a changelog file in the ARIA-AT repository that will be automatically updated with the relevant information at the moment of the deployment
17:19:28 [jugglinmike]: Matt_King: That sounds good. We'll get this merged, and we'll test out this new process
17:21:15 [jugglinmike]: Topic: Enhancements to APG support tables
17:21:22 [jugglinmike]: https://github.com/orgs/w3c/projects/74
17:21:59 [jugglinmike]: Matt_King: This is in response to the feedback that we received last year when we added AT support tables to the APG for the first time
17:22:12 [jugglinmike]: Matt_King: Lots of people were involved in that
17:22:28 [jugglinmike]: Matt_King: There was some public feedback about the understandability of those reports and what the high-level data meant
17:22:49 [jugglinmike]: Matt_King: one of the changes we've been working on in response to that concerns "MUST", "SHOULD", and "MAY"
17:23:04 [jugglinmike]: Matt_King: We're trying to surface two numbers at the highest level for each pattern
17:23:19 [jugglinmike]: Matt_King: The AT support table will show both the numbers for the "MUST"s and the "SHOULD"s
17:23:35 [jugglinmike]: Matt_King: ...and we will have a link in the tables to a page which explains what this data means
17:23:55 [jugglinmike]: Matt_King: It sounds like the code could be ready as soon as end-of-day tomorrow. That's what we were discussing yesterday
17:24:03 [jugglinmike]: present+ carmen
17:24:13 [jugglinmike]: carmen: That's correct; we're testing today and anticipating a release tomorrow
17:24:34 [jugglinmike]: Matt_King: So we should be able to make a pull request to APG after that and review it in the APG meeting next week
17:24:48 [jugglinmike]: Matt_King: That will also enable us to create this page with the explanation of how to read the report
17:25:55 [jugglinmike]: howard-e: Someone at Bocoup will create the pull request to APG once the requisite changes are ready on the "staging" server
17:26:13 [jugglinmike]: Matt_King: We also want to create a page for the "about" section of the APG that explains how to read the reports
17:26:32 [jugglinmike]: carmen: Boaz is working on the copy for that page. I will ask him about it today
17:26:50 [jugglinmike]: Matt_King: Okay, maybe just add a link to Boaz's document in the issue you've created about this, carmen
17:26:53 [jugglinmike]: carmen: Got it
17:27:36 [jugglinmike]: Matt_King: Ideally, we'll get this into the May 7th publication of APG. That's my target
17:28:23 [jugglinmike]: Matt_King: We can think about one or more blog posts after that. I was hoping for Global Accessibility Awareness Day, but that might be difficult... Especially since publishing a post will require coordination with W3C
17:29:38 [jugglinmike]: Matt_King: We want to make sure that the content of the reports is understandable by anybody who might come to the APG and read it. I really want to make the editorial text as understandable and approachable as possible
17:29:52 [jugglinmike]: Topic: Automation V2 project
17:30:23 [SamShaw]: jugglinmike: Matt created an issue for this
17:30:23 [jugglinmike]: https://docs.google.com/document/d/19kJWkF2IrWYAwg3s3VN_e7irCh_7AFUSMfGhNGkeTmA/edit
17:30:28 [jugglinmike]: scribe+ SamShaw
17:31:16 [SamShaw]: jugglinmike I shared this to the mailing list a few weeks ago, I'm not sure we are ready to discuss today. It would be good to get some confirmation from people that this is inline with their understanding
17:31:37 [SamShaw]: Matt_King Can you give people an understanding of what its all about?
17:32:18 [SamShaw]: jugglinmike Sure this is a description of the project to enhance the integration of automated AT response collection so
17:32:18 [SamShaw]: that it can be more fully integrated into the community group's processes.
17:32:42 [SamShaw]: jugglinmike It lays out the problem, background, goals, deliverables, and requirements
17:33:45 [SamShaw]: Matt_King VO and Firefox are listed as separate items. If you add firefox support, and you add JAWS support, would the automation be ready for both? Or would you have to create a pairing for automation to use?
17:35:00 [SamShaw]: jugglinmike On the technical level, it could be either way. I was considering it would be available across the board. I think we can decide what we want to do for each case
17:35:28 [SamShaw]: Matt_King I wasn't sure when you mention Firefox, is that available for Mac too?
17:35:37 [SamShaw]: jugglinmike No, we are still working on Mac
17:35:53 [SamShaw]: Matt_King Okay that part was unclear in the deliverables section
17:36:19 [SamShaw]: Matt_King okay there weren't any priorities on things, like reusing verdicts.
17:36:43 [SamShaw]: Matt_King Alyssa I was curious of your experience with the NVDA bot?
17:37:04 [SamShaw]: present+ Alyssa
17:37:36 [SamShaw]: Alyssa: Tests 1 through 4 for tab and shift tab with focus mode on, for radio button, the app didn't record the output. I had to manually add it
17:38:14 [SamShaw]: Matt_King I thought Mike when the output wasn't captured, that it actually had collected responses
17:38:32 [SamShaw]: Alyssa: should I send you the test that it didn't record the output?
17:38:36 [SamShaw]: jugglinmike yes please
17:38:57 [SamShaw]: Matt_King for the tests that it did collect output, was that helpful? Was that faster?
17:39:07 [SamShaw]: Alyssa: Yes, I found it to be very helpful
17:40:01 [SamShaw]: jugglinmike: In this case the bot recorded that there was no output, (rather than missing the output)
17:40:10 [SamShaw]: Matt_King So we need a way to make that more reliable?
17:40:32 [SamShaw]: jugglinmike yes. There is one area we know certain keys aren't being recorded and we are investigating
17:40:48 [SamShaw]: jugglinmike We don't think this issue is limited to firefox
17:41:38 [SamShaw]: Matt_King So one of the next things it could do, if the output is the same as what was previously recorded, then it can also check off the verdicts
17:41:47 [SamShaw]: Matt_King When I was thinking about this I thought of something worth discussing
17:42:17 [jugglinmike]: Alyssa: for tests 1 through 4 for tab and shift+tab with focus mode the bot recorded no output
17:42:20 [SamShaw]: Matt_King We changed our form to be checkboxes for assertions, before there were 3 choices, when we made it two choices we changed to check boxes
17:42:22 [jugglinmike]: Alyssa: 5 through6 bot recorded no output for up arrow with browse mode and shift tab with focus mode
17:42:26 [jugglinmike]: Alyssa: 11 bot reported no output for space with focus mode
17:42:29 [jugglinmike]: Alyssa: 12 and 13 bot reported no output for both cases
17:43:29 [SamShaw]: Matt_King We actually don't know if one has been checked. Should we go change to yes/no answers for everything? For the bots, we would know which ones were and were not recorded. and if a human tester missed one
17:43:47 [SamShaw]: Matt_King Not checked could mean you skipped it
17:43:57 [jugglinmike]: https://github.com/w3c/aria-at-app/issues/1045
17:43:57 [SamShaw]: jugglinmike Glad you raised this Matt, there is an issue for this
17:44:16 [SamShaw]: jugglinmike UI cannot accurately describe the state of assertion verdicts #1045
17:44:27 [SamShaw]: Matt_King So we are both thinking along the same lines
17:45:12 [SamShaw]: Matt_King I think the reason it was complicated with the radio buttons was that it was confusing with a table [to show our report], but now we dont have a table so I think we could change this
17:45:22 [SamShaw]: Matt_King Do other people running tests thing this would be a good change?
17:45:32 [SamShaw]: present+ Isa
17:45:47 [SamShaw]: Isa: I think so
17:46:16 [jugglinmike]: s/Isa:/IsaDC:/
17:46:37 [SamShaw]: Cool I will use IsaDC
17:48:55 [SamShaw]: Joe_Humbert So I should be capturing the default output with usage hints
17:49:21 [SamShaw]: Matt_King Okay so there is this document with the goals for the next phase of the automation project, please review and comment
17:49:32 [SamShaw]: Matt_King Mike lets work on 1045 and create an action
17:50:29 [SamShaw]: jugglinmike I created another issue that we need to fix as well, I'll paste a link to it
17:50:54 [jugglinmike]: https://github.com/w3c/aria-at-app/issues/1046
17:50:58 [SamShaw]: jugglinmike I think we need to discuss how to modify the statuses
17:51:03 [SamShaw]: Matt_King Okay we can discuss
17:51:12 [SamShaw]: Matt_King lets move on to the Dialog Test plan
17:51:18 [jugglinmike]: scribe+ jugglinmike
17:51:33 [jugglinmike]: Topic: Dialog test plan
17:51:48 [jugglinmike]: Matt_King: one of the things that we're talking about testing is related to testing the boundaries of a modal dialog
17:51:59 [jugglinmike]: Matt_King: e.g. not letting you fall outside of it when you are reading
17:52:13 [jugglinmike]: IsaDC: had a question in the issue, so I added an agenda item to discuss it here today
17:52:29 [jugglinmike]: IsaDC: How should we structure the test? How far do we want to test that?
17:52:51 [jugglinmike]: Matt_King: I encourage people to open the link in the agenda to review the draft test plan
17:52:58 [jugglinmike]: https://deploy-preview-1049--aria-at.netlify.app/review/modal-dialog
17:53:09 [jugglinmike]: Matt_King: Test six is "navigate to the end of the modal dialog"
17:53:46 [jugglinmike]: Matt_King: if you open the page for test six, run the test setup, I think one thing we might want to change is that right now, the focus goes to the input field labeled "street"
17:54:11 [jugglinmike]: Matt_King: I think if we were to set the focus on the button "verify address", then screen readers would automatically be in reading mode rather than browse mode
17:54:23 [jugglinmike]: Matt_King: And this test is about behavior in reading mode, after all
17:55:14 [jugglinmike]: Matt_King: I've been thinking that we could have another command, called something like "navigation beyond dialog boundary" where of course what we want is to observe this as being impossilbe
17:55:42 [jugglinmike]: Matt_King: The command sequence would be CTRL+END followed by down arrow, followed by up arrow
17:56:33 [jugglinmike]: Matt_King: That would have one assertion: "the name of the button 'add' is conveyed" or, more explicitly, "screen reader cursor is on the 'add' button" (possibly with better wording)
17:56:46 [jugglinmike]: s/impossilbe/impossible/
17:57:11 [jugglinmike]: Matt_King: Does that make sense to people
17:57:16 [jugglinmike]: IsaDC: I think it could be confusing
17:57:27 [jugglinmike]: Alyssa: What about trying to tab or shift+tab out of it?
17:57:46 [jugglinmike]: Matt_King: We have a separate test for that. A test for navigating to the beginning and the end
17:58:04 [jugglinmike]: James_Scholes: I don't know if we need such a test because that's controlled by the page
17:58:28 [jugglinmike]: James_Scholes: Oh, so we're not testing that the tab trapping is implementing correctly; we're testing the screen reader's response to the tab trapping
17:58:32 [jugglinmike]: Matt_King: Correct
17:58:35 [jugglinmike]: IsaDC: Yes
17:59:30 [jugglinmike]: James_Scholes: Ctrl+End is a special case because the dialog appears at the bottom of the page. Maybe we need to move the dialog so that something else exists below it
18:01:00 [jugglinmike]: jugglinmike: We're out of time
18:01:11 [jugglinmike]: James_Scholes: I think this requires additional discussion
18:01:22 [jugglinmike]: IsaDC: Matt_King can you draft your idea as a proposal?
18:01:24 [jugglinmike]: Matt_King: Sure
18:02:50 [jugglinmike]: Zakim, end the meeting
18:02:50 [Zakim]: As of this point the attendees have been Matt_King, jugglinmike, howard-e, lola, James_Scholes, Joe_Humbert, IsaDC, carmen, Alyssa
18:02:52 [Zakim]: RRSAgent, please draft minutes
18:02:53 [RRSAgent]: I have made the request to generate https://www.w3.org/2024/04/24-aria-at-minutes.html Zakim
18:03:00 [jugglinmike]: RRSAgent, leave
18:03:00 [RRSAgent]: I see no action items
18:03:00 [Zakim]: I am happy to have been of service, jugglinmike; please remember to excuse RRSAgent. Goodbye
18:03:00 [Zakim]: Zakim has left #aria-at