20:08:38 <RRSAgent> RRSAgent has joined #aria-at
20:08:38 <RRSAgent> logging to https://www.w3.org/2021/02/11-aria-at-irc
20:08:55 <Matt_King> MEETING: ARIA And Assistive Technology Community Group
20:09:06 <Matt_King> CHAIR: Seth Thompson
20:09:15 <Matt_King> rrsagent, make log public
20:09:22 <Matt_King> rrsagent, make minutes
20:09:22 <RRSAgent> I have made the request to generate https://www.w3.org/2021/02/11-aria-at-minutes.html Matt_King
20:09:28 <Matt_King> present+
20:09:34 <boazsender> present+
20:09:35 <jscholes> present+
20:09:38 <michael_fairchild> present+
20:09:38 <s3ththompson> present+
20:09:47 <jesdaigle> present+
20:09:57 <boazsender> scribenick: boazsender
20:10:04 <Matt_King> present+ Hadi
20:10:30 <boazsender> scribe: boazsender
20:13:59 <boazsender> s3ththompson: we're kicking off our 2021 work on ARIA-AT project in our standard way at Bocoup, but this year we're going to be running the project in public.
20:14:03 <Jemma> present+
20:14:07 <boazsender> TOPIC: introductiosn
20:14:20 <boazsender> s/introductiosn/introductions
20:15:15 <boazsender> s3ththompson: we have myself Seth Thompson (he/him), working for Bocoup as Product Owner, Jes Daigle (she/her), eng manager at Bocoup, and Boaz Sender (he/him) supporting the project from Bocoup.
20:15:31 <boazsender> s3ththompson: we also have Mat King who is the project sponsor
20:15:49 <boazsender> s3ththompson reads the Bocoup Code of Conduct summary, and references the w3c CEPC
20:15:59 <boazsender> TOPIC: Scope
20:16:30 <boazsender> s3ththompson: (Reading out the goal from the agenda) our goal is to design and implementation of a hybrid manual and automated testing architecture in the aria-at.w3.org app and github.com/w3c/aria-at test suite. Improvements to the existing manual system. Strategic changes to test authoring tools that help test-authors scale up output. Support for new strategic test suite development.
20:18:21 <boazsender> Matt_King: I want to clarify that this is the app and test runner, but that Sina is running a different project writing tests, and Bocoup is also running another separate project to run automation.
20:18:27 <boazsender> s3ththompson: good point
20:19:16 <boazsender> s3ththompson: this project scope is the app running at https://aria-at.w3.org, the database and backend that stores results from that app, and the reports we're making with those results
20:19:21 <boazsender> TOPIC: success criteria
20:19:43 <boazsender> s/success criteria/Success Criteria
20:19:57 <boazsender> s3ththompson: like we said this is currently a manual test running app
20:20:43 <boazsender> s3ththompson: and we also want to automate, which we'll get to in our success criteria
20:21:01 <boazsender> s3ththompson: our first criteria is about supporting material outside of APG: "Integrate automated tests alongside manual tests and extend tests beyond APG"
20:21:35 <boazsender> s3ththompson: our second criteria is to scale our design to support a lot more tests and test results: "Prove that our app scales as test corpus and results matrix increases in size"
20:22:24 <boazsender> s3ththompson: our third success criteria is about making a more unified experience across all our connected projects in this area, and deal with our acronym soup better: " Unify identity across WAI, ARIA-AT, and APG"
20:23:14 <boazsender> s3ththompson: our fourth success criteria is to make sure that the data from this app are available to web devs, browser teams, and AT makers: "Bring ARIA-AT testing insights to web developers, browser implementers, and AT vendors"
20:23:29 <boazsender> s3ththompson: and our last criteria is to "Involve screen readers and marginalized AT users, manual testers, and others as co-designers"
20:23:46 <boazsender> s3ththompson: anyone else have anything to add to the success criteria?
20:23:58 <boazsender> michael_fairchild: I think these are great, and I don't have others to add.
20:24:17 <jongund> jongund has joined #aria-at
20:24:18 <boazsender> Jemma: as someone working on APG, I'm really happy to see the integragtion.
20:24:51 <boazsender> Matt_King: I think we might want to make some of these more specific, once we are ready to discuss our sucess metrics
20:25:29 <boazsender> Matt_King: it might be good for us to add key success metrics to the aria-at app
20:25:44 <boazsender> s3ththompson: I like that idea
20:25:58 <Jemma> One of measurements can be the percentage of integration rate with exisiting APG examples
20:26:13 <boazsender> the next section is priorities, let me add a measurment dashboard to that priorities list
20:26:53 <jongund> present+ jongund
20:26:56 <boazsender> s/the next section is priorities, let me add a measurment dashboard to that priorities list/s3ththompson: the next section is priorities, let me add a measurment dashboard to that priorities list
20:27:21 <boazsender> s3ththompson: there was also sina's idea about number of key strokes
20:27:23 <boazsender> Matt_King: yes
20:27:49 <boazsender> Matt_King: I think it would be great if 2-3 times this year we could stop, take a breath and share with the world where we are
20:28:02 <boazsender> s3ththompson: yes, great, lets make a priority for that, potentiall
20:28:11 <boazsender> TOPIC: Priorities
20:28:18 <boazsender> s3ththompson: let me read through the prios
20:28:47 <boazsender> I went through all the work we had in our backlog at the end of last year, and the above criteria to write these
20:29:09 <boazsender> s3ththompson: first, we'd like to focus on the API: "Build a better API foundation and data model that encompasses alternate test sources and automated tests"
20:29:30 <boazsender> s3ththompson: right now we assume all tests are manual and from APG, and we might change that
20:30:21 <boazsender> s3ththompson: next we might do something that to "Improve smart updates for evergreen tests", and this really speaks to the ability to make small changes to test results with minimal overhead
20:30:39 <boazsender> s3ththompson: next we'd liket to "Evolve ARIA-AT working mode and improve alignment with App publication workflow." this goes back to the bar for merging tests.
20:31:08 <boazsender> s3ththompson: the working group might be able improve workflow based on the state of a test
20:31:47 <boazsender> s3ththompson: the next priority is to " Develop rendering engine for test runner as external library (remove iFrames from App)." this is in order to improve accessibility for AT users by getting rid of iframes.
20:32:07 <boazsender> this could also be for other consumers, not just the app
20:32:26 <boazsender> s/this could also be for other consumers, not just the app/s3ththompson: this could also be for other consumers, not just the app
20:32:38 <boazsender> s3ththompson: next priority is "Add filtering and advanced analysis features to accessible Reports page."
20:32:48 <Jemma> yay for integrating with APG site!
20:32:52 <boazsender> s3ththompson: next we'd like to "Build embeddable version of Reports page for MDN and APG."
20:33:09 <boazsender> s3ththompson: next we'd like to "Upgrade App to support hybrid manual and automated test results." This will be complex.
20:33:33 <boazsender> s3ththompson: next priority is to "Upgrade App to support hybrid manual and automated test results." so again this about allowing us to test non-apg patterns.
20:33:49 <boazsender> s3ththompson: finally, wed like to "Add accessible UX affordances to handle increased scale of data, including pagination and search"
20:34:44 <Jemma> It sounds like a good idea to start with backlog from the previous project.
20:34:48 <boazsender> s3ththompson: do folks have additional priorities in mind?
20:35:01 <boazsender> Matt_King: I think this is the most important part of the meeting.
20:35:24 <Jemma> The flexibility of AT app embed is great.
20:35:36 <boazsender> Matt_King: we really want folks input on this.
20:35:54 <Jemma> but I do see the challenge for builiding API itself.
20:35:55 <boazsender> Matt_King: I really want alignment on these pieces.
20:36:21 <boazsender> s3ththompson: this is about the big pieces of the work for us to align on.
20:37:12 <boazsender> s3ththompson: I see a comment in IRC from Jemma about starting with the backlog. Which is good. We did start there, and this comes from that. Thanks!
20:40:08 <boazsender> Matt_King: let me start with something very specific: it would be amazing if we could prioritize the working mode changes, and merging a test to the main branch easliy.
20:40:50 <Jemma> "Parallel velocity" is really a good key word to remember.
20:41:28 <boazsender> s3ththompson: ok, I'll move that up.
20:41:51 <boazsender> Matt_King: my next question is, is there a feature in the app that is slowing down automation.
20:42:08 <boazsender> s/Matt_King: my next question is, is there a feature in the app that is slowing down automation./Matt_King: my next question is, is there a feature in the app that is slowing down automation?
20:44:35 <boazsender> Matt_King: my 10k foot view is that humans will run this the first time always.
20:44:48 <boazsender> boazsender: I was thinking sometimes we could start with automation
20:45:10 <boazsender> jscholes: I think we need a human tester at the beginning
20:45:28 <boazsender> michael_fairchild: I agree, but I still think we can automate them earlier, like boazsender is saying
20:45:57 <boazsender> s3ththompson: I wonder if we are conflating the tool as an app for test writers and one for test runners, and perhaps there are two different workflows.
20:46:07 <boazsender> q+
20:46:25 <boazsender> Matt_King: I was hoping the app could manage this
20:46:36 <boazsender> Matt_King: manage some of the writing
20:46:43 <boazsender> s3ththompson: I'll clarify that in the priorities
20:46:53 <boazsender> s3ththompson: is there anything else we're missing?
20:46:59 <boazsender> q-
20:48:06 <Jemma> +1 for Boazsender
20:48:54 <Jemma> agenda?
20:48:56 <boazsender> Matt_King: do we have a bucket of priorities around usability and accessibility?
20:49:07 <boazsender> agenda: https://github.com/w3c/aria-at/wiki/February-11%2C-2021-Community-Group-Meeting
20:49:30 <boazsender> Matt_King: do we have something to revisit the ux assumptions after research?
20:49:45 <boazsender> s3ththompson: great point, I'll add that.
20:50:13 <boazsender> jscholes: how do we communicate priorities we have for the script/runner changes?
20:50:20 <boazsender> jscholes: does that go here?
20:51:01 <boazsender> s3ththompson: yes, this is the right place. We can add a priority to focus on API changes.
20:51:40 <boazsender> Matt_King: I think this is related to things like reusable tests, and the way we compile key commends
20:51:52 <boazsender> jscholes: yes, but also github actions, and test formats
20:53:04 <boazsender> boazsender: maybe the prio could be "meet with test writers, catch their pain points, work them into the project"
20:53:14 <boazsender> jscholes: we could name all our issues in the next test writing meeting
20:54:10 <boazsender> s3ththompson: I think some of this is also covered by our named priorities, like expanding past APG, to include material from ARIA attrs, etc
20:54:26 <boazsender> Matt_King: agree
20:54:30 <boazsender> jscholes: agree
20:54:38 <boazsender> jscholes: its the data model
20:54:43 <boazsender> s3ththompson: exactly
20:54:54 <boazsender> Matt_King: could we make this the focus of the next test writing meeting?
20:54:57 <boazsender> s3ththompson: sure!
20:55:32 <boazsender> s3ththompson: ok, wrapping up priorities, please let know after the call
20:55:46 <boazsender> s/s3ththompson: ok, wrapping up priorities, please let know after the call/s3ththompson: ok, wrapping up priorities, please let know after the call if you have anything to add
20:56:05 <boazsender> TOPIC: Logistics
20:56:28 <boazsender> s3ththompson: we're starting work at Bocoup on February 16th.
20:56:44 <boazsender> s3ththompson: what time is the test writing meeting? is it the same time?
20:56:53 <boazsender> Matt_King: no, it's different every week.
20:57:09 <boazsender> s3ththompson: ok, can everyone who wants to attend this regularly send me their calendars?
20:57:37 <boazsender> Matt_King: one possibility might be the hour before this meeting. 11-12 PST on Thursdays
20:58:17 <boazsender> s3ththompson: ok, we'll look at that
20:59:03 <boazsender> s3ththompson: other things to share: we're moving our task management to public github.
20:59:41 <boazsender> scribe: jongund
21:00:30 <jongund> Seth: that last time we did work together, there was sprint planning and then share with Matt
21:00:42 <jongund> Secth: Is there a better way to share weekly sprints
21:00:43 <boazsender> rrsagent: generate
21:00:43 <RRSAgent> I'm logging. I don't understand 'generate', boazsender.  Try /msg RRSAgent help
21:01:19 <jongund> James: We are interested in the schedule to know what's happening so we can comment if needed
21:01:33 <boazsender> rrsagent: please create the minutes
21:01:33 <RRSAgent> I have made the request to generate https://www.w3.org/2021/02/11-aria-at-minutes.html boazsender
21:01:41 <jongund> Seth: Publish expected sprint plan for comment
21:01:52 <jongund> Seth: I think we can do that
21:02:37 <jongund> Matt: Especially for the first couple of priorities this is important, we want James involved the sprint
21:02:49 <jongund> James; I would be happy to attend
21:03:16 <jongund> Matt: That would be a stronger alignment between app and test development
21:03:52 <jongund> Seth: Previously we would make changes to the App and assign someone on the team to review
21:04:16 <jongund> Seth: Is anybody willing to review
21:04:55 <jongund> Matt: I don't want external dependencies, but we want to do planning so external people have a chance to review
21:05:16 <jongund> James: I would be happy to look at accessibility
21:05:59 <jongund> Jon: I would be interested too, I have accessibility concerns
21:06:57 <jongund> Matt: I would like to consider the process for the accessibility and usability, that has a longer time line, put in staging and get feedback, deploy after feedback is addressed
21:07:09 <jongund> Seth: I put some work into the design process
21:07:47 <jongund> James: It is a lot easier to review the insitu, the actual app
21:08:40 <jongund> Matt: That speaks James to the idea, the accessibility and usability of the reports page, break up development into 6 small steps that we all agree are great
21:09:10 <jongund> Matt: I think the community group can handle a1 month pace, people will appreciate the monthly progress
21:09:26 <jongund> Seth: New things on a monthly schedule
21:09:47 <jongund> Matt: Mostly the UI stuff, the backend can work on its own schedule
21:09:55 <jongund> Seth: I like that framing
21:10:31 <jongund> Seth: it involves our deployment process with staging and data
21:11:01 <jongund> James: The UI might be at a different pace than the data model, we can do some testing with the UI
21:11:51 <jongund> Seth: This is a bigger thing than what we were talking about, because we are using a W3C server, its not quite so easy than using commercial deploy
21:12:37 <jongund> Matt: I don't want that be a major topic, let's do the best we can, humans are going to need to adapt to the infrastructure
21:13:08 <jongund> Seth: I won't invent new work items, that is what I had for work plan and communication
21:13:48 <jongund> Seth: Even though we have two separate repositories, but they are all one project
21:14:01 <jongund> Seth: any other questions be fore action items?
21:14:52 <s3ththompson> Action Item: Email James to set up time to create shared priorities with test writing (s3ththompson)
21:15:38 <s3ththompson> Action Item: Develop project plan from kickoff priorities / roadmap (Bocoup)
21:16:05 <s3ththompson> Action Item: Confirm AT Automation time with Simon and coordinate meeting invitation (s3ththompson)
21:16:42 <jongund> Seth: One zoom link for all meetings?
21:16:50 <jongund> Matt: the Deque one
21:17:02 <jongund> Seth: I will use that link for all ARIA AT meetings
21:17:19 <jongund> Seth: When are they sent out, Monday?
21:17:35 <s3ththompson> Action Item: Seth to send out regular meeting invites for ARIA-AT on Wednesday, with tentative sprint plan (s3ththompson)
21:17:43 <jongund> Matt: We usually do Wednesday, but they can be earlier
21:18:03 <jongund> Seth: Probably Wednesday will have best informatino
21:18:27 <jongund> Matt: Seth you need to become a cohost
21:18:37 <s3ththompson> Action Item: Confirm that Seth can be co-host for meetings (s3ththompson)
21:18:58 <jongund> Action Item: Seth confirm with Michael you can be a cohost
21:19:25 <jongund> Seth: Any other topics?
21:19:51 <s3ththompson> Action Item: Confirm time for weekly ARIA-AT workstream, tentatively 2pm ET Thursdays (s3ththompson)
21:20:07 <jongund> Matt: Next week there will be three meetings, automation on Wednesday,
21:20:21 <jongund> Matt: How long before a project plan?
21:20:41 <jongund> Seth: This will probably start in about 3 weeks, look for the invitation
21:21:02 <jongund> Matt: There will only be two meetings next week
21:21:16 <jongund> rrsagent, draft minutes
21:21:16 <RRSAgent> I have made the request to generate https://www.w3.org/2021/02/11-aria-at-minutes.html jongund
21:40:14 <boazsender> it doesn't seem like the minutes worked...
21:56:45 <boazsender> ah, they are here https://www.w3.org/2021/02/11-aria-at-minutes.html
23:12:32 <Zakim> Zakim has left #aria-at