16:05:34 <RRSAgent> RRSAgent has joined #aria-at
16:05:34 <RRSAgent> logging to https://www.w3.org/2019/09/04-aria-at-irc
16:05:43 <mck> mck has joined #aria-at
16:07:11 <mck> MEETING: ARIA and Assistive Tech Community Group Telecon
16:07:37 <mck> CHAIR: Matt King
16:07:45 <mck> rrsagent, make log public
16:07:48 <spectranaut> spectranaut has joined #aria-at
16:07:54 <mck> rrsagent, make minutes
16:07:54 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/04-aria-at-minutes.html mck
16:08:05 <mck> present+ Matt_King
16:08:16 <michael_fairchild> present+
16:08:24 <spectranaut> present+ valerie_young
16:10:16 <isaac> isaac has joined #aria-at
16:11:17 <jongund> present+ jongund
16:13:26 <michael_fairchild> scribe: michael_fairchild
16:15:04 <mck> Discussion of roadmap for prototype: https://github.com/w3c/aria-at/wiki/Fall-2019-Research-and-Prototype-Roadmap
16:15:46 <michael_fairchild> Topic: Prototype roadmap
16:15:50 <mck> https://github.com/w3c/aria-at/wiki/Fall-2019-Research-and-Prototype-Roadmap
16:17:18 <michael_fairchild> Valerie: The first step is to decide on a generalized language for testing across different screen readers
16:17:54 <michael_fairchild> MK: I'm looking at it this way: what should the column names be and what should they mean?
16:18:06 <michael_fairchild> Valerie: Matt, can you make a first draft of that?
16:18:23 <michael_fairchild> Valerie: Yes, I can do the first draft.
16:18:52 <michael_fairchild> MK: This will be higher level language and not really screen reader commands
16:19:19 <michael_fairchild> Valerie: Should we have issues related to each wiki page to discuss changes?
16:19:42 <michael_fairchild> MK: Yes, that's a good idea. I'd like to figure out a better structure for the wiki too.
16:20:49 <michael_fairchild> Valerie: We also need to decide the actual structure of the test. So far we have been using excel documents, but hopefully we will have a better defined structure that we can lint and import into our testing software
16:21:58 <michael_fairchild> MK: we also need to decide if we will be importing into existing software or something bespoke
16:22:49 <michael_fairchild> Valerie: I imagine there will be a difference between the tests that are authored and what a tester uses to record results. I don't think the tests should be coupled with whatever software we use.
16:23:31 <michael_fairchild> MK: That's more clear to me now. What are the elements within a test vs what is specific to a execution or run of a test?
16:24:46 <michael_fairchild> Valerie: There is also an image of this where we are currently writing tests in an excel document, but I think we want to have them version controlled in git so that we can support pull requests.
16:25:48 <michael_fairchild> MK: So the anatomy of a test itself is really important. Because someone might make a pull request for how to run a specific test vs the test itself.
16:26:36 <michael_fairchild> MK: So when you say design a structured test format, outside of the excel format we are not doing this.
16:26:45 <michael_fairchild> Valerie: We could use a wiki page
16:27:00 <Jemma_> Jemma_ has joined #aria-at
16:27:29 <michael_fairchild> Valerie: This afternoon I could make a wiki page to describe this information. Thinking about this as input to a system.
16:28:30 <michael_fairchild> Valerie: 1.3 is author test material. This is a set of tests that a user could perform, in the structure that we have defined.
16:28:57 <Talita> Talita has joined #aria-at
16:29:09 <Jemma_> present+ JaEunJemmaKu
16:29:54 <michael_fairchild> Valerie: 1.4 needs some explanations. In tests of specs (this isn't a test of a spec), there are usually tens of thousands of tests, but they were written without a means to track the coverage of the spec. There isn't a spec for screen readers, but we could track coverage of the APG examples, ARIA, etc.
16:30:14 <michael_fairchild> MK: I think we should do it the same way we do coverage for regression testing in the APG
16:31:56 <michael_fairchild> Valerie: I think this might be something to figure out once we have a better understanding of how the tests will be structured
16:33:05 <michael_fairchild> Valerie: 2 is tightly related to number 1. Result data model depends on number 1.
16:34:21 <michael_fairchild> Valerie: item 3: is the test author contributing guidelines. I imagine that lots of people will be contributing test material. Since there is no standard for AT, this is about coming to consensus on how AT should behave.
16:34:36 <michael_fairchild> Valerie: I think that is something we should have an initial version of in the next few months
16:35:14 <michael_fairchild> Valerie: item 3.3: a list of features to be tested.
16:35:20 <michael_fairchild> MK: What features?
16:35:35 <michael_fairchild> Valerie: Probably a list of APG examples that are in scope for the project.
16:36:32 <michael_fairchild> Valerie: item 3.4: maybe I should move this one. guidelines for contributions. Describe how to cover a new APG example.
16:37:35 <michael_fairchild> MK: There might be two parts: 1. agnostic tests (same for all screen readers), 2. specific vendor tests
16:38:31 <michael_fairchild> MK: is 3.4 about how to executing a specific test with a specific technology?
16:38:39 <michael_fairchild> Valerie: that's a good question.
16:40:15 <michael_fairchild> Valerie: 3.5 is 3.4 but written out a bit more.
16:40:59 <michael_fairchild> Valerie: I'll write this last 3 points in section 3 more clearly
16:41:42 <michael_fairchild> Valerie: 4. tester application: I'd like if we had a more concrete list of features for the testing app, from nice to have to must have.
16:42:02 <michael_fairchild> Valerie: we have a lot of questions in issue 8 related to this
16:43:03 <michael_fairchild> Valerie: 4.3 - draft of tester and training materials, we should have an idea of how we are communicating instructions to testers.
16:43:43 <michael_fairchild> Valerie: 5 - report mockup, we want to be able to have a simple way to show results of the tests that we have done so far. Whats passing, what is failing, etc.
16:44:05 <michael_fairchild> Valerie: and thats what we need to figure out in the next few months, and then I'll build a prototype.
16:47:14 <michael_fairchild> MK: this means that any part of ARIA that doesn't have an example in the APG won't be included in our results. But if we start our coverage of the APG we will get a good foundation, but we won't get a good understanding of the entire spec. I see those as future tasks.
16:48:08 <michael_fairchild> Valerie: I wonder if we can map APG examples to the ARIA spec so that we can have a better understanding of spec coverage
16:48:14 <michael_fairchild> MK: Yes, we are working on that
16:49:28 <michael_fairchild> Jemma: Are we planning on having a caniuse type document for ARIA? Have you seen the documentation by Level Access? They have some good documentation.
16:49:53 <michael_fairchild> Valerie: do you have a link?
16:51:22 <Jemma_> https://www.powermapper.com/tests/screen-readers/aria/
16:54:00 <michael_fairchild> Topic: next priorities
16:54:25 <michael_fairchild> MK: we need to discuss priorities for Valerie
16:55:41 <michael_fairchild> MK: A top priority should be to research existing testing frameworks and software
16:56:48 <michael_fairchild> Valerie: there are not very many options, so this should be fairly quick. There might be two viable options that are open source. The next step will be to install them and to determine how configurable they are.
16:57:44 <michael_fairchild> MK: we also might be able to explore licensed software. If open source isn't an option, we need to decide if we want to build our own or use licensed software.
16:59:00 <michael_fairchild> MK: so we need to figure out if something open source will meet our needs
17:11:59 <spectranaut> exit
18:43:21 <Zakim> Zakim has left #aria-at