<JoeHumbert> is there a web-based agenda?

<JoeHumbert> hello

<shimizuyohta> Hi. According to the mail from Michael yesterday, tentative agenda is as follows:

<shimizuyohta> 1.Set the agenda 2.Discuss and create a test for the simple checkbox example

<JoeHumbert> right I saw the agenda in the email

We will set the agenda in a little bit

<JoeHumbert> present*

<scribe> scribe: mf

introductions and questions

matt: some background and history. We are still in the exploratory phase (trying to figure out what we need to know in order to develop a prototype).
... Yohta has dome some work in assessing the menubar example, using VO with chrome/safari, started with JAWS

<shimizuyohta> https://drive.google.com/drive/folders/1pOwoCWpT9uJgZZxFd36RHn9iBOiW9Tae

Yohta: I have been working updating some of those assertions - see above URL
... after the meeting, please provide feedback on those assertions if you have time. I want to hear your feedback

<shimizuyohta> File name:19_0402Menubar_assertion_role.xlsx

Matt: one of the things we want to do as soon as we can is to develop our data model. Once we have a reasonable idea of what the data model would look like, we can start developing a backend to support testing and collecting test results.
... we have come to some basic conclusions on methodology, which will help us define some use cases for testing
... basic conclusions on methodology are that we need metadata to do a test. 1. we need a name of the test. 2. we need a url to the test. 3. we need the AT, browser names and versions.
... for the test themselves, we have assertions for one of two categories (reading mode and interaction mode categories)
... assertions will have preconditions, such as mode and screen reader settings (to keep it simple we are going to start with default screen reader settings)
... the second thing is that every single assertion is going to start with a command being executed by the user (such as a screen reader command to navigate to the next menu item or read the next line)
... and then the assertion is that the screen reader behaves is a certain way, but there could be multiple assertions for a single behavior.
... (provided an example assertion for the menubar pattern)
... we want to create assertions for anything in ARIA that provides semantics.
... for every single assertion we will have an accessibility semantic that we will be testing.
... that boils down to the mode, the keyboard command, the assertion, and the attribute that we are testing

<JoeHumbert> is that required info documented somewhere?

Jon: a fundamental assertion is that if something has a role then the role is conveyed. Will we be testing all of the different ways that a user can change or move screen reader focus?

Matt: yes, because different commands to move screen reader focus can yield different and incorrect results
... so we are not testing if it gets focus, we are testing if information is conveyed correctly for each possible interaction

Jon: do we care if focus is not moved correctly?

Matt: no, because that would be a bug in the APG example, not the screen reader

Jon: how do we describe this pre-condition? such as navigating within a menu (down arrow on last item moves focus to the first item)

Matt: maybe one pre condition that we have to state is that we have define where focus or the screen reader carrot is before the associated command is executed
... state where both DOM focus is and where the reading cursor position is
... but the DOM focus location may not matter for reading mode tests. So what needs to be defined might depend on the assertion

Joe: are required information going to be documented anywhere (at/browser versions, etc)?

Matt: it will be. We have notes, and we will have it documented when we start creating the data model

Jon: after the command for the assertion is executed, do we want to document changes in DOM focus and/or screen reader cursor?

Matt: I'm not sure. I don't think we need to. (menubar example)
... (concerns about documenting those results being too verbose and taking too much time)
... I don't think we need to document every single instance of a test. We instead look for all the different ways we can test right arrow and the applicable assertions, and record where something fails.
... does that make sense?

Jon: what is the assertion? Is it "im in a menu item that is in the context of a menubar", then "move focus to another item in the menu bar" and not say how focus was moved?
... there might be a hierarchy of assertions. There might be a general assertion of "whenever an element of a role gets focus, the role is conveyed"

Matt: not always true; groups for example

Jon: true, but we could still provide a general assertion and override it for a specific case like a group
... some attributes such as aria-haspopup can change how a role is conveyed

Corbb: I'm not sure that we need to document where focus goes

Matt: yes. the APG already has defined keyboard behaviors and where focus will go
... I don't see a hierarchy because behaviors and expectations can be different from screen reader to screen reader. This is where we get into "must haves", "should have" or "may have"

Jon: So in general, we document the mode, focus and carrot position, keyboard command?
... described a special case where no output would be expected

Matt: well, something should happen in that example. In VoiceOver if you reach a boundary, voice over will make a "bonk" noise

Michael: 15min left

Matt: I'd like to talk about next steps

Next steps

Matt: I started an assessment of the checkbox example. Its not is a shareable state yet, but will be soon.
... but from all of this work I want to get to a place to build a data model
... is there someone that can take the lead on documenting this and propose how we can build it? (JSON, vs database, etc)

question: how would the data model be different from how a spreadsheet is structured?

Matt: JSON or whatever would house meta information at a higher level (a single title for the assessment, with multiple arrays of results)

Corbb: if we go that route we might limit our testing pool. That might be a barrier to entry for testers

Matt: We would have a front end for collecting data
... testers would just fill out a form

Corbb: that makes sense

Seth: I'd be happy to help with the data model

Michael: I have a data model already created, which is similar but different

Matt: what are those differences?

Michael: I can walk folks through what I have now in the next call

Matt: and I can finish my checkbox assessment
... we could also create a spreadsheet format so that we can start collecting some results before we have a data model created

Corbb: that makes sense. Then we could get some feedback on the data model before the actual modal is finished. We will get a lot of feedback on that.

Matt: Yep. And given Yohta's results and my results, we can start developing a data model

Michael: 5min

<SethKovanic> That works for me.

Matt: for next week, I will finish the checkbox. Yohta will work on assertions. Next week, we will compare with what Michael has created

This is scribe.perl Revision: 1.154  of Date: 2018/09/25 16:35:56  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Present: Matt-King michael_fairchild westont JoeHumbert shimizuyohta Jean-Francois_Hector Jon_Gunderson corbb
No ScribeNick specified.  Guessing ScribeNick: michael_fairchild
Found Scribe: mf

WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: 

WARNING: Input appears to use implicit continuation lines.
You may need the "-implicitContinuations" option.


WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)