Meeting minutes
Next patterns to write tests for
JS: Few weeks ago developed a list of patterns, and we've finished the ones we can wrk on at this time.
Done with buttons, started horizontal slider, nav tree under review.
blocked on other items on list b/c they are getting updated.
includes vertical sliders
Are there others we should avoid.
Or should PAC make a list of suggestions.
MK: not aware of any to avoid.
Hadi: why we call simple button a command button.
js: to distinguish between toggle and simple, steering clear of just button which has multiple examples.
JS: Propose alert, alertDialog, and breadcrumb can start now.
We can work through the list from the top alphabetically.
MK: Propose we discuss which patterns at start of meetig each week
Test Format
js: Deep dive based on initial proposal that Bocoup put together
This is what we call hybrid format, supporting both manual and automated testing.
Added list of issues that need to be addressed by the format proposal to the agenda, probably can't deep dive into each issue today.
Seth: start by talking about the proposal details. We have identified numerous dependent issues and a few subtopics.
Including what data is in the test and how it is encoded.
Want to have a phased approach to updating existing tests.
Start with overview of proposal.
Want to understand how this is coming across, how understandable, or need additional detail? Do we need to break it apart into more digestable pices.
Both an encoding format and content spec.
Currently have a few csv files that reference each other.
They generate a bunch a bunch of html files that embed data into json
Bit question is how we can make info in csv tables encode the same meaning for manual test writing but also accomodate more info needed by an automated runner.
Current format has a csv file called tests.csv that includes 3 key things: instructions for humans, machine readable kb commands (referenced by an ID in commands.csv), and seies of assertionswhich are stated in natural language for a human to understand.
What part of this format breaks if trying to have a machine run the tests?
Assertions need to become some sort of pseudo code for a machine.
js: the machine does not need to know anything about the intent of the test.
it needs to know to press x to jump to checkbox. It needs to know what key to press, but it does not need to know intent.
Seth: Let me back up and make my observation more generic. Where test format has separate human andmachine readable content, our goal is to differentiate between a part that the human needs to read and a part that machine needs to consume or parts that could be both.
Yes, the the machine does not need to know the purpose of the command, but there is some benefit to standardizing on a syntax that can be used consistently for all the info in the table.
One of the big changes in proposal is that we are thinking about merging instructions, commands, and assertions into a single element.
js: not understanding why the assertions need to be machine readable. The human assertions can each by matches to a given output.
How would we templatize something like "presence of autocomplete is conveyed.
Theoutput is what determines what passes or fails. We can state good output.
I don't think the tests list should be responsible for listing all the ways to achieve a given output. There are a lot of ways to get a given result, e.g., speak a checkbox.
Seth: recognize this is a departure that comes with a cost.
Want Alex to give some toughts on this?
Alex: Talking what each cell might look like in a row of the csv.
Want to describe accurately without getting muddled in details.
When we say Press:X, that doesn't have to be a specific key. X could be a list of commands.
Sina: what is the overarching problem we are tyring to solve.
mk: Do I understand right that X could be a command that represents in kind or amount of input?
Alex: yes
JS, Sina : that helps
Sina: Is it the case that most of the time we expect the speech output for a variety of similar commands, e.g., x, f, tab to go to checkbox, to result the same output.
JS: no
So, the 2 variables that most often effect output are AT and mode
Sina: What are the problems not being solved here?
James: templatized reusable assertions
js: Think we need iterative approach to the problems that addresses some of our immediate needs to make test writing easier and more understandable.
that could add more understanding to the conversation.
mk: seems like we need a workshop of some kind.
Seth: agree, le't cut meeting here. sounds like we need a slightly different approach to making decisions, maybe a workshop.
Sina: we have a known list of simple changes, workshop can move forward from that point.
Seth: sounds good, we don't need to use workshop time on that.
js: Before workshop, put together some simple examples, and propose how it would look in the end
Seth: did try to show before and after but maybe tring to make moves that were too big
Seth: great feedback; we'll be in touch about next steps.