<rob_fentress> +present
<scribe> scribe: jongund
MK: Reviewing open pull
requests
... Community group project board, I have not done anything
yet
... Some issues from last week
... Standard work stream updates
... Is Seth on?
ST: Yeah folks
MK: What do we need to cover
today?
... Updates on each of the work streams, what are the
priorities
ST: That may need some
discussion, and added deep dive call discussion, two action
items to discuss
... Title versus the task
... We should give a summary to the group
MK: A summary of where we stand,
test writing updates Sina for test writing
... App work stream update
... Update on the automation
... Then test writing
ST: Issue 337
SP: 349 is about automation and test format
MK: Several things going on in the app space
ST: Quick update
... Ezak is doing some usability testing on interface
... The engineering team is getting the design right about the
report page
... We need to spend more time on the structure of the table,
working with Sina et al.
... Looking for a nice simple implemenation soon
MK: For this year, the reports page, where we have outstanding work?
ST: That's correct, and we will spend some time fixing highest priority usability issues
MK: We will not have anytime as
the group to talk about the issues
... Usability testing results next week?
ST: yes
MK: We can talk about that on the
17th, but the issues maybe resolved by then
... What is the schedule for work as we approach the holidays,
we will meet on the 17th, how much work after the 17th
ST: 6 days of work
MK: There will be a little bit of time for anything discussed on the 17th
ST: Some time before the end of the year other things may need to wait until next year
MK: Any other questions?
MK: Simon you have a lot of progress
SP: Proposed a test format for
instructions and assertions, implemented that in NVDA
... Working in NVDA, here is a link
<zcorpan_> https://docs.google.com/document/d/1YxHAf6r3E2RG4REaYcmM14DuxfBgP4XoWRdfSk5aAQM/edit
SP: In issue 349, describes this
test format, some questions
...
https://github.com/w3c/aria-at/issues/349
<zcorpan_> open questions: https://github.com/w3c/aria-at/issues/349#issuecomment-737902356
MK: Should talk about this now or wait until 337
SP: People should look at the format and give feedback
MK: What does it feel like runnin... how do you feed this test file to NVDA
ST: It essentially opens chrome
and then a driver intercepts NVDA to get the text strings
... you run it from the comand line, you see things happening
on chrome, but the speech is being intercepted
... A server could be used to collect results and generate
reports
MK: I also wonder what you think about, if we should try to do this with another screen reader, to see what the differences and constraints maybe
SP: Yes maybe
SB: Is this dependent on NVDA, is it just a speech driver
SP: Not sure
SB: We could write a SAPI driver,
then it would work for NVDA and JAWS
... It could be screen reader agnostics
... You could even capture the audio
JS: If you want to tie in some annotations in the result, some string based agnostic
<Jemma> SAPI:Speech Application Programming Interface
MK: I am thinking about, the
reason I asked that question, it may affect at one level we
want to abstract
... This driver can drive any screen reader, ... the implement
for all screen readers
SB: It will not work on macOS
MK: The test format should be independent of the screen reader, but what does a driver looks like
SP: The test format can be the same across all screen readers...
JS: We should do an abstrct, if
its JAWS do this, if it is voiceOver do this
... If we want next item, and then a central command reference
that is screen reader
MK: That's a different type of
abstraction than what I was thinking about
... Back to this basic question about what we do right now.
before we nail down to many things, should we repeat with
another screen reader
SP: We can make decisions on the
information we have now
... We may need to revisit with other screen readers
MF: I have done some command abstraction, and I can share
SP: I am interested
<michael_fairchild> https://github.com/accessibilitysupported/a11ysupport.io/blob/master/data/ATBrowsers.json rendered here https://a11ysupport.io/learn/commands
SP: Any other thoughts on this topic?
SB: We alos want to monitor braille output in addtion to speech
MF: This made me think
... Is it important, the SAPI, we get the text, but speech is
sometimes speaks things differently
SB: For example 123 can be 1 2 3 or two hundred twenty three
JS: There are usability coincerns
SB: We are asking if it is recorded, not is it recorded the way we like
<westont> For anyone curious, here's how NVDA "system" tests are emulating key presses https://github.com/nvaccess/nvda/blob/master/tests/system/libraries/SystemTestSpy/speechSpyGlobalPlugin.py#L218
MK: Our job is definitely not to
get all synthesizers to do the same thing
... braille displays are separate products
SB: I was talking about the speech
<westont> A practicality consideration is that this NVDA plugin has a way of knowing when NVDA is "finished" processing the keyboard input. Without that internal information, it becomes quite hard to know when the SAPI plugin should stop listening, and assume nothing was announced
MF: I am talking about human testing, if the speech is different than the text, e.g. synthiszer doesn't recognized by the synth
SB: There is some string replacement
JS: There are synthesizer dependent behaviors
<westont> Here's NVDA's symbol substitution dictionary https://github.com/nvaccess/nvda/blob/master/source/locale/en/symbols.dic
MK: Add screen reader and it gets pretty complicated
SB: The answer is no
MK: I agree
SB: We have been writing some
test
... Feedback is welcome
JS: We have 6 ready, there are 3
pull requests waiting
... 7 or 8 plans being written
MK: I was spending time on select
only combobox today, it is easy to see in the preview to look
at
... It is in github.io open in either form you like
... The hard part is making line by line comments
... I was going to write comments in the comment page
SB: Yes please, easier for screen reader users
MK: All the assertions are in the same row, so it is messy
SB: Another point if you have a META issue, we can look at repeated issues
https://github.com/w3c/aria-at/pull/338
https://raw.githack.com/w3c/aria-at/tests/combobox-select-only/index.html
MK: This is some great progress
<zcorpan_> https://github.com/w3c/aria-at/issues/337
ST: A few orverlapping issues in
the test format
... Simon brought up the firt issue
... We need to be more precise in our terminology
... Assertions can have more than one part
... Doing a string comparison, breaking things down into the
parts, that has issues with testing duplication and order, but
better than full string matching
... Continue designing the automated test format, defining
instructions..
... See the issue for more details
MK: The action there was
specifically was on the instruction piece
... It is about how we write the instructions in the test
... There is still some complication when the ...
JS: If we are telling someone to
navigate to a grid, if they press T they only hear the size of
the table, they need to press down arrow
... When we need sequential commands how do we represent
this
SB: This can be abstracted for specific screen readers
JS: We don't want to miss a case, for example navigating into a table
MK: It is similar to combobox
issues
... This is an example of the kind o thing we need to
capture
JS: Even with the same screen reader there are inconsistencies
MK: We will see this in examples like the carousel
JS: In highlights when there are roles inside of roles
MK: There is a part of me that sequential navigation should be multiple tests?
SB: You will have a jagged matrix, that will be sparsely populated
MK: One of the things Seth, did we come to an agreement that the whole string is what we are testing, satisfies all the assertions, and there are no additional unexpected behaviors
ST: It we don;t look at sub strings now, we might not be able to do it in the future
SP: We probably specific assertions for role ann accName, and also compare the entire output
MK: So we want to do both, we
will always do the equal, ...
... If the equal passes, then everything should pass
NK: It it doesn't pass the sub tests give an indication of why it didn't pass
SB: I got my role, I got my state, if an exact string doesn't match, but it is still ok, we want to capture that
SP: The sub test could pass, but the whole string may not match
discussion of breaking examples...
SB: When doing this automatically we do not have to be human readible, the labels could be tokens, rather than something humans understand
MK: We still have some difficult
things for sequential testing, multiple tests
... Most tests will have one press that you get the output for
the tets
... Then other tests have a sequence and then there might be
speech along the way
SB: More information the second time, you need a way tfor people to skip over redundant content
MK: Something like start here and then nend here to do the assertions
SP: You can use the press commands, the output is what you have after the last press
MK: Let's say you are doing
sequential navigation, as soon as you navigate past that, then
you start collection
... The other thing you should be able to do, press down arrow
is on the combobox
... It expected to take 2 down arrows, but maybe it took 3
presses
JS: It may also addresses changes to screen reader releases
SB: I was also thinking about the menubar example, in those three pieces of information for the assertions
MK: These are the output of the test things I want to test
SB: You want to do grouping
JS: The table is completely different
MK: Press down arrow until they get tot he widget
JS: We have been doing that
SB: A boole can be used for automated testing
MK: As long as the screen reader says the role
SP: I am struggling to understand
JS: For example you navigate to the table, so JAWS will read info about table, then the down arrow to go to the first cell,
This is scribe.perl Revision of Date Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00) Present: Jemma Juliette_McShane Matt_King_ jesdaigle jongund michael_fairchild rob_fentress s3ththompson westont zcorpan_ boazsender Found Scribe: jongund Inferring ScribeNick: jongund WARNING: No date found! Assuming today. (Hint: Specify the W3C IRC log URL, and the date will be determined from that.) Or specify the date like this: <dbooth> Date: 12 Sep 2002 People with action items: WARNING: Input appears to use implicit continuation lines. You may need the "-implicitContinuations" option. WARNING: IRC log location not specified! (You can ignore this warning if you do not want the generated minutes to contain a link to the original IRC log.)[End of scribe.perl diagnostic output]