<scribe> scribe:shimizuyohta

Isaac: I was able to create a wiki page for use cases, and incorporated the feedback Matt provided. They are in the wiki, and we can continue discussion on issue.
... I'm matching the wording, and adjusting the flow.

Roles and process in the working mode document

<zcorpan> working mode https://docs.google.com/document/d/1CXNRSDj1rFsVPtCu4YFww7I0OUEGRIXY6i_s453uaak/edit?usp=sharing

Simon: Just shared a link to the google doc. Added new section for definitions.
... This processes are mostly about developing and getting feedback.
... Describing roles and processes based on the google doc.

Matt: In terms of role, do we know which of these roles end up roles in the system yet?

Isaac: For use cases, I included three roles,for test runner.

tester,community group member...

Simon: Proces 1:(Test developer) Select a GitHub issue specifying an ARIA design pattern implementation for which a test plan is needed from the “AT Test Plan Development” GitHub project.
... Proces 2:(Test developer) Read the relevant parts of the ARIA specification and the APG to understand the intent of the accessibility semantics and features
... Proces 3:(Test developer) If there are native OS components with similar accessibility semantics, study how AT behaves when using those components.
... Proces 4:(Test developer) Research any special AT configuration requirements.

Matt: These steps are directly related to the feedback from SR developers.
... Landmark regions are not announcing on JAWS.
... which I realized with Glen from Freedom Scientific in the last meeting.
... There's going to have to be some setting made by users in the AT configuration. Again this is not the default setting.
... We have to research this, document, and incorporated them into our instruction.
... We need to include setting change if the assertion require a setting to be changed.
... That is where we need consensus from across the community group.
... This might affects with your work, Jon and Valerie in terms of scripts.

Michael: If we're instructing testers to change setting. We also have to guide them to put it back to defoult for next text.

Valerie: I agree with Jon that it's different from what we have now. Essentially we need data mapping between test and SR. We should always have instruction to use defoult setting/or other setting.

Simon: There is a issue for instruction for defoult setting, and separate issue for non-defoult setting.

Valerie: It's sounds like a new feature, and I'm not sure how we can add that for now.

Matt: Valerie, to what extent do you think this chagne of defoult would impact our test?
... This is edge case, so we decided to keep discuss this on issue.

Michael: Another aspect is how do we display this information in testing result.

Matt: It would have to be something like foot-note

Michael: The way I handle this is I say "supported" with asterisk, to describe what the setting is. There is a note to describe the setting in my case.

Matt: We can flag them in the summary section of report.
... At some point, that affect us for test writing.
... One thing we want to arrive sooner is whether we allow something other than defoult.

Simon: Step5.(Test developer) Write a test plan that proposes assertions, assertion priorities, and expected user settings, that:

-Are aligned with the intent of the ARIA specification and APG documentation -Are aligned with what ATs already do or what AT developers are willing to implement

Matt: What did you think of my comment earlier today?
... Has the consensus of other test developers of community group.So we're peer reviewing before step 6.
... I was responding to a step 6-7, about WPT format.

Simon: I agree with your response

Matt: Do we just delete step 6, or rewrite?

Simon: I think we'll rewrite the test to edit WPT part.

*See the updated step 6 from google doc.

Matt: For now I want to address people from outside of group to review. Test writing > Review within the group > Review from outside of the group.

Simon: After internal review, we have step 7.(AT developer, optionally Web developer) Review the proposed assertions, assertion priorities, and expected user settings (Timebox: X)

Matt: There's some room to interpret this. When we say timebox, do we skip this phase if we're tight in schedule not having enough time for feedback?
... And do we need to explicitly address that?

Simon: That was my intention.
... Reastically, I think AT developers are going to review this test when everything is ready.

Matt: Make sense. So in the document we can say (within X days prior to next round of testing)

Simon: And the rest of steps are dependent on the X

Jon: I still have some questions for setup scripts for example to reliably set them up. So 10 sounds little too aspirational.

Matt: So we're talking about now to June.Let's put it 7-10 for now. Even if we did 1 every 2 weeks, we won't be reaching 10.

Discussing the number of test we are going to shoot until June.

Matt: There's alos conflict resolution required, and it may took more than 3 weeks. If we're not counting conflict, 3 weeks sounds reasonable.
... There's one thing related to versioning of AT.
... When we do report, report becoming test round, and it tied to a specific version.
... Does that answer one of the versioning question?

Michael: Yes, but also brings me another quesion. If JAWS adjusted the setting to enable a feature to one version to next.How do we deal with merge of test round?

Matt: I don't think those would affect to test plan.

Simon: This is good point, and we should record what the git revision is for entire repo, when runnign the test.

Matt: Yes, we shoudl archive them.

Jon: In APG, is there one reference to JAWS compatibility in APG?

Matt: Is always to be latest.It says versions is, but they are always latest.

Update on script for test writing

Jon: Menubar test is in master now. I have PR to combobox.
... Yohta's working on converting checkbox text on csv.
... I have some challenges in setup script.

Valerie: Maybe we can have a separate time to debug.

Matt: I'm thinkig all these tests are in the APG, and when we run them, it will be popup version of APG example.

Valerie: Eventually this test should be APG, as we discussed.
... We might want to have example code to make it easier to realize automation.

<Matt_King> rrsag3ent, make minutes]

This is scribe.perl Revision: 1.154  of Date: 2018/09/25 16:35:56  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Present: Matt_King michael_fairchild jongund zcorpan Erika_Miguel shimizuyohta spectranaut
Found Scribe: shimizuyohta
Inferring ScribeNick: shimizuyohta

WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: 

WARNING: Input appears to use implicit continuation lines.
You may need the "-implicitContinuations" option.


WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)