ARIA 1.1 Post-Mortem
(Redirected from Meetings/TPAC 2017/ARIA 1.1 Post-Mortem)
Rec-track status
- ARIA 1.1 is a Proposed Recommendation.
- Core AAM 1.1 is a Proposed Recommendation.
- DPub ARIA 1.0 is a Proposed Recommendation.
- DPub AAM 1.0 is a Proposed Recommendation.
- AccName AAM 1.1 is not yet a Candidate Recommendation.
Workflow
- Features added to ARIA spec. ARIA spec entered CR late October 2016.
- Testable statements for ARIA spec started, but couldn't be fully completed due to many missing mappings.
- Missing mappings sought from platform accessibility API owners for Core AAM.
- Testable statements for ARIA spec resumed, written by hand, in a wiki.
- Testable statements for ARIA spec corrected because many were written incorrectly.
- Testable statements for Core AAM generated via quick-and-dirty JavaScript.
- Core AAM spec entered CR late September 2017, work stopping primarily to meet publishing deadlines.
- Initial results from automated testing obtained for some platforms, revealing quite a few missing implementations.
- Bugs filed, but did not result in implementors doing the implementations. And in some cases we received pushback from them.
- All test results for ARIA and Core AAM obtained mid-October 2017. (But not all results were immediately usable.)
- Both specs entered PR early November 2017.
Consequences of the 1.1 workflow
- Disconnect between what we want and what implementors want discovered after entering CR.
- Critical bugs blocking implementation discovered after entering CR.
- Errors in AAM discovered after entering CR.
- Platform owners cannot make changes in mappings without risking exiting CR.
- Milestones slipped multiple times.
- Many Working Group members unable to contribute after Workflow step 1.
- Others?
Lessons learned
- ARIA features, mappings, implementation, and testing (at least) need to happen together.
- Implementors (user agent and assistive technology) need to become more involved.
- Using Core AAM as THE means to show implementability of ARIA is problematic.
- Treating Core AAM just like "any other (static) spec" is problematic.
- Using the wiki (and especially wiki table markup) for creating testable statements introduced more problems than it solved.
- Programmatic generation of testable statements from specs saves a lot of time and human resources!
- Automated testing tools save a lot of time and human resources! -- when they exist and work as expected.
- Others?
Conclusions (or at least aspirations)
- IF
- we adjust our workflow, and
- we finish and refine our tools for automating test writing and running, and
- we document and provide training and support to Working Group members, and
- we find a way to make the Core AAM a non-static, but still normative, spec
- THEN
- we'll catch where we lack consensus with implementors before it's "too late", and
- we'll all be able to contribute towards advancement along the Rec track, and
- we'll meet our milestones because we'll always be ready to ship, and
- we'll have specs which reliably reflect what implementors are expected to do