Submitting, accepting and refuting tests
Dimitris Dimitriadis
dimitris.dimitriadis@improve.se
W3C DOM WG
Contact for DOM TS Issues
W3C DOM TS - General
- Publically developed (to involve both developers and
compaines
- www-dom-ts@w3.o
rg (public mailing list for the development of the DOM TS
- There will be a TS for each level of the DOM
- Jointly developed by W3C and NIST (level 1 being finlaized, future
levels will be based on experience made in this effort)
W3C DOM TS - Process
- Procdural issues
- We need a stable technical architecture (for version handling,
savind, documentation and so forth)
- More importantly, we need a stable mechanism for submitting,
resolving errors in the tests and interpretation of the
specifications
- Process document
W3C DOM TS
- A test, or a series of tests, get submitted to the DOM TS. Receipt
is acknowledged and a mail is sent to the DOM TS mailing list (a sanity
check before submitting is required).
- The moderator checks the test(s) against the relevant DOM
specifications.
- The test(s) receive the "received" status
W3C DOM TS - Review
- Reviewing tests
- Relevance of the submitted test.
- Part of the specification being testes (also indicated in
the XML test document)
- Quality of code
- Scenario of test(s) (Semantic requirements)
- If the test meets all criteria, it receives the "reviewed
and stable" status. It becomes part of the DOM TS.
- If the test does not meet all criteria (except in some cases the
scenario criterion), it receives the "inappropriate"
status.
W3C DOM TS - Reevaluation
- Reevaluating tests
- Received and stable tests that are found to be erroneous:
- The DOM WG representative checks the test(s) again. The final
authority is the DOM WG in case this is needed.
- Moving between states
- Tests that have been received the "received and stable"
status and are at a later stage found to be erroneous are lifted out of
hte TS and evaluated by the TS moderator.
- In difficult cases, where the test seems OK but won' work,
or in cases where the specification is being questioned, the WG studies
the test.
- If, after consideration, the test is considered to be OK, it
again forms part of the DOM TS.
- If, after consideration, the test is considered to fail, it
receives the "inappropriate" status.
- In any case, notification to public mailing list.
W3C DOM TS
- DOM TS Constituents
- Submitted, reviewed and stable tests.
- There wil be different versions of the DOM TS, since there are
different levels of the DOM
- Up-to-date list of existing and wanted tests
- www-dom-ts@w3.o
rg
- Deliverables
- DOM TS (process, architecture, scope, design)
- TS design guidelines
- Documentation of tests, expected results, scenarios, etc.
W3C DOM TS - QA
- DOM TS QA
- Will monitor guidelines developed in the QA activity (if
pursued)
- Process presupposes sanity check before submitting tests
- Generic XML format for tests includes information on part of
specification being tested (if atomic), expected reluts, scenario (if
scenario-driven test)
W3C DOM TS
- Open questions
- Testing the test suite (done by moderator/author at this
point)
- Ensured if granular enough test format is developed (vis.
expected results)
- Language-neutral definition of tests (to generate other than the
two current language bindings, using eg. XSLT to generate the specific
language binding)
- We're looking into automatic generation of tests based on
the specification
- Moving between states, efficiancy of current model.
W3C DOM TS