<ege> https://github.com/egekorkan/wot/blob/master/testing/Presentation.pdf
<kaz> https://github.com/w3c/wot/pull/444
<kaz> scribenick: mlagally_
<inserted> PR 138
McCool: <presents tool to extract
normative assertions>
... PR is available, not yet merged
Ege: where will the assertions be shown
McCool: "npm runassertion" produces a html result file: assertions.html
<inserted> Ege's slides on Thing Testing
Ege: topic could be named TD
based thing validation implementation
... test a thing by just using it's TD
... simplified V-model, it is a continuous process
... want to use a TD in all parts of the development
cycle
... we test a thing via network interfaces (input and
output)
... this is blackbox-testing
... main idea is to rely on JSON-schema embedded in TD
... JSON schema entries specify the
datastructure
... illustrates schema validation >
... we use existing tools for JSON validation
... we do JSON value generation (JSON-fakers)
... feel free to play with the online-tools (see preso)
... testbench will be ready for next plug fest
McCool: what does interop testing mean? A device can interact with another?
Ege: a device can send fake values
McCool: interop: you create a fake
thing, similar to the things in the Oracle device
simulator
... this is pretty close to what we already have in the
simulator
Ege: can the simulator define functions and change values dynamically ?
Lagally: yes, you can define dynamic behaviour via functions
scribenick: kaz
McCool: suggests Ege and
Michael Lagally work together
... would like to go back to trigger for events
scribenick: dsr
Koster: we may not need to be able to trigger events - we may just add a test event that we can trigger
McCool: are we testing framework or a thing?
Koster: you don't need to test hardware, etc, just the event generating mechanism
McCool: we use descriptions of
existing devices- cannot trigger events
... for framework we can do standardized tests
Koster: we can add a specific trigger mechanism
McCool: we can define a pattern to make things more testable
Ege: domain is to test the scripting API implementation
<inserted> F-Interop site
<inserted> Dave's slides
Dave: EU project, runs until
September
... so far they are testing at protocol level
... W3C brought in higher level (applciation level)
testing
... interop testing of WoT platforms
... e.g. node-WoT and another platform
... you have a suite of tests that can check WoT interop
... we create a test app for each platform
... they expose and consume things via platform API
... also have a backchannel using F-interop
... framework is very similar to what Ege was presenting
... we have declarative tests
... can be derived from specs
... we need machine-interpretable version of the test
cases
... test apps need also access to the tests
Koster: we plan to use F-interop for
Wishi as well
... this is part VPN, part script replay
McCool: what is the smallest piece to test a framework
Dave: a TD with a single property,
e.g. boolean
... simple TD, run a couple of tests both from the consumed and
exposed side of a thing
... for events you have to register
... we are testing on a much higher level than on protocols,
i.e. API level
McCool: any concrete example to
better understand?
... effort?
Dave: 2 months work between May
and September
... show sth. at TPAC
McCool: we can start immediately after plugfest, can we have a demonstrator of the strategy at the plugfest
Dave: will have a document by end
of June
... need to figure out how to collaborate
Koster: can we use that for general online plugfest?
Dave: there are docker-based setups
McCool: if we can have a node-WoT
setup in docker it is easy to reproduce
... next time dig deeper into interop testing
Kaz: interop testing is one of
the test efforts - first priority is spec validation and
implementation testing
... need to clarify relation between F-interop and
plugfest
... which part of framework could be used and vice versa?
Dave: this is discussion in testing TF
McCool: validation needed to
demonstrate interop between two different implementations
... interorganisational dependencies can be an issue
... need to discuss in next call
... licensing to be clarified
... we need official liaison with F-interop
Ege: last time it was quite
difficult to use
... you had to hack to make it work
McCool: a brief demo would be helpful
Dave: perhaps ask Frederico
Ege: can ask him
... targeting next week
scribenick: kaz
Lagally: wonder if anybody else than Ege is interested in Oracle simulator
Kaz: want to have a separate call?
McCool: we can do that during the testing call itself
[adjourned]