IRC log of webmachinelearning on 2022-04-07
Timestamps are in UTC.
- 13:55:08 [RRSAgent]
- RRSAgent has joined #webmachinelearning
- 13:55:08 [RRSAgent]
- logging to https://www.w3.org/2022/04/07-webmachinelearning-irc
- 13:55:09 [Zakim]
- RRSAgent, make logs Public
- 13:55:11 [Zakim]
- please title this meeting ("meeting: ..."), anssik
- 13:55:14 [anssik]
- Meeting: WebML WG Teleconference – 7 April 2022
- 13:55:19 [anssik]
- Chair: Anssi
- 13:55:23 [anssik]
- Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2022-04-07-wg-agenda.md
- 13:55:27 [anssik]
- Scribe: Anssi
- 13:55:33 [anssik]
- scribeNick: anssik
- 13:55:37 [anssik]
- scribe+ dom
- 13:55:41 [anssik]
- Present+ Anssi_Kostiainen
- 13:55:46 [anssik]
- Present+ Dominique_Hazael-Massieux
- 13:55:55 [anssik]
- Present+ James_Fletcher
- 13:56:05 [anssik]
- RRSAgent, draft minutes
- 13:56:05 [RRSAgent]
- I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
- 14:00:01 [ningxin_hu]
- ningxin_hu has joined #webmachinelearning
- 14:02:26 [bbcjames]
- bbcjames has joined #webmachinelearning
- 14:03:02 [anssik]
- anssik has joined #webmachinelearning
- 14:03:06 [anssik]
- anssik has joined #webmachinelearning
- 14:03:27 [anssik]
- RRSAgent, draft minutes
- 14:03:27 [RRSAgent]
- I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
- 14:03:57 [rama]
- rama has joined #webmachinelearning
- 14:05:08 [RafaelCintron]
- RafaelCintron has joined #webmachinelearning
- 14:05:26 [anssik]
- scribe+ anssik_
- 14:05:55 [anssik]
- Topic: Ethical Web Machine Learning
- 14:05:56 [ningxin_hu]
- present+
- 14:06:35 [dom]
- james: we've published a 2nd of a WG note on ethical considerations for ML https://webmachinelearning.github.io/ethical-webmachinelearning/
- 14:06:49 [anssik_]
- RRSAgent, draft minutes
- 14:06:49 [RRSAgent]
- I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik_
- 14:06:57 [dom]
- ... we ran 2 brainstorm sessions for feedback & input on risks & mitigations this week
- 14:07:21 [dom]
- ... both to generate content and set up a process for continuous concrete input on these topics
- 14:07:32 [dom]
- ... we have 8 people on Monday, 10-12 on Tuesday
- 14:07:45 [dom]
- ... they went really well and generated lots of useful input and positive feedback
- 14:08:10 [dom]
- ... the next step will be to integrate the feedback into the document
- 14:08:46 [dom]
- ... and bring content into the operationalization section that would include a white label version of that workshop as a tool to continue thinking on these topics
- 14:09:31 [dom]
- ... we plan to bring the doc for approval by WG for publication as a note - I'll be pausing my involvement for a few months, but hope the work can continue forward
- 14:09:41 [anssik]
- q?
- 14:10:02 [anssik]
- Topic: Proposed new use cases
- 14:10:42 [anssik]
- Subtopic: WebNN API in gaming scenarios
- 14:10:43 [dom]
- Anssi: introducing Tracy, senior engineer from Unity, interested in bringing WebNN into their Web backend
- 14:11:35 [dom]
- Tracy: I was at Microsoft, most recently working on DirectML & +
- 14:11:49 [dom]
- ... moved to Unity in January to work on Barracuda, a lightweight runtime for inferences
- 14:11:59 [dom]
- ... running on phones, handheld consoles, games consoles, PCs
- 14:12:10 [dom]
- ... it also supports a WebGL-based Web player
- 14:12:28 [anssik]
- RRSAgent, draft minutes
- 14:12:28 [RRSAgent]
- I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
- 14:12:48 [dom]
- ... It is used for a variety of use cases internally: reinforcement learning on objects to bring more intelligent behaviors
- 14:12:54 [anssik]
- Present+ Rafael_Cintron
- 14:12:55 [dom]
- ... lots of R&D with neural rendering
- 14:13:16 [dom]
- ... public API for unity customers to use for the ML use cases
- 14:13:23 [anssik]
- Present+ Tracy_Sharpe
- 14:13:35 [anssik]
- Present+ Ganesan_Ramalingam
- 14:13:37 [dom]
- ... Some of the use cases we also have is where Unity is both the editor and the runtime player
- 14:13:53 [dom]
- ... the content tools are done with web pages - it allows to distribute them more quickly to end users
- 14:14:10 [dom]
- ... some of the neural rendering techniques are using WebGL, which can't keep up with what would be expected from a desktop GPU
- 14:14:46 [dom]
- ... WebGPU would already help with improvements, but Chai mentioned WebNN could provide another target to achieve best performance while operating in Web pages
- 14:14:58 [anssik]
- RRSAgent, draft minutes
- 14:14:58 [RRSAgent]
- I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
- 14:15:03 [dom]
- ... we're exploring the topic, may be doing prototyping and benchmarking performance
- 14:15:05 [anssik]
- q?
- 14:15:44 [dom]
- Ningxin: very exciting use case!
- 14:16:25 [dom]
- ... do you have any special requirement in terms of ML functionality to work with the rendering component to work together?
- 14:16:45 [dom]
- ... for example, real-time super-resolution in games
- 14:17:22 [anssik]
- Present+ Jonathan_Bingham
- 14:17:23 [dom]
- Tracy: operating with a game engine is a constrained environment, even in non Web cases
- 14:17:32 [Jonathan]
- Jonathan has joined #webmachinelearning
- 14:17:32 [dom]
- ... the frame budget is restricted
- 14:18:03 [dom]
- ... many of the models in the game cases (e.g. smart agent) end up running on CPU to avoid interfering with the GPU time budget
- 14:18:08 [dom]
- ... this needs careful management
- 14:18:21 [dom]
- ... in terms of the editor experience (where we're using the Web today) don't have the same constraints
- 14:18:32 [dom]
- q+
- 14:19:10 [dom]
- ningxin: one of the target for WebNN aims to serve as backend for frameworks such as tf.js or onnx
- 14:19:19 [dom]
- ... it would be interesting to collaborate on experiments
- 14:20:34 [anssik]
- dom: any requirements for storage or memory management in the editor context on desktop? is that a constraint you're hitting
- 14:20:49 [dom]
- dom: any particular constraint on memory & storage in your use case
- 14:20:51 [ningxin_hu]
- q+
- 14:20:57 [anssik]
- q?
- 14:20:59 [anssik]
- ack dom
- 14:21:00 [RafaelCintron]
- q+
- 14:21:02 [anssik]
- ack ningxin_hu
- 14:21:03 [dom]
- tracy: not that I know of atm
- 14:21:37 [dom]
- ningxin: we are discussing having an async API vs a synchronous API only within a web worker
- 14:22:13 [dom]
- ... based on our prototypes with other frameworks transpiled from C++ into WASM, we see the requirement to call API synchronously
- 14:22:30 [dom]
- ... but the sync API can have issues, e.g. making the API janky if running in the main thread
- 14:23:00 [dom]
- ... in your framework, is there any requirement for sync / async calls? main thread vs worker?
- 14:23:19 [dom]
- Tracy: everything is asynchronous in our API
- 14:24:07 [dom]
- ... nothing should block the rendering of the game thread
- 14:24:43 [dom]
- ningxin: with the work done in the worker - is the code running the inference sync or async?
- 14:24:54 [dom]
- tracy: this would be flexible, depending on the backend
- 14:25:52 [dom]
- ningxin: ok, so no restriction on sync vs async
- 14:26:35 [dom]
- anssi: the WebNN API started with a bunch of use cases; are we getting new requirements from this scenario? or does the API satisfy its requirements?
- 14:27:19 [dom]
- tracy: I'll review the spec to give a more informed answer on potential gaps
- 14:27:38 [anssik]
- q?
- 14:27:42 [anssik]
- ack RafaelCintron
- 14:28:24 [dom]
- RafaelCintron: in your usage of ML in Unity - is the expectations that models will be pre-set, or will the graph be updated during runtime?
- 14:28:37 [dom]
- Tracy: more of the former; some of the weights might change, but mostly baked
- 14:28:48 [anssik]
- q?
- 14:36:08 [anssik]
- Topic: Context-based graph execution methods for different threading models
- 14:36:31 [dom]
- Rafael: I won't be on the call in two weeks - the crux of the question is how much dependency we want to have on WebGPU
- 14:36:51 [dom]
- ningxin_hu: we need a clear boundary between WebNN & WebGPU
- 14:36:57 [anssik]
- -> Context-based graph execution methods for different threading models PR #257 https://github.com/webmachinelearning/webnn/pull/257
- 14:37:32 [dom]
- ... we should target the WebNN based on typical use; if WebNN is going to be expected to be used mostly in standalone mode, WebGPU integration should be hidden
- 14:37:50 [dom]
- ... MLContext.compute & MLContext.asyncCompute would be the primary usage
- 14:38:01 [anssik]
- RRSAgent, draft minutes
- 14:38:01 [RRSAgent]
- I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
- 14:38:02 [dom]
- ... for WebGPU integration, I would support the typical WebGPU usage
- 14:38:31 [dom]
- ... with a common buffer and queue that a WebGPU developer would be familiar with to make WebNN more friendly to them
- 14:38:48 [dom]
- ... MLCommandBuffer or MLCommandEncoder would be closer to that approach
- 14:39:08 [dom]
- ... we haven't had much investigation on the integration with the WebGL pipeline - more work may be needed there
- 14:39:24 [dom]
- ... we should target typical usages of the API
- 14:39:35 [anssik]
- q?
- 14:39:36 [dom]
- ... this will help orient the developers toward the right path
- 14:41:17 [dom]
- RafaelCintron: this is a useful classification; remains to be seen how much this needs to be reflected into different API shapes
- 14:42:09 [dom]
- ... there is a discussion in the WebGPU WG on similar topics
- 14:42:32 [dom]
- ... on async to avoid jank
- 14:42:46 [dom]
- ... with similarly people porting from native wanting to have a sync version
- 14:43:23 [dom]
- ... e.g. a Snap funny hat implementatio
- 14:43:26 [dom]
- s/io/ion
- 14:43:52 [dom]
- ... the group is leaning towards sync only in worker; but struggling with very similar discussions indeed
- 14:44:19 [anssik]
- q?
- 14:44:26 [dom]
- s/fredue/freude/
- 14:44:44 [anssik]
- Topic: Candidate Recommendation readiness
- 14:45:50 [dom]
- anssi: in terms of use cases - do we feel that we have captured the use cases for this API? any major gap?
- 14:46:06 [dom]
- ... e.g. anything emerging from the background blur discussion that should be reflected in the use cases?
- 14:46:11 [RafaelCintron]
- WebGPU Sync vs Async Github Issue: mapSync on Workers - and possibly on the main thread (https://github.com/gpuweb/gpuweb/issues/2217 )
- 14:46:30 [anssik]
- -> Integration with real-time video processing https://www.w3.org/TR/webnn/#usecase-real-time-video-processing
- 14:47:14 [dom]
- ... please review use cases for potential gaps
- 14:47:30 [ningxin_hu]
- q+
- 14:48:46 [anssik]
- ack ningxin_hu
- 14:48:52 [dom]
- dom: the best demonstration will be having frameworks that run with a WebNN backend, with associated demonstrated performance benefits
- 14:49:41 [anssik]
- RRSAgent, draft minutes
- 14:49:41 [RRSAgent]
- I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
- 14:49:59 [dom]
- ningxin_hu: the video processing prototype has a dependency to WebGPU integration - right now it relies on the WebGPU API to import video frame, pre-process it with a WebGPU shader into a WebGPU buffer on which the inference is running
- 14:50:47 [dom]
- ... but there is also an alternative where VideoFrame becomes a directly usable primitive for WebNN, as Chai suggested in the discussion
- 14:51:35 [dom]
- ... There may be value in supporting other models beyond the one we're using for segmentation
- 14:51:52 [anssik]
- -> The first-wave models https://github.com/webmachinelearning/webnn/blob/main/op_compatibility/first_wave_models.md
- 14:52:00 [dom]
- ... we may want to revisit the model list to integrate such a support
- 14:52:44 [anssik]
- -> WebNN should support int8 quantized models #128 https://github.com/webmachinelearning/webnn/issues/128
- 14:52:56 [dom]
- ... For Data Types, there are open issues about quantized and int8 models - we need to investigate this in more depth
- 14:53:16 [dom]
- ... Typical AI accelerators in GPU hardware optimize for quantized data types
- 14:53:30 [dom]
- ... Performance benefits are key to the work
- 14:53:48 [dom]
- q+
- 14:53:58 [anssik]
- ack dom
- 14:55:49 [ningxin_hu]
- +1
- 14:56:08 [dom]
- dom: maybe useful to derive semi-formal requirements from the use cases, incl performance
- 14:57:09 [dom]
- anssi: I've assumed documenting them as issues was enough, but would not oppose to document them formally if people find this useful
- 14:57:18 [dom]
- ... maybe we can label issues as requirements
- 14:57:37 [anssik]
- q?
- 14:58:26 [anssik]
- -> Ethical Considerations https://www.w3.org/TR/webnn/#ethics
- 14:58:45 [dom]
- anssi: we'll have need ethical considerations
- 14:58:59 [dom]
- ... and document implementation experience
- 14:59:24 [dom]
- ... supported by Web Platform Tests, the webnn-baseline and a browser implementation (targeting Chromium atm)
- 15:00:19 [dom]
- dom: note that implementation experience is not a requirement to get into CR
- 15:00:36 [dom]
- ... (although double implementation experience is definitely need to get out of CR)
- 15:01:07 [dom]
- ... in terms of wide review, we had TAG review, first security review (now tagged for horizontal review)
- 15:01:24 [dom]
- ... we'll do another privacy review, and will have to lauch accessibility and internationalization
- 15:01:30 [dom]
- ... and then WebGPU integration review
- 15:01:33 [anssik]
- q?
- 15:01:35 [dom]
- ... we're on good track
- 15:01:49 [anssik]
- Present+ Wonsuk_Lee
- 15:03:27 [anssik]
- RRSAgent, draft minutes
- 15:03:27 [RRSAgent]
- I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
- 15:13:20 [anssik]
- s|Anssi: introducing Tracy|slideset: https://lists.w3.org/Archives/Public/www-archive/2022Apr/att-0000/WebML_WG_Intro.pdf\\nAnssi: introducing Tracy|
- 15:13:23 [anssik]
- RRSAgent, draft minutes
- 15:13:23 [RRSAgent]
- I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
- 15:16:39 [anssik]
- s|Slideset: https://lists.w3.org/Archives/Public/www-archive/2022Apr/att-0000/WebML_WG_Intro.pdf\\nAnssi: introducing Tracy|Anssi: introducing Tracy|
- 15:16:41 [anssik]
- RRSAgent, draft minutes
- 15:16:41 [RRSAgent]
- I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
- 15:17:27 [anssik]
- s|slideset: https://lists.w3.org/Archives/Public/www-archive/2022Apr/att-0000/WebML_WG_Intro.pdf\\nAnssi: introducing Tracy|Anssi: introducing Tracy|
- 15:17:31 [anssik]
- RRSAgent, draft minutes
- 15:17:31 [RRSAgent]
- I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
- 15:30:04 [dom]
- i|Anssi: introducing Tracy|Slideset: https://lists.w3.org/Archives/Public/www-archive/2022Apr/att-0000/WebML_WG_Intro.pdf
- 15:36:43 [dom]
- RRSAgent, draft minutes
- 15:36:43 [RRSAgent]
- I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html dom
- 15:40:08 [dom]
- i|Tracy: I was at Microsoft,|[slide 1]
- 15:40:36 [dom]
- i|... moved to Unity in January|[slide 2]
- 15:40:54 [dom]
- i|... It is used for|[slide 3]
- 15:41:49 [dom]
- i|... the content tools|[slide 4]
- 15:41:51 [dom]
- RRSAgent, draft minutes
- 15:41:51 [RRSAgent]
- I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html dom
- 17:02:42 [Zakim]
- Zakim has left #webmachinelearning