13:55:08 RRSAgent has joined #webmachinelearning 13:55:08 logging to https://www.w3.org/2022/04/07-webmachinelearning-irc 13:55:09 RRSAgent, make logs Public 13:55:11 please title this meeting ("meeting: ..."), anssik 13:55:14 Meeting: WebML WG Teleconference – 7 April 2022 13:55:19 Chair: Anssi 13:55:23 Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2022-04-07-wg-agenda.md 13:55:27 Scribe: Anssi 13:55:33 scribeNick: anssik 13:55:37 scribe+ dom 13:55:41 Present+ Anssi_Kostiainen 13:55:46 Present+ Dominique_Hazael-Massieux 13:55:55 Present+ James_Fletcher 13:56:05 RRSAgent, draft minutes 13:56:05 I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik 14:00:01 ningxin_hu has joined #webmachinelearning 14:02:26 bbcjames has joined #webmachinelearning 14:03:02 anssik has joined #webmachinelearning 14:03:06 anssik has joined #webmachinelearning 14:03:27 RRSAgent, draft minutes 14:03:27 I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik 14:03:57 rama has joined #webmachinelearning 14:05:08 RafaelCintron has joined #webmachinelearning 14:05:26 scribe+ anssik_ 14:05:55 Topic: Ethical Web Machine Learning 14:05:56 present+ 14:06:35 james: we've published a 2nd of a WG note on ethical considerations for ML https://webmachinelearning.github.io/ethical-webmachinelearning/ 14:06:49 RRSAgent, draft minutes 14:06:49 I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik_ 14:06:57 ... we ran 2 brainstorm sessions for feedback & input on risks & mitigations this week 14:07:21 ... both to generate content and set up a process for continuous concrete input on these topics 14:07:32 ... we have 8 people on Monday, 10-12 on Tuesday 14:07:45 ... they went really well and generated lots of useful input and positive feedback 14:08:10 ... the next step will be to integrate the feedback into the document 14:08:46 ... and bring content into the operationalization section that would include a white label version of that workshop as a tool to continue thinking on these topics 14:09:31 ... we plan to bring the doc for approval by WG for publication as a note - I'll be pausing my involvement for a few months, but hope the work can continue forward 14:09:41 q? 14:10:02 Topic: Proposed new use cases 14:10:42 Subtopic: WebNN API in gaming scenarios 14:10:43 Anssi: introducing Tracy, senior engineer from Unity, interested in bringing WebNN into their Web backend 14:11:35 Tracy: I was at Microsoft, most recently working on DirectML & + 14:11:49 ... moved to Unity in January to work on Barracuda, a lightweight runtime for inferences 14:11:59 ... running on phones, handheld consoles, games consoles, PCs 14:12:10 ... it also supports a WebGL-based Web player 14:12:28 RRSAgent, draft minutes 14:12:28 I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik 14:12:48 ... It is used for a variety of use cases internally: reinforcement learning on objects to bring more intelligent behaviors 14:12:54 Present+ Rafael_Cintron 14:12:55 ... lots of R&D with neural rendering 14:13:16 ... public API for unity customers to use for the ML use cases 14:13:23 Present+ Tracy_Sharpe 14:13:35 Present+ Ganesan_Ramalingam 14:13:37 ... Some of the use cases we also have is where Unity is both the editor and the runtime player 14:13:53 ... the content tools are done with web pages - it allows to distribute them more quickly to end users 14:14:10 ... some of the neural rendering techniques are using WebGL, which can't keep up with what would be expected from a desktop GPU 14:14:46 ... WebGPU would already help with improvements, but Chai mentioned WebNN could provide another target to achieve best performance while operating in Web pages 14:14:58 RRSAgent, draft minutes 14:14:58 I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik 14:15:03 ... we're exploring the topic, may be doing prototyping and benchmarking performance 14:15:05 q? 14:15:44 Ningxin: very exciting use case! 14:16:25 ... do you have any special requirement in terms of ML functionality to work with the rendering component to work together? 14:16:45 ... for example, real-time super-resolution in games 14:17:22 Present+ Jonathan_Bingham 14:17:23 Tracy: operating with a game engine is a constrained environment, even in non Web cases 14:17:32 Jonathan has joined #webmachinelearning 14:17:32 ... the frame budget is restricted 14:18:03 ... many of the models in the game cases (e.g. smart agent) end up running on CPU to avoid interfering with the GPU time budget 14:18:08 ... this needs careful management 14:18:21 ... in terms of the editor experience (where we're using the Web today) don't have the same constraints 14:18:32 q+ 14:19:10 ningxin: one of the target for WebNN aims to serve as backend for frameworks such as tf.js or onnx 14:19:19 ... it would be interesting to collaborate on experiments 14:20:34 dom: any requirements for storage or memory management in the editor context on desktop? is that a constraint you're hitting 14:20:49 dom: any particular constraint on memory & storage in your use case 14:20:51 q+ 14:20:57 q? 14:20:59 ack dom 14:21:00 q+ 14:21:02 ack ningxin_hu 14:21:03 tracy: not that I know of atm 14:21:37 ningxin: we are discussing having an async API vs a synchronous API only within a web worker 14:22:13 ... based on our prototypes with other frameworks transpiled from C++ into WASM, we see the requirement to call API synchronously 14:22:30 ... but the sync API can have issues, e.g. making the API janky if running in the main thread 14:23:00 ... in your framework, is there any requirement for sync / async calls? main thread vs worker? 14:23:19 Tracy: everything is asynchronous in our API 14:24:07 ... nothing should block the rendering of the game thread 14:24:43 ningxin: with the work done in the worker - is the code running the inference sync or async? 14:24:54 tracy: this would be flexible, depending on the backend 14:25:52 ningxin: ok, so no restriction on sync vs async 14:26:35 anssi: the WebNN API started with a bunch of use cases; are we getting new requirements from this scenario? or does the API satisfy its requirements? 14:27:19 tracy: I'll review the spec to give a more informed answer on potential gaps 14:27:38 q? 14:27:42 ack RafaelCintron 14:28:24 RafaelCintron: in your usage of ML in Unity - is the expectations that models will be pre-set, or will the graph be updated during runtime? 14:28:37 Tracy: more of the former; some of the weights might change, but mostly baked 14:28:48 q? 14:36:08 Topic: Context-based graph execution methods for different threading models 14:36:31 Rafael: I won't be on the call in two weeks - the crux of the question is how much dependency we want to have on WebGPU 14:36:51 ningxin_hu: we need a clear boundary between WebNN & WebGPU 14:36:57 -> Context-based graph execution methods for different threading models PR #257 https://github.com/webmachinelearning/webnn/pull/257 14:37:32 ... we should target the WebNN based on typical use; if WebNN is going to be expected to be used mostly in standalone mode, WebGPU integration should be hidden 14:37:50 ... MLContext.compute & MLContext.asyncCompute would be the primary usage 14:38:01 RRSAgent, draft minutes 14:38:01 I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik 14:38:02 ... for WebGPU integration, I would support the typical WebGPU usage 14:38:31 ... with a common buffer and queue that a WebGPU developer would be familiar with to make WebNN more friendly to them 14:38:48 ... MLCommandBuffer or MLCommandEncoder would be closer to that approach 14:39:08 ... we haven't had much investigation on the integration with the WebGL pipeline - more work may be needed there 14:39:24 ... we should target typical usages of the API 14:39:35 q? 14:39:36 ... this will help orient the developers toward the right path 14:41:17 RafaelCintron: this is a useful classification; remains to be seen how much this needs to be reflected into different API shapes 14:42:09 ... there is a discussion in the WebGPU WG on similar topics 14:42:32 ... on async to avoid jank 14:42:46 ... with similarly people porting from native wanting to have a sync version 14:43:23 ... e.g. a Snap funny hat implementatio 14:43:26 s/io/ion 14:43:52 ... the group is leaning towards sync only in worker; but struggling with very similar discussions indeed 14:44:19 q? 14:44:26 s/fredue/freude/ 14:44:44 Topic: Candidate Recommendation readiness 14:45:50 anssi: in terms of use cases - do we feel that we have captured the use cases for this API? any major gap? 14:46:06 ... e.g. anything emerging from the background blur discussion that should be reflected in the use cases? 14:46:11 WebGPU Sync vs Async Github Issue: mapSync on Workers - and possibly on the main thread (https://github.com/gpuweb/gpuweb/issues/2217 ) 14:46:30 -> Integration with real-time video processing https://www.w3.org/TR/webnn/#usecase-real-time-video-processing 14:47:14 ... please review use cases for potential gaps 14:47:30 q+ 14:48:46 ack ningxin_hu 14:48:52 dom: the best demonstration will be having frameworks that run with a WebNN backend, with associated demonstrated performance benefits 14:49:41 RRSAgent, draft minutes 14:49:41 I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik 14:49:59 ningxin_hu: the video processing prototype has a dependency to WebGPU integration - right now it relies on the WebGPU API to import video frame, pre-process it with a WebGPU shader into a WebGPU buffer on which the inference is running 14:50:47 ... but there is also an alternative where VideoFrame becomes a directly usable primitive for WebNN, as Chai suggested in the discussion 14:51:35 ... There may be value in supporting other models beyond the one we're using for segmentation 14:51:52 -> The first-wave models https://github.com/webmachinelearning/webnn/blob/main/op_compatibility/first_wave_models.md 14:52:00 ... we may want to revisit the model list to integrate such a support 14:52:44 -> WebNN should support int8 quantized models #128 https://github.com/webmachinelearning/webnn/issues/128 14:52:56 ... For Data Types, there are open issues about quantized and int8 models - we need to investigate this in more depth 14:53:16 ... Typical AI accelerators in GPU hardware optimize for quantized data types 14:53:30 ... Performance benefits are key to the work 14:53:48 q+ 14:53:58 ack dom 14:55:49 +1 14:56:08 dom: maybe useful to derive semi-formal requirements from the use cases, incl performance 14:57:09 anssi: I've assumed documenting them as issues was enough, but would not oppose to document them formally if people find this useful 14:57:18 ... maybe we can label issues as requirements 14:57:37 q? 14:58:26 -> Ethical Considerations https://www.w3.org/TR/webnn/#ethics 14:58:45 anssi: we'll have need ethical considerations 14:58:59 ... and document implementation experience 14:59:24 ... supported by Web Platform Tests, the webnn-baseline and a browser implementation (targeting Chromium atm) 15:00:19 dom: note that implementation experience is not a requirement to get into CR 15:00:36 ... (although double implementation experience is definitely need to get out of CR) 15:01:07 ... in terms of wide review, we had TAG review, first security review (now tagged for horizontal review) 15:01:24 ... we'll do another privacy review, and will have to lauch accessibility and internationalization 15:01:30 ... and then WebGPU integration review 15:01:33 q? 15:01:35 ... we're on good track 15:01:49 Present+ Wonsuk_Lee 15:03:27 RRSAgent, draft minutes 15:03:27 I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik 15:13:20 s|Anssi: introducing Tracy|slideset: https://lists.w3.org/Archives/Public/www-archive/2022Apr/att-0000/WebML_WG_Intro.pdf\\nAnssi: introducing Tracy| 15:13:23 RRSAgent, draft minutes 15:13:23 I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik 15:16:39 s|Slideset: https://lists.w3.org/Archives/Public/www-archive/2022Apr/att-0000/WebML_WG_Intro.pdf\\nAnssi: introducing Tracy|Anssi: introducing Tracy| 15:16:41 RRSAgent, draft minutes 15:16:41 I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik 15:17:27 s|slideset: https://lists.w3.org/Archives/Public/www-archive/2022Apr/att-0000/WebML_WG_Intro.pdf\\nAnssi: introducing Tracy|Anssi: introducing Tracy| 15:17:31 RRSAgent, draft minutes 15:17:31 I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik 15:30:04 i|Anssi: introducing Tracy|Slideset: https://lists.w3.org/Archives/Public/www-archive/2022Apr/att-0000/WebML_WG_Intro.pdf 15:36:43 RRSAgent, draft minutes 15:36:43 I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html dom 15:40:08 i|Tracy: I was at Microsoft,|[slide 1] 15:40:36 i|... moved to Unity in January|[slide 2] 15:40:54 i|... It is used for|[slide 3] 15:41:49 i|... the content tools|[slide 4] 15:41:51 RRSAgent, draft minutes 15:41:51 I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html dom 17:02:42 Zakim has left #webmachinelearning