IRC log of webmachinelearning on 2022-04-07

Timestamps are in UTC.

13:55:08 [RRSAgent]
RRSAgent has joined #webmachinelearning
13:55:08 [RRSAgent]
logging to https://www.w3.org/2022/04/07-webmachinelearning-irc
13:55:09 [Zakim]
RRSAgent, make logs Public
13:55:11 [Zakim]
please title this meeting ("meeting: ..."), anssik
13:55:14 [anssik]
Meeting: WebML WG Teleconference – 7 April 2022
13:55:19 [anssik]
Chair: Anssi
13:55:23 [anssik]
Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2022-04-07-wg-agenda.md
13:55:27 [anssik]
Scribe: Anssi
13:55:33 [anssik]
scribeNick: anssik
13:55:37 [anssik]
scribe+ dom
13:55:41 [anssik]
Present+ Anssi_Kostiainen
13:55:46 [anssik]
Present+ Dominique_Hazael-Massieux
13:55:55 [anssik]
Present+ James_Fletcher
13:56:05 [anssik]
RRSAgent, draft minutes
13:56:05 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
14:00:01 [ningxin_hu]
ningxin_hu has joined #webmachinelearning
14:02:26 [bbcjames]
bbcjames has joined #webmachinelearning
14:03:02 [anssik]
anssik has joined #webmachinelearning
14:03:06 [anssik]
anssik has joined #webmachinelearning
14:03:27 [anssik]
RRSAgent, draft minutes
14:03:27 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
14:03:57 [rama]
rama has joined #webmachinelearning
14:05:08 [RafaelCintron]
RafaelCintron has joined #webmachinelearning
14:05:26 [anssik]
scribe+ anssik_
14:05:55 [anssik]
Topic: Ethical Web Machine Learning
14:05:56 [ningxin_hu]
present+
14:06:35 [dom]
james: we've published a 2nd of a WG note on ethical considerations for ML https://webmachinelearning.github.io/ethical-webmachinelearning/
14:06:49 [anssik_]
RRSAgent, draft minutes
14:06:49 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik_
14:06:57 [dom]
... we ran 2 brainstorm sessions for feedback & input on risks & mitigations this week
14:07:21 [dom]
... both to generate content and set up a process for continuous concrete input on these topics
14:07:32 [dom]
... we have 8 people on Monday, 10-12 on Tuesday
14:07:45 [dom]
... they went really well and generated lots of useful input and positive feedback
14:08:10 [dom]
... the next step will be to integrate the feedback into the document
14:08:46 [dom]
... and bring content into the operationalization section that would include a white label version of that workshop as a tool to continue thinking on these topics
14:09:31 [dom]
... we plan to bring the doc for approval by WG for publication as a note - I'll be pausing my involvement for a few months, but hope the work can continue forward
14:09:41 [anssik]
q?
14:10:02 [anssik]
Topic: Proposed new use cases
14:10:42 [anssik]
Subtopic: WebNN API in gaming scenarios
14:10:43 [dom]
Anssi: introducing Tracy, senior engineer from Unity, interested in bringing WebNN into their Web backend
14:11:35 [dom]
Tracy: I was at Microsoft, most recently working on DirectML & +
14:11:49 [dom]
... moved to Unity in January to work on Barracuda, a lightweight runtime for inferences
14:11:59 [dom]
... running on phones, handheld consoles, games consoles, PCs
14:12:10 [dom]
... it also supports a WebGL-based Web player
14:12:28 [anssik]
RRSAgent, draft minutes
14:12:28 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
14:12:48 [dom]
... It is used for a variety of use cases internally: reinforcement learning on objects to bring more intelligent behaviors
14:12:54 [anssik]
Present+ Rafael_Cintron
14:12:55 [dom]
... lots of R&D with neural rendering
14:13:16 [dom]
... public API for unity customers to use for the ML use cases
14:13:23 [anssik]
Present+ Tracy_Sharpe
14:13:35 [anssik]
Present+ Ganesan_Ramalingam
14:13:37 [dom]
... Some of the use cases we also have is where Unity is both the editor and the runtime player
14:13:53 [dom]
... the content tools are done with web pages - it allows to distribute them more quickly to end users
14:14:10 [dom]
... some of the neural rendering techniques are using WebGL, which can't keep up with what would be expected from a desktop GPU
14:14:46 [dom]
... WebGPU would already help with improvements, but Chai mentioned WebNN could provide another target to achieve best performance while operating in Web pages
14:14:58 [anssik]
RRSAgent, draft minutes
14:14:58 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
14:15:03 [dom]
... we're exploring the topic, may be doing prototyping and benchmarking performance
14:15:05 [anssik]
q?
14:15:44 [dom]
Ningxin: very exciting use case!
14:16:25 [dom]
... do you have any special requirement in terms of ML functionality to work with the rendering component to work together?
14:16:45 [dom]
... for example, real-time super-resolution in games
14:17:22 [anssik]
Present+ Jonathan_Bingham
14:17:23 [dom]
Tracy: operating with a game engine is a constrained environment, even in non Web cases
14:17:32 [Jonathan]
Jonathan has joined #webmachinelearning
14:17:32 [dom]
... the frame budget is restricted
14:18:03 [dom]
... many of the models in the game cases (e.g. smart agent) end up running on CPU to avoid interfering with the GPU time budget
14:18:08 [dom]
... this needs careful management
14:18:21 [dom]
... in terms of the editor experience (where we're using the Web today) don't have the same constraints
14:18:32 [dom]
q+
14:19:10 [dom]
ningxin: one of the target for WebNN aims to serve as backend for frameworks such as tf.js or onnx
14:19:19 [dom]
... it would be interesting to collaborate on experiments
14:20:34 [anssik]
dom: any requirements for storage or memory management in the editor context on desktop? is that a constraint you're hitting
14:20:49 [dom]
dom: any particular constraint on memory & storage in your use case
14:20:51 [ningxin_hu]
q+
14:20:57 [anssik]
q?
14:20:59 [anssik]
ack dom
14:21:00 [RafaelCintron]
q+
14:21:02 [anssik]
ack ningxin_hu
14:21:03 [dom]
tracy: not that I know of atm
14:21:37 [dom]
ningxin: we are discussing having an async API vs a synchronous API only within a web worker
14:22:13 [dom]
... based on our prototypes with other frameworks transpiled from C++ into WASM, we see the requirement to call API synchronously
14:22:30 [dom]
... but the sync API can have issues, e.g. making the API janky if running in the main thread
14:23:00 [dom]
... in your framework, is there any requirement for sync / async calls? main thread vs worker?
14:23:19 [dom]
Tracy: everything is asynchronous in our API
14:24:07 [dom]
... nothing should block the rendering of the game thread
14:24:43 [dom]
ningxin: with the work done in the worker - is the code running the inference sync or async?
14:24:54 [dom]
tracy: this would be flexible, depending on the backend
14:25:52 [dom]
ningxin: ok, so no restriction on sync vs async
14:26:35 [dom]
anssi: the WebNN API started with a bunch of use cases; are we getting new requirements from this scenario? or does the API satisfy its requirements?
14:27:19 [dom]
tracy: I'll review the spec to give a more informed answer on potential gaps
14:27:38 [anssik]
q?
14:27:42 [anssik]
ack RafaelCintron
14:28:24 [dom]
RafaelCintron: in your usage of ML in Unity - is the expectations that models will be pre-set, or will the graph be updated during runtime?
14:28:37 [dom]
Tracy: more of the former; some of the weights might change, but mostly baked
14:28:48 [anssik]
q?
14:36:08 [anssik]
Topic: Context-based graph execution methods for different threading models
14:36:31 [dom]
Rafael: I won't be on the call in two weeks - the crux of the question is how much dependency we want to have on WebGPU
14:36:51 [dom]
ningxin_hu: we need a clear boundary between WebNN & WebGPU
14:36:57 [anssik]
-> Context-based graph execution methods for different threading models PR #257 https://github.com/webmachinelearning/webnn/pull/257
14:37:32 [dom]
... we should target the WebNN based on typical use; if WebNN is going to be expected to be used mostly in standalone mode, WebGPU integration should be hidden
14:37:50 [dom]
... MLContext.compute & MLContext.asyncCompute would be the primary usage
14:38:01 [anssik]
RRSAgent, draft minutes
14:38:01 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
14:38:02 [dom]
... for WebGPU integration, I would support the typical WebGPU usage
14:38:31 [dom]
... with a common buffer and queue that a WebGPU developer would be familiar with to make WebNN more friendly to them
14:38:48 [dom]
... MLCommandBuffer or MLCommandEncoder would be closer to that approach
14:39:08 [dom]
... we haven't had much investigation on the integration with the WebGL pipeline - more work may be needed there
14:39:24 [dom]
... we should target typical usages of the API
14:39:35 [anssik]
q?
14:39:36 [dom]
... this will help orient the developers toward the right path
14:41:17 [dom]
RafaelCintron: this is a useful classification; remains to be seen how much this needs to be reflected into different API shapes
14:42:09 [dom]
... there is a discussion in the WebGPU WG on similar topics
14:42:32 [dom]
... on async to avoid jank
14:42:46 [dom]
... with similarly people porting from native wanting to have a sync version
14:43:23 [dom]
... e.g. a Snap funny hat implementatio
14:43:26 [dom]
s/io/ion
14:43:52 [dom]
... the group is leaning towards sync only in worker; but struggling with very similar discussions indeed
14:44:19 [anssik]
q?
14:44:26 [dom]
s/fredue/freude/
14:44:44 [anssik]
Topic: Candidate Recommendation readiness
14:45:50 [dom]
anssi: in terms of use cases - do we feel that we have captured the use cases for this API? any major gap?
14:46:06 [dom]
... e.g. anything emerging from the background blur discussion that should be reflected in the use cases?
14:46:11 [RafaelCintron]
WebGPU Sync vs Async Github Issue: mapSync on Workers - and possibly on the main thread (https://github.com/gpuweb/gpuweb/issues/2217 )
14:46:30 [anssik]
-> Integration with real-time video processing https://www.w3.org/TR/webnn/#usecase-real-time-video-processing
14:47:14 [dom]
... please review use cases for potential gaps
14:47:30 [ningxin_hu]
q+
14:48:46 [anssik]
ack ningxin_hu
14:48:52 [dom]
dom: the best demonstration will be having frameworks that run with a WebNN backend, with associated demonstrated performance benefits
14:49:41 [anssik]
RRSAgent, draft minutes
14:49:41 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
14:49:59 [dom]
ningxin_hu: the video processing prototype has a dependency to WebGPU integration - right now it relies on the WebGPU API to import video frame, pre-process it with a WebGPU shader into a WebGPU buffer on which the inference is running
14:50:47 [dom]
... but there is also an alternative where VideoFrame becomes a directly usable primitive for WebNN, as Chai suggested in the discussion
14:51:35 [dom]
... There may be value in supporting other models beyond the one we're using for segmentation
14:51:52 [anssik]
-> The first-wave models https://github.com/webmachinelearning/webnn/blob/main/op_compatibility/first_wave_models.md
14:52:00 [dom]
... we may want to revisit the model list to integrate such a support
14:52:44 [anssik]
-> WebNN should support int8 quantized models #128 https://github.com/webmachinelearning/webnn/issues/128
14:52:56 [dom]
... For Data Types, there are open issues about quantized and int8 models - we need to investigate this in more depth
14:53:16 [dom]
... Typical AI accelerators in GPU hardware optimize for quantized data types
14:53:30 [dom]
... Performance benefits are key to the work
14:53:48 [dom]
q+
14:53:58 [anssik]
ack dom
14:55:49 [ningxin_hu]
+1
14:56:08 [dom]
dom: maybe useful to derive semi-formal requirements from the use cases, incl performance
14:57:09 [dom]
anssi: I've assumed documenting them as issues was enough, but would not oppose to document them formally if people find this useful
14:57:18 [dom]
... maybe we can label issues as requirements
14:57:37 [anssik]
q?
14:58:26 [anssik]
-> Ethical Considerations https://www.w3.org/TR/webnn/#ethics
14:58:45 [dom]
anssi: we'll have need ethical considerations
14:58:59 [dom]
... and document implementation experience
14:59:24 [dom]
... supported by Web Platform Tests, the webnn-baseline and a browser implementation (targeting Chromium atm)
15:00:19 [dom]
dom: note that implementation experience is not a requirement to get into CR
15:00:36 [dom]
... (although double implementation experience is definitely need to get out of CR)
15:01:07 [dom]
... in terms of wide review, we had TAG review, first security review (now tagged for horizontal review)
15:01:24 [dom]
... we'll do another privacy review, and will have to lauch accessibility and internationalization
15:01:30 [dom]
... and then WebGPU integration review
15:01:33 [anssik]
q?
15:01:35 [dom]
... we're on good track
15:01:49 [anssik]
Present+ Wonsuk_Lee
15:03:27 [anssik]
RRSAgent, draft minutes
15:03:27 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
15:13:20 [anssik]
s|Anssi: introducing Tracy|slideset: https://lists.w3.org/Archives/Public/www-archive/2022Apr/att-0000/WebML_WG_Intro.pdf\\nAnssi: introducing Tracy|
15:13:23 [anssik]
RRSAgent, draft minutes
15:13:23 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
15:16:39 [anssik]
s|Slideset: https://lists.w3.org/Archives/Public/www-archive/2022Apr/att-0000/WebML_WG_Intro.pdf\\nAnssi: introducing Tracy|Anssi: introducing Tracy|
15:16:41 [anssik]
RRSAgent, draft minutes
15:16:41 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
15:17:27 [anssik]
s|slideset: https://lists.w3.org/Archives/Public/www-archive/2022Apr/att-0000/WebML_WG_Intro.pdf\\nAnssi: introducing Tracy|Anssi: introducing Tracy|
15:17:31 [anssik]
RRSAgent, draft minutes
15:17:31 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html anssik
15:30:04 [dom]
i|Anssi: introducing Tracy|Slideset: https://lists.w3.org/Archives/Public/www-archive/2022Apr/att-0000/WebML_WG_Intro.pdf
15:36:43 [dom]
RRSAgent, draft minutes
15:36:43 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html dom
15:40:08 [dom]
i|Tracy: I was at Microsoft,|[slide 1]
15:40:36 [dom]
i|... moved to Unity in January|[slide 2]
15:40:54 [dom]
i|... It is used for|[slide 3]
15:41:49 [dom]
i|... the content tools|[slide 4]
15:41:51 [dom]
RRSAgent, draft minutes
15:41:51 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/04/07-webmachinelearning-minutes.html dom
17:02:42 [Zakim]
Zakim has left #webmachinelearning