13:56:46 RRSAgent has joined #webmachinelearning 13:56:50 logging to https://www.w3.org/2024/06/27-webmachinelearning-irc 13:56:50 RRSAgent, make logs Public 13:56:51 please title this meeting ("meeting: ..."), anssik 13:56:53 Chair: Anssi 13:57:06 Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2024-06-27-wg-agenda.md 13:57:13 Scribe: Anssi 13:57:16 scribeNick: anssik 13:57:27 gb, this is webmachinelearning/webnn 13:57:27 anssik, OK. 13:57:35 Present+ Anssi_Kostiainen 13:57:41 Present+ Michael_McCool 13:57:46 RRSAgent, draft minutes 13:57:47 I have made the request to generate https://www.w3.org/2024/06/27-webmachinelearning-minutes.html anssik 13:59:44 Present+ Joshua_Lochner 13:59:51 Present+ Dwayne_Robinson 13:59:52 jsbell has joined #webmachinelearning 14:00:07 Regrets+ Rafael_Cintron 14:00:46 ningxin has joined #webmachinelearning 14:00:55 Present+ Dominique_Hazael-Massieux 14:00:57 Present+ Deepti_Gandluri 14:01:16 Present+ Zoltan_Kis 14:01:28 Present+ Joshua_Bell 14:01:48 Present+ Bryan_Bernhart 14:01:50 Regrets+ Ningxin_Hu 14:02:00 RRSAgent, draft minutes 14:02:01 I have made the request to generate https://www.w3.org/2024/06/27-webmachinelearning-minutes.html anssik 14:02:08 dwayner has joined #webmachinelearning 14:02:24 Meeting: WebML WG Teleconference – 27 June 2024 14:02:25 zolkid has joined #webmachinelearning 14:02:26 RRSAgent, draft minutes 14:02:27 I have made the request to generate https://www.w3.org/2024/06/27-webmachinelearning-minutes.html anssik 14:02:36 zolkid has left #webmachinelearning 14:02:43 asully has joined #webmachinelearning 14:02:52 Present+ Austin_Sullivan 14:03:01 RobKochman has joined #webmachinelearning 14:03:04 zolkis has joined #webmachinelearning 14:03:12 anssik: please welcome our most recent new participants Natasha Gaitonde and Andrew Nolan to the WebML WG, both representing Microsoft 14:03:22 Present+ Rob_Kochman 14:03:35 Topic: Announcements 14:03:44 Subtopic: WebML WG F2F at TPAC 2024 14:03:50 -> W3C TPAC 2024 main site https://www.w3.org/2024/09/TPAC/ 14:04:10 -> WebML WG F2F at TPAC 2024 issue for questions, comments, early agenda ideation https://github.com/webmachinelearning/meetings/issues/23 14:04:10 https://github.com/webmachinelearning/meetings/issues/23 -> Issue 23 WebML WG Hybrid Meeting at TPAC 2024 (by reillyeon) 14:04:35 anssik: W3C TPAC 2024 takes place in Anaheim, CA, USA at the Hilton Anaheim on 23–27 September 2024 14:05:32 ... The WebML WG will meet F2F on Monday, 23 September 2024, 09:00–18:00 PDT. We can split this time flexibly to cater for possible remote participants, please let me know any special requirements timing-wise 14:05:38 ... e.g. preference for mornings vs evenings for particular topics 14:05:54 ... WebML WG meets on the first day of the TPAC week 14:06:05 I'll be present in person. Conflicts with Web Apps on the 23rd, so may end up bouncing back and forth as specific agenda firms up. 14:06:21 ... I'd like to hear from you how you'd like to use our F2F time together and start ideating on the broad strokes our F2F agenda 14:06:32 ... good TPAC topics are higher-level topics that benefit from sync discussion 14:06:46 ... the expectation is not to do a line-by-line PR previews at the meeting, those are better done in an async manner 14:07:22 ... TPAC is of course an opportunity to network and resolve bigger technical and also social issues, including across the groups 14:07:24 Joshua_Lochner has joined #webmachinelearning 14:07:34 ... I've seen many long-running debates being settled at F2F meetings 14:08:04 ... the W3C-level expectation is we'll announce the detailed F2F agenda latest two weeks in advance, but we can definitely start drafting already now, it helps people who may want to join but don't follow our group's work closely 14:08:39 dom: agenda sent in advance helps if people are participating remotely, they can ask to adjust timing 14:09:27 anssik: if this is your first TPAC, please check the schedule for other meetings of interest that take place during the TPAC week and feel free to request to join them as an observer 14:09:52 -> TPAC 2024 calendar https://www.w3.org/calendar/tpac2024/ 14:10:36 dom: on Wednesday there's a full day of breakout sessions 14:10:44 -> TPAC 2024 - Breakout sessions https://github.com/w3c/tpac2024-breakouts 14:11:09 +1 14:11:14 dom: this when new ideas and proposals are discussed, if you've never been at TPAC, I strongly encourage you to be there on Wednesday at least 14:11:14 q? 14:11:46 zolkis has joined #webmachinelearning 14:11:47 anssik: usually, the event hotel sells out fast so please check the hotel booking instructions, I've heard there are other events at the same time in the city 14:11:52 -> Hotel booking w/ special rate https://www.w3.org/2024/09/TPAC/venue.html 14:11:59 anssik: instruction on how to register with a special rate are available at the TPAC site: 14:12:03 -> Registration for TPAC 2024 https://www.w3.org/register/tpac2024 14:12:20 anssik: the most urgent things to do are hotel, possible flight bookings, and registration 14:12:32 anssik: hotel cancellation policy is 72 hours prior to arrival, gives you flexibility should your plans change 14:12:43 anssik: I propose we use the existing TPAC meeting issue to solicit proposals for meeting agenda: 14:12:48 -> WebML WG F2F at TPAC 2024 issue https://github.com/webmachinelearning/meetings/issues/23 14:12:48 https://github.com/webmachinelearning/meetings/issues/23 -> Issue 23 WebML WG Hybrid Meeting at TPAC 2024 (by reillyeon) 14:13:09 anssik: in August I plan to produce a more baked in agenda to be shared with the entire W3C community 14:13:42 ... any questions, comments at this stage, higher-level topics you'd like to see on our F2F agenda or things you'd be interested to drive? 14:14:08 ... we can also provide slots for a few presentations on relevant topics to prime discussions, but traditionally TPAC F2F meetings have been interactive discussions on interesting topics rather than one-way presentations 14:14:11 q? 14:15:07 Subtopic: Summer break 14:15:41 anssik: we'll take a break in July and will resume our WG telcons 8 August 2024 14:15:41 ... as usual GH remains open 24/7 and welcomes your contributions while the meetings are on a pause 14:15:44 ... non-controversial changes that receive appropriate review are OK to merge while our meetings are on a pause 14:15:48 ... we'll do a debrief of PRs merged in our Aug telcon to keep everyone on top 14:16:08 Topic: Transformers.js update 14:16:43 Slideset: @@@ 14:17:18 [slide 1] 14:17:35 [slide 2] 14:17:39 [slide 3] 14:18:16 [slide 4] 14:18:40 RafaelCintron has joined #webmachinelearning 14:18:57 [slide 5] 14:19:01 [slide 6] 14:19:32 [slide 7] 14:20:13 Joshua: was about to achieve 16x speedup on my Windows 10 with RTX 14:20:15 [slide 8] 14:20:28 Joshua: with fp16, got 100x speedup (when available) 14:20:30 [slide 9] 14:20:37 [slide 10] 14:20:43 [slide 11] 14:22:00 -> https://huggingface.co/spaces/Xenova/whisper-webgpu Whisper WebGPU demo 14:22:37 [slide 12] 14:22:50 [slide 13] 14:23:50 [slide 14] 14:24:56 [slide 15] 14:25:55 Joshua: with Depth Anything v2, we get real time depth estimation in the browser 14:25:58 [slide 16] 14:27:02 Joshua: WebGPU brought 50x speed up on background removal 14:27:05 [slide 17] 14:27:36 q+ 14:27:44 [slide 18] 14:27:54 anssik: Thanks Joshua for this exciting update! Questions? 14:27:59 q+ 14:28:03 q? 14:28:07 ack McCool 14:28:25 McCool: are you aware of MLBuffer proposal? Plan to WebNN backend? 14:28:37 s/Plan to/Plan for 14:29:20 Joshua_Lochner: have been thinking of alternative WebGPU ML inference library backends, currently relying a lot on ONNX Runtime Web implementation, not exactly on top of MLBuffer developments 14:29:45 q- 14:29:59 ... WebNN support to be handled by ONNX RT 14:30:20 ... working with ONNX RT contributions on the WebNN backend for ONNX RT 14:30:21 q? 14:30:23 ack jsbell 14:30:37 jsbell: what is needed from WebNN and it sounds like ONNX RT Web support? 14:30:39 q+ 14:30:48 Joshua_Lochner: correct 14:31:15 q+ 14:31:25 q? 14:31:28 ack ningxin 14:31:53 ningxin: dynamic shapes, some models require them, but WebNN is static shape, is this a gap we should address? 14:32:22 Joshua_Lochner: one useful benefit for dynamic shapes in for vision models, for user to select the image size they want to process, precompile ONNX model with static shapes 14:32:40 ... decoder models that pass key value, dynamic shape important if you want to use dynamic cache 14:33:31 ... maybe there are way to export models so users export a model, or define it to take small images, in my case for depth anything, could select what range to use, there are other ways to define the model with static shapes that work for your use case 14:33:31 q? 14:33:34 ack dom 14:33:53 dom: you described perf improvements, compared to Wasm? 14:33:57 Joshua_Lochner: correct 14:34:11 dom: any guesses how much perf boost you'd get from WebNN? 14:34:55 Joshua_Lochner: you'd get low-level access for perf would be a lot better, but haven't been able to experiment it with yet, depends on ONNX RT implementation optimization, will collaborate ONNX RT team on benchmarks 14:35:36 dom: as part of TPAC another effort is to collect recorded videos of demos of technologies group's are working, but if we are able to have some early benchmark showing performance improvements applied to transformers it's be great 14:35:58 Joshua_Lochner: we can talk with ONNX RT team about this opportunity 14:36:05 q? 14:36:36 Topic: Device selection 14:37:05 anssik: on our last call we agreed to evolve MLContextOptions and other API controls for device selection informed by further implementation experience and new use cases from the wider web community 14:37:12 ... the spec has a Device selection section and defines MLContextOptions API surface for hints: 14:37:16 -> Device selection https://www.w3.org/TR/webnn/#programming-model-device-selection 14:37:22 -> MLContextOptions https://www.w3.org/TR/webnn/#dictdef-mlcontextoptions 14:37:45 anssik: per our earlier decision, we concluded MLContextOptions is under active development and added an explicit inline issue to the spec to solicit further feedback 14:37:49 -> "MLContextOptions is under active development" spec issue https://www.w3.org/TR/webnn/#issue-a5abcae7 14:38:07 anssik: I'd like to now kick off a new workstream to further explore future-proof device selection mechanisms and assess early design proposals 14:38:13 ... (there's a PR #712 to add "device selection" GH label to help us keep track of proposals) 14:38:14 https://github.com/webmachinelearning/webnn/pull/712 -> Pull Request 712 Process: Add "device selection" label (by anssiko) 14:38:32 anssik: as discussed, a number of early proposals and ideas are already documented in issue #623 14:38:33 https://github.com/webmachinelearning/webnn/issues/623 -> Issue 623 WebNN should support NPU and QDQ operations (by wchao1115) [v2] [opset] [feature request] 14:38:47 ... additional proposals are welcome in that issue, or in new issues if they're self-contained 14:39:04 ... for example, MikeW provided feedback in PR #712 that should be its own "device selection" related issue 14:39:15 ... MikeW please open a separate issue for your feedback so the group can discuss it carefully, this comment: 14:39:18 -> https://github.com/webmachinelearning/webnn/pull/696#pullrequestreview-2123677923 14:39:19 https://github.com/webmachinelearning/webnn/pull/696 -> MERGED Pull Request 696 Add MLDeviceType npu (by fdwr) 14:39:46 anssik: the proposals shared with the group for consideration in issue #623 include: 14:39:50 ... - a fallback device 14:40:00 ... - multiple devices in a preferred order 14:40:10 ... - an exclusion of a specific device 14:40:17 ... also discussed in this context were: 14:40:21 ... - error handling 14:40:32 ... - ultimate fallback 14:40:36 ... - quantized operators 14:41:04 ... I'd like to ask interested group participants to evaluate these proposals and explore them further so we can discuss them in the group to drive toward consensus 14:41:28 ... I'd also want to encourage participants to identify and share additional use cases for device selection 14:41:35 ... Ningxin shared the following device selection use cases earlier: 14:41:53 ... - compute offloading: a game engine may want to run ML tasks on CPU to avoid interfering with the GPU time budget. See WebNN API in gaming scenarios discussion of WG 2022/04/07. 14:41:58 -> WebML WG 2022-04-07 telcon: WebNN API in gaming scenarios https://www.w3.org/2022/04/07-webmachinelearning-minutes.html#t03 14:42:43 ... - op fallback: a ML framework may want to create a CPU context with fallback to Wasm option, that would avoid expensive cross-device tensor data copy between WebNN graph inference and Wasm operators execution. Custom operations #6 and Support CPU - WebAssembly scenario of the op level execution use case #156 14:42:44 https://github.com/webmachinelearning/webnn/issues/6 -> Issue 6 Custom operations (by dsmilkov) [v2] 14:42:44 https://github.com/webmachinelearning/webnn/issues/156 -> CLOSED Issue 156 Support CPU - WebAssembly scenario of the op level execution use case (by huningxin) 14:43:14 q? 14:43:52 q? 14:43:59 q+ 14:44:03 ack RafaelCintron 14:44:31 q+ 14:44:33 RafaelCintron: in Chromium the has been CLs landed for "npu" device type, we're testing with independent HW vendors on how it works across models 14:44:35 q? 14:44:42 ack ningxin 14:45:17 ningxin: WebNN samples have landed "npu" data type for image classification fp16 for MobileNet, SqueezeNet, ResNet 14:45:30 ... you can test with those samples already 14:45:47 ... object detection, part of WebNN samples, also has fp16 model for NPU device type testing 14:45:47 q? 14:46:14 q? 14:46:22 https://webmachinelearning.github.io/webnn-samples/image_classification/index.html has npu device type and fp16 models support 14:46:55 Topic: MLBuffer 14:47:01 Subtopic: Direct buffer sharing proposal 14:47:05 anssik: issue #688 14:47:06 https://github.com/webmachinelearning/webnn/issues/688 -> Issue 688 [MLBuffer] Support interop with WebGPU (by bbernhar) [webgpu interop] 14:47:09 ... I believe our next step is still to talk to the WebGPU group 14:47:17 ... Rafael, Bryan, MikeW to be the messengers toward WebGPU group 14:47:23 ... anything specific to discuss for the direct buffer sharing proposal since the last discussion? 14:47:44 Bryan: correct, another member of my team has been working on the implementation proposal 14:47:51 https://webmachinelearning.github.io/webnn-samples/object_detection/ also has npu device type and fp16 model (SSD MobileNet) for testing 14:48:19 Subtopic: MLGraphBuilder and MLBuffer construction alignment proposal 14:48:25 anssik: issue #697 14:48:25 https://github.com/webmachinelearning/webnn/issues/697 -> Issue 697 Inconsistency between MLGraphBuilder and MLBuffer construction (by reillyeon) [webgpu interop] 14:48:28 ... the MLGraphBuilder and MLBuffer construction alignment proposal has received a proposal from Bryan: 14:48:34 ... createBuffer() return a promise 14:48:46 ... It would: 14:48:50 ... 1. Make MLBuffer construction consistent. 14:48:54 ... 2. Offer a better OOM debugging experience. 14:48:58 ... feedback or comments? Can we more forward with this proposal? Ningxin gave thumbs up. 14:49:10 Bryan: feedback would be great 14:49:11 q? 14:49:28 q? 14:49:47 Topic: TAG feedback: resource contention considerations for GPU/CPU/accelerators (last call revised) 14:49:52 -> TAG review feedback issue https://github.com/w3ctag/design-reviews/issues/933#issuecomment-2126805061 14:49:52 https://github.com/w3ctag/design-reviews/issues/933 -> CLOSED Issue 933 Updated review of WebNN API (by dontcallmedom) [Priority: urgent] [Progress: review complete] [Review type: small delta] [Review type: horizontal review] [Venue: WebML CG] [Resolution: satisfied with concerns] [Mode: breakout] 14:49:52 … [Topic: Machine Learning] [Focus: Web architecture (pending)] [Focus: Security (pending)] [Focus: Privacy (pending)] 14:50:00 anssik: TAG was happy to see our group was responsive to their feedback 14:50:10 ... special thanks to Rafael and Reilly for chiming in on that TAG review issue 14:50:18 ... TAG review is now resolved as "satisfied with concerns" 14:50:23 -> TAG review resolution "satisfied with concerns" https://github.com/w3ctag/design-reviews/issues/933#issuecomment-2192643962 14:50:27 anssik: quoting the TAG response for the concerns part: 14:50:33 "Our main concern is: has this API considered the full range of hardware that it might need to run on? We see this running on CPUs without neural processing extensions, GPUs without extensions, CPUs with extensions, GPUs with extensions, and dedicated ML hardware. What steps have you taken to ensure that this runs across all of these targets, considering the range of hardware that exists and might exist?" 14:51:08 "Our second and related concern is about multi-implementer support. If this is going to be ubiquitous as an approach to "do NN on the web" then it really needs to be implemented across different platforms and different hardware." 14:51:54 anssik: I believe the first concern we can address by demonstrating wide implementation experience across hardware, we're doing this already and should keep on diversifying our implementations 14:52:28 ... for the second concern, we are well positioned as a group, we have participants from nearly all major browsers (except Mozilla), all major OS vendors, and nearly all hardware vendors who have products in this space, and the group is open for new participants from the ecosystem who wish to join 14:52:55 +1 to what Anssi said 14:53:03 anssik: concretely, to satisfy TAG review feedback, it looks like we should add the following to the WebNN spec: 14:53:09 q+ 14:53:21 ... - implementation considerations for GPU but also for CPU and ML accelerators based on Rafael's and Reilly's contributions 14:53:32 ... - a security considerations for denial of service attacks similar to WebGPU 14:53:49 ... other things we should continue to do is to further diversify our implementation experience and our participation -- we're doing the right things on this front so we just have to keep up the momentum 14:54:21 ... any comments on our TAG review response? Who is the first to craft a PR to address the TAG review feedback will surely get a virtual TAG Thank You Acknowledgement :-) 14:54:26 q? 14:54:30 ack dom 14:55:18 dom: quick reaction to the first concern, maybe there's something deeper to look into, what we do from an architecture perspective to make sure WebNN will adapt as new hardware emerges 14:55:33 ... extending to NPU, other future purpose-built hardware 14:55:38 jsbell has joined #webmachinelearning 14:56:06 ... how well does WebNN fallback if requested hardware does not exists? 14:56:27 ... the impact of WebNN low-end devices? that don't have fancy hardware acceleration capabilitites? 14:56:29 q? 14:57:07 anssik: should our response be in the explainer or in the spec? 14:57:25 dom: depends, good to first write it up and then see where to land it 14:57:50 q? 14:58:16 Topic: Open issue and PRs 14:58:20 Subtopic: Debrief on PRs merged recently 14:58:24 -> Recently merged PRs https://github.com/webmachinelearning/webnn/pulls?q=is%3Apr+is%3Amerged 14:58:28 anssik: issue n/a fixed by PR #708 14:58:29 https://github.com/webmachinelearning/webnn/pull/708 -> MERGED Pull Request 708 Link fix: Link to "is empty" not "empty" when an adjective is intended (by inexorabletash) 14:58:32 ... issue n/a fixed by PR #709 14:58:33 https://github.com/webmachinelearning/webnn/pull/709 -> MERGED Pull Request 709 Write example decompositions as complete functions (by inexorabletash) 14:58:36 ... issue n/a fixed by PR #710 14:58:36 https://github.com/webmachinelearning/webnn/pull/710 -> MERGED Pull Request 710 Style: Consistently use backticks for expressions in prose (by inexorabletash) 14:58:40 ... issue n/a fixed by PR #711 14:58:40 https://github.com/webmachinelearning/webnn/pull/711 -> MERGED Pull Request 711 Remove unnecessary subsections, make order consistent (by inexorabletash) 14:58:50 ... thanks to Josh for these PRs and Dwayne, Ningxin, Zoltan, others for timely review 14:59:25 jsbell: nothing big, no need to discuss these PRs today 14:59:45 q? 15:00:07 Subtopic: See you in August 15:00:14 anssik: Thank You all for all the great work done in the first half of 2024! 15:00:18 ... it's been super busy time for our group and we've accomplished a lot e.g.: 15:00:23 ... the group delivered a major spec update in a new Candidate Recommendation with transformers support 15:00:27 ... the group participation continued to grow and diversify 15:00:32 ... the group participants advanced and diversified implementations significantly 15:00:45 ... the group's work got acknowledged outside this WG in major events and keynotes, quite a compliment and very humbling 15:01:01 ... the second half looks super exciting, and we'll have a F2F coming up in September! 15:01:05 ... it is my privilege to chair this group of dedicated, professional and ambitious people who like solve hard problems and deliver 15:01:18 ... thank you all, have a relaxing summer those of you who'll take time off during this period 15:01:24 ... See you all in August! 15:01:29 RRSAgent, draft minutes 15:01:30 I have made the request to generate https://www.w3.org/2024/06/27-webmachinelearning-minutes.html anssik 15:02:09 RRSAgent, draft minutes 15:02:10 I have made the request to generate https://www.w3.org/2024/06/27-webmachinelearning-minutes.html anssik 15:19:38 s|@@@|https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0004/Transformers.js-update.pdf 15:19:44 RRSAgent, draft minutes 15:19:46 I have made the request to generate https://www.w3.org/2024/06/27-webmachinelearning-minutes.html anssik 15:20:32 s/contributions on/contributors on 15:21:35 s/WebNN and/WebNN? 15:22:08 s/important/is important 15:23:01 s/for perf/so perf 15:23:21 s/collaborate/collaborate with 15:23:43 s/working,/working on, 15:26:27 s/the has been/there has been 15:29:08 s/low-end/on low-end 15:29:22 s/capabilitites/capabilities 15:30:23 RRSAgent, draft minutes 15:30:25 I have made the request to generate https://www.w3.org/2024/06/27-webmachinelearning-minutes.html anssik 15:31:17 s/PR previews/PR reviews 15:32:20 s/this when/this is when 15:33:44 RRSAgent, draft minutes 15:33:45 I have made the request to generate https://www.w3.org/2024/06/27-webmachinelearning-minutes.html anssik 17:08:56 Zakim has left #webmachinelearning