13:56:11 RRSAgent has joined #webmachinelearning 13:56:16 logging to https://www.w3.org/2024/09/05-webmachinelearning-irc 13:56:16 Meeting: WebML WG Teleconference – 5 September 2024 13:56:16 RRSAgent, make logs Public 13:56:17 please title this meeting ("meeting: ..."), anssik 13:56:17 Chair: Anssi 13:56:21 Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2024-09-05-wg-agenda.md 13:58:56 Scribe: Anssi 13:59:00 scribeNick: anssik 13:59:09 gb, this is webmachinelearning/webnn 13:59:09 anssik, OK. 13:59:13 ningxin has joined #webmachinelearning 13:59:15 RRSAgent, draft minutes 13:59:17 I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik 14:01:29 RafaelCintron has joined #webmachinelearning 14:01:32 Present+ Joshua_Lochner 14:01:33 Mike_Wyrzykowski has joined #webmachinelearning 14:01:38 Present+ Michael_McCool 14:01:42 Present+ Ningxin_Hu 14:01:53 Present+ Mike_Wyrzykowski 14:02:03 Present+ Austin_Sullivan 14:02:05 dom has joined #webmachinelearning 14:02:24 Present+ Dwayne_Robinson 14:02:51 Present+ Dominique_Hazael-Massieux 14:02:54 asully has joined #webmachinelearning 14:03:53 RRSAgent, draft minutes 14:03:54 I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik 14:04:50 Topic: TPAC F2F agenda published 14:05:16 Issue 180 not found 14:05:18 -> https://github.com/webmachinelearning/meetings/issues/25 14:05:18 https://github.com/webmachinelearning/meetings/issues/25 -> Issue 25 WebML WG - TPAC 2024 agenda (by anssiko) 14:05:59 anssik: TPAC F2F agenda has been published. We'll do fine-tuning and gardening: confirm our topic leads, timeboxes, discuss group contributions that'd be welcome by F2F. 14:07:13 anssik: current F2F plan is as follows 14:07:34 McCool has joined #webmachinelearning 14:07:45 ... Start meeting 9:00 local / 16:00 UTC 14:08:04 Joshua_Lochner has joined #webmachinelearning 14:08:42 ... 15-sec WG participant intros: name, affiliation, top interest [10 mins] 14:08:50 ... currently 20 group participants and 17 observers registered in person 14:09:02 ... Agenda bashing for last-minute drops/adds, not much wiggle room to add things [5 mins] 14:09:25 ... Charter orientation mainly to inform observers and folks from other WGs [5 mins] 14:09:58 Subtopic: Ethics 14:10:16 anssik: - 9:30-10:00 local / 16:30-17:00 UTC in-browser explainability libraries and viz tools (Jay Wang, OpenAI / Georgia Tech) [30 mins] 14:10:22 ... for a sneak-peek, see Jay's PhD defense: 14:10:32 -> Democratizing Human-Centered AI with Visual Explanation and Interactive Guidance https://zijie.wang/phd-thesis/ 14:11:15 Subtopic: Spec orientation [30 mins] 14:12:03 anssik: - Triage pass through open issues: breaking changes, priorities, next steps for the issue (All) 14:12:10 ... to prep for this triage exercise, please bring with you your favorite issues and prepare to share your suggestions for next steps 14:12:14 ... focus on issues w/o agenda time already allocated 14:12:19 ... first come, first served 14:12:40 -> Triage guidance https://github.com/webmachinelearning/webnn/blob/main/docs/IssueTriage.md 14:13:46 Subtopic: New features 14:14:05 ... - A refreshed analysis of popular models, operator & data type gaps 14:14:12 ... Dwayne would you be able to present a summary that fits into 20 mins? 14:14:46 Dwayne: 15 minutes is good 14:15:25 ... - Quantization and dequantization (QDQ), a bag of multiple issues 14:16:15 Austin: someone from Google can present this 14:17:42 anssik: - Platform capability detection 14:17:57 ... spec definition and prototype done! 14:18:04 ... discuss impl experience, deferred rankRange feature 14:19:00 Austin: someone from Google side can talk to this, 5 min, TBD if needs discussion 14:19:23 ... - Future-proof device selection abstractions 14:19:36 s/... -/anssik: ... - 14:20:10 ... MikeW, can prepare a concrete proposal that'd work for WebKit for the group to discuss? 14:20:29 Mike: of course, can talk to this issue and provide a proposal, 5 minutes should be fine 14:21:08 Subtopic: Customer feedback & collaborations 14:21:17 anssik: 13:30-14:00 local - WebLLM and MLC 14:21:28 ... Tianqi Chen to share with us the latest from the WebLLM, MLC, Apache TVM 14:21:40 ... he was interested in WebNN-WebGPU interop progress, so good to share that with him 14:22:26 ... - Transformers.js WebNN backend 14:24:07 JoshuaL: will attend virtually, will showcase Transformers.js WebNN demo, EP selection via ONNX Runtime, demo in the works with up-to-date visual models 14:25:10 ... 10-15 mins user-centric demonstration, not too low-level on the details 14:25:54 ... - ONNX Runtime Web & WebNN EP 14:26:06 JoshuaL: can ask Gunther if he could present on this topic 14:26:22 ... 10-15 mins 14:26:37 anssik: - Google Chrome Feedback revisited #453 14:26:38 https://github.com/webmachinelearning/webnn/issues/453 -> Issue 453 Google Chrome Feedback on WebNN: aiming for broad device coverage and maintainability (by vsekhar) [process] [opset] [use case] 14:27:57 Austin: a lot of the points in this are overlapping with other issues, so we will check if we should keep this on the agenda 14:28:06 ... not need a lot of time regardless 14:28:22 Subtopic: Interop and cross-group coordination 14:28:33 ... - Interop issues across different backends 14:28:36 ... Ningxin, how much time to allocate for this? 14:28:43 https://github.com/webmachinelearning/webnn/issues?q=is%3Aissue+is%3Aopen+label%3Ainterop 14:29:11 ningxin: I want to work through issues labeled with "interop", some are covered in "device selection", ConstantOperand also has a dedicated issue 14:29:32 ... int64 is one interesting issue with MikeW feedback 14:29:52 ... zero-sized dimensions 14:30:07 ... want to triage all "interop" issue for discussion 14:30:43 ... - Core operator #573 14:30:43 https://github.com/webmachinelearning/webnn/issues/573 -> Issue 573 Core operator set (by philloooo) [question] [opset] 14:30:52 s/... - /anssik: - 14:31:13 Austin: related to custom op incubation, we can merge this with custom op 14:31:47 ningxin: we may reveal other options in incubation section, core op set is a good foundation 14:32:05 Austin: we can have a few minutes for this 14:33:08 ... - MLTensor #754 14:33:09 https://github.com/webmachinelearning/webnn/pull/754 -> Pull Request 754 Add MLTensor explainer (by a-sully) [webgpu interop] 14:33:26 ... Austin, possibly Corentin from WebGPU joining 14:33:50 s/... - /anssik: - 14:34:09 ... - MLConstantOperand #668 14:34:09 https://github.com/webmachinelearning/webnn/issues/668 -> Issue 668 Do we need an `MLConstantOperand`? (by a-sully) [question] [interop] 14:34:21 Subtopic: Implementation plans and trials 14:34:33 anssik: Next step for implementations, Origin Trial or equivalent and align with framework developer feedback 14:35:03 ningxin: we're at CR "call for implementations", use this as an opportunity to get signals from implementers 14:35:25 Subtopic: Advancement on the W3C Rec Track 14:35:52 anssik: - Wide review status, close on TAG review feedback 14:36:08 ... - W3C “living standards” expectations and plan with Dom 14:36:35 Dom: personally would suggest to not go to living CR model 14:36:36 Joshua_Lochner has joined #webmachinelearning 14:37:00 Subtopic: Incubation 14:37:05 anssik: - Custom ops 14:37:18 ... Ningxin, 20 mins 14:37:29 ... - Built-in APIs for translation and prompting 14:37:38 ... Dominic, ~20 mins 14:37:55 ... - Model management 14:38:01 ... Michael McCool, ~10-15 mins 14:38:15 McCool: will have an additional breakout session 14:38:23 Subtopic: Wrap up & Dinner 14:38:39 anssik: our F2F group is quite big so we're not able to book one restaurant 14:38:40 -> https://github.com/w3c/tpac2024-breakouts/issues?q=is%3Aissue+is%3Aopen+label%3A%22track%3A+AI%22 AI-related TPAC breakouts 14:38:44 ... one option would be to go to Anaheim Packing District 2.5 miles from the meeting venue 14:38:54 ... "There are several ways to enjoy the culinary artistry from our small businesses, including takeout, delivery, indoor dining and outdoor dining." 14:39:05 ... a wide range of options so could split by food preference 14:39:14 ... if some Anaheim local foodie has recommendations, please share! 14:39:20 -> https://www.anaheimpackingdistrict.com/merchants 14:40:06 RafaelCintron has joined #webmachinelearning 14:40:18 Topic: Device selection abstractions 14:40:47 anssik: We agreed to evolve MLContextOptions and other API controls for device selection informed by further implementation experience and new use cases from the wider web community. 14:40:51 ... I wanted to discuss how frameworks and backends approach configuring inference sessions, understand framework-level use cases and requirements. Distill learnings to help evolve MLContextOptions. 14:40:56 ... ONNX Runtime Web does provide some knobs, which vary by backend 14:41:00 -> Session Options API https://onnxruntime.ai/docs/api/js/interfaces/InferenceSession.SessionOptions.html 14:41:06 anssik: for example: 14:41:15 ... executionMode?: "sequential" | "parallel" - Controls whether multiple operators in the graph across nodes run sequentially or in parallel 14:41:26 ... graphOptimizationLevel?: "basic" | "all" | "disabled" | "extended" - graph-level transformations, ranging from small graph simplifications and node eliminations to more complex node fusions and layout optimizations 14:41:32 ... there's also WebNN EP Options 14:41:42 -> WebNN EP Session Options API https://onnxruntime.ai/docs/api/js/types/InferenceSession.WebNNExecutionProviderOption.html 14:41:52 anssik: WebNNExecutionProviderOption: WebNNOptionsWithoutMLContext | WebNNOptionsWithMLContext | WebNNOptionsWebGpu 14:41:58 ... interface WebNNOptionsWithMLContext extends MLContextOptions with numThreads 14:42:02 ... I wanted to understand whether all these knobs are purely ONNX implementation details or whether some of these could be WebNN MLContextOptions hints. 14:42:15 JoshuaL: would like to have Gunther involved, he'd be an expert in this 14:42:29 ... number of thread user for Wasm EP, proxy by WebWorker 14:42:48 ... I talked with Gunther that ORT docs are not up to date 14:43:17 ... for Transformers.js we set some good defaults, set num of threads 14:44:19 anssik: I also wanted to remind us of CoreML limitations and on what is implementable on Apple platforms, see issue #749 where MikeW provided information about CoreML MLComputeUnits configuration limitations 14:44:19 https://github.com/webmachinelearning/webnn/issues/749 -> Issue 749 MLContextOptions.deviceType seems unnecessary outside of conformance testing (by mwyrzykowski) [device selection] 14:44:23 -> https://developer.apple.com/documentation/coreml/mlcomputeunits/ 14:44:28 Here's the env: https://github.com/microsoft/onnxruntime/blob/main/js/common/lib/env.ts 14:44:45 anssik: the current WebNN API device selection mechanism we declared subject to change is: 14:44:49 -> Device selection https://www.w3.org/TR/webnn/#programming-model-device-selection 14:44:53 -> MLContextOptions https://www.w3.org/TR/webnn/#dictdef-mlcontextoptions 14:45:24 MikeW: not closely following framework configuration options, but will check this ahead of TPAC discussion 14:45:42 q? 14:46:02 Topic: MLTensor 14:46:06 anssik: MLTensor explainer PR #754 received a lot of great feedback and review, thank you! 14:46:06 https://github.com/webmachinelearning/webnn/pull/754 -> Pull Request 754 Add MLTensor explainer (by a-sully) [webgpu interop] 14:46:09 -> MLTensor Explainer (preview) https://github.com/webmachinelearning/webnn/blob/d3e2be575d1879c7c7cd5f438ffc6b6f7d7b3c30/mltensor-explainer.md 14:46:15 Austin: thanks for all the feedback! 14:46:58 ... re Corentin's feedback, MLTensor makes an assumption the MLTensorOptions whether readable from script (or not) is sufficient to know where to allocate the tensor on the system 14:47:12 ... open questions if this is true, if device selection is tied to context 14:47:33 ... Corentin pointed out there's issues with making GPU wait on a fence from another place in the system 14:47:52 ... there's still some open questions, with more clarify on device selection not sure how much progress we can make 14:48:45 anssik: it looks like GH feedback from Rafael, Dwayne, Ningxin addressed 14:48:58 ... GH feedback from Bryan, Phillis and Corentin is still being worked on 14:49:12 anssik: merging an explainer is lower bar than merging a spec PR 14:49:21 Austin: will add notes for Corentin's feedback in the explainer 14:49:32 Dom: what would be the next step before getting this into the spec prose and IDL? 14:49:37 ... is it device selection clarification? 14:49:47 Austin: we need to specify timelines 14:49:55 ... especially for WebGPU interop 14:50:21 ... with WebGPU has its own concept of timelines and need to define how WebNN interacts with that timeline 14:50:42 ... when WebNN rents MLTensor WebNN timeline needs to wait when returned 14:50:55 ... WebNN timeline relationship with WebGPU timeline 14:51:23 ... for context, Chai had an earlier PR that said WebNN's timelime is same as WebGPU CommandBuffer 14:51:34 ... that matches DirectML model, but not CoreML, TFLite 14:51:49 ... from spec perspective timelines must be a bit more decoupled 14:52:14 q? 14:53:01 McCool: I was planning to prototype caching based on MLBuffer/MLTensor, weights from model that can be restored 14:53:33 Austin: MLTensor used for input and outputs, the challenge is, sharing weights across models is not something that is supported by DirectML 14:56:03 RRSAgent, draft minutes 14:56:04 I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik 14:56:56 -> https://www.w3.org/wiki/MeetingTaxis Meeting Taxis for TPAC 2024 14:57:46 q? 14:57:56 Topic: See you at TPAC F2F! 14:58:04 anssik: see you at TPAC F2F on 23 September 2024 for a day of exciting discussions and more! 14:58:11 ... I'll cancel the bi-weekly telcons that have close proximity with the TPAC week: 14:58:15 ... - cancel WebML WG Teleconference – 19 September 2024 14:58:22 ... - cancel WebML WG Teleconference – 3 October 2024 14:58:33 ... Our next call after TPAC will be 17 October 2024 14:58:35 RRSAgent, draft minutes 14:58:36 I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik 15:00:36 s/... - A refreshed/anssik: - A refreshed 15:00:38 RRSAgent, draft minutes 15:00:39 I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik 15:01:04 s/Spec orientation [30 mins]/Spec orientation 15:01:06 RRSAgent, draft minutes 15:01:07 I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik 15:02:00 s/Dominic/Domenic 15:03:41 s/user for/used for 15:05:36 s/with WebGPU has/WebGPU has 15:09:08 s/not need/do not need 15:13:26 s/thread user/threads used 15:19:55 RRSAgent, draft minutes 15:19:56 I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik 15:21:08 s/number of thread/number of threads 15:21:20 RRSAgent, draft minutes 15:21:22 I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik 17:17:11 Zakim has left #webmachinelearning