14:03:53 RRSAgent has joined #webmachinelearning 14:03:58 logging to https://www.w3.org/2024/08/08-webmachinelearning-irc 14:03:58 RRSAgent, make logs Public 14:03:59 please title this meeting ("meeting: ..."), anssik 14:03:59 Meeting: WebML WG Teleconference – 8 August 2024 14:04:01 Chair: Anssi 14:04:05 Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2024-08-08-wg-agenda.md 14:04:11 RRSAgent, draft minutes 14:04:12 I have made the request to generate https://www.w3.org/2024/08/08-webmachinelearning-minutes.html anssik 14:04:28 Present+ Anssi_Kostiainen 14:04:42 Present+ Joshua_Bell 14:04:46 Regrets+ Rafael_Cintron 14:04:52 Present+ Dwayne_Robinson 14:04:58 Present+ Geoff_Gustafson 14:05:07 Present+ Ilya_Rezvov 14:05:16 Present+ Joshua_Lochner 14:05:21 Present+ Michael_McCool 14:05:26 Present+ Mike_Wyrzykowski 14:05:34 Regrets- Ningxin_Hu 14:05:43 Present+ Ningxin_Hu 14:05:49 RRSAgent, draft minutes 14:05:51 I have made the request to generate https://www.w3.org/2024/08/08-webmachinelearning-minutes.html anssik 14:06:09 Present+ Austin_Sullivan 14:06:12 anssik: Welcome back after the summer break! 14:06:24 ... and thanks for nudging things forward over the course of July 14:06:36 Topic: TPAC 2024 registration and WebML WG F2F agenda building 14:07:05 anssik: As announced in June, W3C TPAC 2024 takes place in Anaheim, CA, USA at the Hilton Anaheim on 23–27 September 2024. 14:07:12 ... And our group WebML WG will meet F2F on Monday, 23 September 2024, 09:00–18:00 PDT. 14:07:22 ... that's the first day of the TPAC meetings week that runs until Friday 14:07:31 ... I have two asks for you all regarding TPAC and WebML WG F2F 14:07:34 Subtopic: Registration 14:07:41 ... 1) Complete the registration form: 14:07:45 -> https://www.w3.org/2024/09/TPAC/#registration 14:08:03 anssik: the sooner the better to help organizers figure our space requirements and onsite/remote split for logistics 14:08:26 ... also please note W3C has made fee waivers available to remove any barriers to participation 14:08:32 -> https://www.w3.org/2024/09/TPAC/registration.html#waiver 14:08:46 anssik: any questions on registration? 14:09:30 Subtopic: F2F agenda 14:09:35 anssik: 2) Review the in development F2F agenda and provide your suggestions in the meetings GH issue: 14:09:40 -> https://github.com/webmachinelearning/meetings/issues/25 14:09:40 https://github.com/webmachinelearning/meetings/issues/25 -> Issue 25 WebML WG - TPAC 2024 agenda (by anssiko) 14:09:43 RafaelCintron has joined #webmachinelearning 14:09:52 anssik: we've already received great suggestions and proposals from both WG participants and folks outside the WG 14:10:44 ... I'm sure we'll get those last-minute ideas as usual, but thinking ahead helps especially folks outside out group who may want to join as observers to get an idea what we'll cover, also help slot agenda to allow smoother remote participation experience 14:11:03 ... let me first outline what has been proposed and then open the floor for discussion on further proposals 14:11:38 ... - proposal from Domenic to discuss built-in APIs for translation and prompting, from the Chrome team 14:12:14 ... I'd support inviting Domenic to join our F2F meeting to introduce these proposals to have a discussion on future direction, I think there's a lot of good cross-pollination of ideas to happen and these APIs could also get into this WG in the future 14:12:20 q? 14:12:32 anssik: any Googlers in WG aware of this work? 14:12:42 JoshuaB: aware of this work 14:13:21 q? 14:14:39 ... - JoshB proposed an extensive triage pass through open issues at F2F, weed out breaking changes, priorities, next steps for the issues -- a lot of work but happy to dedicate F2F time for this, identifying issues that benefit from real-time discussion as a pre-work 14:16:09 jsbell: would love to hear from others on this proposal, I know some other WGs do this, important to timebox this exercise, maybe an hour 14:16:10 q? 14:17:56 ... - quantization and dequantization (QDQ) related issues: encapsulate vs dedicated QDQ ops, quant data types int4? int2? 14:18:00 ... a good topic for F2F too 14:18:50 jsbell: I think some prework has been done, Dwayne looked at quantized linear and delinear 14:19:13 ... not sure if these two ops are enough to support important models 14:19:14 q? 14:19:22 DwayneR: two two ops are good first step 14:19:35 ... have done analysis between different backends not looked at these two ops yet 14:19:55 ningxin: we're also examining ONNX models to understand the requirements from frameworks perspetive 14:20:34 -> https://github.com/webmachinelearning/meetings/issues/25 14:20:35 https://github.com/webmachinelearning/meetings/issues/25 -> Issue 25 WebML WG - TPAC 2024 agenda (by anssiko) 14:21:41 anssik: jsbell, your thoughts on v1? 14:21:49 jsbell: not a big fan of versioning! 14:22:24 ... we want to see an Origin Trial milestone and this API to be good to be used with real users 14:22:43 ... QDQ are needed for that milestone 14:23:01 q? 14:23:14 ... - W3C “living standards” topic 14:23:47 ... we're currently periodically publishing CR Snapshots to secure RF commitments and as such we are basically already following the "living standards" path and continue to perpetually maintain the spec 14:24:31 ... happy to discuss this at F2F as well, also want to integrated W3C staff view from Dom 14:25:16 jsbell: agree with Anssi's formulation 14:27:35 Dwayne: no immediate feedback, will look at our open issues for proposals 14:27:55 ningxin: if we can have participants from implementers role that'd be good to discuss interop issues across different backends 14:27:57 MikeWyrzykowski has joined #webmachinelearning 14:28:41 ... also the next step for implementation, Origin Trial or equivalent and align with framework developer feedback 14:29:46 MikeW: I will at TPAC 14:29:49 anssik: great to hear 14:31:09 MikeW: something along the lines of MLBuffer and WebGPU interop would be an interesting F2F topic 14:31:25 q? 14:32:02 anssik: one potential topic could be platform capability detection (a la #463) 14:32:02 https://github.com/webmachinelearning/webnn/issues/463 -> Issue 463 Allow checking whether operators/types are supported for a backend before creating a graph (by huningxin) [feature request] 14:32:45 ... it looks like some sort of platform capability detection is required by frameworks that sit atop WebNN -- historically capability detection has caused some unrest on the web platform for privacy and future-proofing reasons, but we seem to have good use cases at hand and I'm confident can work out any issues 14:33:00 +1 to discuss opSupportLimits 14:33:30 anssik: another proposal: customer-centric session 14:34:00 ... that is, I'd like to make sure this WG listens to its customers carefully i.e. frameworks that integrate with WebNN API 14:34:06 ... I'd be happy to invite any such folks to present similarly to what JoshuaL did with Transformers.js in June 14:34:31 ... who from ONNX Runtime Web team be the best person to reach out to for this opportunity? Other frameworks of interest? 14:34:43 q? 14:35:14 q+ 14:35:18 ack RRSAgent 14:35:21 ack RafaelCintron 14:35:58 q+ 14:36:02 RafaelCintron: there's parallel workstream on WebNN EP we do receive feedback from ONNX RT team, not sure they will be going to TPAC 14:36:14 ... could invite them to call in if they're not joining F2F 14:36:44 DwayneR: Wanming has been adding kernels and operators to ONNX RT 14:37:37 ningxin: we can discuss with folks and have a plan 14:38:55 ... and perhaps we can get JoshuaL to re-run his great Transformers.js presentation for new folks at F2F? 14:38:56 q? 14:39:35 Joshua_Lochner: I'm mostly traveling to California soon so may overlap 14:40:07 ... I will be there for PyTorch conf 18-19 September 2024 14:40:37 ... - "ML ethics for engineers" :-) 14:40:39 q+ 14:40:45 ack ningxin 14:41:19 ningxin: for TPAC plans, checking for incubation for customer ops, is it more appropriate for breakout day? 14:42:06 +1; discussing incubation in WG session is fine. Breakout casts a wide net, but you are competing with all the other Breakouts 14:42:15 anssik: I can review your proposal to advice 14:42:21 breakout session on AI Model Management https://github.com/w3c/tpac2024-breakouts/issues/15 14:42:21 https://github.com/w3c/tpac2024-breakouts/issues/15 -> Issue 15 AI Model Management (by mmccool) [session] 14:42:43 McCool: want to propose model management topic 14:43:13 ... - "ML ethics for engineers" redux :-) 14:43:19 incubation for custom ops 14:43:23 ... I'd also like to reinvigorate work around ML ethics with a pragmatic approach 14:43:29 ... as you may recall, we have produced a doc that discuss ethical issues associated with using ML 14:43:33 -> Ethical Principles for Web Machine Learning https://www.w3.org/TR/webmachinelearning-ethics/ 14:43:49 anssik: since we're largely engineers in this group, I was thinking of inviting folks who work on tools in this space to make this ethics topic more approachable for us, so we ca see running code to help solve some of these issues 14:44:07 ... I was planning to invite folks from Georgia Tech working on WebSHAP, a JS library that can explain any machine learning models on the Web, running completely in the browser using ORT 14:44:12 -> WebSHAP https://poloclub.github.io/webshap/ 14:44:17 anssik: this tool helps with "transparency and explainability" ethical principles we identified in our ethical principles doc 14:44:20 -> Principle 6: Transparency and explainability https://www.w3.org/TR/webmachinelearning-ethics/#principle-6-transparency-and-explainability 14:45:23 Topic: Transformers.js update takeaways 14:45:36 anssik: We received an exciting and action-packed Transformers.js update presentation from Joshua Lochner on our June call. As a refresher, please check the animated presentation: 14:45:40 -> Transformers.js update https://docs.google.com/presentation/d/1BCkLdzB5cR1ZfHwFXkIDuV8BCw8-KIWICbsZ1O00H2Q/view 14:45:57 ... as discussed, Transformers.js informed greatly the WebNN expansion into transformers 14:46:12 ... to continue this productive collaboration, I wanted to revisit the feedback from June discussion to ensure it is reflected in our WebNN work, issues and priorities appropriately. 14:46:15 -> June minutes with Q&A at the end https://www.w3.org/2024/06/27-webmachinelearning-minutes.html#t04 14:46:25 anssik: our June Q&A touched on: 14:46:44 ... - ONNX Runtime Web dependency: WebNN EP backend, implementation optimization 14:47:07 ... - dynamic shapes benefits for vision models, WebNN now static shape 14:47:20 ... - perf improvements WebNN vs Wasm/WebGPU 14:47:34 ... wanted to check if folks have additional topics to record? 14:48:24 Joshua_Lochner: I'm today releasing Transformers.js v3 NPM bold 14:48:35 ... that will provide feedback directly on making sure WebNN support is working 14:48:48 ... we had a call last week to discuss WebNN Transformers.js integration in more detail 14:48:56 ... I can follow up on more concrete things we discussed 14:49:13 ... we'll add WebNN EP as an option 14:49:45 ... differences e.g. static vs. dynamic shapes, config.json specifies that we can't use dynamic 14:50:09 ... that's the first step format, four models can initially run on WebNN, vision models being the easiest because se have config file with input dimensions 14:50:24 ... all that will be done via ONNX Runtime Web and happy to follow up with more details 14:51:23 anssik: demo opportunity by TPAC to showcase Transformers.js WebNN backend? 14:51:34 Joshua_Lochner: certainly will be able to put out a demo 14:51:54 would love to! 14:52:17 q? 14:52:25 ack McCool 14:53:13 Topic: Device selection + quantization & dequantization 14:53:33 anssik: We agreed to evolve MLContextOptions and other API controls for device selection informed by further implementation experience and new use cases from the wider web community 14:53:48 ... To assist in this effort, the "npu" device type landed in Chromium and samples for fp16 models with NPU device type have been made available for experimentation. 14:53:53 -> Chromium NPU device type https://chromium-review.googlesource.com/c/chromium/src/+/5330647 14:53:57 -> NPU + fp16 samples: image classification https://webmachinelearning.github.io/webnn-samples/image_classification/ 14:54:01 -> NPU + fp16 samples: object detection https://webmachinelearning.github.io/webnn-samples/object_detection/ 14:54:07 MikeWyrzykowski has joined #webmachinelearning 14:54:11 anssik: the current device selection mechanism is specified in: 14:54:15 -> Device selection https://www.w3.org/TR/webnn/#programming-model-device-selection 14:54:19 -> MLContextOptions https://www.w3.org/TR/webnn/#dictdef-mlcontextoptions 14:54:43 anssik: we added an in-line block to MLContextOptions to set expectations that MLContextOptions is under active development, and the design is expected to change, informed by further implementation experience and new use cases from the wider web community 14:54:48 ... I note feedback from MikeW / Apple in the related PR: 14:54:52 -> https://github.com/webmachinelearning/webnn/pull/696#pullrequestreview-2123677923 14:54:53 https://github.com/webmachinelearning/webnn/pull/696 -> MERGED Pull Request 696 Add MLDeviceType npu (by fdwr) 14:54:56 "it would be preferable to remove the `If this type cannot be satisfied, an "{{OperationError}}" {{DOMException}} is thrown`, wording as I'm not sure how that is compatible with the sentence preceding it `and allow the implementation to better select the most appropriate underlying execution device for the workload.`" 14:55:56 anssik: MikeW please open a separate issue for this 14:55:59 MikeW: will do 14:56:23 anssik: to avoid ossification of the current MLContextOptions beyond this developer preview phase, I want us to discuss future-proof device selection abstractions 14:56:41 ... issue #623 mentions: 14:56:42 https://github.com/webmachinelearning/webnn/issues/623 -> Issue 623 WebNN should support NPU and QDQ operations (by wchao1115) [v2] [opset] [feature request] [device selection] 14:56:49 ... - fallback device 14:56:55 ... - multiple devices in a preferred order 14:57:00 ... - an exclusion of a specific device 14:57:11 ... - quantization & dequantization (QDQ) data types 14:57:15 ... - error handling 14:57:19 ... - ultimate fallback 14:57:46 ... I believe it is beneficial to be use case-driven here, so any additional use cases for device selection welcome, currently documented are, to recap: 14:57:46 ... - compute offloading 14:57:58 ... - op fallback 14:58:38 q? 14:58:51 anssik: potential TPAC discussion topic? 14:59:42 MikeW: I'll add more context in the issue, there's some pushback on specifying hardware device type, will provide feedback though issue 14:59:56 q? 15:00:14 RRSAgent, draft minutes 15:00:15 I have made the request to generate https://www.w3.org/2024/08/08-webmachinelearning-minutes.html anssik 15:03:02 s/McCool: I'd also like/anssik: I'd also like 15:03:05 RRSAgent, draft minutes 15:03:06 I have made the request to generate https://www.w3.org/2024/08/08-webmachinelearning-minutes.html anssik 15:05:57 s/... I'd also like to reinvigorate/anssik: I'd also like to reinvigorate 15:05:58 RRSAgent, draft minutes 15:06:00 I have made the request to generate https://www.w3.org/2024/08/08-webmachinelearning-minutes.html anssik 15:06:38 s/we ca see/we can see 15:08:35 s/se have/have 15:38:40 s/though/through/ 15:40:25 s/... - quantization and dequantization/anssik: quantization and dequantization 15:40:28 RRSAgent, draft minutes 15:40:29 I have made the request to generate https://www.w3.org/2024/08/08-webmachinelearning-minutes.html anssik 15:41:00 s/two two ops/those two ops 15:41:50 s/... - W3C “living standards” topic/anssik: - W3C “living standards” topic 15:41:53 RRSAgent, draft minutes 15:41:54 I have made the request to generate https://www.w3.org/2024/08/08-webmachinelearning-minutes.html anssik 15:42:41 s/I will at TPAC/I will be at TPAC 15:43:52 s/… and perhaps we/anssik: and perhaps we 15:44:13 s/… - "ML ethics for engineers" :-)// 15:44:16 RRSAgent, draft minutes 15:44:18 I have made the request to generate https://www.w3.org/2024/08/08-webmachinelearning-minutes.html anssik 15:46:56 s/we discussed/we can discuss 15:47:46 RRSAgent, draft minutes 15:47:47 I have made the request to generate https://www.w3.org/2024/08/08-webmachinelearning-minutes.html anssik 17:02:51 Zakim has left #webmachinelearning