13:48:13 RRSAgent has joined #webmachinelearning 13:48:17 logging to https://www.w3.org/2024/08/22-webmachinelearning-irc 13:48:17 Meeting: WebML WG Teleconference – 22 August 2024 13:48:17 RRSAgent, make logs Public 13:48:18 please title this meeting ("meeting: ..."), anssik 13:48:19 Chair: Anssi 13:48:23 Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2024-08-22-wg-agenda.md 13:48:28 Scribe: Anssi 13:48:32 scribeNick: anssik 13:48:44 gb, this is webmachinelearning/webnn 13:48:44 anssik, OK. 13:48:50 Present+ Anssi_Kostiainen 13:48:58 RRSAgent, draft minutes 13:48:59 I have made the request to generate https://www.w3.org/2024/08/22-webmachinelearning-minutes.html anssik 13:55:17 AramZS has joined #webmachinelearning 13:57:02 Present+ Rafael_Cintron 13:57:08 Present+ Dwayne_Robinson 13:58:44 zkis has joined #webmachinelearning 13:58:45 Present+ Zoltan_Kis 13:59:29 Present+ Joshua_Lochner 13:59:46 dwayner has joined #webmachinelearning 14:00:40 ningxin has joined #webmachinelearning 14:01:11 Present+ Ningxin_Hu 14:01:37 Present+ Michael_McCool 14:02:08 McCool has joined #webmachinelearning 14:02:43 anssik: I'm seeing many of our agenda topics receive your contributions over the past few week, thanks for moving things forward! 14:02:50 Topic: Last Call: TPAC 2024 registration and WebML WG F2F agenda 14:02:52 RafaelCintron has joined #webmachinelearning 14:03:29 anssik: W3C TPAC 2024 takes place in Anaheim, CA, USA at the Hilton Anaheim on 23–27 September 2024. The WebML WG will meet F2F on Monday, 23 September 2024, 09:00–18:00 PDT. 14:03:52 ... you should have received a TPAC meeting invite some hours ago, if not, please let me know 14:03:54 Joshua_Lochner has joined #webmachinelearning 14:04:09 ... this TPAC invite is titled "Web Machine Learning Working Group" to differentiate from our bi-weekly teleconferences 14:04:14 ... we'll add the Zoom URL in an update, note it'll be different from the usual Zoom URL 14:04:25 ... the registration is open until 13 September 2024 14:04:47 ... the meeting hotel is now sold out on 23th, alternative hotels are still available: 14:04:52 -> Alternative hotels https://www.w3.org/2024/09/TPAC/venue.html 14:05:13 ... two things to do: 14:05:17 ... 1) Please complete the registration form: https://www.w3.org/2024/09/TPAC/#registration 14:05:21 ... if you participate remotely, please choose the "I will attend remotely" option 14:05:30 ... 2) Review the in-development F2F agenda and provide your suggestions: 14:05:48 ... -> WebML WG - TPAC 2024 agenda https://github.com/webmachinelearning/meetings/issues/25 14:05:49 https://github.com/webmachinelearning/meetings/issues/25 -> Issue 25 WebML WG - TPAC 2024 agenda (by anssiko) 14:06:38 ... Our agenda is shaping up nicely and we'll have a number of guests joining and presenting on interesting topics 14:07:10 ... now would be a good time to share any additional proposals as next we'll start to map the proposals to timeslots 14:07:21 ... Please share your agenda suggestions in the GH issue by the end of your workday, or chime in now 14:07:31 MikeWyrzykowski has joined #webmachinelearning 14:07:41 anssik: current F2F topics grouped: 14:08:00 Present+ Mike_Wyrzykowski 14:08:20 ... ethics: 14:08:47 ... - in-browser explainability libraries and viz tools (Jay Wang, OpenAI / Georgia Tech) 14:09:24 ... process-y and scope: 14:10:07 ... - triage pass through open issues at F2F: breaking changes, priorities, next steps for the issue (All) 14:10:34 ... - wide review status, close on TAG review feedback 14:10:49 ... - W3C “living standards” topic (Dom?) 14:11:21 ... - a refreshed analysis of popular models, operator & data type gaps (Dwayne et al.) 14:11:46 ... new features: 14:11:59 ... - Quantization and dequantization (QDQ), a bag of multiple issues (JoshB) 14:12:19 ... - platform capability detection (Phillis et al.?) 14:12:41 ... - future-proof device selection abstractions 14:12:55 McCool has joined #webmachinelearning 14:13:17 ... interop and cross-group coordination: 14:13:27 ... - MLBuffer and WebGPU interop (MikeW) 14:14:14 MikeW: we are willing to contribute WebKit perspective to this discussion 14:14:43 ... customer feedback and trials: 14:14:56 ... - Transformers.js WebNN backend (JoshuaL) 14:15:06 https://github.com/xenova/transformers.js/pull/890 14:15:06 https://github.com/xenova/transformers.js/pull/890 -> MERGED Pull Request 890 Initial WebNN Support (by ibelem) 14:15:11 ... - ONNX Runtime Web & WebNN EP 14:15:23 ... - WebLLM + MLC discussion (TQ) 14:15:42 ... - interop issues across different backends (Ningxin et al.) 14:15:57 ... - next step for implementations, Origin Trial or equivalent and align with framework developer feedback 14:16:08 ... - core operator set #573 14:16:08 https://github.com/webmachinelearning/webnn/issues/573 -> Issue 573 Core operator set (by philloooo) [question] [opset] 14:16:54 ... incubations: 14:17:06 ... - Built-in APIs for translation and prompting (Domenic) 14:17:24 ... - custom ops (Ningxin) 14:17:27 WebNN is now supported in Transformers.js V3 (see https://github.com/xenova/transformers.js/pull/890#issuecomment-2295602778). The last thing to do is to add this to the config.json so that the user doesn't need to specify freeDimensionOverrides 14:17:38 ... - model management (McCool) 14:18:11 ... now I'd like to understand if there is preference for any specific session(s) to happen AM or PM? 14:18:33 ... my priority is to make sure any remote participants who are driving topics have as reasonable working hours as possible 14:18:37 -> Event Times Around the World https://www.timeanddate.com/worldclock/fixedtime.html?iso=20240923T1600 14:18:57 q? 14:19:09 Topic: Device selection abstractions 14:19:33 anssik: We agreed to evolve MLContextOptions and other API controls for device selection informed by further implementation experience and new use cases from the wider web community. 14:19:40 ... thank you MikeW for submitting feedback on behalf of the WebKit project, we'll discuss it today 14:19:49 ... issue #749 14:19:49 https://github.com/webmachinelearning/webnn/issues/749 -> Issue 749 MLContextOptions.deviceType seems unnecessary outside of conformance testing (by mwyrzykowski) [device selection] 14:19:56 anssik: key points from the feedback: 14:20:03 ... - MLContextOptions.deviceType is currently unimplementable via CoreML 14:20:14 ... - MLContextOptions.deviceType, as currently specified, would lead to additional fragmentation, works on some devices, fails on others 14:20:28 ... - The browser has better insight into workloads than the website author 14:20:46 anssik: MikeW, I'll let you expand on this WebKit feedback and then we can discuss better device selection abstractions you may have in find 14:20:48 q? 14:21:29 MikeW: if we must run a model on NPU and fails, not optimal, hints system would be better 14:21:51 ... to have a device selection would lead to fingerprinting concerns 14:22:50 q+ 14:23:27 MikeW: power preference could be perhaps combined with the device type hints 14:24:10 ... CoreML build on the hints concept, MLComputeUnits 14:24:11 q? 14:24:17 ack McCool 14:24:47 McCool: timing attacks concerns, a particular GPU can be detected 14:24:49 ?+ 14:24:52 q+ 14:24:56 ack MikeWyrzykowski 14:25:21 MikeW: discussed in WebGPU WG, mitigation strategies for browser vendors is to add tiny bit of noise to calculations 14:25:43 ... timing can be solved on the browser level to some level similar to canvas noise injection 14:25:57 ... browser vendor can device what level of privacy protection to add 14:26:19 q+ 14:26:20 ... unless browser leaves it to LCD harder to implement in a privacy sensitive manner 14:26:22 q? 14:26:49 ack RafaelCintron 14:27:27 RafaelCintron: for fingerprinting, it is a concern but I think we're somewhat limited in what we can do without compromising web developer's ability to do cool things 14:28:02 ... a lot of capability bits that tell what extensions you can use with your GPU 14:28:26 ... cannot be so that web developer knows nothing about the hardware capabilitities 14:28:41 This already exposes quite a bit of info, no need for timing comparisons - https://developer.mozilla.org/en-US/docs/Web/API/GPUSupportedLimits 14:28:48 ... wanted to ask MikeW about CoreML, we know CoreML has hints too and they were added for a reason 14:29:07 q+ 14:29:07 ... if they're needed for writing native code on Apple platform it looks like we need something similar on the web platform that is more diverse 14:29:46 ... sometimes GPUs are better or worse, we should explore hinting mechanisms and see how people use them to inform our designs 14:30:00 ... defer hints later to model creation time would complicate things 14:30:03 q? 14:30:06 ack MikeWyrzykowski 14:30:38 MikeW: the hints generally seem fine, the difference between hint and a requirement is important, some overlap between power preference and device type, do you need both? 14:30:51 RafaelCintron: would it be OK to have hints and know what was picked? 14:31:05 MikeW: don't understand why the website should know what was picked (by the implementation) 14:31:26 ... browser could say I support types CPU and NPU 14:31:27 q? 14:33:17 anssik: how do you mitigate against WebGPU extensions fingerprint? 14:33:47 MikeW: by bucketization, this is a problem that we try to mitigate by grouping similar devices 14:33:59 ... we have the freedom to change the buckets later 14:34:47 ... for extension detection is a more of a problem in Intel Mac machines that have more extensions they support, even there many extensions are very similar than Mac silicon Macs so cannot tell which Mac silicon Mac is used 14:35:27 q+ 14:35:29 ack ningxin 14:35:53 ningxin: thanks for the good discussion, want to provide another aspect I shared in #623 14:35:54 https://github.com/webmachinelearning/webnn/issues/623 -> Issue 623 WebNN should support NPU and QDQ operations (by wchao1115) [v2] [opset] [feature request] [device selection] 14:36:14 ... frameworks may want to make WebGPU subgraph, e.g. ONNX Runtime runs a model on a Wasm EP 14:36:55 ... some of this subgraph can be accelerated on WebNN, some ops run on Wasm EP, want the WebNN graphs to be executed on CPU device so tensor data exchange could be on the same device without perf cliff due to copying data back and forth 14:37:14 ... WebNN context can be created from GPU device 14:37:40 ... want to mention this mixed execution device scenario, subgraph execution and efficient data exchange 14:37:42 q? 14:38:16 MikeW: I think these use cases are great, and browser implementations have insights to shift compute to the most appropriate devices 14:38:39 ... ahead of time some indication can be provided that you want to run it on multiple devices, WebKit is open to that in abstract, some concerns with the design specifics 14:38:43 q? 14:38:46 q+ 14:38:51 ack dwayner 14:39:31 dwayner: currently there's only concrete device types, one option could be to add "do whatever is the best" device type, that's relax the device type to device type hint 14:39:53 ... I see some cases where you want to define GPU explicitly, maybe discrete vs integrated 14:40:29 ... website author may have more insights into the workload than the browser, especially when classical apps are becoming web apps 14:40:41 ... MLComputeUnits could expose more low level controls 14:42:03 q? 14:43:05 Topic: Open issue and PRs 14:43:10 anssik: We did not have time to acknowledge great work done during July. Belated thank you! 14:43:14 -> Recently merged PRs https://github.com/webmachinelearning/webnn/pulls?q=is%3Apr+is%3Amerged 14:43:21 anssik: here's what landed since end of June while our calls were on a break: 14:43:25 ... anssik: normative/breaking changes 14:43:30 ... issues #442 #678 fixed by PR #647 14:43:30 https://github.com/webmachinelearning/webnn/pull/647 -> MERGED Pull Request 647 Introduce MLNumber for specifying numeric inputs of any type (by inexorabletash) 14:43:30 https://github.com/webmachinelearning/webnn/issues/442 -> CLOSED Issue 442 Type of some parameters should match the input data type (by Honry) [feature request] [operator specific] 14:43:30 https://github.com/webmachinelearning/webnn/issues/678 -> CLOSED Issue 678 Specifies scalar values casted to match input type. (by philloooo) [feature request] 14:43:33 ... issue #466 fixed by PR #649 14:43:34 https://github.com/webmachinelearning/webnn/pull/649 -> MERGED Pull Request 649 Add axis argument to softmax() (by inexorabletash) 14:43:35 https://github.com/webmachinelearning/webnn/issues/466 -> CLOSED Issue 466 Softmax axis absent (by fdwr) [operator specific] 14:43:38 ... as proposed in issue #689 fixed by PR #718 14:43:39 https://github.com/webmachinelearning/webnn/pull/718 -> MERGED Pull Request 718 Replace MLActivation with MLRecurrentNetworkActivation (by a-sully) 14:43:41 https://github.com/webmachinelearning/webnn/issues/689 -> Issue 689 Consider removing `lstm` and `gru` operators (by a-sully) [question] [operator specific] 14:43:41 ... issue #629 fixed by PR #724 14:43:44 ... issue from code review fixed by PR #719 - rename 14:43:45 https://github.com/webmachinelearning/webnn/pull/724 -> MERGED Pull Request 724 change argmin/argmax to take scalar axis (by philloooo) 14:43:48 ... issue #652 fixed by PR #722 14:43:48 https://github.com/webmachinelearning/webnn/issues/629 -> CLOSED Issue 629 argMax/Min only support scalar axis in TFLite runtime (by fujunwei) [operator specific] [interop] 14:43:51 ... issue #531 fixed by PR #723 - error handling 14:43:52 https://github.com/webmachinelearning/webnn/pull/719 -> MERGED Pull Request 719 Rename where()'s parameters "input" and "other" to "trueValue" and "falseValue" (by shiyi9801) 14:43:55 anssik: tools related: 14:43:56 https://github.com/webmachinelearning/webnn/pull/722 -> MERGED Pull Request 722 Remove argmin/max selectLastIndex parameter (by philloooo) 14:43:58 ... issue n/a fixed by PR #720 - tools 14:43:59 https://github.com/webmachinelearning/webnn/issues/652 -> CLOSED Issue 652 ArgMax/Min `selectLastIndex` is not supported on CoreML (by philloooo) [bug] [operator specific] [interop] 14:44:02 ... issue n/a fixed by PR #727 - lint 14:44:03 https://github.com/webmachinelearning/webnn/pull/723 -> MERGED Pull Request 723 Define error handling of MLNamedArrayBufferViews transfer algorithm (by inexorabletash) 14:44:07 ... issue n/a fixed by PR #728 - lint workflow 14:44:07 https://github.com/webmachinelearning/webnn/issues/531 -> CLOSED Issue 531 Drop the support of synchronous execution (by huningxin) 14:44:10 https://github.com/webmachinelearning/webnn/pull/720 -> MERGED Pull Request 720 Ensure every method dfn is correctly associated with an interface (by a-sully) 14:44:10 ... issue n/a fixed by PR #731 - lint 14:44:13 https://github.com/webmachinelearning/webnn/pull/727 -> MERGED Pull Request 727 Lint: Improve logic for validating algorithm steps. (by inexorabletash) 14:44:14 anssik: other issues, editorial, clarifications etc. 14:44:17 https://github.com/webmachinelearning/webnn/pull/728 -> MERGED Pull Request 728 Workflow: Add steps to run tools/lint.mjs after spec is built (by inexorabletash) 14:44:20 https://github.com/webmachinelearning/webnn/pull/731 -> MERGED Pull Request 731 Update documentation and fix some minor issues with lint tool. (by inexorabletash) [editorial] 14:44:20 ... issue #713 fixed by PR #715 14:44:23 ... issue #567 fixed by PR #717 14:44:24 https://github.com/webmachinelearning/webnn/pull/715 -> MERGED Pull Request 715 Fix #713: DOMString to USVString (by zolkis) 14:44:26 ... issue #574 fixed by PR #721 14:44:27 https://github.com/webmachinelearning/webnn/issues/713 -> CLOSED Issue 713 Use USVString for operand name (by philloooo) [bug] 14:44:30 ... issue #443 fixed by PR #725 - security considerations 14:44:31 https://github.com/webmachinelearning/webnn/pull/717 -> MERGED Pull Request 717 Allow MLGraphBuilder.build() to be called only once (by a-sully) 14:44:33 ... issue #489 fixed by PR #726 14:44:34 https://github.com/webmachinelearning/webnn/issues/567 -> CLOSED Issue 567 Can an MLGraphBuilder be reused? (by reillyeon) [question] 14:44:36 ... issue n/a fixed by PR #729 14:44:38 https://github.com/webmachinelearning/webnn/pull/721 -> MERGED Pull Request 721 Style: Link method argument definitions (by inexorabletash) 14:44:39 ... issue #653 fixed by PR #730 - add dict member 14:44:41 https://github.com/webmachinelearning/webnn/issues/574 -> CLOSED Issue 574 Consider alternate styling/linking for method argument definitions (by inexorabletash) [question] [editorial] [conventions] 14:44:43 ... issue #351 fixed by PR #732 14:44:45 https://github.com/webmachinelearning/webnn/pull/725 -> MERGED Pull Request 725 Add security consideration for computation control-flow attacks (by inexorabletash) 14:44:48 https://github.com/webmachinelearning/webnn/issues/443 -> CLOSED Issue 443 Add security consideration for computation control-flow attack based on weights / constants change (by huningxin) [editorial] 14:44:50 ... issue n/a fixed by PR #733 14:44:52 https://github.com/webmachinelearning/webnn/pull/726 -> MERGED Pull Request 726 Clarify the cast() op behavior between different data types (by inexorabletash) 14:44:54 ... issue n/a fixed by PR #735 14:44:56 https://github.com/webmachinelearning/webnn/issues/489 -> CLOSED Issue 489 Clarify the casting behavior from floating-point / signed integers <-> unsigned integers (by huningxin) [operator specific] [interop] 14:44:57 ... issue n/a fixed by PR #735 14:45:00 https://github.com/webmachinelearning/webnn/pull/729 -> MERGED Pull Request 729 Avoid "sequence" in prose, linkify use in types (by inexorabletash) [editorial] [conventions] 14:45:01 ... issue #734 fixed by PR #738 14:45:04 https://github.com/webmachinelearning/webnn/pull/730 -> MERGED Pull Request 730 Add outputDataType to argmin/argmax (by philloooo) 14:45:04 ... issue #740 fixed by PR #741 14:45:07 https://github.com/webmachinelearning/webnn/issues/653 -> CLOSED Issue 653 Consider changing output type of ArgMax/Argmin to int32, or allow passing output_type (by philloooo) [bug] [operator specific] [interop] 14:45:08 ... issue #585 fixed by PR #742 14:45:11 https://github.com/webmachinelearning/webnn/pull/732 -> MERGED Pull Request 732 Reference newly landed WebIDL "transferable" definition (by inexorabletash) 14:45:11 ... issue #590 fixed by PR #743 14:45:14 https://github.com/webmachinelearning/webnn/issues/351 -> CLOSED Issue 351 Need to define error handling of MLNamedArrayBufferViews transfer algorithm (by huningxin) [bug] 14:45:14 ... issue #378 fixed by PR #745 14:45:17 ... issue n/a fixed by PR #746 14:45:18 https://github.com/webmachinelearning/webnn/pull/733 -> MERGED Pull Request 733 Editorial: Fix an instance of "nchw" enum member styling/linking (by inexorabletash) [editorial] 14:45:22 https://github.com/webmachinelearning/webnn/pull/735 -> MERGED Pull Request 735 Editorial: Don't reference IDL boolean for values in tensors (by inexorabletash) [editorial] 14:45:23 ... thanks Josh, Ningxin, Dwayne, Zoltan, Austin, Phillis, Shiyi, Bruce, others for PRs & reviews! 14:45:25 https://github.com/webmachinelearning/webnn/pull/738 -> MERGED Pull Request 738 Update dimension valid range to signed integer (by philloooo) 14:45:28 ... a lot of progress both in spec, but also in tools, lint, workflow improvements that will make editing this spec even more fun :-) 14:45:29 https://github.com/webmachinelearning/webnn/issues/734 -> CLOSED Issue 734 Restrict dimensions size to [0, INT32_MAX] (by philloooo) [interop] 14:45:32 https://github.com/webmachinelearning/webnn/issues/740 -> CLOSED Issue 740 Question on reduction with empty axes and scalar input (by philloooo) [question] [editorial] 14:45:36 https://github.com/webmachinelearning/webnn/pull/741 -> MERGED Pull Request 741 Clarify reduction with empty axes and scalar input (by inexorabletash) [editorial] 14:45:40 https://github.com/webmachinelearning/webnn/pull/742 -> MERGED Pull Request 742 Add optional operator labels for more diagnosable error messages (by inexorabletash) 14:45:43 https://github.com/webmachinelearning/webnn/issues/585 -> CLOSED Issue 585 Consider adding node `label`s for more diagnosable error messages for async errors. (by philloooo) [feature request] 14:45:47 https://github.com/webmachinelearning/webnn/pull/743 -> MERGED Pull Request 743 Make prelu() bidirectionally broadcast, improve broadcast wording (by inexorabletash) 14:45:51 https://github.com/webmachinelearning/webnn/issues/590 -> CLOSED Issue 590 Consider adopting new broadcasting rules (by a-sully) [question] [interop] 14:45:54 https://github.com/webmachinelearning/webnn/pull/745 -> MERGED Pull Request 745 Add blurb explaining broadcasting in more detail (by inexorabletash) [editorial] 14:45:58 https://github.com/webmachinelearning/webnn/issues/378 -> CLOSED Issue 378 Add samples of shape broadcasting (by huningxin) [testing] 14:46:01 https://github.com/webmachinelearning/webnn/pull/746 -> MERGED Pull Request 746 Bug fix: softmax()'s axis argument should EnforceRange (by inexorabletash) 14:46:32 Zoltan: interested in issue #754 14:46:33 https://github.com/webmachinelearning/webnn/pull/754 -> Pull Request 754 Add MLTensor explainer (by a-sully) 14:46:44 Subtopic: MLTensor explainer 14:47:21 ningxin: MLTensor is a rename of MLBuffer, captures all the discussion around MLBuffer 14:47:31 ... captures the consensus on different MLBuffer issues 14:47:35 ... my +1 14:47:54 ... I need Bryan, Rafael, Dwayne and others to take a look 14:48:05 Zoltan: good to have consensus before TPAC 14:49:20 ningxin: I think the current MLBuffer issues are good, we can consider MLTensor a rename of MLBuffer 14:49:53 ... Bryan has an issue to rename MLBuffer to MLTensor, I already gave my thumbs up and Austin too 14:50:11 ... MLTensor is an evolved MLBuffer, all MLBuffer discussions prior are still relevant 14:50:32 ... we can probably close some MLBuffer issues that have been addressed 14:50:53 Zoltan: explainer is a public summary of all the arguments 14:51:25 ningxin: ONNX RT feedback for MLTensor with Chromium, this explainer will help solicit that 14:51:35 ... that's why Austin volunteered to put this explainer in place 14:52:11 ... another things, with MLTensor and dispatch, we can probably thing of deprecation of compute() a breaking change, we can discuss that with the MLTensor dispatch proposal 14:52:25 q? 14:53:26 q+ 14:53:50 RRSAgent, draft minutes 14:53:51 I have made the request to generate https://www.w3.org/2024/08/22-webmachinelearning-minutes.html anssik 14:54:42 Subtopic: [operator specific] How to define the algorithm of L2_Pool2d? 14:54:46 anssik: issue #278 14:54:47 https://github.com/webmachinelearning/webnn/issues/278 -> Issue 278 How to define the algorithm of L2_Pool2d? (by mingmingtasd) [question] [operator specific] 14:55:04 ningxin: update from Jianwei, TFLite engineer assigned to fix the related bug 14:55:11 ... this issue was blocking the Chromium TFLite backend development for l2Pool2d 15:01:08 RRSAgent, draft minutes 15:01:09 I have made the request to generate https://www.w3.org/2024/08/22-webmachinelearning-minutes.html anssik 15:23:31 s/in find/thought about 15:23:46 s/and fails/and it fails 15:24:37 s/can device/can decide 15:28:18 s/that's relax/that'd relax 15:28:50 s/We did not/on our last call we did not 15:31:32 s/things,/thing, 15:31:53 s/thing of/think of 15:34:22 s/Jianwei/Jiewei 15:34:41 RRSAgent, draft minutes 15:34:43 I have made the request to generate https://www.w3.org/2024/08/22-webmachinelearning-minutes.html anssik 17:04:09 Zakim has left #webmachinelearning