14:56:53 RRSAgent has joined #webmachinelearning 14:56:57 logging to https://www.w3.org/2024/02/08-webmachinelearning-irc 14:56:57 RRSAgent, make logs Public 14:56:58 please title this meeting ("meeting: ..."), anssik 14:56:59 Meeting: WebML WG Teleconference – 8 February 2024 14:57:04 Chair: Anssi 14:57:09 Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2024-02-08-wg-agenda.md 14:58:13 jsbell has joined #webmachinelearning 14:58:17 geoff has joined #webmachinelearning 14:58:36 Scribe: Anssi 14:58:40 scribeNick: anssik 14:58:47 Present+ Anssi_Kostiainen 14:58:52 Present+ Joshua_Bell 14:58:56 Present+ Geoff_Gustafson 14:59:53 Present+ Bryan_Bernhart 15:00:06 Present+ Dwayne_Robinson 15:00:41 Present+ Joshua_Lochner 15:00:47 Present+ Dominique_Hazael-Massieux 15:00:55 Ningxin_Hu has joined #webmachinelearning 15:01:00 Present+ Reilly_Grant 15:01:04 asully has joined #webmachinelearning 15:01:16 Present+ Ningxin_Hu 15:01:17 Joshua_Lochner has joined #webmachinelearning 15:01:35 RRSAgent, draft minutes 15:01:36 I have made the request to generate https://www.w3.org/2024/02/08-webmachinelearning-minutes.html anssik 15:01:47 Deepti has joined #webmachinelearning 15:02:30 Present+ Michael_McCool 15:02:34 Present+ Deepti_Gandluri 15:02:59 chai has joined #webmachinelearning 15:03:05 Present+ Chai_Chaoweeraprasit 15:03:13 RRSAgent, draft minutes 15:03:14 I have made the request to generate https://www.w3.org/2024/02/08-webmachinelearning-minutes.html anssik 15:03:38 Topic: Announcements 15:03:46 Subtopic: Editor change 15:03:54 anssik: first, I want to make an announcement regarding an editor change for the WebNN API 15:04:03 ... Chai will take on additional responsibilities at Microsoft and will step down from his editor role 15:04:07 RafaelCintron has joined #webmachinelearning 15:04:12 ... he will continue as a WG participant and as a fan of this WG 15:04:40 ... on behalf of the WG, I want to thank Chai for his transformative contributions to this WG as a co-editor of the WebNN API spec, including all the guidance and technical insight shared with the group 15:04:57 ... when I asked Chai to join as a co-editor in 2020 we were in the incubation phase 15:05:19 ... with Chai's sustained contributions over the last 4 years we evolved into a WG and the WebNN API advanced from an early proposal into a Candidate Recommendation, and today the work of this WG is widely recognized in the industry, quite a journey! 15:05:24 ... thanks Chai for all the work you've done for the WG! 15:06:21 chai: thank you Anssi! It has been an interesting journey for me, we started early 2020 and I was convinced that the web platform needs WebNN to reach native perf 15:06:35 ... this work has to happen, super excited where we are now in capable hands 15:07:39 ... WebNN remains a high-priority effort for Microsoft 15:07:49 Rachel has joined #webmachinelearning 15:07:58 Present+ Rachel_Yager 15:08:19 chai: thank you Anssi and Ningxin for inviting me early on into this effort 15:08:50 Thanks so much Chai! 15:09:08 ... to fill in the vacant role, I'm pleased to appoint Dwayne Robinson as a new co-editor, joining Ningxin 15:09:22 ... Dwayne is fully dedicated to help WebNN API spec advance and brings in a wealth of relevant experience, he's a brilliant engineer 15:09:29 ... Dwayne has my full support and I'm committed to ensure this transition is as frictionless as possible 15:09:34 ... thank you Dwayne for taking on this important role 15:09:42 ... I will push a PR to acknowledge this editor change, config our repo, this change is effective immediately 15:10:07 Thank you Dwayne! 15:10:26 https://github.com/webmachinelearning/webnn/pull/561 15:10:27 https://github.com/webmachinelearning/webnn/pull/561 -> Pull Request 561 Editor change (by anssiko) 15:11:37 Subtopic: Welcome new participants 15:11:42 anssik: I have also another announcement to make 15:11:50 ... our WG continues to grow, and I'm pleased to welcome our latest new participants 15:12:06 ... Mike Wyrzykowski, Anne van Kesteren, Benjamin Poulain, Theresa O'Connor, Cameron McCormack, Marcos Caceres, from Apple joined the WG this week, welcome all! 15:12:25 ... I'm also pleased to welcome Geoff Gustafson from Intel to the WG 15:12:30 ... also joining us as a guest today is Michael McCool from Intel 15:12:38 ... Geoff and Michael are currently exploring hybrid AI space, related use cases and open issues 15:12:50 ... I will invite them to share their hybrid AI exploration findings in our future meeting to allow WG participants provide guidance on future directions to explore beneficial to this WG 15:13:20 q? 15:13:44 Topic: WebNN API delta wide review and CR Snapshot readiness 15:13:51 anssik: In short, we're on track. Thank you all. 15:14:01 ... all delta wide review requests are in flight: Architecture/TAG, Privacy, a11y, i18n, Security, no concerns heard 15:14:10 ... we plan to publish a new CR Snapshot next month. 15:14:24 ... #240 tracks our progress toward CR readiness and a high-level status is staged in #532, we'll merge this PR when we branch for release 15:14:24 https://github.com/webmachinelearning/webnn/pull/532 -> Pull Request 532 Update Status of this document for CR Snapshot (by anssiko) 15:14:25 https://github.com/webmachinelearning/webnn/issues/240 -> Issue 240 Candidate Recommendation readiness tracker (by anssiko) [process] 15:14:28 q? 15:15:08 zkis has joined #webmachinelearning 15:15:17 present+ Zoltan_Kis 15:15:25 dom: CR transition approval typically takes a week, couple of days to make sure the document is ready to be published 15:16:12 Topic: Wording changes debrief 15:16:20 anssik: jsbell has submitted a number of "Wording change" PRs (aka editorial change), I've helped merge some to take some workload off the editors 15:16:24 ... I asked jsbell to provide a quick overview to make it easier for everyone to catch up with these changes. 15:16:44 Subtopic: Parameter Types 15:17:06 jsbell: - Drop arg type assertions for methods defined by WebIDL 15:17:15 ... - Convert internal algorithm type assertions into parameter types 15:17:34 ... - Add types to all internal algorithm members 15:17:38 Subtopic: Simplification 15:17:44 jsbell: - Bundle validation (power pref, device type, etc) right into context creation, rather than making it a separate step. 15:18:00 ... - The spec had WebIDL-specified enums for device type and power preference as well as internal enums for the same thing, with usage that required these were identical. Dropped the internal-only versions and just use the WebIDL-specified types. 15:18:25 ... - For dictionaries, Bikeshed automatically annotates members with types and defaults, so repeating that in the prose is extra work and extra noise. Also simplified the text by removing "Specifies" where it didn't add anything, and refer to lists rather than sequences in prose. 15:18:33 Subtopic: Fixes 15:19:16 jsbell: - There were a few places that linked to "object" which Bikeshed matched to the FileAPI's blob constructor, which is wrong. To fix this, introduced new definitions for "platform operator" and "platform operand" which also reduce the number of "implementation-defined" references in the spec. 15:19:31 ... We're still linking to "number" wrong but that's in reshape(), which has some issues. 15:19:57 Thanks much Josh, these PRs are very helpful! 15:20:37 q? 15:20:45 yep! 15:21:02 Topic: Issue prioritization (cont'd) 15:21:16 anssik: With our WG growing we have new participants who are looking for guidance on how to best focus their contributions 15:21:38 ... to help with that, we initiated an issue prioritization exercise, started document triage guidance and started labelling issues to test drive this work mode 15:21:46 ... much thanks to jsbell for helping drive this effort! 15:22:08 ... Josh joined me in the Triage team, others interested please get in touch with me. 15:22:22 ... Josh, please introduce the triage guidance proposal and outline the outstanding questions for discussion 15:22:26 https://github.com/webmachinelearning/webnn/pull/533 15:22:27 https://github.com/webmachinelearning/webnn/pull/533 -> Pull Request 533 Process: Add documentation for labels, current and proposed (by inexorabletash) 15:22:27 Subtopic: Triage guidance for WebNN 15:22:31 #533 15:22:31 https://github.com/webmachinelearning/webnn/pull/533 -> Pull Request 533 Process: Add documentation for labels, current and proposed (by inexorabletash) 15:22:35 -> Current labels https://github.com/webmachinelearning/webnn/labels 15:22:53 jsbell: Apologies that last telecon I was rushed due to a power outage 15:23:35 ... My goal is to make the next steps for the API proposal overall clearer - separate the important high level issues - like bounds on interop, or the minimal operator set, etc - from stylistic issues, or issues about specific operators. 15:24:38 ... Some issues contain proposals and are just waiting for a PR, some require more discussion. It's hard to tell how far we are from being complete at a spec level, and where to focus attention. 15:24:49 ... PR #533 has documentation for labels and proposals for a few changes. Anssi has implemented several of the proposals, which include: 15:24:57 ... Added "feature request" 15:24:57 ... Renamed to "opset" 15:24:57 ... Renamed to "conventions" 15:24:57 ... Dropped unused labels ("dependencies", "tag") 15:24:57 ... Consensus on using "bug" for "the spec has a flaw that must be fixed" 15:25:02 Subtopic: Outstanding questions 15:25:12 jsbell: Proposed removals 15:25:16 ... - "enhancement" as ambiguous - unclear if this is a feature request or editorial improvement. 15:25:21 ... - "help wanted" unclear who can help - is this an outstanding question, PR needed, etc. 15:25:25 ... Proposed additions 15:25:31 q+ 15:25:32 ... - "op specific" for issues scoped to a single op (or closely related ops) - about 37 issues, listed in PR 15:25:37 ... - Common from other specs: "has PR", "needs PR", "has WPT", "needs WPT" - signal next steps 15:25:41 q? 15:25:46 ... - Do we like "v2" or should we consider Milestones? Probably defer this, wait for more experience/feedback. 15:25:52 ... - Do we want markdown files in the top level directory, or a docs/ directory, or …? 15:26:14 RafaelCintron: support milestones 15:26:44 Dwayne: granularity of the milestones? 15:26:54 jsbell: up for discussion 15:27:10 ... v2 is usually "future work" 15:27:17 ... "before the next CR" 15:27:37 Reilly: should discuss what we'd consider features that should be in the first release in browser engines 15:27:49 ... we have folks from multiple browser vendors in the WG now 15:27:54 q? 15:27:58 ack RafaelCintron 15:28:36 PR #533 15:28:36 https://github.com/webmachinelearning/webnn/pull/533 -> Pull Request 533 Process: Add documentation for labels, current and proposed (by inexorabletash) 15:29:03 docs/ dir sgtm 15:29:19 jsbell: indeed, PTAL PR #533 15:29:34 Topic: New features 15:29:39 Subtopic: MLBuffer 15:29:45 anssik: first this is a big and complex topic 15:29:53 ... thanks Bryan, Austin and others for all the exploration and documentation, prototyping 15:30:09 ... we have documented the following spec explorations: 15:30:12 ... MLBuffer proposal #482 broken into: 15:30:12 https://github.com/webmachinelearning/webnn/issues/482 -> Issue 482 Support for device-based tensor storage objects (by bbernhar) [webgpu interop] 15:30:17 ... - Creation and representing MLBuffer on XPU devices #542 15:30:17 https://github.com/webmachinelearning/webnn/issues/542 -> Issue 542 [MLBuffer] Creation and representing MLBuffer on a XPU devices (by bbernhar) [webgpu interop] 15:30:20 ... - Uploading/downloading tensor data #543 15:30:20 https://github.com/webmachinelearning/webnn/issues/543 -> Issue 543 [MLBuffer] Uploading/downloading tensor data (by bbernhar) [webgpu interop] 15:30:23 ... - Support for MLBuffer in graph execution #544 15:30:24 https://github.com/webmachinelearning/webnn/issues/544 -> Issue 544 [MLBuffer] Support for MLBuffer in graph execution (by bbernhar) [webgpu interop] 15:30:30 anssik: MLBuffer exploration #541 15:30:31 https://github.com/webmachinelearning/webnn/pull/541 -> Pull Request 541 Add MLBuffer exploration doc (by a-sully) [webgpu interop] 15:30:38 -> MLBuffer exploration (HTML preview) https://github.com/a-sully/webnn/blob/mlbuffer-exploration/mlbuffer-exploration.md 15:30:44 anssik: MLBuffer exploration discusses: 15:31:24 ... 1) Goals 15:31:24 ... 2) Overarching Questions 15:31:24 ... 3) Use Case: Chained Inference 15:31:24 ... - MLBuffer creation 15:31:24 ... - Writing to an MLBuffer 15:31:26 ... - Execute an MLGraph 15:31:30 ... - Read back data from an MLBuffer 15:31:50 Present+ Austin_Sullivan 15:31:58 ... 4) Use Case: WebGPU Interop 15:32:03 ... - Rent out an MLBuffer to WebGPU 15:32:06 ... - Return a rented-out MLBuffer back to WebNN 15:32:07 +1 to merge it into explainer 15:32:11 ... also the Chromium prototype has a lot of informative discussion 15:32:15 -> Chromium implementation https://chromium-review.googlesource.com/c/chromium/src/+/5173676 15:32:26 anssik: where would you like to focus today Bryan, Austin? 15:32:46 q? 15:33:05 Bryan: overarching questions are important, need to be discussed 15:33:46 #542 15:33:47 https://github.com/webmachinelearning/webnn/issues/542 -> Issue 542 [MLBuffer] Creation and representing MLBuffer on a XPU devices (by bbernhar) [webgpu interop] 15:35:20 Bryan: is the goal of WebNN to make it look like WebGPU but works on more than GPU backend, e.g. NPU backend 15:36:29 Reilly: I would focus on the use cases, we have an understanding there are places where apps using WebNN want to have interop between work they do in WebNN and in WebGPU, e.g. Zoom web client 15:36:58 ... semantic segmentation use case running on an NPU, pass result to GPU, render to a display with GPU 15:37:25 ... desire for anything with image processing and using features that cannot be implemented with WebNN to pass data back and forth between WebNN and WebGPU 15:37:41 ... developers passing data between WebNN and Wasm similarly, another similar use case 15:37:43 q+ 15:38:10 Reilly: driven by those use cases, we can should have interop for MLBuffer for those APIs 15:38:35 ... re whether MLBuffer should be a unifying interface for all these backends, internal to WebNN 15:38:35 q+ 15:38:50 ... MLBuffer should work with NPU, should work with all the contexts and configs 15:38:51 q? 15:38:58 ack zkis 15:39:31 zkis: I have question, what is the main thing in MLBuffer, identify buffers you can map between backends to avoid data copies? 15:39:45 ... or enable hybrid execution across XPUs? 15:40:10 ... another question, do we design the API so it is a generic design that can adapt to future use cases such as possible hybrid execution 15:40:10 q? 15:40:23 Reilly: the use cases are very important 15:40:53 q? 15:40:58 ack RafaelCintron 15:41:25 RafaelCintron: to answer Bryan's questions, MLBuffer to work with GPUBuffer, it should work, makes it straightforward to know what to do 15:42:17 ... for use cases, key are 1. WebGPU interop both ways to and from WebNN starting 15:42:20 q+ 15:42:48 ... 2. chained inference 15:43:29 ack Ningxin_Hu 15:43:50 Ningxin_Hu: for CPU I support device agnostic API 15:44:13 ... CPU has optimized mem layout, e.g. SIMD, requires tensor mem layout 15:44:35 ... in chained inference use case we can pass previous inference output and pass it as an input to the next inference 15:44:53 ... no mem relayout between the inferences 15:45:25 ... for use cases I'm interested in chained inference, because perf overhead especially in transformers models, e.g. T5 and SD models 15:45:40 q+ 15:45:49 ... many iterations to run one graph to complete one inference, needs several iterations to go through one sequence 15:46:26 ... with WebGPU and GPUBuffer data transfer with good performance, 2x, with read back to CPU with T5 model through ONNX RT I/O banding 15:47:04 ... noticed an open issue #559 to ask for control flow ops, similar use case 15:47:05 https://github.com/webmachinelearning/webnn/issues/559 -> Issue 559 Control flow operations: if, while (by philloooo) [opset] [feature request] 15:47:27 ... before adding while op, if we have MLBuffer we can use JS-control flow and fulfill this requirement 15:47:28 q? 15:47:31 ack chai 15:48:00 chai: MLBuffer's role in GPU interop space, the question whether it should be explicitly controlled by WebNN or managed by WebNN implementation 15:48:08 ... thinking of NPU it is different from GPU 15:48:23 ... NPU is an accelerator, does not have breadth of support unlike GPU that is general purpose 15:48:56 ... because if this there needs to be a fallback that happens within the graph, it is not a wholesale change from one processor to the next but similar to how CPU and GPU interaction works 15:49:15 ... fallback needs graph resource and workload scheduling done in the OS itself to be super efficient 15:49:54 ... my view is the surfacing of NPU capability may not be in full control of the backend but hybrid execution between NPU and GPU with backend controlling scheduling 15:50:06 ... this is my hunch we need this design for efficiency 15:50:10 q? 15:51:28 Bryan: what makes sense in WebGPU may not make sense in WebNN, thinking about this 15:51:52 ... we have map buffers, CPU accessible GPU buffer, do we need similar mapping for WebNN and WebGPU? 15:52:04 ... staging memory does it make sense for WebNN? 15:52:21 ... do we throw the unified I/O path out of the window? 15:52:40 ... inspiration from ONNX RT for I/O bindings 15:52:45 q+ 15:52:53 ... big questions, does it have to be like WebGPU? 15:52:53 q? 15:52:55 ack asully 15:53:09 asully: I've been advocating WebGPU-style 15:53:22 ... no sync readbacks like in WebGL 15:53:43 ... if MLBuffer is not CPU not necessity for asyncronizity 15:54:03 ... what if the buffer is on NPU or opaque? 15:54:04 q+ 15:54:17 q+ 15:54:30 ... the most flexible approach I think is to have a WebGPU-like approach with everything being mapping nothing can be done in sync from content timeline 15:54:31 q? 15:54:42 ack reillyg 15:55:10 Reilly: I said this should be like WebGPU and that was mainly in the context of if we expose similar semantics it should be like WebGPU 15:55:57 ... if we don't support mapping we can be different from WebGPU 15:56:00 q? 15:56:26 Bryan: in WebGPU you don't map a buffer, you tell GPU to copy it over, WebNN has to spec a copy API to be similar 15:56:48 ... or overload dispatch, or map a buffer 15:56:55 ... I need machinery to do the mapping 15:56:55 q? 15:57:10 #542 15:57:11 https://github.com/webmachinelearning/webnn/issues/542 -> Issue 542 [MLBuffer] Creation and representing MLBuffer on a XPU devices (by bbernhar) [webgpu interop] 15:57:14 ack RafaelCintron 15:57:35 RafaelCintron: WebGPU and buffer mapping took long to design! 15:57:46 ... don't feel bad if it takes long for you too 15:59:08 ... vendors with GPUs with unified memory would like to enhance the buffer in WebGPU 15:59:31 ... MLBuffer should work with a variety of devices 15:59:34 q? 16:00:37 #541 16:00:37 https://github.com/webmachinelearning/webnn/pull/541 -> Pull Request 541 Add MLBuffer exploration doc (by a-sully) [webgpu interop] 16:01:16 q? 16:02:00 RRSAgent, draft minutes 16:02:01 I have made the request to generate https://www.w3.org/2024/02/08-webmachinelearning-minutes.html anssik 16:02:09 Thanks Anssi! 16:02:34 anssik: Happy Chinese New Year Ningxin! 16:02:36 RRSAgent, draft minutes 16:02:37 I have made the request to generate https://www.w3.org/2024/02/08-webmachinelearning-minutes.html anssik 16:11:58 AramZS has joined #webmachinelearning 16:14:28 s/... to fill/anssik: to fill 16:14:32 RRSAgent, draft minutes 16:14:33 I have made the request to generate https://www.w3.org/2024/02/08-webmachinelearning-minutes.html anssik 17:48:05 s/can should/should 17:51:04 s/banding/binding 17:53:32 RRSAgent, draft minutes 17:53:33 I have made the request to generate https://www.w3.org/2024/02/08-webmachinelearning-minutes.html anssik 18:02:14 Zakim has left #webmachinelearning