13:54:36 RRSAgent has joined #webmachinelearning 13:54:40 logging to https://www.w3.org/2023/09/21-webmachinelearning-irc 13:54:40 RRSAgent, make logs Public 13:55:11 Meeting: WebML WG Teleconference – 21 September 2023 13:55:11 please title this meeting ("meeting: ..."), anssik 13:55:13 Chair: Anssi 13:55:13 Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2023-09-21-wg-agenda.md 13:55:13 Scribe: Anssi 13:55:13 scribeNick: anssik 13:55:13 gb, this is webmachinelearning/webnn 13:55:13 anssik, OK. 13:55:13 Present+ Anssi_Kostiainen 13:55:23 RRSAgent, draft minutes 13:55:24 I have made the request to generate https://www.w3.org/2023/09/21-webmachinelearning-minutes.html anssik 13:56:59 etienne has joined #webmachinelearning 13:58:42 Present+ Etienne_Noel 14:00:31 jsbell has joined #webmachinelearning 14:00:39 Regrets+ Dominique_Hazael-Massieux 14:00:45 Present+ Joshua_Bell 14:00:50 ningxin_hu has joined #webmachinelearning 14:00:53 Present+ Joshua_Lochner 14:01:12 Present+ Ningxin_Hu 14:03:10 Joshua_Lochner has joined #webmachinelearning 14:03:13 anssik: please welcome Etienne Noël from Google to the WG! SW Eng Manager at Google Chrome, has worked on Project Fugu among other things, also experience with Storage APIs, caching all the things, Pacific timezone 14:03:31 Present+ Chai_Chaoweeraprasit 14:03:40 Present+ Dwayne_Robinson 14:04:05 Present+ Rafael_Cintron 14:04:13 Topic: Update from TPAC 2023 14:04:42 RafaelCintron has joined #webmachinelearning 14:05:02 anssik: As discussed and agreed, the W3C's Web Machine Learning WG did not formally meet at this year's TPAC. However, the WG's work received a lot of interest during the hallway track and in other meetings. 14:05:10 ... AI/ML topics were discussed in the Advisory Committee meeting and in multiple breakout sessions: 14:05:22 -> AI @ W3C (AC meeting) https://www.w3.org/2023/Talks/TPAC/ac-activities-update/ 14:05:22 chai has joined #webmachinelearning 14:05:48 -> Ethical AI (breakout) https://www.w3.org/2023/09/TPAC/breakouts.html#b-9f1667d4-a23b-4893-80aa-06d5ecca8434 14:05:52 -> Generative AI (breakout) https://www.w3.org/2023/09/TPAC/breakouts.html#b-53380d18-eb3f-47f2-a6ac-435ce6294e0c 14:06:08 ... in the AC meeting, W3C's Dom gave an update on "AI @ W3C". 14:06:42 ... WebNN API and Ethical Principles work noted explicitly as key efforts in this space, also our latest work on transformer-based generative models and the broader impact of generative AI was discussed 14:06:53 ... generative AI discussion was centered around impacts: 14:07:03 ... - transforming content creation 14:07:09 ... - interaction with content and services 14:07:17 ... - new centralized risks 14:07:26 ... - web-scraped content re-use (for training) 14:07:37 ... questions for W3C roadmap that came up: 14:07:51 ... - marking content as AI generated 14:08:00 ... - marking content usable for training 14:08:14 ... - exposing browser-provided ML models through Web APIs 14:08:35 present+ 14:08:42 ... - large asset download/caching/sharing from browsers 14:08:56 ... - delegating partial model execution to the network 14:09:11 ... operational questions for W3C: 14:09:19 ... generative AI role in: 14:09:22 ... - spec writing 14:09:26 ... - test writing 14:09:30 ... - docs writing 14:10:01 anssik: in both the breakout sessions on Ethical AI and Generative AI the discussion was pretty much around ethical issues, and not so much on technical solutions 14:10:33 ... in a WG chair capacity I invited interested folks who were interested to join the WG and help contribute to the Ethical Principles guidelines 14:10:44 ... I shared that the WG is currently focused on delivering on its WebNN API v2 technical goals and we'd welcome new participants to help drive the ethical work forward. 14:11:17 ... that's my summary from TPAC relevant to this WG, questions? 14:11:59 q+ 14:12:11 ack RachelY 14:12:27 RachelY: it was great to see Anssi IRL 14:13:01 ... we had a breakout session on computational intelligence too, Joshua Lochner joined and university researchers 14:13:33 -> Unleashing the Power of Computational Intelligence on the Web https://www.w3.org/events/meetings/ab0de26f-c84d-40b6-80a7-79f3020da3aa/ 14:14:57 ... I'd like to ask this WG's participants to support the Community Group we're starting for Computational Intelligence 14:16:12 q? 14:16:25 Topic: WebNN v2: Proposed model targets 14:16:39 anssik: Over the last few months the WG has identified the following as its proposed v2 model targets: 14:16:44 ... - Text-to-image: Stable Diffusion unet/VAE/text encoder 14:16:55 ... - Image segmentation: Segment Everything decoder 14:16:59 ... - Speech-to-text: Whisper Tiny 14:17:09 ... - Text-to-text generation (encoder-decoder): t5 and m2m100 14:17:16 ... - Text-generation (decoder-only): llama 14:17:28 q+ 14:17:43 ... the participants have proposed these targets that are backed with implementation experience, from the WebNN Chromium DirectML prototype and Transformers.js 14:17:46 ack chai 14:18:17 chai: in the topic of transformer support, want to solicit feedback from the WG, I think this will be iterations 14:18:31 ... we can start with the models now proposed, SD, Segment Everything 14:18:41 ... we want to acknowledge adding this work will be incremental 14:18:44 q? 14:19:40 ... I think SD and Segment Everything are good start 14:20:05 ... adding these to the spec will help the browsers improve other browser infrastructure such as the caching system 14:20:19 ... SD 1.5 is a good starting point, not too big 14:21:07 ... there is also interest in running SD on NPU, NPU support is separate from transformers support, but they're actually almost intertwined 14:21:22 ... good opportunity to tack NPU support at the same time start small 14:21:49 q? 14:21:55 q+ 14:21:58 ack ningxin_hu 14:22:36 ningxin_hu: initial targets are good, with my collegue Wanming we started op breakdown work, want to provide that as input to the WG in the coming days 14:22:54 ... in this analysis we observed almost all these models in ONNX leverage dynamic shaping for input size 14:23:05 ... today WebNN supports only static size tensors 14:23:28 ... we looked at how WebNN implementation can handle this, leave dynamic shape to framework 14:23:43 ... let WebNN deal with static shapes, doing this investigation in parallel 14:23:58 ... we'll also provide that input, Jiawei also contributed to this work 14:24:19 @Rachel @Anssi Link to my TPAC presentation (I reused the template from my previous talk, and updated the content to include the other HF JavaScript libraries): https://docs.google.com/presentation/d/1hx-Y1HFw_F88FJiqZnJ3FYlRnj7HKKwuYkJIRL0TcU0/edit?usp=sharing 14:24:19 https://github.com/Rachel -> @Rachel 14:24:19 … https://github.com/Anssi -> @Anssi 14:24:34 q? 14:25:01 No concerns, just excited! 14:25:12 I submitted a proposal for a business group with several university research groups and companies joining as new members. You are invited to support the creation of this group: http://www.w3.org/community/groups/proposed#computationalibg I would appreciate your input on how this community group can contribute to new use cases and learn from the good work of the Web ML WG. 14:25:17 ... if there are no concerns with this set of model targets I'd ask the WG to initiate work on an op breakdown to better understand what is common across these architectures to inform WebNN API v2 priorities 14:25:27 ... this work is similar to what we did for the so-called first-wave models a few years back when "v1" API effort was kicked off 14:25:32 -> The first-wave models op breakdown https://github.com/webmachinelearning/webnn/blob/main/op_compatibility/first_wave_models.md 14:25:38 ... at that time we looked at the following models: 14:25:45 ... - Image classification: SqueezeNet, MobileNet, ResNet 14:25:51 ... - Object detection: TinyYOLO 14:25:55 ... - Noise suppression: RNNoise, NSNet 14:26:20 ... we don't need to copy this approach exactly, the important part is to conduct this work in public to allow the public to review and participate 14:26:24 ... issue #375 is the place for discussions, suggestions and feedback 14:26:25 https://github.com/webmachinelearning/webnn/issues/375 -> Issue 375 Support for transformers (by dontcallmedom) [v2] [operation set] 14:26:41 ... from that issue you'll also find Dwayne's initial proposal for v2 ops for the proposed text-to-image, image segmentation and speech-to-text models 14:26:57 ... comments? it seems we're ready to move forward with this plan? 14:27:13 RRSAgent, draft minutes 14:27:14 I have made the request to generate https://www.w3.org/2023/09/21-webmachinelearning-minutes.html anssik 14:27:21 q? 14:27:33 Topic: Storage APIs for caching large models 14:28:04 anssik: I'd like to have a discussion on issues related to caching large models in the browser, current Storage APIs, specific issues and workarounds for those APIs. 14:28:14 ... first, I'd like to set the expectation and then let Joshua share his insights on Storage APIs in general 14:28:22 ... to frame this discussion: 14:28:44 ... I'm not proposing we make any changes to the WebNN API to support caching, I'd like us to discuss issues related to current Storage APIs that could be submitted as feedback to the respective Storage API specs 14:28:59 ... Storage APIs (plural) in browsers mean many things: 14:29:22 ... for browsers we have: IndexedDB, localStorage, sessionStorage, Service Worker Cache API, applicationCache (deprecated, superseded by Service Worker Cache API) 14:29:39 q+ 14:29:40 ... there's also the Storage Standard that defines APIs for persistent storage and quota estimates across all these Storage APIs. 14:29:47 -> IndexedDB https://www.w3.org/TR/IndexedDB/ 14:29:50 -> localStorage https://html.spec.whatwg.org/multipage/webstorage.html#the-localstorage-attribute 14:29:53 -> sessionStorage https://html.spec.whatwg.org/multipage/webstorage.html#dom-sessionstorage 14:29:57 -> Service Worker Cache API https://w3c.github.io/ServiceWorker/#cache-interface 14:30:02 -> applicationCache (deprecated, background info) https://web.dev/appcache-removal/ 14:30:05 -> Storage Standard https://storage.spec.whatwg.org/ 14:30:27 ... I'd like to Joshua to give an intro to this topic and then we can discuss the pain points with current APIs for caching large models. 14:30:48 jsbell: some background info then handover to Etienne 14:31:11 ... first, we need to distinguish imperative from network caching 14:31:28 ... latter is not explicitly controlled, by hinted by HTTP Headers 14:31:41 ... network caching in confusing to web developers 14:31:57 ... x-site resources could be cached and it make loading pages faster 14:32:18 ... we've learned that is a privacy concern, so using separate network caches nowadays 14:32:50 ... today two top-level site a.com and b.net, even if the same resource is requested from CDN they receive separate resources 14:33:32 ... there has been research on whether we can maintain privacy properties with transparent caches 14:34:19 ... this WG wants to cache GBs, more pressing need to transparent caching or something that the browser caches and can be shared 14:34:38 ... this leads to sites using imperative APIs to ensure data is not downloaded more than once 14:34:56 ... a few new Storage APIs added include: File System API 14:35:11 ... there's also a similarly names older API, not to be confuesd 14:35:45 ... this new File System API aka Origin-Private File System (OPFS) ships in all browsers 14:36:00 ... POSIX file semantics, accessible synchronously from workers 14:36:14 ... works with Wasm, Emscipten native support 14:36:31 ... AppCache is gone as Anssi mentioned, we can forget it 14:36:58 ... SW Cache API is a replacement for AppCache, resource on any site pulled down, they're immutable resources, Cache API may be a good fit for those 14:37:17 ... API fully accessible from Worker and Window contexts, please use it also outside ServiceWorker context 14:37:32 ... IndexedDB has complexity on top of KVS 14:37:35 q? 14:37:40 ack jsbell 14:37:59 etienne: I've researched memory impact of downloading large file 14:38:21 ... tested Cache API, OPFS 14:38:59 q+ 14:39:14 ... downloading without hitting memory important, wanted to hear issues and what can we do to improve the existing APIs, do we need to inform developers better, do these APIs not meet the reqs of web developers? 14:39:20 ack Joshua_Lochner 14:39:59 Joshua_Lochner: great to meet you Etienne! I've created Transformers.js and for the most part it's been way simpler that people would expect to cache, I have >1GB model cached 14:40:16 q+ 14:40:48 ... released today is in-browser close of HF Chat UI that is 800M param model, using SW Cache API, Transformers.js only uses the Cache API 14:41:22 ... early creation of Transformers.js requested HF Hub and stored model downloaded with Cache API on the client 14:41:55 ... top-level requirements, share store, manage multigig models 14:42:13 ... key issue for me is x-site sharing of models 14:42:34 ... now have to download model A on sites 1, 2 and 3 and share across the sites. 14:42:50 ... workaround is to use a browser extension 14:43:57 ... I created a sample browser extension and had issues with IndexedDB, when set active tab permission in manifest.json, running IndexedDB in the context of the web page had issues 14:44:26 ... I serialized model data with base64 to work around this which of course increased the model size to work around the sharing limitation 14:44:50 ... File System API is also another alternative for sharing across domain, the problem is it requires a buy-in from the user 14:44:53 q+ 14:45:07 ... I want to make this completely invisible to the user where the model is stored and how it is stored 14:45:24 ... needing to ask the user where the model is shared in his system is a problem 14:45:25 q? 14:45:41 ack jsbell 14:46:11 jsbell: thanks for the good news using the Cache API just worked with large models 14:46:26 ... web developers don't understand always Cache API has improved over time so much 14:46:45 ... back in the day only localStorage and sessionStorage with 5MB was available to web developers 14:47:00 ... Chrome has improved quota and other browsers have followed 14:47:46 ... limited storage wrt native apps was a big gap, in Chrome we piloted an upper limit of sites able to use up to 80% of the available disk space, we clean up that pool 14:47:56 ... sites may compete for storage but that is not common 14:48:19 ... Chrome, Safari, Firefox, Edge all can store gigabyte models today 14:48:51 ... querying quota is also supported, Safari 17 supports the Storage API and its quota features 14:49:24 ... JoshuaL talked about browser extensions to mediate between sites, going beyond web proper, but we want to learn how web developers work around these issues 14:49:34 ... good way to prototype these things 14:49:50 ... Anssi mentioned at TPAC, what if browser shipped common models 14:50:20 ... interesting idea, wonderful exploration 14:51:18 ... re IndexedDB issues, each top-level site sees its own storage, extension uses the extension's domains and proxies the data from the extension to the individual pages 14:51:38 ... using base64'd strings to pass data shouldn't be needed 14:52:12 ... new File System API has access to local files user can interact with, allows image editor on the web 14:52:36 ... internals of this API and file handles have access to this OPFS, similar to IndexedDB, hidden from the user 14:53:06 ... the privacy FS is hidden from the user, partitioned by top-level site, does not help with x-site sharing of cached models 14:53:07 q? 14:53:13 ack chai 14:53:15 Thanks so much Josh B! 14:53:55 Joshua L. I definitely want to follow up with you to understand better why serializing was necessary, we can discuss offline 14:54:11 Sounds good! 14:54:11 chai: this reminds me of the earlier days of web browser that video file is too large to be cached, reminds me of the evolution of media, over time problems are mitigated by ecosystem, web sites and browsers, e.g. video streaming fixes the video download problem 14:54:38 ... LLM download issue may solve itself in creative ways in the future 14:54:39 q? 14:54:50 RRSAgent, draft minutes 14:54:51 I have made the request to generate https://www.w3.org/2023/09/21-webmachinelearning-minutes.html anssik 14:55:17 q? 14:55:23 q+ 14:55:43 q? 14:55:51 ack RafaelCintron 14:56:23 RafaelCintron: JoshuaB, what about the domain restrictions? Can JL do something to make users live easier? 14:56:40 jsbell: privacy protection is the most important 14:56:57 q? 14:57:29 Topic: Intel Innovation update 14:58:39 chai: Intel Innovation 23 was this week, I was invited to give a talk on WebNN, I'll share the presentation 14:58:42 ... a few important announcements, Gaudi datacenter, also AI chip in Intel MTL platform aka Intel Core Ultra 14:59:48 ... I sense people want to see models running on NPU now, when we talked about WebNN, we have three processor types, Wasm for CPU SIMD, WebGPU for GPU, WebNN as the standard solution to accelerate on GPU and NPU, this was well received 15:00:00 AramZS has joined #webmachinelearning 15:00:24 ... implementation work in Chromium on DirectML was also discussed 15:01:07 ... the web is coming up, AI on the web can be accelerated similarly to what happens outside the web 15:01:18 anssik: thanks for the update Chai! 15:01:57 The link to the deck is here: https://static.rainfocus.com/intel/innv2023/sess/1689805601141001Xyjy/supmat/DevelopingWebBasedAIApps_Final_1694887505835001jjpL.pdf 15:03:13 RRSAgent, draft minutes 15:03:14 I have made the request to generate https://www.w3.org/2023/09/21-webmachinelearning-minutes.html anssik 15:03:48 q? 15:52:39 s/Intel Innovation update/Update from Intel Innovation 15:52:41 RRSAgent, draft minutes 15:52:42 I have made the request to generate https://www.w3.org/2023/09/21-webmachinelearning-minutes.html anssik 16:24:14 AramZS has joined #webmachinelearning 17:24:12 Zakim has left #webmachinelearning