Meeting minutes
anssik: Welcome back after the summer break!
… and thanks for nudging things forward over the course of July
TPAC 2024 registration and WebML WG F2F agenda building
anssik: As announced in June, W3C TPAC 2024 takes place in Anaheim, CA, USA at the Hilton Anaheim on 23–27 September 2024.
… And our group WebML WG will meet F2F on Monday, 23 September 2024, 09:00–18:00 PDT.
… that's the first day of the TPAC meetings week that runs until Friday
… I have two asks for you all regarding TPAC and WebML WG F2F
Registration
anssik: 1) Complete the registration form:
https://
anssik: the sooner the better to help organizers figure our space requirements and onsite/remote split for logistics
… also please note W3C has made fee waivers available to remove any barriers to participation
https://
anssik: any questions on registration?
F2F agenda
anssik: 2) Review the in development F2F agenda and provide your suggestions in the meetings GH issue:
webmachinelearning/
<gb> Issue 25 WebML WG - TPAC 2024 agenda (by anssiko)
anssik: we've already received great suggestions and proposals from both WG participants and folks outside the WG
… I'm sure we'll get those last-minute ideas as usual, but thinking ahead helps especially folks outside out group who may want to join as observers to get an idea what we'll cover, also help slot agenda to allow smoother remote participation experience
… let me first outline what has been proposed and then open the floor for discussion on further proposals
… - proposal from Domenic to discuss built-in APIs for translation and prompting, from the Chrome team
… I'd support inviting Domenic to join our F2F meeting to introduce these proposals to have a discussion on future direction, I think there's a lot of good cross-pollination of ideas to happen and these APIs could also get into this WG in the future
anssik: any Googlers in WG aware of this work?
JoshuaB: aware of this work
… - JoshB proposed an extensive triage pass through open issues at F2F, weed out breaking changes, priorities, next steps for the issues -- a lot of work but happy to dedicate F2F time for this, identifying issues that benefit from real-time discussion as a pre-work
jsbell: would love to hear from others on this proposal, I know some other WGs do this, important to timebox this exercise, maybe an hour
anssik: quantization and dequantization (QDQ) related issues: encapsulate vs dedicated QDQ ops, quant data types int4? int2?
… a good topic for F2F too
jsbell: I think some prework has been done, Dwayne looked at quantized linear and delinear
… not sure if these two ops are enough to support important models
DwayneR: those two ops are good first step
… have done analysis between different backends not looked at these two ops yet
ningxin: we're also examining ONNX models to understand the requirements from frameworks perspetive
webmachinelearning/
<gb> Issue 25 WebML WG - TPAC 2024 agenda (by anssiko)
anssik: jsbell, your thoughts on v1?
jsbell: not a big fan of versioning!
… we want to see an Origin Trial milestone and this API to be good to be used with real users
… QDQ are needed for that milestone
anssik: - W3C “living standards” topic
… we're currently periodically publishing CR Snapshots to secure RF commitments and as such we are basically already following the "living standards" path and continue to perpetually maintain the spec
… happy to discuss this at F2F as well, also want to integrated W3C staff view from Dom
jsbell: agree with Anssi's formulation
Dwayne: no immediate feedback, will look at our open issues for proposals
ningxin: if we can have participants from implementers role that'd be good to discuss interop issues across different backends
… also the next step for implementation, Origin Trial or equivalent and align with framework developer feedback
MikeW: I will be at TPAC
anssik: great to hear
MikeW: something along the lines of MLBuffer and WebGPU interop would be an interesting F2F topic
anssik: one potential topic could be platform capability detection (a la #463)
<gb> Issue 463 Allow checking whether operators/types are supported for a backend before creating a graph (by huningxin) [feature request]
anssik: it looks like some sort of platform capability detection is required by frameworks that sit atop WebNN -- historically capability detection has caused some unrest on the web platform for privacy and future-proofing reasons, but we seem to have good use cases at hand and I'm confident can work out any issues
<ningxin> +1 to discuss opSupportLimits
anssik: another proposal: customer-centric session
… that is, I'd like to make sure this WG listens to its customers carefully i.e. frameworks that integrate with WebNN API
… I'd be happy to invite any such folks to present similarly to what JoshuaL did with Transformers.js in June
… who from ONNX Runtime Web team be the best person to reach out to for this opportunity? Other frameworks of interest?
RafaelCintron: there's parallel workstream on WebNN EP we do receive feedback from ONNX RT team, not sure they will be going to TPAC
… could invite them to call in if they're not joining F2F
DwayneR: Wanming has been adding kernels and operators to ONNX RT
ningxin: we can discuss with folks and have a plan
… and perhaps we can get JoshuaL to re-run his great Transformers.js presentation for new folks at F2F?
Joshua_Lochner: I'm mostly traveling to California soon so may overlap
… I will be there for PyTorch conf 18-19 September 2024
… - "ML ethics for engineers" :-)
ningxin: for TPAC plans, checking for incubation for customer ops, is it more appropriate for breakout day?
<jsbell> +1; discussing incubation in WG session is fine. Breakout casts a wide net, but you are competing with all the other Breakouts
anssik: I can review your proposal to advice
<McCool> breakout session on AI Model Management w3c/
<gb> Issue 15 AI Model Management (by mmccool) [session]
McCool: want to propose model management topic
… - "ML ethics for engineers" redux :-)
<ningxin> incubation for custom ops
anssik: I'd also like to reinvigorate work around ML ethics with a pragmatic approach
… as you may recall, we have produced a doc that discuss ethical issues associated with using ML
Ethical Principles for Web Machine Learning
anssik: since we're largely engineers in this group, I was thinking of inviting folks who work on tools in this space to make this ethics topic more approachable for us, so we can see running code to help solve some of these issues
… I was planning to invite folks from Georgia Tech working on WebSHAP, a JS library that can explain any machine learning models on the Web, running completely in the browser using ORT
anssik: this tool helps with "transparency and explainability" ethical principles we identified in our ethical principles doc
Transformers.js update takeaways
anssik: We received an exciting and action-packed Transformers.js update presentation from Joshua Lochner on our June call. As a refresher, please check the animated presentation:
Transformers.js update
… as discussed, Transformers.js informed greatly the WebNN expansion into transformers
… to continue this productive collaboration, I wanted to revisit the feedback from June discussion to ensure it is reflected in our WebNN work, issues and priorities appropriately.
June minutes with Q&A at the end
anssik: our June Q&A touched on:
… - ONNX Runtime Web dependency: WebNN EP backend, implementation optimization
… - dynamic shapes benefits for vision models, WebNN now static shape
… - perf improvements WebNN vs Wasm/WebGPU
… wanted to check if folks have additional topics to record?
Joshua_Lochner: I'm today releasing Transformers.js v3 NPM bold
… that will provide feedback directly on making sure WebNN support is working
… we had a call last week to discuss WebNN Transformers.js integration in more detail
… I can follow up on more concrete things we can discuss
… we'll add WebNN EP as an option
… differences e.g. static vs. dynamic shapes, config.json specifies that we can't use dynamic
… that's the first step format, four models can initially run on WebNN, vision models being the easiest because have config file with input dimensions
… all that will be done via ONNX Runtime Web and happy to follow up with more details
anssik: demo opportunity by TPAC to showcase Transformers.js WebNN backend?
Joshua_Lochner: certainly will be able to put out a demo
<Joshua_Lochner> would love to!
Device selection + quantization & dequantization
anssik: We agreed to evolve MLContextOptions and other API controls for device selection informed by further implementation experience and new use cases from the wider web community
… To assist in this effort, the "npu" device type landed in Chromium and samples for fp16 models with NPU device type have been made available for experimentation.
NPU + fp16 samples: image classification
NPU + fp16 samples: object detection
anssik: the current device selection mechanism is specified in:
anssik: we added an in-line block to MLContextOptions to set expectations that MLContextOptions is under active development, and the design is expected to change, informed by further implementation experience and new use cases from the wider web community
… I note feedback from MikeW / Apple in the related PR:
https://
<gb> MERGED Pull Request 696 Add MLDeviceType npu (by fdwr)
"it would be preferable to remove the `If this type cannot be satisfied, an "{{OperationError}}" {{DOMException}} is thrown`, wording as I'm not sure how that is compatible with the sentence preceding it `and allow the implementation to better select the most appropriate underlying execution device for the workload.`"
anssik: MikeW please open a separate issue for this
MikeW: will do
anssik: to avoid ossification of the current MLContextOptions beyond this developer preview phase, I want us to discuss future-proof device selection abstractions
… issue #623 mentions:
<gb> Issue 623 WebNN should support NPU and QDQ operations (by wchao1115) [v2] [opset] [feature request] [device selection]
anssik: - fallback device
… - multiple devices in a preferred order
… - an exclusion of a specific device
… - quantization & dequantization (QDQ) data types
… - error handling
… - ultimate fallback
… I believe it is beneficial to be use case-driven here, so any additional use cases for device selection welcome, currently documented are, to recap:
… - compute offloading
… - op fallback
anssik: potential TPAC discussion topic?
MikeW: I'll add more context in the issue, there's some pushback on specifying hardware device type, will provide feedback through issue
<anssik> s/McCool: I'd also like/anssik: I'd also like
<anssik> s/… and perhaps we/anssik: and perhaps we
<anssik> s/… - "ML ethics for engineers" :-)//