13:58:02 <RRSAgent> RRSAgent has joined #webmachinelearning
13:58:03 <RRSAgent> logging to https://www.w3.org/2022/09/22-webmachinelearning-irc
13:58:04 <Zakim> RRSAgent, make logs Public
13:58:05 <Zakim> please title this meeting ("meeting: ..."), anssik
13:58:40 <anssik> Meeting: WebML WG Teleconference – 22 September 2022
13:58:45 <anssik> Chair: Anssi
13:58:51 <anssik> Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2022-09-22-wg-agenda.md
13:58:58 <anssik> Scribe: Anssi
13:59:03 <anssik> scribeNick: anssik
13:59:09 <anssik> ghurlbot, this is webmachinelearning/webnn
13:59:09 <ghurlbot> anssik, OK
13:59:15 <anssik> Present+ Anssi_Kostiainen
13:59:50 <anssik> Regrets+ Dominique_Hazael-Massieux
14:00:08 <anssik> RRSAgent, draft minutes
14:00:08 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/22-webmachinelearning-minutes.html anssik
14:00:46 <bruce_dai> bruce_dai has joined #webmachinelearning
14:00:49 <ningxin_hu> ningxin_hu has joined #webmachinelearning
14:00:52 <anssik> Present+ Bruce_Dai
14:00:59 <anssik> Present+ Ningxin_Hu
14:01:36 <anssik> Present+ Chai_Chaoweeraprasit
14:03:44 <Jonathan> Jonathan has joined #webmachinelearning
14:03:56 <anssik> Present+ Jonathan_Bingham
14:04:56 <chai> chai has joined #webmachinelearning
14:05:36 <anssik> RRSAgent, draft minutes
14:05:37 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/22-webmachinelearning-minutes.html anssik
14:06:35 <anssik> ghurlbot, this is webmachinelearning/webnn
14:06:36 <ghurlbot> anssik, OK
14:07:36 <anssik> Present+ Rafael_Cintron
14:07:43 <anssik> Topic: WebNN API Candidate Recommendation open issues
14:08:11 <anssik> -> Current CR issues https://github.com/webmachinelearning/webnn/issues?q=is%3Aissue+is%3Aopen+label%3Acr
14:08:43 <anssik> anssik: Review and discuss the current CR issues, work out a plan to address the issues prior to the expected CR publication.
14:08:52 <anssik> Subtopic: Support asynchronous context creation
14:08:56 <anssik> #272
14:08:57 <ghurlbot> https://github.com/webmachinelearning/webnn/issues/272 -> Issue 272 Support asynchronous context creation (huningxin) cr
14:09:10 <anssik> anssik: Awaits W3C TAG recommendation on the design, no WG action required.
14:09:16 <anssik> -> Delta review (to CR) of Web Neural Network API https://github.com/w3ctag/design-reviews/issues/771
14:09:17 <RafaelCintron> RafaelCintron has joined #webmachinelearning
14:09:28 <anssik> anssik: I submitted a "delta" TAG review request per our resolution
14:09:35 <anssik> anssik: and asked TAG for a recommendation on three topics:
14:09:41 <anssik> ... 1) Naming of the sync and async methods
14:09:53 <anssik> ... 2) Sync/async API split design
14:10:18 <anssik> ... 3) Decision to Drop support for WebGL and focused on WebGPU interoperability
14:10:29 <chai> brb
14:10:39 <anssik> ... currently this TAG review issue is "untriaged" in the TAG tracker.
14:10:52 <anssik> ... please check that the request I authored is accurate and let me know any corrections.
14:11:05 <chai> back
14:11:14 <anssik> ... I asked TAG to deliver its recommendation by the end of Oct 2022 latest.
14:11:34 <anssik> q?
14:12:03 <ningxin_hu> it's accurate, thanks anssi
14:12:07 <anssik> Subtopic: Web platform tests
14:12:17 <anssik> -> Test plan https://github.com/webmachinelearning/webnn/issues/265#issuecomment-1246622380
14:12:49 <anssik> anssik: WebNN CR readiness depends on web-platform-tests test suite to demonstrate implementation experience
14:13:40 <anssik> anssik: Bruce published a test plan, this is a call for review
14:13:58 <anssik> ... I noted Chai is close to producing an initial list of recommended ULP tolerance for the ops
14:14:08 <anssik> ... any other contributions, please chime in on the issue.
14:14:11 <anssik> q?
14:14:43 <chai> q+
14:14:49 <anssik> ack chai
14:15:11 <anssik> chai: we are pretty close to producing a recommendation to fill in the blanks in Bruce's ULP table
14:15:24 <anssik> ... some work in complex math related ops
14:15:49 <anssik> ... ops such as conv and gemm require a ULP tolerance that is compexity of the environment
14:16:14 <anssik> ... for those ops, it's not a single number, but depends on the test case, the size of conv filter, for example, the tolerance is a function of the complexity
14:16:25 <anssik> ... we hope to stand the test of time principle
14:17:10 <anssik> ... maybe even after the initial work, there may be cases we need to tweak the tolerances to accommodate future
14:17:32 <anssik> ... tweaking too many times can be a bit messy, we're working on this to have it our end of this week early next week
14:18:21 <anssik> anssik: test suite maintenance is a positive thing
14:18:22 <anssik> q?
14:18:24 <ningxin_hu> q+
14:18:30 <anssik> ack ningxin_hu
14:19:07 <anssik> ningxin_hu: question regarding the ULP table, the comment mentions webnn-baseline to generate a reference
14:19:19 <anssik> ... I'm not sure that table captures the gap in webnn-baseline implementation
14:19:32 <anssik> ... there's a col on WTP and polyfill, no webnn-baseline status in there
14:19:39 <anssik> ... should we capture that in the table?
14:20:34 <chai> q+
14:20:55 <anssik> bruce_dai: webnn-baseline for 1st wave ops have been implemented, my plan for WPT is to cover these 1st wave ops first
14:21:06 <anssik> ningxin_hu: thanks for the update
14:21:19 <anssik> ... we should include webnn-baseline in the plan to help give an overview
14:21:30 <anssik> bruce_dai: I will update the table with a column for webnn-baseline
14:21:32 <anssik> q?
14:21:34 <anssik> ack chai
14:22:05 <anssik> chai: ningxin_hu's questions is important, tolerance is relative to the baseline, in our case when we have the tolerance out, it is the delta to the ideal baseline
14:22:19 <anssik> ... in DML, we implement the IDL baseline in double precision FP
14:22:30 <anssik> ... the diff between the baseline gives us a fixed point
14:22:55 <anssik> ... done with the tolerances, we're open to collaborate on the implementation of the baseline
14:23:13 <anssik> ... this is an important point, we should not use an arbitrary impl on some platform as the baseline
14:23:13 <anssik> q?
14:23:19 <ningxin_hu> +1
14:24:22 <anssik> anssik: bruce_dai clear on the next steps?
14:24:25 <anssik> bruce_dai: yes thanks
14:24:35 <anssik> Subtopic: Add method steps and normative algorithms to operations
14:24:42 <anssik> #210
14:24:43 <ghurlbot> https://github.com/webmachinelearning/webnn/issues/210 -> Issue 210 Add method steps to operations (anssiko) cr
14:24:46 <anssik> #211
14:24:47 <ghurlbot> https://github.com/webmachinelearning/webnn/issues/211 -> Issue 211 Define algorithms for dictionaries with lists as default values (anssiko)
14:26:05 <ningxin_hu> q+
14:26:09 <anssik> ack ningxin_hu
14:26:44 <anssik> ningxin_hu: I agree atomic PRs are preferred, so they will not conflict with others PRs
14:26:58 <anssik> ... a good start could be internal slots and MLOperand interface and MLOperator interface
14:27:22 <anssik> ... those internal slots we can define in the algorithm steps that describe how graph building happens
14:27:31 <anssik> ... this proposal is based on implementation experience
14:27:56 <anssik> ... we implement GraphBuilder in Chromium open source project and there we define internal slots, in implementation those are class members
14:28:21 <anssik> ... we can write code to use these private members and do the wiring
14:28:48 <anssik> anssik: starting with internal slot definitions sounds good
14:29:40 <anssik> q?
14:29:45 <ningxin_hu> q+
14:29:50 <anssik> ack ningxin_hu
14:30:51 <anssik> ningxin_hu: another suggestion, because we want to use algorithm steps to specify the operation, we have a declarative them, in this transition we add algorithm steps, probably we can keep the both for a while to help the reviewer compare the declarative to the algorithmic prose
14:31:07 <anssik> ... declarative should be ultimately replaced with algorithmic steps
14:32:29 <anssik> anssik: some specs mark declarative as informative and make algorithmic steps normative, we could consider the same
14:32:30 <anssik> q?
14:32:42 <anssik> q?
14:33:08 <anssik> Subtopic: Support for int8 quantized models
14:33:15 <anssik> #128
14:33:15 <ghurlbot> https://github.com/webmachinelearning/webnn/issues/128 -> Issue 128 WebNN should support int8 quantized models (wchao1115) cr
14:33:25 <anssik> anssik: I wanted to discuss the two design alternatives:
14:33:45 <anssik> ... 1) quantized ops supported by a new device type (e.g. NPU) OR
14:33:51 <anssik> ... 2) add support for all device types.
14:34:01 <anssik> q?
14:34:27 <anssik> ... Dom recommended last time: "it is good idea to evaluate whether our API can work with new device types such as NPUs, not sure if NPUs should be in CR release scope due to required implementation experience"
14:34:37 <anssik> ... this suggests this feature might be too early to commit for CR
14:34:54 <chai> q+
14:34:58 <anssik> q?
14:35:05 <anssik> ack chai
14:36:54 <anssik> chai: I think we should at least try this feature, it is true quantized ops are implementable on CPUs today, NPU device type is a new one, if we run out of time can drop
14:37:43 <anssik> anssik: can you help with the evaluation?
14:37:52 <anssik> chai: can help with quantized ops and NPU device type
14:37:55 <anssik> q?
14:38:44 <anssik> Jonathan: I need to follow up with folks internally at Google
14:39:18 <anssik> ... many teams at Google, some are interested in this feature and some are opposed
14:40:05 <anssik> ... I think the WebNN API is great, in particular WebGPU team has not been very much involved and their position is less clear, I will dig more into this
14:40:19 <anssik> q?
14:40:27 <anssik> RRSAgent, draft minutes
14:40:27 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/22-webmachinelearning-minutes.html anssik
14:40:33 <anssik> Topic: WebGPU Working Group review request and WebNN-WebGPU interop
14:41:14 <anssik> anssik: I wanted to discuss WG's response to the WebGPU review request, focus on WebNN-WebGPU interop requirements and issues.
14:41:18 <anssik> -> Review request https://lists.w3.org/Archives/Public/public-webmachinelearning-wg/2022Jul/0000.html
14:41:27 <anssik> ... review DL is 15 Oct, see our earlier discussion:
14:41:31 <anssik> -> 11 August 2022 discussion https://www.w3.org/2022/08/11-webmachinelearning-minutes.html#t08
14:41:51 <anssik> ... silence is considered consent
14:41:58 <Jonathan> about NPUs: some folks at Google had previously said that NPU support was the most compelling reason to create a new API like WebNN, and that CPU or GPU might not be sufficient given WASM+SIMD and WebGPU
14:42:08 <anssik> ... we agreed that our response should focus on WebNN-WebGPU interop issues
14:42:15 <anssik> #264
14:42:16 <ghurlbot> https://github.com/webmachinelearning/webnn/issues/264 -> Issue 264 CommandBuffer usage clarification: internal, external, both? (bbernhar)
14:42:34 <anssik> -> gpuweb/issues/2500 https://github.com/gpuweb/gpuweb/issues/2500
14:43:23 <anssik> ... Rafael, anything to report from discussion with Bryan on #264? Resolving that issue would help Ningxin deliver an update in https://github.com/gpuweb/gpuweb/issues/2500
14:43:35 <anssik> q?
14:44:06 <anssik> RafaelCintron: not much progress with Brian, last time WebGPU WG discussed this and are heads down with v1
14:44:16 <anssik> ... they said, any feedback, let us know
14:44:34 <anssik> ... the crux is, to implement WebNN, there needs to be a deep interaction between WebNN and WebGPU code
14:45:00 <anssik> ... to have the API take GPUBuffers but them into graph and work beautifully, these systems will need to work nicely together
14:45:53 <anssik> ... when we gain implementation experience we may need to change things, unclear how NPUs will work with GPUs, can we do transfers etc.
14:47:01 <anssik> ... the last time this came up in WebGPU WG they postponed discussion post-V1, IIRC
14:47:56 <anssik> s/Brian/Bryan
14:48:52 <anssik> anssik: when will WebGPU API ship in Chrome?
14:49:15 <anssik> RafaelCintron: no ship date, but "ASAP", new features are deferred to v.next now
14:49:42 <anssik> anssik: all of WebGPU WG in agreement with v1 feature scope?
14:50:04 <anssik> RafaelCintron: it seems so, it's a matter of how much to polish
14:50:55 <anssik> ... there are some things to resolve, but pressure to ship, issues with interop with other APIs
14:51:00 <ningxin_hu> q+
14:51:04 <anssik> ack ningxin_hu
14:51:44 <anssik> ningxin_hu: WebGPU interop is post-v1, our spec has CommandBuffer interface for this feature, not sure how this would impact the CR target for this year?
14:52:14 <anssik> ... should we make a branch without CommandBuffer interface to get to CR?
14:52:36 <anssik> RafaelCintron: we should get the CR scope implemented and see what issues emerge, those things should block CR
14:53:23 <anssik> q?
14:53:57 <anssik> ningxin_hu: my concern is this interop does not have consensus of the WebGPU WG that owns the WebGPU API
14:56:27 <anssik> ... for WebNN itself, we have control and can decide which features to add, this feature (CommandBuffer) has an external dependency
14:57:03 <anssik> ... if interop feature has a dependency but not resolved, what does this mean?
14:59:11 <anssik> anssik: how big an effort it would be to produce a WebNN-WebGPU interop implementation?
14:59:40 <anssik> ningxin_hu: we have prototyped this for real-time video processing use case
15:00:05 <anssik> ... we need WebNN and WebGPU implementation work closely, not one way dependency, but bi-directional interactions to ensure they work together
15:00:16 <anssik> ... I believe I shared that earlier
15:01:21 <anssik> ... related, WebGPU WG is also rechartering
15:01:26 <anssik> -> WIP PR for WebGPU WG charter 2022-2024 changes https://github.com/gpuweb/admin/pull/18
15:01:36 <anssik> ... we can influence this charter with our feedback e.g. we could be more explicit with our WebNN-WebGPU coordination expectations, currently reads
15:01:48 <anssik> Web Machine Learning Working Group
15:01:48 <anssik> 
15:01:48 <anssik> This Working Group develops a dedicated low-level Web API for enabling efficient machine learning inference in the browser.
15:02:41 <anssik> RRSAgent, draft minutes
15:02:41 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/22-webmachinelearning-minutes.html anssik
15:04:24 <anssik> q?
17:27:05 <Zakim> Zakim has left #webmachinelearning