IRC log of webmachinelearning on 2020-12-10

Timestamps are in UTC.

14:59:35 [RRSAgent]
RRSAgent has joined #webmachinelearning
14:59:35 [RRSAgent]
logging to https://www.w3.org/2020/12/10-webmachinelearning-irc
14:59:37 [Zakim]
RRSAgent, make logs Public
14:59:39 [Zakim]
please title this meeting ("meeting: ..."), anssik
14:59:40 [anssik]
Meeting: WebML CG Teleconference – 10 December 2020
14:59:47 [anssik]
Chair: Anssi
14:59:58 [anssik]
RRSAgent, draft minutes v2
14:59:58 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/12/10-webmachinelearning-minutes.html anssik
15:00:05 [anssik]
Scribe: Anssi
15:00:10 [anssik]
scribeNick: anssik
15:00:23 [anssik]
Present+ Anssi_Kostiainen
15:00:27 [anssik]
Present+ Ningxin_Hu
15:00:31 [anssik]
Present+ Chai_Chaoweeraprasit
15:00:36 [anssik]
Present+ Rafael_Cintron
15:00:44 [anssik]
Present+ Dom
15:01:07 [anssik]
Present+ Ganesan_Ramalingam
15:01:21 [anssik]
Present+ Bruce_Dai
15:01:28 [Rama]
Rama has joined #webmachinelearning
15:01:40 [anssik]
Present+ Wen_He
15:01:56 [dom]
Scribe+
15:02:00 [anssik]
RRSAgent, draft minutes v2
15:02:00 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/12/10-webmachinelearning-minutes.html anssik
15:02:09 [anssik]
TOPIC: WG Charter feedback
15:02:30 [wenhe]
wenhe has joined #webmachinelearning
15:02:37 [Chai]
Chai has joined #webmachinelearning
15:02:41 [dom]
Anssi: the idea was to go through the feedback that's been provided for the charter proposal
15:02:45 [anssik]
https://github.com/w3c/machine-learning-charter/issues
15:02:47 [dom]
... thank you all for the active engagement in the repo
15:03:08 [anssik]
https://github.com/w3c/machine-learning-charter/issues/8
15:03:10 [RafaelCintron]
RafaelCintron has joined #webmachinelearning
15:03:16 [dom]
... first I would like to discuss one issue that seemed to be pretty core to the most of the discussions, #8, opened by Jonathan
15:03:29 [dom]
Present+ Oguz_Derin
15:03:44 [dom]
... Jonathan wanted to make sure the charter gives flexibility to accommodate potential future work
15:03:48 [anssik]
https://github.com/w3c/machine-learning-charter/pull/9
15:04:08 [dom]
... I submitted pull request #9 to address this desire with a list of 2 tentative specification
15:04:18 [dom]
... the first one is the model loader API for which we have an explainer
15:04:36 [dom]
... the second one is a proposal "WebNN API Lower Level Instruction Set"
15:04:48 [zkis]
zkis has joined #webmachinelearning
15:04:53 [anssik]
https://www.irccloud.com/pastebin/6T8DY8hm/
15:05:35 [dom]
""A lower-level instruction set for the Web Neural Network API defines a complementary set of primitives out of which higher-level operations defined by the Web Neural Network API can be constructed. This instruction set is vendor-neutral and is expected to draw inspiration from efforts such as XLA HLO, Tensor Compute Primitives, TensorFlow RISC, and Tensor Operator Set Architecture.""
15:06:02 [BruceDai]
BruceDai has joined #webmachinelearning
15:06:06 [dom]
Anssi: the discussion in the repo shows that Google has been working on these 4 ideas, but not shipping yet
15:06:17 [dom]
... they're basically sitting at a lower level than current WebNN
15:06:45 [dom]
... Based on the discussion on the repo, the pull request would address this issue, and the other issues that seem to mostly derive from this core issue
15:07:07 [dom]
... This seems acceptable to Google, and I would like to get confirmation this is acceptable to others
15:07:40 [dom]
... For clarity sake, these would be tenative specs, which the WG could but does't need to work on, and would be based on prior incubation in the CG
15:07:57 [anssik]
q?
15:08:05 [RafaelCintron]
q+
15:08:08 [dom]
... I'm particularly interested if anyone has concerns about this approach
15:08:10 [anssik]
ack RafaelCintron
15:08:54 [dom]
Rafael: rechartering requires going through legal review in MS, so I would request we keep the old charter
15:08:56 [dom]
q+
15:10:02 [anssik]
ScribeNick: anssik
15:10:28 [anssik]
dom: there are two charters, CG Charter and WG Charter proposal
15:10:44 [anssik]
... did you already get feedback on the WG Charter review?
15:11:08 [anssik]
RafaelCintron: I haven't reviewed the WG Charter yet
15:11:20 [anssik]
... are the CG and WG similar apart from OR #9 changes?
15:11:49 [anssik]
dom: IANAL, they should be similar, the WG is expected to be a strict subset of CG
15:12:07 [Chai]
q+
15:12:14 [dom]
q-
15:12:20 [anssik]
ack dom
15:12:23 [anssik]
ack Chai
15:12:40 [dom]
Chai: I have questions
15:13:02 [dom]
... reading this issue, it seems like the request would be extended to support low level instructions
15:13:22 [dom]
... there has been a lot of discussions on low vs high level already
15:13:46 [dom]
... and it has been pointed out that the "low level" set is not really lower level than what WebNN providers
15:14:48 [dom]
... I also don't understand the relation to subsetting WebNN and this low-vs-high level
15:15:06 [dom]
... I understand there was discussion on whether WebNN would be low level enough and whether a graph API is the right approach
15:15:23 [jonathan]
jonathan has joined #webmachinelearning
15:15:24 [dom]
... I'm not sure how this addresses that question (which I think has been otherwise addressed in the discussion)
15:16:01 [dom]
Present+ Jonathan_Bingham
15:16:31 [dom]
Anssi: Jonathan you had suggested shipping a subset of WebNN that would perform better - can you speak to that?
15:16:58 [anssik]
RRSAgent, draft minutes v2
15:16:58 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/12/10-webmachinelearning-minutes.html anssik
15:17:04 [dom]
Jonathan: 3 topics: whether the operations are low-level enough - I agree with Chair that low level operations can be addressed in a graph API, so that issue can be closed
15:17:37 [anssik]
https://github.com/w3c/machine-learning-charter/issues
15:17:53 [dom]
Anssi: so there is an issue that we can close?
15:18:00 [dom]
Jonathan: issue #2 can be closed
15:19:00 [dom]
... my question about the operation subset was in the context of discussing an alternative to a graph API: we could have a convolution 2D API on its own, independent of any graph or model - because it is a well-known expensive operation
15:19:25 [dom]
... The excitement has been mostly about model loader and graph API, but I was asking if we could do that?
15:19:37 [anssik]
q?
15:20:34 [anssik]
ScribeNick: dom
15:20:51 [Chai]
q+
15:21:01 [anssik]
dom: the WG could decide to work on a subset of the WebNN API and work on it as a separate work item
15:21:12 [anssik]
... given consensu
15:21:40 [dom]
anssi: indeed, a spec can be split into as many pieces as we feel is appropriate
15:21:55 [anssik]
RRSAgent, draft minutes v2
15:21:55 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/12/10-webmachinelearning-minutes.html anssik
15:21:58 [anssik]
q?
15:22:00 [anssik]
ack Chai
15:22:21 [dom]
chai: I want to point out that the way the WebNN is being specified right now is specified as a graph builder API
15:23:17 [dom]
... an immediately-executable stand-alone eager-mode operation would not be a strict subset of WebNN from that perspective
15:23:20 [anssik]
q?
15:23:27 [dom]
Jonathan: +1 on that description
15:24:05 [dom]
Anssi: so how strongly do you feel we need to put that in scope of the WG (assuming if it isn't)
15:24:52 [dom]
Jonathan: some of the feedback I've gotten is that 3 individual operations give you 90% of the perf, then it's worth considering such an approach
15:25:02 [dom]
... which we had discussed in the early days of the CG
15:25:35 [dom]
... this made me wonder how much room the charter leaves to us for this
15:26:15 [anssik]
q?
15:27:00 [anssik]
ScribeNick: anssik
15:27:01 [anssik]
dom: from formal perspective IANAL we could work on both the approaches in the WG, the scope is broad enough to accommodate that, different way to organize the work
15:28:05 [anssik]
... alternatively, if this is still a significant open question, maybe we need to be explore this in CG before starting the WG charter review?
15:29:03 [jonathan]
i'd rather not delay the working group for this :)
15:29:06 [dom]
anssi: practically, exploring this further would delay the start of the WG
15:29:40 [jonathan]
how about i file an issue in the CG and we can discuss in parallel?
15:30:42 [dom]
Anssi: the WG will be chartered for 2 years - so no matter what, we will recalibrate our scope in 2 years
15:30:56 [dom]
... that's another way to pace evolutions in our understanding of the space
15:31:25 [anssik]
https://github.com/w3c/machine-learning-charter/pull/9
15:31:27 [anssik]
q?
15:32:08 [dom]
Anssi: would like to see if we can close issues and get consensus on what the PR should contain
15:32:32 [dom]
... Chai made the point that "the lower level" proposal isn't so clearly lower level
15:32:44 [anssik]
https://github.com/w3c/machine-learning-charter/issues/7
15:33:07 [anssik]
TOSA comparison https://github.com/w3c/machine-learning-charter/issues/7#issuecomment-729487005
15:33:34 [dom]
chai: we haven't found that operations in TOSA were particularly lower level than WebNN
15:33:58 [dom]
... we use XLA / HLO quite a bit to ensure mappability with WebNN
15:34:18 [dom]
... some operators are graph/networks on their own, while others are really instruction sets
15:34:22 [dom]
... we cover both in WebNN
15:35:01 [anssik]
Present+ Mehmet_Oguz_Derin
15:35:03 [dom]
... the process we ensure this is by checking each time we specific a network-like operator, we show how it would decompose in lower-level operators that we define too
15:35:05 [anssik]
Present+ Zoltan_Kis
15:35:17 [anssik]
Present+ Jonathan_Bingham
15:35:24 [jonathan]
q+
15:35:41 [dom]
... as a result, this gives us confidence that we consciously design an API that includes low level operations
15:35:55 [anssik]
ack jonathan
15:36:44 [dom]
Jonathan: I agree with you that the approach seems to address the problem; the Android NN API in launching their 120 operations felt and feels that what they came up with is not low-level enough all what ML practionners want to do on Android
15:37:25 [dom]
... they're exploring a major re-architecuring rather than decomposing lower-level instructions, although I'm not sure why, but they seem convinced
15:37:52 [dom]
... for WebNN, since the group has been looking at all these alternative instructions set, this seems a pretty robust
15:37:58 [dom]
... #7 can be closed as well
15:38:18 [Chai]
afk (need to step out for a few mins)
15:38:21 [anssik]
https://github.com/w3c/machine-learning-charter/issues/6
15:38:21 [RafaelCintron]
q+
15:38:37 [anssik]
ScribeNick: anssik
15:40:53 [Chai]
(im back)
15:43:07 [anssik]
https://github.com/w3c/machine-learning-charter/issues/6
15:43:22 [anssik]
ScribeNick: anssik
15:43:27 [anssik]
https://github.com/w3c/machine-learning-charter/issues/6#issuecomment-736004531
15:43:35 [jonathan]
q+
15:43:47 [RafaelCintron]
q-
15:43:51 [anssik]
ack jonathan
15:44:35 [anssik]
jonathan: the question in issue #6, are we doing graph API because we think it's the best abstraction, or because it takes too long to do a model loader with an agreed upon model format
15:44:59 [anssik]
q?
15:45:17 [anssik]
q?
15:45:41 [anssik]
q?
15:46:08 [RafaelCintron]
q+
15:46:12 [anssik]
ack RafaelCintron
15:46:40 [anssik]
RafaelCintron: graph API is format agnostic, there are many such formats
15:46:59 [anssik]
... maybe graph is better for future scenarios such as training
15:47:12 [Chai]
q+
15:47:16 [anssik]
... also OK to do load model first if inferencing is our primary target
15:47:22 [anssik]
ack Chai
15:47:55 [anssik]
Chai: I wrote the explainer, so should explain :)
15:48:08 [anssik]
... re issue #6, do we believe the graph API is the right abstraction
15:48:36 [anssik]
... the motivation is to reflect what is already out there, we are not reinventing graph API for WebNN, we acknowledge everywhere you look is a graph API
15:48:51 [anssik]
... TensorFlow, ONNX, XLA, TOSA all are reflections of what's out there