14:59:35 RRSAgent has joined #webmachinelearning 14:59:35 logging to https://www.w3.org/2020/12/10-webmachinelearning-irc 14:59:37 RRSAgent, make logs Public 14:59:39 please title this meeting ("meeting: ..."), anssik 14:59:40 Meeting: WebML CG Teleconference – 10 December 2020 14:59:47 Chair: Anssi 14:59:58 RRSAgent, draft minutes v2 14:59:58 I have made the request to generate https://www.w3.org/2020/12/10-webmachinelearning-minutes.html anssik 15:00:05 Scribe: Anssi 15:00:10 scribeNick: anssik 15:00:23 Present+ Anssi_Kostiainen 15:00:27 Present+ Ningxin_Hu 15:00:31 Present+ Chai_Chaoweeraprasit 15:00:36 Present+ Rafael_Cintron 15:00:44 Present+ Dom 15:01:07 Present+ Ganesan_Ramalingam 15:01:21 Present+ Bruce_Dai 15:01:28 Rama has joined #webmachinelearning 15:01:40 Present+ Wen_He 15:01:56 Scribe+ 15:02:00 RRSAgent, draft minutes v2 15:02:00 I have made the request to generate https://www.w3.org/2020/12/10-webmachinelearning-minutes.html anssik 15:02:09 TOPIC: WG Charter feedback 15:02:30 wenhe has joined #webmachinelearning 15:02:37 Chai has joined #webmachinelearning 15:02:41 Anssi: the idea was to go through the feedback that's been provided for the charter proposal 15:02:45 https://github.com/w3c/machine-learning-charter/issues 15:02:47 ... thank you all for the active engagement in the repo 15:03:08 https://github.com/w3c/machine-learning-charter/issues/8 15:03:10 RafaelCintron has joined #webmachinelearning 15:03:16 ... first I would like to discuss one issue that seemed to be pretty core to the most of the discussions, #8, opened by Jonathan 15:03:29 Present+ Oguz_Derin 15:03:44 ... Jonathan wanted to make sure the charter gives flexibility to accommodate potential future work 15:03:48 https://github.com/w3c/machine-learning-charter/pull/9 15:04:08 ... I submitted pull request #9 to address this desire with a list of 2 tentative specification 15:04:18 ... the first one is the model loader API for which we have an explainer 15:04:36 ... the second one is a proposal "WebNN API Lower Level Instruction Set" 15:04:48 zkis has joined #webmachinelearning 15:04:53 https://www.irccloud.com/pastebin/6T8DY8hm/ 15:05:35 ""A lower-level instruction set for the Web Neural Network API defines a complementary set of primitives out of which higher-level operations defined by the Web Neural Network API can be constructed. This instruction set is vendor-neutral and is expected to draw inspiration from efforts such as XLA HLO, Tensor Compute Primitives, TensorFlow RISC, and Tensor Operator Set Architecture."" 15:06:02 BruceDai has joined #webmachinelearning 15:06:06 Anssi: the discussion in the repo shows that Google has been working on these 4 ideas, but not shipping yet 15:06:17 ... they're basically sitting at a lower level than current WebNN 15:06:45 ... Based on the discussion on the repo, the pull request would address this issue, and the other issues that seem to mostly derive from this core issue 15:07:07 ... This seems acceptable to Google, and I would like to get confirmation this is acceptable to others 15:07:40 ... For clarity sake, these would be tenative specs, which the WG could but does't need to work on, and would be based on prior incubation in the CG 15:07:57 q? 15:08:05 q+ 15:08:08 ... I'm particularly interested if anyone has concerns about this approach 15:08:10 ack RafaelCintron 15:08:54 Rafael: rechartering requires going through legal review in MS, so I would request we keep the old charter 15:08:56 q+ 15:10:02 ScribeNick: anssik 15:10:28 dom: there are two charters, CG Charter and WG Charter proposal 15:10:44 ... did you already get feedback on the WG Charter review? 15:11:08 RafaelCintron: I haven't reviewed the WG Charter yet 15:11:20 ... are the CG and WG similar apart from OR #9 changes? 15:11:49 dom: IANAL, they should be similar, the WG is expected to be a strict subset of CG 15:12:07 q+ 15:12:14 q- 15:12:20 ack dom 15:12:23 ack Chai 15:12:40 Chai: I have questions 15:13:02 ... reading this issue, it seems like the request would be extended to support low level instructions 15:13:22 ... there has been a lot of discussions on low vs high level already 15:13:46 ... and it has been pointed out that the "low level" set is not really lower level than what WebNN providers 15:14:48 ... I also don't understand the relation to subsetting WebNN and this low-vs-high level 15:15:06 ... I understand there was discussion on whether WebNN would be low level enough and whether a graph API is the right approach 15:15:23 jonathan has joined #webmachinelearning 15:15:24 ... I'm not sure how this addresses that question (which I think has been otherwise addressed in the discussion) 15:16:01 Present+ Jonathan_Bingham 15:16:31 Anssi: Jonathan you had suggested shipping a subset of WebNN that would perform better - can you speak to that? 15:16:58 RRSAgent, draft minutes v2 15:16:58 I have made the request to generate https://www.w3.org/2020/12/10-webmachinelearning-minutes.html anssik 15:17:04 Jonathan: 3 topics: whether the operations are low-level enough - I agree with Chair that low level operations can be addressed in a graph API, so that issue can be closed 15:17:37 https://github.com/w3c/machine-learning-charter/issues 15:17:53 Anssi: so there is an issue that we can close? 15:18:00 Jonathan: issue #2 can be closed 15:19:00 ... my question about the operation subset was in the context of discussing an alternative to a graph API: we could have a convolution 2D API on its own, independent of any graph or model - because it is a well-known expensive operation 15:19:25 ... The excitement has been mostly about model loader and graph API, but I was asking if we could do that? 15:19:37 q? 15:20:34 ScribeNick: dom 15:20:51 q+ 15:21:01 dom: the WG could decide to work on a subset of the WebNN API and work on it as a separate work item 15:21:12 ... given consensu 15:21:40 anssi: indeed, a spec can be split into as many pieces as we feel is appropriate 15:21:55 RRSAgent, draft minutes v2 15:21:55 I have made the request to generate https://www.w3.org/2020/12/10-webmachinelearning-minutes.html anssik 15:21:58 q? 15:22:00 ack Chai 15:22:21 chai: I want to point out that the way the WebNN is being specified right now is specified as a graph builder API 15:23:17 ... an immediately-executable stand-alone eager-mode operation would not be a strict subset of WebNN from that perspective 15:23:20 q? 15:23:27 Jonathan: +1 on that description 15:24:05 Anssi: so how strongly do you feel we need to put that in scope of the WG (assuming if it isn't) 15:24:52 Jonathan: some of the feedback I've gotten is that 3 individual operations give you 90% of the perf, then it's worth considering such an approach 15:25:02 ... which we had discussed in the early days of the CG 15:25:35 ... this made me wonder how much room the charter leaves to us for this 15:26:15 q? 15:27:00 ScribeNick: anssik 15:27:01 dom: from formal perspective IANAL we could work on both the approaches in the WG, the scope is broad enough to accommodate that, different way to organize the work 15:28:05 ... alternatively, if this is still a significant open question, maybe we need to be explore this in CG before starting the WG charter review? 15:29:03 i'd rather not delay the working group for this :) 15:29:06 anssi: practically, exploring this further would delay the start of the WG 15:29:40 how about i file an issue in the CG and we can discuss in parallel? 15:30:42 Anssi: the WG will be chartered for 2 years - so no matter what, we will recalibrate our scope in 2 years 15:30:56 ... that's another way to pace evolutions in our understanding of the space 15:31:25 https://github.com/w3c/machine-learning-charter/pull/9 15:31:27 q? 15:32:08 Anssi: would like to see if we can close issues and get consensus on what the PR should contain 15:32:32 ... Chai made the point that "the lower level" proposal isn't so clearly lower level 15:32:44 https://github.com/w3c/machine-learning-charter/issues/7 15:33:07 TOSA comparison https://github.com/w3c/machine-learning-charter/issues/7#issuecomment-729487005 15:33:34 chai: we haven't found that operations in TOSA were particularly lower level than WebNN 15:33:58 ... we use XLA / HLO quite a bit to ensure mappability with WebNN 15:34:18 ... some operators are graph/networks on their own, while others are really instruction sets 15:34:22 ... we cover both in WebNN 15:35:01 Present+ Mehmet_Oguz_Derin 15:35:03 ... the process we ensure this is by checking each time we specific a network-like operator, we show how it would decompose in lower-level operators that we define too 15:35:05 Present+ Zoltan_Kis 15:35:17 Present+ Jonathan_Bingham 15:35:24 q+ 15:35:41 ... as a result, this gives us confidence that we consciously design an API that includes low level operations 15:35:55 ack jonathan 15:36:44 Jonathan: I agree with you that the approach seems to address the problem; the Android NN API in launching their 120 operations felt and feels that what they came up with is not low-level enough all what ML practionners want to do on Android 15:37:25 ... they're exploring a major re-architecuring rather than decomposing lower-level instructions, although I'm not sure why, but they seem convinced 15:37:52 ... for WebNN, since the group has been looking at all these alternative instructions set, this seems a pretty robust 15:37:58 ... #7 can be closed as well 15:38:18 afk (need to step out for a few mins) 15:38:21 https://github.com/w3c/machine-learning-charter/issues/6 15:38:21 q+ 15:38:37 ScribeNick: anssik 15:40:53 (im back) 15:43:07 https://github.com/w3c/machine-learning-charter/issues/6 15:43:22 ScribeNick: anssik 15:43:27 https://github.com/w3c/machine-learning-charter/issues/6#issuecomment-736004531 15:43:35 q+ 15:43:47 q- 15:43:51 ack jonathan 15:44:35 jonathan: the question in issue #6, are we doing graph API because we think it's the best abstraction, or because it takes too long to do a model loader with an agreed upon model format 15:44:59 q? 15:45:17 q? 15:45:41 q? 15:46:08 q+ 15:46:12 ack RafaelCintron 15:46:40 RafaelCintron: graph API is format agnostic, there are many such formats 15:46:59 ... maybe graph is better for future scenarios such as training 15:47:12 q+ 15:47:16 ... also OK to do load model first if inferencing is our primary target 15:47:22 ack Chai 15:47:55 Chai: I wrote the explainer, so should explain :) 15:48:08 ... re issue #6, do we believe the graph API is the right abstraction 15:48:36 ... the motivation is to reflect what is already out there, we are not reinventing graph API for WebNN, we acknowledge everywhere you look is a graph API 15:48:51 ... TensorFlow, ONNX, XLA, TOSA all are reflections of what's out there