Meeting minutes
Operation-specific APIs proposal
anssik: all are probably familiar with the background already, discussed in issue by Jonathan and Ping:
operation-specific APIs proposal
anssik: discusses also key use cases
… today I want to continue discuss and review work-in-progress features that satisfy the requirements of this operation-specific APIs proposal
… I listed 3 topics in the agenda, but please chime in if I missed some requirements not satisfied with these
… 1) WebAssembly scenario of the op level execution use case (Ningxin)
… 2) Clarify at which point weights are used in the compilation (Rafael)
… 3) Discuss how the caller using an op-specific API can resource upload and download (Chai)
anssik: let's first discuss Wasm op level execution, Ningxin opened in issue with use case and requirements
Support CPU - WebAssembly scenario of the op level execution use case (issue #156)
anssik: Ningxin, please introduce this use case and two requirements identified
Ningxin: Use case is one scenario of op-level execution per Jonathan et al. proposal
… use case is basically a JS ML framework executing ops on the CPU device with WebAssembly
… conv2d, matmul to execute op with ML specific instructions available in CPU device, e.g. VNNI
… two requirements:
… 1) WebNN API should allow JS ML frameworks create an MLContext for CPU device
… avoid cross device copies, e.g. CPU to GPU
… currently supported is Power Preference
… 2) WebNN API should allow JS ML frameworks control when the output data is available for access
… motivation: WebNN interacts with Wasm and WebGPU/GL shaders and tensor layout conversion between these is expensive
… for op-level execution scenario, JS framework likes to exec ops one by one and need to create multiple single op ML graphs to execute this ML graph, and execute them in a sequential way, using output of the previous as input to the next
… for this scenario it means, WebNN implementation needs to converse memory layouts and that'd hurt performance
… the framework would not use the intermediate results between these ops
… this means WebNN API need to allow developers to controls when the memory layout conversion happens
RafaelCintron: about frameworks, why can't they just create a graph when they don't need an intermediate representation?
ningxin_hu: good question, if you look at PR #148 there's some discussion about this
… it seems frameworks provide a similar op-level API
… and need to execute the ops, but they don't know the graph beforehand
RafaelCintron: could there be just a single-op graph in such a case?
ningxin_hu: do you suggest this is a framework level optimization?
RafaelCintron: if the user of the framework gives the fw one op at a time, could that work out with the current API?
ningxin_hu: WebNN API is a graph API today, the output data of that graph may not be needed for the user code
RafaelCintron: if we just ship a graph API, would it satisfy this scenario?
ningxin_hu: I'm not sure if I can answer this, should ask Ping and Jonathan to give framework perspective.
anssik: Chai had a comment on WebAssembly.Memory object that allow efficient interop with JS ArrayBuffer
WebAssembly.Memory interop with ArrayBuffer
Wasm-ArrayBuffer interop examples
ningxin_hu: I think this is great pointer for Wasm interop
… I think we're fine, given WebNN accepts ArrayBuffer for input and output
… and addresses Wasm interop requirement
… the open is to allow Wasm to ask WebNN create a CPU context
… to avoid data copies as much as possible
anssik: it seems Chai proposed to address (1) MLContext for CPU device with a preference in MLContextOptions
anssik: ack RafaelCintron
RafaelCintron: want to point out, if weights are copied before compilation, you're going to have a copy
ningxin_hu: I agree with RafaelCintron about data copy, would like to add that memory layout conversion issue is important to address, given there may be platform-specific layouts at play
<ningxin_hu> sounds good
ningxin_hu: 2) Clarify at which point weights are used in the compilation (Rafael)
Clarify at which point weights are used in the compilation (issue #157)
RafaelCintron: my point here is, say you're using model builder API and have a bunch of ops and weights and say WebNN compile this for me
… then you change one weight, is the change considered in the compilation?
… is the change ignored, hopefully not?
… we need to make this well-defined, when the ArrayBuffers are used in the compilation
… I think it is good to change weights up until you say compile after which they're frozen
… also need to consider platform specific conversion of layout
… if they make a copy they can do whatever they want
ningxin_hu: I'd like to get the required behavior clarified a bit
… when calling compile, before promise resolves, the data is copied by the implementation
… when the weights are changed, would not affect the compilation?
RafaelCintron: right before compile() returns control back to developer you can change the weights, makes sense?
ningxin_hu: I think that makes sense, that's how webnn-polyfill implements compile()
… if no objections, we should specify in the spec
anssik: do you have a rough idea how the PR should look like to address #157?
ningxin_hu: now compile() is build() I think in the latest spec update
Zoltan: it is standard move semantics in an API
<ningxin_hu> I'll make a PR and invite Rafael to review
Zoltan: 3) Discuss how the caller using an op-specific API can resource upload and download (Chai)
[defer when Chai available]
TAG review feedback - open PRs
anssik: let's take a look at open PRs addressing TAG review feedback
[tag-tracker] NamedOutput mechanism clarification (issue #140)
anssik: reviews completed, after resolving the merge conflict and we're ready. Ningxin feel free to resolve and merge.
<ningxin_hu> i'll fix the conflicts
anssik: thanks!
TAG review feedback - open issues without associated PRs
anssik: there's a few open issues without PRs, let's check them out together to see if there are blockers
[tag-tracker] Define a common term for logical tensor changes?
anssik: "Looking at this PR, wouldn't it make sense to define a common term for logical tensor changes (e.g. views?) somewhere early in the document so that concept can be re-used?"
anssik: any reactions on that?
[tag-tracker] Isomorphic JS story, worker scope exposure?
<RafaelCintron> I have nothing new to add.
[tag-tracker] Ergonomics of the JS examples
anssik: any new information to add for this? Seems we could close this by stating that the API consumer is primarily framework
[tag-tracker] String enum for activations
anssik: "If there are layers that will be taking activations as string enums, there should simply be a string enum for activations rather than have it just in RecurrentNetworkActivation. (One may argue that hyperbolic tangent is RNN specific, but..."
<ningxin_hu> +1 to Chai's comment
anssik: any further feedback beyond Chai's comment?
anssik: we could address this along the lines of Chai's comments
Privacy review feedback - normative change proposals
anssik: let's review the proposal to make WebNN API a policy-controlled feature, per PING review feedback
anssik: Proposal: The feature is allowed in documents in top-level browsing contexts by default, and when allowed, is allowed by default to same-origin domain documents in child browsing contexts, but is disallowed by default in cross-origin documents in child browsing contexts
anssik: Criteo folks may be interested in this
Alexandre: I don't have anything to comment at this time
Zoltan: one question, it should mention what threat it addresses?
anssik: this is from https://
Zoltan: behind the scenes you may want to run some model in the cloud
… I see normal same-origin policy is used, should explain why this mitigation is needed
anssik: any other comments on the privacy suggestion?
… please provide comments in the issue
RafaelCintron: I'm OK with this feature policy, the goal is to not allow web developers use powerful features in iframes unless the main page allows that explicitly
… we shouldn't ask user but feature policy controlled is OK