IRC log of immersive-web on 2020-09-22

Timestamps are in UTC.

18:41:22 [RRSAgent]
RRSAgent has joined #immersive-web
18:41:22 [RRSAgent]
logging to https://www.w3.org/2020/09/22-immersive-web-irc
18:41:27 [atsushi]
zakim, clear agenda
18:41:27 [Zakim]
agenda cleared
18:46:22 [atsushi]
agenda+ webxr-hand-input#50 Move to WG [Manishearth]
18:46:46 [atsushi]
agenda+ webxr-input-profiles#178 Provide assets for the left and right hand [cabanier]
18:56:31 [atsushi]
meeting: Immersive Web Community Group Teleconference
18:56:36 [atsushi]
date: 22 Sep 2020
18:56:47 [atsushi]
atsushi has changed the topic to: https://github.com/immersive-web/administrivia/blob/main/meetings/cg/2020-09-22-Immersive_Web_Community_Group_Teleconference-agenda.md
18:56:52 [atsushi]
agenda: https://github.com/immersive-web/administrivia/blob/main/meetings/cg/2020-09-22-Immersive_Web_Community_Group_Teleconference-agenda.md
18:57:39 [atsushi]
previous meeting: https://www.w3.org/2020/09/15-immersive-web-minutes.html
18:57:48 [atsushi]
chair: Ada
18:57:55 [atsushi]
rrsagent, make log public
18:58:02 [atsushi]
rrsagent, publish minutes v2
18:58:02 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
19:00:01 [cabanier]
cabanier has joined #immersive-web
19:00:07 [cabanier]
present+
19:00:31 [cwilso]
present+
19:01:14 [bajones_]
bajones_ has joined #Immersive-Web
19:01:25 [Brett]
present+
19:01:35 [atsushi]
present+
19:02:58 [Manishearth]
present+
19:03:36 [Lachlan_Ford]
Lachlan_Ford has joined #immersive-web
19:03:45 [kip]
kip has joined #immersive-web
19:06:05 [kip]
present+
19:06:06 [cwilso]
zakim, choose a victim
19:06:06 [Zakim]
Not knowing who is chairing or who scribed recently, I propose alexturn
19:06:19 [cabanier]
scribenick: cabanier
19:06:29 [bajones_]
present+
19:06:36 [Lachlan_Ford]
present+
19:06:48 [atsushi]
zakim, take up agendum 1
19:06:48 [Zakim]
agendum 1. "webxr-hand-input#50 Move to WG" taken up [from Manishearth via atsushi]
19:06:50 [dino]
present+
19:06:58 [yonet]
yonet has joined #immersive-web
19:07:00 [cabanier]
Manishearth: turns out we already did this
19:07:07 [cabanier]
cwilso: that was easy
19:07:21 [cabanier]
Manishearth: we want to move to fpwd so we will send out a call to consensus
19:07:40 [cabanier]
cwilso: you can send out the call and we'll use that
19:07:48 [cwilso]
https://github.com/immersive-web/webxr-input-profiles/issues/178
19:07:56 [atsushi]
zakim, take up agendum 2
19:07:56 [Zakim]
agendum 2. "webxr-input-profiles#178 Provide assets for the left and right hand" taken up [from cabanier via atsushi]
19:08:10 [Manishearth]
https://github.com/immersive-web/webxr-input-profiles/issues/178
19:08:11 [lgombos]
lgombos has joined #immersive-web
19:08:23 [cabanier]
cabanier: I logged this one and there was some feedback from alex and Manishearth
19:08:40 [cabanier]
Lachlan_Ford: as Alex said, it's an hierarchy
19:08:54 [cwilso]
chair: yonet
19:08:54 [cabanier]
... the way we do is different than oculus
19:09:14 [cabanier]
... for us it's based on raw triangles
19:09:39 [cabanier]
... the extension we provide in openxr, exposes indices and vertices
19:10:10 [cabanier]
bajones_: it seems intense when it comes doing this for every frame
19:10:21 [cabanier]
... but you do have a skeletal aspect?
19:10:43 [cabanier]
Lachlan_Ford: yes. it's derived from the mesh
19:10:56 [Manishearth]
cabanier: so from the oculus perspective, it's the same as hl
19:11:06 [Manishearth]
... we also have the hand mesh and skeletal information
19:11:15 [Manishearth]
... but this is to drive the joints spec
19:11:23 [Manishearth]
... not a mesh spec (which has been discussed a bit)
19:11:37 [Manishearth]
... knowing that there is a hand, and knowing that there are joints, you just want to draw a hand
19:11:50 [bajones_]
q+
19:11:52 [Manishearth]
... as brandon said meshes are kinda expensive for xr, so joints make better sense
19:12:09 [Manishearth]
... we should have *a* model, not one for each vendor
19:12:15 [Manishearth]
... espeically since we all track the same joints
19:12:16 [Manishearth]
q+
19:12:27 [Manishearth]
Lachlan_Ford: questionable what an app would do differently based on which hand they get
19:12:58 [Manishearth]
Lachlan_Ford: on the point of a new mesh every frame: you can get away with it because you can map it straight into a vertex buffer
19:13:04 [yonet]
ack bajones_
19:13:07 [Manishearth]
bajones_: a vertex and index buffer per frame?
19:13:13 [Manishearth]
Lachlan_Ford: no, a fixed number of indices per frame
19:13:18 [nick-8thwall]
nick-8thwall has joined #immersive-web
19:13:26 [cabanier]
q+
19:13:37 [nick-8thwall]
q+
19:13:38 [Manishearth]
bajones_: kinda directed at rik, but Lachlan_Ford if you have something that contributes that would help
19:13:52 [Manishearth]
... i know on the oculus side there are assets that are distributed for a canonical hand mesh
19:14:06 [Manishearth]
... curious if you know under what circumstances to use the generated vs canonical mesh
19:14:20 [Manishearth]
... is there a recommendation for why users should one over the other
19:14:26 [Manishearth]
... kinda get a sense of when one is used over the other
19:15:18 [Manishearth]
bajones_: first thing that comes to mind is that if you are using a game you want an appropriately themed hand, so you will use an imported asset
19:15:25 [Manishearth]
... but outside of that, are there cases you know
19:16:24 [Manishearth]
cabanier: so for ex the hands you see in oculus use the mesh, not joints
19:16:28 [yonet]
ack Manishearth
19:16:37 [cabanier]
scribenick: cabanier
19:16:45 [cabanier]
Manishearth: I brought up per vendor hands
19:17:04 [cabanier]
... and mostly because I didn't know what the question was
19:17:06 [cabanier]
q+
19:17:32 [cabanier]
I'm not opposed to it. Just getting one hand one and then we can figure the rest out later
19:17:40 [alexturn]
alexturn has joined #immersive-web
19:17:45 [yonet]
q
19:17:46 [cabanier]
Lachlan_Ford: my understanding was for understanding of gestures
19:17:50 [yonet]
q?
19:17:58 [cabanier]
.. there's gestures support
19:18:13 [cabanier]
... so I was think that was what inputprofiles was for
19:18:18 [cabanier]
Manishearth: no.
19:18:37 [alexturn]
q+
19:18:37 [cabanier]
... it's not just what buttons are supported
19:18:53 [cabanier]
... for webxr select and squeeze are overlapping
19:18:58 [bajones_]
"In the case of hands, you don't have many buttons" [citation needed]
19:19:02 [cabanier]
.... right now there's only one
19:19:18 [cabanier]
... we could add separate meshes
19:19:26 [cabanier]
... but it's an option for ys
19:19:35 [yonet]
ack cabanier
19:19:50 [Manishearth]
cabanier: when it comes to the name, right now when you expose hands as a controller, we call it "Oculus hands"
19:20:00 [Manishearth]
... if an API uses the input profiles repo it won't find that
19:20:19 [Manishearth]
... has there be a discussion about the name?
19:20:27 [Manishearth]
alexturn: where does the string oculus appear?
19:20:42 [Manishearth]
bajones_: the profile array?
19:21:02 [Manishearth]
alexturn: yeah so "oculus hands" isn't a valid profikle name
19:21:11 [Manishearth]
alexturn: we have already defined "generic-hand-select"
19:21:26 [Manishearth]
... if oculus was exposing further data we could have oculus-hand-select and that could fall back
19:21:43 [Manishearth]
... i see this as totally orthogonal though
19:21:54 [cabanier]
q+
19:21:56 [Manishearth]
... e.g. in openxr we have hand+dir and hand+joints
19:22:08 [Manishearth]
... idk if i would tie vendor to the way we do joints
19:22:16 [bajones_]
q+
19:22:22 [alexturn]
q-
19:22:27 [yonet]
ack nick-8thwall
19:22:36 [Manishearth]
nick-8thwall: wanted to inform the discussion with stuff from 8thwall face effects
19:22:40 [Manishearth]
... with a mesh that coverts the face
19:22:45 [Manishearth]
... and renders it
19:23:00 [Manishearth]
... meshes are different, need to be specified ahead of time and covered with properly mapped textures
19:23:13 [Manishearth]
... to come up with assets that cover every encounter
19:23:37 [Manishearth]
... if the end users are still generating meshes the uvs need to be specified ahead of time, so you wouldn't need indices each frame, just vertex positions
19:23:49 [Manishearth]
... a thing that comes up often in face effects is that you need texture effects
19:24:07 [alexturn]
q+
19:24:12 [Manishearth]
... for hands you may need a high fidelity mesh, so you might need an inverse map
19:24:26 [Manishearth]
... some things that may fall into an uncanny valley situation wrt a robocop/themed hand
19:24:52 [Manishearth]
... idk from a UX POV if having a high fidelity hand to go with a prefab tex is strongly preferable to being able to have a generic hand model
19:25:01 [Manishearth]
... that can be appropriately skinned
19:25:09 [Manishearth]
q+
19:25:27 [Manishearth]
nick-8thwall: main thing is, by exposing meshes we have a per-spec multiplication of effort
19:25:36 [Manishearth]
... indices need to be pre baked anyway
19:25:53 [yonet]
ack cabanier
19:26:13 [Manishearth]
cabanier: in response to alexturn , it seems like we should have a common model
19:26:24 [Lachlan_Ford]
q+
19:26:25 [Manishearth]
... and everyone agrees on that
19:26:37 [Manishearth]
... wanna be sure if folks agree it should be based on joints?
19:26:46 [Manishearth]
alexturn: yes, and i think that we should require joints
19:27:03 [Manishearth]
... (if you don't have joints no dice)
19:27:04 [Manishearth]
cabanier: yes
19:27:12 [Manishearth]
cabanier: also, do you have a model? we don't have extra joints
19:27:23 [Manishearth]
alexturn: unfortunately no, because for rendering we use the mesh
19:27:31 [Manishearth]
... with a lot of the properties nick was mentioning
19:27:38 [alexturn]
OpenXR hand mesh (MSFT): https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#XR_MSFT_hand_tracking_mesh
19:27:38 [Manishearth]
... we give you that separately per frame
19:28:02 [Manishearth]
... currently an MSFT standard, not cross-vendor, but it's a way people can do hand meshes
19:28:14 [Manishearth]
... we tend to render the hand mesh every frame
19:28:23 [Manishearth]
q?
19:28:25 [yonet]
ack bajones_
19:28:45 [cabanier]
scribenick: cabanier
19:28:45 [Manishearth]
bajones_: addressing nick's concerns: the facial mesh rendering is an interesting topic here
19:28:52 [Manishearth]
... alcooper has been looking at it on our end
19:29:04 [Manishearth]
... it is a little different: hard to come up with a reasonable representation
19:29:13 [Manishearth]
... just blasting out vertices each frame ends up being more concise
19:29:25 [Manishearth]
... so you don't have quite the same issues as facial rendering, but a lot of the same issues
19:29:41 [Manishearth]
... can imagine that someone who had no other options can do their own hand mesh with their own stable uvs
19:29:48 [Manishearth]
... it just won't line up perfectly
19:30:24 [Manishearth]
bajones_: the case of the oculus default mesh -- doesn't have metacarpals, but i think you can just plug in metacarpals and not skin them to anything
19:30:43 [Manishearth]
... point being if you have a hand mesh that's close enough we can work with that
19:31:07 [Manishearth]
... final thing i want to say is that i would really like to have some asset available in the input-profiles repo. think it would be useful. would not want it to be automatic
19:31:16 [Manishearth]
... the same way controllers do now
19:31:31 [cabanier]
q+
19:31:59 [Manishearth]
... good thing to have it in the library, but to get it to ship out automatically needs a separate impl that is joint aware
19:32:03 [yonet]
ack alexturn
19:32:32 [Manishearth]
alexturn: either joints or hand mesh can get you a good quality result, so if we hit the same articulation bar it should be fine
19:32:50 [Manishearth]
... some differences here bw ar and vr. we believe they can share the same APIs, but apps will build different experiences
19:33:05 [Manishearth]
e.g. in AR it is critical the app match the true size of the hand, in VR there is more flexibility
19:33:13 [Manishearth]
... so you can play a lot more with that
19:33:25 [Manishearth]
... e.g. you might be wearing a big gauntlet
19:33:47 [Manishearth]
... also enables easy retargeting
19:33:57 [Manishearth]
... so in vr you have a lot more flexibility there
19:34:13 [yonet]
ack Manishearth
19:34:15 [nick-8thwall]
q+
19:34:27 [cabanier]
Manishearth: I wanted to mention to Nick
19:34:37 [atsushi]
rrsagent, publish minutes v2
19:34:37 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
19:34:46 [cabanier]
... right now we have no mesh API and we're not adding it to the joints repo
19:35:00 [cabanier]
... we're just figuring out if we should have a mesh in the repo
19:35:16 [cabanier]
... and the hand mesh should be something you opt into
19:35:34 [cabanier]
... We already had issue with performance when it comes to hands
19:36:08 [cabanier]
--- and each mesh should be relative to a wrist
19:36:16 [yonet]
ack Lachlan_Ford
19:36:26 [cabanier]
... I agree with Brandon that we can begin with the Oculus mesh
19:36:45 [cabanier]
Lachlan_Ford: I wanted to have an understanding where this model will live
19:36:55 [cabanier]
... so it's a glb that you pull down and render
19:37:11 [cabanier]
bajones_: yes. we have an assets folder on github.
19:37:18 [cabanier]
... it will live on that same spot
19:37:31 [cabanier]
... and it will be available for everyone to download
19:38:06 [cabanier]
... we need to make sure that it not get automatically pushed. We don't want non-skinned hands to show up there
19:38:25 [cabanier]
... how can we extend that library so hand aware applications can use it
19:38:46 [cabanier]
Lachlan_Ford: I want to know the motivation for standardization
19:38:53 [Manishearth]
q?
19:38:56 [cabanier]
bajones_: it works this way
19:39:24 [Manishearth]
q+
19:39:30 [cabanier]
... it's really framework builders having a appropriately licensed mesh
19:39:43 [cabanier]
... I don't advocate that they will switch to that
19:39:48 [alexturn]
q+
19:40:08 [cabanier]
Lachlan_Ford: the way you describe it, the user can improve upon the baseline
19:40:16 [cabanier]
bajones_: it's often a sanity check
19:40:36 [cabanier]
... we're not advocating an autorative hand
19:41:02 [cabanier]
Manishearth: we aren't standardizing anything. It's just a JS library
19:41:28 [yonet]
ack cabanier
19:41:30 [cabanier]
... we're saying: we just put a mesh here and people can standardize on that
19:41:56 [Manishearth]
cabanier: so i think we talked a little about we don't wanna use this model all the time otherwise we get "claw hands"
19:42:09 [Manishearth]
... i think we only are going to show hands as controllers only if the author opted in to joints
19:42:45 [Manishearth]
cabanier: e.g. on oculus you have controllers and put them down and use their hands
19:43:17 [Manishearth]
... rn you HAVE to pick up controllers to use webxr
19:43:30 [Manishearth]
... unless the user changes some settings
19:44:02 [Manishearth]
... i nthe future we can make it so that ONLY IF the author requests hand-input
19:44:09 [Manishearth]
alexturn: maybe we should have this in the spec
19:44:31 [Manishearth]
alexturn: we might wanna make clear that UAs intend to operate this way
19:44:33 [Manishearth]
q?
19:44:39 [Manishearth]
cabanier: will open an issue
19:44:43 [yonet]
ack nick-8thwall
19:45:08 [Manishearth]
nick-8thwall: just responding to something alexturn said quite a while back: was talking about how we'd enable ar as well as vr for rendering a mesh
19:45:19 [Manishearth]
... for face effects even though you have a high res mesh
19:45:26 [Manishearth]
... it's easier to expose a landmark api
19:45:40 [Manishearth]
... e.g. when you talk about setting a finger to a location
19:45:54 [Manishearth]
... when you have points on the mesh, you have some special landmarks
19:46:14 [Manishearth]
alexturn: and yeah that's the api that we'll have first in webxr
19:46:23 [Manishearth]
.. you get joints plus the tip
19:46:51 [yonet]
ack Manishearth
19:46:51 [Manishearth]
... for collider purposes apps would use joints, and rigging is mostly for rendering
19:47:12 [cabanier]
Manishearth: like Alex said, there is no mesh API
19:47:20 [cabanier]
... it's just a mesh so you can skin the hand
19:47:31 [cabanier]
... we only expose joints. 25 points per hand
19:47:55 [cabanier]
... what oculus and Hololens do is different
19:48:17 [cabanier]
... even if the content doesn't opt into hand, you should expose a hand controller
19:48:38 [cabanier]
Lachlan_Ford: there are cases where you want to render the hand
19:48:46 [cabanier]
Manishearth: not rendering a controller
19:49:06 [cabanier]
alexturn: people will render the gltf
19:49:26 [cabanier]
... any code that uses motion controller is going to have to be updated to the hand model
19:49:56 [cabanier]
???
19:50:33 [yonet]
ack alexturn
19:50:34 [cabanier]
the ua would remove direct delivery for the accessibility feature
19:51:10 [cabanier]
alexturn: I'm torn if the default hand mesh in the input profiles will create confusion
19:51:32 [cabanier]
... you can roll your own hand.
19:51:45 [cabanier]
... maybe it makes more sense to put it in its own location
19:52:16 [cabanier]
yonet: any updates from anyone?
19:52:26 [cabanier]
kip: my last day with mozilla is this week
19:52:41 [cabanier]
... but my github name will stay the same
19:52:45 [atsushi]
i/yonet: any updates from anyone?/topic: AOB
19:53:03 [cabanier]
... other people from mozilla might join especially from the hubs team
19:53:39 [cabanier]
... if you want face-to-face call especially lighting estimation I'll be happy to create some time for that
19:57:48 [alexturn]
present
19:57:49 [alexturn]
present+
20:01:30 [atsushi]
i/cabanier: so from the oculus perspective,/scribenick: Manishearth/
20:01:48 [atsushi]
i/cabanier: when it comes to the name, right now when you expose hands/scribenick: Manishearth/
20:02:07 [atsushi]
i/bajones_: addressing nick's concerns: the facial mesh/scribenick: Manishearth/
20:02:16 [cabanier]
atsushi: thanks for patching up the minutes!
20:02:29 [atsushi]
i/Manishearth: I wanted to mention to Nick/scribenick: cabanier/
20:02:47 [atsushi]
i/cabanier: so i think we talked a little about/scribenick: Manishearth/
20:03:03 [atsushi]
i/Manishearth: like Alex said,/scribenick: cabanier/
20:03:12 [atsushi]
rrsagent, publish minutes v2
20:03:12 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
20:03:20 [atsushi]
np ;)
20:06:33 [atsushi]
s/e.g. in AR it is critical the app match the true size of the hand, in VR there is more flexibility/... e.g. in AR it is critical the app match the true size of the hand, in VR there is more flexibility/
20:06:58 [atsushi]
s/--- and each mesh should be relative to a wrist/... and each mesh should be relative to a wrist/
20:07:28 [atsushi]
s/the ua would remove direct delivery for the accessibility feature/... the ua would remove direct delivery for the accessibility feature/
20:07:38 [atsushi]
rrsagent, publish minutes v2
20:07:38 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
20:08:45 [atsushi]
present+ nick-8thwall, yonet
20:08:46 [atsushi]
rrsagent, publish minutes v2
20:08:46 [RRSAgent]
I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
20:09:33 [atsushi]
rrsagent, bye
20:09:33 [RRSAgent]
I see no action items