IRC log of immersive-web on 2020-09-22
Timestamps are in UTC.
- 18:41:22 [RRSAgent]
- RRSAgent has joined #immersive-web
- 18:41:22 [RRSAgent]
- logging to https://www.w3.org/2020/09/22-immersive-web-irc
- 18:41:27 [atsushi]
- zakim, clear agenda
- 18:41:27 [Zakim]
- agenda cleared
- 18:46:22 [atsushi]
- agenda+ webxr-hand-input#50 Move to WG [Manishearth]
- 18:46:46 [atsushi]
- agenda+ webxr-input-profiles#178 Provide assets for the left and right hand [cabanier]
- 18:56:31 [atsushi]
- meeting: Immersive Web Community Group Teleconference
- 18:56:36 [atsushi]
- date: 22 Sep 2020
- 18:56:47 [atsushi]
- atsushi has changed the topic to: https://github.com/immersive-web/administrivia/blob/main/meetings/cg/2020-09-22-Immersive_Web_Community_Group_Teleconference-agenda.md
- 18:56:52 [atsushi]
- agenda: https://github.com/immersive-web/administrivia/blob/main/meetings/cg/2020-09-22-Immersive_Web_Community_Group_Teleconference-agenda.md
- 18:57:39 [atsushi]
- previous meeting: https://www.w3.org/2020/09/15-immersive-web-minutes.html
- 18:57:48 [atsushi]
- chair: Ada
- 18:57:55 [atsushi]
- rrsagent, make log public
- 18:58:02 [atsushi]
- rrsagent, publish minutes v2
- 18:58:02 [RRSAgent]
- I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
- 19:00:01 [cabanier]
- cabanier has joined #immersive-web
- 19:00:07 [cabanier]
- present+
- 19:00:31 [cwilso]
- present+
- 19:01:14 [bajones_]
- bajones_ has joined #Immersive-Web
- 19:01:25 [Brett]
- present+
- 19:01:35 [atsushi]
- present+
- 19:02:58 [Manishearth]
- present+
- 19:03:36 [Lachlan_Ford]
- Lachlan_Ford has joined #immersive-web
- 19:03:45 [kip]
- kip has joined #immersive-web
- 19:06:05 [kip]
- present+
- 19:06:06 [cwilso]
- zakim, choose a victim
- 19:06:06 [Zakim]
- Not knowing who is chairing or who scribed recently, I propose alexturn
- 19:06:19 [cabanier]
- scribenick: cabanier
- 19:06:29 [bajones_]
- present+
- 19:06:36 [Lachlan_Ford]
- present+
- 19:06:48 [atsushi]
- zakim, take up agendum 1
- 19:06:48 [Zakim]
- agendum 1. "webxr-hand-input#50 Move to WG" taken up [from Manishearth via atsushi]
- 19:06:50 [dino]
- present+
- 19:06:58 [yonet]
- yonet has joined #immersive-web
- 19:07:00 [cabanier]
- Manishearth: turns out we already did this
- 19:07:07 [cabanier]
- cwilso: that was easy
- 19:07:21 [cabanier]
- Manishearth: we want to move to fpwd so we will send out a call to consensus
- 19:07:40 [cabanier]
- cwilso: you can send out the call and we'll use that
- 19:07:48 [cwilso]
- https://github.com/immersive-web/webxr-input-profiles/issues/178
- 19:07:56 [atsushi]
- zakim, take up agendum 2
- 19:07:56 [Zakim]
- agendum 2. "webxr-input-profiles#178 Provide assets for the left and right hand" taken up [from cabanier via atsushi]
- 19:08:10 [Manishearth]
- https://github.com/immersive-web/webxr-input-profiles/issues/178
- 19:08:11 [lgombos]
- lgombos has joined #immersive-web
- 19:08:23 [cabanier]
- cabanier: I logged this one and there was some feedback from alex and Manishearth
- 19:08:40 [cabanier]
- Lachlan_Ford: as Alex said, it's an hierarchy
- 19:08:54 [cwilso]
- chair: yonet
- 19:08:54 [cabanier]
- ... the way we do is different than oculus
- 19:09:14 [cabanier]
- ... for us it's based on raw triangles
- 19:09:39 [cabanier]
- ... the extension we provide in openxr, exposes indices and vertices
- 19:10:10 [cabanier]
- bajones_: it seems intense when it comes doing this for every frame
- 19:10:21 [cabanier]
- ... but you do have a skeletal aspect?
- 19:10:43 [cabanier]
- Lachlan_Ford: yes. it's derived from the mesh
- 19:10:56 [Manishearth]
- cabanier: so from the oculus perspective, it's the same as hl
- 19:11:06 [Manishearth]
- ... we also have the hand mesh and skeletal information
- 19:11:15 [Manishearth]
- ... but this is to drive the joints spec
- 19:11:23 [Manishearth]
- ... not a mesh spec (which has been discussed a bit)
- 19:11:37 [Manishearth]
- ... knowing that there is a hand, and knowing that there are joints, you just want to draw a hand
- 19:11:50 [bajones_]
- q+
- 19:11:52 [Manishearth]
- ... as brandon said meshes are kinda expensive for xr, so joints make better sense
- 19:12:09 [Manishearth]
- ... we should have *a* model, not one for each vendor
- 19:12:15 [Manishearth]
- ... espeically since we all track the same joints
- 19:12:16 [Manishearth]
- q+
- 19:12:27 [Manishearth]
- Lachlan_Ford: questionable what an app would do differently based on which hand they get
- 19:12:58 [Manishearth]
- Lachlan_Ford: on the point of a new mesh every frame: you can get away with it because you can map it straight into a vertex buffer
- 19:13:04 [yonet]
- ack bajones_
- 19:13:07 [Manishearth]
- bajones_: a vertex and index buffer per frame?
- 19:13:13 [Manishearth]
- Lachlan_Ford: no, a fixed number of indices per frame
- 19:13:18 [nick-8thwall]
- nick-8thwall has joined #immersive-web
- 19:13:26 [cabanier]
- q+
- 19:13:37 [nick-8thwall]
- q+
- 19:13:38 [Manishearth]
- bajones_: kinda directed at rik, but Lachlan_Ford if you have something that contributes that would help
- 19:13:52 [Manishearth]
- ... i know on the oculus side there are assets that are distributed for a canonical hand mesh
- 19:14:06 [Manishearth]
- ... curious if you know under what circumstances to use the generated vs canonical mesh
- 19:14:20 [Manishearth]
- ... is there a recommendation for why users should one over the other
- 19:14:26 [Manishearth]
- ... kinda get a sense of when one is used over the other
- 19:15:18 [Manishearth]
- bajones_: first thing that comes to mind is that if you are using a game you want an appropriately themed hand, so you will use an imported asset
- 19:15:25 [Manishearth]
- ... but outside of that, are there cases you know
- 19:16:24 [Manishearth]
- cabanier: so for ex the hands you see in oculus use the mesh, not joints
- 19:16:28 [yonet]
- ack Manishearth
- 19:16:37 [cabanier]
- scribenick: cabanier
- 19:16:45 [cabanier]
- Manishearth: I brought up per vendor hands
- 19:17:04 [cabanier]
- ... and mostly because I didn't know what the question was
- 19:17:06 [cabanier]
- q+
- 19:17:32 [cabanier]
- I'm not opposed to it. Just getting one hand one and then we can figure the rest out later
- 19:17:40 [alexturn]
- alexturn has joined #immersive-web
- 19:17:45 [yonet]
- q
- 19:17:46 [cabanier]
- Lachlan_Ford: my understanding was for understanding of gestures
- 19:17:50 [yonet]
- q?
- 19:17:58 [cabanier]
- .. there's gestures support
- 19:18:13 [cabanier]
- ... so I was think that was what inputprofiles was for
- 19:18:18 [cabanier]
- Manishearth: no.
- 19:18:37 [alexturn]
- q+
- 19:18:37 [cabanier]
- ... it's not just what buttons are supported
- 19:18:53 [cabanier]
- ... for webxr select and squeeze are overlapping
- 19:18:58 [bajones_]
- "In the case of hands, you don't have many buttons" [citation needed]
- 19:19:02 [cabanier]
- .... right now there's only one
- 19:19:18 [cabanier]
- ... we could add separate meshes
- 19:19:26 [cabanier]
- ... but it's an option for ys
- 19:19:35 [yonet]
- ack cabanier
- 19:19:50 [Manishearth]
- cabanier: when it comes to the name, right now when you expose hands as a controller, we call it "Oculus hands"
- 19:20:00 [Manishearth]
- ... if an API uses the input profiles repo it won't find that
- 19:20:19 [Manishearth]
- ... has there be a discussion about the name?
- 19:20:27 [Manishearth]
- alexturn: where does the string oculus appear?
- 19:20:42 [Manishearth]
- bajones_: the profile array?
- 19:21:02 [Manishearth]
- alexturn: yeah so "oculus hands" isn't a valid profikle name
- 19:21:11 [Manishearth]
- alexturn: we have already defined "generic-hand-select"
- 19:21:26 [Manishearth]
- ... if oculus was exposing further data we could have oculus-hand-select and that could fall back
- 19:21:43 [Manishearth]
- ... i see this as totally orthogonal though
- 19:21:54 [cabanier]
- q+
- 19:21:56 [Manishearth]
- ... e.g. in openxr we have hand+dir and hand+joints
- 19:22:08 [Manishearth]
- ... idk if i would tie vendor to the way we do joints
- 19:22:16 [bajones_]
- q+
- 19:22:22 [alexturn]
- q-
- 19:22:27 [yonet]
- ack nick-8thwall
- 19:22:36 [Manishearth]
- nick-8thwall: wanted to inform the discussion with stuff from 8thwall face effects
- 19:22:40 [Manishearth]
- ... with a mesh that coverts the face
- 19:22:45 [Manishearth]
- ... and renders it
- 19:23:00 [Manishearth]
- ... meshes are different, need to be specified ahead of time and covered with properly mapped textures
- 19:23:13 [Manishearth]
- ... to come up with assets that cover every encounter
- 19:23:37 [Manishearth]
- ... if the end users are still generating meshes the uvs need to be specified ahead of time, so you wouldn't need indices each frame, just vertex positions
- 19:23:49 [Manishearth]
- ... a thing that comes up often in face effects is that you need texture effects
- 19:24:07 [alexturn]
- q+
- 19:24:12 [Manishearth]
- ... for hands you may need a high fidelity mesh, so you might need an inverse map
- 19:24:26 [Manishearth]
- ... some things that may fall into an uncanny valley situation wrt a robocop/themed hand
- 19:24:52 [Manishearth]
- ... idk from a UX POV if having a high fidelity hand to go with a prefab tex is strongly preferable to being able to have a generic hand model
- 19:25:01 [Manishearth]
- ... that can be appropriately skinned
- 19:25:09 [Manishearth]
- q+
- 19:25:27 [Manishearth]
- nick-8thwall: main thing is, by exposing meshes we have a per-spec multiplication of effort
- 19:25:36 [Manishearth]
- ... indices need to be pre baked anyway
- 19:25:53 [yonet]
- ack cabanier
- 19:26:13 [Manishearth]
- cabanier: in response to alexturn , it seems like we should have a common model
- 19:26:24 [Lachlan_Ford]
- q+
- 19:26:25 [Manishearth]
- ... and everyone agrees on that
- 19:26:37 [Manishearth]
- ... wanna be sure if folks agree it should be based on joints?
- 19:26:46 [Manishearth]
- alexturn: yes, and i think that we should require joints
- 19:27:03 [Manishearth]
- ... (if you don't have joints no dice)
- 19:27:04 [Manishearth]
- cabanier: yes
- 19:27:12 [Manishearth]
- cabanier: also, do you have a model? we don't have extra joints
- 19:27:23 [Manishearth]
- alexturn: unfortunately no, because for rendering we use the mesh
- 19:27:31 [Manishearth]
- ... with a lot of the properties nick was mentioning
- 19:27:38 [alexturn]
- OpenXR hand mesh (MSFT): https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#XR_MSFT_hand_tracking_mesh
- 19:27:38 [Manishearth]
- ... we give you that separately per frame
- 19:28:02 [Manishearth]
- ... currently an MSFT standard, not cross-vendor, but it's a way people can do hand meshes
- 19:28:14 [Manishearth]
- ... we tend to render the hand mesh every frame
- 19:28:23 [Manishearth]
- q?
- 19:28:25 [yonet]
- ack bajones_
- 19:28:45 [cabanier]
- scribenick: cabanier
- 19:28:45 [Manishearth]
- bajones_: addressing nick's concerns: the facial mesh rendering is an interesting topic here
- 19:28:52 [Manishearth]
- ... alcooper has been looking at it on our end
- 19:29:04 [Manishearth]
- ... it is a little different: hard to come up with a reasonable representation
- 19:29:13 [Manishearth]
- ... just blasting out vertices each frame ends up being more concise
- 19:29:25 [Manishearth]
- ... so you don't have quite the same issues as facial rendering, but a lot of the same issues
- 19:29:41 [Manishearth]
- ... can imagine that someone who had no other options can do their own hand mesh with their own stable uvs
- 19:29:48 [Manishearth]
- ... it just won't line up perfectly
- 19:30:24 [Manishearth]
- bajones_: the case of the oculus default mesh -- doesn't have metacarpals, but i think you can just plug in metacarpals and not skin them to anything
- 19:30:43 [Manishearth]
- ... point being if you have a hand mesh that's close enough we can work with that
- 19:31:07 [Manishearth]
- ... final thing i want to say is that i would really like to have some asset available in the input-profiles repo. think it would be useful. would not want it to be automatic
- 19:31:16 [Manishearth]
- ... the same way controllers do now
- 19:31:31 [cabanier]
- q+
- 19:31:59 [Manishearth]
- ... good thing to have it in the library, but to get it to ship out automatically needs a separate impl that is joint aware
- 19:32:03 [yonet]
- ack alexturn
- 19:32:32 [Manishearth]
- alexturn: either joints or hand mesh can get you a good quality result, so if we hit the same articulation bar it should be fine
- 19:32:50 [Manishearth]
- ... some differences here bw ar and vr. we believe they can share the same APIs, but apps will build different experiences
- 19:33:05 [Manishearth]
- e.g. in AR it is critical the app match the true size of the hand, in VR there is more flexibility
- 19:33:13 [Manishearth]
- ... so you can play a lot more with that
- 19:33:25 [Manishearth]
- ... e.g. you might be wearing a big gauntlet
- 19:33:47 [Manishearth]
- ... also enables easy retargeting
- 19:33:57 [Manishearth]
- ... so in vr you have a lot more flexibility there
- 19:34:13 [yonet]
- ack Manishearth
- 19:34:15 [nick-8thwall]
- q+
- 19:34:27 [cabanier]
- Manishearth: I wanted to mention to Nick
- 19:34:37 [atsushi]
- rrsagent, publish minutes v2
- 19:34:37 [RRSAgent]
- I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
- 19:34:46 [cabanier]
- ... right now we have no mesh API and we're not adding it to the joints repo
- 19:35:00 [cabanier]
- ... we're just figuring out if we should have a mesh in the repo
- 19:35:16 [cabanier]
- ... and the hand mesh should be something you opt into
- 19:35:34 [cabanier]
- ... We already had issue with performance when it comes to hands
- 19:36:08 [cabanier]
- --- and each mesh should be relative to a wrist
- 19:36:16 [yonet]
- ack Lachlan_Ford
- 19:36:26 [cabanier]
- ... I agree with Brandon that we can begin with the Oculus mesh
- 19:36:45 [cabanier]
- Lachlan_Ford: I wanted to have an understanding where this model will live
- 19:36:55 [cabanier]
- ... so it's a glb that you pull down and render
- 19:37:11 [cabanier]
- bajones_: yes. we have an assets folder on github.
- 19:37:18 [cabanier]
- ... it will live on that same spot
- 19:37:31 [cabanier]
- ... and it will be available for everyone to download
- 19:38:06 [cabanier]
- ... we need to make sure that it not get automatically pushed. We don't want non-skinned hands to show up there
- 19:38:25 [cabanier]
- ... how can we extend that library so hand aware applications can use it
- 19:38:46 [cabanier]
- Lachlan_Ford: I want to know the motivation for standardization
- 19:38:53 [Manishearth]
- q?
- 19:38:56 [cabanier]
- bajones_: it works this way
- 19:39:24 [Manishearth]
- q+
- 19:39:30 [cabanier]
- ... it's really framework builders having a appropriately licensed mesh
- 19:39:43 [cabanier]
- ... I don't advocate that they will switch to that
- 19:39:48 [alexturn]
- q+
- 19:40:08 [cabanier]
- Lachlan_Ford: the way you describe it, the user can improve upon the baseline
- 19:40:16 [cabanier]
- bajones_: it's often a sanity check
- 19:40:36 [cabanier]
- ... we're not advocating an autorative hand
- 19:41:02 [cabanier]
- Manishearth: we aren't standardizing anything. It's just a JS library
- 19:41:28 [yonet]
- ack cabanier
- 19:41:30 [cabanier]
- ... we're saying: we just put a mesh here and people can standardize on that
- 19:41:56 [Manishearth]
- cabanier: so i think we talked a little about we don't wanna use this model all the time otherwise we get "claw hands"
- 19:42:09 [Manishearth]
- ... i think we only are going to show hands as controllers only if the author opted in to joints
- 19:42:45 [Manishearth]
- cabanier: e.g. on oculus you have controllers and put them down and use their hands
- 19:43:17 [Manishearth]
- ... rn you HAVE to pick up controllers to use webxr
- 19:43:30 [Manishearth]
- ... unless the user changes some settings
- 19:44:02 [Manishearth]
- ... i nthe future we can make it so that ONLY IF the author requests hand-input
- 19:44:09 [Manishearth]
- alexturn: maybe we should have this in the spec
- 19:44:31 [Manishearth]
- alexturn: we might wanna make clear that UAs intend to operate this way
- 19:44:33 [Manishearth]
- q?
- 19:44:39 [Manishearth]
- cabanier: will open an issue
- 19:44:43 [yonet]
- ack nick-8thwall
- 19:45:08 [Manishearth]
- nick-8thwall: just responding to something alexturn said quite a while back: was talking about how we'd enable ar as well as vr for rendering a mesh
- 19:45:19 [Manishearth]
- ... for face effects even though you have a high res mesh
- 19:45:26 [Manishearth]
- ... it's easier to expose a landmark api
- 19:45:40 [Manishearth]
- ... e.g. when you talk about setting a finger to a location
- 19:45:54 [Manishearth]
- ... when you have points on the mesh, you have some special landmarks
- 19:46:14 [Manishearth]
- alexturn: and yeah that's the api that we'll have first in webxr
- 19:46:23 [Manishearth]
- .. you get joints plus the tip
- 19:46:51 [yonet]
- ack Manishearth
- 19:46:51 [Manishearth]
- ... for collider purposes apps would use joints, and rigging is mostly for rendering
- 19:47:12 [cabanier]
- Manishearth: like Alex said, there is no mesh API
- 19:47:20 [cabanier]
- ... it's just a mesh so you can skin the hand
- 19:47:31 [cabanier]
- ... we only expose joints. 25 points per hand
- 19:47:55 [cabanier]
- ... what oculus and Hololens do is different
- 19:48:17 [cabanier]
- ... even if the content doesn't opt into hand, you should expose a hand controller
- 19:48:38 [cabanier]
- Lachlan_Ford: there are cases where you want to render the hand
- 19:48:46 [cabanier]
- Manishearth: not rendering a controller
- 19:49:06 [cabanier]
- alexturn: people will render the gltf
- 19:49:26 [cabanier]
- ... any code that uses motion controller is going to have to be updated to the hand model
- 19:49:56 [cabanier]
- ???
- 19:50:33 [yonet]
- ack alexturn
- 19:50:34 [cabanier]
- the ua would remove direct delivery for the accessibility feature
- 19:51:10 [cabanier]
- alexturn: I'm torn if the default hand mesh in the input profiles will create confusion
- 19:51:32 [cabanier]
- ... you can roll your own hand.
- 19:51:45 [cabanier]
- ... maybe it makes more sense to put it in its own location
- 19:52:16 [cabanier]
- yonet: any updates from anyone?
- 19:52:26 [cabanier]
- kip: my last day with mozilla is this week
- 19:52:41 [cabanier]
- ... but my github name will stay the same
- 19:52:45 [atsushi]
- i/yonet: any updates from anyone?/topic: AOB
- 19:53:03 [cabanier]
- ... other people from mozilla might join especially from the hubs team
- 19:53:39 [cabanier]
- ... if you want face-to-face call especially lighting estimation I'll be happy to create some time for that
- 19:57:48 [alexturn]
- present
- 19:57:49 [alexturn]
- present+
- 20:01:30 [atsushi]
- i/cabanier: so from the oculus perspective,/scribenick: Manishearth/
- 20:01:48 [atsushi]
- i/cabanier: when it comes to the name, right now when you expose hands/scribenick: Manishearth/
- 20:02:07 [atsushi]
- i/bajones_: addressing nick's concerns: the facial mesh/scribenick: Manishearth/
- 20:02:16 [cabanier]
- atsushi: thanks for patching up the minutes!
- 20:02:29 [atsushi]
- i/Manishearth: I wanted to mention to Nick/scribenick: cabanier/
- 20:02:47 [atsushi]
- i/cabanier: so i think we talked a little about/scribenick: Manishearth/
- 20:03:03 [atsushi]
- i/Manishearth: like Alex said,/scribenick: cabanier/
- 20:03:12 [atsushi]
- rrsagent, publish minutes v2
- 20:03:12 [RRSAgent]
- I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
- 20:03:20 [atsushi]
- np ;)
- 20:06:33 [atsushi]
- s/e.g. in AR it is critical the app match the true size of the hand, in VR there is more flexibility/... e.g. in AR it is critical the app match the true size of the hand, in VR there is more flexibility/
- 20:06:58 [atsushi]
- s/--- and each mesh should be relative to a wrist/... and each mesh should be relative to a wrist/
- 20:07:28 [atsushi]
- s/the ua would remove direct delivery for the accessibility feature/... the ua would remove direct delivery for the accessibility feature/
- 20:07:38 [atsushi]
- rrsagent, publish minutes v2
- 20:07:38 [RRSAgent]
- I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
- 20:08:45 [atsushi]
- present+ nick-8thwall, yonet
- 20:08:46 [atsushi]
- rrsagent, publish minutes v2
- 20:08:46 [RRSAgent]
- I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
- 20:09:33 [atsushi]
- rrsagent, bye
- 20:09:33 [RRSAgent]
- I see no action items