18:41:22 <RRSAgent> RRSAgent has joined #immersive-web
18:41:22 <RRSAgent> logging to https://www.w3.org/2020/09/22-immersive-web-irc
18:41:27 <atsushi> zakim, clear agenda
18:41:27 <Zakim> agenda cleared
18:46:22 <atsushi> agenda+ webxr-hand-input#50 Move to WG [Manishearth]
18:46:46 <atsushi> agenda+ webxr-input-profiles#178 Provide assets for the left and right hand [cabanier]
18:56:31 <atsushi> meeting: Immersive Web Community Group Teleconference
18:56:36 <atsushi> date: 22 Sep 2020
18:56:47 <atsushi> atsushi has changed the topic to: https://github.com/immersive-web/administrivia/blob/main/meetings/cg/2020-09-22-Immersive_Web_Community_Group_Teleconference-agenda.md
18:56:52 <atsushi> agenda: https://github.com/immersive-web/administrivia/blob/main/meetings/cg/2020-09-22-Immersive_Web_Community_Group_Teleconference-agenda.md
18:57:39 <atsushi> previous meeting: https://www.w3.org/2020/09/15-immersive-web-minutes.html
18:57:48 <atsushi> chair: Ada
18:57:55 <atsushi> rrsagent, make log public
18:58:02 <atsushi> rrsagent, publish minutes v2
18:58:02 <RRSAgent> I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
19:00:01 <cabanier> cabanier has joined #immersive-web
19:00:07 <cabanier> present+
19:00:31 <cwilso> present+
19:01:14 <bajones_> bajones_ has joined #Immersive-Web
19:01:25 <Brett> present+
19:01:35 <atsushi> present+
19:02:58 <Manishearth> present+
19:03:36 <Lachlan_Ford> Lachlan_Ford has joined #immersive-web
19:03:45 <kip> kip has joined #immersive-web
19:06:05 <kip> present+
19:06:06 <cwilso> zakim, choose a victim
19:06:06 <Zakim> Not knowing who is chairing or who scribed recently, I propose alexturn
19:06:19 <cabanier> scribenick: cabanier
19:06:29 <bajones_> present+
19:06:36 <Lachlan_Ford> present+
19:06:48 <atsushi> zakim, take up agendum 1
19:06:48 <Zakim> agendum 1. "webxr-hand-input#50 Move to WG" taken up [from Manishearth via atsushi]
19:06:50 <dino> present+
19:06:58 <yonet> yonet has joined #immersive-web
19:07:00 <cabanier> Manishearth: turns out we already did this
19:07:07 <cabanier> cwilso: that was easy
19:07:21 <cabanier> Manishearth: we want to move to fpwd so we will send out a call to consensus
19:07:40 <cabanier> cwilso: you can send out the call and we'll use that
19:07:48 <cwilso> https://github.com/immersive-web/webxr-input-profiles/issues/178
19:07:56 <atsushi> zakim, take up agendum 2
19:07:56 <Zakim> agendum 2. "webxr-input-profiles#178 Provide assets for the left and right hand" taken up [from cabanier via atsushi]
19:08:10 <Manishearth> https://github.com/immersive-web/webxr-input-profiles/issues/178
19:08:11 <lgombos> lgombos has joined #immersive-web
19:08:23 <cabanier> cabanier: I logged this one and there was some feedback from alex and Manishearth
19:08:40 <cabanier> Lachlan_Ford: as Alex said, it's an hierarchy
19:08:54 <cwilso> chair: yonet
19:08:54 <cabanier> ... the way we do is different than oculus
19:09:14 <cabanier> ... for us it's based on raw triangles
19:09:39 <cabanier> ... the extension we provide in openxr, exposes indices and vertices
19:10:10 <cabanier> bajones_: it seems intense when it comes doing this for every frame
19:10:21 <cabanier> ... but you do have a skeletal aspect?
19:10:43 <cabanier> Lachlan_Ford: yes. it's derived from the mesh
19:10:56 <Manishearth> cabanier: so from the oculus perspective, it's the same as hl
19:11:06 <Manishearth> ... we also have the hand mesh and skeletal information
19:11:15 <Manishearth> ... but this is to drive the joints spec
19:11:23 <Manishearth> ... not a mesh spec (which has been discussed a bit)
19:11:37 <Manishearth> ... knowing that there is a hand, and knowing that there are joints, you just want to draw a hand
19:11:50 <bajones_> q+
19:11:52 <Manishearth> ... as brandon said meshes are kinda expensive for xr, so joints make better sense
19:12:09 <Manishearth> ... we should have *a* model, not one for each vendor
19:12:15 <Manishearth> ... espeically since we all track the same joints
19:12:16 <Manishearth> q+
19:12:27 <Manishearth> Lachlan_Ford: questionable what an app would do differently based on which hand they get
19:12:58 <Manishearth> Lachlan_Ford: on the point of a new mesh every frame: you can get away with it because you can map it straight into a vertex buffer
19:13:04 <yonet> ack bajones_
19:13:07 <Manishearth> bajones_: a vertex and index buffer per frame?
19:13:13 <Manishearth> Lachlan_Ford: no, a fixed number of indices per frame
19:13:18 <nick-8thwall> nick-8thwall has joined #immersive-web
19:13:26 <cabanier> q+
19:13:37 <nick-8thwall> q+
19:13:38 <Manishearth> bajones_: kinda directed at rik, but Lachlan_Ford if you have something that contributes that would help
19:13:52 <Manishearth> ... i know on the oculus side there are assets that are distributed for a canonical hand mesh
19:14:06 <Manishearth> ... curious if you know under what circumstances to use the generated vs canonical mesh
19:14:20 <Manishearth> ... is there a recommendation for why users should one over the other
19:14:26 <Manishearth> ... kinda get a sense of when one is used over the other
19:15:18 <Manishearth> bajones_: first thing that comes to mind is that if you are using a game you want an appropriately themed hand, so you will use an imported asset
19:15:25 <Manishearth> ... but outside of that, are there cases you know
19:16:24 <Manishearth> cabanier: so for ex the hands you see in oculus use the mesh, not joints
19:16:28 <yonet> ack Manishearth
19:16:37 <cabanier> scribenick: cabanier
19:16:45 <cabanier> Manishearth: I brought up per vendor hands
19:17:04 <cabanier> ... and mostly because I didn't know what the question was
19:17:06 <cabanier> q+
19:17:32 <cabanier> I'm not opposed to it. Just getting one hand one and then we can figure the rest out later
19:17:40 <alexturn> alexturn has joined #immersive-web
19:17:45 <yonet> q
19:17:46 <cabanier> Lachlan_Ford: my understanding was for understanding of gestures
19:17:50 <yonet> q?
19:17:58 <cabanier> .. there's gestures support
19:18:13 <cabanier> ... so I was think that was what inputprofiles was for
19:18:18 <cabanier> Manishearth: no.
19:18:37 <alexturn> q+
19:18:37 <cabanier> ... it's not just what buttons are supported
19:18:53 <cabanier> ... for webxr select and squeeze are overlapping
19:18:58 <bajones_> "In the case of hands, you don't have many buttons" [citation needed]
19:19:02 <cabanier> .... right now there's only one
19:19:18 <cabanier> ... we could add separate meshes
19:19:26 <cabanier> ... but it's an option for ys
19:19:35 <yonet> ack cabanier
19:19:50 <Manishearth> cabanier: when it comes to the name, right now when you expose hands as a controller, we call it "Oculus hands"
19:20:00 <Manishearth> ... if an API uses the input profiles repo it won't find that
19:20:19 <Manishearth> ... has there be a discussion about the name?
19:20:27 <Manishearth> alexturn: where does the string oculus appear?
19:20:42 <Manishearth> bajones_: the profile array?
19:21:02 <Manishearth> alexturn: yeah so "oculus hands" isn't a valid profikle name
19:21:11 <Manishearth> alexturn: we have already defined "generic-hand-select"
19:21:26 <Manishearth> ... if oculus was exposing further data we could have oculus-hand-select and that could fall back
19:21:43 <Manishearth> ... i see this as totally orthogonal though
19:21:54 <cabanier> q+
19:21:56 <Manishearth> ... e.g. in openxr we have hand+dir and hand+joints
19:22:08 <Manishearth> ... idk if i would tie vendor to the way we do joints
19:22:16 <bajones_> q+
19:22:22 <alexturn> q-
19:22:27 <yonet> ack nick-8thwall
19:22:36 <Manishearth> nick-8thwall: wanted to inform the discussion with stuff from 8thwall face effects
19:22:40 <Manishearth> ... with a mesh that coverts the face
19:22:45 <Manishearth> ... and renders it
19:23:00 <Manishearth> ... meshes are different, need to be specified ahead of time and covered with properly mapped textures
19:23:13 <Manishearth> ... to come up with assets that cover every encounter
19:23:37 <Manishearth> ... if the end users are still generating meshes the uvs need to be specified ahead of time, so you wouldn't need indices each frame, just vertex positions
19:23:49 <Manishearth> ... a thing that comes up often in face effects is that you need texture effects
19:24:07 <alexturn> q+
19:24:12 <Manishearth> ... for hands you may need a high fidelity mesh, so you might need an inverse map
19:24:26 <Manishearth> ... some things that may fall into an uncanny valley situation wrt a robocop/themed hand
19:24:52 <Manishearth> ... idk from a UX POV if having a high fidelity hand to go with a prefab tex is strongly preferable to being able to have a generic hand model
19:25:01 <Manishearth> ... that can be appropriately skinned
19:25:09 <Manishearth> q+
19:25:27 <Manishearth> nick-8thwall: main thing is, by exposing meshes we have a per-spec multiplication of effort
19:25:36 <Manishearth> ... indices need to be pre baked anyway
19:25:53 <yonet> ack cabanier
19:26:13 <Manishearth> cabanier: in response to alexturn , it seems like we should have a common model
19:26:24 <Lachlan_Ford> q+
19:26:25 <Manishearth> ... and everyone agrees on that
19:26:37 <Manishearth> ... wanna be sure if folks agree it should be based on joints?
19:26:46 <Manishearth> alexturn: yes, and i think that we should require joints
19:27:03 <Manishearth> ... (if you don't have joints no dice)
19:27:04 <Manishearth> cabanier: yes
19:27:12 <Manishearth> cabanier: also, do you have a model? we don't have extra joints
19:27:23 <Manishearth> alexturn: unfortunately no, because for rendering we use the mesh
19:27:31 <Manishearth> ... with a lot of the properties nick was mentioning
19:27:38 <alexturn> OpenXR hand mesh (MSFT): https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#XR_MSFT_hand_tracking_mesh
19:27:38 <Manishearth> ... we give you that separately per frame
19:28:02 <Manishearth> ... currently an MSFT standard, not cross-vendor, but it's a way people can do hand meshes
19:28:14 <Manishearth> ... we tend to render the hand mesh every frame
19:28:23 <Manishearth> q?
19:28:25 <yonet> ack bajones_
19:28:45 <cabanier> scribenick: cabanier
19:28:45 <Manishearth> bajones_: addressing nick's concerns: the facial mesh rendering is an interesting topic here
19:28:52 <Manishearth> ... alcooper has been looking at it on our end
19:29:04 <Manishearth> ... it is a little different: hard to come up with a reasonable representation
19:29:13 <Manishearth> ... just blasting out vertices each frame ends up being more concise
19:29:25 <Manishearth> ... so you don't have quite the same issues as facial rendering, but a lot of the same issues
19:29:41 <Manishearth> ... can imagine that someone who had no other options can do their own hand mesh with their own stable uvs
19:29:48 <Manishearth> ... it just won't line up perfectly
19:30:24 <Manishearth> bajones_: the case of the oculus default mesh -- doesn't have metacarpals, but i think you can just plug in metacarpals and not skin them to anything
19:30:43 <Manishearth> ... point being if you have a hand mesh that's close enough we can work with that
19:31:07 <Manishearth> ... final thing i want to say is that i would really like to have some asset available in the input-profiles repo. think it would be useful. would not want it to be automatic
19:31:16 <Manishearth> ... the same way controllers do now
19:31:31 <cabanier> q+
19:31:59 <Manishearth> ... good thing to have it in the library, but to get it to ship out automatically needs a separate impl that is joint aware
19:32:03 <yonet> ack alexturn
19:32:32 <Manishearth> alexturn: either joints or hand mesh can get you a good quality result, so if we hit the same articulation bar it should be fine
19:32:50 <Manishearth> ... some differences here bw ar and vr. we believe they can share the same APIs, but apps will build different experiences
19:33:05 <Manishearth> e.g. in AR it is critical the app match the true size of the hand, in VR there is more flexibility
19:33:13 <Manishearth> ... so you can play a lot more with that
19:33:25 <Manishearth> ... e.g. you might be wearing a big gauntlet
19:33:47 <Manishearth> ... also enables easy retargeting
19:33:57 <Manishearth> ... so in vr you have a lot more flexibility there
19:34:13 <yonet> ack Manishearth
19:34:15 <nick-8thwall> q+
19:34:27 <cabanier> Manishearth: I wanted to mention to Nick
19:34:37 <atsushi> rrsagent, publish minutes v2
19:34:37 <RRSAgent> I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
19:34:46 <cabanier> ... right now we have no mesh API and we're not adding it to the joints repo
19:35:00 <cabanier> ... we're just figuring out if we should have a mesh in the repo
19:35:16 <cabanier> ... and the hand mesh should be something you opt into
19:35:34 <cabanier> ... We already had issue with performance when it comes to hands
19:36:08 <cabanier> --- and each mesh should be relative to a wrist
19:36:16 <yonet> ack Lachlan_Ford
19:36:26 <cabanier> ... I agree with Brandon that we can begin with the Oculus mesh
19:36:45 <cabanier> Lachlan_Ford: I wanted to have an understanding where this model will live
19:36:55 <cabanier> ... so it's a glb that you pull down and render
19:37:11 <cabanier> bajones_: yes. we have an assets folder on github.
19:37:18 <cabanier> ... it will live on that same spot
19:37:31 <cabanier> ... and it will be available for everyone to download
19:38:06 <cabanier> ... we need to make sure that it not get automatically pushed. We don't want non-skinned hands to show up there
19:38:25 <cabanier> ... how can we extend that library so hand aware applications can use it
19:38:46 <cabanier> Lachlan_Ford: I want to know the motivation for standardization
19:38:53 <Manishearth> q?
19:38:56 <cabanier> bajones_: it works this way
19:39:24 <Manishearth> q+
19:39:30 <cabanier> ... it's really framework builders having a appropriately licensed mesh
19:39:43 <cabanier> ... I don't advocate that they will switch to that
19:39:48 <alexturn> q+
19:40:08 <cabanier> Lachlan_Ford: the way you describe it, the user can improve upon the baseline
19:40:16 <cabanier> bajones_: it's often a sanity check
19:40:36 <cabanier> ... we're not advocating an autorative hand
19:41:02 <cabanier> Manishearth: we aren't standardizing anything. It's just a JS library
19:41:28 <yonet> ack cabanier
19:41:30 <cabanier> ... we're saying: we just put a mesh here and people can standardize on that
19:41:56 <Manishearth> cabanier: so i think we talked a little about we don't wanna use this model all the time otherwise we get "claw hands"
19:42:09 <Manishearth> ... i think we only are going to show hands as controllers only if the author opted in to joints
19:42:45 <Manishearth> cabanier: e.g. on oculus you have controllers and put them down and use their hands
19:43:17 <Manishearth> ... rn you HAVE to pick up controllers to use webxr
19:43:30 <Manishearth> ... unless the user changes some settings
19:44:02 <Manishearth> ... i nthe future we can make it so that ONLY IF the author requests hand-input
19:44:09 <Manishearth> alexturn: maybe we should have this in the spec
19:44:31 <Manishearth> alexturn: we might wanna make clear that UAs intend to operate this way
19:44:33 <Manishearth> q?
19:44:39 <Manishearth> cabanier: will open an issue
19:44:43 <yonet> ack nick-8thwall
19:45:08 <Manishearth> nick-8thwall: just responding to something alexturn said quite a while back: was talking about how we'd enable ar as well as vr for rendering a mesh
19:45:19 <Manishearth> ... for face effects even though you have a high res mesh
19:45:26 <Manishearth> ... it's easier to expose a landmark api
19:45:40 <Manishearth> ... e.g. when you talk about setting a finger to a location
19:45:54 <Manishearth> ... when you have points on the mesh, you have some special landmarks
19:46:14 <Manishearth> alexturn: and yeah that's the api that we'll have first in webxr
19:46:23 <Manishearth> .. you get joints plus the tip
19:46:51 <yonet> ack Manishearth
19:46:51 <Manishearth> ... for collider purposes apps would use joints, and rigging is mostly for rendering
19:47:12 <cabanier> Manishearth: like Alex said, there is no mesh API
19:47:20 <cabanier> ... it's just a mesh so you can skin the hand
19:47:31 <cabanier> ... we only expose joints. 25 points per hand
19:47:55 <cabanier> ... what oculus and Hololens do is different
19:48:17 <cabanier> ... even if the content doesn't opt into hand, you should expose a hand controller
19:48:38 <cabanier> Lachlan_Ford: there are cases where you want to render the hand
19:48:46 <cabanier> Manishearth: not rendering a controller
19:49:06 <cabanier> alexturn: people will render the gltf
19:49:26 <cabanier> ... any code that uses motion controller is going to have to be updated to the hand model
19:49:56 <cabanier> ???
19:50:33 <yonet> ack alexturn
19:50:34 <cabanier> the ua would remove direct delivery for the accessibility feature
19:51:10 <cabanier> alexturn: I'm torn if the default hand mesh in the input profiles will create confusion
19:51:32 <cabanier> ... you can roll your own hand.
19:51:45 <cabanier> ... maybe it makes more sense to put it in its own location
19:52:16 <cabanier> yonet: any updates from anyone?
19:52:26 <cabanier> kip: my last day with mozilla is this week
19:52:41 <cabanier> ... but my github name will stay the same
19:52:45 <atsushi> i/yonet: any updates from anyone?/topic: AOB
19:53:03 <cabanier> ... other people from mozilla might join especially from the hubs team
19:53:39 <cabanier> ... if you want face-to-face call especially lighting estimation I'll be happy to create some time for that
19:57:48 <alexturn> present
19:57:49 <alexturn> present+
20:01:30 <atsushi> i/cabanier: so from the oculus perspective,/scribenick: Manishearth/
20:01:48 <atsushi> i/cabanier: when it comes to the name, right now when you expose hands/scribenick: Manishearth/
20:02:07 <atsushi> i/bajones_: addressing nick's concerns: the facial mesh/scribenick: Manishearth/
20:02:16 <cabanier> atsushi: thanks for patching up the minutes!
20:02:29 <atsushi> i/Manishearth: I wanted to mention to Nick/scribenick: cabanier/
20:02:47 <atsushi> i/cabanier: so i think we talked a little about/scribenick: Manishearth/
20:03:03 <atsushi> i/Manishearth: like Alex said,/scribenick: cabanier/
20:03:12 <atsushi> rrsagent, publish minutes v2
20:03:12 <RRSAgent> I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
20:03:20 <atsushi> np ;)
20:06:33 <atsushi> s/e.g. in AR it is critical the app match the true size of the hand, in VR there is more flexibility/... e.g. in AR it is critical the app match the true size of the hand, in VR there is more flexibility/
20:06:58 <atsushi> s/--- and each mesh should be relative to a wrist/... and each mesh should be relative to a wrist/
20:07:28 <atsushi> s/the ua would remove direct delivery for the accessibility feature/... the ua would remove direct delivery for the accessibility feature/
20:07:38 <atsushi> rrsagent, publish minutes v2
20:07:38 <RRSAgent> I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
20:08:45 <atsushi> present+ nick-8thwall, yonet
20:08:46 <atsushi> rrsagent, publish minutes v2
20:08:46 <RRSAgent> I have made the request to generate https://www.w3.org/2020/09/22-immersive-web-minutes.html atsushi
20:09:33 <atsushi> rrsagent, bye
20:09:33 <RRSAgent> I see no action items