16:58:35 RRSAgent has joined #immersive-web 16:58:39 logging to https://www.w3.org/2024/03/26-immersive-web-irc 16:59:38 cabanier has joined #immersive-web 16:59:50 rrsagent, do not start a new log at midnight 16:59:52 present+ 16:59:58 rrsagent, make log public 16:59:59 lgombos has joined #immersive-web 17:00:04 rrsagent, publish minutes 17:00:05 I have made the request to generate https://www.w3.org/2024/03/26-immersive-web-minutes.html atsushi 17:00:14 present+ Laszlo_Gombos 17:00:20 meeting: Immersive Web WG/CG 2024/March face-to-face Day 2 17:01:00 present+ 17:01:12 cabanier has joined #immersive-web 17:01:12 NicolaiIvanov has joined #immersive-web 17:01:12 bemfmmhhj has joined #immersive-web 17:01:12 matrix638 has joined #immersive-web 17:01:12 NickFnBlum has joined #immersive-web 17:01:12 Manishearth has joined #immersive-web 17:01:12 maxass99 has joined #immersive-web 17:01:12 ada has joined #immersive-web 17:01:12 cwilso has joined #immersive-web 17:01:12 iank_ has joined #immersive-web 17:01:12 scheib has joined #immersive-web 17:01:12 alcooper has joined #immersive-web 17:01:28 Omegahed has joined #immersive-web 17:01:40 present+ 17:02:16 yonet has joined #immersive-web 17:02:44 Zakim has joined #immersive-web 17:02:46 <[old]freshgumbubbles> [old]freshgumbubbles has joined #immersive-web 17:02:47 present+ 17:02:59 present+ 17:03:00 present+ 17:03:02 present+ 17:03:03 present+ 17:03:03 present+ 17:03:04 present+ 17:03:17 present+ 17:03:22 webirc54 has joined #immersive-web 17:03:24 rrsagent, publish minutes 17:03:25 I have made the request to generate https://www.w3.org/2024/03/26-immersive-web-minutes.html atsushi 17:03:31 yonet has changed the topic to: https://github.com/immersive-web/administrivia/blob/main/F2F-March-2024/schedule.md 17:03:33 marisha has joined #immersive-web 17:03:45 mblix has joined #immersive-web 17:03:54 present+ 17:03:56 bialpio has joined #immersive-web 17:04:02 present+ 17:04:03 bajones has joined #Immersive-Web 17:04:15 present+ 17:04:20 nick-niantic has joined #immersive-web 17:04:26 webirc54 has left #immersive-web 17:04:27 agenda: https://github.com/immersive-web/administrivia/blob/main/F2F-March-2024/schedule.md 17:04:51 giorgio has joined #immersive-web 17:06:59 Brandel has joined #immersive-web 17:07:03 present+ 17:08:01 topic: Progress on the spec (immersive-web/model-element#78) 17:08:07 rrsagent, publish minutes 17:08:08 I have made the request to generate https://www.w3.org/2024/03/26-immersive-web-minutes.html atsushi 17:08:14 https://github.com/immersive-web/model-element/issues/78 17:08:14 present+ 17:08:18 zakim, pick a victim 17:08:18 Not knowing who is chairing or who scribed recently, I propose mblix 17:08:39 chair: ada 17:08:45 etienne has joined #immersive-web 17:10:14 rik: recently looking at spec, and didn't look like much recent progress recently, any updates? 17:11:00 q+ 17:11:41 Brandel: was recently looking at this. currently have POC implementation in safari but its somewhat old, 2 years old. Looking at now how to make more concrete proposals with educated proposals for implementation, within the last month or two 17:12:19 SergeyRubanov has joined #immersive-web 17:12:23 marisha has joined #immersive-web 17:12:25 present+ 17:13:32 Brandel: Focused on the standards body as well to move forward on the model format support, as well as MaterialX as a way of handling materials for models, to resolve some questions and clarifying 17:13:54 cabanier: should we close a lot of the many opened (old) issues created in the model element repo? 17:14:05 q? 17:14:14 ack Leonard 17:14:29 Brandel: we could potentially do that, often the backlog of GH issues may not be that relevant 17:14:38 q+ 17:15:40 q+ 17:16:07 Leonard: we didn't want to expose an API that provided head tracking data for privacy reasons which is one reason the model element was proposed and created. still skeptical that need model element when alternatives available 17:17:02 sentinel1975 has joined #immersive-web 17:17:49 Brandel: stereoscopic display of 3D model is a significant advantage in a privacy concious way is an advantage, in addition to having the platform take care of rendering has other benefits. This is the perspective of Apple. 17:18:32 Leonard: Are there alternatives that can achieve this, such as by additional specs? Also that could move faster than OS/browser updates 17:19:49 Brandel: no, I don't think alternatives are viable with the privacy + platform advantages of rendering on platform: reflections, stereo, etc. Stereo is a big aspect. don't see a way JS could do that (without privacy concerns) 17:20:21 q? 17:20:23 Leonard: would like to see concerns more detailed in writing, something to point to as reason why 17:20:25 ack bajones 17:22:06 babage has joined #immersive-web 17:22:22 bajones: question about formats, where is the discussion being made. There was a recent meeting in bay area of a format organization talking about formats. Meeting saw benefit of both gltf and usdz in terms of delivery 17:22:29 Note to group from a previous Ada statement: Khronos is looking into procedural textures based on MaterialX 17:23:36 ack alcooper 17:23:37 https://github.com/immersive-web/model-element/issues/70 17:23:50 Brandel: upcoming goal is to figure out interop more, but skeptical of how gltf could be embedded in usdz which some have proposed 17:24:32 A lot of investigation between glTF / USD interoperability is being done in the Metaverse Standards Format. 17:24:36 q? 17:24:43 alcooper: can we get more in the spec itself on reasons for model element in terms of privacy, echoing Leonard 17:25:06 topic: https://github.com/immersive-web/model-element/issues/65 17:25:50 s|https://github.com/immersive-web/model-element/issues/65|High level scene graph manipulation API (immersive-web/model-element#65) 17:27:39 https://matrix.org/blog/2023/06/07/introducing-third-room-tp-2-the-creator-update/ 17:27:44 giorgio: there was a web scene graph api proposal, with attempt to have w3c take over 17:28:01 OverTime has joined #immersive-web 17:28:51 giorgio: wanted to bring up this proposal, and see if could be a springboard for usage/inspiration when controlling a model element 17:30:10 Brandel: scene graph manipulation makes sense, we definitely want, but in context of trying to ship a v1, suggests that we should see how far we can get without taking on this more complicated topic that would be hard to standardize 17:30:48 q? 17:31:32 Brandel: there is room to experiment and play with a lot of possibilities and alternatives on what a scene graph could be with this model element context, rather than trying to standardize at this time 17:32:21 SintayewGashaw has joined #immersive-web 17:32:25 q+ 17:32:26 bialpio has joined #immersive-web 17:32:45 q+ 17:33:06 ack Leonard 17:33:12 Brandel: can come back to this later, after figure out other model tag issues 17:34:05 ack cabanier 17:34:14 Leonard: agree that can wait, and also that this is likely the forum to handle this discussion and development, verses other things like the Metaverse standards forum etc 17:34:28 nick-niantic has joined #immersive-web 17:34:41 q+ 17:35:05 cabanier: doesn't standardizing scene graph conflict with having a model element that supports formats from gltf to usdz to others? 17:36:08 LawrenceKincheloe has joined #immersive-web 17:36:47 Brandel: the formats and what they intrinsically do share enough in that a common scene graph would be able to work / be compatible. converter apps that mostly work demonstrate this, preserving most things 17:37:20 ack nick-niantic 17:37:24 q+ nick-niantic 17:37:38 Brandel: there are a lot of formats, it might be useful to have the UA decide what to support. Don't see any inherent issues 17:37:58 q+ 17:38:15 cabanier: why not just do usdz verses supporting multiple formats? 17:38:27 q+ 17:38:43 q+ Brett 17:39:52 Brandel: different user agents can support different formats. There will be questions of what exactly different UAs and devices and what formats (or subsets) the can support. I.e. maybe not the full usd format might not make sense anytime soon 17:40:48 Brandel: usd has a lot of complicated features, they could be differentially supported 17:40:49 ack nick-niantic 17:41:12 Can we all mute notifications please, it's very distracting 17:41:23 helixhexagon has joined #immersive-web 17:42:26 nick-niantic: Apple has been implicitly developing opinions of 3d models/formats via their visionOS/realitykit work. Is there a world in which there is a realitykit for the web? 17:43:03 helixhexagon has joined #immersive-web 17:43:03 LawrenceKincheloe has joined #immersive-web 17:43:03 SintayewGashaw has joined #immersive-web 17:43:03 OverTime has joined #immersive-web 17:43:03 babage has joined #immersive-web 17:43:03 sentinel1975 has joined #immersive-web 17:43:03 SergeyRubanov has joined #immersive-web 17:43:03 <[old]freshgumbubbles> [old]freshgumbubbles has joined #immersive-web 17:43:03 cabanier has joined #immersive-web 17:43:03 NicolaiIvanov has joined #immersive-web 17:43:03 bemfmmhhj has joined #immersive-web 17:43:03 matrix638 has joined #immersive-web 17:43:03 NickFnBlum has joined #immersive-web 17:43:03 Manishearth has joined #immersive-web 17:43:03 maxass99 has joined #immersive-web 17:43:03 ada has joined #immersive-web 17:43:03 cwilso has joined #immersive-web 17:43:03 iank_ has joined #immersive-web 17:43:03 scheib has joined #immersive-web 17:43:03 alcooper has joined #immersive-web 17:43:53 q? 17:44:19 Brandel: There is the question on what the various levels of the stack to standardize on. Like MaterialX for matrials props for example. We don't want to overindex on an opaque system black box 17:44:41 rzr has joined #immersive-web 17:45:12 nick-niantic: with the model element proposal it has some build in limitations based on how embeeded in page, in a browser. etc. Maybe there's a way we could start from a different point, that doesn't have the same implied limitations 17:45:27 ack cwilso 17:45:48 Brandel: it's an interesting thought that could have something work in a different way, like a web realitykit 17:46:21 vianka has joined #immersive-web 17:47:49 jfernandez has joined #immersive-web 17:48:07 cwilso: the reason model-viewer started with a simple API that didn't have a scene graph and kept simple was to keep it high level. scene-graph level is harder to pin down. Formats effect the scene graph in a big way 17:48:11 ack bajones 17:48:13 sfloradon has joined #immersive-web 17:49:34 bajones: there are tools that make formats interoperate, so could get to a good middle ground. it would be opinionated (a scene graph api), and this is good. Likely would be influenced by whatever format is being implemented and supported first 17:50:53 q+ 17:51:37 webirc45 has joined #immersive-web 17:51:49 KevinBush has joined #immersive-web 17:51:51 bajones: so there is a concern that even unintentionally could tie too much to shape of the first format. This is not without benefits to getting something working sooner, but could be backed into a corner where one file format is essentially or in part made the best format to use. and we would like to avoid that 17:53:17 q+ 17:53:18 bajones: if everyone just used one format i.e. usdz that would be one thing, but vendors don't necessarily want to support this format, they want to use other formats. 17:53:49 etienne has joined #immersive-web 17:54:38 m-alkalbani has joined #immersive-web 17:55:01 bajones: different platforms have built in support for formats like usdz and can make an easy choice to support that, but other platforms don't have that and would like to support something like gltf that was designed as a transmission format. we can't take for granted that all platforms will even eventually get usdz support 17:56:07 bajones: there is early discussion to make usd into a more web centric format. Wants to hear about issues of supporting gltf in usdz, etc. 17:56:13 q- 17:56:18 general_j has joined #immersive-web 17:56:32 q+ 17:56:37 Brandel: one issue is licensing of gltf implementers licence, for instance 17:58:49 Brandel: in a technical level, gltf is simple but doesn't support some use cases involving variants, something supported with usd that can do things more conditionally based on platform capabilities and choices 17:59:51 Brandel: want to support a format that has that capability verses modifying gltf for example to add that capability. 18:00:46 bajones: Examples of conditionality: loading a model in VR would show the build in lights of model, loading in AR would want to ignore and use the real world (inferred) lighting 18:01:02 https://remedy-entertainment.github.io/USDBook/terminology/LIVRPS.html 18:01:48 Brandel: the format plays a part in that discussion. Choosing a format that has laying mechanisms that support uses cases, verses a format like gltf that doesn't or might not have this support currently 18:03:17 bajones: want to understand more specifics of how much of this is required by model format, verses what a page itself could manage. It would be great to have examples 18:05:08 Brandel: Sharing one example where one usdz downloaded from a webpage could itself switch out asset parts dynamically with the platform making decisions on what to do 18:05:47 Chrysippus has joined #immersive-web 18:05:51 ack Brett 18:05:52 bajones: Does the format usd, have this conditionality of if this context do this, else if this context do that? 18:08:13 Brett: Will the model element ever be to moved in front of the webpage in a XR browser context? What other capabilities will be available like multiple models for one model element? 18:08:36 ack nick-niantic 18:08:52 Brandel: not part of the proposal today / at this point. More of a timeline question, focus of getting out a v1 that resolves core uses 18:11:20 nick-niantic: webxr is a low level api, and things like realitykit and model are higher level. where is the potential for poly filling support using webxr (lower level) ? 18:12:02 AnthonySpencer has joined #immersive-web 18:12:26 joraboi445 has joined #immersive-web 18:12:42 Brandel: higher level API helps prevent from over-indexing on assumptions related to formats and features of one platform's 3D 'kit' like realitykit, etc 18:14:43 Brandel: example of webGPU. webGPU benefited from having multiple competting but similar low level APIs to abstract from. For now it doesn't seem like there are multiple platforms with things like RealityKIt, so it's missing the fertile ground to help with that abstraction effort mentally 18:14:57 etropea73101 has joined #immersive-web 18:15:12 ack Leonard 18:15:41 bajones: webGPU took 7 years as well, which is a long time. And has some differences in what was trying to handle for its v1 18:16:12 Leonard: want more information on gltf license issue, want to follow up on 18:17:20 Matthewaway has joined #immersive-web 18:21:00 present+ 18:21:34 present+ 18:22:08 dietrich has joined #immersive-web 18:23:28 fernansd has joined #immersive-web 18:28:02 bialpio_ has joined #immersive-web 18:29:27 bialpio has joined #immersive-web 18:29:32 present+ 18:29:55 marisha has joined #immersive-web 18:31:35 Brett__1 has joined #immersive-web 18:33:14 bajones has joined #Immersive-Web 18:35:40 webirc99 has joined #immersive-web 18:35:51 topic: https://github.com/immersive-web/model-element/issues/71 18:37:06 etienne has joined #immersive-web 18:37:20 Scribe: etienne 18:37:30 Brandel has joined #immersive-web 18:37:36 present+ 18:38:22 present+ 18:38:32 https://github.com/immersive-web/model-element/issues/71 18:39:21 Brandel: Image Based Lighting is very neat and important. Currently we see Model as a portal on a page. And people want to light it in very specific ways. 18:39:32 atsushi has joined #immersive-web 18:39:37 ... we initially wanted to define this on a "root element" but got push back internally 18:40:00 q? 18:40:00 ... still think we should specify this. either as a JS API or ideally in a declarative way. 18:40:09 rrsagent, publish minutes 18:40:10 I have made the request to generate https://www.w3.org/2024/03/26-immersive-web-minutes.html atsushi 18:40:16 q+ 18:40:32 ... there's merit to having a child object inside the tag but don't mind it if we put everything on the top level model element 18:40:35 ack bajones 18:40:46 bajones: I agree that it is neat. 18:42:07 No embed with IBL - no extension in progress 18:42:07 ... of all the options you outlined (all fine), I'm wondering if we could embed these in some of the model formats themselves (USD or an GLTF extension) 18:42:07 ... why should it be separate form the model itself? 18:42:18 i/rik: recently looking at spec, and didn't/scribe: mblix 18:42:24 q+ 18:42:24 lgombos has joined #immersive-web 18:42:30 Brandel: you might want to use different IBL for the same model 18:43:03 ... MaterialX also has the ability to define IBL, but for simplicity it makes sense to specify the IBL separately 18:43:10 bajones: no complaints 18:43:10 ack Leonard 18:43:29 Leonard: in the GLTF case lighting is part of the environment and not the model, for the same reasons Brandel mentioned 18:43:50 ... IBLs are very specific, even changes based on the time of day 18:44:36 Brandel: for USD too, we have Quick Look, and I don't think anybody would want custom backed in IBLs when showing the object in your environment 18:44:54 ... that said the MaterialX route would override this and could be weird (and fun!) 18:45:57 Brandel: bajones said he's less worried about formats here 18:46:08 ... and there was talk at TPAC around HDR image formats too 18:46:10 q? 18:46:19 ... should be part of the broader conversation 18:46:25 topic: https://github.com/immersive-web/model-element/issues/72 18:46:50 s|GLTF|glTF| 18:46:53 webirc99 has left #immersive-web 18:47:23 yonet has joined #immersive-web 18:47:33 Brandel: the PoC implementation of model in iOS/macOS has a camera pitch/yaw/scale 18:47:41 ... inspired by Quick Look 18:47:56 ... but I don't know that it is all that poweful 18:48:12 ... so I've been looking at simply providing a transform 18:48:23 webirc99 has joined #immersive-web 18:48:31 ... it's not a camera but it is moving around the root element, and gives people the ability to pan or orbit 18:49:10 ... while it's appealing to try to make a simpler version for simple models, but for anything more complicated the "simple case" implementation would get in the way 18:49:16 yonet3 has joined #immersive-web 18:49:17 webirc99 has left #immersive-web 18:49:39 ... so I suggest that we use a transform (applying to the "stage"), in the context of a spatial display 18:49:52 q+ 18:49:53 q+ 18:50:02 ... and maybe even that we _do not_ provide basic user interactions 18:50:13 ... since it might not always be appropriate 18:50:21 ... that's my suggestion, wonder what people think 18:50:22 ack cabanier 18:50:42 cabanier: if you manipulate the transform, can you move the model out of the viewport of the portal? 18:50:59 Brandel: yes it's functionally moving the content inside of the view, like an SVG viewport 18:51:02 ... you can make mistakes 18:51:25 ... viewBox (SVG) can be used for incredible things 18:51:52 ... while I'm not proposing a SVG-like scenegraph for now, aligning here would make sense and would be future proof 18:52:00 ... that's why it's not a camera 18:52:20 cabanier: you also mentioning not having default orbit controls, but it sounds like an important thing most people want 18:52:42 Brandel: but the default case is in conflicts of what people would want for more complex models 18:53:08 ... and the default orbit controls are easy to port to the transform, a couple of lines of code 18:53:22 ... and the simplest case might come at the cost of the more complicated one 18:53:26 ... I could be wrong! 18:53:42 cabanier: does USD define where your pivot point is? I think in gLTF you can 18:53:57 Brandel: all 3D files have a system origin, it's up to people to be sensible 18:54:05 cabanier: but that's not the point of pivot for orbit 18:54:24 Brandel: I think most people just use bounding box center etc... 18:54:41 ... some people have _logic_ to find the least silly pivot point 18:54:56 ... but I would expect people putting assets on a page to know where this point should be 18:55:16 ... and without scenegraph inspection we couldn't figure it out 18:55:39 cabanier: I think FBX (AutoDesk) let's you define it 18:56:05 q+ 18:56:07 Brandel: without jumping back into formats... that might be a useful capability to look at and bring back to other formats 18:56:13 ack bajones 18:56:14 ... it sounds neat 18:56:41 bajones: the idea of setting a transform programmatically is a baseline good idea 18:56:46 ... it's going to be extremely useful 18:57:24 ... but if it's strictly required, then we're back into a strictly imperative proposal 18:57:29 ... it removes some of the appeal 18:57:43 ... it also brings up a question about scale, and what the default presentation of the object is going to be 18:58:09 ... if the object itself doesn't define it, you're probably going to have a pretty strict default, that might but you _in_ the model 18:58:26 ... most people will probably expect to zoom out and fully frame the object, for which you need to determine the scale 18:58:53 ... gLTF you have ways to figure it out (Meta had extensions proposal) 18:59:02 ... but without the Scene Graph we're not giving people a way to do that 18:59:25 ... that feels like a lot of work to do ahead of time, people might just go back to model-viewer, you might loose some of target audience 18:59:36 ... no concerns about the capability, it's a powerful thing 18:59:55 ... it opens up possibilities without going into SceneGraph API 19:00:11 ... note: gLTF can include cameras, but models don't always include one 19:00:41 ... other node: people are not always responsible, some models are not well designed 19:00:52 ... "it looked good in my tool" 19:01:21 q+ 19:01:24 ... some of this might be hurdle for people who don't have intimate knowledge of the model itself (think SketchFab) 19:01:46 ... they do a lot of processing anyway, but there are situation where you don't have perfect knowledge about the model 19:01:59 ... you need a way to toss it to the browser and ask for "reasonable defaults" 19:02:20 Brandel: that's all very reasonable. in a Spatial Context, we still get stereo which is neat and better than an image 19:02:37 ... I could be convince that we need a default behavior, but "interactive" might not be a good name for it 19:02:56 ... Quick Look sets the pivot to the bounding box center of the first frame I think, and it's often wrong 19:03:13 bajones: Brandel, in agreement: it's still the least worst guess 19:03:35 Brandel: if we can agree on the default behavior it seems reasonable 19:04:08 bajones: there's still a leap from the default behavior (sometimes wrong) and something slightly more complicated 19:04:28 ... it'll require some basic introspection (dimension and location of the model) 19:04:51 ... it would be nice if we could expose a model min max to facilitate this, maybe just the first frame min max 19:05:05 ... it'll let people pick up from where the default stops 19:05:24 Brandel: there's probably some basic high level scene details that can be shared 19:05:47 bajones: I'm fine with that, and it side steps some of the full Scene Graph API issues. and it's read only 19:06:09 Brandel is a peace with this 19:06:14 s/a/at 19:06:16 q? 19:06:20 ack Leonard 19:06:28 Leonard: there was some mention of FBX as a possible file format 19:06:40 Brandel: just as an example of a strategy 19:06:48 q? 19:06:53 Leonard: there is no future to FBX according to AutoDesk 19:07:01 ack Brett__ 19:07:22 Brett__1: (in person demo), asking about moving model elements via a HTML attribute 19:08:08 marisha: imperatively via JS or via an attribute 19:08:09 ? 19:08:25 bajones: since it's read-only you kind of have to use JavaScript 19:08:37 ... not sure how much we want to go into CSS things there 19:08:44 ... might not be well suited for this environment 19:09:01 Brandel: I'm sympathetic to the idea of using CSS here 19:09:11 ... the imperative API will help us figure out the shape of things 19:09:37 ada: in Brett__1's example you would lookup the bounding box then figure out the transform, in JS 19:09:54 ... to put the element in view 19:10:04 bajones: but that would be the default behavior too 19:10:19 Brett__1: is this an issue with units? 19:10:27 q+ 19:10:38 Brandel: one of the challenge of Model is that it bridges two modalities: the DOM and the model 19:10:46 ... currently discussing things "inside of the model world" 19:10:57 ... CSS top left might not map directly to this 19:11:22 Brett__1: could I shift the vertices while ingesting the model? 19:11:34 Brandel: we're proposing the model as an opaque format for now 19:11:37 You cannot translate on read because of the need to account for scale and rotation. 19:11:54 ... you would need a parser for this 19:12:05 bajones: could be done as an offline process too 19:12:18 ... you would update it before showing it online 19:12:44 Brett__1 (more in-person demo, waving a cookie around) 19:13:19 bajones, ada... discussing CSS units and their weirdness when using a DOMMatrix 19:13:41 bajones: generic 3D environments are typically unitless 19:13:52 ... we could use 1 unit = 1 meter like WebXR 19:14:00 ... but it wouldn't be a good fit for all models 19:14:07 eddy_wong has joined #immersive-web 19:14:11 ... so CSS couldn't easily operate in that context 19:14:22 ... and percentages are hard, because the universe has no clear bounds 19:14:32 Brett__1: but the box will have dimensions 19:14:38 I'd like to raise my hand (sorry first time raising my hand) 19:14:41 bajones: units 19:14:46 q+ eddy_wong 19:15:09 bajones: it generally doesn't matter until you bring it in the real space 19:15:35 bajones: Brandel said it well, manipulating this through CSS is probably not the right place to start 19:15:43 Brandel: all about timelines, we have to start simple 19:15:54 ack bajones 19:16:30 bajones: might be a different topic, but talking about the cameras... that's one half of the equation, the other is the projection property of the viewport itself 19:16:41 ... any thoughts? probably want a reasonable set of defaults too 19:16:50 ... do we want to allow that to be manipulated? 19:17:27 Brandel: the view transforms all the way to ground truth actually leaks where the user is looking 19:17:32 ... so we don't want to expose that 19:17:46 ... one of the rationale for "not providing a camera" is because the camera is your eyes 19:18:01 ... but when not presented spatially it might be useful to have full cameras 19:18:17 ... not sure when it is appropriate 19:18:36 bajones: wasn't thinking about the spatial use case, I see how you derive a lot of information from that 19:18:43 ... but that won't be what most people use 19:19:01 ... so we need a way to line up what I do on a flat page and what I see in a spatial context 19:19:09 ... these shouldn't be completely separate 19:19:25 ... in the flat page baseline, I need some sense of the field of view 19:19:46 ... if it's very narrow I'll have to push the model back a lot, if it's really wide I'll position the model differently 19:19:59 ... I also want a vague idea of where the near field is 19:20:16 ... I'd like to avoid having a "magic value" of 1m, that might actually be different device to device 19:20:48 ... for people to intelligently frame their objects, they need at least to know the defaults 19:21:05 ... we need people to have _some_ context of how the viewport will be presented 19:21:23 Brandel: the near clipping plane is the view plane in our implementation (nothing comes out) 19:21:50 bajones: sure, but we can't put the near plane at 0 (because maths) 19:22:06 ... so presumably you'll have some small value for the near z, we can decide on 19:22:19 Brandel: but the camera is your eye (so the z clipping plane is not at 0) 19:22:27 bajones: you're thinking spatial only again 19:22:50 Brandel: don't have strong feelings right now about the non-spatial display environment 19:23:03 ... and I would personally want to play with FoV etc... 19:23:37 ... the web has "real world units", the window increase in pixel size when you resize it on visionOS too 19:23:55 ... but we don't want people to know how big the window is (in real world units) 19:24:20 +q 19:24:21 ... so we can probably imply defaults 19:24:28 bajones is going to draw on the board 19:24:40 ... fully onboard with not exposing private information 19:24:55 ... but people should be able to express things across modalities 19:27:03 ... first example: non spatial rendering, the browser picks a "eye" to setup the rendering of the model, the author wants it positioned inside of the view's frustum 19:27:15 ... don't care about the physical size of the element actually 19:27:32 ... now the spatial example, with someone's actual eye 19:27:51 ... now we are looking through the page, it's a great effect! 19:28:12 ... but as an author I don't know the view's frustum anymore, and I maybe only tested in a non-spatial context 19:28:29 ... so now the model might not be framed properly 19:28:56 ... so while the effect is cool, we need to mitigate the fact that it might not be setup properly 19:29:12 ... I need to know enough about the environment (in spatial mode) to frame the element 19:29:42 ... probably want to frame my object inside a "magic box" (drawing will be in the minutes) 19:29:59 ... this is challenging but it can be done in a privacy preserving way 19:30:16 ... and I'd like the logic to be mostly the same in the spatial and non-spatial modes 19:30:25 Brandel: this is very useful 19:30:47 ... in the non-spatial mode, scrolling the page might move the view frustum 19:31:01 ... or devices with a webcam could potentially do this too 19:31:30 ... to make things "more spatial" in non-spatial mode 19:32:13 ... in practice the bounding box might be enough to reasonably position the model (in spatial mode) 19:32:23 ... because the page always points at you (on visionOS) 19:32:41 ... not proposing dynamic sheering of the view frustum but it might be an interesting idea 19:32:50 (for non-spatial context) 19:33:29 bajones: the key thing here is that in the non-spatial case, people don't know things are wrong (badly positioned for spatial) 19:33:53 ... we need to nudge people toward doing the right thing for spatial when viewing in non-spatial 19:34:23 Brandel: in principle I don't object to exposing the whole view transform when non-spatial 19:34:52 ... and I agree that it would be good for people to act more consistently between the two contexts 19:35:11 bajones: this goes back to earlier discussions about "magic windows" 19:35:16 ... we should look into past minutes 19:35:19 ack eddy_wong 19:35:42 eddy_wong: want to share some learnings from my prototypes 19:35:55 yonet has joined #immersive-web 19:36:02 ... I prototyped object-fit, like I would use for an image 19:36:05 bajones has joined #Immersive-Web 19:36:11 Leonard has joined #immersive-web 19:36:27 ... the model tag bounds exactly fits the model's bound 19:37:05 ... the benefit is that the web author doesn't need to do the maths, just object-fit: contains, then they can query the transform and manipulate it 19:37:36 ... rotating, scaling etc... 19:38:12 ... so without introspection on the model's bounds, the author can rely on the initial transform (due to the CSS properties) 19:38:23 ... wanted to share this learning, might trigger more ideas 19:38:36 ... and what we see on the whiteboard is a great problem 19:38:47 q+ 19:38:55 ... in our mind the model is still a 2D thing (width: 100px, height: 100px), should it have a depth? 19:39:06 ... would work well with the CSS object-fit 19:39:29 ... and people would be guaranteed the position/scale of the model would be reliable 19:39:45 ... so that would be my answer here, adding depth, but it's a very good question! 19:39:56 ack Brett__ 19:40:13 q+ 19:40:14 Brett__1: the whiteboard illustrates a problem with 3D CSS too 19:40:28 Brandel0 has joined #immersive-web 19:40:53 ack bialpio 19:40:56 ... the non-spatial stuff is hard 19:41:23 bialpio: is allowing the web developer to introspect the model transform risky? 19:41:33 ... could I craft something that would expose head pose 19:42:01 Brandel: maybe, the place where we do the computation probably wouldn't have head pose, just access to the scene geometry 19:42:19 ... shouldn't be not too hard to give a privacy preserving worse answer 19:42:34 bajones: you don't need it to be perfect from all angles 19:42:42 ... just from a reasonable angle / distance 19:43:13 bialpio: I'm doing this from JS and I can create many models 19:43:33 bajones: the fit fonction should be stable even while you scroll etc... 19:43:42 ... optimizing for the middle of the page, straight on etc... 19:43:57 bialpio: once I have the scroll position, and I create another element on top... 19:44:06 bajones: still think we'd only use the model bounding box 19:44:37 Brandel: maybe some timing attacks? putting things that are expensive out of the frame? 19:44:45 ... but there's probably no stopping that 19:44:56 ada: maybe we shouldn't do view based culling 19:45:24 bajones: this is not a new problem for this context, we see this with WebGL too 19:45:49 bajones: we wanted textures that you could sample from but not read-back 19:45:55 ... but people could still do timing attacks 19:46:30 ... that's the reality of using the GPU 19:46:52 Brandel: that's an added reason for putting a firewall between the web content process and what's doing the rendering 19:47:13 ... anything can do a timing attack but you don't have the granularity here 19:47:28 ... more fine grained introspection might bring up new issues 19:47:31 ack nick-niantic 19:48:10 nick-niantic: as we're looking into having the model shrink to the appropriate size, it reminded me of stuff we do for our AR system 19:48:39 ... not physically accurate but more useful. could we scale the content based on the distance of the viewer to the page? 19:49:01 Brandel: making an initial computation based on the bounding box is adequate 19:49:27 ... having a mapping from screen units to meters is probably the best we can do there, not based on the actual distance from the window 19:49:40 ... visionOS expects everything to me 1.5 meters away 19:49:48 q? 19:50:28 I won't join immediately afterwards - another meeting 19:50:56 rrsagent, publish minutes 19:50:58 I have made the request to generate https://www.w3.org/2024/03/26-immersive-web-minutes.html atsushi 20:38:17 atsushi has joined #immersive-web 20:44:30 bajones has joined #Immersive-Web 20:50:44 present+ 20:50:45 present+ 20:50:49 present+ 20:50:50 present+ 20:50:51 present+ Laszlo_Gombos 20:50:55 etienne has joined #immersive-web 20:51:04 rrsagent, publish minutes 20:51:05 I have made the request to generate https://www.w3.org/2024/03/26-immersive-web-minutes.html atsushi 20:51:12 present+ 20:51:23 rrsagent, who is here? 20:51:23 I'm logging. Sorry, nothing found for 'who is here' 20:51:26 Brandel has joined #immersive-web 20:51:29 zakim, who is here? 20:51:29 Present: cwilso, yonet, alcooper, ada, Omegahed, Leonard, cabanier, atsushi, marisha, bialpio, bajones, Brandel, mblix, m-alkalbani, jfernandez, Laszlo_Gombos, etienne 20:51:32 On IRC I see Brandel, etienne, bajones, atsushi, Leonard, eddy_wong, lgombos, Brett__1, bialpio, fernansd, dietrich, Matthewaway, etropea73101, joraboi445, AnthonySpencer, 20:51:32 ... Chrysippus, general_j, KevinBush, sfloradon, jfernandez, vianka, rzr, alcooper, scheib, iank_, cwilso, ada, maxass99, Manishearth, NickFnBlum, matrix638, bemfmmhhj, 20:51:33 present+ 20:51:35 present+ 20:51:36 ... NicolaiIvanov, cabanier, [old]freshgumbubbles, SergeyRubanov, sentinel1975, babage, OverTime, SintayewGashaw, LawrenceKincheloe, helixhexagon, nick-niantic, mblix, Zakim, 20:51:36 ... Omegahed 20:51:44 matlu has joined #immersive-web 20:51:53 present+ 20:52:01 zakim, choose a victim 20:52:01 Not knowing who is chairing or who scribed recently, I propose bajones 20:52:15 marisha has joined #immersive-web 20:52:26 present+ 20:52:28 zakim, choose a victim 20:52:28 Not knowing who is chairing or who scribed recently, I propose Brandel 20:52:32 zakim, choose a victim 20:52:32 Not knowing who is chairing or who scribed recently, I propose etienne 20:52:35 zakim, choose a victim 20:52:35 Not knowing who is chairing or who scribed recently, I propose Omegahed 20:52:42 zakim, choose a victim 20:52:43 Not knowing who is chairing or who scribed recently, I propose etienne 20:52:45 zakim, choose a victim 20:52:45 Not knowing who is chairing or who scribed recently, I propose alcooper 20:52:50 zakim, choose a victim 20:52:51 Not knowing who is chairing or who scribed recently, I propose Laszlo_Gombos 20:52:55 zakim, choose a victim 20:52:55 Not knowing who is chairing or who scribed recently, I propose yonet 20:53:04 zakim, choose a victim 20:53:04 Not knowing who is chairing or who scribed recently, I propose bajones 20:53:06 zakim, choose a victim 20:53:06 Not knowing who is chairing or who scribed recently, I propose atsushi 20:54:01 https://github.com/immersive-web/model-element/issues/76 20:54:39 Scribe: marisha 20:55:26 Brandel: The stereoscopic spatial display of model and the affordances of a model tag are great, but not all devices can do that. There are some thing syou might want to do for a non-spatial non-stereoscopic view 20:56:05 Brandel: On the basis of that divergence, it's good for people to know that's going to be the case. (In a similar vein as rel and Quicklook) 20:56:21 i/Brandel: The stereoscopic spatial display of model and the/topic: Allow a to specify a preference for spatial display (immersive-web/model-element#76)/ 20:56:28 yonet has joined #immersive-web 20:56:29 Brandel: It would beb good to indicate a stereoscopic property on a model 20:57:02 Brandel: For a portal environment it doesn't make sense to put it behind the surface of the page.. but if you do want to do those things, giving folks an option to do that would be desirable 20:57:23 Brandel: Should we consider exposing that in the API, setting a preference for spatial or non-spatial display? 20:57:26 q+ 20:57:34 felix_z has joined #immersive-web 20:57:40 ack bajones 20:57:54 bajones: This seems straightforward but I have some questions 20:58:18 ... It seems the primary purpose is to dictate to show "mono" regardless of environment. Is there a sensible reason to dictate something to be seen in "stereo"? 20:58:50 Brandel: One thing Model can do is for materials and surface discrimination. Knowing that you're not going to do that could be useful. 20:59:20 ... Something being intrinsically spatial might be desirable enough for constructing a page mockup 20:59:43 bajones: The way this is worded makes me think the intention is to give a page a display hint without necessarily being able to read back the actual value 21:00:07 ... so does "knowing" I'm in a particular mode means knowing the preference rather than the actual 21:00:52 Brandel: It's not that we want to prevent people from determining whether a user is on a spatial device, but it could be valuable for content tailoring 21:01:06 q+ 21:01:16 bajones: I broadly agree with not exposing information unless the user can do something useful with it (whether it applies here) 21:01:48 ... For stating something only wants to be displayed in 'mono,' could it be triggered intentionally or otherwise via stylesheets? 21:02:52 ... For scenarios like overlaying text on top of a background, the model might be forced into mono mode 21:03:10 Brandel: Other scenarios to force mono mode might include CSS filters 21:03:26 ... Some effects will only apply to mono models rather than stereo 21:03:30 q? 21:03:33 ack cabanier 21:03:39 cabanier: Why would sepia not work for stereo 21:03:53 Brandel: It would compel a complete rewrite of the CSS stack 21:04:21 ... since in Apple's case Model is not rendered with web but with Reality Kit 21:04:52 ada: We might need a list of CSS filters that would not work in spatial mode 21:05:07 cabanier: Let's say you use reality kit to render the monoscopic model, you still wouldn't be able to use filters 21:05:11 Brandel: That's a matter of plumbing 21:05:38 ... I might need to back out of things if it turns out my understanding is misaligned with reality 21:05:40 rrsagent, publish minutes 21:05:41 I have made the request to generate https://www.w3.org/2024/03/26-immersive-web-minutes.html atsushi 21:05:55 q+ 21:06:15 ... A mono rendered thing may have the ability to respond to CSS filtering depending on the renderer 21:06:36 cabanier: -Asking clarification on Brandon's position- 21:07:03 cabanier: Could stereo or mono be a CSS feature instead of an attribute 21:07:11 Brandel: Maybe. Could possibly be a media query 21:07:58 ... This idea could be used to discriminate between different types of experiences, but should also potentially give the user control 21:08:37 ... Having an indicator of how well something is supported on a platform and for the dev to make decisions about rendering in that context could be valuable. 21:08:52 cabanier: I was thinking properties like 'stereo,' 'mono,' or 'auto' 21:09:07 ... this could apply to other features like a scene graph 21:09:23 Brandel: Looking Glass exists, I don't know how conformant their mechanism for presentation is 21:09:35 ... but stereo may be inadequate to describe things in scenarios like that 21:09:45 ack bajones 21:09:45 ... we may want to be more encompassing of multiple types of contexts 21:10:08 bajones: I would love to see a media query for determining this 21:10:32 felix_z has joined #immersive-web 21:10:47 ... In regard to Looking Glass, I don't see a problem referring to that as 'stereo,' it's generating more than two images but you're still perceiving two images (one for each eye) 21:11:00 q? 21:11:02 Brandel: that makes sense 21:11:32 topic:https://github.com/immersive-web/webxr/pull/1361 21:11:56 cabanier: I will introduce the problem space for this 21:12:13 ... On Quest devices, a feature was introduced to track controllers and hands at the same time, so now there are 4 input sources 21:12:16 s|https://github.com/immersive-web/webxr/pull/1361|First pass at defining tracked sources (immersive-web/webxr#1361) 21:12:21 ... But no WebXR experience supports this properly 21:12:36 ... They draw either only hands or only controllers 21:12:52 ... And it's hard to determine what the user's intent is in regard to what should be rendered when 21:13:32 ... The WebXR spec has the notion of primary and auxiliary input sources 21:13:46 ... and Apple Vision Pro has a problem today with transient pointers, there could be multiple input sources 21:14:36 s|topic: https://github.com/immersive-web/model-element/issues/71|topic: Supply an Image-based Light file (immersive-web/model-element#71)| 21:15:10 s|topic: https://github.com/immersive-web/model-element/issues/72|topic: Specify a camera's pivot location (immersive-web/model-element#72)| 21:15:19 rrsagent, publish minutes 21:15:21 I have made the request to generate https://www.w3.org/2024/03/26-immersive-web-minutes.html atsushi 21:15:29 ada: The decisions about unseen defaults - experiences tend to request both hands first - it means that we get a lot of difficulty determining inputs, it only behaves if you have one hand behind your back, and then you get a tracked pointer and a transient pointer 21:15:52 q+ 21:15:58 q+ 21:16:02 matlu has joined #immersive-web 21:16:06 ada: Regarding your proposal: one of the content that works in Vision Pro: Monster Hands would break because you could no longer see the hands 21:16:42 cabanier: The proposal is to create a new attribute - a list of all tracked sources but do not create input. For Vision Pro this would be hands. In Quest this would be controllers after you put them down 21:17:19 ada: For my personal stance: I don' think we should specify granular input sources like "right and left"- 21:17:27 cabanier: I do think that though 21:17:34 ada: So we differ on that 21:17:56 ada: It would be nice to come up with a sensible agnostic model for inputs 21:17:59 nick-niantic has joined #immersive-web 21:18:04 q+ 21:18:14 q+ 21:18:18 cabanier: I want specifically right and left hands specified though 21:18:46 Brandel: My objection is that it is anthropocentric. 21:18:58 ... Meta overindexes on what people currently already encounter on Meta hardware 21:19:26 ... Handedness has implications about human participants with hands (rather than sightedness or other attributes associated with those inputs) 21:19:43 q+ 21:19:48 ... There could be automated agents, or possibly accessibility tools that abstract away some of these principles 21:20:14 cabanier: But everything is broken everywhere because it's not web compatible 21:20:50 ada: In terms of WebXR's lifetime, I hope we're at the very beginning. There is some pain now, that this proposal will alleviate, and will give some time to work out things to become more generic 21:21:39 ... This change means that the first two inputs that have select events will work, so if there are controllers, they will work 21:22:13 cabanier: It rubs me the wrong way to spec things that is counter to what everyone has implemented. 21:22:16 q? 21:22:59 bajones: If you have multiple primary input sources, if the experience falls back to only recognize the first two sources, it would still be acceptable 21:23:00 ack Brandel 21:23:43 Brandel: In general it's worth making sure the specs refer to real-world things like body plans and things that exist in the world - and then look at those as broadly as possible 21:24:20 ... so ideas like handedness... in reference to the body tracking API, how well does that represent people with different limb configurations 21:24:38 ... There are benefits and costs to reflecting reality vs what is ideal 21:24:49 ... We want to make sure this accounts for all the ways WebXR could be consumed. 21:25:18 cabanier: If you introduce accessibility tools, now you have a third framework that will also not work 21:25:36 ack alcooper q? 21:25:41 ada: Part of the goal here is to buy us time to get out of the current problem space of difficulty 21:25:44 ack alcooper 21:25:51 q? 21:26:33 alcooper: The pattern described of holding the controllers and then switching to becoming tracked sources, that sounds needlessly complicated - can't you just always track all sources and then change which ones emit input events 21:27:15 cabanier: That's mostly how that works but we wanted to enable folks to render the objects separately from input handling 21:27:53 alcooper: The spec doesn't currently say there's only one left and one right; web devs really seem to have overindexed on their own specific implementations 21:28:22 q+ 21:28:41 ... It is unfortunate that they have overindexed on having only two input sources, but I don't love that we just add another attribute as a bandaid to the problem 21:29:06 ack nick-niantic 21:29:34 nick-niantic: In the show One Piece, one character has a sword in each hand and one in his mouth. You could do an experience like this 21:29:42 nick-niantic: I don't know what will change or break here 21:30:01 cabanier: Nothing will break, it will just allow people to adjust things for the Vision Pro 21:30:24 ... Tracked Sources is the new thing. These are objects that are tracked but don't generate input sources 21:31:34 cabanier: -speculation on how API would have to change to glob all objects together regardless of being an input source- 21:31:51 nick-niantic: Why on Oculus can you not generate events with both controllers and hands at the same time 21:32:23 cabanier: That's just the way it is. You can't do that with gamepad right now. 21:33:01 nick-niantic: Just trying to imagine different scenarios where you have different kinds of controller configuration 21:33:25 bajones: When you pick up a controller, do you get a hand object in the tracked sources and then when you put it down they switch places? 21:33:27 cabanier: yes 21:33:41 Brandel: Could you attach multiple controllers to a Quest device, like four controllers? 21:33:51 cabanier: I don't know if that is supported, but conceivably. 21:34:18 ack felix_z 21:34:21 ack bialpio 21:34:46 bialpio: How far off are we from having a WebXR system that can track multiple people? 21:35:01 ... like having multiple players handled by a single WebXR session 21:35:24 ... Having just one display device but having two users 21:35:47 bajones: I can imagine a Looking Glass scenario with multiple users coming up to a large display and their hands being tracked.. 21:35:59 ... That's a scenario where you can't utilize a lot of the drive-by web content out there 21:36:05 ... It would have to be using very specialized hardware 21:36:18 bialpio: I'm just thinking of a future nearer than having robotic arms 21:37:05 bajones: Most specialized systems are not going to be picking up random three.js content - they are implemented for a very specific purpose that does not need to necessarily be served by standard APIs 21:37:27 ada: After the discussion yesterday about secondary views, maybe we should pivot those views to helping enable scenarios that are a bit more elaborate 21:37:54 Brandel: Like playing multiplayer Goldeneye 21:38:36 (on an autostereoscopic, lenticular display like Google project Starline) 21:38:38 bajones: If we have variable expectations for controllers, it might inconvenience but not break you. For requiring multiple views being rendered, that is a lot more constricting 21:39:01 ack ada 21:40:32 ada: Part of this issue is that we conflate input rendering and input interaction. For gamepad it made sense, but one option might be to provide an array of things to render and have a position in 3D space, and to which you should listen for select events 21:40:56 ... It would be a significant re-architecture of the way we do input 21:41:01 q+ 21:41:13 ... Not knowing what was coming has put us in a difficult situation 21:41:13 ack alcooper 21:41:43 alcooper: It looks like the select event should be fired from the session object. What is content doing to ignore events? 21:42:06 bajones: Three specifically has you query controllers by index and then do work to ensure you don't have to think about disconnects and reconnects 21:42:34 ... They do a lot to map input sources to controllers and surface them to the user by index 21:42:47 alcooper: And then Aframe is querying off gamepad events 21:43:05 ada: They started in the WebVR era and never changed it when we moved to WebXR 21:43:56 alcooper: I wish these sites hadn't all overindexed on the current implementation 21:44:02 Brandel: It's not even the current implementation 21:44:10 q+ 21:44:41 ada: Back in the day this was new, and we didn't have a strong design doc for inputs for WebXR, and then for years we've had one model of how inputs are supposed to work - a headset with two sticks 21:45:09 ... and then Android devices came along - it was very hard to get uptake to support WebXR 21:45:22 ... You don't see the same amount of content on handheld devices 21:45:45 ... We haven't shipped WebXR on iPhones yet which means no cross compatibility 21:46:34 ... We're in an awkward era now that headsets are starting to diverge - screens, Vision Pro, TV displays with multiple inputs.. 21:46:58 ... If we can get out of this hurdle we'll be healthier in teh long run.. but this solution is sensible in general because there is new hardware coming out that isn't an input source 21:47:03 q+ 21:47:06 q+ 21:47:26 alcooper: Things like tracked pucks are good candidates for this since they shouldn't be rendered 21:47:31 ack bajones 21:48:17 bajones: When this topic came up I thought this was initially already declared in the spec. In general I feel that we shouldn't change the API in backwards incompatible ways just because pages have decided to do something else 21:48:28 ... but we do see more devices coming online that don't fit the general input mode 21:48:42 ... and we expect to have a lot more accessories and tracking devices 21:49:13 q+ 21:49:28 ... so being able to communicate these in a clear and concise way without imposing unnecessary expectations, and if we can also "fix" some of the existing web content to work on existing devices, sounds like a good path forward. 21:49:31 ack nick-niantic 21:50:05 nick-niantic: If the Apple Vision Pro has an existing implementation and is expected to have a new implementation, we'd like to see that change as quickly as possible 21:50:23 ... People are going to start making content for that device and so time is short to fix this 21:50:31 ada: Yeah the timing is awkward and painful for us 21:50:59 nick-niantic: I would advocate to make the shift while you're still in experimental mode, but also we want you to get out of experimental mode as quickly as possible 21:51:20 nick-niantic: Something else: You mentioned a future when iOS devices would have full webXR support (hypothetically) 21:51:48 ... one of the things we have been fighting is that experiences that are developed for headsets and phones are very different 21:52:11 ... if that hypothetical comes to pass, we would like to know what kind of device we are running on - a phone or headset 21:52:44 ... Android presents itself as a headset to us which is why we do User Agent testing 21:52:55 ... we don't want to have to do the same thing for iOS 21:53:23 Brandel: We already experience this with AR Quicklook 21:54:12 ... [amusing anecdote about having to headbutt a virtual control intended for a phone] 21:54:57 ada: On Vision Pro you do have inputs, it's just not something you wave around. That would work very well today on something like a phone using transient inputs 21:55:30 nick-niantic: You wouldn't build beat saber for phones because you wouldn't get controllers 21:55:59 ack felix_z 21:57:07 felix_z: Why is wrapping so I can listen to input events from a particular source a bad way to do things? I don't have a universe select event that is left-or-right handed. What if I just have a gun in my hand and want to listen to that? 21:57:29 bajones: I think that was the mentality behind why things were originally built how they were 21:58:34 ... the hope was that in a lot of cases, you can get by with fundamental actions starting/stopping, so if I have a play/pause button, and all I really care about is whether the button is pressed, all I have to do is listen to the session 21:58:48 ... The expectation was that we were making it easy to work across the widest amount of hardware 21:59:24 ... The concern is that some of these libraries make that more incompatible mechanism the default (whether it's best for the developer or not) 22:00:08 ... Lots of experiences don't really care where the input is coming from. So determining which input event is coming from where is not the right model for everybody. 22:00:34 Brandel: On the 2D web, we don't listen for events on the mouse - we listen on a target *from* a mouse - they are object-centric 22:01:15 felix_z: Okay I was just wondering why it would be bad for a framework to be this way. The examples you gave are samples, I don't know if they represent the majority 22:02:08 ada: The issue with these frameworks is that they make a lot of assumptions right out of the gate, and they exclude anything that doesn't look like the default "two sticks and a screen" hardware 22:03:04 ... That model has worked well up until now because it's been the predominant available hardware, so it wasn't breaking anybody at the time. But now things are breaking (with some advance warning) and now we have to make the content work for a new paradigm. 22:03:49 bajones: I should say that I've been complaining about frameworks a lot but it's not really fair of me. The frameworks have been doing what they felt was right or even easiest to enable the type of content they were looking to enable. 22:05:23 ... There's a little bit of frustration for the editors here where assumptions didn't match reality of new hardware, and we can complain that they didn't stick closely enough to our own spec.. but no one is being a bad actor. The spec just happened to make a better decision than the libraries did. It wasn't guaranteed to turn out that way 22:05:49 ack alcooper 22:05:59 ... as Rik pointed out, even some of the samples don't work properly here 22:06:27 felix_z has joined #immersive-web 22:06:36 alcooper: If we are thinking about tracked sources including elbow pads or knobs, should they be a new type of input sources? 22:06:54 ada: I think they should still be tracked input sources, with a property about whether they will be rendered or not 22:07:48 ... additional properties: handedness: none, rendered: false, no pointer 22:08:33 alcooper: Was just wondering whether we should still call it an input source 22:09:16 ada: I think it makes sense to expose optional input features for existing input sources 22:10:29 yonet: How do you detect whether controller shouldn't be used as the input source? 22:10:51 cabanier: If you let go of the controller it's no longer detected as an input 22:11:16 ... I think. It's a bit of a black box in the OS 22:35:13 Brandel has joined #immersive-web 22:41:48 scribenick Brandel 22:42:01 marisha has joined #immersive-web 22:42:04 felix_z has joined #immersive-web 22:42:06 present+ 22:42:27 present+ 22:42:32 topic: 2024 recharter (immersive-web/administrivia#204) 22:42:33 q+ 22:42:41 yonet: We need to re-charter in the coming months, and there are some questions that we need to address 22:42:50 ack cwilso 22:43:20 cwilso: for context, a charter at W3C must always be for a fixed term, to allow for fixed goals over that time period. 22:44:01 cwilso: These charters are things that your company needs to be able to assent to. The charters are broad and directional - what is "mostly right" about goals within the timeframe specified 22:44:15 yonet has joined #immersive-web 22:44:43 cwilso: Especially, charters shouldn't be overly ambitious at the risk of driving some companies away 22:45:11 topic: https://github.com/immersive-web/administrivia/issues/204 22:45:32 cwilso:[providing background for a "Living Spec" in the context for potentially adopting it in IWWG] 22:45:57 etienne has joined #immersive-web 22:46:04 q+ to ask about Living Spec in relation to our modules (once Chris is done explaining) 22:46:48 cwilso: This has given rise to the notion of a "Candidate Recommendation Snapshot", which is a way to update a specification in-place, rather than working strictly to a fixed goal 22:47:29 cwilso: There are Intellectual Property distinctions between a V1/2/3 CR model and a living spec-based one 22:48:21 q+ to ask about how changes to CR happen/are expected to be implemented 22:48:24 current draft -> https://w3c.github.io/charter-drafts/2024/immersive-web-wg.html 22:48:31 yonet: Some of the WG's goals seem like a good fit for living spec, and there are some things that the charter may want to aim to promote from CG to WG 22:49:25 bajones: WebGPU is also considering moving to a living Spec, based on the expectation of an ongoing stream of feature/extensions that would be possible to more gracefully accommodate under this model 22:50:08 bajones: Would it be better for us to pursue this approach for the additional _modules_ in this kind of mode, rather than have them scattered through a wider array of documents? 22:50:09 q+ 22:50:12 q+ 22:50:38 ack bajones 22:50:38 bajones, you wanted to ask about Living Spec in relation to our modules (once Chris is done explaining) 22:50:39 yonet: It seems like this would have the greatest impact on the editors and the work of integrating more content into the base specification documents 22:51:17 qq+ 22:51:29 ack cwilso 22:51:29 cwilso, you wanted to react to bajones 22:51:51 ada: One benefit of having separated module specifications is that a given implementation may have no notion of gamepad / hand / etc. If a single spec covers all of the functional contents of the modules, that would limit what is permitted to constitute a "conforming" platform 22:52:05 s/conforming/compliant 22:52:48 cwilso: This is down to the language in the spec describing whether components are mandatory - more that "when hands exist, they exist in this way" rather than "all platforms need hands" 22:53:39 bajones: Going back to the comparison with webGPU, there is a clear baseline compliant implementation - but there are extensions that may be available 22:54:23 bajones: We could spec things in a way that states a minimum implementation - the situation is a little different 22:55:01 cwilso: The language should say "there is a baseline you must meet." The living standard can do that as well 22:55:35 ada: This seems like something we can and should take back to our companies and lawyers to assess the practicality 22:56:07 bajones: There are two questions - whether to livingSpec, and secondly to roll things into a central one rather than pursue modules as independent components 22:56:15 ack alcooper 22:56:15 alcooper, you wanted to ask about how changes to CR happen/are expected to be implemented 22:56:26 q- 22:56:39 alcooper: What does a living spec change, in practice, about accepting a spec change? 22:57:36 cwilso: breaking changes remain important - living standard reflects the reality of how people have worked on and with HTML in the past anyway 22:58:23 cwilso: It tries to allow the use to reflect the ideal, rather than say that strict adherence with a momentary understanding and saying "this is ISO 999" 22:58:49 atsushi: we need to get to a review before getting to a CR. a CR snapshot still needs to undergo IP review 22:59:10 (and other reviews, like horizontal reviews and TAG reviews) 22:59:49 atsushi: This only changes the use of a Rec or a WD - reviews still apply when something becomes CR 23:00:25 alcooper: so This functionally still requires referring to a CR as of a date - it's not the case that any new amendments are automatically CR 23:01:08 atsushi: a CRS should reflect an update 6-24 months after a "substantive change" 23:01:44 ack bialpio 23:01:53 atsushi: the CRS should be republished in a "timely manner" after "reasonable substantive changes" 23:02:36 bialpio: It seems like it would make things easier if modules are rolled into a single spec because it would be easier to detect cross-cutting changes 23:03:46 bialpio: To Ada's point, extra modules tend to have a strong statement of optionality - "IF your device supports things, this is the way to do it". not that they all _must_ - integrating them into a single document shouldn't reflect that 23:03:51 s/reflect/change 23:05:04 bialpio: Affecting more webIDL definitions may require updating more things for more people, but most new parameters are specified as nullable 23:05:33 bialpio: It's not clear that splitting into modules has a bearing on the amount of work that vendors are obliged to do 23:06:18 yonet: And what do we move to WG? WebGPU bindings was mentioned 23:07:03 ada: We have also indicated that a WG charter goal should require two implementers to support, given that a CR ultimately needs two implementations to pursue 23:07:37 [bajones and cabanier indicate they would support webGPU bindings moving to WG] 23:08:12 https://www.w3.org/immersive-web/list_spec.html 23:08:27 ada [raising the 'real-world meshing' feature as a promotion candidate] 23:09:37 bajones: Real-world _Geometry_ is the name of the plane system on Android and Quest. As an aside, a single repo would mean we don't have to come up with different names! 23:10:19 bajones: Real-world meshing is the process of identifying the three-dimensional _shape_ of the world, rather than just surfaces (or 2D bounding geometry) 23:11:51 ada: It sounds like we may want to check with our people to decide if it's reasonable (inside 2 years' time) 23:12:15 atsushi: We already have RW Geometry as a candidate - isn't that enough? 23:12:43 alcooper: I think we can promote from CG to WG mid-charter if it's already on the general radar for the group(s) 23:13:05 bialpio: We did this for some of the geometry work over the last charter window 23:14:14 ada: The charter is often used by the legal departments of vendors to check for the risk to IP portfolios. As long as we have adequate mention in the WG of those keywords that most interest the lawyers we should be okay 23:14:38 ada: [enumerating the list of candidates in the CG listing] 23:15:26 yonet: We should consider a mechanism to demote CG modules / interests to indicate they are archived or no longer pursued 23:15:51 alcooper: This is a place where there may be a distinction between the contents of the charter and what happens to have a repo today 23:16:25 ada: Could we indicate that by removing the 'w3c.json' such that it's no longer regarded as an active proposal? 23:17:19 yonet: We also have tags to denote incubation to indicate that there's no expectation that implementations are incoming 23:18:01 atsushi: We can mark documents as "Articles", such that it's not a recommendation and no active development expected 23:18:26 s/active/there is no active 23:18:46 ada: returning to the list of candidates - "capture?" recording of sessions? 23:19:11 ada: No work has progressed on it, but it seems like there is a valid need for it 23:19:27 ada: Demote "computer vision", as it became "raw camera access" 23:19:38 ada: Demote the 3D CSS stuff for the time being 23:21:30 Brett: detached elements are easy from a specification perspective, it's easy to say what they should do 23:22:11 ada: I was interested in this while at Samsung, but then moved to Apple - so the two parties are now Apple. If there are rumblings about it we can keep it on the docket 23:22:50 bajones: Should we not put this in the charter and CSS decides they _would_ like to pursue it - would we be upset about that? 23:23:02 ada: I think we would be _so_ happy if that happened. 23:23:37 brett: Anyone could develop an interest in VR at any moment in time 23:24:17 ada: would there be any objection to adding it, running the risk that there may be no progress in the 2-year window? 23:24:46 cabanier: It was added 5 years ago to CSS and then removed already 23:25:03 ada: Front-facing camera? keep it where it is (yes) 23:25:37 ada: Geo alignment? 23:26:05 yonet: It's not identical to shared anchors. People are interested in it, but the people responsible for making past progress in MS are no longer assigned to it 23:27:10 alcooper: this concept of demotions should be considered on the basis of moving to WG or not 23:27:31 ada: Marker tracking? (yes, promote) 23:28:18 alcooper: The last disagreement was over the type for marker tracking, QR vs other objects 23:30:06 ada: Model? (promote/ add to list of 'potential deliverables') 23:30:44 ada: I think we should move "Navigation" to "potential" 23:30:54 cabanier: We shipped it, but there's no spec 23:31:42 yonet: Under potential deliverables, we have anchors, model element and body tracking. 23:32:17 yonet: Navigation to move to WG? (move to charter, don't move it to WG) 23:32:44 bajones has joined #Immersive-Web 23:32:57 nick-niantic: re: Splats - I would want to know why we add splats into a specific place - e.g. webGPU, model etc. 23:33:23 nick-niantic: I'm fine to say it, but want to know why _we_ are saying it rather than things we could just do _with_ the technology 23:34:35 nick-niantic: This is like scene graphs, in that we are generally interested but not sure what to do with. 23:34:40 https://w3c.github.io/immersive-web-wg-charter/immersive-web-wg-charter.html 23:35:14 https://w3c.github.io/charter-drafts/2024/immersive-web-wg.html 23:35:31 ada: note that you don't have to be comfortable answering now - but take this to your people so we can have a substantive conversation soon 23:36:02 topic: Update on Metaverse Standards Forum (immersive-web/administrivia#206) 23:36:15 rrsagent, publish minutes 23:36:17 I have made the request to generate https://www.w3.org/2024/03/26-immersive-web-minutes.html atsushi 23:36:41 scribenick: bialpio 23:36:55 topic: https://github.com/immersive-web/administrivia/issues/206 23:36:55 i/yonet: We need to re-charter in the coming/scribe+ Brandel/ 23:36:58 rrsagent, publish minutes 23:36:59 I have made the request to generate https://www.w3.org/2024/03/26-immersive-web-minutes.html atsushi 23:37:06 scribenick bialpio 23:37:43 matlu: [presenting slides] 23:39:24 ... metaverse standards forum is ~1yr old, it is not a standards organization but the ambition is to create open/interoperable metaverse 23:39:43 ... would like to assist standards organizations 23:40:35 ... idea is to get input from industry and feed it back to standardization bodies to help with use cases and accelerate the standardization work 23:40:58 felix_z has joined #immersive-web 23:41:52 ... various membership levels, including free membership, first board meeting coming up soon 23:42:30 ... members can suggest creating exploratory groups & working groups 23:43:08 ... multiple domains the forum focuses on, 3d interoperability among others 23:43:52 ... accessibility also moves to be an exploratory group 23:44:44 ... forum also creates standards register to help figure out where standardization happens 23:44:57 ... plenty presentations available 23:45:17 ... glTF USD Asset Interoperability WG is also active 23:45:37 ... interoperable characters / avatars WG 23:45:57 ... [more working groups described] 23:48:07 q? 23:48:24 Brett: where do you meet? 23:48:39 matlu: zoom calls, f2f meetings 23:50:41 Go here and click on the small padlock icon on the top right to join with you organization email 23:50:41 https://metaverse-standards.org/ 23:50:46 hi 23:50:47 felix_z has joined #immersive-web 23:50:54 webirc83 has joined #immersive-web 23:51:11 topic: https://github.com/immersive-web/proposals/issues/79 23:51:19 hello this is Diego 23:51:48 webirc83: apple released transient pointer 23:52:03 ... it allows integrating with gaze to WebXR 23:52:06 q+ 23:52:35 ... but the missing part is the lack of possibility to highlight UI elements on hover using gaze 23:52:50 ... what'd be the acceptable path to making something like this possible? 23:53:01 ack ada 23:53:31 ada: I understand this is an issue 23:54:02 ... apple's general stance on how the hover should be done is out-of-process glow (?) 23:54:29 Brett4 has joined #immersive-web 23:54:45 ... I'd like to move this forward but I don't know yet what path we'd be able to take 23:54:52 q+ 23:55:09 ... and until I figure this out I don't think we'd be able to come up with a proposal 23:55:35 q+ 23:55:39 ... unlikely that gaze tracking is going to be exposed, it is not currently exposed on AVP even to native apps 23:55:51 ... very privacy invasive 23:56:34 ... other browsers may disagree but we're not likely to implement a gaze tracking api if it were exposed in WebXR 23:57:06 bajones: when mozilla was more active, they also had pretty strong stance on this as well, they did not want to expose gaze vectors 23:57:18 ack Brett_ 23:57:45 Brett4: we'd like to be able to look eye to eye in VR 23:58:07 ... to solve the problem, could there be a half-pinch that's distinctive from full pinch 23:58:54 ada: what I suggest to the authors is that when selectstart arrives, they can enter the state that marks the start of the gaze gesture 23:59:04 q+ 23:59:08 ... and selectend is when the interaction is processed 23:59:11 ack Brett 23:59:16 ack cabanier 23:59:17 ... it is a workaround but not too bad 23:59:56 cabanier: we chatted about accessibility tree yesterday - in theory, device like AVP could use the tree to render highlights over UI elements 00:00:16 ack cabanier 00:00:26 q+ 00:00:27 ada: with the accessibility buffer, we wouldn't know what to do, but may be doable with accessibility tree 00:00:51 etienne: safari does not draw the glow itself 00:01:00 An app might want to highlight 2D UI elements and also 3D objects in the scene that the user can interact with 00:01:48 Brandel: there is a possibilty that ui elements can be vectorized 00:02:11 ... safari is a shared mode app, unlike WebXR / unity which is rendered in fully exclusive mode, it may have different rendering paths 00:02:26 ... there's work for us to do but we don't yet know how it'll look like 00:03:07 ada: it'd require OS changes and I have ideas and fallback ideas, as soon as we figure out what we can do I'll let the group know 00:03:30 cabanier: to summarize, exposing the gaze is a no-go but we'll try to have some alternative 00:03:42 a small demo I was building with A-Frame that would love to port to AVP https://aframe.io/aframe/examples/showcase/spatial-ui/ 00:04:07 ada: start highlightling on selectstart + raycast to know what to highlight and fire the UI element event on selectend 00:05:15 webirc83: there is a laser pointer mode that you could enable? 00:05:27 Brandel: that's an accessibility setting, yes 00:05:42 webirc83: is this something that could be exposed? 00:05:55 Brandel: we haven't looked into exposing it 00:06:05 ack nick-niantic 00:06:23 nick-niantic: we talked about scene description and challenges w/ solutions 00:06:34 ... hover may be one of the benefits 00:07:11 ... rather than specifying detailed information, we could look into specifying more coarse grained data that could help here 00:07:41 +q 00:07:47 ada: I'd rather have accessibility buffers first, vectorized data 2nd, and triangles last 00:08:13 Brandel: is there anything else that you're targeting here? 00:08:25 webirc83: I'm considering general user experience 00:08:52 ... later we'd like to have avatars w/ eye tracking but that's a separate problem 00:09:26 Brandel: treating user input as 2-step process allows it to be cancellable 00:10:31 Brandel: can OS construct such a view where we could intervene? can the OS make changes in an opaque ways to the session? 00:11:08 ada: we don't have anything in the spec that'd disallow the OS to show its UI on top of the session 00:11:29 bajones: correct, we should not have any language like that but I can double-check 00:12:17 ada: we're looking into how to make all of this work 00:12:33 bajones: spec has "trusted immersive UI" concept 00:12:40 ... [reads the spec] 00:13:05 ... this is not explicitly forbidden by spec 00:13:14 alcooper: [reads another part of the spec] 00:13:35 bajones: this is probably not something that is generally spoofable 00:13:39 ack yonet 00:13:51 Section I quoted: https://immersive-web.github.io/webxr/#exclusive-access 00:14:14 Section Brandon quoted: https://immersive-web.github.io/webxr/#trusted-ui 00:14:18 https://avibarzeev.medium.com/for-xr-the-eyes-are-the-prize-25d43a533f2a 00:14:25 yonet: eyes are windows to the soul, and that's exactly why we don't want to expose them 00:14:30 ... there are dangers of exposing them 00:14:34 ack Brett_ 00:15:13 ack Brett 00:15:58 Brett4: having eye tracking work would be a great carrot to encourage devs to use accessibility tree 00:16:35 ada: accessibility is also important w/o being a carrot but it's nice when things work out together like that 00:17:24 Brett4: [setting up presentation] 00:26:30 q+ 00:29:34 q+ 00:29:37 ack marisha 00:30:00 marisha: question / clarification: 628mb on the cube, what is that? 00:30:09 Brett4: that's just bytes 00:30:21 bajones: is it text only or binary? 00:30:27 Brett4: it has both 00:30:40 bajones: so 628 bytes was ascii or binary? 00:30:50 Brett4: ascii 00:31:30 Brandel: simple - yes, but simple for whom? there are reasons why we don't use pbm for images 00:33:13 bajones: nice companion presentation for splats introduced yesterday 00:33:38 ... I don't see anything about materials here - is this something that people put alongside .ply? 00:34:10 ack nick-niantic 00:34:15 Brett4: there's no standards body, the data can be placed in the file and it'd be up to the viewer to interpret what's inside 00:34:37 nick-niantic: people use .ply for splats because it's easy representation 00:35:03 ... they omit some required data but include other data, and it is something that's driven by consensus 00:35:35 ... ply is simple and understood widely, but they are rarely used when something else than lowest common denominator is used 00:35:50 ... the files themselves can be huge 00:36:49 ... the reason why gltf exists is because there's plenty other information that people want to use and that don't map nicely to vertex data 00:37:49 ... we talked about other proprietary formats and why they are proprietary is because they contain a lot of cleverness related to data representation 00:38:27 ... although there are formats like webp which is open 00:38:34 ... but not simple 00:38:45 ... simplicity has both costs and benefits 00:39:19 Brandel: having a body that discusses what is a standard implementation is actually a benefit for us 00:39:41 Brett4: being able to agree between ourselves is beneficial 00:40:27 Brandel: I don't know if web standards group is the place where we need to come up with formats 00:41:04 bajones: lesson learned from working on browser is that there's big difference between getting things first working and working properly 00:41:24 ... first prototype for webvr took me ~1 week 00:41:34 ... and that was a very fun time 00:41:55 ... and then it took another 6 years 00:42:14 ada: WebVR was 2yrs young when it got deprecated 00:42:30 bajones: and it was 10 years ago 00:42:55 ... there's a draw to simplicity, but we will need to go far beyond what it takes to implement an MVP 00:43:14 marisha: the simplicity and the fact is that it's from '90s is interesting 00:43:41 ... ply looks like a primitive format that could have grown alongside the web 00:43:49 ... but didn't 00:45:26 bajones: it's easy to polyfill 00:45:32 Brandel has joined #immersive-web 00:45:56 ... we could try something out and prove this concept 00:46:35 ... if this is something you're enthusiastic about, you don't need a permission from anyone in this room 00:48:01 ada: it should be possible for you to polyfill ply, convert it to usd and people would be happy with it, that'd be a signal for us 00:51:50 marisha: mode switch in order to view e.g. an image is a limitation 00:54:13 bajones: I'd like to see a way to feed the buffers directly to a model tag to display, we'd not have to have blessed formats then 00:56:19 rrsagent, publish minutes 00:56:21 I have made the request to generate https://www.w3.org/2024/03/26-immersive-web-minutes.html atsushi 00:56:35 rrsagent, bye 00:56:35 I see no action items