<josh_marinacci> the link in the calendar that you updated on the 8th doesn't seem to be working

<josh_marinacci> it says the meeting has been canceled or ended

<trevorfsmith> Hmmm. I just used it.

<trevorfsmith> And there are others in it.

<josh_marinacci> /mit.webex.com/mit/j.php?MTID=mfb8383ef0796cd6999844e1626d7fee6///mit.webex.com/mit/j.php?MTID=mfb8383ef0796cd6999844e1626d7fee6

<josh_marinacci> perhaps my calendar invite hasn't updated. :( could someone send me the current link?

<trevorfsmith> Please don't paste it into this public log.

<trevorfsmith> Josh, I'll send it hold on.

<adrian> trevorfsmith, I'm also looking for the new link

<blair> me too, trevor. My cal still has the old one

<trevorfsmith> Dang. Ok, everyone who can't get in let me know and I'll DM you the link

<Nick-8thWall> me

<trevorfsmith> Google calendar is really crap at this.

<adrian> me

scribenick cwilso

<scribe> scribenick: cwilso

trevor: Welcome back!
... we have an agenda:
... subtitles and spatial media (Ada)
... computer vision, issue 4 (Nell)
... 3D rendering and constraints and goals for web extension issue 43 (Trevor)
... lighting estimation repo (Trevor), needs an owner. Any takers?
... If lightning estimation sounds like something you'd like to work on - write an explainer, etc - please contact Trevor
... Taking up subtitles and spatial media...

Ada: we had a call with the Media and Entertainment Interest Group last week, they're interested in subtitles in media, and 360 videos are a popular use of VR.
... one issue they raised is how to do subtitles (where do you place them relative to the user, who's talking (could be behind the user), how can they be displayed clearly and crisply...

<Leonard> +q

Ada: in November they opened up a couple of issues in the proposals repo, I wanted to raise everyone's attention to this issue.

https://github.com/immersive-web/proposals/issues/39

https://github.com/immersive-web/proposals/issues/40

scribe: Please take a look, let's try to get the conversation going.

Leonard: Is this for prerecorded or realtime content?

Ada: They were talking about prerecorded, but live is another great topic. Could you raise in the issues?

Leonard: would transcription be handled elsewhere?

Ada: Yes, this is just display.

Trevor: Are they doing any thinking beyond spherical video? Subtitles in CG scenes?

Ada: just video content for the moment. I think CG scenes could also make use of subtitles, but not sure if it's the same feature. They were focused on 360 video; I did raise 3D video having other issues (placing subtitles at infinity, e.g.), but that's not the focus right now.
... this may also tie in with the quad layer/ overlay issue

Trevor: let's take up 3D Rendering in WebExtension XR (aka WEXR): Issue 43

https://github.com/immersive-web/proposals/issues/43

scribe: there seem to be lots of cross-purposes on the rendering side of this long-lived app system - Blair et al, you'd pointed out that one path is that XRSession could be the "system" that you hang off of.

<blair> FWIW, for small/simple things, I actually tend to be in the glTF side, as you describe it.

scribe: but another view is that in order for the browser to exercise a lot of control and management of 3D items, it might be better to hand off management of 3D assets to a single system.

<blair> (I can't remember how to queue)

blair: I think both are actually required for different things - for Holograms in Hololens, and some of the [Landscape apps] in Magic Leap, a gltf-like solution would make sense.
... gltfs for simple notifications or labels are interesting.
... hologram cube (?) or magic leap prisms - regions of a page - are also quite interesting.
... I think there's some middle ground here. Being able to render multiple full-screen apps at the same time is likely not tenable.
... both of those platforms will likely evolve to have much more powerful top-level UI management.
... this is also quite possibly related to the concept of dioramas.
... https://github.com/immersive-web/proposals/issues/31

cwilso: One more point - we should ensure that whatever model we have for these "mini-apps", prisms, whatever, they have the ability to interact and be programmatic, not just declarative, or the model will be very limited.

trevor: yes - may need to do a back-and-forth model where the declarative system puts in hooks for when the dynamic changes can happen, and call into dynamic call points.
... would love anyone who's into the graphics side could weigh in, now or later.

<josh_marinacci> I can't hear kip anymore

<josh_marinacci> too much distortion

kip: I think this is a deep rabbit hole, but has two primary issues: 1) if you have something running async, you may need to define ways to change composition. 2) this falls into the same category as defining declarative things.

<kip> Will summarize here:

<kip> 1: Managing differences in performance between composited contexts

<kip> - What to do when one runs slower and kicks in something like 45hz space warp

<kip> - What to do when content is slow.. Do overlaid items move asynchronously or stay frame-locked?

<kip> 2: Rendering intent (eg, realism, non-lit, flat shaded)

<kip> - Lighting models vary greatly depending on if scene is mostly static or dynamic

Alberto: I wish for the most basic implementation possible, injecting JS and basic gltf into browser environment. Would be good to kickstart ideas.

<kip> 3: Lighting model (eg, Would PBR + a shared spherical harmonic encoded Precomputed Radiance Transfer still hold up in 5 years?)

trevor: let's take up computer vision issue

<kip> 4: Alpha transparency. Unable to use depth buffer sorting, limiting compositing unless moving to declarative content only

blair: I wrote this up a while ago. I'm not sure what Nell wanted to cover here, but I've talked with folks at Intel and Google about this. The big question for computer vision is how can we expose camera video inside webXR?
... it's always felt like we should be able to leverage WebRTC, but it has a few limitations. WebRTC has been talking about adding metadata to streams, though, like timestamps, maybe even pose etc.
... could work like this: when you open an XR session, camera becomes available, and RTC streams come out with enough metadata so you could sync.

For smartphone-like devices with one feed, the sync is pretty straightforward. For other devices like HMDs, you'll need to roughly sync timestamps, since the framerates will (likely) be different.

scribe: I'm wondering if folks involved in WebRTC would be amenable to adding this stuff.
... I think we should consider whether this is really where we want to be adding support - doing CV in JS will likely light CPUs on fire.

(even in WebASM)

scribe: maybe we should look for decimated image pyramids or something like that, not just frames.

<trevorfsmith> cwilso: I know the WebRTC group here and can help with that conversation. There are interesting security issues that we'll need to address.

<trevorfsmith> blair: Like what?

<trevorfsmith> cwilso: Like permissions fatigue.

<trevorfsmith> It would be if users just let anything get access to their camera feed.

nick: it sounds like people have been worried about complexities. Chris asked us to think about what would potential use cases. I laid some out in the issue: https://github.com/immersive-web/proposals/issues/4
... specifically, https://github.com/immersive-web/proposals/issues/4#issuecomment-461596067

WebXR may want to expose the HMD cameras directly, rather than RTC. You just need access to the camera feed, interaction to generate hit tests, ...

scribe: for mobile AR, you also need synchronized drawing of CG content with camera frame.
... I don't know if this last one was part of XRFrame design
... displaying on top of a camera feed from 100ms ago might not be tenable.

<trevorfsmith> cwilso: What we're thinking of as XR cases are really augmented camera cases.

<trevorfsmith> ... If I'm trying to draw a mustache on people's face it might be a different category from XR.

blair: I mostly agree with what Nick said - I've gotten a lot of pushback to adding API surface area to WebXR that somewhat duplicates RTC's API surface area.
... we didn't do what Nick suggested - syncing camera frames - because of how we implemented the prototype.
... what Chris said about augmented camera feed - wanting to do something that works everywhere vs using the platform (e.g. face tracker in platforms).

Chris: yep, that. There's a current incubation for shape detection to use the platform for augmented camera feeds - https://github.com/WICG/shape-detection-api.

Nick: I don't want to preclude the moustache app. But we also want more powerful use cases.

<trevorfsmith> close the queue

trevor: we're out of time, so let's take it to the issue. I'll ping Nell as well.
... there was a topic brought up about VR-to-VR navigation; if you get a chance take a look at that issue.
... thanks all!

This is scribe.perl Revision: 1.154  of Date: 2018/09/25 16:35:56  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Succeeded: s/https://mit.webex.com/mit/j.php?MTID=mfb8383ef0796cd6999844e1626d7fee6//
Succeeded: s/are/is/
Present: trevorfsmith cwilso alexis_menard adarose alexturn Tony_Brainwaive
Found ScribeNick: cwilso
Inferring Scribes: cwilso

WARNING: No "Topic:" lines found.


WARNING: No date found!  Assuming today.  (Hint: Specify
the W3C IRC log URL, and the date will be determined from that.)
Or specify the date like this:
<dbooth> Date: 12 Sep 2002

People with action items: 

WARNING: No "Topic: ..." lines found!  
Resulting HTML may have an empty (invalid) <ol>...</ol>.

Explanation: "Topic: ..." lines are used to indicate the start of 
new discussion topics or agenda items, such as:
<dbooth> Topic: Review of Amy's report


WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)