15:44:30 RRSAgent has joined #mediawg 15:44:30 logging to https://www.w3.org/2022/06/14-mediawg-irc 15:44:32 Zakim has joined #mediawg 15:44:42 RRSAgent, make logs public 15:44:53 Meeting: Media WG meeting 15:45:37 Agenda: https://github.com/w3c/media-wg/blob/main/meetings/2022-06-14-Media_Working_Group_Teleconference-agenda.md 15:56:25 Matt_Wolenetz has joined #mediawg 16:00:27 present+ 16:00:32 Present+ Francois_Daoust 16:02:39 cpn has joined #mediawg 16:03:36 present+ Joey_Parrish, Chris_Needham, Gary_Katsevman 16:03:43 present+ Tommy_Steiner 16:04:19 Agenda: https://www.w3.org/events/meetings/6195de3d-2af0-471c-853d-b63396c562b6 16:04:22 scribe+ cpn 16:06:46 Topic: MSE #306 16:07:10 Matt: Chrome implementation is based on Serializable. 16:07:32 Chris: Was hoping to be able to discuss today with other implementers 16:07:54 Matt: Getting feedback for another approach would be unfortunate at this point 16:09:09 ... The implemention relies on WebCodecs VideoFrame. The hope is Serializable is not blockable. Keep it restricted at the beginning, then relax those if needed. 16:09:21 ... If it needs to become Transferable, we can add that on later 16:09:58 Chris: Do we need meeting time? 16:10:21 Matt: We've tried a couple of times. Getting responses from Mozilla, but having Apple chime in would be good 16:11:06 Chris: I'll follow up with Jer 16:11:20 Topic: Media Session 16:12:08 s/Keep it restricted at the beginning/Approach is to initially keep the API narrow and restrict flexibility (to not "bake-in" API that allows too much only to restrict it later)/ 16:12:18 Chris: Welcome Tommy and Youenn as co-editors 16:12:32 Tommy: I haven't sent a change to update the editors but will do soon 16:12:52 Regrets: Youenn_Fablet 16:13:19 Chris: Current discussion on Media Session and WebRTC Capture Handle Actions 16:14:11 ... Question about the appropriate scope for Media Session. In particular. whether next slide / previous slide actions should be in Media Session? 16:14:39 Tommy: In my mind, it doesn't include that, doesn't really match. But open to be convinced otherwise 16:15:48 Chris: Is that because the actions come from the page rather than browser chrome? 16:16:13 Tommy: Chrome sees Media Session being related to Audio Focus, so including slide decks doesn't really fit that 16:16:33 ... On whether they should be sent between origins, I don't have strong feelings, but Chrome security folks may have 16:17:34 Chris: I don't block progress on the WebRTC side, so determining the scope would help unblock 16:18:08 Tommy: If they're of a similar opinion, I'd recommend WebRTC progress with its own solution 16:18:40 Chris: There's a number of open issues, for different considerations 16:19:09 ... I'd like to review those to see which parts do fit Media Session 16:20:10 ... Open issues at https://github.com/w3c/mediasession/issues from 275 onward. Please do review and comment on those 16:21:25 Chris: What's the current thinking on Audio Focus, mentioned in #277 16:21:33 Tommy: I'll have to check 16:23:46 Chris: Is the Media Session spec clear about a video conference session vs a media playback session? 16:24:40 Tommy: It depends on the user agent, depending on what surface is used. We use some last focused logic to route actions 16:24:53 ... Up to the user agent to decide which actions to route to which tab 16:26:46 Chris: Want to get to a proposed resolution for Capture Handle Actions and identify the appropriate integration points with Media Session 16:27:11 Jer: Doesn't seem different in Safari. Tab most recently played back with a user gesture will be those to receive Media Session commands 16:27:53 Chris: Routing of telephony related actions 16:28:35 Jer: That's missing from the web generally. Audio Focus, lets the page determine which kind of audio is happening. Media Session would be the first place to introduce that 16:29:09 ... Doesn't seem to make sense. Audio Focus could be used by Media Session, useful outside of Media Session. It's just an explainer, not even in WICG 16:29:33 ... We've had requests for using an audio element for a notification sound, to change how it ducks or interrupts other playing audio 16:30:48 ... Media Session wouldn't help there. My opinion as an implementer, is if we want to add some signalling that audio playback isn't long form audio, Audio Focus would be a better place to do that, different kinds of generated audio 16:31:17 Tommy: That's useful, thanks for the historical context 16:31:34 Jer: I believe Becca was working on it, but don't know if someone else picked it up 16:33:30 Chris: Need for a video conferencing session concept, and where should that be specced. Is it in scope here? 16:34:05 Jer: A UA knows who's doing capturing and in what tab. It should also know if a page has registered capture related session actions 16:34:56 ... Do we need any explicit signalling from the page? If no capture is happening (if you're the remote listener, recipient of frames), that's the one place that may not know there's a conference happening 16:36:29 Chris: Please also have a look at the open issues, 274 onwards 16:37:06 Topic: MSE #306 (revisited) 16:37:20 Matt: We have some feedback from Mozilla on MSE in workers using a handle 16:37:48 ... I've landed the implementation described in spec. I have a response last night from Mozilla, make it Transferable for real, not use Serializable 16:38:12 ... From VideoFrame, we found Serializable has the necessary hooks to apply the restrictions I need for the handle, one per MediaSource 16:38:25 ... When you postMessage it, it can't be postMessaged further 16:39:13 ... As soon as the handle is set on a srcObject attribute, it's marked as used. There's a race condition. Once set, prevent serialization 16:39:30 ... The underlying MediaSource can enforce it's only attached once, and once simultaneuosly 16:40:18 ... Seems a no-brainer that it works with serialization. Issues with making it Transferable, harder to implement a transferable MediaSourceHandle 16:40:52 ... Have a narrowly defined case where it works, can only use the handle once. If we need it to be transferable, make it narrow now, relax later 16:41:14 ... Would like your input on Transferable vs Serializable. Folks are clamouring for this 16:41:31 Jer: What behaviour would be different between those? 16:42:30 Matt: Can serialize, which can throw exceptions, InvalidState. The app would use the handle and assign it to a srcObject attribute on a media element 16:42:46 ... That triggers the resource selection. You don't get a synchronous load attempt, that happens later 16:43:08 ... The approach I took to prevent surprises and interop issues is: when it's assiged to a srcObject, the handle cannot be serialized again 16:43:56 ... If it's ever been attached. If we change it later, would be surprising. If we want multiple handles from a MediaSource, could do later 16:44:17 Jer: For the use cases where someone wants to swap sources on a media element, you can't use the same object URL twice? 16:44:31 Matt: Yes. Does MSE in Workers say you can get multiple handles? 16:44:42 Jer: No, you'd need a different MediaSource 16:45:15 Jer: There's a mechanism where websites do ad insertion, creating an ad source. Can do today using multiple object URLs 16:45:24 Matt: So you'd have to create a new MediaSource 16:45:40 ... When you detach, all SourceBuffers are removed, goes into the closed state 16:45:58 ... So we may have over-restricted, but that's not in contention now 16:46:26 Jer: It's related insofar as if you want to support the same behaviour, you'd need a spec to create multiple object handles 16:47:02 Matt: One object per media source should help apps. not like shared ownership of MediaSource. The at most one handle for the MediaSource also has a reference to it 16:47:22 ... Helps implementers clarify, and developers understand what they need to free up 16:48:27 Jer: Works differently to setting srcObject null? There's a quirk in HTML about src with empty string, where you have to explicitly invoke the resource selection algorithm 16:48:46 Matt: May be a difference in how we implement. Any time the src is set we try to load it. 16:49:37 Jer: I'll have to check that quirk. We may have to change Safari's implemntation 16:50:07 ... So the big difference between Serializable and Transferable is with Serializable we have to add logic 16:50:44 ... Is there a page-visible difference between Serializable and Transferable 16:51:08 Matt: There are broadcast channels etc where there may be differences 16:51:43 Jer: Is this a spec argument, or how its used in practice? 16:52:01 Matt: It's a spec interop question. Want to avoid the pitfalls we hit in WebCodecs VideoFrame 16:52:32 Jer: That suggests there is a page-visible difference 16:52:50 Matt: When I say interop, I mean is it implementable in Chrome 16:52:57 Jer: I'm thinking of cross-browser interop 16:53:24 Matt: The way you do the transfer is signalled by the app. postMessage takes Transferables 16:53:58 ... Another way is if it's Transferable and not Serializable, but disagreement or misunderstanding on that 16:54:42 Jer: It seems, if you're making something Serializable but same as Transferable, weird as an implementer to have Transferable Serializable 16:55:03 ... I don't know if I care that much 16:55:32 Matt: Wanted to have move semantics for VideoFrame in WebCodecs, but this detail made it impossible to implement in practice 16:55:49 ... You cannot practically implement move semantics 16:56:33 ... So if you trap it at the serialization point, or if already assigned to a srcObject. Don't feel there's an issue on those constraints. Is Transferable practically going to be possible 16:57:53 Jer: A handle can exist in any context, so you'd need to be able to transfer to a shared worker and back 16:58:29 Matt: The interface is only on DedicatedWorker and Window. Would be interesting to know if the IDL bindings in Chrome prevent SharedWorker 16:58:56 ... If you're not doing a postMessage it won't work, makes no sense to put a handle into a database 16:59:12 Jer: Frameworks do weird things, it's use cases someone might try 17:00:03 ... I don't have a strong opinion. From a high level point of view, Transferable makes sense as the semantics match 17:00:44 ... If it's unimplementable, then fine. I'm a little worried that we'll face resistance if we need to change it later, if handles need to be both Transferable and Serializable 17:01:05 ... I could pull in some of the JavaScript people for input 17:03:42 Topic: TPAC planning 17:05:22 Chris: Planning Media WG for Friday afternoon, joint meetings with WebRTC, TTWG, MEIG on DataCue 17:06:01 Jer: There's also capabilities and CSS, not necessarily for a joint meeting 17:07:12 ... Audio playback capabilities, channels supported by user agents, in Audio WG. Spatialization flag in MC API, if you plug and unplug headphones, could change 17:09:03 s/Matt: Yes. Does MSE in Workers say you can get multiple handles?/Matt: Yes. Jer: Does MSE in Workers say you can get multiple handles?/ 17:09:23 s/Jer: No, you'd need a different MediaSource/Matt: No, you'd need a different MediaSource/ 17:13:49 s/There's a race condition./Otherwise There's a race condition./ 17:14:18 ... CSS doesn't handle audio 17:14:48 Chris: There's the HDR video plane issue open with CSS. Could request time in their agenda rather than a joint meeting 17:15:41 ... Suggestions welcome for how best to use our meeting time. Prioritised issues, new incubations (could be for the M&E IG meeting) 17:15:49 Jer: Those suggestions sound reasonable 17:16:01 Topic: Next meeting 17:16:30 Chris: July 12. Possibly another joint meeting before to follow up with WebRTC on Capture Handle Actions 17:16:37 [adjourned] 17:16:43 rrsagent, draft minutes 17:16:43 I have made the request to generate https://www.w3.org/2022/06/14-mediawg-minutes.html cpn 17:16:50 rrsagent, make log public 17:18:02 Chair: Chris, Jer 17:18:20 present+ Jer_Noble 17:18:22 rrsagent, draft minutes 17:18:22 I have made the request to generate https://www.w3.org/2022/06/14-mediawg-minutes.html cpn