IRC log of mediawg on 2022-06-14

Timestamps are in UTC.

15:44:30 [RRSAgent]
RRSAgent has joined #mediawg
15:44:30 [RRSAgent]
logging to https://www.w3.org/2022/06/14-mediawg-irc
15:44:32 [Zakim]
Zakim has joined #mediawg
15:44:42 [tidoust]
RRSAgent, make logs public
15:44:53 [tidoust]
Meeting: Media WG meeting
15:45:37 [tidoust]
Agenda: https://github.com/w3c/media-wg/blob/main/meetings/2022-06-14-Media_Working_Group_Teleconference-agenda.md
15:56:25 [Matt_Wolenetz]
Matt_Wolenetz has joined #mediawg
16:00:27 [Matt_Wolenetz]
present+
16:00:32 [tidoust]
Present+ Francois_Daoust
16:02:39 [cpn]
cpn has joined #mediawg
16:03:36 [tidoust]
present+ Joey_Parrish, Chris_Needham, Gary_Katsevman
16:03:43 [tidoust]
present+ Tommy_Steiner
16:04:19 [cpn]
Agenda: https://www.w3.org/events/meetings/6195de3d-2af0-471c-853d-b63396c562b6
16:04:22 [cpn]
scribe+ cpn
16:06:46 [cpn]
Topic: MSE #306
16:07:10 [cpn]
Matt: Chrome implementation is based on Serializable.
16:07:32 [cpn]
Chris: Was hoping to be able to discuss today with other implementers
16:07:54 [cpn]
Matt: Getting feedback for another approach would be unfortunate at this point
16:09:09 [cpn]
... The implemention relies on WebCodecs VideoFrame. The hope is Serializable is not blockable. Keep it restricted at the beginning, then relax those if needed.
16:09:21 [cpn]
... If it needs to become Transferable, we can add that on later
16:09:58 [cpn]
Chris: Do we need meeting time?
16:10:21 [cpn]
Matt: We've tried a couple of times. Getting responses from Mozilla, but having Apple chime in would be good
16:11:06 [cpn]
Chris: I'll follow up with Jer
16:11:20 [cpn]
Topic: Media Session
16:12:08 [Matt_Wolenetz]
s/Keep it restricted at the beginning/Approach is to initially keep the API narrow and restrict flexibility (to not "bake-in" API that allows too much only to restrict it later)/
16:12:18 [cpn]
Chris: Welcome Tommy and Youenn as co-editors
16:12:32 [cpn]
Tommy: I haven't sent a change to update the editors but will do soon
16:12:52 [cpn]
Regrets: Youenn_Fablet
16:13:19 [cpn]
Chris: Current discussion on Media Session and WebRTC Capture Handle Actions
16:14:11 [cpn]
... Question about the appropriate scope for Media Session. In particular. whether next slide / previous slide actions should be in Media Session?
16:14:39 [cpn]
Tommy: In my mind, it doesn't include that, doesn't really match. But open to be convinced otherwise
16:15:48 [cpn]
Chris: Is that because the actions come from the page rather than browser chrome?
16:16:13 [cpn]
Tommy: Chrome sees Media Session being related to Audio Focus, so including slide decks doesn't really fit that
16:16:33 [cpn]
... On whether they should be sent between origins, I don't have strong feelings, but Chrome security folks may have
16:17:34 [cpn]
Chris: I don't block progress on the WebRTC side, so determining the scope would help unblock
16:18:08 [cpn]
Tommy: If they're of a similar opinion, I'd recommend WebRTC progress with its own solution
16:18:40 [cpn]
Chris: There's a number of open issues, for different considerations
16:19:09 [cpn]
... I'd like to review those to see which parts do fit Media Session
16:20:10 [cpn]
... Open issues at https://github.com/w3c/mediasession/issues from 275 onward. Please do review and comment on those
16:21:25 [cpn]
Chris: What's the current thinking on Audio Focus, mentioned in #277
16:21:33 [cpn]
Tommy: I'll have to check
16:23:46 [cpn]
Chris: Is the Media Session spec clear about a video conference session vs a media playback session?
16:24:40 [cpn]
Tommy: It depends on the user agent, depending on what surface is used. We use some last focused logic to route actions
16:24:53 [cpn]
... Up to the user agent to decide which actions to route to which tab
16:26:46 [cpn]
Chris: Want to get to a proposed resolution for Capture Handle Actions and identify the appropriate integration points with Media Session
16:27:11 [cpn]
Jer: Doesn't seem different in Safari. Tab most recently played back with a user gesture will be those to receive Media Session commands
16:27:53 [cpn]
Chris: Routing of telephony related actions
16:28:35 [cpn]
Jer: That's missing from the web generally. Audio Focus, lets the page determine which kind of audio is happening. Media Session would be the first place to introduce that
16:29:09 [cpn]
... Doesn't seem to make sense. Audio Focus could be used by Media Session, useful outside of Media Session. It's just an explainer, not even in WICG
16:29:33 [cpn]
... We've had requests for using an audio element for a notification sound, to change how it ducks or interrupts other playing audio
16:30:48 [cpn]
... Media Session wouldn't help there. My opinion as an implementer, is if we want to add some signalling that audio playback isn't long form audio, Audio Focus would be a better place to do that, different kinds of generated audio
16:31:17 [cpn]
Tommy: That's useful, thanks for the historical context
16:31:34 [cpn]
Jer: I believe Becca was working on it, but don't know if someone else picked it up
16:33:30 [cpn]
Chris: Need for a video conferencing session concept, and where should that be specced. Is it in scope here?
16:34:05 [cpn]
Jer: A UA knows who's doing capturing and in what tab. It should also know if a page has registered capture related session actions
16:34:56 [cpn]
... Do we need any explicit signalling from the page? If no capture is happening (if you're the remote listener, recipient of frames), that's the one place that may not know there's a conference happening
16:36:29 [cpn]
Chris: Please also have a look at the open issues, 274 onwards
16:37:06 [cpn]
Topic: MSE #306 (revisited)
16:37:20 [cpn]
Matt: We have some feedback from Mozilla on MSE in workers using a handle
16:37:48 [cpn]
... I've landed the implementation described in spec. I have a response last night from Mozilla, make it Transferable for real, not use Serializable
16:38:12 [cpn]
... From VideoFrame, we found Serializable has the necessary hooks to apply the restrictions I need for the handle, one per MediaSource
16:38:25 [cpn]
... When you postMessage it, it can't be postMessaged further
16:39:13 [cpn]
... As soon as the handle is set on a srcObject attribute, it's marked as used. There's a race condition. Once set, prevent serialization
16:39:30 [cpn]
... The underlying MediaSource can enforce it's only attached once, and once simultaneuosly
16:40:18 [cpn]
... Seems a no-brainer that it works with serialization. Issues with making it Transferable, harder to implement a transferable MediaSourceHandle
16:40:52 [cpn]
... Have a narrowly defined case where it works, can only use the handle once. If we need it to be transferable, make it narrow now, relax later
16:41:14 [cpn]
... Would like your input on Transferable vs Serializable. Folks are clamouring for this
16:41:31 [cpn]
Jer: What behaviour would be different between those?
16:42:30 [cpn]
Matt: Can serialize, which can throw exceptions, InvalidState. The app would use the handle and assign it to a srcObject attribute on a media element
16:42:46 [cpn]
... That triggers the resource selection. You don't get a synchronous load attempt, that happens later
16:43:08 [cpn]
... The approach I took to prevent surprises and interop issues is: when it's assiged to a srcObject, the handle cannot be serialized again
16:43:56 [cpn]
... If it's ever been attached. If we change it later, would be surprising. If we want multiple handles from a MediaSource, could do later
16:44:17 [cpn]
Jer: For the use cases where someone wants to swap sources on a media element, you can't use the same object URL twice?
16:44:31 [cpn]
Matt: Yes. Does MSE in Workers say you can get multiple handles?
16:44:42 [cpn]
Jer: No, you'd need a different MediaSource
16:45:15 [cpn]
Jer: There's a mechanism where websites do ad insertion, creating an ad source. Can do today using multiple object URLs
16:45:24 [cpn]
Matt: So you'd have to create a new MediaSource
16:45:40 [cpn]
... When you detach, all SourceBuffers are removed, goes into the closed state
16:45:58 [cpn]
... So we may have over-restricted, but that's not in contention now
16:46:26 [cpn]
Jer: It's related insofar as if you want to support the same behaviour, you'd need a spec to create multiple object handles
16:47:02 [cpn]
Matt: One object per media source should help apps. not like shared ownership of MediaSource. The at most one handle for the MediaSource also has a reference to it
16:47:22 [cpn]
... Helps implementers clarify, and developers understand what they need to free up
16:48:27 [cpn]
Jer: Works differently to setting srcObject null? There's a quirk in HTML about src with empty string, where you have to explicitly invoke the resource selection algorithm
16:48:46 [cpn]
Matt: May be a difference in how we implement. Any time the src is set we try to load it.
16:49:37 [cpn]
Jer: I'll have to check that quirk. We may have to change Safari's implemntation
16:50:07 [cpn]
... So the big difference between Serializable and Transferable is with Serializable we have to add logic
16:50:44 [cpn]
... Is there a page-visible difference between Serializable and Transferable
16:51:08 [cpn]
Matt: There are broadcast channels etc where there may be differences
16:51:43 [cpn]
Jer: Is this a spec argument, or how its used in practice?
16:52:01 [cpn]
Matt: It's a spec interop question. Want to avoid the pitfalls we hit in WebCodecs VideoFrame
16:52:32 [cpn]
Jer: That suggests there is a page-visible difference
16:52:50 [cpn]
Matt: When I say interop, I mean is it implementable in Chrome
16:52:57 [cpn]
Jer: I'm thinking of cross-browser interop
16:53:24 [cpn]
Matt: The way you do the transfer is signalled by the app. postMessage takes Transferables
16:53:58 [cpn]
... Another way is if it's Transferable and not Serializable, but disagreement or misunderstanding on that
16:54:42 [cpn]
Jer: It seems, if you're making something Serializable but same as Transferable, weird as an implementer to have Transferable Serializable
16:55:03 [cpn]
... I don't know if I care that much
16:55:32 [cpn]
Matt: Wanted to have move semantics for VideoFrame in WebCodecs, but this detail made it impossible to implement in practice
16:55:49 [cpn]
... You cannot practically implement move semantics
16:56:33 [cpn]
... So if you trap it at the serialization point, or if already assigned to a srcObject. Don't feel there's an issue on those constraints. Is Transferable practically going to be possible
16:57:53 [cpn]
Jer: A handle can exist in any context, so you'd need to be able to transfer to a shared worker and back
16:58:29 [cpn]
Matt: The interface is only on DedicatedWorker and Window. Would be interesting to know if the IDL bindings in Chrome prevent SharedWorker
16:58:56 [cpn]
... If you're not doing a postMessage it won't work, makes no sense to put a handle into a database
16:59:12 [cpn]
Jer: Frameworks do weird things, it's use cases someone might try
17:00:03 [cpn]
... I don't have a strong opinion. From a high level point of view, Transferable makes sense as the semantics match
17:00:44 [cpn]
... If it's unimplementable, then fine. I'm a little worried that we'll face resistance if we need to change it later, if handles need to be both Transferable and Serializable
17:01:05 [cpn]
... I could pull in some of the JavaScript people for input
17:03:42 [cpn]
Topic: TPAC planning
17:05:22 [cpn]
Chris: Planning Media WG for Friday afternoon, joint meetings with WebRTC, TTWG, MEIG on DataCue
17:06:01 [cpn]
Jer: There's also capabilities and CSS, not necessarily for a joint meeting
17:07:12 [cpn]
... Audio playback capabilities, channels supported by user agents, in Audio WG. Spatialization flag in MC API, if you plug and unplug headphones, could change
17:09:03 [Matt_Wolenetz]
s/Matt: Yes. Does MSE in Workers say you can get multiple handles?/Matt: Yes. Jer: Does MSE in Workers say you can get multiple handles?/
17:09:23 [Matt_Wolenetz]
s/Jer: No, you'd need a different MediaSource/Matt: No, you'd need a different MediaSource/
17:13:49 [Matt_Wolenetz]
s/There's a race condition./Otherwise There's a race condition./
17:14:18 [cpn]
... CSS doesn't handle audio
17:14:48 [cpn]
Chris: There's the HDR video plane issue open with CSS. Could request time in their agenda rather than a joint meeting
17:15:41 [cpn]
... Suggestions welcome for how best to use our meeting time. Prioritised issues, new incubations (could be for the M&E IG meeting)
17:15:49 [cpn]
Jer: Those suggestions sound reasonable
17:16:01 [cpn]
Topic: Next meeting
17:16:30 [cpn]
Chris: July 12. Possibly another joint meeting before to follow up with WebRTC on Capture Handle Actions
17:16:37 [cpn]
[adjourned]
17:16:43 [cpn]
rrsagent, draft minutes
17:16:43 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/06/14-mediawg-minutes.html cpn
17:16:50 [cpn]
rrsagent, make log public
17:18:02 [cpn]
Chair: Chris, Jer
17:18:20 [cpn]
present+ Jer_Noble
17:18:22 [cpn]
rrsagent, draft minutes
17:18:22 [RRSAgent]
I have made the request to generate https://www.w3.org/2022/06/14-mediawg-minutes.html cpn