21:58:30 RRSAgent has joined #mediawg 21:58:34 logging to https://www.w3.org/2024/01/09-mediawg-irc 21:58:37 Zakim has joined #mediawg 21:58:53 Meeting: Media WG 21:59:08 Agenda: https://www.w3.org/events/meetings/45b57f00-30f0-498b-ae71-b9241a515314/ 21:59:35 RRSAgent, make logs public 22:02:11 present+ Chris_Needham, Jan-Ivar Bruaroey, Bernard_Aboba, Mark_Watson, Youenn_Fablet, Francois_Daoust, Sun_Shin, Johannes_Kron, Thomas_Guilbert, Jean-Yves_Avenard 22:02:33 Chair: Chris_Needham, Marcos_Caceres 22:02:44 present+ Marcos_Caceres, Jer_Noble 22:03:26 present+ Mark_Foltz 22:03:29 markw has joined #mediawg 22:03:37 present+ Eugene_Zemtov 22:03:42 Present+ Tommy_Steimel, Mark_Foltz 22:03:48 mfoltzgoogle has joined #mediawg 22:03:54 Present+ Mark_Foltz 22:03:59 present+ markw 22:04:14 present+ Tommy_Steimel 22:05:28 scribe+ 22:05:41 marcos has joined #mediawg 22:05:50 present+ Dale_Curtis 22:07:07 Topic: Media Session 22:07:27 subtopic: Relationship between toggle microphone/camera actions and MediaStreamTrack mute/unmute events 22:07:54 jan-ivar: See -> https://github.com/w3c/mediasession/issues/307 issue #307 22:08:09 ... To my knowledge, no relationship for now, which is a bit unfortunate. 22:08:45 ... Also implementation in Chrome. Chrome never mutes the track. 22:09:17 ... When user clicks the buttons, these buttons are 100% javascript controlled. Which might be fine for picture-in-picture. 22:09:51 ... Users may have some expectation that a web page cannot toggle camera when they used the buttons. 22:10:47 ... User agents could decide to mute tracks based on whether the user clicked the toggle buttons, but there are some challenges with the API because the API can maintain the state. 22:10:58 ... Can we trust the web page with that? 22:11:17 ... If we don't, we end up with the double mute issue, which has some advantages and disadvantages. 22:11:22 ... Some proposals in the issue. 22:12:19 Youenn: I would maybe add that it would be interesting for us to use the Media Session API to mute/unmute capture. That seems reasonable. Right now, focused in PiP. Safari UI is pretty similar though. 22:12:51 ... The Safari UA does not trust the web page though. This is different from implementation in Chrome. 22:13:10 ... One model where UA would trust the web page, one model where UA would not trust the web page. 22:13:34 ... I would prefer to move to a world where the UA does not trust the web page, but that may be for another time. 22:14:05 ... It seems that we are mostly aligned on extending the API for the model where the UA does not trust the web page. 22:14:40 Tommy: So, setMicrophone(active) would return a Promise and maybe fail? 22:14:47 Jan-Ivar: Right? 22:14:52 s/Right?/Right. 22:14:58 Tommy: That seems fine. 22:15:38 Youenn: We might want to tighten the spec too, e.g., mute/unmute events might fire after the callback, these kinds of things. 22:16:19 ... When you get tracks, each track has a muted flag, with capture flag has well. 22:16:50 Tommy: So mute event is really there to tell the web page that you're about to mute, but there's nothing they can do. 22:16:57 Youenn: They could stop capture. 22:17:22 Tommy: In the Chrome case, the application would be responsible. In the Safari case, the UA would be. 22:17:25 Youenn: Right. 22:18:03 Jan-Ivar: So the goal is to synchronize the mute cases to reduce the double mute problem. 22:18:09 Youenn: Yes. 22:18:21 Tommy: I don't have a problem with that. 22:18:51 s/setMicrophone(active)/setMicrophoneActive() 22:19:03 Youenn: OK, then I may prepare two PRs. 22:19:49 Tommy: Sometimes, web sites will continue to listen to understand when someone is speaking while muted. 22:20:30 Youenn: Longer plan would be to have a dedicated event in muted tracks to alert when users are speaking while the track is muted. Some support at the OS level. Not spec-ed yet though. 22:20:39 Tommy: OK, that seems reasonable to me. 22:22:31 Youenn: The toggleMicrophone thing might become ambiguous. When you toggle, you might now know what the current state is. In the future, we may want to expose additional info, especially in the mode where the page is not trusted. 22:22:52 Tommy: Yes, there could be a race here. 22:22:59 Youenn: I'll file another issue for that. 22:23:25 ... One PR for the API change setMicrophoneActive, and one PR for mentioning that the mute events can fire after. 22:23:44 cpn: This issue was spawned as part of the double mute issue on media capture. 22:23:54 ... To be followed up in the WebRTC WG. 22:24:25 Youenn: Yes, I think the WebRTC WG should discuss this. 22:25:13 Jan-Ivar: the PR should also address -> https://github.com/w3c/mediasession/issues/279 #279 22:25:41 Youenn: To be closed once the PR is ready and merged. 22:25:47 Topic: Media Capabilities 22:26:27 cpn: I did an issue triage before Christmas. I added labels to some of these. Happy to adjust things based on your own prioritization. 22:26:48 ... For the V1 milestone, really clarification issues. 22:27:17 ... For the V2 milestone, extensions to the current API, such as the transition API, text track capabilities, and audio rendering capabilities. 22:27:33 ... Depending on priorities, we may want to move issues around between V2 and V1. 22:28:49 cpn: Next thing we talked about last time were the privacy issue raised by PING. We need to review that and answer their question more specifically about why we have a capabilities API to start with, as opposed to letting the user agent pick up the right option. 22:29:20 ... Call for people to help get started on this 22:30:02 Bernard: I can help with some of it. Some differences, but some similarities. In WebRTC, media is negotiated. 22:30:26 cpn: Thanks, that would be useful. I haven't looked at the current questionnaire. 22:30:39 Bernard: I've had the same questions come up in the WebRTC SVC case. 22:31:15 cpn: If you can start something and then we can look at it from an adaptive streaming perspective. 22:31:25 Bernard: Yes, these two areas may have slightly different answers. 22:32:12 ... Did the same questions come up in WebCodecs with isTypeSupported() 22:32:25 Eugene: I don't think we requested review from PING yet. 22:32:41 cpn: I'll check the status. 22:34:01 cpn: Given that we have you and Youenn, I'd like to explore whether it makes sense to have separate capabilities APIs. 22:34:34 Jer: Do they return the same information? 22:35:02 Youenn: Some differences between isConfigSupported and media capabilities. Related but not exactly the same thing. 22:35:18 ... Smoothness and power efficiency are good examples. 22:35:48 ... We could add a note on how developers could approach this (real time or not real time scenarios). 22:37:24 Eugene: Example: Encode video in 60fps in high definition. Most devices won't support that. Currently the spec does not say that it shoul be rejected because frame rate won't be supported. 22:37:28 ... But it could be extended. 22:38:17 ... WebCodecs allows people to do lots of different things. Different people mean different things when they use WebCodecs. I wouldn't add anything to Media Capabilities related to WebCodecs. 22:39:00 Jer: For a quick overview of isConfigSupported, it seems that it could be possible to define it in terms of the Media Capabilities. 22:39:33 ... With the update on smoothness linked to real-time scenarios. 22:39:53 Bernard: Except that it returns a configuration. 22:40:22 Eugene: If UA does not support some new key, it will return the configuration that it understands without the options that the developer requested. 22:40:58 Jer: OK, that seems similar to an issue we discussed last time, where Safari returns the dictionary paramters the way it understood them. 22:41:57 Bernard: In WebCodecs, you can have codec specific stuff that won't be in Media Capabilities. Per-frame QP for instance. 22:42:09 ... Not exposed through MIME type parameters. 22:42:19 ... It gives information at a more detailed level. 22:42:32 ... About what an encoder/decoder can do. 22:42:45 Jer: OK, I think I haven't looked into this into details. 22:43:29 Bernard: Side example of a feature for AV1, that we may want to revisit. 22:43:53 Jer: Another API that encodes media may want to expose the same information that we expose in WebCodecs. 22:44:17 Eugene: WebCodecs is designed to be low-level. Other media APIs are more for people who want video to just work. 22:44:45 Jer: I understand that. But it would be problematic to force people into WebCodecs if they need the detailed information for some reason. 22:44:52 ... We can talk about that in the future. 22:45:20 Bernard: Yes, example of contentHints in WebRTC and WebCodecs and they're not quite the same in both contexts. 22:46:10 Dale: The Media Capabilities API could perhaps need to ingest WebCodecs stuff. 22:46:35 ... Because there are so many WebCodecs parameters. 22:46:59 Jer: Unifying places where you can get answers about media capabilities seems good to me. 22:47:20 Bernard: The input could look very different for WebRTC and WebCodecs. 22:47:42 ... It's not clear to me that we wouldn't complicate Media Capabilities. 22:48:38 ... An example from WebRTC is resolution, handled by the user agent, and you might get lower resolution because CPU is busy for instance. 22:48:47 ... A little different from what you would get from WebCodecs. 22:49:43 Youenn: If I understand things, we're thinking that adding WebCodecs to Media Capabilities is something big and potentially difficult. Perhaps worth doing in the future. 22:50:16 ... We may still get developers who may want to express media capabilities in terms of WebCodecs parameters. 22:51:26 cpn: Next issue that I prioritized is around AudioConfiguration, and whether we should be distinguishing the decode capabilities from the rendering capabilities, as done for video. 22:52:00 ... For the channel configuration, right now it's a string, and I don't know what you can query through this. 22:52:25 ... Some developer need to tell whether they can deliver multi channel or whether the browser is going to downmix that to stereo. 22:52:49 ... Chris Cunningham put together a document at the time. 22:52:56 ... Is this something that we want to work on? 22:53:15 ... Perhaps for today, we can just look at it from a priorization point of view. 22:54:00 ... The document explores how we might add this through Media Audio Output Devices API. 22:55:38 Dale: The stuff about channel count is more on the Web Audio side. Developers may want to have something where they can specify through a dictionary whis channel is which. 22:56:22 jer: We heard through Netflix the need to know support for advance Dolby channels. 22:57:02 ... We have received feedback on spatial rendering but not on the number of channels per se. Multichannel or stereo, essentially. 22:58:45 jan-ivar: If there's a need to expose more information in the Audio Output Devices API, that would be in the WebRTC WG. 22:59:25 cpn: Also some issue on trimming down the AudioConfiguration, with a proposal to remove features not used in Chromium. 22:59:46 ... Other implementations may still find them useful, my suggestion is to leave them in. 23:01:03 cpn: If anybody has editorial capacity to work on some of these things, there's a number of things where it's just about working on spec text. 23:01:20 ... I labeled them with a "pr-needed" label. 23:01:59 Marcosc has joined #mediawg 23:02:10 ... There are a number of other issues for which we may need a wider feedback on. I propose to bring that to the Media & Entertainment IG to collect industry input. 23:02:45 Topic: Next call 23:03:11 cpn: Next call on 13 February. 23:03:15 RRSAgent, draft minutes 23:03:16 I have made the request to generate https://www.w3.org/2024/01/09-mediawg-minutes.html tidoust