20:53:13 RRSAgent has joined #me 20:53:17 logging to https://www.w3.org/2023/07/25-me-irc 20:53:17 Zakim has joined #me 20:57:10 Meeting: MEIG meeting 20:57:18 Chair: Chris_Needham 20:57:19 scribe+ cpn 20:57:44 Agenda: https://www.w3.org/events/meetings/b96c0bbd-7923-4c84-9acf-28c063bc7050/ 20:58:46 nhk-ryo has joined #me 20:59:20 present+ Yoshiharu_Dewa, Hisayuki_Ohmata, Jean-Yves_Avenard 20:59:37 ohmata has joined #me 21:00:22 present+ Kaz_Ashimura 21:00:36 present+ John_Riviello, Alicia_Boya, Louay_Bassbouss 21:01:32 present+ Chris_Needham 21:01:37 present+ Kazuyuki_Ashimura, Tobias_Heinl, Javier_Arellano 21:02:04 present+ Ryo_Yasuoka 21:02:16 present+ Satoshi_Nishimura 21:02:32 rrsagent, make log public 21:02:36 rrsagent, draft minutes 21:02:37 I have made the request to generate https://www.w3.org/2023/07/25-me-minutes.html kaz 21:03:03 tidoust has joined #me 21:03:59 present+ Francois_Daoust 21:04:34 Louay has joined #me 21:04:34 agenda: https://lists.w3.org/Archives/Public/public-web-and-tv/2023Jul/0017.html 21:04:39 JohnRiv has joined #me 21:04:47 present+ 21:04:53 jya has joined #me 21:04:57 aboya has joined #me 21:05:09 present+ Tatsuya_Igarashi 21:05:32 present+ Joey_Parrish 21:05:33 wschildbach has joined #me 21:05:37 rrsagent, draft minutes 21:05:38 I have made the request to generate https://www.w3.org/2023/07/25-me-minutes.html kaz 21:05:44 igarashi has joined #me 21:05:46 Topic: Managed Media Source 21:06:07 present+ 21:06:13 jya: The primary reason we came up with MMS is that Apple wanted to add MSE support for iPhone 21:06:16 Present+ Louay_bassbouss 21:06:31 ... It has been supported on all Apple devices: iPad OS, Mac, WebKit for quite a while 21:07:02 .. But since MSE was introduced it was never allowed on iPhone. It has HLS which gives 100% control to the device on how we're going to fetch the content 21:07:16 present+ Mike_Bergman 21:07:24 ... The device can detect the resolution to be used, has deeper knowledge on what the user has asked, there's less privacy concern 21:07:45 ... WebKit is open source, we wanted to have MSE. The issue was how to introduce it on iPhone, primary concern to reduce power usage 21:08:01 present+ Wolfgang_Scildbach 21:08:06 rrsagent, draft minutes 21:08:07 I have made the request to generate https://www.w3.org/2023/07/25-me-minutes.html kaz 21:08:13 ... Test showed that power usage would go up with MSE, so viewing time would reduce. We can never introduce a regression on iPhone 21:08:36 ... We were given constraints: if we play a Youtube video on iPhone with MSE enable, we shouldn't see a power regression 21:09:07 ... We tried various ways, and what we found was the primary reason why the power is used, is websites get one segment at a time rather than downloading in burst 21:09:07 present+ Yasser_Syed 21:09:22 present+ Zachary_Cava 21:09:28 rrsagent, draft minutes 21:09:29 I have made the request to generate https://www.w3.org/2023/07/25-me-minutes.html kaz 21:10:06 ... The main power usage is when the cellular modem is on. So we looked at how HLS downloads data, it downloads quite a big chunk, then stop 21:10:06 ... 5G is more hungry than 4G or 3G, it will download more seconds then shut down 21:10:27 ... Apple still wanted to have control, provide hints. we didn't enable MSE iPhone, as we'd have those issues 21:10:40 myles_ has joined #me 21:10:40 ... MMS provides hints, start fetching content, i have enough, stop fetching content 21:11:13 ... We wanted to enable on 5G, do we need a carrot and stick? If website does the wrong thing, and find a way around it, shut it off completely, and don't enable MSE and fallback to HLS 21:11:36 ... We found, If we disable the 5G modem, we found the power usage the same if using HLS vs MSE 21:11:46 ... The website can do what it wants and it will still be alright 21:12:20 ... MMS evolved to being simply trying to force the site to detecting when internally when to enable the 5G modem 21:12:40 ... If the page sticks within those events and only enable 5G during those times, the power usage is acceptable and there's no regression over HLS 21:13:04 ... MMS allows Webkit to be power efficient on iphone, allolwing easy transition from your website so MMS 21:13:10 s/so/to/ 21:13:36 ... We needed something different to MSE to avoid regression, so there's a new API which is MMS which can be dealt with independently 21:14:24 ... Webkit didn't want regression from a user perspecitive, e.g., airplay not compatible with MSE. We want the site to be aware, they must disable remote playback 21:14:36 ... So it's on purpose that you introduce the regression 21:15:10 ... MMS will ship on iPad OS and Mac, it's behind an experimental flag on iPhone. iOS17 I think, in Safari settings there's a feature flag to enable 21:15:30 ... That's how webkit came up with MMS. For implementation it was designed to allow easy transition from one to the other 21:15:43 ... From discussion with HLS.js folks, we looked at how we can help with things missing in MSE 21:16:11 ... Having more detail on what's evicted and what's added was useful. It's common to all players, they need a way to scan the buffer range, timeranges 21:16:22 ... It's easier for the UA to perform that operation than the JS player 21:16:42 ... There are things we wanted to add, with HLS the phone can know the user has selected low data usage, so we can limit the resolution to fetch 21:17:09 ... Unfortunately, providing the quality attribute in the spec may not be possible from a privacy perspective, could be a fingerprinting vector 21:17:20 ... So we have to find a way that can be done or drop it from the spec 21:17:40 ... That the inner detail on how we came up with MMS, we've attended FOMS and TPAC meetings where we got feedback on what's missing in MSE 21:18:06 ... We tried to add many of those features, in particular on SourceBuffer, so I'm hoping we achieve that goal, not 100% if that's the case or not 21:18:51 present+ Thasso_Griebel 21:19:14 Mike: If a player switches to MMS, how much work is dont in the player and how much in the implementation? How much cnage inthe player? 21:19:54 s/is dont/is done/ 21:19:55 jya: very little change. If a MSE player wants to work on iphone, you just need to change the call to ManagedMediaSource, add the disable remote playback flag, use plain mp4 or HLS 21:20:32 ... That doesn't mean you get the best performance, that's the minimum to get it working. Requirement for an alternate player isn't part of the spec 21:21:22 ... The existing HTML5 spec for the video allows you to provide a src attribute, a with different links, one for your MSE source and for somehting compatible with remote playback, iPhone currently plays 21:22:21 ... If in your existing player you listen to the start/endstraming events and you only fetch data between those, you'll get access to the 5G modem, so faster start time and higher quality playback and better power 21:23:03 Alicia: Thanks for this work. Something I noticed is there seems to be three different problems it's trying to fix 21:23:41 ... not related problems: one is performance problem with fetching media reducing power usage. I like the start/stop streaming events 21:24:10 ... because right now it's a big challenge for anyone making an MSE player. They have to make that choice but they're not in the best place to make that choice, so handholding is good 21:24:44 ... The airplay alternative is a nice thing, but I don't see it as related. Why couldn't we have alternative playback urls and non-managed MediaSource? 21:25:12 ... Also, I saw the buffer change events, it would be good to have as events instead having ht eclient check periodically. I think it should exist in fullblown MSE too 21:25:28 ... Haven't seen if those will be added to both? 21:25:56 s/ht eclient/the client/ 21:25:57 jya: The reuqirement for an Airplay source alternative is not something I'm super happy with, it's not an engineering decision 21:26:37 Alicia: I get that 21:27:06 ... Could the events work with both MMS and MSE? 21:27:23 ... So the limitation is only on the iPhone implementation, not part of the spec? 21:27:58 jya: Was added in Safari last year, hasn't changed in the spec. I'd like to refer you to the GitHub issue for your last question for the changed event, Mark Watson asked the same question 21:28:07 ... https://github.com/w3c/media-source/issues/320 21:28:45 cpn: Please do leave comments in the issue 21:29:41 rrsagent, draft minutes 21:29:42 I have made the request to generate https://www.w3.org/2023/07/25-me-minutes.html kaz 21:29:48 Joey: I was wondering how the notifications of how to fetch content, what interval are they on and how does it interact with low latency live streams? 21:30:04 jya: Interval is up to the UA to define, for iPhone, the low water mark is about 10 seconds, high is 30 21:30:26 ... For live streaming, the expected behaviour is that if you're in a live stream, you won't follow the start/end streaming events, you'll fetch all the time 21:30:51 ... But if you fetch outside those times, the 5G modem on iPhone is disabled and it's like MSE, using 4G 21:31:00 ... Whether other UAs will do that, I don't know 21:31:16 Joey: Is expectation that if you're at the live edge, those will be overruled? 21:31:35 jya: The spec has language that appends could fail if you append when it doesn't want you to 21:32:01 ... It's a remnant of the stick approach. We left it in the proposal, if you don't append between start/end streaming the UA is allowed to deny the append (throw) 21:32:33 ... That's not something webkit has implemented. It's also mentioned in the GH issue, if it's to be removed, we'll have a follow up discussion on whether to remove or not 21:32:57 Joey: As a player developer sohuld we ignore the events when straming at live edge or if there's a buffer underflow? What#s the guidance? 21:33:19 jya: the events are hints, it's only going to affect how it works under the hood, so enabling of the 5G modem 21:33:49 ... If you want the lowest latency possible and you start appending constantly, it's a strong signal that you're playing at the live edge 21:34:24 ... we discussed internally and at previous FOMS about having a dedicated live mode, but we found with just start/end streaming and if player respects those events to determine if the content is live 21:34:57 Joey: Would you be willing to write guidance for player developers? Would be good to encourage players to use the API in ways implementers prefer to use it 21:35:05 jya: I touched on those in the WWDC video 21:35:12 ... https://developer.apple.com/videos/play/wwdc2023/10122/ 21:35:30 Joey: It woundn't be in the spec, just a way to help the community do the right thing 21:36:00 jya: In summary, if you respect the start/end events you get the 5G model. We can see abuot a blog post 21:36:44 Alicia: Is there any performance gain for other than mobile phones, e..g, for set top boxes? 21:37:05 jya: On Mac you get better power usage regardless 21:37:54 Alicia: For a desktop or laptop on wifi, similar for STB. Any gains in those kinds of devices? 21:38:20 jya: Frrom a performacne perspective, Twitch on Firefox, as FF doesn't support MPEG-TS you append one frame at a time 21:38:43 ... If you do that, the CPU is working all the time, has to check if each frame is in existing buffered range 21:39:06 ... If they were to batch their content, but twitch wants lowest latency possible 21:39:40 Alicia: May be not so much a technical problem, but pushing them to do the right thing 21:39:59 jya: It's a fundamental issue between VOD and live, you can't have it all 21:40:21 Alicia: More pracically, many of our customers work with STBs, I work on maintaining implementations of MSE in WebKit for media boxes 21:40:43 ... Could I have good things to sell them on MMS, or nothing to offer beyond followign the standard? 21:41:07 jya: Probably not. One benefit is having the latest webkit version. STBs we know don't update often 21:41:40 Alicia: They also have their own performance constraints. In STBs power can be used in content switching done in the browser 21:42:13 jya: If you do things in batch and increase latency, you'll increase performance. MMS favours doing things in batch 21:42:31 Alicia: And batching also increases memory usage, which is an issue on STBs 21:43:04 cpn: Alicia, what kind of improvement to MSE would you be looking for? 21:43:28 Alicia: Don't know if it handles running out of memory, was that mentioned in MMS? 21:44:02 jya: It is, that's a good point. MMS is allowed to evict at any time, not just during appendBuffer, specifically for low memory devices 21:44:31 Alicia: Extremely useful, you don't know how much RAM the device has. You can have the problem where a website stores too much in memory, the STB might start swapping 21:44:54 jya: Forcing a lower memory footprint in the player can be added on the UA side if using MMS 21:45:46 Alicia: I recall we have some settings to force memory usage precisely because this is a rpoblem. video players assume you have as much RAM as on a desktop, so couldn't really work following the spec 21:45:54 ... Good to have hints more than surprise sticks 21:46:08 jya: MMS allows eviction then the bufferchange event is fired 21:46:47 Thasso: Coming back to Joey's point, the core issue I see in the GH issue is discrepancy between when to download media and when not to, fine for 5G usage 21:47:36 ... but gives the ability for the UA to block the append. Maybe this could be removed in a follow up PR, but could it be removed from the proposal, it's a big concern if reject the append depending on when you fetch the data? 21:48:14 jya: I guess it's a matter of you never know. it may be it'll never be used 21:48:33 Thasso: So could it be removed from the proposal now and add it later if they need that ability 21:48:38 jya: I don't know 21:48:55 Alicia: Would it conflict with the live playback case? 21:49:25 Thasso: It doesn't, I'm good if i fetch between the events, but if UA decides later to reject, I'll have the problem and the player gets more complex 21:49:44 ... It also affects VOD playback where players maintain their own buffers outside of MSE, as buffer managemnet in MSE is tricky 21:49:54 ... If the implementation isn't there, why allow it in the spec in the first place? 21:50:50 jya: It's an ongoing discussion in the issue, may be best to take it offline 21:51:11 cpn: Please raise this in the GH issue, and the Media WG will follow up on it 21:53:33 ... the GH thread is getting long so we may want to break it into specific issues for different questions 21:54:59 rrsagent, draft minutes 21:55:00 I have made the request to generate https://www.w3.org/2023/07/25-me-minutes.html kaz 21:55:19 [adjourned] 21:55:29 rrsagent, draft minutes 21:55:31 I have made the request to generate https://www.w3.org/2023/07/25-me-minutes.html kaz