IRC log of me on 2023-07-25

Timestamps are in UTC.

20:53:13 [RRSAgent]
RRSAgent has joined #me
20:53:17 [RRSAgent]
logging to https://www.w3.org/2023/07/25-me-irc
20:53:17 [Zakim]
Zakim has joined #me
20:57:10 [cpn]
Meeting: MEIG meeting
20:57:18 [cpn]
Chair: Chris_Needham
20:57:19 [cpn]
scribe+ cpn
20:57:44 [cpn]
Agenda: https://www.w3.org/events/meetings/b96c0bbd-7923-4c84-9acf-28c063bc7050/
20:58:46 [nhk-ryo]
nhk-ryo has joined #me
20:59:20 [cpn]
present+ Yoshiharu_Dewa, Hisayuki_Ohmata, Jean-Yves_Avenard
20:59:37 [ohmata]
ohmata has joined #me
21:00:22 [kaz]
present+ Kaz_Ashimura
21:00:36 [cpn]
present+ John_Riviello, Alicia_Boya, Louay_Bassbouss
21:01:32 [kaz]
present+ Chris_Needham
21:01:37 [cpn]
present+ Kazuyuki_Ashimura, Tobias_Heinl, Javier_Arellano
21:02:04 [kaz]
present+ Ryo_Yasuoka
21:02:16 [kaz]
present+ Satoshi_Nishimura
21:02:32 [kaz]
rrsagent, make log public
21:02:36 [kaz]
rrsagent, draft minutes
21:02:37 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/07/25-me-minutes.html kaz
21:03:03 [tidoust]
tidoust has joined #me
21:03:59 [kaz]
present+ Francois_Daoust
21:04:34 [Louay]
Louay has joined #me
21:04:34 [kaz]
agenda: https://lists.w3.org/Archives/Public/public-web-and-tv/2023Jul/0017.html
21:04:39 [JohnRiv]
JohnRiv has joined #me
21:04:47 [JohnRiv]
present+
21:04:53 [jya]
jya has joined #me
21:04:57 [aboya]
aboya has joined #me
21:05:09 [kaz]
present+ Tatsuya_Igarashi
21:05:32 [kaz]
present+ Joey_Parrish
21:05:33 [wschildbach]
wschildbach has joined #me
21:05:37 [kaz]
rrsagent, draft minutes
21:05:38 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/07/25-me-minutes.html kaz
21:05:44 [igarashi]
igarashi has joined #me
21:05:46 [cpn]
Topic: Managed Media Source
21:06:07 [igarashi]
present+
21:06:13 [cpn]
jya: The primary reason we came up with MMS is that Apple wanted to add MSE support for iPhone
21:06:16 [Louay]
Present+ Louay_bassbouss
21:06:31 [cpn]
... It has been supported on all Apple devices: iPad OS, Mac, WebKit for quite a while
21:07:02 [cpn]
.. But since MSE was introduced it was never allowed on iPhone. It has HLS which gives 100% control to the device on how we're going to fetch the content
21:07:16 [kaz]
present+ Mike_Bergman
21:07:24 [cpn]
... The device can detect the resolution to be used, has deeper knowledge on what the user has asked, there's less privacy concern
21:07:45 [cpn]
... WebKit is open source, we wanted to have MSE. The issue was how to introduce it on iPhone, primary concern to reduce power usage
21:08:01 [kaz]
present+ Wolfgang_Scildbach
21:08:06 [kaz]
rrsagent, draft minutes
21:08:07 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/07/25-me-minutes.html kaz
21:08:13 [cpn]
... Test showed that power usage would go up with MSE, so viewing time would reduce. We can never introduce a regression on iPhone
21:08:36 [cpn]
... We were given constraints: if we play a Youtube video on iPhone with MSE enable, we shouldn't see a power regression
21:09:07 [cpn]
... We tried various ways, and what we found was the primary reason why the power is used, is websites get one segment at a time rather than downloading in burst
21:09:07 [kaz]
present+ Yasser_Syed
21:09:22 [kaz]
present+ Zachary_Cava
21:09:28 [kaz]
rrsagent, draft minutes
21:09:29 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/07/25-me-minutes.html kaz
21:10:06 [cpn]
... The main power usage is when the cellular modem is on. So we looked at how HLS downloads data, it downloads quite a big chunk, then stop
21:10:06 [cpn]
... 5G is more hungry than 4G or 3G, it will download more seconds then shut down
21:10:27 [cpn]
... Apple still wanted to have control, provide hints. we didn't enable MSE iPhone, as we'd have those issues
21:10:40 [myles_]
myles_ has joined #me
21:10:40 [cpn]
... MMS provides hints, start fetching content, i have enough, stop fetching content
21:11:13 [cpn]
... We wanted to enable on 5G, do we need a carrot and stick? If website does the wrong thing, and find a way around it, shut it off completely, and don't enable MSE and fallback to HLS
21:11:36 [cpn]
... We found, If we disable the 5G modem, we found the power usage the same if using HLS vs MSE
21:11:46 [cpn]
... The website can do what it wants and it will still be alright
21:12:20 [cpn]
... MMS evolved to being simply trying to force the site to detecting when internally when to enable the 5G modem
21:12:40 [cpn]
... If the page sticks within those events and only enable 5G during those times, the power usage is acceptable and there's no regression over HLS
21:13:04 [cpn]
... MMS allows Webkit to be power efficient on iphone, allolwing easy transition from your website so MMS
21:13:10 [cpn]
s/so/to/
21:13:36 [cpn]
... We needed something different to MSE to avoid regression, so there's a new API which is MMS which can be dealt with independently
21:14:24 [cpn]
... Webkit didn't want regression from a user perspecitive, e.g., airplay not compatible with MSE. We want the site to be aware, they must disable remote playback
21:14:36 [cpn]
... So it's on purpose that you introduce the regression
21:15:10 [cpn]
... MMS will ship on iPad OS and Mac, it's behind an experimental flag on iPhone. iOS17 I think, in Safari settings there's a feature flag to enable
21:15:30 [cpn]
... That's how webkit came up with MMS. For implementation it was designed to allow easy transition from one to the other
21:15:43 [cpn]
... From discussion with HLS.js folks, we looked at how we can help with things missing in MSE
21:16:11 [cpn]
... Having more detail on what's evicted and what's added was useful. It's common to all players, they need a way to scan the buffer range, timeranges
21:16:22 [cpn]
... It's easier for the UA to perform that operation than the JS player
21:16:42 [cpn]
... There are things we wanted to add, with HLS the phone can know the user has selected low data usage, so we can limit the resolution to fetch
21:17:09 [cpn]
... Unfortunately, providing the quality attribute in the spec may not be possible from a privacy perspective, could be a fingerprinting vector
21:17:20 [cpn]
... So we have to find a way that can be done or drop it from the spec
21:17:40 [cpn]
... That the inner detail on how we came up with MMS, we've attended FOMS and TPAC meetings where we got feedback on what's missing in MSE
21:18:06 [cpn]
... We tried to add many of those features, in particular on SourceBuffer, so I'm hoping we achieve that goal, not 100% if that's the case or not
21:18:51 [kaz]
present+ Thasso_Griebel
21:19:14 [cpn]
Mike: If a player switches to MMS, how much work is dont in the player and how much in the implementation? How much cnage inthe player?
21:19:54 [kaz]
s/is dont/is done/
21:19:55 [cpn]
jya: very little change. If a MSE player wants to work on iphone, you just need to change the call to ManagedMediaSource, add the disable remote playback flag, use plain mp4 or HLS
21:20:32 [cpn]
... That doesn't mean you get the best performance, that's the minimum to get it working. Requirement for an alternate player isn't part of the spec
21:21:22 [cpn]
... The existing HTML5 spec for the video allows you to provide a src attribute, a <source> with different links, one for your MSE source and for somehting compatible with remote playback, iPhone currently plays
21:22:21 [cpn]
... If in your existing player you listen to the start/endstraming events and you only fetch data between those, you'll get access to the 5G modem, so faster start time and higher quality playback and better power
21:23:03 [cpn]
Alicia: Thanks for this work. Something I noticed is there seems to be three different problems it's trying to fix
21:23:41 [cpn]
... not related problems: one is performance problem with fetching media reducing power usage. I like the start/stop streaming events
21:24:10 [cpn]
... because right now it's a big challenge for anyone making an MSE player. They have to make that choice but they're not in the best place to make that choice, so handholding is good
21:24:44 [cpn]
... The airplay alternative is a nice thing, but I don't see it as related. Why couldn't we have alternative playback urls and non-managed MediaSource?
21:25:12 [cpn]
... Also, I saw the buffer change events, it would be good to have as events instead having ht eclient check periodically. I think it should exist in fullblown MSE too
21:25:28 [cpn]
... Haven't seen if those will be added to both?
21:25:56 [kaz]
s/ht eclient/the client/
21:25:57 [cpn]
jya: The reuqirement for an Airplay source alternative is not something I'm super happy with, it's not an engineering decision
21:26:37 [cpn]
Alicia: I get that
21:27:06 [cpn]
... Could the events work with both MMS and MSE?
21:27:23 [cpn]
... So the limitation is only on the iPhone implementation, not part of the spec?
21:27:58 [cpn]
jya: Was added in Safari last year, hasn't changed in the spec. I'd like to refer you to the GitHub issue for your last question for the changed event, Mark Watson asked the same question
21:28:07 [cpn]
... https://github.com/w3c/media-source/issues/320
21:28:45 [cpn]
cpn: Please do leave comments in the issue
21:29:41 [kaz]
rrsagent, draft minutes
21:29:42 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/07/25-me-minutes.html kaz
21:29:48 [cpn]
Joey: I was wondering how the notifications of how to fetch content, what interval are they on and how does it interact with low latency live streams?
21:30:04 [cpn]
jya: Interval is up to the UA to define, for iPhone, the low water mark is about 10 seconds, high is 30
21:30:26 [cpn]
... For live streaming, the expected behaviour is that if you're in a live stream, you won't follow the start/end streaming events, you'll fetch all the time
21:30:51 [cpn]
... But if you fetch outside those times, the 5G modem on iPhone is disabled and it's like MSE, using 4G
21:31:00 [cpn]
... Whether other UAs will do that, I don't know
21:31:16 [cpn]
Joey: Is expectation that if you're at the live edge, those will be overruled?
21:31:35 [cpn]
jya: The spec has language that appends could fail if you append when it doesn't want you to
21:32:01 [cpn]
... It's a remnant of the stick approach. We left it in the proposal, if you don't append between start/end streaming the UA is allowed to deny the append (throw)
21:32:33 [cpn]
... That's not something webkit has implemented. It's also mentioned in the GH issue, if it's to be removed, we'll have a follow up discussion on whether to remove or not
21:32:57 [cpn]
Joey: As a player developer sohuld we ignore the events when straming at live edge or if there's a buffer underflow? What#s the guidance?
21:33:19 [cpn]
jya: the events are hints, it's only going to affect how it works under the hood, so enabling of the 5G modem
21:33:49 [cpn]
... If you want the lowest latency possible and you start appending constantly, it's a strong signal that you're playing at the live edge
21:34:24 [cpn]
... we discussed internally and at previous FOMS about having a dedicated live mode, but we found with just start/end streaming and if player respects those events to determine if the content is live
21:34:57 [cpn]
Joey: Would you be willing to write guidance for player developers? Would be good to encourage players to use the API in ways implementers prefer to use it
21:35:05 [cpn]
jya: I touched on those in the WWDC video
21:35:12 [cpn]
... https://developer.apple.com/videos/play/wwdc2023/10122/
21:35:30 [cpn]
Joey: It woundn't be in the spec, just a way to help the community do the right thing
21:36:00 [cpn]
jya: In summary, if you respect the start/end events you get the 5G model. We can see abuot a blog post
21:36:44 [cpn]
Alicia: Is there any performance gain for other than mobile phones, e..g, for set top boxes?
21:37:05 [cpn]
jya: On Mac you get better power usage regardless
21:37:54 [cpn]
Alicia: For a desktop or laptop on wifi, similar for STB. Any gains in those kinds of devices?
21:38:20 [cpn]
jya: Frrom a performacne perspective, Twitch on Firefox, as FF doesn't support MPEG-TS you append one frame at a time
21:38:43 [cpn]
... If you do that, the CPU is working all the time, has to check if each frame is in existing buffered range
21:39:06 [cpn]
... If they were to batch their content, but twitch wants lowest latency possible
21:39:40 [cpn]
Alicia: May be not so much a technical problem, but pushing them to do the right thing
21:39:59 [cpn]
jya: It's a fundamental issue between VOD and live, you can't have it all
21:40:21 [cpn]
Alicia: More pracically, many of our customers work with STBs, I work on maintaining implementations of MSE in WebKit for media boxes
21:40:43 [cpn]
... Could I have good things to sell them on MMS, or nothing to offer beyond followign the standard?
21:41:07 [cpn]
jya: Probably not. One benefit is having the latest webkit version. STBs we know don't update often
21:41:40 [cpn]
Alicia: They also have their own performance constraints. In STBs power can be used in content switching done in the browser
21:42:13 [cpn]
jya: If you do things in batch and increase latency, you'll increase performance. MMS favours doing things in batch
21:42:31 [cpn]
Alicia: And batching also increases memory usage, which is an issue on STBs
21:43:04 [cpn]
cpn: Alicia, what kind of improvement to MSE would you be looking for?
21:43:28 [cpn]
Alicia: Don't know if it handles running out of memory, was that mentioned in MMS?
21:44:02 [cpn]
jya: It is, that's a good point. MMS is allowed to evict at any time, not just during appendBuffer, specifically for low memory devices
21:44:31 [cpn]
Alicia: Extremely useful, you don't know how much RAM the device has. You can have the problem where a website stores too much in memory, the STB might start swapping
21:44:54 [cpn]
jya: Forcing a lower memory footprint in the player can be added on the UA side if using MMS
21:45:46 [cpn]
Alicia: I recall we have some settings to force memory usage precisely because this is a rpoblem. video players assume you have as much RAM as on a desktop, so couldn't really work following the spec
21:45:54 [cpn]
... Good to have hints more than surprise sticks
21:46:08 [cpn]
jya: MMS allows eviction then the bufferchange event is fired
21:46:47 [cpn]
Thasso: Coming back to Joey's point, the core issue I see in the GH issue is discrepancy between when to download media and when not to, fine for 5G usage
21:47:36 [cpn]
... but gives the ability for the UA to block the append. Maybe this could be removed in a follow up PR, but could it be removed from the proposal, it's a big concern if reject the append depending on when you fetch the data?
21:48:14 [cpn]
jya: I guess it's a matter of you never know. it may be it'll never be used
21:48:33 [cpn]
Thasso: So could it be removed from the proposal now and add it later if they need that ability
21:48:38 [cpn]
jya: I don't know
21:48:55 [cpn]
Alicia: Would it conflict with the live playback case?
21:49:25 [cpn]
Thasso: It doesn't, I'm good if i fetch between the events, but if UA decides later to reject, I'll have the problem and the player gets more complex
21:49:44 [cpn]
... It also affects VOD playback where players maintain their own buffers outside of MSE, as buffer managemnet in MSE is tricky
21:49:54 [cpn]
... If the implementation isn't there, why allow it in the spec in the first place?
21:50:50 [cpn]
jya: It's an ongoing discussion in the issue, may be best to take it offline
21:51:11 [cpn]
cpn: Please raise this in the GH issue, and the Media WG will follow up on it
21:53:33 [cpn]
... the GH thread is getting long so we may want to break it into specific issues for different questions
21:54:59 [kaz]
rrsagent, draft minutes
21:55:00 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/07/25-me-minutes.html kaz
21:55:19 [cpn]
[adjourned]
21:55:29 [kaz]
rrsagent, draft minutes
21:55:31 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/07/25-me-minutes.html kaz