IRC log of mediawg on 2023-04-11

Timestamps are in UTC.

20:54:49 [RRSAgent]
RRSAgent has joined #mediawg
20:54:53 [RRSAgent]
logging to https://www.w3.org/2023/04/11-mediawg-irc
20:54:53 [Zakim]
Zakim has joined #mediawg
20:54:56 [tidoust]
RRSAgent, make logs public
20:55:08 [tidoust]
Meeting: Media WG call
20:55:28 [tidoust]
Agenda: https://github.com/w3c/media-wg/blob/main/meetings/2023-04-11-Media_Working_Group_Teleconference-agenda.md
20:58:23 [cpn]
cpn has joined #mediawg
21:01:20 [tidoust]
present+ Chris_Needham, Francois_Daoust, Bernard_Aboba
21:03:33 [jernoble]
jernoble has joined #mediawg
21:04:00 [tidoust]
present+ Jer_Noble
21:04:13 [tidoust]
present+ Dale_Curtis
21:04:39 [tidoust]
present+ Eugene_Zemtsov
21:06:40 [tidoust]
present+ Peter_Thatcher
21:06:47 [tidoust]
repo: w3c/webcodecs
21:06:47 [ghurlbot]
OK.
21:07:23 [tidoust]
Topic: Allow decoder to ignore corrupted frames
21:07:23 [cpn]
scribe+ cpn
21:08:28 [tidoust]
Slideset: @@
21:08:36 [tidoust]
[Slide 2]
21:08:47 [tidoust]
[Slide 3]
21:08:48 [cpn]
Bernard: s/@@/https://docs.google.com/presentation/d/1KbOE1VtD1fzvjmsYl-SA18l2kpVOL-uFB_h4h7s2Pdw/edit/
21:09:13 [cpn]
Bernard: WebCodecs encoder and decoder errors are fatal. It queue's a task to close the encoder/decoder
21:09:25 [cpn]
... The issue is: Is there some way to not close?
21:09:28 [tidoust]
#656
21:09:30 [ghurlbot]
https://github.com/w3c/webcodecs/issues/656 -> Issue 656 Allow decoder to ignore corrupted frames (matanui159) agenda
21:09:41 [tidoust]
[Slide 4]
21:09:57 [cpn]
... Dale clarified that perhaps the text could clarify fatal vs non-fatal
21:10:14 [cpn]
... The safest thing to do could be to close it
21:10:28 [cpn]
Dale: All errors are fatal, not sure why I wrote just software there
21:10:50 [tidoust]
s|Bernard: s/@@/https://docs.google.com/presentation/d/1KbOE1VtD1fzvjmsYl-SA18l2kpVOL-uFB_h4h7s2Pdw/edit/||
21:11:03 [cpn]
Bernard: It doesn't interfere with resiliance, FEC, redundant error coding, etc
21:11:03 [tidoust]
s|@@|https://docs.google.com/presentation/d/1KbOE1VtD1fzvjmsYl-SA18l2kpVOL-uFB_h4h7s2Pdw/edit/
21:11:18 [cpn]
... Discussed if we could have more tests for sending errors
21:11:31 [cpn]
... Chromium reports the wrong error type
21:11:40 [cpn]
Dale: We have a bug open to fix that
21:11:52 [cpn]
Bernard: Is it a bug? Is more info needed in the error?
21:12:09 [cpn]
... Question: should all errors be fatal? Dale makes a good point why they should be
21:12:44 [cpn]
... Potential issues with security team review if we don't make it fatal
21:13:00 [cpn]
Dale: If the author wants to handle the error and resume the decoding, could be for them to decide
21:13:07 [cpn]
Bernard: But they can't if it's closed
21:13:16 [cpn]
Dale: They can create a new decoder
21:13:19 [tidoust]
s|https://docs.google.com/presentation/d/1KbOE1VtD1fzvjmsYl-SA18l2kpVOL-uFB_h4h7s2Pdw/edit/|https://lists.w3.org/Archives/Public/www-archive/2023Apr/att-0000/MEDIAWG-04-11-23.pdf
21:13:23 [tidoust]
RRSAgent, draft minutes
21:13:25 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/04/11-mediawg-minutes.html tidoust
21:13:32 [cpn]
Bernard: Yes. That would require a keyframe
21:13:56 [cpn]
... Second question: There are various reasons why you could get an error. Hardware decoders may error where a sofware decoder wouldn't
21:14:22 [cpn]
... Hardware resources could be acquired by another app. Reconfigure with prefer-hardware could then fail, then you'd have to fall back to software
21:14:40 [cpn]
... Paul asked what's the difference between reset and close, and impact on performance?
21:15:22 [cpn]
Bernard: Any opinions? I've had developers ask whether there's truly been a decoder error, or something else, such as a GPU crash
21:15:32 [cpn]
... Does only having EncodingError provide enough info?
21:16:01 [cpn]
Dale: We're limited on the information available to us. Where a software decoder is more permissive it's in a way non-compliant to the spec
21:16:33 [cpn]
... I though we decided among editors that it should be fatal
21:16:43 [cpn]
Bernard: Is there any objection in the WG to that?
21:16:47 [cpn]
(none)
21:17:08 [cpn]
Bernard: So what to do next? Reconfiguring with prefer-hardware could fail.
21:17:52 [cpn]
Dale: Developers would have to handle it either way. We could provide more docs to advise on use a more professional analysis tool
21:18:06 [cpn]
... or ffmpeg
21:18:30 [cpn]
Bernard: Is EncodingError right?
21:18:56 [cpn]
... Any other changes to the spec?
21:19:18 [cpn]
Dale: No spec change, just MDN documentation improvements. My team have been working on that
21:19:55 [cpn]
Eugene: Could add a new optional exception such as corrupted stream or corrupted chunk, to say it's something to do with the stream
21:20:07 [cpn]
... and not some kind of infrastructure issue underneath
21:20:14 [cpn]
Bernard: That would be helpful though
21:20:48 [cpn]
... Developers would appreciate that. Would it be an error message inside EncoderError?
21:21:07 [cpn]
Eugene: That error type doesn't sound right
21:22:10 [cpn]
Dale: I'm not sure why we didn't add that. Technically it's an error in the encoding...
21:22:25 [cpn]
Bernard: Not sure it's a requirement to change the type, but the extra info would be useful
21:22:47 [tidoust]
i/Bernard: So what to do next?/[Slide 5]
21:22:58 [cpn]
Eugene: If we can see the data is noncompliant, and maybe an OperationError when it's a GPU crash or out of resources, would be useful distinction
21:23:18 [cpn]
... Hardware decoders don't always give the reason for error though
21:23:35 [cpn]
Bernard: Next step would be to see if that's feasible and prepare a PR if so
21:23:40 [cpn]
Eugene: I can do that
21:24:02 [cpn]
Topic: Allow configuration of AV1 screen content coding tools
21:24:10 [tidoust]
[Slide 6]
21:24:15 [tidoust]
#646
21:24:16 [ghurlbot]
https://github.com/w3c/webcodecs/issues/646 -> Issue 646 Support for Screen Content Coding (aboba) PR exists
21:24:16 [cpn]
Bernard: We've added a PR to initialise the AV1 quantizer
21:24:26 [tidoust]
PR #662
21:24:27 [ghurlbot]
https://github.com/w3c/webcodecs/issues/662 -> Pull Request 662 Enable configuration of AV1 screen content coding tools (aboba)
21:24:29 [cpn]
... This PR adds a boolean for forceScreenContentTools. Default is false
21:24:56 [cpn]
... when true, the AV1 spec sets a seek force screen content tools, then it uses the palette and block copy tools
21:25:09 [tidoust]
[Slide 7]
21:25:15 [cpn]
... The PR adds this boolean attribute, default false, with an explanation
21:25:30 [cpn]
Eugene: Difference with the other PR?
21:25:50 [cpn]
Bernard: I rebased it due to a merge conflict
21:26:30 [cpn]
Eugene: Another one proposed adding the flag to VideoEncodingConfig. The distinction is you configure it once, but now you'd set it per-frame
21:26:39 [cpn]
Bernard: Quantizer is per-frame as well
21:26:57 [cpn]
Eugene: So need to decide where it goes: per frame or not
21:27:25 [cpn]
Bernard: Should be per frame. Content could change during screen capture, e.g., go from slides to a video presentation where you'd disable screen content tools
21:27:48 [cpn]
... I closed the other PR
21:28:05 [cpn]
Eugene: It belongs per-frame. I checked libaom, it does allow setting per-frame
21:28:25 [cpn]
Bernard: If can change, but how you know to change it is a different story...
21:29:05 [cpn]
Dale: Quantizer makes sense as an encoder, but the screen tools feels like a per-frame metadata thing
21:29:15 [cpn]
... Then under the hood we'd automatically do the right thing
21:29:48 [cpn]
Bernard: WebRTC does it that way, by checking whether it comes from a screen - but could be a game or sports event which are not amenable to screen content tools
21:30:09 [cpn]
Eugene: IIRC, we'll have the same for VP9
21:30:35 [cpn]
Bernard: I guess so, the AV1 tools are more sophisticated. Would that use the same kind of parameter?
21:30:56 [cpn]
Eugene: As far as I know there's a global setting, not per frame
21:31:24 [cpn]
Bernard: HEVC has screen content tools, but it's hardware only, so doesn't make sense to add it, as it woulnd't be used
21:31:43 [cpn]
... Looking at the WebRTC code, it mostly changed the quantizer
21:33:05 [cpn]
Chris: So proposed resolution is to add this per-frame for AV1, then consider VP9 separately. It's very much codec-specific
21:33:11 [tidoust]
Topic: Extend EncodedVideoChunk metadata for SVC
21:33:24 [tidoust]
[Slide 8]
21:33:37 [tidoust]
#619
21:33:38 [ghurlbot]
https://github.com/w3c/webcodecs/issues/619 -> Issue 619 Consistent SVC metadata between WebCodecs and Encoded Transform API (aboba) agenda, PR exists
21:33:40 [tidoust]
PR #654
21:33:41 [ghurlbot]
https://github.com/w3c/webcodecs/issues/654 -> Pull Request 654 Extend EncodedVideoChunkMetadata for Spatial SVC (aboba)
21:34:08 [cpn]
Bernard: For background: In WebCodecs we support temporal scalablity, and WebRTC supports temporal and spatial scalability
21:34:31 [cpn]
... In the WebRTC encoded transform, we provide metadata for these encoded frames
21:35:00 [cpn]
... WebCodecs has an encodeded metadata, but there's a mismatch - if you want temporal scalability you need to use the WebRTC API
21:35:16 [cpn]
... Since it's in WebRTC, why not bring it also to WebCodecs?
21:35:19 [tidoust]
[Slide 9]
21:35:51 [cpn]
... First question is compare WebCodecs and WebRTC APIs. WebCodecs API is more structured, not just one big blob of stuff as in WebRTC
21:36:36 [tidoust]
[Slide 10]
21:36:59 [cpn]
... The SVC sub-dictionary design has a frame number (unsigned short), not the same as frame id
21:37:08 [cpn]
... frameId is a globally unique id for the frame
21:37:51 [cpn]
... It's something you want to serialize on the wire, so the sender and receiver could want the information. Frame number is a modulo 2^16 of the frame id, to use less space in the wire serialization
21:38:09 [cpn]
... When you describe dependencies you are referencing a series of frame numbers
21:38:37 [cpn]
... In real life you typically don't have 2^16 or 2^32 frame-ago dependencies
21:38:55 [cpn]
... Different names compares to the WebRTC version
21:39:41 [cpn]
... Decode targets and chain links. A forwarder keeps state, and the frame rate and spatial resolution for a particular client. This determines what layers I forward to that client
21:40:14 [cpn]
... It's useful to have this from the encoder, as the forwarder has state about the target, and compare against the frame itself
21:40:39 [cpn]
... It makes it possible forwarder to quickly decide whether to forward or not
21:41:22 [cpn]
... Protect the WebCodecs decoder against things that would cause a decoder error
21:41:49 [cpn]
... Chain links. If you get a frame as a receiver, with dependencies, is it true that if I submit it ot the WebCodecs decoder I should get an error?
21:42:00 [cpn]
... The dependencies might have dependencies? So you'd still get a decoder error
21:42:17 [cpn]
... Chain links look at the whole chain of dependencies, and see if you'd get an error
21:42:47 [cpn]
... Easier for the encoder to send the data than for the receiver to calculate the dependency graph, and avoid duplicate work across each client
21:43:04 [cpn]
... We're thinking this should go in the SVC dictionary
21:43:36 [cpn]
... One thing is there's no unsigned short, just unsigned long. Is this headed in the right direction?
21:44:05 [cpn]
Eugene: As a superficial comment, we should everything as unsigned long. It wouldn't cost anything
21:44:08 [cpn]
Bernard: Agree
21:44:31 [cpn]
Eugene: I'm not sure when this would be implemented
21:45:06 [cpn]
Bernard: It's in the Chromium code base. The SVC modes and depedency descriptor is there
21:45:44 [cpn]
Dale: I think we'd need to check what our encoders produce. We should get one software encoder working before landing the spec change
21:46:19 [tidoust]
[Slide 11]
21:46:22 [tidoust]
[Slide 12]
21:46:24 [tidoust]
[Slide 13]
21:46:31 [cpn]
Bernard: I think it's possible to enable it. Next steps?
21:46:44 [cpn]
Dale: Can you share a link to the Chromium code?
21:46:50 [cpn]
Bernard: Will do
21:47:37 [cpn]
Chris: Prototyping as Dale suggested?
21:47:59 [cpn]
Bernard: It's in WebRTC spec, concern about the alternative version showing up in WebRTC
21:48:27 [cpn]
Dale: I feel like having it working end to end, either WebRTC or WebCodecs, would be good
21:48:51 [cpn]
Bernard: If the WG approves the approach, could follow it up in WebRTC
21:49:03 [cpn]
Dale: Would prefer to see it working first though
21:49:51 [cpn]
Eugene: We only have the temporal layer ID, as that's the only thing implemented, and we can add more dictionary entries when we're happy
21:50:43 [cpn]
Bernard: We could submit a PR to remove from WebRTC, then when it's implemented add it in both places
21:51:51 [cpn]
... It's dangeous for both to be out of sync
21:52:14 [cpn]
Chris: Has this shipped in WebRTC or is it only a spec change for now?
21:52:22 [cpn]
Bernard: Would have to check
21:52:35 [cpn]
Chris: Happy to organise joint meeting discussion between both groups
21:53:28 [cpn]
Chris: So check if shipped, get working end to end in either WebRTC and WebCodecs, then ensure specs are consistent across both
21:53:54 [cpn]
Topic: Media WG rechartering
21:54:04 [tidoust]
scribe+ tidoust
21:54:16 [tidoust]
repo: w3c/charter-media-wg
21:54:16 [ghurlbot]
OK.
21:54:23 [tidoust]
#38
21:54:24 [ghurlbot]
https://github.com/w3c/charter-media-wg/issues/38 -> Issue 38 Document Picture-in-Picture (steimelchrome)
21:54:39 [Github]
https://github.com/w3c/media-wg/pull/38 : Add webcodecs quantizer mode to agenda listing.
21:54:39 [tidoust]
cpn: Open question related to rechartering and the Document Picture-in-Picture.
21:55:11 [tidoust]
... The Media WG was suggested in the TAG review as venue for Recommendation track progress.
21:55:58 [tidoust]
... Suggestion at this stage would not be that it become a WG deliverable but rather a potential normative deliverable as we've done in the past with the Audio Focus API.
21:56:20 [tidoust]
... With the goal being to avoid rechartering when spec is ready to enter the Recommendation track.
21:56:36 [tidoust]
... Question here is whether we should consider the document to our scope as a potential normative spec.
21:57:08 [tidoust]
... François pointed out that this would change the scope of the WG a little, because the group is focused on media related features.
21:57:22 [tidoust]
... Document PiP is broader in scope.
21:57:43 [tidoust]
... While it has interest from media companies to using it for media content, it's not restricted to that at all.
21:58:04 [tidoust]
... Question is: is the Media WG the right venue to continue work on the spec?
21:58:20 [tidoust]
... In favor: Picture-in-Picture is developed by the Media WG
21:58:43 [tidoust]
... I have a concern about the broader scope though.
21:59:29 [tidoust]
jernoble: The original media element supported fullscreen mode. The Fullscreen API is a more general purpose API. I think that is a WHATWG deliverable and that there are lots of parallel there.
21:59:49 [tidoust]
... As an implementer, I don't know that working on the spec in the Media WG makes sense.
22:00:14 [tidoust]
cpn: I think François suggested an alternate ohme for the spec, WebApps WG.
22:00:27 [tidoust]
... I don't think there's a barrier to landing that on the Recommendation track somewhere.
22:00:46 [tidoust]
... I'll just start an email thread with Tommy and chairs to thrash this out.
22:01:22 [tidoust]
... We don't want to hold up too much on that because draft charter is mostly ready otherwise.
22:01:57 [tidoust]
Dale: Most use cases are media-related but argument and comparison with Fullscreen makes sense to me.
22:02:47 [tidoust]
jernoble: If it becomes necessary to integrate with other specs, such as CSS, etc. it also makes sense to use a group that's more used to doing that.
22:03:01 [tidoust]
cpn: I agree. My proposal would be not to do it here.
22:03:21 [tidoust]
jernoble: Even more importantly, I think it would be even more successful in another group.
22:03:33 [tidoust]
Topic: TPAC 2023 joint meetings
22:04:13 [tidoust]
cpn: I think what I'm going to propose for our group is similar to last year: full morning or full afternoon to go through discussions. The other thing I'm interested in exploring is joint meetings.
22:05:13 [tidoust]
Bernard: Ongoing poll to figure out who's going to show up in-person. If not enough people, my sense is that W3C guidelines are not to request TPAC time.
22:05:22 [tidoust]
... I'll be remote.
22:05:30 [tidoust]
cpn: I'm planning to be there in-person.
22:06:07 [cpn]
tidoust: I wouldn't worry too much about the requirement for number of people. The venue is large enough
22:07:07 [tidoust]
Bernard: Joint meeting with WebRTC will be useful. Lots of ongoing discussions.
22:07:21 [tidoust]
cpn: I'll follow-up via email on the specifics of that.
22:07:35 [tidoust]
RRSAgent, draft minutes
22:07:36 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/04/11-mediawg-minutes.html tidoust
23:25:33 [Zakim]
Zakim has left #mediawg