IRC log of mediawg on 2023-04-11
Timestamps are in UTC.
- 20:54:49 [RRSAgent]
- RRSAgent has joined #mediawg
- 20:54:53 [RRSAgent]
- logging to https://www.w3.org/2023/04/11-mediawg-irc
- 20:54:53 [Zakim]
- Zakim has joined #mediawg
- 20:54:56 [tidoust]
- RRSAgent, make logs public
- 20:55:08 [tidoust]
- Meeting: Media WG call
- 20:55:28 [tidoust]
- Agenda: https://github.com/w3c/media-wg/blob/main/meetings/2023-04-11-Media_Working_Group_Teleconference-agenda.md
- 20:58:23 [cpn]
- cpn has joined #mediawg
- 21:01:20 [tidoust]
- present+ Chris_Needham, Francois_Daoust, Bernard_Aboba
- 21:03:33 [jernoble]
- jernoble has joined #mediawg
- 21:04:00 [tidoust]
- present+ Jer_Noble
- 21:04:13 [tidoust]
- present+ Dale_Curtis
- 21:04:39 [tidoust]
- present+ Eugene_Zemtsov
- 21:06:40 [tidoust]
- present+ Peter_Thatcher
- 21:06:47 [tidoust]
- repo: w3c/webcodecs
- 21:06:47 [ghurlbot]
- OK.
- 21:07:23 [tidoust]
- Topic: Allow decoder to ignore corrupted frames
- 21:07:23 [cpn]
- scribe+ cpn
- 21:08:28 [tidoust]
- Slideset: @@
- 21:08:36 [tidoust]
- [Slide 2]
- 21:08:47 [tidoust]
- [Slide 3]
- 21:08:48 [cpn]
- Bernard: s/@@/https://docs.google.com/presentation/d/1KbOE1VtD1fzvjmsYl-SA18l2kpVOL-uFB_h4h7s2Pdw/edit/
- 21:09:13 [cpn]
- Bernard: WebCodecs encoder and decoder errors are fatal. It queue's a task to close the encoder/decoder
- 21:09:25 [cpn]
- ... The issue is: Is there some way to not close?
- 21:09:28 [tidoust]
- #656
- 21:09:30 [ghurlbot]
- https://github.com/w3c/webcodecs/issues/656 -> Issue 656 Allow decoder to ignore corrupted frames (matanui159) agenda
- 21:09:41 [tidoust]
- [Slide 4]
- 21:09:57 [cpn]
- ... Dale clarified that perhaps the text could clarify fatal vs non-fatal
- 21:10:14 [cpn]
- ... The safest thing to do could be to close it
- 21:10:28 [cpn]
- Dale: All errors are fatal, not sure why I wrote just software there
- 21:10:50 [tidoust]
- s|Bernard: s/@@/https://docs.google.com/presentation/d/1KbOE1VtD1fzvjmsYl-SA18l2kpVOL-uFB_h4h7s2Pdw/edit/||
- 21:11:03 [cpn]
- Bernard: It doesn't interfere with resiliance, FEC, redundant error coding, etc
- 21:11:03 [tidoust]
- s|@@|https://docs.google.com/presentation/d/1KbOE1VtD1fzvjmsYl-SA18l2kpVOL-uFB_h4h7s2Pdw/edit/
- 21:11:18 [cpn]
- ... Discussed if we could have more tests for sending errors
- 21:11:31 [cpn]
- ... Chromium reports the wrong error type
- 21:11:40 [cpn]
- Dale: We have a bug open to fix that
- 21:11:52 [cpn]
- Bernard: Is it a bug? Is more info needed in the error?
- 21:12:09 [cpn]
- ... Question: should all errors be fatal? Dale makes a good point why they should be
- 21:12:44 [cpn]
- ... Potential issues with security team review if we don't make it fatal
- 21:13:00 [cpn]
- Dale: If the author wants to handle the error and resume the decoding, could be for them to decide
- 21:13:07 [cpn]
- Bernard: But they can't if it's closed
- 21:13:16 [cpn]
- Dale: They can create a new decoder
- 21:13:19 [tidoust]
- s|https://docs.google.com/presentation/d/1KbOE1VtD1fzvjmsYl-SA18l2kpVOL-uFB_h4h7s2Pdw/edit/|https://lists.w3.org/Archives/Public/www-archive/2023Apr/att-0000/MEDIAWG-04-11-23.pdf
- 21:13:23 [tidoust]
- RRSAgent, draft minutes
- 21:13:25 [RRSAgent]
- I have made the request to generate https://www.w3.org/2023/04/11-mediawg-minutes.html tidoust
- 21:13:32 [cpn]
- Bernard: Yes. That would require a keyframe
- 21:13:56 [cpn]
- ... Second question: There are various reasons why you could get an error. Hardware decoders may error where a sofware decoder wouldn't
- 21:14:22 [cpn]
- ... Hardware resources could be acquired by another app. Reconfigure with prefer-hardware could then fail, then you'd have to fall back to software
- 21:14:40 [cpn]
- ... Paul asked what's the difference between reset and close, and impact on performance?
- 21:15:22 [cpn]
- Bernard: Any opinions? I've had developers ask whether there's truly been a decoder error, or something else, such as a GPU crash
- 21:15:32 [cpn]
- ... Does only having EncodingError provide enough info?
- 21:16:01 [cpn]
- Dale: We're limited on the information available to us. Where a software decoder is more permissive it's in a way non-compliant to the spec
- 21:16:33 [cpn]
- ... I though we decided among editors that it should be fatal
- 21:16:43 [cpn]
- Bernard: Is there any objection in the WG to that?
- 21:16:47 [cpn]
- (none)
- 21:17:08 [cpn]
- Bernard: So what to do next? Reconfiguring with prefer-hardware could fail.
- 21:17:52 [cpn]
- Dale: Developers would have to handle it either way. We could provide more docs to advise on use a more professional analysis tool
- 21:18:06 [cpn]
- ... or ffmpeg
- 21:18:30 [cpn]
- Bernard: Is EncodingError right?
- 21:18:56 [cpn]
- ... Any other changes to the spec?
- 21:19:18 [cpn]
- Dale: No spec change, just MDN documentation improvements. My team have been working on that
- 21:19:55 [cpn]
- Eugene: Could add a new optional exception such as corrupted stream or corrupted chunk, to say it's something to do with the stream
- 21:20:07 [cpn]
- ... and not some kind of infrastructure issue underneath
- 21:20:14 [cpn]
- Bernard: That would be helpful though
- 21:20:48 [cpn]
- ... Developers would appreciate that. Would it be an error message inside EncoderError?
- 21:21:07 [cpn]
- Eugene: That error type doesn't sound right
- 21:22:10 [cpn]
- Dale: I'm not sure why we didn't add that. Technically it's an error in the encoding...
- 21:22:25 [cpn]
- Bernard: Not sure it's a requirement to change the type, but the extra info would be useful
- 21:22:47 [tidoust]
- i/Bernard: So what to do next?/[Slide 5]
- 21:22:58 [cpn]
- Eugene: If we can see the data is noncompliant, and maybe an OperationError when it's a GPU crash or out of resources, would be useful distinction
- 21:23:18 [cpn]
- ... Hardware decoders don't always give the reason for error though
- 21:23:35 [cpn]
- Bernard: Next step would be to see if that's feasible and prepare a PR if so
- 21:23:40 [cpn]
- Eugene: I can do that
- 21:24:02 [cpn]
- Topic: Allow configuration of AV1 screen content coding tools
- 21:24:10 [tidoust]
- [Slide 6]
- 21:24:15 [tidoust]
- #646
- 21:24:16 [ghurlbot]
- https://github.com/w3c/webcodecs/issues/646 -> Issue 646 Support for Screen Content Coding (aboba) PR exists
- 21:24:16 [cpn]
- Bernard: We've added a PR to initialise the AV1 quantizer
- 21:24:26 [tidoust]
- PR #662
- 21:24:27 [ghurlbot]
- https://github.com/w3c/webcodecs/issues/662 -> Pull Request 662 Enable configuration of AV1 screen content coding tools (aboba)
- 21:24:29 [cpn]
- ... This PR adds a boolean for forceScreenContentTools. Default is false
- 21:24:56 [cpn]
- ... when true, the AV1 spec sets a seek force screen content tools, then it uses the palette and block copy tools
- 21:25:09 [tidoust]
- [Slide 7]
- 21:25:15 [cpn]
- ... The PR adds this boolean attribute, default false, with an explanation
- 21:25:30 [cpn]
- Eugene: Difference with the other PR?
- 21:25:50 [cpn]
- Bernard: I rebased it due to a merge conflict
- 21:26:30 [cpn]
- Eugene: Another one proposed adding the flag to VideoEncodingConfig. The distinction is you configure it once, but now you'd set it per-frame
- 21:26:39 [cpn]
- Bernard: Quantizer is per-frame as well
- 21:26:57 [cpn]
- Eugene: So need to decide where it goes: per frame or not
- 21:27:25 [cpn]
- Bernard: Should be per frame. Content could change during screen capture, e.g., go from slides to a video presentation where you'd disable screen content tools
- 21:27:48 [cpn]
- ... I closed the other PR
- 21:28:05 [cpn]
- Eugene: It belongs per-frame. I checked libaom, it does allow setting per-frame
- 21:28:25 [cpn]
- Bernard: If can change, but how you know to change it is a different story...
- 21:29:05 [cpn]
- Dale: Quantizer makes sense as an encoder, but the screen tools feels like a per-frame metadata thing
- 21:29:15 [cpn]
- ... Then under the hood we'd automatically do the right thing
- 21:29:48 [cpn]
- Bernard: WebRTC does it that way, by checking whether it comes from a screen - but could be a game or sports event which are not amenable to screen content tools
- 21:30:09 [cpn]
- Eugene: IIRC, we'll have the same for VP9
- 21:30:35 [cpn]
- Bernard: I guess so, the AV1 tools are more sophisticated. Would that use the same kind of parameter?
- 21:30:56 [cpn]
- Eugene: As far as I know there's a global setting, not per frame
- 21:31:24 [cpn]
- Bernard: HEVC has screen content tools, but it's hardware only, so doesn't make sense to add it, as it woulnd't be used
- 21:31:43 [cpn]
- ... Looking at the WebRTC code, it mostly changed the quantizer
- 21:33:05 [cpn]
- Chris: So proposed resolution is to add this per-frame for AV1, then consider VP9 separately. It's very much codec-specific
- 21:33:11 [tidoust]
- Topic: Extend EncodedVideoChunk metadata for SVC
- 21:33:24 [tidoust]
- [Slide 8]
- 21:33:37 [tidoust]
- #619
- 21:33:38 [ghurlbot]
- https://github.com/w3c/webcodecs/issues/619 -> Issue 619 Consistent SVC metadata between WebCodecs and Encoded Transform API (aboba) agenda, PR exists
- 21:33:40 [tidoust]
- PR #654
- 21:33:41 [ghurlbot]
- https://github.com/w3c/webcodecs/issues/654 -> Pull Request 654 Extend EncodedVideoChunkMetadata for Spatial SVC (aboba)
- 21:34:08 [cpn]
- Bernard: For background: In WebCodecs we support temporal scalablity, and WebRTC supports temporal and spatial scalability
- 21:34:31 [cpn]
- ... In the WebRTC encoded transform, we provide metadata for these encoded frames
- 21:35:00 [cpn]
- ... WebCodecs has an encodeded metadata, but there's a mismatch - if you want temporal scalability you need to use the WebRTC API
- 21:35:16 [cpn]
- ... Since it's in WebRTC, why not bring it also to WebCodecs?
- 21:35:19 [tidoust]
- [Slide 9]
- 21:35:51 [cpn]
- ... First question is compare WebCodecs and WebRTC APIs. WebCodecs API is more structured, not just one big blob of stuff as in WebRTC
- 21:36:36 [tidoust]
- [Slide 10]
- 21:36:59 [cpn]
- ... The SVC sub-dictionary design has a frame number (unsigned short), not the same as frame id
- 21:37:08 [cpn]
- ... frameId is a globally unique id for the frame
- 21:37:51 [cpn]
- ... It's something you want to serialize on the wire, so the sender and receiver could want the information. Frame number is a modulo 2^16 of the frame id, to use less space in the wire serialization
- 21:38:09 [cpn]
- ... When you describe dependencies you are referencing a series of frame numbers
- 21:38:37 [cpn]
- ... In real life you typically don't have 2^16 or 2^32 frame-ago dependencies
- 21:38:55 [cpn]
- ... Different names compares to the WebRTC version
- 21:39:41 [cpn]
- ... Decode targets and chain links. A forwarder keeps state, and the frame rate and spatial resolution for a particular client. This determines what layers I forward to that client
- 21:40:14 [cpn]
- ... It's useful to have this from the encoder, as the forwarder has state about the target, and compare against the frame itself
- 21:40:39 [cpn]
- ... It makes it possible forwarder to quickly decide whether to forward or not
- 21:41:22 [cpn]
- ... Protect the WebCodecs decoder against things that would cause a decoder error
- 21:41:49 [cpn]
- ... Chain links. If you get a frame as a receiver, with dependencies, is it true that if I submit it ot the WebCodecs decoder I should get an error?
- 21:42:00 [cpn]
- ... The dependencies might have dependencies? So you'd still get a decoder error
- 21:42:17 [cpn]
- ... Chain links look at the whole chain of dependencies, and see if you'd get an error
- 21:42:47 [cpn]
- ... Easier for the encoder to send the data than for the receiver to calculate the dependency graph, and avoid duplicate work across each client
- 21:43:04 [cpn]
- ... We're thinking this should go in the SVC dictionary
- 21:43:36 [cpn]
- ... One thing is there's no unsigned short, just unsigned long. Is this headed in the right direction?
- 21:44:05 [cpn]
- Eugene: As a superficial comment, we should everything as unsigned long. It wouldn't cost anything
- 21:44:08 [cpn]
- Bernard: Agree
- 21:44:31 [cpn]
- Eugene: I'm not sure when this would be implemented
- 21:45:06 [cpn]
- Bernard: It's in the Chromium code base. The SVC modes and depedency descriptor is there
- 21:45:44 [cpn]
- Dale: I think we'd need to check what our encoders produce. We should get one software encoder working before landing the spec change
- 21:46:19 [tidoust]
- [Slide 11]
- 21:46:22 [tidoust]
- [Slide 12]
- 21:46:24 [tidoust]
- [Slide 13]
- 21:46:31 [cpn]
- Bernard: I think it's possible to enable it. Next steps?
- 21:46:44 [cpn]
- Dale: Can you share a link to the Chromium code?
- 21:46:50 [cpn]
- Bernard: Will do
- 21:47:37 [cpn]
- Chris: Prototyping as Dale suggested?
- 21:47:59 [cpn]
- Bernard: It's in WebRTC spec, concern about the alternative version showing up in WebRTC
- 21:48:27 [cpn]
- Dale: I feel like having it working end to end, either WebRTC or WebCodecs, would be good
- 21:48:51 [cpn]
- Bernard: If the WG approves the approach, could follow it up in WebRTC
- 21:49:03 [cpn]
- Dale: Would prefer to see it working first though
- 21:49:51 [cpn]
- Eugene: We only have the temporal layer ID, as that's the only thing implemented, and we can add more dictionary entries when we're happy
- 21:50:43 [cpn]
- Bernard: We could submit a PR to remove from WebRTC, then when it's implemented add it in both places
- 21:51:51 [cpn]
- ... It's dangeous for both to be out of sync
- 21:52:14 [cpn]
- Chris: Has this shipped in WebRTC or is it only a spec change for now?
- 21:52:22 [cpn]
- Bernard: Would have to check
- 21:52:35 [cpn]
- Chris: Happy to organise joint meeting discussion between both groups
- 21:53:28 [cpn]
- Chris: So check if shipped, get working end to end in either WebRTC and WebCodecs, then ensure specs are consistent across both
- 21:53:54 [cpn]
- Topic: Media WG rechartering
- 21:54:04 [tidoust]
- scribe+ tidoust
- 21:54:16 [tidoust]
- repo: w3c/charter-media-wg
- 21:54:16 [ghurlbot]
- OK.
- 21:54:23 [tidoust]
- #38
- 21:54:24 [ghurlbot]
- https://github.com/w3c/charter-media-wg/issues/38 -> Issue 38 Document Picture-in-Picture (steimelchrome)
- 21:54:39 [Github]
- https://github.com/w3c/media-wg/pull/38 : Add webcodecs quantizer mode to agenda listing.
- 21:54:39 [tidoust]
- cpn: Open question related to rechartering and the Document Picture-in-Picture.
- 21:55:11 [tidoust]
- ... The Media WG was suggested in the TAG review as venue for Recommendation track progress.
- 21:55:58 [tidoust]
- ... Suggestion at this stage would not be that it become a WG deliverable but rather a potential normative deliverable as we've done in the past with the Audio Focus API.
- 21:56:20 [tidoust]
- ... With the goal being to avoid rechartering when spec is ready to enter the Recommendation track.
- 21:56:36 [tidoust]
- ... Question here is whether we should consider the document to our scope as a potential normative spec.
- 21:57:08 [tidoust]
- ... François pointed out that this would change the scope of the WG a little, because the group is focused on media related features.
- 21:57:22 [tidoust]
- ... Document PiP is broader in scope.
- 21:57:43 [tidoust]
- ... While it has interest from media companies to using it for media content, it's not restricted to that at all.
- 21:58:04 [tidoust]
- ... Question is: is the Media WG the right venue to continue work on the spec?
- 21:58:20 [tidoust]
- ... In favor: Picture-in-Picture is developed by the Media WG
- 21:58:43 [tidoust]
- ... I have a concern about the broader scope though.
- 21:59:29 [tidoust]
- jernoble: The original media element supported fullscreen mode. The Fullscreen API is a more general purpose API. I think that is a WHATWG deliverable and that there are lots of parallel there.
- 21:59:49 [tidoust]
- ... As an implementer, I don't know that working on the spec in the Media WG makes sense.
- 22:00:14 [tidoust]
- cpn: I think François suggested an alternate ohme for the spec, WebApps WG.
- 22:00:27 [tidoust]
- ... I don't think there's a barrier to landing that on the Recommendation track somewhere.
- 22:00:46 [tidoust]
- ... I'll just start an email thread with Tommy and chairs to thrash this out.
- 22:01:22 [tidoust]
- ... We don't want to hold up too much on that because draft charter is mostly ready otherwise.
- 22:01:57 [tidoust]
- Dale: Most use cases are media-related but argument and comparison with Fullscreen makes sense to me.
- 22:02:47 [tidoust]
- jernoble: If it becomes necessary to integrate with other specs, such as CSS, etc. it also makes sense to use a group that's more used to doing that.
- 22:03:01 [tidoust]
- cpn: I agree. My proposal would be not to do it here.
- 22:03:21 [tidoust]
- jernoble: Even more importantly, I think it would be even more successful in another group.
- 22:03:33 [tidoust]
- Topic: TPAC 2023 joint meetings
- 22:04:13 [tidoust]
- cpn: I think what I'm going to propose for our group is similar to last year: full morning or full afternoon to go through discussions. The other thing I'm interested in exploring is joint meetings.
- 22:05:13 [tidoust]
- Bernard: Ongoing poll to figure out who's going to show up in-person. If not enough people, my sense is that W3C guidelines are not to request TPAC time.
- 22:05:22 [tidoust]
- ... I'll be remote.
- 22:05:30 [tidoust]
- cpn: I'm planning to be there in-person.
- 22:06:07 [cpn]
- tidoust: I wouldn't worry too much about the requirement for number of people. The venue is large enough
- 22:07:07 [tidoust]
- Bernard: Joint meeting with WebRTC will be useful. Lots of ongoing discussions.
- 22:07:21 [tidoust]
- cpn: I'll follow-up via email on the specifics of that.
- 22:07:35 [tidoust]
- RRSAgent, draft minutes
- 22:07:36 [RRSAgent]
- I have made the request to generate https://www.w3.org/2023/04/11-mediawg-minutes.html tidoust
- 23:25:33 [Zakim]
- Zakim has left #mediawg