Meeting minutes
Open Screen Protocol
mfoltzgoogle: We may not need a lot of time on Open Screen Protocol. Not a lot of new technical proposals to go through.
… Most open issues have resolutions from Fukuoka.
Track CFRG PAKE competition outcome #242
mfoltzgoogle: We need to track a pairing code. Looking at better security for authentication. At the time we drafte the spec, the best candidate for this was PAKE2, which is being used in some Google products
… At the same time, the IETF cryptographic group have been working on recommendations for which PAKE to use.
… They wrapped up this work some time ago, and recommended something different. Two protocols: CPACE and OPAQUE.
… The first one is balanced: two sides share a secret and communicate symmetrically. The second one is for client/server use.
… My assumption is that the balanced PAKE is more appropriate for our use case.
… It seems to be relatively new, no IETF RFC for now, but it is being shaped.
… I wanted to update the group on this.
… We need to take a deeper dive on the internals of the protocol and report to the group.
… We may want to support two PAKEs.
anssik: Is it too early to take a resolution?
mfoltzgoogle: Any feedback is welcome. The next time we have an opportunity to meet, we should probably come up with a proposal.
cpn: My colleague Nigel was interested in this aspect. We'll look into it and see if we have any thoughts.
Add a back pressure signal for media remoting #241
mfoltzgoogle: During last TPAC, we spent some point discussing cases where one endpoint would use streaming, e.g. through MSE on the controlling side. We wondered about ways to signal back pressure from the receiver to the remote playback sender.
… Not sure we want to do that yet, but it makes sense to provide a signal in the protocol.
… The resolved action item was a little vague. There are a few ways we could add a back pressure signal.
… I took a stab at a pull request
mfoltzgoogle: Summary: whenever the receiver gets back to the controller, it can add one of 3 statuses: enough-data, insufficient-data, too-much-data, which the sender can use to adjust the rate of the data it sends.
… This probably deserves a little bit of discussion.
mfoltzgoogle: Please have a look at the PR and provide feedback.
… The point that was brought up in Fukuoka is that without signal brought back to the sender, the network congestion path from the network to the sender is typically different than the one between the sender and the receiver.
… There may be differences of memory, e.g. sender with GB of memory whereas an HDMI dongle may have only 128MB
… The goal with that protocol is to enable senders and receivers to negotiate the rate of data transfer.
… That may impact the JS API as well if that requires cooperation from the application
cpn: Are these states enough or do you need some buffer size indication? Would numeric measure be useful?
mfoltzgoogle: It could. I'll have to think a bit more. I know that our implementation has a pacing algorithm, and knowing the buffer size could indeed help.
… Part of the motivation for the state was the way that it could be exposed to scripts.
… That would provide some insight as to how sites could use it.
… Folks used to MSE might be able to provide feedback on whether that's useful or not.
anssik: Action on the group: review the PR and provide feedback
Action: People to review the PR and provide feedback
Codec switching when remoting
mfoltzgoogle: This topic generated some discussion in Fukuoka. I wanted to test the group's temperature on whether that is a priority.
… When we're implementing remoting, the media element can switch to another source at any time. There could be implementations that might want to transcode to another codec.
… We discussed this in some length in Fukuoka. We could add additional messages to OSP to enable codec switching.
… If we wanted to expose this to JS, we'll need to sping off the discussion to the Remote Playback API.
… If we wanted to take on the script work, I would want to wait until that discussion progresses.
mfoltzgoogle: It can roughly be framed as Media capabilities for remote playback.
anssik: Feedback from Media WG participants?
… Relationship with Media Capabilities? Dependency on this API or something different?
mfoltzgoogle: I don't think so. I don't think that the discussion in Fukuoka was sufficiently concrete to go to that level of details.
… Idea is to provide feedback so that the implementation could decide what sources can be remoted.
mounir: I don't have a lot of context here. I'm aware of this issue from TPAC last year.
eric_carlson: It sounds to me that something like #2 will be required. I don't know whether this will work with Media Capabilities. Somebody needs to do a bit of research.
anssik: I'm hearing that #2 needs research. Is the Media WG aware of that issue? Or is this just our internal thinking at this point?
mfoltzgoogle: I think it depends on whether it is a high priority.
eric_carlson: After taking a quick look at the capabilities spec, I'm confident that it would work as-is. Not certain, but seems fine. Again, somebody needs to do a little bit of research. I don't think it would take much.
mounir: The Media Capabilities API could work for something like this. It's an async API.
… I'm feeling that the Remote Playback API may need to expose that somehow.
… We don't expose any device on the Remote Playback API, so I don't know where to expose the info.
anssik: My summary is that someone who understands the issue needs to look at the Media Capabilities spec should look into this.
… Let's add "needs research" on the issue for #2.
Action: Mark to add "needs research" on issue #223 for second point
mfoltzgoogle: If there are folks interested in exploring exposing this to scripts, that would be good.
Capabilities for HDR rendering and display #194
mfoltzgoogle: We want to allow an endpoint that receives media to signal its capabilities to render and display HDR media.
… We decided for the work on HDR media capabilities in the Media WG to conclude.
… I propose to align OSP with Media Capabilities issue #118, and maybe wait on #119 if it's still being discussed.
… There's an open PR for #119 from Vi.
mounir: I thought we were done with this. I believe Vi is no longer on this either. I'll check with Chris.
mfoltzgoogle: It seems #119 is really on display capabilities as opposed to media capabilities. Adding attributes to the screen object.
mounir: OK, got it, then the discussion is still ongoing. Talked about that with the CSS WG.
… A few competing proposals. I would recommend to wait unless you're in a hurry.
mfoltzgoogle: I'll propose a PR to align OSP with #118 in the meantime.
Action: Mark to propose a PR to align OSP with Media Capabilities issue #118
Refinements to Remote Playback protocol #159
mfoltzgoogle: Not worth the group's time. A bunch of todos.
Publication as FPWD
anssik: The spec is in good shape. Any objection to publish the spec as FPWD?
mfoltzgoogle: What does that entail?
anssik: It entails that the scope is roughly known
tidoust: FPWD triggers a call for patent exclusions, so you may want to do that as soon as possible
mfoltzgoogle: OK, no objection to that.
… We may want to do an editorial pass on the spec before publication
PROPOSED RESOLUTION: publish the Open Screen Protocol as First Public Working Draft
[no objections]
Resolution: publish the Open Screen Protocol as First Public Working Draft
[coffee break]
Presentation API
Tab Self Mirroring API
Mark: Recently, Chrome launched a feature whereby you could cast the content of a Google meeting to a Chromecast device
… We implemented this by allowing the site to ask the browser to mirror its own tab
… We did this for a number of reasons. It would have been difficult for the video conferencing app to run natively on the cast device, so mirroring was a good solution for sharing on a larger display
… We've had the capability in Google Slides for a while. This was done in the cast SDK using non-public APIs
… With these use cases, it would be a good time to explore what a proper API would look like
… It's early stage, not decided on API shape, but we have a good idea on requirements
… In the last few years, Chrome has shipped tabs as a source for getDisplayMedia, so there's a precedent for sites to be able to request sharing tabs with another device or application
… The main difference between this and tab mirroring is that the site wants to customize it in a couple of ways
… Chrome can control the latency, and for meetings we wanted a low latency experience for better A/V sync
… Also, if the user is not participating in the conference, so just watching, we want the ability to just route the audio to the target display. So we don't have to worry about echo cancellation in that case.
… This covers the use cases and requirements we want to achieve.
… The user would see a cast button in the page, a list of device that can accept the mirrored content. The page doesn't change, it's just mirrored to the TV as well as being shown locally
… It builds on the Presentation API, with a special URL to indicate that the site requests to present its own tab. A couple of properties on the connection object can be used to adjust the mirroring - for audio, and latency customization
… Any questions?
Anssi: In the non-goals, you say mirroring to wired displays.
Mark: It's tricky to find an API that would support both. For video conferencing, you don't have the same constraints around latecy and A/V sync
… If there's a strong use case, it could be addressed by other work in the group. It could be extended to wired displays as well.
Anssi: People may not have had time to look at this in detail yet. If there's support in the group, are you interested in integrating into the Presentation API?
Mark: We'll seek developer feedback, then use that to decide whether to add to the Presentation API. There's another proposal from Takumi that makes it more like Remote Playback.
… This could make sense, as the Presentation API is designed for a second device with a connection to it. We need to think about the right API surface for this, taking into account the use cases.
Anssi: Any views on the API surface?
Eric: I haven't thought about this yet. There's an issue against MediaCapture screen share #149 [and a companion pull request #148], proposing a new API to create a media stream from the current tab.
… Some interesting discussion, in mailing list and in GitHub on the privacy issues
Mounir: About MediaCapture, would the four goals be met by screen capture? Is there an approach where we could compose an API, something between screen capture and Presentation API?
Mark: If some of the dynamic settings could be exposed via screen capture, and find a way to compose with Remote Playback or Presentation API to handle device permissions, could be a good way to think about this
Anssi: Please take a look at the explainer. How to give feedback?
Mark: It's in GitHub, so please raise issues, or email me directly
Anssi: Before we adopt this, we need to ensure it's in WG scope. We could open a tracking issue in the Presentation API repo?
Mark: When the time's right, we can figure out how to bring it into the group
Same screen presentation, issue #476, PR #479
dlibby: The Powerpoint team has been looking to use the Presentation API.
dlibby: For a single screen device they want to enter a slideshow view, for parity with the native app
… They want some ergonomic improvements, alongside remote presentation, with same API and events etc
… Looking at the spec, nothing specifically disallowed it. Want group feedback, could modify Presentation API to support this?
Mark: If the Presentation API supports this, and the user chooses their own screen from the list, what behaviour would you want?
dlibby: A full screen secondary window with the requested content, and the user could alt-tab back to the original page.
… Could window.open that propagates a permission token work? There's benefit to authors to have just one code path for this scenario
Mark: If the application can live in the current constraints of the Presentation API, it's compatible with the scope of the API. I wouldn't mandate browser to support but they could
… For the PR, would we need to change the availability API to support this? I'm not convinced that's the case yet
dlibby: We wouldn't want to change the availability algorithm, since single screen is always available, running in a separate process to the request itself
Mark: If instead we allowed user activation to create a full screen window, would that also address the use case?
dlibby: From a user scenario point of view, you could build the necessary abstraction over those modes. This is an exploration of whether Presentation API is right way to go for these scenarios
Takumi: Would this skip the selection dialog?
Mark: No, the way a site could create a fullscreen window requires two steps - one to open, then make full screen. The proposal is to do that with a single user gesture
dlibby: Not clear that's a change HTML would be willing to take
Mark: That could be difficult, as there's potential for abuse of fullscreen. We try to ensure users are aware when the go / have gone full screen
… Making it easier for that to happen (e.g., clickjacking) could be hard to find the balance
… I would like to hear about the window placement work, does that also address some of the use cases. Are you looking at wired and also wireless displays?
dlibby: The main use case is for wired displays. We're looking into implications of wireless
Anssi: We could look at this in the multi-screen placement discussion tomorrow
… That spec might address this use case
… We also need to pay attention to the privacy concerns. Can we learn from the fullscreen API in that regard?
Mike: I think there's merit in considering how the use case can be solved in the Presentation API, but also consider how to solve using multi-screen window placement
Requesting enhancements to the Presentation API, Issue #477
Mark: The requester would like to ask the presentation be opened in a new window that shares the same browsing context as the controlling page, as their application has an authentication step to show the presenting page.
… They want a new document, remote displays aren't a priority for their use case.
… They also want persistent, so the site remembers their choice.
… This could be done through the Presentation API, but if the focus is on creating new windows in the same browsing profile, I thought it could be in scope for window placement
Mike: It sounds like it's in scope of multi-screen window placement
Mark: Please add any feedback to the issue
Mike: Will do
Publishing a Presentation API update
Mark: It's been some time since we published a formal update to the Presentation API. Is there a need to publish? We've merged some editorial PRs on the boilerplate
Anssi: Under Process 2020 we can more easily publish updated CRs
Francois: We're not automatically switched to Process 2020. There'll be call for review sent to the AC for a number of WGs. We'll move to P2020 soon, but not yet
… Previous process doesn't stop us from publishing if we want to
… It's slightly easier in P2020
Post-meeting note from Francois: Previous comment is actually wrong. The group did switch to Process 2020. We just haven't switched to the new Patent Policy, but in our case, that does not change much and we can publish a Candidate Recommendation Draft right away under the new process.
Anssi: What are the substantial changes?
Mark: There were some changes to clarify the receiving browsing context. Also some algorithm fixes, mostly clarifications and not new features, I think.
… So there have been some normative changes, but making the spec more precise.
PROPOSED RESOLUTION: Look into publishing a new CR for the Presentation API
Resolution: Look into publishing a new CR for the Presentation API
Remote Playback API
Remote Playback State enum can be misleading when changing media source, Issue #125
Mark: This is how we allow the browser and playback device to figure out what source to play when doing remote playback
… It's more applicable in the flinging case, where you send a URL
… Discussion to find a way for the playback device to signal to the page that the source is not compatible with remote playback
… The conclusion was that no API changes were needed, but the page should list all compatible sources and leave it to the protocol to do source selection
… From a protocol point of view, it's compatible with what we have today
… The sender can send multiple URLs with metadata, e.g., extended mime type
… From the spec point of view, it requires a PR to clarify that the sender should send all the sources to the playback device, and that either device could do source selection
… I created PR #141, please review
Anssi: There's one normative change.
Eric: I'll ask Jer for feedback on this
Anssi: We'll wait for review feedback before landing this. Any other views?
Remote playback API test automation, Issue 92
Anssi: If Honry isn't able to work on this at the moment, how to move it forward?
… It would block advancing to PR, we need to demonstrate implementability, and test automation is important for that
Mark: Is this web driver API part of the spec?
Anssi: I believe the plan is it should be part of the spec. Some W3C specs put it into the spec proper, e.g., Generic Sensor API
Mounir: For WebXR we have a test API where the intent is to simulate a device. Could be interesting to look at.
DrAlex: In the new WebRTC spec we have a mock capture device that doesn't touch WebDriver at all
Mounir: The WebXR test API is exposed when running WPT
Anssi: Are there similar test APIs for other APIs, or is this the first?
Mounir: Possible things like WebUSB have similar
Anssi: The Generic Sensor API defines mock interfaces
Mounir: Here's the info on Web USB.
Louay: I shared yesterday a presentation at the MEIG about extending WPT test runner for embedded devices. It's in the Web Media API CG
… The extension is now available in the official WPT project. It may help in this case, the aim is to run on embedded devices, we run on Chromecast for example
… I can share the documentation and links to review
… There's a new command for starting the test runner to target embedded devices where there's no WebDriver support
Anssi: Thank you. Do you still have a student project ongoing, could this be an option for us?
Louay: I can check if that's possible, for the next semester. Could consider in collaboration with the Web Media API group
Anssi: Please add some info to issue #92
… We need volunteers to help with test automation
Publish Remote Playback API as Proposed Recommendation
Anssi: There's a list of things to do to get to PR in issue #130
… One is test automation, then issue #125 - we could land that PR soon
… We decided to publish PR, but we need to clear the blocking issues
Mark: Is the blocker to define the test API, or is it having two implementations?
Anssi: We don't need the test suite if we can demonstrate two independent implementations. It could be manually tested
… We just need the evidence, but it doesn't need to be automated
Mark: Is it the automation, or having the API that enables automation, that's important?
Francois: There's nothing that requires the group to automate the tests. We can make do with a manual test suite, and don't add test API surface to the spec
<Louay_> Device Platform Tests / Web Platform Tests Slides
Anssi: Do we have volunteers to work on test implementation? If not, move forward with manual testing
PROPOSED RESOLUTION: Figure out if we want to move forward with test automation or manual testing
PROPOSED RESOLUTION: To unblock PR publication, figure out if we want to move forward with test automation or manual testing
Francois: This looks more like an action than a resolution.
Action: Louay to figure out if we want to move forward with test automation or manual testing, to unblock PR publication
Anssi: Please update issue #92, when you know more
AOB for today?
(none)
Anssi: Thank you everyone for joining. We're back at the same time tomorrow