– DRAFT –
Second Screen CG - TPAC 2021 vF2F

Multi-Screen Window Placement

<anssik> Multi-Screen Window Placement spec

<anssik> Multi-Screen Window Placement Explainer

anssik: Spec has been shaping up since our last meeting. Explainer completed as well.
… Mike or Joshua, could you highlight key issues?

Mike: General point is that the API is sufficiently capable. Its shape is stable and extensible enough.
… Our aim is to get a stable release in Chromium sometime in the near future.
… Some fairly minor issues on the spec, including TAG feedback to rename some of the properties.
… I think we'll just go with TAG suggestions there, although we can bikeshed on that.
… Most of the issues raised are about extending the API. Still need to go through some of them but I'm confortable that the API can accommodate the extensions afterwards if needed.
… Maybe one big open question surrounding the use of Wayland in Linux.
… Our API is partly about extending window.open coordinate values.
… Wayland was willingly designed to prevent applications from knowing where windows are positioned on screen. As things stand, it breaks window primitive relative to positioning and moving.
… In most other systems, it's not possible for windows to specify their bounds.
… Windows get placed in certain arrangements.
… Wayland is definitely an interesting risk. It does seem to be gaining popularity as a replacement for X as a window management.
… I believe KDE does not have plan to deprecate X at least for now but other plans in progress.
… I would appreciate feedback.

anssik: Is that captured in an issue?

Mike: Not yet. More of an internal risk study for now. Planning to publish that soon.

anssik: I'm aware of Wayland contributors in my company.

anssik: Second question is assessing whether Multi-Screen could be promoted as a WG deliverable in the next charter.

<anssik> Extensions to the Window Interface

Mike: Breaking it down, our proposal is to extend CSSOM View. Additional argument for Fullscreen API and the main entry point for getting the array of available screens. It could be part of CSSOM View interfaces or defined as its own API.

<anssik> Open issues

Mike: A new issue for Wayland is certainly appropriate. A couple of issues around naming could benefit from feedback as well.

<msw> https://github.com/webscreens/window-placement/issues/54

<msw> https://github.com/webscreens/window-placement/issues/50

Mike: [goes through a couple of bikeshedding issues]

anssik: I'm looking at the TAG review repo. There seem to be multiple issues.

<msw> https://github.com/w3ctag/design-reviews/issues/602

<anssik> Multi-Screen Window Placement API TAG design review

anssik: How much of the TAG feedback is reflected in the spec?

Mike: The main points are really about the naming of the entry points.

anssik: From a design perspective, that seems like an OK.
… Have you discussed with security and privacy folks already at this stage?

Mike: We've gone through multiple rounds of internal security and privacy reviews.
… As part of original trial.
… No specific changes requested. It would be interesting to understand how some of the user affordances that we're considering may impact things.
… E.g. partner asking to open window in fullscreen.

anssik: Back in the days, there was a proposal on gesture delegation.

Mike: Partners have requested the ability to open multiple pop-ups from a single gesture without triggering the pop-up blocker. And potentially be able to request fullscreen for the pop-up or request multiple fullscreens for the pop-ups.
… I don't think that it makes sense to require the user to click "fullscreen" 7 times on multiple days, but this can be added later on.

<anssik> Capability Delegation API

Mike: You mentioned a delegation API. One person I'm working with is actively pursuing an API that would allow an iframe to pass a token to another to delegate permission.

mfoltzgoogle: Back to the issue around Wayland. Potential extension to the current API that would either let a site detect that the underlying window system basically won't allow window movement, or some way to tell the app that window movement would fail if they tried.
… Might be preferable as getting Wayland to support this would likely take time.

Mike: That's great feedback. I think it would be valuable. In one way, folks can call moveTo and see that it doesn't work, but it would be beneficial to expose somewhere whether window placement is allowed.
… Dealing with different window managers is interesting because they sometimes have unique behaviors.
… Never quite sure whether the window manager is done processing the latest request that you sent to it.
… I would like a Promise-based mechanism that could resolve/reject. But being able to resolve the Promise once the window manager is done is actually a challenge.

jsbell: About Wayland, approach is very privacy-forward in the sense that windows don't know where they are.
… I can imagine different use cases for the APIs.
… Multiple screens, e.g. financial interface.
… Other scenario: pop-ups with relative coordinates.
… First tied to window manager, the other would work.
… In terms of the issue tracker, we've provided a lot of feedback to partners, but we haven't categorized issues as "future add", "to be done", etc.
… We also have some internal partners that are interested and that hasn't been reflected in the issue tracker yet.

anssik: OK, making that more explicit is helpful.

jsbell: If it's going to be promoted as a WG deliverable, is explicit support from other implementers needed?

anssik: W3C Process does not mandate that.

<anssik> tidoust: that is correct, in general we like to see a general agreement on an API we standardize to ensure there's no dead end, process-wise no requirement for all major browsers to support a proposal before WG adoption

Mike: Related adoption by the WG or going through the CSS WG, aspects should be reflected in the CSSOM View spec, but it may be very well appropriate to write the rest in a separate spec

mfoltzgoogle: You don't want to end up being split between two groups to minimize dependencies.

Mike: By defining coordinate views in CSSOM View functions. If those coordinates were not defined in main screen coordinates, we would need a dedicated API for that.
… In that sense, it might be appropriate for getScreens to be exposing the placement of individual screens, and coordinates to all be defined in the same spec.
… That said, part of what we're exposing will also be used by the Fullscreen API.

anssik: I'm hearing that this is a candidate for adoption by the WG.
… We'll be looking into it. What is the Chrome's expected timeline for this spec? Would you like to see the spec move sooner rather than later in terms of standardization?

Mike: Some time in Q1 2022

anssik: OK, we probably want this to move forward then.

mfoltzgoogle: I think that's reasonable to adopt, but charter needs to be explicit about how split is expected with CSS WG.

anssik: Yes, I could see a future meeting where we sit down with the CSS WG about this spec.

jsbell: Anyone liaising with the CSS WG already?

anssik: That's a good question.

mfoltzgoogle: We did reach out around the Presentation API a few years ago. Nothing recent.

anssik: We can establish a connection.

mfoltzgoogle: Closer coordination between the Media WG and CSS WG around media capabilities.

cpn: Was about to make the same point.
… CSS has a large number of specs in development. You may find it challenging to get time in their agenda.
… Not meant to say that we had difficulty with that, just acknowledging that they have lots on their plate.

PROPOSED RESOLUTION: CG would like the WG to consider adopting the Multi-Screen Window Placement API as a deliverable

Resolution: CG would like the WG to consider adopting the Multi-Screen Window Placement API as a deliverable

anssik: We will work with the WG to try to implement this.

Window Segments Enumeration deprecation & new Viewport Segments Property

anssik: Please give us an update dlibby.

<dlibby> https://webscreens.github.io/form-factors/

dlibby: Concepts around foldables, dual screens, multi-window configurations.
… Clarify concepts in API. What viewport segments get exposed to developers.

<anssik> New Viewport Segments Property explainer

dlibby: Came to the conclusion to move segments to Javascript, in the visualViewport API.

<anssik> Form Factors Explained

dlibby: has scale, pinch-zoom, related concepts.

<dlibby> https://wicg.github.io/visual-viewport/

dlibby: Since last meeting, moved API into visualViewport, which is in WICG.

<dlibby> https://github.com/w3c/csswg-drafts/issues/6339

dlibby: has shipped in major browsers.
… Tentative plan is to adopt in the CSSOM View WG.

<dlibby> https://drafts.csswg.org/mediaqueries-5/#mf-horizontal-viewport-segments

dlibby: Next step is to get to Editors Draft, then onto TR.

<dlibby> https://drafts.csswg.org/css-env-1/#viewport-segments

dlibby: More ergonomic API surfaces.

anssik: Thank you.

<anssik> mfoltzgoogle: it is on a todo list to import the spec Visual Viewport into CSS WG soonish

anssik: Does CSSWG need to recharter to adopt new specs?

dlibby: Charter is flexible and wide-ranging

anssik: Can you speak to experimental plans?

dlibby: Running an origin trial in Edge. The Android support library is still in beta. Some nitty-gitty things like binary size.

[ CSS WG charter has this about adopting new deliverables: "This list of modules is not exclusive: The WG may also create new CSS modules, within its scope. Also, it may split or merge CSS modules. If no participant in the group believes a proposed module is out of scope, and the group has consensus to add it, the group may add a new module. If the participants who object sustain their objection after discussion, a re-charter to clarify the scope may

be needed." https://www.w3.org/2020/12/css-wg-charter.html#deliverables ]

dlibby: for Edge, dual-screen devices, it is live. On Android, a positive reception.
… circle back and finish off integration into Chromium. Last thing, crossing T's with TAG review.

anssik: Expect that as part of the visualViewport spec, the segments property would be a feature?

dlibby: The property is coming along as a "more experimental" feature.

anssik: Propose we deprecate the old Window Segments Enumeration API.

<anssik> Add deprecation message #15

anssik: PR #15 is still open

Action: dlibby to merge PR#15.

darktears: Assuming by the Android support lib, the window manager?
… The posture could be hard coded in the window manager.
… Are you connected with Samsung?

Action: darktears to send Samsung contact to dlibby

Presentation API v2 feature: Tab Self Mirroring API

mfoltzgoogle: I'll introduce this briefly. Takumi will go into details.

<takumif> Explainer: https://github.com/mfoltzgoogle/tab-self-mirroring/blob/main/explainer.md

mfoltzgoogle: To set a bit of context. Some partners or sites simply want to mirror themselves to a target device as opposed to having to create a new document and show it on the other device.
… Since Chrome has a tab mirroring feature which predates the Presentation API, we activated this through a custom URI.
… We'd like to integrate this mechanism in the Presentation API.
… We made some changes recently.

takumif: See the explainer. Currently, with Chrome, we have Google Meet and Google Slides that use the functionality. And one other product currently working on an implementation to support this.
… Talking to them, right now, the capabilities that they need: 1) allow them to choose whether to do the video playback on the controller or receiver side. 2) choose to change the latency of the stream that they're doing to lower it when needed.
… Use case is when audio is played back by the controlling side.
… There are also additional features that they would like to see in the future: choose the capabilities of the device that they choose to mirror to. Only mirror to devices with video output capabilities for instance.
… e.g. for slides
… In the explainer right now, the proposal is to change the constructor of the PresentationRequest. Currently, it takes a URL or an array of URLs. The proposal would have the constructor also take an array of objects with additional parameters such as capture latency
… We're looking for feedback on whether this makes sense, or whether this should be moved to a different location.

anssik: Could the API be standalone?
… How much overlap is there in terms of implementation infrastructure with the Presentation API?

takumif: I think that there is a large overlap, making it an argument to have it as part of the Presentation API.

anssik: We could make it an extension to the Presentation API, 2 specs with a dependency.

mfoltzgoogle: I did want to mention that we've been in discussions with partners about a use case: depending on the device, they want to do self-mirroring or launch a receiver application.
… In order for that to work without multiple buttons on the page, the best way to do that is to extend PresentationRequest.
… Otherwise, it gets more complicated to support for the same device availability and user gesture.

anssik: What is the stability of this feature? Link to WG rechartering?
… We don't need to explicitly mention the feature in the charter as far as I can tell.

anssik: What is the implementation status for this? You mentioned partners.

takumif: We have a private API that is different from what we're proposing here that is currently being used.

Mike: Thanks for sharing the explainer. Two alternatives considered: Presentation API and Remote Playback API.
… What about messaging for the streams?

takumif: We're not exposing the streams to the apps at all.
… The mirroring part will be part of implementations.

mfoltzgoogle: Most of the implementation is the new API shape. The functionality already exists otherwise. Most of the implementation work would be in defining the API and wiring it up.
… The other question was around stability. I would say that it's still fairly early. We may want to do some more prototyping.

anssik: any plan to move it to the CG space?

mfoltzgoogle: We'll probably move the explainer. I don't expect a spec to appear until we have a little bit more experience working with it.

Action: takumif to move Tab Self Mirroring API explainer to Second Screen CG repository

AOB

anssik: Thank you for joining us today. Stay tuned for WG rechartering.
… We will have calls once in a while. Let the chairs know when you feel that there's a topic that would benefit from synchronous discussions.

[some discussion on the hope to organize a real F2F soon-ish]

<anssik> proposal was for 9 OR 23 Nov 2021 21:00 UTC for a joint meeting with Media WG

cpn: We talked previously about organizing joint Media WG discussions around HDR capabilities discussions
… Is that still needed?

<anssik> Joint meeting with Second Screen WG #33

anssik: Two possibilities in November.

<anssik> [Remote Playback] Capabilities for HDR rendering and display openscreenprotocol #194

<anssik> Use the same codec names as the media APIs. openscreenprotocol #233

anssik: Two issues to discuss. One on HDR capabilities, the other on codec names.

mfoltzgoogle: I won't be available on 23 Nov. Around Thanksgiving. Better in December or January.

cpn: December 14 or January 11 would be the Media WG meetings.