IRC log of webscreens on 2019-09-16

Timestamps are in UTC.

23:20:18 [RRSAgent]
RRSAgent has joined #webscreens
23:20:18 [RRSAgent]
logging to https://www.w3.org/2019/09/16-webscreens-irc
23:20:49 [mfoltzgoogle]
rrsagent, bye
23:31:12 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
23:33:20 [tidoust]
tidoust has joined #webscreens
23:33:45 [kzms2]
kzms2 has joined #webscreens
23:34:40 [tidoust]
RRSAgent, pointer?
23:34:40 [RRSAgent]
See https://www.w3.org/2019/09/16-webscreens-irc#T23-34-40
23:35:47 [tidoust]
RRSAgent, draft minutes v2
23:35:47 [RRSAgent]
I have made the request to generate https://www.w3.org/2019/09/16-webscreens-minutes.html tidoust
23:36:08 [tidoust]
RRSAgent, make logs public
23:36:37 [tidoust]
Meeting: Second Screen WG F2F - Day 2/2
23:36:40 [tidoust]
Chair: Anssi
23:36:48 [tidoust]
Agenda: https://www.w3.org/wiki/Second_Screen/Meetings/September_2019_F2F#Agenda
23:38:23 [tidoust]
Present+ Chris_Needham, Daniel_Libby, Anssi_Kostiainen, Peter_Thatcher, Takumi_Fujimoto, Mark_Foltz, Francois_Daoust, Mike_Wasserman
23:38:38 [cpn]
cpn has joined #webscreens
23:38:39 [ericc]
ericc has joined #webscreens
23:39:47 [MasayaIkeo]
MasayaIkeo has joined #webscreens
23:40:35 [anssik]
anssik has joined #webscreens
23:40:50 [dlibby]
dlibby has joined #webscreens
23:40:55 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
23:41:52 [takumif]
takumif has joined #webscreens
23:42:02 [tidoust]
scribe: tidoust
23:42:35 [tidoust]
RRSagent, this meeting spans midnight
23:44:15 [takio]
takio has joined #webscreens
23:44:39 [tidoust]
Present+ Yuki_Yoshida, Takio_Yamaoka, Eric_Carlson
23:45:27 [tidoust]
Anssi: We covered all day 1 topics yesterday. Recent additions to day 2 are around accessibility and display enumerations and positioning.
23:46:02 [tidoust]
... Starting the day with new API features for Remote Playback API.
23:46:16 [tidoust]
... A bit of planning at the end as we need to recharter by the end of this year.
23:46:48 [tidoust]
... We should wrapup at noon, as I have a hard stop.
23:47:20 [tidoust]
Topic: New Remote Playback API proposals
23:49:25 [tidoust]
Topic: Remote buffer state for Remote Playback + MSE
23:49:35 [msw_]
msw_ has joined #webscreens
23:49:57 [tidoust]
takumif: There are some conditions under which the media playback on the receiver side may not be smooth, for instance if the buffer on the receiver is too small and the controller keeps pushing.
23:50:14 [tidoust]
... Or if the bandwidth is smaller than the media the controller is pushing onto the buffer.
23:50:27 [MasayaIkeo]
MasayaIkeo has joined #webscreens
23:50:45 [tidoust]
... Two ways to solve these: new API, or use exiting API, also solve at the protocol level.
23:50:52 [MasayaIkeo]
MasayaIkeo has joined #webscreens
23:52:02 [tidoust]
... If the controller knows that the receiver buffer is small, it can limit the transmission to the receiver. For the bandwidth issue, we can use the MediaElement.buffered and readyState attributes. That is, we can synchronize the buffered state and readyState state on the two devices, so that the controller knows that this happens.
23:52:23 [tidoust]
... Alternatively, I thought about adding a new state attribute to the remote playback object.
23:53:01 [tidoust]
... "remotingBufferState" attribute that tells whether it has enough data or too much data.
23:53:25 [tidoust]
Present+ Mounir_Lamouri, Masaya_Ikeo
23:54:15 [tidoust]
takumif: To know whether the buffer is full, we need some info at the protocol level.
23:55:26 [tidoust]
Peter: If part of the solution of the first problem is to cause the sender not to send as much data, then from a JavaScript perspective, it looks like the two problems are the same.
23:55:37 [yy]
yy has joined #webscreens
23:56:06 [tidoust]
anssik: Wondering how AirPlay and other implementations handle this situation.
23:56:42 [tidoust]
ericc: It's different because it doesn't load the data on the sender, unless you're doing screen scraping, where I don't know how that works.
23:57:05 [tidoust]
mfoltzgoogle: We don't have these features implemented in our current implementation.
23:59:03 [tidoust]
... we don't hit the problem of buffer running out of space in the mirroring case in practice.
23:59:25 [tidoust]
... Here, we need a backpressure signal.
23:59:42 [tidoust]
Mounir: Why not exposing the buffer size?
00:00:02 [tidoust]
Present+ Jer_Noble
00:00:23 [tidoust]
Peter: I'm not sure we're going to expose that at the API level.
00:00:58 [tidoust]
... A new API is a possibility though.
00:01:16 [tidoust]
... I agree we should add something to the protocol.
00:01:50 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
00:02:31 [tidoust]
ericc: The application may not even know that it is remoting.
00:02:52 [tidoust]
mfoltzgoogle: If we can model that in a way that complies with the MSE API, that would be good.
00:03:22 [tidoust]
Peter: I assume that when applications call appendBuffer, things go to the buffer directly.
00:04:06 [tidoust]
... How can we expect people to be sophisticated enough to use the existing API?
00:04:27 [tidoust]
... "Do you know that sometimes you call appendBuffer it won't go to the buffer?"
00:04:57 [tidoust]
... Remoting may happen today without the application knowing it?
00:05:05 [tidoust]
ericc: Yes.
00:05:25 [tidoust]
... With Airplay, you know when remote playback is active, whether you requested it or not.
00:05:33 [tidoust]
Peter: You can check remote.state or whatever.
00:05:38 [tidoust]
ericc: Yes.
00:06:08 [tidoust]
mfoltzgoogle: When transition happens from local to remote playback, will there be a change in buffered and so on?
00:06:48 [HirokiENDO]
HirokiENDO has joined #webscreens
00:06:58 [tidoust]
ericc: It would be better if applications could be aware that things could be made much more efficient.
00:07:21 [tidoust]
... Have a remote state and fire an event when the state changes.
00:07:56 [tidoust]
[looking at the remotingBufferState proposal]
00:08:13 [tidoust]
Mounir: If buffered range is exposed, why would you need that on top of it?
00:09:11 [tidoust]
takumif: the sender doesn't know about decode time, how long it takes to push things over.
00:09:31 [tidoust]
anssik: What is the common case for most developers?
00:09:43 [tidoust]
Peter: To make it easy, we should add something like this.
00:10:09 [tidoust]
Mounir: I don't see the need for "insufficient-data" and "enough-data".
00:10:25 [tidoust]
Peter: It can be about network bandwidth.
00:10:43 [tidoust]
mfoltzgoogle: Two variables: buffer size and network conditions.
00:11:15 [tidoust]
... "insufficient-data" would be one way to specify that network bandwidth is not enough.
00:11:52 [tidoust]
Mounir: Make sure, as much as we can, that we want things to work automatically without requiring changes in existing applications.
00:12:20 [tidoust]
takumif: maybe we just need "too-much-data".
00:13:07 [tidoust]
... too low can be handled by the buffered time range.
00:13:34 [tidoust]
Mounir: Could that just be an event?
00:14:21 [tidoust]
mfoltzgoogle: If we do the protocol change, do we need to expose that to the API?
00:14:41 [tidoust]
Peter: I think that's the question. And if we don't make it easy, will it be used?
00:16:12 [tidoust]
... Roughly, it seems we want to add something but smaller than that.
00:17:04 [tidoust]
PROPOSED RESOLUTION: Add a backpressure signal to the OSP
00:17:23 [tidoust]
RESOLUTION: Add a backpressure signal to the OSP
00:18:27 [tidoust]
mfoltzgoogle: I think we want more feedback on the second part.
00:18:49 [tidoust]
... What's the best way to expose the info to developers? How to make the mechanism consistent with existing APIs?
00:19:08 [cpn_]
cpn_ has joined #webscreens
00:19:33 [tidoust]
takumif: The Remote Playback API says that which of the media element attributes is sync-ed is up to the application. This would require that buffered time range is sync-ed.
00:20:30 [tidoust]
Peter: [details typical computation on buffered time range]
00:20:56 [tidoust]
Mounir: Isn't it the case that if you care that much, you would use the Presentation API?
00:21:22 [tidoust]
Peter: Stepping back a bit, this whole conversation started trying to find a way to do MSE + Remote Playback API.
00:21:48 [stepsteg]
stepsteg has joined #webscreens
00:23:04 [tidoust]
Topic: One prompt for Presentation and Remote Playback APIs
00:24:05 [tidoust]
takumif: Each of those APIs have their own way to prompt. Prompt may list different devices. It would be nice to prompt the whole list and do either remote playback API or presentation API depending on selected display.
00:24:15 [tidoust]
... [showing example code]
00:25:22 [tidoust]
anssik: What is the concrete use case behind this proposal?
00:26:13 [tidoust]
takumif: Some receivers may only support Presentation API or Remote Playback API. If a site wants to support both of those, right now the site would need to show two buttons.
00:26:37 [tidoust]
Jer: And no way for the user to know which button to click.
00:26:55 [tidoust]
mfoltzgoogle: The "prompt" is permission to use the device once?
00:27:14 [tidoust]
takumif: Yes, we would want this permission to expire almost immediately.
00:27:24 [tidoust]
anssik: Is this a new pattern on the Web platform?
00:27:55 [tidoust]
mfoltzgoogle: There is user gesture, and user activation. This seems a little bit different.
00:28:09 [tidoust]
ericc: I'm not sure. It's data that is valid for the duration of the Promise callback.
00:28:23 [tidoust]
Jer: Why do we want this to die this immediately?
00:28:35 [tidoust]
anssik: So that it happens in context, not one hour after the user selected a display.
00:29:26 [tidoust]
Jer: Problem is that if people use weird JS frameworks that post messages all over the place, you'll end up with things broken.
00:29:55 [tidoust]
ericc: But if that's the behavior from the ground up, then that's fine. It breaks when you change behavior.
00:30:27 [tidoust]
... We can restrict it at start, and if we find out that there are too many problems, we could relax things later on.
00:31:24 [tidoust]
Jer: I'd like to see prompt take an array. In the future, we may have more things that can be remoted.
00:31:31 [tidoust]
mfoltzgoogle: A dictionary might be better.
00:31:45 [tidoust]
anssik: I'm hearing support for the proposal with a dictionary parameter.
00:32:19 [tidoust]
... I'm slightly surprised that no one used the same design.
00:32:34 [tidoust]
mfoltzgoogle: A use case that comes to mind is media capture.
00:33:59 [HirokiEn_]
HirokiEn_ has joined #webscreens
00:34:08 [tidoust]
tidoust: Do we need to get feedback from other groups on this design? Privacy perhaps, WebRTC on media capture, etc.
00:34:22 [tidoust]
Jer: Yes, we may be missing some context in which this is used.
00:34:51 [tidoust]
anssik: A Pull Request would help get more feedback.
00:35:40 [tidoust]
Mike: Folks in WebXR are talking about bundling permissions. Similar question about requesting access to device capabilities simultaneously as part of XR request.
00:36:46 [tidoust]
takumif: OK, I'll just make a pull request on the Remote Playback API.
00:37:06 [tidoust]
mfoltzgoogle: My suggestion would be to add it to the Presentation API and reuse the namespace there.
00:38:23 [tidoust]
... It's fine to have a pull request for now and decide afterwards whether it's V1 or V2.
00:38:42 [tidoust]
Mounir: More a Chrome question, is there a real use case from Web developers?
00:39:02 [tidoust]
takumif: Not aware of specific feedback.
00:39:24 [tidoust]
Mounir: Just wanted to point out that we probably don't want to specify features that are not triggered by actual needs.
00:39:48 [tidoust]
Peter: Need will become more important as we expose devices through the APIs.
00:40:30 [tidoust]
Mounir: No concern about the use case, more concerned about whether it's pressing.
00:41:09 [tidoust]
PROPOSED RESOLUTION: Create a pull request for a common prompt along the lines presented and gather feedback on the design from other groups
00:41:31 [tidoust]
RESOLUTION: Create a pull request for a common prompt along the lines presented and gather feedback on the design from other groups
00:42:32 [tidoust]
Topic: Presentation receiver friendly name
00:42:36 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
00:43:14 [tidoust]
takumif: We can add the friendly name to either the controller or receiver side, on PresentationConnection or PresentationReceiver.
00:44:07 [tidoust]
... It is only necessary to expose the friendly name on one side, since both sides can communicate. It makes more sense to expose it on the controller side.
00:44:23 [tidoust]
ericc: It is only available after the user chooses the device?
00:44:26 [tidoust]
takumif: Yes.
00:45:29 [tidoust]
ericc: In Airplay, we replace the video element with a message that says "playing on [foo]" with the name of the device, so I can imagine that being useful to Web applications.
00:45:53 [tidoust]
cpn: Is there a privacy issue even after you gave permission to use the device?
00:46:43 [tidoust]
ericc: For media capture, it's possible to enumerate the capture devices that are attached to a machine. As originally written, a script could just enumerate the devices and get the display names of the devices without a user prompt. Sites were using that for fingerprinting. We changed the spec.
00:47:23 [tidoust]
... Names are always empty until user grant permission through a prompt and we lie about devices.
00:47:59 [tidoust]
... Point is we expose localized string provided by the device only after the user grants permission to capture, which I think is similar to what is being proposed here.
00:48:59 [tidoust]
Mounir: For the Presentation API, you may already have access to that name in 2-UA mode.
00:49:28 [tidoust]
ericc: Definitely something that needs to be reviewed by privacy folks, but my guess is that it is OK after prompt.
00:50:47 [tidoust]
anssik: It seems that we have support for the "receiverName" and that we should refer back to the media capture enumeration API and reach out to PING.
00:51:59 [tidoust]
PROPOSED RESOLUTION: Add "receiverName" to "PresentationConnection" and seek privacy review from PING noting similarity to "enumerateDevices"
00:52:19 [tidoust]
ericc: Note in enumerateDevices, attribute is named "label".
00:53:19 [tidoust]
Jer: If you're concerned about privacy, you can add a mitigation that user agents can use a generic name.
00:53:55 [tidoust]
PROPOSED RESOLUTION: Add "receiverName" to "PresentationConnection" and seek privacy review from PING noting similarity to "enumerateDevices" in "MediaDeviceInfo.label"
00:54:10 [tidoust]
PROPOSED RESOLUTION: Add "receiverName" to "PresentationConnection" and seek privacy review from PING noting similarity to "MediaDeviceInfo.label" in "enumerateDevices"
00:54:23 [tidoust]
RESOLUTION: Add "receiverName" to "PresentationConnection" and seek privacy review from PING noting similarity to "MediaDeviceInfo.label" in "enumerateDevices"
00:57:58 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
01:32:53 [tidoust]
tidoust has joined #webscreens
01:33:02 [tidoust]
RRSAgent, draft minutes
01:33:02 [RRSAgent]
I have made the request to generate https://www.w3.org/2019/09/16-webscreens-minutes.html tidoust
01:33:51 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
01:34:28 [takumif]
takumif has joined #webscreens
01:34:42 [tidoust]
Topic: RemotePlaybackState enum can become misleading when changing media.src (#125)
01:34:57 [tidoust]
-> https://github.com/w3c/remote-playback/issues/125 Issue #125
01:35:59 [takio]
takio has joined #webscreens
01:36:23 [tidoust]
Jer: On both Youtube and Netflix, different behaviors that result in broken experiences with users willing to use Airplay.
01:37:35 [tidoust]
... Youtube will have a single video element and wait for an event that the user picked an Airplay display. Netflix uses wireless presentation changed.
01:37:51 [tidoust]
... Even big web sites wan get this wrong.
01:38:23 [tidoust]
... when display is not compatible.
01:38:52 [tidoust]
... In Airplay, we fire connecting then disconnected, but applications need to understand that this means not compatible.
01:39:10 [tidoust]
... Straw man proposal would be to have an "unsupported" message.
01:39:24 [tidoust]
... Mounir made a counter proposal.
01:40:11 [tidoust]
Mounir: Counter proposal is to provide some information in the "disconnected" event that explains why you got disconnected. If we create a new state, you have to handle state transitions.
01:40:21 [tidoust]
s/"unsupported" message/"unsupported" state
01:40:41 [tidoust]
Jer: From our perspective, it's a very common use case, if not 100% of use cases.
01:41:28 [tidoust]
... If you add a message to "disconnected", you have to prompt the user again, and user may select the same display. Not a great user experience.
01:42:11 [tidoust]
... The new state would allow to retry without prompting again.
01:42:43 [tidoust]
mfoltzgoogle: In Chrome, we mark remote playback as unavailable when no compatible devices are available (e.g. when MSE is used).
01:43:54 [tidoust]
Mounir: If you are an MSE player, you have to be aware that some devices don't support remote MSE, so you need to have a file fallback.
01:43:59 [MasayaIkeo]
MasayaIkeo has joined #webscreens
01:44:19 [tidoust]
Jer: All clients have to create multiple video elements to support all of these?
01:44:42 [tidoust]
Mounir: If you want to do that as a fallback, yes.
01:45:08 [tidoust]
... The benefit for us an API is that you make all decisions before prompting. You check availability.
01:46:07 [dlibby]
dlibby has joined #webscreens
01:46:09 [tidoust]
ericc: The idea with the way that it works in Webkit right now is that we're firing an event that we'll try to remote the playback. If the application knows it won't work, it is responsible for changing the src.
01:46:33 [tidoust]
Jer: Will fail with ads, and playlists. My point is that the scenario is complex even for the biggest video providers.
01:47:07 [tidoust]
ericc: Also, they may not have a second encoding. They need to be able to detect that user wanted to play remotely and that it failed.
01:47:57 [tidoust]
... The expectation is that, once a remote session is started, everything you play locally actually plays remotely.
01:48:20 [tidoust]
mfoltzgoogle: No fallback to screen rendering when playback does not work?
01:48:23 [tidoust]
ericc: No.
01:48:56 [tidoust]
... On Safari, when remote playback does not work, playback plays locally.
01:49:10 [tidoust]
Jer: Which triggers a number of feedback from users.
01:50:09 [tidoust]
ericc: Assumption is that the system plays the video. Not really the case with MSE.
01:50:44 [tidoust]
... Vimeo is another example where it doesn't always work right for the same issue.
01:50:55 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
01:51:32 [HirokiEndo]
HirokiEndo has joined #webscreens
01:52:25 [tidoust]
[Some providers use two video elements, to control ability to switch between sources, and preserve MSE state]
01:53:03 [tidoust]
Mounir: Same issue as codec capabilities. Remote Playback API solves this before connection.
01:53:15 [tidoust]
Peter: Problem is that you don't have a way to filter the list?
01:53:31 [tidoust]
ericc: Right. Before the page is even loaded, the list is established.
01:54:17 [tidoust]
Mounir: Could we have an event that just says "unsupported codec"?
01:54:42 [tidoust]
... You don't want to switch to "disconnected" because you're not, the event would solve the problem.
01:55:26 [tidoust]
Present+ Staphany_Park
01:56:02 [tidoust]
Peter: You might be in a remote session even before the page is loaded?
01:56:20 [tidoust]
ericc: Correct. The way we handle it is by firing an event.
01:56:40 [Joshue108]
Joshue108 has joined #webscreens
01:57:02 [tidoust]
Peter: Issue is you want to convey that you're switching from connecting to connected but cannot play.
01:57:04 [tidoust]
ericc: Yes.
01:57:16 [Joshue108]
present+
01:57:39 [tidoust]
Peter: Use case is somehow the user engages remote playback from the system while or before the page loads.
01:58:06 [tidoust]
Jer: Also transition between e.g. an MP4 file to an MSE-based source.
01:58:36 [tidoust]
Mounir: I wouldn't be surprised if we went to "disconnected" whenever a video stops.
01:59:14 [tidoust]
Jer: Nothing that says that state should go to "connecting" when source changes. Unspecified.
01:59:33 [tidoust]
Peter: So question is about the meaning of "connected".
01:59:58 [tidoust]
ericc: Yes, from my perspective, we are connected. Connected but incompatible.
02:00:07 [tidoust]
Peter: We use the term unavailable.
02:00:36 [hendo]
hendo has joined #webscreens
02:00:38 [tidoust]
Mounir: The spec has not_supported_error and not_found_error.
02:01:21 [tidoust]
mfoltzgoogle: The main question I had is what can the app do about it.
02:01:42 [tidoust]
Jer: You could imagine a separate API that tries to resume playback but doesn't require a prompt.
02:02:06 [tidoust]
mfoltzgoogle: If the application needs to do more than changing the src, how would it know that the new src would be compatible with remote playback?
02:02:10 [tidoust]
Jer: That's a good question.
02:04:00 [tidoust]
[discussion on monitoring display availability as specified in the spec]
02:04:31 [tidoust]
anssik: I think we have understanding that it is a wide issue.
02:04:55 [tidoust]
Peter: Option A would be to add a state that is connected-but-unavailable. To get from there to connected, you change the src.
02:05:18 [tidoust]
... Option B would be disconnected state with a reason. To get back to connected, there's some reconnection mechanism without prompt.
02:06:10 [tidoust]
Jer: Either one would be one. I note it's not even clear that changing src is supported by the spec. Is it unspecified on purpose?
02:06:32 [tidoust]
Mounir: Instead of creating a new state, could we go back to "connecting"?
02:07:12 [tidoust]
Jer: You can just imagine people showing a spinner forever if we stay at "connecting". A bit weird to go to connected then immediately back to connecting.
02:07:37 [tidoust]
... If you're stuck in connecting, you'd need a separate event to tell the application that it needs to do something.
02:07:47 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
02:07:51 [tidoust]
Peter: So Option C is new event
02:08:35 [tidoust]
[Note that availability in the spec is for the union of all devices]
02:09:59 [tidoust]
Peter: With option C, you'd say with connected, and so app that doesn't listen to the new event would just think everything's fine.
02:10:02 [tidoust]
Jer: Yes.
02:10:05 [tidoust]
Peter: Fine with that?
02:10:13 [tidoust]
Jer: That matches current behavior.
02:10:20 [tidoust]
Peter: Some preference for Option C.
02:10:50 [tidoust]
mfoltzgoogle: That seems like a media error state.
02:11:01 [tidoust]
Peter: But you also need to change that when you change the src.
02:11:59 [tidoust]
PROPOSED RESOLUTION: look at how user agent should behave in the case of source switching in Remote Playback API
02:12:36 [tidoust]
Mounir: We may have to keep in mind that it may be optional.
02:14:00 [tidoust]
RESOLUTION: look at how user agent should behave in the case of source switching in the Remote Playback API
02:14:59 [tidoust]
Topic: Window placement & Screen Enumeration
02:15:21 [tidoust]
Mike: We're exploring a window placement & screen enumeration APIs.
02:15:31 [tidoust]
... Native apps often use multiple screens (lots of use cases)
02:15:35 [cpn_]
scribenick: cpn
02:15:35 [tidoust]
scribe: cpn
02:15:50 [cpn_]
Mike: Sites want to be able to control placement on the second screen
02:16:04 [cpn_]
... The OS desktop comprises multiple phyiscal displays
02:16:11 [cpn_]
... [photos of examples]
02:16:28 [cpn_]
... Displays can be high DPI, wide color gamut
02:16:41 [cpn_]
... The Screen interface will describe the current display that the content is presented on
02:16:54 [cpn_]
... Pages don't have a way to introspect the other connected displays in a similar way
02:17:22 [cpn_]
... The existing APIs for opening, moving, resizing windows are limited, due to implementation specific behaviour around moving windows between displays etc
02:17:31 [cpn_]
... There's a gap between what native apps and web apps can do
02:17:52 [cpn_]
... Want to provide the ability to introspect connected displays via screen enumeration API
02:18:06 [cpn_]
... Also support window placement across any display
02:18:17 [cpn_]
Mark: What about virtual desktops?
02:18:53 [cpn_]
Mike: Not considered yet, initial reaction: we wouldn't want sites to be able to move pages to other workspaces
02:19:40 [mounir]
Present+
02:19:41 [cpn_]
... Also thinking about how pages want to be able to control, opening maximised, child or model windows
02:20:02 [cpn_]
... Some of this is exploration, opening multiple windows as in a dashboard display
02:20:15 [cpn_]
... And events
02:20:27 [cpn_]
... [possible screen enumeration API]
02:20:55 [cpn_]
... Returns an array of objects similar to the current Screen object, with width, height, orientation, scale factor, etc
02:21:23 [cpn_]
... Sites could use this together with a window placement API, to allow opening of presentation on one display with notes on another
02:21:41 [cpn_]
... Also, show an advert across 5 displays
02:21:54 [cpn_]
... Show medical imaging app on high bit depth display
02:22:11 [cpn_]
... [possible window placement API]
02:22:37 [cpn_]
... similar to window.open, but more structured and helpful
02:22:50 [cpn_]
Mounir: How does it help move across displays?
02:23:30 [cpn_]
Mike: Looking for input and feedback on that. Existing APIs could be sufficient, could be accessed with permission
02:23:52 [cpn_]
Mounir: Are left and top here the absolute position of each display?
02:24:35 [cpn_]
Mike: It's in screen space, already exposed to the web
02:24:56 [cpn_]
Mounir: Could pass in a display object instead of passing left and top
02:25:23 [cpn_]
Mike: Do you have to specify the screen space or display space coordinates?
02:26:10 [cpn_]
Mounir: You can't moveTo a different display today
02:26:19 [cpn_]
Mike: You can't, but the coordinates are in screen space
02:27:19 [cpn_]
Mike: Have considered this in the explainer
02:27:52 [cpn_]
Mounir: Want to avoid incompatibility, with existing content, and between implementations
02:28:56 [cpn_]
Mark: What window types would this API to?
02:29:17 [cpn_]
Mike: We've mostly been thinking about popup windows
02:29:23 [cpn_]
?: Also stand-alone windows
02:29:58 [cpn_]
Mike: Could add a new API for disambiguating display specific coordinates
02:30:13 [cpn_]
... [Additional explorations]
02:30:38 [cpn_]
... A number of issues come up when movement is disjoint from sizing, could be an opportunity to address
02:30:58 [cpn_]
... Similar to requestFullScreen, could have a request maximized state
02:31:26 [cpn_]
... Important to know when a window has been moved, so the web app could store and then restore their window placement
02:32:22 [cpn_]
Jer: Risk in enumerating devices
02:33:06 [cpn_]
Mike: Permission request prompt
02:33:23 [cpn_]
Mounir: Don't like use of permission prompts except for privacy and security
02:33:53 [cpn_]
... It'll make using the API a pain
02:34:55 [cpn_]
Mike: [demo of Chrome OS with two virtual displays]
02:36:01 [cpn_]
... Can specify coordinates to open on current or second display
02:36:10 [cpn_]
... Simple use case is to open a presentation on the second display
02:36:44 [cpn_]
Anssi: Please share the slides with the group
02:36:54 [tidoust]
-> https://github.com/spark008/screen-enumeration Screen enumeration proposal
02:37:05 [tidoust]
-> https://github.com/spark008/window-placement Window Placement API proposal
02:37:22 [cpn_]
Mike: We're working on the explainers, have sent intent to implement for screen enumaration
02:37:33 [msw_]
https://docs.google.com/presentation/d/1We32ddl5X0EuWGISYr3z6oa7bVI5kN-KQzRQmeISEgM
02:37:41 [cpn_]
Stephanie: There's a WICG discourse thread
02:37:49 [tidoust]
-> https://discourse.wicg.io/t/proposal-screen-enumeration-api/3861 Discourse thread on Screen enumeration
02:38:20 [cpn_]
Anssi: WICG is the appropriate next step for this, or we could recharter the Second Screen CG to add this
02:38:42 [cpn_]
Mark: I'd like to incubate this further, then look at group consensus on whether to adopt into the CG
02:39:19 [cpn_]
Mounir: The Screen interface is in CSS, so it could go there, and the Window stuff is in HTML
02:39:58 [cpn_]
Thomas: Does this group think this is a reasonable addition to the web?
02:40:14 [cpn_]
Eric: The privacy implications need to be considered
02:40:32 [cpn_]
... Want to consult with PING and privacy experts
02:41:55 [tidoust]
Present+ Victor_Costan
02:42:28 [cpn_]
Mounir: Want to avoid having the top of the window off screen
02:42:49 [cpn_]
Eric: Also windows that are that are too small, so the user is unaware
02:43:14 [cpn_]
Mounir: Also always on top windows
02:43:34 [cpn_]
Mark: Get input from Philip Jagenstadt and ? would be helpful
02:43:58 [mounir]
s/? would be /Avi Drisman would be /
02:44:08 [tidoust]
Present+ Thomas_Nattestad
02:44:14 [cpn_]
Anssi: Thank you for this, we'll provide feedback in WICG
02:45:49 [cpn_]
Topic: Accessible RTC use cases
02:46:23 [cpn_]
Josh: Working in APA WG, have use cases in WebRTC, but relate to other groups, e.g., Second Screen
02:46:28 [cpn_]
... Some things for consideration here
02:46:44 [Joshue108]
https://www.w3.org/WAI/APA/wiki/Accessible_RTC_Use_Cases
02:46:51 [ericc_]
ericc_ has joined #webscreens
02:47:10 [cpn_]
Josh: Scenarios and user needs for real-time communication
02:47:39 [takeru]
takeru has joined #webscreens
02:48:05 [cpn_]
... Some specific user needs
02:48:47 [cpn_]
... A screen reader user, actually a navigation and reading device. The user may have many audio devices to manage, want to route output to them
02:49:01 [cpn_]
... We're using the term Second Screen to refer to any output devices, e.g, braille
02:49:10 [TakeruYamada]
TakeruYamada has joined #webscreens
02:49:18 [cpn_]
... A user may have multiple sound cards to manage
02:49:30 [tidoust]
scribe: tidoust
02:50:11 [tidoust]
Josh: [detailing mixing modalities]
02:50:44 [tidoust]
mfoltzgoogle: Web browser consuming content marked up for accessibility, then screen reader and accessible tools, and output devices.
02:51:05 [tidoust]
Josh: Yes, and there might be different types of displays and content.
02:51:38 [tidoust]
... [going through other scenarios]
02:52:34 [tidoust]
... These are some of the use cases that we think are related to some of the work you're doing.
02:52:58 [tidoust]
... We have a table on the page where we map scenarios to specifications and groups.
02:53:17 [tidoust]
anssik: Want us to consider these use cases? Feedback from our side?
02:53:21 [tidoust]
Josh: Both would be great.
02:53:55 [tidoust]
anssik: Maybe we can revise accessibility review with these new use cases in mind.
02:54:12 [Takeru_Yamada]
Takeru_Yamada has joined #webscreens
02:54:12 [tidoust]
mfoltzgoogle: The Remote Playback API has the most overlap with these use cases.
02:54:27 [anssik]
Remote Playback API a11y review https://lists.w3.org/Archives/Public/public-apa/2016Nov/0017.html
02:54:59 [tidoust]
... Media has different tracks. If you want to consider different routing for different tracks, then we'd need to consider that.
02:55:43 [tidoust]
anssik: Let's consider that as part of wide review for the upcoming revised CR release of Remote Playback API.
02:56:29 [tidoust]
Josh: That's great, thank you.
02:56:30 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
02:57:15 [tidoust]
Topic: Planning
02:58:51 [tidoust]
mfoltzgoogle: We discussed earlier on having some kind of explainer for the OSP. No major update since Berlin as I didn't have time to finish it before TPAC.
02:59:15 [tidoust]
... Problems OSP is trying to solve, key items that the TAG might be interested to hear about.
02:59:21 [tidoust]
anssik: yes, including CBOR.
02:59:42 [tidoust]
mfoltzgoogle: Yes. I think I can handle it, but happy to take feedback on the document.
02:59:55 [tidoust]
... If we finish all of the 1.0 issues, the question I have is "what is the next step"?
03:00:55 [tidoust]
[discussion on publishing a final CG report]
03:02:30 [hendo_]
hendo_ has joined #webscreens
03:02:35 [tidoust]
mfoltzgoogle: Review from TAG, WebAppSec, PING, and accessibility would be good.
03:02:55 [tidoust]
anssik: We shouldn't feel blocked by lack of review, not a mandatory step for CGs and no guarantee we'll get reviews.
03:03:24 [tidoust]
mfoltzgoogle: Also Media WG comes to mind
03:04:24 [tidoust]
... For accessibility, it may be a level below what they usually look at.
03:04:33 [tidoust]
tidoust: Josh would be the right point of contact there
03:04:44 [tidoust]
anssik: Yes, we can ask for review. Again, not blocking.
03:05:09 [tidoust]
mfoltzgoogle: So end of november could be target date for publication.
03:05:58 [tidoust]
... If there are issues to look at and resolve, we can schedule a telco.
03:06:19 [tidoust]
... Moving on to SSWG rechartering
03:06:25 [tidoust]
... We need to go through the rechartering process.
03:07:07 [tidoust]
... Two pull requests: an update of the language to reflect on what the group is actually doing (new terms) with no change of scope.
03:07:33 [anssik]
Copy editing: https://github.com/w3c/secondscreen-charter/pull/10
03:07:45 [anssik]
Material changes: https://github.com/w3c/secondscreen-charter/pull/11
03:08:00 [tidoust]
... Second one is material changes to the scope. Features that I think we should consider is that I'd like to explore presentation of part of an HTML document.
03:08:44 [tidoust]
... Similar to fullscreen that can work with any element.
03:08:55 [tidoust]
anssik: We may want to add that as concrete example.
03:09:42 [tidoust]
mfoltzgoogle: The second thing is remote playback features for OSP. Encompasses some of the proposals that we reviewed here today. I can tweak that a bit, for instance to mention receiverName.
03:10:17 [tidoust]
... Also single prompt for both presentation and remote playback
03:10:40 [anssik]
HTML diff of material changes: https://services.w3.org/htmldiff?doc1=https%3A%2F%2Fraw.githack.com%2Fw3c%2Fsecondscreen-charter%2Fmfoltz-draft-2019%2Findex.html&doc2=https%3A%2F%2Fraw.githack.com%2Fw3c%2Fsecondscreen-charter%2Fmfoltz-draft-2019-2%2Findex.html
03:13:15 [tidoust]
mfoltzgoogle: Question about whether to integrate the OSP in the scope of Second Screen WG
03:13:52 [tidoust]
tidoust: We may get feedback that protocols are IETF and APIs W3C. More importantly, what matters is what people want to do.
03:15:13 [tidoust]
mfoltzgoogle: The application level protocol is really only of interest to folks at W3C. My gut feeling is to continue this in the WG and split the parts that are not application-level to IETF if there's interest there. Most is applying existing protocols to our use case.
03:16:28 [tidoust]
anssik: Practically speaking, we're talking about the same people, so it would make sense. Transitioning to the Rec track would make sense.
03:18:33 [tidoust]
tidoust: Companion question if we take that in scope of the WG is whether to mandate support for OSP
03:18:53 [tidoust]
mfoltzgoogle: Cannot tell whether that's a shared goal by everyone.
03:21:17 [tidoust]
tidoust: Let me have a discussion with Strategy team on taking the OSP in scope of the WG.
03:21:31 [tidoust]
mfoltzgoogle: Can we have feedback before we need to take a decision, end of October?
03:21:36 [tidoust]
tidoust: Definitely.
03:21:56 [tidoust]
mfoltzgoogle: We may need a short extension to bring OSP on board before we recharter properly
03:22:21 [tidoust]
tidoust: Doable without going through the membership if justified and scoped to 2-3 months.
03:24:12 [tidoust]
Peter: Going to dispatch in IETF would be a good way to gauge interest either on the entire OSP, or IoT-type pairing stuff.
03:24:22 [tidoust]
... Next meeting is in November. In March 2020 after that.
03:26:49 [tidoust]
anssik: General question on what we would like to gain.
03:27:16 [tidoust]
Peter: Reviews from experts, e.g. on mDNS, QUIC, CBOR.
03:27:32 [tidoust]
anssik: OK, we need reviews, but not necessarily an IETF-track document.
03:28:23 [tidoust]
... If we are seeking feedback from IETF individuals, it would be easier if the OSP remained in the CG as they may not be members.
03:29:12 [tidoust]
Peter: The IETF has a review process, you'll have more chance to get feedback going that route than pulling people in.
03:29:51 [tidoust]
... It may be good to have feedback from a W3C/IETF perspective, at the organizational level.
03:29:55 [tidoust]
tidoust: OK, will work on that.
03:31:36 [tidoust]
mfoltzgoogle: We need the feedback before end of October so that we can adjust things correctly.
03:32:18 [tidoust]
mfoltzgoogle: Another change is that I consider content mirroring to no longer be out of scope, since that seems to be what presentation of part of an HTML page is about.
03:34:13 [tidoust]
... If we can decide on the network protocol question, I would support two year extension.
03:34:35 [tidoust]
tidoust: Usually good practice not to create too short charters. Group can recharter before the end of its charter.
03:35:06 [tidoust]
ACTION: mfoltzgoogle to update proposed charter end date to end of 2021
03:35:18 [tidoust]
ACTION: anssik to make sure that draft charter gets reviewed by WG participants.
03:36:02 [tidoust]
ACTION: tidoust to run internal discussions with Strategy team about possible onboarding of the Open Screen Protocol.
03:37:26 [tidoust]
RRSAgent, draft minutes
03:37:26 [RRSAgent]
I have made the request to generate https://www.w3.org/2019/09/16-webscreens-minutes.html tidoust
03:38:00 [tidoust]
RRSAgent, draft minutes v2
03:38:00 [RRSAgent]
I have made the request to generate https://www.w3.org/2019/09/16-webscreens-minutes.html tidoust
03:38:26 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
03:57:21 [ericc]
ericc has joined #webscreens
04:09:55 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
04:18:31 [hendo]
hendo has joined #webscreens
04:27:46 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
04:31:46 [msw_]
msw_ has joined #webscreens
04:36:49 [tidoust]
tidoust has joined #webscreens
04:37:19 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
04:40:14 [tidoust]
Present+ Grisha_Lyuskin, Alex_Russel, Raymes_Khoury
04:40:18 [dlibby_]
dlibby_ has joined #webscreens
04:40:32 [tidoust]
Topic: Window segments
04:41:51 [tidoust]
Daniel: Not really second sreen, but kind of. Main scenario we want to enable is to position elements on specific views. The rectangle of your main viewport may not be in the same display.
04:42:32 [tidoust]
... The other problem we're trying to solve is things like keyboard that pop up. Currently no way for applications to reposition their content, e.g. to scroll things into view.
04:44:01 [tidoust]
... We came up with a proposal for getWindowSegments() that return the list of segments, areas of your layout viewport that exists in different regions.
04:44:35 [tidoust]
Thomas: Does that include device with two screens (on the front and the back).
04:45:01 [tidoust]
Daniel: Not necessarily, we're encapsulating when your browser viewport encompasses these different displays.
04:45:19 [tidoust]
... One question is how it could combine with the screen enumeration API.
04:46:40 [tidoust]
Alex: What's new is knowing that these segments exist, because that's not exposed for now.
04:46:44 [tidoust]
Daniel: That's correct.
04:47:22 [tidoust]
... Foldable displays.
04:47:49 [tidoust]
... Logical thinking about what is a screen and what is a segment.
04:48:47 [tidoust]
Mike: What I've been looking at is existing APIs and what they can bring for screen enumeration. I'm curious about the info that is already available.
04:49:16 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
04:49:23 [tidoust]
Daniel: I can't say, but being able to handle these scenarios in a global way would be good.
04:49:40 [tidoust]
Thomas: Does this go beyond foldable displays?
04:49:47 [tidoust]
Daniel: That's the main use case.
04:51:20 [tidoust]
mfoltzgoogle: Any event on the Web platform that surfaces changes? For tablet mode, there's a hack, but for foldable, you'd want something as well.
04:51:30 [tidoust]
Daniel: We do have an eventing model.
04:52:15 [dlibby_]
link to the explainer
04:52:15 [dlibby_]
https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/master/Foldables/explainer.md
04:52:18 [tidoust]
Mike: Similarly, for screen enumeration, we feel that an eventing model is needed.
04:53:31 [tidoust]
... Screen enumeration and window placement originated from Service Worker. It could be exclusive to a Service Worker.
04:53:46 [tidoust]
Thomas: I would hope not to have to install a Service Worker.
04:53:54 [tidoust]
Mike: There is a lot of history of abuse with window.open
04:54:25 [tidoust]
... I almost see a parallel between these two thoughts and Remote Playback API and Presentation API doing similar things.
04:55:01 [tidoust]
mfoltzgoogle: Presentation will allow targeting wired displays on top of wireless displays, and that's the main overlap with the scenarios that you're considering.
04:55:13 [tidoust]
... The goals are slightly different, I think.
04:56:26 [tidoust]
Daniel: The window segment proposal is more a reactive thing that a proactive thing.
04:56:41 [tidoust]
mfoltzgoogle: Can you learn that you're on a foldable now without this API?
04:56:49 [tidoust]
Daniel: I don't think so.
04:57:58 [tidoust]
mfoltzgoogle: If you're on a mobile and get a window resize, you can probably quickly infer that you have a foldable.
04:58:19 [tidoust]
... Gut feeling is that it doesn't add much info from a fingerprinting perspective.
04:59:01 [tidoust]
Mike: Curious to tell what happens today when keyboard pops up.
04:59:11 [tidoust]
mfoltzgoogle: viewport change event.
04:59:43 [tidoust]
Mike: Would you expect a resize event when keyboard appear on the bottom left?
04:59:51 [tidoust]
Daniel: No, since not a regular shape.
05:00:25 [tidoust]
Mike: If there's no way that your site can know that it's been occluded by a virtual keyboard, that seems like a gap to fill.
05:00:34 [tidoust]
Daniel: Right, I think that's the issue.
05:01:18 [tidoust]
... There is an issue in the CSS WG tracker on declarative. We're leaving that on the side for now.
05:02:46 [tidoust]
mfoltzgoogle: Two cases, tiles and foldable. Different mechanisms, although the tile one should cover most of the needs.
05:03:31 [tidoust]
... As long as the Web application can derive the tiling from other sources, we don't have so much of a fingerprinting issue.
05:04:07 [tidoust]
Alex: I would like to push back on solving all problems in user land.
05:05:16 [tidoust]
Mike: With regards to split screen devices, what would the system expose?
05:05:25 [tidoust]
... Separate displays?
05:05:45 [tidoust]
... What does it look like for one application frame to span multiple displays there?
05:06:30 [tidoust]
... I know Chrome OS is kind of strange in this regard, it creates one virtual display that spans the physical displays.
05:07:39 [tidoust]
Daniel: Breakout session scheduled tomorrow to go deeper into details.
05:07:59 [tidoust]
Thomas: I'm excited to know what native is doing, because I'm sure Samsung had to pull it into Android.
05:08:03 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
05:08:46 [tidoust]
Daniel: Another option looking at screen enumeration was that a display could be represented as a set of segments. Some overlap.
05:09:57 [tidoust]
Mike: There is this complete lack of capabilities to see the available displays.
05:10:46 [tidoust]
... If we were to explore something along the lines of segments without screen enumerations, it would not capture use cases such as overlapping windows, difference of resolutions between displays, etc.
05:11:43 [tidoust]
... I'm curious about how displays work on a Samsung DEX.
05:13:15 [msw_]
msw_ has joined #webscreens
05:13:24 [tidoust]
RRSAgent, draft minutes v2
05:13:24 [RRSAgent]
I have made the request to generate https://www.w3.org/2019/09/16-webscreens-minutes.html tidoust
05:14:23 [mfoltzgoogle]
mfoltzgoogle has joined #webscreens
05:23:50 [msw_]
msw_ has joined #webscreens
05:37:29 [ericc]
ericc has joined #webscreens
05:54:03 [tidoust]
tidoust has joined #webscreens
06:13:34 [MasayaIkeo]
MasayaIkeo has joined #webscreens
06:18:36 [tidoust]
scribe+ cpn_
06:18:42 [tidoust]
RRSAgent, draft minutes v2
06:18:42 [RRSAgent]
I have made the request to generate https://www.w3.org/2019/09/16-webscreens-minutes.html tidoust
06:19:59 [tidoust]
i/Mike: Sites want to be able/scribe: cpn_
06:20:14 [tidoust]
i/Josh: [detailing mixing modalities]/scribe: tidoust
06:20:17 [tidoust]
RRSAgent, draft minutes v2
06:20:17 [RRSAgent]
I have made the request to generate https://www.w3.org/2019/09/16-webscreens-minutes.html tidoust
06:20:45 [tidoust]
RRSAgent, bye
06:20:45 [RRSAgent]
I see 3 open action items saved in https://www.w3.org/2019/09/17-webscreens-actions.rdf :
06:20:45 [RRSAgent]
ACTION: mfoltzgoogle to update proposed charter end date to end of 2021 [1]
06:20:45 [RRSAgent]
recorded in https://www.w3.org/2019/09/16-webscreens-irc#T03-35-06
06:20:45 [RRSAgent]
ACTION: anssik to make sure that draft charter gets reviewed by WG participants. [2]
06:20:45 [RRSAgent]
recorded in https://www.w3.org/2019/09/16-webscreens-irc#T03-35-18
06:20:45 [RRSAgent]
ACTION: tidoust to run internal discussions with Strategy team about possible onboarding of the Open Screen Protocol. [3]
06:20:45 [RRSAgent]
recorded in https://www.w3.org/2019/09/16-webscreens-irc#T03-36-02