Web & TV Convergence

Workshop Report

Contents

• Executive Summary
• Participants
• Workshop Discussions
  • Session 1 — Welcome
  • Session 2 — Hybrid TV
  • Session 3 — Multi-screen 1
  • Session 4 — Panel: W3C and SDO alignment, moderated by Giuseppe Pascale
  • Session 5 — Multi-screen 2
  • Session 6 — Panel: Key issues in web media
  • Session 7 — More Challenges for Web Media
  • Session 8 — Wrap up & Next steps
• Future Steps
  • Synchronization of Video and (meta) data, video & audio
  • Testing devices based on web technologies
  • Rendering and control of linear video using <video> (so-called «Tuner API»)
  • Misc gaps around delivery & rendering of IP video, mostly integration issues for TV devices
  • Discovery and communication between two UAs or a UA and another device/service
  • Performance measurement (benchmarks) for web technologies/animations
  • Accessibility features
  • Pluggable CDM for EME

---

Note: For more details of the discussions the full workshop minutes are available.

Executive Summary

Since its inception in 2010, the W3C Web and TV Interest Group has been turning TV-related web requirements into reality, helped by workshops in Tokyo, Berlin and Hollywood. Recent developments both in technology and business, however, have meant more changes are needed for the web to fully integrate with TV and similar entertainment. The fourth Web and TV workshop was held in March 2014 to meet this challenge. The location was Munich, Germany, at the headquarters of IRT who were also generous with logistical and technical support. The workshop theme was "Web and TV Convergence: Aligning Global Standards and What's Next for Web and TV". To that end, the roughly 130 attendees included many representatives of both key stakeholders in the TV industry as well as regional standards organizations. Broadcasters, in particular from Europe, were well represented but we were fortunate to have participants and speakers from other industries and areas of the globe as well.

The topics for discussion were decided after a public call for papers, following which almost 30 were received. The majority concentrated on multi-screen applications and hybrid TV, showing that these topics are still very relevant and a key issue for W3C to address. Over the two days we also engaged in discussions covering other areas such as testing, audio, accessibility, metadata and more, as you can read in more detail below.

The outcome of the workshop was a selection of topics for investigation and future work. Some required guidance for interested parties of where discussions should take place, such as synchronisation of video and metadata or discovery and communication between devices (multi-screen). Other topics were issues that the Web and TV Interest Group should address directly such as enabling testing requirements to be fulfilled, ensuring accessibility requirements are met in emerging standards, and raising the importance of performance improvements and benchmarks. In addition, the Interest Group should be aware of and monitor related Community Groups, including creating a new one to draft a Tuner or Channel Control API.

Participants

There were 126 participants from the following 88 organizations, both W3C members and non-members:

Abertis Telecom, Alcatel-Lucent, Apple, ARD, ARM, AT&T, Bayerischer Rundfunk, BBC, BLM (Bayerische Landeszentrale für neue Medien), BSkyB, C-DAC (Centre for Development of Advanced Computing), CableLabs, Cisco, Conax, Conbox, Condat, CreateCtrl, Czech Television, Deutsche Telekom, DFKI, Digital TV Labs, DIT (Deggendorf Institute of Technology), Dolby, Dong-Eui University, EBU (European Broadcasting Union), EchoStar, Elgato Systems, Ericsson, Espial, ETRI (Electronic and Telecommunications Research Institute), France Télévisions, Fraunhofer FOKUS, Freesat, Fuji Television Network, Huawei Technologies, Igalia, Irdeto, IRT, Japan Ministry of Internal Affairs and Communications, Kaonmedia, KDDI, LG Electronics, LTFE, Luxunda, M-net Telekommunikations, MDR (Mitteldeutscher Rundfunk), MNITI, MODUL University, MPAA, MStar Semiconductor, NBCUniversal, NDR (Norddeutscher Rundfunk), NHK (Japan Broadcasting Corporation), Nippon TV, NPO (Netherlands Public Broadcasting), NTT (Nippon Telegraph and Telephone Corporation), Opera, Orange, Oriental Cable Network, Panasonic AVC Networks, Ph.D Student, Polytechnic University of Bari, RAI Research Center, RTL Group, RTV Slovenia, Samsung Electronics, SES Platform Services, Sky Deutschland, Sky Italy, SmarDTV, Sony Corporation, St. Pölten University of Applied Sciences, Strategy & Technology, Swisscom, TBS (Tokyo Broadcasting System), Télécom ParisTech, Thomson Video Networks, TNO (Netherlands Organization for Applied Scientific Research), Tomo-Digi, Toshiba Corporation, Toshiba Information Systems UK, TP Vision, TV Asahi, TV TOKYO, University of Almeria, W3C, WDR (Westdeutscher Rundfunk), WOWOW

Workshop Discussions

Session 1 — Welcome

Links to slides

The workshop started with a welcome by Philipp Hoschka (W3C) and Ralf Neudel (IRT) and continued with two introductory talks:

Since some participants were not familiar with W3C, Philipp Hoschka introduced the W3C and its standardisation track covering the process from initial ideas generated by workshops and Community Groups to a standard's publication and subsequent maintenance. There are certain types of group involved in this process which are Community Groups, Business Groups, Interest Groups and Working Groups. The only one of these that can create a full standard is Working Group and the steps required are Working Draft, Last Call Working Draft, Candidate Recommendation, Proposed Recommendation and finally Recommendation.

This was followed by a further explanation of W3C work but specifically relating to the TV industry by Giuseppe Pascale (Opera Software). The Web and TV Interest Group is the focal point of this work so he started by explaining the goals of the group, specifically:

• Identification of requirements for tighter support of media-centric applications on the Web Platform;
• Identification of gaps in the Web Platform that do not allow the identified requirements to be met;
• Review of deliverables under development by other W3C groups that are relevant to the IG scope and report bugs as appropriate;
• Liaison with other organizations in the media industry that are using the Web Platform for their technical specifications and/or their services to foster alignment and interoperability on a global scale.

To reach these goals the Interest Group gathers use cases, extracts requirements and creates a gap analysis to guide further standardisation efforts, but it should be remembered that the Interest Group itself cannot create specifications. That is the responsibility of Working Groups, possibly using input from Community Groups and Business Groups. Since 2010 the Web and TV Interest Group has been the source of several additions to HTML and other specifications mostly through the use of Task Forces.

Session 2 — Hybrid TV

Links to papers and slides

As the theme of the workshop was "Web and TV convergence", there were reports of real-world experience of hybrid TV by members of HbbTV and IPTV Forum Japan. They included technical details of implementations as well as usage statistics.

HbbTV

For HbbTV, hybrid TV business drivers come from analog being switched off, consumer demand, broadcaster innovation, governments and pay-TV operators. HbbTV adoption is mostly concentrated around Europe but there are trials being run in East Asia as well as interest and adoption in other regions. All deployments of HbbTV in 2014 will be version 1.5 which is replacing version 1.0 and includes MPEG DASH and DRM APIs. Version 2.0 is in development with improved support for HTML5, content synchronisation and ad insertion.

As part of this development, bugs have been filed with W3C HTML WG (#24859, #24860, #24861) concerning the "controls" attribute of the video and audio elements, which, as a boolean attribute, does not offer enough control to web apps. Other issues include:

• HTML5 MediaError not detailed enough
• HTML5 VideoTrack and AudioTrack support too limited a set of metadata: W3C bug #24863
• Missing short-lived (instantaneous) cues: W3C bug #24161
• Unclear in spec if DOM FocusEvents are generated when the UA gains and loses focus: W3C bug #23271

An overview of the HbbTV testing system was also covered. This is important for aligning the expectations of both viewers and broadcasters. A comparison of HbbTV testing requirements and W3C work in this area showed gaps and areas to be addressed.

IPTV Forum Japan

Hybrid TV is also being developed in Japan and NHK launched their Hybridcast service to the public in September 2013 with several manufacturers are producing Hybridcast-ready TVs. Commercial broadcasters are also carrying out trial services and workshop participants heard a summary of the progress from each broadcaster present. The latest specifications include guidelines for applying HTML5 and some extension APIs. Broadcast channels are shown within an "object" element using type "video/iptvf-broadcast" and CSS is used to specify coordination and z-order. An English version of the specification is now available: www.iptvforum.jp/en/download/

Session 3 — Multi-screen 1

Links to papers and slides

Multi-screen is still a hot topic accounting for many of the position papers received. It was split into two sessions over the two days allowing several different approaches to be presented.

Enabling Second Display Use Cases on the Web

Louay Bassbouss (Fraunhofer FOKUS) presented on behalf of the W3C Second Screen Presentation CG. Its goal is to "define simple APIs to request displaying an HTML page on the secondary screen and some means for the first screen to communicate with and control the second page, wherever it is rendered". Use cases include presentations, video and image sharing, gaming, and media flinging to multiple screens. Intel has contributed an initial draft API which has subsequently been edited based on feedback from a variety of contributors. The API itself uses a "navigator.presentation" object which enables a search for available screens using existing connection methods, both wired and wireless.

A Flexible Multi-Screen Solution Based on UPnP

Universal Plug-­n-Play Forum has created a new and open multi-­screen specification leveraging UPnP devices and W3C specifications to enable primary and secondary screen devices to interact with each other in a synchronized fashion over IP networks. Some gaps in W3C specifications were identified and the new UPnP specification was described as a solution. The DCP (Device Control Protocol) has a Screen Control Point component which is used to provide interactive services to other display devices which are implemented with the Screen Device component.

Multiscreen Service in Shanghai

Oriental Cable Network offered an overview of the network architecture and interactive TV service which is in public use with millions of users. There was also a description of their multi-screen service deployment. This includes video-on-demand and video messaging. Both the XMPP protocol and HTML5 can be used within the system but there was concern that metadata specifications for video are fragmented. Currently cable operators follow CableLabs’ ADI metadata standard system but Internet video service providers use their own metadata format.

Second Screen User Experiences

A proposal was offered for a second-screen system based on use cases such as friends sharing content remotely through a social network. As well as W3C specifications such as Service Workers and Web Sockets, a JavaScript library is mentioned which exposes a social API and shared context. Existing technologies already address issues such as local peer to peer discovery and messaging but it was argued that a server-based approach is simpler and more reliable. A demo was also available to show the synchronisation in action.

Session 4 — Panel: W3C and SDO alignment

Links to slides

A panel discussion moderated by Giuseppe Pascale. Panelists were:

• Jean-Pierre Evain, EBU
• Jon Piesing, TP Vision (representing HbbTV/OIPF)
• Kinji Matsumura, NHK (representing IPTVFJ)
• Clarke Stevens, CableLabs (representing UPnP/DLNA)
• Philipp Hoschka, W3C

Following short introductory presentations from EBU, HbbTV, IPTV Forum Japan and UPnP/DLNA, the discussion evolved around the following issues:

• Why do you choose HTML5 as a platform?
  • Ability to re-use web developers' knowledge;
  • Only real option because of HTML browsers in TVs.
• What is the challenge of referencing specs from W3C including HTML5?
  • TV has specific requirements (e.g. no windowing, remote controllers);
  • Organizational - what spec version do we refer to if it's not a Recommendation? How does that impact testing & certification?
  • Difficulty of adding own functionalities and then aligning with the spec.
• Why is the existing web industry able to handle spec fragmentation but not the TV industry?
  • Things like Timed Text are still not finished so platforms have to create their own extensions;
  • TV standards are traditionally very stable but the web is a frequently changing world;
• When you run into spec gaps in your SDO what's the resolution process?
  • We bring requirements back to the W3C;
  • We add our own extensions for regional broadcast requirements;
  • Small gaps are submitted to W3C's bug tracking system: www.w3.org/Bugs/Public/
  • Sometimes difficult, e.g. what a broadcaster sees as a video might not be what the W3C calls a video.
• What are some of the success and failures? Is there a different between working on the "inside" as a member and bringing input as an outsider?
  • The TV community communicated their video element requirements well;
  • The testing effort has been disappointing;
  • There's always a balance with the investment required and whether it's worth bringing something to the W3C table.
• A lot can be done through Community Groups. What gaps are the most difficult ones? Real-time linear content; access to EPG or metadata; harmonising and third-party integration?
  • Dealing with linear content is a critical area. We could use an abstraction layer whereby we can indicate a URL.
  • A lot of work has been done on MPEG-DASH. What do we need at W3C to fit with that?
• Is it possible to have a basic profile quickly that focuses on basic things like a video player to create a unique standard that can have mass deployment?
  • If it's a good compromise, it takes time;
  • To do something quickly, you need a small group of people and ruthless focus. If you add more people and more ambition it takes longer.
• Testing - The need is still there. Is there anything SDOs can do?
  • SDOs would be pleased to use W3C tests if they exist and are good enough;
  • The actual writing of tests takes less time than other parts of test suite creation, including peer review;
  • There may be many existing tests but working out which are good is non-trivial;
  • The W3C effort to raise funding has not really worked;
  • Is there a possibility that testing could move forward through domain-specific efforts?
  • Test The Web Forward has some resources and assets that a Web and TV testing community could use.
• What are some quick wins for web and TV going forward?
  • Make it easier for people to join W3C from broadcasting world and for them to speak up;
  • There are several small possible wins already submitted by HbbTV in the W3C bug tracking system;
  • Specific issues where you've several dedicated people in a room to fix a problem with aligned opinions.
• At least one of the slides showed a bewildering number of standards, which reference testing. The bodies are slightly different in how they implement specifications. Is that a problem? Should W3C look at that?
  • The two words "reference implementation" cause concern. They could give a particular vendor an advantage;
  • I think you have to look at each organization. There's also operability testing.

Session 5 — Multi-screen 2

Links to papers and slides

Challenges for enabling targeted multi-screen advertisement for interactive TV services

There are various challenges for developing interactive TV services, connecting TV and companion devices and adaptive streaming in the browser. An API was proposed for building multi-screen applications using Web technologies. There is also an implementation called FAMIUM18 which is a proof of concept. The components of the FAMIUM framework dealing with browser-based DASH and DRM are FAMIUM.transcoder which converts existing media (on-demand or live content) to DASH-compliant content, and FAMIUM.player which is a Web-based DASH and DRM-capable media player. For encrypted content, FAMIUM.cenc relies on underlying DRM platforms that can be accessed via dedicated Content Decryption Modules (CDM).

Linking Web Content Seamlessly with Broadcast Television: Issues and Lessons Learned

Linked Television is the seamless interweaving of TV and Web content into a single, integrated experience. The LinkedTV workflow consists of seed content, media fragments, annotation, enriched related content, curation and finally multiscreen viewing personalisation. This works with web video but there are gaps in broadcast TV integration, such as:

• Support of Media Fragments URI specification
• Locator schemes and attached annotations
• Extension of TV metadata standards
• Integration into Broadcast publishing workflows

Inter-Device Media Synchronization in Multi-Screen Environment

Issues relating to inter-device media synchronization in a multi-screen environment were discussed, as was the general architecture for inter-device synchronization. Technology standards are important to guarantee interoperability for inter-media synchronization in a multi-screen environment. Based on several examples of content sharing, the main issues are:

• Direct or indirectly network device discovery (cross-browser)
• Direct communication or indirect communication for exchanging timing information (cross-browser)
• Signalling information to migrate service components seamlessly
• Component-based web application authoring mechanism to migrate specific component to other screen

Three challenges for Web&TV

For users to enjoy content seamlessly irrespective of device or video distribution mechanism, three areas exist where work is yet to be done.

• How to do large scale, low delay (and low jitter) video delivery to the browser
• How to integrate synchronisation capabilities into the browser
• How to manage the home situation when the STB browser is in the cloud (both home network and privacy)

There is also a distinction between what the web can mean: Sometimes 'web' refers to web technology and other times 'web' indicates that realm where inspired and skillful people forge brilliant new applications.

Session 6 — Panel: Key issues in web media

Links to papers and slides

A panel moderated by JC Verdié with short talks on varied subjects, followed by discussion and questions from the floor.

Commercial Content in Browsers

The future situation for commercial content service providers is anti-competitive because of the nature of DRM support in different browsers. In addition the end-user experience is likely to suffer since the user may either have to use a non-preferred browser or a native application. Also a vision to view prime content on any browser may not become mainstream except those integrating multiple DRM systems at a costly factor. Application based content viewing will continue to be the preferred choice. In order to avoid the above issue it is suggested that it is further discussed and a clear industry direction is set.

Web Distribution Formats for Subtitles and Captions

A clear, easy-to-understand explanation of the status quo in which two timed text specifications exist for video captioning. It was argued that antagonism between these two types of Timed Text standards causes confusion in the industry and there is a risk of a division between professional content providers and browser manufacturers that slows down the adoption of accessibility services. A further question is how different interests in the market can be coordinated and which position and power the browser manufacturers have in the definition of new web standards.

Discovery and Second Screen, HTML5 video and MSE

Based on requirements from the Home Network Task Force of the Web and TV IG, discovery is required and is key to many usages in the home. Also needed is UPnP and Bonjour compatibility, including for legacy devices. However, it was argued that the W3C Network Discovery API is in trouble, because the current spec is limited, using CORS makes it incompatible with legacy devices, browsers won't implement it (because of UPnP legacy), and work on it is more or less stalled. Four options were presented:

• Option 1: Hide URLs and IPs, manage messaging
• Option 2: Make browsers mutually discoverable
• Option 3: Define safe entities à la webscreens
• Option 4: Use object sharing

Session 7 — More Challenges for Web Media

Links to papers and slides

Network-Assistance and Server Management in Adaptive Streaming on the Internet

A proposal for server-based adaptive streaming was offered to address issues with client-managed DASH, such as:

• No Network/Server involvement in adaptation management
• No guaranteed and coherent QoE, when involving many clients of different types and screens from different vendors
• No global optimization, in allocating network and server resources across many different clients
• No Network QoS support for differentiating users and services to facilitate viable business models for network operators
• No incentives for network operators to participate, to guarantee QoS

Two areas were identified as needing to be standardised by W3C: Parameters (subscriber information; service description; network QoS information) and interfaces/protocols (support for server adaptation management requests and responses; support for network QoS assistance request and responses).

The Audio Definition Model

Audio is a critical element of media content, and progress is being made in improving the audience experience in terms of quality, immersiveness and interactivity. The key to ensuring audio can be correctly reproduced for the user is good metadata that is tied closely to the audio. Up until now there lacked any metadata model that sufficiently described the format of audio that is sufficient for these future approaches. Therefore the EBU developed the Audio Definition Model (ADM) (EBU Tech 3364) which can give a complete technical description of the audio within a file to allow it to be correctly rendered. ITU contributions are also being made in this area. A list of future work includes:

• A list of standard configurations will be drawn together (Database; Reference XML file);
• Audio Object parameters need continual refinement;
• Libraries/APIs for parsing and generating ADM metadata to be developed;
• Look at streaming methods.

PARS – A Multiscreen Web Application Platform

Pars is a multiscreen web application platform. Although the design and implementation of Pars platform is still immature, leaving research opportunities in security, for example, it proves the feasibility of a web-based multiscreen application platform. The approach is a distributed application with such benefits as:

• No pre-installation;
• Roles of devices can be re-organized during runtime;
• PARS, COLTRAM, and other works from KAIST, NTT, etc.

To meet this goal, standards are required such as local discovery, P2P messaging and remote execution. The proposed Second Screen Presentation CG's API seems to offer these. Also important are interoperability, time synchronisation, and a balance between security/privacy and usability.

IPTV using P2PSP and HTML5+WebRTC

An implementation of the P2PSP (Peer-to-Peer Straightforward Protocol) over a WebRTC/HTML5 framework was presented. This results in a Web client that can be used to retrieve in real-time multimedia content from a streaming server. This solution, compared to non P2P-based ones, minimizes the bandwidth consumption at the server side and provides a good quality of service as well as avoiding the need to download plugins or third party software. In other words, HTML5 and WebRTC should be considered for this and other P2P video streaming protocols.

Session 8 — Wrap up & Next steps

Links to slides

Glenn Deen (NBCUniversal) gave a short introduction to new project called GGIE - The Glass to Glass Internet Ecosystem. Still in its early stage, the project is not a new standards developing organization (SDO) - rather it's a collaboration of experts and volunteers to develop use cases and gap analyses in collaboration with existing SDOs. They will then be free to use the results of this work.

Finally, the workshop closed with a session chaired by Giuseppe Pascale to address the next steps. See next section.

Future Steps

The format for discussing next steps for interested parties was:

• Go through the list of identified gaps;
• Check that they are correctly defined;
• Is the W3C the right place for a follow-up?
• (if yes) is the IG the right place to have a follow-up?
• Assess interest in working on it in W3C.

The following topics were discussed one by one, and also in a follow-up conference call on April 16th.

Synchronization of Video and (meta) data, video & audio

There is partial coverage from existing specs (Media Fragments, HTML5 multitrack API, HTML5 MediaController) but not for stream events and exposure of multiple timelines. There's also an open issue of whether this is this something for W3C to handle or is it an integration issue? Are there gaps in the existing specs (e.g. HTML5)?

• Interested parties need to raise bugs against relevant specs
• Send Bug report to IG list and ask for support
• Use the TV IG list as a way to share issues and ask for support

Testing devices based on web technologies

Not surprisingly there was agreement that testing is very important and that it would be ideal to have a common testing solution for all HTML-based products. While there is testing work being done within W3C, there needs to be more information on the existing material and tools, making sure that they're suitable for the TV industry, as well as more test cases.

There are also issues regarding differing definitions of testing and what is required for a full test suite. Operational issues also exist around test development, approval and maintenance within the W3C. Options going forward include:

• Reviving the Testing Task Force to look at the existing tools and material and see if they are suitable for the TV industry (and if not, work to address the missing features);
• A Test The Web Forward event for TV-related specifications
• Getting the TV IG to facilitate a conversation among interested parties

Rendering and control of linear video using <video> (so-called «Tuner API»)

There is no common API to render and control linear content via the HTML video element. The HTML5 API may not be currently covering all the requirements needed to render linear content, e.g. access to list of services, parental access control, etc.

• A new Community Group has been proposed to draft a technical solution to be later moved to a WG;
• This will be followed-up in the TV IG;
• There have since been offers from other organizations to use their existing APIs as a basis for a new API.

Misc gaps around delivery & rendering of IP video, mostly integration issues for TV devices

Jon Piesing, representing HbbTV, has submitted several bugs to the HTML WG regarding current gaps. The TV IG should help other SDOs to also send their issues and requirements to the HTML WG. This is to be followed-up in the TV IG as well as:

• How can the TV IG support these SDOs?
• Should the SDOs monitor the mailing list and try to contribute to the discussion if they are facing the same issues?
• Would it help for the IG to have phone conferences where bugs are discussed?
• Maybe we should use the IG to keep track of TV related bugs and their status.

Discovery and communication between two UAs or a UA and another device/service

There is already ongoing work in this area:

• Second Screen Presentation CG
• System Applications WG
• Network Service Discovery API (in DAP WG)

It was decided that the discussions should continue within their respective groups with the option of calling upon the TV IG to help if necessary.

Performance measurement (benchmarks) for web technologies/animations

It's difficult to guarantee a good experience on low-end devices such as TVs and STBs. To address this, the workshop discussed whether there should be a reference benchmark. The topic has been raised in Web Performance WG but not prioritized. Can/should the TV IG help with that? There is a risk that people optimise for tests and real world apps still suffer from performance issues. Is doing nothing worst than doing something that is not perfect?

Maybe now is the right time to raise the issue again with the Web Performance WG and perhaps also the gaming community. The TV IG should collect requirements from interested parties to be passed on to the appropriate groups.

Accessibility features

Accessibility for TV tends to focus on captioning and subtitles but there are many other issues beyond this. The Web Accessibility group is producing media accessibility requirements guidelines and in the US there is also a Communications and Video Accessibility Act: transition.fcc.gov/cgb/dro/cvaa.html

• The TV IG should work on collecting use cases and requirements for accessibility features other then captioning and subtitles;
• The Web Accessibility group are interested in engaging with the TV IG. The W3C team contacts should work on this collaboration.

Pluggable CDM for EME

This concerns the issues in the "Commercial Content in Browsers" talk by Ericsson (see session 6 above)

• Is this something for the TV IG or better raised directly with Media TF of the HTML WG (working on EME)?
• Is the W3C the right place to discuss it?

TV IG participants should raise the issue within the Media Task Force of the HTML WG.

---

Questions? team-webtv-ws-submit@w3.org
Photo credit: Petr Kratochvil