Requirement Document Draft

(ONLY THE EDITOR SHOULD CHANGE THIS DOCUMENT)

This section contains the draft for a requirement document as discussed by Home Network TF. This is a best-effort attempt to summarize the consensus reached so far by the TF but is still to be considered a draft to subject to change at any time.

Abstract

There is an increasing amount of personal and streaming (broadcast) content that users would like to be able to access from any device in the home (personal computers, tablets, mobile phones, TVs and others).

Growing numbers of consumer electronic devices such as TVs and mobile phones can access internet based services as well as content originating from both home network devices and broadcast services.

There is also growing interest in providing richer and novel user experiences that provide a cohesive experience across multiple devices including TVs, mobile phones, tablets and others.

This document lists the design goals and requirements that proposed W3C recommendation should support in order to enable access to services and content provided by home network devices on other devices, including the discovery and playback of content available to those devices, both from services such as traditional broadcast media and internet based services but also from the home network.

Status of This Document

This document is merely a public working draft of requirement(s) for potential W3C recommendations. It has no official standing of any kind and does not represent the support or consensus of any standards organization.

Introduction

There is an increasing amount of personal and streaming (broadcast) content that users would like to be able to access from any device in the home (personal computers, tablets, mobile phones, TVs and others).

Growing numbers of consumer electronic devices such as TVs and mobile phones can access internet based services as well as content originating from both home network devices and broadcast services. For example, many commercial video providers currently provide the ability for a user to access content stored on a home network device (e.g. a digital video recorder, DVR) or from a network connected set-top-box, STB. A home network content discovery and control protocol is used by the DVR and STB to provide this access, through a native user interface on the device.

The dominant scenario today is for a home network device to both discover and playback home network content. Examples of these devices may include a personal computer or a connected television. This is commonly referred to as a 2-Box model. An emerging scenario is for the content discovery and control to take place on a separate handheld device, such as a smartphone or tablet computer, and for the handheld device to instruct a content player, a connected television for instance, to playback content from a content server such as a DVR. This is referred to as the 3-Box model.

In all use cases, security mechanisms are made available to protect user privacy and content owners’ rights.

There is currently no formally standardized way to build a web application that is able to use content discovery and control protocols to get access to the content available on different devices in the home network. This document lists the design goals and requirements that future W3C recommendations need to address in order to cover such gap. This would enable content providers to deliver web applications to any conforming device in order to enhance and harmonize the user experience

Conformance

As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.

The key words must, must not, required, should, should not, recommended, may, and optional in this specification are to be interpreted as described in [RFC2119].

This specification only applies to one class of product: W3C Technical Reports. A number of specifications may be created to address the requirements enumerated in this document. In some cases the union of multiple parts of different specifications may be needed to address a single requirement. Nevertheless, this document speaks only of a conforming specification.

A conforming specification is one that addresses one or more requirements listed in this document. A conforming specification should attempt to address requirements marked with the keywords "should" or "may" unless there is a technically valid reason not to do so.

Terminology

programme (program)

A programme (program) comprises a single period of audio visual content. It is usually expected to be labeled in content directories or television programme guides as a single entity. This might include an episode of a television programme, a radio programme, or a movie.

device

For the purposes of this document, a device is any piece of equipment connected to the home network. Note that this is a generic term that includes also television definition below.

television

A device that presents programme content from a variety of sources - such as received via broadcast (cable, satellite, terrestrial), on-demand streaming services, or streamed from other devices in the home (e.g. PVR). Within the scope of this document, it is presumed that a television is connected to the home network.

service

For the purposes of this document, a service is any functionality available on a device, such as the ability to playback audio/video content, record content, print documents etc.

document

An individual resource that adheres to a certain content type, ranging from short static documents to long essays or reports with rich multimedia. For simplicity, terms such as shown, displayed, and visible might sometimes be used when referring to the way a document is rendered to the user. These terms are not meant to imply a visual medium; they must be considered to apply to other media in equivalent ways.

application

For the purposes of this document, the term "Application" refers to a collection of documents (not necessarily delivered over HTTP) which use server-side or client-side processing (e.g. JavaScript) to provide an "application-like" experience within a Web browser or other kinds of Web run-time. Applications may include locally executable elements of interactivity and persistent state. Note that locally stored applications like W3C Widgets are also covered by this definition.

home network

For the purposes of this document, the term "home network" refers to the networking infrastructure that facilitates Internet Protocol communications between devices within the home. This may range from a single legacy IPv4 subnet to multiple IPv4 subnets and dual stack or IPv6 environments and will typically (but not always) be connected to the Internet.

content item

For the purposes of this document, the term “content item” refers to metadata describing or more binary versions of media. The media described by an item may either be stored or streamed. A single content item may refer to multiple media binaries that represent different formats for the content being described.

channel item

For the purposes of this document, the term “channel item” refers to an “item” describing a streaming source. A channel item may contain metadata describing the channel source such as channel number, distribution network, etc. A channel may not be available on the home network, i.e. the “channel item” may refer to a channel that can only be locally tuned. In these cases “channel item” metadata will not provide network addressing information (URLs) to connect to the channel source.

epg (electronic programming guide) item

For the purposes of this document, the term “epg item” refers to an item that may be available with some defined time range. An “epg item” may contain metadata describing the channel source for the content and the time range the channel media will be available.

record schedule (item)

For the purposes of this document, the term “record schedule” refers to an item which contains metadata describing a request to a network recording service to record content. “Record schedule items” may include: simple time/duration requests, recording of a specific “epg item”, and recording dynamically selected “epg items” based metadata properties of desired content. “Record schedule items” additionally include metadata describing the status of the recording request.

record task (item)

For the purposes of this document, the term “record task item” refers to an item which contains metadata describing a pending or completed network recording operation. “Record task items” are created by a network recording service and allow network record service clients to control pending recording operations. Additionally, “record task items” include metadata describing the status of the recording request and the identity of “content items” that represent the results of network recording operations.

Requirements

This section enumerates the requirements that conforming specification(s) would need to address to enable discovery and control of devices and services in the home network by a user agent. These requirements are directly motivated by the #Use cases described in this document and are based on an interactive process of feedback from the discussions in the Home-Network Task Force of the Web & TV IG

General

Compatibility with widely deployed standards

Several home network protocols (e.g. UPnP, M-DNS/DNS-SD) are in popular use today for advertising and sharing content and services in a home network environment. A conforming specification shall set out requirements for an API that would enable interaction with those protocols.

Discovery and Advertising

Service Discovery

Conforming specifications should provide a means for applications to discover devices and applications in the home network which advertise services. Details of the advertising protocols are out of scope for this document and the type and number of supported discovery protocols is user agent dependent. Nevertheless conforming specifications should provide a means for application search for specific services and to be able to determine the means by which it should communicate with that service.

Application Discovery

Conforming specifications should provide a means for applications running in different user-agents to discover each other directly via the home network. Details of the advertising protocol are out of scope for this document.

Content Discovery

Conforming specifications should provide a means for applications to discover devices in the home network capable of serving content (content servers). In addition, conforming specifications should provide a mechanism to retrieve a list of the available content, provide additional information (metadata) about the content and to support negotiation of a supported content format between content servers and content players.

Content Advertisement

Conforming specifications should provide a means for application to expose content description to a service on the home network. Other devices or applications on the home network should be able to playback the content served by the application and retrieve additional metadata about the served content.

Services Advertisement

Conforming specifications should provide a means for application to expose and advertise services on the home network.

Media Identification

Conforming specifications should provide applications with a means to identify and thereby recognize a program that may be available from devices or services on the home network, when such an identifier is available. Such identifiers should be unique to a program, but common across multiple instance of the same program; and should be consistent across different services and devices in different home networks.

Control and Communication

Control of content players

Conforming specifications should enable applications to know what content that can be presented by a device or service on the home network and control the playback of that content presented, in such a way that the application does not have to handle:

issues of codec, container format, or transport protocol compatibility
differences in the mechanisms by which content is delivered to the rendering device (e.g. received from broadcast vs. streamed from a media server)

Control of content recorders

Conforming specifications should enable applications to control recording of content presented by devices or services within the home network, in such a way that the application does not have to handle:

issues of codec, container format, or transport protocol compatibility
differences in the mechanisms by which content is delivered to the rendering device (e.g. received from broadcast vs. streamed from a media server)

Playback of content

Conforming specifications should provide a means to playback content advertised by a home network content server.

Playback of live content

Conforming specifications should provide a means to playback live content advertised by a home network content server.

3 Box model

Conforming specifications should provide a means for controlling the playback of content from home network content servers to home network content players. Conforming specifications should support means to negotiate a supported content type format that is available on the content server and is suitable for being played on the content player.

Time-synchronization

User-agents should provide applications with a means to co-time the presentation of their own content (audio, video or other) with the timeline of a programme being played back on another device, including programmes being received via broadcast. Conforming specifications should provide applications with a means to be aware of the progress (time within the programme) of the playback of media content on a remote device/service, including programmes being received via broadcast

Accurate time-synchronization

Conforming specifications should support any protocols or optional protocol features that enable applications to be aware of the progress (time within the programme) of the playback of a programme on a remote device to within frame accuracy (1/25th or 1/30th second) or better. Conforming specifications should support the ability to determine an estimate of the level of accuracy with which timing information about media playback on a remove device/service is conveyed to applications.

Service Communication

Conforming specifications should provide a means for an application to exchange messages directly via the home network with services discovered in the home network.

Application communication

Conforming specifications should provide a means for applications to exchange messages directly via the home network with other applications running on a different user agent in the home network.

Content Protection

Conforming specifications should support the content protection mechanism for a content item used by a content server in order to playback that content item. Conforming specifications must provide a graceful failure model when a content protection mechanism is not supported.

Migration

Push migration

Conforming specifications should provide a means to transfer an application to a different user agent in the home network without requiring support of an application server.

Pull migration

Conforming specifications should support transfer an application to a different user agent in the home network without requiring support of an application server. The request is initiated by the target user agent.

Security and Privacy

Application trust level

Conforming specifications provide a means to define the trust level of an application. Determining the trust level may involve interaction with the user and the use of security mechanisms such as password, PIN etc.

Access to home network services

Conforming specifications should allow access to home network devices and services only to trusted applications

Prevent leaking of information

Conforming specifications should prevent unauthorized transfer of information outside the home network.

Use cases

This section is a non-exhaustive list of use cases that would be enabled by one (or more) specifications implementing the requirements listed above. Each use case is written according to the following template:

Ux. <TITLE>
Original Proposal: ISSUE-n

Description: 
* high level description/overview of the goals of the use case
* Schematic Illustration (devices involved, work flows, etc.) (Optional)

Motivation: 
* Explanation of benefit to ecosystem
* Why were you not able to use only existing standards to accomplish this?

Dependencies:
* other use cases, proposals or other ongoing standardization activities which this use case is dependent on or related to

Requirements:
* List of requirements implied by this Use Case.

U1. Service User Interface

Original Proposal: ISSUE-4

Description: An application as an interface to a service: the application provides a remote user interface for a device (light switch, HiFi volume control, radio station chooser, etc.) or a service on a device (remote control on the media player software on a computer).

Possible implementation:

the application discovers services with a discovery protocol, selects one service of a certain expected type, e.g. a media renderer service.
the application binds to it and provides a UI for it.
on user interaction, the application sends the messages to set the content url to play then play, pause, ... to the media renderer service, with appropriate parameters.

Motivation: There is no standard interface to discovery protocols in existing standards implemented in connected devices. There is no standard interface to service communication protocols in existing standards implemented in connected devices. What should be standardized is:

an interface to get a list of discovered services;
an interface to get the list of messages exposed by a discovered service;
an interface to send a message to a discovered service;
an interface to set a listener for messages from a discovered service.

Dependencies: In order to interface with a service, the application first needs to discover the service (or the device, then the service on the device).

Requirements:

Low Level	High Level
Determine list of discovered services	#Service Discovery
Determine list of messages exposed by a discovered service	#Service Discovery
Send a message to a discovered service	#Service Communication
Listen to messages (possibly unsolicited) from a discovered service.	#Service Communication

U2. Discovered content host

Original Proposal: ISSUE-5

Description: A document as host for discovered content: e.g. the document describe content provided by a local, discovered device or service.

Motivation: Rendering a media on another device than the one hosting is the basic step to enable more complex use cases.

Requirements:

Low Level	High Level
Support a document which displays discovered content	#Playback of content

U3. Service Migration

Original Proposal: ISSUE-7

Description: An application moving across HNTF user agents in a decentralized situation (local application, without a server).

A radio service is split into a radio application which resides typically on a HiFi, TV, personal music player or computer, and a radio controller application which resides on a device with intuitive interaction capability such as a phone, computer or tablet. The radio application is implemented as a document exposing a service interface on the network, and has a state that needs preserving if the document is migrated. The radio controller application is a pure interface, and does not have its own state.

The radio application does not have a visual interface. The radio application exposes a service of type JCDsRadioApp with seven possible messages:

setURI(radioStationURI, radioName),
play(),
stop(),
setVolume(n),
getURI(),
getRadioName(),
getPlayState(),
getVolume().

The radio application is running on a TV set.

The radio controller application has a visual interface, comprising:

a button to choose the radio station.
a play button.
a stop button.
a label for the current radio station name.
a volume bar to display the current volume.
two buttons to increase and decrease the volume.

The radio controller application is running on a smartphone. The radio controller application looks for a service called JCDsRadioApp by using the HNTF discovery mechanism, then establishes a connection with the above application. The radio controller application gets the radio name, volume, play state of the radio application and displays that information.

The stage is now set.

The user wants to move from the TV in the main room to the computer in her office:

In the HNTF user agent on the TV, the user requests to migrate the radio application.
The TV HNTF UA looks for other HNTF UAs on the network, by discovering service "HNTFUserAgentv1.0".
The result of the discovery is a list of HNTF UAs: the one on the phone, the one on the computer, possibly other ones.
The user selects the HNTF UA running on the computer.
The TV HNTF UA sends the message migrate(appURI, contextURI) to the computer HNTF UA.
The computer HNTF UA requests the appURI to get the radio application, then requests the contextURI, which contains all the information needed to place the radio application in the same state it was in on the TV.
The computer HNTF UA loads the radio application, then the context. Loading the context first sets the volume to half and the station to "Jazz".
Then the computer HNTF UA exposes a service of type JCDsRadioApp.
Then the smartphone HNTF UA understands that its connection to the radio application in the TV is down, so it starts again a discovery of service type JCDsRadioApp. It finds one on the computer HNTF UA, and sends a connection request.
The computer HNTF UA receives a connection request from the device mentioned in the migration context as formerly connected. It accepts the connection without requiring a user validation.

End of migration.

When moving the radio document, the following needs to be preserved:

the service needs to be restarted and exposed on the new device
the connection to the (same) radio controller needs to be reestablished
the radio needs to be playing the same station, with the same volume, etc.

Another possible example is a multi-user game, e.g. with lots of players sending location and activity information about their character/cars. Upon migration, the game state needs to be transferred to the new device, and the connections to other players need to be reestablished.

Motivation:

no way to discover a service from an application
no way to expose a service from an application
no way to move an application to another device and restart it while keeping an execution context

Dependencies: This use case depends on discovery. This is a refinement of ISSUE-15, where the migrated document uses a service rather than exposes a service.

Requirements:

Low Level	High Level
User-Agents support a service to transfer a running Application to different User-Agent.	#Push migration
Applications support the ability to save their current state and provide this information via a contextURI.	#Push migration
Applications support the ability to restore their state on a different User-Agent using state information provided by a contextURI.	#Push migration

U4. Service Distribution

Original Proposal: ISSUE-8

Description: An application spawning other applications on other devices and communicating with them: e.g. the TV set receives and renders an application implementing some voting; the application discovers multiple phones in the home, proposes to activate voting interfaces on each of the viewers' phones, communicates with the voting interfaces, collates votes and sends them back to the channel's voting service.

Possible implementation:

The TV receives a connected TV application for voting, which starts automatically.
The application exposes a service on the HN to collect votes and forward them to the TV channel.
The application discovers personal interactive devices (phones and tablets for our case) running the HNTF document UA which exposes a recognizable service.
The application sends a suggest message with a URL to a voting interface.
Willing users accept the suggested migration and their UA loads the voting interface.
The voting interface document loads and discovers and binds to the vote collecting service.
The user votes
The voting interface document sends a vote message to the vote collecting service.
The vote collecting service exposed by the TV application receives the vote message and forwards them for a certain voting period.
At the end of the show (or of the voting period), the application is killed by the TV system, so the voting collection service disappears.
Seeing the voting collection service disappear, the voting interfaces may shut down.

This use case does not require new technology, but reuses technology required by other use cases. Technologies required by other use cases are:

the ability to discover
the ability to expose a service

This use case requires:

the HNTF UAs to implement a service which could be called "HNTF service", so that documents can discover other "HNTF UAs"
the HNTF service to include a "suggest" message telling the UA to please render a document given by URL
maybe the HNTF service to include a message allowing a document to determine its capabilities, to be able to segregate between devices appropriate for voting or not.

A few sketches for clarification:

Motivation:

This is the generic version of a crucial "second screen" usage scenario.

As there are more devices in the home, some generic and some task-specific, and with varying capabilities (including different UI methods), there is a growing need to spread an application across different devices to achieve service distribution. But the service usually "enters" the home network through one particular device. The service running entirely on the initial device, as part of other use cases, can discover its environment and determine that other devices could meaningfully contribute to the quality of experience. From then on, the service needs to send part of "itself" to other devices.

Dependencies: Depends on Service UI, Document Responding to Requests, Document Exposing a Service, Document Discovering a Service

Requirements:

Low Level	High Level
A document exposes an interface to a service.	#Service Discovery, #Service Communication

U5. 3-Box model

Original Proposal: ISSUE-10

Description: A document as an interface coordinating action between other services. In the most obvious example, a document discovers media content sources and media players. The document allows the user to select a source and a player, then control playback (Play, pause, rewind, etc.) of the content to the player.

Motivation: Assets offered by one service (on one device) can be consumed by another service (on another device) and controlled by a separate controlling document (on a third device). Vendors can offer control services to manage services across the whole home.

Dependencies: This is similar to Service User Interface (ISSUE-4) but it explicitly manages services between independent devices, multiple simultaneous connections.

Requirements:

Low Level	High Level
An application may invoke services to control devices on other User-Agents separate from the User-Agent issuing the service requests.	#3 Box model

U6. Application Exposing a Service

Original Proposal: ISSUE-12

Description: An application exposes a service on the home network. In order to allow this with some technologies, it may be necessary for the HNTF user agent to advertise itself on the HN as a device. For example, an application rendered on a connected TV has access to the connected TV API for EPG information. Other devices on the HN do not have access to this information. The application implements a service exposing the EPG information and makes it discoverable by other devices.

Possible implementation:

the application A is a TV-broadcast-related application, which is allowed to call the function returning the description of the next program.
the application A exposes a service with one message getNextProgramDescription.
application B on another device discovers the service (not knowing that it is an application, it is just another service to application B).

Neither application A or B know the actual nature of the other. They may have an IP address, and they share a service type.

Motivation: Allowing sharing resources other than content, such as a capability (a large screen, a sensor) or an "authorization" (permission to use a restricted API).

Dependencies: This is a prerequisite to applications discovering each other without a "middle man": one of the two applications exposes a service that the other application may discover.

Requirements:

Low Level	High Level
An application can cause a User-Agent to act as a device which exposes services.	#Services Advertisement

U7. Application Discovering a Service

Original Proposal: ISSUE-14

Description: An application discovers a service. Discovery means that after discovery, the application has:

an identification of a device (maybe) and a service;
a means to know the messages that can be exchanged, possibly with their parameters;
a means to send messages to the service, if the service may receive messages;
a means to receive messages from the service, if the service may send messages.

Motivation: Very basic use case, on which all other use cases depend. There is no existing way for an application to discover services and communicate with them, if the author did not have the address of the service.

Dependencies: None

Requirements:

Low Level	High Level
An application can determine allowable messages and parameters provided by a service.	#Service Discovery
An application can send messages to a service.	#Service Communication
An application can receive messages from a service	#Service Communication

U8. Application Push-Migration

Original Proposal: ISSUE-15

Description: An application moving across devices in a decentralized situation (local application, without a server).

I start using my phone for the user interface to a device and then my tablet becomes available, so I move the interface application to the tablet. The application in question is an HNTF application, which means it has discovered services and has connections to some services and a history of execution. Just restarting the application on another device would lose the execution state, the existing connections, etc...

I start using my phone as a UI of a service on my TV, e.g. EPG, but the screen is small.
My tablet finally becomes available.
I request the migration of the current application: this is a user agent function.
The user agent discovers its peers on the network, i.e. other user agents capable of running the current document.
The user agent proposes to me a list of suitable user agents, including the one running on the tablet.
I choose the user agent running on the tablet.
The UA on the phone binds to the UA on the tablet, and sends a migration message with the URL of the document, and the URL of the execution state.
The UA on the tablet receives the migration message, fetches the application and the execution state and loads both.
The UA on the tablet signals to the UA on the phone that migration is complete, by replying "success" to the migration message.
The UA on the phone stops rendering the migrated application.
I pick up the tablet and go on with the service at the point I left it on the phone.

Motivation: There is no way with current standards and in the absence of a central server to achieve the saving and transfer of the execution state of a widget, so there is no way to start an application on a device, switch devices and restart the application on the new device, keeping the exact same execution state.

Dependencies: This use case depends on discovery.

Requirements:

Low Level	High Level
User-Agents support a service to transfer a runningapplication from another User-Agent to itself.	#Push migration
Applications support the ability to save their current state and provide this information via a contextURI.	#Push migration

U9. Application Pull-Migration

Original Proposal: ISSUE-25

Description: An application moving across devices in a decentralized situation (local application, without a server), the migration being initiated on the target device.

I start using my phone for the user interface to a device and then my tablet becomes available, so I move the interface application to the tablet by interacting with the tablet. The application in question is an HNTF application, which means it has discovered services and has connections to some services and a history of execution. Just restarting the application on another device would lose the execution state, the existing connections, etc...

I start using my phone as a UI of a service on my TV, e.g. EPG, but the screen is small.
My tablet finally becomes available.
I drop my phone.
I take the tablet, start the HNTFUA and request a list of discoverable HNTFUAs on the tablet: this is a user agent function.
I start the HNTFUA of the phone and request from it a list of pullable applications and I choose one in the list.
The UA on the tablet sends a pull-migration message with the identifier of the wanted application chosen from the above list.
The UA on the tablet receives the response to the pull-migration message, i.e. the url of the application and the url of the execution state, fetches the application and the execution state and loads both.
The UA on the tablet signals to the UA on the phone that migration is complete.
The UA on the phone stops rendering the migrated application.

Motivation: Same as ISSUE-15, with the additional twist of triggering the migration from the target device rather than the source device.

Requirements:

Low Level	High Level
User-Agents support a service to notify a running application of a request to transfer itself to another User-Agent.	#Pull migration
Applications support the ability to save their current state and provide this information via a contextURI.	#Pull migration

U10. Media Identification

Original Proposal: ISSUE-19

Description: Where available, applications should be able to determine and refer to programmes using a unique and consistent identifier. If the same programme is available from multiple devices or services, the programme should have the same identifier associated with it across all devices or services.

Scenario: An application, wants to present more information associated with a particular programme that the user is currently viewing on their television, without the television acting as an intermediary that serves that additional information.

The application discovers an appropriate service
The application sends a request to the service to return an identifier for the programme currently playing on the television
The application discovers a service that can return metadata for an identified programme
The application sends a request to the service to retrieve a unique identifier for the programme that was assigned by the original broadcaster, content publisher or content creator
The application contacts an internet service that can resolve identifiers to additional programme metadata
The internet service returns additional metadata for the programme

Motivation:

Allows anyone (not just the original content creator or distributor) to create an application that can associate programmes with metadata and web based content; without that metadata or content having to be delivered, or referenced by data served from services on the television or set top box.
Recommendations to be fed to any groups defining home network protocols to add global content identifier support where not present
Areas for standardization: (Application specific API/protocols)
- Requirement for any high level media playback API abstractions (see issue-20) should include the ability to:
  - expose such identifiers as part of programme metadata
  - search for programmes using such identifiers
  - request playback of a programme using such an identifier
- This use case does not suggest mandating a particular identifier format.

Dependencies: TV Querying and Control use case (for outlined user scenario)

Requirements:

Low Level	High Level
Provide consistent program identifiers which are meaningful across multiple devices.	#Media Identification
Provide services to provide metadata corresponding to program identifiers.	#Media Identification

U11. TV Control

Original Proposal: ISSUE-20

Description: An API, or service, for simplified control of key functionality of television devices, including those with integrated broadcast receivers and limited or no media streaming capabilities. The application defines its own user interface, independent of the user interface of the television device.

Application developers would benefit from a simplified API or services that provide:

A single unified directory of programmes that a given rendering device (e.g. television display) is able to obtain and play. The application does not have to check, for itself, across multiple sources of content and for any issues of codec, container format, and transport protocol compatibility.
A single simple method for requesting a programme or channel to be presented by a rendering device, irrespective of the above factors.

It is also desirable to be able to control other common aspects of television functionality through the same API.

The target of the API would be a processing engine on the home network. For a Javascript API this processing engine could be incorporated into the user agent. Alternatively, it could be implemented by any other device on the home network, including the television device itself. For the latter case, the user agent must be able to discover and communicate with the processing engine in order to execute the functions of the API.

The processing engine may, when appropriate, utilize existing home networking protocols to perform its function. For example: it may use UPnP AV services to discover and stream content.

Scenario 1: Basic programme guide application: The application uses the API to perform the following tasks:

The application sends a request to discover television devices on the home network
The application sends a request to ask for the identity of the programme currently being presented on the television, and the source (e.g. television channel)
The application sends a request to retrieve basic programme metadata (title, description, start and end times) and displays this for the user
The application sends a request to retrieve a list of sources (e.g. channels) of programmes that can be displayed on the television
The application sends a request to retrieve a list of programmes, and their basic metadata for those sources for presentation to the user
The application sends a request to cause a given channel or programme be played on the television.

Extension to scenario 1: Control of other television functions:

The application can send a request to change the volume of the television
The application can enable or disable accessibility features such as subtitles or audio-description
The application can request to seek (jump to a different time point) within the programme being played

Scenario 2: A web site with material supporting a particular programme wants to be able to offer to play the programme on a television if it is available. A possible interaction (via the API):

The application sends a request to discover television devices on the home network
The application makes a request to query if a specific programme is available to the television. If it is available, the application offers the choice to the user to display it.
If the user accepts, the application requests that programme be played on the television.

Each of the interactions described should be ideally achievable a single call to an API or service method.

Motivation:

Simplify creation of applications that will work with a range of devices and variety of means by which a given programme might played on the television.
Standardization could facilitate a new ecosystem of interfaces.
Existing standards for home network communication are not commonly available from the browser context and expose functionality at a lower level.

What could be standardized:
- Type of use case: Application specific services / protocols / APIs
- A Javascript API
- Service(s) for television type devices on the home network

Dependencies:

Requires Application discovering a service (issue 14)
Scenario 2 requires 'Media Identification (issue 19)

Requirements:

Low Level	High Level
Provide a directory of programs a rendering device can play.	#Control of content players
Provide a service to select a program on a rendering device to play.	#Control of content players
Provide services to control other aspects of a rendering device such as Seek, Volume, etc.	#Control of content players

U12. Time Synchronization

Original Proposal: ISSUE-21

Description: Applications should be able to time synchronize their activity with the time-line of a programme being played on a television or other home media rendering device.

Application developers will benefit from having a single simple API that provides a unified interface to this functionality, irrespective of the means by which the programme is being delivered to, or played by, the television device.

The application is running on a different device from where the programme is being played. This could be a laptop, mobile phone, tablet, etc. Both devices are on the same home network. The device playing the programme may be a television or other media rendering device. The programme being played may be being obtained by a variety of different means:

from a live broadcast via an integrated tuner
playback from local storage (e.g. a Personal Video Recorder device)
streaming from another device on the home network
streaming from an on-demand internet service

The target of the API would be a processing engine on the home network. For a Javascript API this processing engine could be incorporated into the user agent. Alternatively, it could be implemented by any other device on the home network, including the television device itself. For the latter case, the user agent must be able to discover and communicate with the processing engine in order to execute the functions of the API.

The processing engine may, when appropriate, utilize existing home networking protocols to perform its function. For example: it may use UPnP AV services to query and set the playback time index.

An API meeting these requirements may enable an application to:

request the current playback time index of the programme being played on a given device
command the device to jump to a different playback time index
request notification when programme playback reaches a particular moment or time interval in the programme timeline.

Scenario: An application wants to present a slideshow of content relating to the programme the user is watching on their television. As the programme progresses, the slideshow automatically moves onto the correct slide for any given point in the programme. If it is possible to seek within the programme timeline (because the programme is not being watched from a live broadcast), the user can use the application to jump to a different point in the presentation and the television will also seek to the corresponding segment of the programme.

Motivation:

Simplify creation of applications that will work with a range of devices and variety of means by which a given programme might played on the television.
Enable the creation of added-value viewing experience for the user where additional information is available in a timely fashion without interrupting the TV viewing experience or increasing on screen 'clutter'.
What could be standardized:
- Type of use case: Application specific services / protocols / APIs
- A Javascript API
- Services or protocols implemented by devices on the home network

Dependencies:

Application discovering a service (issue 14)
Media Identification (issue 19) - required for an application to know which content to provide for the programme being watched in the described scenario
This use case could be considered an extension of the TV Control (issue 20) use case

Requirements:

Low Level	High Level
Provide ability to generate notification to application when a particular playback time is reached.	#Time-synchronization
Provide the ability to position playback to a specified time-index.	#Time-synchronization
Obtain the current playback time-index.	#Time-synchronization

U13. Lip Sync Time Synchronization

Original Proposal: ISSUE-22

Description:

This use case is a specialization of the use case: Time Synchronisation (issue-21)

An application should be able to synchronize the presentation of its own content with a high degree of accuracy to the timeline of a programme being played on a television or other home media rendering device. The level of accuracy should be sufficient to achieve 'lip-sync' between audio and video (typically within 1 to 2 video frames or tens of milliseconds).

In addition to the functions described in the Time Synchronisation use case; the API used by the application should be able to:

report an estimate of the timing synchronism accuracy that is achievable
provide synchronization to within lip-sync accuracy if all devices involved support it

The processing engine that implements the API may utilize existing home networking protocols to achieve the required accuracy. A likely necessary component will be the synchronization of clocks between the television or media rendering device and the device on which the application is running. Possible protocols that might be used to achieve this include: Precision Time Protocol (IEEE 1588-2002) IEEE1588, IEEE 802.11AS, UPnP Precision Time Synchronization UPNPAV4.

Applications will benefit if the API presents a simple unified set of functions such that the application does not need know which protocols are being used.

Scenario: An application plays alternative audio that is time synchronized to a programme that the user is watching. The synchronization is sufficiently accurate to maintain 'lip-sync' between the alternative audio and the programme video. The alternative audio may be delivered to the application independently of the television device - such as via a direct stream from a broadcaster's internet based server.

Motivation:

Simplify creation of applications requiring lip-sync accurate time synchronization that will work with a range of devices and variety of means by which a given programme might be played on the television.
What could be standardized:
- Type of use case: Application specific services / protocols / APIs
- A Javascript API
- Services or protocols implemented by devices on the home network

Dependencies:

This use case is a specialization of the Time Synchronization use case

Requirements:

Low Level	High Level
Determine the time synchronization capabilities of home-network devices.	#Accurate time-synchronization
Provide the capability for applications to synchronize with playback content to approximately frame resolution.	#Accurate time-synchronization
Provide the capability to precisely synchronize multiple content streams	#Accurate time-synchronization

U14: Local Link of Web Applications

Original Proposal: ISSUE-24

Description:

This use case is about the bi-directional communication between web applications via the local IP network e.g. Home IP network
Assumptions:
- User devices are connected to the same local IP network
- The applications on user devices know each other and can exchange messages for communications
- Note that the applications may not necessarily be provided by the same distributor or service provider
- A user uses the IR remote controller to operate the HTML5 browser on the TV device (10 feet UI)
- A user uses the touch screen to operate the HTML5 browser on the tablet device or smart phone (Touch UI)
- Note that the application may not necessarily to be a web application as long as it communicates by the same network protocols as the user-agent does.
User Scenario A - Single User
- Bob operates the TV device to watch movies on an online video service
- Bob also operates the tablet device to access a social network service about movies
- Bob views web pages of the social network service and find a favorite movie
- Bob clicks the “your device” button on the web page of the social network service
- The application on the TV device show the web page of the video service about the movie, then Bob starts to watch the movie
User Scenario B - Multiple Users
- Bob operates his smart phone to use an application for a picture service
- Alice operates her smart phone to access another picture application
- Bob selects his favorite picture and clicks the “Sharing photo with other device” on the application
- The application on Bob’s device shows the device name list and he selects the Alice’s device
- The picture file is transferred from the Bob’s device to the Alice’s device while each application shows the progress bar of file transfer
- After the transfer is completed, the application on the Alice’s device shows the Bob’s favorite picture
User Scenario: Interactive quiz
- Alice and Bob operates the TV device to watch a quiz programme, from live broadcast
- Alice and Bob also operate their smart phones to access a quiz application
- Note: The smart phone quiz application knows that the quiz programme is being watched on the TV and establishes communication with an interactive application on the TV that is active during the quiz programme)
- The TV application overlays on-screen information informing Alice and Bob that they are now taking part in the quiz.
- As a question is asked in the quiz programme, the interactive TV application instructs Alice and Bob's smart phone quiz applications to ask Alice and Bob the same questions. Alice and Bob enter answers to the questions using their smart phones, which are relayed back to the interactive TV application.
- Between quiz rounds, Alice and Bob's scores are displayed on their smart phones and overlayed on screen by the TV interactive application. Scores are compared to those of the contestants featuring in the programme.
- Note: Regarding need/justification: Using local-link communication to an interactive application provided as part of a broadcast stream provides a way around scalability concerns if participation is substantial for a particularly popular programme.
Use case (System Interactions)
- An User-Agent discover other Users agent on the same IP-sub network
- Via API, the User-Agent provides the discovered application list which contains the applications which are running on other user-agent, e.g. application id, the URL for message exchange, physical device name(for user selection)
- The application establishes the bi-directional communication for message exchanges with the application on the other user-agent via API
Implementation Examples
- For the local discovery, the HTML5 browser implements UPnP Discovery to provide the generic discovery API to find applications on the same IP sub-network
- For the message exchange, the HTML5 browser implements the extended WebSocket protocol and APIs for bi-directional communications
- The HTML5 browser should implement some mechanism to confirm the user consensus to permit the device discovery and message exchanges for the applications
Additional comments:
- The above user scenarios are examples. As long as the message exchange API is service/application agnostic, many rich user experience can be realized as the open web platform does

Motivation

The “Local Link” enables applications to exchange message, e.g. String, Blob, via the local network directly. Without the “local Link”, the applications which are running on different devices can in-directly communicate via a service on the Internet. But, the “Local Link” has the following benefits
- Adhoc: Without any syndication of services, the applications provided by different services can communicate each other's in ad-hoc. Also, all devices are not required to be bound to the same service. This is a kind of service mash-up using multiple devices
- Efficient: A large size of files can be transferred between devices via high bandwidth local network connection.
- Off-line: Even if the Internet access is not available, the applications can still communicate
- Use-centric: Users can easily recognize which devices communicate. Only devices on the same network are listed and the communications of different user device are based on the user consent
There are some standards of local discovery, e.g. UPnP, Bonjour, WS-Discovery, but there is no standard of the JavaScript API which enables communications between Web applications in the manner of application/service agnostic
What needs to be standardized:
- Type of use case: Generic APIs
  - General service discovery and messaging APIs should be standardized
  - Network protocols for the APIs should be specified

Dependencies:

Any other use cases related to the local IP network discovery since the discovery is also application/service agnostic

Requirements:

Low Level	High Level
Support applications communicating with each-other via ad-hoc messages.	#Application communication, #Application Discovery

U15. Home Network Enabled User-Agent - Network Media Player

Original Proposal: ISSUE-26

Description:

Enable a User-Agent to act as a Home Network Media Player

Scenarios

The User-Agent acts as a Home Network Media Player device. For example:

List available Home Network Media Server devices. Steps
List available content on a Home Network Media Server. Steps
List available content on a Home Network Media Server matching specified metadata criterion. Steps
Determine capabilities of User-Agent playback facilities provided to Home Network Media Player. Ex: supported content formats, number of streams, supported resolution(s).
Play content from a Home Network Media Server. Steps
View EPG data from a Home Network Media Server. Steps
Tune and play live programs from a Home Network Media Server capable of streaming live content. Steps
Select and play recorded content from a Home Network Recording device. Steps
Select and play time-shifted content. Steps

Steps:

The User-Agent acts as a Home Network Media Player device.
1. List available Home Network Media Server devices.
  1. Web page issues a method to discover home network devices
  2. User-Agent gets an event indicating the list is ready.
  3. User-Agent retrieves a list of handles representing the discovered devices
  4. Web page issues a method to get information about each discovered device
  5. Web page retains handles for devices of the appropriate type.
  6. Web page displays available devices to end-user.
  7. End-User selects one of the displayed devices.
  8. Web page issues method to access the device.
  9. User-Agent responds that a password is required.
  10. Web page obtains the password from the end-user (or secure password store) and issues method to provide the password to the User-Agent
  11. Web page can now access the device.
2. List available content on a Home Network Media Server.
  1. Web page formats an action request to browse the Home Network device metadata containers for EPG Items matching desired channel/time ranges.
  2. Web page issues method to send action request to selected Home Network device
  3. User-Agent sends request to Home Network Media Server device.
  4. User-Agent generates event when Home Network device responds (or times-out)
  5. Web page gets response (XML document) from Home Network Media Server device.
  6. Web page displays containers and items as indicated in XML document
    This depends on the data representation of Home Network Media Server content metadata.
  7. Web page displays the information in the Home Media Server response.
3. List available content on a Home Network Media Server matching specified metadata criterion.
  1. Web page formats an action request to determine searchable metadata on the selected Home Network device metadata store.
  2. Web page issues method to send action request to selected Home Network device.
  3. Web page formats an action request to search the Home Network device metadata store.
  4. Web page issues method to send action request to selected Home Network device
  5. User-Agent sends request to Home Network Media Server device.
  6. User-Agent generates event when Home Network device responds (or times-out)
  7. Web page gets response (XML document) from Home Network Media Server device.
  8. Web page displays containers and items as indicated in XML document
4. Play content from a Home Network Media Server.
  1. End-user select displayed Home Network Media Server item for playback.
  2. Web page formats an action request to browse metadata for selected item.
  3. User-Agent sends request to Home Network Media Server device.
  4. User-Agent generates event when Home Network device responds (or times-out)
  5. Web page gets response (XML document) from Home Network Media Server device.
  6. Web page selects media binary format that is compatible with User-Agent.
  7. Web page displays Media Player using HTML5 <video> element.
  8. Web page transfers item URL to <video> element @src (or creates <source> element as child-element)
5. View EPG data from a Home Network Media Server.
  1. Web page formats an action request to determine root container(s) for EPG data on the selected Home Network device metadata store.
  2. Web page issues method to send action request to selected Home Network device.
  3. Web page formats an action request to search the Home Network device metadata store.
  4. Web page issues method to send action request to selected Home Network device
  5. User-Agent sends request to Home Network Media Server device.
  6. User-Agent generates event when Home Network device responds (or times-out)
  7. Web page gets response (XML document) from Home Network Media Server device.
  8. Web page displays EPG containers and EPG items as indicated in XML document
6. Tune and play live programs from a Home Network Media Server capable of streaming live content.
  1. Web page formats an action request to determine root container(s) for Channel Lineup metadata on the selected Home Network device.
    Notes:
    - Multiple Channel Lineup containers may represent either an aggregate of multiple channel sources (Broadcast, Cable, Satellite) or selected channel sources (“Favorite Channels” for instance).
    - Channel Lineup containers contain Channel Items which are metadata bundles of channel related properties. Some example of channel properties are: Channel Name, Channel Number, Channel Icon, Channel Preview Stream.
    - A Channel Item contains metadata describing one or more URLs which stream the channel's binary content. Access to one of these URL implies any tuning operations necessary to make the requested channel source available.
  2. Web page formats a channel listing based on Channel Items found. In addition the Web page may construct a subset of available Channel Items based previously stored end-user selections.
  3. Web page may support a Channel Up/Down notion based on an ordering of channels selected by the end-user of by using metadata properties of the Channel Items (Channel Number, Channel Name, etc.)
  4. Web page will typically offer the end-user the ability to select a channel for playback. The Web page will generate a <video> element referencing the channel's streaming URL provided by the Channel Item metadata.
7. Select and play recorded content from a Home Network Recording device.
  1. Web page formats an action request to determine root container(s) for Recorded Content metadata on the selected Home Network device.
    Notes:
    - Recorded Content Items are similar to stored Content Items. However, Recorded Content Items may contain metadata containing the ID of the Recording Service request that resulted in the creation of this item.
  2. Web page display information about recorded content item.
  3. Web page offers end-user the ability to play recorded item.
  4. Web page offers end-user the ability to manipulate the Recording request which created the item.
8. Select and play Time-Shift recorded content from a Home Network Recording device.
  Notes:
  - Time-shift recording is an important variant of Network Recording. For Time-Shift recording no explicit request is required to record content; instead, the recording device automatically records the currently tuned channel (or favorite channel) in a circular buffer. Recorded content may be organized as one of more blocks of continuous content or as a list of separate captured programs.
  - Time-shift recording generates temporary Content Items (bundles of content metadata). These content items are typically pointed to by the Channel Item that the content was captured from. For Time-shift Content Items some additional metadata properties are defined to indicate that the recording of an item may not be complete, and/or that a Content Item may represent a partial recording the original content.
  1. Web page access Channel Items (as described in prior use-case)
  2. Web page detects Channel Item lists to one or more Content Items as preserved content.
  3. Web page displays preserved Content Items which is available for the channel.
  4. Web page offers user ability to play preserved content by accessing URL in preserved Content Item.
  5. Web page offers user ability to convert preserved content to permanently recorded content by offering preserved Content Item(s) to the Network Recording service.

Motivation:

Existing Home Network interfaces do not (by scope/tradition) standardize user interface control. However, the combination of HTML markup and Javascript allow both access to Home Network devices functions and access to very capable user interface resources.
Why were you not able to use only existing standards to accomplish this?
- HTML User-Agents do not provided the abstractions necessary for Javascript to issue discover and issue commands to home network devices.
- Basic access methods to home network devices need to be standardized so that developers can implement pages to control Home Network devices using Javascript/HTML.
What might you suggest could be standardized?
- Provide generic access methods to enable a User-Agent to discover and command home network devices.
- Provide a means for applications to control some of the parameters that may need to be exposed to support well-established home network protocols. A more detailed analysis is needed to identify such parameters and a way to specify them in a transport agnostic way
- Provide access and privacy controls for untrusted pages accessing home network devices.
- Allow pages to discover functional differences (optional functionality) that may be implemented in some but not all Home Network ecosystem devices.

Dependencies:

This use-case provides the basic framework for generic access to home network devices. It is "agnostic" towards the underlying functionality of the devices being controlled.

Requirements:

Low Level	High Level
Application can obtain and display descriptive metadata for playback content.	#Service Discovery, #Content Discovery
Application can obtain and display other descriptive metadata such as Electronic Programming Guides.	#Service Discovery, #Content Discovery
Application can select content binary formats which match the User-Agent’s supported playback formats.	#Playback of content
Application can play content from Home Network stored content sources.	#Playback of content
Applications can play live content from Home Network streaming sources.	#Playback of live content
Applications can play content from Home Network recording sources.	#Playback of content

U16. Home Network Enabled User-Agent - Network Media Server

Original Proposal: ISSUE-27

Description:

Enable a User-Agent to act as a Home Network Media Server

Scenarios

The User-Agent acts as a Home Network Media Server device. For example:
- Create a discoverable Home Network Media Server device. Steps
- List available content that the User-Agent chooses to make available on the Home Network Steps
- Provide descriptive metadata (title, description) for this content. Steps
- Provide multiple binary formats for this content. Steps
- Provide functionality to allow other User-Agents to search for content matching specified metadata criteria. Steps
- Provide capability for Home Network Media Server to notify clients when list of available content changes. Steps

Steps:

The User-Agent acts as a Home Network Media Server device. For example:
1. Create a "discoverable" Media Server device on the Home Network.
  1. Web page provides User-Agent descriptive metadata for the device to be created.
  2. Web page provides User-Agent access to programs to service requests made to Home Network Media Server.
  3. Web page requests User-Agent to make device visible on Home Network
2. List available content that the User-Agent chooses to make available on the Home Network
  1. Created Media Server device receives request to list Media Server content.
  2. User-Agent invokes previously registered program (php?) to service request
  3. User-Agent forwards results of registered program to requesting end-point providing identifiers for (Content Items) describing available content.
3. Provide descriptive metadata (title, description) for this content.
  1. Created Media Server device receives a request to provide metadata for a Content Item
  2. User-Agent invokes a previously registered program (php?) to provide metadata corresponding to a Content Item
  3. User-Agent forwards results of registered program to requesting end-point.
4. Provide multiple binary formats for this content.
  1. Content Items contain metadata providing one or more alternative formats for content represented by a content item. This metadata consists of a URL which serves the content item and additional descriptive metadata for the content binary served by the corresponding URL.
5. Provide functionality to allow other User-Agents to search for content matching specified metadata criteria.
  1. Created Media Server device receives a request to provide Content Item identifiers whose metadata matches criteria provided by the end-point.
  2. User-Agent invokes a previously registered program (php?) to provide identifiers of Content Items which match the search criteria provided.
  3. User-Agent forwards results of registered program to requesting end-point.
6. Notify User-Agents of changes to available content.
  1. User-Agent receives request from end-point(s) to report changes to available content.
  2. User-Agent receives notification that available content has changed
  3. User-Agent forwards notification to end-point(s) which have registered for this notification

Motivation:

Existing Home Network interfaces do not (by scope/tradition) standardize user interface control. However, the combination of HTML markup and Javascript allow both access to Home Network devices functions and access to very capable user interface resources.
Why were you not able to use only existing standards to accomplish this?
- HTML User-Agents do not provided the abstractions necessary for Javascript to issue discover and issue commands to home network devices.
- Basic access methods to home network devices need to be standardized so that developers can implement pages to control Home Network devices using Javascript/HTML.
What might you suggest could be standardized?
- Provide generic access methods to enable a User-Agent to discover and command home network devices.
- Provide a mean for applications to control some of the parameters that may need to be exposed to support well-established home network protocols. A more detailed analysis is needed to identify such parameters and a way to specify them in a transport agnostic way
- Provide access and privacy controls for untrusted pages accessing home network devices.
- Allow pages to discover functional differences (optional functionality) that may be implemented in some but not all Home Network ecosystem devices.

Dependencies:

This use-case provides the basic framework for generic access to home network devices. It is "agnostic" towards the underlying functionality of the devices being controlled.

Requirements:

Low Level	High Level
Applications can cause a User-Agent to act as a Home Network media server.	#Content Advertisement
Applications support services which provide metadata describing available content.	#Content Advertisement
Application supports services which allow searching for content metadata matching specified criteria.	#Content Advertisement
Application supports services which notify clients of changes in available content or content metadata.	#Content Advertisement
Content descriptions support offering content in multiple binary formats.	#Content Advertisement

U17. Home Network Enabled User-Agent

Original Proposal: ISSUE-28

Description:

Enable a User-Agent to control other media oriented Home Network Devices (3-box model)

Scenarios

The User-Agent commands other media oriented home network devices. For example:
- Determine the playback capabilities of a Home Network Media Rendering device. For example: supported content formats, number of streams, supported resolution(s)
- Direct a Home Network Media Rendering device to play compatible content stored on a Home Network Media Server.
- Direct a Home Network Media Rendering device to play live content from a Home Network Media Server capable of streaming content.
- Control Home Network Media Renderer attributes: Brightness, Contrast, Volume, etc.
- Control Home Network Media Renderer multiplex stream selection: Closed Captioning, Language, Subtitling and Camera Angles.

Steps:

The User-Agent commands other media oriented home network devices. For example:
1. Direct a Home Network Media Rendering device to play content stored from a Home Network Media Server.
  1. Web page requests User-Agent discover Media Rendering devices on Home Network
  2. Web page allows end-user to select a Media Rendering device
  3. Web page issues request to Media Rendering device to determine content formats it supports
  4. Web page requests User-Agent discover Media Server devices on Home Network
  5. Web page allows end-user to select a Media Server device
  6. Web page obtains list of Content Items available on Media Server
  7. Web page removes items from list which do not have formats compatible with selected Media Renderer device
  8. Web page allows end-user to select from remaining Content Items.
  9. Web page selects Content Item URL that provides "best" playback experience on Media Renderer.
  10. Web page sends selected Content Item URL to Media Renderer device
  11. Web page sends command to Media Renderer device to begin playback
2. Direct a Home Network Media Rendering device to play live content from a Home Network Media Server capable of streaming content.
  1. Note: Steps are similar to above use-case. However, Media Rendering device needs to modify buffer sizes to account for content being served at playback rate.
3. Control Home Network Media Renderer attributes: Brightness, Contrast, Volume, etc.
  1. Web page issues request to Media Renderer device to get default settings
  2. Web page issues request to Media Renderer device to modify default settings.
  3. Web page issues request to Media Renderer device to modify settings for current Content Item (if any).
4. Control Home Network Media Renderer multiplex stream selection: Closed Captioning, Language, Subtitling and Camera Angles.
  1. Note: Steps are followed to the point where Media Render is provided the Content Item URL.
  2. Web page issues request to Media Renderer to determine what playback attributes can be modified for the selected Content Item.
  3. Web page may request to Media Renderer device to select desired playback settings for the selected Content Item if user wishes to override default settings.

Motivation:

Existing Home Network interfaces do not (by scope/tradition) standardize user interface control. However, the combination of HTML markup and Javascript allow both access to Home Network devices functions and access to very capable user interface resources.
Why were you not able to use only existing standards to accomplish this?
- HTML User-Agents do not provided the abstractions necessary for Javascript to issue discover and issue commands to home network devices.
- Basic access methods to home network devices need to be standardized so that developers can implement pages to control Home Network devices using Javascript/HTML.
What might you suggest could be standardized?
- Provide generic access methods to enable a User-Agent to discover and command home network devices.
- Provide a mean for applications to control some of the parameters that may need to be exposed to support well-established home network protocols. A more detailed analysis is needed to identify such parameters and a way to specify them in a transport agnostic way
- Provide access and privacy controls for untrusted pages accessing home network devices.
- Allow pages to discover functional differences (optional functionality) that may be implemented in some but not all Home Network ecosystem devices.

Dependencies:

This use-case provides the basic framework for generic access to home network devices. It is "agnostic" towards the underlying functionality of the devices being controlled.

Requirements:

Low Level	High Level
Application allows a User-Agent to discover and control media playback on other Home-Network Devices.	#3 Box model, #Control of content players

U18. Home Network Enabled User-Agent - Network Record Controller

Original Proposal: ISSUE-29

Description:

Enable a User-Agent to control a Home Network Recording Device

Scenarios

The User-Agent controls a Home Network Recording device. For example:
- Request the recording device capture content based on simple time/date criteria.
- Request the recording device capture content based recurring time/date criteria.
- Request the recording device capture content based on an EPG description.
- Request the recording device capture content based on a metadata description.
- Provide controls to suspend/restart/cancel planned recording activities.
- Provide status information for current/future recording activities.
- Provide access to recorded program descriptions and content.

Steps:

The User-Agent controls a Home Network Recording device. For example:
1. Request the recording device capture content based on simple time/date criteria.
  1. Web page issues a request to determine the capabilities of the Network Recording device.
  2. Web page creates a XML document describing a simple record schedule.
  3. Web page sends XML document to Network Recording device.
  4. Network recording device creates one or more Recording Tasks.
  5. Web page may issue request to Network Recording device to determine status of Recording Tasks.
2. Request the recording device capture content based recurring time/date criteria.
  1. Web page creates a XML document describing a recurring record schedule.
  2. Note: See above steps.
3. Request the recording device capture content based on an EPG description.
  1. Web page obtains a EPG Content Items from Media Server (See prior use-case)
  2. Web page creates a XML document containing identifier of the EPG Item.
    Note: Network Recording service presumes that EPG service resides on same device.
  3. Web page sends XML document to Network Recording device.
  4. Network recording device creates one or more Recording Tasks.
4. Request the recording device capture content based on a metadata description.
  1. Web page creates a XML document describing a recurring record schedule which selects content based on metadata properties.
  2. Web page sends XML document to Network Recording device.
  3. Network recording device creates one or more Recording Tasks.
5. Provide controls to suspend/restart/cancel planned recording activities.
  1. Web page may issue request to Recording Device to get current Recording Tasks.
    Note: Record Tasks are generated from XML Record Schedule documents submitted to Network Recording Device.
  2. Web page may issue request to suspend, restart or cancel Record Tasks or Record Schedules.
6. Provide status information for current/future recording activities.
  1. Web page may issue request to Recording Device to get current Recording Task documents. Record Tasks contain various status information indicating whether recording is pending (not started), active (started but not completed), or done (completed).
7. Provide access to recorded program descriptions and content.
  1. The Network Recording device (Record Tasks) creates Content Items.
  2. The Record Task contains identifiers to the created Content Item(s) for the Record Task.
  3. The Content Items contain metadata describing what was recorded.

Motivation:

Existing Home Network interfaces do not (by scope/tradition) standardize user interface control. However, the combination of HTML markup and Javascript allow both access to Home Network devices functions and access to very capable user interface resources.
Why were you not able to use only existing standards to accomplish this?
- HTML User-Agents do not provided the abstractions necessary for Javascript to issue discover and issue commands to home network devices.
- Basic access methods to home network devices need to be standardized so that developers can implement pages to control Home Network devices using Javascript/HTML.
What might you suggest could be standardized?
- Provide generic access methods to enable a User-Agent to discover and command home network devices.
- Provide a means for applications to control some of the parameters that may be needed to be exposed to support well-established home network protocols. A more detailed analysis is needed to identify such parameters and a way to specify them in a transport agnostic way
- Provide access and privacy controls for untrusted pages accessing home network devices.
- Allow pages to discover functional differences (optional functionality) that may be implemented in some but not all Home Network ecosystem devices.

Dependencies:

This use-case provides the basic framework for generic access to home network devices. It is "agnostic" towards the underlying functionality of the devices being controlled.

Requirements:

Low Level	High Level
Application supports various recording schedules: Time of Day, Electronic Programming Guide, Metadata matching.	#Control of content recorders
Application provides status information for current and pending recording activities.	#Control of content recorders
Application provides ability to cancel, suspend, and restart pending recording activities.	#Control of content recorders
Application provides access to recorded content.	#Control of content recorders

U19. UPnP / DLNA ecosystem support

Original Proposal: ISSUE-23

Description: Use Cases U15, U16, U17, U18, and U1 describe functionality implemented by existing UPnP/DLNA home network devices in general terms. However, there is an existing install base of UPnP/DLNA compliant devices and the associated SDOs (UPnP Forum, DLNA) continue actively develop additional device related specifications. It would be beneficial to the end-user community if W3C User-Agents provided access to UPnP/DLNA devices without requiring any modification to the existing install base of UPnP/DLNA compliant devices.

Scenarios:

user agents support applications which can discover UPnP/DLNA compliant devices.
user agents support applications retrieving UPnP Device and Service Description XML documents.
user agents support application issuing of SOAP actions to control UPnP/DLNA devices.
user agents support applications subscribing to UPnP Service Events.
user agents support applications inspection of UPnP Service State Variables.
user agents support applications including DLNA specified HTTP transport headers to obtain DLNA content metadata and to control playback features (such as trick-play, time-based positioning).

A summary of UPnP specifications relevant to this use case is provided in the #Universal Plug'n Play (UPnP) Reference section.

Requirements:

Low Level	High Level
Application supports access to UPnP/DLNA devices without modification of existing deployed devices.	#Compatibility with widely deployed standards
Application support UPnP devices not related to content playback.	#Compatibility with widely deployed standards

Security

Tracker : ISSUE-3

Enabling application access to Home Network devices and content available on such devices raise several security and privacy concerns. Addressing user security and privacy is of primary importance for a conforming specification. This implies that some features may have to be "restricted" in order not to compromise user security. This may have an impact on the flexibility of these specifications, so a delicate balance between user experience and security needs to be found.

This section list possible threats that a conforming specification may have to deal with and possible solutions to reduce risks for the user. This section is not intended as an exhaustive list of problems and solutions but is intended to provide some directions for further investigation

Threats

Security

Malicious attacks
- An external server can control an Home Network device (e.g. send spam to your printer)
- A vulnerability in one of the Home Network devices can be used as a back door to get access to the Home Network
- Denial of service - repeated requests for resources could render a Home Network device/application/service unusable
Leaking of information from the home network to the Internet (e.g. content, content metadata)
Some users inside the home network may (inadvertently) allow external application(s) unwanted access to home network devices (e.g. a family may wish to prevent any website that the children view on their PCs or phones from being able to query and/or control other devices)

Privacy

3rd party visibility into content (e.g. movies) available in the HN.
3rd party visibility of types of devices available in the home network
3rd party visibility of applications in use/installed (e.g. don't expose the fact that my home security system is part of my HN)

Potential solutions

The list of techniques listed in this section are not mutually exclusive but can be combined to provide more flexibility in handling user security.

Home Network access allowed only to Trusted applications

In order for the application(s) to be able to access Home Network functionalities exposed by the UA, it needs to be trusted (by default is untrusted)
There could be different way to identify if an application is trusted and also different trust levels (and associated permissions).

How to determine the trust level of an application (several may apply)

Asking the user to authorize the application (via a dialog, a white list or other means)
Using a white list that is provided by the platform (e.g. surrendering control to service providers)

Trust levels options (mutually exclusive)

just 2 levels (trusted/untrusted)
- Trusted have full access to the Home Network capabilities
- Untrusted have NO access to the Home Network capabilities
multiple trust levels and associated permissions (like user in OSes)
- identify different subgroups of capabilities that the UA is able to expose
- associate these capabilities to a permission and each group to a subset of permissions
don't define generic trust levels but let the user decide on a "per action" base (see Modular access to HN Services and Content below)

Device Pairing

Pairing operation (using a PIN/password) is needed before devices can communicate
- The user is prompted to authorize the communication. Note that this could lead to a Potential denial-of-service.
- The user need to enter a (numeric) PIN or alphanumeric password
  - Numeric PIN may be easier to enter with traditional remote controls
  - A "default" PIN could be defined as common in other standards.
- The pairing could be one way (the control device make a "log in" into the controlled device) or two ways (devices needs to authorize each other)
Pairing Expiration: having pairing expire after a while could provide more security. In some scenarios thought, having the ability to configure it permanently allow for a better user experience. For example a user may have the Home Network devices configured by a technician. This is also very important for people with disabilities that may be unable to configure devices themselves.

Device Information only visible to the user

Exposing information about services, content and devices in the Home Network to an application is a potential privacy leak. This could be minimized allowing the application to only get access to:

an identifier for the device/content/service
an API to ask the UA to show more information about the content/service/device in some specific use cases, e.g.:
- ask the user to authorize a specific device to perform a given action
- show the user additional details about a specific device/ piece of content

This mechanism could limit the application flexibility but increases the user privacy.

An alternative approach could be to limit the application ability to access this information until the application has been explicitly authorized by the User, or has successfully paired with the device.

Modular access to Home Network Services and Content

One possible approach could be to consider each service as a separate "functionality" that the device has to authorize, in a similar fashion as Apps on mobile phones need to be authorized to access specific functionalities provided by the platform. The main difference in this case is that services available in the Home Network change over time, and also there isn't an installation process for web applications. Furthermore such approach would need a way to uniquely identify services (potentially across different home networking protocols) and expose relevant information for the user to understand what is actually authorizing. Such a technique would be then specification what would decide to enable a level interface for communication between devices/applications, which is not explicitly aware of different services

Categorization of Requirements and next steps

One of the deliverable of the Home Network Taskforce, according to its charter, is a set of recommendation to W3C Director for recommendation track work fulfilling some (or all) of the identified requirements.

This section summarizes the consensus of the TF about the desirable next steps and also to try to give an indication of relative priorities of the identified requirements.

Requirement Categorization

The HNTF has made an attempt to categorize requirements based on the priority of addressing them to enable some of the use cases that are already available via other technologies or that are felt particularly important to have an enhanced user experience. Go without saying that all identified requirements are considered important by the Task Force; so the criteria followed is more the degree to which each requirements enables use cases that cannot be covered today rather than a value judgment of the importance of one requirement versus another (which differs by individual). By this "enablement" criterion, a feature that enables more use cases is higher priority than a feature that enables a limited subset of use cases.

The following requirements underpin most (all) of the use cases listed in this document, so they are considered the starting point for a technical work:

#Service Discovery, #Content Discovery

These alone are insufficient to fulfill most of the use cases and user scenarios identified. The use cases that seem most straightforward to address next and which also show the most immediate utility are those that support the ability to control and query media playback

#Compatibility with widely deployed standards, #Control of content players, #Playback of content, #Playback of live content, #3 Box model, #Time-synchronization, #Service Communication

The requirements listed above focus on enabling applications to utilize existing home network services. The remaining requirements all enable use cases and scenarios that are of great interest and could be considered a "next step" of making it possible for applications to advertise and/or offer services. These requirements would allow leveraging web technologies and creating new types of application which mashes up Web and home networked devices.

#Services Advertisement, #Content Advertisement, #Application communication,#Application Discovery, #Push migration, #Pull migration,#Accurate time-synchronization

While security requirements are of really high priority in general, some of the requirement listed in this document may or may not be covered by a conforming specification depending on the particular technical solution that will be designed to cover the requirements listed above. That is why the following requirements to no fall into the High/Low priority scheme:

#Application trust level, #Access to home network services, #Prevent leaking of information

Media Identification is a prerequisite for applications to be able to properly relate items of media content to a wider web of data/content. On the other end this requirement is orthogonal to the other discussed in this document and could be probably discussed in parallel with the work on the other requirements. That is why the following requirement is also not categorized.

#Media Identification

Identified Gaps

A close analysis and discussion of the use-cases and requirements described in this document, lead the group to conclude that the major gap in the Open Web platform is the lack of means for an application to discover services and applications available on the home network. Filling such a gap will probably allow the coverage of most (if not all) requirements listed in this documents.

Some concerns were raised if the discovery alone is enough to address advertisements requirements especially for support of existing protocols such as UPnP. In fact, advertisements itself could be implemented as a service so that applications can discover it and then register to be advertised on the network. Such a service may be provided by the UA itself (i.e. the functionality is provided by the UA but exposed as a service) or by an external device. It is not clear though (lacking implementations of this) if this approach will be enough flexible and efficient

Furthermore to have an interoperable solution, a common advertising service would have to be standardized or JS binding to already in use advertising services/protocols would have to be provided.

The discussion will have to be deferred to the point in time when a UA supporting discovery will be available.

Next Steps

The HNTF believes that all the identified use cases and requirements are important to achieve a better convergence between the Web and TV. The TF also believes that all of them have an impact on the current web platform architecture so a recommendation track work would be needed to make sure that the identified requirements are covered in an open and interoperable way.

The HNTF believes that many of the requirements identified here may match the deliverables in the Device API (DAP) WG charter. Therefore it is proposed to provide this requirement document as an input for further work to the DAP WG. However, if after a more detailed analysis, requirements detailed in this document are found to be out-of-scope for the Device API (DAP) WG, then the HNTF recommends that additional W3C Working Groups be chartered to address the remaining requirements.

Open Issues

During his work, the HNTF did not have time to explore topics like the access to the home network from outside the home or concepts like group of homes. The TF believes that this is an area of extreme interest and would suggest the DAP WG (or any other working group working on the requirements extracted from this document) further investigate these topics. The Web & TV IG itself may consider discussing these topics and provide feedbacks to one or more W3C Working Groups.

Related works (informative)

This section include references to related works (prototypes, draft APIs, etc.) that have been produced by IG participants and are related to the use cases outlined in this document. Note that this section is informative and that the Web and TV IG do not endorse any of the works listed in this section. Nevertheless the Web and TV IG thinks that these related works contain useful ideas that could be discussed and expanded by a working group writing specifications addressing requirements enumerated in this document.

CableLabs

CableLabs is a not-for-profit research and development consortium that has cable operators as its members. While working on defining use cases and requirements as member of the Home Network TF of the Web&TV IG, some work has been done by CableLabs to experiment with possible ways to support use cases discussed by the Home Network Task Force in an HTML5 user agent. The latest draft of such proposal is available at http://www.w3.org/2011/webtv/HNTF/CableLabs_Revised_API_20110727-2.pdf

Opera

Opera proposed a low-level API to address some of the use cases outlined in this document and discussed by the Home Network Task Force of the Web&TV IG. This section contains an outline of the proposal as presented to the DAP WG.

Problem

A number of devices connected within a local network today advertise service API endpoints to the network, the primary purpose of which is to allow other devices within the same local network to connect, control and interact with those services through the provided interfaces.

It is possible today to build native applications that can both discover and communicate with such local-networked services. There is currently no standard or, indeed, any particularly easy way to discover or interact with local devices and services from a web application without significant technical hackery on the part of the user (see 'current practice' below).

We need a way for a user agent, acting as a broker between a user and a web application to discover services via common discovery protocols (i.e. SSDP and DNS-SD) within the local network, allow web applications to request a particular service type (or class of service) that it wishes to interact with and then for a user to authorize that connectivity based on services found in their local network matching the requested service type. The solution should then enable communication to occur between the authorized web application and the local-networked service, bypassing or overriding the same-origin policy rule if necessary to allow for safe and secure cross-domain communication.

Current Practice

Assuming the user is able to obtain a local-networked service's host and port information, which in itself is a hard task for both technical and non-technical users alike, the user must then provide that host and port information to the web application by entering such information in e.g. a web form and submitting said form. Having got this far the web application, despite having all the information required to communicate with a local-networked device/service, is then likely to encounter another issue, namely the same-origin policy rule which prevents a web application from communicating with a networked device that does not share the same host and port origin combination as itself.

As a solution to this, CORS has been developed to enable cross-origin messaging from web applications. Within the home network, if the target networked device happened to provide the HTTP header [Access-Control-Allow-Origin: *] then there is little problem with the browser now being able to communicate with the provided host and port.

However, the majority of local-networked services do not implement CORS and because [Access-Control-Allow-Origin: *] is generally a bad idea to enable within a local network, it is not really the ideal future-facing solution within local networks to ensure access is granted only to web applications that a user has explicitly authorized access to.

In the vast majority of use cases communication from a web application to a local-networked service is blocked or inherently insecure today due to one or more of the issues above being present in modern web browsers.

The current practice therefore does not fulfill the requirements of the initial problem and so we began to consider alternative approaches.

Proposed Solution

To counter the issues identified in the current practice above we developed a low-level API, produced incrementally over the course of the last few months, that allows a web application to request a particular service type (or class of service type) and for the user to then be able to authorize and connect a discovered matching service to the requesting web application. This enables a user-authorized web application to control, interact and synchronize content (and otherwise generally communicate) with home-networked services, negating the operational issues present in browsers today.

The proposed solution is available, in its full form, here:

http://people.opera.com/richt/release/specs/discovery/Overview.html

A note on CORS: this API is intended to work with or without CORS support enabled in local-networked services. That is to say, CORS in combination with e.g. HTTP authentication schemes, assuming such a model is deployed widely in the future, may provide a good user authorization model for this API (perhaps even to the point of negating the need for any specialized Service Discovery user interfaces at all). As it stands, the proposed solution is intended to work with existing services deployed within local networks without shoe-horning all communications in to a future-only CORS model.

Disclaimer: This API proposal does not represent the consensus of any standards group and should not be referenced as anything other than an unofficial draft which is a work in progress.

BBC

The BBC is non-profit, publicly funded broadcaster based in the UK that also engages in technical research. As part of its participation in the Home Network TF, the BBC has shared its own experimental API work that enables clients, including HTML user agents, to discover and communicate with television style devices within the home network.

This API defines a RESTful web service to be implemented by a television, set-top-box, or similar device. The data model represents programmes and sources of programmes in a relatively abstract way. The API in turn defines a uniform way to discover, identify, acquire (e.g. record) and control playback of programme content. There is also provision for starting and communicating with web or native applications running on the television. Applications using this API avoid having to deal with issues of codec, container format or transport protocol compatibility.

Documents providing background, an introductory overview and a draft specification of the API are available here:

The BBC has prototyped various remote control and second screen experiences that use this API. These include:

The presentation of time-synchronised slideshows
Near-lip-synced alternative audio
A Mobile phone as a remote control with support for blind users.
Enhancement of websites with awareness of current TV viewing

The following documents briefly describe some of these applications:

Telecom ParisTech

Telecom ParisTech is a French academic institution where research relevant to HNTF on combining W3C widgets, SVG and UPnP went on from 2008 to 2010. The work has been implemented as part of the open source multimedia player GPAC ( http://gpac.wp.institut-telecom.fr/ ) and is publically available. All use cases contributed by Telecom ParisTech to HNTF have been implemented in that system, but for the "pull migration" use case.

This work is described in the following papers:

C. Concolato, J. C. Dufourd, J. Le Feuvre, K. Park and J. Song, "Communicating and migratable interactive multimedia documents", Multimedia Tools and Applications, May 2011 available at http://biblio.telecom-paristech.fr/cgi-bin/download.cgi?id=11187
J. C. Dufourd, C. Concolato and J. Le Feuvre, "SVG Communicating Widgets", SVG Open, Mountain View, CA, USA, October 2009 available at http://www.svgopen.org/2009/papers/28-SVG_Communicating_Widgets/
C. Concolato, J. Le Feuvre and J. C. Dufourd, "Declarative Interfaces for Dynamic Widgets Communications", Document Engineering, Munich, Germany, September 2009, pp. 241-244 available at http://biblio.telecom-paristech.fr/cgi-bin/download.cgi?id=9919
J. Le Feuvre, C. Concolato and J. C. Dufourd, "Widgets Mobility", International Conference on Mobile Technology, Applications and Systems, Nice, France, September 2009 available at http://biblio.telecom-paristech.fr/cgi-bin/download.cgi?id=9917

CEA-2014 Web-based Protocol and Framework for Remote User Interface - Overview

CEA-2014 describes web based Remote UI applications for CE devices. CEA-2014 is referenced by multiple Standards Development Organizations to provide ecosystem specific Remote UI solutions.

CEA-2014 leverages existing web technology standards including XHTML for screen layout and user interactions; and ECMAScript for interactive application support. In addition, CEA-2014 provides additional standardized interfaces addressing the following areas:

Remote UI Application Listing
Client Capabilities Matching
Standardized Audio-Video Player
Digital Rights Management Framework
Content Download Management Framework
UPnP / DLNA Home Network Framework
SVG (Scalable Vector Graphics) integration with XHTML
Server to Client Notification Framework
Remote UI Save and Restore Framework

Remote UI Application Listing

CEA-2014 defines a mechanism for a Remote UI server to list available applications. This mechanism is based on an XML schema (XML UI Listing) which provides application descriptive metadata. In addition, a CEA-2014 server may include metadata for multiple versions of the same Remote UI application with differing client capability requirements. A Remote UI client system may then choose a version of a Remote UI application which best matches its capabilities. See Client Capability Matching in the following section

Client Capabilities Matching

CEA-2014 defines XML metadata describing the capabilities of a connecting Remote UI client system. These capabilities include Input Support (Remote Control, Keyboard, etc.), AV Player Capabilities, Client Display Capabilities (resolution, color space) and Digital Rights Management capabilities. When a Remote UI client connects to a server, it will first examine the capability requirements of the Remote UI applications offered and will connect to a version that best matches its capabilities. During connection, the Remote UI Client provides its capabilities to the server. The server may modify the web-page provided to better match the connecting client's capabilities or may reject the connection if it cannot tailor the page to support the client's capabilities.

Standardized Audio-Video Player

CEA-2014 defines a standardized Audio-Video Player object. The AV Player object provides methods for control of AV playback (Play, Stop, Pause, Seek and Loop). In addition the AV player provides a "lock-out" mechanism disables end-users from skipping "required content". AV player callbacks are defined to notify web-pages of various playback related state-changes. The CEA specification defines a state-machine for the AV player so web-pages can rely on predictable AV player state transition behavior.

Digital Rights Management Framework

CEA-2014 defines a framework to allow a Remote UI application (web-page) to communicate with DRM Agent(s) installed on a Remote UI client. This DRM framework allows generic messaging between the web-page and DRM Agents installed on the Remote UI client.
Note: CEA-2014 does not define the specifics of any particular DRM system beyond listing of available Digital Rights systems in Remote UI client capabilities.

Content Download Management Framework

CEA-2014 defines a framework for Remote UI applications to initiate the download of content. Once the download process is started the Remote UI client can switch to other Remote UI applications. In addition to providing status of download operations, CEA-2014 defines a Content Access Descriptor XML schema to describe content to be downloaded. Since Content Access Descriptors are XML documents they may be associated with URLs and embedded into Remote UI applications.

UPnP / DLNA Home Network Framework

CEA-2014 defines a UPnP / DLNA binding so that web-pages can discover and control home network devices. The binding allows Remote UI applications discover, manage events and command UPnP and DLNA devices. In conjunction with the UPnP / DLNA Home Network Framework the Standardized AV Player is required to comply with DLNA media transport requirements. The combination makes CEA-2014 Remote UI clients suitable to support both UPnP and DLNA applications.

The ability for Remote UI applications (originating from outside the home) to access home network devices presents significant security and privacy concerns. CEA-2014 requires implementation of a variety of security mechanisms to protect end-users Remote UI applications performing unauthorized access to Home Network devices.

SVG (Scalable Vector Graphics) integration with XHTML

CEA-2014 supports the embedding of SVG markup within XHTML pages and vice versa. CEA-2014 addresses some issues with regard to differing DOM models and event handling between SVG and XHTML resulting from the combination of SVG and XHTML content on the same web-page.

Server to Client Notification Framework

CEA-2014 provides the capability for servers to request Remote UI clients display web-pages. An "event" mechanism is defined so that Remote UI clients and servers can classify these Notification messages. In addition CEA-2014 clients may recover previously sent notifications that may have been missed while the Remote UI client was not present on the network.

Remote UI Save and Restore Framework

CEA-2014 provides a framework for a Remote UI application to save its state and POST it to a Remote UI server. The saved information consists of a reference to the original Remote UI application URL and metadata describing the saved state of the application. The save state may be restored by a different device effectively allowing the application to be moved to a different Remote UI client. The save-restore process may be triggered by the Remote UI application itself or by a request initiated external to the Remote UI client.

Universal Plug'n Play (UPnP) Reference

Introduction

This section discusses devices compliant to UPnP (Universal Plug'n Play) architecture. This architecture has been developed over +10 years by member companies of the UPnP Forum. Although this section focuses on UPnP device models defined for AV (Audio-Visual) devices, UPnP defines a number of other standard device models for Home Network Gateways, Telephony, Remote Access, and Home Automation applications.

Further information concerning membership in the UPnP Forum may be obtained at: http://www.upnp.org

UPnP AV Architecture

UPnP AV (Audio-Visual) Architecture provides an informative overview of the UPnP model for media serving and media rendering devices. It is a good starting point for readers becoming familiar with UPnP architecture related to audio-video device support.

AVArchitecture:2, UPnP Forum, December 31, 2010.
Available at: http://www.upnp.org/specs/av/UPnP-av-AVArchitecture-v2-20101231.pdf.
Latest version available at: http://www.upnp.org/specs/av/UPnP-av-AVArchitecture-v2.pdf.

UPnP Device Architecture

UPnP Device Architecture provides the basic mechanisms for discovering, advertising, and commanding of UPnP devices. All UPnP Devices implement and comply with the requirements in this specification.

UPnP Device Architecture, version 1.0, UPnP Forum, October 15, 2008.
Available at: http://www.upnp.org/specs/arch/UPnP-arch-DeviceArchitecture-v1.0-20081015.pdf.
Latest version available at: http://www.upnp.org/specs/arch/UPnP-arch-DeviceArchitecture-v1.0.pdf.

UPnP Media Renderer Device

The UPnP MediaRenderer specification defines the required and optional services of a standard UPnP MediaRenderer device.

MediaRenderer:3, UPnP Forum, December 31, 2010.
Available at: http://www.upnp.org/specs/av/UPnP-av-MediaRenderer-v3-Device-20101231.pdf.
Latest version available at: http://www.upnp.org/specs/av/UPnP-AV-MediaRenderer-v3-Device.pdf.

The following references define the required and optional services provided by a standard UPnP MediaRenderer device:
- UPnP Rendering Control Service
  The UPnP RenderingControl specification defines actions and variables which control presentation of content on a standard UPnP MediaRenderer device. Examples may include: controlling display brightness, audio volume, soundtrack selection, display pan/zoom, etc..
  
  RenderingControl:3, UPnP Forum, December 31, 2010.
  Available at: http://www.upnp.org/specs/av/UPnP-av-RenderingControl-v3-Service-20101231.pdf.
  Latest version available at: http://www.upnp.org/specs/av/UPnP-av-RenderingControl-v3-Service.pdf.
- UPnP AVTransport Service
  The UPnP AVTransport specification defines actions and variables which control connection to and playback of AV content on a standard UPnP MediaRenderer device. Examples may include: setting a playback URL, issuing Play, Stop, Seek, and Pause operations and determining the current state of a rendering device (Playing, Stopped, Paused, etc).
  
  AVTransport:3, UPnP Forum, December 31, 2010.
  Available at: http://www.upnp.org/specs/av/UPnP-av-AVTransport-v3-Service-20101231.pdf.
  Latest version available at: http://www.upnp.org/specs/av/UPnP-av-AVTransport-v3-Service.pdf.
- UPnP Connection Manager
  The UPnP ConnectionManager service defines actions and variables to allocate and control multiple sessions on a standard UPnP MediaRenderer device. Examples may include allocating a Picture-in-Picture window on a rendering device, or allocating an additional audio player instance on a rendering device.
  
  ConnectionManager:3, UPnP Forum, December 31, 2010.
  Available at: http://www.upnp.org/specs/av/UPnP-av-ConnectionManager-v3-Service-20101231.pdf.
  Latest version available at: http://www.upnp.org/specs/av/UPnP-av-ConnectionManager-v3-Service.pdf.

UPnP Media Server Device

The UPnP MediaServer device specification defines the required and optional services of a standard UPnP MediaServer device.

MediaServer:4, UPnP Forum, December 31, 2010.
Available at: http://www.upnp.org/specs/av/UPnP-av-MediaServer-v4-Device-20101231.pdf.
Latest version available at: http://www.upnp.org/specs/av/UPnP-AV-MediaServer-v4-Device.pdf.

The following references define services provided by a standard UPnP MediaServer device:
- UPnP Content Directory Service
  The UPnP ContentDirectory service publishes metadata for stored content, broadcast content, channel lineup(s) and electronic programming guide(s). A UPnP MediaServer device may optionally serve content described by ContentDirectory service metadata or the ContentDirectory service may provide metadata referring to content residing on other home network servers.
  
  ContentDirectory:4, UPnP Forum, December 31, 2010.
  Available at: http://www.upnp.org/specs/av/UPnP-av-ContentDirectory-v4-Service-20101231.pdf.
  Latest version available at: http://www.upnp.org/specs/av/UPnP-av-ContentDirectory-v4-Service.pdf.
- UPnP Connection Manager Service
  The UPnP ConnectionManager service defines actions and variables controlling multiple sessions allocated on a standard UPnP server device. The ConnectionManager service runs on both standard UPnP MediaRender and UPnP MediaServer devices.
  
  ConnectionManager:3, UPnP Forum, December 31, 2010.
  Available at: http://www.upnp.org/specs/av/UPnP-av-ConnectionManager-v3-Service-20101231.pdf.
  Latest version available at: http://www.upnp.org/specs/av/UPnP-av-ConnectionManager-v3-Service.pdf.
- UPnP Device Protection Service
  The UPnP DeviceProtection service specification provides secure communication to UPnP devices. For a UPnP MediaServer, the UPnP DeviceProtection service controls access to the UPnP MediaServer device services.
  
  DeviceProtection:1, UPnP Forum, December 31, 2010.
  Available at: http://www.upnp.org/specs/gw/UPnP-gw-DeviceProtection-v1-Service-20110224.pdf.
  Latest version available at: http://www.upnp.org/specs/gw/UPnP-gw-DeviceProtection-v1-Service.pdf.
- UPnP Scheduled Recording Service
  The UPnP ScheduledRecording service defines actions and variables which control creating and tracking of recording requests for a recording device. Examples may include: creating a recording request based time or electronic programming guide metadata, displaying programs already recorded and displaying and controlling pending recording activities. The ScheduledRecording service supports recording from both network (URLs) and non-network sources (such as analog and digital tuners).
  
  ScheduledRecording:2, UPnP Forum, December 31, 2010.
  Available at: http://www.upnp.org/specs/av/UPnP-av-ScheduledRecording-v2-Service-20101231.pdf.
  Latest version available at: http://www.upnp.org/specs/av/UPnP-av-ScheduledRecording-v2-Service.pdf.

Ericsson Labs

Ericsson Labs is the part of Ericsson Research that focuses on iterative open innovation and experiments with new technologies, services and business models. The purpose of Ericsson Labs is to support Ericsson’s vision of the Networked Society by working with partners including technology providers and third-party developers.

Ericsson Labs provides access to experimental APIs in their infancy so that developers can create new, innovative services. "Web Technologies" is one of the major categories of experimental APIs and the category contains API named "Web Device Connectivity".

Ericsson Labs: https://labs.ericsson.com/
Web Technologies: https://labs.ericsson.com/apis?api_category=203
Web Device Connectivity: https://labs.ericsson.com/apis/web-device-connectivity

The Web Device Connectivity API enables you to connect end users’ devices to the Web. It provides end users with new experiences of devices in their home networks being a part of the Web and web sites can seamlessly make use of those home devices. For example, an web site or a blog can access the viewer's DLNA enabled TV and let the viewer choose his/her DLNA enabled TV to play the contents on the web site.

Demo video: https://www.youtube.com/watch?v=fwQA4K9UdTM&feature=player_embedded

Web Device Connectivity platform consists of a software running inside each end users' home network and interacting with home devices, in-cloud servers aggregating device information and enforcing security policy, and a Javascript library providing API for web sites. Developers can create web sites, by including the Javascript library, which accesses devices in end users' home networks through the Ericsson Labs Web Device Connectivity platform.