W3C

Media Access Events

W3C Working Draft 13 October 2006

This version:
http://www.w3.org/TR/2006/WD-MediaAccessEvents-20061013/
Latest version:
http://www.w3.org/TR/MediaAccessEvents/
Editors:
Ola Andersson (Ikivo) <ola.andersson@ikivo.com>
Jean-Claude Dufourd (Streamezzo) <jean-claude.dufourd@streamezzo.com>
Roland Lundblad (Ikivo) <roland.lundblad@ikivo.com>

Abstract

This specification defines a set of DOM 3 Events [DOM3EV] compatible events and DOM APIs related to media access. The defined events and APIs enables detailed monitoring of media stream buffering and initialization. They are designed to be easily combined with the SVG Tiny 1.2 [SVGT12] uDOM but can equally well be combined with other, larger DOM APIs.

Status of this Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This is a First Public Working Draft of "Media Access Events".

This specification has been released for public feedback regarding the overall design and functionality; in particular, whether it meets the needs of OMA, ISO SC 29, SVG, and SMIL for event-driven access to streaming media. It should not be implemented at this stage.

This document was developed by the SVG WG as part of the W3C Graphics Activity. The Working Group expects to advance this Working Draft to Recommendation Status.

Web content and browser developers are encouraged to review this draft. Please send comments to www-svg@w3.org, the public email list for issues related to SVG. Archives of the list are available.

Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

Table of Contents

1. Introduction

This section is informative.

Streamable media, such as audio and video, is streamed over the network and is processed on the client as it arrives. This leads to different behaviour compared to downloaded media. The client normally holds a buffer where it stores part of the media prior to processing it. The state of this buffer affects the client. For example, if the buffer fills too slowly in relation to client processing the client will have to stall processing while it waits for the buffer to fill. It is often of interest to monitor the state of the buffer and client, and the purpose of correlating media access events introduced in this specification is to facilitate such monitoring. Initialising a session containing one or several media streams often takes a considerable amount of time. The author may wish to give the user some feedback that this setup is in progress and, when setup is done, present information of the available media streams.

2. Definitions

The Media Access Events namespace is http://www.w3.org/ns/media-access-event# .

The examples in this specification use a namespace prefix of me to denote the Media Access Events namespace .

A session is a set of synchronized streams rendered by a single media element.

3. State chart

This section is informative.

The following state chart illustrates the flow of events generated when streaming a media resource. Even if multiple streams are rendered by a single media element within one session , only one event is fired at each stage.

State chart

This chart can be explained as follows:

4. The MediaAccessEvent interface

The Media Access Event module contains basic event types associated with media access. This interface inherits from the Event interface in the SVG uDOM [SVGT12].

The MediaAccessEvent interface

interface MediaAccessEvent : events::Event {

  // MediaAccessEvent
  const unsigned short      STATUS_OK                           = 0;
  const unsigned short      STATUS_ERROR_ON_SETUP               = 1;
  const unsigned short      STATUS_ERROR_UNSUPPORTED_MEDIA      = 2;
  const unsigned short      STATUS_ERROR_CORRUPT_STREAM         = 3;

  readonly attribute boolean         bufferLevelValid;
  readonly attribute unsigned long   bufferLevel;
  readonly attribute float           bufferRemainingTime;
  readonly attribute unsigned short  status;
  DOMString           getSessionName();
  MediaStreamInfoList getMediaStreams();
};

Definition Group MediaAccessEvent

STATUS_OK
No error has occured.
STATUS_ERROR_ON_SETUP
Initialization of a session failed.
STATUS_ERROR_UNSUPPORTED_MEDIA
One of more of the streams within this session contains media not supported by the User Agent.
STATUS_ERROR_CORRUPT_STREAM
One of more of the streams within this session is corrupt and playback is stopped.

Attributes

bufferLevel of type unsigned long, readonly

Specifies a percentage value of the current buffer level. This value is only of relevance for 'DataReceptionProgress' events. If the session contains multiple streams, then the reported value is the minimum buffer level over all the streams of the session .

bufferLevelValid of type boolean, readonly

If false the buffer level is unknown and the value of bufferLevel SHOULD be ignored.

bufferRemainingTime of type float, readonly

Specifies the playing time (in seconds) of the current content of buffer. This value is only of relevance for 'DataReceptionProgress' events. If the session contains multiple streams, then the reported value is the minimum remaining time over all the streams of the session .

status of type unsigned short, readonly

Specifies the error that has occured. One of STATUS_OK , STATUS_ERROR_ON_SETUP , STATUS_ERROR_UNSUPPORTED_MEDIA or STATUS_ERROR_CORRUPT_STREAM MUST be STATUS_OK for every event type except 'Error' .

Methods

getMediaStreams

Returns information about the different media streams in this session .

No Parameters
Return Value
The list of streams in this session .
No Exceptions
getSessionName

Returns the name of the media access session .

No Parameters
Return Value
The media access session name.
No Exceptions

The Media Access event types are listed below.

Even if a state is not relevant to a particular delivery protocol or implementation, the corresponding events MUST be fired. For example, if there is no session setup phase, then events 'BeginSessionSetup' , 'EndSessionSetup' and 'DataRequest' MUST be fired together in that order.

BeginSessionSetup
Type BeginSessionSetup
Namespace http://www.w3.org/ns/media-access-event#
Interface MediaAccessEvent
Cancelable Yes
Bubbles Yes
Target the media element holding the URL to the delivered content
Context info none

A 'BeginSessionSetup' event MUST be fired at the beginning of a streaming session .

EndSessionSetup
Type EndSessionSetup
Namespace http://www.w3.org/ns/media-access-event#
Interface MediaAccessEvent
Cancelable Yes
Bubbles Yes
Target the media element holding the URL to the delivered content
Context info none

An 'EndSessionSetup' event MUST be fired after a 'BeginSessionSetup' event when the initialisation of a streaming session has been completed.

DataRequest
Type DataRequest
Namespace http://www.w3.org/ns/media-access-event#
Interface MediaAccessEvent
Cancelable Yes
Bubbles Yes
Target the media element holding the URL to the delivered content
Context info none

A 'DataRequest' event MUST be fired when the UA sends the request for data to the server.

Playable
Type Playable
Namespace http://www.w3.org/ns/media-access-event#
Interface MediaAccessEvent
Cancelable Yes
Bubbles Yes
Target the media element holding the URL to the delivered content
Context info none

An 'Playable' event MUST be fired after a 'DataRequest' event when enough data is available to start or resume playing.

DataReceptionProgress
Type DataReceptionProgress
Namespace http://www.w3.org/ns/media-access-event#
Interface MediaAccessEvent
Cancelable Yes
Bubbles Yes
Target the media element holding the URL to the delivered content
Context info none

The User Agent MAY dispatch 'DataReceptionProgress' when data is being received. Information on the current buffer level SHOULD be available.

NotPlayable
Type NotPlayable
Namespace http://www.w3.org/ns/media-access-event#
Interface MediaAccessEvent
Cancelable Yes
Bubbles Yes
Target the media element holding the URL to the delivered content
Context info none

An 'NotPlayable' event MUST be fired when not enough data is available to continue playing.

EndOfDataReception
Type EndOfDataReception
Namespace http://www.w3.org/ns/media-access-event#
Interface MediaAccessEvent
Cancelable Yes
Bubbles Yes
Target the media element holding the URL to the delivered content
Context info none

An 'EndOfDataReception' event MUST be fired when the data reception has finished.

Stop
Type Stop
Namespace http://www.w3.org/ns/media-access-event#
Interface MediaAccessEvent
Cancelable Yes
Bubbles Yes
Target the media element holding the URL to the delivered content
Context info none

A 'Stop' event MUST be fired when the data reception has finished and all received data has been presented.

An 'Stop' event MUST be fired when an end condition has been met within the media element, e.g. one expression in the end attribute has become true or endElement() has been called on the media element. In that case, the 'Stop' event MUST be sent before the endEvent to be dispatched from the media element.

Error
Type Error
Namespace http://www.w3.org/ns/media-access-event#
Interface MediaAccessEvent
Cancelable Yes
Bubbles Yes
Target the media element holding the URL to the delivered content
Context info none

An 'Error' event MUST be fired when an error occurs. The error code provides information about which kind of error has occured.

There MUST be at most one event of each of the following types sent within one session : 'BeginSessionSetup' , 'EndSessionSetup' , 'DataRequest' , 'EndOfDataReception' , 'Stop' , 'Error'

There MAY be more than one event of each of the following types sent within one session : 'DataReceptionProgress' , 'Playable' , 'NotPlayable'

5. The MediaStreamInfo interface

The MediaStreamInfo interface stores information about a media stream.

The MediaStreamInfo interface

interface MediaStreamInfo {
  readonly attribute DOMString  mediaType;
  readonly attribute DOMString  transport;
  readonly attribute DOMString  format;
};

Attributes

format of type DOMString, readonly

The format of the media in this stream. E.g. "H264". This attribute MUST match the format as specified by IANA , see listing [MIME-Media-Types].

mediaType of type DOMString, readonly

The type of media in this stream, e.g. "video" or "audio". [Editor's note: this wording is under discussion]

transport of type DOMString, readonly

The protocol this stream is being delivered over, e.g. "RTP" or "HTTP". [Editor's note: this wording is under discussion]

6. The MediaStreamInfoList interface

Returns the index th item in the collection. If index is greater than or equal to the number of MediaStreamInfo s in the list, this returns null .

The MediaStreamInfoList interface

interface MediaStreamInfoList {
  readonly attribute unsigned long  length;
  MediaStreamInfo    item(in unsigned long index);
};

Attributes

length of type unsigned long, readonly

Returns the number of items in the collection.

Methods

item

Returns the index th item in the collection. If index is greater than or equal to the number of MediaStreamInfo s in the list, this returns null .

Parameters
index of type unsigned long
Index into the collection.
Return Value
The MediaStreamInfo at the index th position in the MediaStreamInfoList , or null if that is not a valid index .
No Exceptions

7. Conformance

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].

Unless otherwise specified immediately following the header, all sections in this document - to the exclusion of examples which are all informative - are normative.

8. Examples of usage

This section is informative.

Three examples to illustrate the usage of streaming events.

8.1. Simple state information display

The following example shows the streaming of video. When the playback of the video is stalled due to network congestion, a simple text is displayed. When the playback resumes, the text is removed.

Example: Using MediaAccessEvents in SVG to display a simple message
<svg xmlns="http://www.w3.org/2000/svg" version="1.2" baseProfile="tiny"
xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
xmlns:se="http://www.w3.org/ns/media-access-event#" width="100%" height="100%"
viewBox="0 0 400 300">
    <video id="exampleMovie" type="video/h264" xlink:href="rtsp://example.org/movies/exampleMovie.sdp"
           x="0" y="0" width="400" height="300"/>
    <text display="none" id="information" fill="white" x="50" y="50" font-size="30">
         <set attributeName="display" to="inline" begin="exampleMovie.NotPlayable" fill="freeze"/>
         <set attributeName="display" to="none" begin="exampleMovie.Playable" fill="freeze"/>
        Buffering
    </text>
</svg>

8.2. Buffering progress bar

The following example shows streaming of video within svg. When playback of the video is stalled due to a lack of video data a progress bar is displayed. The progress bar indicates how much buffering that is needed before playback can be resumed. The example is not runnable as is since the intention is to show an example of using streaming events, not to give a complete svg example.

Example: Using MediaAccessEvents in SVG to display a buffering progress bar
<svg xmlns="http://www.w3.org/2000/svg" version="1.2" baseProfile="tiny"
xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
xmlns:se="http://www.w3.org/ns/media-access-event#">
    <script type="application/ecmascript"><![CDATA[
        function initSetup (evt) {
            var setupAnimation = document.getElementById('setupAnimation');
            setupAnimation.setAttributeNS("http://www.w3.org/2000/svg", "visibility", "visible");
            setupAnimation.beginElement();
        }

        function exitSetup (evt) {
            var setupAnimation = document.getElementById('setupAnimation');
            setupAnimation.setAttributeNS("http://www.w3.org/2000/svg", "visibility", "hidden");
            setupAnimation.endElement();
        }

        function bufferStart (evt) {
            var progressBar = document.getElementById('progressBar');
            progressBar.setAttributeNS("http://www.w3.org/2000/svg", "width", "0");
            progressBar.setAttributeNS("http://www.w3.org/2000/svg", "visibility", "visible");
            var bufferAnimation = document.getElementById('bufferAnimation');
            bufferAnimation.setAttributeNS("http://www.w3.org/2000/svg", "visibility", "visible");
            bufferAnimation.beginElement();
        }

        function bufferProgress (evt) {
            var progressBar = document.getElementById('progressBar');
            progressBar.setAttributeNS("http://www.w3.org/2000/svg", "width", 100*(evt.bufferFillLevel));
        }

        function bufferComplete (evt) {
            var progressBar = document.getElementById('progressBar');
            progressBar.setAttributeNS("http://www.w3.org/2000/svg", "visibility", "hidden");
            var bufferAnimation = document.getElementById('bufferAnimation');
            bufferAnimation.setAttributeNS("http://www.w3.org/2000/svg", "visibility", "hidden");
            bufferAnimation.endElement();
        }
    ]]></script>

    <video id="exampleMovie" type="video/h264" xlink:href="rtsp://example.org/movies/exampleMovie.sdp"
           x="0" y="0" width="400" height="300">
        <handler type="application/ecmascript" ev:event="me:BeginSessionSetup">
            initSetup(evt);
        </handler>

        <handler type="application/ecmascript" ev:event="me:EndSessionSetup">
            exitSetup(evt);
        </handler>

        <handler type="application/ecmascript" ev:event="me:NotPlayable">
            bufferStart(evt);
        </handler>

        <handler type="application/ecmascript" ev:event="me:DataRequest">
            bufferStart(evt);
        </handler>

        <handler type="application/ecmascript" ev:event="me:DataReceptionProgress">
            bufferProgress(evt);
        </handler>

        <handler type="application/ecmascript" ev:event="me:Playable">
            bufferComplete(evt);
        </handler>
    </video>

    <g> ...
        <animate id="bufferAnimation" ... />
    </g>

    <g> ...
        <animate id="setupAnimation" ... />
    </g>

    <rect id="progressBar" ... />
</svg>

8.3. Stream info box

This example shows streaming of video within svg. An svg-button is available and when clicked it displays a box with information about the video stream. As with the example above this one isn't runnable as is since the svg markup isn't complete. The image to the right shows a possible rendering of the info box in this example. Example of info box
Example: Using MediaAccessEvents in SVG to display a stream info box
<svg xmlns="http://www.w3.org/2000/svg" version="1.2" baseProfile="tiny"
xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
xmlns:se="http://www.w3.org/ns/media-access-event#">
    <script type="application/ecmascript"><![CDATA[
        function sessionSetupComplete (evt) {
            var sessionName, sessionProtocol;
            var streamFormat, mediaType;
            sessionName = evt.sessionName;
            if (sessionName == 0) sessionName = "No session name available";
            var streamList = evt.getMediaStreams();
            var stream = streamList.item(0);
            if (stream == 0){
                sessionProtocol = "N/A";
                streamFormat = "N/A";
                mediaType = "N/A";
            }
            else{
                sessionProtocol = stream.transport;
                streamFormat = stream.format;
                mediaType = stream.mediaType;
            }
            document.getElementById('sessionNameText').textContent = "Session name: " + sessionName;
            document.getElementById('sessionProtocolText').textContent = "Streaming protocol: " + sessionProtocol;
            document.getElementById('sessionTypeText').textContent = "Streaming media: " + mediaType;
            document.getElementById('sessionFormatText').textContent = "Format/codec: " + streamFormat;
        }

        function showInfo (evt) {
            var infoBox = document.getElementById('infoBox');
            setupAnimation.setAttributeNS("http://www.w3.org/2000/svg", "visibility", "visible");
        }

        function hideInfo (evt) {
            var infoBox = document.getElementById('infoBox');
            setupAnimation.setAttributeNS("http://www.w3.org/2000/svg", "visibility", "hidden");
        }
    ]]></script>

    <video id="exampleMovie" type="video/h264" xlink:href="rtsp://example.org/movies/exampleMovie.sdp"
           x="0" y="0" width="400" height="300">
        <handler type="application/ecmascript" ev:event="me:EndSessionSetup">
            sessionSetupComplete(evt);
        </handler>
    </video>

    <g id="infoButton"> ...
        <handler type="application/ecmascript" ev:event="ev:DOMActivate">
            showInfo(evt);
        </handler>
    </g>

    <g id="infoBox"> ...
        <text id="sessionNameText" ...></text>
        <text id="sessionProtocolText" ...></text>
        <text id="sessionTypeText" ...></text>
        <text id="sessionFormatText" ...></text>

        <g id="okButton"> ...
            <handler type="application/ecmascript" ev:event="ev:DOMActivate">
                hideInfo(evt);
            </handler>
        </g>
    </g>
</svg>

9. Acknowledgements

This section is informative.

The authors would like to thank Cyril Concolato for his thorough review and excellent contribution. We would also like to thank Robin Berjon for ReSpec, the editorial framework upon which this specification is written.

A. References

DOM3EV
Document Object Model (DOM) Level 3 Events Specification, Philippe Le Hégaret (W3C), and Tom Pixley (Netscape Communications Corporation).
MIME-Media-Types
IANA MIME Media Types,
RFC2119
Key words for use in RFCs to Indicate Requirement Levels, S. Bradner.
SVGT12
Scalable Vector Graphics (SVG) Tiny 1.2 Specification, Ola Andersson <ola.andersson@ikivo.com>, Robin Berjon <robin.berjon@expway.fr>, Jon Ferraiolo <jon.ferraiolo@adobe.com>, Vincent Hardy <vincent.hardy@sun.com>, Scott Hayman, Dean Jackson <dean@w3.org>, Craig Northway <craign@cisra.canon.com.au>, and Antoine Quint <aq@fuchsia-design.com>.