W3C

Synchronized Multimedia Modules
based upon SMIL 1.0

W3C Note 23 February 1999

This version: http://www.w3.org/TR/1999/NOTE-SYMM-modules-19990223
Latest version: http://www.w3.org/TR/NOTE-SYMM-modules
Editors: Patrick Schmitz ()
Ted Wugofski ()
Warner ten Kate ()


Status of this document

This Note was produced by the W3C SYMM Interest Group. It describes a snapshot of certain aspects of the discussions on future work on SYMM and SMIL. Following the approach spearheaded by the W3C HTML Working group, the Note describes how SMIL 1.0 functionality and future SYMM functionality can be represented as a set of markup modules. The intention was to study the feasibility of such an approach, and the details of the proposed modularisation are preliminary only. Please send comments to the public mailing list www-smil@w3.org.

This document is a NOTE made available by W3C for discussion only. This indicates no endorsement of its content, nor that W3C has, is, or will be allocating any resources to the issues addressed by the NOTE.

Table of Contents

Abstract
1. Introduction
2. Synchronized Multimedia Modules
3. Module Integration
4. Multimedia Profiles
5. Issues
References

Abstract

This document describes a proposed modularization of synchronized multimedia functionality, based upon the SMIL 1.0 Recommendation.  The proposed modules complement HTML/CSS modules to support the definition of multimedia profiles that combine SYMM and HTML/CSS modules, as well as other XML-based languages.  A set of example profiles for multimedia use is described. The document is intended to form a basis for work by a future SYMM Working Group.

1. Introduction

The first W3C Working Group on Synchronized Multimedia (SYMM) developed SMIL - Synchronized Multimedia Integration Language. This XML-based language is used to express synchronization relationships among media elements.  SMIL 1.0 documents describe multimedia presentations that can be played in SMIL-conformant viewers.

SMIL 1.0 provides limited means for integrating with HTML, CSS or other XML languages.  This integration has been an area of considerable interest within the W3C as well as other external organizations (such as the ATSC T3/S17 specialist group). The W3C SYMM Interest Group has been tracking these issues in preparation for future work on the SMIL language..

The W3C HTML Working Group is working towards a modularization of HTML.  The W3C CSS Working Group has had similar discussions. One reason that the HTML Working Group is pursuing modularization is to support the use of HTML in a wider range of applications. In support of this activity, the HTML Working Group and the SYMM Interest Group have been discussing strategies for integrating multimedia functionality into HTML. Several proposals have been submitted and are being discussed (HTML+TIME and BHTML). One approach to solving this problem is to modularize synchronized multimedia functionality, and make these modules available to other languages.

1.1. Goals and Requirements

There are four goals for modularizing synchronized multimedia functionality:

In essence, modularizing SMIL is the first step towards the integration of timing and other key aspects of SMIL with other Web languages. This modularization provides a means for other languages, such as HTML and an XML-based vector-graphics format, to benefit from timing and synchronization functionality while reusing the lessons learned and work embodied in SMIL. This becomes especially important for the development of cross-language tools which can readily transcode syntax but generally have a difficult time porting content to different timing and content models.

This Note describes the current approach to this problem by the SYMM Interest Group. The modules described herein satisfy the following design requirements:

Several additional requirements will inform the work of a future SYMM working group:

The purpose of the requirements is to ensure that the modularization makes no significant changes to the SMIL 1.0 language while laying a foundation for integration with HTML, CSS, and other Web Languages. This Note provides a discussion of current integration strategies and proposals.

The SYMM Interest Group recommends that the results presented in this Note form the basis for some of the work of a future SYMM Working Group.

1.2. Modules and Profiles

Modularization of synchronized multimedia functionality provides a building block for language, application, and platform developers. This Note recognizes that in addition to functional modules, there exists the notion of profiles and platforms.

For the purposes of this Note, we define the terms module, profile, and platform:

module
A module is a collection of elements. Each element is in one and only one module.
profile
A profile is a collection of modules particular to an application domain or language. For example: The SMIL profile corresponds to the collection of modules that make up the SMIL language. A enhanced television profile would correspond to the collection of modules for media-enhancement of broadcast television.
platform
A platform is a collection of profiles, non-Web technologies, policies, resource requirements, etc.

This Note focuses on modules and profiles, and is not concerned with specifying or characterizing platforms. Nevertheless, it is worth noting the distinction between a profile and a platform: a profile generally describes a document type and its usage, whereas a platform encompasses the entire environment. (The term "profile" as defined by the HTML WG is likely to bear a wider scope than used in this Note.)

2. Synchronized Multimedia Modules

This Note proposes a set of modules for SYMM functionality.  The first section describes the modules associated with functionality defined in SMIL 1.0.  A second section describes proposed extensions to SMIL 1.0. 

2.1. SMIL 1.0 Modules

Structure
Includes the <smil>, <head> and <body> elements.  These elements will be used in documents based upon traditional SMIL profiles.
 
Meta
Includes the <meta> element. The elements contain information describing the document, either to inform the human user or to assist some automation. The element will be used in documents based upon traditional SMIL profiles, possibly in concert with meta data modules available from other efforts.
 
Layout
Includes the <layout>, <root-layout> and <region> elements, and related attributes. This module supports the layout mechanism described in SMIL 1.0.  These elements will all be used in documents based upon the SMIL profile.
 
Timing and Synchronization markup
This module includes timing structure (e.g. sequence and parallel timelines) and timing control properties (e.g. begin, end, and repeat) that describe the timing relationships among elements.   This module also includes mechanisms to control the performed synchronization of the document (described in SMIL 1.0 as soft and hard).
This is separated from the media so that it can be applied to HTML and XML applications independent of support for media.
Note that in some integration profiles, the specific syntax for the markup may vary slightly with the profile (e.g. the SMIL 1.0 <seq> element may be represented as a sequence attribute if a CSS style-based integration is used).  These differences notwithstanding, the module semantics and the timing model will be consistent across all profiles. 
The specific syntax and mechanism of integration with HTML and XML applications is under review. 
 
Media Elements
This includes the media declaration elements <ref>, <animation>, <audio>, <img>, <video>, <text> and <textstream>.  The media declaration may relate to the object element in HTML (this needs attention).
 
Linking
Includes the <a> and <anchor> elements.
The relation of this module to other linking modules, particularly XLink, XPointer and HTML linking, is under review.  For example when integrating with HTML, timing applied to the (HTML) <a> and <area> elements could provide much or all of the SMIL 1.0 defined functionality.
 
Content control
This includes the <switch> element and test attributes introduced in SMIL 1.0.  This is of general utility to authors, independent of the multimedia functionality.

2.2. Proposed Extensions

Layout (extensions)
Additional functionality will likely be added to this module as part of ongoing work.   Relationship of this Layout module to other Layout mechanisms, including CSS, is to be considered.
 
Timing and Synchronization markup  (extensions)
Additional functionality will likely be added to the module as part of ongoing work.   Proposed extensions include event-based timing constructs, e.g. to support user-interaction and stream events, and more detailed mechanisms to control the performed synchronization of the document.
 
Transitions
This module includes new functionality to support transition effects on elements.  These may be media elements, or they may be traditional HTML (or XML) elements.   A CSS-style syntax under consideration might represent transitions as in this HTML-based example:
<html>
  <head>
    <title>Powerpoint Animation in HTML</title>
    <style>
      p.caption { transition-style: wipe; font-size: big }
      li { transition-style: dissolve }
    </style>
  </head>
  <body>
    <p class="caption">Multimedia Requirements</p>
    <ul>
      <li>Timing the Display of Bullet Items</li>
      <li>Transition Effects</li>
    </ul>
    </body>
</html>
      

Specifics of the module syntax is under review. There is a desire to present a simple syntax for typical use-cases, and also to support more complex functionality by modeling transitions as filters with timing support. The various proposals will be considered and refined as work proceeds.

Animation
This module includes functionality to support common simple animation like motion paths.  Where Transitions apply rendering filters, Animation behaviors manipulate Object Model properties like position, color, size or rotation (e.g. in an XML-based vector-graphics format).
 
Resource Management
This module includes support for resource (e.g. bandwidth) management hints.  These can facilitate user agent optimizations, and can control the degradation strategy of the user agents.  For example, the author may assign priorities to elements to inform a user-agent that will dynamically balance network bandwidth. Other proposed markup includes hints for cueing or preloading media and a means of specifying that an individual media element is delivered as a substream within an aggregated media stream (for large streamed presentations).

3. Module Integration

The HTML Working Group is still reviewing formal mechanisms to define or specify profiles.   Several approaches are under consideration including DTDs, XML Schemas, and RDF-based mechanisms.  The SYMM Activity will provide associated requirements and use-cases to the HTML Working Group, and will follow the recommendation of the HTML Working Group for profile specification.

Several mechanisms for integrating SYMM functionality with other languages (especially HTML) have been discussed.  It is possible that a given module may be represented with different syntax depending upon the profile and integration.  Nevertheless, the SYMM modules will define consistent semantics for all profiles, regardless of syntax variants.

The current variants and issues for integrating synchronized multimedia functionality with HTML and XML languages are:

4. Multimedia Profiles

A range of profiles have been discussed that would make use of different sets of modules. Below is an initial list of profiles that could integrate SYMM and HTML functionality in different ways.  This is not intended to be a final statement, but rather an indication of how modules would be used.  It provides a backdrop for the discussion of modularization.

In addition, some specific proposals can be described in terms of the relevant modules:

4.1. Example Profiles

Leightweight presentations
A basic profile would handle simple presentations, supporting timing of text content. The simplest version of this could be used to sequence stock quotes or headlines on restricted platforms like palmtop devices or smart phones.  The Mobile Access group is looking at related profiles.  An HTML-based profile would combine HTML modules (probably a subset of HTML) with the timing module. It might include the transitions as well.
 
Timeline-Centric Multimedia presentations
This describes the profile associated most closely with SMIL 1.0.  This does not integrate HTML, and uses SMIL layout rather than the flow layout model associated with HTML .  Typical use-cases for this profile are multimedia presentations that include time-based and streaming media. This would include most or all of the SMIL modules.
 
HTML integrated Multimedia presentations
This is a variant on the Traditional Multimedia profile that mixes multimedia and HTML.   It mixes HTML layout (i.e. flow layout) and CSS positioning.  Use-cases for this profile are presentations that use HTML as a media type in concert with time-based and streaming media (e.g. PowerPoint-like presentations with time-based media). This profile would include (most or all of) the HTML modules, the Timing, Media, Transition, Animation, Resource Management and Content control modules (i.e. all but the Structure, Meta and Layout modules).  The linking functionality would overlap between the Linking module and a similar HTML module.
 
Web Enhancement of Broadcast, On-Demand Video and DVD
This describes support for web presentations tied to broadcast and/or on demand media.   The primary media will often define the main timeline. The main additional requirements here are for stream event support and the ability to tie the page clock to the primary media. Stream event support requires event-based timing of elements (and timelines), and a means of associating events with document-external (stream) triggers. E.g. where the stream could be a TV-broadcast, it would be possible to refer to the events by referencing the TV broadcast stream or player-object.  It must be possible to ensure that the multimedia (enhancement) timeline is strictly synced to or driven by the primary (broadcast) media.

Note that where "webcasting" may associate Web-pages to a TV-broadcast, in "enhanced television" additional media form an integral part with the main video presentation, providing enhancement of and interaction with the program.

While the typical use-cases may be dominated with the primary media (e.g. streaming video broadcast or DVD video), the authoring model is still closely tied to the Web and HTML.   This profile would likely use the same modules as the HTML integrated Multimedia presentations profile, but with the additional timing and synchronization requirements described.

5. Issues

Syntax

The details of the elements and properties still need to be worked out.  In particular, there are open issues on the syntax for timing markup, and the mechanism for integration with HTML and XML applications, and stylesheet tools like CSS and XSL.

Accessibility syntax

An additional issue that must be coordinated with representatives of the Web Accessibility Initiative (WAI) concerns accessibility support.  This includes elements and attributes which assist the accessibility of content declared in the presentation. In SMIL 1.0, this concerns the title, alt and longdesc attributes. 

It is not clear whether this should be an additional module, or additional support in specific modules.   In particular, the intent is not to define a new set of Accessibility mechanisms, but to integrate current work of expert groups in this area.


References

Broadcast HTML
A Modular Hypertext Markup Language for Broadcast Applications. HTML working group contributed paper. Ted Wugofski. This is available at: http://toocan.philabs.research.philips.com/misc/atsc/bhtml/
HTML+TIME
Timed Interactive Multimedia Extensions for HTML (HTML+TIME) Note 18 September 1998, Patrick Schmitz, Jin Yu, Peter Santangeli. This is available at: http://www.w3.org/TR/NOTE-HTMLplusTIME
Reformulating HTML in XML
W3C Working Draft 5 December 1998, Dave Raggett, Frank Boumphrey, Murray Altheim, Ted Wugofski. This is available at: http://www.w3.org/TR/1998/WD-html-in-xml-19981205
SMIL 1.0
Synchronized Multimedia Integration Language (SMIL) 1.0 Specification 15 June 1998, Philipp Hoschka. This is available at: http://www.w3.org/TR/REC-smil