SL16: Providing Script-Embedded Text Captions for MediaElement Content

The objective of this technique is to use text captioning that is embedded in the stream with media displayed in a Silverlight MediaElement, and present that text captioning in a separate Silverlight control or text element.

This particular technique uses scripting files with a TimelineMarkers collection that are embedded directly within the media file. When text captioning is embedded directly in the streams, synchonization of the scripting stream versus the video content stream is done automatically by the MediaElement component. Each time the MarkerReached event fires, that is an indication that a synch point in the video that corresponds to a script marker entry has been reached. Silverlight application authors can obtain the text from the relevant timeline marker entry through their event handler implementations, and can display captions in the user interface area where the text captions are displayed. Typical Silverlight controls that can be used for displaying text captions include TextBlock (nonfocusable), TextBox, or RichTextBox. A typical interface design would place the caption-display control in close proximity to the MediaElement control that is being captioned, for example might place the captions directly underneath the MediaElement "screen".

Script-embedded captions are captions that are stored directly in the media file as metadata, rather than as a separate file. For information about techniques for captions in separate files, see SL28: Using Separate Text-Format Text Captions for MediaElement Content.

Tools

Producing the media file with TimelineMarkers captions directly in embedded scripting can be accomplished using the Microsoft Expression Encoder tool. Online help for the procedure of encoding scripting with text captions in the stream are available in the offline Help file that installs with the Microsoft Expression 4 Encoder products. For more information, see Expression Encoder Pro Overview.

There is a public API for introducing Markers into a WMV file, as part of the Windows Media Format SDK. Using Expression Encoder is the way that the task of directly embedding TimelineMarkers is presented and taught in Microsoft's available instructional material on Silverlight. However, because the mechanism is public, it is possible that other tools exist or will exist that can also produce media with script-encoded TimelineMarkers.

Examples

Example 1: MediaElement handles MarkerReached, displays marker text in existing TextBox

This example has a UI definition in XAML and interaction logic in C#. The following is the basic UI in XAML. This example is deliberately simple and does not include AutomationProperties for identification or user instructions. The most relevant part of this example is that the Silverlight author declares a handler for the event MarkerReached. This event fires potentially hundreds of times, once for each caption in the stream. Each time the event fires, the event handler runs and adds the text to the dedicated TextBox in the user interface.

<UserControl x:Class="MediaTimelineMarkers.MainPage"
   xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
   xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
>
   <StackPanel x:Name="LayoutRoot" Background="White">
       <MediaElement MarkerReached="OnMarkerReached"
       HorizontalAlignment="Left"
       Source="/spacetime.wmv"
       Width="300" Height="200" />
       <ScrollViewer>
           <TextBox Name="captionText" Height="40"
           IsReadOnly="true" AcceptsReturn="true"/>
       </ScrollViewer>
   </StackPanel>
 </UserControl>

private void OnMarkerReached(object sender, TimelineMarkerRoutedEventArgs e)
{
   captionText.Focus();
   captionText.SelectedText = e.Marker.Text.ToString() + "\n";
}

This example is shown in operation in the working example of Media Timeline Markers.

Resources

Resources are for information purposes only, no endorsement implied.

Audio and Video Overview
MediaElement Class
Accessible Media Project - a reference implementation MediaPlayer control from the Silverlight product team that includes several accessibility features including captioning; note that the codebase might not be updated to Silverlight version 4

Related Techniques

Tests

Procedure

Using a browser that supports Silverlight, open an HTML page that references a Silverlight application through an object tag. The application plays media that is expected to have text captioning.
Check that a text area in the user interface shows captions for the media.

Expected Results

# 2 is true.

If this is a sufficient technique for a success criterion, failing this test procedure does not necessarily mean that the success criterion has not been satisfied in some other way, only that this technique has not been successfully implemented and can not be used to claim conformance.