EPUB Media Overlays 3.2

Final Community Group Specification

Editors:
Marisa DeMeglio (DAISY Consortium)
Daniel Weck (DAISY Consortium)
Participate:
GitHub w3c/publ-epub-revision
File a bug
Commit history
Pull requests

Abstract

This specification defines a usage of [SMIL3] (Synchronized Multimedia Integration Language), the Package Document, CSS Style Sheets, and EPUB® Content Documents for representation of audio synchronized with the EPUB Content Document.

The text and audio synchronization enabled by Media Overlays provides enhanced accessibility for any user who has difficulty following the text of a traditional book. Media Overlays also provide a continuous listening experience for readers who are unable to read the text for any reason, something that traditional audio embedding techniques cannot offer. They are even useful for purposes not traditionally considered accessibility concerns (e.g., for language learning or reading of commercial audio books).

This specification is one of a family of specifications that compose EPUB 3 [EPUB32], an interchange and delivery format for digital publications based on XML and Web Standards. It is meant to be read and understood in concert with the other specifications that make up EPUB 3.

Refer to [EPUB32Changes] for more information on the differences between this specification and its predecessor.

Status of This Document

This specification was published by the EPUB 3 Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA) other conditions apply. Learn more about W3C Community and Business Groups.

If you wish to make comments regarding this document, please send them to public-epub3@w3.org (subscribe, archives).

1. Introduction

1.1 Relationship to Other Specifications

This section is non-normative.

1.1.1 Relationship to SMIL

This specification relies on a subset of [SMIL3], from which the EPUB Media Overlays elements and attributes defined in Media Overlay Document Definition are derived.

1.2 Terminology

Terms with meanings specific to EPUB 3 are capitalized in this document (e.g., "Author", "Reading System"). A complete list of these terms and definitions is provided in [EPUB32].

Only the first instance of a term in a section is linked to its definition.

1.3 Conformance

As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.

The key words MAY, MUST, MUST NOT, NOT REQUIRED, OPTIONAL, RECOMMENDED, REQUIRED, SHOULD, and SHOULD NOT are to be interpreted as described in [RFC2119].

1.4 Namespace prefix mappings

For convenience, the following namespace prefixes [XML-NAMES] are used in this specification without explicitly being declared. To use any of these prefixes, a declaration is REQUIRED.

prefix URI
epub http://www.idpf.org/2007/ops

2. Media Overlay Document Definition

2.1 Introduction

This section is non-normative.

Synchronized audio narration is found in mainstream ebooks, educational tools and ebooks formatted for persons with print disabilities. In EPUB 3, these types of books are created using Media Overlay Documents to describe the timing for the pre-recorded audio narration and how it relates to the EPUB Content Document markup. The file format for Media Overlays is defined as a subset of [SMIL3], a W3C recommendation for representing synchronized multimedia information in XML.

The Media Overlays feature is designed to be transparent to EPUB Reading Systems that do not support the feature. The inclusion of Media Overlays in an EPUB Publication has no impact on the ability of Media Overlay-unaware Reading Systems to render the EPUB Publication as though the Media Overlays are not present.

Although future versions of this specification might incorporate support for video media (e.g., synchronized text/sign-language books), this version supports only synchronizing audio media with the EPUB Content Document.

2.2 Content Conformance

A Media Overlay Document MUST meet all of the following criteria:

Document Properties

  It MUST meet the conformance constraints for XML documents defined in XML Conformance [EPUB32].

  It MUST be valid to the Media Overlays schema as defined in Appendix A, Media Overlays Schema and conform to all content conformance constraints expressed in Media Overlay Document Definition.

  It MUST be authored to reflect the structure of the EPUB Content Document with which it is associated, as stated in Structure.

  It MAY refer to more than one EPUB Content Document, but an EPUB Content Document MUST NOT be referenced by more than one Media Overlay Document.

  It MUST adhere to the requirements for Embedded Media.

  It SHOULD use semantic markup where appropriate, as described in Semantic Inflection.

  It MUST be packaged with the EPUB Publication as shown in Packaging.

File Properties

  The Media Overlay Document filename SHOULD use the file extension .smil.

2.3 Reading System Conformance

EPUB Reading System support for Media Overlays is OPTIONAL. A Reading System that supports Media Overlays MUST meet the following criteria:

A Reading System that does not support Media Overlays MUST meet the following criteria:

2.4 Media Overlay Document Definition

All elements [XML] defined in this section are in the https://www.w3.org/ns/SMIL namespace [XML-NAMES] unless otherwise specified.

2.4.1 The smil Element

The smil element is the root element of all Media Overlay Documents.

Element Name

smil

Usage

The smil element is the root element of the Media Overlay Document.

Attributes
version [required]

Specifies the version number of the [SMIL3] specification to which the Media Overlay adheres.

This attribute MUST have the value "3.0".

id [optional]

The ID [XML] of the element, which MUST be unique within the document scope.

epub:prefix [optional]

Declares additional metadata vocabulary prefixes.

Refer to Semantic Inflection for more information.

Content Model

In this order:

2.4.2 The head Element

The head element is the container for metadata in the Media Overlay Document.

Element Name

head

Usage

The head element is the OPTIONAL first child of the smil element.

Attributes

None

Content Model

metadata [0 or 1]

As this specification does not define any metadata properties that have to occur in the Media Overlay Document, the head element is OPTIONAL.

2.4.3 The metadata Element

The metadata element represents metadata for the Media Overlay Document. The metadata element is an extension point that allows the inclusion of metadata from any metainformation structuring language.

Element Name

metadata

Usage

As a child of the head element.

Attributes

None

Content Model

[0 or more] elements from any namespace

This specification defines no metadata properties that MUST occur in the Media Overlay Document; the metadata element is provided for custom metadata requirements.

2.4.4 The body Element

The body element is the starting point for the presentation contained in the Media Overlay Document. It contains the main sequence of par and seq elements.

Element Name

body

Usage

The body element is the REQUIRED second child of the smil element.

Attributes
epub:type [optional]

An expression of the structural semantics of the corresponding element in the EPUB Content Document.

The value is a white space separated list of property [Packages32] types. Refer to Semantic Inflection for more information.

id [optional]

The ID [XML] of the element, which MUST be unique within the document scope.

epub:textref [optional]

The relative IRI reference [RFC3987] of the corresponding EPUB Content Document, including a fragment identifier that references the specific element as per the [XPTRSH].

Content Model

In any order:

  • seq [0 or more]

  • par [0 or more]

At least one par or seq is REQUIRED.

2.4.5 The seq Element

The seq element contains media objects which are to be rendered sequentially.

Element Name

seq

Usage

One or more seq elements MAY occur as children of the body element and of the seq element.

Attributes
epub:type [optional]

An expression of the structural semantics of the corresponding element in the EPUB Content Document.

The value is a white space separated list of property [Packages32] types. Refer to Semantic Inflection for more information.

id [optional]

The ID [XML] of the element, which MUST be unique within the document scope.

epub:textref [required]

The relative IRI reference [RFC3987] of the corresponding EPUB Content Document, including a fragment identifier that references the specific element as per the [XPTRSH].

Refer to Structure for more information.

Content Model

In any order:

  • seq [0 or more]

  • par [0 or more]

At least one par or seq is REQUIRED.

2.4.6 The par Element

The par element contains media objects which are to be rendered in parallel.

Element Name

par

Usage

One or more par elements MAY occur as children of the body and seq elements.

Attributes
epub:type [optional]

An expression of the structural semantics of the corresponding element in the EPUB Content Document.

The value is a white space separated list of property [Packages32] types. Refer to Semantic Inflection for more information.

id [optional]

The ID [XML] of the element, which MUST be unique within the document scope.

Content Model

In any order:

The audio element is OPTIONAL only if its sibling text element refers to audio or video media (see Embedded Media), or to textual content intended for rendering via Text-to-Speech (TTS).

2.4.7 The text Element

The text element references an element in the EPUB Content Document. A text element typically refers to a textual element, but can also refer to other EPUB Content Document media elements (see Embedded Media).

Element Name

text

Usage

As a REQUIRED child of the par element.

Attributes
src [required]

The relative IRI reference [RFC3987] of the corresponding EPUB Content Document, including a fragment identifier that references the specific element as per the [XPTRSH].

id [optional]

The ID [XML] of the element, which MUST be unique within the document scope.

Content Model

Empty

2.4.8 The audio Element

The audio element represents a clip of audio media.

Element Name

audio

Usage

A REQUIRED child of the par element unless its sibling text element refers to audio or video media, or to textual content intended for rendering via Text-to-Speech (TTS), in which case it is OPTIONAL (see Embedded Media).

Attributes
id [optional]

The ID [XML] of the element, which MUST be unique within the document scope.

src [required]

The relative or absolute IRI reference [RFC3987] of an audio file. The audio file MUST be one of the audio formats listed in the Core Media Type Resources [EPUB32] table.

clipBegin [optional]

A clock value that specifies the offset into the physical media corresponding to the start point of an audio clip.

MUST be a [SMIL3] clock value.

See Appendix B, Examples of Clock Values.

clipEnd [optional]

A clock value that specifies the offset into the physical media corresponding to the end point of an audio clip.

MUST be a [SMIL3] clock value.

See Appendix B, Examples of Clock Values.

The chronological offset of the terminating position MUST be after the starting offset specified in the clipBegin attribute.

Content Model

Empty

3. Creating Media Overlays

3.1 Introduction

This section is non-normative.

A pre-recorded narration of a publication can be represented as a series of audio clips, each corresponding to part of the EPUB Content Document. A single audio clip, for example, typically represents a single phrase or paragraph, but infers no order relative to the other clips or to the text of a document. Media Overlays solve this problem of synchronization by tying the structured audio narration to its corresponding text (or other media) in the EPUB Content Document using [SMIL3] markup. Media Overlays are, in fact, a simplified subset of SMIL 3.0 that allow the playback sequence of these clips to be defined.

The SMIL elements primarily used for structuring Media Overlays are body (used for the main sequence), seq (sequence) and par (parallel). (Refer to Media Overlay Document Definition for more information on these and other SMIL elements.)

The par element is the basic building block of an Overlay and corresponds to a phrase in the EPUB Content Document. The element provides two key pieces of information for synchronizing content: 1) the audio clip containing the narration for the phrase; and 2) a pointer to the associated EPUB Content Document fragment. The par element uses two media element children to represent this information: an audio element and a text element. Since par elements render their children in parallel, the audio clip and EPUB Content Document fragment are played at the same time, resulting in a synchronized presentation.

The text element src attribute references the associated phrase, sentence, or other segment of the EPUB Content Document by its IRI reference. The audio element src attribute similarly references the location of the corresponding audio clip, and adds the OPTIONAL clipBegin and clipEnd attributes to indicate a specific offset within the clip.

par elements are placed together sequentially to form a series of phrases or sentences. Not every element of the EPUB Content Document will have a corresponding par element in the Media Overlay, only those relevant to the audio narration.

par elements can also be added to seq elements to define more complex structures such as parts and chapters (see Structure).

3.2 Relationship to the EPUB Content Document

Note

In this section, the EPUB Content Document is assumed to be an XHTML Content Document. While Media Overlays can be used with SVG Content Documents, playback behavior might not be consistent and therefore interoperability is not guaranteed.

3.2.1 Structure

The body of a Media Overlay Document consists of two elements: the par element and the seq element. The ordering of these elements MUST match the default reading order of the EPUB Content Document.

The par element represents phrases. Each element identifies a text and audio component to sychronize during playback.

The seq element represents sequences. It is used to represent nested containers such as sections, asides, headers, and footnotes. It allows the structure inherent in these containers to be retained in the Media Overlay Document.

The seq element MUST contain an epub:textref attribute. The IRI [RFC3987] value of this attribute MUST reference the corresponding structural element in an EPUB Content Document. As seq elements do not provide synchronization instructions, this attribute allows a Reading System to still match the element to a location in the text.

The following example shows a Media Overlay Document with nested seq elements, representing a chapter with both a section header and a figure.

<smil xmlns="http://www.w3.org/ns/SMIL" xmlns:epub="http://www.idpf.org/2007/ops" version="3.0">
    <body>

        <!-- a chapter -->
        <seq id="id1" epub:textref="chapter1.xhtml#sectionstart" epub:type="chapter">

            <!-- the section title -->
            <par id="id2">
                <text src="chapter1.xhtml#section1_title"/>
                <audio src="chapter1_audio.mp3"
                       clipBegin="0:23:23.84"
                       clipEnd="0:23:34.221"/>
            </par>

            <!-- some sentences in the chapter -->
            <par id="id3">
                <text src="chapter1.xhtml#text1"/>
                <audio src="chapter1_audio.mp3"
                       clipBegin="0:23:34.221"
                       clipEnd="0:23:59.003"/>
            </par>
            <par id="id4">
                <text src="chapter1.xhtml#text2"/>
                <audio src="chapter1_audio.mp3"
                       clipBegin="0:23:59.003"
                       clipEnd="0:24:15.000"/>
            </par>

            <!-- a figure -->
            <seq id="id7" epub:textref="chapter1.xhtml#figure">
                <par id="id8">
                    <text src="chapter1.xhtml#photo"/>
                    <audio src="chapter1_audio.mp3"
                           clipBegin="0:24:18.123"
                           clipEnd="0:24:28.764"/>
                </par>
                <par id="id9">
                    <text src="chapter1.xhtml#caption"/>
                    <audio src="chapter1_audio.mp3"
                           clipBegin="0:24:28.764"
                           clipEnd="0:24:50.010"/>
                </par>
            </seq>

            <!-- more sentences in the chapter (outside the figure) -->
            <par id="id12">
                <text src="chapter1.xhtml#text3"/>
                <audio src="chapter1_audio.mp3"
                       clipBegin="0:25:45.515"
                       clipEnd="0:26:30.203"/>
            </par>
            <par id="id13">
                <text src="chapter1.xhtml#text4"/>
                <audio src="chapter1_audio.mp3"
                       clipBegin="0:26:30.203"
                       clipEnd="0:27:15.000"/>
            </par>

        </seq>
    </body>
</smil>
Note

The reason for grouping structures like sections, figures, tables, and footnotes in a seq element is so that their start and end positions can be identified during playback. Reading Systems can then offer playback options tailored to the layout of the given Rendition, such as jumping past a long figure, turning off rendering of page break announcements (see Skippability and Escapability), or customizing the reading mode to suit structures such as tables.

3.2.2 Granularity

This section is non-normative.

Media Overlay text elements' src attributes refer to EPUB Content Document elements by their IDs [XML]. The granularity level of the Media Overlay therefore depends on how the EPUB Content Document is marked up. If the finest level of markup is at the paragraph level, then that is the finest possible level at which Media Overlay synchronization can be authored. Likewise, if sub-paragraph markup is available, such as [HTML] span elements representing phrases or sentences, then finer granularity is possible in the Media Overlay. Finer granularity gives users more precise results for synchronized playback when navigating by word or phrase and when searching the text, but increases the file size of the Media Overlay Documents.

3.2.3 Embedded Media

Any EPUB Content Document associated with a Media Overlay MAY contain embedded media such as video, audio, and images. The Media Overlay text element MAY be used in such instances to reference the embedded media by its ID [XML] value.

When a text element references embedded media that contains audio, the audio sibling element is OPTIONAL.

Authors SHOULD avoid using scripts to control playback of referenced embedded EPUB Content Document media, as this might conflict with Media Overlays playback behavior.

3.2.4 Text-to-Speech

This specification allows the use of Text-to-Speech (TTS) in addition to pre-recorded audio clips. When a Media Overlay text element with no audio sibling element references an element within the target EPUB Content Document, the contents of that referenced element MUST be appropriate for rendering via TTS. For example, it could be a textual EPUB Content Document element or contain a text fallback.

3.3 Semantic Inflection

In order to express semantic inflections, the epub:type attribute [ContentDocs32] MAY be attached to Media Overlay par, seq, and body elements.

Values for the Media Overlay epub:type attribute are constrained identically to the epub:type attribute in EPUB Content Documents. Refer to Semantic Inflection [ContentDocs32] for details.

The epub:type attribute facilitates Reading System behavior appropriate for the semantic type(s) indicated. Examples of these behaviors are Skippability and Escapability and Note.

This specification adopts the vocabulary association mechanisms defined in Vocabulary Association [ContentDocs32] unmodified. Terms from the default vocabulary MUST be used unprefixed in Overlay Documents.

Media Overlays MAY use additional vocabularies by defining them in the epub:prefix attribute on the root smil element.

3.4 Associating Style Information

Visual rendering information for the currently-playing EPUB Content Document element MAY be expressed in the CSS Style Sheet using author-defined classes. These author-defined class names SHOULD be declared in the Package Document metadata using the metadata properties active-class and playback-active-class. The class names are then discoverable by Reading Systems.

The active-class and playback-active-class properties MUST NOT be used in conjunction with a refines attribute [Packages32], as they are always considered to apply to the entire Rendition.

This example demonstrates how Authors can associate style information with the currently-playing EPUB Content Document.

Note

Although this example uses the class names -epub-media-overlay-active and -epub-media-overlay-playing, any class names are permitted. The class names chosen can be used along with any supported CSS features.

In this example, the Reading System would apply the author-defined -epub-media-overlay-active class to each text element in the EPUB Content Document as it became active during playback. Conversely, the class name is removed when the element is no longer active. The user would see each EPUB Content Document element styled with a yellow background for the duration of that element's playback.

The Reading System would also apply the author-defined -epub-media-overlay-playing class to the document element of the EPUB Content Document when Media Overlays playback begins. The class name is removed when playback stops. In the case of an XHTML Content Document, the class name would be applied to the html element. In the case of an SVG Content Document, it would be applied to the svg element. The user would see all the inactive text elements turn gray during Media Overlays playback. When playback stopped, the elements’ colors would return to their defaults.

3.5 Packaging

3.5.1 Including Media Overlays

If an EPUB Content Document is wholly or partially referenced by a Media Overlay, then its manifest item element [Packages32] MUST include a media-overlay attribute. The attribute MUST reference the ID [XML] of the manifest item for the corresponding Media Overlay Document.

The media-overlay attribute MUST NOT be attached to manifest item elements that do not reference EPUB Content Documents.

Manifest items for Media Overlay Documents MUST have the media type application/smil+xml.

3.5.2 Package Metadata

The Package Document MUST include the duration of the entire Rendition in a meta element with the duration property.

In addition, the duration of each EPUB Content Document with an associated Media Overlay MUST be provided. The refines attribute [Packages32] is used to associate each duration declaration to the corresponding manifest item [Packages32].

Authors also MAY include narrator information in the Package Document, as well as author-defined CSS class names to be applied to the currently-playing EPUB Content Document element.

Note

The prefix media: is reserved by [Packages32] for the inclusion of these properties in package metadata.

4. Playback Behaviors

4.1 Loading the Media Overlay

When an EPUB Reading System loads a Package Document, it MUST refer to the Manifest item elements' media-overlay attributes to discover the corresponding Media Overlays for EPUB Content Documents. Playback MUST start at the Media Overlay element which corresponds to the desired EPUB Content Document starting point. Note that the start of an EPUB Content Document MAY correspond to an element at the start or in the middle of a Media Overlay. When the Media Overlay Document has finished playing, the Reading System SHOULD load the next EPUB Content Document (as specified in the Package Document spine) and also load its corresponding Media Overlay Document, provided that one is given.

4.2 Basic Playback

4.2.1 Timing and Synchronization

Reading Systems MUST render immediate children of the body element in a sequence. A seq element's children MUST be rendered in sequence, and playback completes when the last child has finished playing. A par element's children MUST be rendered in parallel (with each starting at the same time), and playback completes when all the children have finished playing. When the body element's last child has finished playing, playback of the Media Overlay Document is done.

4.2.2 Rendering Audio

When presented with a Media Overlay audio element, Reading Systems MUST play the audio resource referenced by the src attribute, starting at the clip offset time given by the clipBegin attribute and ending at the clip offset time given by the clipEnd attribute. The following rules MUST be observed:

  • If clipBegin is not specified, its value is assumed to be "0".

  • If clipEnd is not specified, its value is assumed to be the full duration of the physical media.

  • If clipEnd exceeds the full duration of the physical media, then its value is assumed to be the full duration of the physical media.

User-controllable audio playback options SHOULD include timescale modification, in which the playback rate is altered without distorting the pitch. The suggested range is half-speed to double-speed.

4.2.3 Rendering EPUB Content Document Elements

When presented with a Media Overlay text element, Reading Systems SHOULD ensure the EPUB Content Document element referenced by the src attribute is visible in the Viewport. During Media Overlays playback, Reading Systems with a Viewport SHOULD add the class names given by the metadata properties active-class and playback-active-class to the appropriate elements in the EPUB Content Document. Conversely, the class names SHOULD be removed when the playback state changes, as described in Associating Style Information.

The active-class and playback-active-class metadata properties are OPTIONAL, and if omitted, Reading System behavior is implementation-specific.

4.3 Interacting with the EPUB Content Document

4.3.1 Navigation

Because the Media Overlay is closely linked to the EPUB Content Document, it is very easy for Reading Systems to locate a position in the EPUB Content Document based on the current position in the Media Overlay playback. If the user pauses synchronized playback and navigates to a different part of the EPUB Publication, synchronized playback MUST resume at that point. For example, if a specific page number in the EPUB Content Document is the desired location, then this same point is located in the Media Overlay and playback started there.

This same approach allows for synchronizing the Media Overlay playback with user selection of a navigation point in the EPUB Navigation Document. The Reading System loads the Media Overlay for that file and finds the correct point for starting playback based on the ID [XML] of the navigation point target.

Note

A Media Overlay Document can be associated directly with a Navigation Document in order to provide synchronized playback of its contents, regardless of whether the XHTML Content Document in which it resides is included in the spine. See EPUB Navigation Document for more information.

Note

Media Overlay Document elements can be associated with EPUB Content Document structures such as tables. Reading Systems need to ensure that Media Overlay playback remains synchronized with user navigation of table rows and cells. The Reading System might also play the corresponding table header preceding the contents of the cell.

4.3.2 Embedded Audio and Video

An EPUB Content Document with which a Media Overlay is associated MAY itself contain embedded video and audio media, which MAY be pointed to by Media Overlay elements. Unlike text and images, video and audio media has an intrinsic duration. Consequently, when a Reading System renders the synchronization described by a Media Overlay, the default playback behavior of audio and video media embedded within the associated EPUB Content Document MUST be overridden.

Note that the rules below apply only to referenced [HTML] video or audio elements within the associated EPUB Content Document. That is to say, the rules apply to only those elements pointed to by text elements within the Media Overlay (i.e., via the src attribute). Embedded media that is not referenced by Media Overlay elements is not subject to these rules.

  • All referenced audio and video media embedded within an EPUB Content Document MUST have their public playback interface deactivated (typically: play/pause control, time slider, volume level, etc.). This behavior is necessary to avoid interference between the scheduled playback sequence defined by the Media Overlay, and the arbitrary playback behavior due to user interaction or script execution. As a result, when the Reading System is in playback mode, it SHOULD:

    • Hide the individual video/audio UI controls from the page, which overrides the default behavior defined by the [HTML] controls attribute.

    • Prevent scripts embedded within the EPUB Content Document from invoking the JavaScript audio/video playback API (i.e., authored as part of the default behavior). It is RECOMMENDED that content producers avoid publishing embedded scripts dedicated to controlling the playback of embedded audio/video media. The published Media Overlay can then retain full control of the synchronized presentation without any risk of interference from script-enabled custom behaviors.

  • All referenced audio and video media embedded within an EPUB Content Document MUST be initialized to their "stopped" state, and be ready to be played from the zero-position within their content stream (possibly displaying the image specified using the [HTML] poster attribute). This requirement overrides the default behavior defined by the [HTML] autoplay attribute.

  • When an EPUB Content Document element becomes active, the CSS Style Sheet visual highlighting rules apply regardless of the content type referred to by that element's src attribute (e.g., the CSS class name defined by the active-class metadata property SHOULD be applied to visible video and audio player controls within the host EPUB Content Document).

  • In addition to the default behavior of Media Overlay activation for textual fragments and images, audio and video playback MUST be started and stopped according to the duration implied by the authored Media Overlay synchronization (as per the standard [SMIL3] timing model). There are two possible scenarios:

    • When a Media Overlay text element has no audio sibling within its par parent container, the referenced EPUB Content Document audio or video media MUST play until it ends, at which point the text element's lifespan terminates. In this case, the implicit duration of the text element (and by inference, of the parent par container) is that of the referenced audio or video clip.

    • When a Media Overlay text element has an audio sibling within its par parent container, the playback duration of the referenced EPUB Content Document audio or video media MUST be constrained by the duration of the audio sibling. In this case, the actual duration of the parent par container is that of the child audio clip, regardless of the duration of the video or audio media pointed to by the text element. This behavior can result in embedded video or audio media ending playback prematurely (before reaching its full duration), or ending before the playback of the parallel Media Overlay audio is finished (in which case the last-played video frame SHOULD remain visible until the parent par container finally ends). This behavior is equivalent of the Media Overlay audio element implicitly carrying the behavior of the [SMIL3] endsync attribute.

      Furthermore, Reading Systems SHOULD expose user controls for the volume levels of each independent audio track (i.e., from the audio element of the Media Overlay, and from the embedded audio or video media within the EPUB Content Document), so that audio output can be adjusted to match listeners' requirements. Note that having overlapping audio tracks is typically an authoring-time concern: content producers usually add a layer of audio information over a video track for description purposes. It is RECOMMENDED that overlapping audio situations are carefully examined and dealt with at production stage, as Reading Systems are NOT REQUIRED to handle simultaneous volume levels in any particular way.

  • When a text element becomes inactive in the Media Overlay, and when it points to embedded video or audio media, that referenced media MUST be reset to its initial "stopped" state, ready to be played from the zero-position within their content stream (possibly displaying the poster image specified using the [HTML] markup).

4.3.3 Text-to-Speech

When a Media Overlay text element with no audio sibling element references text within the target EPUB Content Document, Reading Systems capable of Text-to-Speech (TTS) SHOULD render the referenced text using TTS.

As per Reading System conformance requirements, the speech-related information provided in the target EPUB Content Document SHOULD be used to play the audio stream as part of the Media Overlay rendering. See Reading System Text-to-Speech Conformance Requirements [EPUB32].

The Media Overlay text element's lifespan corresponds to the rendering time of the associated speech synthesis. The implicit duration of the text element (and by inference, of the parent par element) is therefore determined by the execution of the Text-to-Speech engine, and cannot be known at authoring time (factors like speech rate, pauses and other prosody parameters influence the audio output).

4.4 Skippability and Escapability

4.4.1 Skippability

While reading, users might want to turn on or off certain features of the content, such as footnotes, page numbers, or other types of secondary content. This feature is called skippability. Reading Systems SHOULD use the semantic information provided by Media Overlay elements' epub:type attribute to determine when to offer users the option of skippable features.

The following non-exhaustive list represents terms from the [EPUB-SSV] for which Reading Systems might offer the option of skippability:

  • footnote

  • endnote

  • pagebreak

Reading System support for skippability based on epub:type values SHOULD NOT be assumed.

4.4.2 Escapability

Escapable items are nested structures, such as tables and lists, that users might wish to skip over, continuing to read from the point immediately after the nested structure. The escapability feature differs from the skippability feature in that it does not enable or disable entire types of items, but provides an exit from them (e.g., a user can listen to some of the content before choosing to escape).

Reading Systems SHOULD allow escaping of nested structures. Reading Systems MUST determine the start of nested structures by the value of the epub:type attribute and SHOULD offer users the option to skip playback of that structure and resume with whatever content comes after it.

The following non-exhaustive list represents terms from the [EPUB-SSV] for which Reading Systems might offer the option of escapability:

  • table

  • table-row

  • table-cell

  • list

  • list-item

  • figure

5. EPUB Navigation Document

The EPUB Navigation Document is an XHTML Content Document and can therefore be associated with an audio Media Overlay. Unlike traditional XHTML Content Documents, however, Reading Systems have to present the EPUB Navigation Document to users even when it is not included in the spine (see EPUB Navigation Documents — Reading System Conformance [Packages32]). As a result, the method in which an associated Media Overlay behaves can change depending on the context:

Note

Specific implementation details are beyond the scope of this specification. The DAISY Media Overlays Playback Requirements document describes best practices for Authors, and provides recommendations for Reading System developers.

A. Media Overlays Schema

The schema for Media Overlays is available at https://github.com/w3c/epubcheck/tree/master/src/main/resources/com/adobe/epubcheck/schema/30/media-overlay-30.nvdl.

Validation using this schema requires a processor that supports [NVDL], [RelaxNG-Schema], [ISOSchematron] and [XMLSCHEMA-2].

Note

The NVDL schema layer can be substituted by a multi-pass validation using the embedded RELAX NG and ISO Schematron schemas alone.

B. Examples of Clock Values

This section is non-normative.

The following are examples of allowed clock values:

C. Media Overlays Metadata Vocabulary

C.1 Overview

C.1.1 About this Vocabulary

This section is non-normative.

The properties in this vocabulary are usable in the meta element's property attribute [Packages32].

C.1.2 Referencing

The base IRI for referencing this vocabulary is http://www.idpf.org/epub/vocab/overlays/#.

The prefix "media:" is reserved for use [Packages] with properties in this vocabulary and does not have to be declared in the Package Document.

C.2 Media Overlays Properties

C.2.1 active-class

Name: active-class
Description: Author-defined CSS class name to apply to the currently-playing EPUB Content Document element.
Allowed value(s): xsd:string
Cardinality: Zero or one
Example: <meta property="media:active-class">-epub-media-overlay-active</meta>

C.2.2 duration

Name: duration
Description: The duration of the entire presentation or of a specific Media Overlay. The specified durations account for the audio clips known at authoring time, and so exclude live streaming from external resources and speech synthesis.
Allowed value(s):

MUST be a [SMIL3] clock value.

Cardinality: Exactly one for a given Rendition and for each Media Overlay.
Example: <meta property="media:duration">1:36:20</meta>

C.2.3 narrator

Name: narrator
Description: Name of the narrator.
Allowed value(s): xsd:string
Cardinality: Zero or more
Example: <meta property="media:narrator">Joe Speaker</meta>

C.2.4 playback-active-class

Name: playback-active-class
Description: Author-defined CSS class name to apply to the EPUB Content Document's document element when playback is active.
Allowed value(s): xsd:string
Cardinality: Zero or one
Example: <meta property="media:playback-active-class">-epub-media-overlay-playing</meta>

D. Acknowledgements and Contributors

This section is non-normative.

EPUB 3 is developed by the W3C's EPUB 3 Community Group in coordination with the Publishing Business Group.

The EPUB 3.2 revision was led by:

In addition to the editors, this version of EPUB would not have been possible without significant contributions from:

Special thanks go to the former members of the International Digital Publishing Forum, particularly Markus Gylling and Bill McCoy, without whom EPUB would not have become a reality.

E. References

E.1 Normative references

[ContentDocs32]
EPUB Content Documents 3.2. URL: epub-contentdocs.html
[EPUB-SSV]
EPUB Structural Semantics Vocabulary. IDPF. URL: http://www.idpf.org/epub/vocab/structure/
[EPUB32]
EPUB 3.2. URL: epub-spec.html
[HTML]
HTML. W3C. W3C Recommendation. URL: https://www.w3.org/TR/html/
[ISOSchematron]
ISO/IEC 19757-3: Rule-based validation — Schematron. 2006-06-01. URL: http://standards.iso.org/ittf/PubliclyAvailableStandards/c040833_ISO_IEC_19757-3_2006(E).zip
[NVDL]
ISO/IEC 19757-4: NVDL (Namespace-based Validation Dispatching Language). 2006-06-01. URL: http://standards.iso.org/ittf/PubliclyAvailableStandards/c038615_ISO_IEC_19757-4_2006(E).zip
[Packages]
EPUB Packages 3. URL: http://www.idpf.org/epub3/latest/packages
[Packages32]
EPUB Packages 3.2. URL: epub-packages.html
[RelaxNG-Schema]
Information technology -- Document Schema Definition Language (DSDL) -- Part 2: Regular-grammar-based validation -- RELAX NG. ISO/IEC. 2008. URL: http://standards.iso.org/ittf/PubliclyAvailableStandards/c052348_ISO_IEC_19757-2_2008(E).zip
[RFC2119]
Key words for use in RFCs to Indicate Requirement Levels. S. Bradner. IETF. March 1997. Best Current Practice. URL: https://tools.ietf.org/html/rfc2119
[RFC3987]
Internationalized Resource Identifiers (IRIs). M. Duerst; M. Suignard. IETF. January 2005. Proposed Standard. URL: https://tools.ietf.org/html/rfc3987
[SMIL3]
Synchronized Multimedia Integration Language (SMIL 3.0). Dick Bulterman. W3C. 1 December 2008. W3C Recommendation. URL: https://www.w3.org/TR/SMIL3/
[XML]
Extensible Markup Language (XML) 1.0 (Fifth Edition). Tim Bray; Jean Paoli; Michael Sperberg-McQueen; Eve Maler; François Yergeau et al. W3C. 26 November 2008. W3C Recommendation. URL: https://www.w3.org/TR/xml/
[XML-NAMES]
Namespaces in XML 1.0 (Third Edition). Tim Bray; Dave Hollander; Andrew Layman; Richard Tobin; Henry Thompson et al. W3C. 8 December 2009. W3C Recommendation. URL: https://www.w3.org/TR/xml-names/
[XMLSCHEMA-2]
XML Schema Part 2: Datatypes Second Edition. Paul V. Biron; Ashok Malhotra. W3C. 28 October 2004. W3C Recommendation. URL: https://www.w3.org/TR/xmlschema-2/
[XPTRSH]
XPointer Shorthand Notation. 25 March 2003. URL: https://www.w3.org/TR/2003/REC-xptr-framework-20030325/

E.2 Informative references

[EPUB32Changes]
EPUB 3.2 Changes. URL: epub-changes.html