Web Animations/Play Control Review

Simplified Time Sources Review

This is a review of: https://docs.google.com/document/d/1jntqP6E8MxD2CS89ggo6Zyhl2kmjzVYTFpiQdW5-FDE/edit#heading=h.824qgh7xkpyw (I'm not sure if that link is public, but if it's not, hopefully someone will make it so and update this!)

Strong points:

We need to resolve play control on nested elements and restricting it to the root of the tree is a solid solution.
Having a different kind of object at the root of the tree makes the relationship of play control clear—there is only one point where you can pause/play things. No unexpected side effects.

Concerns with TimedEffect approach:

Play control behaviour is dependent on properties of the TimedEffect
More moving parts—initial learning curve is steeper and useability may be impacted

Play control behaviour is dependent on properties of the TimedEffect

Play control is based on the time of an item after applying its start time since we decided that currentTime is zero at the item's start time (See http://lists.w3.org/Archives/Public/public-fx/2012JulSep/0049.html item 2).

Also, we decided that pausing doesn't take effect until the earliest of two moments: the start time or when the animation interval begins (See http://lists.w3.org/Archives/Public/public-fx/2012JulSep/0117.html item 5). As a result pausing depends on the start delay too.

We can't move start time and start delay from TimedEffect to Animation since each child can provide its own start time and delay (the former is necessary for SVG integration and the latter for a wide range of effects).

So that leaves the play control calculations in Animation dependent on a (nullable) member. These calculations will need to be written in terms of checking if timedItem is set and then poking into its properties.

This seems like an awkward place to break the classes apart. It prevents, for example, Animation taking multiple TimedItems in a future version.

Learning curve and useability

For the simple case of creating an animation there are two moving parts: Animation and TimingEffect. This makes simple things a little more complex. For example to modify an animation effect after creating, the chain is anim.timedItem.effect.

And how does the constructor look? Do you have:

var anim = document.createAnimation(new TimedEffect(elem, { anim }, 3));

We could make a constructor that flattened this, but then what about adding anim to group? Would it be something like:

group.appendChild(anim.timedItem)

Again, we could add convenience behaviour there to take an Animation but I'm concerned we'd be adding a lot of this convenience behaviour which undermines the concepts this architecture is supposed to emphasise.

As a further example, walking up the tree from a given TimedEffect (I expect the result of Element.getActiveAnimations will return the specific thing that is animation the element not the root of an arbitrarily complex tree) to get to the play control is probably inconvenient. The proposal mentions the possibility of adding remoting methods to simplify this but once again, doing this would obscure the relationship of play control and the primary benefit of this approach would be undermined.

Adding a animation property might work if it returned an AnimationProxy object that remained valid after re-parenting.

API research

In seeking alternative approaches to this, I have prepared a summary of how other APIs tackle this problem.

Alternative proposal

In summary:

Move play control to an interface that TimedItem implements
Play control behaves differently for TimedItems slaved to a controller/group
(Possibly) rename groups to controllers

Rationale

This proposal is based on the principle of, "It should be easy to do simple things; possible to do complex things; and impossible, or at least difficult, to do wrong things."[1]

Given that most animation script libraries do not provide grouping (none of the surveyed script libraries provided it although some had queues which are like sequence groups), this can be considered a more advanced use. The case of creating a simple animation and playing it possibly accounts for 80% or more of usage.

With this proposal, such usage is simple:

 var anim = document.createAnimation(elem, { width: 10% }, 3);
 anim.play();

When authors come to use groups they may encounter an additional concept: play()/pause() behave differently on a child of a group. Authors only need learn this if they use groups and only if they try to play()/pause() children. As a result the learning curve is gradual.

Class diagram

One possibility. See notes below.

Changes

Move play control to an interface on TimedItem

This is primarily for didactic purposes. It is not important to the operation of the model. Having a separate interface simply emphasises that this is a different facet of the item's operation.

As an interface, TimedItem's 'implement' its behaviour—they don't inherit it. As a result it is quite reasonable that TimedItem.play() and TimedItem.pause() depend on the state of TimedItem (in particular its startTime and startDelay).

It might make sense to even have just 'Animation' and 'TimingGroup' implement PlayControl individually but I'm not sure that's necessary.

Play control behaves differently for TimedItems slaved to a controller/group

There are a few possible courses of action with regards to play control on TimedItems that have a parent TimeController/TimeGroup:

Ignore it: pause(), play(), seeking, etc. all either silently fail or throw an exception. This causes less unexpected side effects since an author can't accidentally pause a whole tree.
Remote it: pause(), play(), reverse() etc. basically walk up the tree and call the equivalent method on the rootmost TimedItem. Seeking, however, won't work due to non-invertible time transformations so it would either work by approximation or would just throw.
Partially allow it: This is basically what HTML does. pause() works; play() catches up; seeking throws; playbackRate is ignored; On further investigation, I actually think this might be quite straightforward. The main feature is that play() catches up so you don't end up with a distinction between 'scheduled' time and 'runtime' time.

A further possibility is a combination of the first two options where, for example, the PlayControl interface includes both pause (ignored when parent is a group), and pauseTree (does the remoting).

I think all options are reasonable but, unless I'm missing some insurmountable problem with matching HTML's behaviour, that would be my preference. Otherwise you have a situation where different kinds of controllers dictate different limitations on the play control of their children. It's simpler if you can just say, "play() does A, unless the item is slaved to a controller, in which case it does B."

(Possibly) rename groups to controllers

If having play control behave differently for items attached to a group is considered confusing we could potentially rename TimingGroups to TimingControllers. This provides a more obvious parallel with HTML and perhaps expresses the fact that these items are slaved to a controller, not merely bundled up.

I prefer not making this change but it's ok.

For reference, here are what other APIs call group-like things:

QML: ParallelAnimation/SequenceAnimation
CA: CAAnimationGroup
Android: AnimationSet
WPF: TimelineGroup/ParallelTimeline/etc.

Integration with media

There are three cases to consider:

controlling HTMLMediaElements with Web Animations groups
controlling HTMLMediaControllers with Web Animations groups
controlling Web Animations animations (and possibly groups) with HTMLMediaControllers

Controlling HTMLMediaElements with Web Animations groups

Just set HTMLMediaElement.controller to the WebAnimations TimingController. If we choose to mimic the behaviour of slaved HTMLMediaElements with regards to how play(), pause(), seeking etc. work then the only thing left to define with regards to play control would be to import the blocking behaviour so that buffering media blocks the controller.

Otherwise we'd need to consider what happens when you pause a video in such a case. If we say that play() is ignored, then the UA would probably need to make sure the UI for a video element slaved to a TimingController did the remoting.

Regarding actual playback, HTMLMediaElements don't actually read the time samples per se. Instead there is a process that produces a time offset and an effective (average) playbackRate and sets that on the media element. This is because not many implementations will be able to apply an easing function to the playback of video/audio. We would probably also not require implementations to play things backwards.

Controlling HTMLMediaControllers with Web Animations group

I think we can safely just not support this. Media controllers don't nest.

Controlling Web Animations animations (and possibly groups) with HTMLMediaControllers

How important is this? I suspect we could not support this.

If we decide to support this, we could add:

 TimedItem.attachTo((TimeSource or MediaController) parent)

When the parent is a TimeGroup is simply appends to the groups children. (Alternatively we could actually make it just track the group's time without becoming a child in the sense of affecting group layout.)

This has the disadvantage that there are two ways to do tree surgery—from the parent TimingGroup/TimingController interfaces (which are possibly needed for controlling position within the group) or from the child. However, it has the advantage of providing an easy point of extension for other gesture-based time sources (although we can easily add that later).

 Although it would be preferably if tree surgery was done purely through the TimingController/TimingGroup interfaces we need a way to attach animations to HTMLMediaControllers.

But, I suspect we can leave this feature out for the time being anyway.

Commentary

Strong points:

Simple things are simple. Complex things are possible.
Consistency with HTML. (APIs must coexist peacefully with the platform, so do what is customary.[2])

Concerns:

Play control is conflated with animation definition
Methods do not behave identically in all situations

Play control is conflated with animation definition

Play control can be thought of independently of the (static) definition of a timed effect. Lumping them together in the one interface will create confusion with authors.

a) I think this is of minor concern. I have never heard anyone say, "It's weird that I can pause an animation" and the survey of other animation APIs reveals that putting play control directly on the animation is commonplace. In the survey, WPF is the only exception.

b) I have tried to address this by separating out PlayControl into a separate interface. The primary purpose of this separation is didactic.

Methods do not behave identically in all situations

HTML already has this so from a learning point of view, it should not be surprising.

We can make the slaving behaviour more obvious by renaming attach() (if we have it) to slaveTo(); or TimingGroup to TimingController (SequenceController; ParallelController etc.)

Update

A few additional thoughts:

Do we need to be able to cancel an item in the hierarchy without cancelling the whole tree? E.g. Element.cancelAllAnimations?
Do we need a 'fast-forward to end' feature? A lot of APIs have this. e.g. finish() which seeks to end of active interval?
PauseControl could be extended with attributes like isSlaved:bool or controller:PauseControl so you could pause the timed item only using anim.pause() or the whole tree using anim.controller.pause(). Or even anim.group.pause()? Again, it might be better if this returned PauseControlProxy.

Update #2

Alternative proposal #2:

Have a separate PlayController object
- One per 'tree'
Animations / groups have controller member that returns a PlayControllerProxy—it tracks the PlayController associated with the timed item even when re-parented
Animations have a start() method. This is what you call when you first create an animation.
- Not sure about the semantics here when adding to a group etc., but for simple animations you create them, then call start
play() and pause() are on the controller
TimedItem.currentTime is read-only
PlayController.currentTime is writeable
Run-time playbackRate control (the version that does a compensatory seek) is on the controller
Setting TimedItem.playbackRate causes 're-layout'
Disadvantages: Different to HTML media stuff

Update #3

SVG's requirements introduces some interesting restrictions:

Firstly, I'm pretty sure we want to be able to pause/seek/reverse root-level SVG <animate>/<par>/<seq>-produced animations.
However, <svg> elements also have their own play control:

  pauseAnimations()
  unpauseAnimations()
  animationsPaused()
  getCurrentTime()
  setCurrentTime()

This applies to root-level <svg> elements:

Since an ‘svg’ is sometimes the root of the XML document tree and other times can be a component of a parent XML grammar, the document begin for a given SVG document fragment is defined to be the exact time at which the ‘svg’ element's SVGLoad event is triggered. The document end of an SVG document fragment is the point at which the document fragment has been released and is no longer being processed by the user agent. However, nested ‘svg’ elements within an SVG document do not constitute document fragments in this sense, and do not define a separate document begin; all times within the nested SVG fragment are relative to the document time defined for the root ‘svg’ element. (http://www.w3.org/TR/SVG11/animate.html#DocumentBegin)

Basically, each rootmost SVG fragment establishes a new independently seekable/pausable group.

It's not really clear what nested <svg> elements do in terms of play control. Times are relative to the root <svg> but what about play control?

Perhaps the most important question here is implementation status:

Test	Chrome 25	Safari (Windows) 5.1.7	Opera 12.12	Firefox 20
Pausing independent SVG fragments in HTML	OK	n/a (HTML5 parser not supported?)	OK	Ok
Seeking independent SVG fragments in HTML	OK	n/a	OK	OK
Pausing nested SVG in SVG	Pause applies separately. Pausing root SVG does not pause SVG nested and vice versa.	Pause applies separately. Pausing root SVG does not pause SVG nested and vice versa.	Pause only applies on root SVG, pause on nested SVG is remoted to root SVG.	Pause only applies on root SVG, ignored on nested SVG.
Seeking nested SVG in SVG	Seeking applies separately. Seeking root SVG does not seek nested SVG and vice versa.	n/a. Seeking not supported.	Seeking only applies on root SVG, seek on nested SVG is remoted to root SVG.	Seeking only applies on root SVG, ignored on nested SVG.
Pausing independent SVG children in SVG	Pausing applies separately. Pausing one nested SVG does not pause the other.	Pausing applies separately. Pausing one nested SVG does not pause the other.	Pause on nested SVG is remoted to root, so pausing one nested SVG pauses the other.	Pause on nested SVG is ignored so pausing a nested SVG has no effect.
Seeking independent SVG children in SVG	Seeking applies separately. Seeking on nested SVG does not seek the other.	n/a. Seeking not supported.	Seeking a nested SVG is remoted to root, so seeking one nested SVG seeks the other.	Seeking a nested SVG is ignored so seeking a nested SVG has no effect.

Basically WebKit treats nested SVG as entirely independent time groups whilst Presto and Gecko adopt nested SVG into the rootmost SVG's time group which seems to be what the spec requires. Presto and Gecko differ on how they treat play control on nested SVG.

So basically, I think we need two levels of play control for SVG. Outermost SVG element level play control plus play control for SVG elements that don't have a parent group.

How to solve?

Add another type of root-level container that doesn't repeat but whose children can have play control?