<kaz> Meeting: Media and Entertainment IG - Media Timed Events TF
<scribe> scribe: tidoust
<inserted> https://docs.google.com/presentation/d/1KPX8OYQzKbNXG_AqpDgktMkS9aKwei_ffj_3G0E3ygk/edit Slides
cpn: Two calls in one. IG Note,
the document is, I believe, complete. Needs for improved
synchronization capabilities were pursued in WHATWG, and then
datacue for need to expose media timed events (in-band or
not).
... Question that remains is whether we believe there is more
for the TF to do.
... Second topic is the Datacue incubation itself.
... I'd like feedback to guide the specification.
cpn: [overview of spec
sections]
... In the last few weeks, we've done some work to address
remaining issues. Everything's merged, no open issue at the
moment.
... Do we think from our TF perspective that this document is
complete?
... I'd like us to focus our attention on the DataCue API
itself as next stage.
... Assuming you agree, I'd like to send a Call for Consensus
to the IG, which we need to publish an updated IG Note.
... Peter, it may be worth hearing about your use case, as it
may be something that needs to be added to the document.
... Do others who contributed to the document think that it is
complete?
RobSmith: I think that it is very complete.
[No other comment]
cpn: OK, let's go through your use case then, Peter.
PeterThoPesch_IRT_: I don't think
that there may be additional requirements derived from this use
case.
... The use case is about graphical elements that we would like
to transport separately from the video. Things like TV graphics
where the name of a guest is added to the picture, or sports
graphics.
... For the time being, these graphics are integrated in the
video and streamed to all devices.
... But video ratio may depend on the device (landscape,
portrait, 1:1). We'd like to make the video experience more
responsive by separating the video and the graphics which could
then be adapted to the video ratio and positioned
correctly.
... We didn't find any proper solution except to use text
tracks to connect that with the video.
cpn: How do you expect to deliver the images?
PeterThoPesch_IRT_: That's a
different problem. We could imagine having an HTML/CSS
animation on top of the video, or some (proprietary) JSON
format to describe the graphic.
... There's one company I know who does that with a proprietary
JSON format to render HTML content.
... Also example of body moving in Adobe.
cpn: I was wondering whether you were considering web resources tracks that go inside your MP4 container.
PeterThoPesch_IRT_: That could be
a solution. And then it would be synchronized by DASH
streams.
... One problem is that the player may not be Web-based. It may
be a native player.
... There may be multiple delivery channels.
cpn: It sounds like DataCue as
we're discussing here could be a solution to this, as a way to
associate metadata with a position in the media timeline.
... The other part of the solution would be the actual
rendering which is something that we touch on in this document
in the gap analysis around synchronized rendering.
... The "requestVideoFrameCallback" proposal would help.
PeterThoPesch_IRT_: Synchronization needs are probably stronger for graphical elements than regular text captioning. There may be vector graphic animations for instance.
Description of responsive graphics synced with video use case
PeterThoPesch_IRT_: I don't know if we should add the use case to the document.
cpn: The requirements are covered
although use cases are slightly different.
... Part of me wants to conclude work on this document.
... If you feel strongly that this should be included, we
can.
PeterThoPesch_IRT_: I trust that
the requirements are covered. I could perhaps draft a section
by tomorrow noon?
... If it's going to slow things down, I would say no.
cpn: It doesn't slow things down
by a lot. Not a big problem, really.
... The main thing is that I want to capture the correct
recommendations. Please check the part on rendering.
PeterThoPesch_IRT_: OK, I'll look into it by noon tomorrow.
<kaz> https://www.w3.org/TR/2011/NOTE-hnreq-20111201/
tidoust: +1 to including the use case, it seems more mainstream than the use case that's in the document that covers the same needs.
kaz: OK to include, wondering
about the title of the document.
... E.g. "Requirements for Media Timed Events"
... Also consider a shortname update, maybe?
tidoust: doable but a slight pain given that document has already been published to /TR/
rob: Another way to capture the use case would also be to open an issue in the DataCue repo
cpn: Right, I agree with including the use case in the IG Note. We need to be a little bit careful about the explainer of DataCue
cpn: We haven't started to draft
a spec yet. Focus is on the explainer. In the next week or so,
I hope to have a first draft of a spec.
... We have a good collaboration starting now with the
DASH-IF.
... [showing an architecture diagram from DASH-IF]
... You can see how the DataCue API interfaces with the rest,
with timed event buffers, the manifest parser.
... The DASH player can subscribe to receiving specific timed
events. And then we may have events that are specific to a Web
application. The Web application can subscribe to specific
media timed events.
... "onreceive" lets you know that an event has been received
in advance, allowing to do things such as fetching external
resources before the event takes place.
... "onstart" is triggered when the event fires.
... Also worth looking at a "Type 1" player, which is not
standardized at W3C, but is common. Any API that we come up
with, I'd like us to support both Type 3 and Type 1, although
Type 1 is more the focus.
... Anyone that wants Type 1 integration would likely have to
specify it outside of W3C.
... That's the architectural overview of where I think the
DataCue API fits.
-> Updating cue time attributes can generate unexpected events (#19)
<kaz> Issue 19
cpn: Regarding issue #19, there
is this potential case where, if you change the timing of a
cue, the platform may trigger duplicate "enter" and "exit"
events.
... The current outcome is that maybe we don't need to propose
a change to the HTML spec for now, but need to be something
that we keep in our heads when we specify the API.
RobSmith: That's correct.
cpn: Thanks a lot for the description. That crystallizes things quite nicely.
DataCue or a more specific API for DASH and emsg events
cpn: Introduce TextTrackCue.text,
DataCue blob data, and then thinking a DASH events in
particular, should we have a dedicated emsg cue
interface?
... TextTrackCue does not have a constructor, and there is no
way to attach the actual data payload.
... VTTCue has a text string which you could use to carry the
data, and a constructor. But it also has a bunch or positioning
properties that we don't need for DataCue. Not a very good fit
for the requirements that we have.
... What remains is the DataCue interface.
... As specified in HTML5, it would have a "data" attribute,
and then extensions implemented in WebKit: constructor, the
actual payload in a "value" attribute and the type of payload
that is being carried in a "type" attribute.
... It would be interesting to see how the mapping between
"type" and "value" works in practice. ID3 tags are supported in
WebKit, I believe, for instance.
... Another option would be to create an EmsgCue
interface.
... Similar to DataCue, you'd have a scheme ID URI to identify
the type of value.
... I don't think that we need a specific EmsgCue. The DataCue
API, as specified, would provide the right abstraction provided
that we define the "type" mapping properly.
RobSmith: I have a limited understanding of the emsg interface, but my feeling is that it's a good example of actual data, so including that in the DataCue design seems important.
cpn: In the explainer document, there is a section about mapping between emsg events and DataCue.
<kaz> DataCue Explainer
<PeterThoPesch_IRT_> How can I put myself on the speaker queue?
cpn: I think this can work, and I know that the DASH-IF group has some very useful work that we can point at that talks about timing scales (timing is represented differently in emsg events)
<PeterThoPesch_IRT_> ok, thanks :-)
cpn: Can we align closely with
Webkit implementation.
... Or should we depart. Feedback from Webkit developers would
be welcome.
RobSmith: Also a good way to
testing out their design. I'm wary of reinventing something
that's already been done. If we can reuse, that would be
great.
... Also, application-generated vs. user-agent generated cues
is going to be an interested use case.
cpn: Yes, I'm seeing the
application generated use case, as something that would be
correctly supported by the DataCue API as designed here.
... The "any value" would cover all possible needs.
... The emsg boxes and in-band events are where I see
complexity in defining the API and processing rules.
... The "data" and "value" attributes are redundant and
possibly confusing. There may be a need to preserve that.
Conversation to be had with implementers.
... I encourage you to look at open issues
<PeterThoPesch_IRT_> Just a thought: Might there be an event for seeking? Like onTimelineDiscontinued? If so, would that affect the solution?
cpn: A note to say that W3C members are welcome to DASH-IF calls every second Friday.
Ali: Just to confirm you're welcome. We'll also probably have updates based on ongoing MPEG discussions.
cpn: May, 18th 2020.
... One call per month. We can revise that if that feels
needed.
This is scribe.perl Revision: 1.154 of Date: 2018/09/25 16:35:56 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00) Succeeded: i/Two/topic: Agenda Succeeded: s/redudant/redundant/ Succeeded: i/Two/https://docs.google.com/presentation/d/1KPX8OYQzKbNXG_AqpDgktMkS9aKwei_ffj_3G0E3ygk/edit Slides Present: Gary_Katsevman Peter_Tho_Pesch Francois_Daoust Ali_Begen Rob_Smith Kaz_Ashimura Chris_Needham John_Simmons So_Vang Found Scribe: tidoust Inferring ScribeNick: tidoust Agenda: https://lists.w3.org/Archives/Public/public-web-and-tv/2020Apr/0024.html WARNING: No date found! Assuming today. (Hint: Specify the W3C IRC log URL, and the date will be determined from that.) Or specify the date like this: <dbooth> Date: 12 Sep 2002 People with action items: WARNING: IRC log location not specified! (You can ignore this warning if you do not want the generated minutes to contain a link to the original IRC log.)[End of scribe.perl diagnostic output]