IRC log of datacue on 2019-09-18

Timestamps are in UTC.

01:42:11 [RRSAgent]
RRSAgent has joined #datacue
01:42:11 [RRSAgent]
logging to https://www.w3.org/2019/09/18-datacue-irc
01:42:17 [dom]
RRSAgent, make log public
01:45:02 [dom]
Meeting: DataCue and "Time marches on" in HTML
01:50:36 [koalie]
koalie has joined #datacue
01:50:42 [koalie]
koalie has changed the topic to: https://w3c.github.io/tpac-breakouts/sessions.html
01:50:45 [koalie]
koalie has left #datacue
01:56:37 [takio]
takio has joined #datacue
01:57:22 [dom]
dom has joined #datacue
01:58:34 [romain]
romain has joined #datacue
01:58:38 [cyril]
cyril has joined #datacue
01:58:46 [cyril]
rrsagent, start meeting
01:58:46 [RRSAgent]
I'm logging. I don't understand 'start meeting', cyril. Try /msg RRSAgent help
01:58:56 [romain]
present+
01:59:01 [gkatsev]
gkatsev has joined #datacue
01:59:04 [cyril]
Chair: Chris Needham
01:59:08 [cyril]
scribe: Cyril
02:00:16 [cyril]
meeting: DataCue and "Time marches on" in HTML
02:00:21 [gkatsev]
present+
02:00:22 [pal]
pal has joined #datacue
02:00:25 [cyril]
rrsagent, draft minutes
02:00:25 [RRSAgent]
I have made the request to generate https://www.w3.org/2019/09/18-datacue-minutes.html cyril
02:00:27 [chcunningham]
chcunningham has joined #datacue
02:00:27 [pal]
present+
02:00:30 [chcunningham]
present+
02:00:44 [ericc]
ericc has joined #datacue
02:00:46 [jkamata]
jkamata has joined #datacue
02:00:57 [ericc]
present+
02:01:19 [stepsteg]
stepsteg has joined #datacue
02:01:24 [cyril]
chris: topics are discuss the DataCue API and the Time marches on algorithm
02:01:33 [cyril]
Topic: DataCue API
02:01:33 [tidoust]
tidoust has joined #datacue
02:01:41 [jkamata]
present+
02:01:42 [cyril]
chris: if anybody is not familiar so far
02:01:51 [cyril]
... our goal is we want to introduce native user agent support
02:01:55 [cyril]
... for DASH events
02:02:11 [cyril]
... as past of support for MPEG CMAF content
02:02:24 [cyril]
... alongside if UA has native user support for DASH playback
02:02:33 [cyril]
... we would like to support out of band MPD events
02:02:45 [cyril]
... HbbTV is an example of a player that has nativd DASH support
02:02:50 [stepsteg]
present+
02:03:11 [atai]
atai has joined #datacue
02:03:13 [cyril]
... I'd like to discuss implementer interest for other metadata cue formats
02:03:20 [cyril]
... for example Safari has support for ID3
02:03:41 [cyril]
... we also want API support for application-generated timed metadata cues
02:03:49 [cyril]
... when generated by a player
02:04:03 [cyril]
... the existing approach is to use the VTT cue
02:04:16 [cyril]
... either inline or as a reference in the VTTCue object
02:04:38 [cyril]
... having a more convenient datacue API that let us store in the preferred format would be better
02:04:50 [cyril]
... our goals are: sync arbitrary data with video
02:05:08 [cyril]
... e.g. dashcams sensor data
02:05:36 [cyril]
... there is a lot of interest from the Open Geo Consortium
02:05:53 [cyril]
... the applicability is broader than for the M&E IG
02:06:04 [cyril]
... [shows current support for DataCue API]
02:06:18 [mounir]
mounir has joined #datacue
02:07:01 [cyril]
... basically no support so far in Chrome or Firefox
02:07:14 [cyril]
... some support in Safari with an extended API
02:07:41 [nigel]
nigel has joined #datacue
02:07:52 [nigel]
Present+ Nigel
02:07:57 [nigel]
rrsagent, pointer?
02:07:57 [RRSAgent]
See https://www.w3.org/2019/09/18-datacue-irc#T02-07-57
02:08:44 [cyril]
... [showing the data structure for emsg]
02:08:54 [cyril]
... there a 2 versions that differ wrt the timing
02:09:14 [cyril]
... there would need a mapping between the esmg timing and the cue timing
02:09:17 [tk]
tk has joined #datacue
02:09:26 [MasayaIkeo]
MasayaIkeo has joined #datacue
02:09:36 [cyril]
... in v0 the timing is relative
02:10:02 [cyril]
... the data is a byte array, so you need a schema to identify the data
02:10:29 [cyril]
... DASH-IF is working in parallel around specifying a delivery and processing model for DASH events
02:10:41 [cyril]
... they are considering more types of players, not web platform only
02:10:53 [cyril]
... one of the requirements they have identified
02:11:07 [cyril]
... is that in order for an application to get prepared for presenting a cue, e.g. a video overlay
02:11:15 [cyril]
... that may require fetching other resources
02:11:19 [marisa]
marisa has joined #datacue
02:11:28 [cyril]
... they signal the event to the application ahead of time
02:11:40 [cyril]
... to be able to render at the appropriate time
02:11:51 [cyril]
... so we have 2 events: onreceive and onstart
02:12:03 [cyril]
... I have a number of questions:
02:12:25 [cyril]
... it relates to the early discussion around this, should in band events be exposed as a byte array
02:12:37 [cyril]
... or should they be exposed as objects
02:12:46 [cyril]
... the second approach makes it easier for app devs
02:12:57 [cyril]
... this may be desirables for cues that are commonly used
02:13:05 [cyril]
... for example within DASH players
02:13:26 [cyril]
... the emsg can be used for application specific events
02:13:37 [cyril]
... and we don't need support from browsers for those
02:13:51 [cyril]
... there is a question of how we identify inband tracks
02:13:56 [cyril]
... there are various fields
02:14:20 [cyril]
... all of them seem to enable identifying the kind of metadata
02:14:31 [cyril]
... it is not clear to me reading the spec and comparing implementation
02:14:39 [cyril]
... what the level of support is
02:14:59 [cyril]
chris_c: on your first bullet
02:15:23 [cyril]
... is it a reasonable behavior to fallback to the opaque array buffer when you don't understand the type?
02:15:36 [cyril]
chris: I'd like to understand what common subset can be supported?
02:15:58 [cyril]
... but the fallback could be a good approach if we have an API for that
02:16:24 [cyril]
francois: youll end up with 2 representations for the same data
02:16:36 [cyril]
... so in the end the devs have to handle the opaque case
02:16:49 [cyril]
... so in this case we shouldn't bother about the structured object
02:16:55 [cyril]
eric: I disagree strongly
02:17:11 [cyril]
... there are metadata formats that are very difficult for JS to parse
02:17:15 [cyril]
... correctly
02:17:29 [cyril]
... so that's why I added the implementation to WebKit
02:17:39 [cyril]
... because we had lots of requests to support datacues
02:17:50 [cyril]
... and just supporting arrays is not doing web authors a servie
02:18:06 [cyril]
francois: it is better to have a system that works across browsers
02:18:21 [cyril]
eric: if we ddecide that structured data is important
02:18:27 [cyril]
... we need to agree on a set of types
02:18:38 [cyril]
... that we want to support
02:18:56 [cyril]
... there will always be custom metadata
02:19:06 [cyril]
... people can put anything a container format
02:19:13 [cyril]
... and they want to have access to them
02:19:25 [igarashi_]
igarashi_ has joined #datacue
02:19:26 [cyril]
... it does not make sense to have support for limited set
02:19:41 [cyril]
chris: I can imagine a world where an impl wants to provide access to ID3 and another not
02:19:51 [cyril]
... it's the responsability of the dev to know that
02:20:06 [cyril]
eric: I agree that we should not end up in this situation
02:20:16 [cyril]
mounir: is there benefit in trying to avoid that?
02:20:22 [cyril]
eric: I think so
02:20:28 [cyril]
... we don't have to end up there
02:20:36 [cyril]
... if we can come up with a way to describe the cue
02:20:50 [cyril]
... and require that a browser that uses that identifier have a structured data
02:21:44 [cyril]
mounir: there could be security issues and different parsing if the browser do it themself
02:22:20 [cyril]
richard: if the parsing within the browser and use webassembly, how could there be a security issue?
02:22:32 [cyril]
mounir: if you use webassembly that's ok
02:22:49 [cyril]
... we try to avoid doing parsing in C++
02:23:04 [cyril]
chris_c: I'm trying to understand what the fallback would look like
02:23:10 [cyril]
... maybe the ID3 would not be contentious
02:23:49 [cyril]
chris: having an API structure that lets the application introspect the cue
02:24:11 [cyril]
gkatsev: ID3 in HLS, safari parses it, but in other browsers you have to do it yoursefl
02:24:25 [cyril]
nigel: is the data in the array buffer a registered type
02:24:50 [cyril]
ericc: no the data has no indication
02:24:51 [niwamoto]
niwamoto has joined #datacue
02:24:54 [cyril]
cyril: no magic number?
02:24:56 [cyril]
ericc: no
02:25:13 [cyril]
chris: the emsg also indicates the scheme id
02:25:27 [cyril]
ericc: with the current data cue api, the array buffer would have that whole thing from start to end
02:25:42 [cyril]
... and you'd have to snfiff the bits to figure out if it's an emsg or id3
02:26:01 [cyril]
... and it's going to have to parse it to determine if it's a emsg or not
02:26:47 [cyril]
nigel: imagine that we expose this data through MSE
02:26:54 [dontcallmeDOM]
dontcallmeDOM has joined #datacue
02:27:11 [cyril]
... the bytestream would be identifiable
02:27:46 [cyril]
ericc: the UA, thing that parses the raw media container, does have a signal about what kind of metadata it is
02:28:00 [cyril]
... if the data cue had a scheme and identifier for the type of metadata and an array buffer
02:28:10 [cyril]
... then in theory it could know how to parse it
02:28:20 [cyril]
... the reason I decided that was not practical for us
02:28:37 [cyril]
...is that there are metadata values that are extremely complex to parse
02:28:44 [cyril]
...like HLS has a pList
02:28:52 [cyril]
... writing a parser for a binary pList in JS
02:29:00 [cyril]
... is not easy
02:29:23 [cyril]
... pratically speaking, WebKit does not have access to the raw pList
02:29:29 [cyril]
... the low level does the parsing
02:29:36 [cyril]
... and we get it as a native object
02:29:46 [cyril]
... a representation of the data
02:29:56 [cyril]
... which WebKit converts into a JS object attached to the datacue
02:30:18 [cyril]
greg: most of the conversation is about inband
02:30:30 [cyril]
... I can see datacue useful for out of band use cases
02:30:37 [cyril]
ericc: that is a part of this
02:30:50 [cyril]
... from script you can make a new data cue with start/end
02:30:53 [cyril]
... and attach anything
02:31:14 [cyril]
chris: the explainer is incomplete and in a very early stage
02:31:17 [cyril]
... it does not explain everything
02:31:30 [cyril]
ericc: any solution we come up with has to support cues from script
02:31:43 [nigel]
scribe: nigel
02:31:51 [nigel]
cyril: Comment on synchronisation
02:32:02 [nigel]
.. The payload of the metadata may trigger behaviour with unbounded complexity
02:32:15 [nigel]
.. so that's why you probably need to process it in advance and to know in advance the practical bound.
02:32:23 [nigel]
.. To me this is similar to how video content is processed.
02:32:34 [nigel]
.. We don't have two timestamps, one for receiving, the other for presenting.
02:32:41 [nigel]
.. The implementation has to know when to preprocess things.
02:32:42 [NJ_]
NJ_ has joined #datacue
02:32:50 [nigel]
.. So I'm not convinced that having two events is a good approach.
02:32:57 [nigel]
ericc: I agree and am strongly opposed to having two.
02:32:58 [cyril]
eric: I strongly oppose to having 2 timestamps
02:33:03 [nigel]
scribe: cyril
02:33:21 [cyril]
... in addition you cannot predict how much it is going to take in the app to do the processing
02:33:40 [cyril]
... if what you are suggested is that a cue should be delivered as soon as it is available
02:33:46 [cyril]
... that's going to vary widely
02:33:56 [cyril]
.. depending on where the parsing happens
02:34:19 [cyril]
francois: perhaps it's useful to look at why
02:34:43 [nigel]
cyril: I agree, 3 categories of event:
02:34:49 [nigel]
.. Overlay, maybe after js processing.
02:34:58 [nigel]
.. Network impact, like making requests or sending messages
02:35:02 [nigel]
.. 3rd, modifying the DOM
02:35:07 [nigel]
s/Ov/1. Ov
02:35:11 [nigel]
s/Ne/2. Ne
02:35:31 [nigel]
.. The 3rd category - you should be able to pre-render in advance and keep your frames until they're ready
02:35:36 [nigel]
.. The other two I'm not sure about yet.
02:35:46 [nigel]
i/cyril:/scribe: nigel
02:35:50 [nigel]
scribe: cyril
02:36:04 [cyril]
chris: I'd like to move on to the next part, synchronization
02:36:07 [cyril]
Topic: Time marches on
02:36:19 [cyril]
chris: web apps use the oncuechange
02:36:30 [cyril]
... triggered by the time marches on
02:36:40 [cyril]
... and the spec says there is an upper limit
02:36:54 [cyril]
... but in practice some implementations do follow the upper limit
02:37:09 [cyril]
... this means that it is possible for an application to miss a short duration cue entirely
02:37:24 [cyril]
... the cuechange event is fired, the app inspects the active cues list
02:37:27 [cyril]
... and acts
02:37:47 [cyril]
... it's quite possible that in between cues triggered there are cues that app don't see it
02:38:02 [cyril]
... there is a bug report raised by Jon Piesing, HbbTV
02:38:12 [cyril]
... and the recommendation is not to create short cues
02:38:18 [cyril]
... but it's worse than that
02:38:31 [cyril]
... you have to take execution time into account
02:38:43 [cyril]
... use of oncuechange is problematic for handling cues
02:38:54 [cyril]
... the good news is that if you want to avoid missng cues
02:39:07 [cyril]
... you can attach events to onenter and on exit
02:39:20 [mounir]
q+ foolip
02:39:29 [cyril]
nigel: but if it was missed, enter/exit are triggered at the same time
02:39:51 [cyril]
... and if there are visual changes they will be missed
02:40:07 [cyril]
foolip: the time marches on step are not defined to run every 250ms
02:40:22 [cyril]
... it's meant to be continuous
02:40:35 [cyril]
... only the event are triggered every 250ms
02:40:41 [cyril]
chris: that's not my readinfg of the spec
02:40:59 [cyril]
foolip: the problem is that implementations are not following the spec because that's easier to do
02:41:09 [cyril]
ericc: if you run a test to look at the variance
02:41:15 [cyril]
... you'll see 10-20ms
02:41:25 [cyril]
because we don't use the time marches on
02:41:29 [cyril]
... but look at the cues
02:41:37 [cyril]
... this is a quality of implementation issue
02:41:48 [cyril]
nigel: this is a spec question
02:41:53 [cyril]
... [reading the spec]
02:42:13 [cyril]
foolip: it's just for the timeupdate event
02:42:17 [cyril]
... not for the cue events
02:42:55 [cyril]
foolip: [explaining how it worked in Presto]
02:43:06 [cyril]
nigel: chrome does it this way
02:43:13 [cyril]
foolip: not because the spec is wrong
02:43:24 [cyril]
nigel: but the spec allows it
02:43:35 [cyril]
chris: we need a follow-up to understand that
02:43:47 [cyril]
foolip: maybe open a bug in chromium
02:44:04 [cyril]
chris_n: the spec does not mandate 250ms
02:44:19 [cyril]
ericc: so that we are not firing timeupdate events to not overload the system
02:44:30 [cyril]
... we could, but that would cause other issues
02:44:32 [cyril]
ack
02:44:39 [cyril]
ack foolip
02:44:42 [cyril]
q?
02:45:10 [cyril]
pierre: the spec guarantees that every single cue will be fired
02:45:21 [cyril]
... regardless of the algorithm ?
02:45:28 [cyril]
gkatsev: no some have been missed
02:45:46 [cyril]
scott: the text says some cues can be skipped
02:46:29 [cyril]
ericc: cues can be dropped
02:47:04 [cyril]
pal: if I have a cue that has a duration of d is there a req that difference between onenter and onexit is close to d?
02:47:06 [cyril]
ericc: no
02:47:18 [cyril]
pal: you could get them simulatenously
02:47:38 [cyril]
ericc: but if there is onenter/onexit it should be fired
02:47:43 [cyril]
foolip: that's a good idea
02:47:58 [cyril]
chris: another related issue is
02:48:06 [cyril]
... we want a more accurate firing of these events
02:48:19 [cyril]
... driven by the need to align captions with shots or scene changes in the video
02:48:28 [cyril]
... and we came up with a number of 20ms
02:48:43 [cyril]
... that gives a chance to the application
02:48:53 [cyril]
nigel: you want to replace the number 250 with 20?
02:48:56 [cyril]
chris: no
02:49:22 [cyril]
richard: the shorter the time limit goes down, it's exponential the power you're going to have
02:49:47 [cyril]
foolip: the reason the schedulig is poor isnot for battery saving
02:49:59 [cyril]
ericc: it was because it was simpler to write
02:50:19 [cyril]
... it's not possible to guarantee any kind specific latency
02:50:32 [cyril]
... because the browser is under the same constraints as anything else
02:50:58 [cyril]
nigel: that depends on the frame rate
02:51:07 [cyril]
ericc: cues are not tied to frames
02:51:17 [cyril]
foolip: but frames have time stamos
02:51:33 [cyril]
ericc: in my system the frames are rendered by a different subsystem
02:52:03 [cyril]
foolip: there is a quality of implementation issue
02:52:18 [cyril]
ericc: no matter what wording we put in the spec
02:52:22 [cyril]
... it won't help you
02:52:31 [cyril]
... you have to file bugs to get what you need
02:52:53 [cyril]
chris_n: I wouldn't close an issue because it is ok with the spec
02:53:09 [cyril]
chris: I see some inconsistencies between implementations
02:53:17 [cyril]
... when the application moves cues around in the timeline
02:53:25 [cyril]
... if you change time of the cues
02:53:28 [cyril]
... and if you seek the media
02:53:35 [cyril]
... and seek over some cues
02:53:43 [cyril]
ericc: have you filed bugs?
02:53:46 [cyril]
chris: not yet
02:54:13 [cyril]
chris_n: a spec update is not necessary but it may be useful to avoid others doing the same mistake
02:54:22 [cyril]
ericc: we should not wait for TPAC to file bugs
02:54:35 [cyril]
... if we want to have the issue fixed quickly
02:54:43 [cyril]
nigel: it's hard to file a bug with the given spec
02:55:07 [cyril]
ericc: if you file a bug with an example and it is not good enough even if it matches the spec we should fix it
02:55:20 [cyril]
... we could get the spec improved
02:55:36 [cyril]
foolip: all specs are wrong every other paragraph!
02:56:09 [cyril]
richard: sometimes I've asked to fix an implementation but been told that the impl is within the spec
02:56:33 [cyril]
foolip: it happens that implementers consider the spec as untouchable but you should escalate
02:57:07 [cyril]
chris: [showing a waverform library demo]
02:57:21 [cyril]
... I'm using VTTCues
02:57:40 [cyril]
... adjusting the times on cues
02:57:48 [cyril]
... it's not the only use case
02:58:34 [cyril]
... [showing a table of what events get fired in practice]
02:59:30 [cyril]
ericc: you should file a bug
03:00:34 [cyril]
chris: the next stage is the meeting on Friday, joint Media WG and Timed Text
03:00:44 [cyril]
... we should figure out how to use that time productively
03:00:46 [nigel]
-> https://bugs.chromium.org/p/chromium/issues/detail?id=576310 Blink bug
03:00:55 [nigel]
-> https://bugs.chromium.org/p/chromium/issues/detail?id=907459 blink bug
03:00:58 [cyril]
... it seems filing bug is the recommendation
03:01:08 [cyril]
rrsagent, make minutes
03:01:09 [RRSAgent]
I have made the request to generate https://www.w3.org/2019/09/18-datacue-minutes.html cyril
03:10:35 [ericc]
ericc has joined #datacue
03:20:30 [ericc_]
ericc_ has joined #datacue
03:34:32 [ericc]
ericc has joined #datacue
03:37:46 [pal]
pal has joined #datacue
03:39:51 [ericc]
ericc has joined #datacue
04:18:39 [tidoust]
tidoust has joined #datacue
04:22:15 [marisa]
marisa has joined #datacue
04:27:59 [marisa]
marisa has joined #datacue
04:28:25 [romain]
romain has joined #datacue
04:28:32 [romain]
romain has left #datacue
04:29:20 [atai]
atai has joined #datacue
04:29:39 [dontcallmeDOM]
dontcallmeDOM has joined #datacue
04:30:51 [ericc_]
ericc_ has joined #datacue
04:34:10 [MasayaIkeo]
MasayaIkeo has joined #datacue
04:43:52 [dom]
dom has left #datacue
04:51:38 [atai]
atai has joined #datacue
05:26:03 [MasayaIkeo]
MasayaIkeo has joined #datacue
05:27:47 [ericc]
ericc has joined #datacue
05:31:19 [tidoust]
tidoust has joined #datacue
05:32:56 [pal]
pal has joined #datacue
05:35:58 [atai]
atai has joined #datacue
05:35:59 [marisa]
marisa has joined #datacue
05:36:16 [marisa]
marisa has joined #datacue
05:36:19 [marisa]
marisa has left #datacue
05:39:12 [MasayaIkeo]
MasayaIkeo has joined #datacue
06:32:05 [MasayaIkeo]
MasayaIkeo has joined #datacue
06:44:06 [MasayaIkeo]
MasayaIkeo has joined #datacue
07:18:47 [tidoust]
tidoust has joined #datacue
07:27:28 [ericc]
ericc has joined #datacue
07:31:52 [tidoust_]
tidoust_ has joined #datacue
07:45:25 [atai]
atai has joined #datacue
07:47:17 [MasayaIkeo]
MasayaIkeo has joined #datacue
07:51:57 [tidoust_]
tidoust_ has joined #datacue
07:59:56 [tidoust]
tidoust has joined #datacue
08:00:52 [atai1]
atai1 has joined #datacue
08:31:24 [tidoust]
tidoust has joined #datacue
08:34:44 [ericc]
ericc has joined #datacue
08:37:41 [MasayaIkeo]
MasayaIkeo has joined #datacue
08:38:14 [MasayaIkeo]
MasayaIkeo has joined #datacue
09:33:21 [MasayaIk_]
MasayaIk_ has joined #datacue
10:06:29 [MasayaIkeo]
MasayaIkeo has joined #datacue
11:55:32 [tidoust]
tidoust has joined #datacue