01:42:11 <RRSAgent> RRSAgent has joined #datacue
01:42:11 <RRSAgent> logging to https://www.w3.org/2019/09/18-datacue-irc
01:42:17 <dom> RRSAgent, make log public
01:45:02 <dom> Meeting: DataCue and "Time marches on" in HTML
01:50:36 <koalie> koalie has joined #datacue
01:50:42 <koalie> koalie has changed the topic to: https://w3c.github.io/tpac-breakouts/sessions.html
01:50:45 <koalie> koalie has left #datacue
01:56:37 <takio> takio has joined #datacue
01:57:22 <dom> dom has joined #datacue
01:58:34 <romain> romain has joined #datacue
01:58:38 <cyril> cyril has joined #datacue
01:58:46 <cyril> rrsagent, start meeting
01:58:46 <RRSAgent> I'm logging. I don't understand 'start meeting', cyril.  Try /msg RRSAgent help
01:58:56 <romain> present+
01:59:01 <gkatsev> gkatsev has joined #datacue
01:59:04 <cyril> Chair: Chris Needham
01:59:08 <cyril> scribe: Cyril
02:00:16 <cyril> meeting: DataCue and "Time marches on" in HTML
02:00:21 <gkatsev> present+
02:00:22 <pal> pal has joined #datacue
02:00:25 <cyril> rrsagent, draft minutes
02:00:25 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/18-datacue-minutes.html cyril
02:00:27 <chcunningham> chcunningham has joined #datacue
02:00:27 <pal> present+
02:00:30 <chcunningham> present+
02:00:44 <ericc> ericc has joined #datacue
02:00:46 <jkamata> jkamata has joined #datacue
02:00:57 <ericc> present+
02:01:19 <stepsteg> stepsteg has joined #datacue
02:01:24 <cyril> chris: topics are discuss the DataCue API and the Time marches on algorithm
02:01:33 <cyril> Topic: DataCue API
02:01:33 <tidoust> tidoust has joined #datacue
02:01:41 <jkamata> present+
02:01:42 <cyril> chris: if anybody is not familiar so far
02:01:51 <cyril> ... our goal is we want to introduce native user agent support
02:01:55 <cyril> ... for DASH events
02:02:11 <cyril> ... as past of support for MPEG CMAF content
02:02:24 <cyril> ... alongside if UA has native user support for DASH playback
02:02:33 <cyril> ... we would like to support out of band MPD events
02:02:45 <cyril> ... HbbTV is an example of a player that has nativd DASH support
02:02:50 <stepsteg> present+
02:03:11 <atai> atai has joined #datacue
02:03:13 <cyril> ... I'd like to discuss implementer interest for other metadata cue formats
02:03:20 <cyril> ... for example Safari has support for ID3
02:03:41 <cyril> ... we also want API support for application-generated timed metadata cues
02:03:49 <cyril> ... when generated by a player
02:04:03 <cyril> ... the existing approach is to use the VTT cue
02:04:16 <cyril> ... either inline or as a reference in the VTTCue object
02:04:38 <cyril> ... having a more convenient datacue API that let us store in the preferred format would be better
02:04:50 <cyril> ... our goals are: sync arbitrary data with video
02:05:08 <cyril> ... e.g. dashcams sensor data
02:05:36 <cyril> ... there is a lot of interest from the Open Geo Consortium
02:05:53 <cyril> ... the applicability is broader than for the M&E IG
02:06:04 <cyril> ... [shows current support for DataCue API]
02:06:18 <mounir> mounir has joined #datacue
02:07:01 <cyril> ... basically no support so far in Chrome or Firefox
02:07:14 <cyril> ... some support in Safari with an extended API
02:07:41 <nigel> nigel has joined #datacue
02:07:52 <nigel> Present+ Nigel
02:07:57 <nigel> rrsagent, pointer?
02:07:57 <RRSAgent> See https://www.w3.org/2019/09/18-datacue-irc#T02-07-57
02:08:44 <cyril> ... [showing the data structure for emsg]
02:08:54 <cyril> ... there a 2 versions that differ wrt the timing
02:09:14 <cyril> ... there would need a mapping between the esmg timing and the cue timing
02:09:17 <tk> tk has joined #datacue
02:09:26 <MasayaIkeo> MasayaIkeo has joined #datacue
02:09:36 <cyril> ... in v0 the timing is relative
02:10:02 <cyril> ... the data is a byte array, so you need a schema to identify the data
02:10:29 <cyril> ... DASH-IF is working in parallel around specifying a delivery and processing model for DASH events
02:10:41 <cyril> ... they are considering more types of players, not web platform only
02:10:53 <cyril> ... one of the requirements they have identified
02:11:07 <cyril> ... is that in order for an application to get prepared for presenting a cue, e.g. a video overlay
02:11:15 <cyril> ... that may require fetching other resources
02:11:19 <marisa> marisa has joined #datacue
02:11:28 <cyril> ... they signal the event to the application ahead of time
02:11:40 <cyril> ... to be able to render at the appropriate time
02:11:51 <cyril> ... so we have 2 events: onreceive and onstart
02:12:03 <cyril> ... I have a number of questions:
02:12:25 <cyril> ... it relates to the early discussion around this, should in band events be exposed as a byte array
02:12:37 <cyril> ... or should they be exposed as objects
02:12:46 <cyril> ... the second approach makes it easier for app devs
02:12:57 <cyril> ... this may be desirables for cues that are commonly used
02:13:05 <cyril> ... for example within DASH players
02:13:26 <cyril> ... the emsg can be used for application specific events
02:13:37 <cyril> ... and we don't need support from browsers for those
02:13:51 <cyril> ... there is a question of how we identify inband tracks
02:13:56 <cyril> ... there are various fields
02:14:20 <cyril> ... all of them seem to enable identifying the kind of metadata
02:14:31 <cyril> ... it is not clear to me reading the spec and comparing implementation
02:14:39 <cyril> ... what the level of support is
02:14:59 <cyril> chris_c: on your first bullet
02:15:23 <cyril> ... is it a reasonable behavior to fallback to the opaque array buffer when you don't understand the type?
02:15:36 <cyril> chris: I'd like to understand what common subset can be supported?
02:15:58 <cyril> ... but the fallback could be a good approach if we have an API for that
02:16:24 <cyril> francois: youll end up with 2 representations for the same data
02:16:36 <cyril> ... so in the end the devs have to handle the opaque case
02:16:49 <cyril> ... so in this case we shouldn't bother about the structured object
02:16:55 <cyril> eric: I disagree strongly
02:17:11 <cyril> ... there are metadata formats that are very difficult for JS to parse
02:17:15 <cyril> ... correctly
02:17:29 <cyril> ... so that's why I added the implementation to WebKit
02:17:39 <cyril> ... because we had lots of requests to support datacues
02:17:50 <cyril> ... and just supporting arrays is not doing web authors a servie
02:18:06 <cyril> francois: it is better to have a system that works across browsers
02:18:21 <cyril> eric: if we ddecide that structured data is important
02:18:27 <cyril> ... we need to agree on a set of types
02:18:38 <cyril> ... that we want to support
02:18:56 <cyril> ... there will always be custom metadata
02:19:06 <cyril> ... people can put anything a container format
02:19:13 <cyril> ... and they want to have access to them
02:19:25 <igarashi_> igarashi_ has joined #datacue
02:19:26 <cyril> ... it does not make sense to have support for limited set
02:19:41 <cyril> chris: I can imagine a world where an impl wants to provide access to ID3 and another not
02:19:51 <cyril> ... it's the responsability of the dev to know that
02:20:06 <cyril> eric: I agree that we should not end up in this situation
02:20:16 <cyril> mounir: is there benefit in trying to avoid that?
02:20:22 <cyril> eric: I think so
02:20:28 <cyril> ... we don't have to end up there
02:20:36 <cyril> ... if we can come up with a way to describe the cue
02:20:50 <cyril> ... and require that a browser that uses that identifier have a structured data
02:21:44 <cyril> mounir: there could be security issues and different parsing if the browser do it themself
02:22:20 <cyril> richard: if the parsing within the browser and use webassembly, how could there be a security issue?
02:22:32 <cyril> mounir: if you use webassembly that's ok
02:22:49 <cyril> ... we try to avoid doing parsing in C++
02:23:04 <cyril> chris_c: I'm trying to understand what the fallback would look like
02:23:10 <cyril> ... maybe the ID3 would not be contentious
02:23:49 <cyril> chris: having an API structure that lets the application introspect the cue
02:24:11 <cyril> gkatsev: ID3 in HLS, safari parses it, but in other browsers you have to do it yoursefl
02:24:25 <cyril> nigel: is the data in the array buffer a registered type
02:24:50 <cyril> ericc: no the data has no indication
02:24:51 <niwamoto> niwamoto has joined #datacue
02:24:54 <cyril> cyril: no magic number?
02:24:56 <cyril> ericc: no
02:25:13 <cyril> chris: the emsg also indicates the scheme id
02:25:27 <cyril> ericc: with the current data cue api, the array buffer would have that whole thing from start to end
02:25:42 <cyril> ... and you'd have to snfiff the bits to figure out if it's an emsg or id3
02:26:01 <cyril> ... and it's going to have to parse it to determine if it's a emsg or not
02:26:47 <cyril> nigel: imagine that we expose this data through MSE
02:26:54 <dontcallmeDOM> dontcallmeDOM has joined #datacue
02:27:11 <cyril> ... the bytestream would be identifiable
02:27:46 <cyril> ericc: the UA, thing that parses the raw media container, does have a signal about what kind of metadata it is
02:28:00 <cyril> ... if the data cue had a scheme and identifier for the type of metadata and an array buffer
02:28:10 <cyril> ... then in theory it could know how to parse it
02:28:20 <cyril> ... the reason I decided that was not practical for us
02:28:37 <cyril> ...is that there are metadata values that are extremely complex to parse
02:28:44 <cyril> ...like HLS has a pList
02:28:52 <cyril> ... writing a parser for a binary pList in JS
02:29:00 <cyril> ... is not easy
02:29:23 <cyril> ... pratically speaking, WebKit does not have access to the raw pList
02:29:29 <cyril> ... the low level does the parsing
02:29:36 <cyril> ... and we get it as a native object
02:29:46 <cyril> ... a representation of the data
02:29:56 <cyril> ... which WebKit converts into a JS object attached to the datacue
02:30:18 <cyril> greg: most of the conversation is about inband
02:30:30 <cyril> ... I can see datacue useful for out of band use cases
02:30:37 <cyril> ericc: that is a part of this
02:30:50 <cyril> ... from script you can make a new data cue with start/end
02:30:53 <cyril> ... and attach anything
02:31:14 <cyril> chris: the explainer is incomplete and in a very early stage
02:31:17 <cyril> ... it does not explain everything
02:31:30 <cyril> ericc: any solution we come up with has to support cues from script
02:31:43 <nigel> scribe: nigel
02:31:51 <nigel> cyril: Comment on synchronisation
02:32:02 <nigel> .. The payload of the metadata may trigger behaviour with unbounded complexity
02:32:15 <nigel> .. so that's why you probably need to process it in advance and to know in advance the practical bound.
02:32:23 <nigel> .. To me this is similar to how video content is processed.
02:32:34 <nigel> .. We don't have two timestamps, one for receiving, the other for presenting.
02:32:41 <nigel> .. The implementation has to know when to preprocess things.
02:32:42 <NJ_> NJ_ has joined #datacue
02:32:50 <nigel> .. So I'm not convinced that having two events is a good approach.
02:32:57 <nigel> ericc: I agree and am strongly opposed to having two.
02:32:58 <cyril> eric: I strongly oppose to having 2 timestamps
02:33:03 <nigel> scribe: cyril
02:33:21 <cyril> ... in addition you cannot predict how much it is going to take in the app to do the processing
02:33:40 <cyril> ... if what you are suggested is that a cue should be delivered as soon as it is available
02:33:46 <cyril> ... that's going to vary widely
02:33:56 <cyril> .. depending on where the parsing happens
02:34:19 <cyril> francois: perhaps it's useful to look at why
02:34:43 <nigel> cyril: I agree, 3 categories of event:
02:34:49 <nigel> .. Overlay, maybe after js processing.
02:34:58 <nigel> .. Network impact, like making requests or sending messages
02:35:02 <nigel> .. 3rd, modifying the DOM
02:35:07 <nigel> s/Ov/1. Ov
02:35:11 <nigel> s/Ne/2. Ne
02:35:31 <nigel> .. The 3rd category - you should be able to pre-render in advance and keep your frames until they're ready
02:35:36 <nigel> .. The other two I'm not sure about yet.
02:35:46 <nigel> i/cyril:/scribe: nigel
02:35:50 <nigel> scribe: cyril
02:36:04 <cyril> chris: I'd like to move on to the next part, synchronization
02:36:07 <cyril> Topic: Time marches on
02:36:19 <cyril> chris: web apps use the oncuechange
02:36:30 <cyril> ... triggered by the time marches on
02:36:40 <cyril> ... and the spec says there is an upper limit
02:36:54 <cyril> ... but in practice some implementations do follow the upper limit
02:37:09 <cyril> ... this means that it is possible for an application to miss a short duration cue entirely
02:37:24 <cyril> ... the cuechange event is fired, the app inspects the active cues list
02:37:27 <cyril> ... and acts
02:37:47 <cyril> ... it's quite possible that in between cues triggered there are cues that app don't see it
02:38:02 <cyril> ... there is a bug report raised by Jon Piesing, HbbTV
02:38:12 <cyril> ... and the recommendation is not to create short cues
02:38:18 <cyril> ... but it's worse than that
02:38:31 <cyril> ... you have to take execution time into account
02:38:43 <cyril> ... use of oncuechange is problematic for handling cues
02:38:54 <cyril> ... the good news is that if you want to avoid missng cues
02:39:07 <cyril> ... you can attach events to onenter and on exit
02:39:20 <mounir> q+ foolip
02:39:29 <cyril> nigel: but if it was missed, enter/exit are triggered at the same time
02:39:51 <cyril> ... and if there are visual changes they will be missed
02:40:07 <cyril> foolip: the time marches on step are not defined to run every 250ms
02:40:22 <cyril> ... it's meant to be continuous
02:40:35 <cyril> ... only the event are triggered every 250ms
02:40:41 <cyril> chris: that's not my readinfg of the spec
02:40:59 <cyril> foolip: the problem is that implementations are not following the spec because that's easier to do
02:41:09 <cyril> ericc: if you run a test to look at the variance
02:41:15 <cyril> ... you'll see 10-20ms
02:41:25 <cyril> because we don't use the time marches on
02:41:29 <cyril> ... but look at the cues
02:41:37 <cyril> ... this is a quality of implementation issue
02:41:48 <cyril> nigel: this is a spec question
02:41:53 <cyril> ... [reading the spec]
02:42:13 <cyril> foolip: it's just for the timeupdate event
02:42:17 <cyril> ... not for the cue events
02:42:55 <cyril> foolip: [explaining how it worked in Presto]
02:43:06 <cyril> nigel: chrome does it this way
02:43:13 <cyril> foolip: not because the spec is wrong
02:43:24 <cyril> nigel: but the spec allows it
02:43:35 <cyril> chris: we need a follow-up to understand that
02:43:47 <cyril> foolip: maybe open a bug in chromium
02:44:04 <cyril> chris_n: the spec does not mandate 250ms
02:44:19 <cyril> ericc: so that we are not firing timeupdate events to not overload the system
02:44:30 <cyril> ... we could, but that would cause other issues
02:44:32 <cyril> ack
02:44:39 <cyril> ack foolip
02:44:42 <cyril> q?
02:45:10 <cyril> pierre: the spec guarantees that every single cue will be fired
02:45:21 <cyril> ... regardless of the algorithm ?
02:45:28 <cyril> gkatsev: no some have been missed
02:45:46 <cyril> scott: the text says some cues can be skipped
02:46:29 <cyril> ericc: cues can be dropped
02:47:04 <cyril> pal: if I have a cue that has a duration of d is there a req that difference between onenter and onexit is close to d?
02:47:06 <cyril> ericc: no
02:47:18 <cyril> pal: you could get them simulatenously
02:47:38 <cyril> ericc: but if there is onenter/onexit it should be fired
02:47:43 <cyril> foolip: that's a good idea
02:47:58 <cyril> chris: another related issue is
02:48:06 <cyril> ... we want a more accurate firing of these events
02:48:19 <cyril> ... driven by the need to align captions with shots or scene changes in the video
02:48:28 <cyril> ... and we came up with a number of 20ms
02:48:43 <cyril> ... that gives a chance to the application
02:48:53 <cyril> nigel: you want to replace the number 250 with 20?
02:48:56 <cyril> chris: no
02:49:22 <cyril> richard: the shorter the time limit goes down, it's exponential the power you're going to have
02:49:47 <cyril> foolip: the reason the schedulig is poor isnot for battery saving
02:49:59 <cyril> ericc: it was because it was simpler to write
02:50:19 <cyril> ... it's not possible to guarantee any kind specific latency
02:50:32 <cyril> ... because the browser is under the same constraints as anything else
02:50:58 <cyril> nigel: that depends on the frame rate
02:51:07 <cyril> ericc: cues are not tied to frames
02:51:17 <cyril> foolip: but frames have time stamos
02:51:33 <cyril> ericc: in my system the frames are rendered by a different subsystem
02:52:03 <cyril> foolip: there is a quality of implementation issue
02:52:18 <cyril> ericc: no matter what wording we put in the spec
02:52:22 <cyril> ... it won't help you
02:52:31 <cyril> ... you have to file bugs to get what you need
02:52:53 <cyril> chris_n: I wouldn't close an issue because it is ok with the spec
02:53:09 <cyril> chris: I see some inconsistencies between implementations
02:53:17 <cyril> ... when the application moves cues around in the timeline
02:53:25 <cyril> ... if you change time of the cues
02:53:28 <cyril> ... and if you seek the media
02:53:35 <cyril> ... and seek over some cues
02:53:43 <cyril> ericc: have you filed bugs?
02:53:46 <cyril> chris: not yet
02:54:13 <cyril> chris_n: a spec update is not necessary but it may be useful to avoid others doing the same mistake
02:54:22 <cyril> ericc: we should not wait for TPAC to file bugs
02:54:35 <cyril> ... if we want to have the issue fixed quickly
02:54:43 <cyril> nigel: it's hard to file a bug with the given spec
02:55:07 <cyril> ericc: if you file a bug with an example and it is not good enough even if it matches the spec we should fix it
02:55:20 <cyril> ... we could get the spec improved
02:55:36 <cyril> foolip: all specs are wrong every other paragraph!
02:56:09 <cyril> richard: sometimes I've asked to fix an implementation but been told that the impl is within the spec
02:56:33 <cyril> foolip: it happens that implementers consider the spec as untouchable but you should escalate
02:57:07 <cyril> chris: [showing a waverform library demo]
02:57:21 <cyril> ... I'm using VTTCues
02:57:40 <cyril> ... adjusting the times on cues
02:57:48 <cyril> ... it's not the only use case
02:58:34 <cyril> ... [showing a table of what events get fired in practice]
02:59:30 <cyril> ericc: you should file a bug
03:00:34 <cyril> chris: the next stage is the meeting on Friday, joint Media WG and Timed Text
03:00:44 <cyril> ... we should figure out how to use that time productively
03:00:46 <nigel> -> https://bugs.chromium.org/p/chromium/issues/detail?id=576310 Blink bug
03:00:55 <nigel> -> https://bugs.chromium.org/p/chromium/issues/detail?id=907459 blink bug
03:00:58 <cyril> ... it seems filing bug is the recommendation
03:01:08 <cyril> rrsagent, make minutes
03:01:09 <RRSAgent> I have made the request to generate https://www.w3.org/2019/09/18-datacue-minutes.html cyril
03:10:35 <ericc> ericc has joined #datacue
03:20:30 <ericc_> ericc_ has joined #datacue
03:34:32 <ericc> ericc has joined #datacue
03:37:46 <pal> pal has joined #datacue
03:39:51 <ericc> ericc has joined #datacue
04:18:39 <tidoust> tidoust has joined #datacue
04:22:15 <marisa> marisa has joined #datacue
04:27:59 <marisa> marisa has joined #datacue
04:28:25 <romain> romain has joined #datacue
04:28:32 <romain> romain has left #datacue
04:29:20 <atai> atai has joined #datacue
04:29:39 <dontcallmeDOM> dontcallmeDOM has joined #datacue
04:30:51 <ericc_> ericc_ has joined #datacue
04:34:10 <MasayaIkeo> MasayaIkeo has joined #datacue
04:43:52 <dom> dom has left #datacue
04:51:38 <atai> atai has joined #datacue
05:26:03 <MasayaIkeo> MasayaIkeo has joined #datacue
05:27:47 <ericc> ericc has joined #datacue
05:31:19 <tidoust> tidoust has joined #datacue
05:32:56 <pal> pal has joined #datacue
05:35:58 <atai> atai has joined #datacue
05:35:59 <marisa> marisa has joined #datacue
05:36:16 <marisa> marisa has joined #datacue
05:36:19 <marisa> marisa has left #datacue
05:39:12 <MasayaIkeo> MasayaIkeo has joined #datacue
06:32:05 <MasayaIkeo> MasayaIkeo has joined #datacue
06:44:06 <MasayaIkeo> MasayaIkeo has joined #datacue
07:18:47 <tidoust> tidoust has joined #datacue
07:27:28 <ericc> ericc has joined #datacue
07:31:52 <tidoust_> tidoust_ has joined #datacue
07:45:25 <atai> atai has joined #datacue
07:47:17 <MasayaIkeo> MasayaIkeo has joined #datacue
07:51:57 <tidoust_> tidoust_ has joined #datacue
07:59:56 <tidoust> tidoust has joined #datacue
08:00:52 <atai1> atai1 has joined #datacue
08:31:24 <tidoust> tidoust has joined #datacue
08:34:44 <ericc> ericc has joined #datacue
08:37:41 <MasayaIkeo> MasayaIkeo has joined #datacue
08:38:14 <MasayaIkeo> MasayaIkeo has joined #datacue
09:33:21 <MasayaIk_> MasayaIk_ has joined #datacue
10:06:29 <MasayaIkeo> MasayaIkeo has joined #datacue
11:55:32 <tidoust> tidoust has joined #datacue