Meeting minutes
Welcome, Agenda etc
Nigel: Welcome everyone to TTWG at TPAC 2023.
… [introduces people in the room]
… Agenda for today:
… Any reflections from yesterday's joint meeting with MEIG, and planning for Thursday's joint meeting
… with MEIG and APA.
… [iterates through agenda]
Nigel: Any other business, or points to make sure we cover?
Evan: Is anyone working on adding spec feature to WebVTT for audio description?
Nigel: Not as far as i know - I know Apple did a TTS implementation using it,
… but there are no proposals for any changes to the spec, e.g. to support SSML, voice selection etc.
… As far as I know!
… And it doesn't seem like anyone wants to use it to play back pre-recorded audio, it's TTS only.
Evan: We've done some prototyping in Chrome
Nigel: Just a reminder at the beginning of the session.
… As always our work is done a friendly professional way, as per the code of conduct
Reflections from Monday's MEIG joint meeting
<tidoust> Draft Media & Entertainment IG minutes
Nigel: We covered the IMSC-HRM and DAPT, and had an intro from Eric about
… the TextTrackCue proposal they will make later today.
… Any observations? (there don't have to be!)
Andreas: 2 things that come to mind.
… 1st, the presentation from Fraunhofer about CTA Wave tests
… We observed that they need tests for subtitles and test content
… They're testing device capabilities.
… Subtitles are left out but they plan an iteration to add subtitles
… Should we actively approach CTA Wave and offer our help?
Nigel: What sort of help might we offer?
Andreas: For the Media API snapshot, defining a baseline of what should be supported in browsers,
… maybe more WebVTT related.
… The other is about test content, what parameters, vectors to test, we could help with.
Francois: It was surprising that they didn't mention WebVTT or TTML
… In my understanding the TTML tests are not in WPT. To run them, how do you do that?
… The work they had to do was to adjust the test runner to work on TV sets.
Andreas: I don't know
Francois: Someone has to do the work and make a test runner work on TVs
Nigel: The IMSC and TTML tests are static resources, including test documents and
… for presentation tests, reference renders.
… The imscjs library includes a test runner - it brings in the tests and generates renders from them.
… And then diffs the renders against the references.
Francois: The Web Media API doesn't include TTML because that's not directly referenced and used in browsers.
Atsushi: WebVTT tests are in WPT.
Francois: Right, but CTA WAVE might not be interested in the TTML tests.
Nigel: CTA is interested in client side stuff, and ATSC 3 and CMAF defines IMSC as the subtitle format
… so they should be interested!
Francois: Yes, just specifically related to web media API, it would not be in scope.
Andreas: That was my recollection, I was not sure if TTML was in our out of scope.
… We can ask CTA Wave how this could fit.
… Testing it for smart TVs should be in scope of CTA's work.
Francois: Web Media API is a list of specs that the main core browser engines support,
… and therefore browsers in TV sets must support.
… It's a "baseline" published once every year for the last 5 or 6 years.
… Once in a while they add exceptions to say particular features do not necessarily need to be supported.
<tidoust> Web Media API Snapshot 2022
Nigel: Observation: Smart TV manufacturers are required to support TTML subtitles by ATSC 3, HbbTV etc
… so they're getting conflicting messages from W3C and elsewhere.
Francois: Yes, the Media API only lists what is commonly supported in browsers, so although
… they do have those requirements they aren't part of the common web requirements.
Andreas: Maybe it needs a rethink. There's a tendency to support some technologies by polyfills.
… We can ask our WAVE colleagues how to deal with that and how TTML should fit in this picture.
Xiaohan: Is TTML relatively new?
Nigel: No, it's been a Rec since 2010.
Evan_Liu: Is that the one with multiple versions?
Nigel: No, that's TTML1, that's the base spec.
… TTML2 adds a lot of features for global script layout.
… IMSC has 3 versions, first rec in 2016.
Xiaohan: If you want IMSC implemented in browsers then I can talk to the team.
… It's a chicken and egg problem - we need to know people want to use it too.
Nigel: BBC and Netflix use it a lot.
Andreas: It's widely used for high quality requirements, including speaker identifiers and other features.
… It's used a lot, often with polyfills that implement IMSC.
… Native support would be beneficial, and we talked about it in the past.
… Possibly we can talk about it later on when Eric and Marcos present the TextTrackCue proposal..
Xiaohan: It's supported now by TVs or cinema?
Andreas: There are standards that mandate support such as ATSC 3, HbbTV.
Nigel: By the way, you can upload IMSC to YouTube, and Exoplayer plays them back, so it's already in some Google products.
Evan_Liu: Is it better in every way than WebVTT?
Nigel: [attempts to give a balanced answer] It depends on the user group, I would say yes, but that's from
… an enterprise perspective for the BBC. However for some users wanting to quickly type a subtitle file,
… WebVTT is much more forgiving of errors. However the implementations focus on lowest common
… denominator for WebVTT, so bottom centre white subtitles are fine, but not all the other options.
… And that's not really good enough. Also WebVTT can't do rubys and text decorations and some other
… detailed rendering features that are important.
Xiaohan: A resource showing the differences and the advantages of supporting IMSC would be helpful.
Andreas: It's also important to be aware of the history of the development. We have 2 formats that
… want to solve the same problem. TTML predates WebVTT, but some W3C stakeholders stepped away
… from XML, founded WHATWG, and in that step also moved WebVTT there initially, when TTML
… was close to Rec. Now we have these two formats. Which is the better is as Nigel said, in the eye of the beholder.
… For TTML, a lot of requirements are supported, for different stakeholders.
… The issue at the moment is that some of the bigger companies can afford to build their own
… renderers for TTML to meet all the quality features, but a lot of people just use what is natively
… supported in the browser, WebVTT, which is not sufficient, so we see a lot of degradation in the
… quality experience for the audience. For example a lot of German broadcasters use WebVTT for
… browsers but don't use colour, which is required for speaker identification.
Evan_Liu: Do a lot of big broadcasters use TTML then?
Nigel: I don't have survey results, but when you look at the standards landscape that broadcasters use
… it's dominated by IMSC or the EBU subset of IMSC, EBU-TT-D. WebVTT is at best an optional "also".
Evan_Liu: When we look at usage for WebVTT it is very low so it is hard to justify spending time
… improving support for it.
Francois: Even some renderers of IMSC would use VTTCue to schedule timing, so you might not even be
… able to tell if the subtitles were WebVTT.
Nigel: If you can measure usage of the <track> element that would do it.
Evan_Liu: It looks like 0.15% of all WebVTT scripts include embedded styling, so nobody is using it,
… maybe because it isn't used in other browsers, even though it's in the spec, and Chrome has supported
… it for 4 years.
Xiaohan: do the big VOD players use WebVTT?
Nigel: I don't think so, even YouTube doesn't use it.
Evan_Liu: That's right, there are features that are needed that aren't supported, and some
… users want vertical text, like for Japanese, which isn't in WebVTT.
Nigel: It is in IMSC!
Andreas: Comment to Francois: polyfills must use VTTCue because the generic TextTrackCue
… is not implemented. Only VTTCue can be constructed.
… Regarding the style blocks, it's relatively recent to use them in WebVTT especially in an embedding context.
… It's not used because browsers don't support it enough. People try it out, see it doesn't work,
… then revert to using WebVTT like SRT, just timecode and text.
Matt: Background if it's of use. Our approach is to revert to WebVTT more than I would hope because
… as Nigel said earlier, lowest common denominator is applicable.
… It works well enough on connected TVs even with some styling information.
… We had difficulty with some TTML formats.
… We had difficulty using one format all the way through. It's just not posssible.
… Even when there is support between formats, there is a lack of fidelity in the contents of the file,
… so what you put in isn't what you get out.
… We lack good bodies of test material so developers can check they're propagating all the features
… needed from beginning to end of the chain.
… That's for ITV in the UK, which is also ITV Studios globally. We're a public service broadcaster in the UK
… and a fairly large creator of TV content, and our own streaming platform ITVX. Anything we try to
… do globally is broken by these challenges. We have to be more tactical than I would like..
Nigel: I would love to encourage TTML support in browsers. It would really help a lot of stakeholders and users.
Xiaohan: It would be great to have better explanatory materials to explain why we should spend the effort
… on it. If we implement it and then nobody uses it would not be a good story.
Francois: One aspect of this is to meet regulatory requirements in some regions.
Nigel: Xiaohan if there's anything I can do to support you in having that conversation, let's talk about it.
Andreas: One other topic from MEIG. The comment about all the profiles and formats.
… Yesterday Francois asked about the profiles of TTML. Possibly we need a short explainer.
… I think we may have a list of the profiles, but it may not be clear enough. Usually we get questions
… about the relationships between different formats. Maybe we should work on a document and a
… visualisation of that so it's clearer.
Nigel: Maybe a WG Note with backing across the two communities could help, being sensitive to
… not wanting to be antagonistic to any one community.
Andreas: Having a clear picture for those outside the community would be very helpful in answering
… the questions, which are always very similar.
Xiaohan: If I were writing a document like that I'd want to know the difference from the user
… perspective as well as the broadcaster perspective, and what the differences are.
… Some recommendation about what browsers should do would be helpful.
… As you can see, even though Evan is owning this in Chromium, we're a bit clueless, and there's a
… disconnection between this WG and our team.
Nigel: You're welcome to join, I think things may have improved since last time Google folk were involved.
… Moving on to planning for Thursday's joint meeting with MEIG and APA.
Nigel: [iterates through the agenda]
… From a TTWG perspective, I'm not sure which things will affect us.
… The SSML in DAPT and HTML one will be especially interesting.
… It's directly relevant as we think about how to manage potential additional SSML vocabulary via TTML,
… or if we can mix in directly.
… Anything anyone wants to mention/ask/make sure we cover in Thursday's joint meeting?
Andreas: I will present the work based on DVB's liaison today but this will possibly also affect APA,
… but I will not be able to be present on Thursday.
… Can we make sure we raise it on Thursday?
Nigel: Yes that's an important point, will do.
Break
Nigel: Let's break now until 1130.
[Francois leaves]
HDR in PNG document updates
Using the ITU BT.2100 PQ EOTF with the PNG Format
Nigel: This was a WG Note from 2017 back from the time when we needed to support image based
… subtitles in IMSC Image Profile but PNG did not have native support for HDR.
Github: w3c/
Nigel: Chris Lilley raised this issue because over the years things have changed and this approach
… now appears to have problems.
Nigel: [reviews changes in the related pull request]
Nigel: Correct to say that there are no other comments for now from TTWG?
… I see that this has had review from Cyril Concolato, Chris Needham, Pierre Lemieux.
no comments from the group
SUMMARY: TTWG appears to be happy with this change.
… I've approved the pull request.
Liaison from DVB regarding accessibility services
Nigel: We received an incoming liaison from DVB, visible to members at https://
… Andreas, I think you're able to discuss this?
Andreas: shows slides
Andreas: DVB has two standards soon to be updated, DVB-I and TV-Anytime.
… They are planning to consolidate and extend the signalling
… This presentation shows the generic content, but IPR issues mean we will not be able to minute
… discussions later.
[slide 2]
Andreas: DVB-I allows clients to seamlessly switch between broadcast and IP services.
… It's a metadata specification and system architecture that enables implementation of clientss.
… The main work is in ETSI TS 103 770, which has had one version published.
… The update is planned to be finalized this year.
[slide 3]
Andreas: TV Anytime is ETSI TS 102 833-2-1 and is a metadata specification
… for supporting search, select and use of TV content
… It has limited capability to signal access services.
[slide 4]
Andreas: One driver for change is EU 2019/882 also known as the European Accessibility Act (EEA)
… which mandates access to accessible access to AV media services.
… It will be enforced by June 2025.
… DVB wants to update their specification because of this too.
… The proposed approach is intended to harmonize this.
… There was no standard way to signal accessibility services provided by apps esp HbbTV apps.
[slide 5]
Andreas: Updating and harmonizing AD, subtitling, dialogue enhancement, siggning.
… For subtitling, need to enhance the purpose and type signalling.
[slide 6]
Andreas: New HbbTV signalling capabilities for apps.
[slide 7]
Andreas: Both specs will include work on accessibility signalling.
… Expected to influence other DVB specs than DVB-I in the long term.
… Advantage for client implementers is consistency of signalling.
… Feedback requested to liaison by October 2nd.
… Want to make sure we don't break anything, either implemented or best practices.
[slide 8]
Andreas: The chair of the DVB-I group is Paul Higgs from Huawei.
… Before I go into more details on the value list, any questions on the scope of that work?
Nigel: There's already a list of purposes in the DVB TTML specification. Are they reusing that or wanting
… to change it?
Andreas: Maybe a question for the detailed discussion.
… Worth noting that non-broadcast clients can access DVB-I services.
… There's an HTML5 reference application for DVB-I.
… So this applies more broadly on the web.
Eric: This is meant to be a signal sent along with the content describing the accessibility characteristics of the content?
Andreas: It's not distributed with the content at playout.
… It's ahead of that, a registry that tells clients what accessibility characteristics they could expect alongside
… any particular stream or service.
… It allows the client to select the most appropriate services.
Eric: It presupposes that the user knows the user's preferences?
Andreas: Yes
Eric: Or it presents a menu of options and allows the user to pick what they want?
Andreas: Yes. It also has a programme guide to help the user select a channel.
… Each service should indicate e.g. that it has a subtitle track in a particular language and intended for hard of hearing
Eric: Orthogonal question: have we looked at the European accessibility requirements from
… the perspective of what it's possible to express with the TextTrack specs that we have?
Nigel: You mean the kind attribute in the track element?
Eric: Yes. Do we need more data, or to extend kind?
… I've been blissfully unaware of the act.
Nigel: In this group I think we have not done a formal analysis of that.
Eric: Sounds like we probably should.
Andreas: The main thing that's of interest to this group is to ensure that the UI is accessible.
… It's a subset of WCAG.
… What this technically means will be defined by another ETSI spec, accessibility requirements for ICT products.
… The update of this specification will also be important for this group.
Nigel: Worth reminding that the Directive defers technical details to one or more of the European
… standards bodies before it can come into action.
Andreas: And then each nation in the EU needs to put it into their own laws. It's helpful but not necessary
… to have the technical specification at that time. It's a bit like "safe harbor" in FCC, a minimum set
… of requirements.
… Now I need to go through some details that cannot be documented publicly. These are in the
… liaison that they sent, in the additional material.
[further slides]
New TV Anytime elements (member only)
DVB TTML specification - see table 2
Nigel: Looking at the DVB TTML spec list, I think it's unclear if "same-lang-dialogue" includes
… all dialogue in it's original language or only the dialogue in a single language, with any other language
… text just omitted.
Eric: If there are multiple audio tracks in different languages then you cannot resolve what
… "same lang" or "other lang" means.
… It needs to be useful to the client, so that it can accurately apply these labels to allow the user to make a selection.
…
… I wrote an algorithm for matching up audio track and subtitle track in Webkit, that's open source.
… The code isn't that hard to read.
Andreas: There's also a DECE algorithm.
Nigel: It's public, quite complicated - there's a scoring system.
<MattS> hi all - joining later for the DAPT session...
Break for lunch
Nigel: Back at 1430
WebVTT issues
VTT Metadata Cue format is ambiguous; some metadata may be unintentionally presented to the user in a context outside HTML w3c/webvtt#511
github: w3c/
<jcraig> Slides... https://
James: Hi everyone, we've been talking about this topic for at least a couple of years, we think we
… have a way forward thanks to a suggestion from Gary.
… I'm going to cover problems with VTT metadata today,
… Proposed solution with examples,
… and a Specific new use case for strobing
… [slide 3]
… Example of thumbnails metadata.
… Problem today is ambiguity
… [slide 5]
… Can't tell if it's metadata
… [slide 6]
… Or what type of metadata, e.g. key value pair vs JSON
… [slide 7]
… Proposal: ATTRIBUTES in VTT
… More or less the same as Gary's suggestion, a different name.
… [slide 8]
… Example: Dialog-Only "Subtitles"
… minimal usage is ATTRIBUTES block with a kind: attribute, should be the same as the HTML video track element's kind attribute.
… That definition in HTML is handled, but there is nothing to define VTT as subtitles outside
… that use case, e.g. in an MPEG container.
… srclang is one of the suggestions, here es-mx with a label: Español
… The difference between subtitles and captions being whether sound effects are included for
… the deaf and hard of hearing.
… [slide 9 (?)]
… Another example is descriptions aka Audio Description
… Using text to speech, or text to Braille.
… Users who cannot see the media but want to watch alongside friends and family.
… Or hearing viewers who do not want to disrupt their co-watchers, as Leonie Watson
… mentioned last year.
… The label in this case would be in an audio menu not a subtitle menu.
… [slide 10]
… Metadata example from before. kind: metadata.
… That's why we didn't choose METADATA.
… We introduced "type" where we're maintain a regisitry, TTWG would be a good home for that.
… I chose video-thumbnails here, but it doesn't have an accessible label.
… [slide 11]
… JSON version, with multiple languages of alt text in different languages.
… This allows the previous example to be accessible.
… The video-thumbnails could be in the registry pointing to a spec that defines the JSON format.
… [slide 12] Use Case: Video Strobing
… If you look at the description of #511, this proposal comes further down.
<jcraig> w3c/
James: Also see #512 is the impetus for this particular discussion.
… Apple released a feature in the Spring called Dim Flashing Lights which is a way
… to mitigate flash patterns as they happen in media, for people with light based discomfort or
… epilepsy. We'd like to timecode the risk times with WebVTT metadata.
… [slide 13] "Warning..." that refers to a few seconds of flashing at an unknown point in a 2 hour movie.
… People tell me their partner has to watch ahead to find the flashing section so they can skip over it on second watch.
… This metadata exists, but we'd like to push it forward to viewers.
… I have a small video which I'll show that has some flashing in it.
… [slide 14] genuine warning
… If you're sensitive, cover the bottom left portion of the screen.
… [slide 15, shows video]
… You can see the risk estimation is a lot lower on the right side than the lefft.
… Dim Flashing Lights is on GitHub.
… [slide 16] Open source links
… We auto-mitigate this ourselves without the need for this API by looking ahead in the frame buffer.
… We can't do it on 3rd party hardware though, e.g. AppleTV+ on 3rd party machines where we don't
… have access to the lower level GP level frameworks.
… In addition to the mitigation there are other user level features, like allowing the user to skip
… the sections they don't want. We have shipped something similar for HLS but would like it on the web
… and more standardised in VTT.
… [slide 17] Example metadata for flashing
… We have type: video-strobing which would point to a registry
… We also have intensity, flash type and algoriithm
… There are 3 types of flashing, general, spatial pattern or red. They're all listed in the WCAG.
… Our algorithm can identify general flash.
… This example has us intermingling our algorithm with others.
Nigel: Is your idea that one VTT file identifies flashes discovered by multiple algorithms?
James: Potentially yes
Nigel: And why does the user care about the algorithm?
James: The user probably would not, but it might be useful in choosing behaviour to work around the flash.
Eric: The user agent can use the type of flashing to work out whether to skip or mitigate a different way.
Nigel: That's type rather than algorithm?
James: Ideally this intensity value should be agreed on and testable.
… In some cases different algorithms give different intensity levels for the same flashing.
… The goal of listing the algorithm is to reconcile where that number came from.
… Ideally if we thought it was 90% no matter which one it came from, but if they have very different
… values we might actually trust the worst case.
Evan_Liu: Is this part of the proposal?
James: The proposal today is for the ATTRIBUTES block and this is our first use case.
… [Slide 18]
… Other potential Use cases
… Physical sensitivity warnings, content trigger warnings
… Motion induced vertigo, jump scares etc.
… I've seen these warnings in video games as well, e.g. upcoming violence or suicide.
… We could mark up a bunch of different things depending on what the video or game industry wants.
… [slide 19] Questions?
… That's all of the proposal today.
… We're proposing the ATTRIBUTES block.
… If that's useful I can help with the PR, and Eric could write an implementation.
… We could use this flash pattern as the first point outward from the registry.
Evan_Liu: For your video scrubbing example, it would be up to the content provider to decide when to apply these mitigations?
James: I can't speak for all, but typically for Apple's media library, the content provider just
… submits Yes/No for sensitivity. The goal would be to add this as another asset alongside ingest
… into Apple's library.
… I don't know how others do it.
… Apple produces content too so we'd do it too.
Andreas: I would support both parts of this.
… First question/comment: I think it's necessary because WebVTT is used outside the web context,
… so it's really a requirement to have this.
… For the ATTRIBUTE block you are adding a new attribute, "type". Should this also be added back
… to the <track> element, which has media data but no type.
Eric: A further question: If we do have this, we need to define the processing rules for
… a UA if the kind or type attributes in the HTML and the WebVTT document don't agree.
… If I know there's an ATTRIBUTES block do I even need the attributes in the track element?
… The only benefit to having the attributes on the track element is if someone is looking at the source to the web page..
Nigel: Doesn't that force the UA to download all the track element resources on page load to work out what to do?
Eric: That's a very good point
… Right now the mode by default is disabled which means nothing is loaded, but you still need
… to construct al the UI
Gary: I think that answers what to do if there are conflicting track attributes.
Eric: Yes, the track element should always win
James: That happens in SVG images with an img element where both have a label, the local one wins.
Gary: I think we didn't really answer if the idea is that we want a new type attribute in the HTML
Andreas: I wanted to add that - do we need attributes at all in the VTT - they can be used for other formats.
Eric: I think we do. Maybe type is too generic a name for it.
… As we proposed it here it is metadata type
Gary: It probably is better for the name in the VTT file to have a similar name to the HTML attribute
… but it doesn't actually have to be.
Eric: It will confuse people if they're different
Gary: The bikeshed name is metadata_type.
Eric: That makes sense
James: If that's the case we would have the same metadata_type name in both places.
… kind: metadata
… metadata_type: video_strobing
Evan_Liu: An enumerated list?
James: A registry of some kind, can be an informative resource.
… Some table that reserves a type value and has a pointer to where it is defined.
Andreas: In TTML, possibly in HTML too, you can add private values if you prefix with x-...
Eric: Yes it would be a good idea to have a rule for "official" types e.g. it cannot start
… with x- or whatever, or to override, then it must start with something like data-
… Having one place where people can go to find out how to author is a good idea.
James: I like the idea of supporting prototyping and then adding to the registry
Gary: Would the registry point to a note or a spec, or just reserve the name
Eric: It should point to a spec or a note
James: I think it should point to a spec
Nigel: Typically with a Registry we need to define the rules for adding values, so we could require
… a pointer to a document with a URL, and whatever other changes.
Gary: As long as this group is not responsible for writing those documents.
Andreas: This new block is for defining new attributes, and I can see people using it.
… Would you allow others to add their own new attributes to the block?
Eric: As long as we define the parsing rules in the same way as VTT does now,
… like take the first word, look in the list, if it's not there skip to the next line etc.
… Or an older user agent would skip things it didn't know about.
Gary: That's probably fine. My main concern is if someone uses some attribute that we want to add
… then the value might break.
James: That's why we have an interop problem in HTML now.
Nigel: Question about syntax. What level of complexity do we need to plan to support.
… How did you get to that key: value syntax?
James: I just thought most folk would want key value pairs, we could use javascript
Eric: Most of the time we aren't going to need multiple lines
James: In an attributes block I can't think of one
Eric: You might want something like alt text that describes the content of the file
… you could have a different block for that though
James: That's not defined by this spec?
Eric: Or a licence
Andreas: In attributes you cannot put multiple lines
Nigel: You can put escaped new line characters in
Andreas: Do we need that here?
Nigel: That's the question
Gary: You don't want multi-line attribute values
Nigel: alt text is the classic example
James: Like you said, you can use \n
Nigel: That's a question for the syntax specification, do we want to escape new lines
Gary: Does WebVTT support that now?
Eric: Not right now I don't think it does
James: One of the goals is to not break existing parsers.
Eric: You can use the existing parser for that as long as you don't have a blank line within a block
Andreas: One important question is if this new addition should be in the Rec track version of the document.
… Is this a proposal to add to the CR or the next upcoming stable version?
Gary: One thing I've been for a while trying to do is to try and get just what is currently
… implemented in VTT out as Rec. Adding new features would make that harder.
… That said, it doesn't mean we can't work on the feature.
… It might be worth adding it anyway before Rec.
… For me, and I should check how to do it, there's the evergreen spec stuff where small features
… can be added into Rec more easily and this seems like a good candidate for that.
Eric: I agree, it would be a shame to postpone Rec any more than we have to.
… Not having it in v1 of the spec isn't going to keep us from adding that to the feature set.
James: It can be in the ED or WD as soon as the group thinks it's a good idea.
… As soon as everyone who has a stake in that thinks it is ready to go we can take a flag off it.
Andreas: The registry can be created anyway.
Atsushi: I think we decided to bring the spec solely into TTWG? I haven't seen anything on that for a few years.
… Mostly the spec was developed under CG.
Nigel: I'm puzzled by the question
Gary: The CG still exists but noone is involved in advancing it from that side.
… Maybe we should close the CG?
Atsushi: When I looked a few years ago there was an objection to closing the CG.
Andreas: CGs can only work on CG drafts, so only we can work on the Rec track document
Atsushi: That is one of my naive questions. When I tried to update the repo to be just TTWG I believe
… someone objected.
Gary: I haven't heard any objections
SUMMARY: Strong support for this new ATTRIBUTE block but we probably don't want this to hold up the current version of WebVTT from progressing to Rec
Evan_Liu: If the UA does not support these new features does it need to parse the new block?
Eric: According to the current parsing algorithm it will not break anything
James: We'll work on a PR. Thank you.
Gary: Thanks.
DAPT issues
Eric and James and Evan_Liu leave
Matt and Cyril join
Cyril: Worth summarising the situation with Wide Review.
Nigel: Good idea
Nigel: It's on APA's agenda, not sure when they will get around to it or if they hope to discuss it with us
… in the joint meeting on Thursday
… Internationalisation: We have not made the review request yet.
Cyril: No, the template needs a pointer to the Explainer. Now that we have the Explainer
… there is no blocker, I just need to do it.
Nigel: I encourage everyone to check through this
Cyril: There's an explainer for how to write an explainer, so we used that as a template.
… We didn't invent a lot of new text.
Nigel: No, though I didn't do a comparison with other text elsewhere.
… Privacy and Security reviews are both requested
Nigel: Lastly, TAG review
Nigel: Has this been done?
Cyril: No not yet
… It's on my list - TAG and i18n are on my list
Nigel: That's the Horizontal Review part of Wide Review.
… The other part, contacting external organisations etc. is logged at
Nigel: In terms of responses, we did not receive many.
… ARIB responded to say they have no comments.
… And we got several acknowledgements of receipt of our messages
Cyril: I added Disney/Hulu, Warner, Apple and CableLabs.
… I will have a follow-up with Disney, they want to understand a bit more.
… I actually sent a message to James Craig about this to Apple, he hasn't responded yet.
Nigel: I have had very positive responses from two organisations, one wanting to be the first
… implementers! Another said they had already started using it, and it has been very smooth.
Cyril: I'm going to raise the remaining HR issues. How long should I give them - a month and a half, say?
Nigel: Sounds reasonable to me.
Nigel: [looks through the pull requests]
… Seems like there's nothing to discuss on these now, they need some work to be done.
Cyril: The best use of our time is to review those offline.
Nigel: Looking at the issues...
Consider removal of distinction between script and transcript in chapter 2.1 w3c/dapt#175
github: w3c/
Andreas: I'm not sure if my point was clear or if you understood it?
Cyril: My understanding is that you're saying "DAPT Script", because it can refer either to
… a transcript or a script, is confusing. Is that right?
Andreas: You say in §2.1 that script and transcript are mutually exclusive in their meanings, but
… then you use one of them as a superset of the other.
… This basic description of transcript and script doesn't seem necessary. You always label
… a specific script with something else like "pre-recording script". The categorisation doesn't work for me,
… language-wise.
Cyril: Trying to see how to address your concern.
… One way is to remove the paragraph and just call everything a script?
Andreas: Yes that would work for me
Cyril: Or if we change DAPT Script to something else that doesn't use the word "Script" would that resolve it?
Andreas: I'm not sure if "DAPT Script" is the only use of "script" to make both.
Cyril: The first document you can produce is a transcript.
… Start by transcribing the original language, for dubbing.
… Then by adjusting and translating the transcript you produce the dub script.
Andreas: Ok
Cyril: I actually like the text, it's clear what is a transcript and a script. I can understand that
… "DAPT Script" being possibly a transcript could be confusing. That's why I was asking if changing that
… term could be less confusing.
Matt: From a linguistic point of view you can only ever transcribe something that already exists.
… You could end up with a script that is a transcript. You're unlikely to need a transcript of audio description,
… with the workflow you would try not to be in a scenario where you have no script for audio description.
… I can see the confusion but to my mind the origination of a transcript is always very clear.
… A script is forward looking, a transcript is backward looking.
Andreas: That was a comment I made long ago. By reading through it and making sense of it
… without being an expert in the domain of localisation. If there's a clear understanding of this
… differentiation then there may not be an issue. I just wanted to make sure that this is clear to others.
… If it's used this way in localisation it should be kept this way.
Matt: I definitely think they are not used interchangeable. But "script" has a broader range of meanings
… than "transcript".
Andreas: Would you say a "transcript" is not a "script"?
Matt: I would say a "transcript" is a kind of "script".
… From a workflow point of view a script is forward looking, but linguistically a transcript is a
… form of script.
Cyril: How do we resolve this issue?
… Is modifying the first paragraph to say that a transcript is a special type of script - would that help?
Andreas: That would be clearer to me at least, but then you have to rewrite the whole paragraph.
… Currently they seem to be mutually exclusive.
Nigel: Ok, I think I can do that.
SUMMARY: Reword §2.1 para 1 to allow for "script" to include "transcript" as a special case.
Reconsidering the role of character styles w3c/dapt#124
github: w3c/
Cyril: The current text is kind of confusing because styles can be used for two things.
… One is indicating colors and font etc. when recording the script, for example in the dubbing application
… when the voice actor is recording.
… The other is when you're producing subtitles from transcripts for example, to preserve styles.
… The ambiguity is because the spec isn't clear which use case it is trying to resolve.
… We talked about it, and I discussed it with my Netflix colleagues.
… At this point I would be happy to remove every mention of styles from the spec,
… or just have a note that says that if you transform a DAPT Script into subtitles or captions you can
… preserve styles from a DAPT script.
… But really a DAPT script is not meant to be displayed as-is.
Nigel: When we talked about it Cyril we made the distinction between styling for thousands of audience memberss
… vs styling for individual voice artists.
Matt: When I saw style I was a bit confused.
Cyril: So removing styles would be okay?
Matt: I think so yes
Cyril: One of our use cases is to put Japanese text for someone on the right. If they could describe
… the positioning while transcribing then that can be used for subtitles later.
Matt: I can see similar examples avoiding graphics.
… I would agree, from a workflow point of view we wouldn't be getting the same people to do the styling.
SUMMARY: Simplify the specification by removing the styles, adding a note that styles may be used.
Nigel: That's assigned to you Cyril.
Consider identifying the original language on top of the current language w3c/dapt#148
github: w3c/
Cyril: I don't know where we ended up with this.
… I proposed having a langSrc="pivot" value.
Nigel: Then if you reused a translation as a pivot then you'd have to change the value of "langSrc" everywhere.
Cyril: What's the other proposal?
Nigel: I proposed to identify the language from which the Text was translated.
group discussion
Atsushi: Strong +1 to keeping a trail of the source language from which text was translated.
SUMMARY: Change it so omitting langSrc implies it is original, otherwise the value of langSrc is the translation source language code.
Break
Nigel: Back at 1700
<MattS> have dropped off the call now - have a great rest of session!
TextTrackCue presentation
Matt has left
Marcos, Pierre and Andreas Bovens join
Evan Liu returns
Marcos: I will present for 20 minutes, will post the slides
… Enhancements to TextTrackCue and WebVTT
… Why? Accessibility, Privacy and User control
… Users can define caption styles with system accessibility settings in OSes,
… e.g. font size, text and background color
… We want to support custom format but have them rendered by the browser engine
… For privacy, styles must not be leaked to the page
… Consistency across sites and overall experiences is the goal, including in Picture in Picture,
… live captioned video calls etc.
… Demo by Eric
Eric: [shares screen]
… What we have here is a cartoon I downloaded from YouTube. I downloaded the captions
… that YouTube generated.
… I generated an IMSC file from them.
… When we play it you see it's basically the YouTube style captions.
… The one supplied by the website.
… However if I go into my system settings, accessibility, captions I have my machine configured
… to show the subtitles a different way. If these captions were WebVTT based then the browser
… would render them and style them.
… I have a trial implementation of what we will propose, in Webkit.
… The way these are rendered uses imsc.js.
… I modified it to tag parts of the cue to identify the foreground and background as it's parsed.
… When I click this button I hide the imscJS output and instead show the text track populated
… with the DOM node generated by imscJS. I added attributes to the foreground and background.
… Even though the browser didn't parse it it is rendering the cues and using the style I chose in the system settings.
… But imscJS can't know anything about it and instead the authored styles are preserved.
Marcos: Changes to HTML
… TextTrackCue interface changes. Add a constructor with startTime and endTime and document fragment
… docFragment restricted to br, img, div, p, rb, rt, rtc, ruby, span, i, b which allows us to support
… all of WebVTT.
… usage example: we steal the first child element.
… Requirement is to have a cue and a background, can be the same element.
… 2 new global attributes: cuebackground and cue
… Serves as ::cue and ::cue-background pseudo elements
… Add .getCueAsHTML() method to TextTrackCue.
… May be able to move it down and have it backwards compatible.
… Or override getCueAsHTML()
Nigel: Move down?
Eric: Currently method on VTTCue, proposing to move it to TextTrackCue unless it causes a problem
… in which case have it on both classes and override in VTTCue.
Marcos: Add a private [[legacyUpdateFragment]](docFrag) internal method.
… Legacy to support WebVTT's .text attribute mutations.
… We don't want to support this mutation, there's no need for it.
Andreas: I don't understand this mutation
Eric: VTTCue's text attribute is read/write. You can modify it after construction.
… In hindsight that was probably not a good idea. We don't want to support it on TextTrackCue
Nigel: Why was it not a good idea?
Marcos: It's expensive, and there seems to be no use case in place of creating a new one.
Andreas: What about in an editor environment?
Marcos: TextTrackCue takes a document fragment and validates it so we don't want to support mutating it.
Eric: Internally the browser will copy that fragment, so even if you did change it nothing would happen.
Marcos: In the WebVTT world there was no assumption that there would be DOM nodes, but in reality
… that's how everyone is using it.
… [shows CSS styling using pseudo selectors]
… Make the constructor call into TextTrackCue. It should "call super". It's missing now, not a big deal.
… All parsed VTT results in a parsed HTML fragment by default.
Eric: It's just spec text change, you wouldn't be able to observe the difference.
Marcos: Relocate ::cue CSS spec text to CSS pseudo elements specification and specify ::cue-background
… I taked to fantasai - she thinks CSS behaviour should be moved to CSS specs.
… Then it's in an independent spec that others can use.
… Let the minutes reflect that we love and trust the CSS WG.
… Open questions.
… In WebKit today, TextTrackCue supports more than WebVTT.
… br, img, div, p, rb, rt, rtc, ruby, span
… WebVTT doesn't support img, for instance, should we reconcile this?
Eric: It was a low effort "week of code" work to implement.
Marcos: If this looks good we can open a pull request with no pressure, and we'd like a second implementer
https://
Andreas: Thanks a lot for the presentation.
… We have discussed this for several years, it would be great to see this moving.
… One thing you mentioned is that it covers everything in WebVTT plus some more vocabulary
Andreas_Bovers and Marcos_Caceres leave
Andreas: Does it support everything in IMSC?
Eric: That's how I came up with this list. The list that's there is based on the node types
… that imscJS uses.
Gary: I was going to say that images in WebVTT has been brought up before?
Eric: It has
Gary: I don't see a reason not to allow it
… at least we might want something similar, e.g. to support non-HTML processors.
… It might need a similar allotment for images.
Eric: Wouldn't we have to extend the syntax.
… There's no markup to signal insertion of an image in WebVTT.
Gary: I was imagining an image tag like thing
… There are also privacy implications, that can be mitigated by inlining the image.
Eric: Yes, we'd have to define the processing rules, cross site behaviour etc.
Nigel: I would steer clear of images unless you have a strong use case.
Eric: I talked to Tess about it, thought about the pros and cons, thought we should add.
… I agree it would be better not to because it causes accessibility problems and is a hot button issue
… for some people.
Gary: We could always add it later.
Eric: Sure, if there's a strong use case.
… It makes it easier not to.
Nigel: There would be other options too, like IMSC or some other format.
Andreas: It's only about adding it to WebVTT?
Eric: It's just for parity. There is a reason not to add image, but it's the same as not adding to TextTrackCue.
Andreas: The case for images is where there's no font support for a language.
Eric: The issue is you cannot restrict it to that case.
Andreas: Any data points about usage of image?
Pierre: There are bug reports on imscJS asking for support for images, so it is absolutely in use.
Cyril: The link to the explainer in the minutes is broken. Also please share the slides.
Eric: Marcos is going to share the slides - he had to run to another group he's sharing.
Cyril: The question in terms of API: is there a way for a website to opt out of system level override?
… The Netflix use case is to apply a user's Netflix-wide caption settings rather than the system settings.
Eric: Then why use this API at all?
… The answer is an emphatic "no" because it's important to make it difficult for sites to opt out.
Gary: When you give it the document fragment does it require the cue and cuebackground attributes?
Eric: Yes, it throws if they aren't there.
… If you test this you'll find it generally works but isn't shippable yet. The interaction between the
… user's styles and the inline styles isn't quite right. The specificity of the CSS can lead to it doing the wrong thing
… but that's not going to be hard to fix.
<eric_carlson> https://
Nigel: I want to know more about those attributes, and if there are any other constraints.
… Is there a maximum number of elements with each attribute?
Eric: they're just used for CSS styling, so it's fine to have any number.
Gary: Just like having two cues with the same time period.
Eric: Yes
Nigel: In the past we've had some queries about how to understand what the source styling is.
… I have a spidey-sense that that could be the detail in which the devil resides. Can we write down
… the expected behaviour here?
Eric: That's a system implementation issue.
… The "window" colour is applied to the element with the cue-background attribute, and the
… "background" is applies to the element with the cue attribute.
… In Apple's system settings.
Andreas: This would be a considerable improvement in the situation.
… This morning we also had some discussion of native support of formats like IMSC.
… I'm wondering how this could help for that.
… As Marco mentioned, WebVTT already defines how to translate itself into HTML.
… In case there's interest in the browser industry to support other formats, would a similar mapping
… from IMSC to HTML help?
Eric: I don't think this is going to make it any easier. When I added the support we have in WebKit
… it was 2 days of work or something. If a source base supports WebVTT and displays it using the shadow dom,
… as all the browsers do, it's easy to take the fragment and put it into the shadow DOM,
… It's just coming from a different place.
… From script rather than the VTT parser.
… The issue is the parser. They're notoriously hard to get right and a major source of security
… vulnerabilities. The major effort in supporting any format is the parser, getting it right and doing it safely.
… It would be many orders of magnitude more complex.
… I may be overestimating the work, it's significantly more than this.
Andreas: Do you expect that the document fragment would be generated outside the browser?
Eric: Yes, and it sanitises it and then sticks it in the shadow DOM
Nigel: What sanitising?
Eric: There's an API for this now, making sure it has no scripts etc.
… Fixes up dangling nodes, makes sure it's safe and filters out the things that are permitted.
Pierre: Quick question. I think you talked about this at the beginning. Is it to merely match the
… capabilities or mimics a subset of WebVTT, or to support an intersection of WebVTT and IMSC. In your
… mind what's the scope of this?
Eric: In my mind it's to allow people who can't use WebVTT to do what they're doing now but
… to hand over rendering as well as scheduling to the browser, so we can display the captions in PiP,
… full screen etc to make the experience better for the user and the developer.
… The list of node types in there now are based on what is in imscJS. That's how I came up with the list.
Pierre: There's one missing though, region, which is why I'm asking.
… It's useful to semantically associate subtitles across events.
Eric: If it's important we can add it.
Pierre: Going from memory, things like multi-row align, and others that are considered essential
… for subtitles or need to be added into CSS.
Eric: If it's not supported in CSS that's where the discussion needs to happen.
… What we should do is make it possible for captions to work given the constraints we have with CSS
… If there are problems with CSS we should fix that rather than going around it.
Pierre: Exactly, that's why I'm asking.
Gary: Hopefully when CSS gets the feature then this is an opportunity to get it.
Pierre: Right, this is HTML though, so if there's a feature missing in HTML and CSS this would be the
… right time to address it.
Eric: I wholeheartedly agree.
Gary: Yes it does help make the case that these should be revisited.
Pierre: linePadding is another one. You can polyfill in imscJS
Nigel: At the moment imscJS does things to support fillLineGap and linePadding, but it can only do it
… by manipulating the content in the DOM. If it no longer has access to that then it can't support those
… features unless the CSS layer natively supports them.
… Is there a way around that?
Eric: The script simply can't access the shadow DOM. We're going to have to figure it out.
<Zakim> nigel, you wanted to ask about speaker colours
Gary: Two questions.
… How free are the styles the document fragment can have?
… Is it all of CSS?
Eric: Currently, yes, but it's an open question as to whether it should be or not.
… We will have to define that because the VTT spec defines what you can apply using the pseudo elements.
… Maybe that's wrong, it happens automatically and I use the same pseudo class to define it as for VTT.
… It's possible some things are disallowed.
… If you had a style element...
Eric: It would throw, it's not allowed.
Gary: Second question, related to what Pierre brought up, around regions and VTT regions.
… It would be good to see how that could be implemented.
Eric: Yes, and there are new things in CSS that make more sense than what's in WebVTT
Gary: In a live scenario you may not have all the cues available ahead of time.
… Maybe the region API would be an extension of the VTT Region API.
Eric: maybe
Nigel: I'm wondering what the path forward is to being able to provide speaker identification data
… so that in the settings e.g. Apple's ones you don't just have the override all vs don't override options.
Eric: It's the user's choice
Nigel: That's not good enough - it breaks the usability
Eric: If you want to prevent color overrides that's a mistake
Nigel: That's not my target, it's to allow users to override per-speaker color, not just have a single choice.
Andreas: The immersive captions community group has produced a similar set of examples.
… An enhancement would be to provide a set of favoured colours and replace them.
Eric: That's a lot of user interface complexity.
Nigel: It's a system implementation issue but if the input data isn't there then there's no hope of them implementing it at all.
Andreas: Two things. First, adding bleeding edge CSS functionality that's not in IMSC or WebVTT would be a good thing
… for example the proposal from BBC to support styling that's in CSS but not translated into TTML styles yet.
… Maybe even if some styles are not defined in WebVTT, but they would be supported, that's a good
… thing.
Eric: Let's come up with a list
Nigel: I think support everything, why not?
Andreas: If they're not causing any harm, yes
Eric: Some things like background-image could be harmful. We need a list.
Andreas: Note that the player also has some user settings too and there's a need for how this is
… implemented and best practices.
Nigel: Want to register support for this.
Gary: Also support.
… WebVTT has these Node objects that can be quite complex to target with CSS.
… Do we want to support that?
Eric: ??
Gary: There's a voice object, e.g. ::cue(v.mary) - you would potentially want to support a user
… writing CSS to target that through this API.
Eric: That's a great point, I hadn't thought about that.
… We should work out how to identify those nodes.
Gary: Some are easy, a b element is just a b element. Others like voice are trickier.
Eric: Given that we can use inline style we may not need it, I haven't thought about it.
Gary: I'm thinking of it as a developer with an API, where the user is using our player on a page and
… they want to target CSS like saying mary should always be green.
Eric: Would that work just with class?
Gary: It should, but the same selector should work with either WebVTT or TextTrackCue
Eric: Let's look into that
… It should do, yes
Nigel: Do we need to discuss webvtt#516?
Eric: If we define the cue-background attribute and the CSS pseudo element then that issue
… will be solved.
Nigel: But it's not in WebVTT now?
Gary: This is to support window color. It would need a change in WebVTT.
… Doing that would align it more closely with the regulations, so there is a benefit to doing it.
… This also falls under the great changes we want in WebVTT but shouldn't block move to CR.
… I think the constructor one can go in as long as its backwards compatible.
Eric: It should be backwards compatible so it's just the semantics of the text.
… It won't change any implementation, it's just fixing up the spec text.
Nigel: Is the ability to construct a TextTrackCue not an implementation change?
Eric: I'm just talking about updating the wording of the text in the spec to describe what
… happens behind the scenes.
Gary: The wording could say something like "you might create a TextTrackCue if the ctor is available" potentially,
… wording to be figured out exactly.
Nigel: Just musing over the richer customisation options question, you can use CSS selectors to select
… based on attribute values or presence, so if there were some industry convergence on a set of
… attributes against which styling could be customised, then that could be extended to improve the
… options available to users.
Eric: Yes.
Andreas: Just to get my head around the changes proposed.
… As I understood, there will be a TextTrackCue ctor, so that will be a change in the HTML spec?
Eric: Yes.
… And we need to change both specs if its compatible to move the getCueAsHtml method to TextTrackCue
Gary: As a follow-up to that, would it include the special attribute tags?
Eric: yes
Gary: If you create a VTTCue and all getCueAsHtml() should that include the attribute tags?
Eric: Yes. Your VTT file would have to define the background. It's an implementation detail.
Gary: There's a chance that if there are issues then VTTCue.getCueAsHtml() wouldn't have those,
… that could potentially cause an issue.
Eric: I don't think there will be. We already have it as a private pseudo-class that goes onto the
… DOM node that holds those cues so we can apply the styles.
Gary: There's a potential issue if someone is not expecting these new attributes and their
… code breaks if they suddenly appear.
Nigel: Related to that, what you get back isn't what you put in, but is the sanitised version?
Eric: Yes, it's a sanitised cloned version so the script doesn't have access to the shadow DOM.
Gary: I'd expect it to be as close to what is being rendered as possible.
Eric: Yes, in our implementation I clone the node from the shadow DOM.
Nigel: Thanks for this, it's been a great discussion.
AOB Andreas
Andreas: Just want to follow up on discussions. There is a need to increase interoperability,
… and Pierre and I are looking at organising a meeting at NAB next year to bring industry together.
… Good to align with an other event. We are testing the water and checking interest in a half day
… event at NAB where the time could be used as a meeting or alternatively as a kind of subtitling plugfest
… where implementations of browsers or TVs can be tested against content by different providers.
… We will be happy to receive thoughts on this. I will send a message to the TTWG reflector, and yesterday
… I also sent one to the MEIG.
Meeting close
Gary: That was a good discussion there, I'm happy that there's some life in WebVTT still!
Nigel: Yes, thanks everyone for your great input today, we've had some great discussions and they
… weren't all ones I expected at the beginning of the day.
… We meet again on Thursday afternoon with APA and MEIG.
Gary: Thanks for taking minutes Nigel!
Nigel: Thanks!
… [adjourns meeting]