Meeting minutes
Agenda
- Welcome
- Brief recap of media discussions at TPAC : 5 mins
- Hybridcast Updates with NHK: 10 mins
- NHK work and proposals for Application Development for Consumer Products TF: 15mins
- AOB
Brief review of media topics atTPAC 2022
<cpn> slideset:
https://
ChrisN: During the MEIG
meeting there was
… discussion on TV application
development.
… Includes the developer experience, performance
issues, and media device specific APIs.
… Would like to move that forward, invite
interested people.
… We will talk more about that today.
… Then, CTA WAVE HTML Application TF and Web Meda
API snapshot updates.
… WebRTC receive capability is being added,
possibly WASM in a future updates.
… There was a presentation from Fraunhofer on next
generation audio codec support.
… During the joint meeting with WebRTC and Media
WG, discussed new activity starting up
… focusing on the media handling architecture in
browsers, and any possible causes of inefficiency or API
difficulties, e.g., where media is passed between different Web
APIs, such as WebRTC to WebCodecs, or to WebGPU, etc.
… There is a new CG on Audiovisual Media Formats,
initiated by Dolby, looking at browser support for commercial media
formats.
… Everybody is welcome to join to participate in
the discussion
… In the Media WG meeting we discussed MSE feature
prioritization,
… including pre-emptive eviction for low memory
platforms, which we hope unlocks MSE on more platforms
… Also DASH emsg support, and follow up discussion
in DASH-IF.
… May still be too early to standardise emsg
handling in MSE, want to get implementation feedback based on
JavaScript players.
… For EME v2, aiming for FPWD during the Charter
period (May 2023)
… Media Session API, agreed to add actions to
support slide presentations over WebRTC, next/previous
slide
… Audio Focus API relates to Media
Session.
… Example use case of playing music vs notification
sounds.
… Interest from Apple and Mozilla to develop an
API
… WebCodecs and VideoFrame associated metadata,
e.g., from analyzing the video for face detection.
… Agreed to use a registry approach.
… Introduction for Document PiP API, use case might
be a video player with custom player controls in a PiP
window.
… That is a brief summary, I haven't covered
everything, though.
Kaz: Regarding, consumer device TF discussion, as I mentioned at TPAC it would make sense to collect current practices, including from NHK
Hybridcast updates
slideset:
https://
Sato: ToC
… This is an update on IPTVF, then I'll cover some
of NHK's R&D activities
… [slide 3]
… Updates on IPTV Forum Japan
… [slide 4]
… Hybridcast diffusion status
… Annual shipments and total number of Hybridcast
TVs
… Hybridcast is available on over 300 TV models,
with 20 million units sold.
… [slide 5]
… NHK's R&D activity
… [slide 6]
… Media service for various devices. Hybridcast
Connect supports broadcast and online video.
… MTE allows triggering of emergency alerts. We're
looking at how these can be used in a connected home environment
with WoT.
… [slide 7]
… Video streaming (diagram of their
system)
… Broadcast station
Hybridcast TV
… [slide 8]
… Software tools for video streaming
… We have developed open source software tools,
published in March 2022
… Basjoo.js is a DASH player for the IPTVF Japan
profile on TVs
… Relay-miffe is for inserting Media Timed Events
(MTE) with CMAF ultra-low latency
… Delivery-miffe is a CMAF-ULL streaming server.
Please take a look at these in GitHub
… [slide 9]
… Device collaboration
… Broadcast station -> Hybridcast TV ->
Hybridcast Connect -> Devices
… [slide 10]
… Linkage between Hybridcast and WoT
… We presented on WoT-enabled TV services using
Hybirdcast Connect at TPAC 2022
…
… And gave a demo during the WoT
breakout
… [slide 11]
… OSS for Hybridcast Connect learners
… We have open source software for Hybridcast
Connect as well.
… The Hybridcast Connect TV emulator implements the
Hybridcast Connect APIs
… We have a reference SDK in Java and JavaScript,
and client sample application.
… This allows a client device to connect to the
Hybridcast TV and interact with it using the Hybridcast Connect
protocol
… [slide 12]
… Resources
… You can find more information in our NHK STRL
OPEN HOUSE 2022 web page
… https://
… and in our GitHub repository at https://
… Thank you!
rrsagnt, draft minutes
ChrisN: Really
interesting, thank you.
… Can you explain a bit more about the WoT
demo?
… What's happening here?
Sato: (shows their slides on the demo itself)
https://
Sato: We gave this
presentation during the WoT demo breakout by Endo from NHK
… The demonstration includes a soccer program on TV
and an air cleaner
… The air cleaner's noise would be lower depending
on the soccer game's status, e.g., when a goal is scored
ChrisN: Thank you
Sato: (shows the demo
content within the slides)
… An event message sent from the TV side to the air
cleaner side depending on the soccer game's situation, e.g., to
reduce the noise level when a goal is scored.
ChrisN: Is this MTE-based?
Sato: Yes
Ohmata: It's originally
based on the event message from the TV program
… There are trigger messages, e.g., start, goal,
replay, etc,
… handled by WoT and Hybridcast
protocol.
ChrisN: That makes
sense.
… It would be interesting to see the detail of the
mechanism,
… but don't have enough time today, though. Perhaps
we can organise some follow up meetings to explore
further.
… Do you have a WoT Thing Description to control
the TV?
… I'm wondering if we should discuss the detail
during the future calls?
Ohmata: WoT protocol and Hybridcast protocol handle the message
Kaz: Maybe you can show the p9 from Endo-san's slide set to describe the mechanism
Kaz: The TV also has a WoT Thing Description. Two devices using the WoT mechanism
Kaz: This would be a
good start for collaboration with MEIG and WoT. We discussed with
WebTransport people at TPAC, a use case is surveillance cameras,
adding metadata annotations, handling related devices using WoT and
MTE.
… That kind of collaboration would be
useful.
ChrisN: OK, let's plan to discuss this in more detail
NHK work and proposals for Application Development for Consumer Products TF
ChrisN: MEIG recently
produced a charter for futher work on this topic in a TF.
… We're interested to hear your experiences, and
opinions.
Yasuoka: Case studies
of Hybridcast application development...
… [slide 2]
… Hybridcast has been in service for around 10
years, in 300 TV models, 20 million units sold
… [slide 3]
… Stakeholders and their relationships
… IPTV Forum Japan generates Hybridcast standards,
operational guidelines, and test suites
… for various TV manufacturers and application
developers (e.g., brodcasters)
… [slide 4]
… Performance difference
… To achieve the expected behaviour in TV
applications, the performance difference among various devices is a
big issue.
… Typically, each broadcaster has to consult with
each manufacturer individually to resolve issues.
… [slide 6]
… We have two case studies to present.
… Case 1: Animation rendering
performance
… We wanted to use CSS animation effects, e.g.,
when changing the focused element, but found that rendering is not
smooth in some TVs.
… We asked the TV manufactures to improve the
device, but couldn't.
… So our only option was to avoid using the CSS
transition properties, to improve the performance.
… [slide 7]
… Case 2: Process when switching
services
… For linear video services, there is latency when
switching between services, which affects the service
quality.
… This also affects switching between broadcast and
streaming, and vice versa.
… Switching delay may prevent users from watching a
part of video program.
… It also has a risk that emergency notifications
may not be displayed promptly.
… When switching services, quick switching after
getting the trigger is expected
… But it's difficult to control what will be
displayed on the screen, when launching the TV application, due to
the performance difference.
… We want to avoid showing a blank
screen.
… [slide 9]
… Performance test activities (by the
IPTVF-J)
… Our goal is to improve the UX for Hybridcast
services, so we are sharing issues and experiences between
manufacturers and broadcasters.
… Test categories include screen rendering,
application launch, interactions with companion devices, and memory
usage
… [slide 10]
… Test 1: Animation by sprite sheet
… We created a test sequence, 8 seconds horizontal
movement of an image, with 4 animation frames.
… We measured it 10 times on 6 different
TVs
… The latency results vary between different TVs
and by execution timing.
… The expectation was 8 seconds every time, but the
graph shows the measured variation in elapsed time.
… [slide 11]
… Test 2: Browser launch time
… Here we measure the time span between pressing
the button to launch the application, and the application window
being ready.
… This includes a request to fetch the AIT file,
and a request for the HTML content.
… The time varies between TVs, especially in each
execution process
… [slide 12]
… Conclusion and discussion
… We presented past and ongoing challenges on
performance issues.
… How can application developers and TV vendors
contribute to the TF?
… What can they expect from the TF?
… We can provide our use cases based on our
experience.
… Questions? Comments?
ChrisN: Thanks! ChrisL, any comments?
ChrisL: Interesting
discussion on pushing Web and browsers for applications on TV
… It's major challenge, need to increase the
performance for TV apps.
… For transitions and animations, the performance
is not enough,
… and we also need to reduce the animations,
etc.
… Not sure if you're aware of Lightning framework
which uses WebGL rendering
… let me post the resource
<ChrisLorenzo>
Developer Experience -
https://
<ChrisLorenzo>
https://
<ChrisLorenzo>
Framework for performance on TV apps - https://
ChrisN: There are some
aspects here that we've not considered yet for the TF
discussion
… What you describe is very much consistent with
what ChrisL has been mentioning
… One of the techniques we use is detecting the
device model and selectively enabling the features,
… which allows some features to be used if we have
tested that the performance is good enough.
… The framework allows us to conditionally
able/disable the feature.
… If less capable, disable the
feature.
… The other thing interesting is start up
performance time for Hybridcast applications, which we haven't
talked about before.
Yasuoka: (shows the Test 2 results again)
ChrisN: Any insights to
what is the cause of the delays here?
… How does it vary with the browser
implementations?
Yasuoka: It's difficult to describe the detail
ChrisN: I wonder if
there are additional measurements or metrics we could use to see
where the TV is spending the time?
… e.g., the browser itself is a large application
binary and needs to load a lot into memory at start up
… Could there be a way to instrument it to get more
details?
Yasuoka: In the graph,
the green part depends on the network connection
… We would need to dig into the detailed behavior
of Hybridcast implementation
ChrisL: So the user
press the launch button at time 0
… and the graph shows the time until the browser is
ready receive the response
… There's an HTTP connection, then load the page
content
… Browsers do that quickly, actually
… What the difference between A-J here?
Yasuoka: Those are the different TV models, available in Japan
ChrisL: OK. This is
very similar what we have seen with smartphone variation
… A and J are very fast
ChrisN: Yes, that's
interesting. Are those faster ones the latest models,
… or more expensive ones?
Yasuoka: The newer ones are faster
ChrisL: WAVE guys are
doing similar tests,
… some TVs have powerful chips, and some
don't.
… Then there's the idea to have something like a
performance label that can be part of the marketing of a
device.
ChrisN: A interesting
approach, I'm not sure if something like that would be within the
scope of W3C, though,
… similar to the CTA WAVE's activity.
Kaz: TV device
performance may be out of scope, but getting this kind of
infomation from various areas and SDOs should be useful.
… We can look into the detailed processing time in
this kind of procedure, not the device or network part, but the
browser processing time should be clarified.
… If we can get that kind of clarification and see
the actual cause of time delay, we can look with browser vendors to
improve
… As ChrisN metioned, maybe NHK could bring this to
CTA WAVE
… MEIG could look at the results in more
detail
… such as in TF calls
ChrisN: This is good
discussion, happy to spend more time today.
… One of the interesting things is Web Performance
WG's activity on providing APIs to allow websites to measure
performance
… Things like rendering stability, how much content
moves, time to first input. Chrome call this the "Core Web
Vitals"
… Is this something interesting from our viewpoint?
What metrics may be useful for TV specific
applications?
… Also, have you started to look at alternate
rendering mechanisms, e.g., Lightning.js which are Canvas and WebGL
or WebGPU based?
… How would they impact application
development?
Yasuoka: This testing
activity is not done for those kind of mechanisms yet,
… would like to try them in the
future.
… Currently, Hybridcast doesn't support WebGPU or
WebRTC.
… We'd like to try them but it's kind of difficult
at the moment.
ChrisN: It can be
difficult to introduce those new features. The WAVE Web Media API
Snapshot.
… What kind of help do you expect from
W3C?
Yasuoka: If those
mechanisms could be actual standards for media handling
… that would be great
… would like to proceed with considering those new
mechanisms
Ohmata: Hybridcast
currently doesn't define those mechanisms as mandated
… but think it would be useful
ChrisN: Coming back to
your other question,
… how can app/product developers contribute to the
TF?
… I suggest raising your topics in the MEIG GitHub
so that we can discuss each one individually,
… for eaxample, the discussion about launching
time, etc.
Kaz: We can start with
gathering information on pain points and workarounds based on
industry experience
… We have another example from Chris Lorenzo at
Comcast. We can summarise the description also and put them
together
… We can also ask someone from BBC and Fraunhofer
about their experience
ChrisN: Yes, it is helpful to gather more experience reports
ChrisN: Your other
question was about expected outputs from the TF,
… what can app/product developers expect from the
TF?
… As Chris said, we want to encourage continued use
of Web technologies
… If performance improvements can be achieved based
on the current Web technology,
… we should identify what can be done.
… This could be an area to discuss with Web
Performance WG as well
… The idea of having tests by application
developers and device manufacturers
… on achievable performance goals, could even
become a marketing tool
… Having a representative test suite may be a
valuable thing.
… The other area is looking at alternative
rendering mechanisms
… instead of DOM-based rendering
mechanism.
… We could start to look at canvas-based,
WebGL-based, etc.
… A new rendering standard in the future could be
developed
… How that easily could that be adopted, many
companies are building HTML-based applications today.
… Support for web new technologies in
TVs.
… All this would require new open
discussions.
… I suggest creating GitHub issues to capture your
points,
… and we can see how to follow up each of
them.
… Would that make sense?
Kaz: +1
ChrisL: I also agree
ChrisN: So I'll propose
to do that.
… This has been really good input,
… thank you for your presentation!
… We'd like to continue our collaboration with
you.
Yasuoka: Thank YOU!
ChrisN: I want to
invite you to continue to the discussion from topics,
… from both of your presentations.
… We would continue to work with you via
Kaz.
… Anything else for today?
(none)
ChrisN: We look forward to further collaboration. Thanks!
[adjourned]