<kaz> scribenick: kaz
<kaz> Recorded video from this call
Chris: Can I check if it's ok to record the call as usual?
(no objection)
(kaz starts recording on WebEx)
<cpn> scribenick: cpn
<kaz> Francois' slides
Francois: The goal is to report from
the workshop in June around web games.
... W3C runs workshops as a way to investigate a new space or
problem, as a starting point for further discussion.
... We looked at web games as a whole, how web technology can be
used to develop web games today, and what new technologies could be
useful.
... I'll report on the outcomes from the event.
... There's a published report online.
Francois: I'll skim over the less
relevant topics from our group's perspective.
... There are some use cases that are related, some use cases from
gaming are also useful from a streaming perspective.
Francois: We had a previous
workshop on games in 2011, wanted to check on progress. Lots of new
technologies emerged and standardised.
... Web games create new paradigms not in the native world. There's
no install, open a URL and start the experience.
... Native games involve possibly large installation for all the
assets.
... WebAssembly is very popular, allows cross compilation of games
from C++ or Rust into a package that can be run in the
browser.
... The interaction with web APIs is an interesting question
here.
... Recent announcements around cloud gaming. The game is run in
the cloud on high-end servers with plenty of CPU, GPU, memory
power.
... The result is streamed to an end device, can be lower-end.
Provided the client can send input events to the cloud, can provide
gaming experiences without requiring features on the client.
<scribe> ... New environments such as Facebook Instant games etc, a hosted environment where web technologies can be used.
Francois: Useful for 3D rendering,
but it was highlighted a specific need. When you want to render 3D
content, or 3D media rendering, there's a scene graph.
... Before you can send commands to the GPU to render, you need to
process the graph in various ways, e.g., frustrum culling. This
needs to happen on the CPU not the GPU.
... It can be parallelised, but hard to do on the web, as Workers
use message passing, they can share unstructured memory using
SharedArrayBuffer.
... This was removed from browsers after Spectre and
Meltdown.
... It's coming back, should be supported by all browsers.
... Can't share a structured object with the worker, must serialize
and then deserialize it.
... Doing that 60 or 100 times per second doesn't work.
... So a way to share structured data across threads is needed.
Requires going deep into JavaScript-land, which is a single
threaded model.
... It's a need that was pushed by many people at the
workshop.
... Another need is that you can currently only start a worker via
a file. It would be much more useful to be able to start workers
using a function that inherits the calling scope.
<kaz> scribenick: kaz
Chris: What was the conclusion from the workshop, is the fact that you need a separate JS file a design constraint, or could it be made to work with a function too?
<cpn> scribenick: cpn
Francois: I believe it was done this
was as it was easier. For a worker to take a function, you need to
specify what gets inherited, e.g., shared memory, so it's not as
easy as adding a new method to Workers.
... But it's not impossible.
Francois: Another topic discussed was
WebAssembly, the main issue that could be improved from a game
perspective is calling Web APIs from WASM code is inefficient right
now.
... Consider a game engine that uses C or C++ such as Unity or
Unreal Engine for example. It's hard to integrate with web APIs
efficiently from the C++ code.
... Another problem is around debugging. Once your C++ code is
compiled to WASM, it's hard to debug the C++ that's running in the
browser as WASM.
... Luke Wagner presented the Web Assembly Bindings document. it's
now called the Interface Types proposal. I won't go into details
here.
... This would make calling web APIs more efficient.
https://github.com/WebAssembly/interface-types/blob/master/proposals/interface-types/Explainer.md
Francois: There's also DWARF for Web Assembly debugging, using usual toolchains.
https://yurydelendik.github.io/webassembly-dwarf/
Francois: On the web, this is not on
par with native. WebGL is old, misses functions that would improve
rendering, and is less performant.
... There are different texture formats supported by chipsets. The
WebGPU API provides a common abstraction over different
hardware.
... There's an active CG working on this, will transition to WG.
One issue before that can happen is choice of shading
language.
... Two main contenders, and some disagreement on which to choose,
so it's an ongoing discussion.
... A presentation from the Khronos Group on glTF. There's a
discussion there on compressed texture transmission format, a
common format that can be supported across platforms.
... This is good news for developers, to avoid having to ship
different texture formats for their models.
... Should there be a 3D scene element on the web? It's been in the
air for years, without leading to anything.
... The workshop helped trigger discussion in the Immersive Web CG
on glTF.
Francois: For accesibility of 3D scenes, need semantic information, there's joint work that could happen between Khronos Group and W3C.
Francois: Web games are
no-install experience. But still need to load assets before
playing. 20% of players give up if there's a 10 megabyte
download.
... There's a need to address the loading process differently on
the web.
... Storing assets locally, but restrictions make it hard to store
permanently on the device, e.g., high scores, vs a cache for game
assets that could be downloaded again if needed.
... Discussion at the workshop suggested not much needs to change
for the loading experience. It's more about educating games
developers about downloading things in the background, and
postponing downloads.
... For storage, an idea is to allow multiple buckets per origin. A
difficulty is how to make the user aware of what's going on, what's
permanent and what isn't.
Francois: Games are meant to be
challenging, so games are inaccessible by definition in some way.
There's no one recipe to follow for accessibility in games. The
technique would vary from game to game.
... Some questions remain, e.g., how to provide semantic
information to assistive technology when there's no DOM, i.e., when
using which is an opaque set of pixels.
... Problematic when the code is not JavaScript, cross compiled
from C++, how to provide the semantic information from C++
code?
Kaz: There's standardisation work on WoT to extend the capability of web applications to devices such as Amazon Alexa, could be of interest.
<Zakim> kaz, you wanted to suggest we think about WoT as a possible tool for accessibility
Francois: We discussed the devices
that people with motor impairment could be using, accessible input
devices, what are the consequences for game design?
... There's an incubation in WICG called the Accessibiltiy Object
Model (AOM), it's a way to expose semantic information when there's
no DOM.
Francois: Combined with the WebIDL
binding proposal, could be a way to integrate with C++. The two
things in combination could be useful to address accessibility
questions.
... There are accessibilityy guidelines available, presented by Ian
Hamilton and Matthew from The Paciello Group
Kaz: Also, personal and privacy information. It might make sense to have some joint discussion at TPAC on these related topics.
Francois: Matthew is going to follow up in the APA WG, this is a logical place to discuss.
Francois: There are lots of C and C++
audio libraries, professional code, used in native games for audio
effects etc.
... This code is hard to port to the web, because the Web Audio
model doesn't work well in this case. There's a lack of a low level
audio input/output API.
... To support this, need better threading support, and there's a
proposal in the Audio CG from Google: Audio Device Client for low
level I/O
... This would help with porting of existing code to the web
Francois: The Gamepad API has not
made progress recently. Buttons may be exposed differently between
browsers, so there's a combinatorial explosion to understand which
button was pressed.
... Controllers have modern features, touch surfaces, light
indicators. You cloud make the gamepad blink the lights to indicate
which one to use.
... Also haptics support. These modern features link to the
immersive world, there are advanced VR controllers.
... Want to finalise the first version of the Gamepad API, close
the exsting issues, then start work on a v2 API to support the
modern features.
Francois: For cloud gaming, need to
send inputs to the cloud for rendering. Input events are aligned to
rAF for scheduling.
... This isn't suitable for cloud gaming. All input events need to
go through the main thread, leads to congestion issues, helpful to
process some events in a worker.
... Mouse events are accelerating and in FPS games, it makes it
hard to make precise movements.
... There's a Google proposal for Input to Workers and
Worklets.
Francois: Also raw events, to get
events as soon as possible. Evaluating whether the cost is worth
it.
... Also unaccelerated mouse events.
Francois: Problem with WebRTC
solution is that you can't fine tune the encoding/decoding
parameters, also congestion control.
... Two ideas discussed, WebTransport based on QUIC, a UDP version
of Web Sockets. Allows developers to write their own congestion
control.
... Incubated in WICG. Also the WebCodecs proposal, not yet in
WICG, needs support. This would expose the encoding and decoding
capabilities of the browser.
Francois: Some APIs might not be
available in hosted environemnts, and some APIs are specific to
hosted environments.
... Different companies have different hosted environments. Could
be beneficial to converge these.
Francois: Easy for things to
disappear on the web. Metadata formats, such as schema.org, for
game capabilities: categories, input controllers, offline support,
what monetization schemes does it implement?
... Monetization on the web is broader than games. Watching ads to
add features to the game.
... Micro-transaction proposal in WICG.
Francois: Agreement at the workshop
for further activities, could be a new IG. Could it fit in scope
for the M&E IG, maybe as a new TF?
... It's a different topic for this IG. Is there enough support for
some new activity?
Francois: There's one on mini-apps,
hosted environments. Also WebTransport and WebCodecs, and WebGPU. A
3D HTML element, from someone not at the workshop
... and the Input for Workers/Worklets
Francois: The Audio WG will meet, and WebApps WG will talk about the Gamepad API
<Barbara_H> Latency is a key challenge. What about a discussion at the Networking IG on latency?
Barbara: So latency is key. Could discuss at the Web and Networks IG on latency and the impact on gaming?
Francois: It could be, yes. WebTransport would be good to discuss there, also other solutions, especially thinking about 5G. Good point, yes.
Barbara: Is it this group, or another
group? High-end gaming will push the graphics aspects. There's also
mainstream gaming happening.
... We'd need to look at the gaming model segmentation. It's OK to
start by grouping them together, but the needs of a high end
dedicated gamers vs casual gamers will start to grow.
... Looking at the business model, high-end dedicated gaming
segment will grow, but they have different expectations to move to
the web.
... The casual gaming segment is the growing market. Different
approach for established market moving to the web vs an emerging
market.
Francois: You're right. We struggled
to get AAA producers to the workshop, although we had the main game
engines, but that's more technical.
... They see the web as an important platform, but they're not
prioritising it.
... Then there are the indie game developers, more keen on web
technology, for reach.
... And in the middle are the game platforms, more low end, but
sharing high end needs.
... Happy to take suggestions.
Barbara: About machine learning,
games will be impacted in the future, it's a common theme.
... Another thing to monitor going forward, intersection with cloud
gaming.
Francois: We left machine learning out of scope, we'll have a future workshop on that.
Will: WebRTC was passed over in terms of improvements. Why not fix some of those issues with WebRTC vs inventing a new distribution scheme? It has a huge penetration now, allowing customisable FEC or congestion control would be good improvements.
Francois: The WebTransport proposal can be plugged into WebRTC. If you expose congestion control to WebRTC, you end up with a similar solution.
Peter: The work on WebCodec and WebTransport originated from WebRTC, trying to provide more control to the application. It grew out of the WebRTC WG because of the applications outside of RTC that were finding these new APIs useful.
<kaz> scribenick: kaz
Chris: I think this is a good topic
for us to follow up on. We have two breakout sessions at
TPAC.
... Should we schedule something after TPAC to go into more detail?
Not everyone will be at TPAC.
... We can follow up offline about scheduling a call on Web
Transport and Web Codec.
<scribe> ACTION: Chris to arrange IG call on Web Codecs and Web Transport
<cpn> scribenick: cpn
Rob: To Barbara's comments, there's
an open source animation package called Blender. A great idea they
had was to create open projects, films or animations that are
community sourced.
... An open project for a game could be a good idea, to prove out
the technology. So while you're building the APIs, you're also
building a game and involving the community. It would showcase the
benefits, highlight any weaknesses or problems, get feedback and
buy-in from the community during the develoment.
... At the end you'd have the system, with examples of how to use
it. It would help with the hard-core vs casual argument, it would
become clear which areas it would support well and which might be
problematic, e.g., can you produce a 3D shooter due to latency
issues?
... Would save effort on things that wouldn't work.
<RobSmith> https://cloud.blender.org/open-projects
<kaz> scribenick: kaz
Chris: What support is there from participants at the workshop? There was a lot of interest, but what about taking these topics forward? Is there a push for creating a TF or IG?
Francois: Have discussed with some of
the participants. Some are positive about the idea, but none were
really willing to drive it.
... We need to find one or two organisations who would drive the
discussion, otherwise it's hard to create a group and do something
useful.
<cpn> scribenick: cpn
Francois: We created a games CG after
the first workshop, but it gradually died. So we need people to be
able to commit to running the effort.
... I'm talking mainly with people at Facebook, they have a good
over-arching perspective, could be a good first company with an
incentive to push an activity, no commitment yet.
... If you're interested to help, please get in touch.
... An IG is harder to create, needs a scope and charter, getting
member support. A TF is more lightweight, is it a good idea to
create in the M&E IG? Some of the topics are foreign to this
group until now.
<Zakim> kaz, you wanted to suggest we think about WoT, DID (and possibly DAS) as well for TPAC
Kaz: I agree with other comments made here, and want to suggest having joint discussion with WoT, DID, DAS, etc. at TPAC
<kaz> scribenick: kaz
Chris: TPAC is a good opportunity so
discuss creating a group.
... I think the key is having people to lead the activity, and
following that we can talk about the venue, either this IG or a new
group.
... I'm very happy to participate in those conversations.
Francois: That makes sense.
Chris: Our IG meeting will be on
Monday, Sep. 16. If you can't be there, we'll have a Webex open to
join remotely.
... The agenda is mostly finalised.
Chris: We have a joint meeting with
Second Screen WG/CG.
... We'll discuss accessibility and captioning in VR and 360
video.
... We have open discussion during afternoon, looking at the next
steps for media on the web.
... Things like Web Codecs and Web Transport feed into this.
... Now that we have a Media WG, there's an opportunity for the IG
to look at its contribution to those APIs, but also to look beyond
that to use cases that drive further development of media support
on the web.
... If you're coming, I look forward to meeting you there.
... If you can't make the meeting in person, you're welcome to join
remotely.
... We have some media related breakouts on Wednesday, a lot of
interesting topics.
... This should be a good meeting.
<tidoust> Proposed breakout sessions at TPAC
(none)
Chris: Thanks for joining, and thank you for reporting, Francois.
Kaz: Next call on Oct. 8?
Chris: Not sure, I'll be away. I'll discuss with the IG co-chairs.
[adjourned]