IRC log of web-networks on 2021-06-18

Timestamps are in UTC.

14:01:47 [RRSAgent]
RRSAgent has joined #web-networks
14:01:47 [RRSAgent]
logging to https://www.w3.org/2021/06/18-web-networks-irc
14:01:50 [Zakim]
Zakim has joined #web-networks
14:02:11 [cpn]
cpn has joined #web-networks
14:02:59 [dom]
Present+ Dom, AliBegen, Arpit_Bansal, ChrisNeedham, DanDruta, EdLambert, EricSiow, JonasSvennebring, PiersOHanlon, SantoshRedy, Satya, SOngXu, Sudeep, ThoharisCharitidis, YajunChen
14:03:24 [dom]
Present+ LarryZhao
14:03:29 [dom]
Chair: SongXu
14:03:55 [dom]
Meeting: Network Emulation Tool in Browser Developer Tools
14:04:22 [dom]
SongXu: our meeting is about Network Emulation Tool in Browser Developer Tools
14:04:40 [dom]
... our agenda: some background of this work and goals
14:04:51 [dom]
... overview of existing tools that are relevant to similar functionalities
14:05:02 [dom]
... followed by discussions on the models & requirements
14:05:07 [dom]
Topic: Background & Goal
14:05:24 [dom]
Song: During TPAC, there was interest showed around network condition emulation
14:05:38 [dom]
... we want to review concrete proposal to improve network emulation tools for web developers
14:05:45 [dom]
... and identify standardization opportunities for trace format
14:06:18 [dom]
... our initial roadmap on this was to standardization efforts around trace format in Q2 2021
14:06:23 [dom]
... the landscape analysis too
14:06:34 [dom]
... have use cases defined in Q3 2021, along with emulation methodology
14:06:42 [dom]
Topic: Quick Overview about existing tools
14:06:55 [dom]
Song: WebPageTest, LPP Network Trace Tools and Throttle
14:07:24 [dom]
LarryZhao: WebPageTest (aka WPT) - a web performance tool providing diagnostic under a variety of conditions
14:07:36 [dom]
... it helps find performance issues, based on test run from a different set of locations around the world
14:08:14 [dom]
... You can also pick a type of connection - e.g. LTE
14:08:28 [dom]
... the documentation shows that this is done using NetEm and dummynet
14:09:00 [dom]
... netem is an option enabled in the linux kernel, e.g. with packet delay, loss, duplication, reordering
14:09:18 [dom]
... When WPT simulates network, it works at the network level directly
14:09:28 [dom]
Subtopic: LPP Network Trace Tools
14:09:39 [dom]
LarryZhao: we heard from Jonas on this before
14:09:52 [dom]
... it features 3 important parts: trace capture, emulator and a file format
14:10:02 [dom]
... it is presented as a chrome extension
14:10:07 [dom]
... [shows demo of LPP]
14:10:29 [dom]
... it utilizes the Chrome Dev Tool Protocol
14:10:37 [dom]
... so it operates at the level of application network
14:10:49 [dom]
... it only has effects on the Web site you visit via Google Chrome
14:11:16 [dom]
... that's a major difference between WPT and LPP - network level vs app level
14:11:20 [dom]
Subtopic: Throttle
14:11:48 [dom]
LarryZhao: Similar to WPT, from sitespeed.io to simulate slow network conditions
14:11:56 [dom]
... it tool works at the network level
14:12:05 [dom]
... very similar to WPT
14:12:14 [dom]
i/LarryZhao: WebPageTest/Subtopic: WPT
14:12:46 [dom]
Sudeep: is the Throttle tool similar to traffic control (tc) tools?
14:12:51 [dom]
LarryZhao: it is based on TC
14:14:49 [dom]
Arpit: how about the Mac version?
14:15:01 [dom]
Song: they use the same core library across Mac/Linux
14:15:24 [dom]
Piers: it uses dummynet underneath afaik
14:15:50 [dom]
... throttle uses both pfctl & dummynet
14:16:01 [dom]
... to provide something similar to tc & netem
14:16:06 [dom]
SubTopic: Gap Analysis
14:16:34 [dom]
Song: some of the tools can only emulate limited network conditions
14:16:55 [dom]
... but it's useful to be able to test under network switch (e.g. from WIFI to LTE)
14:17:19 [dom]
... with WebRTC & cloud gaming scenarios, you need access to more conditions (e.g. package dropped, different loss rates)
14:17:55 [dom]
... there are features common across tools: up & down bandwidth, latency / rtt
14:18:14 [dom]
... how much interop is needed for these features? a standardized trace format would help in that case
14:18:23 [dom]
Topic: Use cases
14:18:34 [dom]
Song: I've collected use cases that would make use of this
14:19:12 [dom]
... one is test automation - a developer could test their sites under various real life network conditions previously captured
14:19:24 [dom]
... WPT has a collection of such traces
14:20:19 [dom]
... Another use case around test & development
14:20:28 [dom]
Jonas: this is one use case we've considered
14:20:52 [dom]
... Our main intention was to use it for standards Web application testing to get a feeling how the connection is in different parts of the world
14:21:13 [dom]
... we're studing extensions for MPEG dash, cloud gaming, real-time media
14:21:20 [dom]
s/stud/study/
14:22:44 [dom]
Song: another use case is for PWA testing
14:23:13 [dom]
Sudeep: the input on this is also available on the github repo
14:23:37 [dom]
-> https://github.com/w3c/network-emulation/issues/3 Network emulation Use Cases
14:24:27 [dom]
Sudeep: testing scenarios where some requests randomly timeout has proved difficult in the context of an app experience
14:24:39 [dom]
... and testing robutness depending on the frequency of these timeouts
14:25:15 [dom]
... Another thing they mentioned is testing how service workers react to these varying network conditions, e.g. when using background sync APIs
14:25:49 [dom]
... having a way to automate this testing in local environments would go a long way
14:26:14 [dom]
... the last item is emulating online/unreliable/offline states
14:27:15 [dom]
... there can be scenarios where a given request will go fine, but then another will fail due to a e.g. a proxy redirect
14:27:26 [dom]
... this creates delays in how the web app will load & render
14:27:41 [dom]
... and this can't be dealt with today at the level of browser developer tools
14:28:28 [dom]
Topic: Models or requirements
14:28:52 [dom]
Song: Opening the floor to discuss what features of emulation, what parameters to monitor, what methods to use to emulate
14:29:04 [dom]
Present+ DapengMax
14:29:41 [dom]
Ali: I've added two slides to discuss other use cases
14:29:49 [dom]
Subtopic: testing http streaming clients
14:30:07 [dom]
Ali: we use network emulation to test streaming scenarios
14:30:20 [dom]
... HTTP streaming is very popular, consumes a lot of bandwidth and triggers a lot of research
14:30:29 [dom]
... we depend quite heavily on network emulation tools
14:30:47 [dom]
... at a high level, the model is that a streaming client fetches media segments from a server
14:30:59 [dom]
... based on information known in advance on what to request when to request
14:31:25 [dom]
... the stream is available in different resolutions & bitrates, allowing to adapt the stream (hence the name adaptive streaming)
14:31:38 [dom]
... the most important parameter to determine what bitrate is bandwidth evaluation
14:31:45 [dom]
... there can be others (e.g. CPU & memory)
14:31:57 [dom]
... but the general case is the network is the main bottleneck
14:32:19 [dom]
... the client needs to measure its active bandwidth and predict what the right segment will be
14:32:37 [dom]
... the measure is done based on the data being fetched
14:32:53 [dom]
... but this can be pretty noisy given the different sizes of segment, the impact of round trips, etc
14:33:12 [dom]
... what most clients do is that they smoothed out the value (e.g. average, percentiles)
14:33:34 [dom]
... some clients make prediction out of this
14:33:47 [dom]
... this gets done over & over during the streaming session
14:34:16 [dom]
... my understanding is that LPP can be used to help with this
14:34:55 [dom]
... in terms of measuring and predicting the bandwidth, although not about how the client should behave
14:35:04 [dom]
... This leads to a number of requirements:
14:35:14 [dom]
... network emulation is a big part of our testing in our daily life
14:35:24 [dom]
... we usually set different upload & download conditions
14:35:31 [dom]
... with different network congestion
14:35:56 [dom]
... our main focus is on bandwidth throttling, we're mostly ignoring e.g. packet loss or re-ordering
14:36:47 [dom]
... when dealing with live streaming, we need throttling on short time scales too
14:37:04 [dom]
... Reproducibility is a very important requirement for us
14:37:16 [dom]
... we need to be able to test these conditions across different devices & content
14:37:41 [dom]
... this means it should be easy to design programmatic traces, including modifications to existing traces (e.G. scale up the bandwidth)
14:37:56 [dom]
... e.g. just having to adjust a scale factor
14:38:05 [dom]
... likewise being to scale the timescale on which it runs
14:38:15 [dom]
... the files should be small, easy to edit and share
14:38:44 [dom]
Song: thanks for the excellent input
14:39:14 [dom]
Ali: we heavily use the built-in throttling tool in Chrome - it has proved easy to script, but it doesn't have all the features we want
14:39:21 [dom]
... we would be more than happy to switch to a better tool
14:39:31 [dom]
Jonas: the scaling aspect is a good one to add
14:39:42 [dom]
Ali: yes, both in terms of magnitude & time
14:40:19 [dom]
... sometimes, we design a trace that we just want to replicate under different scales
14:40:36 [dom]
Jonas: that indeed helps with sensitivity tests
14:40:59 [dom]
... With regard to streaming, I agree that LPP shouldn't be involved in how the estimations and predictions are used
14:41:23 [dom]
... we're exploring different type of behaviors of what we can do with these traces - of course not forcing them on any one
14:41:59 [dom]
... we're planning to publish a set of examples that demonstrate different type of optimizations (e.g maximizing quality vs minimizing buffering)
14:42:14 [dom]
RRSAgent, draft minutes
14:42:14 [RRSAgent]
I have made the request to generate https://www.w3.org/2021/06/18-web-networks-minutes.html dom
14:42:14 [dom]
RRSAgent, make log public
14:43:05 [dom]
Sudeep: how do know which use cases would benefit the most when we start emulating network conditions?
14:43:30 [dom]
... are there use cases that have no solution to manage the quality experience e.g. under congestion control?
14:43:40 [dom]
... how do cloud gaming deal with these?
14:44:14 [dom]
Piers: WebRTC has P2P traffic
14:44:31 [dom]
... not sure how Chrome throttling applies to that
14:44:49 [dom]
Sudeep: I tried it but it didn't seem to work
14:45:29 [dom]
Piers: another question is network emulation in the context of newer congestion algorithms like BBR which are delay based
14:46:18 [dom]
Ali: I mentioned we didn't care much about packet loss & reordering - that's true for HTTP 1.1 & HTTP 2 since they're based on TCP
14:46:37 [dom]
... but as we start testing HTTP3 at some point in the future, these things might become more relevant - hard to say how much at this point
14:47:02 [dom]
... but as QUIC replaces TCP, we will be interested to access these aspects too
14:47:33 [dom]
... that aspect will also be more relevant for cloud gaming & webrtc
14:47:55 [dom]
RRSAgent, draft minutes
14:47:55 [RRSAgent]
I have made the request to generate https://www.w3.org/2021/06/18-web-networks-minutes.html dom
14:48:53 [dom]
Topic: Next steps
14:49:12 [dom]
Song: I'll bring these topics we've identified to our github repo
14:49:31 [dom]
Jonas: in terms of trace format standardization, it was scheduled for Q2 which is about finished
14:49:38 [dom]
Song: yes, this will have to be revisited
14:49:46 [dom]
... probably more of a Q3 now
14:50:03 [dom]
Jonas: I guess the schedule depends on the interest around standardizing such a format
14:51:22 [dom]
Dom: which producers / consumers of such a format would need to be involved in these discussions?
14:52:38 [dom]
Ed: is there a timeframe for getting feedback on trace format standardization?
14:52:49 [dom]
Song: I would like to say by the end of July
14:53:36 [dom]
Piers: has there been an assessment of other commercial & open source approaches in this space?
14:54:23 [dom]
Song: we did part of this stuff in the past, but we didn't look at commercial implementations yet
14:54:36 [dom]
Piers: there are quite a few network emulators out there, I'm not sure about their format
14:54:43 [dom]
... hotnet, NS (?)
14:54:52 [dom]
... they're often programmatic rather than declarative
14:55:06 [dom]
... in streaming papers, they tend to use traces from the FCC and other sources
14:55:12 [dom]
... but they tend to be just CSV files
14:55:20 [Yajun_Chen]
Yajun_Chen has joined #web-networks
14:55:49 [dom]
Sudeep: in a last meeting, two tools were mentioned: crowdad and network management labs
14:56:14 [dom]
piers: indeed - a new one is emerging from Standford, Puffer (?) that provides anonymized logs of clients
14:56:24 [dom]
... there are a number of repositories traces outthere
14:56:44 [dom]
-> https://puffer.stanford.edu/ Puffer
14:57:08 [dom]
Jonas: Crowdad seems like a mix of formats
14:57:16 [dom]
-> http://crawdad.org/ CRAWDAD
14:57:20 [dom]
s/Crow/Craw/g
14:57:29 [Larry_Zhao]
Larry_Zhao has joined #web-networks
14:57:50 [dom]
-> https://www.measurementlab.net/#/ Measurement Lab
14:57:53 [Larry_Zhao]
Larry_Zhao has left #web-networks
14:58:06 [dom]
Sudeep: it makes sense to look indeed at existing formats & repositories
14:58:46 [dom]
Song: thank you all for joining this meeting
14:59:05 [dom]
... let's continue discussions in github
14:59:20 [dom]
RRSAgent, draft minutes
14:59:20 [RRSAgent]
I have made the request to generate https://www.w3.org/2021/06/18-web-networks-minutes.html dom