IRC log of web-networks on 2021-06-18
Timestamps are in UTC.
- 14:01:47 [RRSAgent]
- RRSAgent has joined #web-networks
- 14:01:47 [RRSAgent]
- logging to https://www.w3.org/2021/06/18-web-networks-irc
- 14:01:50 [Zakim]
- Zakim has joined #web-networks
- 14:02:11 [cpn]
- cpn has joined #web-networks
- 14:02:59 [dom]
- Present+ Dom, AliBegen, Arpit_Bansal, ChrisNeedham, DanDruta, EdLambert, EricSiow, JonasSvennebring, PiersOHanlon, SantoshRedy, Satya, SOngXu, Sudeep, ThoharisCharitidis, YajunChen
- 14:03:24 [dom]
- Present+ LarryZhao
- 14:03:29 [dom]
- Chair: SongXu
- 14:03:55 [dom]
- Meeting: Network Emulation Tool in Browser Developer Tools
- 14:04:22 [dom]
- SongXu: our meeting is about Network Emulation Tool in Browser Developer Tools
- 14:04:40 [dom]
- ... our agenda: some background of this work and goals
- 14:04:51 [dom]
- ... overview of existing tools that are relevant to similar functionalities
- 14:05:02 [dom]
- ... followed by discussions on the models & requirements
- 14:05:07 [dom]
- Topic: Background & Goal
- 14:05:24 [dom]
- Song: During TPAC, there was interest showed around network condition emulation
- 14:05:38 [dom]
- ... we want to review concrete proposal to improve network emulation tools for web developers
- 14:05:45 [dom]
- ... and identify standardization opportunities for trace format
- 14:06:18 [dom]
- ... our initial roadmap on this was to standardization efforts around trace format in Q2 2021
- 14:06:23 [dom]
- ... the landscape analysis too
- 14:06:34 [dom]
- ... have use cases defined in Q3 2021, along with emulation methodology
- 14:06:42 [dom]
- Topic: Quick Overview about existing tools
- 14:06:55 [dom]
- Song: WebPageTest, LPP Network Trace Tools and Throttle
- 14:07:24 [dom]
- LarryZhao: WebPageTest (aka WPT) - a web performance tool providing diagnostic under a variety of conditions
- 14:07:36 [dom]
- ... it helps find performance issues, based on test run from a different set of locations around the world
- 14:08:14 [dom]
- ... You can also pick a type of connection - e.g. LTE
- 14:08:28 [dom]
- ... the documentation shows that this is done using NetEm and dummynet
- 14:09:00 [dom]
- ... netem is an option enabled in the linux kernel, e.g. with packet delay, loss, duplication, reordering
- 14:09:18 [dom]
- ... When WPT simulates network, it works at the network level directly
- 14:09:28 [dom]
- Subtopic: LPP Network Trace Tools
- 14:09:39 [dom]
- LarryZhao: we heard from Jonas on this before
- 14:09:52 [dom]
- ... it features 3 important parts: trace capture, emulator and a file format
- 14:10:02 [dom]
- ... it is presented as a chrome extension
- 14:10:07 [dom]
- ... [shows demo of LPP]
- 14:10:29 [dom]
- ... it utilizes the Chrome Dev Tool Protocol
- 14:10:37 [dom]
- ... so it operates at the level of application network
- 14:10:49 [dom]
- ... it only has effects on the Web site you visit via Google Chrome
- 14:11:16 [dom]
- ... that's a major difference between WPT and LPP - network level vs app level
- 14:11:20 [dom]
- Subtopic: Throttle
- 14:11:48 [dom]
- LarryZhao: Similar to WPT, from sitespeed.io to simulate slow network conditions
- 14:11:56 [dom]
- ... it tool works at the network level
- 14:12:05 [dom]
- ... very similar to WPT
- 14:12:14 [dom]
- i/LarryZhao: WebPageTest/Subtopic: WPT
- 14:12:46 [dom]
- Sudeep: is the Throttle tool similar to traffic control (tc) tools?
- 14:12:51 [dom]
- LarryZhao: it is based on TC
- 14:14:49 [dom]
- Arpit: how about the Mac version?
- 14:15:01 [dom]
- Song: they use the same core library across Mac/Linux
- 14:15:24 [dom]
- Piers: it uses dummynet underneath afaik
- 14:15:50 [dom]
- ... throttle uses both pfctl & dummynet
- 14:16:01 [dom]
- ... to provide something similar to tc & netem
- 14:16:06 [dom]
- SubTopic: Gap Analysis
- 14:16:34 [dom]
- Song: some of the tools can only emulate limited network conditions
- 14:16:55 [dom]
- ... but it's useful to be able to test under network switch (e.g. from WIFI to LTE)
- 14:17:19 [dom]
- ... with WebRTC & cloud gaming scenarios, you need access to more conditions (e.g. package dropped, different loss rates)
- 14:17:55 [dom]
- ... there are features common across tools: up & down bandwidth, latency / rtt
- 14:18:14 [dom]
- ... how much interop is needed for these features? a standardized trace format would help in that case
- 14:18:23 [dom]
- Topic: Use cases
- 14:18:34 [dom]
- Song: I've collected use cases that would make use of this
- 14:19:12 [dom]
- ... one is test automation - a developer could test their sites under various real life network conditions previously captured
- 14:19:24 [dom]
- ... WPT has a collection of such traces
- 14:20:19 [dom]
- ... Another use case around test & development
- 14:20:28 [dom]
- Jonas: this is one use case we've considered
- 14:20:52 [dom]
- ... Our main intention was to use it for standards Web application testing to get a feeling how the connection is in different parts of the world
- 14:21:13 [dom]
- ... we're studing extensions for MPEG dash, cloud gaming, real-time media
- 14:21:20 [dom]
- s/stud/study/
- 14:22:44 [dom]
- Song: another use case is for PWA testing
- 14:23:13 [dom]
- Sudeep: the input on this is also available on the github repo
- 14:23:37 [dom]
- -> https://github.com/w3c/network-emulation/issues/3 Network emulation Use Cases
- 14:24:27 [dom]
- Sudeep: testing scenarios where some requests randomly timeout has proved difficult in the context of an app experience
- 14:24:39 [dom]
- ... and testing robutness depending on the frequency of these timeouts
- 14:25:15 [dom]
- ... Another thing they mentioned is testing how service workers react to these varying network conditions, e.g. when using background sync APIs
- 14:25:49 [dom]
- ... having a way to automate this testing in local environments would go a long way
- 14:26:14 [dom]
- ... the last item is emulating online/unreliable/offline states
- 14:27:15 [dom]
- ... there can be scenarios where a given request will go fine, but then another will fail due to a e.g. a proxy redirect
- 14:27:26 [dom]
- ... this creates delays in how the web app will load & render
- 14:27:41 [dom]
- ... and this can't be dealt with today at the level of browser developer tools
- 14:28:28 [dom]
- Topic: Models or requirements
- 14:28:52 [dom]
- Song: Opening the floor to discuss what features of emulation, what parameters to monitor, what methods to use to emulate
- 14:29:04 [dom]
- Present+ DapengMax
- 14:29:41 [dom]
- Ali: I've added two slides to discuss other use cases
- 14:29:49 [dom]
- Subtopic: testing http streaming clients
- 14:30:07 [dom]
- Ali: we use network emulation to test streaming scenarios
- 14:30:20 [dom]
- ... HTTP streaming is very popular, consumes a lot of bandwidth and triggers a lot of research
- 14:30:29 [dom]
- ... we depend quite heavily on network emulation tools
- 14:30:47 [dom]
- ... at a high level, the model is that a streaming client fetches media segments from a server
- 14:30:59 [dom]
- ... based on information known in advance on what to request when to request
- 14:31:25 [dom]
- ... the stream is available in different resolutions & bitrates, allowing to adapt the stream (hence the name adaptive streaming)
- 14:31:38 [dom]
- ... the most important parameter to determine what bitrate is bandwidth evaluation
- 14:31:45 [dom]
- ... there can be others (e.g. CPU & memory)
- 14:31:57 [dom]
- ... but the general case is the network is the main bottleneck
- 14:32:19 [dom]
- ... the client needs to measure its active bandwidth and predict what the right segment will be
- 14:32:37 [dom]
- ... the measure is done based on the data being fetched
- 14:32:53 [dom]
- ... but this can be pretty noisy given the different sizes of segment, the impact of round trips, etc
- 14:33:12 [dom]
- ... what most clients do is that they smoothed out the value (e.g. average, percentiles)
- 14:33:34 [dom]
- ... some clients make prediction out of this
- 14:33:47 [dom]
- ... this gets done over & over during the streaming session
- 14:34:16 [dom]
- ... my understanding is that LPP can be used to help with this
- 14:34:55 [dom]
- ... in terms of measuring and predicting the bandwidth, although not about how the client should behave
- 14:35:04 [dom]
- ... This leads to a number of requirements:
- 14:35:14 [dom]
- ... network emulation is a big part of our testing in our daily life
- 14:35:24 [dom]
- ... we usually set different upload & download conditions
- 14:35:31 [dom]
- ... with different network congestion
- 14:35:56 [dom]
- ... our main focus is on bandwidth throttling, we're mostly ignoring e.g. packet loss or re-ordering
- 14:36:47 [dom]
- ... when dealing with live streaming, we need throttling on short time scales too
- 14:37:04 [dom]
- ... Reproducibility is a very important requirement for us
- 14:37:16 [dom]
- ... we need to be able to test these conditions across different devices & content
- 14:37:41 [dom]
- ... this means it should be easy to design programmatic traces, including modifications to existing traces (e.G. scale up the bandwidth)
- 14:37:56 [dom]
- ... e.g. just having to adjust a scale factor
- 14:38:05 [dom]
- ... likewise being to scale the timescale on which it runs
- 14:38:15 [dom]
- ... the files should be small, easy to edit and share
- 14:38:44 [dom]
- Song: thanks for the excellent input
- 14:39:14 [dom]
- Ali: we heavily use the built-in throttling tool in Chrome - it has proved easy to script, but it doesn't have all the features we want
- 14:39:21 [dom]
- ... we would be more than happy to switch to a better tool
- 14:39:31 [dom]
- Jonas: the scaling aspect is a good one to add
- 14:39:42 [dom]
- Ali: yes, both in terms of magnitude & time
- 14:40:19 [dom]
- ... sometimes, we design a trace that we just want to replicate under different scales
- 14:40:36 [dom]
- Jonas: that indeed helps with sensitivity tests
- 14:40:59 [dom]
- ... With regard to streaming, I agree that LPP shouldn't be involved in how the estimations and predictions are used
- 14:41:23 [dom]
- ... we're exploring different type of behaviors of what we can do with these traces - of course not forcing them on any one
- 14:41:59 [dom]
- ... we're planning to publish a set of examples that demonstrate different type of optimizations (e.g maximizing quality vs minimizing buffering)
- 14:42:14 [dom]
- RRSAgent, draft minutes
- 14:42:14 [RRSAgent]
- I have made the request to generate https://www.w3.org/2021/06/18-web-networks-minutes.html dom
- 14:42:14 [dom]
- RRSAgent, make log public
- 14:43:05 [dom]
- Sudeep: how do know which use cases would benefit the most when we start emulating network conditions?
- 14:43:30 [dom]
- ... are there use cases that have no solution to manage the quality experience e.g. under congestion control?
- 14:43:40 [dom]
- ... how do cloud gaming deal with these?
- 14:44:14 [dom]
- Piers: WebRTC has P2P traffic
- 14:44:31 [dom]
- ... not sure how Chrome throttling applies to that
- 14:44:49 [dom]
- Sudeep: I tried it but it didn't seem to work
- 14:45:29 [dom]
- Piers: another question is network emulation in the context of newer congestion algorithms like BBR which are delay based
- 14:46:18 [dom]
- Ali: I mentioned we didn't care much about packet loss & reordering - that's true for HTTP 1.1 & HTTP 2 since they're based on TCP
- 14:46:37 [dom]
- ... but as we start testing HTTP3 at some point in the future, these things might become more relevant - hard to say how much at this point
- 14:47:02 [dom]
- ... but as QUIC replaces TCP, we will be interested to access these aspects too
- 14:47:33 [dom]
- ... that aspect will also be more relevant for cloud gaming & webrtc
- 14:47:55 [dom]
- RRSAgent, draft minutes
- 14:47:55 [RRSAgent]
- I have made the request to generate https://www.w3.org/2021/06/18-web-networks-minutes.html dom
- 14:48:53 [dom]
- Topic: Next steps
- 14:49:12 [dom]
- Song: I'll bring these topics we've identified to our github repo
- 14:49:31 [dom]
- Jonas: in terms of trace format standardization, it was scheduled for Q2 which is about finished
- 14:49:38 [dom]
- Song: yes, this will have to be revisited
- 14:49:46 [dom]
- ... probably more of a Q3 now
- 14:50:03 [dom]
- Jonas: I guess the schedule depends on the interest around standardizing such a format
- 14:51:22 [dom]
- Dom: which producers / consumers of such a format would need to be involved in these discussions?
- 14:52:38 [dom]
- Ed: is there a timeframe for getting feedback on trace format standardization?
- 14:52:49 [dom]
- Song: I would like to say by the end of July
- 14:53:36 [dom]
- Piers: has there been an assessment of other commercial & open source approaches in this space?
- 14:54:23 [dom]
- Song: we did part of this stuff in the past, but we didn't look at commercial implementations yet
- 14:54:36 [dom]
- Piers: there are quite a few network emulators out there, I'm not sure about their format
- 14:54:43 [dom]
- ... hotnet, NS (?)
- 14:54:52 [dom]
- ... they're often programmatic rather than declarative
- 14:55:06 [dom]
- ... in streaming papers, they tend to use traces from the FCC and other sources
- 14:55:12 [dom]
- ... but they tend to be just CSV files
- 14:55:20 [Yajun_Chen]
- Yajun_Chen has joined #web-networks
- 14:55:49 [dom]
- Sudeep: in a last meeting, two tools were mentioned: crowdad and network management labs
- 14:56:14 [dom]
- piers: indeed - a new one is emerging from Standford, Puffer (?) that provides anonymized logs of clients
- 14:56:24 [dom]
- ... there are a number of repositories traces outthere
- 14:56:44 [dom]
- -> https://puffer.stanford.edu/ Puffer
- 14:57:08 [dom]
- Jonas: Crowdad seems like a mix of formats
- 14:57:16 [dom]
- -> http://crawdad.org/ CRAWDAD
- 14:57:20 [dom]
- s/Crow/Craw/g
- 14:57:29 [Larry_Zhao]
- Larry_Zhao has joined #web-networks
- 14:57:50 [dom]
- -> https://www.measurementlab.net/#/ Measurement Lab
- 14:57:53 [Larry_Zhao]
- Larry_Zhao has left #web-networks
- 14:58:06 [dom]
- Sudeep: it makes sense to look indeed at existing formats & repositories
- 14:58:46 [dom]
- Song: thank you all for joining this meeting
- 14:59:05 [dom]
- ... let's continue discussions in github
- 14:59:20 [dom]
- RRSAgent, draft minutes
- 14:59:20 [RRSAgent]
- I have made the request to generate https://www.w3.org/2021/06/18-web-networks-minutes.html dom