14:01:47 <RRSAgent> RRSAgent has joined #web-networks
14:01:47 <RRSAgent> logging to https://www.w3.org/2021/06/18-web-networks-irc
14:01:50 <Zakim> Zakim has joined #web-networks
14:02:11 <cpn> cpn has joined #web-networks
14:02:59 <dom> Present+ Dom, AliBegen, Arpit_Bansal, ChrisNeedham, DanDruta, EdLambert, EricSiow, JonasSvennebring, PiersOHanlon, SantoshRedy, Satya, SOngXu, Sudeep, ThoharisCharitidis, YajunChen
14:03:24 <dom> Present+ LarryZhao
14:03:29 <dom> Chair: SongXu
14:03:55 <dom> Meeting: Network Emulation Tool in Browser Developer Tools
14:04:22 <dom> SongXu: our meeting is about Network Emulation Tool in Browser Developer Tools
14:04:40 <dom> ... our agenda: some background of this work and goals
14:04:51 <dom> ... overview of existing tools that are relevant to similar functionalities
14:05:02 <dom> ... followed by discussions on the models & requirements
14:05:07 <dom> Topic: Background & Goal
14:05:24 <dom> Song: During TPAC, there was interest showed around network condition emulation
14:05:38 <dom> ... we want to review concrete proposal to improve network emulation tools for web developers
14:05:45 <dom> ... and identify standardization opportunities for trace format
14:06:18 <dom> ... our initial roadmap on this was to standardization efforts around trace format in Q2 2021
14:06:23 <dom> ... the landscape analysis too
14:06:34 <dom> ... have use cases defined in Q3 2021, along with emulation methodology
14:06:42 <dom> Topic: Quick Overview about existing tools
14:06:55 <dom> Song: WebPageTest, LPP Network Trace Tools and Throttle
14:07:24 <dom> LarryZhao: WebPageTest (aka WPT) - a web performance tool providing diagnostic under a variety of conditions
14:07:36 <dom> ... it helps find performance issues, based on test run from a different set of locations around the world
14:08:14 <dom> ... You can also pick a type of connection - e.g. LTE
14:08:28 <dom> ... the documentation shows that this is done using NetEm and dummynet
14:09:00 <dom> ... netem is an option enabled in the linux kernel, e.g. with packet delay, loss, duplication, reordering
14:09:18 <dom> ... When WPT simulates network, it works at the network level directly
14:09:28 <dom> Subtopic: LPP Network Trace Tools
14:09:39 <dom> LarryZhao: we heard from Jonas on this before
14:09:52 <dom> ... it features 3 important parts: trace capture, emulator and a file format
14:10:02 <dom> ... it is presented as a chrome extension
14:10:07 <dom> ... [shows demo of LPP]
14:10:29 <dom> ... it utilizes the Chrome Dev Tool Protocol
14:10:37 <dom> ... so it operates at the level of application network
14:10:49 <dom> ... it only has effects on the Web site you visit via Google Chrome
14:11:16 <dom> ... that's a major difference between WPT and LPP - network level vs app level
14:11:20 <dom> Subtopic: Throttle
14:11:48 <dom> LarryZhao: Similar to WPT, from sitespeed.io to simulate slow network conditions
14:11:56 <dom> ... it tool works at the network level
14:12:05 <dom> ... very similar to WPT
14:12:14 <dom> i/LarryZhao: WebPageTest/Subtopic: WPT
14:12:46 <dom> Sudeep: is the Throttle tool similar to traffic control (tc) tools?
14:12:51 <dom> LarryZhao: it is based on TC
14:14:49 <dom> Arpit: how about the Mac version?
14:15:01 <dom> Song: they use the same core library across Mac/Linux
14:15:24 <dom> Piers: it uses dummynet underneath afaik
14:15:50 <dom> ... throttle uses both pfctl & dummynet
14:16:01 <dom> ... to provide something similar to tc & netem
14:16:06 <dom> SubTopic: Gap Analysis
14:16:34 <dom> Song: some of the tools can only emulate limited network conditions
14:16:55 <dom> ... but it's useful to be able to test under network switch (e.g. from WIFI to LTE)
14:17:19 <dom> ... with WebRTC & cloud gaming scenarios, you need access to more conditions (e.g. package dropped, different loss rates)
14:17:55 <dom> ... there are features common across tools: up & down bandwidth, latency / rtt
14:18:14 <dom> ... how much interop is needed for these features? a standardized trace format would help in that case
14:18:23 <dom> Topic: Use cases
14:18:34 <dom> Song: I've collected use cases that would make use of this
14:19:12 <dom> ... one is test automation - a developer could test their sites under various real life network conditions previously captured
14:19:24 <dom> ... WPT has a collection of such traces
14:20:19 <dom> ... Another use case around test & development
14:20:28 <dom> Jonas: this is one use case we've considered
14:20:52 <dom> ... Our main intention was to use it for standards Web application testing to get a feeling how the connection is in different parts of the world
14:21:13 <dom> ... we're studing extensions for MPEG dash, cloud gaming, real-time media
14:21:20 <dom> s/stud/study/
14:22:44 <dom> Song: another use case is for PWA testing
14:23:13 <dom> Sudeep: the input on this is also available on the github repo
14:23:37 <dom> -> https://github.com/w3c/network-emulation/issues/3 Network emulation Use Cases
14:24:27 <dom> Sudeep: testing scenarios where some requests randomly timeout has proved difficult in the context of an app experience
14:24:39 <dom> ... and testing robutness depending on the frequency of these timeouts
14:25:15 <dom> ... Another thing they mentioned is testing how service workers react to these varying network conditions, e.g. when using background sync APIs
14:25:49 <dom> ... having a way to automate this testing in local environments would go a long way
14:26:14 <dom> ... the last item is emulating online/unreliable/offline states
14:27:15 <dom> ... there can be scenarios where a given request will go fine, but then another will fail due to a e.g. a proxy redirect
14:27:26 <dom> ... this creates delays in how the web app will load & render
14:27:41 <dom> ... and this can't be dealt with today at the level of browser developer tools
14:28:28 <dom> Topic: Models or requirements
14:28:52 <dom> Song: Opening the floor to discuss what features of emulation, what parameters to monitor, what methods to use to emulate
14:29:04 <dom> Present+ DapengMax
14:29:41 <dom> Ali: I've added two slides to discuss other use cases
14:29:49 <dom> Subtopic: testing http streaming clients
14:30:07 <dom> Ali: we use network emulation to test streaming scenarios
14:30:20 <dom> ... HTTP streaming is very popular, consumes a lot of bandwidth and triggers a lot of research
14:30:29 <dom> ... we depend quite heavily on network emulation tools
14:30:47 <dom> ... at a high level, the model is that a streaming client fetches media segments from a server
14:30:59 <dom> ... based on information known in advance on what to request when to request
14:31:25 <dom> ... the stream is available in different resolutions & bitrates, allowing to adapt the stream (hence the name adaptive streaming)
14:31:38 <dom> ... the most important parameter to determine what bitrate is bandwidth evaluation
14:31:45 <dom> ... there can be others (e.g. CPU & memory)
14:31:57 <dom> ... but the general case is the network is the main bottleneck
14:32:19 <dom> ... the client needs to measure its active bandwidth and predict what the right segment will be
14:32:37 <dom> ... the measure is done based on the data being fetched
14:32:53 <dom> ... but this can be pretty noisy given the different sizes of segment, the impact of round trips, etc
14:33:12 <dom> ... what most clients do is that they smoothed out the value (e.g. average, percentiles)
14:33:34 <dom> ... some clients make prediction out of this
14:33:47 <dom> ... this gets done over & over during the streaming session
14:34:16 <dom> ... my understanding is that LPP can be used to help with this
14:34:55 <dom> ... in terms of measuring and predicting the bandwidth, although not about how the client should behave
14:35:04 <dom> ... This leads to a number of requirements:
14:35:14 <dom> ... network emulation is a big part of our testing in our daily life
14:35:24 <dom> ... we usually set different upload & download conditions
14:35:31 <dom> ... with different network congestion
14:35:56 <dom> ... our main focus is on bandwidth throttling, we're mostly ignoring e.g. packet loss or re-ordering
14:36:47 <dom> ... when dealing with live streaming, we need throttling on short time scales too
14:37:04 <dom> ... Reproducibility is a very important requirement for us
14:37:16 <dom> ... we need to be able to test these conditions across different devices & content
14:37:41 <dom> ... this means it should be easy to design programmatic traces, including modifications to existing traces (e.G. scale up the bandwidth)
14:37:56 <dom> ... e.g. just having to adjust a scale factor
14:38:05 <dom> ... likewise being to scale the timescale on which it runs
14:38:15 <dom> ... the files should be small, easy to edit and share
14:38:44 <dom> Song: thanks for the excellent input
14:39:14 <dom> Ali: we heavily use the built-in throttling tool in Chrome - it has proved easy to script, but it doesn't have all the features we want
14:39:21 <dom> ... we would be more than happy to switch to a better tool
14:39:31 <dom> Jonas: the scaling aspect is a good one to add
14:39:42 <dom> Ali: yes, both in terms of magnitude & time
14:40:19 <dom> ... sometimes, we design a trace that we just want to replicate under different scales
14:40:36 <dom> Jonas: that indeed helps with sensitivity tests
14:40:59 <dom> ... With regard to streaming, I agree that LPP shouldn't be involved in how the estimations and predictions are used
14:41:23 <dom> ... we're exploring different type of behaviors of what we can do with these traces - of course not forcing them on any one
14:41:59 <dom> ... we're planning to publish a set of examples that demonstrate different type of optimizations (e.g maximizing quality vs minimizing buffering)
14:42:14 <dom> RRSAgent, draft minutes
14:42:14 <RRSAgent> I have made the request to generate https://www.w3.org/2021/06/18-web-networks-minutes.html dom
14:42:14 <dom> RRSAgent, make log public
14:43:05 <dom> Sudeep: how do know which use cases would benefit the most when we start emulating network conditions?
14:43:30 <dom> ... are there use cases that have no solution to manage the quality experience e.g. under congestion control?
14:43:40 <dom> ... how do cloud gaming deal with these?
14:44:14 <dom> Piers: WebRTC has P2P traffic
14:44:31 <dom> ... not sure how Chrome throttling applies to that
14:44:49 <dom> Sudeep: I tried it but it didn't seem to work
14:45:29 <dom> Piers: another question is network emulation in the context of newer congestion algorithms like BBR which are delay based
14:46:18 <dom> Ali: I mentioned we didn't care much about packet loss & reordering - that's true for HTTP 1.1 & HTTP 2 since they're based on TCP
14:46:37 <dom> ... but as we start testing HTTP3 at some point in the future, these things might become more relevant - hard to say how much at this point
14:47:02 <dom> ... but as QUIC replaces TCP, we will be interested to access these aspects too
14:47:33 <dom> ... that aspect will also be more relevant for cloud gaming & webrtc
14:47:55 <dom> RRSAgent, draft minutes
14:47:55 <RRSAgent> I have made the request to generate https://www.w3.org/2021/06/18-web-networks-minutes.html dom
14:48:53 <dom> Topic: Next steps
14:49:12 <dom> Song: I'll bring these topics we've identified to our github repo
14:49:31 <dom> Jonas: in terms of trace format standardization, it was scheduled for Q2 which is about finished
14:49:38 <dom> Song: yes, this will have to be revisited
14:49:46 <dom> ... probably more of a Q3 now
14:50:03 <dom> Jonas: I guess the schedule depends on the interest around standardizing such a format
14:51:22 <dom> Dom: which producers / consumers of such a format would need to be involved in these discussions?
14:52:38 <dom> Ed: is there a timeframe for getting feedback on trace format standardization?
14:52:49 <dom> Song: I would like to say by the end of July
14:53:36 <dom> Piers: has there been an assessment of other commercial & open source approaches in this space?
14:54:23 <dom> Song: we did part of this stuff in the past, but we didn't look at commercial implementations yet
14:54:36 <dom> Piers: there are quite a few network emulators out there, I'm not sure about their format
14:54:43 <dom> ... hotnet, NS (?)
14:54:52 <dom> ... they're often programmatic rather than declarative
14:55:06 <dom> ... in streaming papers, they tend to use traces from the FCC and other sources
14:55:12 <dom> ... but they tend to be just CSV files
14:55:20 <Yajun_Chen> Yajun_Chen has joined #web-networks
14:55:49 <dom> Sudeep: in a last meeting, two tools were mentioned: crowdad and network management labs
14:56:14 <dom> piers: indeed - a new one is emerging from Standford, Puffer (?) that provides anonymized logs of clients
14:56:24 <dom> ... there are a number of repositories traces outthere
14:56:44 <dom> -> https://puffer.stanford.edu/ Puffer
14:57:08 <dom> Jonas: Crowdad seems like a mix of formats
14:57:16 <dom> -> http://crawdad.org/ CRAWDAD
14:57:20 <dom> s/Crow/Craw/g
14:57:29 <Larry_Zhao> Larry_Zhao has joined #web-networks
14:57:50 <dom> -> https://www.measurementlab.net/#/ Measurement Lab
14:57:53 <Larry_Zhao> Larry_Zhao has left #web-networks
14:58:06 <dom> Sudeep: it makes sense to look indeed at existing formats & repositories
14:58:46 <dom> Song: thank you all for joining this meeting
14:59:05 <dom> ... let's continue discussions in github
14:59:20 <dom> RRSAgent, draft minutes
14:59:20 <RRSAgent> I have made the request to generate https://www.w3.org/2021/06/18-web-networks-minutes.html dom