Meeting minutes
<bts> start this meeting
Brandon: will be discussing compression streams and adding brotli.
Brandon: comrpession streams allows compressing/decompressing data
Brandon: issue opened for adding brotli
Brandon: already have support for zlib.
Brandon: one of the main points is around how we want to move forward for adding support and if compression and decompression support is needed.
Brandon: from online discussion it seems like whether to add compression + decomp or just one.
Skef: can we talk about the concrete issues first?
Skef: the main issue around compression is around the dictionary
Adam: has some impact on binary size, been trying to assess exactly what that impact is.
Randell: understanding from chrome is that z standard is that the encoder was about 600k vs 200k of the decoder.
Chrome has added encoder for some cases, and the increase was about 400kb
Skef: my understanding of brotli is there is a dictionary that can be referenced. Used by both. Is there a differenece in the dictionary in the implementation?
Adam: yes, I'm not sure of the specifics.
Adam: brotli has a peculiar feature that has some knowledge of english (eg. pluralize a word).
Adam: not just some data, it has some intelligence.
Adam: might actually be the code size and not just the dictionary.
Brandon: is it possible to compress without?
Adam: yes, not sure if there's an existing option but it's possible to just not mention the dictionary.
Adam: could be a platform predictability pattern if some platforms haven't and some don't
Adam: we should avoid surprising developers.
Skef: if we had to characterize the tradeoffs. Cost of binary vs cost of memory use. Maybe you could take what's used by decoder and recreate what the encoder needs.
Adam: I discussed a bit with the brotli team but they didn't seem to think it was an option.
Adam: it's not just a blob of data.
Skef: how big is it?
Adam: 190kb
Adam: if code size is also 400kb it is less of an issue.
Adam: before this week I tried to get some idea of what the criteria would be, but haven't been able to get an answer from the right people.
Adam: so we are kind of stuck
Brandon: talk is about impl perspective, haven't seen web dev perspective. Would dev expect to have both, would it be surprising/poor experience?
Adam: last year in spain I took position it's too confusing, but it's very common in the video codec world. So maybe it's not surprising. Also got feedback that decompression is still useful.
Adam: data point is compression is being used more then decompression. eg. for uploads
Adam: 6% of pages using streams and 0.12% using decompression streams
Brandon: if brotli could provide a better compression ratio would that be useful?
Adam: yes
The size concern might not affect all browsers.
should there be an allowance for optional support and have it be queryable
there is a desire on everyones side to have it and we're being tripped up. Maybe should make a compromise and move forward.
Brandon: there's also js runtimes (eg. cdn's) outside of browsers that would benefit.
Adam: not an expert on whether optional features are acceptable. I find the option attractive. If data later comes in its widely adopted that could help jsutify the size increase.
Adam: the immediate benefit of shipping is 0. If users in another browser are using then it demonstrates immediate benefit.
Adam: so open to it but not sure if more standards oriented people will be ok.
Adam: kind of a foot gun, but least worse kind (browser will throw exception immediately).
Adam: you could put uncompressed data in a brotli wrapper. Then devs wouldn't find out about it till later. That would be a worse deviation.
Skef: in trying to think about this there's two broad common cases. One the person just wants to transmit in a smaller size. If they're neutral on the particulars they just want what's smallest. So in those case if support is partial they may not use it since its easier to pick what's always there. Two you need brotli specifically. For example
you're making a WOFF.
Skef: in that case you'd need to polyfill. They can be clever what you load changes, for those optional interface is very attractive. If they can tell what's there they bring over what they need.
Skef: so I think it makes sense to have optionality.
Adam: constructor is simple (construct and see if it throws an error)
Randell: two comments there's another usecase which is storing stuff in local storage. In which case more comp is better but not critical. Same comment for uploading. Soft failover without polyfill is viable route.
Adam: I propose that at the friday webperf meeting we bring the proposal of an optional interface. We might have a proposal for optional codecs in general.
Adam: then see what they say.
Brandon: is that optional decompression as well or just compression?
Adam: adding decompression is easy.
Skef: decoders are in all browsers for woff2
Adam: in chrome we're not in a position to ship compression at the moment. Friday I should have a better idea, but I think the answer is no.
Adam: personal feeling is optional support is positive for web and chrome.
Adam: if it does get used we can ship it.
Brandon: concern it might be confusing for web devs. For future algorithms will it be the same thing?
Adam: agree it's confusing but I've come around to the position that providing just decompression is sufficient value to make up for the confusion.
Skef: there's two potential levels having one but not the other, or the optional support.
Brandon: from using an api you might expect both are available.
Brandon: lot of discussion has been around impl details.
Brandon: impl is leading the api.
Randell: fundemental issue is whether to expand the options.
Randell: what is the justification it. For decomp could include leading compressed things on the web and has higher ratio then existing streams. For compression its getting a higher level of compression for things like storage.
Randell: how critical are either of those, are there hard drivers that need brotli decompression.
<bts> Garrett: Building incremental font transfer Brotli decompression required
Garret: as an example if you're building an incremental font transfer client you need brotli specifically.
Skef: for the cases in which you're trying to decompress something if theres a standard, you don't always have control over the standard its helpful to have decompression more widely supported.
Skef: and incremental font transfer is one of those cases.
Randell: associated with that the argument against these theres no argument about code size since all browsers already have decompression.
Randell: that argument has more bearing on compression side.
Randell: you're adding a tax on all browsers and users.
Brandon: many users may have data caps and limited service. If they're loading a page and want to save bytes, could save those users greatly. While a borwser increases in size but all users could see a significant decrease in size.
Randell: brotli isn't dramatically better but is an improvment.
Vlad: brotli has a slider for compression level.
Randell: are you proposing as part of the compression api exposing the slider.
?
Adam: always been the plan to include compression levels. Was stuck on how to represen them. Now that we are all settled on 1 to 9.
Adam: brotli may have some additional knobs.
<bts> Garret: Extension to Brotli allows you to a different dictionary.
Adam: the other codecs have similar options so does make sense.
Adam: if the compressor doesn't have knowledge of the pattern you can supply a small dictionary relevant to your content.
Adam: that's mostly for small messages.
Garret: also useful for constructing patches.
Randell: firefox and chrome are shipping this as compression dictionaries.
Adam: we are aware of benefits of compression dictionaries and looking to expose. I'm leaning towards exposing as much as possible.
Adam: we'll take an incremental approach.
Skef: if there is this scale commonality its good to have reasonable default and then particulars for each protocol. So clean default, neutral adjustment, and then particulars.
Brandon: exposing the compression levels is a different topic then brotli support.
Adam: we're also in similar position with z-standard. Can ship decomp cheaply, we have it on desktop but not android.
Adam: as to the point of how much improvement. In terms of how much you get for your cpu usage both are astrononmically better then zlib. You get more compression per instruction.
Adam: and I think its beneficial to provide modern algorithms. deflate only has 32kb window which is small by modern standards.
Randell: I do wonder about the average size of things compressed by the api.
Adam: I suspect some cases are only 100 bytes at a time.
Skef: I don't have actual empirical data, but I would guess if you were to draw the curve for 1 through 9 I wouldguess the change in both processor usage is all different.
Adam: brotli 11 is very expensive.
Adam: brotli 11 does an exhaustive search.
Adam: z-standard has a bunch of different tradeoffs.
Adam: does byte at a time compression.
Adam: back references without huffman encoding but may be wrong.
Adam: has -3 which gives poor compression quickly. Competitive with other byte at a time codecs.
Kagami: comment that if someone makes a website without testing on all browsers. Then it may break on other browsers.
Kagami: not against making it optional. Would be open to providing the dictionary.
Brandon: other comments or questions?
Skef: to reiterate the point of why we're here our most basic desire is don't let reluctance about compression side interfere with decompression side. If that desire is enough to push compression in that's great but if not we strongly prefer some sort of solution. Hard split where only one is available, failing that have compression be optional,
failing that only decompression is available.
Brandon: mentioned bringing a proposal to whatwg?
Brandon: still have certain concerns about web developers being confused, especially given the wide use of compression in the data.
Brandon: including it would allow developers to really decrease sizes in limited bandwidth cases.
Adam: why we care about bandwidth. Many users with cheap android phones with minimal storage thats usually full. Makes it hard to update chrome.
Adam: and they get stuck on old versions of chrome. So binary size is very important.
Skef: being doing opentype standards work and seeing the same thing. Desire to get storage size (not transfer size) down.
Nidhi: 400kb I mentioned earlier is on desktop. Only 200kb on android
Randell: Know why that is?
Adam: on android we compile optimized for size, but not always for the compression impl.
Skef: for android case is there really nothing on android that compresses brotli?
Adam: we don't like to lay on system libraries. I can look into it. Whether the compressor is already shipped.
Brandon: will bve discussed further with proposal on Friday.
<aprotyas> joint meeting link: https://