Meeting minutes
Update on Custom Font Diff Brotli Encoder
<Vlad> https://
<jpamental> jpamental present+
Garret: showing the results of work on a custom brotli encoder. This encoder is aware ofthe structure of the font and uses the subset plan information from the subsetter to produce diffs for glyf, loca, hmtx, and vmtx without having to actually access the raw glyf table bytes (except for new glyfs). This results in a significant speedup vs the generic brotli encoder (130ms to 3.6 ms for a 10,000 codepoint base) while only adding ~1kb to the pat[CUT]
Additionally I combined this with the immutable layout method shown last meeting and that yielded further speedups getting subset times down to 2.9 ms and reducing patch sizes.
For the non custom diffed tables the generic brotli encoder is used with a partial dictionary but produces a stream that compresses using the full dictionary.
Vlad: how much of this work do you plan to make part of the spec?
Garret: don't need spec changes to do this, but the spec should at least mention this technique.
Garret: should be able to link out to a document/reference implementation.
Garret: next steps are to finish up the document I started about this and add the final results and consider spec updates to note the method should probably be used.
Skef: status update. So we've been having some internal discussions. Our service is not regional (only caching is) and there's increasing enthusiasm for a solution that allows 100% serving from the cache.
Skef: and the design seems to be converging on something that would not be the range request specification, but might meet the same use cases as with range request.
Skef: you preprocess the font into the form where it becomes self sufficient on the client side.
Skef: I'm starting to look into this. In some number of months (no promises) if that comes to fruition we might be able to help turn that part of the spec into something that's ready to go.
Skef: I can't gaurantee it meets all these cases. Once I'm ready with a bsic draft I can share it.
Skef: someone can contact the now missing side of that work and see if it's a way forward.
Vlad: that makes sense, I would like to add that so far range request has been a general idea but we have a bunch of issues that need to be addressed before it becomes a spec.
Vlad: if you can beef it up that seems like a positive thing.
Vlad: we only need to review and agree on this as part of the group.
Vlad: so don't feel limited to get someone else approval to investigate.
Skef: nothing scheduled but I plan to do work on this in the short time.
Skef: we break things into clusters. The current range request spec has more logic on the client side. So you preprocess into several chunks that can be loaded seperately.
Skef: can possibly also do this with features.
Skef: break it up into these chunks by codepoint and if the client can load independent glyphs into the two tables then all it needs is the mapping of the chunks.
Skef: so yeah you need to load the mapping first after that you just load what you need out of flat files.
Skef: this will be completely regional cache compatible.
Vlad: regional version of range request to allow benefits from incxfer without logic on the server side.
Skef: so we could extend range request by doing even more preprocessing.
Jason: author experience is important we need to make it easy to adopt.
Skef: yes
Open issues (https://github.com/w3c/IFT/issues)
Vlad: ok reviewing issues nowl.
Garret: client conformance tests, no update.
Garret: table in the font no update yet. WIll look into next.
Skef: next is my issue #121
Skef: so first thing is we shouldn't have the patch format by convention.
Skef: there's a set of patch formats that the client indicates it can accept.
Skef: I think that when we've set a subset in some format it would be good to be able to continue in that format.
Skef: so since things can be cached so the client indicates it supports VCDIFF + Brotli, server sends VCDIFF, and then next the server sends Brotli.
Skef: so it's a matter of what matches what we previously used.
Skef: we have this model where you start by patching the null file and that works with the two formats. But say there's a format where you start with a woff and maybe the client keeps some extra.
Garret: ok I think we might need some small spec updates to allow on patch requests the client to specify only one format and not be forced to include VCDIFF.
Skef: #3 has to do with local caching and thinking about that in terms of how things work with normal fonts.
Skef: for example lets say a font is encoded in a woff do browsers when they download a woff they'll have to unpack to use.
Skef: do browsers cache the woff or the decoded woff.
Garret: I believe that it's the woff in the http cache, the rendering process may cache the ttf.
Skef: so section 4.4.1 talks about the font subset but doesn't specify the exact format.
Skef: so say you have a model that relies on a baseline subset, so you put in 20% of the glyphs and you'll patch beyond that and you want to beregional cacheable.
Skef: so if you download incrementally you'll always patch to the baseline.
Skef: in that case the thing you need to cache is the baseline subset, but that isn't what you put into rendering.
Skef: that should be possible
Skef: so basically you want to be able to have the patch application give back two things, 1. something to give back to the patch algorithm for future patch applications and 2. the actual produced font.
Garret: not opposed to this, will need to look at how we can change the spec to allow this to be an option.
Skef: #4 there's a seemingly mandatory checksum requirement on top of another checksum requirement. There should be an option if the endfile is checksumed.
Garret: 3 checksums in the protocol original font, the checksum from client to server to make sure the server produces the right base, and third one checks the patch application produced the right result. The third is optional.
Skef: the 3rd checksum is the one I'm worried about.
Garret: sounds good I think that we can remove it. It doesn't serve much of a purpose currently.
Vlad: maybe next we could talk about issue #103.
Skef: the al a carte issue. The way I understand it is the way we're righting the spec we are imagining a javascript being part of a separate process analgous to the webfonts loading api. The al a carte stuff is waiting on the api
Skef: until we have the format of the api.
Garret: agreed.
Skef: so maybe a good starting point is to build out the prototype api as part of the web assembly client.
Garret: update on the shared brotli is expired issue. The compression team has updated the version on the draft so it's no longer expired, but longer term we can consider just including the single paragraph from that spec which is relevant to shared brotli in our spec.
Skef: issue #94 waiting on spec changes.
Skef: provisions for regional caching has been pretty stable. Obviously the stuff I mentioned earlier has issues. Otherwise the main thing we're banking on is query.
Garret: no update, next step is to contact their wg.
Skef: current range request loads the first part of the file completely. There might not be a protocol selection at all.
Skef: it's more like loading a font file that tells you more.
Skef: it doesn't start out the same way as a communication with the server.
Skef: the experience of using this other format would much more like loading a font.
Skef: instead of saying I want to run IFT the first step is to download this baseline file.
Skef: so no need for dual tech specifiers.
<jpamental> Sorry - I have to drop for another call (and can't meet next week either!)
Vlad: let's do next call on Nov. 8th.