Brotli is a relatively-new compression algorithm from Google that's pretty awesome for web stuff. Firefox, Chrome, and Opera now support it and advertise it via Accept-Encoding. There's a C++ shlib and an nginx module already, too. Our options here would be:
- The simplest option to implement: We could switch off gzipping in Varnish entirely and move gzipping functionality to nginx first. It would then probably be trivial, to load the nginx brotli module and support both algorithms. I don't know how substantial the CPU hit would be, but this is the simplest option. It also removes all gzip complexity from varnish, which has been a source of past bugs and question-marks. Another big downside is varnish's effective storage size would be diminished due to lack of gzipped storage, at least on text. On the other hand, we already have more storage than we need there, and this doesn't really affect the bulk of cache_upload's objects anyways.
- The hard/questionable option: Write a varnish patch or vmod so that it can use brotli in addition to gzip if the client supports it, using the existing varnish gzip support mechanisms. This might be tricky for a number of reasons: the current gzip-support code assumes normalization to one compessed encoding, and also assumes it's a win to store objects in compressed form and hope we don't have to decompress them often as well, since most clients support gzip. We'd have to either double-store objects in both formats, or store in one and decomp->recomp to the other, or always store uncompressed and commonly compress outputs on the fly.
- If we think brotli is only supported by a minority of clients - Hybrid preferring gzip? Leave varnish doing gzip as it does now (so we get storage savings and compression-cpu savings in the common case), have nginx force "AE: gzip" just to normalize everything for varnish (and decompress gzip on the fly for non-gzip clients), and then also have nginx decode->recode for brotli on the fly when the client supports it.
- If we think brotli is supported by the majority of the clients, we can reverse (3) and mix it with (2): patch varnish to be able to replace gzip-support with brotli support universally (now we get even better storage savings than before), and have nginx on-the-fly decomp->recomp to gzip for older clients.
Probably the first thing to do here is to gauge brotli support level by sampling AE headers before varnish has normalized them to just gzip.
