Shrinking Your Web Pages With Brotli

Brotli is an open source, lossless compression algorithm developed by Google. Originally designed to decrease the size of web fonts, Brotli now works as a general-purpose format for compressed data. Brotli acts as an alternative to gzip and Deflate, generating files that are smaller and decompress just as quickly.

A Brief History of Brotli

Brotli was originally made to compress WOFF2 web fonts. It is a continuation of Zopfli, another Google-developed compression algorithm built for backwards compatibility with gzip, zlib, and Deflate. Zopfli and Brotli both provide much better compression rates than gzip, zlib, or Deflate, albeit with much higher CPU usage. Over time, Brotli dropped backwards compatibility in favor of its own file format, which uses the .br extension.

Brotli is unique in that it can only be used over encrypted connections. This is due to intermediaries (or "middle boxes") intercepting and reading unencrypted HTTP traffic. When faced with a Brotli file, some of these intermediaries tried to pass the file through as gzip by changing it's Content-Encoding header from br to gzip. By limiting Brotli to HTTPS, Google prevents this interference to allow faster adoption of Brotli.

Benchmarks

Brotli offers significantly higher compression ratios than Deflate at the cost of higher CPU usage. In a 2015 study by Google, Brotli's highest compression level offered a 29% higher compression ratio than Deflate's highest level. However, Brotli's compression and decompression speeds at this level were 97% and 17% slower than Deflate's respectively (0.5 and 289.5 MB/s compared to 15.5 and 347.3 MB/s).

Because of its relatively slow compression speeds, companies are using Brotli to compress files in advance. In 2017, Dropbox converted the static content on dropbox.com to Brotli, decreasing the size of their assets by an average of 20–25%. Because the content is static, Dropbox can pre-compress files during the build process and serve them directly to users.

How Do I Use Brotli?

Several of the most popular web servers support Brotli either natively or using modules. Several CDNs also offer support for Brotli.

Like gzip, zlib, and Deflate, Brotli offers multiple compression levels. Higher levels generally provide better compression rates at slower speeds, with level 1 being the fastest and 11 being the slowest. Level 5 is recommended as a good balance between file size and speed, especially for dynamic compression.

Apache

Apache supports Brotli through the mod_brotli module, which was added in version 2.4.26. To enable Brotli, add the following to your virtual host configuration:

# Compress common text-based MIME types
AddOutputFilterByType BROTLI_COMPRESS text/html text/plain text/xml text/css text/javascript application/javascript

# Skip content that's already compressed
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-brotli

# Sets the compression level (defaults to 5)
BrotliCompressionQuality 6

mod_brotli recompresses content on each request. To serve pre-compressed files, copy your Brotli files to a folder accessible by Apache. Then, add a rewrite condition to your virtual host configuration that serves these files in place of their gzip equivalents.

# Serve brotli compressed CSS files if they exist and the client accepts brotli.
RewriteCond "%{HTTP:Accept-encoding}" "br"
RewriteCond "%{REQUEST_FILENAME}\.br" "-s"
RewriteRule "^(.*)\.css" "$1\.css\.br" [QSA]

# Serve brotli compressed JS files if they exist and the client accepts brotli.
RewriteCond "%{HTTP:Accept-encoding}" "br"
RewriteCond "%{REQUEST_FILENAME}\.br" "-s"
RewriteRule "^(.*)\.js" "$1\.js\.br" [QSA]

# Serve correct content types, and prevent double compression.
RewriteRule "\.css\.br$" "-" [T=text/css,E=no-brotli:1]
RewriteRule "\.js\.br$" "-" [T=text/javascript,E=no-brotli:1]

<FilesMatch "(\.js\.br|\.css\.br)$">
    # Serve correct encoding type.
    Header append Content-Encoding br

    # Force proxies to cache brotli &
    # non-brotli css/js files separately.
    Header append Vary Accept-Encoding
</FilesMatch>

Nginx

Nginx supports Brotli through the ngx_brotli module. The module actually consists of two separate modules: one for on-the-fly compression, and another for pre-compressed files. You will have to compile Nginx with the module included in order to use Brotli.

brotli on; # Enables on-the-fly compression.
brotli_static on; # Enables serving pre-compressed files.
brotli_types *; # Enables compression for all MIME types. text/html responses are always compressed.
brotli_comp_level 6; # Sets the compression level (defaults to 6).

Microsoft IIS

You can use Brotli in IIS through the open source Brotli Compression Scheme Plugin module. Using the installation script automatically installs and configures Brotli for all IIS sites. You can configure the compression parameters by opening your applicationHost.config file and navigating to the <httpCompression> section:

<httpCompression directory="%SystemDrive%\inetpub\temp\IIS Temporary Compressed Files">
  <scheme name="br" dll="%windir%\system32\inetsrv\brotli.dll" dynamicCompressionLevel="5" staticCompressionLevel="11" />
  <scheme name="gzip" dll="%windir%\system32\inetsrv\gzip.dll" dynamicCompressionLevel="4" staticCompressionLevel="9" />
  <staticTypes>
    <add mimeType="text/*" enabled="true" />
    ...
  </staticTypes>
  <dynamicTypes>
    <add mimeType="text/*" enabled="true" />
    ...
  </dynamicTypes>
</httpCompression>

As of this writing, the only configurable options are the static and dynamic compression levels.

CDNs

Several CDNs offer transparent support for Brotli including KeyCDN and Akamai. Other CDNs, however, don't recognize Brotli-encoded files and may even attempt to fall back to gzip. When LinkedIn rolled out Brotli across their site in 2017, they found that all five of their CDNs reverted the Accept-Encoding header from Vary to gzip even when they were serving Brotli files. The CDNs also couldn't cache the content, since LinkedIn's content servers contained Content-Encoding: br in their responses, which the CDNs couldn't understand.

To work around this, LinkedIn moved their Brotli files to a completely separate URL and stripped the Content-Encoding: br header. After the file is cached by the CDN, they add the header back in each response to the client. While it is a workaround, it improved performance by 2–6.5%, with mobile users seeing the biggest gains.

Browsers

Brotli is supported by most modern browsers with the exception of Internet Explorer 11 and Opera Mini. Chrome added support in version 50, Firefox in version 44, Edge in version 15, Opera in version 38, and Safari in version 11. 84.25% of browsers globally accept Brotli, while 83.42% of browsers in the United States support it.

Fortunately, browsers that don't support Brotli will fall back to alternative compression algorithms by simply not requesting Brotli in the Accept-Encoding header.

Conclusion

Brotli represents a new generation of data compression and storage. With the potential to decrease web page sizes by 20%, Brotli will lead to a faster, smaller web. You can learn more about Brotli on the official GitHub page.

• ← How JavaScript's Async and Defer Tags Let You Load Pages Faster
• What HTTP/2 Means for Frontend Development →