12 Days of Christmas Performance
brought to you by REVSYS

Varnish

Good timings we bring
To you and your kin
We wish you a Faster Website
And a Happy New Year

Now bring us some friggin' traffic
Now bring us some friggin' traffic
And a cup of good cheer
Santa's SEO Song

Varnish is the perfect gift this holiday season. It's the Tickle Me Elmo for us performance geeks. Especially for those of us who are permanently on Santa's Performance naughty list and are making arrangements to receive a metric ton of coal.

So what's Varnish? It's a caching reverse proxy. Everyone's familiar with browser cache and we've already covered using things like far future expires headers to take better advantage of it. Browser cache is per user however, a distributed cache if you will. Varnish is our site's cache.

It's a small super faster daemon that usually sits between nginx and your application server. Based on rules you configure, it listens to the traffic being proxied through it and caches the results in memory or on disk for exactly how long you want. Serving up that content back to future users without having to bother your application server with it.

Some things Varnish can do for us:

  • Serve cached content, taking load off our app server(s)
  • Serve fresh or stale content based on complicated rules we set
  • Store and serve fragments of whole HTML pages
  • Prevents the dog piling problem
  • Serve stale content if our app server is down

We're not going to lie, configuring Varnish using VCL (Varnish Configuration Language) isn't exactly easy to get right. It's complicated becacuse you likely want to cache certain pages for differing amounts of time, probably based on the logged in status of your user, and you want to avoid user's noticing stale caching issues.

So your exact configuration is going to vary based on things like:

  • Presence or absence of particular cookies
  • Cache related headers coming from your app
  • Possibly other headers from your app
  • Whether or not your site is internationalized to deliver content in multiple languages
  • Your tolerance level for stale content in different sections of your site

Everyone's exact situation is different so we won't dive into how to set it up in detail. Instead, we're going to explain different ways you can take advantage of Varnish instead.

Varnish Use Cases

Whole Site Caching

The simplest use case for Varnish is your typical not very dynamic content heavy site. We use Varnish in front of this site, www.revsys.com in a very simple way. Our site is built with Django of course, and while it's dynamic, the content itself doesn't change all that frequently.

So we configured Varnish to ignore all cookies and cache headers and simply cache all pages for 5 minutes. We exclude a few more dynamic URLs of our site, for example the Django admin, but otherwise Varnish caches everything for us.

For your typical CMS or blog site this works beautifully. We can write absolutely terrible, from a performance standpoint, code and varnish over the mess.

Think about what this does for us, for the 99.99% of page views that are cached it removes the following from the request/response cycle:

  • The overhead of every single database query
  • The overhead of your template engine to generate the page
  • The over head of a half dozen or more memcached or redis calls
  • The overhead of additional app server processes to handle your load

It essentially turns your dynamic, but not frequently updated site, into a stupid fast static site. Easily capable of handling being on the front page of Reddit and HackerNews at the same time with a 1-2GB cloud instance.

REST APIs

Putting Varnish in front of your REST API is a another perfect use case. Not only for the usual this bit of data doesn't change that often scenario, but also to help protect you against bad actors much like you achieve with throttling. In fact, if you're heavily using Varnish in front of your API you arguably get rid of throttling entirely.

You can configure Varnish to listen for a special HTTP PURGE method to evict specific URLs or URLs matching a pattern from the cache. Keep in mind this isn't specific to REST APIs and can be used with any URL of your site. This let's us have the best of both worlds, caching when we need it and nearly instant updates when the underlying data changes.

Let's talk about a specific example, Twitter. If you were building Twitter two of the least changing API ends points, on a per user basis, are the lists of who a user follows and who follows them. Compared to the actual Tweet stream, this rarely changes. Let's assume these end points look like this:

/api/v2/<user>/follows
/api/v2/<user>/following

Let's assume we're building this with Django and django-friendship. While django-friendship emits django signals whenever the follower/following relationships change, we can also just listen for the underlying model to change. We can then PURGE these two end points using something like this:

import requests
from django.db.models import post_save
from django.dispatch import receiver

from friendship.models import Follow

@reciever(post_save, sender=Follower):
def bust_follower_apis(*args, **kwargs):
    instance = kwargs['instance']

    end_point1 = '/api/v2/{}/follows'.format(instance.follower.username)
    requests.request("PURGE", end_point1)

    end_point2 = '/api/v2/{}/following'.format(instance.followee.username)
    requests.request("PURGE", end_point2)

Now we can cache these end points for minutes, hours, days, hell even years since we are confident the cache will be invalidated anytime the data changes.

Keep in mind that in your Varnish configuration you will want to restrict these PURGE requests to only be allowed from your internal applications. Possibly also requiring a shared secret header of some sort to ensure the malicious public doesn't purge your cache for you.

ESI – Edge Side Includes

Have you ever noticed that most sites and even apps have only a couple of spots that are really dynamic with the rest of the pages being relatively static content? There are lots of bits of content that appear on other pages and maybe in slightly different forms.

Edge Side Includes work basically just like the old Server Side Includes we all used back in the CGI days. Your app, well specifically the markup it emits as this can be applied truly static pages as well, spits out something like:

<div class="navbar">
    <esi:include src="http://our-backend/navbar.html">
</div>

And the content of that URL is inserted verbatim into the markup at that point in the page.

Let's do another quick example to visualize how you can use this. We'll use Medium.com as an example. On each individual post, there is the post itself and a small bio about the author. These two elements obviously change, but pretty independantly from each other.

If we setup our code to return these markup fragments individually we can do something like this:

<div class="body">
    <esi:include src="http://our-backend/esi/body/12345678/">
</div>
<div class="bio">
    <esi:include src="http://our-backend/esi/user/bio/987654/">
</div>

Now when a user vists http://our-frontend.com/post/awesome-post all our application needs to do is lookup the post and user IDs that correspond to the awesome-post slug and render that small bit of ESI markup. Varnish will take care of grabbing those two URLs, stitching them into the page. If we're caching these end points, and we should be, then the stitching process is just another cache lookup and string concatenation.

Now when user #987654 logs in and changes the text of their bio, we simply PURGE /esi/user/bio/987654 and the new info is from then on stitched into all of the posts on the site.

One benefit aside from the modularity this bring us is this cuts down on the amount of storage, both in memory and on disk, of the Varnish cache allowing us to cache more of our site and for longer periods of time.

We all know cache invalidation is one of the hard problems in computer science. Varnish isn't a panacea, but if we construct our ESI URL schemes in a smart hierarchical manner we can make our lives easier. Consider the following:

/esi/user/1234/bio/full
/esi/user/1234/bio/short
/esi/user/1234/profile
/esi/user/1234/picture/large
/esi/user/1234/picture/medium
/esi/user/1234/picture/small

This HTML fragments could be being used all over our site in various places. But now if user #1234 changes anything about their data we can just issue:

PURGE /esi/user/1234/*

And bust ALL of fragments related to user #1234. As you can see this pattern can be extremely flexible, powerful, and fast at the same time.

Commercial Options

There are a couple of commercial options you can consider if you don't feel like running Varnish yourself. Honestly, running Varnish isn't hard, it's the configuration that is hard to get exactly right.

Fastly for example is essentially CDN style distributed Varnish in the cloud. You can even setup more complicated than normal rules using VCL.

Many other more advanced CDNs, for example Akamai have features in place that give you all of the advantages of Varnish, just without Varnish. You wouldn't be digging into VCL, but instead docs for their particular product. However, the use cases and techniques outlined above all still apply.

Real World Varnish

Varnish is extremely popular and used by sites such as Wired, Urban Dictionary, MacRumors, Zappos, and Disqus to name drop a few. You can read more on how Disqus scaled Django to 8 billion page views in part by using Varnish.

Hope you learned something new and can take advantage of Varnish in one of your projects. Happy Holidays!

 Day 6 - Caching Django Sessions
Day 8 - Testing