LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Articles > Jeremy's Magazine Articles
User Name
Password

Notices

By jeremy at 2008-03-14 11:18
Get Glossy with Varnish
Jeremy Garcia
Linux Magazine

In many of the recent" Tech Support" columns, I've shown you how to optimize and scale your Web infrastructure. This has ranged from fine tuning PHP and MySQL, to load balancing with Perlbal or Pound. To properly scale your infrastructure, you need to evaluate your applications and meet their specific needs. You have a variety of fantastic open source Web infrastructure components to choose from, and picking the correct components is as difficult as ever. To help sort through the cruft, I want to introduce you to a particularly useful app called Varnish, a high-performance HTTP accelerator.

What type of applications would benefits from Varnish? The primary target is dynamic applications that are modified relatively infrequently when compared to reads. Apps such as blogs and online news publications are great examples, but you'll need to look at your exact usage patterns to be sure. Keep in mind that HTTP accelerator is really just a buzzword for a caching reverse proxy-- which is to say, an application that sits in front of a server and routes requests to the server or handles the request itself.

The reverse proxies I've covered previously, Pound and Perlbal, are meant more for load balancing and do not perform any kind of content caching. Unlike Squid, which added Web acceleration as an additional feature to a product meant primarily as a forward proxy, Varnish is designed from the ground up to be an HTTP accelerator. It's available from http://varnish.projects.linpro.no/ under a BSD license.

Getting Varnish

Binary packages for varnish are available for most major distributions. If binaries aren't available for your distribution, or you'd like to compile from source, installation is the standard ./configure && make && make install. Varnish uses the Varnish Configuration Language (VCL), for all config directives. VCL is the heart of varnish, so you'll need to get to know it very well to get the most out of your cache. The VCL syntax is fairly simple, and deliberately similar to C and Perl. Blocks are delimited by curly braces, statements end with semicolons, and comments may be written as in C, C++ or Perl according to your own preferences. Regular expressions are used heavily.

The first item you'll need to add to your Varnish configuration is which server (s) you'd like to process backend requests. You can have any number of backend content servers, for simplicity the example below has only one.

Code:
backend default {
 set backend.host = "192.168.1.1";
  set backend.port = "80";
}
With the backend server defined, we can now move on to access control lists (ACLs). You can set up ACLs to bypass the cache for certain machines in an office, test the cache from only your workstation or even for more advanced traffic control. The following ACL will allow 192.168.1.0/24 and deny 192.168.2.0/24.

Code:
acl linuxmag {
  "192.168.1.0/24";
  ! "192.168.2.0/24";
}
The remaining VCL directives allow you to define request handling and document caching policies. You have the following blocks available to you: vcl_recv, vcl_pipe, vcl_pass, vcl_hash, vcl_hit, vcl_miss, vcl_fetch, vcl_deliver, vcl_timeout and vcl_discard. Covering each directive in detail is beyond the scope of this article, but the VCL man page has an explanation of each along with some example configuration code.

I will cover a couple brief examples to get you started, but would like to reiterate that for you to get the most out of Varnish you'll need to have both a good understanding of your application behavior and VCL.

vcl_recv is one of the main VCL directives. Its purpose is to decide whether or not to serve the request, how to do it, and, if applicable, which backend to use. To cache all images and CSS, use a block similar to the following.

Code:
sub vcl_recv {
  if (req.url ~ "\.(css|gif|png|jpg)$") {
            lookup;
    }
}
You can use multiple directives and set a default actions as follows.

Code:
sub vcl_recv {
  if (req.request != "GET" && req.request != "HEAD") {
               pipe;
      }
  if (req.http.Authenticate || req.http.Cookie) {
            pass;
      }
  lookup;
}
Once you have a Varnish policy you're happy with, it's time to start varnishd.

Code:
# /path/to/varnishd -a 192.168.3.1:80 -f /path/to/VCL.config-g nobody -u nobody -T localhost:2222 -s file,/var/varnish/storage.bin,1G
The above will start varnish as the nobody user on 192.168.3.1 port 80 with the management interface running on port 2222 of localhost. It'll use a 1GB cache file named storage.bin in /var/varnish using the policy you've defined in /path/to/VCL.config. For additional startup parameters, see man varnishd.

The management interface is a command line interface that can be accessed using telnet. It allows you to get statistics, dynamically reload VCL policies, stop/start the varnishd process and much more. The full details are covered in the varnishd man page.

Properly setting up and configuring Varnish is far from trivial. You'll need to know a fair amount about your application usage and get familiar with VCL. If you're a high traffic Web site, however, the learning curve will be well worth it.


  



All times are GMT -5. The time now is 12:30 AM.

Main Menu
Advertisement

My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration