Varnish is a reverse caching proxy. It is used to reduce the load on a server when the same website is being repeatedly requested.
This guide is for CentOS.
Quick Introduction to varnish:
Quick Links:
Install
Config Files
Commands
Configuring varnish
Master-Slave – configuration a varnish master slave configuration
Logs
Varnish: Port 80
Installation:
yum install varnish
service varnish start
Varnish config files:
/etc/sysconfig/varnish
The configuration files found in /etc/sysconfig/varnish and are used to specify details such as: Varnish listening port and varnish storage (files, memory, size).
/etc/varnish/default.vcl
The configuration file /etc/varnish/default is used to control vanish behaviour such as: Where to send traffic, how to handle requests and more.
Varnish Commands
Service varnish stop
service varnish start
service varnish restart clears the varnish cache and forcefully reloads new vcl
service varnish reload tests the syntax and will exit is there is an error. It will then apply the new vcl without clearing the cache
curl -I 127.0.0.1:80 | grep -i “varnish”
or
curl -I domain.co.uk | grep -i “varnish” this tests to see if varnish has been configured correctly
varnishd -C -f /etc/varnish/default.vcl this command will test to make sure default.vcl is configured correctly. It if is not it will return and tell you what line the error occurred. Note: sometimes syntax errors will be a different line to the one stated with the compiler (proceed with caution when making changed etc.)
varnishlog -r /var/log/varnish/varnish.log human readable binary varnishlogs (if they have been enabled)
varnishstat will show you a lot of statistics such as quick view of cache hit rate (most common use), connection counts and more.
varnishtop will show you a running count of the items varnishlog is reading. You can also combine it with -I $SOME_REGEX on the commandline to show a specific item
Configuring varnish (generic part)
We are now going to edit /etc/sysconfig/varnish to listen to the correct port and assign an amount of memory for varnish to use.
You can specify if you would like varnish to cache using filesystem or memory. If you are using varnish for performance then you should configure it to use memory.
We need to change the following:
VARNISH_LISTEN_PORT=80
VARNISH_STORAGE_SIZE=256M
Configuring Varnish Backend
For this you will need to edit /etc/varnish/default.vcl and change it to listen to port 8080. The section should look like:
backend default {
.host = "127.0.0.1";
.port = "8080";
}
An full example of a varnish default.vlc file can be found here.
If you are looking into varnish x-forwarded-for please visit my guide here.
Now you will need to configuring the web server. Please see below for apache and nginx.
You can run the following command to make sure you have no errors in your configuration: varnishd -C -f /etc/varnish/default.vcl
The command will return with an error and a line number if you have an issue in the file. Note: sometimes syntax errors will be a different line to the one stated with the compiler (proceed with caution when making changed etc.)
Apache:
You will need to make sure that apache is listening to port 8080, to do this you need to edit the config file /etc/httpd/conf/httpd.conf.
Example:
| BEFORE Listen *:80 NameVirtualHost *:80 <VistualHost *:80> … |
AFTER Listen *:8080 NameVirtualHost *:8080 <VirtualHost *:8080> … |
|---|
Note: you will need to change the listening port in all of your vhosts to port 8080
Nginx:
You will need to change the port of your server blocks to port 8080.
Note: You may also need to change the port in the file: vim /etc/nginx/conf.d/default.conf
Restart apache / nginx!
Reload varnish for the new changes to take effect with service varnish reload and you’re done!
You can test varnish is working on the server by performing the following (on the varnish server):
curl -I 127.0.0.1:80 | grep -i "varnish"
Configuring varnish – backend master
——–This section is still under construction. Please ignore for now———-
Further varnish configuration
Edit the /etc/varnish/default.vcl file again and add the following code below the backend default section:
backend master {
.host = "10.x.x.x";
.port = "80";
}
Varnish Access Control List (ACL)
——–This section is still under construction. Please ignore for now———-
This can be used to control PURGE requests. A PURGE request is an HTTP request that an application can send to varnish to expire an item from cache.
acl purge {
"localhost";
}
Varnish logging is not enabled by default when varnish is installed. There are two different types of logging and we will lightly explore both.
Using varnishncsa (apache style) logging
service varnishncsa start
chkconfig varnishncsa on
Logs will format to /var/log/varnish/varnishncsa.log
Using varnishlog (varnish syle) logging
Note: it is NOT advisable to have these logs turned on all of the time because they produce soo much information that they can cause too much disk I/O overhead on busy sites. This logging should be turned on for debugging purposes and then turned off after.
service varnishlog start
chkconfig varnishlog on (would advise against this)
varnishlog will produce logs in binary format to /var/log/varnish/varnish.log
To print the contents of the log file in human readable format you can use the following command:
varnishlog -r /var/log/varnish/varnish.log
Varnish Error Handling
404
Scenario: New content has been uploaded to the master server. A request for the new content is sent to the slave server but the content is not there! 404 is produced. With the following config it will retry the request and produce the correct content (without 404)
sub vcl_fetch {
if (beresp.status == 404 && req.restarts == 0) {
return(restart);
}