Nginx and W3 Total Cache

With the increasing popularity of the lightweight nginx server, the latest developer build of W3 Total Cache, includes support for nginx. As nginx does not have directory specify files (e.g. .htaccess), the settings are intended to be added to one’s nginx.conf file.

Before preceding, I simply cannot pass up an opportunity to extol the virtues of this plugin – its optimizations are multifaceted, and the end results are beyond compare – many thanks to the developer, W3 EDGE for all the hard work developing such an outstanding performance plugin.

The following nginx configuration is generated by W3 Total Cache, with a few modifications.

Note: the setup is as follows:
nginx (0.8.53) frontend with apache (2.2.16) backend
W3 Total cache (0.9.1.4b) with page cache, browser cache, minify, and url rewriting (and using a CDN, but that doesn’t affect the nginx rules)

server {
	server_name  www.example.com example.com;
	access_log  /var/www/html/example.com/logs/nginx_access.log;
	error_log  /var/www/html/example.com/logs/nginx_error.log debug;
	root   /var/www/html/example.com/public_html/;

	# BEGIN W3TC Page Cache core
		set $w3tc_rewrite 1;
	if ($request_method = POST) {
		set $w3tc_rewrite 0;
	}
	if ($query_string != "") {
		set $w3tc_rewrite 0;
	}
	if ($request_uri !~ \/$) {
		set $w3tc_rewrite 0;
	}
	set $w3tc_rewrite2 1;
	if ($request_uri ~* "(\/wp-admin\/|\/xmlrpc.php|\/wp-(app|cron|login|register|mail)\.php|wp-.*\.php|index\.php)") {
		set $w3tc_rewrite2 0;
	}
	if ($request_uri ~* "(wp\-comments\-popup\.php|wp\-links\-opml\.php|wp\-locations\.php)") {
		set $w3tc_rewrite2 1;
	}
	if ($w3tc_rewrite2 != 1) {
		set $w3tc_rewrite 0;
	}
	if ($http_cookie ~* "(comment_author|wp\-postpass|wordpress_\[a\-f0\-9\]\+|wordpress_logged_in)") {
		set $w3tc_rewrite 0;
	}
	set $w3tc_ua "";
	if ($http_user_agent ~* "(2\.0\ mmp|240x320|alcatel|amoi|asus|au\-mic|audiovox|avantgo|benq|bird|blackberry|blazer|cdm|cellphone|danger|ddipocket|docomo|dopod|elaine/3\.0|ericsson|eudoraweb|fly|haier|hiptop|hp\.ipaq|htc|huawei|i\-mobile|iemobile|j\-phone|kddi|konka|kwc|kyocera/wx310k|lenovo|lg|lg/u990|lge\ vx|midp|midp\-2\.0|mmef20|mmp|mobilephone|mot\-v|motorola|netfront|newgen|newt|nintendo\ ds|nintendo\ wii|nitro|nokia|novarra|o2|openweb|opera\ mobi|opera\.mobi|palm|panasonic|pantech|pdxgw|pg|philips|phone|playstation\ portable|portalmmm|ppc|proxinet|psp|pt|qtek|sagem|samsung|sanyo|sch|sec|sendo|sgh|sharp|sharp\-tq\-gx10|small|smartphone|softbank|sonyericsson|sph|symbian|symbian\ os|symbianos|toshiba|treo|ts21i\-10|up\.browser|up\.link|uts|vertu|vodafone|wap|willcome|windows\ ce|windows\.ce|winwap|xda|zte)") {
		set $w3tc_ua _low;
	}
	if ($http_user_agent ~* "(acer\ s100|android|archos5|blackberry9500|blackberry9530|blackberry9550|cupcake|docomo\ ht\-03a|dream|htc\ hero|htc\ magic|htc_dream|htc_magic|incognito|ipad|iphone|ipod|lg\-gw620|liquid\ build|maemo|mot\-mb200|mot\-mb300|nexus\ one|opera\ mini|samsung\-s8000|series60.*webkit|series60/5\.0|sonyericssone10|sonyericssonu20|sonyericssonx10|t\-mobile\ mytouch\ 3g|t\-mobile\ opal|tattoo|webmate|webos)") {
		set $w3tc_ua _high;
	}
	set $w3tc_ref "";
	set $w3tc_ssl "";
	if ($scheme = https) {
		set $w3tc_ssl _ssl;
	}
	set $w3tc_enc "";
	if ($http_accept_encoding ~ gzip) {
		set $w3tc_enc .gzip;
	}
	#set $w3tc_rewrite 1;
	if (!-f "/var/www/html/example.com/public_html/wp-content/w3tc/pgcache/$request_uri/_index$w3tc_ua$w3tc_ref$w3tc_ssl.html$w3tc_enc") {
		set $w3tc_rewrite 0;
	}
	if ($w3tc_rewrite = 1) {
		rewrite (.*) "/wp-content/w3tc/pgcache/$1/_index$w3tc_ua$w3tc_ref$w3tc_ssl.html$w3tc_enc" last;
	}
	# END W3TC Page Cache core

	# BEGIN W3TC Page Cache cache
	location ~ /wp-content/w3tc/pgcache.*html$ {
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b (Cached)";
		add_header Vary "Accept-Encoding, Cookie";
		add_header Pragma "public";
		add_header Cache-Control "public, must-revalidate, proxy-revalidate";
		expires 7d;
	}
	location ~ /wp-content/w3tc/pgcache.*gzip$ {
		gzip off;
		types {}
		default_type text/html;
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b (Cached, Gzipped)";
		add_header Vary "Accept-Encoding, Cookie";
		add_header Content-Encoding gzip;
		add_header Pragma "public";
		add_header Cache-Control "public, must-revalidate, proxy-revalidate";
		expires 7d;
	}
	# END W3TC Page Cache cache

	# BEGIN W3TC Browser Cache - Part 1
	gzip on;
	gzip_types text/css application/x-javascript text/richtext image/svg+xml text/plain text/xsd text/xsl text/xml image/x-icon;
	# END W3TC Browser Cache - Part 1

	# BEGIN W3TC Minify core
	set $w3tc_enc "";
	if ($http_accept_encoding ~ gzip) {
		set $w3tc_enc .gzip;
	}
	if (-f $request_filename$w3tc_enc) {
		rewrite (.*) $1$w3tc_enc break;
	}
	rewrite ^/wp-content/w3tc/min/([a-f0-9]+)\/(.+)\.(include(\-(footer|body))?(-nb)?)\.[0-9]+\.(css|js)$ /wp-content/w3tc/min/index.php?tt=$1&gg=$2&g=$3&t=$7 last;
	# END W3TC Minify core

	# BEGIN W3TC Minify cache
	location ~ /wp-content/w3tc/min.*\.js$ {
		types {}
		default_type application/x-javascript;
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b (Minify)";
		add_header Vary "Accept-Encoding";
	}
	location ~ /wp-content/w3tc/min.*\.css$ {
		types {}
		default_type text/css;
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b (Minify)";
		add_header Vary "Accept-Encoding";
	}
	location ~ /wp-content/w3tc/min.*js\.gzip$ {
		gzip off;
		types {}
		default_type application/x-javascript;
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b (Minify, Gzipped)";
		add_header Vary "Accept-Encoding";
		add_header Content-Encoding gzip;
	}
	location ~ /wp-content/w3tc/min.*css\.gzip$ {
		gzip off;
		types {}
		default_type text/css;
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b (Minify, Gzipped)";
		add_header Vary "Accept-Encoding";
		add_header Content-Encoding gzip;
	}
	# END W3TC Minify cache

	# BEGIN W3TC Browser Cache - Part 2

	location ~ \.(css|js)$ {
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b";
		add_header Pragma "public";
		add_header Cache-Control "public, must-revalidate, proxy-revalidate";
		expires 7d;
	}
	location ~ \.(html|htm|rtf|rtx|svg|svgz|txt|xsd|xsl|xml)$ {
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b";
		add_header Pragma "public";
		add_header Cache-Control "public, must-revalidate, proxy-revalidate";
		expires 1d;
	}
	location ~ \.(asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|png|pot|pps|ppt|pptx|ra|ram|swf|tar|tif|tiff|wav|wma|wri|xla|xls|xlsx|xlt|xlw|zip)$ {
		add_header X-Powered-By "W3 Total Cache/0.9.1.4b";
		add_header Pragma "public";
		add_header Cache-Control "public, must-revalidate, proxy-revalidate";
		expires 30d;
	}

	# END W3TC Browser Cache - Part 2

	try_files $uri $uri/ /index.php?q=$request_uri;

	location ~ \.php$ {
		access_log off;
		proxy_pass   http://127.0.0.1:8080;
	}

	location ~ /\.ht {
		deny all;
	}
}

The following changes were made from the default configuration in order for things to work:

• Change paths from absolute to relative (with respect to document root) – the only path(s) that remain absolute are those checking whether or not a file exists.
• Expires headers added based on personal preference.
• X-Powered-By header modified to show action performed.
• Some additional Cache-Control and Pragma headers added.
• The browser cache block was split into two.
  • The reason being that if the browser cache block precedes any of the other blocks that serve gzipped files, the gzip extension is matched and the Content-type is over-ridden. The solution is to either remove the gzip extension from the browser cache list; or to move the processing of the browser cache extensions after all the gzipped files have been processed.
• Finally, the last few lines, pass any files not handled by W3 Total Cache onto WordPress, and proxy PHP scripts to apache running in the background. The last stanza denies access to .htaccess files that are still used by apache.
• The simplest proof of this setup was to turn apache off – if the page hadn’t been previously loaded (and no cached copy was present), a 502-Bad gateway error was returned; if however a cached copy existed, the page loaded without any necessity for apache or mysql, exactly as it should.

One other change that I think might prove beneficial is to prevent the minify code from redirecting root (/). That is to say, replacing:

if (-f $request_filename$w3tc_enc) {
	rewrite (.*) $1$w3tc_enc break;
}

with:

set $min_rewrite 1;
if ($request_uri ~ ^\/$){
	set $min_rewrite 0;
}

if (!-f $request_filename$w3tc_enc) {
	set $min_rewrite 0;
}

if ($min_rewrite = 1){
	rewrite (.*) $1$w3tc_enc break;
}

The reason for this suggestion is that WordPress redirects index.php back to root, which, while functional, will result in a ‘logical’ redirect loop (i.e. nginx redirects / to index.php, WordPress (apache) redirects index.php back to /). Since two separate entities do the redirects, there isn’t actually a loop, but from the perspective of the request, a redirect is issued pointing to itself (which by definition is a loop). (This behaviour was brought to my attention by the fact that my Pingdom monitor for this site kept notifying me of downtime due to an “Infinite redirection loop”).

In order for W3 Total Cache to treat the server as nginx instead of apache (in this case, apache is running PHP), I modified wp-content/plugins/w3-total-cache/inc/define.php, such that:
function w3_is_apache() always returned false (line: ~410) and
function w3_is_nginx() always returned true (line: ~420).

For interest sake, a quick run of apachebench (ab) shows processing times averaging 10ms for a cached page:

# ab -n 100 -c 10 http://www.thatsgeeky.com/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www.thatsgeeky.com (be patient).....done

Server Software:        nginx/0.8.53
Server Hostname:        www.thatsgeeky.com
Server Port:            80

Document Path:          /
Document Length:        37139 bytes

Concurrency Level:      10
Time taken for tests:   0.145 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      3784101 bytes
HTML transferred:       3739964 bytes
Requests per second:    688.83 [#/sec] (mean)
Time per request:       14.517 [ms] (mean)
Time per request:       1.452 [ms] (mean, across all concurrent requests)
Transfer rate:          25455.22 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1    2   0.9      2       5
Processing:     9   12   1.3     12      15
Waiting:        1    3   0.7      3       5
Total:         10   14   1.1     14      17

Percentage of the requests served within a certain time (ms)
  50%     14
  66%     14
  75%     15
  80%     15
  90%     15
  95%     16
  98%     17
  99%     17
 100%     17 (longest request)

There remain some issues to be resolved with the setup of W3 Total Cache (minify paths for instance), but the parts relating to nginx are (finally) working as they should.