This is duplicate of my original post: https://troubleshoot-coltpython.blogspot.com/2015/05/ubuntu-nginx-php5-fpm-mariadb-varnish.html

This is my cheat sheet for future reference in case I forget. It’s composed from various blogs around the interwebz and modified to suit my needs. I thought someone might find this useful.
I won’t go into much details about config definitions, it’s cheat sheet after all 🙂

We’ll setup Varnish caching engine with Nginx. Nginx will also serve as reverse proxy for SSL (https) requests.

My working environment:

Ubuntu 14.04.2 LTS | Trusty
Nginx 1.4.6
PHP 5.5.9
Varnish: 4.0.3-2
MariaDB: 5.5.43

Important config sections are highlighted. You need to change this.

Varnish will listen on port 80 and forward requests to backend (Nginx) listening on port 8080.

We are doing this as root:

sudo -s

1. Install Varnish

apt-get install apt-transport-https

curl https://repo.varnish-cache.org/GPG-key.txt | apt-key add -

echo "deb https://repo.varnish-cache.org/ubuntu/ trusty varnish-4.0" >> /etc/apt/sources.list.d/varnish-cache.list

apt-get update

apt-get install varnish

2. Install Nginx + Naxsi

echo "deb http://ppa.launchpad.net/nginx/stable/ubuntu $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/nginx-stable.list

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C300EE8C

apt-get update

apt-get install nginx

apt-get install nginx-naxsi

apt-get install libpcre3-dev libssl-dev

3. Install PHP5-FPM

apt-get install php5-fpm

4. Install MariaDB server

apt-get install mariadb-server

5. Configure Nginx (I also use CloudFlare in front)

Generate certificates for SSL sessions:

mkdir -p /etc/nginx/ssl

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/ssl/domain.example.key -out /etc/nginx/ssl/domain.example.crt

Edit nginx.conf:

nano /etc/nginx/nginx.conf

user www-data;
worker_processes 4;
pid /run/nginx.pid;

events {
worker_connections 768;
multi_accept on;
use epoll;
}

http {

#Basic Settings

#Cloudflare real IP
set_real_ip_from 199.27.128.0/21;
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 162.158.0.0/15;
set_real_ip_from 104.16.0.0/12;
set_real_ip_from 172.64.0.0/13;
set_real_ip_from 2400:cb00::/32;
set_real_ip_from 2606:4700::/32;
set_real_ip_from 2803:f800::/32;
set_real_ip_from 2405:b500::/32;
set_real_ip_from 2405:8100::/32;
real_ip_header CF-Connecting-IP;

#Varnish get real IP
real_ip_header X-Forwarded-For;
set_real_ip_from 127.0.0.1;

#TCP and buffer settings
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 20;
client_max_body_size 15m;
client_body_timeout 60;
client_header_timeout 60;
#client_body_buffer_size 1K;
#client_header_buffer_size 1k;
#large_client_header_buffers 4 8k;
send_timeout 60;
reset_timedout_connection on;
types_hash_max_size 2048;
server_tokens off;

include /etc/nginx/mime.types;
default_type application/octet-stream;

#Logging Settings

access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;

#Gzip Settings
gzip on;
gzip_disable “msie6”;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_min_length 512;
gzip_buffers 16 8k;
gzip_http_version 1.1;
#gzip_types text/css text/javascript text/xml text/plain text/x-component application/javascript application/x-javascript application/json application/xml application/rss+xml font/truetype application/x-font-ttf font/opentype application/vnd.ms-fontobject image/svg+xml;


#Uncomment it if you installed nginx-naxsi
include /etc/nginx/naxsi_core.rules;

#Virtual Host Configs
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}

nano /etc/nginx/sites-enabled/example.domain.conf

server {
listen 127.0.0.1:8080;
server_name example.domain.com www.example.domain.com;
root “/home/example.domain/public_html”;

index index.php;
client_max_body_size 10m;

access_log /home/example.domain/_logs/access.log;
error_log /home/example.domain/_logs/error.log;

if ($http_user_agent ~* (Baiduspider|webalta|nikto|wkito|pikto|scan|acunetix|morfeus|webcollage|youdao) ) {
return 401;
}

if ($http_user_agent ~* (HTTrack|clshttp|archiver|loader|email|harvest|extract|grab|miner) ) {
return 401;
}

location ~ /(\.|wp-config.php|readme.html|license.txt) {
return 404;
}

# Add trailing slash to */wp-admin requests.
rewrite /wp-admin$ $scheme://$host$uri/ permanent;

location / {
include /etc/nginx/naxsi.rules;
try_files $uri $uri/ /index.php$uri?$args;
}

#Needed by Naxsi
location /RequestDenied {
return 403;
}

location ~ “^(.+\.php)($|/)” {
fastcgi_split_path_info ^(.+\.php)(.*)$;

fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
fastcgi_param SERVER_NAME $host;

if ($uri !~ “^/uploads/”) {
fastcgi_pass unix:/var/run/example.domain_fpm.sock; #You can use TCP connection instead or single sock for all sites. You can define this (pools) in /etc/php5/fpm/pool.d/ 
}
include fastcgi_params;
}

location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
expires max;
log_not_found off;
access_log off;
}

location ~* \.(html|htm)$ {
expires 30m;
}

location ~* /\.(ht|git|svn) {
deny all;
}

#Add compression
gzip on;
gzip_comp_level 2;
gzip_min_length 1000;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain application/x-javascript text/xml text/css application/xml;

}

server {
listen 443 ssl;
server_name example.domain.com www.example.domain.com;
ssl_certificate /etc/nginx/ssl/example.domain.crt;
ssl_certificate_key /etc/nginx/ssl/example.domain.key;

location / {

## Pass the request on to Varnish.
location / {
proxy_pass http://127.0.0.1:80;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Port 443;
proxy_set_header Host $host;
proxy_redirect off;
                }
         }
}

6. Configure naxsi for Nginx (Optimized for WordPress):

nano /etc/nginx/naxsi.rules

# Sample rules file for default vhost.

#LearningMode;
SecRulesEnabled;
#SecRulesDisabled;
DeniedUrl “/RequestDenied”;

## check rules
CheckRule “$SQL >= 8” BLOCK;
CheckRule “$RFI >= 8” BLOCK;
CheckRule “$TRAVERSAL >= 4” BLOCK;
CheckRule “$EVADE >= 4” BLOCK;
CheckRule “$XSS >= 8” BLOCK;

# WordPress naxsi rules

### HEADERS
BasicRule wl:1000,1001,1005,1007,1010,1011,1013,1100,1200,1308,1309,1310,1311,1315 “mz:$HEADERS_VAR:cookie”;
# xmlrpc
BasicRule wl:1402 “mz:$HEADERS_VAR:content-type”;

### simple BODY (POST)
BasicRule wl:1001,1015,1009,1311,1310,1101,1016 “mz:$URL:/|$BODY_VAR:customized”;
# comments
BasicRule wl:1000,1010,1011,1013,1015,1200,1310,1311 “mz:$BODY_VAR:post_title”;
BasicRule wl:1000 “mz:$BODY_VAR:original_publish”;
BasicRule wl:1000 “mz:$BODY_VAR:save”;
BasicRule wl:1008,1010,1011,1013,1015 “mz:$BODY_VAR:sk2_my_js_payload”;
BasicRule wl:1001,1009,1005,1016,1100,1310 “mz:$BODY_VAR:url”;
BasicRule wl:1009,1100 “mz:$BODY_VAR:referredby”;
BasicRule wl:1009,1100 “mz:$BODY_VAR:_wp_original_http_referer”;
BasicRule wl:1000,1001,1005,1008,1007,1009,1010,1011,1013,1015,1016,1100,1200,1302,1303,1310,1311,1315,1400 “mz:$BODY_VAR:comment”;
BasicRule wl:1100 “mz:$BODY_VAR:redirect_to”;
BasicRule wl:1000,1009,1315 “mz:$BODY_VAR:_wp_http_referer”;
BasicRule wl:1000 “mz:$BODY_VAR:action”;
BasicRule wl:1001,1013 “mz:$BODY_VAR:blogname”;
BasicRule wl:1015,1013 “mz:$BODY_VAR:blogdescription”;
BasicRule wl:1015 “mz:$BODY_VAR:date_format_custom”;
BasicRule wl:1015 “mz:$BODY_VAR:date_format”;
BasicRule wl:1015 “mz:$BODY_VAR:tax_input%5bpost_tag%5d”;
BasicRule wl:1015 “mz:$BODY_VAR:tax_input[post_tag]”;
BasicRule wl:1100 “mz:$BODY_VAR:siteurl”;
BasicRule wl:1100 “mz:$BODY_VAR:home”;
BasicRule wl:1000,1015 “mz:$BODY_VAR:submit”;
# news content matches pretty much everything
BasicRule wl:0 “mz:$BODY_VAR:content”;
BasicRule wl:1000 “mz:$BODY_VAR:delete_option”;
BasicRule wl:1000 “mz:$BODY_VAR:prowl-msg-message”;
BasicRule wl:1100 “mz:$BODY_VAR:_url”;
BasicRule wl:1001,1009 “mz:$BODY_VAR:c2c_text_replace%5btext_to_replace%5d”;
BasicRule wl:1200 “mz:$BODY_VAR:ppn_post_note”;
BasicRule wl:1100 “mz:$BODY_VAR:author”;
BasicRule wl:1001,1015 “mz:$BODY_VAR:excerpt”;
BasicRule wl:1015 “mz:$BODY_VAR:catslist”;
BasicRule wl:1005,1008,1009,1010,1011,1015,1315 “mz:$BODY_VAR:cookie”;
BasicRule wl:1101 “mz:$BODY_VAR:googleplus”;
BasicRule wl:1007 “mz:$BODY_VAR:name”;
BasicRule wl:1007 “mz:$BODY_VAR:action”;
BasicRule wl:1100 “mz:$BODY_VAR:attachment%5burl%5d”;
BasicRule wl:1100 “mz:$BODY_VAR:attachment_url”;
BasicRule wl:1001,1009,1100,1302,1303,1310,1311 “mz:$BODY_VAR:html”;
BasicRule wl:1015 “mz:$BODY_VAR:title”;
BasicRule wl:1001,1009,1015 “mz:$BODY_VAR:recaptcha_challenge_field”;
BasicRule wl:1011 “mz:$BODY_VAR:pwd”;
BasicRule wl:1000 “mz:$BODY_VAR:excerpt”;

### BODY|NAME
BasicRule wl:1000 “mz:$BODY_VAR:delete_option|NAME”;
BasicRule wl:1000 “mz:$BODY_VAR:from|NAME”;

### Simple ARGS (GET)
# WP login screen
BasicRule wl:1100 “mz:$ARGS_VAR:redirect_to”;
BasicRule wl:1000,1009 “mz:$ARGS_VAR:_wp_http_referer”;
BasicRule wl:1000 “mz:$ARGS_VAR:wp_http_referer”;
BasicRule wl:1000 “mz:$ARGS_VAR:action”;
BasicRule wl:1000 “mz:$ARGS_VAR:action2”;
# load and load[] GET variable
BasicRule wl:1000,1015 “mz:$ARGS_VAR:load”;
BasicRule wl:1000,1015 “mz:$ARGS_VAR:load[]”;
BasicRule wl:1015 “mz:$ARGS_VAR:q”;
BasicRule wl:1000,1015 “mz:$ARGS_VAR:load%5b%5d”;

### URL
BasicRule wl:1000 “mz:URL|$URL:/wp-admin/update-core.php”;
BasicRule wl:1000 “mz:URL|$URL:/wp-admin/update.php”;
# URL|BODY
BasicRule wl:1009,1100 “mz:$URL:/wp-admin/post.php|$BODY_VAR:_wp_http_referer”;
BasicRule wl:1016 “mz:$URL:/wp-admin/post.php|$BODY_VAR:metakeyselect”;
BasicRule wl:11 “mz:$URL:/xmlrpc.php|BODY”;
BasicRule wl:11 “mz:$URL:/wp-cron.php|BODY”;
BasicRule wl:2 “mz:$URL:/wp-admin/async-upload.php|BODY”;
# URL|BODY|NAME
BasicRule wl:1100 “mz:$URL:/wp-admin/post.php|$BODY_VAR:_wp_original_http_referer|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/post.php|$BODY_VAR:metakeyselect|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/user-edit.php|$BODY_VAR:from|NAME”;
BasicRule wl:1100 “mz:$URL:/wp-admin/admin-ajax.php|$BODY_VAR:attachment%5burl%5d|NAME”;
BasicRule wl:1100 “mz:$URL:/wp-admin/post.php|$BODY_VAR:attachment_url|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/plugins.php|$BODY_VAR:verify-delete|NAME”;
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/post.php|$BODY_VAR:post_category[]|NAME”;
BasicRule wl:1311 “mz:$URL:/wp-admin/post.php|$BODY_VAR:post_category|NAME”;
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/post.php|$BODY_VAR:tax_input[post_tag]|NAME”;
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/post.php|$BODY_VAR:newtag[post_tag]|NAME”;
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/users.php|$BODY_VAR:users[]|NAME”;
# URL|ARGS|NAME
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/load-scripts.php|$ARGS_VAR:load[]|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/users.php|$ARGS_VAR:delete_count|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/users.php|$ARGS_VAR:update|NAME”;

# plain WP site
BasicRule wl:1000 “mz:URL|$URL:/wp-admin/update-core.php”;
BasicRule wl:1000 “mz:URL|$URL:/wp-admin/update.php”;
# URL|BODY
BasicRule wl:1009,1100 “mz:$URL:/wp-admin/post.php|$BODY_VAR:_wp_http_referer”;
BasicRule wl:1016 “mz:$URL:/wp-admin/post.php|$BODY_VAR:metakeyselect”;
BasicRule wl:11 “mz:$URL:/xmlrpc.php|BODY”;
BasicRule wl:11 “mz:$URL:/wp-cron.php|BODY”;
# URL|BODY|NAME
BasicRule wl:1100 “mz:$URL:/wp-admin/post.php|$BODY_VAR:_wp_original_http_referer|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/post.php|$BODY_VAR:metakeyselect|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/user-edit.php|$BODY_VAR:from|NAME”;
BasicRule wl:1100 “mz:$URL:/wp-admin/admin-ajax.php|$BODY_VAR:attachment%5burl%5d|NAME”;
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/admin-ajax.php|$BODY_VAR:data[wp-auth-check]|NAME”;
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/admin-ajax.php|$BODY_VAR:data[wp-check-locked-posts][]|NAME”;
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/update-core.php|$BODY_VAR:checked[]|NAME”;
# URL|ARGS|NAME
BasicRule wl:1310,1311 “mz:$URL:/wp-admin/load-scripts.php|$ARGS_VAR:load[]|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/users.php|$ARGS_VAR:delete_count|NAME”;
BasicRule wl:1000 “mz:$URL:/wp-admin/users.php|$ARGS_VAR:update|NAME”;

### Plugins
#WP Minify
BasicRule wl:1015 “mz:$URL:/wp-content/plugins/bwp-minify/min/|$ARGS_VAR:f”;

7. Configure Varnish:

nano /etc/varnish/default

# Configuration file for varnish
#
# /etc/init.d/varnish expects the variables $DAEMON_OPTS, $NFILES and $MEMLOCK
# to be set from this shell script fragment.
#
# Note: If systemd is installed, this file is obsolete and ignored. You will
# need to copy /lib/systemd/system/varnish.service to /etc/systemd/system/ and
# edit that file.

# Should we start varnishd at boot? Set to “no” to disable.
START=yes

# Maximum number of open files (for ulimit -n)
NFILES=131072

# Maximum locked memory size (for ulimit -l)
# Used for locking the shared memory log in memory. If you increase log size,
# you need to increase this number as well
MEMLOCK=82000

# Default varnish instance name is the local nodename. Can be overridden with
# the -n switch, to have more instances on a single server.
# INSTANCE=$(uname -n)

# This file contains 4 alternatives, please use only one.

## Alternative 1, Minimal configuration, no VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# content server on localhost:8080. Use a 1GB fixed-size cache file.
#
# DAEMON_OPTS=”-a :6081 \
# -T localhost:6082 \
# -b localhost:8080 \
# -u varnish -g varnish \
# -S /etc/varnish/secret \
# -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G”


## Alternative 2, Configuration with VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# one content server selected by the vcl file, based on the request.
# Use a 256MB memory based cache.
#
DAEMON_OPTS=”-a :80 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,256m”


## Alternative 3, Advanced configuration
#
# See varnishd(1) for more information.
#
# # Main configuration file. You probably want to change it 🙂
# VARNISH_VCL_CONF=/etc/varnish/default.vcl
#
# # Default address and port to bind to
# # Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# # a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
# VARNISH_LISTEN_ADDRESS=
# VARNISH_LISTEN_PORT=6081
#
# # Telnet admin interface listen address and port
# VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
# VARNISH_ADMIN_LISTEN_PORT=6082
#
# # The minimum number of worker threads to start
# VARNISH_MIN_THREADS=1
#
# # The Maximum number of worker threads to start
# VARNISH_MAX_THREADS=1000
#
# # Idle timeout for worker threads
# VARNISH_THREAD_TIMEOUT=120
#
# # Cache file location
# VARNISH_STORAGE_FILE=/var/lib/varnish/$INSTANCE/varnish_storage.bin
#
# # Cache file size: in bytes, optionally using k / M / G / T suffix,
# # or in percentage of available disk space using the % suffix.
# VARNISH_STORAGE_SIZE=1G
#
# # File containing administration secret
# VARNISH_SECRET_FILE=/etc/varnish/secret
#
# # Backend storage specification
# VARNISH_STORAGE=”file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}”
#
# # Default TTL used when the backend does not specify one
# VARNISH_TTL=120
#
# # DAEMON_OPTS is used by the init script. If you add or remove options, make
# # sure you update this section, too.
# DAEMON_OPTS=”-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
# -f ${VARNISH_VCL_CONF} \
# -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
# -t ${VARNISH_TTL} \
# -p thread_pool_min=${VARNISH_MIN_THREADS} \
# -p thread_pool_max=${VARNISH_MAX_THREADS} \
# -p thread_pool_timeout=${VARNISH_THREAD_TIMEOUT} \
# -S ${VARNISH_SECRET_FILE} \
# -s ${VARNISH_STORAGE}”
#


## Alternative 4, Do It Yourself
#
# DAEMON_OPTS=””

nano /etc/varnish/default.vcl

# Configuration file for varnish (optimized for WordPress)

vcl 4.0;

##########BACKEND##########
backend server1 {
.host = “127.0.0.1”;
.port = “8080”;
.connect_timeout = 600s;
.first_byte_timeout = 600s;
.between_bytes_timeout = 600s;
.max_connections = 800;
}

#Only allow purging from specific IPs
acl purge {
“localhost”;
“127.0.0.1”;
}

# This function is used when a request is send by a HTTP client (Browser)
##########START SUB VCL_RECV##########
sub vcl_recv {
#Normalize the header, remove the port (in case you’re testing this on various TCP ports)
set req.http.Host = regsub(req.http.Host, “:[0-9]+”, “”);

#Which pages to cache
if(!(req.http.host ~ “domain1.com“) &&
!(req.http.host ~ “domain2.com“) &&
!(req.http.host ~ “domain3.com“)) {
return (pass);
}

#FORCE SSL
if ( (req.http.host ~ “^(?i)domain1.com” || req.http.host ~ “^(?i)www.domain1.com“) && req.http.X-Forwarded-Proto !~ “(?i)https”) {
return (synth(751, “”));
}

if ( (req.http.host ~ “^(?i)domain2.com” || req.http.host ~ “^(?i)www.domain2.com“) && req.http.X-Forwarded-Proto !~ “(?i)https”) {
return (synth(750, “”));
}

if ( (req.http.host ~ “^(?i)domain3.com” || req.http.host ~ “^(?i)www.domain3.com“) && req.http.X-Forwarded-Proto !~ “(?i)https”) {
return (synth(752, “”));
}


#Set client IP to headers
if (req.restarts == 0) {
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For + “, ” + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}


#Allow purging from ACL
if (req.method == “PURGE”) {
#If not allowed then a error 405 is returned
if (!client.ip ~ purge) {
return(synth(405, “This IP is not allowed to send PURGE requests.”));
}
#If allowed, do a cache_lookup -> vlc_hit() or vlc_miss()
return (purge);
}

#Post requests will not be cached
if (req.http.Authorization || req.method == “POST”) {
return (pass);
}

#———-Wordpress specific configuration———-“

#Do not cache the RSS feed
if (req.url ~ “/feed”) {
return (pass);
}

#Blitz hack
if (req.url ~ “/mu-.*”) {
return (pass);
}


#Do not cache the admin and login pages
if (req.url ~ “/wp-(login|admin)”) {
return (pass);
}

#Do not cache the WooCommerce pages
### REMOVE IT IF YOU DO NOT USE WOOCOMMERCE ###
if (req.url ~ “/(cart|my-account|checkout|addons|/?add-to-cart=)”) {
return (pass);
}

#Remove the “has_js” cookie
set req.http.Cookie = regsuball(req.http.Cookie, “has_js=[^;]+(; )?”, “”);

#Remove any Google Analytics based cookies
set req.http.Cookie = regsuball(req.http.Cookie, “__utm.=[^;]+(; )?”, “”);

#Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, “__qc.=[^;]+(; )?”, “”);

#Remove the wp-settings-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, “wp-settings-1=[^;]+(; )?”, “”);

#Remove the wp-settings-time-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, “wp-settings-time-1=[^;]+(; )?”, “”);

#Remove the wp test cookie
set req.http.Cookie = regsuball(req.http.Cookie, “wordpress_test_cookie=[^;]+(; )?”, “”);

#Are there cookies left with only spaces or that are empty?
if (req.http.cookie ~ “^ *$”) {
unset req.http.cookie;
}

#Cache the following files extensions
if (req.url ~ “\.(css|js|png|gif|jp(e)?g|swf|ico)”) {
unset req.http.cookie;
}

#Normalize Accept-Encoding header and compression
if (req.http.Accept-Encoding) {
#Do not compress compressed files …
if (req.url ~ “\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$”) {
unset req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ “gzip”) {
set req.http.Accept-Encoding = “gzip”;
} elsif (req.http.Accept-Encoding ~ “deflate”) {
set req.http.Accept-Encoding = “deflate”;
} else {
unset req.http.Accept-Encoding;
}
}

#Check the cookies for WordPress-specific items
if (req.http.Cookie ~ “wordpress_” || req.http.Cookie ~ “comment_”) {
return (pass);
}
if (!req.http.cookie) {
unset req.http.cookie;
}

#———-End of WordPress specific configuration———-#

#Do not cache HTTP authentication and HTTP Cookie
if (req.http.Authorization || req.http.Cookie) {
# Not cacheable by default
return (pass);
}

#Cache all others requests
return (hash);
}
#########END SUB_VCL_RECV##########

#PART OF FORCE SSL
sub vcl_synth {
if (resp.status == 751) {
set resp.status = 301;
set resp.http.Location = “https://domain1.com” + req.url;
return(deliver);
}

if (resp.status == 750) {
set resp.status = 301;
set resp.http.Location = “https://domain2.com” + req.url;
return(deliver);
}

if (resp.status == 752) {
set resp.status = 301;
set resp.http.Location = “https://domain3.com“ + req.url;
return(deliver);
}


}


sub vcl_pipe {
return (pipe);
}

sub vcl_pass {
return (fetch);
}

#The data on which the hashing will take place
sub vcl_hash {
hash_data(req.url);
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}

#Include the X-Forward-Proto header, since we want to treat HTTPS
# requests differently, and make sure this header is always passed
# properly to the backend server.
if (req.http.X-Forwarded-Proto) {
hash_data(req.http.X-Forwarded-Proto);
}

#If the client supports compression, keep that in a different cache
if (req.http.Accept-Encoding) {
hash_data(req.http.Accept-Encoding);
}
return (lookup);
}

#This function is used when a request is sent by our backend (Nginx server)
##########START VCL_BACKEND_RESPONSE##########
sub vcl_backend_response {
#Remove some headers we never want to see
unset beresp.http.Server;
unset beresp.http.X-Powered-By;

#For static content strip all backend cookies
if (bereq.url ~ “\.(css|js|png|gif|jp(e?)g)|swf|ico”) {
unset beresp.http.cookie;
}

#Only allow cookies to be set if we’re in admin area
if (beresp.http.Set-Cookie && bereq.url !~ “^/wp-(login|admin)”) {
unset beresp.http.Set-Cookie;
}

#Do not cache response to posted requests or those with basic auth
if ( bereq.method == “POST” || bereq.http.Authorization ) {
set beresp.uncacheable = true;
set beresp.ttl = 120s;
return (deliver);
}

#Do not cache search results
if ( bereq.url ~ “\?s=” ){
set beresp.uncacheable = true;
set beresp.ttl = 120s;
return (deliver);
}

#Only cache status ok
if ( beresp.status != 200 ) {
set beresp.uncacheable = true;
set beresp.ttl = 120s;
return (deliver);
}

#A TTL of 24h
set beresp.ttl = 24h;
#Define the default grace period to serve cached content
set beresp.grace = 30s;

return (deliver);
}
##########END VCL_BACKEND_RESPONSE##########

##########START VCL_DELIVER##########
sub vcl_deliver {
if (obj.hits > 0) {
set resp.http.X-Cache = “cached”;
} else {
set resp.http.x-Cache = “uncached”;
}

#Remove some headers: PHP version
unset resp.http.X-Powered-By;

#Remove some headers: Apache\Nginx version & OS
unset resp.http.Server;

#Remove some headers: Varnish
unset resp.http.Via;
unset resp.http.X-Varnish;

return (deliver);
}
##########END VCL_DELIVER##########

sub vcl_init {
return (ok);
}

sub vcl_fini {
return (ok);
}

8. Set other config files:

nano /etc/php5/fpm/php.ini

[PHP]
display_errors = off
log_errors: on
max_input_time: 60
output_buffering: 4096
register_argc_argv: off
request_order: GP
session.bug_compat_42: off
session.bug_compat_warn: off
session.gc_divisor: 1000
session.hash_bits_per_character: 5
short_open_tag: off
variables_order: GPCS
engine = On
short_open_tag = Off
asp_tags = Off
precision = 14
output_buffering = 4096
zlib.output_compression = Off
implicit_flush = Off
unserialize_callback_func =
serialize_precision = 17
disable_functions=pcntl_alarm,pcntl_fork,pcntl_waitpid,pcntl_wait,pcntl_wifexited,pcntl_wifstopped,pcntl_wifsignaled,pcntl_wexitstatus,pcntl_wtermsig,pcntl_wstopsig,pcntl_signal,pcntl_signal_dispatch,pcntl_get_last_error,pcntl_strerror,pcntl_sigprocmask,
pcntl_sigwaitinfo,pcntl_sigtimedwait,pcntl_exec,pcntl_getpriority,pcntl_setpriority,system,exec,shell_exec,passthru,proc_close,proc_get_status,proc_nice,proc_open,proc_terminate,popen,pclose,phpinfo,eval
disable_classes =
ignore_user_abort = Off
zend.enable_gc = On
expose_php = Off
max_execution_time = 120
max_input_time = 60
memory_limit = 64M
error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT
display_errors = Off
display_startup_errors = Off
log_errors = On
log_errors_max_len = 1024
ignore_repeated_errors = Off
ignore_repeated_source = Off
report_memleaks = On
track_errors = Off
html_errors = On
variables_order = “GPCS”
request_order = “GP”
register_argc_argv = Off
auto_globals_jit = On
post_max_size = 15M
auto_prepend_file =
auto_append_file =
default_mimetype = “text/html”
default_charset = “UTF-8”
doc_root =
user_dir =
enable_dl = Off
cgi.fix_pathinfo=1
file_uploads = On
upload_max_filesize = 15M
max_file_uploads = 20
allow_url_fopen = Off
allow_url_include = Off
default_socket_timeout = 30
[CLI Server]
cli_server.color = On
pdo_mysql.cache_size = 2000
pdo_mysql.default_socket=
[mail function]
SMTP = localhost
smtp_port = 25
mail.add_x_header = On
[SQL]
sql.safe_mode = Off
[ODBC]
odbc.allow_persistent = On
odbc.check_persistent = On
odbc.max_persistent = -1
odbc.max_links = -1
odbc.defaultlrl = 4096
odbc.defaultbinmode = 1
[Interbase]
ibase.allow_persistent = 1
ibase.max_persistent = -1
ibase.max_links = -1
ibase.timestampformat = “%Y-%m-%d %H:%M:%S”
ibase.dateformat = “%Y-%m-%d”
ibase.timeformat = “%H:%M:%S”
[MySQL]
mysql.allow_local_infile = On
mysql.allow_persistent = Off
mysql.cache_size = 2000
mysql.max_persistent = -1
mysql.max_links = -1
mysql.default_port =
mysql.default_socket =
mysql.default_host =
mysql.default_user =
mysql.default_password =
mysql.connect_timeout = 60
mysql.trace_mode = Off
[MySQLi]
mysqli.max_persistent = -1
mysqli.allow_persistent = On
mysqli.max_links = -1
mysqli.cache_size = 2000
mysqli.default_port = 3306
mysqli.default_socket =
mysqli.default_host =
mysqli.default_user =
mysqli.default_pw =
mysqli.reconnect = Off
[mysqlnd]
mysqlnd.collect_statistics = On
mysqlnd.collect_memory_statistics = Off
[PostgreSQL]
pgsql.allow_persistent = On
pgsql.auto_reset_persistent = Off
pgsql.max_persistent = -1
pgsql.max_links = -1
pgsql.ignore_notice = 0
pgsql.log_notice = 0
[Sybase-CT]
sybct.allow_persistent = On
sybct.max_persistent = -1
sybct.max_links = -1
sybct.min_server_severity = 10
sybct.min_client_severity = 10
[bcmath]
bcmath.scale = 0
[Session]
session.save_handler = files
session.use_strict_mode = 0
session.use_cookies = 1
session.use_only_cookies = 1
session.name = PHPSESSID
session.auto_start = 0
session.cookie_lifetime = 0
session.cookie_path = /
session.cookie_domain =
session.cookie_httponly =
session.serialize_handler = php
session.gc_probability = 0
session.gc_divisor = 1000
session.gc_maxlifetime = 1440
session.bug_compat_42 = Off
session.bug_compat_warn = Off
session.referer_check =
session.cache_limiter = nocache
session.cache_expire = 180
session.use_trans_sid = 0
session.hash_function = 0
session.hash_bits_per_character = 5
url_rewriter.tags = “a=href,area=href,frame=src,input=src,form=fakeentry”
[MSSQL]
mssql.allow_persistent = On
mssql.max_persistent = -1
mssql.max_links = -1
mssql.min_error_severity = 10
mssql.min_message_severity = 10
mssql.compatibility_mode = Off
mssql.secure_connection = Off
[Tidy]
tidy.clean_output = Off
[soap]
soap.wsdl_cache_enabled=1
soap.wsdl_cache_dir=”/tmp”
soap.wsdl_cache_ttl=86400
soap.wsdl_cache_limit = 5
[ldap]
ldap.max_links = -1

nano /etc/php5/fpm/pool.d/domain.example.pool.conf

; Use this config only if you want to run separate php5-fpm process per user
; Pool name, the variable $pool can be used in any directive and will be replaced by the
; pool name (‘www’ here)
[user]

; Per pool prefix
; It only applies on the following directives:
; – ‘slowlog’
; – ‘listen’ (unixsocket)
; – ‘chroot’
; – ‘chdir’
; – ‘php_values’
; – ‘php_admin_values’
; When not set, the global prefix (or /usr) applies instead.
; Note: This directive can also be relative to the global prefix.
; Default Value: none
;prefix = /path/to/pools/$pool

; The address on which to accept FastCGI requests.
; Valid syntaxes are:
; ‘ip.add.re.ss:port’ – to listen on a TCP socket to a specific address on
; a specific port;
; ‘port’ – to listen on a TCP socket to all addresses on a
; specific port;
; ‘/path/to/unix/socket’ – to listen on a unix socket.
; Note: This value is mandatory.
listen = /var/run/example.domain_fpm.sock

; Set listen(2) backlog. A value of ‘-1’ means unlimited.
; Default Value: 128 (-1 on FreeBSD and OpenBSD)
;listen.backlog = -1

; List of ipv4 addresses of FastCGI clients which are allowed to connect.
; Equivalent to the FCGI_WEB_SERVER_ADDRS environment variable in the original
; PHP FCGI (5.2.2+). Makes sense only with a tcp listening socket. Each address
; must be separated by a comma. If this value is left blank, connections will be
; accepted from any ip address.
; Default Value: any
;listen.allowed_clients = 127.0.0.1

; Set permissions for unix socket, if one is used. In Linux, read/write
; permissions must be set in order to allow connections from a web server. Many
; BSD-derived systems allow connections regardless of permissions.
; Default Values: user and group are set as the running user
; mode is set to 0666
listen.owner = user
listen.group = user
listen.mode = 0660

; Unix user/group of processes
; Note: The user is mandatory. If the group is not set, the default user’s group
; will be used.
user = user
group = user

; Choose how the process manager will control the number of child processes.
; Possible Values:
; static – a fixed number (pm.max_children) of child processes;
; dynamic – the number of child processes are set dynamically based on the
; following directives:
; pm.max_children – the maximum number of children that can
; be alive at the same time.
; pm.start_servers – the number of children created on startup.
; pm.min_spare_servers – the minimum number of children in ‘idle’
; state (waiting to process). If the number
; of ‘idle’ processes is less than this
; number then some children will be created.
; pm.max_spare_servers – the maximum number of children in ‘idle’
; state (waiting to process). If the number
; of ‘idle’ processes is greater than this
; number then some children will be killed.
; Note: This value is mandatory.
pm = dynamic

; The number of child processes to be created when pm is set to ‘static’ and the
; maximum number of child processes to be created when pm is set to ‘dynamic’.
; This value sets the limit on the number of simultaneous requests that will be
; served. Equivalent to the ApacheMaxClients directive with mpm_prefork.
; Equivalent to the PHP_FCGI_CHILDREN environment variable in the original PHP
; CGI.
; Note: Used when pm is set to either ‘static’ or ‘dynamic’
; Note: This value is mandatory.
pm.max_children = 1

; The number of child processes created on startup.
; Note: Used only when pm is set to ‘dynamic’
; Default Value: min_spare_servers + (max_spare_servers – min_spare_servers) / 2
pm.start_servers = 1

; The desired minimum number of idle server processes.
; Note: Used only when pm is set to ‘dynamic’
; Note: Mandatory when pm is set to ‘dynamic’
pm.min_spare_servers = 1

; The desired maximum number of idle server processes.
; Note: Used only when pm is set to ‘dynamic’
; Note: Mandatory when pm is set to ‘dynamic’
pm.max_spare_servers = 1

; The number of requests each child process should execute before respawning.
; This can be useful to work around memory leaks in 3rd party libraries. For
; endless request processing specify ‘0’. Equivalent to PHP_FCGI_MAX_REQUESTS.
; Default Value: 0
pm.max_requests = 500

; The URI to view the FPM status page. If this value is not set, no URI will be
; recognized as a status page. By default, the status page shows the following
; information:
; accepted conn – the number of request accepted by the pool;
; pool – the name of the pool;
; process manager – static or dynamic;
; idle processes – the number of idle processes;
; active processes – the number of active processes;
; total processes – the number of idle + active processes.
; max children reached – number of times, the process limit has been reached,
; when pm tries to start more children (works only for
; pm ‘dynamic’)
; The values of ‘idle processes’, ‘active processes’ and ‘total processes’ are
; updated each second. The value of ‘accepted conn’ is updated in real time.
; Example output:
; accepted conn: 12073
; pool: www
; process manager: static
; idle processes: 35
; active processes: 65
; total processes: 100
; max children reached: 1
; By default the status page output is formatted as text/plain. Passing either
; ‘html’, ‘xml’ or ‘json’ as a query string will return the corresponding output
; syntax. Example:
; http://www.foo.bar/status
; http://www.foo.bar/status?json
; http://www.foo.bar/status?html
; http://www.foo.bar/status?xml
; Note: The value must start with a leading slash (/). The value can be
; anything, but it may not be a good idea to use the .php extension or it
; may conflict with a real PHP file.
; Default Value: not set
;pm.status_path = /status

; The ping URI to call the monitoring page of FPM. If this value is not set, no
; URI will be recognized as a ping page. This could be used to test from outside
; that FPM is alive and responding, or to
; – create a graph of FPM availability (rrd or such);
; – remove a server from a group if it is not responding (load balancing);
; – trigger alerts for the operating team (24/7).
; Note: The value must start with a leading slash (/). The value can be
; anything, but it may not be a good idea to use the .php extension or it
; may conflict with a real PHP file.
; Default Value: not set
;ping.path = /ping

; This directive may be used to customize the response of a ping request. The
; response is formatted as text/plain with a 200 response code.
; Default Value: pong
;ping.response = pong

; The access log file
; Default: not set
;access.log = log/$pool.access.log

; The access log format.
; The following syntax is allowed
; %%: the ‘%’ character
; %C: %CPU used by the request
; it can accept the following format:
; – %{user}C for user CPU only
; – %{system}C for system CPU only
; – %{total}C for user + system CPU (default)
; %d: time taken to serve the request
; it can accept the following format:
; – %{seconds}d (default)
; – %{miliseconds}d
; – %{mili}d
; – %{microseconds}d
; – %{micro}d
; %e: an environment variable (same as $_ENV or $_SERVER)
; it must be associated with embraces to specify the name of the env
; variable. Some exemples:
; – server specifics like: %{REQUEST_METHOD}e or %{SERVER_PROTOCOL}e
; – HTTP headers like: %{HTTP_HOST}e or %{HTTP_USER_AGENT}e
; %f: script filename
; %l: content-length of the request (for POST request only)
; %m: request method
; %M: peak of memory allocated by PHP
; it can accept the following format:
; – %{bytes}M (default)
; – %{kilobytes}M
; – %{kilo}M
; – %{megabytes}M
; – %{mega}M
; %n: pool name
; %o: ouput header
; it must be associated with embraces to specify the name of the header:
; – %{Content-Type}o
; – %{X-Powered-By}o
; – %{Transfert-Encoding}o
; – ….
; %p: PID of the child that serviced the request
; %P: PID of the parent of the child that serviced the request
; %q: the query string
; %Q: the ‘?’ character if query string exists
; %r: the request URI (without the query string, see %q and %Q)
; %R: remote IP address
; %s: status (response code)
; %t: server time the request was received
; it can accept a strftime(3) format:
; %d/%b/%Y:%H:%M:%S %z (default)
; %T: time the log has been written (the request has finished)
; it can accept a strftime(3) format:
; %d/%b/%Y:%H:%M:%S %z (default)
; %u: remote user
;
; Default: “%R – %u %t \”%m %r\” %s”
;access.format = %R – %u %t “%m %r%Q%q” %s %f %{mili}d %{kilo}M %C%%

; The timeout for serving a single request after which the worker process will
; be killed. This option should be used when the ‘max_execution_time’ ini option
; does not stop script execution for some reason. A value of ‘0’ means ‘off’.
; Available units: s(econds)(default), m(inutes), h(ours), or d(ays)
; Default Value: 0
request_terminate_timeout = 30s

; The timeout for serving a single request after which a PHP backtrace will be
; dumped to the ‘slowlog’ file. A value of ‘0s’ means ‘off’.
; Available units: s(econds)(default), m(inutes), h(ours), or d(ays)
; Default Value: 0
;request_slowlog_timeout = 0

; The log file for slow requests
; Default Value: not set
; Note: slowlog is mandatory if request_slowlog_timeout is set
;slowlog = log/$pool.log.slow

; Set open file descriptor rlimit.
; Default Value: system defined value
;rlimit_files = 1024

; Set max core size rlimit.
; Possible Values: ‘unlimited’ or an integer greater or equal to 0
; Default Value: system defined value
;rlimit_core = 0

; Chroot to this directory at the start. This value must be defined as an
; absolute path. When this value is not set, chroot is not used.
; Note: you can prefix with ‘$prefix’ to chroot to the pool prefix or one
; of its subdirectories. If the pool prefix is not set, the global prefix
; will be used instead.
; Note: chrooting is a great security feature and should be used whenever
; possible. However, all PHP paths will be relative to the chroot
; (error_log, sessions.save_path, …).
; Default Value: not set
;chroot =

; Chdir to this directory at the start.
; Note: relative path can be used.
; Default Value: current directory or / when chroot
chdir = /

; Redirect worker stdout and stderr into main error log. If not set, stdout and
; stderr will be redirected to /dev/null according to FastCGI specs.
; Note: on highloaded environement, this can cause some delay in the page
; process time (several ms).
; Default Value: no
;catch_workers_output = yes

; Pass environment variables like LD_LIBRARY_PATH. All $VARIABLEs are taken from
; the current environment.
; Default Value: clean env
;env[HOSTNAME] = $HOSTNAME
;env[PATH] = /usr/local/bin:/usr/bin:/bin
;env[TMP] = /tmp
;env[TMPDIR] = /tmp
;env[TEMP] = /tmp

; Additional php.ini defines, specific to this pool of workers. These settings
; overwrite the values previously defined in the php.ini. The directives are the
; same as the PHP SAPI:
; php_value/php_flag – you can set classic ini defines which can
; be overwritten from PHP call ‘ini_set’.
; php_admin_value/php_admin_flag – these directives won’t be overwritten by
; PHP call ‘ini_set’
; For php_*flag, valid values are on, off, 1, 0, true, false, yes or no.

; Defining ‘extension’ will load the corresponding shared extension from
; extension_dir. Defining ‘disable_functions’ or ‘disable_classes’ will not
; overwrite previously defined php.ini values, but will append the new value
; instead.

; Note: path INI options can be relative and will be expanded with the prefix
; (pool, global or /usr)

; Default Value: nothing is defined by default except the values in php.ini and
; specified at startup with the -d argument
;php_admin_value[sendmail_path] = /usr/sbin/sendmail -t -i -f www@my.domain.com
;php_flag[display_errors] = off
;php_admin_value[error_log] = /var/log/fpm-php.www.log
;php_admin_flag[log_errors] = on
;php_admin_value[memory_limit] = 32M
php_admin_value[session.save_path] = “/home/user/_sessions”
php_admin_value[open_basedir] = “/home/user:/usr/share/pear:/usr/share/php:/tmp:/usr/local/lib/php”

php_flag[display_errors] = off
php_admin_value[error_reporting] = 0
php_admin_value[error_log] = /var/log/php5-fpm.log
php_admin_flag[log_errors] = on
php_admin_value[memory_limit] = 128M

9. Add SSL to WordPress:

Go to your WordPress web folder, open file wp-config.php and add:

define(‘FORCE_SSL_ADMIN’, true);

define(‘FORCE_SSL_LOGIN’, true);

if ($_SERVER[‘HTTP_X_FORWARDED_PROTO’] == ‘https’)

$_SERVER[‘HTTPS’]=’on’;

Important! Add this before the line “require_once(ABSPATH . ‘wp-settings.php’);”, otherwise you’ll get “You do not have sufficient permissions to access this page” message when trying to login into admin (wp-admin).

10. Restart services

service nginx restart

service php5-fpm restart

service varnish restart

Duplicate of my original post: https://troubleshoot-coltpython.blogspot.com/2013/11/bash-script-domain-expiration-check.html

This script checks if domain has expired and also supports e-mail notice. 

Original script is from Matt (domain-check). I modifed already modified script from Vivek (http://www.cyberciti.biz/tips/domain-check-script.html).

List of changes:

– Added support for Slovenian (.si) domains.
– Added VAR for “mail from:”.
– Increased time between whois queries to 7 seconds (Slovenian whois only allows 10 queries per minute).
– Mail subject and mail data are translated to Slovene language.
– Changed default VAR path for mail (/bin/mail/ to /usr/bin/mail)

Get the script:

wget http://krejzi.si/files/domain-check;mv domain-check domain-check.sh;chmod +x domain-check.sh

Mirror: https://pastebin.com/pQe3RmDz

Usage:

Usage: ./domain-check.sh [ -e email ] [ -x expir_days ] [ -q ] [ -a ] [ -h ] {[ -d domain_namee ]} || { -f domainfile}

  -a                                   : Send a warning message through email
  -d domain                            : Domain to analyze (interactive mode)
  -e email address                     : Email address to send expiration notices
  -f domain file                       : File with a list of domains
  -h                                   : Print this screen
  -s whois server                      : Whois sever to query for information
  -q                                   : Don't print anything on the console
  -x days                              : Domain expiration interval (eg. if domain_date < days)

Check single domain:

./domain-check.sh -d domain.si

Check single domain and send e-mail notice if domain will expire in less than defined in WARNDAYS:

./domain-check.sh -a -d domain.si

You can also check multiple domains. Create a file domains.txt and add your domains like:

domain1.com
domain2.com
domain3.com

Check multiple domains:

./domain-check.sh -f domains.txt

Check multiple domains and send e-mail notice if any domain will expire in less than defined in WARNDAYS:

./domain-check.sh -a -f domains.txt

Set your own parameters from command line. Next command will check domains in domains.txt with expiration date less than 60 days and send e-mail notice to admin@domain.si

./domain-check.sh -a -f domains.txt -e admin@domain.si -x 60 

If you are going to use this script with cron, use -q option or change QUIET=”FALSE” to QUIET=”TRUE”

Let’s say we want to run this script every day at 4 AM. Open crontab with crontab -e (Ubuntu) and add the following line:

0 4 * * * /path/to/your/script/domain-check.sh -q -a -f /path/to/your/list/domains.txt

Duplicate of original post: https://troubleshoot-coltpython.blogspot.com/2013/11/ubuntu-fail2ban-fails-to-parse-apache.html

I was trying to setup Fail2ban to block WordPress login bruteforce attacks, but Fail2ban somehow failed to parse access.log

When parsing log file with command:

fail2ban-regex /var/log/apache2/access.log /etc/fail2ban/filter.d/apache-wp-login.conf

CPU rises to 100% usage until I kill the process. I made a quick fix – workaround with redirecting needed content from access.log to another log file.

My setup is as follows:

OS: Ubuntu Server 12.04 LTS
Fail2ban: 0.8.6-3wheezy2build0.12.04.1
Python: 2.7.3-0ubuntu2.2

Fail2ban configuration:

 nano /etc/fail2ban/fail2ban.conf 

# Fail2Ban configuration file
# Author: Cyril Jaquier
# $Revision$


[Definition]

# Option: loglevel
# Notes.: Set the log level output.
# 1 = ERROR
# 2 = WARN
# 3 = INFO
# 4 = DEBUG
# Values: NUM Default: 3
#

loglevel = 4

# Option: logtarget
# Notes.: Set the log target. This could be a file, SYSLOG, STDERR or STDOUT.
# Only one log target can be specified.
# Values: STDOUT STDERR SYSLOG file Default: /var/log/fail2ban.log
#

logtarget = /var/log/fail2ban.log

# Option: socket
# Notes.: Set the socket file. This is used to communicate with the daemon. Do
# not remove this file when Fail2ban runs. It will not be possible to
# communicate with the server afterwards.
# Values: FILE Default: /var/run/fail2ban/fail2ban.sock
#
socket = /var/run/fail2ban/fail2ban.sock

We add a new entry for WordPress

Add configuration to /etc/fail2ban/jail.conf

[apache-wp-login]

enabled = true
port = http,https
filter = apache-wp-login
logpath = /var/log/apache2/apache-wp-login.log
maxretry = 3
findtime = 60

Now me make new filter for WordPress. Some servers logs are in different format so we have to make different regex entries. These are two most common configurations:

Log format example 1:

www.domain.si:80 188.65.115.90 - - [08/Nov/2013:13:20:46 +0100] "POST /en/wp-login.php HTTP/1.1" 200 1784 "http://www.domain.si/en/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1623.0 Safari/537.36"

Log format example 2:

89.222.15.152 - - [08/Nov/2013:13:40:12 +0000] "POST /wp-login.php HTTP/1.1" 200 1756 "http://domain.wordpress.com/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0"

Create /etc/fail2ban/filter.d/apache-wp-login.conf

For log format example 1 use:

[Definition]
failregex = ^[^\:]+\:80 <HOST> .* "POST
ignoreregex =

ATTENTION! Don’t apply this filter to default access.log, it would block all POST request, not only WordPress. Only use it with modified log file we will create bellow.

For log format example 2 use:

failregex = <HOST>.*] "POST /wp-login.php

ATTENTION! Don’t apply this filter to log format example 1, it would block your own server, because <HOST> regex would match your domain (www.domain.si).

Now we create new log file for parsing (/var/log/apache2/apache-wp-login.log). We only want to filter out POST requests for wp-login.php and write them to new log file called apache-wp-login.log

tail --follow=name /var/log/apache2/access.log | grep --line-buffered wp-login.php > /var/log/apache2/apache-wp-login.log &

Add this command to /etc/rc.local so it would run at reboot:

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

# Fail2ban WordPress Login
tail -f /var/log/apache2/access.log | grep --line-buffered wp-login.php > /var/log/apache2/apache-wp-login.log &

exit 0

/etc/init.d/fail2ban restart