Tag Archives: wordpress

Rewrite HTTP host request with Varnish

I had a stubborn WordPress plugin that somehow was remembering the URL of the site it was installed on. It became a problem when I changed the site URL. Despite changing the URL everywhere I could think of, this particular plugin was calling CSS files for the URL of the original site. I did a search and replace in the site database and searched all files for any reference to that site but couldn’t find anything. I never did find the culprit. My workaround was to use varnish to rewrite the request before it hit the browser.

Thanks to this answer by Jorge Nerin on Stack Overflow, I found my answer on how to do this.

backend www {
  .host = "www.example.com";
  .port = "http";
}

sub vcl_recv {
  if (req.http.host ~ "(?i)^(www.)?example.com$") {
    set req.backend_hint = www;
  }
}

In my case I had a default backend (no other backends configured) so my varnish config was simply adding these line in sub vcs_recv (varnish 4 syntax)

if (req.http.host ~ "(?i)^(www.)?old.host.name$") {
     set req.backend_hint = default;
}

That did the trick!

Fix wordpress PHP change was reverted error

Since WordPress 4.9 I’ve had a peculiar issue when trying to edit theme files using the web GUI. Whenever I tried to save changes I would get this error message:

Unable to communicate back with site to check for fatal errors, so the PHP change was reverted. You will need to upload your PHP file change by some other means, such as by using SFTP.

After following this long thread I saw the suggestion to install and use the Health Check plugin to get more information into why this is happening. In my case I kept getting this error message:

The loopback request to your site failed, this may prevent WP_Cron from working, along with theme and plugin editors.<br>Error encountered: (0) cURL error 28: Connection timed out after 10001 milliseconds

I researched what a loopback request is in this case. It’s the webserver reaching out to its own site’s url to talk to itself. My webserver was being denied internet access, which included its own URL, so it couldn’t complete the loopback request.

One solution, mentioned here, is to edit the hosts file on your webserver to point to 127.0.0.1 for the URL of your site. My solution was to open up the firewall to allow my server to connect to its URL. I then ran into a different problem:

The loopback request to your site failed, this may prevent WP_Cron from working, along with theme and plugin editors.<br>Error encountered: (0) cURL error 60: Peer's Certificate issuer is not recognized.

After digging for a while I found this site which explains how to edit php.ini to point to an acceptable certificate list. To fix this on my Cent7 machine I edited /etc/php.ini and added this line (you could also add it to /etc/php.d/curl.ini)

curl.cainfo="/etc/pki/tls/cert.pem"

This caused php’s curl module to use the same certificate trust store that the underlying OS uses.

Then restart php-fpm if you’re using it:

sudo systemctl restart php-fpm

Success! Loopback connections now work properly.


Update 7/16/2018: I still had a wordpress site that was giving me certificate grief despite the above fix. After MUCH frustration I finally found this post where André Gayle points out that wordpress ships with its own certificate bundle, independent of even curl’s ca bundle! It’s located in your wordpress directory/wp-includes/certificates folder.

My solution to this extremely frustrating problem was to remove their bundle and symlink to my own (Cent 7 box – adjust your path to match where your wordpress install and certificate trust store is located)

sudo mv /var/www/html/wordpress/wp-includes/certificates/ca-bundle.crt /var/www/html/wordpress/wp-includes/certificates/ca-bundle.crt.old
sudo ln -s /etc/pki/tls/cert.pem /var/www/html/wordpress/wp-includes/certificates/ca-bundle.crt

FINALLY no more loopback errors in the Health Check plugin, and thus the ability to edit theme files in the editor.

Whitelist IPs with varnish

I recently needed to restrict which IP addresses can access wp-login.php for a wordpress site of mine. This site is sitting behind varnish cache for speed. Modifying htaccess doesn’t work in this case so I have to modify the varnish configuration in order to get this to work.

The varnish documentation is actually quite good at telling you what you need to do.  You have to first specify an acl and then in the vcl_recv function specify the action when these IPs are used.

I kept running into a problem where varnish wouldn’t compile. I kept receiving this error:

"Expected CSTR got 'admin_net'" (C String?)

It turns out my load balancer does not support the PROXY protocol, so client.ip is always the IP of the load balancer, not the IP of the person making the request.

The solution was finally found here where it was explained that in absence of PROXY protocol you can use the std.ip() function to convert a string containing an IP address to the value varnish expects an IP address to be, in order to check it against an ACL (see here for syntax reference.)

I then had to take it a step further to trim all the extraneous commas and quotes from X-Forwarded-For so that the std.ip() function would work:

# set realIP by trimming CloudFlare IP which will be used for various checks
set req.http.X-Actual-IP = regsub(req.http.X-Forwarded-For, "[, ].*$", "");

With these three bits combined I was able to properly restrict access to wp-login.php to a specified whitelist of IP addresses:

acl admin {
 "10.0.0.0/24";
 "10.0.1.0/24";
 "10.0.2.0/24";
}

...

sub vcl_recv {
...

 # set realIP by trimming CloudFlare IP which will be used for various checks
 set req.http.X-Actual-IP = regsub(req.http.X-Forwarded-For, "[, ].*$", "");

 #Deny wp-login.php access if not in admin ACL
 if ((std.ip(req.http.X-Actual-IP, "0.0.0.0") !~ admin) && req.url ~ "^/wp-login.php") {
  return(synth(403, "Access denied."));
 }
...
}

Success.

Fix WordPress “Sorry, you are not allowed to access this page.”

I recently came across an issue with my WordPress installation. It’s situated behind a load balancer where SSL is terminated. The load balancer takes HTTPS traffic, then forwards it as HTTP on port 80 to the wordpress server.

I was running issues with a redirect loop after installing wordpress. The solution was to add this bit of code to wp-config.php:

define('FORCE_SSL_ADMIN', true);
if ($_SERVER['HTTP_X_FORWARDED_PROTO'] == 'https')
 $_SERVER['HTTPS']='on';

This solves the redirect loop issue but then I ran into a different problem. When I tried to sign into wp-admin I would get this message:

Sorry, you are not allowed to access this page.

After much digging I found this post which emphasizes that you must place that code BEFORE anything else in wp-config.php (except for the beginning PHP tag.) Success!

Install wordpress on CentOS7 with nginx & caching

Lately I’ve become interested in the LEMP stack (as opposed to the LAMP stack.) As such I’ve decided to set up a wordpress site running CentOS 7, nginx, mariadb, php-fpm, zend-opcache, apc, and varnish. This writeup will borrow heavily from two of my other writeups, Install Wordpress on CentOS 7 with  SELinux and Speed up WordPress in CentOS7 using caching. This will be a mashup of those two with an nginx twist with guidance from digitalocean. Let’s begin.

Repositories

To install the required addons we will need to have the epel repository enabled:

yum -y install epel-release

nginx

Install necessary packages:

sudo yum -y install nginx
sudo systemctl enable nginx

Optional: symlink /usr/share/nginx/ to /var/www/ (for those of us who are used to apache)

sudo ln -s /usr/share/nginx/ /var/www

Open necessary firewall ports:

sudo firewall-cmd --add-service=http --permanent
sudo systemctl restart firewalld

start nginx:

sudo systemctl start nginx

Navigate to your new site to make sure it brings up the default page properly.

MariaDB

Install:

sudo yum -y install mariadb-server mariadb
sudo systemctl enable mariadb

Run initial mysql configuration to set database root password

sudo systemctl start mariadb
sudo mysql_secure_installation

Configure:

Create a wordpress database and user:

mysql -u root -p 
#enter your mysql root password here
create user wordpress;
create database wordpress;
GRANT ALL PRIVILEGES ON wordpress.* To 'wordpress'@'localhost' IDENTIFIED BY 'password';
quit;

php-fpm

Install:

sudo yum -y install php-fpm php-mysql php-pclzip
sudo systemctl enable php-fpm

Configure:

Uncomment cgi.fix_pathinfo and change value to 0:

sudo sed -i 's/\;\(cgi.fix_pathinfo=\)1/\10/g' /etc/php.ini

Modify the listen= parameter to listen to UNIX socket instead of TCP:

sudo sed -i 's/\(listen =\).*/\1 \/var\/run\/php-fpm\/php-fpm.sock/g' /etc/php-fpm.d/www.conf

Change listen.owner and listen.group to nobody:

sudo sed -i 's/\(listen.owner = \).*/\1nobody/g; s/\(listen.group = \).*/\1nobody/g' /etc/php-fpm.d/www.conf

Change running user & group from apache to nginx:

sudo sed -i 's/\(^user = \).*/\1nginx/g; s/\(^group = \).*/\1nginx/g' /etc/php-fpm.d/www.conf

Start php-fpm:

sudo systemctl start php-fpm

Caching

To speed up wordpress further we need to install a few bits of caching software. Accept defaults when prompted.

zend-opcache & apc:

Install necessary packages:

sudo yum -y install php-pecl-zendopcache php-pecl-apcu php-devel gcc
sudo pecl install apc

Add apc extension to php configuration:

sudo sh -c "echo '\

#Add apc extension 
extension=apc.so' >> /etc/php.ini"

Restart php-fpm:

sudo systemctl restart php-fpm

Varnish

Install:

sudo yum -y install varnish
sudo systemctl enable varnish

Configure nginx to listen on port 8080 instead of port 80:

sudo sed -i /etc/nginx/nginx.conf -e 's/listen.*80/&80 /'

Change varnish to listen on port 80 instead of port 6081:

sudo sed -i /etc/varnish/varnish.params -e 's/\(VARNISH_LISTEN_PORT=\).*/\180/g'

Optional: change varnish to cache files in memory with a limit of 256M (caching to memory is much faster than caching to disk)

sudo sed -i /etc/varnish/varnish.params -e 's/\(VARNISH_STORAGE=\).*/\1\"malloc,256M\"/g'

Add varnish configuration to work with caching wordpress sites:

Update 2/25/2018 added section to allow facebook to properly scrape varnish cached sites.

sudo sh -c 'echo "/* SET THE HOST AND PORT OF WORDPRESS
 * *********************************************************/
vcl 4.0;
import std;

backend default {
 .host = \"127.0.0.1\";
 .port = \"8080\";
 .first_byte_timeout = 60s;
 .connect_timeout = 300s;
}
 
# SET THE ALLOWED IP OF PURGE REQUESTS
# ##########################################################
acl purge {
 \"localhost\";
 \"127.0.0.1\";
}

#THE RECV FUNCTION
# ##########################################################
sub vcl_recv {

#Facebook workaround to allow proper shares
if (req.http.user-agent ~ \"facebookexternalhit\")
        {
        return(pipe);
        }

# set realIP by trimming CloudFlare IP which will be used for various checks
set req.http.X-Actual-IP = regsub(req.http.X-Forwarded-For, \"[, ].*$\", \"\"); 

 # FORWARD THE IP OF THE REQUEST
 if (req.restarts == 0) {
 if (req.http.x-forwarded-for) {
 set req.http.X-Forwarded-For =
 req.http.X-Forwarded-For + \", \" + client.ip;
 } else {
 set req.http.X-Forwarded-For = client.ip;
 }
 }

 # Purge request check sections for hash_always_miss, purge and ban
 # BLOCK IF NOT IP is not in purge acl
 # ##########################################################

 # Enable smart refreshing using hash_always_miss
if (req.http.Cache-Control ~ \"no-cache\") {
 if (client.ip ~ purge) {
 set req.hash_always_miss = true;
 }
}

if (req.method == \"PURGE\") {
 if (!client.ip ~ purge) {
 return(synth(405,\"Not allowed.\"));
 }

 ban (\"req.url ~ \"+req.url);
 return(synth(200,\"Purged.\"));

}

if (req.method == \"BAN\") {
 # Same ACL check as above:
 if (!client.ip ~ purge) {
 return(synth(403, \"Not allowed.\"));
 }
 ban(\"req.http.host == \" + req.http.host +
 \" && req.url == \" + req.url);

 # Throw a synthetic page so the
 # request wont go to the backend.
 return(synth(200, \"Ban added\"));
}

# Unset cloudflare cookies
# Remove has_js and CloudFlare/Google Analytics __* cookies.
 set req.http.Cookie = regsuball(req.http.Cookie, \"(^|;\s*)(_[_a-z]+|has_js)=[^;]*\", \"\");
 # Remove a \";\" prefix, if present.
 set req.http.Cookie = regsub(req.http.Cookie, \"^;\s*\", \"\");

 # For Testing: If you want to test with Varnish passing (not caching) uncomment
 # return( pass );

# DO NOT CACHE RSS FEED
 if (req.url ~ \"/feed(/)?\") {
 return ( pass ); 
}

#Pass wp-cron

if (req.url ~ \"wp-cron\.php.*\") {
 return ( pass );
}

## Do not cache search results, comment these 3 lines if you do want to cache them

if (req.url ~ \"/\?s\=\") {
 return ( pass ); 
}

# CLEAN UP THE ENCODING HEADER.
 # SET TO GZIP, DEFLATE, OR REMOVE ENTIRELY. WITH VARY ACCEPT-ENCODING
 # VARNISH WILL CREATE SEPARATE CACHES FOR EACH
 # DO NOT ACCEPT-ENCODING IMAGES, ZIPPED FILES, AUDIO, ETC.
 # ##########################################################
 if (req.http.Accept-Encoding) {
 if (req.url ~ \"\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$\") {
 # No point in compressing these
 unset req.http.Accept-Encoding;
 } elsif (req.http.Accept-Encoding ~ \"gzip\") {
 set req.http.Accept-Encoding = \"gzip\";
 } elsif (req.http.Accept-Encoding ~ \"deflate\") {
 set req.http.Accept-Encoding = \"deflate\";
 } else {
 # unknown algorithm
 unset req.http.Accept-Encoding;
 }
 }

 # PIPE ALL NON-STANDARD REQUESTS
 # ##########################################################
 if (req.method != \"GET\" &&
 req.method != \"HEAD\" &&
 req.method != \"PUT\" && 
 req.method != \"POST\" &&
 req.method != \"TRACE\" &&
 req.method != \"OPTIONS\" &&
 req.method != \"DELETE\") {
 return (pipe);
 }
 
 # ONLY CACHE GET AND HEAD REQUESTS
 # ##########################################################
 if (req.method != \"GET\" && req.method != \"HEAD\") {
 return (pass);
 }
 
 # OPTIONAL: DO NOT CACHE LOGGED IN USERS (THIS OCCURS IN FETCH TOO, EITHER
 # COMMENT OR UNCOMMENT BOTH
 # ##########################################################
 if ( req.http.cookie ~ \"wordpress_logged_in|resetpass\" ) {
 return( pass );
 }

 #fix CloudFlare Mixed Content with Flexible SSL
 if (req.http.X-Forwarded-Proto) {
 return(hash);
 }

 # IF THE REQUEST IS NOT FOR A PREVIEW, WP-ADMIN OR WP-LOGIN
 # THEN UNSET THE COOKIES
 # ##########################################################
 if (!(req.url ~ \"wp-(login|admin)\") 
 && !(req.url ~ \"&preview=true\" ) 
 ){
 unset req.http.cookie;
 }

 # IF BASIC AUTH IS ON THEN DO NOT CACHE
 # ##########################################################
 if (req.http.Authorization || req.http.Cookie) {
 return (pass);
 }
 
 # IF YOU GET HERE THEN THIS REQUEST SHOULD BE CACHED
 # ##########################################################
 return (hash);
 # This is for phpmyadmin
if (req.http.Host == \"pmadomain.com\") {
return (pass);
}
}

sub vcl_hash {

if (req.http.X-Forwarded-Proto) {
 hash_data(req.http.X-Forwarded-Proto);
 }
}


# HIT FUNCTION
# ##########################################################
sub vcl_hit {
 return (deliver);
}

# MISS FUNCTION
# ##########################################################
sub vcl_miss {
 return (fetch);
}

# FETCH FUNCTION
# ##########################################################
sub vcl_backend_response {
 # I SET THE VARY TO ACCEPT-ENCODING, THIS OVERRIDES W3TC 
 # TENDANCY TO SET VARY USER-AGENT. YOU MAY OR MAY NOT WANT
 # TO DO THIS
 # ##########################################################
 set beresp.http.Vary = \"Accept-Encoding\";

 # IF NOT WP-ADMIN THEN UNSET COOKIES AND SET THE AMOUNT OF 
 # TIME THIS PAGE WILL STAY CACHED (TTL), add other locations or subdomains you do not want to cache here in case they set cookies
 # ##########################################################
 if (!(bereq.url ~ \"wp-(login|admin)\") && !bereq.http.cookie ~ \"wordpress_logged_in|resetpass\" ) {
 unset beresp.http.set-cookie;
 set beresp.ttl = 1w;
 set beresp.grace =3d;
 }

 if (beresp.ttl <= 0s ||
 beresp.http.Set-Cookie ||
 beresp.http.Vary == \"*\") {
 set beresp.ttl = 120 s;
 # set beresp.ttl = 120s;
 set beresp.uncacheable = true;
 return (deliver);
 }

 return (deliver);
}

# DELIVER FUNCTION
# ##########################################################
sub vcl_deliver {
 # IF THIS PAGE IS ALREADY CACHED THEN RETURN A HIT TEXT 
 # IN THE HEADER (GREAT FOR DEBUGGING)
 # ##########################################################
 if (obj.hits > 0) {
 set resp.http.X-Cache = \"HIT\";
 # IF THIS IS A MISS RETURN THAT IN THE HEADER
 # ##########################################################
 } else {
 set resp.http.X-Cache = \"MISS\";
 }
}" > /etc/varnish/default.vcl'

Restart varnish & nginx:

sudo systemctl restart nginx varnish

Logging

By default varnish does not log its traffic. This means that your apache log will only log things varnish does not cache. We have to configure varnish to log traffic so you don’t lose insight into who is visiting your site.

Update 2/14/2017:  I’ve discovered a better way to do this. The old way is still included below, but you really should use this other way.

New way:

CentOS ships with some systemd scripts for you. You can use them out of the box by simply issuing

systemctl start varnishncsa
systemctl enable varnishncsa

If you are behind a reverse proxy then you will want to tweak the varnishncsa output a bit to reflect x-forwarded-for header values (thanks to this github discussion for the guidance.) Accomplish this by appending a modified log output format string to /lib/systemd/system/varnishncsa.service:

sudo sed -i /lib/systemd/system/varnishncsa.service -e "s/ExecStart.*/& -F \'%%{X-Forwarded-For}i %%l %%u %%t \"%%r\" %%s %%b \"%%{Referer}i\" \"%%{User-agent}i\"\' /g"

Lastly, reload systemd configuration, enable, and start the varnishncsa service:

sudo systemctl daemon-reload
sudo systemctl enable varnishncsa
sudo systemctl start varnishncsa

 


Old way:

First, enable rc.local

sudo chmod +x /etc/rc.local
sudo systemctl enable rc-local

Next, add this entry to the rc.local file:

sudo sh -c 'echo "varnishncsa -a -w /var/log/varnish/access.log -D -P /var/run/varnishncsa.pid" >> /etc/rc.local'

If your varnish server is behind a reverse proxy (like a web application firewall) then modify the above code slightly (thanks to this site for the information on how to do so)

sudo sh -c "echo varnishncsa -a -F \'%{X-Forwarded-For}i %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\"\' -w /var/log/varnish/access.log -D -P /var/run/varnishncsa.pid >> /etc/rc.local"

Once configuration is in place, start the rc-local service

sudo systemctl start rc-local

WordPress

Download, extract, and set permissions for your wordpress installation (this assumes your wordpress site is the only site on the server)

wget https://wordpress.org/latest.zip
sudo unzip latest.zip -d /usr/share/nginx/html/
sudo chown nginx:nginx -R /usr/share/nginx/html/

Configure nginx

Follow best practice for nginx by creating a new configuration file for your wordpress site. In this example I’ve created a file wordpress.conf inside the /etc/nginx/conf.d directory.

Create the file:

sudo vim /etc/nginx/conf.d/wordpress.conf

Insert the following, making sure to modify the server_name directive:

server {
    listen 8080;
    listen [::]:8080;
    server_name wordpress;
    root /usr/share/nginx/html/wordpress;
    port_in_redirect off;


    location / {
        index index.php;
        try_files $uri $uri/ /index.php?q=$uri&$args;
    }

    location ~ \.php$ {
        fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }

    error_page 404 /404.html;     
    location = /40x.html {
     }

    error_page 500 502 503 504 /50x.html;
    location = /50x.html {
    }
}

Restart nginx

sudo systemctl restart nginx

Configure upload directory

If you want users to upload content, then you will want to assign the http_sys_rw_content_t selinux security context for the wp-uploads directory (create it if it doesn’t exist)

sudo mkdir /usr/share/nginx/html/wordpress/wp-content/uploads
sudo chown nginx:nginx /usr/share/nginx/html/wordpress/wp-content/uploads
sudo semanage fcontext -a -t httpd_sys_rw_content_t "/usr/share/nginx/html/wordpress/wp-content/uploads(/.*)?"
sudo restorecon -Rv /usr/share/nginx/html/wordpress/wp-content/uploads

Run the wizard

In order for the wizard to run properly we need to temporarily give the wordpress directory httpd_sys_rw_content_t selinux context

sudo chcon -t httpd_sys_rw_content_t /usr/share/nginx/html/wordpress/

Now navigate to your new website in a browser and follow the wizard, which will create a wp-config.php file inside the wordpress directory.

W3 Total Cache

Once your wordpress site is set up you will want to configure it to communicate with varnish. This will make sure the cache is always up to date when changes are made.

Install the W3 Total Cache wordpress plugin and configure it as follows:

opcode-database-object

Opcode cache: Opcode:Zend Opcache

Database cache: Check enable, select Opcode: Alternative PHP Cache (APC / APCu)

Object cache: Check enable, select Opcode: Alternative PHP Cache (APC / APCu)

Fragment cache: Opcode: Alternative PHP cache (APC / APCu)

proxy

Reverse Proxy: Check “Enable reverse proxy caching via varnish”
Specify 127.0.0.1 in the varnish servers box. Click save all settings.

Wrapping up

Once your site is properly set up, restore the original security context for the wordpress directory:

sudo restorecon -v /usr/share/nginx/html/wordpress/

Lastly restart nginx and varnish:

sudo systemctl restart nginx varnish

Success! Everything is working within the proper SELinux contexts and caching configuration.

Troubleshooting

403 forbidden

I received this error after setting everything up. After some digging I came across this site which explained what could be happening.

For me this meant that nginx couldn’t find an index file and was trying to default to a directory listing, which is not allowed by default. This is fixed by inserting a proper directive to find index files, in my case, index.php. Make sure you have “index index.php” in your nginx.conf inside the location / block:

    location / {
        index index.php;
    }

 Accessing wp-admin redirects you to port 8080, times out

If you find going to /wp-admin redirects you to a wrong port and times out, it’s because nginx is forwarding the portnumber. We want to turn that off (thanks to this site for the help.) Add this to your nginx.conf:

  port_in_redirect off;

Script to change WordPress URL

I wrote up a little script to run when you migrate a wordpress installation from one host to another (hostname change.)  Once this script is run you can access the site via the hostname of the server it’s running on and then change the URL to whatever you like. This comes in handy for when you want to migrate one internal host to another, then specify an external hostname once things are looking how you like them.

Change SQL_COMMAND to reflect the name of the wordpress table in the destination server. Thanks to this site for the guidance in writing the script.

#!/bin/bash

#A simple script to update the wordpress database to reflect a change in hostname
#Run this after changing the hostname / IP of a wordpress server

#Prompt for mysql root password
read -s -p "Enter mysql root password: " SQL_PASSWORD

SQL_COMMAND="mysql -u root -p$SQL_PASSWORD wordpress -e"

#Determine what the old URL was and save to variable
OLD_URL=$(mysql -u root -p$SQL_PASSWORD wordpress -e 'select option_value from wp_options where option_id = 1;' | grep http)
#Get current hostname
HOST=$(hostname)

#SQL statements to update database to new hostname
$SQL_COMMAND "UPDATE wp_options SET option_value = replace(option_value, '$OLD_URL', 'http://$HOST') WHERE option_name = 'home' OR option_name = 'siteurl';"
$SQL_COMMAND "UPDATE wp_posts SET guid = replace(guid, '$OLD_URL','http://$HOST');"
$SQL_COMMAND "UPDATE wp_posts SET post_content = replace(post_content, '$OLD_URL', 'http://$HOST');"
$SQL_COMMAND "UPDATE wp_postmeta SET meta_value = replace(meta_value,'$OLD_URL','http://$HOST');"

Speed up WordPress in CentOS7 using caching

It has recently come to my attention that WordPress has a serious design flaw: it’s trivially easy to execute a denial of service attack against it in its default state. One simply has to hold down F5 to cause the site to slow to a crawl and in many cases crash entirely.

The reason behind this is that every single page request turns into the server parsing PHP code and executing a database call. Holding down F5 initiates hundreds of page view requests, which turn into hundreds of PHP code execution threads (each taking up CPU and memory) and hundreds of database calls. The server becomes quickly overwhelmed and runs out of resources.

The solution to this problem (on CentOS 7 at least) is a combination of php-fpm, zendopcache, APC, varnish and W3 Total Cache. It’s definitely more complicated but it eliminates this problem and massively increases site load times and general responsiveness.

Repositories

To install the required addons we will need to have the epel repository enabled:

yum -y install epel-release

zend-opcache

This caches PHP opcode to greatly speed up PHP code execution. It’s included in later versions of PHP but alas CentOS 7 is stuck on PHP 5.4, which does not include such caching. You have to install it manually. Thanks to this site for the information.

sudo yum -y install php-pecl-zendopcache

apc

This is another kind of of cache – this time for database operations.

sudo yum -y install php-pecl-apcu php-devel gcc
sudo pecl install apc
#accept defaults when prompted

php-fpm

php-fpm is a different way to serve up PHP code. Instead of apache running a module to interpret php code, it will send all php requests to a separate PHP server, optimized for speed. That php server will interpret the code and return the results to your browser.

sudo yum -y install php-fpm
sudo systemctl enable php-fpm

Modify your apache config to forward all php requests to php-fpm. Be sure to modify this to match your site URL setup:

sudo sh -c "echo '
    <LocationMatch \"^/(.*\.php(/.*)?)$\"> 
        ProxyPass fcgi://127.0.0.1:9000/var/www/html/\$1 
    </LocationMatch>' >> /etc/httpd/conf/httpd.conf"

Varnish

Varnish is a reverse proxy server optimized for caching pages. It sits between your visitors and your webserver and caches whatever it can to increase responsiveness and speed. This site pointed me in the right direction for configuring Varnish  in CentOS 7.

sudo yum -y install varnish
sudo systemctl enable varnish

Change apache to listen on port 8080 instead of port 80:

sudo sed -i /etc/httpd/conf/httpd.conf -e 's/Listen 80/&80/'

Change varnish to listen on port 80 instead of port 6081:

sudo sed -i /etc/varnish/varnish.params -e 's/VARNISH_LISTEN_PORT=6081/VARNISH_LISTEN_PORT=80/g'

Now we need to configure Varnish to properly cache wordpress sites. I found the configuration from this site to be the most helpful. I normally include code blocks to copy and paste but the configuration file is pretty big.

Instead, click here for the configuration code, then copy the whole page and paste it into your terminal.

Update 12/28/2016: I’ve updated the varnish configuration code slightly to allow the “purge all caches” button of W3 Total cache to work. Thanks to this site for pointing me in the right direction and this thread for getting me there.

After varnish has been configured, restart your new PHP / caching stack:

sudo systemctl restart httpd varnish php-fpm

Logging

Update: added this section on 11/4/2016

By default varnish does not log its traffic. This means that your apache log will only log things varnish does not cache. We have to configure varnish to log traffic so you don’t lose insight into who is visiting your site.

Update 2/14/2017:  I’ve discovered a better way to do this. The old way is still included below, but you really should use this other way.

New way:

CentOS ships with some systemd scripts for you. You can use them out of the box by simply issuing

systemctl start varnishncsa
systemctl enable varnishncsa

If you are behind a reverse proxy then you will want to tweak the varnishncsa output a bit to reflect x-forwarded-for header values (thanks to this github discussion for the guidance.) Accomplish this by appending a modified log output format string to /lib/systemd/system/varnishncsa.service:

sudo sed -i /lib/systemd/system/varnishncsa.service -e "s/ExecStart.*/& -F \'%%{X-Forwarded-For}i %%l %%u %%t \"%%r\" %%s %%b \"%%{Referer}i\" \"%%{User-agent}i\"\' /g"

Lastly, reload systemd configuration, enable, and start the varnishncsa service:

sudo systemctl daemon-reload
sudo systemctl enable varnishncsa
sudo systemctl start varnishncsa

Old way:

First, enable rc.local

sudo chmod +x /etc/rc.local
sudo systemctl enable rc-local #you can ignore errors here

Next, add this entry to the rc.local file:

sudo sh -c 'echo "varnishncsa -a -w /var/log/varnish/access.log -D -P /var/run/varnishncsa.pid" >> /etc/rc.local'

If your varnish server is behind a reverse proxy (like a web application firewall) then modify the above code slightly (thanks to this site for the information on how to do so)

sudo sh -c "echo varnishncsa -a -F \'%{X-Forwarded-For}i %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\"\' -w /var/log/varnish/access.log -D -P /var/run/varnishncsa.pid >> /etc/rc.local"

W3 Total Cache

The improvements above will greatly increase your speed and eliminate the F5 denial of service issue. The last bit to make it even sweeter is to install the W3 Total Cache wordpress plugin. Thanks to this site for the information that pointed me in the right direction.

There are a ton of options in W3 Total cache that are beyond the scope of this tutorial. For our purposes we will enable the following options in the General Settings tab of the plugin:

opcode-database-object

Opcode cache: Opcode:Zend Opcache

Database cache: Check enable, select Opcode: Alternative PHP Cache (APC / APCu)

Object cache: Check enable, select Opcode: Alternative PHP Cache (APC / APCu)

proxy

Reverse Proxy: Check “Enable reverse proxy caching via varnish”
Specify 127.0.0.1 in the varnish servers box. Click save all settings.

Full speed ahead

With all of these pieces into place your site is definitely more complicated, but it is also much faster to load. Enjoy.

Troubleshooting

If you go through all these steps only to see this very non-descriptive message:

File not found

it means you have PHP forwarding to the wrong directory. Modify the LocationMatch section you inserted at the bottom of /etc/httpd/conf/httpd.conf earlier to ensure the correct directory structure is passed for php files.

Install WordPress on CentOS 7 with SELinux

I’ve been frustrated lately with the number of tutorials for installing WordPress on CentOS 7 that say to simply turn SELinux off. This is an unacceptable workaround. It’s simply lazy administration. SELinux is not designed to make your job harder (although it can do that pretty well), it’s there to make your system safer.

I have scoured the internet and pieced together bits of information from several sources – mainly here, here, and here to put together a tutorial that walks you through how to install WordPress in CentOS7 while keeping SELinux turned on.

It took a lot of reading to understand SELinux, but once you understand it, it makes a whole lot more sense. Go figure.

Apache

Install required packages:

sudo yum -y install httpd
sudo systemctl enable httpd

Modify apache config to allow mod_rewrite:

sudo sed -i /etc/httpd/conf/httpd.conf -e 's/AllowOverride None/AllowOverride All/g'

Open necessary firewall ports:

sudo firewall-cmd --add-service=http --permanent
sudo systemctl restart firewalld

Start apache:

sudo systemctl start httpd

Navigate to your new site to make sure the testing page comes up.

Update 11/4/2016: If you are behind a reverse proxy such as varnish or a web application firewall, you will want to modify your apache configuration to log x-forwarded-for IPs to make the logs more meaningful:

sudo sed -i /etc/httpd/conf/httpd.conf -e 's/%h/%{X-Forwarded-For}i/g'

MariaDB

Install:

sudo yum -y install mariadb-server mariadb
sudo systemctl enable mariadb

Run initial mysql configuration to set database root password

sudo systemctl start mariadb
sudo mysql_secure_installation

Create a wordpress database and user:

mysql -u root -p 
#enter your mysql root password here
create user wordpress;
create database wordpress;
GRANT ALL PRIVILEGES ON wordpress.* To 'wordpress'@'localhost' IDENTIFIED BY 'password';
quit;

WordPress

Install PHP and restart apache

sudo yum -y install php php-mysql php-gd php-ldap php-odbc php-pear php-xml php-xmlrpc php-mbstring php-snmp php-soap curl
sudo systemctl restart httpd

Configure base wordpress directory

Download, extract, and set permissions for your wordpress installation:

wget https://wordpress.org/latest.zip
sudo unzip latest.zip -d /var/www/html
sudo chown apache:apache -R /var/www/html/wordpress

Optional: Change Apache document root so you don’t need to tack /wordpress at the end of the url:

sudo sed -i /etc/httpd/conf/httpd.conf -e 's/DocumentRoot \"\/var\/www\/html/&\/wordpress/g'
sudo systemctl restart httpd

Configure upload directory

If you want users to upload content, then you will want to assign the http_sys_rw_content_t selinux security context for the wp-uploads directory (create it if it doesn’t exist)

sudo mkdir /var/www/html/wordpress/wp-content/uploads
sudo chown apache:apache /var/www/html/wordpress/wp-content/uploads
sudo semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/html/wordpress/wp-content/uploads(/.*)?"
sudo restorecon -Rv /var/www/html/wordpress/wp-content/uploads

Run the wizard

In order for the wizard to run properly we need to temporarily give the wordpress directory httpd_sys_rw_content_t selinux context

sudo chcon -t httpd_sys_rw_content_t /var/www/html/wordpress/

Now navigate to your new website in a browser and follow the wizard, which will create a wp-config.php file inside the wordpress directory. Once your site is properly set up, restore the original security context for the wordpress directory:

sudo restorecon -v /var/www/html/wordpress/

Success! Everything is working within the proper SELinux contexts.

Troubleshooting

Permission denied when accessing /wordpress

The obvious thing to check is to make sure the directory /var/www/html/wordpress has the ownership set to apache. That didn’t fix my issue, though. Thanks to serverfault I narrowed this down to a selinux permissions issue.  Changing the selinux context to httpd_sys_content_t for the wordpress folder fixed the issue.

sudo semanage fcontext -a -t httpd_sys_content_t /var/www/html/wordpress/
sudo restorecon -v /var/www/html/wordpress/

Sorry, but I can’t write the wp-config.php file.

“You can create the wp-config.php manually and paste the following text into it.”

I had a suspicion that writing files was also a selinux issue. This was confirmed when I decided to tail /var/log/audit/audit.log and found this when the wordpress installer tried to write wp-config.php:

type=AVC msg=audit(1475596102.558:16868): avc: denied { write } for pid=5751 comm="httpd" name="wordpress" dev="dm-0" ino=68494934 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:httpd_sys_content_t:s0 tclass=dir
type=SYSCALL msg=audit(1475596102.558:16868): arch=c000003e syscall=21 success=no exit=-13 a0=7f9f4b931478 a1=2 a2=0 a3=1 items=0 ppid=5740 pid=5751 auid=4294967295 uid=48 gid=48 euid=48 suid=48 fsuid=48 egid=48 sgid=48 fsgid=48 tty=(none) ses=4294967295 comm="httpd" exe="/usr/sbin/httpd" subj=system_u:system_r:httpd_t:s0 key=(null)

This led me to find this great explanation from Francis Pereira on how to get wordpress and selinux to peacefully coexist. Also thanks to the excellent Redhat Manual for more information on what to do, combined with this very helpful guide from serverlab.

First, we need to temporarily grant httpd_sys_rw_content_t to the wordpress directory. This is to allow the initial wizard to create the wp-config.php file.

sudo chcon -t httpd_sys_rw_content_t /var/www/html/wordpress/

Now you can run the wizard and it will work properly.

Once your site is set up, restore the context to http_sys_content_t

sudo restorecon -v /var/www/html/wordpress/

Two factor authentication in WordPress with Authy

With data breaches as rampant as they are I’ve decided to get more serious about security and implement two factor authentication. Authy is a great way to add this to WordPress, and it’s free (or at least most of its features are.) This information comes from their blog.

  1. Install the Authy plugin from here
  2. Create an account at https://dashboard.authy.com
  3. Add an application for your blog to the Authy dashboard and copy the API key given to you
  4. Activate the Authy wordpress plugin, go into settings and paste in the API key
  5. Activate two factor authentication for your user by mousing over the top right corner and selcting “Edit my profile”, scroll down to the bottom, and click “Enable/Disable Authy”

When I did this I had forgotten that I had a different login plugin running – Login Lockdown. With both these enabled I could no longer log in! There was some sort of conflict between the two plugins. I had to disable both plugins by following this guide.

  1. Navigate to your wordpress directory and go to wp-content/plugins
  2. Rename the offending plugin directory to something like pluginname-disabled
  3. Log into WordPress and go to your plugins page, it will generate an error
  4. Now that you’re logged in, you can rename those folders back to their original name to either re-activate or delete those plugins.

Now you are much more secure. Even if someone has your password they will not be able to log in unless they also have your phone.