WP Shield Security PRO – Release 18.4

ShieldPRO 18.4 is quick release after our recent 18.3 release, a couple of weeks ago.

Our main focus with this release is on performance.

We found, and receive reports from some members, that page loading was noticeably degrading over time. We’ve spent the last week or so digging into whether this is related to Shield and what the precise causes might have been.

In this article we’ll outline what we found and what we’ve changed with this release.

Was ShieldPRO Impacting Performance?

We discovered that yes, Shield was impacting the performance in 1 key area: Time To First Byte.

If you’re not familiar with how WordPress pages are loaded and sent to the visitor’s browser, here is a very simple summary:

1. Visitor requests a page (e.g. via their web browser)
2. The browser performs a DNS lookup to find out “where” exactly the webserver is that hosts this website.
3. The browser gets the IP address of the server and then sends an HTTP request to the server for the given page URL.
   • I haven’t gone into details here because it’s not relevant, but this involves much more than is outlined here. It also includes SSL/TLS negotiation & connection in most cases.
4. The server receives the request and directs it to the appropriate handler – typically apache/nginx which in-turn iniates the PHP subsystem.
5. Loading of WordPress then begins.
6. WordPress loads and builds the response.
7. This response is then sent back to the client (web browser) and the content for the page is received.
8. The browser then renders the page for the visitor, also loading anything else that’s required, such as images/stylesheets/javascript etc.

Each stage, if the performance is degraded, could severely impact the page load speed.

For example, the very first step is one of the most overlooked, yet critical, areas for performance optimisation. Ensuring you have fast and reliable DNS will offer huge gains in page load speeds for your visitors. We recommend CloudFlare for this, if you use them for nothing else!

The step we want to focus on is #6, where WordPress builds the response. This determines how long it takes for visitors to start receiving data from a site – the “time to first byte” (TTFB).

Our testing revealed that in some cases, running Shield could add up to 0.5s to this metric.

How Shield Security Increased WordPress Page Load Times

When WordPress builds a response, Shield has a bit of work to do and must assess each request in a number of different ways.

One critical areas is the status of the visitor IP address, and whether any rules should be applied.

Is the IP address:

• “whitelisted”/”bypassed” – i.e. has unfettered access to the site and all requests are allowed.
• “blocked” – i.e. has zero access to the site and all requests are blocked.
• on the “offense” list – i.e. has the IP committed an offense and is this current request also an offense?

Shield uses the database to store these IP rules. Shield queries the rules for any that match the conditions above.

Before our work to integrate CrowdSec, the IP Rules database was small, even for high traffic sites – perhaps 100s of IP addresses, at most. IP lookups were exceptionally fast and had no impact on page load speeds at all.

CrowdSec provides extensive lists of IP Block Rules and as-such, the IP Rules tables can get large over time. Shield always prunes this data keeping it as small as it can be. However, for example, on the server that you’re reading this article, the IP Rules lists currently sits at around ~39,000 entries.

That’s a lot of IP address.

Actually, it’s not that big, and this alone doesn’t impact performance. The IP Rules database is optimised so that lookups for individual IP addresses is indexed and very fast.

The problem lies in the fact that IP addresses can also be stored as “ranges”. Given some hosting restrictions, we can’t search IP ranges directly on the database.

So to summarise we perform 2 separate steps when querying IP rules:

1. Check whether the single IP address is present on the IP Rules table (very fast)
2. Request all ranges and check if the IP address is within any of the ranges (relatively slower than #1)

We found that it was the 2nd step that was causing the impact on page speeds. Querying the database for all ranges and then searching them, added a significant amount of time to the page load for large IP Rules tables.

How We’ve Optimised WordPress Page Loading

We’ve taken a multi-faceted approach to addressing this issue, but at it core, we use caching. Here are the 3 areas we’ve optimised through caching:

• IP Range caching – we query for all ranges and then store this result separately so that we have much faster access to this data on each page load
• “No IP Rules” caching – we query for individual IP address as-normal, but when there are no rules for the IP, we cache this also reducing the need to look for it on every load.
• “Bypass Rules” caching – we query for all bypass/whitelisted IP address and cache these also.

The most significant of these is the IP Range cache. This ensures that we have very fast access to the ranges – the slowest part of the process.

We decided to add the other 2 optimisations as they were quick wins that would help with improving performance, albeit marginally.

Do The Changes Help Speed-Up WordPress Page Loading?

Absolutely, yes!

For affected sites, caching the IP Ranges makes a big improvement to the page load times. If your site has been affected, it’ll feel more responsive almost immediately.

Other Improvements & Fixes

There are a number of further improvements with this release that also pertain to performance. We pushed a few operations that normally occur earlier in the page load to the end of the page loading process. These changes won’t have as big an impact as those discussed already, but they all add up.

We’ve also fixed a number of smaller bugs, such as display of the Google Authenticator QR codes, IP Rules table building and more.

Comments, Suggestions and Feedback

We always welcome your thoughts and feedback so please leave your comments below and we’ll get back to you. We’d love to hear whether these changes have made any noticeable improvements for you and your WordPress sites.