Most of the bots that are visiting your website are far from friendly. Whether you’re worried about viruses or web scrapers, your sites would benefit from having a filter in front of it to weed out the bad bots. Cloudflare is the most popular solution, and although it’s free, it isn’t the best.
Cloudflare protection from SQL injections is limited, and legitimate visitors sometimes get “site offline” messages or captchas. Incapsula and Distil are two intriguing alternatives that, unlike Cloudflare, don’t require you to hand over your DNS.
Incapsula claims that 50% of your site visitors are actually not human. These bots are at the very least sapping your bandwidth and at worst trying get access to your sites (through SQL Injection, XSS, OWASP and other tactics).
Incapsula positions itself as your website’s gatekeeper. They use a sophisticated visitor classification technology to allow legitimate traffic into your website while keeping the bad bots out.
Incapsula’s free plan is actually quite generous. You can monitor unlimited domains and enjoy the following benefits:
- Advanced website security
- CDN & website optimization
- Backdoor hack protection:
Setting up Incapsula is very easy. Just go into your Advanced DNS Editor (in cPanel) for your website and set the following:
- Change your site’s IP (A record) to the IP Incapsula gives you.
- Change www (cname) to point to the address Incapsula gives you.
Although setup is simple, one thing I noticed was that Incapsula was putting “www” in front of my domain name in my dashboard, which was odd since I didn’t specify the www form of my domain name during setup. After doing a little digging on their blog, you apparently need the www form of your domain to be authoritative to get the full effect of their CDN service. So to get the full Incapsula content delivery effect (your data routed to their nearest datacenter & CDN), you need to have:
- Have http://example.com as the root of your site.
- Setup the www cname as mentioned above in the setup stage.
- Flip on the domain redirection switch for the www to be the canonical URL for your domain in the Incapsula dashboard. (located in your site’s general settings):
I hate www in front of my domain name so I didn’t turn this on. For my site, Incapsula is doing some basic caching and I’ve setup MaxCDN to do the content delivery.
Incapsula’s dashboard is impressive. You can see what percentage of your visitors are real people versus bots. Incapsula is not only good for monitoring but for blocking bad bots and malicious users. You can block by IP or at the country level, and you can automatically block “Bad Bots” as defined through Incapsula’s intelligence.
You’ll notice in the picture above that some features are not available unless you upgrade. These include SQL injection, cross site scripting and DDoS protection. Incapsula’s Enterprise-Grade Web Application Firewall is only available on their $59/month Business plan. The Business plan is also required to get email support, while advanced anti scraping is only available on their Enterprise plans. Any paid plan will get your Incapsula CDN upgraded so that it performs advanced dynamic content caching.
Both Incapsula and Distil have the same concern (bad bots), but different reasons as to why they want to block them. While Incapsula’s value proposition is based around blocking hackers from getting into your site, Distil positions their product around stopping web scrapers from stealing your site content. Incapsula mentions their advanced anti scraping technology only in relation to their Enterprise plans which is probably out of the reach of most webmasters.
Distil is actually a Content Protection Network (CPN) that filters incoming requests to ensure visitors are valid end-users and not bots or scrapers. The Distil CPN uses behavioral-based learning to continually identify bad bots. It’s an adaptive algorithm: the more you use Distil, the more it learns about the types of visitors and the better it’s able to provide protection. Any requests for your web pages will get routed through Distil’s cloud CPN servers to vet and fingerprint all visitors. Any website visitor that looks to be performing content scraping will be identified by Distil’s behavioral-based identification algorithms. A further strength of Distil’s offering is that you can configure how offenders are handled: you can block offending users, or use a gradual stepped response resulting in an eventual ban. Captchas can also be selected to allow suspicious users a chance to verify before proceeding. All Distil’s servers share information regarding malicious signatures, so there is an element of crowdsourced intelligence being used. An attacker that is identified by Distil on one site will have its unique signature distributed and flagged for all sites under its protection. Distil also accelerates your website’s performance, indirectly through bandwidth saving and directly through content caching.
Much of the Distil dashboard focuses on analytics that tell who your visitors are and what they are doing on your site:
as well as how many pages users view per minute:
After Distil finds the threats to your site, you can block them using various options:
Distil allows you to block by country:
block by referrer:
Distil also allows you to block all proxies accessing your site:
Value vs. Cost – Incapsula vs. Distil
Much of my perceived value of the paid versions of both services is influenced by two observations:
- Incapsula’s free plan is full of useful features, while Distil doesn’t offer a free plan.
- Neither Distil or Incapsula’s paid plans scale well for webmasters who own more than 5 sites.
Considering that Incapsula’s $9 Personal plan or $59 Business plan only covers one site ($9 or $19 for additional sites, depending on the plan), Incapsula doesn’t scale well for someone who owns multiple sites. Someone who owns 20 sites using Incapsula’s Business plan would be paying $420 a month. The same could be said for Distil: a $99/month plan is available for up to 5 sites, and a $499/month Plus plan for up to 25 sites. The Plus plan and Pro plan (50 sites / $1499) pricing is confusing: you’d end up saving $500 a month by stacking 10 Standard plans together to monitor 50 domains. The bigger plans do get extra monthly page views, but Distil’s pricing seems regressive the more sites you add past the first five.
My recommendation: Incapsula’s free plan is a step up from Cloudflare. I would recommend keeping most of your sites there for the bandwidth reduction and the bad bot filtering. If you have sites that are getting hacked or under DDoS (or if you own an important site that can’t afford either of these things happening), then I recommend that you selectively upgrade sites to Incapsula’s Business plan to take advantage of their firewall protection. If you have content-heavy sites that you suspect are getting scraped, I’d recommend Distil’s Standard $99/month plan for up to 5 sites. You can head over to http://www.distil.it and sign up for a 14 day trial.