Block referrer spam from hosting platforms
Over the past few months we’ve been seeing an ever increasing amount of “automated” and botnet generated referrer spam coming into websites hosted on our hosting platform and onto sites that we manage on other hosting services on behalf of our customers.
This is a global problem affecting websites regardless of the type of site as the “visits” are all automated and generated randomly – so if you have a website and it’s accessible online then you could be a target. One of the major contributors to this increase in automated traffic is an SEO startup called SEMalt based in Ukraine, although they are by no means alone in their use of this technique.
Why is referrer spam and automated traffic a problem
While there is no direct security issue with this type of traffic – genuine search engines such as Google and Yahoo! use this method to “crawl” your website to index it’s pages and make it searchable – the problem comes when the traffic is uncontrolled and doesn’t abide by the instructions sent out from servers (like ours) that dictate how often a site should be accessed by automated processes which can result in a website becoming saturated with automated traffic and disrupt access for genuine visitors.
The other problem, and the main subject of this post, is that there has been a recent increase in instances of “referrer spam” (automated visits with the intention of showing a website address in your visitor logs and analytic data) which, in some cases, can render your website visitor analytics data useless due to the sheer volume of automated visits drowning out useful data created by actual, human, visitors.
From today we’re increasing our measures to mitigate this issue in two ways:
- We’re actively monitoring server-wide visitor traffic logs for instances of referrer spamming and taking steps to block this traffic from accessing the websites of customer who use our website hosting service.
- We’re monitoring server-wide data that feeds into our Piwik website analytics system and periodically removing this automated traffic from customer analytics accounts and email reports.
We are also currently running a clean-up process on all historic analytics data to remove automated visits that are a result of referrer spamming attempts. This will mean that from tomorrow your analytics data will no longer include this malicious traffic and so when you run reports on our Piwik service you may see a drop in visitor numbers going back several months (this is to be expected and the new reports will more accurately reflect your visitor numbers as they won’t include the automated visits)
Blocking SEMalt or other referrer spam traffic on your own website
If you’re website is not hosted by O’Brien Media you can use the following code, places in a .htaccess file in the root/base directory of your website to block SEMalt traffic (there are two examples from sites we’ve seen a lot of traffic from the the past few weeks)
SetEnvIfNoCase Referer 7makemoneyonline.com spam=yes SetEnvIfNoCase Referer buttons-for-website.com spam=yes Order allow,deny Allow from all Deny from env=spam
Whenever you notice a new referrer in your analytics data that you want to block just add a new line below the top two lines, as below, replacing example.com with the domain name shown in the referrer section of your analytics software:
SetEnvIfNoCase Referer example.com spam=yes
If you have any questions about this or any aspect of our service please don’t hesitate to get in touch.