"X-Moz: prefetch" and skewed page-hits

Earlier today I installed a WordPress plugin recommend for tracking the popularity of posts. The plugin is unsurprisingly named "Recently Popular". After installing the plugin I ran some quick tests and found that I was getting extra hits recorded. I spent a bit of time back-tracking to find the source and after systematically disabling all other plugins and page elements found that it was firing in wp_head() in the page header.

After some more digging, I noticed that the extra hit was for the chronologically next published post and that the problem occurred in both WordPress and WordPressMU. This wasn't making a lot of sense so I decided to try a different browser - more of a sanity test than anything. That's when I found it didn't occur in Chrome, or Opera - just Firefox 3.5.6 that I'd upgraded to a few hours earlier.

I fired up the Live HTTP Headers add-on and checked out the requests Firefox was making. It was definitely making both post requests. I took a closer look at the second request and noticed the extra header "X-Moz: prefetch".

A quick search for X-Moz: prefetch turns up Mozilla's Link prefetching FAQ which gives a good description of what is happening and why. WordPress creates a tag similar to the following when wp_head() is executed:

<link rel='next' title='The Next Post' href='http://your_domain/year/month/day/the_next_post/' />

I am unaware of anyway to disable the prefetch hints. You could edit your header.php and remove the wp_head() statement, but many plugins rely on the execution of this function so results could be unexpected and undesirable. The issue for me was not that the hint was published but that the prefetch hits were being counted as real post requests, as well as the actual request when I clicked through a second or two later. This would seriously skew the perceived popularity of posts.

My solution was to ensure that the Recently Popular plugin ignored post requests that passed the "X-Moz: prefetch" header. Depending on your server configuration, the method of checking the header exists may differ - apache_request_headers() (alias getallheaders()) is only supported when PHP is installed as an Apache module. Most servers should support checking for $_SERVER['HTTP_X_MOZ'].

I wonder how many other people will wonder why their page hit stats have mysteriously increased without any increase in ad impressions, etc.

I will contact the plugin author to suggest an update once I've published this post.

Tracking Email Clicks in Analytics

Many of us are using Google Analytics to measure the usual web metrics, but most of use are only using a small portion of this tool's functionality.

It's common practice to send registered users an email to confirm account activity, to keep them up to date via a newsletter, or to try and encourage return activity. In many cases we are not really measuring how effective these mailings are or how they impact on our website traffic.

At first glance it looks like a tricky problem, mail client applications will generally not pass a referrer and browser mail will be recorded as one of the hundreds of mail domains in use. Link Tagging is the simple solution, although there are a few options depending on how deep you want to go.

Source and Medium

By appending utm_source and utm_medium parameters to your links you can easily track who many visits are directly attributable to your mailings and see them in the All Traffic Sources report.

Here's an example of how your links should look:
http://www.yoursite.com/somepage.html?utm_source=Newsletter&utm_medium=email

Setting the utm_source value will replace any referrer value as the Traffic Source so random browser domains will be consolidated under one value, along with any email clicks with no referrer which would usually be classed as "(direct)". This is the only required parameter of this type, any other utm_xxxx fields used in conjunction with utm_source are optional.

Using utm_medium=email is recommended, especially if you are using more than one utm_source value in different email types (e.g. Newsletter, AdminEmail, ReferAFriend) so that you can easily filter the results on the All Traffic Sources report.

Campaigns

Specifying a utm_campaign value can help group your links in a more meaningful way. This could be a sub group of your source categories (e.g. utm_campaign=200907 to identify this is the monthly Newsletter for July 2009) or you could use a campaign like utm_campaign=Winter-Sale across many sources (email, banner, CPC, etc). It all depends on what you want or need to measure. Whatever you choose, any utm_campaign values tracked will be displayed on the Traffic Sources->Campaigns report.

Content and Terms

These options are less common but still useful. Setting the utm_content parameter could help identify if text or html emails are getting more clicks. Alternatively you could track the comparative success of different creatives from the same campaign. utm_content values tracked will be displayed on the Traffic Sources->Ad Versions report.

I've included the utm_term here just for completeness. It's usually used to identify search terms or keywords purchased. utm_term values tracked will be displayed on the Traffic Sources->Keywords report.

Handy Hint

Even if you're already tracking your email clicks with another solution, it's probably worth adding these parameters (or at least some of them). As long as they are passed to the landing page it doesn't matter if you add them to the pre or post tracking URL. You may do some special tweaks so that your tracking solution passes utm_xxxx parameters on to the destination URL.

Eliciting Site Referrals

One of the best ways to increase the reach of your website is through word of mouth. Personal recommendations from existing visitors carry a lot of weight. To make this easier and more frequent, it can be a good idea to give your users a nudge in the right direction with a "Share this with a friend" call to action.

The simplest way for visitors to share great content is to copy and paste a link to a friend (via email or chat). To help this process work well it's a good idea to have readable URLs to encourage clicking. For example http://yoursite.com/item/17326/The_Meaning_Of_Life looks a lot more interesting than http://yoursite.com/item=17326

Another approach to sharing content is the "Send this to a friend" model where the visitor enters a friends email address and the hosting site send the email on behalf. Many web-savvy visitors will avoid this rather than reveal the email address of a friend. In my opinion it's an out dated model that should only be used for "locked" content or simliar special circumstances.

A more powerful way to be recommended is by plugging into social networks or news networks where the link exposure is much greater than one-to-one. An easy way to implement this is via the AddThis.com bookmarking and sharing service buttons. It's a free service that allows a certain amount of customisation and also features analytics of what is being shared. There's a nice introduction video on the tour page.



Bonus Tip
Don't rely on the <title> tag to get the right title shared. To get the title to be picked up properly by Facebook, and a handful of other sites, you'll need to add a Meta Title tag:
<meta name="title" content="Dolphins rampage through Venice" />.

Analytics Debunks Charlatan

Yesterday I overheard a friend's phonecall with a salesman for an internet listing service. She runs a modest business with her website as the only advertising and bringing in enough customers to keep her booked weeks in advance.

The listing service was claiming that they could increase traffic to her site because they specialised in listing companies in her specialised field. I suggested that she should think carefully before throwing her money their way, since her website is fairly well optimised for search engines.

As it turned out, the listing service had offered her a one month free trial (which had just expired) and had been allegedly sending traffic her way already. I decided to spend a few minutes helping her evaluate the trial.

The first step was to look at their website. Sadly their homepage failed to load as most of the content was blocked by ABP - not a good start. Next we found her listing on their site, mostly content pasted from her homepage, although her business name was spelt incorrectly (twice). By this stage I was feeling underwhelmed.

So we decided to check out the traffic they've been sending to her site. Google Analytics had been in place for some time so we could easily measure the impact. The first thing we did was check the Traffic Sources report. Indeed there were 27 visits in the last month, although they never peaked higher than 2 per day and the bounce rate seemed pretty high to me.

I suggested we check on where these visitors were coming from and see if we could find out a little more about them so we set up a Custom Segment where Source contains the listing site's domain. We could see that almost all of the traffic came from London, except for 3 visits from Australia, coincidentally where the business was based. Digging further into the New vs. Returning Visitors report showed that all but one of the London visitors was the same person returning every day or so to generate traffic.

In my opinion this kind of listing service is a waste of money if you have followed the most basic SEO principals. Needless to say, my friend will not be engaging their services.