Here’s How Vanity Metrics Are Outsmarting Alexa

The term “vanity metric” refers to high-level indicators of popularity. Take an Instagrammer with a huge following, for example. Superficially, her large follower count will be alluring to brands who want to do influencer marketing campaigns. However, if this huge following is made up of mainly inactive users, or even bought (fake) followers, that influencer won’t be able to deliver the ROI that her stats would lead brands to believe.

Many brands get suckered into working with people like that because they’re not looking at the full picture. They need to dig deeper than these vanity metrics before working with someone. If you want to know what signals to look out for, check out our post on vanity metrics, which dives into specific ways that these metrics can be manipulated. In short, content creators can buy followers, engagement, comments, and traffic; in fact, just about any metric is susceptible to manipulation.

How Vanity Metrics Play Into Influencer Marketing

While researching vanity metrics, I stumbled across this post by the IFB. It targets bloggers and urges them NOT to lie about their stats because of tools like Alexa, Compete, and Quantcast. These sites provide traffic estimations for any site out there and many brands/agencies rely on them for validating the stats that bloggers send to them in media kits.

The reason this post struck a chord with me was because I actually had the OPPOSITE experience. I had to question if a blogger was inflating her stats based on the data Compete.com provided. But after that entire experience, I learned a pretty valuable lesson, so I’ll dive into that first.

Here’s how Compete.com led us astray… 

A few years ago, my partner and I were building a sales-alert app, targeting shoppers who were looking for deals. When it was time to launch our app, we ran a series of very successful campaigns with fashion bloggers to help spread the word.

We chose three bloggers for our campaigns: One who did the post for free, one who charged $150, and one who charged $400. At that time, we were just a two person team, no salaries, and combined savings was getting close to the “danger-zone”… so $400 for a blog post was a huge decision for us.

Before committing to the $400 campaign, I scanned through this blogger’s last 10 months of posts. I checked out what brands were working with her, how many of them were using her repeatedly, how many comments she was getting per post, and the quality of those comments.

And she checked out on all fronts. Except one.

When I looked her up on Compete.com, her traffic was so low that I actually decided to negotiate with her. The $400 charge just didn’t make sense for the traffic that Compete.com was estimating.

Well, she didn’t know anything about Compete.com, but she assured me that those stats were way off, and the ROI that we were after wouldn’t be a problem…

Her confidence was reassuring, so we decided go for it and “roll the hard six”… any BSG fans??!

Turns out, she was right. On the morning that our post went live, her blog sent us 9,000 clicks by 10AM. And after that first day, we had 3,000 signups!! That’s more than we’d ever had to date. Over the course of the next month, another 3,000 people found our little sale-app through her same post and signed up…AND, as if that wasn’t good enough, we had 10 more bloggers join our site and then write their OWN blog posts about our app.

She knocked our ROI goals so far out of the park…

The clicks that we received from her site within the first few hours of the post being live were equal to what Compete.com estimated for her entire site’s traffic over the course of a MONTH!!! Their estimates weren’t just a little off, they were off by like 15X!

And the funny thing is that I almost didn’t hire her because of the inaccurate stats I found on Compete.com. This was definitely a great learning experience!

Getting over Compete.com

Prior to this incident, I was never concerned with typical vanity metrics, like follower counts, as it’s so darn easy to just buy followers these days. But up until that experience with the $400 blogger, I was always a devout Compete.com user. I regarded its traffic estimations as gospel. When a blogger would share traffic stats that differed from what Compete.com told me, I would always believe Compete.com over the blogger.

After that incident though, I started looking for other meaningful qualities in the bloggers that I wanted to work with:

• First and foremost, I would make sure that the blogger’s audience would be interested in our app (fashion lovers who liked deals).
• After that, I looked at their engagement, the quality of their comments, what brands were sponsoring their content, and how many of them were repeat customers.
• Then, once I decided on a blogger, I’d simply ask them about their traffic. Pretty much everyone will at least give you a ballpark but many will go so far as to send you a Google Analytics screenshot.
• Lastly, and probably most importantly, I’d let them know what ROI I was hoping to achieve, and I’d straight up ask whether or not they felt this was feasible. I don’t think it’s fair to ask anyone to guarantee results and I’m pretty sure that most of them won’t give you a guarantee. However, it’s more than fair to ask them if they feel your goals are attainable. They know their audience better than anyone else does.

Two years later: 
Giving these traffic estimation sites a second chance…

Buying followers is easy and something we’ve observed many bloggers do. Our site gives you a rundown on follower growth as well as engagement stats.

Buying traffic, however, doesn’t seem to be as prevalent, but that doesn’t mean that people are going to be totally forthcoming about it. Most bloggers aren’t exactly handing out business cards with their Google Analytics login credentials.

So when a brand asks about traffic, it’s pretty easy to lie about it or at least twist the numbers to look more enticing. We’ve seen many bloggers send over skewed data and act as though it’s the norm, like a snapshot of their traffic during fashion week when they might have had an enormous spike due to extra press coverage. This site goes into a pretty brutal rundown of some very well-known bloggers out there who have gotten into trouble for not being honest about their metrics.

Brands and agencies are totally aware that influencers will sometimes provide misleading stats, which is why they’re still using sites like Compete.com and Alexa to validate the numbers that influencers are providing. It totally makes sense from their end.

As the co-founder of an influencer marketing platform, I’ve had the opportunity to talk to hundreds of brands and PR agencies. I can now attest to the claims made in the IFB article. In fact, I’ve come across more brands and agencies who are using traffic estimation tools than those who are not. In fact, our most requested feature has been integrating Compete.com or Alexa into our profile views.

I’ve already done pretty thorough testing with Compete.com, so I decided to test out Alexa before completely writing off these traffic estimator sites.

For those of you who don’t know, Alexa is a handy little browser extension. You just install it in your browser, and then whenever you are on a website, you can click it to see what “Alexa Rank” that particular website has. The rank tells you where this site sits within the lineup of ALL websites in the world.

The goal is to have the lowest Alexa Rank possible. Above you can see that Pinterest is ranked the 15th most popular website. Twitter is the 8th. CNN is 24th, which seems a little odd…The lower the rank, the higher the traffic.

So, assuming that these rankings are accurate, if you have a handful of bloggers you’re thinking about working with, all you have to do is figure out which of those bloggers has the lowest (aka best) Alexa score, right?

To check on this hypothesis, I used ROI data from a past campaign that we ran with four bloggers.

We had pretty good ROI from each of those four bloggers. Blogger A and Blogger B were by far the best though in terms of the return on investment they brought in! Those two bloggers sent us 80% of this campaign’s total ROI, which means that Blogger C and Blogger D only sent 20%. See the chart to the right.

With the ROI in mind, I then looked up the Alexa score for each of these bloggers to see if the scores matched up with the different ROI amounts.

The chart below shows the somewhat surprising results. Again, keep in mind that high Alexa scores = low traffic…sort of like in a race, where 1st, 2nd, and 3rd place winners are above everyone else. You want your Alexa score to be nice and low because you’re striving to become 1st place, just as you would in a race!

This diagram shows the Alexa score in grey, while the block of color represents the traffic estimation.

The blogger who did the BEST for us (Blogger A) had the WORST Alexa ranking out of everyone (430,316). Her ranking was actually 2X worse than the ranking of the blogger who did the WORST for us, whose Alexa score was 211,769. The blogger who was the second worst clocked in at 60,815,   which is eight times better than the one who did the best.

Anyway, that’s a lot of math and this inverse-scoring thing that Alexa does is sort of confusing…so, just to bring everything home, here is a diagram showing the ROI on the left, next to the traffic estimation on the right.

If we had based our decisions off of Alexa, we would have completely ignored the blogger who performed the best, and we would have probably double-downed on some of the under-performing bloggers. 45% of our ROI came through Blogger A, but according to Alexa, her traffic would be around eight TIMES lower than Blogger C. And Blogger C only gave us ⅓ the ROI that blogger A gave us.

That’s a very non-trivial discrepancy. It’s HUGE.

In case you need more convincing…

Before concluding my above experiment, just for kicks, I looked up my own site, The Shelf, because I know what our traffic is.

And according to Alexa, our site is getting more traffic than any of those bloggers!!

Can you believe that?

If you do, you shouldn’t. That data is totally off. We’re not getting anywhere near the amount of traffic those four bloggers are getting. When we did campaigns with them, they sent more traffic in a matter of hours than what we get in a week.

So, if you’ve managed to make sense of all of those diagrams and math, then I think we’re all on the same page about this. Even though influencers might be manipulating their numbers, there are many influencers out there who are unjustly being accused of misrepresenting their stats, even though they are not. Sites like Alexa and Compete.com are just making it LOOK like they are cheating.

I realize that my little experiment above isn’t exactly scientific so if you’re still unsure about what to do, here are two more Alexa-bashing blog posts that you might want to check out. These people also ran experiments…

Numbers, Rankings, and Why I Think Alexa Is Big a Waste of Time

Alexa Sucks and Here’s why

It’s also easy enough for you to just check it out yourself. Figure out what your website traffic is, then find two or three friends who can give you their website traffic stats, then compare that to the Alexa/Compete stats.

Where is their data coming from?

Since I wanted this post to be nice and thorough, I decided to look into HOW Alexa is coming up with these estimates, and this is what they say on their site:

“Alexa’s Traffic Ranks are based on the traffic data provided by users in Alexa’s global data panel over a rolling 3 month period. Traffic Ranks are updated daily. A site’s ranking is based on a combined measure of Unique Visitors and Pageviews.”

So basically, in order for them to measure traffic, their toolbar needs to be installed on a ton of browsers all over the world. Then they just look to see which sites those toolbar-users are visiting. If you have a toolbar on your browser and you go over to Blogger A’s blog, then your visit will count towards her traffic. 

BTW, if you want your own handy Alexa browser extension, you can download one here.

Superficially this entire setup makes a lot of sense.

But there’s one key part that sort of screws everything up.

You need to consider WHO is actually taking the time to install the Alexa toolbar…marketers, PR professionals, SEOs, website admins…that’s about it. Most people have never even heard of it. In the case of our fashion blogger example, how many fashionistas out there do you think are installing that browser extension? (Probably some. But disproportionately less than other vertical.)

If you want to work with influencers in the marketing or tech space, the rankings are going to be far more accurate because your target demographic is one in the same with the Toolbar-installers.

If, on the other hand, you’re reaching out to fashion bloggers, or mommy bloggers, I’d think twice before putting a lot of stock into the traffic stats being surfaced by these traffic estimation sites. These unreliable metrics might cause you to ignore bloggers who could potentially be wonderful for you to work with. And, even worse, they could produce a false positive, and lead you to throw money into a campaign that generates a whole lot of nothing.

How easy is it to manipulate your Alexa score?

If all of that info didn’t convince you of the perils of relying on these traffic estimators, let me tell you about one final experiment that I did…(because I’m super thorough like that).

I read an article by Yaro Starak: How To Boost Your Alexa Ranking In One Easy Step, which basically explains how to “game Alexa”. His article explained that he wanted to see how much his OWN traffic would affect his Alexa score. So he installed the Alexa toolbar on his browser, and then he went on his site every day and turned the toolbar on. Within a few weeks, his Alexa score improved by a whopping 20%. 20 PERCENT!!

I was curious to see how this could possibly be true, because one person should not be able to cause such a huge shift in their score. So I installed the Alexa toolbar on my OWN browser on March 13, 2015, a month and a half ago. Our score was 73,298.

Below is the progression that our Score went through over a 10 day period of time.

In just ten days, I saw a 15% improvement.

After that, I was sort of bored with this Alexa experiment, so I stopped taking screenshots of my score’s improvement. But it’s been steadily headed downward (which is good)…

Today, April 27, 2015, our score is 45,455.

Sort of crazy that one person (myself), is able to cause a score to change by 27,843 points!! This is a whopping 38% change in a month and a half. I wonder what would happen if I got each of my employees to do this too. Ha! We’d probably be up there next to Pinterest by now! 😉

Wrapping up.

It all boils down to your business. If you’re working with probable Alexa-bar users, then the Alexa widget can be your best friend but if you’re working with fashion/lifestyle/mommy bloggers, then you might want to reconsider your selection criteria.

Here’s some extra reading in case you’re looking for more:

• A post on Vanity Metrics.
• A post mapping out what your very first campaign should look like.
• A post about why bloggers charge and what to expect.

In the end, it’s all about targeting. You need to find the bloggers who are the best match to your brand in terms of demographic. And then approach the ones who work with good brands and get good engagement on their various posts. Once you get a conversation going, then pop the question about traffic and ROI goals.

It’s best to look at influencers as a sum of many parts.

It’s not just about engagement.

It’s not just about followers.

You need to look at an influencer from many different angles and then decide whether or not that person will have the power to influence your very specific audience.

P.S. I am by no means an Alexa expert. And I realize that some people are super passionate about these traffic tools. If I missed anything in this post, I’d love to hear what criteria you all use to select influencers, as well as any experiences you might have using traffic estimation sites! Let’s discuss in the comments!! 🙂