What is Bing It On?
Bing It On is a valiant marketing effort by Microsoft search engine Bing. The campaign claims that People Chose Bing Web Search Results Over Google Nearly 2 to 1 in Blind Comparison Tests and is fueled by BingItOn.com, a website where people could see for themselves in a blind comparison test (similar to the old Pepsi Challenge comparing Coca-Cola and Pepsi) which search results they prefer, Google’s or Bing’s. The website strips formatting of the SERPs (although it doesn’t do a great job of this as things like author pictures still appear for Google) so that users are unable to distinguish between which search engine is on the left or right. The user can search using a custom search term or use the search suggestions provided by Bing. Search engines switch sides randomly, and the user indicates which side’s results they prefer. They can also answer that it was a “draw” if they cannot choose a winner for that round.
Bing Wins 2:1?
I was a bit perplexed by this campaign as I was certain about the superiority of Google, having compared the two of them before with my own side-to-side searches. However, I was open to testing objectively which search engine I actually preferred when using the Bing It On website. I was also interested in which engine my peers would choose.
My Personal Results: 3 Google, 1 Bing, 1 Draw
Bing’s study conducted by an independent research company tested “nearly 1000 participants [and] 57.4% chose Bing more often, 30.2% chose Google more often; 12.4 % resulted in a draw”. These statistics were intriguing, but I still didn’t quite believe the numbers seeing as the results of myself and a few friends all had results in favor of Google. Then it dawned on me that there was quite a bit of public data out there on Twitter. Bing was having people tweet with the hashtag #BingItOn. Finding the answer to which search engine people actually preferred would be as simple as extracting the outcomes from Twitter’s search function for the hashtag. This also confused me as I would assume that if the results did NOT favor Bing, then they wouldn’t want people tweeting this information since it would be contrary to their claims. I couldn’t understand why Microsoft would spend so much money promoting the Bing It On campaign on television and all over the internet if there was data that said otherwise.
What the Twitter Data Says About Which Search Engine Wins?
I was able to extract data from Twitter (ranging from 9/07/2012 to 9/11/2012) using an advanced search term for sample size of n=286. This data included foreign language tweets as well, and the results were determined using Google Translator. The data from Twitter showed quite a different picture than the data that Bing is claiming:
The Actual Results of Bing It On
|
Google Wins |
Bing Wins |
Draws |
|
| Number of Tweets |
208 |
69 |
9 |
| Percent |
72.73% |
24.13% |
3.14% |
While Bing is saying that people prefer Bing’s search results almost 2 to 1 compared to Google, my data shows that people actually prefer Google’s search results close to 3 to 1.
#BingItOn Results From Twitter
You may be asking yourself why Bing would promote this campaign so heavily if there is public data available that says differently? The answer is that the campaign carefully masked this data through an intelligent social media campaign.
Obfuscation of Twitter Results With @BingRewards Contest
I had expected that it would be as easy to collect the data as searching for the Bing It On campaign hashtag. However when I actually tried this, I found that I could hardly find any results. As it turns out Bing brilliantly handled the social media aspect of this campaign. Bing ran another (might have been more than one) campaign in tandem with Bing It On, all of which used the BingItOn hashtag. One of which was a contest which overshadowed results for the Bing It On campaign. In order to compete, it was necessary to tweet a reason why you started using Bing. With the myriad of tweets using the BingItOn hashtag, it was hard to clearly see the results to the Google v. Bing challenge.
See Tweet below:
Why did you start using Bing? Visit binged.it/QzI0rO & answer for a chance to win 100k @bingrewards credits! #BingItOn
— Bing Rewards (@BingRewards) September 10, 2012
It was necessary to use the following advanced search query to filter out unrelated tweets (after utilizing the advanced search terms, results required sifting with Excel):
#BingItOn -@BingRewards -"win Xbox" -"2 win Surface" -"People chose Bing nearly 2 to 1 in a blind test" -"some great prizes" -"bing rewards" -"The most popular searched question on Bing" -#spon
Another interesting thing to note is when you receive the results from BingItOn.com, you are presented with social sharing options. Sharing via this method however, does not share your results; it only shares the link for the campaign. It was necessary for the user to take manual action to share their results. Many people were sharing what they thought were results, but instead was only an invitation for their friends to try Bing It On.
Not important, but this video had 173 Likes and 678 Dislikes. The response has been mostly negative.
Which search engine’s results did you prefer: Google’s or Bing’s? I’d be interested to see what your results were in the comments below.
UPDATE: Barry Schwartz conducted a poll that yielded similar results to my study. He found that Google won 55% of the time, Bing won 38% of the time, and 7% experience a tie.
UPDATE: A post published on The Bing search blog pointed to a study conducted by SurveyMonkey (mentioned on Search Engine Land) where “more people preferred Bing results labeled as Google than Google results labeled as Google” as a result of a Confirmation Bias (Google is the best search engine).
UPDATE (10/3/2013): Freakonomics posted about a study conducted by a Yale professor, finding evidence contrary to the claims of Bing. See comments on Search Engine Land and (Bing’s Response).