Below is a heated conversation that transpired between Dan Zarrella and Olivier Blanchard over an article written by Zarrella and published by the Harvard Business Review. Zarrella’s article, entitled “How to Calculate the Value of a Like” presents a formula for placing ROI value on a Facebook like. Blanchard claims the formula is too much an approximation and is not very useful. Blanchard explains gives an example how he to attribute ROI to a social media campaign, but Zarrella is concerned about data leakage. Both Blanchard and Zarrella are brilliant minds in the social media industry, but can’t seem to agree on this topic. I recommend reading this discussion, as one can learn a great deal about assessing the ROI of a social media campaign in the process.
See the Twitter debate bellow (presented with Storify):
Debate Between @thebrandbuilder & @danzarrella
Dan Zarrella Wrote "How to Calculate the Value of a Like". Oliver Blanchard doesn’t like it.
Storified by Paul Shapiro · Mon, Nov 26 2012 11:41:05
How to Calculate the Value of a LikeOver my decade working in web marketing, I’ve spent a ton of time at various marketing conferences, and I’ve read countless books and blo…
Oliver Blanchard calls out Dan Zarrella on his article
Okay, sorry, but I’m going to have to start cursing on Twitter if people keep publishing shit like this: http://bit.ly/V8wd8lOlivier Blanchard
@thebrandbuilder You don’t like it? Why not?Dan Zarrella
@danzarrella Well, for starters, because it doesn’t actually measure the value of a like.Olivier Blanchard
@thebrandbuilder How so? It may not measure the entire value of a like, but it’s a great start/baseline, no?Dan Zarrella
@danzarrella Dan… 1. If it doesn’t measure the entire value of a like, then it doesn’t measure the value of a like. Let’s start with that.Olivier Blanchard
@thebrandbuilder Not true. Is the value of any marketing measurable?Dan Zarrella
@danzarrella 2. No, it’s not a great start/baseline. It’s a mess of random measurements thrown into an equation that makes no sense.Olivier Blanchard
@thebrandbuilder Again, not true. Months were spent working on this with a bunch of smart people.Dan Zarrella
@danzarrella 😀 Re: "Months were spent working on this with a bunch of smart people." Oh, well it must be legit then.Olivier Blanchard
@danzarrella By the way, when you counter with "Is the value of any marketing measurable?" what do you think that says about your math?Olivier Blanchard
@thebrandbuilder You’re trying to say it’s a random pile of measurements that don’t make sense, after what, 5 minutes looking at it?Dan Zarrella
@thebrandbuilder I’m asking you for an example of a marketing channel that is measurable to your likes.Dan Zarrella
@danzarrella For starters, Dan, I sure as shit wouldn’t waste my time measuring the "value" of likes. And if I did, I would do it right.Olivier Blanchard
@thebrandbuilder Ahh, so there’s the crux, you believe VOAL isn’t worth calculating.Dan Zarrella
@thebrandbuilder Btw, your tone about this whole thing is very unbecoming, FWIW.Dan Zarrella
@danzarrella No need for a pivot just yet, Dan. Whether I believe it’s worth calculating or not has no bearing on your equation.Olivier Blanchard
@thebrandbuilder So give me an example of a channel you believe can be measured effectively.Dan Zarrella
@danzarrella My tone? 😀 I think you mean my calling out the absurdity of your equation. I understand how it might be unpleasant.Olivier Blanchard
@thebrandbuilder Disagreeing with me is great, I love that. But you know what I mean. Anyways, got an example for me?Dan Zarrella
@danzarrella Every channel can be measured effectively. And this may come as a surprise, but it’s fairly easy to do.Olivier Blanchard
@thebrandbuilder So can you give me a specific example of a method of measuring?Dan Zarrella
@danzarrella I’m working with a municipality right now that spends resources on seven key channels. We know the $ value of each one.Olivier Blanchard
@thebrandbuilder Ok, so give me one specific example, please?Dan Zarrella
@danzarrella Yeah, I could. Though this is stuff you should already know how to do.Olivier Blanchard
@thebrandbuilder haha, but you won’t.Dan Zarrella
@danzarrella Come to Greenville, SC. I’ll set aside an afternoon to teach you this stuff. Free of charge.Olivier Blanchard
@thebrandbuilder So you don’t have a single specific example for this discussion?Dan Zarrella
@danzarrella I have plenty of examples, Dan. And none involve a crap equation. It’s simple ROI and channel-driven conversions. It’s simple.Olivier Blanchard
@thebrandbuilder So why won’t you share a single specific example?Dan Zarrella
@danzarrella I don’t measure "likes" because "likes" don’t get asses in seats at a theater or feet in a store, for starters.Olivier Blanchard
@thebrandbuilder So there’s no correlation between likes and sales?Dan Zarrella
@danzarrella Nope. Zero correlation between likes and sales. None. The "value" of a like isn’t only unrelated to sales…Olivier Blanchard
@danzarrella … it’s also inconsistent across brands, individual consumers, and time. The "value" of a like is a constantly moving target.Olivier Blanchard
@thebrandbuilder Can you point me to that research? I’d love to check that out.Dan Zarrella
@thebrandbuilder Hence why companies need to use their own metrics in the calculation.Dan Zarrella
@danzarrella You could start by reading my blog. In regards to calculating like/fan/follower value, start here: http://bit.ly/Q17SKhOlivier Blanchard
@danzarrella Here’s the thing, man. I think it’s great that you’re trying to figure this out, but your equation makes no sense. It’s wrong.Olivier Blanchard
@danzarrella And from your answers to me earlier, I think that you already know that. So why publish it if you know it’s wrong?Olivier Blanchard
@danzarrella I have examples, Dan. I have examples going back six years. B2B, B2C, enterprise, mom & pop, govt… That’s not the point.Olivier Blanchard
@danzarrella What would my explaining to you how retailer X measured the value of Facebook vs. email in a campaign help your math, Dan?Olivier Blanchard
@thebrandbuilder You don’t have a specific example for discussion, let me know when you do.Dan Zarrella
@thebrandbuilder Give me the example and you’ll see.Dan Zarrella
Be prepared to learn something…
@danzarrella All right. here we go. Small specialty retailer. Local market. Channels: print advertising, FB, a blog, Twitter and email.Olivier Blanchard
@thebrandbuilder So how do you measure the value of their Twitter account?Dan Zarrella
@danzarrella The biz was spending 80% of its $ on print. Didn’t think that was good. Wanted to figure out the value of each channel…Olivier Blanchard
@danzarrella So we started embedding promo codes in each post, link, update and ad. The promo codes were unique to each channel.Olivier Blanchard
@thebrandbuilder What about all the people who saw a retweet of their content and walked into the store?Dan Zarrella
@danzarrella We started measuring the impact of unique promotions (per channel) to foot traffic in his stores. (He has no e-commerce biz.)Olivier Blanchard
@danzarrella Let me finish. I’m explaining this to you.Olivier Blanchard
@danzarrella We knew exactly how much $ (including man hours, not just media buy) we spent on each channel. Following me so far?Olivier Blanchard
@thebrandbuilder Yep.Dan Zarrella
@danzarrella At specific intervals (weekend sales/promotions) we collected promo codes at the point of purchase.Olivier Blanchard
@danzarrella Remember that each channel had its own promo codes. This information was added to the transaction info in his CRM system.Olivier Blanchard
@thebrandbuilder And the people who didn’t use the codes?Dan Zarrella
@danzarrella This allowed us to tie specific $$$ amounts to individual channels AND to unique customers, to see where we were reaching them.Olivier Blanchard
@danzarrella Those who didn’t have the codes were asked by sales associates where they heard about the promotion.Olivier Blanchard
@thebrandbuilder But you missed customers who didn’t use promo codes, and misatributed those who follow on multiple channels.Dan Zarrella
@danzarrella Whichever channel came up first was the one we used. Those who didn’t know about the promotion were entered blank.Olivier Blanchard
@thebrandbuilder And those that couldn’t remember, didn’t answer, didn’t answer correctly, or cited one of the multiple channels they’re on?Dan Zarrella
@danzarrella The blanks were an important value because it told us what % of customers we weren’t yet reaching yet.Olivier Blanchard
@thebrandbuilder Seems like there’s a lot of value leak in that.Dan Zarrella
@danzarrella There’s no value leak. We repeated the process regularly and often. This allowed us to track shifts in channel value and usage.Olivier Blanchard
@thebrandbuilder Do you agree that there were some people who weren’t tracked correctly?Dan Zarrella
@danzarrella We tracked the value of outbound channel activity relative to transactions, Dan. We weren’t tracking people or likes.Olivier Blanchard
@thebrandbuilder But if you missed some people, you missed some value.Dan Zarrella
@danzarrella What it created for us was a value matrix, starting with ROI by channel. It told us the net reach of our channels…Olivier Blanchard
@danzarrella It also told us what channels were most likely to trigger visits to the store and subsequent transactions…Olivier Blanchard
@thebrandbuilder Can you agree that you missed some customers that weren’t tracked correctly?Dan Zarrella
@danzarrella It even told us what channels our biggest spenders preferred. We actually started customizing content to match their needs.Olivier Blanchard
@danzarrella We asked the "blanks" (those customers we weren’t effectively reaching yet) what channel they wanted us to use to reach them.Olivier Blanchard
@thebrandbuilder And you don’t think a single person you "reached" with Twitter was a blank?Dan Zarrella
@danzarrella They told us. We saw an almost 30% increase in in-store responses to promotions in the first month following our first test.Olivier Blanchard
@danzarrella Dan, you’re still missing the point. If our message "reached" someone on a channel but they didn’t share, come to the store…Olivier Blanchard
@danzarrella … or ultimately buy something, it was a dud. We didn’t count that as reach. We counted that as not reached.Olivier Blanchard
@thebrandbuilder You don’t think you missed a single person you "reached" that also came into the store?Dan Zarrella
@danzarrella Your "equation" and this discussion are about "value." Well, that value is ultimately measured in transaction $$$.Olivier Blanchard
@thebrandbuilder You’re dodging my question, you know what I’m asking you and you’re trying to obfuscate it. Your method had leaks.Dan Zarrella
@danzarrella What we measured (without convoluted equations) was that value. It told us, for instance, that email was our most valuable chnlOlivier Blanchard
@thebrandbuilder But you missed a bunch of customers, who didn’t use promo codes, or didn’t self-report correctly.Dan Zarrella
@danzarrella Dan… you’re still not understanding this, man. The people who didn’t use the codes were people we didn’t reach.Olivier Blanchard
@thebrandbuilder Haha, that’s some silly logic. There were people who saw the Twitter account who bought stuff that didn’t use codes.Dan Zarrella
@danzarrella Anyone who was just there on their own, unprompted, was counted as "not reached." That initial number was about 65%.Olivier Blanchard
@thebrandbuilder I think you need a quick lesson on accuracy vs precision. Maybe try wikipedia?Dan Zarrella
@thebrandbuilder There were people who came because of twitter that didn’t use a code.Dan Zarrella
@danzarrella By month #4, the number of non reached during promos was down to about 25%. Are you paying attention?Olivier Blanchard
@thebrandbuilder You made up a bunch of different categories to account for leakiness, but the leakiness was still there.Dan Zarrella
@danzarrella People who came because of twtr and didn’t have a code just told the sales associates that they had heard of the offer on twtr.Olivier Blanchard
@thebrandbuilder My point is that there are no perfectly accurate measurements in marketing, but there are lots of usefully precise metrics.Dan Zarrella
@danzarrella They had no reason to lie, man. 😀 We verified it over the next few months anyway. It was accurate.Olivier Blanchard
@thebrandbuilder haha and you seriously think 100% of them correctly self-reported?Dan Zarrella
@danzarrella Yeah, actually. 1. We have the numbers to prove it. 2. We have their transaction history to prove it. 3. …Olivier Blanchard
@thebrandbuilder You were relying on self-reporting. Horribly leaky.Dan Zarrella
@danzarrella 3. Since we engaged with them directly on the channels they provided for us, we know it’s true. So yes. It was accurate.Olivier Blanchard
@danzarrella @thebrandbuilder So this discussion amounts to about an hour of both your time. What’s the value of that? ;)david barrowcliff
@dbarrowcliff The value to me: Teaching @danzarrella and everyone paying attention the fundamentals of how to do this properly. :)Olivier Blanchard
@thebrandbuilder I can see the customer leak all over the floor of your customer’s store.Dan Zarrella
@danzarrella Not "leaky" at all, no. As an aside, most of these folks ended up engaging with us across multiple channels anyway.Olivier Blanchard
@thebrandbuilder Yeah, so they were either missed or incorrectly attributed. Super leaky!Dan Zarrella
@danzarrella You know, you can be a smartass all you want. You asked for an example. I gave you a good one. Learn or don’t. That’s your callOlivier Blanchard
@thebrandbuilder Yes, and I showed you all the leak in it. Just as leaky as any other measurement.Dan Zarrella
@danzarrella Maybe @dbarrowcliff was right after all. This was a complete waste of my time. Good luck with your "like" equation, Dan.Olivier Blanchard
@danzarrella When you’re ready to stop peddling bullshit and feel ready to learn how to do this properly, give me a call. I’ll help you.Olivier Blanchard
After reading all this, IÂ would like to nominate Oliver Blanchard and Dan Zarrella to go head to head in a debate at SXSW.
Â
BTW, I applied the formula to our business and it’s incredibly inaccurate. We measure ROI from Facebook as a lead-generation source as combining our customer fundamentals. Again, it’s breaks down to a simple ROI formula… not any of this quadratic equation business.I worry for the others who feel like they magically can cast a spell for their clients to “explain the ROI away.”
Wow.
Who’s making the T-shirts. 😉  Team Olivier!  Seriously, my brain hurts after reading that. Well done, sir.
@belllindsay Love it! I recommend reading the comments on the HBR article as well. They are pretty entertaining.
@pshapiro Oh, I’ve been all over the post since yesterday. LOL  Gold, Jerry, gold!! Thanks for linking to Olivier’s update. 😀
@pshapiro I have also bookmarked this page. Needless to say.
Here is Oliver Blanchard’s response to the debate on his blog:
http://thebrandbuilder.wordpress.com/2012/11/27/how-not-to-measure-the-value-of-a-like/
thanks for pulling this together, Paul.
I would side with Olivier on this one. I think it cannot be argued that no marketing measuring tool is perfect, the question is how accurate they are. The @brandbuilder method seems pretty accurate to me, it does what it aims to do – it can track the efficiency of each channel used so the ad budget could be optimalized. It’s clever and it’s simple. I don’t really see what Dan’s convoluted formula is good for even if it actually worked, and I don’t think it does. Olivier’s blog post (http://thebrandbuilder.wordpress.com/2012/11/27/how-not-to-measure-the-value-of-a-like/) explains very well why.
Lacking clear, transparent, and transferable metrics is a key reason holding many folks back from the big investment in SoMe that hey should be making.
I’m sorry to say it, but I’m disappointed in the rudeness and tone of superiority displayed by one party in this debate. Neither side is without blame, but one was just over the top. I understand and appreciate the passion this topic instils, but the high dudgeon and comments bordering on personal insult only serve to obfuscate the points being made.
@markfrisk I agree, and not that it is an excuse, but Oliver often speaks in this sort of tone. It was not restricted to this conversation. I too would have liked the debate to have been more professional. Thanks for your opinion.
@pshapiro I know he often uses this tone. It’s unappealing and it’s definitely not doing him any favors.
Â
None of which is to take away from your job in capturing the debate. On that I say, Well done!
Thanks so much for pulling this together. Will be using this as an example in a future class since the content,and, yes, even the tone,
are great “reaching moments.” Thank you.
Thank you for putting this together. Without having read the piece or the model that Dan Zarrella put together, I’d say that it sounds like another equation that puts together data for data’s sake. We are inundated with data in social media, but attributing that data to results (sales) is an entirely different challenge. We have to be very clear about why we’re measuring certain metrics and what they actually mean to sales, which is what Oliver was getting at. You might not like his tone and I’ve been turned off by it in the past as well, but his points here are well made.
What a great debate, fascinating to read through it and see how data was being used. And how data can be analyzed differently by people.