Incremental Facebook ROI / ROAS – Measurement options

Facebook is one of the most well-known places for running digital ads, especially for e-commerce. Still, lots of companies struggle with answering the question: “How much ROI does my Facebook campaign have?”. You may ask, why even ask ?! In Facebook panel there are all information ve all the statistic – spent, transaction value. Everyone can calculate it easily!

But is it so easy? Let’s dive a little deeper.

One more hint – in whole article I write about Facebook, however it is really the same for advertismenent on Instagram.

 

Deduplication in online marketing

In general, deduplication means removing duplicates. What does it mean regarding online marketing? It means that you measure each transaction on your site only once. Let’s say, that one person:

1. Saw your Ad on YouTube (mobile)
2. Checked your company Facebook profile (mobile)
3. Saw your Ad on Facebook and clicked it (mobile)
4. Clicked Google Shopping campaign (desktop)
5. Made a purchase (desktop)

 

In such a case in Google Analytics you will see that Google Shopping had a transaction. Google Analytics will count a session from Facebook. However, it will be different cookies*. In Facebook panel, however, you will get information that (thanks to your ad) there was a purchase. So in the end you had one transaction – Facebook shows it’s because of Fb ad, and Google Analytics shows it’s from Google Shopping.

The issue is that Facebook does not deduplicate transactions, meaning in Facebook panel you will always see that it’s Facebook purchase if a user had any engagement with your ad. Btw this is the answer to a very popular question on interviews (for e-commerce ads junior position) “Why Facebook shows much more transaction than Google Analytics”. Next question – which statistics should I believe:

• from Google Analytics – which shows, that I should stop spending on Facebook immediately
• from Facebook panel – which shows, that I should move all marketing budget into Facebook

Article as a whole shows how to approach it, as this is a very important question.

 

*Because of the difference in cookies you will not be able to see Facebook on a path to conversion, or assisted-conversion reports. What’s more regarding cookies is that in your smartphone you have two cookies – one is a standard web browser (for example Chrome) and the second one is a … build-in browser into Facebook app. Meaning that if someone clicked the site from Facebook ad. Went through a page, left it. And after an hour (or any other time) came back this time on standard browser … Google Analytics will not see Facebook on the path to transactions. As those are different cookies (build into Facebook app web browser and standard web browser).

 

Attribution window(s) on Facebook

The very first thing which you can check is the attribution window. It is the time frame in which Facebook counts a transaction as his. Here are the possibilities on Facebook:

As you can see there are two types of attribution windows view and click. What does it mean? It means if it is enough for a user to view the ad to be counted as a transaction, or is it required that the ad was clicked. Let’s say you are checking also view (which is a default setting) this way in Facebook panel a transaction will be shown even if the user only saw it. It could scroll through it for 1 second, and still, in Facebook panel, you will get that it is Facebook’s contribution to your business.

The second thing is a time frame. If you choose 28 days-view, it means that if someone saw (without clicking) your ad on Facebook, the 1st of January 2021, and made a purchase on the 28th of January in Facebook panel it will be shown as Facebook transaction.

Which attribution window should you choose? Let’s read and find out 🙂

Incremental transaction(s) in e-commerce advertisement

An interesting concept regarding the measurement of e-commerce ads is the incremental approach. What does it even mean? Here it is all about the comparison if the ads (really) work. Let’s imagine that you can exclude 50% of your audience from targeting and then you are able to measure a number of transactions for each group:

• 50% – people who could get your ad (of course no all of them got it, but they could) (group A)
• 50% – people who were excluded from any campaign (group B)

 

Let’s say group A made 100 transactions, and group B 80 transactions. In such a case your ads provided you with 20 transactions. It does not matter what Facebook panel shows you. The only thing which matters is the “real world” – how many transactions do you (really/incrementally) get from running your ads.

In short incremental transaction (of ad) means the transaction which would not took place if there was no ad. It does not take into account post-click / post-view measurement.

 

Facebook conversion lift

Such a test in Facebook nomenclature is called conversion lift or holdout test. You are checking if there is (and “if”, how great) a lift regarding transactions for a group of users who could get your ads. To put it on the graph it looks like:

So as described earlier your audience is divided into two groups. One can get the and the second cannot. Then you run your campaign(s) in a standard way. Facebook will not provide any ad to a control group. After the test, you get results – how many transactions you got from each group. After that, you can calculate how many additionals transactions you had because of running ads on Facebook. That’s the difference between “number of transactions from the group with possible ads” minus “number of transactions from the control group (without ad)”. Now you take your costs into account and calculate “real” (incremental) cost per transaction. Of course, it’s possible to calculate ROI / ROAS as well – you need to take into account the incremental value of transactions.

 

What are the disadvantages of Facebook conversion lift? There are at least a couple of them:

1. The main disadvantage is that you allow the same system to run a test, which profits from running ads on it. No one should be a judge in his case.
2. Facebook automatic testing takes ~15% as a control group. Often it is too little to get statistically significant results

What other options are there to test the same – incremental (real) impact which those ads have on your business. There is an option it is GEO testing. What is this concept about?

 

Geo testing for Facebook

Let’s assume that your company is an e-commerce shop selling in the whole USA. What you can do is to check which states are similar to each other. By similar, I do not mean its sale volume is close to one another. What I mean here is that both have a similar trend. So the sale of those two states can look like this:

Both locations are different regarding the number of sales. However, they are very similar regarding trends. To calculate a trend you can use for example Pearson statistics (in Excel / Google sheets it is “pearson” function).

Let’s assume those 19 weeks were standard activity without any test – ads were running in two locations. And in week number 20 you started running the test (targeting only one state – the one in red on the graph). You may ask here why the red one? Because this one is smaller, meaning you need less budget to move it a little bit to find that your ad provided you with additional sales. Let’s assume you run your test for 4 weeks and the results are:

It’s visible at a first glance, that your campaign provided you with transactions, which would not happen if there were no ads (as there were no ads in State 1, and the difference between the states shirnk significantly). On a graph there are 3 lines:

• Blue: State 1 – (without the ad) (control)
• Red: State 2 – with ads – testing
• Green: State 2 (probable results without a test) – calculated results if the difference between State1 and State 2 would stay the same.
• The incremental number of transactions is green line minus red line. Those are transactions that would not happen if you haven’t run the ads.

Of course, you should calculate cost per transaction and ROI/ROAS based on that data.

Remarketing / (pure) prospecting – Facebook paid campaigns

There is one more level for all the test – targeting. Remarketing groups behave differently compared to pure prospecting. What do I mean by pure prospecting? Any audience in which you exclude ALL the people who were on your site. So you are targeting people who (probably) don’t know your brand at all. At least that’s the idea.

How to approach testing with this additional layer? The best way is to test one thing at a time. For example, if you would like to use Facebook hold-out (lift) test, you can run remarketing on one account and prospecting on another account, as this type of test(s) is on the account level. A similar approach is possible with GEO testing.

 

Using Facebook panel for evaluation of campaign results

After running a test (of course, you should reexamine results once in a while) you should get the information – how many INCREMENTAL transactions did Facebook provide. Let’s say Facebook panel shows 500 transactions* and your test (after calculating the results) shows 100 transactions. So in the future, you can calculate your REAL Facebook impact on your business as 1/5 of what panel shows.

*And here is the answer to the attribution window. You should choose the one which is:

• logical for your campaign goal – if you sell cars you need a longer time
• the shorter the conversion window the better as your reporting will be more convenient – no time is needed to wait till all the data gets collected. And sometimes you would lose the results if you pause the campaign
• it should correlate with your incremental results. So let’s say in two years you run such a test 3 times. Look at different attribution windows and compare them to real results. Choose the one which correlates

 

Other e-commerce advertising which needs incremental test

In my opinion, E-commerce should test all the ads it run with an incremental approach to runs only those, which adds to the business*. If it doesn’t it is “a waste of money” which could be spent elsewhere. However, Facebook is not the only medium where incremental measurement is more than welcome. The other(s) are for example:

• Dynamic Remarketing – did user purchased after making a click or would they made an acquisition event without the ad (?). Those people were already on the website. They knew the site. That is an interesting question – if dynamic remarketing incrementally contributes to your business.
• Search Brand campaign – if there are no paid searches for your brand keywords this traffic can be retaken by organic traffic. Is it worth spending money on search brand (?)
• YouTube ads – in most cases Google Analytics will not evaluate many transaction from youtube advertising. Similar situation with Google Ads panel. So here is the opposite to Facebook panel. With youtube (probably) no report will show you great results. Still, YouTube is a very popular platform, and it needs proper evaluation.

Those are different examples and I would like to discuss the appropriate approach to each of them individually.

*I do understand that e-commerce also runs branding campaigns. But this article focus on performance ads, which the only goal is to sell. Still there are other ways to save money – for example on analytical tools.

Summary – Incremental Facebook Ads measurement

I would not believe in Facebook panel to compare its data to other traffic sources like SEO, Shopping, Affiliation, Marketing automation, and so on. From my experience, it is a good practice to run incremental (hold-out) test / geo test. Still, Facebook panel is not useless. It is really valuable to compare different targeting and ads to each other based on Facebook panel.

An additional thing not covered in this article is lifetime values of the user. Performance (e-commerce) marketing has lots of aspects to dig into 🙂