Measurement Is Crucial
0 likes147 views
With the recent launch of the Split Testing and Lift APIs, applying the scientific method to your marketing is now a reality.
![85
Set up conversion lift study
SET UP A LIFT STUDY WITH ONE TEST GROUP
curl
-F 'name="new study"'
-F 'description="description of my study"'
-F 'start_time=1435622400'
-F 'end_time=1436918400'
-F 'cooldown_start_time=1433116800'
-F 'observation_end_time=1438300800'
-F 'viewers=[<USER_ID1>, <USER_ID2>]'
-F 'type=LIFT'
-F 'cells=[{name:"test group",description:"description of my test group",treatment_percentage:89.75,control_percentage:
10.25,adaccounts:[<ACCOUNT_ID1>,<ACCOUNT_ID2>]}]'
-F 'objectives=[{name:"new objective",is_primary:true,type:"MAI",applications:[{id:<APP_ID>}]}]'
-F 'confidence_level=0.9'
-F 'access_token=<ACCESS_TOKEN>'
https://graph.facebook.com/<API_VERSION>/<BUSINESS_ID>/ad_studies](https://image.slidesharecdn.com/measurementiscrucial-190228092443/85/Measurement-Is-Crucial-13-320.jpg)
![86
Multiple groups
curl
-F 'name="new study"'
-F 'description="description of my study"'
-F 'start_time=1435622400'
-F 'end_time=1436918400'
-F 'cooldown_start_time=1433116800'
-F 'observation_end_time=1438300800'
-F 'viewers=[<USER_ID1>, <USER_ID2>]'
-F 'type=LIFT'
-F 'cells=[{name:"group A",description:"description of group A",treatment_percentage:50,control_percentage:20,campaigns:
[<CAMPAIGN_ID1>]},{name:"group B",description:"description of group B",treatment_percentage:20,control_percentage:
10,campaigns:[<CAMPAIGN_ID2>]}]'
-F 'objectives=[{name:"new objective",is_primary:true,type:"MAI",applications:[{id:<APP_ID>}]}]'
-F 'access_token=<ACCESS_TOKEN>'
https://graph.facebook.com/<API_VERSION>/<BUSINESS_ID>/ad_studies
SET UP A LIFT STUDY WITH MULTIPLE TEST GROUPS
-F 'cells=[{name:"group A",description:"description of group A",treatment_percentage:50,control_percentage:20,campaigns:
[<CAMPAIGN_ID1>]},{name:"group B",description:"description of group B",treatment_percentage:20,control_percentage:
10,campaigns:[<CAMPAIGN_ID2>]}]'](https://image.slidesharecdn.com/measurementiscrucial-190228092443/85/Measurement-Is-Crucial-14-320.jpg)
![88
Lift results sample
{
"id": "<STUDY_OBJECTIVE_ID>",
"results": [{
"population.test": 1349292,
"population.control": 336567,
"polutation.reached": 738219,
"impressions": 3349184,
"spend": 23443.12,
"frequency": 4.5368434028385,
"buyers.test": 212300,
"buyers.control": 50823,
"buyers.scaled": 203748.63642603,
"buyers.incremental": 8551.3635739689,
"buyers.delta": 1536.450929913,
"buyers.pValue": 0,
"buyers.isStatSig": 1,
"buyers.reached": 111199,
"buyers.reachedPercent": 0.52378238341969,
"buyers.baseline": 102647.63642603,
"buyers.lift": 0.08330794426164,
"conversions.test": 1473655,
"conversions.control": 362552,
"conversions.scaled": 1453465.4710177,
"conversions.incremental": 20189.528982342,
"conversions.delta": 29532.640850362,
"conversions.pValue": 0.26076775078954,
"conversions.isStatSig": 0,
"conversions.reached": 735235,
"conversions.reachedPercent": 0.49891935358004,
"conversions.baseline": 715045.47101766,
"conversions.lift": 0.028235307824002,
"incrementalROAS": -0.1998564046528,
"sales.test": 2112564.35,
"sales.control": 528126.12,
"sales.scaled": 2117249.607677,
"sales.incremental": -4685.2576770443,
"sales.delta": 113045.85347285,
"sales.pValue": 0.94564373701777,
"sales.isStateSig": 0,
"sales.reached": 1103736.31,
"sales.baseline": 1108421.567677,
"sales.lift": -0.0042269636514411,
}],
}](https://image.slidesharecdn.com/measurementiscrucial-190228092443/85/Measurement-Is-Crucial-16-320.jpg)
![89
Ad studies breakdowns
curl -G
-d 'access_token=<ACCESS_TOKEN>'
https://graph.facebook.com/<API_VERSION>/<STUDY_OBJECTIVE_ID>?fields=results&breakdowns=['cell_id']
{
"id": "<STUDY_OBJECTIVE_ID>",
"results": [
{
"cell_id": "<CELL_ID1>",
...
Default fields where the values are specific to the <CELL_ID1> breakdown
...
},
{
"cell_id": "<CELL_ID2>",
...
Default fields where the values are specific to the <CELL_ID2> breakdown
...
}],
}](https://image.slidesharecdn.com/measurementiscrucial-190228092443/85/Measurement-Is-Crucial-17-320.jpg)
![97
Ad studies and cells
Creation example
curl
-F 'name="Grumpy Faces Study"'
-F 'description="Will test which planet is best"'
-F 'start_time=1435622400'
-F 'start_time=1436918400'
-F 'type=SPLIT_TEST'
-F 'cells=[
{name: Blue Planet", treatment_percentage: 50, adsets: [<AD_SET_ID>]},
{name: Red Planet", treatment_percentage: 50, adsets: [<AD_SET_ID>]}
]'
-F 'access_token=<ACCESS_TOKEN>'
https://graph.facebook.com/v2.7/<BUSINESS_ID>/ad_studies](https://image.slidesharecdn.com/measurementiscrucial-190228092443/85/Measurement-Is-Crucial-25-320.jpg)
Measurement Is Crucial
- 2. 74 Half the money I spend on advertising is wasted; the trouble is I dont know which half John Wanamaker
- 3. Users are exposed to ads through multiple channels 75
- 4. Correlation doesn't imply causation Source: http://www.tylervigen.com/spurious-correlations 400 500 600 700 800 9 9.5 10 10.5 11 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 Civil engineering doctorates awarded Per capita consumption of mozzarella cheese 76
- 5. Testing Facebook Ads Using marketing API Split testingConversion lift measurement Test Control Ad impressions No Ad impressions Test Control Facebook population0 100% 0 50%Split #1 50% 100%Split #2 Test group #1 (Campaign IDs or ad set IDs) Test group #2 (Other campaign IDs or ad set IDs) 77
- 6. Test Conversion Lift and the scientific method Control 78
- 7. Conversion lift AD IMPRESSION NO AD IMPRESSION TEST CONTROL TEST CONTROL 79
- 8. Defining an experiment CAMPAIGN A NO CAMPAIGNVERSUS Single cell study Multi cell study CAMPAIGN A NO CAMPAIGNVERSUS CAMPAIGN B NO CAMPAIGNVERSUS CAMPAIGN A + CAMPAIGN B NO CAMPAIGNVERSUS
- 9. Study duration and cooldown periods Pre-study quiet period Post-campaign quiet period Campaign duration APRIL 15 17 MAY 1 17 MAY 8 17TODAY MEASUREMENT PERIOD 81
- 10. How is percentage lift calculated? TEST CONTROL 3k conversions (exposed) 2k conversions (unexposed) 4k conversions (scaled) 82
- 11. Formula reached conv. - incremental conv. (3000) (1000) incremental conv. (1000) 50%= percentage lift 83
- 12. 84 How do I get started?
- 13. 85 Set up conversion lift study SET UP A LIFT STUDY WITH ONE TEST GROUP curl -F 'name="new study"' -F 'description="description of my study"' -F 'start_time=1435622400' -F 'end_time=1436918400' -F 'cooldown_start_time=1433116800' -F 'observation_end_time=1438300800' -F 'viewers=[<USER_ID1>, <USER_ID2>]' -F 'type=LIFT' -F 'cells=[{name:"test group",description:"description of my test group",treatment_percentage:89.75,control_percentage: 10.25,adaccounts:[<ACCOUNT_ID1>,<ACCOUNT_ID2>]}]' -F 'objectives=[{name:"new objective",is_primary:true,type:"MAI",applications:[{id:<APP_ID>}]}]' -F 'confidence_level=0.9' -F 'access_token=<ACCESS_TOKEN>' https://graph.facebook.com/<API_VERSION>/<BUSINESS_ID>/ad_studies
- 14. 86 Multiple groups curl -F 'name="new study"' -F 'description="description of my study"' -F 'start_time=1435622400' -F 'end_time=1436918400' -F 'cooldown_start_time=1433116800' -F 'observation_end_time=1438300800' -F 'viewers=[<USER_ID1>, <USER_ID2>]' -F 'type=LIFT' -F 'cells=[{name:"group A",description:"description of group A",treatment_percentage:50,control_percentage:20,campaigns: [<CAMPAIGN_ID1>]},{name:"group B",description:"description of group B",treatment_percentage:20,control_percentage: 10,campaigns:[<CAMPAIGN_ID2>]}]' -F 'objectives=[{name:"new objective",is_primary:true,type:"MAI",applications:[{id:<APP_ID>}]}]' -F 'access_token=<ACCESS_TOKEN>' https://graph.facebook.com/<API_VERSION>/<BUSINESS_ID>/ad_studies SET UP A LIFT STUDY WITH MULTIPLE TEST GROUPS -F 'cells=[{name:"group A",description:"description of group A",treatment_percentage:50,control_percentage:20,campaigns: [<CAMPAIGN_ID1>]},{name:"group B",description:"description of group B",treatment_percentage:20,control_percentage: 10,campaigns:[<CAMPAIGN_ID2>]}]'
- 15. 87 Lift results API curl -G -d 'access_token=<ACCESS_TOKEN>' https://graph.facebook.com/<API_VERSION>/<STUDY_OBJECTIVE_ID>?fields=results
- 16. 88 Lift results sample { "id": "<STUDY_OBJECTIVE_ID>", "results": [{ "population.test": 1349292, "population.control": 336567, "polutation.reached": 738219, "impressions": 3349184, "spend": 23443.12, "frequency": 4.5368434028385, "buyers.test": 212300, "buyers.control": 50823, "buyers.scaled": 203748.63642603, "buyers.incremental": 8551.3635739689, "buyers.delta": 1536.450929913, "buyers.pValue": 0, "buyers.isStatSig": 1, "buyers.reached": 111199, "buyers.reachedPercent": 0.52378238341969, "buyers.baseline": 102647.63642603, "buyers.lift": 0.08330794426164, "conversions.test": 1473655, "conversions.control": 362552, "conversions.scaled": 1453465.4710177, "conversions.incremental": 20189.528982342, "conversions.delta": 29532.640850362, "conversions.pValue": 0.26076775078954, "conversions.isStatSig": 0, "conversions.reached": 735235, "conversions.reachedPercent": 0.49891935358004, "conversions.baseline": 715045.47101766, "conversions.lift": 0.028235307824002, "incrementalROAS": -0.1998564046528, "sales.test": 2112564.35, "sales.control": 528126.12, "sales.scaled": 2117249.607677, "sales.incremental": -4685.2576770443, "sales.delta": 113045.85347285, "sales.pValue": 0.94564373701777, "sales.isStateSig": 0, "sales.reached": 1103736.31, "sales.baseline": 1108421.567677, "sales.lift": -0.0042269636514411, }], }
- 17. 89 Ad studies breakdowns curl -G -d 'access_token=<ACCESS_TOKEN>' https://graph.facebook.com/<API_VERSION>/<STUDY_OBJECTIVE_ID>?fields=results&breakdowns=['cell_id'] { "id": "<STUDY_OBJECTIVE_ID>", "results": [ { "cell_id": "<CELL_ID1>", ... Default fields where the values are specific to the <CELL_ID1> breakdown ... }, { "cell_id": "<CELL_ID2>", ... Default fields where the values are specific to the <CELL_ID2> breakdown ... }], }
- 18. Can run multiple studies at once Use findings to apply adjustments to attribution figures Retesting period depends on purchase consideration cycle length 90
- 19. 91 How do I get better results from my Facebook campaigns?
- 20. One of these has 3x the conversions? Same CTR 92
- 21. Group A A/B test Group B 93
- 22. Ad studies and cells Split testing provides an easy way to run experiments on how different factors affect the performance of Facebook ads Ad formats Landing pages Creatives Bid types Targeting 9494
- 23. Defining an experiment HEADLINE A HEADLINE BVERSUS Single cell A/B test FACEBOOK INSTAGRAMVERSUS VERSUS FACEBOOK + INSTAGRAM Multi cell A/B test 95
- 24. One of these has 3x the conversions? Same CTR 96
- 25. 97 Ad studies and cells Creation example curl -F 'name="Grumpy Faces Study"' -F 'description="Will test which planet is best"' -F 'start_time=1435622400' -F 'start_time=1436918400' -F 'type=SPLIT_TEST' -F 'cells=[ {name: Blue Planet", treatment_percentage: 50, adsets: [<AD_SET_ID>]}, {name: Red Planet", treatment_percentage: 50, adsets: [<AD_SET_ID>]} ]' -F 'access_token=<ACCESS_TOKEN>' https://graph.facebook.com/v2.7/<BUSINESS_ID>/ad_studies
- 26. 98 Key takeaways Test using the scientific method Conversion Lift to prove FB value Split testing for optimisation within FB