Thanks for taking the time to have a look at this. My company has been approached by external consultants offering to give us a view of the incremental lift of running certain marketing campaigns. I understand such a thing would be complex and we need to limit confounding variables etc. where possible, but they’re making it out to be rocket science and I don’t buy it.
I was pointed onto the following blog post by a friend some time ago: https://www.kaushik.net/avinash/controlled-experiments-measuring-incrementality/ which is a very good explication of the concept but doesn’t deal with the nuts and bolts.
My thought is: how much specialist statistics/data science (I hesitate to use the buzzword but bear with me) knowledge is really needed to successfully execute a controlled experiment and get reasonably accurate results. By reasonably accurate I mean be able to tell with a high degree of confidence that X campaign drove uplift in sales with a small margin of error.
These consultants are telling us we need a data scientist who uses R in order to do this, smells like BS to me. Or at least I don’t see why we can’t get a confident view that our campaigns are working using simpler statistical methods.
Does anybody have experience running these in marketing, or in some analogous field? What’s your experience/opinion on my question?