We start with a combination of geo-based experimentation where results are fed into a marketing mix model in the form of Bayesian priors and an internal multi touch attribution model. These two pieces are combined and compared to the experiment results to give us a general idea of what the high level incremental ROI of our marketing efforts are. We use this data to create an adjustment factor that gets reapplied to the marketing mix model, which contains a diminishing return functional specification for our spend/response for each channel. Using these corrected diminishing return functions it becomes an optimization problem where we can optimize over different response metrics across different geo-areas.
Even this approach has issues - geo testing can take quite a while and there can be seasonal confounding between different campaign types that isn't captured since we're not running the same campaigns the entire year. Even a bigger problem is how we disentangle intrinsic customer behavior from what we're doing in marketing - heterogeneity in customer cohorts is a huge issue when it comes to working with downfunnel metrics with a MMM.
This article is one that's been in my mind for quite a while now - I've been toying with a directed graph approach where a MMM is only informing top level acquisition while downstream activation, customer cohort, CLV, and attrition models are used to build out the ultimate downfunnel analysis.
I appreciate the detailed response! Myself and another DS are building a marketing analytics program, with a focus on revenue (we are rev, not ops). It has been a massive undertaking and it has been a long time, well 5 years, since I have looked at an MMM. I read this article The Death of MMM's, which tbh I'm only taking half seriously, but it given my time away from marketing made me wonder. I definitely appreciate the MMM is limited to diminishing returns and adstock, but I don't see how that makes it "dead". One of the comments in that article (from Ashish S.) talked about using Vector Autoregressive Models, which I have been thinking about testing as well. He make a very compelling argument for them.
I work in B2B, so the n-size we deal with is very low. We are definitely taking the a priori approach right now as well. We are still at the back-of-the-envelope math stage, but I'm laying the framework for a Multitouch Attribution + Markov Chain, just need to get spend mapped and in a good state.
I appreciate you sharing that article! We are approaching MBR (FML lol), so I haven't had a chance to read it. Do you have any other go-to sources you recommend?
Very carefully.
We start with a combination of geo-based experimentation where results are fed into a marketing mix model in the form of Bayesian priors and an internal multi touch attribution model. These two pieces are combined and compared to the experiment results to give us a general idea of what the high level incremental ROI of our marketing efforts are. We use this data to create an adjustment factor that gets reapplied to the marketing mix model, which contains a diminishing return functional specification for our spend/response for each channel. Using these corrected diminishing return functions it becomes an optimization problem where we can optimize over different response metrics across different geo-areas.
Even this approach has issues - geo testing can take quite a while and there can be seasonal confounding between different campaign types that isn't captured since we're not running the same campaigns the entire year. Even a bigger problem is how we disentangle intrinsic customer behavior from what we're doing in marketing - heterogeneity in customer cohorts is a huge issue when it comes to working with downfunnel metrics with a MMM.
This article is one that's been in my mind for quite a while now - I've been toying with a directed graph approach where a MMM is only informing top level acquisition while downstream activation, customer cohort, CLV, and attrition models are used to build out the ultimate downfunnel analysis.
I appreciate the detailed response! Myself and another DS are building a marketing analytics program, with a focus on revenue (we are rev, not ops). It has been a massive undertaking and it has been a long time, well 5 years, since I have looked at an MMM. I read this article The Death of MMM's, which tbh I'm only taking half seriously, but it given my time away from marketing made me wonder. I definitely appreciate the MMM is limited to diminishing returns and adstock, but I don't see how that makes it "dead". One of the comments in that article (from Ashish S.) talked about using Vector Autoregressive Models, which I have been thinking about testing as well. He make a very compelling argument for them.
I work in B2B, so the n-size we deal with is very low. We are definitely taking the a priori approach right now as well. We are still at the back-of-the-envelope math stage, but I'm laying the framework for a Multitouch Attribution + Markov Chain, just need to get spend mapped and in a good state.
I appreciate you sharing that article! We are approaching MBR (FML lol), so I haven't had a chance to read it. Do you have any other go-to sources you recommend?