Right now online advertisers and marketers have a lot of interest in attribution analysis. Marketers are struggling to determine the impact of each individual campaign, across various channels, that make up their online marketing efforts. While they can discern the aggregate effects, it is much more difficult to disentangle the individual effectiveness from any given campaign or marketing channel.
This is actually a tough problem, and it is similar to the problem faced when attempting to optimize over multiple customer touch points across your own organization.
For basic single touch point tests, it is fine to just run an AB style test. Conductrics, like other solutions, learns to the best performing options by keeping track of the following:
- the number of times each test option is presented to users.
- the total value of the goals achieved (e.g. value of shopping cart) after each test option.
As you can see, it’s not too complicated really. Of course, when user targeting is activated, things get a bit more complicated under the hood, but the optimizer still uses essentially just these two bits of information to drive the optimization forward.
Things change a bit when we are solving muti-touch optimization problems.
Here is why. Not only do we need to know many goals happen after each test option, but we also need to learn how often the user goes to the other touch-points in your application. This means that we need to learn a bit more about the dynamics of the site and how the test impacts that dynamic.
Site Dynamics
Learning the dynamics means keeping track of not only when users convert, but also keeping track of how they move between the different touch points. For example, let’s say we want to run tests on three pages simultaneously. The following is a simple graph of what that might look like.
Okay, so this is a little messy looking, but bear with me. We have three pages, each running an A/B style test. The arrows represent the probability (how often) a user will move from one page to another after being exposed to one of the test options. The actual conversion event, or goal, happens off of Page C. By linking the three touch-points and the three tests, Conductrics will not only learn the option on Page C that has the highest goal conversion, but will also learn which options on Page A and Page B, will ultimately drive the most traffic to Page C, sort of acting like lead generators for Page C.
Isn’t this Just Multivariate Testing?
In some ways, yes. Like Multivariate testing, we are trying to optimize with more than one decision variable. However, Conductrics takes advantage of the fact that in multi-touch optimization, decisions take place at different times and places. That lets us employ a type of artificial intelligence, sometimes known as approximate dynamic programing or reinforcement learning.
Its sort of like learning how to play a good game of Backgammon, the site may need to make a series of decisions before the user reaches a conversion event. Or there may be multiple conversion events so that Conductrics needs to learn and harmonize the decisions taken at different locations in the site.
What is neat is that you don’t have to really do anything special to setup Conductrics to learn this much more complicated type of problem. You can create a multi-touch agent, just as you would a normal AB style test agent. You just tell us the decision-points, and the decisions for each decision point. That is it, we keep track of the connections and math so you can just focus on improving your customers experiences.
Can I also use targeting?
You bet you can. Not only can you add targeting, but you are also free to set up multivariate like tests at the touch-points – that’s what is so great about the Conductrics’ agent method, it gives you total flexibility in setting up your optimization efforts.
This table gives you some idea of the types of agents that you can set up, once you have signed up!
As you can see, AB Testing, Multi-Armed Bandits, and Behavioral Targeting, can all be seen as just different versions of the same intelligent agent. Cool, right?! Please comment with any thoughts or questions and don’t forget to go sign up for your free trial account.
Tags: AB Testing, Attribution, Dynamic Programing, Multivariate Testing, Targeting