Author Archives: Matt Gershoff
Segmentation and Shrinkage
In our last post, we introduced the idea of shrinkage. In this post we are going to extend that idea to improve our results when we segment our data by customer. Often what we really want is to discover what digital experience is working best for each customer. A major problem is that as we segment […]
Posted in Uncategorized
Leave a comment
Prediction, Pooling, and Shrinkage
As some of you may have noticed, there are often little skirmishes that occasionally break out in digital testing and optimization. There are the AB test vs multi-armed bandits debate (both are good, depending on task), standard vs multivariate testing (same, both good), and the Frequentist vs. Bayesian testing argument (also, both good). In the […]
Posted in Uncategorized
Leave a comment
Easy Introduction to AB Testing and P-Values
A version of this post was originally published over at Conversion XL For all of the talk about how awesome (and big, don’t forget big) Big data is, one of the favorite tools in the conversion optimization toolkit, AB Testing, is decidedly small data. Optimization, winners and losers, Lean this that or the other thing, at […]
Posted in Analytics, Testing and Data Science, Uncategorized
2 Comments
Predictive Targeting: Managing Complexity
Personalization, one to one, predictive targeting, whatever you call it. Serving the optimal digital experience for each customer is often touted as the pinnacle of digital marketing efficacy. But if predictive targeting is so great, why isn’t everyone doing it right now? The reason is that while targeting can be incredibility valuable, many in […]
Posted in Analytics, Testing and Data Science, Uncategorized
3 Comments
Big Data is Really About the Very Small
Awhile back I put together a fun list of the top 7 data scientists before there was Data Science. I got some great feedback on others that should be on the list (Tukey, Hopper, and even Florence Nightingale). In hindsight I probably should have also included Edgar Codd. While at IBM, Codd developed the relational […]
Posted in Uncategorized
Leave a comment
AB Testing: When Tests Collide
Normally, when we talk about AB Tests (standard or Bandit style), we tend to focus on things like the different test options, the reporting, the significance levels, etc. However, once we start implementing tests, especially at scale, it becomes clear that we need a way to manage how we assign users to each test. There […]
Posted in Analytics, Reporting, Testing and Data Science
Leave a comment
The World’s Top 7 Data Scientists before there was Data Science
I am often a bit late to the party and only recently saw Tim O’Reilly’s “The Worlds’ 7 most powerful Data Scientists”. As data science has become a big deal, there have been a several top data science lists that have been floating around. So for fun, I thought I would put together my own […]
Posted in Testing and Data Science, Uncategorized
10 Comments
Big Data or Big Distraction
Contrary to what you have heard, the unfolding technological transformation we are witnessing isn’t really about data, not directly at any rate. It’s not that data isn’t important, but the focus on data is obscuring the real nature of change, which is the transition from a world driven by essentially static and reactive systems to […]
Intelligent Agents: AB Testing, User Targeting, and Machine Learning
(Updated: Aug 2020) Whether you are in a large company running marketing campaigns, or in digital analytics, or a data scientist at a tech startup, you probably are on board with the importance of analytical decision-making. Go to any related conference, blog, meet up, industry slack and you will hear many of the following terms: […]
List of Machine Learning and Data Science Resources – Part 2
This is a follow up to a post to the list of Machine Learning and Data Sciences resources I put up a little while ago. This post contains some links to resources on clustering and Reinforcement Learning that I didn’t get to in the first post. Like the first one, it’s a bit haphazard, and […]
Posted in Testing and Data Science
1 Comment
Segmentation and Shrinkage
In our last post, we introduced the idea of shrinkage. In this post we are going to extend that idea to improve our results when we segment our data by customer. Often what we really want is to discover what digital experience is working best for each customer. A major problem is that as we segment […]
Posted in Uncategorized
Leave a comment
Prediction, Pooling, and Shrinkage
As some of you may have noticed, there are often little skirmishes that occasionally break out in digital testing and optimization. There are the AB test vs multi-armed bandits debate (both are good, depending on task), standard vs multivariate testing (same, both good), and the Frequentist vs. Bayesian testing argument (also, both good). In the […]
Posted in Uncategorized
Leave a comment
Easy Introduction to AB Testing and P-Values
A version of this post was originally published over at Conversion XL For all of the talk about how awesome (and big, don’t forget big) Big data is, one of the favorite tools in the conversion optimization toolkit, AB Testing, is decidedly small data. Optimization, winners and losers, Lean this that or the other thing, at […]
Posted in Analytics, Testing and Data Science, Uncategorized
2 Comments
Predictive Targeting: Managing Complexity
Personalization, one to one, predictive targeting, whatever you call it. Serving the optimal digital experience for each customer is often touted as the pinnacle of digital marketing efficacy. But if predictive targeting is so great, why isn’t everyone doing it right now? The reason is that while targeting can be incredibility valuable, many in […]
Posted in Analytics, Testing and Data Science, Uncategorized
3 Comments
Big Data is Really About the Very Small
Awhile back I put together a fun list of the top 7 data scientists before there was Data Science. I got some great feedback on others that should be on the list (Tukey, Hopper, and even Florence Nightingale). In hindsight I probably should have also included Edgar Codd. While at IBM, Codd developed the relational […]
Posted in Uncategorized
Leave a comment
AB Testing: When Tests Collide
Normally, when we talk about AB Tests (standard or Bandit style), we tend to focus on things like the different test options, the reporting, the significance levels, etc. However, once we start implementing tests, especially at scale, it becomes clear that we need a way to manage how we assign users to each test. There […]
Posted in Analytics, Reporting, Testing and Data Science
Leave a comment
The World’s Top 7 Data Scientists before there was Data Science
I am often a bit late to the party and only recently saw Tim O’Reilly’s “The Worlds’ 7 most powerful Data Scientists”. As data science has become a big deal, there have been a several top data science lists that have been floating around. So for fun, I thought I would put together my own […]
Posted in Testing and Data Science, Uncategorized
10 Comments
Big Data or Big Distraction
Contrary to what you have heard, the unfolding technological transformation we are witnessing isn’t really about data, not directly at any rate. It’s not that data isn’t important, but the focus on data is obscuring the real nature of change, which is the transition from a world driven by essentially static and reactive systems to […]
Intelligent Agents: AB Testing, User Targeting, and Machine Learning
(Updated: Aug 2020) Whether you are in a large company running marketing campaigns, or in digital analytics, or a data scientist at a tech startup, you probably are on board with the importance of analytical decision-making. Go to any related conference, blog, meet up, industry slack and you will hear many of the following terms: […]
List of Machine Learning and Data Science Resources – Part 2
This is a follow up to a post to the list of Machine Learning and Data Sciences resources I put up a little while ago. This post contains some links to resources on clustering and Reinforcement Learning that I didn’t get to in the first post. Like the first one, it’s a bit haphazard, and […]
Posted in Testing and Data Science
1 Comment
