Author Archives: Matt Gershoff

AB Testing: Ruling Out To Conclude

Seemingly simple ideas underpinning AB Testing are confusing. Rather than getting into the weeds around the definitions of p-values and significance, perhaps AB Testing might be easier to understand if we reframe it as a simple ruling out procedure. Ruling Out What? There are two things we are trying to rule out when we run […]

Posted in Uncategorized | Leave a comment

Some are Useful: AB Testing Programs

As AB testing becomes more commonplace, companies are moving beyond thinking about how to best run experiments to how to best set up and run experimentation programs. Unless the required time, effort, and expertise is invested into designing and running the AB Testing program, experimentation is unlikely to be useful. Interestingly, some of the best […]

Posted in Uncategorized | Leave a comment

Conductrics and ITP

What’s the impact on your Conductrics implementation? As you are likely aware, many browsers have begun to restrict the ability to “track” visitor behavior, in an effort to protect visitor privacy. The focus is especially on third-party scripts that could track users as they move between totally unrelated sites on the Internet. Apple’s efforts in […]

Posted in Uncategorized | Leave a comment

Conductrics Announces Search Discovery as a Premier Partner

December 1, 2020 – Austin, Texas –  Conductrics, a digital experimentation and artificial intelligence (AI) SaaS company, announces its partnership with Search Discovery, a premier data transformation company. Search Discovery will now offer Conductrics optimization technology along with industry-leading optimization consulting support. Together, the two companies will offer clients a superior integrated solution that satisfies […]

Posted in Uncategorized | Leave a comment

Video: Conductrics Platform Overview

Nate Weiss, Conductrics CTO, demonstrates key new features and functionalities of the latest software release.  This latest release focuses on streamlining and improving the user experience for faster and easier execution of optimization programs. The new features that are demonstrated include:  A revamped and simplified user experience Streamlined A/B Testing/Optimization workflows Improved Program Management tools […]

Posted in Uncategorized | Leave a comment

Conductrics Announces Updated Release of its SaaS Experimentation Platform

 SEPTEMBER 24, 2020 – AUSTIN, TX – Digital experimentation and Artificial Intelligence (AI) SaaS company Conductrics, today announced its latest major release of its experience optimization platform built expressly for marketers, developers, and IT professionals. The updated platform, which shares the company’s name, is a cloud-based A/B testing and decision optimization engine. This latest release […]

Posted in Uncategorized | Leave a comment

Headline Optimization at Scale: Conductrics Macros

Conductrics has a history of providing innovative approaches to experimentation, testing and optimization.  In 2010, we introduced one of the industry’s first REST API for experimentation which also supported multi-arm bandits and predictive targeting.  Continuing our goal to provide innovative solutions, we’ve developed Express Macros and Templates to safely and easily build tests at scale. […]

Posted in Uncategorized | Leave a comment

Getting Past Statistical Significance: Foundations of AB Testing and Experimentation

How often is AB Testing reduced to the following question:  ‘what sample size do I need to reach statistical significance for my AB Test?’  On the face of it, this question sounds reasonable. However, unless you know why you want to run a test at particular significance level, or what the relationship is between sample […]

Posted in Uncategorized | Leave a comment

What is the value of my AB Testing Program?

Occasionally we are asked by companies how they should best assess the value of running their AB testing programs. I thought it might be useful to put down in writing some of the points to consider if you find yourselves asked this question. With respect to hypothesis tests, there are two main sources of value: […]

Posted in Uncategorized | Leave a comment

Do No Harm or AB Testing without P-Values

A few weeks ago I was talking with Kelly Wortham during her excellent AB Testing webinar series.  During the conversation, one of the attendees asked if they just wanted to pick between A and B, did they really need to run standard significance tests at a 90% or 95% confidence levels? The simple answer is […]

Posted in Uncategorized | 5 Comments