Most everyone knows that using a propensity model to predict customers’ likelihood of buying, engaging, leaving and so on, produces a larger target audience. That is too bad, for what everyone knows is incorrect.
A Field of Dreams
In the movie Field of Dreams, actor Kevin Costner (whom people often confuse me with), plays an Iowa corn farmer struggling to pay the mortgage on his homestead. He is out farming one day and hears a voice like a loud whisper in the wind:
If you build it, they will come.
And in the end he does build it and they do come.
Models are not like that. In fact, they reduce the size of the customer population that has a high propensity to do whatever. If you heard a voice in this context it would say:
If you build it, fewer will come.
Wow, that is really bad! So why do we use models?
Actually, it is not bad—that is what we want models to do. But you must think I am outside my mind for saying so. However, modeling the propensity of customers to buy, engage, etc. is like prospecting for gold. There is a lot of rock and sand that has to be filtered out to get the “pure” gold dust.
A rocker box (also known as a cradle) is a gold mining implement for separating alluvial placer gold from sand and gravel which was used in placer mining in the 19th century. It consists of a high-sided box, which is open on one end and on top, and was placed on rockers.
The inside bottom of the box is lined fles (usually glass or plastic) and a usually a carpet similar to a sluice. On top of the box is a classifier sieve or grizzly (usually with half-inch or quarter-inch openings) which screens-out larger pieces of rock and other material, allowing only finer sand and gravel through. Between the sieve and the lower sluice section is a baffle, which acts as another trap for fine gold and also ensures that the aggregate material being processed is evenly distributed before it enters the sluice section. It sits at an angle and points towards the closed back of the box. Traditionally, the baffle consisted of a flexible apron or “blanket riffle” made of canvas or a similar material, which had a sag of about an inch and a half in the center, to act as a collection pocket for fine gold. Later rockers (including most modern ones) dispensed with the flexible apron and used a pair of solid wood or metal baffle boards. These are sometimes covered with carpet to trap fine gold. The entire device sits on rockers at a slight gradient, which allows it to be rocked side to side.
A propensity model is like a rocker box: what comes out with a high propensity score is your gold dust customers—the ones that are rich with propensity to do whatever it is that you are modeling. The debris scores low in the model. If you have 10,000,000 customers, chances are that only 10,000 (or fewer) have a high propensity to buy product X. But those customers are your gold dust. I also use the analogy of filtered funnel, where the top has a large opening and the bottom a much smaller one, and where my “gold dust” customers are small enough to pass through the tiny funnel opening at the bottom. Many go in, but few come out.
It is incumbent on us modelers to manage the expectations of our customers. Though quite a few already understand this model phenomenon, we must establish with the customer at the beginning of a model project, even before we think about data, model functional form and so on, what to anticipate as an outcome.
So, the voice that said, “If you build it they will come”, was not talking to me (even though I strongly resemble Kevin Costner). Instead, my voice said, “If you build it, fewer will come.”
Jeffrey Strickland, Ph.D.
Jeffrey Strickland, Ph.D., is the Author of “Predictive Analytics Using R” and a Senior Analytics Scientist with Clarity Solution Group. He has performed predictive modeling, simulation and analysis for the Department of Defense, NASA, the Missile Defense Agency, and the Financial and Insurance Industries for over 20 years. Jeff is a Certified Modeling and Simulation professional (CMSP) and an Associate Systems Engineering Professional. He has published nearly 200 blogs on LinkedIn, is also a frequently invited guest speaker and the author of 20 books including:
- Discrete Event simulation using ExtendSim
- Crime Analysis and Mapping
- Missile Flight Simulation
- Mathematical modeling of Warfare and Combat Phenomenon
- Predictive Modeling and Analytics
- Using Math to Defeat the Enemy
- Verification and Validation for Modeling and Simulation
- Simulation Conceptual Modeling
- System Engineering Process and Practices
- Weird Scientist: the Creators of Quantum Physics
- Albert Einstein: No one expected me to lay a golden eggs
- The Men of Manhattan: the Creators of the Nuclear Era
- Fundamentals of Combat Modeling