What is Predictive Analytics Missing?

What is Predictive Analytics Missing

#Simulation! Oh sure, there are pockets of analysts using simulation in #predictive #analytics and pockets of companies that use the results, but the use of simulation to predict behavior (e.g., propensity to do something) is not widespread.

I am taking a break today from the “how to” and “X reasons why” articles and offer instead a little editorial contemplation.

Several industries rely heavily upon simulation models: Aeronautics, Aerospace, Defense, Transportation, Medical and Maritime for example. So, why do these industries use simulation, while the Financial and Insurance industries do not?

Advantages of using simulation

Simulation provides a method for checking your understanding of the world around you and helps you produce better results, faster. It is an important tool you can use to:

  • Predict the course and results of certain actions
  • Understand why observed events occur
  • Identify problem areas before implementation
  • Explore the effects of modifications
  • Confirm that all variables are known
  • Evaluate ideas and identify inefficiencies
  • Gain insight and stimulate creative thinking
  • Communicate the integrity and feasibility of your plans

What are the Insurance and Financial industries missing?

I see, read about, and create statistical models for predictive analytics in this realm. It is amazing there are so many people who will accept a predictive model based on internal or third party data. Even if they realize how bad their data really is, they’re still locked into statistical models. I know of at least two instances (two different companies) where their whole approach to predictive analytics is based upon time-series analysis—nothing else is considered.

So, your financial institution may be using time-series analysis solely to predict propensity, deploying the model and operating based on its results. Wow! What faith it takes to do something like that! There is very little understanding of why certain behaviors or events occur and very little insight into potential problems before implementation. And the data is blindly trusted—“it must be good since we collected it!”

What could they do differently?

Perhaps there is a fear of the unknown, and perhaps the unknown is anything outside of the realm of logistic regression or time-series analysis. This is of course what I do, so I am talking about impacting my livelihood. Moreover, I know, as George Box would echo if her were still with us, that my models are wrong! In order to get the closer to “right” (that is like trying to reach infinity), I know I need simulation. Simulate to ensure all the variables are known. Simulate to know the variables are reasonably distributed and experiment with alternatives. Simulate to gain insight into a particular propensity phenomenon. Simulate to…

We are doing just fine

The response to this might be that they already understand these phenomena well and they already know what variables are important. They probably did at one time. However, I know for sure my spending behavior, engaging behavior, borrowing behavior and so on, has been altered drastically by a struggling economy. “Oh, but our economy is healthy,” you might say. So then, why do products and services cost more and I get paid less. Ten years ago, a Senior Operations Research Analyst was worth $X thousand, but today he or she is worth $(X – Y) thousand, but that energy bill keeps rising without bound. Should we not even test the idea that the world has changed and what we used to know is now just a mystery? However, we are doing just fine, aren’t we?

The One-eye Man in the Kingdom of the Blind

You may have read a post by this title or at least seen the quote that this comes from, but for those of you who have no idea what I am talking about, here is an explanation. Kurt Vonnegut, author of Player Piano, said this in his book:

Almost nobody’s competent, Paul. It’s enough to make you cry to see how bad most people are at their jobs. If you can do a half-assed job of anything, you’re a one-eyed man in the kingdom of the blind.”

The Operations Research analyst should have one good eye and hence lead the blind to safety. I argued before about how OR analysts must have a holistic approach to problem solving, and that includes simulations. “Oh, but I have not delved into simulation before”, one might say. My response is, “what are you waiting for?” LPs, IPs, and MILPs (if you are an OR analyst you know what these are) are not going to solve every problem we face, but it seems to be the focus in many OR educational programs. Get out of your comfort zone and get busy being a holistic problem solver.

Break the rules!

Every OR analyst should have a little bit of rebel inside them. Our approaches must be novel in many cases and the rules that say regression models must only be broken if we are to be worth our weight in salt (since we are not on a gold standard). When our models are not producing much lift (or net lift), something may be broken. It could be the data or the preconceived notion of customer propensity. Simulation can help find the root causes, for you can simulate a system or phenomenon without ever touching the real, operational system.

A little at a time

That financial industry might not be ready to pour the weight of their analytics resources into simulation, and that is understandable. So, perhaps start with a pilot study, get some results and show them how simulation can help them. Imagine where we might be if the financial industry had been doing this kind of “What if” analysis in the early part of this century. Would we have seen the events of 2008-ish coming?

Will it crash?

We do not fly launch vehicles in space without simulation, nor launch test missiles without simulation. It is too expensive—in people and material—to launch and fail. In our early years of space exploration, we experienced many crashes before the first human was launched into space, and we have had some failures since then. But a lot of time, effort and money go into simulation for these programs. We used simulation before executing Operation Iraqi Freedom, and one senior officer was quoted saying that we had fought that battle over and over in simulation, prior to deploying, which made the fight look easy (even though there is no “easy” in warfare). Imagine how many lives were saved by simulation. Will it crash? Of course it will, but to simulate the crash is much better than experiencing the crash in real time.


I have thus far failed to impress my financial institution customer with the importance of simulation. Technically, I am a failure as an OR analyst. Sure, I have given my customer the product they asked for, but sometimes the customer does not know what they really need. It is incumbent on us to help them to ask the right questions, develop the best business cases, and use the correct solution method—and that may be simulation.

Jeffrey StricklandAuthored by:
Jeffrey Strickland, Ph.D.

Jeffrey Strickland, Ph.D., is the Author of “Predictive Analytics Using R” and a Senior Analytics Scientist with Clarity Solution Group. He has performed predictive modeling, simulation and analysis for the Department of Defense, NASA, the Missile Defense Agency, and the Financial and Insurance Industries for over 20 years. Jeff is a Certified Modeling and Simulation professional (CMSP) and an Associate Systems Engineering Professional. He has published nearly 200 blogs on LinkedIn, is also a frequently invited guest speaker and the author of 20 books including:

  • Discrete Event simulation using ExtendSim
  • Crime Analysis and Mapping
  • Missile Flight Simulation
  • Mathematical modeling of Warfare and Combat Phenomenon
  • Predictive Modeling and Analytics
  • Using Math to Defeat the Enemy
  • Verification and Validation for Modeling and Simulation
  • Simulation Conceptual Modeling
  • System Engineering Process and Practices
  • Weird Scientist: the Creators of Quantum Physics
  • Albert Einstein: No one expected me to lay a golden eggs
  • The Men of Manhattan: the Creators of the Nuclear Era
  • Fundamentals of Combat Modeling

Connect with Jeffrey Strickland
Contact Jeffrey Strickland

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s