Hey, we no longer need statistics! Cancel all college and university statistics programs…fire all the statistics faculty…shut down SAS…stop wasting time on R and Python…the end has come…there is a new beginning…ALL DATA.

Why do we no longer need Statistics?

I am sure you have heard of BIG DATA, but have you heard of ALL DATA? Probably not, because I think I may have just invented the term. Apparently there are some major players who actually believe that when they have all of the data, then they no longer have to take samples, and that statistics only applies to samples. Since we do not need to take samples anymore, then we do not need statistics anymore. By the way, Statisticians, we do not need you anymore. Go find a nice job as a greeter at Walmart, not that there is anything wrong with Walmart greeting—I kind of like the special touch.

What’s wrong with this Picture?

First, the CEO’s, CTO’s, and other O’s that are espousing this view absolutely have no earthy (or parallel universe) idea of what they are talking about. I would call them naïve, but they are really somewhere between “so full of themselves” and just plain dumb! I think their issue is the lack of understanding statistics, statistics anxiety, or a combination of the two. But, ALL DATA will take care of that, since it will not require any statistics.

Second, ALL DATA does not equal PERFECT DATA. I deal with humongous data sets. In my experience roughly 80% of the data is useless when building a model of some phenomenon, like the propensity to buy product X or the probability that the O-ring on component Y of the latest space launch vehicle will fail. Moreover, of the 80% that is useless, about half of that is really bad data. Of the twenty percent that is useable, about 10% of actually contains strong predictors. And this stuff is pretty easy to figure out using Weight of Evidence (WOE) and information Value (IV) algorithms. However I suppose ALL DATA morphed into ALL-PERFECT DATA!

Third, I think it is great that we have ALL-PERFECT DATA, but how are you going to process it? I know, you’ll just push a button to get a prediction. And inside that “black box” there are no statistics involved. There is no regressing, no averaging, no transforming, no imputing, on so on. The data is just speaking for itself. Oh wait, we have gone from ALL-PERFECT DATA to something much better, TALKING DATA.

Fourth, this implies we do not have to analyze the results. I suppose that is due to our new TALKING DATA. It speaks the results itself! No more histogram, means, variances, Skewness, etc., because we do not care. Our TALKING DATA will tell us what decisions to make. Wait a minute…if our TALKING DATA is making decisions, then it is really THINKING DATA. So now we do not need any decision makers! Fire all the CEO’s, CTO’s, and other O’s. Wait another minute! Who is going to implement the decisions made by our THINKING DATA? I know, it’s the MAGICAL DATA (probably in the form of unicorns). Wait another minute, if we have MAGICAL DATA, we do not need anyone else, for magic can do it all. Layoff everyone. No one ever need to work again, thanks to MIGICAL DATA!

Fifth, I do not really have a fifth so I am going to ask my BIG, ALL, PERFECT, TALKING, MAGICAL DATA for its opinion. What was that? Our UNICORN said, “all those people who think we do not need statistics are uninformed!” Now, do not get angry with me. I did not say that. It came for the UNICORN you created…

How do we fix the Picture?

That is an easy one, fire all the brainless CEO’s, CTO’s and other O’s who wouldn’t know a statistic if it slapped them in the face. But that would not work. Who would create the UNICORN?

Who wrote this Article?

Jeffrey Strickland believes in Unicorns and Martians. He has used BIG, ALL, PERFECT, TALKING, MAGICAL DATA (UNICORNS) to build space launch vehicles, missile defense systems, unmanned aerial vehicles, and some other really cool stuff. He has used UNICORNS to predict the outcome of battles, casualty rates, and cyber-attacks. He has also used UNICORNS to predict the propensity to purchase product X and to drop product Y. His next project with the UNICORN will be to precisely predict the weather for the next 150 years, four months, six days, eleven hours and 17 minutes. He might even publish his 21^st books on using the UNICORN and publish his 255th LinkedIn post. Wait, we do not need Jeffrey, because we have the UNICORN…

Connect with Jeffrey (or his Unicorn)

Authored by:
Jeffrey Strickland, Ph.D.

Jeffrey Strickland, Ph.D., is the Author of Predictive Analytics Using R and a Senior Analytics Scientist with Clarity Solution Group. He has performed predictive modeling, simulation and analysis for the Department of Defense, NASA, the Missile Defense Agency, and the Financial and Insurance Industries for over 20 years. Jeff is a Certified Modeling and Simulation professional (CMSP) and an Associate Systems Engineering Professional (ASEP). He has published nearly 200 blogs on LinkedIn, is also a frequently invited guest speaker and the author of 20 books including:

• Operations Research using Open-Source Tools
• Discrete Event simulation using ExtendSim
• Crime Analysis and Mapping
• Missile Flight Simulation
• Mathematical Modeling of Warfare and Combat Phenomenon
• Predictive Modeling and Analytics
• Using Math to Defeat the Enemy
• Verification and Validation for Modeling and Simulation
• Simulation Conceptual Modeling
• System Engineering Process and Practices

Connect with Jeffrey Strickland
Contact Jeffrey Strickland