Data lakes didn’t quite pan out so now it’s all about an abstraction layer and machine learning to save the day. Hopefully, machine learning can cleanse your data on the fly since humans have proved repeatedly they aren’t meticulous enough.
Will there ever be a technology that can fix decades of poor data hygiene? Probably not, but that isn’t going to stop technology vendors from trying. The good news: Machine learning may come closest to saving your data management hide. Data hygiene isn’t easy. You can’t hire enough interns to even come close to rectifying past mistakes. The reality is enterprises haven’t been creating data dictionaries, meta data and clean information for years.
Author: Larry Dignan