Databases are powerful abstractions that allow people and programs to manipulate large amounts of data in a predictable, repeatable way. There’s a good reason why SQL is so important to modern data processing. But in some circumstances, a database is the wrong tool for the job. That was definitely the case at LinkedIn, says Apache Kafka co-creator Jun Rao.

According to Rao’s keynote speech at Kafka Summit yesterday, LinkedIn‘s data architecture in the early 2010s didn’t differ much from the typical enterprise. There was a hefty dose of relational databases and SQL, two technologies that had proven themselves in countless deployments over the past two decades. “The initial architecture for data in LinkedIn was pretty standard, in SQL,” Rao told a couple of thousand Kafka Summit attendees Monday morning. “In the middle was a centralized database. That’s actually where LinkedIn put all its data.”

Source: datanami.com
Author: Alex Woodie