The war drums have long been beating for elimination of data silos, with big data, big wigs like Ben Porterfield, CEO of Looker, and Edd Wilder-James, Google’s Open Source Program Lead, espousing the importance of democratizing data in order to gain a “real and essential competitive advantage” in today’s digitized economy. Despite the concerted united front against the data silo scourge, it does not appear that business leaders are taking heed, with only 2% of organizations claiming to be “completely effective at data sharing,” per a recent DZone survey. So, what’s the hold up? Well, like any with any information technology challenge, there are pros and cons to change, and in this blog post we’ll analyze both sides of the coin.
THE GOOD
Silos are cheap (in the short run). Eliminating them, however, can get expensive, as companies will usually need to get their disparate systems to speak to one another via API development. Thumbtack estimates the average hourly cost of API Development to be between $75.00 and $150.00 USD, and with an integration project with a major platform like Amazon MWS estimated to take north of 50 hours, it’s easy to see how these initiatives can quickly turn into a money pit.
Jeff Sauer, Professor of Data Driven Marketing at St. Thomas University, once wrote, “for every $20 you spend on web analytics tools, you should spend $80 on the brains to make sense of the data.” This statement belies the fact that the cost of eliminating data silos to make data accessible across your enterprise is really only the tip of the iceberg. For a business to truly democratize their data, they must also invest in training programs to educate “lay” employees on how to use analytics tools. This is a serious outlay.
Beyond cost, there are also security implications to eliminating data silos. Imagine, for instance, that finance team’s data suddenly becomes accessible to marketing team members. Without role-based access control systems in place, the scenario could easily arise where a digital marketing is inadvertently employing sensitive data to build an ad audience. In a haphazard way, silos often act as natural security barriers, and preserve data integrity by walling it off from general access. It’s therefore critical that data governance strategy is top-of-mind when considering the elimination of data silos.
THE BAD
Inherently, data quality fluctuates across departments within a large enterprise, which why businesses implement data quality management (DQM) programs to enforce a defined standard. The existence of data silos within an enterprise, however, presents a significant barrier to achieving effective data stewardship, as they cause “redundant and unsynchronzied” data sets to proliferate, uncontrollably.
If it becomes impossible to achieve some level of data quality consistency across an enterprise, certain business units may become stranded with bad data. Stating the obvious: bad data leads to ill-informed business decisions, which ultimately impacts the broader organization’s bottom line.
Another side of effect of data silos is that they limit the ability of companies to use AI for deep learning initiatives, as silos can impair algorithms from discovering patterns and correlations across massive data sets.
THE UGLY
The most serious consequence of retaining data silos is that they reinforce silo mentalities. A corporate culture in which employees are reluctant to share information and collaborate, often leads to operational inefficiencies, and in some cases a toxic work environment. The ability to share data streams across corporate functions is within itself a move towards workplace transparency and trust. By eliminating data silos, businesses are empowered to develop a unified vision that all departments can work and aspire towards. When data is shared, success can also be.