While there's already talk of big-data disillusionment, research shows most companies are only in the early stages of adopting this far-reaching technology: Inflated expectations aimed for blurry goals and traveled unmapped territory.
Earlier this year, Gartner analyst Svetlana Sicular wrote that big-data is "falling into the trough of disillusionment." It's up to IT leaders to build bridges over this quagmire and prevent their organization from sliding into a ditch of their own making. The technologies are here. The tools are here. For big-data to work, you really need to work closely with your business peers, explain your requirements, and mutually agree on common goals, deadlines, and intentions.
Your letter might go something like this...
Dear Business Leader:
Like you, we in IT are very excited about big-data. We know this technology can empower our organization to become more competitive, more innovative and productive, and recognize that we have to be among the 81 percent of companies that include big-data and advanced analytics projects among our top five priorities for this year.
Even though there's a shortage of experts in big-data and analytics, we're doing our part. We're educating our staff and hiring new people expert in Hadoop, SQL, databases, and integration. But big-data success doesn't rest solely on technology. It takes commitment from everyone.
In a recent study by big-data cloud services provider Infochimps, many of the participating 300 IT department respondents said some element of "time" was the biggest issue surrounding big-data implementations. That is, companies didn't always allot enough hours, days, or weeks to components of their big-data implementation, whether it was doing recon pre-design, training end-users, integrating these data silos, or all of the above and more. Allowing adequate time upfront will save us countless dollars and frustrations later.
Many failed big-data projects don't succeed because they lacked data context and didn't connect the dots; in other words, there was a data-analysis talent gap, the survey found. We have a lot of information scattered throughout siloed repositories; without an accurate assessment, we cannot create a cohesive big-data solution that meets our organization's needs. We need to do that together.
As Jim Kaskade, CEO of Infochimps, said: "Companies need to start with the business problem first to properly scope their projects. Too many organizations are building big-data platforms intended to meet the entire organization's needs. Unless they understand specific use-cases first, many will find such an approach falls short."
The most critical elements of a big-data platform include: The ability to scale; ease of management; flexible architecture; speed to deployment; and security, Infochimps study found. As we work together to create our organization's big-data solution, we must keep all these criteria in mind.
It won't be an easy road. It won't be a short journey. But at the end of this trip, we'll have a big-data solution that will make all employees more productive, more informed, and more autonomous. So let's get it done together.
Your IT Team