Hype Vs Reality in Big Data
“We ourselves sometimes speak about ‘Big Data solutions,’ but in reality what our Stratecast Big Data & Analytics practice is about are data management solutions, some of which are beginning to effectively manage Big Data. And, contrary to what is fast becoming conventional wisdom, some organizations are in fact deploying data management solutions addressing Big Data. However, especially noteworthy given the benefits that harnessing Big Data has to offer, the rate of adoption is low. One reason is a lack of clarity in the value proposition: too many providers and hopeful solution buyers are not making the case effectively for why those who sign the checks should invest.
“Deploying ‘the right stuff’ and not just‘more stuff’ alsoenabled the retailer to slash its IT footprint,and as a result, it is now saving more than $1.4 million annually”
The benefits include the opportunity to create an agile, data-driven organization; leveraging data to support KPIs; and transforming fear of Big Data into employee empowerment, offering the insights people need, when they need them, to make smarter decisions. All of that translates directly into revenue generation and retention, and while vendors obviously hope to capitalize on this by selling more products, we see examples all the time where companies actually save money by deploying a Big Data solution. For example, a bricks-and-mortar mega retailer in Europe just markedly enhanced its competitive position with Big Data-driven real-time analytics. Deploying ‘the right stuff’ and not just ‘more stuff’ also enabled the retailer to slash its IT footprint, and as a result, it is now saving more than $1.4 million annually—while understanding its business better than ever before, and responding faster to the needs of its customers. “Stratecast estimates the market for Big Data, analytics, and Business Intelligence (BI) solutions at $25 billion for 2013, increasing at a Compound Annual Growth Rate (CAGR) of 12.7 percent to $40 billion by 2017.”
Developing Big DataCapabilities In-House Vs.Outsourcing
“While some organizations are certainly crafting their own Big Data systems, they are often better served by engaging a professional team to either provide a Big Data solution outright, or, at minimum, work with the organization on the initiative. This can easily cost the organization less than doing the job in-house. We urge buyers to consider the following direct, indirect, and opportunity costs in constructing a business case:
• “Failure to capture missioncritical data–with a poorly planned Big Data implementation, the business will not capture all relevant data to compete effectively in the marketplace and retain existing customers. This can render the business vulnerable to competitors, leading to missed opportunities, lost revenue and higher churn. For the business case analysis, calculate the likely and potential top line and opportunity costs of continuing to operate in the absence of complete data.
• “Taking multiple ‘shots’ at Big Data can delay implementation– implementing a Big Data system is a massive undertaking that touches every area of the business; and the downstream impact of any delay is magnified. From decades of experience optimizing IT infrastructures, and more recent experience implementing Big Data solutions, expert consultants understand where the vulnerabilities and risks are, and plan accordingly. With proper upfront planning comes lower exposure to risk, decreased chance of unexpected disruptions, on-time project completion, and an agile business that seizes new opportunities instead of missing market windows.
• “Resource drain on internal team– most IT and Data Science teams are under pressure to maintain daily operations and incorporate new business-enhancing technologies, without increasing staff. By hiring a team of experts, they can keep internal staff doing the things they need to do and avoid the costs of delaying or deferring essential tasks. • “Implementing Big Data architecture confines costs to hardware–while merely adding servers will not manage Big Data, it is sometimes necessary to add servers when implementing a Big Data infrastructure. Even so, if the organization has worked with an experienced professional service team to incorporate the Big Data Architecture elements in Stratecast’s Data Management Model, the cost of the servers themselves should be the only incremental cost.
Making Big Data an Integral Part of Business Strategy
SMBs and below should have their heads in the clouds. Where ‘Big Data for the rich’ may leave midsized companies is in cloud-based Big Data solutions. These solutions can turn the embedded/legacy IT model on its head, and help companies do more with less. Reducing the IT footprint of legacy technology reduces both Capital Expenditures (CAPEX: upfront investment) and ongoing Operating Expenses (OPEX). Some providers who are providing more affordable solutions, some entirely cloudbased, some premisesbased, and some a hybrid of technologies, include Birst, GoodData, Starcounter, GridGain, Platfora, and GigaSpaces. Do they have huge market share at present? Compared to the market leaders— No. Yet Birst has more than 1,000 customers including Citrix, YMCA, and CBS Interactive; Starcounter has 80 large customers including Hyundai, Telia, Canal+, European mega-retailer Gekas Ullared, and Verizon Wireless; and the others in this grouping also have robust customer bases.
Another challenge to the growth of this market is the ‘IT glass walls’ phenomenon: at present, it still requires too much specialized expertise, in the form of the data scientist or CIO and team, for business users to quickly get the data they need. The cloud-based data management providers and other innovators in this space are making headway by providing solutions that are not only affordable but also geared to enabling business users to get the insights they need without having to beg IT for information.
Seriously consider Hadoop—but get all the Hadoop you need. We hear too many companies in effect saying, ‘I’d like to order one Hadoop, please,’ as if that is going to solve the Big Data riddle for them. It is certainly not an exact analogy, but I liken Hadoop to my Firefox browser: I love it, but only with my favorite add-ons like Tab Mix Plus, FireShot, Google Translate, QuickTime, and Adobe Shockwave. Similarly with Hadoop, to get the most out of your data management solution you need modules that have been developed by other Apache Software Foundation working groups: modules such as Pig to accommodate semistructured data and simplify various tasks. Hive to use Hadoop as an Enterprise Data Warehouse (EDW). Sqoop and Flume to import Web log data into a Hadoop Distributed File System (HDFS), and to integrate structured data into the mix. Ambarri, Whirr, Zookeeper, and Ozzie to accelerate deployment, simplify ongoing management, manage workflow, and maintain data synchronization.
