Posts Tagged analytics
While the buzz in the big data market is around massive amounts of data and what you can do with that data, in truth companies are more interested in the small stuff. Let me explain. One of the greatest benefits of getting hold of lots of data is that a company can see patterns and anomalies that would have gone undetected with a smaller set of data.
Companies are interested in big data not so much because of what that data represents in terms of volume, variety and velocity, but what the analysis of that data can do for the business. As business decision makers begin to recognize the potential of big data, they realize that the biggest insights and hidden gems of knowledge are found when big data actually becomes quite small. However, the analysis must start with very large volumes of data. Exploring the world of big data takes business leaders beyond the data found in their organization’s traditional databases. Researchers, data scientists, and marketers, are analyzing very large volumes of unstructured data from previously untapped sources such as e-mails, customer service records, sensor data, and security logs.
Now that companies have the ability to deal with so much more data, they are free to incorporate a greater variety of data into the mix. For example, they are incorporating social media, mobile phone location, traffic, and weather data into more traditional analysis. But what is the goal? Add more data and more elements so that those patterns emerge. Armed with the insight from analysis, business leaders can take a more precise set of data and compare it to data from a data warehouse or a system of record. As a result, the research becomes more targeted and directed to fit in with the context of the business.
Why do companies need to take this targeted approach to leveraging big data? In essence, companies want to use big data analysis to make a personalized offer that is just right for the customer when that customer is ready to buy. In some cases, the answer may be related not to selling but to diagnosing problems with a manufacturing system or a patient with an unexplained illness.
How do you move from big data analysis to small data insights and personalized action? You need to consider three elements: defining your business problem, defining and analyzing your data sources, and integrating and incorporating your big data analysis with your operational data.
Defining your business problem. Companies are beginning to ask the traditional questions about customers, products, and partners in new ways. For example, are you looking to manage your customer interactions armed with in-depth and customized knowledge about each individual customer? Companies with a focus on driving continuous improvement in customer service are asking, “How can I delight this customer and anticipate their specific needs?” These are important goals for businesses competing in today’s fast-paced, mobile-driven market.
Defining and analyzing your data sources. What information do you need to make the right offer to your buyer when he deciding on a purchase? What information can you glean from outside sources such as social media data? What big data sources do you have available internally that were previously underutilized? For example, can you use text analytics to gain new insight about customers from call center notes, emails, and voice recordings? One important goal with big data analytics is to look for patterns and relationships that apply to your business and narrow down the data set based on business context. Your big data analysis will help you find the small treasures of information in your big data.
Integrating and incorporating the analysis of you big data with your operational data.
After your big data analysis is complete, you need an approach that will allow you to integrate or incorporate the results of your big data analysis into your business process and real-time business actions. This will require some adjustment to the conventional notion of data integration. In order to bring your big data environments and enterprise data environments together, you will need to incorporate new methods of integration that support Hadoop and other nontraditional big data environments. In addition, if you want to incorporate the results of very fast streaming data into your business process, you will need advanced technology that enables you to make decisions in real-time.
Ultimately, if you want to make good decisions based on the results of your big data analysis you need to deliver information at the right time and with the right context. In order to make the results of your analysis actionable, you need to focus more on the small – targeted and personalized results of big data – than on the large data volumes.
I am looking forward to attending The Smart Governance Forum (23rd meeting of the IBM Data Governance Council) in California on February 1-3, where I will be a panelist for a session on Smart Governance Analytics. As my panel group started to plan for the event, I did some background research on the Council to understand more about them. What kinds of questions were Council members asking about information governance when they began meeting in 2004 and how are things different today? Have they developed best practices that would be useful to other companies working to develop an information governance strategy?
Information governance refers to the methods, policies, and technology that your business deploys to ensure the quality, completeness, and safety of its information. Your approach to information governance must align with the policies, standards, regulations, and laws that you are legally required to follow. When a group of senior executives responsible for information security, risk, and compliance at IBM customer organizations began meeting in 2004, interest in IT governance was high, but there wasn’t as much attention focused specifically on information governance.
Books like “IT Governance: How Top Performers Manage IT Decision Rights for Superior Results” by Peter Weill and Jeanne W Ross helped companies understand the benefit of aligning IT goals with the overall goals and objectives of the business. In addition, there were other publications at this time focused on how to take a balanced scorecard approach to managing business strategy and on best practices for implementing IT governance. These approaches are of critical importance to business success, however there was also a need to develop a framework for understanding, monitoring, and securing the rapidly increasing supply of business data and content.
And that is what a group of IT information focused business leaders and IBM and business partner technology leaders decided to do. The amount of data they needed to collect, aggregate, process, analyze, share, change, store, and retire was growing larger every day. In addition to data stored in traditional data bases and packaged applications like CRM (customer relationship management) systems, they were also concerned about information stored and shared in unstructured formats like documents, spreadsheets, and email.
Having more information about your companies customers, partners, and products creates great opportunity, but more information also means more risk if you don’t manage your information with care. Council members asked each other lots of questions such as:
- How can we be sure that the right people get access to the right information at the right time?
- How can we make sure that the wrong people do not get access to our private information at any time?
- How can we overcome the risks to data quality, consistency, and security increased by the siloed approach to business data ownership that is so prevalent in our organizations?
- How can we create a benchmarking tool for information governance that will help our businesses to increase revenue, lower costs, and reduce risks?
- How can improve our ability to meet the security and protection standards of auditors and regulators?
As a result of its discussions, The Council developed a Maturity Model to help you assess your current state of information governance and provide guidance for developing a roadmap for the future. The Model identifies 11 categories of information governance. The categories cover all the different elements of building an information security strategy such as understanding who in the business/IT is responsible for what information, what policies do you follow to control the flow of your information in your company, what are your methodologies for identifying and mitigating risk, and how do you measure the value of your data and the effectiveness of governance. I read two IBM White Paper’s on the Model that add insight to the questions you need to ask to begin building a path to better information governance, “The IBM Data Governance Council Maturity Model: Building a roadmap for effective data governance” and “The IBM data governance blueprint: Leveraging best practices and proven technologies“.
So, what’s changed? FInancial crises, increasing regulation, high-profile incidents of stolen private data, cloud technology, and other factors have added substance and complexity to the questions you need to ask about information governance. There is much to do. One question we will explore at the conference next week is, How do you measure the effectiveness of your information governance strategy and what analytical measures are appropriate? For example, some companies are using analytical tools to look for patterns of email communication across the company and discover a greater level of insight into how information is flowing and what needs more review. Look for more on analytics and governance after the conference.
With all of the acquisitions happening in the business intelligence space, customers are in a state of confusion. What do all of these changes mean — both short term and long term? In my view, it is best to take a pragmatic approach to this changing market landscape. This acquisition spree in BI is impacting as many as 80,000 customers who use business intelligence software from Hyperion, Business Objects, or Cognos. What should customers be asking their vendors about the future directions of their products? All three BI vendors have either just been acquired or are soon to be acquired by three large IT companies – Oracle, SAP, and IBM, respectively. Is it business as usual or do the IT managers need to alter their business relationships and plans for business intelligence implementations now that a wave of consolidation is underway?
Customer concerns about the re-alignment in the business intelligence area tend to fall into three categories: impact on legacy environments, impact on future buying decisions, and technical innovation. Management issues will vary with how business intelligence software has been deployed at their companies and how well integrated this software has been with other information management software.
Legacy environments. Many companies tend to use BI tools for traditional management reporting. In general, they like to stick with what they know and what their users are trained to use. Therefore, they want to be assured that they can continue using the familiar reporting tools in the same way. While most managers assume that once an acquisition is complete, there will be new operating efficiencies. What they don’t always know is whether those efficiencies will translate to savings. They also would like to understand how the company might benefit from the fact that a larger company with more resources has bought the company they have been dealing with. How will my company benefit from the acquisition? Will there be any changes in the sales and service teams? What about pricing issues and planned upgrades?
Typically, it will take a while before changes are evident. For example, if you are getting good value from your use of Crystal Reports—the Business Objects’ reporting solution designed to support business modeling, analysis, and decision making – it really may not matter to you which company purchased Business Objects.
Ironically, some of the most important issues that might emerge happen when customers merge with each other. For example, one company might have standardized on Cognos for its analysis and reporting while the company being acquired uses Business Objects as a standard. Since large enterprises rely on business intelligence software to gain insight into production, sales, revenue, or other data across divisions and subsidiaries, too many tools may make decision making more difficult.
It is interesting that these acquisitions are hitting the market at the same time that companies are trying to move from a departmental view of data to an enterprise perspective. A unified and standardized approach to information management across the enterprise is becoming a top priority. Companies that have accumulated many different BI vendor software may use this time of change to re-evaluate a BI strategy.
Typically companies are used to managing multiple software components from multiple vendors. The expectation is that the consolidation of two or more of a single vendor will lead to benefits resulting from the tighter integration of the products. This is often the best outcome in terms of support, training, and management. However, this is typically a multi-year effort by the vendors building a unified portfolio based on acquired software. As businesses move from a traditional siloed single purpose data warehouse to information integration and analytics, having one vendor to call is often a welcome change as companies try to simplify the management of its infrastructure.
Impact on Future Buying Decisions
The situation may be a little different if a customer is in the middle of a proof of concept (POC) for a project designed to develop a single view of a company’s customer base. How will the recent acquisitions in the business intelligence market impact how businesses to move forward?
Consider the example of a large bank that had recently made a significant acquisition. The bank wanted to understand its most profitable customers across the newly merged company. Customer history and sales data was retained in a siloed manner by line of business and there was no integration between the data for the two companies. This company used Cognos for many of their executive level reports, but now management had raised concerns about data quality.
In order to develop a single view of customer they needed to look for incompatibilities and inconsistencies in the disparate data sources. These data sources needed to be integrated and IT needed to assure the business that the information was accurate, complete, and trust worthy. The bank selected Informatica to provide the software needed to help with the integration and to improve the quality of its data.
Informatica has a comprehensive solution for data integration and data quality. In addition, just a few months ago Informatica and Cognos announced an expansion of their strategic relationship. This partnership fit well into the CIO’s priority to consolidate all the many disconnected vendors in use in IT and to ensure that the integrations between different applications are as tightly integrated as possible. Now, suddenly Cognos is part of IBM and a direct competitor to Informatica. What should the CIO do? Certainly IBM is committed to supporting all of Cognos’ existing partnerships. You should not have to change your plans because of the acquisition, however there are questions to ask about how things will change in the future.
All the BI vendors mentioned above have well-established partner relationships with emerging information management software companies. There has been a lot of customer support for the creation of tighter integrations between the information management infrastructure and the business intelligence layer. Companies have recognized that the reporting structure becomes meaningless if the supporting data cannot be trusted.
The partnerships have been important because much of the important innovation comes from small emerging companies. Partnerships with major players makes it easier for the company to leverage innovation with lower risk. The established players can provide the integration with the analytics and reporting technology. As companies attempt to unlock much of the data that has been previously unreachable at the enterprise level, it will become much more important to have a unified approach to information quality, information integration, and business intelligence. Market consolidation will help ensure that innovation is better utilized in a predictable manner.