You say you already have a plan in place to guard your company’s data? Are you sure it has you adequately protected? While you certainly understand the need for data security – your sales challenges are tough enough without exposing your customer’s credit card information to a security breech, for example – the chances are good that in 2010 you will consider various options for improving the security of your data.  If you are going to protect your company’s most valuable asset — your data —  you will begin to view data security as a component of a more comprehensive information governance strategy.

The risks of internal or external threats to your company’s data are becoming more complex as the depth and breadth of your information expands rapidly and your data is shared with business partners, suppliers, and customers.  In addition, as companies begin to take advantage of cloud services for some of their workloads, additional complexity is added to the multitude of security concerns. Many companies have deployed a disjointed approach to securing, controlling, and managing its data making it hard to anticipate and prepare for constantly changing security risks. There are lots of different ways that unauthorized users may enter your network or otherwise steal your data.  Many companies typically have a distinct solution to combat each one individually and typcially can’t each of themprotect against all of them and. For example, access control, data encryption, network traffic monitoring, vulnerability testing, and auditing may all be monitored with independent applications.

There is a good reason why many companies find they need to deploy lots of different solutions to effectively govern its information. Some of the most innovative solutions have come from emerging companies who have built a niche around a particular vertical market or some segment of the information security market. So you deploy the best solution for you biggest challenges and move on. However, as you begin to think more holistically about your needs for information governance, you will want to ensure that information security solutions are well integrated. This is one reason why emerging companies with an information security solution have become desirable acquisition candidates for larger software vendors.

Guardium, a privately-held company based in Massachusetts, is one of the most recent examples of this trend. When IBM announced its acquisition of the company in the last week of November, Guardium moved from a fast growing startup to one of the pillars of the IBM information governance strategy.The company’s technology helps clients with some of the most challenging issues around unauthorized access to critical data. Their solutions provide secure access to enterprise data – across many different database environments such as IBM, Oracle, Microsoft, Teradata and others.  In addition, customers can reduce operational costs by automating regulatory compliance tasks. While many companies may have the ability to monitor one database at a time, Guardium brings added value by enabling companies with complex environments to monitor databases across their organization.

This acquisition aligns well with IBM’s strategy to provide customers with a well-integrated and comprehensive approach to information management. IBM has spent in the range of $12 Billion over the past five years to add software assets that will help companies to make more intelligent decisions and realize more business value from their information.

After just returning from IBM’s Information on Demand (IOD) Conference in Las Vegas, I would like to take this opportunity to virtually whisper just one word in the ear of a current day Benjamin Braddock, “analytics”. Many businesses have spent the past 25 years or so automating and streamlining business processes in order to drive improvements in efficiency and productivity.  But now, it is becoming apparent that these businesses expect their future success will increasingly depend on how skillfully they manage, govern, and analyze information. Businesses are applying analytical techniques to business information to help reduce risk and increase the certainty that they are making the right decisions.

IBM has, in fact, spent $12 Billion in software investments (both organic multiple acquisitions like SPSS, Cognos, Filenet, iPhrase, and Ascential Software, just to name a few) over the past 4-5 years to ensure it will be able to support its customers in their quest to unlock the business value of information. In addition, in April of 2009 IBM announced a new organization comprised of 4000 consultants focused on advanced business analytics and business optimization – teams with skills in applying business intelligence technologies like mathematical modeling, simulation, data analytics, and optimization techniques.

In an era of intense competition, tight credit, and cost concerns across global and vertical markets, this focus on getting the most value from the information you have makes a lot of sense. Companies find they are processing more information than ever before, but less of this information is being accurately and adequately used.  The quantity of available data that a business needs to manage and understand has skyrocketed along with the increase in instrumented and intelligent products. For example, RFID tags that are embedded in manufactured products,  plants and animals generate an enormous amount of data in efforts to control inventories and improve security and safety.  Trying to make decisions with inadequate,  inaccurate, or untimely  information is like driving a fast sports car down the highway with a very large blind spot impeding your view of the truck approaching on your side. You need to know about the obstacles that might appear in  your pathway before you try to make a “real-time” correction and steer your car (or your business) of a cliff.  So, students and business leaders alike please take note, I see some “analytics” in your future.

If you are a software developer, the chances are pretty good that you’ve across  IBM’s developer Works. With approximately 8 million registered users this site has received a lot of use by developers looking for resources on coding, standards, and technical details on  software languages like Java or  applications like Lotus Notes.  Lots of great information, but let’s face it –traditional web environments  feel a little static once you begin to use social networking sites like Facebook or Twitter.   This is why I was interested to learn about IBM’’s recent introduction of My developerWorks – a transformation of developer Works that incorporates many of the attributes of social networking.

Why is this important for developers? Technology is changing too fast and the pace of the work environment it too intense for most developers to gain new skills quickly. You can research and learn a lot of material on your own, but you can get a lot smarter and produce higher quality results if you have an effective way of learning from others and collaborating across teams.   Given the economic realities of 2009, many software developers recognize that time spent keeping up with emerging technologies and understanding the business value of IT projects may make a real difference in meeting tight deadlines at work, getting a promotion or even landing a new job. These software developers often gain a lot of their information by following bloggers in their special area of interest or blog themselves to reach out to their peers. They use online reference sources, Linked In, Facebook, and Twitter to find solutions to a tough problem and keep in touch with colleagues.  My developer Works was designed to enable developers to  incorporate many of these resources and tools into one place.

Social networking is changing the way people meet, interact, and share information.  That doesn’t mean you can always make the right connections and get the right  information  in your specific technical area.   You may know you need help with a problem, but you don’t know where to find  the answer.  You may expect that somewhere in your own large company there is an expert with the expert knowledge to help you out, but it take s a week to find him because he is located in another continent.  My developer works is designed to help developers find the technical communities that will help them speed up this process. These are some of the things  that I think developers will like about My developer Works:

• The ability to create your own view by adding  feeds for favorite bloggers and online forums
• Easy ways to identify and find subject matter experts using tools  like virtual business cards
• New ways to share ideas and project details with  work teams
• Use of  keywords and tagging to locate people to help with your research and skill development

My expectation is that the true benefits of My develop Works  to developers will go beyond having one coordinated portal for  research  and blogging or twitter feeds. The unique dynamics of social software enable people to gain value from non-predictable events. It is hard to predict the right formula or set of circumstances that will lead to innovation. You sometimes get great ideas from following unplanned threads or from pulling together information across many different environments. Developers should be able to use My developerWorks to more easily locate  the right circle of  industry experts and business colleagues to help foster personal growth and innovation.

In a fast-paced work environment, time  often feels like your most limited resource. The business needs to get products to market faster and there is pressure on IT to deliver more efficient solutions in shorter time frames, and with  lower budgets.  We often needs answers faster than we can even think of the question. MyDeveloper Works is designed to help developers to become more productive at their jobs and have more fun doing it.

I attended the HP Analyst Meeting in Boston a few weeks ago and had several discussions with the business intelligence (BI) group. It is clear to me that HP is struggling to try to figure out the best way to sell in this type of down market. Obviously, this isn’t easy for anyone.  HP’s approach to solving this problem for its BI, data warehousing, and analytics solutions was to create a BI solutions group consisting of consulting (based on the 2006 Knightsbridge acquisition) and technology (Neoview, an integrated hardware and software platform for enterprise data warehousing).  One of the best articulations of the approach HP has adopted came from a discussion by a sales executive who is on the front lines of trying to convince customers to part with cash. What struck me was the coordinated effort that was necessary to sell to a large global organization with huge data management challenges.  This got me thinking about what it takes to sell in a very tough market.

To be successful, this sales person went all out. The Neoview product team and BI consultants all pulled to together to provide the right solution for the customer. Over the course of about ten weeks they conducted at least 100 interviews with the company to build strategy, roadmaps, and ROI estimates. The  team worked with the customer to create a master plan that showed how HP could help the company with its goal of transforming its business.

The HP sales team leveraged many different resources to make sure they had an excellent understanding of the customer’s needs and that the company understood how HP could help. HP made sure to get executive sponsors in key leadership positions at the customer organization. They also brought in some of HP’s top thought leaders and made sure that happy customers were available to discuss their experiences. In addition, HP leveraged its partner network (including Ab Initio, SAS, and SAP Business Objects) to provide a complete solution.

Neoview was a good fit for the customer’s data management challenges. Problems with inconsistent customer information and disconnected IT systems were so hard to manage that it was becoming impossible for IT to adequately support the business. This customer’s top priorities were to regain control of its existing analytics data store and revamp enterprise customer intelligence and enterprise risk management. They liked the way Neoview was built. It is based on HP’s NonStop engineering expertise that has been used for over 30 years in industries such as financial services (stock exchanges) and telecommunication (switching) where the management of vast amounts of data is essential. Neoview is designed to support hundreds of terabytes of data and over one thousand processors. The customer also had some concerns about issues like getting its team up to speed on the product. HP stepped up to meet their concern by offering training and help with Neoview’s operation to ensure a smooth transition..

As I stepped back from this discussion, it occurred to me that successful technology sales in this type of complex market is incredibly challenging. It is simply not enough to make an announcement and hope for the best. On the one hand,  the ingredients for a successful sale  sound pretty simple – you need to understand the customer’s pain and provide the right solution at the right price. Easy? Try telling that to a team that just implemented a full-scale, coordinated sales push, made all the right moves, and beat out formidable competitors to win the sale. It’s not so easy, particularly with a complicated IT solution in a market where  the business demands fast and cost effective results.And, if it takes such a coordinated effort to win one sale, how realistic is it to expect to sustain these efforts over the long term?

There are three main requirements for selling IT products and solutions in today’s market:

Get the basics right.You need to provide good technology at the right price. Your marketing plan needs to be based on clear and concise messages and your sales team needs to be able to articulate  those messages in a way  that is just right for your customer. This sounds like a good plan in any market. The difference today is that you can’t count on some of the sales that might previously have been considered “low-hanging fruit”. For example, assume you are preparing a proof-of-concept for a prospective customer and several of this company’s developers know you and your product from their work at other companies. Although it is helpful to have strong supporters of your product on the prospect’s IT team, their support can not overcome a product or solution that doesn’t solve real business problems. Today, the business is demanding more from IT – more business value, more trusted data, more control over costs, and more control over project time lines.

Understand your customer’s needs. You need to understand your customer’s challenges and expectations in order to make sure your product/solution is a good fit One of the most common mistakes that software product marketing teams make when preparing marketing materials is to focus on the outstanding and differentiating features of their product from a product-centric instead of a customer-centric point of view. You need to understand what problem you are solving for your customer and how your solution will solve this problem faster-cheaper-or with more flexibility for future changes than your competitors products. An understand of customer needs should happen at two levels. First, it helps to look at customers in a vertical market so your solution and  marketing strategy account for industry specific complexities and challenges. Second, you need to understand the requirements of the prospect at hand -within the context of its industry as well as unique situations such as a recent acquisition or internal changes that may impact their sales decision.

Develop a coordinated and well organized approach. The difference between a good sales effort and a great one is in the way internal teams and business partners collaborate to put knowledge about customer needs and product/solution capabilities together to find the right fit for the customer. For many software vendors, the services or consulting team (internal team or external partnership) has a large role to play to help close the deal. For example, regulatory requirements in industries such as  health care and financial services industries are continuing to change making significant demands on IT environments for companies in these industries. Although, you may have a great solution for handling vast amounts of data and improving data security and governance, you may not have the opportunity to prove it to your prospect without a coordinated sales effort. By bringing your product and sales teams together with a consulting team with deep experience in the regulatory and data requirements for health care and financial services you will be better equipped to show the business value of your solution.

As you try to finalize your deal, it often comes down to very similar issues across different types of customers.  They are all looking for quick, inexpensive fixes to hard problems! The reality is that there are no easy solutions to closing deals in this economy. Understanding what the customer needs is hard, but  what is even harder is making all of this scalable.

Everyone takes for granted these days that all Web services interfaces are the same.  I hate to break the bad news to you, but they aren’t.  This has not been a terrible problem when companies were using Web service interfaces for their internal use but it can cause real problems when they are trying to use these services between companies.

Software and hardware vendors and system integrators have heard loud and clear from their customers that something needs to be done.  Therefore, I was pleased to see that a new organization has been formed called The Web Services Test Forum (WSTF) . It is intended to provide a practical approach to fixing the problem.  The list of companies and participating (as well as those who are not) is quite interesting.  The companies supporting this effort include IBM, Oracle, Active Endpoints, Axway, CISCO, eviware, Fujitsu, Hitachi, Red Hat, Software AG, Teamlog, and TIBCO Software Inc.  The first end-user companies to jump on board are the car companies (Ford Motor Co. and The Automotive Industry Association (AIAG).  Why the automobile industry — its simple…they have been leading in the development of interoperability standards for the last decade because of the need to automate their supply chain.  Noted by their absence are companies like HP, SAP, Microsoft and a host of others too long to list here.

While there has been considerable work over the years on Web services standards, there hasn’t been a standardized and open mechanism to test the results.  Therefore, the WSTF is intended to provide an open testing environment focused on improving the quality of Web services.The goal for the Forum is to accelerate interoperability for Web services standards, simplify integration and improve interoperability for customers in heterogeneous environments.

I like this approach both because it focuses on a key problem  for customers and it provides a pragmatic way to find solutions to issues of interoperability.  So, here are my five reasons why I think this Forum will be important:

1. The WSTF gives the standards creation process a dynamic business focus. Members of the WSTF are expected to provide real-life business cases from their work environments. The context of the business problem will remain central to the testing process created to assess the interoperability of specific technologies.The results of the interoperability research on each business case will be described in a scenario that includes multiple parts: a description of the business case, the architecture used to describe how specific technologies and standards are used in the case, a set of test cases, and the artifacts (such as WSDL, XML schemas) needed to implement the test cases.Participating end-use customers can submit case scenarios to ensure that their highest priority needs are well understood by the Forum. These customer-based scenarios will be used to validate interoperability in a multi-vendor testing environment. By keeping  customer problems at the center of all testing environments,vendors and customers should get the results they need faster.

2. The Forum will put a practical spin on the critically  important, but often lengthy architectural approach to standards creation. The WSTF will test real-life problems in an open multi-vendor testing environment. Software vendors will jump start the collaboration process by including standards  in the testing environment before they have become fully vetted, analyzed and formally approved. This early collaboration will help to identify interoperability issues as soon as possible .There are seemingly endless  different combinations of various end points and Web services interfaces that need to be  tested for interoperability. It makes good business sense for vendors to first test those combinations that are most important to their customers, but it can be hard to make the right choices without a lot of customer involvement.  The  customer/vendor collaboration  in the WSTF is intended to  help vendors create a create a testing environment that is  less  theoretical and more focused on practical business considerations.

3. Best practices useful to members and non-members of the WSTF will develop from real-life test cases submitted by end-use customers. Ford and the Automotive Industry Association (AIAG) are the only end-use customers included in the original group of Forum members. More are expected to join. Understanding the requirements of the auto industry should provide a great starting point  for the WSTF. The industry as a whole was early in its adoption of Web Services technology—using these technologies to facilitate the massive e-commerce interchange of parts and supplies that takes place throughout the disconnected auto industry supply chain. Companies in the auto industry need to connect complex business processes both internally and with a large network of partners. The test cases that members like Ford and AIAG  will  submit are likely to require considerable interoperability between many unique vendor technologies. Improving the quality of messaging within the context of the automotive market will help establish best practices that are widely applicable to other industries.

4. The open nature and focus on scenarios will help end-use customers and vendors to save money and resources in the testing process. Each vendor will host its own testing environment and expose access to its endpoints. This will result in a lower cost way to test products. Having access to testing environments on an ongoing basis will reduce the need for each vendor or customer to set up their own multi-vendor test environments.

5. The structure of the WSTF will enable end-use customers to increase the impact they have on how software vendors approach interoperability . The mission of the WSTF is to change the way interoperability testing is done by creating an environment that encourages collaboration between vendors and customers early in the testing process. There are no dues or other barriers that might exclude customers. The WSTF has stressed that no one vendor will be in charge so that the focus will be on how to interoperate with lots of different vendors. The concept is to create one open place where the vendors can test. The feedback from customers will be fast since vendors and customers will be working together. If the process works as intended this customer/vendor feedback loop will lead to better standards and ultimately more rapid product improvements from vendors.

As I think you can tell, I think the WSTF is off to a good start.  One question I still have is how the relationship between the Web Services Interoperability Organization(WS-I) and the WSTF will work. The expectation is that the two organizations will be complementary, but their goals for improving the quality of Web services standards are fairly similar. The WSTF has the potential to make some real improvements in the way  Web services interoperability testing is handled. However, if this is to work, more users of Service Oriented Architectures (SOA) need to jump on the band wagon and push this effort.  History has shown that standards organizations like these only work when customers and users understand that their businesses will benefit from a standardized approach to both Web services and the testing of those services.

Just after I returned from last month’s IBM SOA IMPACT Conference, the Hurwitz team started to interview companies for case studies to be included in the second edition of SOA For Dummies. So for me, one of the benefits of attending IMPACT was getting a feel for some of the key trends in customer SOA implementations in preparation for this research.

I was particularly interested in the “Smart SOA” theme introduced at IMPACT. It seemed an apt description based on the large number of participants (6000 customers and partners) and the diversity and quantity of customer presentations (over 250 across 8 different subject matter tracks) that took place during the week.

But I must admit, initially I wasn’t quite sure about the significance of using the word smart. Does implementing SOA make you smart? Do you need to get smart before you can bring SOA to your company? Or as Drew Carey, the “special guest” at the conference kickoff, joked about the “Smart SOA” theme by saying something like, “What does IBM mean by this…smart SOA as opposed to the stupid SOA our competitors sell?”

After just a few days in Las Vegas with lots of smart IBM customers and partners it was pretty easy to understand why Smart SOA does a good job of describing the current level of customer adoption of this approach. Based on my observations at IMPACT 2008, I see three major trends supporting the scenario that customers, partners, and vendors are all getting smarter about SOA.

Getting Smart about bringing IT and Business together.

Companies have moved very quickly to create a business driven (rather than IT driven) approach to their SOA initiatives. Sandy Carter, IBM VP SOA and WebSphere, cited some impressive statistics supporting this trend in her IMPACT keynote on SOA deployments. She quoted a recent IBM customer survey which indicated that the percentage of SOA initiatives requiring business involvement jumped from 30% in 2006 up to 70% in 2007. IT executives want to get the business involved with their SOA initiatives from the beginning and this is having an impact on how companies are selecting a SOA vendor. IT-centric approaches are out. Customers are looking to their SOA vendors for help in implementing SOA within the context of the specific requirements and best practices relevant for their own industry. Results of IBM’s survey of customers indicate that the #1 criteria used to select a SOA vendor is business expertise.

And just as customers have gotten smarter about improving the collaboration between business and IT, IBM has gotten smarter about capitalizing on its deep vertical industry experience to help companies achieve success with SOA. IBM reports an 80% win rate on SOA related deals when they engage the customer in a Business Value Assessment. This is a set of tools designed to be used with both the customer LOB and IT management to ensure that the customer receives industry specific ROI business cases, key process models, and a SOA solution proposal. In addition, IBM partners and systems integrators like LiquidHub and Perficient, who both presented at IMPACT, are finding that they have more success by using industry specific knowledge to support customers in developing a SOA Roadmap based on business process and governance.

Getting smart about educating developers, architects, and business analysts about SOA

Just a few years ago, acting in the role of SOA evangelist at a very large organization could get sort of lonely. There weren’t a lot of other people around with a similar focus so sharing ideas about SOA best practices was not all that common. Now there are organizations like the SOA Consortium, a SOA advocacy group comprised of end users, service providers, and technology vendors, that use forums and online collaboration tools to share information on SOA best practices and help establish standards. By the way, the SOA Consortium (part of the OMG) has a SOA case study contest underway right now with submissions due by June 30.

At IMPACT IBM announced a Smart SOA social network to facilitate communication between IT and business executives who would want to explore issues and share ideas about SOA. The social networking experience can help people gain the education they need when they need it. The need for SOA specific education is seen as a number one priority by many companies. Lack of having sufficient staff with the right skills is still seen as a major inhibitor in developing a SOA infrastructure.

If companies are going to have a business focused approach then you need to have more developers who understand the business and business analysts with a stronger understanding of technology. Some companies have been very successful at retraining developers who may have strong mainframe skills, for example, that are no longer in such high demand so they can be participants on the SOA team. Architects are using online collaborations tools like Twitter to expand knowledge of SOA and business skills.

Getting Smart about Scaling SOA

There was a big change this year at IMPACT with many customers beginning to move forward based on the success of their early deployments of business services. I talked to many customers who are now moving to scale with broader SOA implementations across business units. Customers understand that it is wise to select an initial SOA project that services a major need of the business and will show ROI and success relatively quickly. However, scaling SOA requires a different level of attention to enterprise issues such as process integrity and latency.

For example, one insurance company director of enterprise architecture told me about what it takes to scale. This director told me that his organization had deployed its first services seven or eight years ago. He has learned a lot of lessons over the years. For example, he recommends that organizations should not try to scale SOA to the enterprise level without automated governance — regardless of how large and expert the IT team happens to be. His team has found out the hard way that a lack of SOA governance will result in a failed SOA deployment.

Getting smart with SOA is certainly a journey for many of the companies we have talked to over the years. It is easy to get caught up in the excitement. Customers that I talked to at last year’s IMPACT conference were excited that there were other professionals just like them in attendance. This year, there were signs that customers are moving to the next level and maybe even getting smarter.

“Green” in the office or in the home often starts with the addition of recycling bins or changing to more energy efficient light bulbs. For example, an office I know well recently announced a cost cutting measure that would also be environmentally friendly. Signs were posted in all office kitchens indicating that the office was going “green” by eliminating the supply of plastic cups and that everyone should bring in their own mug.

I thought of this symbolic office initiative and how small it is while I was at HP’s Analyst meeting in Boston last week and listened to Randy Mott, HP CIO, describe HP’s very grand “green” data center transformation. The enormity of this transformation really got my attention. A typical enterprise data center consumes about as much energy as a small city. A lot of the energy that enters the data center is actually lost in the cooling process before it can be used to keep the servers and other technology running. HP has placed a high priority on research and innovation around blades, power, and cooling technology so that data centers can use energy more efficiently.

About three years ago, Mark Hurd, HP’s CEO, looked at HP’s 85 globally distributed data centers as energy consuming beasts that needed to be tamed. He challenged Randy Mott and the HP team to make the company’s information infrastructure more flexible and responsive to the business while dramatically reducing the energy consumption of its data centers. And he challenged his team to complete this transformation over an eighteen month period.

HP has made tremendous progress with this transformation effort and plans to be fully operational with its six new data centers in three months time. The new data centers were built in pairs across three U.S. cities (Austin, Atlanta,and Houston). This type of transition from old to new data centers can be painful. You do not want to close down a data center until every last application has been closed out and accounted for. HP has already moved out of 32 major data centers through out the world and closed 150 server rooms.

I am adding a link to Robin Bloor’s blog on this topic. He adds some other really good insights that you’ll find interesting.

What lessons did Mark Hurd have to tell us about HP’s data center transformation? Here is what I took away as best practices:

· Everyone in the company has to play a role in making transformation happen

· Don’t do this in a half-hearted way –- you will have to retire old legacy applications

· The next generation data center need to focus on automating, monitoring and controlling –It is focused on the economics of power in the data center

· You need to think about this in terms of an architected infrastructure – not a single targeted project

It is pretty typically for large global organizations to have data centers distributed throughout the world. These large organizations have all seen an explosion in the amount of data needed to be managed, stored, and shared.

Some of these data centers run the risk of not having enough power to keep running in the near future. While the challenges and expense of taking on such a massive data center transformation in such a short time can not be underestimated, HP’s approach can serve as a guide for other companies facing similar issues.

]]> http://mkaufman.wordpress.com/2008/04/07/how-less-becomes-more-in-hps-transformation-of-its-data-centers/feed/ 0 mkaufman http://mkaufman.wordpress.com/2008/01/16/ten-common-mistakes-companies-make-in-data-integration/ http://mkaufman.wordpress.com/2008/01/16/ten-common-mistakes-companies-make-in-data-integration/#comments Wed, 16 Jan 2008 22:03:02 +0000 mkaufman http://mkaufman.wordpress.com/2008/01/16/ten-common-mistakes-companies-make-in-data-integration/ ]]>

One of the most critical IT management responsibilities is to ensure that the business has access to trusted information. This is actually a very challenging goal for many companies because the data needed to support business decision making is often inconsistent, redundant and of poor quality. Company data sources have become increasingly complex, often trapped in a complex tangle of disparate data stores and technology systems.

Large enterprises have typically approached the management of information in a siloed way. Each division or line of business within an organization such as finance, sales and marketing, operations, or a specific product line has been treated as a unique entity. Each entity requires different business applications and each of those applications has been tightly linked with its own data store. This siloed approach no longer meets the need of business users who need to understand and make decisions across the enterprise as a whole.

Why not? Each business application may need similar data, such as customer, product, or pricing data, but the definitions of these data may vary across departments. In addition, the data from the various data stores may have different structures, different interfaces, and even different semantics. The data on customers, products and services are often tied into a specific line of business. What happens when you want to cross sell across product lines. Are the definitions of the customer the same?

Creating a company wide environment of trusted information requires some data integration. This process requires a well-thought out architectural approach that will provide information about the business as a service to everyone who needs it. This architectural approach will typically require technology for ETL (extract-transform-load), quality management, the creation of a metadata layer, and a strategy for master data management (MDM).

Companies can often identify why data integration is required, but then fall short on implementing the technology in a way that maximizes the benefit to the business. It can be very challenging for companies to manage the data integration process successfully. Some of the biggest problems stem from a lack of understanding about the needs of the business. The process of data integration needs to be considered as part of an overall information management strategy for the business and within the context of the business strategy and priorities. It is important to consider the rules, strategies, and goals of the business as part of the process. Does this approach make sense to the business? Does this approach satisfy the requirements of the business?

If you follow a strictly technical approach to data integration you are likely to make some mistakes and fall short of reaching your desired goal. Successful companies look at information management holistically with an ultimate goal of providing trusted and consistent information about the business to everyone one from the CEO to customer service representatives to external partners and suppliers.

The following are ten common mistakes that should be avoided when planning for data integration.

1. Following a “fire-drill” approach to data integration. It is short sighted to use ETL technology as a tool to solve a one-time data integration problem rather than using this technology as part of a comprehensive approach to information management.
2. Not thinking about data as a shared and reusable resource. It is easier to budget based on getting a single task done. However, it is much more efficient and cost effective to be able to reuse data resources once the second, third and future projects are initiated.
3. Thinking tactically about data integration and missing out on opportunities to improve business process. Companies often implement data integration technology to eliminate time-consuming and labor-intensive processes that have been required to gain a consolidated view across business units. However, it is a mistake to focus on reducing head count and saving time in the data integration process, without also considering a broader strategic view towards improving overall business processes.
4. Not establishing an architectural framework with the capability of providing reusable information services. Once the data is decoupled from the business application, you need to develop a methodology that supports reuse so the data can be shared in different ways as needed. The information as a service approach is designed to ensure that business services are able to consume and deliver the data they need in a trusted, controlled, consistent, and flexible way across the enterprise.
5. Using software code to adjust for differences in definitions about customers, products, and other data types on a one-off project basis. In order to deliver information as a service, there needs to be repeatable way to manage complex processes without the expense and time required for recoding. This can be accomplished with the support of a metadata infrastructure.
6. Integrating data without placing a high priority on data quality. It is critically important that companies establish processes to cleanse and correct data as part of the overall data integration process. Creating standardized and consistent information will ensure that business users are more confident about business information and in a better position to grow the business and remain competitive.
7. Not creating a standardized way to handle data that is common to the various disparate IT systems and business groups. Companies need to understand the commonalities across different data types. This can best be achieved by developing a master data management (MDM) strategy to serve as the system of record for the consuming systems and applications.
8. The technical integration team and the business experts do not communicate effectively. There needs to be a shared and common language describing business processes to enhance communication between business and IT management. The business is more likely to have good quality information they can count on if the IT and the business establish an efficient process for sharing knowledge and requirements.
9. Business owners are reluctant to give up ownership of data. In order to gain the efficiency and accuracy in the data integration process, it is important to establish a consensus among the various data owners regarding data terminology and definitions, and there needs to be a clear understanding of the data lineage and who is responsible for these data over time. This often requires a significant cultural change because individual business experts often have a long history of managing data for their line of business or department as if it was a stand-alone entity. Companies need to find a way to balance the need for individual business experts to maintain control over their own data with the need for centralized management of data within a metadata environment.
10. Trying to do too much in one project. When data is integrated across departmental data silos, previously inaccessible data becomes available to business users. Companies can take on projects that would have been impossible before because of the enormous amounts of hand coding and manual data collection that would have been required. However, these benefits can be lost if companies try to tackle too much at once. Enterprise-wide information management projects must be approached in an incremental way so that there is time to evaluate and improve data quality, understand the needs of the business, and establish repeatable methodologies and processes.

With all of the acquisitions happening in the business intelligence space, customers are in a state of confusion. What do all of these changes mean — both short term and long term? In my view, it is best to take a pragmatic approach to this changing market landscape. This acquisition spree in BI is impacting as many as 80,000 customers who use business intelligence software from Hyperion, Business Objects, or Cognos. What should customers be asking their vendors about the future directions of their products? All three BI vendors have either just been acquired or are soon to be acquired by three large IT companies – Oracle, SAP, and IBM, respectively. Is it business as usual or do the IT managers need to alter their business relationships and plans for business intelligence implementations now that a wave of consolidation is underway?

Customer concerns about the re-alignment in the business intelligence area tend to fall into three categories: impact on legacy environments, impact on future buying decisions, and technical innovation. Management issues will vary with how business intelligence software has been deployed at their companies and how well integrated this software has been with other information management software.

Legacy environments. Many companies tend to use BI tools for traditional management reporting. In general, they like to stick with what they know and what their users are trained to use. Therefore, they want to be assured that they can continue using the familiar reporting tools in the same way. While most managers assume that once an acquisition is complete, there will be new operating efficiencies. What they don’t always know is whether those efficiencies will translate to savings. They also would like to understand how the company might benefit from the fact that a larger company with more resources has bought the company they have been dealing with. How will my company benefit from the acquisition? Will there be any changes in the sales and service teams? What about pricing issues and planned upgrades?

Typically, it will take a while before changes are evident. For example, if you are getting good value from your use of Crystal Reports—the Business Objects’ reporting solution designed to support business modeling, analysis, and decision making – it really may not matter to you which company purchased Business Objects.

Ironically, some of the most important issues that might emerge happen when customers merge with each other. For example, one company might have standardized on Cognos for its analysis and reporting while the company being acquired uses Business Objects as a standard. Since large enterprises rely on business intelligence software to gain insight into production, sales, revenue, or other data across divisions and subsidiaries, too many tools may make decision making more difficult.

It is interesting that these acquisitions are hitting the market at the same time that companies are trying to move from a departmental view of data to an enterprise perspective. A unified and standardized approach to information management across the enterprise is becoming a top priority. Companies that have accumulated many different BI vendor software may use this time of change to re-evaluate a BI strategy.

Typically companies are used to managing multiple software components from multiple vendors. The expectation is that the consolidation of two or more of a single vendor will lead to benefits resulting from the tighter integration of the products. This is often the best outcome in terms of support, training, and management. However, this is typically a multi-year effort by the vendors building a unified portfolio based on acquired software. As businesses move from a traditional siloed single purpose data warehouse to information integration and analytics, having one vendor to call is often a welcome change as companies try to simplify the management of its infrastructure.

Impact on Future Buying Decisions

The situation may be a little different if a customer is in the middle of a proof of concept (POC) for a project designed to develop a single view of a company’s customer base. How will the recent acquisitions in the business intelligence market impact how businesses to move forward?

Consider the example of a large bank that had recently made a significant acquisition. The bank wanted to understand its most profitable customers across the newly merged company. Customer history and sales data was retained in a siloed manner by line of business and there was no integration between the data for the two companies. This company used Cognos for many of their executive level reports, but now management had raised concerns about data quality.

In order to develop a single view of customer they needed to look for incompatibilities and inconsistencies in the disparate data sources. These data sources needed to be integrated and IT needed to assure the business that the information was accurate, complete, and trust worthy. The bank selected Informatica to provide the software needed to help with the integration and to improve the quality of its data.

Informatica has a comprehensive solution for data integration and data quality. In addition, just a few months ago Informatica and Cognos announced an expansion of their strategic relationship. This partnership fit well into the CIO’s priority to consolidate all the many disconnected vendors in use in IT and to ensure that the integrations between different applications are as tightly integrated as possible. Now, suddenly Cognos is part of IBM and a direct competitor to Informatica. What should the CIO do? Certainly IBM is committed to supporting all of Cognos’ existing partnerships. You should not have to change your plans because of the acquisition, however there are questions to ask about how things will change in the future.

Innovation

All the BI vendors mentioned above have well-established partner relationships with emerging information management software companies. There has been a lot of customer support for the creation of tighter integrations between the information management infrastructure and the business intelligence layer. Companies have recognized that the reporting structure becomes meaningless if the supporting data cannot be trusted.

The partnerships have been important because much of the important innovation comes from small emerging companies. Partnerships with major players makes it easier for the company to leverage innovation with lower risk. The established players can provide the integration with the analytics and reporting technology. As companies attempt to unlock much of the data that has been previously unreachable at the enterprise level, it will become much more important to have a unified approach to information quality, information integration, and business intelligence. Market consolidation will help ensure that innovation is better utilized in a predictable manner.

My name is Marcia Kaufman. I have been an analyst, consultant, and researcher in software for many years.  As a partner at Hurwitz & Associates, I focus on emerging software issues related to data management, service oriented architectures, and emerging technologies that help both large and mid-sized companies leverage technology for customer benefit.