Building a business intelligence system requires intelligent building blocks: Buy it or build it, they’ll all want to use it

Some of the most tangible benefits of a knowledge management system have come from the area of decision support, and the tools of decision support collectively build today’s business intelligence systems. With an established history, those data warehousing, mining, analysis and reporting tools offer demonstrable decision support guidance. Ironically, one of the most difficult decisions an organization faces is how to go about deploying such a system.

Analyzing sales data, reviewing market research or weighing demographic statistics can be integral to an organization’s success. Uncertainty is less about whether an organization needs to analyze data as it is about whether to buy or build the system to do it.

Research just released by the analyst group Ovum (www.ovum.com) points out that the market uncertainty is accompanied by significant growth, especially in services. The worldwide business intelligence market is expected to top $50 billion by 2002, according to the report, "Ovum Evaluates: Data Warehousing Tools and Strategies."

"There are a number of underlying trends in the market, but the most important is the shift from build to buy," said Ovum’s business intelligence authority David Wells. "However, the transition to buy is much less straightforward than some vendors would like potential customers to believe."

Fortunately, application development tools from several major BI vendors allow a successful compromise. Off-the-shelf solutions from vendors, such as Brio, Business Objects, Cognos, Hummingbird, IBM, Informix, Platinum, SAS Institute, Seagate and Sybase, allow immediate analysis as well as customization. Further, traditional enterprise application vendors PeopleSoft, SAP and Oracle have introduced their own BI-specific applications.

Even systems built of components from the same vendor must rely on an open architecture to integrate existing desktop applications. An integrated, open platformÑsuch as Microsoft’s SQL Server 7.0 or IBM’s DB2 UDB (Universal Database)Ñallows easy interface with OLAP tools from differing vendors.

"We’ve made a lot of progress as an industry to get these tools to work together," said Bob Walters, a VP in Informix’s (www.informix.com) data warehouse division. "But the more you want them integrated, the higher the risk. Usable integration is not tough, but close integration at the application level is quite a bit tougher."

Integration concerns are not just with traditional BI components of OLAP and transaction processing, data mining and cleansing, but also with applications handling unstructured data. Increasingly, organizations seek to incorporate imaged data, geo-spatial or text data into an analysis system. Solutions such as the SAS Institute’s (www.sas.com) Collaborative Business Intelligence exemplify the emerging market for tools that handle structured and unstructured data with equal efficacy.

The market can expect more of those crossover solutions as structured data vendors like Computer Associates (www.cai.com) and Hummingbird (www.hummingbird.com) integrate their newly acquired unstructured data experts Platinum (www.platinum.com) and PC DOCS/Fulcrum (www.pcdocs.com) respectively.

"In building your own, there’s more project risk, but if your goal is a highly integrated system offering a palpable competitive advantage, this may pay deep dividends," advised Walters.

Building on an open platform facilitates the inclusion of data from multiple sourcesÑlegacy data, transaction data and information from any RDMS. As organizations spread across multiple locations, often with completely separate databases, the need for open integration becomes all the more critical.

"A year ago there were significant changes and shifts going on in how business intelligence solutions were built and maintained," said Ovum’s Wells. "One thing that hasn’t changed, however, is that many IT user organizations still underestimate the importance of the back-end problemÑgetting information into the data warehouse or analytical applicationÑand that some vendors are still encouraging them in this attitude."

The key to designing effective operational systems is ensuring the ability to extract information for analysis. "This is almost always the most expensive and risky part of data warehousing," said Wells. "But some IT user organizations seem to be stuck in denial about the importance of this part of the process, and some vendors seem keen to encourage them."

Historically, business intelligence projects were treated as IT projects, driven by specific line-of-business requirements (e.g. to enable interactive drill-down on quarterly revenue figures). Data warehousing now allows companies to take an enterprisewide approach to valuing its analytical information.

Consistent with that change in the perception of BI tools has come an increased ease of use. Analysis is no longer for the exclusive use of senior executives, by the exclusive toil of the IT department. Increasingly, analysis is done by those who actually need the information.

"As products become easier to use, you push decision making down to the knowledge worker," said Torgeir Braaten, a product strategist with Seagate Software (www.seagate.com). "Ten or 15 years ago, an OLAP system was something a CFO or CIO would have on a desktop," he said. "The concept of scalability did not existÑyou didn’t have the bandwidth."

Braaten stresses the importance of scalability as hundreds or even thousands of reports are rolled out to users, each of whom can make additional ad hoc queries. "You get a lot of feedback very quickly," Braaten said, "questions such as, ÔHow do we get these out to everybody?’ ÔHow do we let users see hundreds of reports?’ ÔHow do I share?’"

Development tools like Seagate Holos offer an environment for building OLAP applications using a multidimensional OLAP database server and a toolkit that address the scalabilty issue.

"In the average workplace, 80% to 90% of the users just want a report," Braaten said. "Farther up the spectrum, some need more ad hoc analysis and want to ask questions. It’s only the top 1% that need sophisticated analysis. Scalability becomes key as sophistication rises. The idea is to bring people and data together."

"In the last year, tool prices have come down and functionality has improved dramatically," the Ovum study reports. "Many data warehouse implementers would benefit from looking at the options again."