BI Datasources - The Great Mash-up
February 8, 2011
Data mash-up within Business Intelligence (BI) applications is one of the latest must have requirements that I have been asked to build into solutions that I have created for some major enterprise clients recently. As mentioned in my last article about the growth of NoSQL use within the BI area I believe that the mash-up can be another useful tool to be added to the BI application - but it must not be utilized to the detriment of the core principles of any BI solution.
The History of Mash-up
The history of mash-up can be backtracked by first understanding the broader context of the history of the Web. For Web 1.0 business model companies stored consumer data on portals and updated them regularly. They controlled all the consumer data and the consumer had to use their products and services to get the information.
With the advent of Web 2.0 a new proposition was created, using Web standards that were commonly and widely adopted across traditional competitors and unlocked the consumer data. At the same time, mash-ups emerged allowing mixing and matching competitor's API to create new services.
The first mash-ups used mapping services or photo services to combine these services with data of any kind and therefore create visualizations of the data. In the beginning, most mash-ups were consumer-based, but recently the mash-up is to be seen as an interesting concept useful also to enterprises. Business mash-ups can combine existing internal data with external services to create new views on the data. (mash-up (web application hybrid))
Discussion of BI Mash-up Applications
The problem with the clamor for the most recent technological advances to be added to the BI application stack is as always the differing understanding between the business user and the developer to how they can be used. The battle I have faced recently is trying to get the people within businesses to understand what, how and where the mash-up can be utilized within their organizations. Mash-ups enable nontechnical users to build dynamic views of disparate data that are personalized, context-rich, role-tailored, and ad hoc to explore this data in greater depth. However the problem with most of the currently available BI Vendors mash-up applications or plug ins is that they simply offer a numerical analysis of data via the normal OLAP cube route and then attach a search bar alongside this analysis to enable a search of separate silos of either textual, web or unstructured content to match up with the data already recovered and analysis.
The ability of a mash-up to pull content from other sources is what most business users are excited about and this combined with the ability to store non-structured data in a NoSQL environment, which allows for rapid search and retrieval and storage of any and all linked data. I have used this build process to provide a system for one of my government clients which allows for normalized data to be indexed, text searched and analyzed to provide link information, which is then utilized in a webbot search to provide any and all information available from the web, allowing the user to analyze how the figures presented in the statistical analysis may be driven.
Most corporations are now requesting that BI systems that I consult on or build have the ability to interrogate social networking sites to find out what is being said about their products - this is a perfect example of the ability of mash-ups to provide information that most marketing mangers and sales teams desperately need to understand to improve business productivity and sales success.
This requirement to link to all types of data also needs to be able to interrogate all systems that are available within the corporate environment - there is no point in having a BI application, which has the ability to mash-up data if it cannot attach to all the clients information. These results should also have the ability to be shown not only in their normal context but also in a context that is easy to understand and use for the customer. One of the best examples currently on the web is the real estate tracking site Zillow.com. This site utilizes a mash-up process to combine data from many disparate spatial sources such as public records, map data (GIS) and statistical analysis to provide the site visitor with a visual report, with the ability to drill through the data and provide graphical and tabular results if required.
Current Versions of BI Mash-up Solutions
Searching the web for vendors providing the ability to utilize mash-ups within BI will provide many applications, which are capable of providing a fully functional mash-up solution. Some that I have had personal experience with are as follows:
Those from Oracle, IBM and Microsoft are usually available within the locally developed BI Solution if the companies' platforms are based on these technologies. Penthao is a very useful open source application, which I have used to build applications from the ground up with much success. I was introduced to InetSoft when investigating a solution for a recent client, which involved linking to NoSQL data and have been impressed by their commitment to keeping ahead of the current trends for data mash-up by providing an end user ability to create data mash-ups.
There are many more vendors out there and I would suggest that as a BI Developer you familiarize yourself with as many as you can.
I see mash-ups extending the traditional data-driven BI solutions to incorporate traditional planned data from a normal rdbms or cube add in unstructured data and access further information from either RSS or the web utilising web services. Most of the modern BI Solutions including Performance Point can solve the first two connections but to connect to the web can require either a web service to be hand coded or the purchase of one of the specific applications mentioned above. I have had practical experience with hand coding both the connection to unstructured data and also the web service and with the availability of pre designed modules it is easily achieved in less than 250 lines of code.
These abilities now being made available from solutions such as Intesoft and Microsoft SharePoint Designer to allow non technical analysts and end users to create a data mash-up and this combined with the current rapid development of applications from multiple vendors does mean that the BI mash-up is here to stay.