Data mash-up within Business
Intelligence (BI) applications is one of the latest must have requirements
that I have been asked to build into solutions that I have created for some
major enterprise clients recently. As mentioned in my last article about
the growth of NoSQL use
within the BI area I believe that the mash-up can be another useful tool
to be added to the BI application – but it must not be utilized to the detriment
of the core principles of any BI solution.
The History of Mash-up
The history of mash-up can be backtracked by first understanding the broader
context of the history of the Web. For Web 1.0
business model companies stored consumer data on portals and updated them
regularly. They controlled all the consumer data and the consumer had to use
their products and services to get the information.
With the advent of Web 2.0 a new proposition was created, using Web standards
that were commonly and widely adopted across traditional competitors and
unlocked the consumer data. At the same time, mash-ups emerged allowing mixing
and matching competitor’s API to create new services.
The first mash-ups used mapping services or photo services to combine these
services with data of any kind and therefore create visualizations of the data.
In the beginning, most mash-ups were consumer-based, but recently the mash-up is
to be seen as an interesting concept useful also to enterprises. Business
mash-ups can combine existing internal data with external services to create new
views on the data. (mash-up (web
application hybrid))
Discussion of BI Mash-up Applications
The problem with the clamor for the most recent technological advances to be
added to the BI application stack is as always the differing understanding
between the business user and the developer to how they can be used. The battle
I have faced recently is trying to get the people within businesses to
understand what, how and where the mash-up can be utilized within their organizations.
Mash-ups enable nontechnical users to build dynamic views of disparate data
that are personalized, context-rich, role-tailored, and ad hoc to explore this
data in greater depth. However the problem with most of the currently available
BI Vendors mash-up applications or plug ins is that they simply offer a
numerical analysis of data via the normal OLAP cube route and then attach a
search bar alongside this analysis to enable a search of separate silos of
either textual, web or unstructured content to match up with the data already
recovered and analysis.
The ability of a mash-up to pull content from other sources is what most
business users are excited about and this combined with the ability to store non-structured
data in a NoSQL environment, which allows for rapid search and retrieval and
storage of any and all linked data. I have used this build process to provide a
system for one of my government clients which allows for normalized data to be
indexed, text searched and analyzed to provide link information, which is then
utilized in a webbot search to provide any and all information available from
the web, allowing the user to analyze how the figures presented in the
statistical analysis may be driven.
Most corporations are now requesting that BI systems that I consult on or
build have the ability to interrogate social networking sites to find out what
is being said about their products – this is a perfect example of the ability
of mash-ups to provide information that most marketing mangers and sales teams
desperately need to understand to improve business productivity and sales
success.
This requirement to link to all types of data also needs to be able to
interrogate all systems that are available within the corporate environment –
there is no point in having a BI application, which has the ability to mash-up
data if it cannot attach to all the clients information. These results should
also have the ability to be shown not only in their normal context but also in
a context that is easy to understand and use for the customer. One of the best
examples currently on the web is the real estate tracking site Zillow.com. This site utilizes a mash-up process
to combine data from many disparate spatial sources such as public records, map
data (GIS) and statistical analysis to provide the site visitor with a visual
report, with the ability to drill through the data and provide graphical and
tabular results if required.
Current Versions of BI Mash-up Solutions
Searching the web for vendors providing the ability to utilize mash-ups within
BI will provide many applications, which are capable of providing a fully
functional mash-up solution. Some that I have had personal experience with are
as follows:
Microsoft
Performance Point Server
Those from Oracle, IBM and Microsoft are usually available within the
locally developed BI Solution if the companies’ platforms are based on these
technologies. Penthao is a very useful open source application, which I have
used to build applications from the ground up with much success. I was
introduced to InetSoft when investigating a solution for a recent client, which
involved linking to NoSQL data and have been impressed by their commitment to
keeping ahead of the current trends for data mash-up by providing an end user
ability to create data mash-ups.
There are many more vendors out there and I would suggest that as a BI
Developer you familiarize yourself with as many as you can.
Conclusion
I see mash-ups extending the traditional data-driven BI solutions to
incorporate traditional planned data from a normal rdbms or cube add in
unstructured data and access further information from either RSS or the web
utilising web services. Most of the modern BI Solutions including Performance
Point can solve the first two connections but to connect to the web can require
either a web service to be hand coded or the purchase of one of the specific
applications mentioned above. I have had practical experience with hand coding
both the connection to unstructured data and also the web service and with the
availability of pre designed modules it is easily achieved in less than 250
lines of code.
These abilities now being made available from solutions such as Intesoft and
Microsoft SharePoint Designer to allow non technical analysts and end users to
create a data mash-up and this combined with the current rapid development of
applications from multiple vendors does mean that the BI mash-up is here to
stay.
