DB2 management, tutorials, scripts, coding, programming and tips for database administrators
What is next for big data? Some experts claim that data "volumes, velocity, variety and veracity" will only increase over time, requiring more data storage, faster machines and more sophisticated analysis tools. However, this is short-sighted, and does not take into account how data degrades over time. Analysis of historical data will always be with us, but generation of the most useful analyses will be done with data we already have. To adapt, most organizations must grow and mature their analytical environments. Here are the steps they must take to prepare for the transition.
Business Intelligence (BI) has matured over the past two decades. The next few years will be critical for the information technology staff, as they attempt to integrate and manage multiple, diverse hardware and software platforms. This article addresses how to meet this need, as users demand greater ability to analyze ever-growing mountains of data, and IT attempts to keep costs down.
Some companies have been slow to acquire big data applications. They discovered that modern hardware platforms and database management systems were more than adequate for most of their business analytics needs. Such needs share several common attributes, including analytics run against operational systems, where the analytics logic and engine were close to the object data. This meant that companies could avoid complex and high-volume data movement and extract-transform-load (ETL) strategies while executing queries against already existing, well-tuned databases. In this article we introduce the concepts of strategic and tactical analytics, and how best to support these methods with your current IT infrastructure.
With promises of incredibly fast queries, many IT shops implemented one or more big data applications in combination with high-performance hardware and software suites. However, few IT enterprises followed the appropriate data governance means and methods of ensuring data quality. What ensued was data of dubious quality being loaded into these applications, calling into question the results. In this article we review data quality methods and metrics for loading big data applications.
Many big data applications are designed, built and installed without a formal load test. This is unfortunate, as load testing gives the database administrator quite a lot of valuable information. It may make the difference between poor and acceptable big data performance. This article reviews big data application implementation practices with some proactive tips on load testing to make your production implementation a success.
Big data applications are now in place to support business analytics processing. However, many standard performance tuning options (such as indexes) may no longer apply. Here we investigate the differences between operational systems and big data systems, and present common options for big data performance tuning.
Many IT enterprises have created big data applications to store and analyze massive amounts of historical data. Should these applications be installed in their own exclusive environment, or tightly integrated with current operational systems?
In its first phase of implementation, the big data application received and stored data from operational systems, allowing business analysts to use business analytics software to analyze the data for trends. Now we are in the next phase. Big data applications must now create value by feeding data back into operational systems. This becomes even more important during the busiest time of the year. What are the most important things for the IT staff to prepare?
Big data applications store huge amounts of data in a massively-parallel storage array and use sophisticated analytical software to return solutions to SQL queries. With multiple applications accessing enterprise data, we now need mechanisms to guarantee that the data is easily available. Here, we discuss a few common techniques to ensure that a big data application does not cause bottlenecks in mission-critical processes.
Most large companies have one or more big data applications, which provide fast access to large stores of customer and sales data. As the IT organization grows new job categories and new tasks are added to the mix. These include big data hardware and software support, business analysts who use analytics to probe and explore the data, and managers who must supervise and prioritize job tasks.
As the holiday season approaches, your organization can expect more BI activity as you take advantage of a significant increase in customer interactions. The smart IT organization should be proactive and prepare now for larger data volumes and more analytics activity by tuning their big data application for high performance.
In many cases, the success of a big data application can be traced to how well it is integrated into your enterprise data warehouse. This article presents several ways to get this done quickly and efficiently from the beginning.