DB2 management, tutorials, scripts, coding, programming and tips for database administrators
Your big data application needs regular extracts from your production systems. While many best practices exist for big data extract, transform and load (ETL) processes, we sometimes forget that these data-intensive procedures can affect the operational environment’s performance. Here are some suggestions to mitigate such performance risks.
Big data applications start big and keep growing. As the masses of big data being analyzed grow both in size and complexity, the hardware and software communities have responded with huge storage offerings and massively parallel storage and retrieval mechanisms. The logical next step is for the IT enterprise to take advantage of these technological innovations for things other than classical big data processing.
While the performance of a mission-critical application sometimes takes center stage, IT staff are always aware that the performance of the overall system affects all applications. Database management systems, operating system software, physical data storage and retrieval, and backup and recovery operations all take part in providing a solid infrastructure for applications. For many customer facing applications, such as on-line transaction processing, the database management system is the key component.
Assume you’ve been given a directive by IT management to "tune" things. What strategies are available and how do you judge which ones are best? Considering the current economic climate, can any be implemented on a shoestring budget? Can any of these tasks be assigned to resources with basic DBA skills and still be productively accomplished?
As budgets tighten, how can database administrators keep up with the rapidly-changing new technology environment? The solution: automation. In this article we discuss various ways to automate common processes and relieve the infrastructure staff from repetitive tasks, so that then can concentrate on more urgent priorities.
Early data warehouse implementations began as collections of financial and customer data that accumulated over time. Modern warehouses have evolved into complex and elegant enterprise analytics platforms, hosting a broad collection of multiple data types, queried by advanced business intelligence software. As the warehouse environment becomes more valuable, capacity planning becomes critical. In this article we present several strategies for managing data warehouse capacity planning and performance tuning.
Lockwood Lyon presents simple SQL statements that the database administrator (DBA) can execute against the DB2 catalog to determine if your DB2 subsystem suffers from common maladies, and what the DBA can do to fix or mitigate potential problems.
Despite the sophistication of the latest DB2 software versions and the power of current IBM z/server technology, it is still possible for performance and data availability to deteriorate due to a variety of things, including increased dataset extents, loss of clustering, index page splits, and other factors. This article presents simple SQL statements that the database administrator (DBA) can execute against the DB2 catalog to determine if one or more application databases suffer from common maladies, and what the DBA can do to fix or mitigate potential problems.
What is next for big data? Some experts claim that data "volumes, velocity, variety and veracity" will only increase over time, requiring more data storage, faster machines and more sophisticated analysis tools. However, this is short-sighted, and does not take into account how data degrades over time. Analysis of historical data will always be with us, but generation of the most useful analyses will be done with data we already have. To adapt, most organizations must grow and mature their analytical environments. Here are the steps they must take to prepare for the transition.
Business Intelligence (BI) has matured over the past two decades. The next few years will be critical for the information technology staff, as they attempt to integrate and manage multiple, diverse hardware and software platforms. This article addresses how to meet this need, as users demand greater ability to analyze ever-growing mountains of data, and IT attempts to keep costs down.
Some companies have been slow to acquire big data applications. They discovered that modern hardware platforms and database management systems were more than adequate for most of their business analytics needs. Such needs share several common attributes, including analytics run against operational systems, where the analytics logic and engine were close to the object data. This meant that companies could avoid complex and high-volume data movement and extract-transform-load (ETL) strategies while executing queries against already existing, well-tuned databases. In this article we introduce the concepts of strategic and tactical analytics, and how best to support these methods with your current IT infrastructure.
With promises of incredibly fast queries, many IT shops implemented one or more big data applications in combination with high-performance hardware and software suites. However, few IT enterprises followed the appropriate data governance means and methods of ensuring data quality. What ensued was data of dubious quality being loaded into these applications, calling into question the results. In this article we review data quality methods and metrics for loading big data applications.