Using Sybase ASE 15.5 In-memory Databases
May 7, 2010By Robert Schneider
This article investigates how the newly-available In-memory Database (IMDB) and Relaxed Durability Database (RDDB) in Sybase ASE 15.5 offer new techniques for supporting performance-intensive applications while not mandating any deviations from traditional Sybase development practices.
With the release of version 15.5 of its Adaptive Server Enterprise (ASE) database platform, Sybase offers new database technology for database architects, application developers, and system administrators. In this article, we investigate how the newly-available In-memory Database (IMDB) and Relaxed Durability Database (RDDB) offer new techniques for supporting performance-intensive applications while not mandating any deviations from traditional Sybase development practices.
We begin by examining the special performance requirements of high-volume modern database-driven applications, and how memory-based data management is an innovative approach that helps deliver the throughput necessary to support these mandates. Next, we delve into the new IMDB and RDDB options offered by ASE 15.5, including a brief overview of how they work. Once that topic has been covered, we move on to a series of inter-related scenarios that chronicle cases where each option is appropriate.
How in-memory databases help modern applications
Given their role as foundational technology for mission-critical applications, traditional relational database engines are designed to support the well-known ACID principles (Atomicity, Consistency, Isolation, and Durability). In particular, they typically implement the Durability principle by using logs that are written to disk whenever a transaction is committed by a user or application. Combined with data page writes, this approach delivers full durability: transactions that have been committed will be permanently registered on physical media, even in the event of a system failure. This architecture has delivered decades of success for relational databases.
However, this protection comes with a rather high price tag, namely the expensive overhead incurred by all of the disk activity necessary to preserve data integrity. Since disk operations are typically 10 times slower than memory-resident activities, these costs can quickly add up.
While many applications require this level of insurance, there are numerous situations where streamlined coverage will suffice. For example, increasing numbers of applications supporting industries as diverse as finance, government, and telecom all generate tremendous amounts of relatively short-lived data. Although each of these situations is different, they all require higher speed than often can be provided by traditional disk-based data persistence.
In response to these trends, ASE 15.5 offers new data processing options that operate entirely in memory, thereby bypassing the durability-focused burden found in conventional disk-based data management. In addition, Sybase has made it easy for developers and administrators to seamlessly take advantage of these new alternatives without needing to rewrite applications or make any other changes.
Understanding new database options
Before we look at some scenarios where in-memory and relaxed durability databases can help, lets briefly explore their architecture as well as how to work with them.
In-memory database (IMDB)
As we describe earlier, traditional disk-based relational databases continually synchronize their memory (data and transaction logs) with disk drives. However, in the case of an in-memory database such as offered by ASE 15.5, neither log nor data is ever written to disk. Instead, both remain in memory at all times. Transaction logging is still performed, but the log remains in memory. As long as no unexpected events (such as a server, database, or application failure) occur, the information contained in this database will remain safeguarded and usable. Furthermore, youre always free to copy information from the in-memory database onto a disk-based database, thus preserving your information beyond the lifespan of a single server instance.
To create an ASE in-memory database, you first run a series of stored procedures that allocate an in-memory storage cache and generate a logical disk device within this cache. Once thats been done, you create the database using the traditional ASE syntax, with an added directive to use the in-memory option. From this point onward, the in-memory database appears (to users and applications alike) just like a traditional database, except that it delivers better performance and diminished durability.
Once you shut down ASE (or an unplanned event occurs), the IMDBs contents are lost. However, note that an empty copy of the database will be re-created upon server re-boot. In fact, you can even create a template as a boot-time blueprint. This template can contain metadata (such as table definitions), data, users, and permissions. This is a great way to bootstrap your IMDB and eliminate manual preparation.
Relaxed durability database (RDDB)
As we described earlier, traditional disk-based databases provide durability by regular flushing of data pages and the transaction log to disk. To deliver their enhanced performance, in-memory databases skip these time-consuming, expensive responsibilities. In an effort to provide a degree of compromise between these two extremes, ASE 15.5 offers another option: relaxed durability databases.
In particular, there are two options at your disposal. The first option delivers similar durability as an in-memory database: when the database server shuts down (whether normally or via an unplanned event such as a crash) your in-memory data is lost. On the other hand, selecting the second option provides enhanced durability if you perform a proper, normal database server shutdown. However, note that if an unplanned shutdown occurs, the data will still be lost.
Relaxed durability databases are fully integrated with ASE, just like in-memory databases. This provides the same look-and-feel, user experience, and language compatibility.
Database usage scenarios
Now that weve outlined the new choices offered in ASE 15.5, lets see how these options might be used in support of a series of real-world scenarios. For the purposes of this illustration, imagine that youve been asked to create a database architecture for a new e-commerce web site. Your design will need to support a collection of important capabilities, including:
To support these requirements, well look at three separate use cases. As youll see, each situation has its own unique necessities that can be addressed by ASE 15.5.
Requirement 1: An online shopping cart
As you begin your design efforts, you face the sobering fact that your new e-commerce site will confront a formidable collection of competitors. To make matters worse, youre well aware that your customers are a highly demanding, fickle lot; they expect very fast responsiveness, and wont hesitate to abandon a shopping cart and go elsewhere if your site isnt available, even for a moment. You also know that once youve lost a customer, theyre unlikely to ever return.
Thus, to help support this essential responsibility and deliver an optimal customer experience, your job is to apply the right mixture of hardware and software technologies. From the hardware perspective, you employ a collection of dedicated, powerful servers in a highly controlled environment. To minimize the possibility of site outages, you employ sophisticated server failover capabilities, along with network redundancy and other techniques to ensure availability.
Given that your processing environment is so robust, and that server outages are highly unlikely, the optimal information management software choice would be to select the ASE in-memory database option. By avoiding disk interaction whenever possible, it will deliver the fastest possible database performance. In addition, by creating a boot-time template with pre-defined tables and data, you minimize the amount of preparatory work necessary to stage your site to accept orders.
Requirement 2: Customer feedback and comments
Product reviews, experiences, and other customer-driven commentary are essential components of any modern e-commerce shopping experience. To support this key capability, you plan to create a well-integrated forum where customers can share their thoughts on your products. Since this intelligence will be a major component of the buying decision, you need to ensure that both performance and data integrity are present in your solution: the data must be available quickly, and newly created comments need to be safely stored on your servers.
By analyzing the various ASE data management options, you decide that an in-memory approach will deliver the performance you need to complete these tasks within the specified response window. However, youre concerned about a potential loss of open comments should a planned server shutdown occur in the middle of the editing process. While the likelihood of a server shutdown affecting many comments is low, you still want to avoid the prospect of a customer carefully entering a review only to see it vanish into the ether. This argues against the pure in-memory database technique, since this option provides no durability in this type of situation.
In this case, the relaxed durability option (specified when creating the database) offers a suitable compromise: it delivers the enhanced performance of an in-memory database while preserving data durability if a planned shutdown takes place while processing is underway. Note that the relaxed durability option does not provide any durability protection if an unplanned server shutdown or application failure occurs.
Requirement 3: Order and inventory management
The amount of time it takes to receive their order, and the accuracy of the shipment, will form the lasting impression that a customer will take away from your site. Thus, an always-on, lightning-fast website is of little use if the order fulfillment process is subpar. However, since budgets are constrained in every organization, its quite likely that the fastest and most available servers will be dedicated to the shopping cart and customer forum, leaving somewhat less-capable hardware to support the back office. While outages and unplanned downtime are still relatively unlikely, they are more prone to occur in this type of environment.
Since data durability outweighs performance requirements for this type of processing, this is a situation where traditional disk-based persistence is the optimal choice. As an added benefit, if additional resources become available to support the order fulfillment process, youre free to revisit your decision: switching to an in-memory database wont require any application changes, which illustrates the beauty of Sybases holistic architectural philosophy.
The new database options provided in ASE 15.5 are worth a look, especially if youre charged with supporting high-volume applications, situations that dont require perfect data durability, or are lucky enough to have highly reliable hardware at your disposal.
Whats especially intriguing about these fresh innovations is that they can be used transparently; you dont need to re-code your applications or make any other time-consuming alterations to take advantage of these new alternatives.
Finally, its important to apply these new selections properly: not every situation requires an in-memory or relaxed durability database; your data integrity requirements may mandate a traditional database instead. However, for those occasions where they make sense, they offer an ideal package of performance and flexibility.
Additional ResourcesSybase Getting Started with In-Memory Databases in Adaptive Server® Enterprise 15.5
Overview of Sybase ASE In-Memory Database Feature
Sybase Sybase ASE In-Memory Databases
About the author
Robert Schneider is a Silicon Valley-based technology consultant. He has written six books and numerous articles on advanced technical topics such as Cloud Computing, Service Oriented Architecture (SOA), and high-performance relational database design and optimization. He may be reached at robert.Schneider@think88.com