Database Benchmarking - Page 2
January 26, 2005
The TPC-C Benchmark
From the TPC Web site, the TPC-C benchmark is defined as:
TPC-C sounds like a reasonable test for a transactional database. However, if you look further into the TPC-C benchmark, you will see that there are versions of the test, so the results of how a system did using one version may not be comparable to the same system using the latest version (5.3). TPC-C is the most popular benchmark test, so it is probably the results you have seen Oracle advertise. As a test of this, perform a search on benchmark at Oracle's home page, and go to the Benchmark Results - Transaction Processing link.
Let's drill down into the first bulleted item (TPC-C: Oracle and HP ... on Linux).
Reading the fine print at the bottom, Oracle cites the Transaction Processing Council results, but look at the specifics of what it took to get those results. On that particular system/platform combination, using the specific test established by the TPC, Oracle exceeded one million transactions per minute. Again, this result is based on fairly exact conditions whose results may - and do - differ when run on other system/platform combinations. Using only the tpmC metric, did anyone else exceed the one million transactions per minute mark?
Now that you know about the TPC and the results it lists, two of the top three tpmC results were obtained by none other than IBM - and one of those spots is first place. See the page at http://www.tpc.org/tpcc/results/tpcc_results.asp?print=false&orderby=tpm&sortby=desc and order by descending tpmC. Oracle's results are impressive, but not nearly so as the results obtained with the IBM eServer/DB2 database on AIX combination and it is nearly three times as much tpmC as Oracle. Further, the IBM competitor has a lower price per tpmC. However, don't try this test at home, boys and girls. If you click on the "IBM eServer p5 595 64p" link in the System column, you may be surprised to see how much this system cost (over 16 million dollars). The results shown at Oracle's Web site only tell part of the story - when using either tpmC or Price/tpmC, Oracle is not the best overall. Nevertheless, technically speaking, the results (using tpmC) shown by Oracle are the best on the market today as the IBM "winner" isn't available until May 2005.