On Aug. 6, 2001, Hewlett-Packard Co. (HP) (Palo Alto, California, U.S.) and Oracle Corp. (Redwood Shores, California) announced a record-breaking data warehousing benchmark for the HP Superdome server, Oracle9i Database and the HP Surestore disk array XP512. This new single-system 1000-GB TPC-H benchmark beat the IBM RS6000 SP550 clustered system — while using half the number of processors — and delivered the best performance per central processing unit (CPU) against results from IBM, NCR and Sun Microsystems.

First, what is TPC? TPC is the Transaction Processing Performance Council, an organization that establishes benchmarks — a set of rules of the game — for computer systems characterized by transaction processing. Some common utility transaction processors are customer information systems (CIS), geographic information systems (GIS) and work management systems (WMS). Large numbers of users (customer service representatives) are continually making inquiries to the system (CIS) to get data (billing information) out of the database. Companies submit test results to TPC. TPC reviews the submittal against the rules and publishes the results — company against company — if accepted.

Second, what is TPC-H? The TPC-H is a decision-support benchmark. It consists of a suite of business-oriented ad-hoc queries and concurrent data modifications. This benchmark illustrates decision-support systems that examine large volumes of data, execute queries with a high degree of complexity and give answers to critical business questions.

A GIS matches the TPC-H benchmark the best, because the benchmark represents environments where users don't know which queries will be executed against a database system-queries are “ad-hoc.” Pre-knowledge of the queries may not be used to optimize the database management system (DBMS). Consequently, query execution times can be very long.

There are other types of benchmarks besides TPC-H. CIS and WMS have much more predictable queries and, therefore, are better matched to other benchmarks.

Third, why should you care? Some people care a lot about who wins at Daytona: Ford, Chevrolet or Dodge. They believe there's a correlation between the company that wins and the durability of the next car they will buy. One thing that you can do with the published TPC-H results is to have a reasonably good apples-to-apples performance comparison between sets of hardware and DBMS for your new or upgraded GIS.

Also, the benchmark will give you an indication of the relative costs of performance. The performance metric reported by TPC-H is called the TPC-H Composite Query-per-Hour Performance Metric. It reflects multiple aspects of the capability of the system to process queries, including the database size, the query processing power when queries are submitted by a single stream, and the query throughput when multiple concurrent users submit queries. The Price/Performance metric is simply the cost divided by the performance metric.

The TPC-H reports on four database sizes: 100 GB, 300 GB, 1000 GB and 3000 GB; a GIS is likely in the latter two.

The best thing about benchmarks is that publishing results shows the world who is “king” — at least for now. Benchmarking drives hardware and software companies to continually improve their products; the price/performance ratio continues to drop steadily. That's good for the T&D business.
Circle 204 on Reader Service Card