http://ibm.biz/demystperf

Just a reminder: this blog is a mirror of the "main" site:

• http://ibm.biz/demystperf
• https://www-304.ibm.com/connections/blogs/performance

But, as the above is planned to be sunset I will publish everything here as well. Just give me enough time to resynch both blogs.

SAPS Olympics: 10 charts for fun and profit

I’ve prepared 10 charts I think they are interesting enough. You may use them for your fun or your professional presentations. Of course the data source is the official SAPS (SD 2-Tier) results posted at http://global.sap.com/solutions/benchmark/sd2tier.epx. Please, feel free to ask for additional charts not included here, as I plan for new ones.

The list here:

1. Count of benchmarked systems by benchmark version.
   
2. All the SAPS results.
   
3. All The SAPS results (logarithmic scale).
   
4. The latest SAPS results (logarithmic scale).
   
5. Top 1 SAPS evolution.
   
6. SAPS evolution (50 benchmarks moving average).
   
7. SAPS evolution (50 benchmarks moving average) (logarithmic scale).
   
8. All the SAPS per core results.
   
9. SAPS per core evolution (50 benchmarks moving average).
   
10. Top 1 SAPS per core evolution.
    

 

Chart #1: Benchmarked systems by benchmark version

 

Chart #2: All the SAPS results

 

Chart #3: All the SAPS results (logarithmic scale)

 

Chart #4: The latest SAPS Results (logarithmic scale)

 

Chart #5: Top 1 SAPS System Evolution

 

 

Chart #6: SAPS evolution (50 benchmarks moving average)

 

 

Chart #7: SAPS evolution (50 benchmarks moving average) (logarithmic scale)

 

 

Chart #8: All the SAPS per core results

 

 

 

Chart #9: SAPS per core evolution (50 benchmarks moving average)

 

 

 

Chart #10: Top 1 SAPS per core evolution

 

 

SAPS Olympics: 10 years ago

Let’s go back around ten years, what did happen in the SAPS arena back then? In this analysis I have considered the 130 systems that were measured with the SAP R/3 Enterprise 4.70 benchmark specification. The first one was in 2003 April 4^th, a Mitsubishi Apricot with certification number 2003032, and the last one was published in Jul 4^th 2005, an Egenera pBlade 950-000084 with certification number 2005037. That is, more than two years of time span. All the numbers and calculations are based on the official SAPS (SD 2-Tier) results posted at http://global.sap.com/solutions/benchmark/sd2tier.epx.

 

Remember that you have to be careful when comparing two SAPS values if they correspond to two different benchmark specs. You have to take into account software release effects and other benchmark specs changes. This is like the need to take into account the inflation rates when comparing the value of year 2008 dollars to year 2014 dollars. Current SAPS are heavier than past ones.

SAP Technology Partners

The SAP Technology Partners (a SAP concept) that were actively benchmarking SAPS. Fujitsu sometimes appears alone and sometimes with Siemens, but I’ve grouped both in the count.

 

By CPU family

The Intel Xeon was the dominating family, and this is a constant in the history of SAPS Olympics. AMD Opteron had strong presence. Intel Itanium was alive, and they were also IBM POWER5, UltraSPARC IV, SPARC64 V, and PA-RISC times.

By Operating System

Operative systems seen: Windows Server 2000 and 2003, IBM AIX 5, Solaris 9, Linux SLES 8, and HP-UX 11.

 

By Relational Database Management System

Relational Database Management Systems seen:  Microsoft SQL Server 2000, IBM DB2 UDB 8 and 9.5, Oracle 9i and SAP DB. All of them transitioning from 32-bit to 64-bit flavors.

Absolute Number of SAPS (SAPS per system)

Gold -> Fujitsu PRIMEPOWER 2500 with 128 SPARC64 V @2080 MHz processors: 105820 SAPS.

Silver-> IBM eServer p5 Model 595 with 64 POWER5 @1900 MHz: 100700 SAPS.

Bronze -> Sun Fire Model E25k with 72 UltraSPARC IV @1200 MHz: 51070 SAPS.

The last ->  Fujitsu Siemens Computers PRIMERGY Model BX300 with 1 Intel Pentium M @1800 MHz: 830 SAPS.

  

 

SAPS per core / per thread

In those days the processor and core terms were managed by marketing and, consequently, blurred and misdefined.  The problem is that sometimes processors are equal to cores, and sometimes they are not. In the SAPS official table the cores (and threads) column are zero, and only the processor column is filled. Thus, I cannot offer a significative analysis unless I take big time analyzing system by system (and that is not in my near scope.

SAPS Olympics: 5 years ago

What did happen in the SAPS arena five years ago? Was the server landscape back then very different from now? I’m extending the SAPS Olympics here, and have in plan to go even earlier in the past (and to perform more subtle analysis). All the numbers and calculations are based on the official SAPS (SD 2-Tier) results posted at http://global.sap.com/solutions/benchmark/sd2tier.epx.

I have considered the 111 systems that were measured with the SAP ERP 6.0 (2005) benchmark specification. The first one was in 2007 May 11^th, a HP ProLiant DL585 G2 with certification number 2007034, and the last one was published in Dec 19^th 2008, an IBM System x3650 M2 with certification number 2008079. That is, more than one and a half year of time span. 

Besides the SAP software version, there are two significant differences in the benchmark specification with respect to the current one: UNICODE was not required (in fact the analyzed list does not contain UNICODE results, even there were a few), and the response time limit was set to 2 seconds instead of  the current 1 second. As many of you already know, you cannot compare two SAPS values if they correspond to two different benchmark specs. You have to take into account software release effects. This is like the need to take into account the inflation rates when comparing the value of year 2008 dollars to year 2014 dollars. Current SAPS are heavier than past ones.

SAP Technology Partners

Which SAP Technology Partners (a SAP concept) were actively benchmarking SAPS? Here are the results. Fujitsu sometimes appears alone and sometimes with Siemens, but I’ve grouped both in the count. Sun and Fujitsu shared the SPARC based systems, and in fact the same system is published twice (one for each brand).

The main changes are: Sun then is Oracle now, Egenera is not currently an active SAP partner system vendor, and Cisco had not presence five years ago.

By CPU family

They were IBM POWER6 times (now is POWER8), and dual / quad core Intel Xeon Harpertown, Tigerton and Gainestown (first Nehalem) generations. Intel Itanium is dead by now, and AMD Opteron has declined since then. Regarding SPARC there were UltraSPARC T2 / T2 Plus and SPARC64 VI and VII.

By Operating System

Operative systems seen: Windows Server 2003 and 2008, IBM AIX 5.3 and 6.1, Solaris 10, Linux RHEL 5.2 and SLES 10, and HP-UX 11i.

By Relational Database Management System

Relational Database Management Systems seen:  Microsoft SQL Server 2003 and 2008, IBM DB2 9 and 9.5, Oracle 10g and SAP MaxDB.

Absolute number of SAPS (SAPS per system)

Gold --> Sun / Fujitsu SPARC Enterprise Server M9000 with 64 SPARC64 VII @2520 MHz processors (256 cores, 512 threads): 196570 SAPS.

Silver --> IBM POWER 595 with 32 POWER6 @5.0 GHz (64 cores, 128 threads): 177950 SAPS.

Bronze --> Sun / Fujitsu SPARC Enterprise Server M9000 with 64 SPARC64 VI @2400 MHz processors (128 cores, 256 threads): 129420 SAPS.

The last --> Sun / Fujitsu SPARC Enterprise Server M3000 with 1 SPARC64 VII @2520 MHz processors (4 cores, 8 threads): 4130 SAPS.

.

SAPS per core

Gold --> IBM System x3650 M2 with 2 chips Intel Xeon Processor X5570 @2930 MHz (4 cores, 8 threads): 3191 SAPS/core.

Silver --> HP ProLiant DL380 G6 with 2 chips Intel Xeon Processor X5570 @2930 MHz (4 cores, 8 threads): 3125 SAPS/core.

Bronze --> Fujitsu Siemens Computers PRIMERGY Model TX300 S5 / RX300 S5 with 2 chips Intel Xeon Processor X5570 @2930 MHz (4 cores, 8 threads): 2956 SAPS/core.

These first positions are held by the Intel Xeon Processor X5570, the by then newcomer Nehalem microarchitecture. The three above systems were benchmarked the latest days of the ERP 6.0 (2005) benchmark spec, just the end of the analyzed interval.

Lowest --> Sun Blade X8450 with 4 chips Quad-Core Intel Xeon Processor E7340 @2400 MHz (16 cores, 16 threads): 657 SAPS/core.

SAPS per thread

Gold --> Fujitsu Siemens Computers PRIMERGY Model TX300 S4 | RX300 S4 with 2 Dual-Core Intel Xeon Processor X5260 @2330 MHz (4 cores, 4 threads): 1845 SAPS/thread.

Silver --> HP ProLiant DL385 G5p with 2 Quad-Core AMD Opteron Processor 2384 @2700 MHz (8 cores, 8 threads): 1722 SAPS/thread.

Bronze --> IBM System x3650 M2 with 2 Intel Xeon Processor X5570 @2930 MHz (8 cores, 16 threads): 1560 SAPS/thread.

Lowest --> Sun / Fujitsu SPARC Enterprise T5440 with 4 UltraSPARC T2 Plus @1400 MHz (32 cores, 256 threads) 147 SAPS/thread.                 

The latest positions are held by UltraSPARC T2 / T2 Plus servers.

Note: this blog is a mirror from http://ibm.biz/demystperf.