BENCHMARKING COMPROMISE

Use a set of several real applications, covering a range of application areas and program structure.

Problems:

• Applications are still not average programs, so results can still not reliably be extrapolated.
• It is misleading to combine performance on different programs since it is unclear how they should be weighed.
• Compiler technology can still make a big difference.
• Still fails to account for the effort needed in manual performance tuning for many parallel architectures

Examples:

• The SPEC suites
• The Perfect Club suites