Front Page

The Liberty Research Group

Experimental Results

The following experimental results were obtained using an OSE prototype built by retrofitting Intel's Electron compiler. A detailed description of the experimental setup can be found in Section 4.1 of the OSE paper.

All the performance numbers reported below correspond to actual running times, measured on real hardware. Experiments have been run on unloaded HP i2000 Itanium 1 machines. Running times for a program refer to the machine cycles spent executing a program's source-code functions; time spent in system calls or system library functions is ignored.

Tuning the OSE compiler

The following results were obtained during the tuning phase of the OSE compiler. They refer to 15 SPECcint2000 and SPECcfp2000 benchmarks. Since this experiment's purpose was tuning rather than evaluation, each benchmark's training inputs were used both for profiling and for evaluating the executable's runtime. In this graph:

  • The red bar corresponds to the performance of OSE with no compile-time pruning and no static performance estimation.
  • The gray bar corresponds to the performance of OSE with no compile-time pruning but with static performance estimation. Notice that performance losses due to estimator inaccuracies are small.
  • The blue bar corresponds to the performance of the full OSE system; it uses both static performance estimation and compile-time pruning. Notice that compile-time pruning does not incur any noticeable performance penalties.

    • OSE compile times

    The following graph shows the compile times incurred by the OSE prototype for the 15 benchmarks used in the previous experiment. The gray and blue bars correspond to OSE's compile times without and with compile-time pruning respectively. Static performance estimation is used in both cases. Notice that compile-time pruning allows big compile-time savings.

    Performance evaluation

    In the following graph, OSE is applied to 14 applications that did not participate in the prototype's tuning phase. The application set includes some SPECint95 benchmarks, some Mediabench benchmarks, and the UNIX application yacc. In this case, separate sets of inputs were used for profiling and for evaluation. Notice that OSE performs much better on these applications than on SPEC, probably because the original Electron compiler was more carefully tuned for SPEC benchmarks.