Entropy-Based Analysis of Benchmarks for Instruction Set Simulators

Instruction-Set Simulators (ISSs) are widely used to simulate the execution of programs for a target architecture on a host machine. They translate the instructions of the program that should be executed into instructions of the host Instruction-Set Architecture (ISA). The performance of ISSs strongly depends on the implementation and the instructions that should be executed. Therefore, benchmarks that are used to compare the performance of ISSs should contain a variety of instructions. Since many benchmarks are written in high-level programming languages, it is usually not clear to the user which instructions are underlying the benchmarks. In this work, we present a tool that can be used to analyze the variety of instructions used in a benchmark. In a multi-stage analysis, the properties of the benchmarks are collected. An entropy-based metric is used to measure the diversity of the instructions used by the benchmark. In a case study, we present results for the benchmarks Whetstone, Dhrystone, Coremark STREAM, and stdcbench. We show the diversity of those benchmarks for different compiler optimizations and indicate which benchmarks should be used to test the general performance of an ISS.