Richard Kilmer wrote:
I don't necessarily agree that microbenchmarks are _NOT_ good indicators. I think a specific microbenchmark is not, but the point of these micro- benchmarks are to judge the speed of specific runtime units. If you run one really fast it is not a good indicator of general performance, but if you run all of them quickly, its a good indication that the general performance will exceed the performance of the runtimes you are comparing yourself to.
Microbenchmarks are a good indicator of performance for the range of features that those microbenchmarks test. If all the microbenchmarks you examine are testing low-level VM operations and fixnum math, they're only good indicators of the performance of low-level VM operations and fixnum math. They may or may not be indicators of general performance, and simply running more of sthen does not improve the quality of the suite. Key features like core class performance, object/memory/GC performance, IO/network performance, and so on are often absent from such benchmarks, and have at least as much bearing on general application performance as low-level VM operations and fixnum math. - Charlie