[MacRuby-devel] Running the experimental branch

[Charles Oliver Nutter]

Richard Kilmer wrote:
> I don't necessarily agree that microbenchmarks are _NOT_ good indicators.
> I think a specific microbenchmark is not, but the point of these micro-
> benchmarks are to judge the speed of specific runtime units.  If you run
> one really fast it is not a good indicator of general performance, but 
> if you
> run all of them quickly, its a good indication that the general performance
> will exceed the performance of the runtimes you are comparing yourself
> to.

Microbenchmarks are a good indicator of performance for the range of 
features that those microbenchmarks test. If all the microbenchmarks you 
examine are testing low-level VM operations and fixnum math, they're 
only good indicators of the performance of low-level VM operations and 
fixnum math. They may or may not be indicators of general performance, 
and simply running more of sthen does not improve the quality of the suite.

Key features like core class performance, object/memory/GC performance, 
IO/network performance, and so on are often absent from such benchmarks, 
and have at least as much bearing on general application performance as 
low-level VM operations and fixnum math.

- Charlie