[MacRuby-devel] Looking for info on MacRuby's JIT

[ravenex]

@Ernie,
 
Thank you for the links. Actually I'm have read MRI 1.9's code
base before, so I was able to find quite a few "entry points"
that I needed to get up to speed with MacRuby. The problem is
I'm not familiar with the Objective-C runtime, and that's big
blocker for me, since MacRuby is tightly integrated with it.
 
@Vincent
 
> There's currently no documentation on the inner workings of MacRuby.
> Any help is welcome ;-)
> Most knowledge is in the code or Laurent's head (and a bit in other
> commiters' head ;-)). Feel free to ask questions on the ML, though
> answers may take some time.
 
Thank you very much for the reply. I'll see if I can understand
enough of what's going on, before I can make myself useful :-)
 
>> The thing I find very confusing is that in vm.cpp, RoxorCore::RoxorCore(),
>> there's a call to ee->DisableLazyCompilation(). This disables LLVM JIT's
>> lazy compilation.
 
> Yes I added it to be sure that LLVM does not use its lazy compilation
> mechanism (that is know not to work correctly in multiple threads).
> It's not needed in MacRuby because we do it our own way.
 
I saw the same changes have been applied to Unladen Swallow on LLVMdev.
Looks like it's common for language implementations to build their own
lazy mechanisms when using LLVM for its JIT.
 
> In JIT mode, the conversion of Ruby code to LLVM code (or more
> strictly speaking LLVM C++ objects) is done as soon as a Ruby script
> is loaded. But LLVM's optimization phase and native code generation of
> a function is only done when this function is called.
 
> The place where it's done is hard to find though: it's done using a
> functionality of the Objective-C runtime: we define the
> "resolveClassMethod:" and "resolveInstanceMethod:" on NSObject (in
> vm.cpp, in Init_PreVM)
> http://developer.apple.com/mac/library/documentation/cocoa/reference/Foundation/Classes/NSObject_Class/Reference/Reference.html#//apple_ref/occ/clm/NSObject/resolveClassMethod:
> There are also a few other cases where we compile a method even if
> it's not called, for instance when a method is aliased or a module
> included.
 
So that's where...yes, I got it!
 
> For blocks it's done in dispatcher.cpp, in rb_vm_prepare_block. That's
> a function called just before any block is given to a function.
 
> In AOT mode, of course, we always compile and optimize everything
> (blocks and methods) beforehand.
 
Just to make sure I got it right:
 
MacRuby's VM is called Roxor. Currently in the code in trunk doesn't use
any interpreter mode; Ruby code is always compiled before is can be run.
Compilation can be done in either JIT or AOT mode.
(I thought there was an experimental branch that actually used LLVM's
interpreter to speed up eval, what happened to that?)
 
JIT mode is the default mode. A translation unit is a Ruby source file.
As a Ruby source file gets loaded, it's first parsed by a modified parser
taken from MRI 1.9, and turned into NODE*, pretty much the same as the
original Ruby AST. Then, RoxorCompiler will compile AST into LLVM IR,
which is kept and JITted later on demand.
Because in MacRuby all Ruby classes inherit from NSObject, MacRuby uses
mechanism from the Objective-C runtime to drive the lazy compilation
behavior: original implementations of NSObject's resolveClassMethod: and
resolveInstanceMethod: are replaced with MacRuby's own ones, which act as
a hook and call the JIT when method implementations are being resolved.
For normal Ruby methods, resolveClassMethod_imp() and
resolveInstanceMethod_imp() both call RoxorCore::resolve_method(), which
calls RoxorCore::resolve_methods(), which calls RoxorCore::resolve_method(),
and that'll call RoxorCore::compile() which compiles LLVM IR into native
code.
 
In AOT mode, Ruby code gets parsed in Ruby AST, compiled into LLVM IR by
RoxorAOTCompiler, and then the LLVM bitcode is dumped to a temporary file.
The bitcode is then translated into assembly by llc, then assembled by
invoking gcc, and then optionally linked with other stuff.
 
I'll continue reading the source for now...
 
Thank you all,
Raven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/macruby-devel/attachments/20091208/aa282536/attachment-0001.html>