Looking for info on MacRuby's JIT
Hi all, Greetings. New to this list, and to MacRuby. I was just skimming over the source code from trunk, but wasn't able to find a document that describes the current design of Roxor. I only found requirements, build instructions, coding styles and the like. The thing I find very confusing is that in vm.cpp, RoxorCore::RoxorCore(), there's a call to ee->DisableLazyCompilation(). This disables LLVM JIT's lazy compilation. Is it the case that Roxor compiles every method called by the toplevel script? Or how is Roxor feeding stuff into the JIT? It seems to me that in main.cpp, main() calls ruby_options(), where the whole source file gets parsed into MRI's Ruby AST. That AST is supposed to go into RoxorCompiler::compile_main_method(), and out comes LLVM IR. And then after optimizations passes, GETCORE()->compile() puts the JIT in action, turns LLVM IR into native code, and then invoke the IMP. I must have missed something...but if Roxor JIT compiles a method upon first invocation, where is this behavior implemented. Any pointers to documents, blog posts or mails would be truely appreciated. Thanks in advance, Raven
Hi Ravenex, On Dec 7, 2009, at 8:36 AM, ravenex wrote:
Greetings. New to this list, and to MacRuby. I was just skimming over the source code from trunk, but wasn't able to find a document that describes the current design of Roxor. I only found requirements, build instructions, coding styles and the like.
I don't have any answers either, but let me try to collect the relevant information, if only to improve my own understanding.
The thing I find very confusing is that in vm.cpp, RoxorCore::RoxorCore(), there's a call to ee->DisableLazyCompilation(). This disables LLVM JIT's lazy compilation. Is it the case that Roxor compiles every method called by the toplevel script? Or how is Roxor feeding stuff into the JIT?
Any pointers to documents, blog posts or mails would be truely appreciated.
As I understand it, the MacRuby compiler is known as Roxor and is defined in: * http://svn.macosforge.org/repository/ruby/MacRuby/trunk/compiler.h * http://svn.macosforge.org/repository/ruby/MacRuby/trunk/compiler.cpp Roxor depends on some language facilities, e.g.: * http://svn.macosforge.org/repository/ruby/MacRuby/trunk/llvm.h * http://svn.macosforge.org/repository/ruby/MacRuby/trunk/objc.h But mostly on the VM (which is essential for breaking the Global Interpreter Lock): * http://svn.macosforge.org/repository/ruby/MacRuby/trunk/vm.h * http://svn.macosforge.org/repository/ruby/MacRuby/trunk/vm.cpp * http://svn.macosforge.org/repository/ruby/MacRuby/trunk/vm_eval.c * http://svn.macosforge.org/repository/ruby/MacRuby/trunk/vm_method.c
It seems to me that in main.cpp, main() calls ruby_options(), where the whole source file gets parsed into MRI's Ruby AST. That AST is supposed to go into RoxorCompiler::compile_main_method(), and out comes LLVM IR. And then after optimizations passes, GETCORE()->compile() puts the JIT in action, turns LLVM IR into native code, and then invoke the IMP. I must have missed something...but if Roxor JIT compiles a method upon first invocation, where is this behavior implemented.
In terms of front-ends, there is both an AOT and JIT compiler. It looks like both are called from: http://svn.macosforge.org/repository/ruby/MacRuby/trunk/main.cpp With the AOT compiler called via: rb_vm_aot_compile -> RoxorAOTCompiler::compile_main_function and the JIT invoked from: ruby_run_node -> RoxorCompiler::compile_main_function If I understand your question, it is what method explicitly does the compilation during JIT? That would seem to be: compile_node * http://svn.macosforge.org/repository/ruby/MacRuby/trunk/compiler.cpp
Function * RoxorCompiler::compile_main_function(NODE *node) { current_instance_method = true;
Value *val = compile_node(node);
If so, then the question becomes "Where is implemented, compile_node and does it cache pre-compiled methods"? I don't know, but hopefully this gives you somewhere to look; or will inspire someone who knows more to provide a better answer.... -- Ernie P.
Greetings. New to this list, and to MacRuby. I was just skimming over the source code from trunk, but wasn't able to find a document that describes the current design of Roxor. I only found requirements, build instructions, coding styles and the like.
There's currently no documentation on the inner workings of MacRuby. Any help is welcome ;-) Most knowledge is in the code or Laurent's head (and a bit in other commiters' head ;-)). Feel free to ask questions on the ML, though answers may take some time.
The thing I find very confusing is that in vm.cpp, RoxorCore::RoxorCore(), there's a call to ee->DisableLazyCompilation(). This disables LLVM JIT's lazy compilation.
Yes I added it to be sure that LLVM does not use its lazy compilation mechanism (that is know not to work correctly in multiple threads). It's not needed in MacRuby because we do it our own way.
I must have missed something...but if Roxor JIT compiles a method upon first invocation, where is this behavior implemented.
In JIT mode, the conversion of Ruby code to LLVM code (or more strictly speaking LLVM C++ objects) is done as soon as a Ruby script is loaded. But LLVM's optimization phase and native code generation of a function is only done when this function is called. The place where it's done is hard to find though: it's done using a functionality of the Objective-C runtime: we define the "resolveClassMethod:" and "resolveInstanceMethod:" on NSObject (in vm.cpp, in Init_PreVM) http://developer.apple.com/mac/library/documentation/cocoa/reference/Foundat...: There are also a few other cases where we compile a method even if it's not called, for instance when a method is aliased or a module included. For blocks it's done in dispatcher.cpp, in rb_vm_prepare_block. That's a function called just before any block is given to a function. In AOT mode, of course, we always compile and optimize everything (blocks and methods) beforehand.
participants (3)
-
Ernest N. Prabhakar, Ph.D.
-
ravenex
-
Vincent Isambart