[MacRuby-devel] Thread safety in apply example?

[Joshua Ballanco]

On Mon, Jan 24, 2011 at 8:20 PM, Charles Oliver Nutter
<headius at headius.com>wrote:

> Yes, forcing the updates to run serially rather than in parallel. The
> poor-man's fork/join.


More like a map-reduce, I would think. In fact, at some point in the past
the "dispatch extras" (i.e. everything in MacRuby dispatch related that
wasn't part of the "base" libdispatch functionality) contained a #map_reduce
function. Actually, I must admin that it's been some time since I've had a
look at the dispatch extras/dispatch gem...and there seems to be less of
them than I recall...hmmm...


> > There's also a mixin available in the Dispatch gem which abstracts away
> the
> > need for a serial queue for assignments by monkey-patching assignments to
> > all happen on a serial queue.
>
> I'm curious what you mean by this. Can you point out the code? Is it
> actually attempting to rewrite local variable or instance variable or
> array modifications in code?
>

Sorry, I mis-remembered. Not a mix-in, but Dispatch::Proxy from the dispatch
gem is what I was thinking of. It uses the delegator pattern to funnel all
access through a serial queue. Admittedly, this technique is a bit heavy
handed (and, I'm just now realizing, somewhat inefficiently implemented...).
Still, it gets the job done...


> I'm building a JRuby-based mimic API of the MacRuby GCD API:
> https://github.com/headius/jcd
>

That sounds like an excellent idea! One issue that we've come across
repeatedly in MacRuby is how to teach Ruby programmers to write "grown-up"
thread-safe code (and, from my half-hearted attempts to follow what's going
on in JRuby, it seems you've been dealing with this as well). That is,
something like "@my_array[i] += 1" is safe enough when you have a GIL, but
is not when you have true low-level reentrancy. (In fact, even @my_array[i]
= foo is not thread-safe!)

Instead of attempting to bring the world of spin-locks and mutexes (and when
to use one instead of the other) into Ruby, I think it's probably more
useful to introduce a transactional programming model. This LtU post comes
to mind: http://lambda-the-ultimate.org/node/4070

Anyway, I'd be glad to lend a hand (and whatever slice of my limited time I
can spare) to the effort. In particular, test suites/specs are difficult in
this space, as the MacRuby team has learned first-hand.


> I started out using java.util.concurrent Executors directly, and later
> moved to using HawtDispatch, a libdispatch-like API. The latter has
> done a reasonably good job of matching the necessary dispatch_*
> functions in an OO API, and for the pieces that exist the wrapping job
> has been trivial.
>
> What I have now is a mostly-complete Queue and Object, but I'm still
> figuring out how to map the other libdispatch primitives to what
> HawtDispatch provides. The library currently is complete enough to run
> a fork of ControlTower I made called ChaosBazaar:
> http://github.com/headius/ChaosBazaar


So, now I'm curious: have you gotten to the dispatch_source_t part of
libdispatch? I wonder, because this is where I would predict the most
difficulty. In OS X, the dispatch sources are made possible through the use
of a kqueue, which is kinda like a select but at the kernel level. Also,
there might be some difficulty in mapping the semantics directly due to the
high degree of asynchrony achieved by libdispatch. For example: the
following is perfectly legal:

myq = Dispatch::Queue.new
myq.async do
  myq.cancel!
  *keep doing stuff on 'myq'*
end

...because cancelation is dispatched, asynchronously, to be executed by the
same queue (or, well, with the same queueing semantics) as the queue you are
canceling.

Also, I'm really excited by the clone of ControlTower. I should warn you
that ControlTower only includes ~40% of the GCD-ishness that was originally
planned for it. I originally intended that reading from/writing to the
socket should occur using dispatch_sources, since that is more efficient
than waiting on a blocking read. In fact, I have (somewhere) a branch with
this implemented, but the concurrency bugs that developed were so cryptic
and difficult to chase down, that I eventually gave up...resulting in what
you have now. I think you'll find the vestiges of these attempts in the
"gcd-ify" branch...but I can't vouch at all for their functional state.

Anyway, I think this could be the start of a great thing for Ruby in
general...

Cheers,

Josh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/macruby-devel/attachments/20110125/8ecf48e8/attachment-0001.html>