On Fri, 11 Sep 2009, Matt Wright wrote:

On a similar note to some of the other things I mentioned. When the project page talks about the kernel support, are we just talking kevent/kqueues here? Or are there more darwin kernel level things at work. I'm looking more for the limitations when targeting Leopard, rather than porting to non-Darwin, and moving towards using libdispatch linked to an application without needing to load kexts/etc on Leopard.

I also found it interesting that you built it so it could run without blocks. I found when writing WW that life was ever so much easier using blocks to my advantage during implementation.

Hi Matt-- I've got libdispatch building on FreeBSD, and am currently debugging it, so I can speak a bit to the dependencies I've identified that aren't (yet) in FreeBSD. This is probably pretty similar to the list of things in Snow Leopard but not Leopard: - libdispatch_init is invoked directly by the libsyscall bootstrap in libSystem. At least on FreeBSD, we can't just switch to an attribute constructor because of the way rtld performs locking around library init routines, so I'm pondering this still. This is an issue for a Leopard port also. - libdispatch includes support for monitoring Mach ports. Not in scope for me on FreeBSD, so I just added a HAVE_MACH configure test. Not an issue for a Leopard port. - libdispatch uses Mach semaphores. In FreeBSD, I've substituted POSIX semaphores. I've run into at least one issue doing so, as libdispatch maintains parallel state and appears not to propagate an initial declared 'value' to the Mach semaphore. Once I understand the problem better, I'll post a more specific query for the Apple folks. Not an issue for a Leopard port. - libdispatch relies on mach_absolute_time, which for the purposes of FreeBSD I've replaced with clock_gettime. There are some semantic issues about which clock to use -- FreeBSD supports CLOCK_UPTIME, which is basically the same, but Linux doesn't, so maybe CLOCK_MONOTONIC, despite meaning something different from mach_absolute_time, is the way to go. Not an issue for a Leopard port. - There are some issues with libdispatch's internal use of the legacy API. My goal has been not to compile in legacy.c, and this has caused a few build nits so far. The only such nit not yet resolved in the libdispatch tree is in the kqueue debugging server built into libdispatch. I think a similar goal of non-support for legacy APIs for a Leopard port is reasonable. - libdispatch uses the pthread_workqueue primitive by default, but can also manage its own thread pool using portable pthread primitives. For the FreeBSD port, we'll do the latter for now, but we have an agenda item for the FreeBSD developer summit later this week to talk about GCD, and in particular, pthread work queues. For a Leopard port, you should just be able to use the "pure" pthread version of the code, which configure picks up in the auto* build system. - FreeBSD requires including pthread_np.h to get non-portable pthread routines. Not an issue for the Leopard port. - libdispatch relies on the preallocation of pthread thread-specific data (TSD) keys for it in Apple's libSystem, and hooks p to them using a non-portable interface. We won't be able to make use of this design choice in FreeBSD or other systems, and I assume Leopard probably also doesn't have those keys reserved. My understanding that this design choice was made for performance reasons and I have a version that uses pthread_create_key in a portable way. Unfortunately, this is what is triggering my rtld bootstrap problems, so I'm still thinking about this. Leopard would also be affected. - libdispatch uses Apple's userspace malloc zone API; for FreeBSD purposes, I've made it a simple wrapper around malloc, which is functionally correct and given jemalloc, possibly as performant. I'll need to benchmark this later. Not an issue for Leopard. - libdispatch relies on EV_DISPATCH and EV_RECEIPT, which don't (yet) appear in FreeBSD's kqueue code. I need to look at the SL xnu bits some more and decide on the right approach, but likely merging them back to FreeBSD is the way to go. EV_RECEIPT is in Leopard, but EV_DISPATCH is not, which may be an issue for the Leopard port. This isn't an issue I fully understand yet but I can probably comment more once I've done a bit more reading. - libdispatch relies on EVFILT_USER to post events to itself via kqueue from one thread to another. This also needs to be merged back to FreeBSD, and isn't in Leopard, so will be an issue. There are probably workarounds that don't involve adding these to the kernel (perhaps using a pipe to do a loopback notification, or something more elegant?) but the cleanest solution is to have the kernel functionality. I guess Apple folk might (or might not) be able to comment on the chances of this appearing in a future Leopard update. - libdispatch does a run-time query of the kernel build version number for debugging reasons. For now I've just ifdef'd __APPLE__ this and crashinfo management for FreeBSD. Not an issue for Leopard, I assume. - libdispatch must be built with at least -march=i486 for the gcc built-in atomics to work. I'm using i686 currently. - libdispatch blocks routines won't work on FreeBSD currently; FreeBSD builds fine with clang/llvm, but some work on the C runtime will be required. In particular, it would be good to have the runtime blocks support in FreeBSD's libgcc, I suspect. Non-blocks will apply to Leopard as well, but Apple has already carefully ifdef'd all the blocks parts. All of the above will also apply to Linux, Solaris, and other more traditional UNIXy platforms. In addition, ports to those platforms will stumble significantly over highly integrated use of kqueue, which is perfectly suited for this purpose but not present on those systems. I've not yet done an analysis of how easily something like epoll (or select?) might be substituted, but I'd guess the specific semantics of kqueue, and especially the modifications in SL, are quite important. I'll post patches once I'm a little further, but right now my goals are: (1) Get the auto* build framework to work on SL. I ran into a problem with the libtool shipped with SL, but have been advised that if I do all the libtoolification on FreeBSD it should then work on Mac OS X. I'll need to add MIG-related bits to the build still. (2) Finish debugging (and resolving) problems bootstrapping libdispatch on FreeBSD so I can get to the point where the unresolved kqueue semantic issues can be dealt with. If things go well, I should be able to post patches this weekend, but debugging the libdispatch startup process involves understanding a lot of subtle issues so we'll see. :-) Robert N M Watson Computer Laboratory University of Cambridge

Matt

On 11 Sep 2009, at 11:23, Shantonu Sen wrote:

...

See Kevin's post in the list archives: <http://lists.macosforge.org/pipermail/libdispatch-dev/2009-September/000000.html>

Building (and deploying) libdispatch requires the rest of the dependencies of the libSystem core system library. This will be supported using the DarwinBuild tool.

Shantonu

Sent from my MacBook

On Sep 11, 2009, at 2:52 AM, Matt Wright wrote:

...

Hey guys,

Right, despite signing up to the mailing list this morning. I managed to miss the introduction message because I had to go to work before I'd done the confirmation email ;).

I'm very interested in contributing, I've had my own implementation of libdispatch's API for a week or so. Spent the morning having a little compare between how I've done mine and how yours is actually written. I don't have a paid ADC membership so much first exposure to GCD was the weekend that Snow Leopard came out, by the following weekend I was enamoured with it to the extent that I wanted to write it for Leopard.

So the first thing that struck me is that it doesn't build. My first guess was that it relied on being built in the presence of the darwin sources? I haven't had enough time to have a look. I'm especially interested in the line in the project description about kernel support. Are we just talking kqueue here? Or an additional kernel module?

From a personal standpoint, I wrote my implementation (http://daagaak.github.com/WiganWallgate/) in order to be able to use libdispatch calls on Leopard. I've several personal projects that I really wanted to use GCD in but without it in Leopard would mean moving my code to be Snow Leopard only. All-in-all writing my own version was a very entertaining weekend, even if it ends up getting dropped in favour of making libdispatch work on Leopard/iPhone/etc.

I wonder if you could shed some light on what kind of environment you need to build libdispatch in to get a successful compile. The brief "open xcode and hit build" attempt that I made this morning was not so much a success.

Matt

--

Matt Wright matt@sysctl.co.uk http://www.sysctl.co.uk _______________________________________________ libdispatch-dev mailing list libdispatch-dev@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/libdispatch-dev

_______________________________________________ libdispatch-dev mailing list libdispatch-dev@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/libdispatch-dev

On Sat, 12 Sep 2009, Kevin Van Vechten wrote:

On Sep 12, 2009, at 2:15 AM, Robert Watson wrote:

...

- libdispatch uses Mach semaphores. In FreeBSD, I've substituted POSIX semaphores. I've run into at least one issue doing so, as libdispatch maintains parallel state and appears not to propagate an initial declared 'value' to the Mach semaphore. Once I understand the problem better, I'll post a more specific query for the Apple folks. Not an issue for a Leopard port.

A count is kept in user space and updated with atomic operations. The general idea is to avoid trapping into the kernel except when blocking is required (or when signaling a blocked thread is required).

There were two things that worried me: (1) The following comment in semaphore.c: // Mach semaphores appear to sometimes spuriously wake up. Therefore, // we keep a parallel count of the number of times a Mach semaphore is // signaled (6880961). (2) That the initial semaphore value, 'value' passed to dispatch_semaphore_create(), isn't propaated to the kernel semaphore when it's allocated on-demand in _dispatch_posix_semaphore_create(). If the kernel semaphore is just the wakeup primitive, this may not be an issue, but I haven't read what's there deeply enough to reason about that yet. An answer from someone who knows already would be most helpful :-). Since POSIX semaphores are already a blended kernel/userspace implementation on FreeBSD, it sounds like I could ditch the userspace mirrored state, although we'll have to see whether that is true of other UNIX systems. Robert N M Watson Computer Laboratory University of Cambridge

On Sep 11, 2009, at 11:12 AM, Matt Wright wrote:

On a similar note to some of the other things I mentioned. When the project page talks about the kernel support, are we just talking kevent/kqueues here? Or are there more darwin kernel level things at work. I'm looking more for the limitations when targeting Leopard, rather than porting to non-Darwin, and moving towards using libdispatch linked to an application without needing to load kexts/ etc on Leopard.

In terms of API, there's not much difference between what is provided by the kernel in Leopard and Snow Leopard. However, there were fixes made in Snow Leopard to issues exposed by libdispatch (for example, to portions of kqueue that weren't heavily utilized before libdispatch). I suspect much of the core functionality of libdispatch could be made to work fine on Leopard, but there may be some problematic edge cases with a few of the dispatch source types.

I also found it interesting that you built it so it could run without blocks. I found when writing WW that life was ever so much easier using blocks to my advantage during implementation.

Blocks are great and we expect virtually all clients of libdispatch will use them because of the convenience they provide. The choice not to use blocks in the implementation of libdispatch itself was due to the desire to avoid any locks or heap allocations in the "fast path" of the dispatch mechanism. Kevin