[libdispatch-dev] I missed the intro message ;)

[Kevin Van Vechten]

On Sep 13, 2009, at 4:20 AM, Robert Watson wrote:

> On Sat, 12 Sep 2009, Kevin Van Vechten wrote:
>
>> On Sep 12, 2009, at 2:15 AM, Robert Watson wrote:
>>
>>> - libdispatch uses Mach semaphores.  In FreeBSD, I've substituted  
>>> POSIX semaphores.  I've run into at least one issue doing so, as  
>>> libdispatch maintains parallel state and appears not to propagate  
>>> an initial declared 'value' to the Mach semaphore.  Once I  
>>> understand the problem better, I'll post a more specific query for  
>>> the Apple folks.  Not an issue for a Leopard port.
>>
>> A count is kept in user space and updated with atomic operations.   
>> The general idea is to avoid trapping into the kernel except when  
>> blocking is required (or when signaling a blocked thread is  
>> required).
>
> There were two things that worried me:
>
> (1) The following comment in semaphore.c:
>
>        // Mach semaphores appear to sometimes spuriously wake up.   
> Therefore,
>        // we keep a parallel count of the number of times a Mach  
> semaphore is
>        // signaled (6880961).

Earlier in the project, we saw some evidence of mach semaphores  
spuriously waking up under very heavy load.  We weren't able to  
conclusively prove this was happening but decided to assume it was  
based on the fact that the primary use of mach semaphores on Mac OS X  
is to back pthread conditionals (pthread_cond_t) which treat  
KERN_ABORTED (Mach equivalent to EINTR) as a spurious wakeup.

More recently we discovered a memory smasher (which has already been  
resolved) that may have also contributed the observed "spurious  
wakeup" behavior, so this remains an unconfirmed hypothesis.

> (2) That the initial semaphore value, 'value' passed to
>    dispatch_semaphore_create(), isn't propaated to the kernel  
> semaphore when
>    it's allocated on-demand in _dispatch_posix_semaphore_create().   
> If the
>    kernel semaphore is just the wakeup primitive, this may not be an  
> issue,
>    but I haven't read what's there deeply enough to reason about  
> that yet.
>    An answer from someone who knows already would be most helpful :-).
>
> Since POSIX semaphores are already a blended kernel/userspace  
> implementation on FreeBSD, it sounds like I could ditch the  
> userspace mirrored state, although we'll have to see whether that is  
> true of other UNIX systems.

Yes, the Mach semaphores are merely a sleep/wakeup primitive.  The  
Mach semaphore backing the dispatch_semaphore_t is lazily initialized  
on demand in _dispatch_semaphore_create_port().  As long as the  
dispatch semaphore value remains non-negative, no Mach semaphore is  
needed.

Ultimately we'd like to enhance dispatch semaphores to share the same  
per-thread cached Mach semaphore that's used for dispatch_sync()  
instead of requiring their own.

Kevin