[libdispatch-dev] Updates regarding the status of libdispatch on Windows

[DrPizza]

> The big question to raise now is wether merging back to macosforge.org
> is possible or not. We had to change a lot of the sources and some of
> the changes do not really contribute to the maintainability of the code,
> e.g. MSVC still does not support named intializers in structs.
> Additionally I split queue.c into several subfiles, such as debug.c and
> so on, to make the different implementations for the different platforms
> easier to find. As such it will be quite difficult to provide small
> patches instead of one or two bigger ones. And there is DrPizza/Peter's
> port as well, although I do not know how much of his work is tested.

I've just pushed a bunch of updates to my codebase [1] that includes ports of 
most of the tests that I pulled from the macosforge git. It's not actually 
clear to me if the macosforge git is current or how closely aligned it is 
with the "official" releases at opensource.apple.com, though I presume 
they're all reasonably close.

The test cases are generally working. The drift test will probably never 
really work properly on Windows. Windows has only millisecond accuracy for 
waits, but the test is demanding sub-tenth of a millisecond accuracy for 
jitter and drift. Such demands are unlikely to ever be met.

I've taken a different tack than you for several parts of the code. Although 
it would be nice to ensure that a common codebase works everywhere, the 
platforms I'm interested in are Windows and Mac OS X (I've not yet built my 
fork on my Mac, but it's on my todo list...), and I want the library to feel 
as native as possible on both platforms. This has led me to do a few things:

1) Not even try to emulate kevent/kqueue, and instead use an I/O Completion 
   Port
2) Create a new OIO source type for overlapped I/O
3) Not yet bother porting READ and WRITE sources (though I may grudgingly 
   implement a select()-based fallback for use with sockets)
4) Create new per-thread queues that are similar in concept to the main queue, 
   but unlike the main queue are not singletons. This is for convenient 
   interoperability with WndProcs; in Windows, _any_ thread can have windows 
   associated with it, not just the main thread.

Some implementation notes for the per-thread queues are available at [2]. 
It's not especially complicated, really; they're essentially just a 
generalization of the main queue. I store a field in each queue now to denote 
that the queue is "manually drained"; the main queue and the threaded queues 
each have this field filled in, everything else leaves it empty. A manually-
drained queue alerts the windowing system each time it has work to do, and 
if the thread ends its TLS destructors will attempt to drain the queue.

I have broken out the Cocoa (main queue)/WndProc interoperability functions 
into a separate header/source file, interop.h/interop.c. There's a certain 
amount of symmetry between them: in each case libdispatch needs to alert the 
windowing toolkit that there are callbacks to be drained (using a Mach port 
for Cocoa, and a window message for Windows), in each case the windowing 
toolkit needs to be able to drain the actual messages, and so on. The Cocoa 
hooks are currently a little more complete--I've not yet created equivalents 
to the begin thread/end thread callbacks--but the principle seems sound enough.

I haven't yet actually _checked_ my per-thread queues; I was concentrating 
first on porting the existing test-cases rather than writing my own. The 
implementation is shared with the main queue, and I've checked that and it 
passes the tests, so I don't think there are any particularly egregious flaws 
remaining.

I am sorely tempted to pull timers out of the kevent system and make them use 
pthread_workqueues directly; this would allow me, on both Windows and Mac OS X 
to use the leeway facility. This change would be very invasive, which is why 
I've not done so yet, but on first glance it looks like an easier approach 
trying to shoehorn leeways into the existing manager queue.

There are quite a few oddities in the original code that I pulled from git. 
For example, at [3], the claim is made that a zero interval means "disabled". 
I do not see which code actually makes that claim true. The interval seems 
only to be used at [4], when calcualting the number of missed intervals for 
periodic timers. A zero interval has a catastrophic result on this calculation.

Similarly, the timer #defines at [5] don't completely make sense to me. 
They're used as if masks at [6], even though one of the values is zero. It 
also appears that intervals are useless on "absolute" timers [7], even though 
the documentation [8] makes no mention of this restriction (and there is no 
obvious reason for such a restriction to exist), and the intervals are 
converted as if they're meant to be used [9].

It's more than possible that I am missing something terribly important here, 
of course, so the above may be hopelessly misguided.

Overall, I've tried to keep the source tree looking as close to the original 
as was practical. The major departure is the creation of the platform 
hierarchy [10], which is where the major support infrastructure required for 
Win32 is kept. This includes Win32 versions of certain portions of pthreads, 
including TLS storage with TLS destructors (an egregious oversight from 
Win32), and a pthread_workqueue wrapper for native Win32 threadpools. There 
are also Win32 versions of POSIX semaphores, though the semaphore class uses 
Win32 semaphores directly now anyway.

I'm not averse to some alternative layout, but I don't think sticking this 
stuff in the shims directory is the best place for it; the pthread, semaphore 
and pthread_workqueue wrappers, in particular, are far more substantial than 
mere shims.

Regards,

Peter

[1]: https://github.com/DrPizza/libdispatch
[2]: http://blog.quiscalusmexicanus.org/post/4929297740/grand-central-dispatch-for-win32-things-still-to-do
[3]: http://libdispatch.macosforge.org/trac/browser/trunk/src/source.c#L646
[4]: http://libdispatch.macosforge.org/trac/browser/trunk/src/source_kevent.c#L871
[5]: http://libdispatch.macosforge.org/trac/browser/trunk/src/source_internal.h#L127
[6]: http://libdispatch.macosforge.org/trac/browser/trunk/src/source_kevent.c#L637
[7]: http://libdispatch.macosforge.org/trac/browser/trunk/src/source_kevent.c#L867
[8]: http://developer.apple.com/library/mac/documentation/Performance/Reference/GCD_libdispatch_Ref/Reference/reference.html#//apple_ref/c/func/dispatch_source_set_timer
[9]: http://libdispatch.macosforge.org/trac/browser/trunk/src/source.c#L688
[10]: https://github.com/DrPizza/libdispatch/tree/master/libdispatch/platform