[libdispatch-dev] libdispatch for Win32

Fri Apr 22 22:41:53 PDT 2011

Hey Peter, 

It is great to discover that much interest into
porting libDispatch to Windows lately. As you might have read while
browsing the archives of this mailing list, I am working on a win32 port
as well - and have already done just the same as you did. Thanks to the
help of Brent Fulgham we can build on MSVC as well. The idea of using
C++0x lambdas as a workaround for missing blocks support occured to me
too - seems as if the number of similarities between our two ports is
not going to end soon. 

As such I would consider it odd if we spent
time and energy (as already happened far too much) into maintaining and
developing two windows variants of libdispatch. I'd love to merge our
two source trees, just have a look at mine by going to
http://opensource.mlba-team.de/svn/xdispatch/trunk/core/ or
opensource.mlba-team.de/xdispatch for more excessive documentation. I -
too - have concurrent and serial queues working and I am currently
fixing the timers on windows. Please note my annotations to your ideas
below. 

Sincerely, Marius 

> On Sat, 23 Apr 2011 03:07:19 +0000,
DrPizza wrote: 

> So, I've got the basics working tolerably well. Is
this something that people 
> care about/want source for?

> What I've
done so far is as follows:

> Get most of the code building properly in
VC++ 2010:
> * Replace C99 named initializers with old-fashioned
aggregate initializers.
> * Replace gcc typeof with real type names.
> *
Replace gcc's fancy macros with less fancy standard C89 ones.
> * VC++
has no equivalent (AFAIK) to gcc's transparent_union, so insert 
> casts
as necessary.
> * Minor bits and pieces like get rid of the ?: gccism.
>
* Provide minimal Win32 equivalents to missing UNIX headers that seem
>
necessary.

Please see the shims folder within my source tree

>
Blocks:
> * Only Microsoft is in a position to produce built-in block
support for 
> VC++ and I'm sure as hell not going to write a
source-source translator. 
> Instead, I have a C++ lambda wrapper that
works in conjunction with the 
> _f function variants. This seems more
than enough for most purposes.

I moved my lambda implementation into
xdispatch, in order to keep libdispatch
as pure c library. It really
seems to work well.

> Port pthread_workqueues to Win32:
> * Built on
top of "new-style" (Vista and up) Win32 threadpools.
> * Reasonably
complete.
> * Reasonably inadequately tested.

Is there a reason for you
not using the readily ported libpthread_workqueue of Mark Heily? By
using
the older style threadpool you can achieve a broader compatibility
as windows xp still seems quite familiar among users.

> Rework
dispatch_sources:
> * The Windows overlapped I/O model is better than
the traditional UNIX one, 
> but doesn't readily support:
>
DISPATCH_SOURCE_TYPE_READ
> DISPATCH_SOURCE_TYPE_WRITE
> * And Windows
in general doesn't have any good analogues to:
>
DISPATCH_SOURCE_TYPE_VNODE
> DISPATCH_SOURCE_TYPE_SIGNAL
>
DISPATCH_SOURCE_TYPE_PROC
> DISPATCH_SOURCE_TYPE_MACH_RECV
>
DISPATCH_SOURCE_TYPE_MACH_SEND
> * But what I do have instead is initial
support
> DISPATCH_SOURCE_TYPE_OIO (for "overlapped I/O")
> example:
https://gist.github.com/938097 [1]
> * Overlapped I/O supports files,
sockets, named pipes, and more. All of 
> these need testing.
> * The
loss of READ/WRITE/SIGNAL/MACH_* is no big deal on Windows, as they 
>
don't really fit into Win32 anyway. Only one PROC feature (EXIT) 
>
translates into Win32, and I'm not seeing any clearly compelling reason

> to replicate it, as it doesn't seem especially useful. However, the
loss 
> of VNODE is unfortunate, as it both have interesting features.
But this 
> may not be fatal. ReadDirectoryChangeNotificationsW supports
overlapped 
> I/O, so should plumb into my existing
DISPATCH_SOURCE_TYPE_OIO with no 
> changes anyway, though the interface
will not be quite as tidy.
> * I still need to test:
>
DISPATCH_SOURCE_TYPE_DATA_ADD
> DISPATCH_SOURCE_TYPE_DATA_OR
> * I need
to fix:
> DISPATCH_SOURCE_TYPE_TIMER
> I ripped out part of the
machinery to make the code clearer temporarily; 
> now I need to add it
back.
> * I also need to examine the ins and outs of cancellation and
suspension 
> and so on and so forth. Win32 doesn't allow a handle to be
detached from 
> an IOCP except by closing the handle, so there are some
sadnesses there. 
> I'm not sure how much impact they'll have in
practice, possibly none.
> * I am toying with the idea of something
along the lines of 
> DISPATCH_SOURCE_TYPE_WAITABLE, which would perform
a wait on any Win32 
> waitable object (so mutex, event, waitable timer
handles, amongst others) 
> and dispatch a message to a queue when that
wait occurs. This would give 
> us back the one PROC scenario that makes
sense in Win32, too, as you can 
> wait on a process handle, and the
wait resumes when the process 
> terminates. Due to the annoying traits
of WaitForMultipleObjects (it's 
> limited to 64 HANDLEs), however, this
might be a little awkward to 
> implement without moving to a rather
wasteful thread-per-wait model.
> * There are almost certainly memory
leaks, bugs, etc.. My focus has been on 
> validating the general
approach more than writing a bunch of test cases.

Did you re-implement
your own version of kqueues or completely exchange the kevent etc.
calls
within your source code? DISPATCH_SOURCE_TYPE_OIO sounds
interesting, as Mark and I already
discussed a similar solution "as the
way to go" on windows.

> Main queues:
> * From my understanding of Mac
OS X, Windows has no real meaningful 
> equivalent of Cocoa's blessed
main queue. Any thread can have a message 
> pump and associated
windows, which it's then responsible for drawing, 
> etc.. Processes
have an M:N model (M threads, controlling N windows), 
> with each
window being affinitized to its own thread.
> * However, the ability to
post a message back into a window's message loop 
> is obviously
invaluable, so I want to create as close a workalike as 
> makes sense
in Win32. Something that captures the spirit, if not the 
> exact same
API. I've not yet written any code for this, but my plan of 
> action is
to do something along these lines:
> 1) allow creation of serial queues
bound to an HWND or HWNDs. 
> Callbacks posted to these queues will be
pumped into the WndProc 
> one-by-one.
> 2) Either a WndProc hook or a
helper function (or both) to respond to 
> the callbacks posted to the
WndProc and execute them.
> 3) Possibly some convenience helpers to
allow the retrieval of a 
> queue given an HWND and so on.
> Distant
future:
> * There would be certain benefits to ripping out the
pthread_workqueues and 
> using the new-style Win32 threadpools
directly. The Win32 threadpools 
> directly support timers, so they
might allow DISPATCH_SOURCE_TYPE_TIMER to 
> be moved off the
dispatch_mgr queue/thread. Likewise, they directly 
> support overlapped
I/O, so might allow DISPATCH_SOURCE_TYPE_OIO to be 
> moved off the
dispatch_mgr thread too. They also directly support 
> waits on handles,
which would greatly simplify 
> DISPATCH_SOURCE_TYPE_WAITABLE (if I do
indeed go down that route). 
> So the advantages would be many--but I am
wary of diverging too far from 
> the existing source, which is why thus
far I've implemented 
> pthread_workqueues instead; it was the easy
solution.
> * Going in the opposite direction, some might prefer
switching to Windows 
> 2000-style thread pools, so as to support
Windows XP instead. This would 
> work (and I think someone on the list
mentioned that they had implemented 
> pthread_workqueue on that API
already), but it also means eliminating the 
> possibility of the
streamlined implementations described above.

I have to disagree. By
using RegisterWaitForSingleObject on a timer handle you can
easily
achieve similar behaviour using the "old" thread pool api without
needing an
additional manager thread.

> Source code:
> My plan was to
dump it into my github, if people find the whole thing 
> interesting,
though I was going to wait until I'd fixed timer sources, 
> since
they're rather important.
>
> Peter
>

That would be interesting
although I hope we can merge our efforts within the near
future.

Links:
------
[1] https://gist.github.com/938097
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/libdispatch-dev/attachments/20110423/d3d0d514/attachment-0001.html>