[libdispatch-dev] libdispatch on VS 2008 and VS 2010 64bit

[DrPizza]

> Unfortunately I disagree with you in some aspects. I cannot see any real
> benefit from using (u)intptr_t within libdispatch as in my eyes we do not
> need a datatype scaling with the architecture but should prefer another one
> promising a fixed range on all architectures.
As of right now, libdispatch on Mac OS X and FreeBSD, and libxdispatch on 
Linux (and Solaris, and any other UNIX) already have this "scaling" behaviour, 
thanks to their use of the LP64 model. Given that all bar one of the platforms 
already scale, why not round out the set and make Windows scale too?

> This should increase
> portability while ensuring feature parity again. As such I clame size_t the
> victim as there is no standard concerning its size - only its minimum size.
> Bad enough it is used within the public api of both kqueues and libdispatch.
> But it can be replaced by (u)int32_t in most other cases within the
> implementation easily. Pointers is another story of course, but those are
> already fixed within the xdispatch windows port.
Using (u)int32_t breaks compatibility with libdispatch, and makes the code 
_less functional_. I honestly do not understand why you regard it as the way 
forward.

And I disagree that it increases portability to fix the size at 32-bit. Fixing 
it at "pointer-size int" increases portability. Fixing it at 32-bit means that 
some things that work on 32-bit platforms don't neatly scale up to 64-bit.

> So why stay with a 32 bit datatype while compiling a 64bit binary? For me the
> main reason to use 64bit is bigger memory and (some) calculations running
> more efficiently. But libdispatch is using neither of them - only
> applications built on top of libdispatch.
I don't think it's quite as simple as that. Consider DATA_ADD sources. 64-bit 
libdispatch allows you to add larger values without risk of rollover. It does 
this today, already, and you can build programs that depend on it. You can 
break up work into larger blocks without having to worry about overflowing the 
counter. Classic advantage of native 64-bit integers, right?

But currently libxdispatch on Win64 breaks programs that do this, because long 
is only 32-bit in Win64.

> As you wrote the limitations of a
> 32bit variable will have no real impact on semaphore counts and priorities. I
> claim they also work very well for the allowed number of queues and the
> suspend/reference count. As such we can reduce future efforts by using a
> fixed size datatype.
But ds_pending_data (and related types) might quite reasonably roll past 4 GB, 
which is why libdispatch uses long for these values, and why replacing long 
with int32 isn't reasonable.

> On Windows the InterlockedIncrement functions and their relatives (except the
> pointer operations of course) will work on 32bit LONG variables on all
> architectures - for working on LONGLONG variables on 64bit architectures we
> would have to use InterlockedIncrement64 and friends. So all is safe when
> using the solution explained above.
It isn't safe; at the moment you are using InterlockedExchangeAdd64 on 32-bit 
longs in 64-bit builds.

> So for me the remaining problem is the kqueue interface needing the
> dispatch_source api to use unsigned long for compatibility and I welcome any
> suggestions solving this. Still I consider it difficult to do public API
> changes to support a single operating system (Windows) when resulting in
> changes on various other operating systems.
Such changes are source and binary compatible on UNIX platforms anyway. It's 
only on Windows that errant uses of long in client code will have to be altered.

> On Windows it is sadly very
> difficult to provide the same sources as on unix platforms (speaking e.g. of
> DISPATCH_SOURCE_TYPE_PROC, DISPATCH_SOURCE_TYPE_SIGNAL,
> DISPATCH_SOURCE_TYPE_VNODE, ...) as well.
Sure. But others (TIMER, DATA_ADD, DATA_OR) are easy. TIMER and DATA_ADD can 
readily overflow a 32-bit value, and there's no reason not to allow DATA_OR 
to use 64-bit values except for the whole "long" issue.

There's no particularly good reason why READ might not indicate more than 4 GB 
of data available to read, though I imagine in practice such uses are rare. 
That said, Apple's source indicates that this is a case specifically tested; 
see line 530 of source_kevent.c [1].

Systematically replacing (unsigned) long with (u)intptr_t will mean that you 
have source code and functional compatibility between both LP64 (UNIX) systems 
and LLP64 (Windows) systems. It's the most straightforward, consistent change 
to make.

> So for me the only practical solution seems to provide a slightly different
> dispatch_source (and possibly kqueue) api on Windows, something I am not
> really happy about.
The big question I have about kqueue on Windows is, do you think that kqueue 
is useful to have for applications other than libxdispatch? If kqueue is a 
generally useful API, then it makes sense to strive to have a workalike API 
on Windows (and IMHO, that includes replacing longs with intptr_ts--if it's 
safe to pass a pointer on UNIX, it must be safe to pass a pointer on Win32).

However, if you think it's likely that the only time kqueue will ever be used 
on Windows is as an implementation detail for libxdispatch on Windows, then 
there's no great value, IMO, in nailing down every last detail of kqueue's 
behaviour if that proves particularly complex. After all, libdispatch simply 
doesn't make use of every single facet of kqueue's behaviour. There's little 
value in working to implement features that never end up getting used.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Ultimately, I guess I must be missing something. The changes required to use 
(u)intptr_t consistently are not that significant. They result in no functional 
change on UNIX platforms, and extend the behaviour of Windows to match UNIX. 
Why is it superior to make UNIX platforms less functional, just so that they 
match the behaviour of Windows, even though there's no good reason to have that 
behaviour in Windows anyway?

I think also it's clear that the intent was to have these things scale. After 
all, libdispatch was written for an LP64 platform. The odd man out is Windows, 
and we can fix up Windows' oddities easily enough. We just have to stop using 
long.

[1]: http://libdispatch.macosforge.org/trac/browser/trunk/src/source_kevent.c#L530