[libdispatch-dev] Lion libdispatch sources posted & next steps for macosforge repo

[Daniel A. Steffen]

Hi Mark,

On Aug 20, 2011, at 10:58 PM, Mark Heily wrote:

> On 08/16/2011 05:32 PM, Daniel A. Steffen wrote:
>> The Lion libdispatch sources are now available:
>> http://www.opensource.apple.com/release/mac-os-x-107/
>> http://www.opensource.apple.com/tarballs/libdispatch/libdispatch-187.5.tar.gz
>> 
>> The configure/make buildsystem has been updated to produce a working
>> libdispatch.dylib on Lion that can be used/installed in place of the
>> stock version, see INSTALL for details.
>> 
> 
> Thanks a lot!
> 
> I notice that there is a new pthread_workqueue priority level named WORKQ_BG_PRIOQUEUE. This will need to be added to libpthread_workqueue in order for the new libdispatch to work. Can anyone tell me briefly what the the semantics of this new priority level are?

pretty much as described in libdispatch queue.h:

 * @constant DISPATCH_QUEUE_PRIORITY_BACKGROUND
 * Items dispatched to the queue will run at background priority, i.e. the queue
 * will be scheduled for execution after all higher priority queues have been
 * scheduled and the system will run items on this queue on a thread with
 * background status as per setpriority(2) (i.e. disk I/O is throttled and the
 * thread's scheduling priority is set to lowest value).

> By the way, the comment in Darwin libc pthread_workqueue.h needs to be updated to reflect the addition of WORKQ_BG_PRIOQUEUE [1]. It currently reads
> 
>    "/* Kernel expected target concurrency of the workqueue clients
>        for the three priority queues */"
> 
> Now there are four priority queues.. yes, I know this is nit-picking :)
> 
> There's another suspicious comment in the same header file that reads
> 
>  "/* WORKQ_HIGH/DEFAULT/LOW_PRIOQUEUE are the only valid values */"
> 
> My guess is that WORKQ_BG_PRIOQUEUE will also be a valid value.

yes, it would be helpful if you could file a bug at bugreporter.apple.com about this so that we can fix the comments for the next OS release.

as you have probably seen, libdispatch falls back to WORKQ_LOW_PRIOQUEUE if WORKQ_BG_PRIOQUEUE is not defined.

>> Next steps are to decide how to integrate the new sources into the
>> macosforge repository; as you will discover, there has been
>> significant refactoring/reorganization, making this not completely
>> trivial.
>> 
>> As a first step I was planning to import the Lion source drop on a
>> branch, and then start reorganizing trunk similarly to make trunk
>> changes that are missing from the branch more apparent.
>> 
> 
> I've taken the pristine Lion sources and created a patchset based on the portability patches that went into the MacOSForge repository. I added some additional patches to work around some new problems introduced with Lion. I'm happy to report that everything builds on Linux and most of the unit tests are working. I've published this work here:
> 
>     https://www.heily.com/trac/libdispatch

awesome!

> This does not include the kqueue abstraction layer that Paolo referred to in a previous post. I'm not willing to make such a large change to the Lion sources when there is no practical benefit (sorry, Paolo). This means that libkqueue will be needed on platforms which lack a native kqueue(2) facility.
> 
> I think the best way forward would be to move the current trunk to the tags/ directory for historical purposes, make a new trunk based on the Lion sources, and apply my patches when they are ready. There's only a dozen patches, and they are all fairly trivial.

I have imported the Lion source drop onto a Lion branch in the macosforge libdispatch repo, and have also added our Lion testsuite there, though that is not hooked up to the buildsystem yet (hoping to finish that up today).
The Lion testsuite is significantly more extensive, and covers both existing and new Lion libdispatch features. Unfortunately we had never pulled in any of the portability changes from the macosforge testsuite, so it will require some porting to run on other platforms.

I'll get onto integrating your patches once I have the testsuite building with the auto* buildsystem.

Ideally I'd also like to know what fixes we need to build on FreeBSD before replacing trunk with the Lion branch.

Daniel