Hi Mark, On Aug 20, 2011, at 10:58 PM, Mark Heily wrote:

On 08/16/2011 05:32 PM, Daniel A. Steffen wrote:

...

The Lion libdispatch sources are now available: http://www.opensource.apple.com/release/mac-os-x-107/ http://www.opensource.apple.com/tarballs/libdispatch/libdispatch-187.5.tar.g...

The configure/make buildsystem has been updated to produce a working libdispatch.dylib on Lion that can be used/installed in place of the stock version, see INSTALL for details.

Thanks a lot!

I notice that there is a new pthread_workqueue priority level named WORKQ_BG_PRIOQUEUE. This will need to be added to libpthread_workqueue in order for the new libdispatch to work. Can anyone tell me briefly what the the semantics of this new priority level are?

pretty much as described in libdispatch queue.h: * @constant DISPATCH_QUEUE_PRIORITY_BACKGROUND * Items dispatched to the queue will run at background priority, i.e. the queue * will be scheduled for execution after all higher priority queues have been * scheduled and the system will run items on this queue on a thread with * background status as per setpriority(2) (i.e. disk I/O is throttled and the * thread's scheduling priority is set to lowest value).

By the way, the comment in Darwin libc pthread_workqueue.h needs to be updated to reflect the addition of WORKQ_BG_PRIOQUEUE [1]. It currently reads

"/* Kernel expected target concurrency of the workqueue clients for the three priority queues */"

Now there are four priority queues.. yes, I know this is nit-picking :)

There's another suspicious comment in the same header file that reads

"/* WORKQ_HIGH/DEFAULT/LOW_PRIOQUEUE are the only valid values */"

My guess is that WORKQ_BG_PRIOQUEUE will also be a valid value.

yes, it would be helpful if you could file a bug at bugreporter.apple.com about this so that we can fix the comments for the next OS release. as you have probably seen, libdispatch falls back to WORKQ_LOW_PRIOQUEUE if WORKQ_BG_PRIOQUEUE is not defined.

...

Next steps are to decide how to integrate the new sources into the macosforge repository; as you will discover, there has been significant refactoring/reorganization, making this not completely trivial.

As a first step I was planning to import the Lion source drop on a branch, and then start reorganizing trunk similarly to make trunk changes that are missing from the branch more apparent.

I've taken the pristine Lion sources and created a patchset based on the portability patches that went into the MacOSForge repository. I added some additional patches to work around some new problems introduced with Lion. I'm happy to report that everything builds on Linux and most of the unit tests are working. I've published this work here:

https://www.heily.com/trac/libdispatch

awesome!

This does not include the kqueue abstraction layer that Paolo referred to in a previous post. I'm not willing to make such a large change to the Lion sources when there is no practical benefit (sorry, Paolo). This means that libkqueue will be needed on platforms which lack a native kqueue(2) facility.

I think the best way forward would be to move the current trunk to the tags/ directory for historical purposes, make a new trunk based on the Lion sources, and apply my patches when they are ready. There's only a dozen patches, and they are all fairly trivial.

I have imported the Lion source drop onto a Lion branch in the macosforge libdispatch repo, and have also added our Lion testsuite there, though that is not hooked up to the buildsystem yet (hoping to finish that up today). The Lion testsuite is significantly more extensive, and covers both existing and new Lion libdispatch features. Unfortunately we had never pulled in any of the portability changes from the macosforge testsuite, so it will require some porting to run on other platforms. I'll get onto integrating your patches once I have the testsuite building with the auto* buildsystem. Ideally I'd also like to know what fixes we need to build on FreeBSD before replacing trunk with the Lion branch. Daniel

On Aug 22, 2011, at 6:51 PM, Mark Heily wrote:

Filed bug #10004102.

thanks!

Rather than maintain our own custom portability shims, I'd like to suggest that we import a private copy of libbsd:

http://libbsd.freedesktop.org/wiki/

It provides things like strlcpy(), fgetln() and arc4random() that are BSD in origin. I've used this library successfully in the Linux port of libdispatch.

sounds good, maybe we should resurrect the toplevel compat directory for this purpose ? shims as used currently (outside of the testsuite) are more of a lightweight wrapper around differing underlying primitives...

...

Ideally I'd also like to know what fixes we need to build on FreeBSD before replacing trunk with the Lion branch.

I've tested my patchset on FreeBSD, and as of r12 the patched version of libdispatch 187.5 compiles cleanly on both Linux and FreeBSD.

excellent, thanks! Daniel

On 08/23/2011 11:55 PM, Daniel A. Steffen wrote:

auto* buildsystem for testsuite committed to Lion branch, 'make check' now builds& runs the complete testsuite on Mac OS X. Also cleaned up some Mac OS X-specific auto* buildsystem bits.

Great! I have rebased my porting efforts on the Lion branch, and done some preliminary work to get the testsuite working. Running 'make' in the top level directory will work on FreeBSD and Linux, but running 'make check' will fail part of the way through. Regards, - Mark