[libdispatch-dev] libdispatch port to FreeBSD (was: [libdispatch-changes] [27] trunk/src)

Mon Sep 14 13:05:10 PDT 2009

Dear all--

I have committed an initial autoconf/automake/libtool build framework to 
libdispatch svn, and a first cut at conditionally compiling Apple-specific 
pieces of libdispatch.  This allows libdispatch to build on both FreeBSD and 
Mac OS X Snow Leopard.

Some notes for FreeBSD hackers:

- Several kqueue extensions (EVFILT_USER, NOTE_TRIGGER, EV_RECEIPT, and
   EV_DISPATCH) are required in the FreeBSD kernel, and hopefully will go into
   the FreeBSD tree in the next couple of days.

- Blocks are definitely not yet supported -- there seems to be a lot of
   interest, so I'm optimistic.  As compilers aren't my forte, any assistance
   in this department would be most appreciated. :-)

- On FreeBSD, there is no support for Apple's pthread work queue primitive, so
   I'm using the fallback worker code in libdispatch.  The FreeBSD kernel
   community needs to have a conversation about the right way to address this
   requirement, but in the mean time the worker model should be fine.

- There are also some kqueue events present in FreeBSD but not in Mac OS X;
   these may require tweaks to libdispatch to fully support by GCD-aware
   applications using these FreeBSD events.

- I have developed and tested on a 32-bit Intel system; the comments below on
   -march on Mac OS X likely apply for FreeBSD/amd64 users as well.

Some notes for Mac OS X hackers:

- For now, you will need to hand-trim the -march line from src/Makefile.
   This should be fixed soon by some further autoconf work.

- When I use the externally-built dylib on Snow Leopard, test programs spin.
   This may be due to interference between the libSystem dispatch code and the
   dylib -- possibly forcing the use of pthread_create_key rather than reserved
   keys would help.

- I have not attempted to configure/build for Leopard as yet, but this change
   should make that a lot easier.  EVFILT_USER is likely the biggest obstacle.

Some notes for Linux hackers:

- The lack of kqueue support is the critical obstacle in a port to Linux.  It
   may be that a kqueue emulation library based on epoll (or even libevent)
   could fill this gap.  If someone wants to look at this, pay very careful
   attention to kqueue semantics, as libdispatch relies on them heavily.

Please consider this a work-in-progress -- don't be surprised when it doesn't 
build, let alone work.  :-)

Finally: this work would not have been possible without the support of Jordan 
Hubbard, Kevin Van Vechten, Dave Zarzycki, Bill Siegrist, and Apple.  I am 
surprised by how easily the porting has gone so far -- it's a credit to their 
careful planning and quality of implementation that it has gone so smoothly.

Thanks also to Peter O'Gorman and Colin Percival for their assistance with 
auto*, my nemesis.

With any luck, there will be more good porting news to report in the near 
future!

Thanks,

Robert N M Watson
Computer Laboratory
University of Cambridge

---------- Forwarded message ----------
Date: Mon, 14 Sep 2009 11:46:19 -0700 (PDT)
From: source_changes at macosforge.org
Reply-To: libdispatch-dev at lists.macosforge.org
To: libdispatch-changes at lists.macosforge.org
Subject: [libdispatch-changes] [27] trunk/src

   Revision
       27

   Author
       robert at fledge.watson.org

   Date
       2009-09-14 11:46:18 -0700 (Mon, 14 Sep 2009)

   Log Message

Port of libdispatch to the FreeBSD operating system, which consisted of
disabling various Mac OS X-specific portions of the library (such as Mach),
providing portable versions of certain services internally (POSIX semaphores
instead of Mach semaphores, clock_gettime() instead of mac_absolute_time()),
and providing an autoconf/automake/libtool build infrastructure.  This is
just a first cut, refinement to follow.

Conditionalize portions of the libdispatch build as follows:

HAVE_MACH
 	If Mach is present, support Mach ports throughout libdispatch.

USE_LIBDISPATCH_INIT_CONSTRUCTOR
 	If defined, add the 'constructor' attribute to libdispatch_init so
 	that it runs as part of library initialization.  This is not needed
 	in Mac OS X libSystem due to explicit invocation of libpthread_init
 	by libsyscall, but is for a stand-alone library.

HAVE_DECL_EVFILT_SESSION, HAVE_DECL_NOTE_SIGNAL, HAVE_DECL_NOTE_REAP,
HAVE_DECL_NOTE_NONE, HAVE_DECL_VQ_UPDATE, HAVE_DECL_VQ_VERYLOWDISK
 	Conditionally support kqueue events that are not present on FreeBSD.

DISPATCH_NO_LEGACY
 	Expand the scope of this existing macro to allow libdispatch to
 	build without legacy.c.

HAVE_MACH_ABSOLUTE_TIME
 	Wrap mach_absolute_time() in _dispatch_absolute_time(), and
 	provide a portable clock_gettime() implementation.  Some further
 	refinment will be required here.

USE_APPLE_SEMAPHORE_OPTIMIZATIONS
 	Only use non-portable assembly optimizations of dispatch semaphores
 	if supported.

HAVE_SEM_INIT
 	Use POSIX semaphores if Mach semaphores aren't present; further
 	ifdef'ing possibly required here.  If neither HAVE_MACH nor
 	HAVE_SEM_INIT are present, the compile will fail.

USE_APPLE_CRASHREPORTER_INFO
 	Only define Mac OS X crashreporter state if requested.

HAVE_PTHREAD_WORKQUEUES
 	Only compile in support for pthread work queues if present;
 	otherwise force use of normal worker threads.

HAVE_PTHREAD_KEY_INIT_NP
 	If defined, use non-portable allocation of pthread keys--otherwise,
 	use pthread_create_key().

USE_APPLE_TSD_OPTIMIZATIONS
 	Only use non-portable assembly optimizations if dispatch thread-
 	specific data if this is defined.

HAVE_MALLOC_CREATE_ZONE
 	If not present, use compat/malloc_zone.h which implements the malloc
 	zone APIs using normal malloc(3).

Also fix some portability nits for building, such as casting before using
long as a format string for several types.  Note that this port will not
build or run on a stock FreeBSD release or svn distribution yet, as it
relies on kqueue extensions not committed back to FreeBSD yet.

There are several known issues, such as how best to map CLOCK_UPTIME or
CLOCK_MONOTONIC struct timespec timestamps into a Mach absolute time
int64_t number space.  Also, the POSIX semaphore ifdefing is adequate but
not complete.

   Modified Paths

  *  trunk/src/benchmark.c
  *  trunk/src/dispatch.h
  *  trunk/src/internal.h
  *  trunk/src/legacy.h
  *  trunk/src/os_shims.h
  *  trunk/src/private.h
  *  trunk/src/queue.c
  *  trunk/src/queue_internal.h
  *  trunk/src/semaphore.c
  *  trunk/src/semaphore_internal.h
  *  trunk/src/shims.c
  *  trunk/src/source.c
  *  trunk/src/source.h
  *  trunk/src/source_private.h
  *  trunk/src/time.c

   Diff

     Modified: trunk/src/benchmark.c (26 => 27)