- libdispatch-dev - lists.macosforge.org

linux pthread_workqueue support
by Pierre Habouzit 07 Feb '12

07 Feb '12

Hi, I've written a proof-of-concept module of what could be a Linux way of supporting thread pool regulation, and in particular pthread_worqueue kind of API. [0] I've published the code on [1] and you can have a look at how the kernel support "works" on [2] (asciidoc generated, may be a bit late from the one in the repository). On [3] there is a small "library", more like hints of how to use the API. I sadly don't have enough time to make a tentative port of mark's libpthread-workqueue using this kernel module, hence wonder if there are people who would be interested into that kind of work. Enjoy… PS: the kernel module has been tested very lightly with stuff like [4], I've, *I think*, found all the possible issues and are reasonably confident with the correctness of the code. Though I'd strongly advise you to use it in a virtual machine at first. It mucks about with the scheduler, very bad things can happen which often results in deadlocked kernel at best, kernel panics at worst… [0] https://plus.google.com/102418560027665797123/posts/jQMUfb53c5n [1] http://git.madism.org/?p=~madcoder/pwqr.git;a=summary [2] http://madism.org/~madcoder/pwqr.html [3] http://git.madism.org/?p=~madcoder/pwqr.git;a=blob;f=lib/libpwqr.c;hb=HEAD [4] http://git.madism.org/?p=~madcoder/pwqr.git;a=blob;f=test/test.c;hb=HEAD -- Intersec <http://www.intersec.com> Pierre Habouzit <pierre.habouzit(a)intersec.com> | Chief Scientist Mob : +33 (0)6 1636 8131 37 Rue Pierre Lhomme 92400 Courbevoie

1 0

libdispatch on Fedora 16
by Mark Heily 27 Dec '11

27 Dec '11

Good news: libdispatch r197 now builds and runs on Fedora Linux. I have published unofficial source RPMs for libdispatch and it's Linux dependencies (libkqueue, libpthread_workqueue, and libBlocksRuntime). This will allow you to build libdispatch on Fedora 16, and hopefully serve as a basis for porting to Red Hat Enterprise Linux and other RPM-based distributions. Here's a link to the release page: http://mark.heily.com/project/libdispatch/release/00197-f16 Cheers, - Mark

1 0

Get access to my pictures and more
by Antonio Carlos de Faria 24 Nov '11

24 Nov '11

Hey, I have created a Netlog profile with my pictures, videos, blogs and events and I want to add you as a friend so you can see it. You first need to register on Netlog! When you log in, you can create your own profile. Take a look: http://en.netlog.com/go/mailurl/-bT0yNTI4Mzc0MTE3Jmw9MSZnbT0xMiZ1PSUyRmdvJT… Cheers, Antonio Carlos de ---------------------------------------------------------------- Don't want to receive invitations from your friends anymore? http://en.netlog.com/go/mailurl/-bT0yNTI4Mzc0MTE3Jmw9MiZnbT0xMiZ1PSUyRmdvJT…

1 0

Static Analysis for leaks with dispatch_objects
by Mike Lewis 26 Oct '11

26 Oct '11

Hi, I was wondering if there were any plans on adding something to the equivalent of the annotations __attribute__((cf_returns_retained)) to GDC function calls for dispatch objects. Since they're not CFTypes I realize support to clang's static analyzer need to be added. After using a lot of the new dispatch_io calls and having manage dispatch_data_t objects, it seems like there is a much larger use case for analysis support. Thanks, Mike -- Michael Lewis lolrus.org mikelikespie(a)gmail.com

3 3

Re: [libdispatch-dev] lib dispatch worker threads may loop depending on compiler optimization
by Paolo Bonzini 20 Sep '11

20 Sep '11

On 09/19/2011 06:32 PM, Joakim Johansson wrote: > <http://libdispatch.macosforge.org/trac/ticket/35#comment:10> Thanks. It turns out it is _not_ a bug in GCC at all, but in libdispatch. Compiling the .i file shows the following: movq %r13, %rax movq $1, 0(%r13) movq %r12, 16(%r13) #APP # 110 "/tb/builds/thd/sbn/2.4/src/thirdparty/libdispatch/197/src/src/queue_internal.h" 1 xchg %rax, 64(%rbx) # 0 "" 2 #NO_APP testq %rax, %rax movq %rbp, 24(%r13) movq $0, 8(%r13) je .L172 The APP/NO_APP markers signify that an asm is being used rather than sync builtins. Looking at the source code indeed reveals this... // GCC generates suboptimal register pressure // LLVM does better, but doesn't support tail calls // 6248590 __sync_*() intrinsics force a gratuitous "lea" instruction, with resulting register pressure #if 0 && defined(__i386__) || defined(__x86_64__) #define dispatch_atomic_xchg(p, n) ({ typeof(*(p)) _r; asm("xchg %0, %1" : "=r" (_r) : "m" (*(p)), "0" (n)); _r; }) #else #define dispatch_atomic_xchg(p, n) ((typeof(*(p)))__sync_lock_test_and_set((p), (n))) #endif ... which is missing parentheses like #if 0 && (defined(__i386__) || defined(__x86_64__)) The asm is wrong, because it doesn't have a clobber for "memory". That fixes the testcase. Paolo

2 1

lib dispatch worker threads may loop depending on compiler optimization
by Joakim Johansson 17 Sep '11

17 Sep '11

We just filed http://libdispatch.macosforge.org/trac/ticket/35 after sometimes encountering spinning worker threads when running under high load on multi-cpu machines. Various versions of gcc will optimize and reorder the code differently, but with 4.5.1 we run into this problem. Is this a fundamental problem with libdispatch itself that will rear it’s ugly head if/when the Mac OS X system compiler is changed in the future (to optimize more aggressively), or is it a bug in the gcc 4.5.1 optimizer? The Lion source code seems more or less unchanged here, so even if we tested this with the current r197 code base, the behavior should be the same when integrating the latest source drop. Thoughts? Thanks, Joakim

10 36

CFLite packages for Linux
by Mark Heily 03 Sep '11

03 Sep '11

Since there are some libdispatch tests that rely on CoreFoundation, I've ported CFLite to Debian Linux and released the unofficial binary and source packages here: http://mark.heily.com/project/CFLite I'm not able to run the tests yet because CFLite does not provide some of the necessary functions that libdispatch needs [1]. These functions are in CFRunLoop.c which is not built by default. I'll add it to the build and see what happens. - Mark [1] clang -DHAVE_CONFIG_H -I. -I../config -I.. -I.. -I../compat/libMachShims/include -Wall -g -fblocks -I../compat/libMachShims/include -I../compat/libkqueue/include -I../compat/libpthread_workqueue/include -MT dispatch_cf_main.o -MD -MP -MF .deps/dispatch_cf_main.Tpo -c -o dispatch_cf_main.o dispatch_cf_main.c dispatch_cf_main.c:38:3: warning: implicit declaration of function 'CFRunLoopPerformBlock' is invalid in C99 [-Wimplicit-function-declaration] CFRunLoopPerformBlock(CFRunLoopGetMain(), ... ^ dispatch_cf_main.c:38:25: warning: implicit declaration of function 'CFRunLoopGetMain' is invalid in C99 [-Wimplicit-function-declaration] CFRunLoopPerformBlock(CFRunLoopGetMain(), ... ^ dispatch_cf_main.c:38:45: error: use of undeclared identifier 'kCFRunLoopDefaultMode' CFRunLoopPerformBlock(CFRunLoopGetMain(), kCFRunLo... ^ dispatch_cf_main.c:56:2: warning: implicit declaration of function 'CFRunLoopRun' is invalid in C99 [-Wimplicit-function-declaration] CFRunLoopRun(); ^ 3 warnings and 1 error generated.

2 2

Lion branch porting status
by Mark Heily 31 Aug '11

31 Aug '11

Just a quick update on the work I've been doing with the Lion branch: As of r30, all of the unit tests compile under Linux. There's a problem with the test harness that prevents 'make check' from running, but I've tried several of the tests individually and they appear to work. Most of the patches are very rough, specific to Linux only, and not ready for submission. I'll polish them up over the next few weeks and start sending them to the list. I've created a compat/ directory and included a private copy of libbsd, libkqueue, libpthread_workqueue, and libMachShims. I wrote libMachShims based on some earlier work porting launchd to Linux. It provides a subset of the Mach headers and emulates some functions like mach_absolute_time(). This avoids the need to sprinkle #ifdefs all around the Mach-specific sections of code. Once things settle down with the libdispatch porting effort, I'd like to take a look at if/how launchd could be combined with libdispatch to serve as a foundation for writing network daemons. Cheers, - Mark

4 6

Lion libdispatch sources posted & next steps for macosforge repo
by Daniel A. Steffen 25 Aug '11

25 Aug '11

The Lion libdispatch sources are now available: http://www.opensource.apple.com/release/mac-os-x-107/ http://www.opensource.apple.com/tarballs/libdispatch/libdispatch-187.5.tar.… The configure/make buildsystem has been updated to produce a working libdispatch.dylib on Lion that can be used/installed in place of the stock version, see INSTALL for details. Next steps are to decide how to integrate the new sources into the macosforge repository; as you will discover, there has been significant refactoring/reorganization, making this not completely trivial. As a first step I was planning to import the Lion source drop on a branch, and then start reorganizing trunk similarly to make trunk changes that are missing from the branch more apparent. Before I get started on that we should make sure that all pending non-controversial patches are integrated onto trunk, does anybody have a list of what is still outstanding? (I guess this is where more consequent use of the trac ticket system on http://libdispatch.macosforge.org/trac/report would be paying off..) Daniel

3 7

Re: [libdispatch-dev] Lion libdispatch sources posted & next steps for macosforge repo
by Daniel A. Steffen 18 Aug '11

18 Aug '11

Thanks, committed with slight amendments for style/whitespace and some simplifications to use existing facilities; in particular on Linux use the main thread as the signal handler thread if it is going to stick around forever anyway rather than spin up a separate signal handler thread from dispatch_main(). Daniel On Aug 17, 2011, at 2:46, Joakim Johansson wrote: > Ooops, reviewing Marks patch it seems he actually integrated the two I submitted as well, so just the ‘linux specific’ ones should be fine (as they weren’t really upon further inspection…) > > So I think just Marks patch cluster from: >> http://lists.macosforge.org/pipermail/libdispatch-dev/2011-June/000532.html > http://lists.macosforge.org/pipermail/libdispatch-dev/2011-June/000559.html > > Should be all that is pending... > > Cheers, > > Joakim > > On 17 aug 2011, at 11:43, Joakim Johansson wrote: > >> That’s great news, thanks! >> >> I think a list of outstanding minor patches (submitted after the large Christmas package from Jordan) which are pending would be: >> >> Fixes for a couple of warnings from our statical code analyzer: >> http://lists.macosforge.org/pipermail/libdispatch-dev/2011-May/000529.html >> http://lists.macosforge.org/pipermail/libdispatch-dev/2011-May/000530.html >> >> And a couple of Linux specific ones from Mark Heily: >> http://lists.macosforge.org/pipermail/libdispatch-dev/2011-June/000532.html >> http://lists.macosforge.org/pipermail/libdispatch-dev/2011-June/000559.html >> >> Sounds like a very good idea to start using the trac system more consistently going forward… >> >> Cheers, >> >> Joakim >> >> On 16 aug 2011, at 23:32, Daniel A. Steffen wrote: >> >>> The Lion libdispatch sources are now available: >>> http://www.opensource.apple.com/release/mac-os-x-107/ >>> http://www.opensource.apple.com/tarballs/libdispatch/libdispatch-187.5.tar.… >>> >>> The configure/make buildsystem has been updated to produce a working libdispatch.dylib on Lion that can be used/installed in place of the stock version, see INSTALL for details. >>> >>> Next steps are to decide how to integrate the new sources into the macosforge repository; as you will discover, there has been significant refactoring/reorganization, making this not completely trivial. >>> >>> As a first step I was planning to import the Lion source drop on a branch, and then start reorganizing trunk similarly to make trunk changes that are missing from the branch more apparent. >>> >>> Before I get started on that we should make sure that all pending non-controversial patches are integrated onto trunk, does anybody have a list of what is still outstanding? (I guess this is where more consequent use of the trac ticket system on http://libdispatch.macosforge.org/trac/report would be paying off..) >>> >>> Daniel >>> _______________________________________________ >>> libdispatch-dev mailing list >>> libdispatch-dev(a)lists.macosforge.org >>> http://lists.macosforge.org/mailman/listinfo.cgi/libdispatch-dev >> >> _______________________________________________ >> libdispatch-dev mailing list >> libdispatch-dev(a)lists.macosforge.org >> http://lists.macosforge.org/mailman/listinfo.cgi/libdispatch-dev >

1 0