- libdispatch-dev - lists.macosforge.org

Updates regarding the status of libdispatch on Windows
by Marius Zwicker 20 Jun '11

20 Jun '11

I am very glad to report good news on the topic libdispatch on windows. Yesterday we managed to eradicate the last obvious bugs and all tests and can now completely build and run libdispatch by using MSVC (Visual Studio 2010) on windows. All tests seem to pass or fail the same way they currently do on Linux, so at least we have reached feature parity ;) - Please see the test results attached [1], the responsible code can be found at http://opensource.mlba-team.de/svn/xdispatch/tags/0.4/ (see the core folder for the libdispatch implementation). The big question to raise now is wether merging back to macosforge.org is possible or not. We had to change a lot of the sources and some of the changes do not really contribute to the maintainability of the code, e.g. MSVC still does not support named intializers in structs. Additionally I split queue.c into several subfiles, such as debug.c and so on, to make the different implementations for the different platforms easier to find. As such it will be quite difficult to provide small patches instead of one or two bigger ones. And there is DrPizza/Peter's port as well, although I do not know how much of his work is tested. What do you think about it? - Marius Zwicker [1] Test results of xdispatch_tests.exe /k (which is a cross-platform implementation of the test-harness and tests in one executable): MUnit Test Framework with independent processes (c) 2011 MLBA Running all Tests ================================== Running test 'dispatch_api'... Passed (1) Running test 'dispatch_simpleFunction'... Passed (2) Running test 'dispatch_sync'... Passed (3) Running test 'dispatch_testDebug'... -> Printing object debugging information: -> Printing done. Passed (4) Running test 'dispatch_apply_function'... Passed (5) Running test 'dispatch_priority'... -> LOW: 0 -> DEFAULT: 64 **************************************** -> HIGH: 128 ******************************************************************************** Passed (6) Running test 'dispatch_priority2'... -> LOW: 64 **************************************** -> DEFAULT: 64 **************************************** -> HIGH: 64 **************************************** Passed (7) Running test 'dispatch_cascade'... -> maxcount = -2 * * * * * * * * * * * * *** * * *** ** * ** * ** * ** * ** ** * * * * *** ** * ** * ** * ** * ** ** * * * * *** ** * ** * ** * ** * ** ** * * * * *** ** * ** * ** * ** * ** ** * * * * *** ** * ** * ** * ** * ** ** * * * * *** ** * ** * ** * ** * ** ** * * * * *** ** * ** * ** * ** * ** ** * * * * *** ** * ** * ** * ** * ** ** * * Passed (8) Running test 'dispatch_apply_serialqueue'... Passed (9) Running test 'dispatch_after'... -> must finish between 5.5s and 6.5s: 0.015614 -> must finish between 1.5s and 2.5s: 0.005205 -> must finish between 0.0s and 0.5s: 0.000000 Passed (10) Running test 'test_queue_finalizer'... Passed (11) Running test 'dispatch_starfish'... -> lap: 10 -> count: 300 -> delta: 27063 ns -> math: 0.149850 ns / lap -> lap: 9 -> count: 300 -> delta: 18223 ns -> math: 0.100903 ns / lap -> lap: 8 -> count: 300 -> delta: 19046 ns -> math: 0.105460 ns / lap -> lap: 7 -> count: 300 -> delta: 18235 ns -> math: 0.100969 ns / lap -> lap: 6 -> count: 300 -> delta: 18317 ns -> math: 0.101423 ns / lap -> lap: 5 -> count: 300 -> delta: 18339 ns -> math: 0.101545 ns / lap -> lap: 4 -> count: 300 -> delta: 18355 ns -> math: 0.101633 ns / lap -> lap: 3 -> count: 300 -> delta: 20610 ns -> math: 0.114120 ns / lap -> lap: 2 -> count: 300 -> delta: 21503 ns -> math: 0.119064 ns / lap -> lap: 1 -> count: 300 -> delta: 18919 ns -> math: 0.104756 ns / lap Passed (12) Running test 'dispatch_memory_use'... -> All workers done. Passed (13) Running test 'dispatch_group_function'... -> sleeping... -> sleeping... -> sleeping... -> done. -> done. -> done. Passed (14) Running test 'cross_blocks'... Passed (15) Running test 'dispatch_plusplus'... Passed (16) Running test 'dispatch_apply_blocks'... Passed (17) Running test 'dispatch_group_blocks'... -> sleeping... -> sleeping... -> sleeping... -> done. -> done. -> done. Passed (18) Running test 'dispatch_after_blocks'... -> must finish between 5.5s and 6.5s: 0.015614 -> must finish between 1.5s and 2.5s: 0.005205 -> must finish between 0s and .5s: 0.000000 Passed (19) Running test 'dispatch_pingpong_blocks'... Passed (20) Running test 'dispatch_semaphore'... Passed (21) Running test 'dispatch_timer_bit31'... Passed (22) Running test 'dispatch_timer_bit63'... -> 0 Passed (23) Running test 'dispatch_drift'... -> 1: jitter 0.000000, drift 0.000000 -> 2: jitter 0.006400, drift 0.006400 -> 3: jitter 0.012800, drift 0.006400 -> 4: jitter 0.003600, drift -0.009200 -> 5: jitter 0.010000, drift 0.006400 -> 6: jitter 0.000799, drift -0.009201 -> 7: jitter 0.007199, drift 0.006400 -> 8: jitter 0.013599, drift 0.006400 -> 9: jitter 0.004399, drift -0.009200 -> 10: jitter 0.010799, drift 0.006400 -> 11: jitter 0.001599, drift -0.009200 -> 12: jitter 0.007998, drift 0.006399 -> 13: jitter 0.014398, drift 0.006400 -> 14: jitter 0.005198, drift -0.009200 -> 15: jitter 0.011598, drift 0.006400 -> 16: jitter 0.002398, drift -0.009200 -> 17: jitter 0.008798, drift 0.006400 -> 18: jitter -0.000403, drift -0.009201 -> 19: jitter 0.005997, drift 0.006400 -> 20: jitter 0.012397, drift 0.006400 -> 21: jitter 0.003197, drift -0.009200 -> 22: jitter 0.009597, drift 0.006400 -> 23: jitter 0.000396, drift -0.009201 -> 24: jitter 0.006796, drift 0.006400 -> 25: jitter 0.013196, drift 0.006400 -> 26: jitter 0.003996, drift -0.009200 -> 27: jitter 0.010396, drift 0.006400 -> 28: jitter 0.001196, drift -0.009200 -> 29: jitter 0.007595, drift 0.006399 -> 30: jitter 0.013995, drift 0.006400 -> 31: jitter 0.004795, drift -0.009200 -> 32: jitter 0.011195, drift 0.006400 -> 33: jitter 0.001995, drift -0.009200 -> 34: jitter 0.008395, drift 0.006400 -> 35: jitter 0.014794, drift 0.006399 -> 36: jitter 0.005594, drift -0.009200 -> 37: jitter 0.011994, drift 0.006400 -> 38: jitter 0.002794, drift -0.009200 -> 39: jitter 0.009194, drift 0.006400 -> 40: jitter -0.000007, drift -0.009201 -> 41: jitter 0.006393, drift 0.006400 -> 42: jitter 0.012793, drift 0.006400 -> 43: jitter 0.003593, drift -0.009200 -> 44: jitter 0.009993, drift 0.006400 -> 45: jitter 0.000793, drift -0.009200 -> 46: jitter 0.007192, drift 0.006399 -> 47: jitter 0.013592, drift 0.006400 -> 48: jitter 0.004392, drift -0.009200 -> 49: jitter 0.010792, drift 0.006400 -> 50: jitter 0.001592, drift -0.009200 -> 51: jitter 0.007992, drift 0.006400 -> 52: jitter 0.014391, drift 0.006399 -> 53: jitter 0.005191, drift -0.009200 -> 54: jitter 0.011591, drift 0.006400 -> 55: jitter 0.002391, drift -0.009200 -> 56: jitter 0.008791, drift 0.006400 -> 57: jitter -0.000410, drift -0.009201 -> 58: jitter 0.005990, drift 0.006400 -> 59: jitter 0.012390, drift 0.006400 -> 60: jitter 0.003190, drift -0.009200 -> 61: jitter 0.009590, drift 0.006400 -> 62: jitter 0.000390, drift -0.009200 -> 63: jitter 0.006789, drift 0.006399 -> 64: jitter 0.013189, drift 0.006400 -> 65: jitter 0.003989, drift -0.009200 -> 66: jitter 0.010389, drift 0.006400 -> 67: jitter 0.001189, drift -0.009200 -> 68: jitter 0.007589, drift 0.006400 -> 69: jitter 0.013988, drift 0.006399 -> 70: jitter 0.004788, drift -0.009200 -> 71: jitter 0.011188, drift 0.006400 -> 72: jitter 0.001988, drift -0.009200 -> 73: jitter 0.008388, drift 0.006400 -> 74: jitter 0.014788, drift 0.006400 -> 75: jitter 0.005587, drift -0.009201 -> 76: jitter 0.011987, drift 0.006400 -> 77: jitter 0.002787, drift -0.009200 -> 78: jitter 0.009187, drift 0.006400 -> 79: jitter -0.000013, drift -0.009200 -> 80: jitter 0.006386, drift 0.006399 -> 81: jitter 0.012786, drift 0.006400 -> 82: jitter 0.003586, drift -0.009200 -> 83: jitter 0.009986, drift 0.006400 -> 84: jitter 0.000786, drift -0.009200 -> 85: jitter 0.007186, drift 0.006400 -> 86: jitter 0.013585, drift 0.006399 -> 87: jitter 0.004385, drift -0.009200 -> 88: jitter 0.010785, drift 0.006400 -> 89: jitter 0.001585, drift -0.009200 -> 90: jitter 0.007985, drift 0.006400 -> 91: jitter 0.014385, drift 0.006400 -> 92: jitter 0.005184, drift -0.009201 -> 93: jitter 0.011584, drift 0.006400 -> 94: jitter 0.002384, drift -0.009200 -> 95: jitter 0.008784, drift 0.006400 -> 96: jitter -0.000416, drift -0.009200 -> 97: jitter 0.005983, drift 0.006399 -> 98: jitter 0.012383, drift 0.006400 -> 99: jitter 0.003183, drift -0.009200 -> 100: jitter 0.009583, drift 0.006400 -> 101: jitter 0.000383, drift -0.009200 -> 102: jitter 0.006783, drift 0.006400 -> 103: jitter 0.013182, drift 0.006399 -> 104: jitter 0.003982, drift -0.009200 -> 105: jitter 0.010382, drift 0.006400 -> 106: jitter 0.001182, drift -0.009200 -> 107: jitter 0.007582, drift 0.006400 -> 108: jitter 0.013982, drift 0.006400 -> 109: jitter 0.004781, drift -0.009201 -> 110: jitter 0.011181, drift 0.006400 -> 111: jitter 0.001981, drift -0.009200 -> 112: jitter 0.008381, drift 0.006400 -> 113: jitter 0.014781, drift 0.006400 -> 114: jitter 0.005580, drift -0.009201 -> 115: jitter 0.011980, drift 0.006400 -> 116: jitter 0.002780, drift -0.009200 -> 117: jitter 0.009180, drift 0.006400 -> 118: jitter -0.000020, drift -0.009200 -> 119: jitter 0.006380, drift 0.006400 -> 120: jitter 0.012779, drift 0.006399 -> 121: jitter 0.003579, drift -0.009200 -> 122: jitter 0.009979, drift 0.006400 -> 123: jitter 0.000779, drift -0.009200 -> 124: jitter 0.007179, drift 0.006400 -> 125: jitter 0.013579, drift 0.006400 -> 126: jitter 0.004378, drift -0.009201 -> 127: jitter 0.010778, drift 0.006400 -> 128: jitter 0.001578, drift -0.009200 -> 129: jitter 0.007978, drift 0.006400 -> 130: jitter 0.014378, drift 0.006400 -> 131: jitter 0.005178, drift -0.009200 -> 132: jitter 0.011577, drift 0.006399 -> 133: jitter 0.002377, drift -0.009200 -> 134: jitter 0.008777, drift 0.006400 -> 135: jitter -0.000423, drift -0.009200 -> 136: jitter 0.005977, drift 0.006400 -> 137: jitter 0.012376, drift 0.006399 -> 138: jitter 0.003176, drift -0.009200 -> 139: jitter 0.009576, drift 0.006400 -> 140: jitter 0.000376, drift -0.009200 -> 141: jitter 0.006776, drift 0.006400 -> 142: jitter 0.013176, drift 0.006400 -> 143: jitter 0.003975, drift -0.009201 -> 144: jitter 0.010375, drift 0.006400 -> 145: jitter 0.001175, drift -0.009200 -> 146: jitter 0.007575, drift 0.006400 -> 147: jitter 0.013975, drift 0.006400 -> 148: jitter 0.004775, drift -0.009200 -> 149: jitter 0.011174, drift 0.006399 -> 150: jitter 0.001974, drift -0.009200 -> 151: jitter 0.008374, drift 0.006400 -> 152: jitter 0.014774, drift 0.006400 -> 153: jitter 0.005574, drift -0.009200 -> 154: jitter 0.011973, drift 0.006399 -> 155: jitter 0.002773, drift -0.009200 -> 156: jitter 0.009173, drift 0.006400 -> 157: jitter -0.000027, drift -0.009200 -> 158: jitter 0.006373, drift 0.006400 -> 159: jitter 0.012773, drift 0.006400 -> 160: jitter 0.003572, drift -0.009201 -> 161: jitter 0.009972, drift 0.006400 -> 162: jitter 0.000772, drift -0.009200 -> 163: jitter 0.007172, drift 0.006400 -> 164: jitter 0.013572, drift 0.006400 -> 165: jitter 0.004372, drift -0.009200 -> 166: jitter 0.010771, drift 0.006399 -> 167: jitter 0.001571, drift -0.009200 -> 168: jitter 0.007971, drift 0.006400 -> 169: jitter 0.014371, drift 0.006400 -> 170: jitter 0.005171, drift -0.009200 -> 171: jitter 0.011570, drift 0.006399 -> 172: jitter 0.002370, drift -0.009200 -> 173: jitter 0.008770, drift 0.006400 -> 174: jitter -0.000430, drift -0.009200 -> 175: jitter 0.005970, drift 0.006400 -> 176: jitter 0.012370, drift 0.006400 -> 177: jitter 0.003169, drift -0.009201 -> 178: jitter 0.009569, drift 0.006400 -> 179: jitter 0.000369, drift -0.009200 -> 180: jitter 0.006769, drift 0.006400 -> 181: jitter 0.013169, drift 0.006400 -> 182: jitter 0.003969, drift -0.009200 -> 183: jitter 0.010368, drift 0.006399 -> 184: jitter 0.001168, drift -0.009200 -> 185: jitter 0.007568, drift 0.006400 -> 186: jitter 0.013968, drift 0.006400 -> 187: jitter 0.004768, drift -0.009200 -> 188: jitter 0.011168, drift 0.006400 -> 189: jitter 0.001967, drift -0.009201 -> 190: jitter 0.008367, drift 0.006400 -> 191: jitter 0.014767, drift 0.006400 -> 192: jitter 0.005567, drift -0.009200 -> 193: jitter 0.011967, drift 0.006400 -> 194: jitter 0.002766, drift -0.009201 -> 195: jitter 0.009166, drift 0.006400 -> 196: jitter -0.000034, drift -0.009200 -> 197: jitter 0.006366, drift 0.006400 -> 198: jitter 0.012766, drift 0.006400 -> 199: jitter 0.003566, drift -0.009200 -> 200: jitter 0.009965, drift 0.006399 -> 201: jitter 0.000765, drift -0.009200 -> 202: jitter 0.007165, drift 0.006400 -> 203: jitter 0.013565, drift 0.006400 -> 204: jitter 0.004365, drift -0.009200 -> 205: jitter 0.010765, drift 0.006400 -> 206: jitter 0.001564, drift -0.009201 -> 207: jitter 0.007964, drift 0.006400 -> 208: jitter 0.014364, drift 0.006400 -> 209: jitter 0.005164, drift -0.009200 -> 210: jitter 0.011564, drift 0.006400 -> 211: jitter 0.002363, drift -0.009201 -> 212: jitter 0.008763, drift 0.006400 -> 213: jitter -0.000437, drift -0.009200 -> 214: jitter 0.005963, drift 0.006400 -> 215: jitter 0.012363, drift 0.006400 -> 216: jitter 0.003163, drift -0.009200 -> 217: jitter 0.009562, drift 0.006399 -> 218: jitter 0.000362, drift -0.009200 -> 219: jitter 0.006762, drift 0.006400 -> 220: jitter 0.013162, drift 0.006400 -> 221: jitter 0.003962, drift -0.009200 -> 222: jitter 0.010362, drift 0.006400 -> 223: jitter 0.001161, drift -0.009201 -> 224: jitter 0.007561, drift 0.006400 -> 225: jitter 0.013961, drift 0.006400 -> 226: jitter 0.004761, drift -0.009200 -> 227: jitter 0.011161, drift 0.006400 -> 228: jitter 0.001960, drift -0.009201 -> 229: jitter 0.008360, drift 0.006400 -> 230: jitter 0.014760, drift 0.006400 -> 231: jitter 0.005560, drift -0.009200 -> 232: jitter 0.011960, drift 0.006400 -> 233: jitter 0.002760, drift -0.009200 -> 234: jitter 0.009159, drift 0.006399 -> 235: jitter -0.000041, drift -0.009200 -> 236: jitter 0.006359, drift 0.006400 -> 237: jitter 0.012759, drift 0.006400 -> 238: jitter 0.003559, drift -0.009200 -> 239: jitter 0.009959, drift 0.006400 -> 240: jitter 0.000758, drift -0.009201 -> 241: jitter 0.007158, drift 0.006400 -> 242: jitter 0.013558, drift 0.006400 -> 243: jitter 0.004358, drift -0.009200 -> 244: jitter 0.010758, drift 0.006400 -> 245: jitter 0.001557, drift -0.009201 -> 246: jitter 0.007957, drift 0.006400 -> 247: jitter 0.014357, drift 0.006400 -> 248: jitter 0.005157, drift -0.009200 -> 249: jitter 0.011557, drift 0.006400 -> 250: jitter 0.002357, drift -0.009200 Failed Test average jitter: fabs(*jittersum) / (double)(*count) < 0.0001 failed (was 0.007169,0.000100), at: ..\..\..\tests\dispatch_drift.cpp:94 (24) Running test 'cxx_dispatch_debug'... -> Begin testing debug output using std::cout xdispatch::queue (com.apple.main-thread) xdispatch::group xdispatch::semaphore -> All output done. (25) Running test 'cxx_dispatch_queue'... Passed (26) Running test 'cxx_dispatch_mainqueue'... Passed (27) Running test 'cxx_dispatch_cascade'... Passed (28) Running test 'cxx_dispatch_group'... -> sleeping... -> sleeping... -> sleeping... -> done. -> done. -> done. Passed (29) Running test 'cxx_dispatch_serialqueue'... Passed (30) Running test 'cxx_dispatch_semaphore'... Passed (31) Running test 'cxx_dispatch_current'... -> Queue should be global default queue xdispatch::queue (com.apple.root.default-priority) -> Queue should be main queue xdispatch::queue (com.apple.main-thread) Passed (32) Running test 'cxx_dispatch_timer'... -> Testing single-shot timer -> Testing periodic timer -> 0 -> 1 -> 2 -> 3 -> 4 -> 5 Passed (33) Running test 'Qt_runnable'... Passed (34) Running test 'Qt_dispatch_debug'... -> Begin testing debug output using QDebug() -> All output done. (35) Running test 'Qt_dispatch_queue'... Passed (36) Running test 'Qt_dispatch_mainqueue'... -> System usage should be zero Passed (37) Running test 'Qt_dispatch_cascade'... Passed (38) Running test 'Qt_dispatch_after'... -> Should finish between 1s and 1.5s: 0.998000 Failed Test watch.elapsed()>=1000 failed (was 0,0), at: ..\..\..\tests\Qt_dispatch_timeout.cpp:48 (39) Running test 'Qt_dispatch_group'... -> signal 'all_finished' emitted Passed (40) Running test 'Qt_dispatch_serialqueue'... Passed (41) Running test 'Qt_dispatch_semaphore'... Passed (42) Running test 'Qt_dispatch_current'... -> Queue should be global default queue -> Queue should be main queue Passed (43) Running test 'Qt_synchronized'... -> Testing 'synchronized' keyword (apply) -> 0.000160 ms per Iteration -> Testing 'synchronized' keyword (for / group) -> 0.000000 ms per Iteration -> Testing 'synchronize' keyword (apply) -> 0.000160 ms per Iteration -> Testing 'synchronize' keyword (for / group) -> 0.000000 ms per Iteration -> Testing with normal mutex (apply) -> 0.002180 ms per Iteration Passed (44) ================================== !!! NOT ALL PASSED !!! ---------------------------------- Passed: 42 Failed: 2

7 19

[PATCH] libdispatch-r197_v1
by Mark Heily 14 Jun '11

14 Jun '11

I have attached a patch against r197 that is a combination of several previous uncommitted patches, plus a few other things. With this patch, I can perform a simple "./configure && make" on Linux, and everything works. Regards, - Mark

2 2

[PATCH] Quiet the warning of unused return value
by Joakim Johansson 28 May '11

28 May '11

I get a warning as the return value is not checked from dispatch_atomic_cmpxchg() - as it is in all other 11 usages of it. Attached is a patch that just discards the return value accordingly to quiet this warning. Joakim Index: src/queue.c =================================================================== --- src/queue.c (revision 197) +++ src/queue.c (working copy) @@ -377,7 +377,7 @@ // The first xchg on the tail will tell the enqueueing thread that it // is safe to blindly write out to the head pointer. A cmpxchg honors // the algorithm. - dispatch_atomic_cmpxchg(&dq->dq_items_head, mediator, NULL); + (void) dispatch_atomic_cmpxchg(&dq->dq_items_head, mediator, NULL); _dispatch_debug("no work on global work queue"); return NULL; }

2 1

[PATCH] bug fix for invalid comparison with unsigned value < 0
by Joakim Johansson 28 May '11

28 May '11

The comparison "_dispatch_hw_config.cc_max_active < 0” is of an unsigned value and thus never true, the attached patch uses and checks failure of sysconf() properly instead. Cheers, Joakim Index: src/queue.c =================================================================== --- src/queue.c (revision 197) +++ src/queue.c (working copy) @@ -474,12 +474,13 @@ _dispatch_hw_config.cc_max_physical = _dispatch_hw_config.cc_max_active; #elif HAVE_SYSCONF && defined(_SC_NPROCESSORS_ONLN) - _dispatch_hw_config.cc_max_active = (int)sysconf(_SC_NPROCESSORS_ONLN); - if (_dispatch_hw_config.cc_max_active < 0) - _dispatch_hw_config.cc_max_active = 1; + int ret; + + ret = (int)sysconf(_SC_NPROCESSORS_ONLN); + _dispatch_hw_config.cc_max_logical = _dispatch_hw_config.cc_max_physical = - _dispatch_hw_config.cc_max_active; + _dispatch_hw_config.cc_max_active = (ret < 0) ? 1 : ret; #else #warning "_dispatch_queue_set_width_init: no supported way to query CPU count" _dispatch_hw_config.cc_max_logical =

1 0

libdispatch object caches
by Joakim Johansson 25 May '11

25 May '11

Hi, I have a question about a possible optimization and would appreciate feedback on whether it would be considered a) worthwhile to investigate b) a candidate for integration if providing good results. The reason for asking is that I’ve seen significantly better (seems to be around 2x) performance on Solaris for e.g. ‘dispatch starfish’ for the lap times, which seems to be malloc-bound (as is also commented in the starfish implementation). libdispatch today uses optimized TSD dispatch continuation object caches with a heavily optimized implementation for Darwin. This cache seems unbounded during the lifetime of a given drain of the queue, after which a forced cleanup is performed (by calling _dispatch_force_cache_cleanup()). The proposed change would be to let _dispatch_continuation_alloc_from_heap() allocate its objects from a magazine based object cache, and let _dispatch_cache_cleanup2() to return objects back to that object cache. This is the approach taken in libumem (see http://www.usenix.org/event/usenix01/full_papers/bonwick/bonwick.pdf for background) - in fact, the easiest approach would possibly be to even depend on libumem being available (https://labs.omniti.com/trac/portableumem exists for non-Solaris platforms) - this could be compile-time optional of course depending on the availability of libumem in such a case, but that would implicitly replace the default memory allocator (if we don’t ‘cherry pick’ out e.g. the magazine layer). It seems that this would allow us to reuse dispatch continuation objects across invocations/threads in an efficient way. So there are a few questions: - Would a magazine-based object cache be interesting for libdispatch? - Have any analysis of typical TSD continuation cache sizes been done already? (anyone who have seen any issues here in GCD-heavy applications?) - What would be a reasonable test for performance validation? - Would a relying on libumem be acceptable for people in general? One other possibility would be to only keep a single item in the TSD and fall back on such an object-cache if that single object would not be available (that is, releasing it back to the object pool in _dispatch_continuation_free, the ‘hot continuation’ trick from _dispatch_continuation_pop() would thus still work) - but this would require more serious investigation (the magazine approach above would be a ’safer’ change). Thoughts? Joakim PS Apple folks also have rdar://4944235 for libumem/OS X, as it would be a nice analogue to the GCD system-level thread management, but for the memory allocation / object cacheing subsystems instead

3 2

[patch] workaround for GNU/Linux pthread_exit() bug
by Mark Heily 17 May '11

17 May '11

See below for a patch to dispatch_main() that is needed on GNU/Linux. This platform cannot call pthread_exit() from the main thread because of an undocumented bug that causes the process to become a zombie. As an alternative, this patch manually calls the TSD destructors, blocks all signals, and parks the main thread using pause(). If there is anything else that needs to be done to make this a viable alternative to calling pthread_exit(), please let me know. Regards, - Mark $ svn di src/queue.c Index: src/queue.c =================================================================== --- src/queue.c (revision 197) +++ src/queue.c (working copy) @@ -941,8 +941,32 @@ if (pthread_main_np()) { #endif _dispatch_program_is_probably_callback_driven = true; +#if defined(__linux__) + /* + * Workaround for a GNU/Linux bug that causes the process to + * become a zombie when the main thread calls pthread_exit(). + */ + sigset_t mask; + void *p; + + p = pthread_getspecific(dispatch_queue_key); + if (p != NULL) + _dispatch_queue_cleanup(p); + p = pthread_getspecific(dispatch_sema4_key); + if (p != NULL) + _dispatch_release(p); + p = pthread_getspecific(dispatch_cache_key); + if (p != NULL) + _dispatch_cache_cleanup2(p); + + sigfillset(&mask); + pthread_sigmask(SIG_SETMASK, &mask, NULL); + pause(); + DISPATCH_CRASH("pause() returned"); +#else /* !defined(__linux__) */ pthread_exit(NULL); DISPATCH_CRASH("pthread_exit() returned"); +#endif #if HAVE_PTHREAD_MAIN_NP } DISPATCH_CLIENT_CRASH("dispatch_main() must be called on the main thread");

2 1

libdispatch for Win32
by DrPizza 30 Apr '11

30 Apr '11

So, I've got the basics working tolerably well. Is this something that people care about/want source for? What I've done so far is as follows: Get most of the code building properly in VC++ 2010: * Replace C99 named initializers with old-fashioned aggregate initializers. * Replace gcc typeof with real type names. * Replace gcc's fancy macros with less fancy standard C89 ones. * VC++ has no equivalent (AFAIK) to gcc's transparent_union, so insert casts as necessary. * Minor bits and pieces like get rid of the ?: gccism. * Provide minimal Win32 equivalents to missing UNIX headers that seem necessary. Blocks: * Only Microsoft is in a position to produce built-in block support for VC++ and I'm sure as hell not going to write a source-source translator. Instead, I have a C++ lambda wrapper that works in conjunction with the _f function variants. This seems more than enough for most purposes. Port pthread_workqueues to Win32: * Built on top of "new-style" (Vista and up) Win32 threadpools. * Reasonably complete. * Reasonably inadequately tested. Rework dispatch_sources: * The Windows overlapped I/O model is better than the traditional UNIX one, but doesn’t readily support: DISPATCH_SOURCE_TYPE_READ DISPATCH_SOURCE_TYPE_WRITE * And Windows in general doesn't have any good analogues to: DISPATCH_SOURCE_TYPE_VNODE DISPATCH_SOURCE_TYPE_SIGNAL DISPATCH_SOURCE_TYPE_PROC DISPATCH_SOURCE_TYPE_MACH_RECV DISPATCH_SOURCE_TYPE_MACH_SEND * But what I do have instead is initial support DISPATCH_SOURCE_TYPE_OIO (for "overlapped I/O") example: https://gist.github.com/938097 * Overlapped I/O supports files, sockets, named pipes, and more. All of these need testing. * The loss of READ/WRITE/SIGNAL/MACH_* is no big deal on Windows, as they don't really fit into Win32 anyway. Only one PROC feature (EXIT) translates into Win32, and I'm not seeing any clearly compelling reason to replicate it, as it doesn't seem especially useful. However, the loss of VNODE is unfortunate, as it both have interesting features. But this may not be fatal. ReadDirectoryChangeNotificationsW supports overlapped I/O, so should plumb into my existing DISPATCH_SOURCE_TYPE_OIO with no changes anyway, though the interface will not be quite as tidy. * I still need to test: DISPATCH_SOURCE_TYPE_DATA_ADD DISPATCH_SOURCE_TYPE_DATA_OR * I need to fix: DISPATCH_SOURCE_TYPE_TIMER I ripped out part of the machinery to make the code clearer temporarily; now I need to add it back. * I also need to examine the ins and outs of cancellation and suspension and so on and so forth. Win32 doesn't allow a handle to be detached from an IOCP except by closing the handle, so there are some sadnesses there. I'm not sure how much impact they'll have in practice, possibly none. * I am toying with the idea of something along the lines of DISPATCH_SOURCE_TYPE_WAITABLE, which would perform a wait on any Win32 waitable object (so mutex, event, waitable timer handles, amongst others) and dispatch a message to a queue when that wait occurs. This would give us back the one PROC scenario that makes sense in Win32, too, as you can wait on a process handle, and the wait resumes when the process terminates. Due to the annoying traits of WaitForMultipleObjects (it's limited to 64 HANDLEs), however, this might be a little awkward to implement without moving to a rather wasteful thread-per-wait model. * There are almost certainly memory leaks, bugs, etc.. My focus has been on validating the general approach more than writing a bunch of test cases. Main queues: * From my understanding of Mac OS X, Windows has no real meaningful equivalent of Cocoa's blessed main queue. Any thread can have a message pump and associated windows, which it's then responsible for drawing, etc.. Processes have an M:N model (M threads, controlling N windows), with each window being affinitized to its own thread. * However, the ability to post a message back into a window's message loop is obviously invaluable, so I want to create as close a workalike as makes sense in Win32. Something that captures the spirit, if not the exact same API. I've not yet written any code for this, but my plan of action is to do something along these lines: 1) allow creation of serial queues bound to an HWND or HWNDs. Callbacks posted to these queues will be pumped into the WndProc one-by-one. 2) Either a WndProc hook or a helper function (or both) to respond to the callbacks posted to the WndProc and execute them. 3) Possibly some convenience helpers to allow the retrieval of a queue given an HWND and so on. Distant future: * There would be certain benefits to ripping out the pthread_workqueues and using the new-style Win32 threadpools directly. The Win32 threadpools directly support timers, so they might allow DISPATCH_SOURCE_TYPE_TIMER to be moved off the dispatch_mgr queue/thread. Likewise, they directly support overlapped I/O, so might allow DISPATCH_SOURCE_TYPE_OIO to be moved off the dispatch_mgr thread too. They also directly support waits on handles, which would greatly simplify DISPATCH_SOURCE_TYPE_WAITABLE (if I do indeed go down that route). So the advantages would be many--but I am wary of diverging too far from the existing source, which is why thus far I've implemented pthread_workqueues instead; it was the easy solution. * Going in the opposite direction, some might prefer switching to Windows 2000-style thread pools, so as to support Windows XP instead. This would work (and I think someone on the list mentioned that they had implemented pthread_workqueue on that API already), but it also means eliminating the possibility of the streamlined implementations described above. Source code: My plan was to dump it into my github, if people find the whole thing interesting, though I was going to wait until I'd fixed timer sources, since they're rather important. Peter

2 4

trivial linux patch + general questions
by "C. Bergström" 22 Apr '11

22 Apr '11

Can someone apply this if it's a proper fix ------------- diff --git a/autogen.sh b/autogen.sh old mode 100644 new mode 100755 diff --git a/testing/Makefile.am b/testing/Makefile.am index 404ced0..103d8bf 100644 --- a/testing/Makefile.am +++ b/testing/Makefile.am @@ -81,7 +81,7 @@ TOOLS= \ noinst_PROGRAMS+=$(TOOLS) INCLUDES=-I$(top_builddir) -I$(top_srcdir) -LDADD=libtest.la ../src/libdispatch.la +LDADD=libtest.la ../src/libdispatch.la ../src/libshims.la CFLAGS=-Wall $(MARCH_FLAGS) $(CBLOCKS_FLAGS) CXXFLAGS=-Wall $(MARCH_FLAGS) $(CXXBLOCKS_FLAGS) --------- I also get a test failure with make check --------- LOW: 68 ******************************************* DEFAULT: 63 **************************************** HIGH: 61 *************************************** Actual: 192 Expected: 192 [PASS] blocks completed Actual: 68 Expected: <61 [FAIL] high priority precedence (../../testing/dispatch_priority.c:83) ../../testing/dispatch_priority.c:83 PASS: dispatch_priority ================================================== [TEST] Dispatch Priority (Set Target Queue) [PID] 541 ================================================== --------- Is this an out of sync test case or a legitimate failure? Last two questions.. Does anyone have ideas on Solaris or Linux non-portable tsd or semaphore optimizations? Thanks! ./C

4 6

Regarding dispatch_get_main_queue() and the thread it is actual executing on
by Marius Zwicker 19 Apr '11

19 Apr '11

Hi everyone, During my work for libDispatch I had to find a way for implementing TSD de-allocators (which are not available on windows). While doing so I did some extensive debugging on OS X / Linux for understanding what is actually happening when calling dispatch_main(). Weird enough it seemed to me as if the main thread would never execute an item from the main queue but simply go to sleep instead. To verify this behaviour, I wrote a test to compare the pthread_t handles during execution: static pthread_t main_thread; void pass(void* dt){ pthread_t now = pthread_self(); assert(pthread_equal(now, main_thread) == 1); exit(0); } void dispatch_api() { dispatch_queue_t q = NULL; main_thread = pthread_self(); q = dispatch_get_main_queue(); assert(q != NULL); dispatch_async_f(q, NULL, pass); ... dispatch_main(); } This test is approving my suspicions, as the pthread_equal(...) assertion is constantly failing. So the big question is: What am I missing? I thought the main queue is available as a mean to be sure some blocks execute on the main thread (e.g. when updating the GUI is only possible while being on the main thread). I welcome any hints or help. - Marius Zwicker

2 4

libdispatch with Visual Studio
by Brent Fulgham 16 Apr '11

16 Apr '11

I spent a few minutes yesterday playing with the libdispatch sources to see how far I could get building under Visual Studio. As expected, it choked when presented with various useful gcc extensions, such as named structure initializers, differences in macro handling, and so forth. All of these differences are fairly easily resolved, with the exception of the gcc-specific 'typeof' operator. I'm not sure how to provide similar behavior under the Visual C++ compiler, as I don't think it has a mechanism to determine the static type of an expression at runtime. Can anyone comment on how this is resolved for the Apple build of libdispatch (i.e., the version that ships with Safari/WebKit)? Thanks, -Brent

4 3