June 2011 - libdispatch-dev - lists.macosforge.org

[PATCH] libdispatch-r197_v2
by Mark Heily 29 Jun '11

29 Jun '11

See attached for a small update to the portability patch I sent out awhile back. This fixes a crash on Linux caused by a glibc bug in the pause() function. See below for the diff between v1 and v2 of the patch. Regards, - Mark @@ -115,8 +119,9 @@ sigfillset(&mask); pthread_sigmask(SIG_SETMASK, &mask, NULL); - pause(); - DISPATCH_CRASH("pause() returned"); + for (;;) { + pause(); + } #else /* !defined(__linux__) */ pthread_exit(NULL); DISPATCH_CRASH("pthread_exit() returned");

1 0

Clarification on default target_queue for custom serial queues
by Daniel Shusta 27 Jun '11

27 Jun '11

When reading about libdispatch, I get the sense that I can create serial queues en masse and it’ll just work out, because the queues will be limited to a system-determined number of threads. The reason behind this: > In fact, serial queues are scheduled using the global queues. Each serial queue has a target queue, which is initially set to the default priority concurrent queue. When a block is first added to an empty serial queue, the queue itself is added to the target queue. -- GCD technical brief, formerly linked from http://developer.apple.com/technologies/mac/snowleopard/gcd.html but is now a dead link. There’s a backup at http://www.ctestlabs.org/hughes_multicore/documents/GrandCentral_TB_brief_2… This information is repeated at the WWDC session talks (#210, about 10 and a half minutes into the video available to developers). Essentially, global queues are limited to a handful of threads, and since serial queues run on the global queues, they're also all funneled down to that handful of threads. But libdispatch doesn’t actually seem to do this. Example code (tested on iOS 4.3, also shows this behavior on Snow Leopard 10.6.7, 10.6.0, and Lion dev previews): NSUInteger queueCount = 50; dispatch_queue_t queueArray[queueCount]; for ( NSUInteger i = 0; i < queueCount; ++i ) queueArray[i] = dispatch_queue_create("com.some.queue", DISPATCH_QUEUE_SERIAL); // Then enqueue expensive blocks on each of these with a dispatch group, wait for the group to finish, and clean up. The process will just go ahead and create 50 threads. On a dual-core machine, this isn’t really what I was hoping for. But two lines of adjustments… NSUInteger queueCount = 50; dispatch_queue_t queueArray[queueCount]; dispatch_queue_t defaultGlobalQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0); for ( NSUInteger i = 0; i < queueCount; ++i ) { queueArray[i] = dispatch_queue_create("com.some.queue", DISPATCH_QUEUE_SERIAL); dispatch_set_target_queue(queueArray[i], defaultGlobalQueue); } // Then enqueue expensive blocks on each of these with a dispatch group, wait for the group to finish, and clean up. …will limit it to a couple of threads. According to the documentation quoted above, these two code samples should produce identical results -- serial queues should already target the default global queue. Yet the actual behaviors are very different. Is this intentional? Am I misunderstanding something? Thanks for your time and whatever response you can give, --Daniel Shusta === PS. this is specific to custom serial queues. Custom concurrent queues in iOS 4.3+ funnel down to a couple of threads. PPS. In libdispatch's queue.c, it does seem like the target queue is supposed to be the default global queue. In _dispatch_queue_init() we see dq->do_targetq = _dispatch_get_root_queue(0, true);

5 7

When do the NDAs expire?
by DrPizza 24 Jun '11

24 Jun '11

I'm not covered by the NDAs (not a registered developer, never agreed to them), but am interested in details of the new I/O mechanisms. The barrier API is also very welcome. When will people be able to talk about these, and when (if ever) will source code be available? Peter

2 4

Updates regarding the status of libdispatch on Windows
by Marius Zwicker 20 Jun '11

20 Jun '11

I am very glad to report good news on the topic libdispatch on windows. Yesterday we managed to eradicate the last obvious bugs and all tests and can now completely build and run libdispatch by using MSVC (Visual Studio 2010) on windows. All tests seem to pass or fail the same way they currently do on Linux, so at least we have reached feature parity ;) - Please see the test results attached [1], the responsible code can be found at http://opensource.mlba-team.de/svn/xdispatch/tags/0.4/ (see the core folder for the libdispatch implementation). The big question to raise now is wether merging back to macosforge.org is possible or not. We had to change a lot of the sources and some of the changes do not really contribute to the maintainability of the code, e.g. MSVC still does not support named intializers in structs. Additionally I split queue.c into several subfiles, such as debug.c and so on, to make the different implementations for the different platforms easier to find. As such it will be quite difficult to provide small patches instead of one or two bigger ones. And there is DrPizza/Peter's port as well, although I do not know how much of his work is tested. What do you think about it? - Marius Zwicker [1] Test results of xdispatch_tests.exe /k (which is a cross-platform implementation of the test-harness and tests in one executable): MUnit Test Framework with independent processes (c) 2011 MLBA Running all Tests ================================== Running test 'dispatch_api'... Passed (1) Running test 'dispatch_simpleFunction'... Passed (2) Running test 'dispatch_sync'... Passed (3) Running test 'dispatch_testDebug'... -> Printing object debugging information: -> Printing done. Passed (4) Running test 'dispatch_apply_function'... Passed (5) Running test 'dispatch_priority'... -> LOW: 0 -> DEFAULT: 64 **************************************** -> HIGH: 128 ******************************************************************************** Passed (6) Running test 'dispatch_priority2'... -> LOW: 64 **************************************** -> DEFAULT: 64 **************************************** -> HIGH: 64 **************************************** Passed (7) Running test 'dispatch_cascade'... -> maxcount = -2 * * * * * * * * * * * * *** * * *** ** * ** * ** * ** * ** ** * * * * *** ** * ** * ** * ** * ** ** * * * * *** ** * ** * ** * ** * ** ** * * * * *** ** * ** * ** * ** * ** ** * * * * *** ** * ** * ** * ** * ** ** * * * * *** ** * ** * ** * ** * ** ** * * * * *** ** * ** * ** * ** * ** ** * * * * *** ** * ** * ** * ** * ** ** * * Passed (8) Running test 'dispatch_apply_serialqueue'... Passed (9) Running test 'dispatch_after'... -> must finish between 5.5s and 6.5s: 0.015614 -> must finish between 1.5s and 2.5s: 0.005205 -> must finish between 0.0s and 0.5s: 0.000000 Passed (10) Running test 'test_queue_finalizer'... Passed (11) Running test 'dispatch_starfish'... -> lap: 10 -> count: 300 -> delta: 27063 ns -> math: 0.149850 ns / lap -> lap: 9 -> count: 300 -> delta: 18223 ns -> math: 0.100903 ns / lap -> lap: 8 -> count: 300 -> delta: 19046 ns -> math: 0.105460 ns / lap -> lap: 7 -> count: 300 -> delta: 18235 ns -> math: 0.100969 ns / lap -> lap: 6 -> count: 300 -> delta: 18317 ns -> math: 0.101423 ns / lap -> lap: 5 -> count: 300 -> delta: 18339 ns -> math: 0.101545 ns / lap -> lap: 4 -> count: 300 -> delta: 18355 ns -> math: 0.101633 ns / lap -> lap: 3 -> count: 300 -> delta: 20610 ns -> math: 0.114120 ns / lap -> lap: 2 -> count: 300 -> delta: 21503 ns -> math: 0.119064 ns / lap -> lap: 1 -> count: 300 -> delta: 18919 ns -> math: 0.104756 ns / lap Passed (12) Running test 'dispatch_memory_use'... -> All workers done. Passed (13) Running test 'dispatch_group_function'... -> sleeping... -> sleeping... -> sleeping... -> done. -> done. -> done. Passed (14) Running test 'cross_blocks'... Passed (15) Running test 'dispatch_plusplus'... Passed (16) Running test 'dispatch_apply_blocks'... Passed (17) Running test 'dispatch_group_blocks'... -> sleeping... -> sleeping... -> sleeping... -> done. -> done. -> done. Passed (18) Running test 'dispatch_after_blocks'... -> must finish between 5.5s and 6.5s: 0.015614 -> must finish between 1.5s and 2.5s: 0.005205 -> must finish between 0s and .5s: 0.000000 Passed (19) Running test 'dispatch_pingpong_blocks'... Passed (20) Running test 'dispatch_semaphore'... Passed (21) Running test 'dispatch_timer_bit31'... Passed (22) Running test 'dispatch_timer_bit63'... -> 0 Passed (23) Running test 'dispatch_drift'... -> 1: jitter 0.000000, drift 0.000000 -> 2: jitter 0.006400, drift 0.006400 -> 3: jitter 0.012800, drift 0.006400 -> 4: jitter 0.003600, drift -0.009200 -> 5: jitter 0.010000, drift 0.006400 -> 6: jitter 0.000799, drift -0.009201 -> 7: jitter 0.007199, drift 0.006400 -> 8: jitter 0.013599, drift 0.006400 -> 9: jitter 0.004399, drift -0.009200 -> 10: jitter 0.010799, drift 0.006400 -> 11: jitter 0.001599, drift -0.009200 -> 12: jitter 0.007998, drift 0.006399 -> 13: jitter 0.014398, drift 0.006400 -> 14: jitter 0.005198, drift -0.009200 -> 15: jitter 0.011598, drift 0.006400 -> 16: jitter 0.002398, drift -0.009200 -> 17: jitter 0.008798, drift 0.006400 -> 18: jitter -0.000403, drift -0.009201 -> 19: jitter 0.005997, drift 0.006400 -> 20: jitter 0.012397, drift 0.006400 -> 21: jitter 0.003197, drift -0.009200 -> 22: jitter 0.009597, drift 0.006400 -> 23: jitter 0.000396, drift -0.009201 -> 24: jitter 0.006796, drift 0.006400 -> 25: jitter 0.013196, drift 0.006400 -> 26: jitter 0.003996, drift -0.009200 -> 27: jitter 0.010396, drift 0.006400 -> 28: jitter 0.001196, drift -0.009200 -> 29: jitter 0.007595, drift 0.006399 -> 30: jitter 0.013995, drift 0.006400 -> 31: jitter 0.004795, drift -0.009200 -> 32: jitter 0.011195, drift 0.006400 -> 33: jitter 0.001995, drift -0.009200 -> 34: jitter 0.008395, drift 0.006400 -> 35: jitter 0.014794, drift 0.006399 -> 36: jitter 0.005594, drift -0.009200 -> 37: jitter 0.011994, drift 0.006400 -> 38: jitter 0.002794, drift -0.009200 -> 39: jitter 0.009194, drift 0.006400 -> 40: jitter -0.000007, drift -0.009201 -> 41: jitter 0.006393, drift 0.006400 -> 42: jitter 0.012793, drift 0.006400 -> 43: jitter 0.003593, drift -0.009200 -> 44: jitter 0.009993, drift 0.006400 -> 45: jitter 0.000793, drift -0.009200 -> 46: jitter 0.007192, drift 0.006399 -> 47: jitter 0.013592, drift 0.006400 -> 48: jitter 0.004392, drift -0.009200 -> 49: jitter 0.010792, drift 0.006400 -> 50: jitter 0.001592, drift -0.009200 -> 51: jitter 0.007992, drift 0.006400 -> 52: jitter 0.014391, drift 0.006399 -> 53: jitter 0.005191, drift -0.009200 -> 54: jitter 0.011591, drift 0.006400 -> 55: jitter 0.002391, drift -0.009200 -> 56: jitter 0.008791, drift 0.006400 -> 57: jitter -0.000410, drift -0.009201 -> 58: jitter 0.005990, drift 0.006400 -> 59: jitter 0.012390, drift 0.006400 -> 60: jitter 0.003190, drift -0.009200 -> 61: jitter 0.009590, drift 0.006400 -> 62: jitter 0.000390, drift -0.009200 -> 63: jitter 0.006789, drift 0.006399 -> 64: jitter 0.013189, drift 0.006400 -> 65: jitter 0.003989, drift -0.009200 -> 66: jitter 0.010389, drift 0.006400 -> 67: jitter 0.001189, drift -0.009200 -> 68: jitter 0.007589, drift 0.006400 -> 69: jitter 0.013988, drift 0.006399 -> 70: jitter 0.004788, drift -0.009200 -> 71: jitter 0.011188, drift 0.006400 -> 72: jitter 0.001988, drift -0.009200 -> 73: jitter 0.008388, drift 0.006400 -> 74: jitter 0.014788, drift 0.006400 -> 75: jitter 0.005587, drift -0.009201 -> 76: jitter 0.011987, drift 0.006400 -> 77: jitter 0.002787, drift -0.009200 -> 78: jitter 0.009187, drift 0.006400 -> 79: jitter -0.000013, drift -0.009200 -> 80: jitter 0.006386, drift 0.006399 -> 81: jitter 0.012786, drift 0.006400 -> 82: jitter 0.003586, drift -0.009200 -> 83: jitter 0.009986, drift 0.006400 -> 84: jitter 0.000786, drift -0.009200 -> 85: jitter 0.007186, drift 0.006400 -> 86: jitter 0.013585, drift 0.006399 -> 87: jitter 0.004385, drift -0.009200 -> 88: jitter 0.010785, drift 0.006400 -> 89: jitter 0.001585, drift -0.009200 -> 90: jitter 0.007985, drift 0.006400 -> 91: jitter 0.014385, drift 0.006400 -> 92: jitter 0.005184, drift -0.009201 -> 93: jitter 0.011584, drift 0.006400 -> 94: jitter 0.002384, drift -0.009200 -> 95: jitter 0.008784, drift 0.006400 -> 96: jitter -0.000416, drift -0.009200 -> 97: jitter 0.005983, drift 0.006399 -> 98: jitter 0.012383, drift 0.006400 -> 99: jitter 0.003183, drift -0.009200 -> 100: jitter 0.009583, drift 0.006400 -> 101: jitter 0.000383, drift -0.009200 -> 102: jitter 0.006783, drift 0.006400 -> 103: jitter 0.013182, drift 0.006399 -> 104: jitter 0.003982, drift -0.009200 -> 105: jitter 0.010382, drift 0.006400 -> 106: jitter 0.001182, drift -0.009200 -> 107: jitter 0.007582, drift 0.006400 -> 108: jitter 0.013982, drift 0.006400 -> 109: jitter 0.004781, drift -0.009201 -> 110: jitter 0.011181, drift 0.006400 -> 111: jitter 0.001981, drift -0.009200 -> 112: jitter 0.008381, drift 0.006400 -> 113: jitter 0.014781, drift 0.006400 -> 114: jitter 0.005580, drift -0.009201 -> 115: jitter 0.011980, drift 0.006400 -> 116: jitter 0.002780, drift -0.009200 -> 117: jitter 0.009180, drift 0.006400 -> 118: jitter -0.000020, drift -0.009200 -> 119: jitter 0.006380, drift 0.006400 -> 120: jitter 0.012779, drift 0.006399 -> 121: jitter 0.003579, drift -0.009200 -> 122: jitter 0.009979, drift 0.006400 -> 123: jitter 0.000779, drift -0.009200 -> 124: jitter 0.007179, drift 0.006400 -> 125: jitter 0.013579, drift 0.006400 -> 126: jitter 0.004378, drift -0.009201 -> 127: jitter 0.010778, drift 0.006400 -> 128: jitter 0.001578, drift -0.009200 -> 129: jitter 0.007978, drift 0.006400 -> 130: jitter 0.014378, drift 0.006400 -> 131: jitter 0.005178, drift -0.009200 -> 132: jitter 0.011577, drift 0.006399 -> 133: jitter 0.002377, drift -0.009200 -> 134: jitter 0.008777, drift 0.006400 -> 135: jitter -0.000423, drift -0.009200 -> 136: jitter 0.005977, drift 0.006400 -> 137: jitter 0.012376, drift 0.006399 -> 138: jitter 0.003176, drift -0.009200 -> 139: jitter 0.009576, drift 0.006400 -> 140: jitter 0.000376, drift -0.009200 -> 141: jitter 0.006776, drift 0.006400 -> 142: jitter 0.013176, drift 0.006400 -> 143: jitter 0.003975, drift -0.009201 -> 144: jitter 0.010375, drift 0.006400 -> 145: jitter 0.001175, drift -0.009200 -> 146: jitter 0.007575, drift 0.006400 -> 147: jitter 0.013975, drift 0.006400 -> 148: jitter 0.004775, drift -0.009200 -> 149: jitter 0.011174, drift 0.006399 -> 150: jitter 0.001974, drift -0.009200 -> 151: jitter 0.008374, drift 0.006400 -> 152: jitter 0.014774, drift 0.006400 -> 153: jitter 0.005574, drift -0.009200 -> 154: jitter 0.011973, drift 0.006399 -> 155: jitter 0.002773, drift -0.009200 -> 156: jitter 0.009173, drift 0.006400 -> 157: jitter -0.000027, drift -0.009200 -> 158: jitter 0.006373, drift 0.006400 -> 159: jitter 0.012773, drift 0.006400 -> 160: jitter 0.003572, drift -0.009201 -> 161: jitter 0.009972, drift 0.006400 -> 162: jitter 0.000772, drift -0.009200 -> 163: jitter 0.007172, drift 0.006400 -> 164: jitter 0.013572, drift 0.006400 -> 165: jitter 0.004372, drift -0.009200 -> 166: jitter 0.010771, drift 0.006399 -> 167: jitter 0.001571, drift -0.009200 -> 168: jitter 0.007971, drift 0.006400 -> 169: jitter 0.014371, drift 0.006400 -> 170: jitter 0.005171, drift -0.009200 -> 171: jitter 0.011570, drift 0.006399 -> 172: jitter 0.002370, drift -0.009200 -> 173: jitter 0.008770, drift 0.006400 -> 174: jitter -0.000430, drift -0.009200 -> 175: jitter 0.005970, drift 0.006400 -> 176: jitter 0.012370, drift 0.006400 -> 177: jitter 0.003169, drift -0.009201 -> 178: jitter 0.009569, drift 0.006400 -> 179: jitter 0.000369, drift -0.009200 -> 180: jitter 0.006769, drift 0.006400 -> 181: jitter 0.013169, drift 0.006400 -> 182: jitter 0.003969, drift -0.009200 -> 183: jitter 0.010368, drift 0.006399 -> 184: jitter 0.001168, drift -0.009200 -> 185: jitter 0.007568, drift 0.006400 -> 186: jitter 0.013968, drift 0.006400 -> 187: jitter 0.004768, drift -0.009200 -> 188: jitter 0.011168, drift 0.006400 -> 189: jitter 0.001967, drift -0.009201 -> 190: jitter 0.008367, drift 0.006400 -> 191: jitter 0.014767, drift 0.006400 -> 192: jitter 0.005567, drift -0.009200 -> 193: jitter 0.011967, drift 0.006400 -> 194: jitter 0.002766, drift -0.009201 -> 195: jitter 0.009166, drift 0.006400 -> 196: jitter -0.000034, drift -0.009200 -> 197: jitter 0.006366, drift 0.006400 -> 198: jitter 0.012766, drift 0.006400 -> 199: jitter 0.003566, drift -0.009200 -> 200: jitter 0.009965, drift 0.006399 -> 201: jitter 0.000765, drift -0.009200 -> 202: jitter 0.007165, drift 0.006400 -> 203: jitter 0.013565, drift 0.006400 -> 204: jitter 0.004365, drift -0.009200 -> 205: jitter 0.010765, drift 0.006400 -> 206: jitter 0.001564, drift -0.009201 -> 207: jitter 0.007964, drift 0.006400 -> 208: jitter 0.014364, drift 0.006400 -> 209: jitter 0.005164, drift -0.009200 -> 210: jitter 0.011564, drift 0.006400 -> 211: jitter 0.002363, drift -0.009201 -> 212: jitter 0.008763, drift 0.006400 -> 213: jitter -0.000437, drift -0.009200 -> 214: jitter 0.005963, drift 0.006400 -> 215: jitter 0.012363, drift 0.006400 -> 216: jitter 0.003163, drift -0.009200 -> 217: jitter 0.009562, drift 0.006399 -> 218: jitter 0.000362, drift -0.009200 -> 219: jitter 0.006762, drift 0.006400 -> 220: jitter 0.013162, drift 0.006400 -> 221: jitter 0.003962, drift -0.009200 -> 222: jitter 0.010362, drift 0.006400 -> 223: jitter 0.001161, drift -0.009201 -> 224: jitter 0.007561, drift 0.006400 -> 225: jitter 0.013961, drift 0.006400 -> 226: jitter 0.004761, drift -0.009200 -> 227: jitter 0.011161, drift 0.006400 -> 228: jitter 0.001960, drift -0.009201 -> 229: jitter 0.008360, drift 0.006400 -> 230: jitter 0.014760, drift 0.006400 -> 231: jitter 0.005560, drift -0.009200 -> 232: jitter 0.011960, drift 0.006400 -> 233: jitter 0.002760, drift -0.009200 -> 234: jitter 0.009159, drift 0.006399 -> 235: jitter -0.000041, drift -0.009200 -> 236: jitter 0.006359, drift 0.006400 -> 237: jitter 0.012759, drift 0.006400 -> 238: jitter 0.003559, drift -0.009200 -> 239: jitter 0.009959, drift 0.006400 -> 240: jitter 0.000758, drift -0.009201 -> 241: jitter 0.007158, drift 0.006400 -> 242: jitter 0.013558, drift 0.006400 -> 243: jitter 0.004358, drift -0.009200 -> 244: jitter 0.010758, drift 0.006400 -> 245: jitter 0.001557, drift -0.009201 -> 246: jitter 0.007957, drift 0.006400 -> 247: jitter 0.014357, drift 0.006400 -> 248: jitter 0.005157, drift -0.009200 -> 249: jitter 0.011557, drift 0.006400 -> 250: jitter 0.002357, drift -0.009200 Failed Test average jitter: fabs(*jittersum) / (double)(*count) < 0.0001 failed (was 0.007169,0.000100), at: ..\..\..\tests\dispatch_drift.cpp:94 (24) Running test 'cxx_dispatch_debug'... -> Begin testing debug output using std::cout xdispatch::queue (com.apple.main-thread) xdispatch::group xdispatch::semaphore -> All output done. (25) Running test 'cxx_dispatch_queue'... Passed (26) Running test 'cxx_dispatch_mainqueue'... Passed (27) Running test 'cxx_dispatch_cascade'... Passed (28) Running test 'cxx_dispatch_group'... -> sleeping... -> sleeping... -> sleeping... -> done. -> done. -> done. Passed (29) Running test 'cxx_dispatch_serialqueue'... Passed (30) Running test 'cxx_dispatch_semaphore'... Passed (31) Running test 'cxx_dispatch_current'... -> Queue should be global default queue xdispatch::queue (com.apple.root.default-priority) -> Queue should be main queue xdispatch::queue (com.apple.main-thread) Passed (32) Running test 'cxx_dispatch_timer'... -> Testing single-shot timer -> Testing periodic timer -> 0 -> 1 -> 2 -> 3 -> 4 -> 5 Passed (33) Running test 'Qt_runnable'... Passed (34) Running test 'Qt_dispatch_debug'... -> Begin testing debug output using QDebug() -> All output done. (35) Running test 'Qt_dispatch_queue'... Passed (36) Running test 'Qt_dispatch_mainqueue'... -> System usage should be zero Passed (37) Running test 'Qt_dispatch_cascade'... Passed (38) Running test 'Qt_dispatch_after'... -> Should finish between 1s and 1.5s: 0.998000 Failed Test watch.elapsed()>=1000 failed (was 0,0), at: ..\..\..\tests\Qt_dispatch_timeout.cpp:48 (39) Running test 'Qt_dispatch_group'... -> signal 'all_finished' emitted Passed (40) Running test 'Qt_dispatch_serialqueue'... Passed (41) Running test 'Qt_dispatch_semaphore'... Passed (42) Running test 'Qt_dispatch_current'... -> Queue should be global default queue -> Queue should be main queue Passed (43) Running test 'Qt_synchronized'... -> Testing 'synchronized' keyword (apply) -> 0.000160 ms per Iteration -> Testing 'synchronized' keyword (for / group) -> 0.000000 ms per Iteration -> Testing 'synchronize' keyword (apply) -> 0.000160 ms per Iteration -> Testing 'synchronize' keyword (for / group) -> 0.000000 ms per Iteration -> Testing with normal mutex (apply) -> 0.002180 ms per Iteration Passed (44) ================================== !!! NOT ALL PASSED !!! ---------------------------------- Passed: 42 Failed: 2

7 19

[PATCH] libdispatch-r197_v1
by Mark Heily 14 Jun '11

14 Jun '11

I have attached a patch against r197 that is a combination of several previous uncommitted patches, plus a few other things. With this patch, I can perform a simple "./configure && make" on Linux, and everything works. Regards, - Mark

2 2