Results from running libdispatch tests on Solaris using libkqueue and libdispatch with patches
Hi, If anyone is interested, the output of the working unit tests for libdispatch on Solaris 10 is attached below. This is done with libdispatch r188 + the patches posted to the list the last few days built together with libkqueue r271 together with a few patches posted on the libkqueue list. dispatch_priority seems to fail spuriously sometimes (even though it passed in the example below) and the starfish latency boundary test sometimes fails - this is on a Xeon 5500 box with 8 cores (of which 4 is HT). When running the test suite, a very large number of threads is created for some tests: 386 during dispatch_priority 35 during dispatch_priority2 257 during starfish It seems a bit excessive, what would be expected on such a configuration not having support for pthread_workqueue? I will try it using Mark Heily’s userspace pthread_workqueue implementation as well and see how that works. Joakim —— node5:~/gcd/trunk/testing> ./dispatch_api ================================================== [TEST] Dispatch (Public) API [PID] 16120 ================================================== Actual: 412850 Expected: 412850 [PASS] dispatch_get_main_queue node5:~/gcd/trunk/testing> ./dispatch_c99 ================================================== [TEST] Dispatch C99 [PID] 16135 ================================================== Actual: 412830 Expected: 412830 [PASS] dispatch_get_main_queue node5:~/gcd/trunk/testing> ./dispatch_cascade ================================================== [TEST] Dispatch Cascade [PID] 16150 ================================================== maxcount = 2464 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ** * * * * * * * * * ** * * * * * * * * * ** * * *** * * * * * node5:~/gcd/trunk/testing> ./dispatch_debug ================================================== [TEST] Dispatch Debug [PID] 16165 ================================================== node5:~/gcd/trunk/testing> ./dispatch_priority ================================================== [TEST] Dispatch Priority [PID] 16180 ================================================== LOW: 64 **************************************** DEFAULT: 60 ************************************** HIGH: 68 ******************************************* Actual: 192 Expected: 192 [PASS] blocks completed Actual: 64 Expected: <68 [PASS] high priority precedence node5:~/gcd/trunk/testing> ./dispatch_priority2 ================================================== [TEST] Dispatch Priority (Set Target Queue) [PID] 16195 ================================================== Actual: 413150 Expected: 413150 [PASS] q[i] Actual: 415a10 Expected: 415a10 [PASS] q[i] Actual: 415ac0 Expected: 415ac0 [PASS] q[i] LOW: 58 ************************************* DEFAULT: 67 ****************************************** HIGH: 67 ****************************************** Actual: 192 Expected: 192 [PASS] blocks completed Actual: 58 Expected: <67 [PASS] high priority precedence node5:~/gcd/trunk/testing> ./dispatch_starfish ================================================== [TEST] Dispatch Starfish [PID] 16210 ================================================== lap: 10 count: 1000 delta: 832448041 ns math: 415.808212 ns / lap Actual: 415 Expected: <1000 [PASS] Latency lap: 9 count: 1000 delta: 789866055 ns math: 394.538489 ns / lap Actual: 394 Expected: <1000 [PASS] Latency lap: 8 count: 1000 delta: 976664387 ns math: 487.844349 ns / lap Actual: 487 Expected: <1000 [PASS] Latency lap: 7 count: 1000 delta: 1625163520 ns math: 811.769990 ns / lap Actual: 811 Expected: <1000 [PASS] Latency lap: 6 count: 1000 delta: 1527922124 ns math: 763.197864 ns / lap Actual: 763 Expected: <1000 [PASS] Latency lap: 5 count: 1000 delta: 1293263436 ns math: 645.985732 ns / lap Actual: 645 Expected: <1000 [PASS] Latency lap: 4 count: 1000 delta: 1755776727 ns math: 877.011352 ns / lap Actual: 877 Expected: <1000 [PASS] Latency lap: 3 count: 1000 delta: 2045621497 ns math: 1021.788960 ns / lap Actual: 1021 Expected: <1000 [FAIL] Latency (dispatch_starfish.c:75) dispatch_starfish.c:75 lap: 2 count: 1000 delta: 997176198 ns math: 498.090009 ns / lap Actual: 498 Expected: <1000 [PASS] Latency lap: 1 count: 1000 delta: 1204968309 ns math: 601.882272 ns / lap Actual: 601 Expected: <1000 [PASS] Latency node5:~/gcd/trunk/testing> ./queue_finalizer ================================================== [TEST] Dispatch Queue Finalizer [PID] 16225 ================================================== Actual: 413390 Expected: 413390 [PASS] dispatch_queue_new Actual: 72f36169c7132463 Expected: 72f36169c7132463 [PASS] finalizer ran node5:~/gcd/trunk/testing>
On 27 jul 2010, at 12.33, Joakim Johansson wrote:
I will try it using Mark Heily’s userspace pthread_workqueue implementation as well and see how that works.
And the answer is… Quite a bit better! I just tried it with libdispatch r188 + Solaris patches + libkqueue r280, results included below (there is an example of the spurious failures mentioned in dispatch_priority2). The number of threads are now around 17, which seems much more reasonable on my machine, and the machine is no longer pushed up to a system load of 75 when running the tests… :-) Latency for starfish is much more consistent and has overall lower values. It would definitely be very nice to have the pthread_workqueue implementation integrated as a fallback as previously discussed between Mark/Jordan over time. Joakim ————————————————— node5:~/gcd/trunk/testing> ./dispatch_api ================================================== [TEST] Dispatch (Public) API [PID] 13047 ================================================== Actual: 412870 Expected: 412870 [PASS] dispatch_get_main_queue node5:~/gcd/trunk/testing> ./dispatch_c99 ================================================== [TEST] Dispatch C99 [PID] 13062 ================================================== Actual: 412870 Expected: 412870 [PASS] dispatch_get_main_queue node5:~/gcd/trunk/testing> ./dispatch_cascade ================================================== [TEST] Dispatch Cascade [PID] 13077 ================================================== maxcount = 1172 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ** ** * * * * * * * * ** ** * * * * * * * * ** ** * * ** * * * * * * * ** ** * **** * * ** * * * * * * * * ** **** **** * * ** * * ** * * * * * * * ** ** **** **** * * **** * ** * * * **** * * ** * * node5:~/gcd/trunk/testing> ./dispatch_debug ================================================== [TEST] Dispatch Debug [PID] 13092 ================================================== node5:~/gcd/trunk/testing> ./dispatch_priority ================================================== [TEST] Dispatch Priority [PID] 13107 ================================================== LOW: 37 ************************ DEFAULT: 71 ********************************************* HIGH: 84 ***************************************************** Actual: 192 Expected: 192 [PASS] blocks completed Actual: 37 Expected: <84 [PASS] high priority precedence node5:~/gcd/trunk/testing> ./dispatch_priority2 ================================================== [TEST] Dispatch Priority (Set Target Queue) [PID] 13122 ================================================== Actual: 413170 Expected: 413170 [PASS] q[i] Actual: 417480 Expected: 417480 [PASS] q[i] Actual: 417530 Expected: 417530 [PASS] q[i] LOW: 63 **************************************** DEFAULT: 64 **************************************** HIGH: 66 ****************************************** Actual: 193 Expected: 192 [FAIL] blocks completed (dispatch_priority.c:82) dispatch_priority.c:82 Actual: 63 Expected: <66 [PASS] high priority precedence node5:~/gcd/trunk/testing> ./dispatch_starfish ================================================== [TEST] Dispatch Starfish [PID] 13137 ================================================== lap: 10 count: 1000 delta: 723891749 ns math: 361.584290 ns / lap Actual: 361 Expected: <1000 [PASS] Latency lap: 9 count: 1000 delta: 699582448 ns math: 349.441782 ns / lap Actual: 349 Expected: <1000 [PASS] Latency lap: 8 count: 1000 delta: 799519154 ns math: 399.360217 ns / lap Actual: 399 Expected: <1000 [PASS] Latency lap: 7 count: 1000 delta: 802168767 ns math: 400.683700 ns / lap Actual: 400 Expected: <1000 [PASS] Latency lap: 6 count: 1000 delta: 720398591 ns math: 359.839456 ns / lap Actual: 359 Expected: <1000 [PASS] Latency lap: 5 count: 1000 delta: 932910394 ns math: 465.989208 ns / lap Actual: 465 Expected: <1000 [PASS] Latency lap: 4 count: 1000 delta: 825487034 ns math: 412.331186 ns / lap Actual: 412 Expected: <1000 [PASS] Latency lap: 3 count: 1000 delta: 772381517 ns math: 385.804954 ns / lap Actual: 385 Expected: <1000 [PASS] Latency lap: 2 count: 1000 delta: 826056037 ns math: 412.615403 ns / lap Actual: 412 Expected: <1000 [PASS] Latency lap: 1 count: 1000 delta: 867257324 ns math: 433.195467 ns / lap Actual: 433 Expected: <1000 [PASS] Latency node5:~/gcd/trunk/testing> ./queue_finalizer ================================================== [TEST] Dispatch Queue Finalizer [PID] 13152 ================================================== Actual: 4133b0 Expected: 4133b0 [PASS] dispatch_queue_new Actual: 8e3ce4e70cba8e97 Expected: 8e3ce4e70cba8e97 [PASS] finalizer ran node5:~/gcd/trunk/testing> /usr/sbin/psrinfo 0 on-line since 04/30/2010 16:29:35 1 on-line since 04/30/2010 16:29:46 2 on-line since 04/30/2010 16:29:46 3 on-line since 04/30/2010 16:29:46 4 on-line since 04/30/2010 16:29:46 5 on-line since 04/30/2010 16:29:46 6 on-line since 04/30/2010 16:29:46 7 on-line since 04/30/2010 16:29:46 node5:~/gcd/trunk/testing> —————————————————
On Tue, 27 Jul 2010, Joakim Johansson wrote:
dispatch_priority seems to fail spuriously sometimes (even though it passed in the example below) and the starfish latency boundary test sometimes fails - this is on a Xeon 5500 box with 8 cores (of which 4 is HT).
The priority regression tests fail because libdispatch doesn't know how to propagate queue priority to thread priority when using the thread pool model rather than pthread work queues. So this is a bug in GCD that doesn't manifest on Mac OS X as it uses a different kernel feature footprint. It would be nice to (a) fix this problem in GCD and (b) get pthread workqueue compatibility on various operating systems. Stacey seems to have gone quiet lately, but I'm CCing him anyway. :-) Robert
Thanks, I've been running with the userspace libpthread_workqueue implementation with better results - it takes significantly more effort to make it fail then, see attached test runs for a typical (successful) run as well as one where it failed (I had to run 20+ iterations to make it fail though). Also, the "blocks completed" test additionally failed due to a bug in the test outlined in my other mail "Subject: [libdispatch-dev] PATCH - bugfix for dispatch_priority.c spurious failures where actual blocks completed != expected". That one should probably fail on Mac OS X / FreeBSD / Linux as well from time to time. Cheers, Joakim Successful run: ------------------- node5:~/gcd/trunk> testing/dispatch_priority ================================================== [TEST] Dispatch Priority [PID] 6495 ================================================== LOW: 11 ******* DEFAULT: 53 ********************************** HIGH: 128 ******************************************************************************** Actual: 192 Expected: 192 [PASS] blocks completed Actual: 11 Expected: <128 [PASS] high priority precedence node5:~/gcd/trunk> testing/dispatch_priority Failed run: ------------------- node5:~/gcd/trunk> testing/dispatch_priority ================================================== [TEST] Dispatch Priority [PID] 6645 ================================================== LOW: 76 ************************************************ DEFAULT: 47 ****************************** HIGH: 69 ******************************************** Actual: 192 Expected: 192 [PASS] blocks completed Actual: 76 Expected: <69 [FAIL] high priority precedence (dispatch_priority.c:83) dispatch_priority.c:83 node5:~/gcd/trunk> testing/dispatch_priority On 30 jul 2010, at 11.19, Robert Watson wrote:
On Tue, 27 Jul 2010, Joakim Johansson wrote:
dispatch_priority seems to fail spuriously sometimes (even though it passed in the example below) and the starfish latency boundary test sometimes fails - this is on a Xeon 5500 box with 8 cores (of which 4 is HT).
The priority regression tests fail because libdispatch doesn't know how to propagate queue priority to thread priority when using the thread pool model rather than pthread work queues. So this is a bug in GCD that doesn't manifest on Mac OS X as it uses a different kernel feature footprint. It would be nice to (a) fix this problem in GCD and (b) get pthread workqueue compatibility on various operating systems.
Stacey seems to have gone quiet lately, but I'm CCing him anyway. :-)
Robert
participants (2)
-
Joakim Johansson
-
Robert Watson