[libdispatch-dev] Clarification on default target_queue for custom serial queues

[Daniel A. Steffen]

Hi Daniel,

On Jun 25, 2011, at 4:16 PM, Daniel Shusta wrote:

> When reading about libdispatch, I get the sense that I can create serial queues en masse and it’ll just work out, because the queues will be limited to a system-determined number of threads. The reason behind this:
> 
>> In fact, serial queues are scheduled using the global queues. Each serial queue has a target queue, which is initially set to the default priority concurrent queue. When a block is first added to an empty serial queue, the queue itself is added to the target queue.
> 
> -- GCD technical brief, formerly linked from http://developer.apple.com/technologies/mac/snowleopard/gcd.html but is now a dead link. There’s a backup at http://www.ctestlabs.org/hughes_multicore/documents/GrandCentral_TB_brief_20090608.pdf 
> 
> This information is repeated at the WWDC session talks (#210, about 10 and a half minutes into the video available to developers).
> 
> Essentially, global queues are limited to a handful of threads, and since serial queues run on the global queues, they're also all funneled down to that handful of threads.
> 
> But libdispatch doesn’t actually seem to do this. Example code (tested on iOS 4.3, also shows this behavior on Snow Leopard 10.6.7, 10.6.0, and Lion dev previews):
> 
> 
> NSUInteger queueCount = 50;
> dispatch_queue_t queueArray[queueCount];
> 
> for ( NSUInteger i = 0; i < queueCount; ++i ) 
> 	queueArray[i] = dispatch_queue_create("com.some.queue", DISPATCH_QUEUE_SERIAL);
> 
> // Then enqueue expensive blocks on each of these with a dispatch group, wait for the group to finish, and clean up.
> 
> 
> The process will just go ahead and create 50 threads. On a dual-core machine, this isn’t really what I was hoping for. But two lines of adjustments…
> 
> 
> NSUInteger queueCount = 50;
> dispatch_queue_t queueArray[queueCount];
> dispatch_queue_t defaultGlobalQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
> 
> for ( NSUInteger i = 0; i < queueCount; ++i ) {
> 	queueArray[i] = dispatch_queue_create("com.some.queue", DISPATCH_QUEUE_SERIAL);		
> 	
> 	dispatch_set_target_queue(queueArray[i], defaultGlobalQueue);
> }
> 
> // Then enqueue expensive blocks on each of these with a dispatch group, wait for the group to finish, and clean up.
> 
> 
> …will limit it to a couple of threads.
> 
> According to the documentation quoted above, these two code samples should produce identical results -- serial queues should already target the default global queue. Yet the actual behaviors are very different.
> 
> Is this intentional? Am I misunderstanding something?
> 
> Thanks for your time and whatever response you can give,
> --Daniel Shusta
> 
> ===
> PS. this is specific to custom serial queues. Custom concurrent queues in iOS 4.3+ funnel down to a couple of threads.
> 
> PPS. In libdispatch's queue.c, it does seem like the target queue is supposed to be the default global queue. In _dispatch_queue_init() we see 
> 
> dq->do_targetq = _dispatch_get_root_queue(0, true);

it is this 'true' here that makes all the difference, the default target queue of a serial queue is the _overcommit_ default-priority global concurrent queue.

For items/queues submitted to an overcommit global queue, the current Mac OS X kernel workqueue mechanism creates threads more eagerly, e.g. even if an n-wide machine is already fully committed with n cpu-busy threads, submitting another item directly to the overcommit global queue or indirectly to a serial queue with default target queue will cause another thread to be created to handle that item (potentially overcommitting the machine, hence the name).

If you wish to avoid this, simply set the target queue of your serial queues to the default priority global queue (i.e. non-overcommit).

The overcommit/non-overcommit distinction is intentionally undocumented and only available in the queue_private.h header because we hope to revise the kernel workqueue mechanism in the future to avoid the need for this distinction.

Daniel