Using libdispatch/GCD in a real-time context
Hi, We are interested to see if libdispatch/GCD could be used in a real-time context, more specifically in JACK audio server (http://jackaudio.org/). JACK audio server can connect a number of different applications to an audio device, as well as allowing them to share audio between themselves. Its clients can run in their own processes (ie. as normal applications), or can they can run within the JACK server (ie. as a "plugin"). JACK allows synchronous execution of all clients, and low latency operation. All JACK applications are connected in a graph and the server (triggered by a CoreAudio audio callback) is responsible to "execute the graph" at each audio cycle, starting from the inputs and transferring "activation" from client to client until the outputs of the graph have been executed. On OSX JACK uses mach semaphores for inter-process communication. Each client start a thread and give it RT behavior (more precisely the thread is a "time-constraint" thread in OSX terminology and inherit the "period, computation, constraints" parameters of the server CoreAudio RT thread). On multi-cores machines, audio clients may themselves want to use additional helper threads, to speed up audio computation. This can be done using a pool of RT threads, but obviously doing like that does not scale well, when several applications do the same thing. We looked at libdispatch/GCD as a possible alternative for this "explicitly managing RT threads" model. We were thinking in completely dropping the client RT thread and replace it by a "dispatch_source" to be triggered by the inter-process semaphore. Then use libdispatch again to replace the need for RT helper threads. Right now there is no way to define a real-time queue in libdispatch, but would this kind of extension be theoreticaly possible? What would be the way to do that? Any comment on the overall idea? Best Regards Stéphane Letz
On 05/12/2010 07:12 AM, Stéphane Letz wrote:
Right now there is no way to define a real-time queue in libdispatch, but would this kind of extension be theoreticaly possible? What would be the way to do that? Any comment on the overall idea?
Hi, I think it would be possible to add realtime support to libdispatch, although you would also need changes to pthread_workqueue(3) libc functions. It sounds like you are using the Mach function thread_policy_set(), which is not portable and also not very general-purpose. Have you tried using pthread_setschedparam(3) with the SCHED_FIFO scheduling class, and would this approach be acceptable for JACK? Regards, - Mark
Le 17 mai 2010 à 05:46, Mark Heily a écrit :
On 05/12/2010 07:12 AM, Stéphane Letz wrote:
Right now there is no way to define a real-time queue in libdispatch, but would this kind of extension be theoreticaly possible? What would be the way to do that? Any comment on the overall idea?
Hi,
I think it would be possible to add realtime support to libdispatch, although you would also need changes to pthread_workqueue(3) libc functions.
It sounds like you are using the Mach function thread_policy_set(), which is not portable and also not very general-purpose. Have you tried using pthread_setschedparam(3) with the SCHED_FIFO scheduling class, and would this approach be acceptable for JACK?
Regards,
- Mark
Using POSIX function is not enough. Real-time threads on OSX are "time constraints" threads (using the thread_policy_set API with THREAD_TIME_CONSTRAINT_POLICY) http://developer.apple.com/mac/library/documentation/Darwin/Conceptual/Kerne... What would be the different steps to follow to add realtime support to libdispatch then ? Regards Stephane Letz
On 05/17/2010 05:20 AM, Stéphane Letz wrote:
What would be the different steps to follow to add realtime support to
libdispatch then ?
I would start by filing an RFE bug to the libdispatch tracker, since this is a complex request that may take some time to implement. Here's the link: http://libdispatch.macosforge.org/trac/newticket The dispatch_queue_create() function and the dispatch_queue_attr struct could be extended to specify a realtime queue [1]. The pthread_workqueue_attr struct could be extended to support a time_constraint_policy for the worker threads [2]. Each realtime dispatch_queue would use a separate pthread_workqueue whose threads would run with the desired time_constraint_policy. For additional context, take a look at the pthread_workqueue code I pulled from the xnu and darwin sources: http://mark.heily.com/src/libpthread_workqueue.tar Regards, - Mark [1] typedef struct dispatch_queue_attr { int thread_policy; struct thread_time_constraint_policy ttcpolicy; } dispatch_queue_attr_t; dispatch_queue_attr_t attr = { .thread_policy = THREAD_TIME_CONSTRAINT_POLICY, .ttcpolicy = { .period = ?, .computation = ?, .constraint = ?, .preemptible = 1, }, }; my_queue = dispatch_queue_create("org.jack.realtime.audio", &attr); [2] typedef struct { uint32_t sig; int queueprio; int overcommit; + int thread_policy; + struct thread_time_constraint_policy *ttcpolicy; unsigned int resv2[13]; } pthread_workqueue_attr_t; +int pthread_workqueue_attr_settimeconstraintpolicy_np( + pthread_workqueue_attr_t * attr, + struct thread_time_constraint_policy *ttcpolicy);
I would start by filing an RFE bug to the libdispatch tracker, since this is a complex request that may take some time to implement. Here's the link:
http://libdispatch.macosforge.org/trac/newticket
The dispatch_queue_create() function and the dispatch_queue_attr struct could be extended to specify a realtime queue [1]. The pthread_workqueue_attr struct could be extended to support a time_constraint_policy for the worker threads [2]. Each realtime dispatch_queue would use a separate pthread_workqueue whose threads would run with the desired time_constraint_policy.
For additional context, take a look at the pthread_workqueue code I pulled from the xnu and darwin sources:
http://mark.heily.com/src/libpthread_workqueue.tar
Regards,
- Mark
[1]
typedef struct dispatch_queue_attr { int thread_policy; struct thread_time_constraint_policy ttcpolicy; } dispatch_queue_attr_t;
dispatch_queue_attr_t attr = { .thread_policy = THREAD_TIME_CONSTRAINT_POLICY, .ttcpolicy = { .period = ?, .computation = ?, .constraint = ?, .preemptible = 1, }, };
my_queue = dispatch_queue_create("org.jack.realtime.audio", &attr);
[2]
typedef struct { uint32_t sig; int queueprio; int overcommit; + int thread_policy; + struct thread_time_constraint_policy *ttcpolicy; unsigned int resv2[13]; } pthread_workqueue_attr_t;
+int pthread_workqueue_attr_settimeconstraintpolicy_np( + pthread_workqueue_attr_t * attr, + struct thread_time_constraint_policy *ttcpolicy);
Thanks. Before going further, I've adapted our "fine-grained" parallel code (that is using an ad-hoc Work Stealing Scheduler and a pool or RT threads..) to use libdispatch. Performances are currently much worse. - I see that using blocks is also allocating memory, which is usually forbidden in RT code (I guess "dispatch_async_f" should be used instead of "dispatch_async" right ?) - The parallel queue is DISPATCH_QUEUE_PRIORITY_HIGH, I guess that I cannot change the GCD thread priorities on the fly right? Thanks. Stéphane Letz
participants (2)
-
Mark Heily
-
Stéphane Letz