Re: [launchd-dev] Diagnosing launchd daemon failure to launch
Ahh, that was the problem. So the real issue is that my daemon is loading, but not responding to my client app. I'm trying to use Distributed Objects on top of UNIX Domain sockets. The directory for the socket is /var/tmp, and the client and server can communicate if I run both programs inside XCode. However, when I move the daemon program to /Library/PrivilegedHelperTools and copy to plist into /Library/LaunchDaemons everything stops working. The client seems to be hanging while trying to connect to the socket. Any ideas what I'm doing wrong? Thanks, Iain At 10:46 -0500 5/11/08, Iain Delaney wrote:
Then I reboot, and run 'launchctl list' in the terminal.
Did you use "sudo launchctl list", or just "launchctl list"? The latter will show you the list of jobs for login session launchd, not for the global launchd. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Relations, Developer Technical Support, Core OS/Hardware
On Nov 6, 2008, at 10:37 AM, Iain Delaney wrote:
Ahh, that was the problem. So the real issue is that my daemon is loading, but not responding to my client app.
I'm trying to use Distributed Objects on top of UNIX Domain sockets. The directory for the socket is /var/tmp, and the client and server can communicate if I run both programs inside XCode. However, when I move the daemon program to /Library/PrivilegedHelperTools and copy to plist into /Library/LaunchDaemons everything stops working. The client seems to be hanging while trying to connect to the socket.
Any ideas what I'm doing wrong?
I would highly recommend choosing a more stable location for your socket, first of all. /var/tmp gets cleaned out periodically; I'd suggest /var/run, which is only cleaned out at boot-time. Also, have you specified the socket in your daemon's launchd.plist(5), and does your daemon check in with launchd to obtain the descriptor for the socket? If the answer to either of these questions is "No", please see the SampleD project. http://developer.apple.com/samplecode/SampleD/listing3.html -- Damien Sorresso BSD Engineering Apple Inc.
Hi Damien, Thanks for the feedback. I've looked at the SampleD code and plist, but there's something I don't understand. I based my domain sockets code on the code in the CFLocalServer sample program, and that code uses a string constant for the socket path, shared between the client and server. What is the purpose of putting the path in the daemon plist? If I put the path in the plist, and the server gets it from launchd, how does the client get to the socket? Thanks for your help, Iain Delaney On 6-Nov-08, at 2:04 PM, Damien Sorresso wrote:
On Nov 6, 2008, at 10:37 AM, Iain Delaney wrote:
Ahh, that was the problem. So the real issue is that my daemon is loading, but not responding to my client app.
I'm trying to use Distributed Objects on top of UNIX Domain sockets. The directory for the socket is /var/tmp, and the client and server can communicate if I run both programs inside XCode. However, when I move the daemon program to /Library/ PrivilegedHelperTools and copy to plist into /Library/LaunchDaemons everything stops working. The client seems to be hanging while trying to connect to the socket.
Any ideas what I'm doing wrong?
I would highly recommend choosing a more stable location for your socket, first of all. /var/tmp gets cleaned out periodically; I'd suggest /var/run, which is only cleaned out at boot-time.
Also, have you specified the socket in your daemon's launchd.plist(5), and does your daemon check in with launchd to obtain the descriptor for the socket? If the answer to either of these questions is "No", please see the SampleD project.
http://developer.apple.com/samplecode/SampleD/listing3.html -- Damien Sorresso BSD Engineering Apple Inc.
On Nov 6, 2008, at 12:17 PM, Iain Delaney wrote:
Hi Damien, Thanks for the feedback. I've looked at the SampleD code and plist, but there's something I don't understand. I based my domain sockets code on the code in the CFLocalServer sample program, and that code uses a string constant for the socket path, shared between the client and server. What is the purpose of putting the path in the daemon plist? If I put the path in the plist, and the server gets it from launchd, how does the client get to the socket?
Iain, This is all illustrated in SampleD. Your daemon obtains a file descriptor directly from launchd. This is so launchd can launch your daemon on-demand when there are data on the socket. You can create a CFSocketRef with that descriptor using CFSocketCreateWithNative() if you wish. -- Damien Sorresso BSD Engineering Apple Inc.
I am confused, I thought the purpose of setting the sticky bit on /var/ tmp was so you could count on your files being persistent there? What comes along and cleans out this directory? This is not to say your suggestion to to move to /var/run is not valid, but the code I inherited uses /var/tmp based on the security concerns at the time. The daemon, now best described as a LaunchAgent on Leopard that is in production I want to move to the launchd model from it's current StartupItems implementation. However I am not quite clear on how to let the domain socket listener know that I have been killed and restarted, is it my job to kill off the listener and restart, or could I create another another launchd task to watch for the creation of a new domain socket and would cause the listener to reconnect? For security reasons you do want to create a new domain socket, right? In testing I have also noticed that I can loop when launched, how would I set a limit or a backoff with a runaway launchd process? Ned Hogan MacCruzIn Enterprises, Inc. Apple Certified System Administrator XSAN/Open Directory Certified Apple Developer Connection Select On Nov 6, 2008, at 11:04 AM, Damien Sorresso wrote:
On Nov 6, 2008, at 10:37 AM, Iain Delaney wrote:
Ahh, that was the problem. So the real issue is that my daemon is loading, but not responding to my client app.
I'm trying to use Distributed Objects on top of UNIX Domain sockets. The directory for the socket is /var/tmp, and the client and server can communicate if I run both programs inside XCode. However, when I move the daemon program to /Library/ PrivilegedHelperTools and copy to plist into /Library/LaunchDaemons everything stops working. The client seems to be hanging while trying to connect to the socket.
Any ideas what I'm doing wrong?
I would highly recommend choosing a more stable location for your socket, first of all. /var/tmp gets cleaned out periodically; I'd suggest /var/run, which is only cleaned out at boot-time.
Also, have you specified the socket in your daemon's launchd.plist(5), and does your daemon check in with launchd to obtain the descriptor for the socket? If the answer to either of these questions is "No", please see the SampleD project.
http://developer.apple.com/samplecode/SampleD/listing3.html -- Damien Sorresso BSD Engineering Apple Inc.
_______________________________________________ launchd-dev mailing list launchd-dev@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/launchd-dev
On Nov 7, 2008, at 7:13 AM, Ned Hogan wrote:
I am confused, I thought the purpose of setting the sticky bit on / var/tmp was so you could count on your files being persistent there? What comes along and cleans out this directory?
D'oh! I was thinking of /private/tmp, not /var/tmp. Sorry.
This is not to say your suggestion to to move to /var/run is not valid, but the code I inherited uses /var/tmp based on the security concerns at the time.
The daemon, now best described as a LaunchAgent on Leopard that is in production I want to move to the launchd model from it's current StartupItems implementation. However I am not quite clear on how to let the domain socket listener know that I have been killed and restarted, is it my job to kill off the listener and restart, or could I create another another launchd task to watch for the creation of a new domain socket and would cause the listener to reconnect? For security reasons you do want to create a new domain socket, right?
No. In the launchd model, launchd will hold a file descriptor for the Unix domain socket and listen on it. When data are available, launchd will start your agent up. Your agent will then check in with launchd and obtain the file descriptor and start reading from it. If your agent dies when there is still unread data on the socket, it'll be started back up, and you can pick up where you left off in the stream. Let launchd worry about the socket. All your agent needs to know is that it was launched because there are data on the socket that need to be read.
In testing I have also noticed that I can loop when launched, how would I set a limit or a backoff with a runaway launchd process?
See launchd.plist(5), specifically, the KeepAlive criteria. -- Damien Sorresso BSD Engineering Apple Inc.
Sorry, I did make myself clear, I have two processes, one that runs as a damon in the background and should remain persistent, and another that is valid only in the aqua context. The first is a traditional UNIX style daemon, called AgentMon, that is a client to a Server in the Cloud that runs in the background as root, all of this code was orginally based on the inetd or xinetd model, similar to rshd or sshd daemon. It's been cleaned up to not do fork children via the tech note. It in turn communicates with a second local user agent, UsrTask, that is launched via a login hook (loginwindow.plist). We can but don't support FUS, so one and only one UsrTask to represent the user along with one and only one the AgentMon to represent the system. This again complicates things as far as simply having it launch in the aqua context, but I should be able to use ~/Library verses /Library The user has the option to quit the UsrTask, the daemon should stay alive in most but not all cases. In the case that AgentMon also needs to exit, I could handle this by having the daemon remove the QueueDirectory toggle file, and then have the LaunchAgent exit. Touch the queue to relaunch. The QueueDirecotry makes sense for AgentMon, and I am happy with the way this works. The question is should I make the QueueDirectories the Domain Socket for UsrTask? This thread now makes me ponder if I should be using the Sockets method instead. Hopefully this makes sense, just looking for some advice, but I am sure I can figure it out one way or another. On Nov 7, 2008, at 10:57 AM, Damien Sorresso wrote:
On Nov 7, 2008, at 7:13 AM, Ned Hogan wrote:
I am confused, I thought the purpose of setting the sticky bit on / var/tmp was so you could count on your files being persistent there? What comes along and cleans out this directory?
D'oh! I was thinking of /private/tmp, not /var/tmp. Sorry.
This is not to say your suggestion to to move to /var/run is not valid, but the code I inherited uses /var/tmp based on the security concerns at the time.
The daemon, now best described as a LaunchAgent on Leopard that is in production I want to move to the launchd model from it's current StartupItems implementation. However I am not quite clear on how to let the domain socket listener know that I have been killed and restarted, is it my job to kill off the listener and restart, or could I create another another launchd task to watch for the creation of a new domain socket and would cause the listener to reconnect? For security reasons you do want to create a new domain socket, right?
No. In the launchd model, launchd will hold a file descriptor for the Unix domain socket and listen on it. When data are available, launchd will start your agent up. Your agent will then check in with launchd and obtain the file descriptor and start reading from it. If your agent dies when there is still unread data on the socket, it'll be started back up, and you can pick up where you left off in the stream.
Let launchd worry about the socket. All your agent needs to know is that it was launched because there are data on the socket that need to be read.
In testing I have also noticed that I can loop when launched, how would I set a limit or a backoff with a runaway launchd process?
See launchd.plist(5), specifically, the KeepAlive criteria. -- Damien Sorresso BSD Engineering Apple Inc.
Iain, Alternatively, given that your code uses Distributed Objects, one can add a Mach service to the launchd property-list and then use this on the server side: id serverBinding = [NSConnection serviceConnectionWithName: @"com.example.exampleMachService" rootObject: exampleObject]; And on the client side: id exampleRemoteObject = [NSConnection connectionWithRegisteredName: @"com.example.exampleMachService" host: nil]; Cheers, davez On Nov 6, 2008, at 10:37 AM, Iain Delaney wrote:
Ahh, that was the problem. So the real issue is that my daemon is loading, but not responding to my client app. I'm trying to use Distributed Objects on top of UNIX Domain sockets. The directory for the socket is /var/tmp, and the client and server can communicate if I run both programs inside XCode. However, when I move the daemon program to /Library/PrivilegedHelperTools and copy to plist into /Library/LaunchDaemons everything stops working. The client seems to be hanging while trying to connect to the socket. Any ideas what I'm doing wrong? Thanks, Iain At 10:46 -0500 5/11/08, Iain Delaney wrote:
Then I reboot, and run 'launchctl list' in the terminal.
Did you use "sudo launchctl list", or just "launchctl list"? The latter will show you the list of jobs for login session launchd, not for the global launchd.
S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/
Apple Developer Relations, Developer Technical Support, Core OS/ Hardware _______________________________________________ launchd-dev mailing list launchd-dev@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/launchd-dev
Hi Dave, Thanks for the input, but if I want to use Mach services, couldn't I just use a named DO connection (like [NSConnection registerName:] )? Then I don't need to put anything about a Mach service in the plist. I tried this earlier and it seemed to work fine. Then I read tech note 2083: Daemons and Agents and found that DTS recommends against using Mach messages and suggests UNIX Domain sockets instead. That's how I ended up with my current problem. Are the Mach APIs and Distributed Objects really as dangerous as the tech note leads me to believe? Thanks for your help, Iain Delaney On 6-Nov-08, at 5:14 PM, Dave Zarzycki wrote:
Iain,
Alternatively, given that your code uses Distributed Objects, one can add a Mach service to the launchd property-list and then use this on the server side:
id serverBinding = [NSConnection serviceConnectionWithName: @"com.example.exampleMachService" rootObject: exampleObject];
And on the client side:
id exampleRemoteObject = [NSConnection connectionWithRegisteredName: @"com.example.exampleMachService" host: nil];
Cheers,
davez
On Nov 6, 2008, at 10:37 AM, Iain Delaney wrote:
Ahh, that was the problem. So the real issue is that my daemon is loading, but not responding to my client app. I'm trying to use Distributed Objects on top of UNIX Domain sockets. The directory for the socket is /var/tmp, and the client and server can communicate if I run both programs inside XCode. However, when I move the daemon program to /Library/ PrivilegedHelperTools and copy to plist into /Library/LaunchDaemons everything stops working. The client seems to be hanging while trying to connect to the socket. Any ideas what I'm doing wrong? Thanks, Iain At 10:46 -0500 5/11/08, Iain Delaney wrote:
Then I reboot, and run 'launchctl list' in the terminal.
Did you use "sudo launchctl list", or just "launchctl list"? The latter will show you the list of jobs for login session launchd, not for the global launchd.
S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/
Apple Developer Relations, Developer Technical Support, Core OS/ Hardware _______________________________________________ launchd-dev mailing list launchd-dev@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/launchd-dev
At 8:37 -0500 7/11/08, Iain Delaney wrote:
Thanks for the input, but if I want to use Mach services, couldn't I just use a named DO connection (like [NSConnection registerName:] )? Then I don't need to put anything about a Mach service in the plist. I tried this earlier and it seemed to work fine.
This will work, but your daemon will have to run all the time. If you take the path that DaveZ suggests, you can launch on demand.
Then I read tech note 2083: Daemons and Agents and found that DTS recommends against using Mach messages and suggests UNIX Domain sockets instead. That's how I ended up with my current problem. Are the Mach APIs and Distributed Objects really as dangerous as the tech note leads me to believe?
In general we recommend staying away from the Mach APIs. However, the situation is not cut and dry. If it was, we'd deprecate the Mach APIs, or eliminate them entirely. The text in TN2083 tries to cover some of the subtleties of this situation, but it's hard to do without discussing specifics. In the specific case of using DO for local IPC, I'd definitely recommend you use Mach for your transport. DO was written with the Mach IPC model in mind. DO-over-sockets has to jump through lots of hoops to make sockets look like Mach ports, and that produces all sorts of wacky edge cases. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Relations, Developer Technical Support, Core OS/Hardware
participants (5)
-
Damien Sorresso
-
Dave Zarzycki
-
Iain Delaney
-
Ned Hogan
-
Quinn