Enforce a maximum time that a launchd process can live...
Hello, I have a process I would like to used launchd to control. Its job is very simple; look for messages that are waiting in a SQLite database and send them synchronously with NSMutableURLRequest via HTTP. The problem I face is occasionally on production machines I have found that the networking request becomes stalled for some reason which causes the process to simply sit waiting. I was hoping that launchd would be able to enforce a maximum time that a process can live using the ExitTimeout key. The documentation isn't very clear to me on this point. At first I took it as this is the maximum length of time the process would live before launchd would send it a SIGKILL. However, it was pointed out to me by someone they believe it is the time between launchd sending a SIGTERM and SIGKILL. This would only occur if the launchd process was stopped or removed and doesn't exit within the specified ExitTimeout causing a SIGKILL to be sent. In the ideal world people would say, "Your problem is a bug and it needs to be fixed." However, in the real world where frameworks are not under my control this isn't an option. In this case I want to work around the problem. I want to specify a maximum amount of time that a process can live and then have it terminated if it exceeds that time. Can I accomplish a process living for a certain amount of time and then being asked to terminate via launchd? If not, what other options do I have? Thanks, Michael
On Jan 26, 2010, at 8:04 AM, Michael Ledford wrote:
Hello,
I have a process I would like to used launchd to control. Its job is very simple; look for messages that are waiting in a SQLite database and send them synchronously with NSMutableURLRequest via HTTP. The problem I face is occasionally on production machines I have found that the networking request becomes stalled for some reason which causes the process to simply sit waiting.
I was hoping that launchd would be able to enforce a maximum time that a process can live using the ExitTimeout key.
In general, launchd doesn't automatically kill jobs except when a session is being torn down.
The documentation isn't very clear to me on this point. At first I took it as this is the maximum length of time the process would live before launchd would send it a SIGKILL. However, it was pointed out to me by someone they believe it is the time between launchd sending a SIGTERM and SIGKILL. This would only occur if the launchd process was stopped or removed and doesn't exit within the specified ExitTimeout causing a SIGKILL to be sent.
Correct.
In the ideal world people would say, "Your problem is a bug and it needs to be fixed." However, in the real world where frameworks are not under my control this isn't an option. In this case I want to work around the problem. I want to specify a maximum amount of time that a process can live and then have it terminated if it exceeds that time.
Is the NSMutableURLRequest encapsulated in some API that you're using which doesn't support a timeout interval? Because NSMutableURLRequest itself does. If this is the case, you can spin the synchronous call out into its own thread/GCD queue and install a timer that exits after 20 seconds (or however long you'd like). Alternatively, you could use the SCNetworkReachability APIs to get a callback when your destination is reachable and attempt the send then. You'd also have the aforementioned timer so that, if the host doesn't become reachable within a certain amount of time, you give up and exit.
Can I accomplish a process living for a certain amount of time and then being asked to terminate via launchd? If not, what other options do I have?
See above. -- Damien Sorresso BSD Engineering Apple Inc.
On Jan 26, 2010, at 12:14 PM, Damien Sorresso wrote:
Is the NSMutableURLRequest encapsulated in some API that you're using which doesn't support a timeout interval? Because NSMutableURLRequest itself does.
No, the problem is with the NSMutableURLRequest itself. Even though it supports a timeout, default is 60 seconds, it simply gets stuck. I spoke with an engineer at WWDC about it and he was perplexed when we connected to a machine with the process and sampled it. In any case this subject is off topic for this list.
If this is the case, you can spin the synchronous call out into its own thread/GCD queue and install a timer that exits after 20 seconds (or however long you'd like).
I was contemplating installing an alarm, using ualarm(), and just allowing the default handler terminate when it gets the SIGALRM. Would this be acceptable? Thanks, Michael
On Jan 26, 2010, at 11:23 AM, Michael Ledford wrote:
On Jan 26, 2010, at 12:14 PM, Damien Sorresso wrote:
Is the NSMutableURLRequest encapsulated in some API that you're using which doesn't support a timeout interval? Because NSMutableURLRequest itself does.
No, the problem is with the NSMutableURLRequest itself. Even though it supports a timeout, default is 60 seconds, it simply gets stuck. I spoke with an engineer at WWDC about it and he was perplexed when we connected to a machine with the process and sampled it. In any case this subject is off topic for this list.
In that case, you should sample the process when it's stuck in NSMutableURLRequest and file a bug.
If this is the case, you can spin the synchronous call out into its own thread/GCD queue and install a timer that exits after 20 seconds (or however long you'd like).
I was contemplating installing an alarm, using ualarm(), and just allowing the default handler terminate when it gets the SIGALRM. Would this be acceptable?
I guess. But you're never really guaranteed that some framework behind the scenes isn't going to hijack SIGALRM. That's why I recommended GCD. -- Damien Sorresso BSD Engineering Apple Inc.
participants (2)
-
Damien Sorresso
-
Michael Ledford