[launchd-dev] QueueDirectories problem

[Michal Taurich]

Hello all,

I have very strange launchd problem. I wrote a small script to watch defined folder and if anything is added to this folder, it copy the file via SCP to different machine and than move the file to another folder. The script looks like following:

#!/bin/bash

in=/in
remote="user at xxx.xxx.xxx.xx:/path"
out=/out

OLDIFS=$IFS
IFS="$(echo -e \"\\n\")"

for i in $(find $in -type f)
do

IFS=$OLDIFS

scp "${i}" "${remote}"

IFS=$OLDIFS

if [ $? = 0 ];
then
mv "${i}" "${out}"
fi;
done

IFS=$OLDIFS

exit 0

I have also launchd .plist for it like follows:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<string>scp_upload</string>
<key>ProgramArguments</key>
<array>
<string>/script</string>
</array>
<key>QueueDirectories</key>
<array>
<string>/in</string>
</array>
<key>RunAtLoad</key>
<true/>
</dict>
</plist>

It works perfectly locally. Unfortunately, when I run it on remote host and connect to shared drive via AFP and try to copy file from my computer to "watched folder" I got errors like "Unsupported character" or "filename too long" or so. I discovered, the launchd starts immediately when filesytem is changed (The Finder shows 4 kb as file size, but in Terminal it shows 0 kb so I think it is copied just header). Imagine following example: You want to receive some incoming files from internet (e.g. several customers send you images) and you want to move it according filenames (filenames are only numbers). So files starts with 1xxxx will be moved to folder 10000-19999. And now comes the tricky situation. Because launchd works like it works, when customer starts uploading the file, there is made a file system record for that file. But because launchd is set as "watching for folder changes" it tries to move the file, even the file is not completed (imagine e.g. 20 MB print quality 
 photos). So launchd fails and write in console log "threat" like "9 times I will fail and I will disable myself". So it is just a question of time when launchd will disable itself. So my question is how to avoid this behaviour. I tried to add an condition in my script based on following:

lsof | grep -c "${in}/${i}"

if [ $? = 0 ];
then
exit 0;
fi 

Basically it checks in list of open files, if the file is opened, if yes, than exit0, otherwise continue. But it also sometimes works and sometimes not (better say, some files are moved and some not and I can wait hours and the files sit on the same place even launchd is running). So I would like to ask someone if have some experience with this problem and can recommend me some workarround or some approved and tested way, how to avoid move/copy/rename or whatever incomplete files. Thanks in advance.

Michal