[CalendarServer-dev] Exception in recvfd --was: Re: continuing work on FreeBSD ports

Axel Rau Axel.Rau at Chaos1.DE
Wed Feb 13 14:25:46 PST 2013


Hi Fred,

thanks a lot for the debug patch.
It works with trial:
---
# trial test/test_sendmsg.py
twext.python.test.test_sendmsg
  SendmsgTestCase
    test_roundtrip ...                                                     [OK]
    test_sendSubProcessFD ...                                           [ERROR]
    test_wrongTypeAncillary ...                                            [OK]

===============================================================================
[ERROR]
Traceback (most recent call last):
Failure: twext.python.test.test_sendmsg.ExitedWithStderr: 
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/Calendar_and_Contacts_Server-4.2-py2.7-freebsd-9.1-RELEASE-amd64.egg/twext/python/test/pullpipe.py", line 22, in <module>
    fd, description = recvfd(int(sys.argv[1]))
  File "/usr/local/lib/python2.7/site-packages/Calendar_and_Contacts_Server-4.2-py2.7-freebsd-9.1-RELEASE-amd64.egg/twext/python/sendfd.py", line 65, in recvfd
    [unpackedFD] = unpack("i", packedFD)
struct.error: unpack requires a string argument of length 4

twext.python.test.test_sendmsg.SendmsgTestCase.test_sendSubProcessFD
-------------------------------------------------------------------------------
Ran 3 tests in 0.122s

FAILED (errors=1, successes=2)
# ls -l /tmp/debug.dat*
total 2
-rw-r--r--  1 root   wheel   31 Feb 13 23:05 debug.dat.48537
-rw-r--r--  1 root   wheel   55 Feb 13 23:05 debug.dat.48538
# cat /tmp/debug.dat.4853*
recvmsg: fd [8] [0] result [d]
recvmsg: fd [8] [0] result [5]
recvmsg: value:
3 0 0 0
# rm /tmp/debug.dat.4853*
---
But produces no output in real life:
---
[caldav-0] 	--- <exception caught here> ---
[caldav-0] 	  File "/usr/local/lib/python2.7/site-packages/twisted/internet/kqreactor.py", line 279, in _doWriteOrRead
[caldav-0] 	    why = selectable.doRead()
[caldav-0] 	  File "/usr/local/lib/python2.7/site-packages/Calendar_and_Contacts_Server-4.2-py2.7-freebsd-9.1-RELEASE-amd64.egg/twext/internet/sendfdport.py", line 281, in doRead
[caldav-0] 	    fd, description = recvfd(self.fd)
[caldav-0] 	  File "/usr/local/lib/python2.7/site-packages/Calendar_and_Contacts_Server-4.2-py2.7-freebsd-9.1-RELEASE-amd64.egg/twext/python/sendfd.py", line 65, in recvfd
[caldav-0] 	    [unpackedFD] = unpack("i", packedFD)
[caldav-0] 	struct.error: unpack requires a string argument of length 4
...
# ls -l /tmp/debug.dat.485*
-rw-r--r--  1 root  wheel  0 Feb 13 23:07 /tmp/debug.dat.48563
-rw-r--r--  1 root  wheel  0 Feb 13 23:07 /tmp/debug.dat.48577
-rw-r--r--  1 root  wheel  0 Feb 13 23:07 /tmp/debug.dat.48578
-rw-r--r--  1 root  wheel  0 Feb 13 23:07 /tmp/debug.dat.48579
-rw-r--r--  1 root  wheel  0 Feb 13 23:07 /tmp/debug.dat.48580
#
---
I have no explanation for that.
Axel

Am 13.02.2013 um 20:20 schrieb Fredrik Unger:

> Hi,
> 
> Ok.
> 
> My problems were not with the kqreactor.py so I have not so much experience of that does but .
> 
> To concentrate on the sendfd.py problem:
> 
> [caldav-0] 	  File "/usr/local/lib/python2.7/site-packages/Calendar_and_Contacts_Server-4.2-py2.7-freebsd-8.2-RELEASE-p9-amd64.egg/twext/python/sendfd.py", line 65, in recvfd
> [caldav-0] 	    [unpackedFD] = unpack("i", packedFD)
> [caldav-0] 	struct.error: unpack requires a string argument of
> length 4
> 
> What I did was to add a debug to file thing in sendmsg.c
> Attached a small test patch (against current trunk).
> Patch it, then you need to compile the sendmsg.c module :
> 
> gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python2.7 -c sendmsg.c -o sendmsg.o
> 
> 
> gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,relro -Wl,-z,relro -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 sendmsg.o -o sendmsg.so
> 
> or build whole calendar server.
> 
> Then run the test
> 
> trial test/test_sendmsg.py
> 
> In ls /tmp/debug.dat.PID should be the output..
> In my case :
> recvmsg: fd [8] [0] result [d]
> 
> Now, you can adapt the things to track your problem and start
> a real calendar server, at the crash there should be several pid files..
> maybe you get some data out of them..
> 
> /Fred
> 
> 
> 
> On 02/13/2013 12:05 AM, Axel Rau wrote:
>> I did not make any progress. I tried
>> - in sendmsg_sendmsg to check for the right type arg (with PyErr_Format, got no error)
>> - to print out the args with PySys_WriteStdout (did not get anything)
>> - I'm not even sure, if this error is related to my problem:
>>     test_sendSubProcessFD ...                         [ERROR]
>> - If I try control_message->cmsg_type = 0;, I get Errno 22 in sendfd, as expected
>> 
>> I need more pointers...
>> 
>> Axel
>> 
>> Am 08.02.2013 um 13:30 schrieb Fredrik Unger:
>> 
>>> Hi,
>>> 
>>> I found some problem with sendfd.c some time ago. [1]
>>> 
>>> My problem was with 64 bit Debian and twext/python/sendmsg.c
>>> 
>>> Have a look and see if there is anything that is FreeBSD non friendly there.
>>> 
>>> You can try with the tests :
>>> cd twext/python
>>> trial test/test_sendmsg.py
>>> That could be easier to debug than the full code.
>>> 
>>> On my setup I failed test_sendSubProcessFD now, but I just did a quick test. Sources are not up to date or anything..
>>> 
>>> You can also add some printouts to the C code, and see if you find something bad.
>>> 
>>> It is most probably some flags or so that might not be covered for BSD ?
>>> and sendmsg.c is the interface to the operating system from python..
>>> 
>>> Just some suggestions. If you need more pointers let me know, but I have no experience with FreeBSD..
>>> 
>>> /Fred
>>> 
>>> [1] https://lists.macosforge.org/pipermail/calendarserver-users/2012-August/002032.html
>>> 
>>> On 02/08/2013 01:04 PM, Axel Rau wrote:
>>>> 
>>>> Am 02.02.2013 um 17:39 schrieb Axel Rau:
>>>> 
>>>>> [caldav-0] 	  File "/usr/local/lib/python2.7/site-packages/Calendar_and_Contacts_Server-4.2-py2.7-freebsd-8.2-RELEASE-p9-amd64.egg/twext/python/sendfd.py", line 65, in recvfd
>>>>> [caldav-0] 	    [unpackedFD] = unpack("i", packedFD)
>>>>> [caldav-0] 	struct.error: unpack requires a string argument of length 4
>>>> This is hard to debug. I did not make much progress.
>>>> If I switch to kqueue-reactor, I'm getting another FD problem when starting listening:
>>>> ---
>>>> Log opened.
>>>> twistd 12.1.0 (/usr/local/bin/python2.7 2.7.3) starting up.
>>>> reactor class: twext.internet.kqreactor.KQueueReactor.
>>>> ControlSocket starting on '/var/db/caldavd/caldavd.sock'
>>>> Traceback (most recent call last):
>>>>   File "/usr/local/bin/twistd", line 14, in<module>
>>>>     run()
>>>>   File "/usr/local/lib/python2.7/site-packages/twisted/scripts/twistd.py", line 27, in run
>>>>     app.run(runApp, ServerOptions)
>>>>   File "/usr/local/lib/python2.7/site-packages/twisted/application/app.py", line 652, in run
>>>>     runApp(config)
>>>>   File "/usr/local/lib/python2.7/site-packages/twisted/scripts/twistd.py", line 23, in runApp
>>>>     _SomeApplicationRunner(config).run()
>>>>   File "/usr/local/lib/python2.7/site-packages/twisted/application/app.py", line 390, in run
>>>>     self.postApplication()
>>>>   File "/usr/local/lib/python2.7/site-packages/twisted/scripts/_twistd_unix.py", line 230, in postApplication
>>>>     self.startApplication(self.application)
>>>>   File "/usr/local/lib/python2.7/site-packages/twisted/scripts/_twistd_unix.py", line 340, in startApplication
>>>>     service.IService(application).privilegedStartService()
>>>>   File "/usr/local/lib/python2.7/site-packages/twisted/application/service.py", line 277, in privilegedStartService
>>>>     service.privilegedStartService()
>>>>   File "/usr/local/lib/python2.7/site-packages/twisted/application/service.py", line 277, in privilegedStartService
>>>>     service.privilegedStartService()
>>>>   File "/usr/local/lib/python2.7/site-packages/Calendar_and_Contacts_Server-4.2-py2.7-freebsd-9.1-RELEASE-amd64.egg/calendarserver/tap/caldav.py", line 487, in privilegedStartService
>>>>     super(GroupOwnedUNIXServer, self).privilegedStartService()
>>>>   File "/usr/local/lib/python2.7/site-packages/twisted/application/internet.py", line 105, in privilegedStartService
>>>>     self._port = self._getPort()
>>>>   File "/usr/local/lib/python2.7/site-packages/twisted/application/internet.py", line 133, in _getPort
>>>>     'listen%s' % (self.method,))(*self.args, **self.kwargs)
>>>>   File "/usr/local/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 406, in listenUNIX
>>>>     p.startListening()
>>>>   File "/usr/local/lib/python2.7/site-packages/twisted/internet/unix.py", line 303, in startListening
>>>>     self.startReading()
>>>>   File "/usr/local/lib/python2.7/site-packages/twisted/internet/abstract.py", line 416, in startReading
>>>>     self.reactor.addReader(self)
>>>>   File "/usr/local/lib/python2.7/site-packages/Calendar_and_Contacts_Server-4.2-py2.7-freebsd-9.1-RELEASE-amd64.egg/twext/internet/kqreactor.py", line 84, in addReader
>>>>     self._updateRegistration(fd, KQ_FILTER_READ, KQ_EV_ADD|KQ_EV_ENABLE)
>>>>   File "/usr/local/lib/python2.7/site-packages/Calendar_and_Contacts_Server-4.2-py2.7-freebsd-9.1-RELEASE-amd64.egg/twext/internet/kqreactor.py", line 74, in _updateRegistration
>>>>     self._kq.control([ev], 0, 0)
>>>> OSError: [Errno 9] Bad file descriptor
>>>> ---
>>>> Any help appreciated,
>>>> Axel
>>>> ---
>>>> PGP-Key:29E99DD6  ☀ +49 151 2300 9283  ☀ computing @ chaos claudius
>>>> 
>>>> _______________________________________________
>>>> calendarserver-dev mailing list
>>>> calendarserver-dev at lists.macosforge.org
>>>> https://lists.macosforge.org/mailman/listinfo/calendarserver-dev
>>> 
>>> _______________________________________________
>>> calendarserver-dev mailing list
>>> calendarserver-dev at lists.macosforge.org
>>> https://lists.macosforge.org/mailman/listinfo/calendarserver-dev
>> 
>> ---
>> PGP-Key:29E99DD6  ☀ +49 151 2300 9283  ☀ computing @ chaos claudius
>> 
>> _______________________________________________
>> calendarserver-dev mailing list
>> calendarserver-dev at lists.macosforge.org
>> https://lists.macosforge.org/mailman/listinfo/calendarserver-dev
> 
> <debug.patch>_______________________________________________
> calendarserver-dev mailing list
> calendarserver-dev at lists.macosforge.org
> https://lists.macosforge.org/mailman/listinfo/calendarserver-dev

---
PGP-Key:29E99DD6  ☀ +49 151 2300 9283  ☀ computing @ chaos claudius



More information about the calendarserver-dev mailing list