[CalendarServer-dev] Migrating from 5.1 to 8.0 --was: Re: Circular reference in 8.0 (cannot import name GroupCacherPollingWork)

[Axel Rau]

> Am 08.06.2016 um 02:29 schrieb Andre LaBranche <dre at apple.com>:
> 
> 
>> On Jun 7, 2016, at 1:20 PM, Axel Rau <Axel.Rau at chaos1.de> wrote:
>> 
>> 
>>> Am 02.06.2016 um 19:42 schrieb Axel Rau <Axel.Rau at chaos1.de>:
>>> 
>>> Now I’m faced with:
>>> 
>>> [twext.enterprise.adbapi2#debug] ConnectionPool: txn busy 'jobqueue.workCheck': free=0, busy=1, waiting=0
>>> [twext.enterprise.adbapi2#debug] ConnectionPool: txn free 'jobqueue.workCheck': free=1, busy=0, waiting=0
>>> [directoryproxy] Usage: twistd [options]
>>> [directoryproxy] Options:
>> This has been resolved by upgrading twisted. )-;
> 
> Great! If you’re not already looking at our requirements.txt files and trying to follow their guidance, I would suggest doing that.
Sure. Sometimes you depend on other packages on your platform (-;
> If you resolved this by upgrading to a version newer than what is stated in our requirements files, that would be good to know also.
I upgraded from 15.2.1 to 15.5.0
> 
>> After setting uid to guid, I could connect to my test server.
>> This should be stressed in the README!
> 
> Noted, thanks.
> 
>> After putting the users e-mail address into short-name, everyone can configure his account by e-mail and pw. (-:
> 
> Yay!
> 
>> What drives me mad is this:
>> - - -
>> [caldav3:local/etc/caldavd] root# python /usr/local/lib/python2.7/site-packages/calendarserver/tools/principals.py --help
>> Traceback (most recent call last):
>> File "/usr/local/lib/python2.7/site-packages/calendarserver/tools/principals.py", line 33, in <module>
>>   from calendarserver.tools.cmdline import utilityMain, WorkerService
>> File "/usr/local/lib/python2.7/site-packages/calendarserver/tools/cmdline.py", line 21, in <module>
>>   from calendarserver.tap.util import checkDirectories, getRootResource
>> File "/usr/local/lib/python2.7/site-packages/calendarserver/tap/util.py", line 38, in <module>
>>   from calendarserver.tools.util import checkDirectory
>> File "/usr/local/lib/python2.7/site-packages/calendarserver/tools/util.py", line 38, in <module>
>>   from twistedcaldav.stdconfig import DEFAULT_CONFIG_FILE
>> File "/usr/local/lib/python2.7/site-packages/twistedcaldav/stdconfig.py", line 1831, in <module>
>>   config.update() 
>> File "/usr/local/lib/python2.7/site-packages/twistedcaldav/config.py", line 262, in update
>>   hook(self._data, reloading=reloading)
>> File "/usr/local/lib/python2.7/site-packages/twistedcaldav/stdconfig.py", line 1457, in _postUpdateResourceService
>>   from twistedcaldav.upgrade import upgradeResourcesXML
>> File "/usr/local/lib/python2.7/site-packages/twistedcaldav/upgrade.py", line 67, in <module>
>>   from calendarserver.tap.util import getRootResource, FakeRequest
>> ImportError: cannot import name getRootResource
>> - - -
>> All tools have this problem!
>> What can I do to help resolving this?
> 
> Do you have anything at /var/db/caldavd? If so, what is there? The default path for ServerRoot is /var/db/caldavd. I think (with help from Cyrus :) what's happening here is that the CLI argument processing ends up reaching into our config object at a time when it hasn't been initialized beyond the defaults values found in twistedcaldavd/stdconfig.py. This is likely a bug in the server, which seems to be brought to the fore by the combination of having stuff at /var/db/caldavd and also using a config file that defines a different ServerRoot.

# ls -l /var/db/caldavd/
total 32
drwxr-xr-x  2 root     wheel    512 Jun  2 10:32 .postgresql
drwxr-x---  2 root     caldavd  512 May 21 11:51 Config
drwxr-xr--  4 caldavd  caldavd  512 Jun  4 11:57 Data
drwxr-xr-x  2 caldavd  caldavd  512 Apr  2  2015 Documents
-rw-r--r--  1 root     caldavd    5 Jun  7 19:21 caldav-instance-0.pid
-rw-r--r--  1 root     caldavd    5 Jun  7 19:21 caldav-instance-1.pid
-rw-r--r--  1 root     caldavd    5 Jun  7 19:21 caldavd.pid
srw-rw----  1 root     caldavd    0 Jun  7 19:21 caldavd.sock
drwxrwxr--  2 root     caldavd  512 Jun  6 13:46 data
srw-rw----  1 root     caldavd    0 Jun  7 19:21 directory-proxy.sock
lrwxr-xr-x  1 root     caldavd    5 Jun  7 19:21 directory-proxy.sock.lock -> 30855
srwx------  1 caldavd  caldavd    0 Jun  7 19:21 memcache.sock
# ls -l /var/db/caldavd/Data/
total 24
drwxr-xr-x   2 caldavd  caldavd   512 Apr  2  2015 Documents
-rw-r--r--   1 root     caldavd   512 Mar 19 10:26 mailgatewaytokens.sqlite-journal
-rwxr-xr-x   1 caldavd  caldavd     6 Apr  7 08:58 memberships_cache
-rw-r--r--   1 root     caldavd     0 Jun  4 11:57 proxies-loaded
-rwxr-xr-x   1 root     caldavd    29 May 24 11:03 resources.xml
drwxr-xr-x  18 root     caldavd  5120 Jun  2 17:06 zoneinfo
# ls -l /var/db/caldavd/data/
total 0

From caldavd.plist:

    <key>ServerRoot</key>
    <string>/var/db/caldavd</string>

How can I instrument the config parsing process to find out what’s going on?

Axel
---
PGP-Key:29E99DD6  ☀  computing @ chaos claudius