Hi, This discussion is more debian related, but it might help trunk as well. This is a list of steps and fixes I made for the packaging of trunk with regards to the base I have for 4.1.1 which in turn builds on the debian 3.2 package. First attempt to build trunk as a package failed in dependency on pycalendar [1]. (upgrading pycalendar to r214 took 3 attempts as it was not clear what branch to use) With the dependencies installed the server died with [2] twext.enterprise.dal.parseschema.ViolatedExpectation: Expected Token.Keyword got None: This was fixed with patch [3] (simple whitespace addition). Now the same error as before appeared. I also got some information from Peter Mogensen (off list) that detailed that he saw the same problem but only when the debian package was installed. (eg, a pure SVN checkout worked, but failed when the debian package was present) He reported it was totally dependent on the presens /usr/share/pyshared/twisted/plugins/caldav.py I did several tests, but at the end I "fixed it" by removing the RuntimeError exception. [4] The logging in the patch did not work, it can be left out. I probably do not use the log as it is meant to be used. I added a printout in the twext/backport/internet/tcp.py and it got printed, so I assume that the patching went fine. Lastly psutil in Debian is version 0.5.1 and version 0.6.0 is needed to have virtual_memory method. [5] I have not yet backported the change to 4.1.1 but do not expect problems, I just wanted to check if the change (removing the exception) is reasonable ? If ok, I will prepare the 4.1.1 debian package and put it online for testing. /Fred [1] File "/usr/lib/python2.7/dist-packages/twistedcaldav/__init__.py", line 70, in <module> PyCalendarProperty.registerDefaultValue("X-CALENDARSERVER-PRIVATE-COMMENT", PyCalendarValue.VALUETYPE_TEXT) AttributeError: type object 'PyCalendarProperty' has no attribute 'registerDefaultValue' [2] File "/usr/bin/twistd", line 14, in <module> run() File "/usr/lib/python2.7/dist-packages/twisted/scripts/twistd.py", line 27, in run app.run(runApp, ServerOptions) File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 647, in run config.parseOptions() File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 614, in parseOptions usage.Options.parseOptions(self, options) File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 261, in parseOptions for (cmd, short, parser, doc) in self.subCommands: File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 631, in subCommands for plug in sorted(plugins, key=attrgetter('tapname')): File "/usr/lib/python2.7/dist-packages/twisted/plugins/caldav.py", line 31, in getProperty return getattr(reflect.namedClass(self.serviceMakerClass), propname) File "/usr/lib/python2.7/dist-packages/twisted/python/reflect.py", line 351, in namedObject module = namedModule('.'.join(classSplit[:-1])) File "/usr/lib/python2.7/dist-packages/twisted/python/reflect.py", line 339, in namedModule topLevel = __import__(name) File "/usr/lib/python2.7/dist-packages/twistedcaldav/scheduling/imip/mailgateway.py", line 26, in <module> from calendarserver.tap.util import getRootResource, directoryFromConfig File "/usr/lib/python2.7/dist-packages/calendarserver/tap/util.py", line 51, in <module> from twistedcaldav.directory import calendaruserproxy File "/usr/lib/python2.7/dist-packages/twistedcaldav/directory/calendaruserproxy.py", line 46, in <module> from twistedcaldav.directory.principal import formatLink File "/usr/lib/python2.7/dist-packages/twistedcaldav/directory/principal.py", line 70, in <module> from twistedcaldav.resource import CalendarPrincipalCollectionResource, CalendarPrincipalResource File "/usr/lib/python2.7/dist-packages/twistedcaldav/resource.py", line 82, in <module> from twistedcaldav.sharing import SharedCollectionMixin, SharedHomeMixin File "/usr/lib/python2.7/dist-packages/twistedcaldav/sharing.py", line 33, in <module> from txdav.common.datastore.sql_tables import _BIND_MODE_OWN, \ File "/usr/lib/python2.7/dist-packages/txdav/common/datastore/sql_tables.py", line 48, in <module> schema = _populateSchema() File "/usr/lib/python2.7/dist-packages/txdav/common/datastore/sql_tables.py", line 44, in _populateSchema return SchemaSyntax(schemaFromPath(pathObj)) File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 92, in schemaFromPath addSQLToSchema(schema, schemaData) File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 121, in addSQLToSchema t = tableFromCreateStatement(schema, stmt) File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 76, in tableFromCreateStatement cp.parse() File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 228, in parse while self.nextColumn(): File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 242, in nextColumn return self.parseConstraint(maybeIdent) File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 314, in parseConstraint expect(self, ttype=Keyword, value='KEY') File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 516, in expect return expectSingle(nextval, **kw) File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 496, in expectSingle raise ViolatedExpectation(ttype, '%s:%r' % (nextval.ttype, nextval)) twext.enterprise.dal.parseschema.ViolatedExpectation: Expected Token.Keyword got None:<Function 'key(HO...' at 0x2340758> [3] Index: txdav/common/datastore/sql_schema/current.sql =================================================================== --- txdav/common/datastore/sql_schema/current.sql (revision 10029) +++ txdav/common/datastore/sql_schema/current.sql (working copy) @@ -35,7 +35,7 @@ PORT integer not null, TIME timestamp not null default timezone('UTC', CURRENT_TIMESTAMP), - primary key(HOSTNAME, PORT) + primary key (HOSTNAME, PORT) ); [4] --- a/twext/patches.py +++ b/twext/patches.py @@ -59,6 +59,9 @@ (i.e., all of C{twext/backport}) should be removed upon upgrading our minimum required Twisted version. """ + from twext.python.log import Logger + log = Logger() + from twext.backport import internet as bpinternet from twisted import internet internet.__path__[:] = bpinternet.__path__ + internet.__path__ @@ -69,9 +72,7 @@ subname = submod.name.split(".")[-1] tiname = 'twisted.internet.' + subname if tiname in sys.modules: - raise RuntimeError( - tiname + "already loaded, cannot load required backport") - + log.msg("%s reported in sys.modules : %s" % (tiname, sys.modules[tiname])) if not _hasIPv6ClientSupport(): [5] File "/usr/lib/python2.7/dist-packages/calendarserver/accesslog.py", line 594, in update mem = psutil.virtual_memory() exceptions.AttributeError: 'module' object has no attribute 'virtual_memory'
On Nov 14, 2012, at 3:04 AM, Fredrik Unger <fred@tree.se> wrote:
This discussion is more debian related, but it might help trunk as well.
Thanks for sharing this update.
This is a list of steps and fixes I made for the packaging of trunk with regards to the base I have for 4.1.1 which in turn builds on the debian 3.2 package.
First attempt to build trunk as a package failed in dependency on pycalendar [1]. (upgrading pycalendar to r214 took 3 attempts as it was not clear what branch to use)
This is "documented" by the call to py_dependency in support/build.sh. I realize that this is possibly not the best place for it. How would it be best to surface that information for you, so you would have noticed it? Where do you typically go looking for that sort of information?
With the dependencies installed the server died with [2] twext.enterprise.dal.parseschema.ViolatedExpectation: Expected Token.Keyword got None:
This is very likely because we depend on a very specific version of python-sqlparse: 0.1.2. Every subsequent version of python-sqlparse has broken something, so we upgrade that dependency very conservatively. What version were you working with?
This was fixed with patch [3] (simple whitespace addition).
Thanks. Can you attach this patch to a bug in Trac instead of a mailing list message? It would be much more likely to get applied that way.
Now the same error as before appeared. I also got some information from Peter Mogensen (off list) that detailed that he saw the same problem but only when the debian package was installed. (eg, a pure SVN checkout worked, but failed when the debian package was present)
He reported it was totally dependent on the presens /usr/share/pyshared/twisted/plugins/caldav.py
This is most likely because './run' is pulling all of its dependencies from its working directory, rather than the system. On your system, something - probably a Twisted plugin from some other package which is neither Twisted nor CalendarServer - is pulling in twisted.internet.address before this code can run. Can you run 'python -vvvv `which twistd` caldavd --help' and look for imports of the twisted.internet.address module?
I did several tests, but at the end I "fixed it" by removing the RuntimeError exception.
Hmm. Rather than removing the exception you should avoid doing the patches as well. This will break IPv6 support but otherwise things should work. You can restore that functionality by just upgrading to a current version of Twisted, anyway.
[4] The logging in the patch did not work, it can be left out. I probably do not use the log as it is meant to be used.
I added a printout in the twext/backport/internet/tcp.py and it got printed, so I assume that the patching went fine.
Lastly psutil in Debian is version 0.5.1 and version 0.6.0 is needed to have virtual_memory method. [5]
Once again, we have a lot of dependencies and we don't support a wide range of versions of them, especially when subsequent versions have features that we need :).
I have not yet backported the change to 4.1.1 but do not expect problems, I just wanted to check if the change (removing the exception) is reasonable ?
If ok, I will prepare the 4.1.1 debian package and put it online for testing.
I guess these changes are OK, but please put everything into Trac so we can eventually integrate your changes (or alternate fixes to those problems) so you do not have to carry a lot of patches in Debian. Thanks, -glyph
On Wed, 14 Nov 2012 13:52:06 -0800 Glyph <glyph@twistedmatrix.com> wrote:
This is "documented" by the call to py_dependency in support/build.sh.
Yes, I know, I forgot to look there. Not a big deal.
.. we depend on a very specific version of python-sqlparse: 0.1.2.
Ok, I understand.
Thanks. Can you attach this patch to a bug in Trac
Patch : https://trac.calendarserver.org/ticket/759 I might have used the wrong milestone and type for the bug, sorry.
This is most likely because './run' is pulling all of its dependencies from its working directory, rather than the system. On your system, something - probably a Twisted plugin from some other package which is neither Twisted nor CalendarServer - is pulling in twisted.internet.address before this code can run.
I think I tracked it down..partially When starting the *subprocesses* the --reactor flag is used. This starts the reactor select early and before the caldav plugin. It eventually imports twisted.internet.tcp Longer I did not trace. The initial start of the caldav process is ok. Why is the reactor select working in the original code ? Is it a different plugin order ? Both should be Twisted 12.0.0. Due to my limited knowledge of twisted and caldav I am stuck here. I am also not sure if it is a bug in any way, or the Debian environment. Below is a detailed log of my testing for reference. /Fred python -vvvv `which twistd` caldavd --help &> caldav.log grep -n 'import twext.patches' caldav.log 4332:import twext.patches # precompiled from /usr/lib/python2.7/dist-packages/twext/patches.pyc grep -n 'import twisted.internet.address' caldav.log 5681:import twisted.internet.address # precompiled from /usr/lib/python2.7/dist-packages/twext/backport/internet/address.pyc It works. BUT, what fails are the subprocesses. I dug into calendarserver/tap/caldav.py and added a simple patch [1], maybe for improved logging. The printout is forwarded by the subprocess logger to the error.log [2] The subprocesses all fail with the exception from patches.py, trying to patch twisted.internet.address. Using the commandline : /usr/bin/python2.7 -vvvv /usr/bin/twistd -u caldavd -g caldavd --reactor=select -n caldav -f /etc/caldavd/caldavd.plist -o ProcessType=Slave -o BindAddresses=,:: -o PIDFile=caldav-instance-0.pid -o ErrorLogFile=None -o ErrorLogEnabled=False -o LogID=0 -o MultiProcess/ProcessCount=2 -o ControlPort=0 -o MetaFD=19 &> failure.log To produce a new log showed the problem : grep -n 'import twisted.internet.address' failure.log 4588:import twisted.internet.address # precompiled from /usr/lib/python2.7/dist-packages/twisted/internet/address.pyc grep -n 'import twext.patches' failure.log 4808:import twext.patches # from /home/fred/dev/src/debian/calendarserver/twext/patches.py 4896:import twext.patches # precompiled from /home/fred/dev/src/debian/calendarserver/twext/patches.pyc 4965:import twext.patches # precompiled from /home/fred/dev/src/debian/calendarserver/twext/patches.pyc 5039:import twext.patches # precompiled from /home/fred/dev/src/debian/calendarserver/twext/patches.pyc 5185:import twext.patches # precompiled from /home/fred/dev/src/debian/calendarserver/twext/patches.pyc 5254:import twext.patches # precompiled from /home/fred/dev/src/debian/calendarserver/twext/patches.pyc 5328:import twext.patches # precompiled from /home/fred/dev/src/debian/calendarserver/twext/patches.pyc 5402:import twext.patches # precompiled from /home/fred/dev/src/debian/calendarserver/twext/patches.pyc The twisted.internet.address is imported earlier in this case. Why ? The only difference to the twisted command is --reactor=select. By removing that for the test : /usr/bin/python2.7 -vvvv /usr/bin/twistd -u caldavd -g caldavd -n caldav -f /etc/caldavd/caldavd.plist -o ProcessType=Slave -o BindAddresses=,:: -o PIDFile=caldav-instance-0.pid -o ErrorLogFile=None -o ErrorLogEnabled=False -o LogID=0 -o MultiProcess/ProcessCount=2 -o ControlPort=0 -o MetaFD=19 &> working.log we get a working patching method again : grep -n 'import twisted.internet.address' working.log 5573:import twisted.internet.address # from /home/fred/dev/src/debian/calendarserver/twext/backport/internet/address.py Now, from where does it come ? 557 : "--reactor=%s" % (config.Twisted.reactor,), in calendarserver/tap/caldav.py (for example). This is set in ./twistedcaldav/stdconfig.py I think 853 "Twisted": { 854 "reactor": "select", 855 }, comparing the two logs (sdiff) show that where select is not used, (also the case with the initial start of caldav under debian) the plugin import twisted.plugins.caldav gets loaded. With the reactor flag import twisted.plugins.twisted_reactors gets loaded BEFORE caldav which in turn loads : twisted.internet.selectreactor which from twisted.internet import posixbase and which from twisted.internet import tcp, udp I guess after tcp is loaded the patching will fail. I did not follow the trail to where address gets loaded. The selectreactor is part of : python-twisted-core: /usr/share/pyshared/twisted/internet/selectreactor.py Package: python-twisted-core Version: 12.0.0-1 [1] Index: calendarserver/tap/caldav.py =================================================================== --- calendarserver/tap/caldav.py (revision 10029) +++ calendarserver/tap/caldav.py (working copy) @@ -1808,7 +1808,7 @@ childFDs.update(procObj.getFileDescriptors()) args = procObj.getCommandLine() - + print p.name+' :>'+' '.join(args) self._reactor.spawnProcess( p, args[0], args, uid=uid, gid=gid, env=env, childFDs=childFDs [2] 2012-11-15 10:09:07+0100 [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Adding notification service 2012-11-15 10:09:07+0100 [-] [calendarserver.tap.caldav.CalDAVServiceMaker#info] Adding group caching service 2012-11-15 10:09:07+0100 [-] caldav-0 :>/usr/bin/python2.7 /usr/bin/twistd -u caldavd -g caldavd --reactor=select -n caldav -f /etc/caldavd/caldavd.plist -o ProcessType=Slave -o BindAddresses=,:: -o PIDFile=caldav-instance-0.pid -o ErrorLogFile=None -o ErrorLogEnabled=False -o LogID=0 -o MultiProcess/ProcessCount=2 -o ControlPort=0 -o MetaFD=19 2012-11-15 10:09:07+0100 [-] caldav-1 :>/usr/bin/python2.7 /usr/bin/twistd -u caldavd -g caldavd --reactor=select -n caldav -f /etc/caldavd/caldavd.plist -o ProcessType=Slave -o BindAddresses=,:: -o PIDFile=caldav-instance-1.pid -o ErrorLogFile=None -o ErrorLogEnabled=False -o LogID=1 -o MultiProcess/ProcessCount=2 -o ControlPort=0 -o MetaFD=21 2012-11-15 10:09:07+0100 [-] notifications :>/usr/bin/python2.7 /usr/bin/twistd -u caldavd -g caldavd --reactor=select -n caldav_notifier -f /etc/caldavd/caldavd.plist 2012-11-15 10:09:07+0100 [-] groupcacher :>/usr/bin/python2.7 /usr/bin/twistd -u caldavd -g caldavd --reactor=select -n caldav_groupcacher -f /etc/caldavd/caldavd.plist -o PIDFile=groupcacher.pid 2012-11-15 10:09:08+0100 [-] [caldav-0] Unhandled Error
participants (2)
-
Fredrik Unger
-
Glyph