[CalendarServer-dev] [debian] testing trunk

Wed Nov 14 03:04:01 PST 2012

Hi,

This discussion is more debian related, but it might help trunk as well. 

This is a list of steps and fixes I made for the packaging of trunk with regards
to the base I have for 4.1.1 which in turn builds on the debian 3.2 package.

First attempt to build trunk as a package failed in dependency on pycalendar [1].
(upgrading pycalendar to r214 took 3 attempts as it was not clear what branch to use)

With the dependencies installed the server died with [2]
twext.enterprise.dal.parseschema.ViolatedExpectation: Expected Token.Keyword got None:

This was fixed with patch [3] (simple whitespace addition).

Now the same error as before appeared.
I also got some information from Peter Mogensen (off list) that detailed that
he saw the same problem but only when the debian package was installed. 
(eg, a pure SVN checkout worked, but failed when the debian package was present)

He reported it was totally dependent on the presens
/usr/share/pyshared/twisted/plugins/caldav.py

I did several tests, but at the end I "fixed it" by removing the RuntimeError
exception. [4] The logging in the patch did not work, it can be left out. 
I probably do not use the log as it is meant to be used. 

I added a printout in the twext/backport/internet/tcp.py
and it got printed, so I assume that the patching went fine.

Lastly psutil in Debian is version 0.5.1 and version 0.6.0 is needed to
have virtual_memory method. [5]

I have not yet backported the change to 4.1.1 but do not expect problems, I just
wanted to check if the change (removing the exception) is reasonable ?

If ok, I will prepare the 4.1.1 debian package and put it online for testing.

/Fred


[1]
  File "/usr/lib/python2.7/dist-packages/twistedcaldav/__init__.py", line 70, in <module>
    PyCalendarProperty.registerDefaultValue("X-CALENDARSERVER-PRIVATE-COMMENT", PyCalendarValue.VALUETYPE_TEXT)
AttributeError: type object 'PyCalendarProperty' has no attribute 'registerDefaultValue'

[2]
  File "/usr/bin/twistd", line 14, in <module>
    run()
  File "/usr/lib/python2.7/dist-packages/twisted/scripts/twistd.py", line 27, in run
    app.run(runApp, ServerOptions)
  File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 647, in run
    config.parseOptions()
  File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 614, in parseOptions
    usage.Options.parseOptions(self, options)
  File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 261, in parseOptions
    for (cmd, short, parser, doc) in self.subCommands:
  File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 631, in subCommands
    for plug in sorted(plugins, key=attrgetter('tapname')):
  File "/usr/lib/python2.7/dist-packages/twisted/plugins/caldav.py", line 31, in getProperty
    return getattr(reflect.namedClass(self.serviceMakerClass), propname)
  File "/usr/lib/python2.7/dist-packages/twisted/python/reflect.py", line 351, in namedObject
    module = namedModule('.'.join(classSplit[:-1]))
  File "/usr/lib/python2.7/dist-packages/twisted/python/reflect.py", line 339, in namedModule
    topLevel = __import__(name)
  File "/usr/lib/python2.7/dist-packages/twistedcaldav/scheduling/imip/mailgateway.py", line 26, in <module>
    from calendarserver.tap.util import getRootResource, directoryFromConfig
  File "/usr/lib/python2.7/dist-packages/calendarserver/tap/util.py", line 51, in <module>
    from twistedcaldav.directory import calendaruserproxy
  File "/usr/lib/python2.7/dist-packages/twistedcaldav/directory/calendaruserproxy.py", line 46, in <module>
    from twistedcaldav.directory.principal import formatLink
  File "/usr/lib/python2.7/dist-packages/twistedcaldav/directory/principal.py", line 70, in <module>
    from twistedcaldav.resource import CalendarPrincipalCollectionResource, CalendarPrincipalResource
  File "/usr/lib/python2.7/dist-packages/twistedcaldav/resource.py", line 82, in <module>
    from twistedcaldav.sharing import SharedCollectionMixin, SharedHomeMixin
  File "/usr/lib/python2.7/dist-packages/twistedcaldav/sharing.py", line 33, in <module>
    from txdav.common.datastore.sql_tables import _BIND_MODE_OWN, \
  File "/usr/lib/python2.7/dist-packages/txdav/common/datastore/sql_tables.py", line 48, in <module>
    schema = _populateSchema()
  File "/usr/lib/python2.7/dist-packages/txdav/common/datastore/sql_tables.py", line 44, in _populateSchema
    return SchemaSyntax(schemaFromPath(pathObj))
  File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 92, in schemaFromPath
    addSQLToSchema(schema, schemaData)
  File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 121, in addSQLToSchema
    t = tableFromCreateStatement(schema, stmt)
  File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 76, in tableFromCreateStatement
    cp.parse()
  File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 228, in parse
    while self.nextColumn():
  File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 242, in nextColumn
    return self.parseConstraint(maybeIdent)
  File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 314, in parseConstraint
    expect(self, ttype=Keyword, value='KEY')
  File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 516, in expect
    return expectSingle(nextval, **kw)
  File "/usr/lib/python2.7/dist-packages/twext/enterprise/dal/parseschema.py", line 496, in expectSingle
    raise ViolatedExpectation(ttype, '%s:%r' % (nextval.ttype, nextval))
twext.enterprise.dal.parseschema.ViolatedExpectation: Expected Token.Keyword got None:<Function 'key(HO...' at 0x2340758>

[3]
Index: txdav/common/datastore/sql_schema/current.sql
===================================================================

--- txdav/common/datastore/sql_schema/current.sql       (revision 10029)
+++ txdav/common/datastore/sql_schema/current.sql       (working copy)
@@ -35,7 +35,7 @@
   PORT      integer not null,
   TIME      timestamp not null default timezone('UTC', CURRENT_TIMESTAMP),
 
-  primary key(HOSTNAME, PORT)
+  primary key (HOSTNAME, PORT)
 );

[4]
--- a/twext/patches.py
+++ b/twext/patches.py
@@ -59,6 +59,9 @@
     (i.e., all of C{twext/backport}) should be removed upon upgrading our
     minimum required Twisted version.
     """
+    from twext.python.log import Logger
+    log = Logger()
+
     from twext.backport import internet as bpinternet
     from twisted import internet
     internet.__path__[:] = bpinternet.__path__ + internet.__path__
@@ -69,9 +72,7 @@
         subname = submod.name.split(".")[-1]
         tiname = 'twisted.internet.' + subname
         if tiname in sys.modules:
-            raise RuntimeError(
-                tiname + "already loaded, cannot load required backport")
-
+            log.msg("%s reported in sys.modules : %s" % (tiname, sys.modules[tiname]))
 
 
 if not _hasIPv6ClientSupport():


[5]
          File "/usr/lib/python2.7/dist-packages/calendarserver/accesslog.py", line 594, in update
            mem = psutil.virtual_memory()
        exceptions.AttributeError: 'module' object has no attribute 'virtual_memory'
