[CalendarServer-changes] [6370] CalendarServer/branches/users/glyph/more-deferreds-7
source_changes at macosforge.org
source_changes at macosforge.org
Thu Sep 23 23:10:58 PDT 2010
Revision: 6370
http://trac.macosforge.org/projects/calendarserver/changeset/6370
Author: glyph at apple.com
Date: 2010-09-23 23:10:56 -0700 (Thu, 23 Sep 2010)
Log Message:
-----------
rebase on trunk
Modified Paths:
--------------
CalendarServer/branches/users/glyph/more-deferreds-7/calendarserver/tools/purge.py
CalendarServer/branches/users/glyph/more-deferreds-7/support/build.sh
CalendarServer/branches/users/glyph/more-deferreds-7/twext/web2/server.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/addressbook.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/calendar.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/opendirectorybacker.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/principal.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/test/test_calendar.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/extensions.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/icaldav.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/put_addressbook_common.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/put_common.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/report_calendar_query.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/report_common.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/resource.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/scheduling/implicit.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/scheduling/processing.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/scheduling/utils.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/sharing.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/storebridge.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_addressbookmultiget.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_addressbookquery.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_calendarquery.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_extensions.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_sharing.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_sql.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_wrapping.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/util.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/test/base.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/test/test_sql.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/file.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/scheduling.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/sql.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/common.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_file.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_sql.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/util.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/icalendarstore.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/file.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/common.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_file.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_sql.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/iaddressbookstore.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_legacy.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/test/test_util.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/test/util.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/util.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/icommondatastore.py
Added Paths:
-----------
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/common.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/index_file.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_index_file.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/index_file.py
CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_index_file.py
Removed Paths:
-------------
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/index.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_index.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_vcardindex.py
CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/vcardindex.py
Property Changed:
----------------
CalendarServer/branches/users/glyph/more-deferreds-7/
Property changes on: CalendarServer/branches/users/glyph/more-deferreds-7
___________________________________________________________________
Modified: svn:mergeinfo
- /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/generic-sqlstore:6167-6191
/CalendarServer/branches/new-store:5594-5934
/CalendarServer/branches/new-store-no-caldavfile:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/more-deferreds-6:6330
/CalendarServer/branches/users/glyph/sendfdport:5388-5424
/CalendarServer/branches/users/glyph/sql-store:5929-6073
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/branches/users/wsanchez/transations:5515-5593
+ /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/generic-sqlstore:6167-6191
/CalendarServer/branches/new-store:5594-5934
/CalendarServer/branches/new-store-no-caldavfile:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
/CalendarServer/branches/users/glyph/sendfdport:5388-5424
/CalendarServer/branches/users/glyph/sql-store:5929-6073
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/branches/users/wsanchez/transations:5515-5593
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/calendarserver/tools/purge.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/calendarserver/tools/purge.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/calendarserver/tools/purge.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -270,7 +270,7 @@
# Get the calendar home
principalCollection = directory.principalCollection
principal = principalCollection.principalForRecord(record)
- calendarHome = principal.calendarHome()
+ calendarHome = yield principal.calendarHome()
if verbose:
print "%s %-15s :" % (record.uid, record.shortNames[0]),
@@ -367,7 +367,7 @@
principalCollection = directory.principalCollection
principal = principalCollection.principalForRecord(record)
- calendarHome = principal.calendarHome()
+ calendarHome = yield principal.calendarHome()
# Anything in the past is left alone
now = datetime.utcnow().strftime("%Y%m%dT%H%M%SZ")
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/support/build.sh
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/support/build.sh 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/support/build.sh 2010-09-24 06:10:56 UTC (rev 6370)
@@ -501,6 +501,10 @@
# Python dependencies
#
+ # First, let's make sure that we ourselves are on PYTHONPATH, in case some
+ # code (like, let's say, trial) decides to chdir somewhere.
+ export PYTHONPATH="${wd}:${PYTHONPATH:-}";
+
local zi="zope.interface-3.3.0";
py_dependency \
"Zope Interface" "zope.interface" "${zi}" \
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twext/web2/server.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twext/web2/server.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twext/web2/server.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -346,6 +346,7 @@
d.addCallback(self._cbFinishRender)
d.addErrback(self._processingFailed)
d.callback(None)
+ return d
def preprocessRequest(self):
"""Do any request processing that doesn't follow the normal
@@ -470,10 +471,10 @@
@raise NoURLForResourceError: if C{resource} has no URL in this request
(because it was not obtained from the request).
"""
- resource = self._urlsByResource.get(resource, None)
- if resource is None:
+ url = self._urlsByResource.get(resource, None)
+ if url is None:
raise NoURLForResourceError(resource)
- return resource
+ return url
def locateResource(self, url):
"""
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/addressbook.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/addressbook.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/addressbook.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -19,7 +19,6 @@
"""
__all__ = [
- "uidsResourceName",
"DirectoryAddressBookHomeProvisioningResource",
"DirectoryAddressBookHomeTypeProvisioningResource",
"DirectoryAddressBookHomeUIDProvisioningResource",
@@ -32,10 +31,14 @@
from twext.web2.http import HTTPError
from twext.web2.http_headers import ETag, MimeType
+from twisted.internet.defer import inlineCallbacks, returnValue
+
from twistedcaldav.config import config
from twistedcaldav.directory.idirectory import IDirectoryService
-from twistedcaldav.directory.resource import DirectoryReverseProxyResource
-from twistedcaldav.directory.util import transactionFromRequest
+
+from twistedcaldav.directory.common import CommonUIDProvisioningResource,\
+ uidsResourceName, CommonHomeTypeProvisioningResource
+
from twistedcaldav.extensions import ReadOnlyResourceMixIn, DAVResource,\
DAVResourceWithChildrenMixin
from twistedcaldav.resource import AddressBookHomeResource
@@ -44,8 +47,6 @@
log = Logger()
-# Use __underbars__ convention to avoid conflicts with directory resource types.
-uidsResourceName = "__uids__"
# FIXME: copied from resource.py to avoid circular dependency
class CalDAVComplianceMixIn(object):
@@ -71,7 +72,9 @@
return MimeType("httpd", "unix-directory")
-class DirectoryAddressBookHomeProvisioningResource (DirectoryAddressBookProvisioningResource):
+class DirectoryAddressBookHomeProvisioningResource (
+ DirectoryAddressBookProvisioningResource
+ ):
"""
Resource which provisions address book home collections as needed.
"""
@@ -134,7 +137,10 @@
return "addressbooks"
-class DirectoryAddressBookHomeTypeProvisioningResource (DirectoryAddressBookProvisioningResource):
+class DirectoryAddressBookHomeTypeProvisioningResource (
+ CommonHomeTypeProvisioningResource,
+ DirectoryAddressBookProvisioningResource
+ ):
"""
Resource which provisions address book home collections of a specific
record type as needed.
@@ -156,18 +162,7 @@
def url(self):
return joinURL(self._parent.url(), self.recordType)
- def locateChild(self, request, segments):
- name = segments[0]
- if name == "":
- return (self, segments[1:])
- record = self.directory.recordWithShortName(self.recordType, name)
- if record is None:
- return None, []
-
- return (self._parent.homeForDirectoryRecord(record, request),
- segments[1:])
-
def listChildren(self):
if config.EnablePrincipalListings:
@@ -206,99 +201,33 @@
return self._parent.principalForRecord(record)
-class DirectoryAddressBookHomeUIDProvisioningResource (DirectoryAddressBookProvisioningResource):
+class DirectoryAddressBookHomeUIDProvisioningResource (
+ CommonUIDProvisioningResource,
+ DirectoryAddressBookProvisioningResource
+ ):
- def __init__(self, parent):
- """
- @param parent: the parent of this resource
- """
- assert parent is not None
+ homeResourceTypeName = 'addressbooks'
- super(DirectoryAddressBookHomeUIDProvisioningResource, self).__init__()
+ enabledAttribute = 'enabledForAddressBooks'
- self.directory = parent.directory
- self.parent = parent
+ def homeResourceCreator(self, record, transaction):
+ return DirectoryAddressBookHomeResource.createHomeResource(
+ self, record, transaction)
- def url(self):
- return joinURL(self.parent.url(), uidsResourceName)
- def locateChild(self, request, segments):
-
- name = segments[0]
- if name == "":
- return (self, ())
-
- record = self.directory.recordWithUID(name)
- if record:
- return (self.homeResourceForRecord(record, request), segments[1:])
- else:
- return (None, ())
-
- def getChild(self, name, record=None):
- raise NotImplementedError("DirectoryAddressBookHomeUIDProvisioningResource.getChild no longer exists.")
-
- def listChildren(self):
- # Not a listable collection
- raise HTTPError(responsecode.FORBIDDEN)
-
- def homeResourceForRecord(self, record, request):
-
- transaction = transactionFromRequest(request, self.parent._newStore)
-
- name = record.uid
-
- if record is None:
- self.log_msg("No directory record with GUID %r" % (name,))
- return None
-
- if not record.enabledForAddressBooks:
- self.log_msg("Directory record %r is not enabled for address books" % (record,))
- return None
-
- assert len(name) > 4, "Directory record has an invalid GUID: %r" % (name,)
-
- if record.locallyHosted():
- child = DirectoryAddressBookHomeResource(self, record, transaction)
- else:
- child = DirectoryReverseProxyResource(self, record)
-
- return child
-
- ##
- # DAV
- ##
-
- def isCollection(self):
- return True
-
- def displayName(self):
- return uidsResourceName
-
- ##
- # ACL
- ##
-
- def principalCollections(self):
- return self.parent.principalCollections()
-
- def principalForRecord(self, record):
- return self.parent.principalForRecord(record)
-
-
class DirectoryAddressBookHomeResource (AddressBookHomeResource):
"""
Address book home collection resource.
"""
- def __init__(self, parent, record, transaction):
- """
- @param path: the path to the file which will back the resource.
- """
- assert parent is not None
- assert record is not None
- assert transaction is not None
+ @classmethod
+ @inlineCallbacks
+ def createHomeResource(cls, parent, record, transaction):
+ self = yield super(DirectoryAddressBookHomeResource, cls).createHomeResource(
+ parent, record.uid, transaction)
self.record = record
- super(DirectoryAddressBookHomeResource, self).__init__(parent, record.uid, transaction)
+ returnValue(self)
+
def principalForRecord(self):
return self.parent.principalForRecord(self.record)
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/calendar.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/calendar.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/calendar.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -20,7 +20,6 @@
"""
__all__ = [
- "uidsResourceName",
"DirectoryCalendarHomeProvisioningResource",
"DirectoryCalendarHomeTypeProvisioningResource",
"DirectoryCalendarHomeUIDProvisioningResource",
@@ -33,12 +32,13 @@
from twext.web2.http import HTTPError
from twext.web2.http_headers import ETag, MimeType
-from twisted.internet.defer import succeed
+from twisted.internet.defer import succeed, inlineCallbacks, returnValue
from twistedcaldav.config import config
from twistedcaldav.directory.idirectory import IDirectoryService
-from twistedcaldav.directory.resource import DirectoryReverseProxyResource
-from twistedcaldav.directory.util import transactionFromRequest
+from twistedcaldav.directory.common import uidsResourceName,\
+ CommonUIDProvisioningResource, CommonHomeTypeProvisioningResource
+
from twistedcaldav.directory.wiki import getWikiACL
from twistedcaldav.extensions import ReadOnlyResourceMixIn, DAVResource,\
DAVResourceWithChildrenMixin
@@ -48,9 +48,6 @@
log = Logger()
-# Use __underbars__ convention to avoid conflicts with directory resource types.
-uidsResourceName = "__uids__"
-
# FIXME: copied from resource.py to avoid circular dependency
class CalDAVComplianceMixIn(object):
def davComplianceClasses(self):
@@ -136,7 +133,10 @@
def displayName(self):
return "calendars"
-class DirectoryCalendarHomeTypeProvisioningResource (DirectoryCalendarProvisioningResource):
+class DirectoryCalendarHomeTypeProvisioningResource(
+ CommonHomeTypeProvisioningResource,
+ DirectoryCalendarProvisioningResource
+ ):
"""
Resource which provisions calendar home collections of a specific
record type as needed.
@@ -158,18 +158,7 @@
def url(self):
return joinURL(self._parent.url(), self.recordType)
- def locateChild(self, request, segments):
- name = segments[0]
- if name == "":
- return (self, segments[1:])
- record = self.directory.recordWithShortName(self.recordType, name)
- if record is None:
- return None, []
-
- return (self._parent.homeForDirectoryRecord(record, request),
- segments[1:])
-
def listChildren(self):
if config.EnablePrincipalListings:
@@ -207,99 +196,35 @@
def principalForRecord(self, record):
return self._parent.principalForRecord(record)
-class DirectoryCalendarHomeUIDProvisioningResource (DirectoryCalendarProvisioningResource):
+class DirectoryCalendarHomeUIDProvisioningResource (
+ CommonUIDProvisioningResource,
+ DirectoryCalendarProvisioningResource
+ ):
- def __init__(self, parent):
- """
- @param parent: the parent of this resource
- """
- assert parent is not None
+ homeResourceTypeName = 'calendars'
- super(DirectoryCalendarHomeUIDProvisioningResource, self).__init__()
+ enabledAttribute = 'enabledForCalendaring'
- self.directory = parent.directory
- self.parent = parent
+ def homeResourceCreator(self, record, transaction):
+ return DirectoryCalendarHomeResource.createHomeResource(
+ self, record, transaction)
- def url(self):
- return joinURL(self.parent.url(), uidsResourceName)
- def locateChild(self, request, segments):
- name = segments[0]
- if name == "":
- return (self, ())
-
- record = self.directory.recordWithUID(name)
- if record:
- return (self.homeResourceForRecord(record, request), segments[1:])
- else:
- return (None, ())
-
- def getChild(self, name, record=None):
- raise NotImplementedError("DirectoryCalendarProvisioningResource.getChild no longer exists.")
-
- def listChildren(self):
- # Not a listable collection
- raise HTTPError(responsecode.FORBIDDEN)
-
- def homeResourceForRecord(self, record, request):
-
- transaction = transactionFromRequest(request, self.parent._newStore)
- name = record.uid
-
- if record is None:
- log.debug("No directory record with GUID %r" % (name,))
- return None
-
- if not record.enabledForCalendaring:
- log.debug("Directory record %r is not enabled for calendaring" % (record,))
- return None
-
- assert len(name) > 4, "Directory record has an invalid GUID: %r" % (name,)
-
- if record.locallyHosted():
- child = DirectoryCalendarHomeResource(self, record, transaction)
- else:
- child = DirectoryReverseProxyResource(self, record)
-
- return child
-
- ##
- # DAV
- ##
-
- def isCollection(self):
- return True
-
- def displayName(self):
- return uidsResourceName
-
- ##
- # ACL
- ##
-
- def principalCollections(self):
- return self.parent.principalCollections()
-
- def principalForRecord(self, record):
- return self.parent.principalForRecord(record)
-
-
class DirectoryCalendarHomeResource (CalendarHomeResource):
"""
Calendar home collection resource.
"""
- def __init__(self, parent, record, transaction):
- """
- @param path: the path to the file which will back the resource.
- """
- assert parent is not None
- assert record is not None
- assert transaction is not None
+ @classmethod
+ @inlineCallbacks
+ def createHomeResource(cls, parent, record, transaction):
+ self = yield super(DirectoryCalendarHomeResource, cls).createHomeResource(
+ parent, record.uid, transaction)
self.record = record
- super(DirectoryCalendarHomeResource, self).__init__(parent, record.uid, transaction)
+ returnValue(self)
+
# Special ACLs for Wiki service
def accessControlList(self, request, inheritance=True, expanding=False, inherited_aces=None):
def gotACL(wikiACL):
Copied: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/common.py (from rev 6368, CalendarServer/branches/users/glyph/more-deferreds-6/twistedcaldav/directory/common.py)
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/common.py (rev 0)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/common.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -0,0 +1,147 @@
+# -*- test-case-name: twistedcaldav.test.test_wrapping,twistedcaldav.directory.test.test_calendar -*-
+##
+# Copyright (c) 2010 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twisted.internet.defer import inlineCallbacks, returnValue
+from twext.web2.http import HTTPError
+from twext.web2 import responsecode
+from twext.web2.dav.util import joinURL
+from twistedcaldav.directory.util import transactionFromRequest
+from twistedcaldav.directory.resource import DirectoryReverseProxyResource
+
+__all__ = [
+ 'uidsResourceName',
+ 'CommonUIDProvisioningResource'
+]
+
+# Use __underbars__ convention to avoid conflicts with directory resource
+# types.
+
+uidsResourceName = "__uids__"
+
+class CommonUIDProvisioningResource(object):
+ """
+ Common ancestor for addressbook/calendar UID provisioning resources.
+
+ Must be mixed in to the hierarchy I{before} the appropriate resource type.
+
+ @ivar homeResourceTypeName: The name of the home resource type ('calendars'
+ or 'addressbooks').
+
+ @ivar enabledAttribute: The name of the attribute of the directory record
+ which determines whether this should be enabled or not.
+ """
+
+ def __init__(self, parent):
+ """
+ @param parent: the parent of this resource
+ """
+
+ super(CommonUIDProvisioningResource, self).__init__()
+
+ self.directory = parent.directory
+ self.parent = parent
+
+
+ @inlineCallbacks
+ def homeResourceForRecord(self, record, request):
+
+ transaction = transactionFromRequest(request, self.parent._newStore)
+ name = record.uid
+
+ if record is None:
+ self.log_msg("No directory record with GUID %r" % (name,))
+ returnValue(None)
+
+ if not getattr(record, self.enabledAttribute):
+ self.log_msg("Directory record %r is not enabled for %s" % (
+ record, self.homeResourceTypeName))
+ returnValue(None)
+
+ assert len(name) > 4, "Directory record has an invalid GUID: %r" % (
+ name,)
+
+ if record.locallyHosted():
+ child = yield self.homeResourceCreator(record, transaction)
+ else:
+ child = DirectoryReverseProxyResource(self, record)
+
+ returnValue(child)
+
+
+ @inlineCallbacks
+ def locateChild(self, request, segments):
+
+ name = segments[0]
+ if name == "":
+ returnValue((self, ()))
+
+ record = self.directory.recordWithUID(name)
+ if record:
+ child = yield self.homeResourceForRecord(record, request)
+ returnValue((child, segments[1:]))
+ else:
+ returnValue((None, ()))
+
+
+ def listChildren(self):
+ # Not a listable collection
+ raise HTTPError(responsecode.FORBIDDEN)
+
+ ##
+ # ACL
+ ##
+
+ def principalCollections(self):
+ return self.parent.principalCollections()
+
+ def principalForRecord(self, record):
+ return self.parent.principalForRecord(record)
+ ##
+ # DAV
+ ##
+
+ def isCollection(self):
+ return True
+
+
+ def getChild(self, name, record=None):
+ raise NotImplementedError(self.__class__.__name__ +
+ ".getChild no longer exists.")
+
+ def displayName(self):
+ return uidsResourceName
+
+ def url(self):
+ return joinURL(self.parent.url(), uidsResourceName)
+
+
+
+class CommonHomeTypeProvisioningResource(object):
+
+ @inlineCallbacks
+ def locateChild(self, request, segments):
+ name = segments[0]
+ if name == "":
+ returnValue((self, segments[1:]))
+
+ record = self.directory.recordWithShortName(self.recordType, name)
+ if record is None:
+ returnValue((None, []))
+
+ child = yield self._parent.homeForDirectoryRecord(record, request)
+ returnValue((child, segments[1:]))
+
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/opendirectorybacker.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/opendirectorybacker.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/opendirectorybacker.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -397,66 +397,7 @@
yield updateLock.release()
updateLock = None
- #FIXME: implement store based cache
-# tmpDirLock = self._tmpDirAddressBookLock
-# self.log_debug("blocking on lock of: \"%s\")" % self._tmpDirAddressBookLockPath)
-# yield tmpDirLock.acquire()
-#
-# try:
-# self.log_info("Filling directory address book")
-# startTime = time.time()
-# newAddressBook = CalDAVFile(makeTmpFilename())
-# yield newAddressBook.createAddressBookCollection()
-# for key, record in records.items():
-# try:
-# vcard = record.vCard()
-# # make up a destination
-#
-# fileName = unquote(record.uriName())
-# destination = CalDAVFile(join(newAddressBook.fp.path, fileName))
-# destination_uri = record.hRef()
-#
-# self.log_debug("Adding \"%s\", uri=\"%s\"" % (fileName, destination_uri, ))
-# self.log_debug("VCard text =\n%s" % (record.vCardText(), ))
-#
-# yield StoreAddressObjectResource( request = None,
-# sourceadbk = False,
-# destinationadbk = True,
-# destination = destination,
-# destination_uri = destination_uri,
-# destinationparent = newAddressBook,
-# vcard = vcard,
-# indexdestination = False,
-# ).run()
-# except:
-# self.log_info("Could not add record %s" % (record,))
-# del records[key]
-# newAddressBookCTag = customxml.GETCTag(str(hash(self.baseGUID + ":" + self.realmName + ":" + "".join(str(hash(records[key])) for key in records.keys()))))
-#
-# self.log_info("Indexing new directory address book")
-# newAddressBook.index().recreate()
-# elaspedTime = time.time()-startTime
-# self.log_info("Timing: Fill address book: %.1f ms (%d vcards, %.2f vcards/sec)" % (elaspedTime*1000, len(records), len(records)/elaspedTime))
-#
-# updateLock = self.updateLock()
-# self.log_debug("blocking on lock of: \"%s\")" % self._updateLockPath)
-# yield updateLock.acquire()
-#
-# self.log_debug("Swapping in new directory address book")
-#
-# # move old address book out of the way
-# if self.directoryBackedAddressBook.fp.exists():
-# os.rename(self.directoryBackedAddressBook.fp.path, makeTmpFilename())
-#
-# #move new one into place
-# os.rename(newAddressBook.fp.path, self.directoryBackedAddressBook.fp.path)
-# self.directoryBackedAddressBook.fp.restat()
-#
-# self.directoryBackedAddressBook.writeDeadProperty(newAddressBookCTag)
-# finally:
-# self.log_debug("unlocking: \"%s\")" % self._tmpDirAddressBookLockPath)
-# yield tmpDirLock.release()
-
+
if not keepLock:
self.log_debug("unlocking: \"%s\")" % self._updateLockPath)
yield updateLock.release()
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/principal.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/principal.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/principal.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -55,6 +55,7 @@
from twistedcaldav.directory import calendaruserproxy
from twistedcaldav.directory import augment
from twistedcaldav.directory.calendaruserproxy import CalendarUserProxyPrincipalResource
+from twistedcaldav.directory.common import uidsResourceName
from twistedcaldav.directory.directory import DirectoryService, DirectoryRecord
from twistedcaldav.extensions import ReadOnlyResourceMixIn, DAVPrincipalResource,\
DAVResourceWithChildrenMixin
@@ -67,9 +68,6 @@
log = Logger()
-# Use __underbars__ convention to avoid conflicts with directory resource types.
-uidsResourceName = "__uids__"
-
class PermissionsMixIn (ReadOnlyResourceMixIn):
def defaultAccessControlList(self):
return authReadACL
@@ -867,26 +865,27 @@
else:
return False
+ @inlineCallbacks
def scheduleInbox(self, request):
- home = self.calendarHome(request)
+ home = yield self.calendarHome(request)
if home is None:
- return succeed(None)
+ returnValue(None)
- inbox = home.getChild("inbox")
+ inbox = yield home.getChild("inbox")
if inbox is None:
- return succeed(None)
+ returnValue(None)
- return succeed(inbox)
+ returnValue(inbox)
+ @inlineCallbacks
def notificationCollection(self, request):
-
+
notification = None
if config.Sharing.Enabled:
- home = self.calendarHome(request)
+ home = yield self.calendarHome(request)
if home is not None:
- notification = home.getChild("notification")
-
- return succeed(notification)
+ notification = yield home.getChild("notification")
+ returnValue(notification)
def calendarHomeURLs(self):
homeURL = self._homeChildURL(None)
@@ -929,14 +928,16 @@
else:
return joinURL(url, name) if name else url
+
def calendarHome(self, request):
# FIXME: self.record.service.calendarHomesCollection smells like a hack
service = self.record.service
if hasattr(service, "calendarHomesCollection"):
return service.calendarHomesCollection.homeForDirectoryRecord(self.record, request)
else:
- return None
+ return succeed(None)
+
def _addressBookHomeChildURL(self, name):
if not hasattr(self, "addressBookHomeURL"):
if not hasattr(self.record.service, "addressBookHomesCollection"):
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/test/test_calendar.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/test/test_calendar.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/test/test_calendar.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -16,30 +16,51 @@
from twisted.internet.defer import inlineCallbacks
from twext.web2.dav import davxml
-from twext.web2.test.test_server import SimpleRequest
from twistedcaldav import caldavxml
from twistedcaldav.test.util import TestCase
+from twext.web2.test.test_server import SimpleRequest
+from twistedcaldav.directory.util import transactionFromRequest
class ProvisionedCalendars (TestCase):
"""
Directory service provisioned principals.
"""
+
+ @inlineCallbacks
def setUp(self):
- super(ProvisionedCalendars, self).setUp()
+ yield super(ProvisionedCalendars, self).setUp()
self.createStockDirectoryService()
self.setupCalendars()
+ def oneRequest(self, uri):
+ req = self._cleanupRequest = SimpleRequest(self.site, "GET", uri)
+ return req
+
+
+ def tearDown(self):
+ """
+ If the request started by this test has a transaction, commit it.
+ Otherwise, don't bother.
+ """
+ class JustForCleanup(object):
+ def newTransaction(self, *whatever):
+ return self
+ def commit(self):
+ return
+ return transactionFromRequest(self._cleanupRequest, JustForCleanup()).commit()
+
+
def test_NonExistentCalendarHome(self):
def _response(resource):
if resource is not None:
self.fail("Incorrect response to GET on non-existent calendar home.")
- request = SimpleRequest(self.site, "GET", "/calendars/users/12345/")
+ request = self.oneRequest("/calendars/users/12345/")
d = request.locateResource(request.uri)
d.addCallback(_response)
@@ -49,7 +70,7 @@
if resource is None:
self.fail("Incorrect response to GET on existent calendar home.")
- request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/")
+ request = self.oneRequest("/calendars/users/wsanchez/")
d = request.locateResource(request.uri)
d.addCallback(_response)
@@ -59,7 +80,7 @@
if resource is None:
self.fail("Incorrect response to GET on existent calendar.")
- request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/calendar/")
+ request = self.oneRequest("/calendars/users/wsanchez/calendar/")
d = request.locateResource(request.uri)
d.addCallback(_response)
@@ -69,14 +90,14 @@
if resource is None:
self.fail("Incorrect response to GET on existent inbox.")
- request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/inbox/")
+ request = self.oneRequest("/calendars/users/wsanchez/inbox/")
d = request.locateResource(request.uri)
d.addCallback(_response)
@inlineCallbacks
def test_CalendarTranspProperty(self):
- request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/calendar/")
+ request = self.oneRequest("/calendars/users/wsanchez/calendar/")
# Get calendar first
calendar = (yield request.locateResource("/calendars/users/wsanchez/calendar/"))
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/extensions.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/extensions.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/extensions.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -38,7 +38,7 @@
from twisted.cred.error import LoginFailed, UnauthorizedLogin
import twext.web2.server
-from twext.web2 import responsecode, iweb, http, server
+from twext.web2 import responsecode, server
from twext.web2.auth.wrapper import UnauthorizedResponse
from twext.web2.http import HTTPError, Response, RedirectResponse
from twext.web2.http import StatusResponse
@@ -734,6 +734,7 @@
self.putChildren = {}
super(DAVResourceWithChildrenMixin, self).__init__(principalCollections=principalCollections)
+
def putChild(self, name, child):
"""
Register a child with the given name with this resource.
@@ -742,9 +743,12 @@
"""
self.putChildren[name] = child
+
def getChild(self, name):
"""
- Look up a child resource.
+ Look up a child resource. First check C{self.putChildren}, then call
+ C{self.makeChild} if no pre-existing children were found.
+
@return: the child of this resource with the given name.
"""
if name == "":
@@ -755,23 +759,34 @@
result = self.makeChild(name)
return result
+
def makeChild(self, name):
- # Subclasses with real children need to override this and return the appropriate object
+ """
+ Called by L{DAVResourceWithChildrenMixin.getChild} to dynamically
+ create children that have not been pre-created with C{putChild}.
+ """
return None
+
def listChildren(self):
"""
@return: a sequence of the names of all known children of this resource.
"""
return self.putChildren.keys()
+
def locateChild(self, req, segments):
"""
- See L{IResource}C{.locateChild}.
+ See L{IResource.locateChild}.
"""
- # If getChild() finds a child resource, return it
- return (self.getChild(segments[0]), segments[1:])
+ thisSegment = segments[0]
+ moreSegments = segments[1:]
+ return maybeDeferred(self.getChild, thisSegment).addCallback(
+ lambda it: (it, moreSegments)
+ )
+
+
class DAVResourceWithoutChildrenMixin (object):
"""
Bits needed from twext.web2.static
@@ -789,6 +804,8 @@
def locateChild(self, request, segments):
return self, server.StopTraversal
+
+
class DAVPrincipalResource (DirectoryPrincipalPropertySearchMixIn,
SuperDAVPrincipalResource, LoggingMixIn,
DirectoryRenderingMixIn):
@@ -888,8 +905,8 @@
if self.deadProperties().contains((dav_namespace, "resourcetype")):
return self.deadProperties().get((dav_namespace, "resourcetype"))
if self.isCollection():
- return davxml.ResourceType.collection
- return davxml.ResourceType.empty
+ return davxml.ResourceType.collection #@UndefinedVariable
+ return davxml.ResourceType.empty #@UndefinedVariable
def render(self, request):
if not self.fp.exists():
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/icaldav.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/icaldav.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/icaldav.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -67,29 +67,25 @@
Create a calendar collection for this resource.
"""
- def iCalendar(name=None):
+ def iCalendar():
"""
Instantiate an iCalendar component object representing this resource or
its child with the given name.
The behavior of this method is not specified if it is called on a
resource that is not a calendar collection or a calendar resource within
a calendar collection.
- @param name: the name of the desired child of this resource, or None
- if this resource is desired. Must be None if this resource is
- not a calendar collection.
+
@return: a L{twistedcaldav.ical.Component} of type C{"VCALENDAR"}.
"""
- def iCalendarText(name=None):
+ def iCalendarText():
"""
Obtains the iCalendar text representing this resource or its child with
the given name.
The behavior of this method is not specified if it is called on a
resource that is not a calendar collection or a calendar resource within
a calendar collection.
- @param name: the name of the desired child of this resource, or None
- if this resource is desired. Must be None if this resource is
- not a calendar collection.
+
@return: a string containing iCalendar text with a top-level component
of type C{"VCALENDAR"}.
"""
Deleted: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/index.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/index.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/index.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -1,1156 +0,0 @@
-# -*- test-case-name: twistedcaldav.test.test_index -*-
-##
-# Copyright (c) 2005-2010 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
-
-"""
-CalDAV Index.
-
-This API is considered private to static.py and is therefore subject to
-change.
-"""
-
-__all__ = [
- "db_basename",
- "ReservationError",
- "MemcachedUIDReserver",
- "Index",
- "IndexSchedule",
- "IndexedSearchException",
-]
-
-import datetime
-import time
-import hashlib
-
-try:
- import sqlite3 as sqlite
-except ImportError:
- from pysqlite2 import dbapi2 as sqlite
-
-from vobject.icalendar import utc
-
-from twisted.internet.defer import maybeDeferred, succeed
-
-from twext.python.log import Logger, LoggingMixIn
-
-from twistedcaldav.ical import Component
-from twistedcaldav.query import calendarquery, calendarqueryfilter
-from twistedcaldav.sql import AbstractSQLDatabase
-from twistedcaldav.sql import db_prefix
-from twistedcaldav.instance import InvalidOverriddenInstanceError
-from twistedcaldav.config import config
-from twistedcaldav.memcachepool import CachePoolUserMixIn
-
-log = Logger()
-
-db_basename = db_prefix + "sqlite"
-schema_version = "10"
-collection_types = {"Calendar": "Regular Calendar Collection", "iTIP": "iTIP Calendar Collection"}
-
-icalfbtype_to_indexfbtype = {
- "FREE" : 'F',
- "BUSY" : 'B',
- "BUSY-UNAVAILABLE": 'U',
- "BUSY-TENTATIVE" : 'T',
-}
-indexfbtype_to_icalfbtype = dict([(v, k) for k,v in icalfbtype_to_indexfbtype.iteritems()])
-
-#
-# Duration into the future through which recurrences are expanded in the index
-# by default. This is a caching parameter which affects the size of the index;
-# it does not affect search results beyond this period, but it may affect
-# performance of such a search.
-#
-default_future_expansion_duration = datetime.timedelta(days=365*1)
-
-#
-# Maximum duration into the future through which recurrences are expanded in the
-# index. This is a caching parameter which affects the size of the index; it
-# does not affect search results beyond this period, but it may affect
-# performance of such a search.
-#
-# When a search is performed on a time span that goes beyond that which is
-# expanded in the index, we have to open each resource which may have data in
-# that time period. In order to avoid doing that multiple times, we want to
-# cache those results. However, we don't necessarily want to cache all
-# occurrences into some obscenely far-in-the-future date, so we cap the caching
-# period. Searches beyond this period will always be relatively expensive for
-# resources with occurrences beyond this period.
-#
-maximum_future_expansion_duration = datetime.timedelta(days=365*5)
-
-class ReservationError(LookupError):
- """
- Attempt to reserve a UID which is already reserved or to unreserve a UID
- which is not reserved.
- """
-
-class IndexedSearchException(ValueError):
- pass
-
-class SyncTokenValidException(ValueError):
- pass
-
-class AbstractCalendarIndex(AbstractSQLDatabase, LoggingMixIn):
- """
- Calendar collection index abstract base class that defines the apis for the index.
- This will be subclassed for the two types of index behaviour we need: one for
- regular calendar collections, one for schedule calendar collections.
- """
-
- def __init__(self, resource):
- """
- @param resource: the L{CalDAVResource} resource to
- index. C{resource} must be a calendar collection (ie.
- C{resource.isPseudoCalendarCollection()} returns C{True}.)
- """
- self.resource = resource
- db_filename = self.resource.fp.child(db_basename).path
- super(AbstractCalendarIndex, self).__init__(db_filename, False)
-
- def create(self):
- """
- Create the index and initialize it.
- """
- self._db()
-
- def reserveUID(self, uid):
- """
- Reserve a UID for this index's resource.
- @param uid: the UID to reserve
- @raise ReservationError: if C{uid} is already reserved
- """
- raise NotImplementedError
-
- def unreserveUID(self, uid):
- """
- Unreserve a UID for this index's resource.
- @param uid: the UID to reserve
- @raise ReservationError: if C{uid} is not reserved
- """
- raise NotImplementedError
-
- def isReservedUID(self, uid):
- """
- Check to see whether a UID is reserved.
- @param uid: the UID to check
- @return: True if C{uid} is reserved, False otherwise.
- """
- raise NotImplementedError
-
- def isAllowedUID(self, uid, *names):
- """
- Checks to see whether to allow an operation with adds the the specified
- UID is allowed to the index. Specifically, the operation may not
- violate the constraint that UIDs must be unique, and the UID must not
- be reserved.
- @param uid: the UID to check
- @param names: the names of resources being replaced or deleted by the
- operation; UIDs associated with these resources are not checked.
- @return: True if the UID is not in the index and is not reserved,
- False otherwise.
- """
- raise NotImplementedError
-
- def resourceNamesForUID(self, uid):
- """
- Looks up the names of the resources with the given UID.
- @param uid: the UID of the resources to look up.
- @return: a list of resource names
- """
- names = self._db_values_for_sql("select NAME from RESOURCE where UID = :1", uid)
-
- #
- # Check that each name exists as a child of self.resource. If not, the
- # resource record is stale.
- #
- resources = []
- for name in names:
- name_utf8 = name.encode("utf-8")
- if name is not None and self.resource.getChild(name_utf8) is None:
- # Clean up
- log.err("Stale resource record found for child %s with UID %s in %s" % (name, uid, self.resource))
- self._delete_from_db(name, uid, False)
- self._db_commit()
- else:
- resources.append(name_utf8)
-
- return resources
-
- def resourceNameForUID(self, uid):
- """
- Looks up the name of the resource with the given UID.
- @param uid: the UID of the resource to look up.
- @return: If the resource is found, its name; C{None} otherwise.
- """
- result = None
-
- for name in self.resourceNamesForUID(uid):
- assert result is None, "More than one resource with UID %s in calendar collection %r" % (uid, self)
- result = name
-
- return result
-
- def resourceUIDForName(self, name):
- """
- Looks up the UID of the resource with the given name.
- @param name: the name of the resource to look up.
- @return: If the resource is found, the UID of the resource; C{None}
- otherwise.
- """
- uid = self._db_value_for_sql("select UID from RESOURCE where NAME = :1", name)
-
- return uid
-
- def addResource(self, name, calendar, fast=False, reCreate=False):
- """
- Adding or updating an existing resource.
- To check for an update we attempt to get an existing UID
- for the resource name. If present, then the index entries for
- that UID are removed. After that the new index entries are added.
- @param name: the name of the resource to add.
- @param calendar: a L{Calendar} object representing the resource
- contents.
- @param fast: if C{True} do not do commit, otherwise do commit.
- """
- oldUID = self.resourceUIDForName(name)
- if oldUID is not None:
- self._delete_from_db(name, oldUID, False)
- self._add_to_db(name, calendar, reCreate=reCreate)
- if not fast:
- self._db_commit()
-
- def deleteResource(self, name):
- """
- Remove this resource from the index.
- @param name: the name of the resource to add.
- @param uid: the UID of the calendar component in the resource.
- """
- uid = self.resourceUIDForName(name)
- if uid is not None:
- self._delete_from_db(name, uid)
- self._db_commit()
-
- def resourceExists(self, name):
- """
- Determines whether the specified resource name exists in the index.
- @param name: the name of the resource to test
- @return: True if the resource exists, False if not
- """
- uid = self._db_value_for_sql("select UID from RESOURCE where NAME = :1", name)
- return uid is not None
-
- def resourcesExist(self, names):
- """
- Determines whether the specified resource name exists in the index.
- @param names: a C{list} containing the names of the resources to test
- @return: a C{list} of all names that exist
- """
- statement = "select NAME from RESOURCE where NAME in ("
- for ctr in (item[0] for item in enumerate(names)):
- if ctr != 0:
- statement += ", "
- statement += ":%s" % (ctr,)
- statement += ")"
- results = self._db_values_for_sql(statement, *names)
- return results
-
-
- def testAndUpdateIndex(self, minDate):
- # Find out if the index is expanded far enough
- names = self.notExpandedBeyond(minDate)
- # Actually expand recurrence max
- for name in names:
- self.log_info("Search falls outside range of index for %s %s" % (name, minDate))
- self.reExpandResource(name, minDate)
-
- def whatchanged(self, revision):
-
- results = [(name.encode("utf-8"), deleted) for name, deleted in self._db_execute("select NAME, DELETED from REVISIONS where REVISION > :1", revision)]
- results.sort(key=lambda x:x[1])
-
- changed = []
- deleted = []
- for name, wasdeleted in results:
- if name:
- if wasdeleted == 'Y':
- if revision:
- deleted.append(name)
- else:
- changed.append(name)
- else:
- raise SyncTokenValidException
-
- return changed, deleted,
-
- def lastRevision(self):
- return self._db_value_for_sql(
- "select REVISION from REVISION_SEQUENCE"
- )
-
- def bumpRevision(self, fast=False):
- self._db_execute(
- """
- update REVISION_SEQUENCE set REVISION = REVISION + 1
- """,
- )
- self._db_commit()
- return self._db_value_for_sql(
- """
- select REVISION from REVISION_SEQUENCE
- """,
- )
-
- def indexedSearch(self, filter, useruid="", fbtype=False):
- """
- Finds resources matching the given qualifiers.
- @param filter: the L{Filter} for the calendar-query to execute.
- @return: an iterable of tuples for each resource matching the
- given C{qualifiers}. The tuples are C{(name, uid, type)}, where
- C{name} is the resource name, C{uid} is the resource UID, and
- C{type} is the resource iCalendar component type.
- """
-
- # Make sure we have a proper Filter element and get the partial SQL
- # statement to use.
- if isinstance(filter, calendarqueryfilter.Filter):
- if fbtype:
- # Lookup the useruid - try the empty (default) one if needed
- dbuseruid = self._db_value_for_sql(
- "select PERUSERID from PERUSER where USERUID == :1",
- useruid,
- )
- else:
- dbuseruid = ""
-
- qualifiers = calendarquery.sqlcalendarquery(filter, None, dbuseruid)
- if qualifiers is not None:
- # Determine how far we need to extend the current expansion of
- # events. If we have an open-ended time-range we will expand one
- # year past the start. That should catch bounded recurrences - unbounded
- # will have been indexed with an "infinite" value always included.
- maxDate, isStartDate = filter.getmaxtimerange()
- if maxDate:
- maxDate = maxDate.date()
- if isStartDate:
- maxDate += datetime.timedelta(days=365)
- self.testAndUpdateIndex(maxDate)
- else:
- # We cannot handler this filter in an indexed search
- raise IndexedSearchException()
-
- else:
- qualifiers = None
-
- # Perform the search
- if qualifiers is None:
- rowiter = self._db_execute("select NAME, UID, TYPE from RESOURCE")
- else:
- if fbtype:
- # Lookup the useruid - try the empty (default) one if needed
- dbuseruid = self._db_value_for_sql(
- "select PERUSERID from PERUSER where USERUID == :1",
- useruid,
- )
-
- # For a free-busy time-range query we return all instances
- rowiter = self._db_execute(
- "select DISTINCT RESOURCE.NAME, RESOURCE.UID, RESOURCE.TYPE, RESOURCE.ORGANIZER, TIMESPAN.FLOAT, TIMESPAN.START, TIMESPAN.END, TIMESPAN.FBTYPE, TIMESPAN.TRANSPARENT, TRANSPARENCY.TRANSPARENT" +
- qualifiers[0],
- *qualifiers[1]
- )
- else:
- rowiter = self._db_execute("select DISTINCT RESOURCE.NAME, RESOURCE.UID, RESOURCE.TYPE" + qualifiers[0], *qualifiers[1])
-
- # Check result for missing resources
-
- for row in rowiter:
- name = row[0]
- if self.resource.getChild(name.encode("utf-8")):
- if fbtype:
- row = list(row)
- if row[9]:
- row[8] = row[9]
- del row[9]
- yield row
- else:
- log.err("Calendar resource %s is missing from %s. Removing from index."
- % (name, self.resource))
- self.deleteResource(name)
-
- def bruteForceSearch(self):
- """
- List the whole index and tests for existence, updating the index
- @return: all resources in the index
- """
- # List all resources
- rowiter = self._db_execute("select NAME, UID, TYPE from RESOURCE")
-
- # Check result for missing resources:
-
- for row in rowiter:
- name = row[0]
- if self.resource.getChild(name.encode("utf-8")):
- yield row
- else:
- log.err("Calendar resource %s is missing from %s. Removing from index."
- % (name, self.resource))
- self.deleteResource(name)
-
-
- def _db_version(self):
- """
- @return: the schema version assigned to this index.
- """
- return schema_version
-
- def _add_to_db(self, name, calendar, cursor=None, expand_until=None, reCreate=False):
- """
- Records the given calendar resource in the index with the given name.
- Resource names and UIDs must both be unique; only one resource name may
- be associated with any given UID and vice versa.
- NB This method does not commit the changes to the db - the caller
- MUST take care of that
- @param name: the name of the resource to add.
- @param calendar: a L{Calendar} object representing the resource
- contents.
- """
- raise NotImplementedError
-
- def _delete_from_db(self, name, uid, dorevision=True):
- """
- Deletes the specified entry from all dbs.
- @param name: the name of the resource to delete.
- @param uid: the uid of the resource to delete.
- """
- raise NotImplementedError
-
-class CalendarIndex (AbstractCalendarIndex):
- """
- Calendar index - abstract class for indexer that indexes calendar objects in a collection.
- """
-
- def __init__(self, resource):
- """
- @param resource: the L{CalDAVResource} resource to
- index.
- """
- super(CalendarIndex, self).__init__(resource)
-
- def _db_init_data_tables_base(self, q, uidunique):
- """
- Initialise the underlying database tables.
- @param q: a database cursor to use.
- """
- #
- # RESOURCE table is the primary index table
- # NAME: Last URI component (eg. <uid>.ics, RESOURCE primary key)
- # UID: iCalendar UID (may or may not be unique)
- # TYPE: iCalendar component type
- # RECURRANCE_MAX: Highest date of recurrence expansion
- # ORGANIZER: cu-address of the Organizer of the event
- #
- q.execute(
- """
- create table RESOURCE (
- RESOURCEID integer primary key autoincrement,
- NAME text unique,
- UID text%s,
- TYPE text,
- RECURRANCE_MAX date,
- ORGANIZER text
- )
- """ % (" unique" if uidunique else "",)
- )
-
- #
- # TIMESPAN table tracks (expanded) time spans for resources
- # NAME: Related resource (RESOURCE foreign key)
- # FLOAT: 'Y' if start/end are floating, 'N' otherwise
- # START: Start date
- # END: End date
- # FBTYPE: FBTYPE value:
- # '?' - unknown
- # 'F' - free
- # 'B' - busy
- # 'U' - busy-unavailable
- # 'T' - busy-tentative
- # TRANSPARENT: Y if transparent, N if opaque (default non-per-user value)
- #
- q.execute(
- """
- create table TIMESPAN (
- INSTANCEID integer primary key autoincrement,
- RESOURCEID integer,
- FLOAT text(1),
- START date,
- END date,
- FBTYPE text(1),
- TRANSPARENT text(1)
- )
- """
- )
- q.execute(
- """
- create index STARTENDFLOAT on TIMESPAN (START, END, FLOAT)
- """
- )
-
- #
- # PERUSER table tracks per-user ids
- # PERUSERID: autoincrement primary key
- # UID: User ID used in calendar data
- #
- q.execute(
- """
- create table PERUSER (
- PERUSERID integer primary key autoincrement,
- USERUID text
- )
- """
- )
- q.execute(
- """
- create index PERUSER_UID on PERUSER (USERUID)
- """
- )
-
- #
- # TRANSPARENCY table tracks per-user per-instance transparency
- # PERUSERID: user id key
- # INSTANCEID: instance id key
- # TRANSPARENT: Y if transparent, N if opaque
- #
- q.execute(
- """
- create table TRANSPARENCY (
- PERUSERID integer,
- INSTANCEID integer,
- TRANSPARENT text(1)
- )
- """
- )
-
- #
- # REVISIONS table tracks changes
- # NAME: Last URI component (eg. <uid>.ics, RESOURCE primary key)
- # REVISION: revision number
- # WASDELETED: Y if revision deleted, N if added or changed
- #
- q.execute(
- """
- create table REVISION_SEQUENCE (
- REVISION integer
- )
- """
- )
- q.execute(
- """
- insert into REVISION_SEQUENCE (REVISION) values (0)
- """
- )
- q.execute(
- """
- create table REVISIONS (
- NAME text unique,
- REVISION integer,
- DELETED text(1)
- )
- """
- )
- q.execute(
- """
- create index REVISION on REVISIONS (REVISION)
- """
- )
-
- if uidunique:
- #
- # RESERVED table tracks reserved UIDs
- # UID: The UID being reserved
- # TIME: When the reservation was made
- #
- q.execute(
- """
- create table RESERVED (
- UID text unique,
- TIME date
- )
- """
- )
-
- # Cascading triggers to help on delete
- q.execute(
- """
- create trigger resourceDelete after delete on RESOURCE
- for each row
- begin
- delete from TIMESPAN where TIMESPAN.RESOURCEID = OLD.RESOURCEID;
- end
- """
- )
- q.execute(
- """
- create trigger timespanDelete after delete on TIMESPAN
- for each row
- begin
- delete from TRANSPARENCY where INSTANCEID = OLD.INSTANCEID;
- end
- """
- )
-
- def _db_can_upgrade(self, old_version):
- """
- Can we do an in-place upgrade
- """
-
- # v10 is a big change - no upgrade possible
- return False
-
- def _db_upgrade_data_tables(self, q, old_version):
- """
- Upgrade the data from an older version of the DB.
- """
-
- # v10 is a big change - no upgrade possible
- pass
-
- def notExpandedBeyond(self, minDate):
- """
- Gives all resources which have not been expanded beyond a given date
- in the index
- """
- return self._db_values_for_sql("select NAME from RESOURCE where RECURRANCE_MAX < :1", minDate)
-
- def reExpandResource(self, name, expand_until):
- """
- Given a resource name, remove it from the database and re-add it
- with a longer expansion.
- """
- calendar = self.resource.getChild(name).iCalendar()
- self._add_to_db(name, calendar, expand_until=expand_until, reCreate=True)
- self._db_commit()
-
- def _add_to_db(self, name, calendar, cursor = None, expand_until=None, reCreate=False):
- """
- Records the given calendar resource in the index with the given name.
- Resource names and UIDs must both be unique; only one resource name may
- be associated with any given UID and vice versa.
- NB This method does not commit the changes to the db - the caller
- MUST take care of that
- @param name: the name of the resource to add.
- @param calendar: a L{Calendar} object representing the resource
- contents.
- """
- uid = calendar.resourceUID()
- organizer = calendar.getOrganizer()
- if not organizer:
- organizer = ""
-
- # Decide how far to expand based on the component
- master = calendar.masterComponent()
- if master is None or not calendar.isRecurring() and not calendar.isRecurringUnbounded():
- # When there is no master we have a set of overridden components - index them all.
- # When there is one instance - index it.
- # When bounded - index all.
- expand = datetime.datetime(2100, 1, 1, 0, 0, 0, tzinfo=utc)
- else:
- if expand_until:
- expand = expand_until
- else:
- expand = datetime.date.today() + default_future_expansion_duration
-
- if expand > (datetime.date.today() + maximum_future_expansion_duration):
- raise IndexedSearchException
-
- try:
- instances = calendar.expandTimeRanges(expand, ignoreInvalidInstances=reCreate)
- except InvalidOverriddenInstanceError, e:
- log.err("Invalid instance %s when indexing %s in %s" % (e.rid, name, self.resource,))
- raise
-
- self._delete_from_db(name, uid, False)
-
- # Add RESOURCE item
- self._db_execute(
- """
- insert into RESOURCE (NAME, UID, TYPE, RECURRANCE_MAX, ORGANIZER)
- values (:1, :2, :3, :4, :5)
- """, name, uid, calendar.resourceType(), instances.limit, organizer
- )
- resourceid = self.lastrowid
-
- # Get a set of all referenced per-user UIDs and map those to entries already
- # in the DB and add new ones as needed
- useruids = calendar.allPerUserUIDs()
- useruids.add("")
- useruidmap = {}
- for useruid in useruids:
- peruserid = self._db_value_for_sql(
- "select PERUSERID from PERUSER where USERUID = :1",
- useruid
- )
- if peruserid is None:
- self._db_execute(
- """
- insert into PERUSER (USERUID)
- values (:1)
- """, useruid
- )
- peruserid = self.lastrowid
- useruidmap[useruid] = peruserid
-
- for key in instances:
- instance = instances[key]
- start = instance.start.replace(tzinfo=utc)
- end = instance.end.replace(tzinfo=utc)
- float = 'Y' if instance.start.tzinfo is None else 'N'
- transp = 'T' if instance.component.propertyValue("TRANSP") == "TRANSPARENT" else 'F'
- self._db_execute(
- """
- insert into TIMESPAN (RESOURCEID, FLOAT, START, END, FBTYPE, TRANSPARENT)
- values (:1, :2, :3, :4, :5, :6)
- """,
- resourceid,
- float,
- start,
- end,
- icalfbtype_to_indexfbtype.get(instance.component.getFBType(), 'F'),
- transp
- )
- instanceid = self.lastrowid
- peruserdata = calendar.perUserTransparency(instance.rid)
- for useruid, transp in peruserdata:
- peruserid = useruidmap[useruid]
- self._db_execute(
- """
- insert into TRANSPARENCY (PERUSERID, INSTANCEID, TRANSPARENT)
- values (:1, :2, :3)
- """, peruserid, instanceid, 'T' if transp else 'F'
- )
-
-
- # Special - for unbounded recurrence we insert a value for "infinity"
- # that will allow an open-ended time-range to always match it.
- if calendar.isRecurringUnbounded():
- start = datetime.datetime(2100, 1, 1, 0, 0, 0, tzinfo=utc)
- end = datetime.datetime(2100, 1, 1, 1, 0, 0, tzinfo=utc)
- float = 'N'
- self._db_execute(
- """
- insert into TIMESPAN (RESOURCEID, FLOAT, START, END, FBTYPE, TRANSPARENT)
- values (:1, :2, :3, :4, :5, :6)
- """, resourceid, float, start, end, '?', '?'
- )
- instanceid = self.lastrowid
- peruserdata = calendar.perUserTransparency(None)
- for useruid, transp in peruserdata:
- peruserid = useruidmap[useruid]
- self._db_execute(
- """
- insert into TRANSPARENCY (PERUSERID, INSTANCEID, TRANSPARENT)
- values (:1, :2, :3)
- """, peruserid, instanceid, 'T' if transp else 'F'
- )
-
- self._db_execute(
- """
- insert or replace into REVISIONS (NAME, REVISION, DELETED)
- values (:1, :2, :3)
- """, name, self.bumpRevision(fast=True), 'N',
- )
-
- def _delete_from_db(self, name, uid, dorevision=True):
- """
- Deletes the specified entry from all dbs.
- @param name: the name of the resource to delete.
- @param uid: the uid of the resource to delete.
- """
- self._db_execute("delete from RESOURCE where NAME = :1", name)
- if dorevision:
- self._db_execute(
- """
- update REVISIONS SET REVISION = :1, DELETED = :2
- where NAME = :3
- """, self.bumpRevision(fast=True), 'Y', name
- )
-
-
-def wrapInDeferred(f):
- def _(*args, **kwargs):
- return maybeDeferred(f, *args, **kwargs)
-
- return _
-
-
-class MemcachedUIDReserver(CachePoolUserMixIn, LoggingMixIn):
- def __init__(self, index, cachePool=None):
- self.index = index
- self._cachePool = cachePool
-
- def _key(self, uid):
- return 'reservation:%s' % (
- hashlib.md5('%s:%s' % (uid,
- self.index.resource.fp.path)).hexdigest())
-
- def reserveUID(self, uid):
- uid = uid.encode('utf-8')
- self.log_debug("Reserving UID %r @ %r" % (
- uid,
- self.index.resource.fp.path))
-
- def _handleFalse(result):
- if result is False:
- raise ReservationError(
- "UID %s already reserved for calendar collection %s."
- % (uid, self.index.resource)
- )
-
- d = self.getCachePool().add(self._key(uid),
- 'reserved',
- expireTime=config.UIDReservationTimeOut)
- d.addCallback(_handleFalse)
- return d
-
-
- def unreserveUID(self, uid):
- uid = uid.encode('utf-8')
- self.log_debug("Unreserving UID %r @ %r" % (
- uid,
- self.index.resource.fp.path))
-
- def _handleFalse(result):
- if result is False:
- raise ReservationError(
- "UID %s is not reserved for calendar collection %s."
- % (uid, self.index.resource)
- )
-
- d =self.getCachePool().delete(self._key(uid))
- d.addCallback(_handleFalse)
- return d
-
-
- def isReservedUID(self, uid):
- uid = uid.encode('utf-8')
- self.log_debug("Is reserved UID %r @ %r" % (
- uid,
- self.index.resource.fp.path))
-
- def _checkValue((flags, value)):
- if value is None:
- return False
- else:
- return True
-
- d = self.getCachePool().get(self._key(uid))
- d.addCallback(_checkValue)
- return d
-
-
-
-class SQLUIDReserver(object):
- def __init__(self, index):
- self.index = index
-
- @wrapInDeferred
- def reserveUID(self, uid):
- """
- Reserve a UID for this index's resource.
- @param uid: the UID to reserve
- @raise ReservationError: if C{uid} is already reserved
- """
-
- try:
- self.index._db_execute("insert into RESERVED (UID, TIME) values (:1, :2)", uid, datetime.datetime.now())
- self.index._db_commit()
- except sqlite.IntegrityError:
- self.index._db_rollback()
- raise ReservationError(
- "UID %s already reserved for calendar collection %s."
- % (uid, self.index.resource)
- )
- except sqlite.Error, e:
- log.err("Unable to reserve UID: %s", (e,))
- self.index._db_rollback()
- raise
-
- def unreserveUID(self, uid):
- """
- Unreserve a UID for this index's resource.
- @param uid: the UID to reserve
- @raise ReservationError: if C{uid} is not reserved
- """
-
- def _cb(result):
- if result == False:
- raise ReservationError(
- "UID %s is not reserved for calendar collection %s."
- % (uid, self.index.resource)
- )
- else:
- try:
- self.index._db_execute(
- "delete from RESERVED where UID = :1", uid)
- self.index._db_commit()
- except sqlite.Error, e:
- log.err("Unable to unreserve UID: %s", (e,))
- self.index._db_rollback()
- raise
-
- d = self.isReservedUID(uid)
- d.addCallback(_cb)
- return d
-
-
- @wrapInDeferred
- def isReservedUID(self, uid):
- """
- Check to see whether a UID is reserved.
- @param uid: the UID to check
- @return: True if C{uid} is reserved, False otherwise.
- """
-
- rowiter = self.index._db_execute("select UID, TIME from RESERVED where UID = :1", uid)
- for uid, attime in rowiter:
- # Double check that the time is within a reasonable period of now
- # otherwise we probably have a stale reservation
- tm = time.strptime(attime[:19], "%Y-%m-%d %H:%M:%S")
- dt = datetime.datetime(year=tm.tm_year, month=tm.tm_mon, day=tm.tm_mday, hour=tm.tm_hour, minute=tm.tm_min, second = tm.tm_sec)
- if datetime.datetime.now() - dt > datetime.timedelta(seconds=config.UIDReservationTimeOut):
- try:
- self.index._db_execute("delete from RESERVED where UID = :1", uid)
- self.index._db_commit()
- except sqlite.Error, e:
- log.err("Unable to unreserve UID: %s", (e,))
- self.index._db_rollback()
- raise
- return False
- else:
- return True
-
- return False
-
-
-
-class Index (CalendarIndex):
- """
- Calendar collection index - regular collection that enforces CalDAV UID uniqueness requirement.
- """
-
- def __init__(self, resource):
- """
- @param resource: the L{CalDAVResource} resource to
- index. C{resource} must be a calendar collection (i.e.
- C{resource.isPseudoCalendarCollection()} returns C{True}.)
- """
- assert resource.isCalendarCollection(), "non-calendar collection resource %s has no index." % (resource,)
- super(Index, self).__init__(resource)
-
- if (
- hasattr(config, "Memcached") and
- config.Memcached.Pools.Default.ClientEnabled
- ):
- self.reserver = MemcachedUIDReserver(self)
- else:
- self.reserver = SQLUIDReserver(self)
-
- #
- # A dict of sets. The dict keys are calendar collection paths,
- # and the sets contains reserved UIDs for each path.
- #
-
- def reserveUID(self, uid):
- return self.reserver.reserveUID(uid)
-
-
- def unreserveUID(self, uid):
- return self.reserver.unreserveUID(uid)
-
-
- def isReservedUID(self, uid):
- return self.reserver.isReservedUID(uid)
-
-
- def isAllowedUID(self, uid, *names):
- """
- Checks to see whether to allow an operation which would add the
- specified UID to the index. Specifically, the operation may not
- violate the constraint that UIDs must be unique.
- @param uid: the UID to check
- @param names: the names of resources being replaced or deleted by the
- operation; UIDs associated with these resources are not checked.
- @return: True if the UID is not in the index and is not reserved,
- False otherwise.
- """
- rname = self.resourceNameForUID(uid)
- return (rname is None or rname in names)
-
- def _db_type(self):
- """
- @return: the collection type assigned to this index.
- """
- return collection_types["Calendar"]
-
- def _db_init_data_tables(self, q):
- """
- Initialise the underlying database tables.
- @param q: a database cursor to use.
- """
-
- # Create database where the RESOURCE table has unique UID column.
- self._db_init_data_tables_base(q, True)
-
- def _db_recreate(self, do_commit=True):
- """
- Re-create the database tables from existing calendar data.
- """
-
- #
- # Populate the DB with data from already existing resources.
- # This allows for index recovery if the DB file gets
- # deleted.
- #
- fp = self.resource.fp
- for name in fp.listdir():
- if name.startswith("."):
- continue
-
- try:
- stream = fp.child(name).open()
- except (IOError, OSError), e:
- log.err("Unable to open resource %s: %s" % (name, e))
- continue
-
- # FIXME: This is blocking I/O
- try:
- calendar = Component.fromStream(stream)
- calendar.validateForCalDAV()
- except ValueError:
- log.err("Non-calendar resource: %s" % (name,))
- else:
- #log.msg("Indexing resource: %s" % (name,))
- self.addResource(name, calendar, True, reCreate=True)
- finally:
- stream.close()
-
- # Do commit outside of the loop for better performance
- if do_commit:
- self._db_commit()
-
-class IndexSchedule (CalendarIndex):
- """
- Schedule collection index - does not require UID uniqueness.
- """
-
- def reserveUID(self, uid): #@UnusedVariable
- """
- Reserve a UID for this index's resource.
- @param uid: the UID to reserve
- @raise ReservationError: if C{uid} is already reserved
- """
-
- # iTIP does not require unique UIDs
- return succeed(None)
-
- def unreserveUID(self, uid): #@UnusedVariable
- """
- Unreserve a UID for this index's resource.
- @param uid: the UID to reserve
- @raise ReservationError: if C{uid} is not reserved
- """
-
- # iTIP does not require unique UIDs
- return succeed(None)
-
- def isReservedUID(self, uid): #@UnusedVariable
- """
- Check to see whether a UID is reserved.
- @param uid: the UID to check
- @return: True if C{uid} is reserved, False otherwise.
- """
-
- # iTIP does not require unique UIDs
- return succeed(False)
-
- def isAllowedUID(self, uid, *names): #@UnusedVariable
- """
- Checks to see whether to allow an operation with adds the the specified
- UID is allowed to the index. Specifically, the operation may not
- violate the constraint that UIDs must be unique, and the UID must not
- be reserved.
- @param uid: the UID to check
- @param names: the names of resources being replaced or deleted by the
- operation; UIDs associated with these resources are not checked.
- @return: True if the UID is not in the index and is not reserved,
- False otherwise.
- """
-
- # iTIP does not require unique UIDs
- return True
-
- def _db_type(self):
- """
- @return: the collection type assigned to this index.
- """
- return collection_types["iTIP"]
-
- def _db_init_data_tables(self, q):
- """
- Initialise the underlying database tables.
- @param q: a database cursor to use.
- """
-
- # Create database where the RESOURCE table has a UID column that is not unique.
- self._db_init_data_tables_base(q, False)
-
- def _db_recreate(self, do_commit=True):
- """
- Re-create the database tables from existing calendar data.
- """
-
- #
- # Populate the DB with data from already existing resources.
- # This allows for index recovery if the DB file gets
- # deleted.
- #
- fp = self.resource.fp
- for name in fp.listdir():
- if name.startswith("."):
- continue
-
- try:
- stream = fp.child(name).open()
- except (IOError, OSError), e:
- log.err("Unable to open resource %s: %s" % (name, e))
- continue
-
- # FIXME: This is blocking I/O
- try:
- calendar = Component.fromStream(stream)
- calendar.validCalendarForCalDAV()
- calendar.validateComponentsForCalDAV(True)
- except ValueError:
- log.err("Non-calendar resource: %s" % (name,))
- else:
- #log.msg("Indexing resource: %s" % (name,))
- self.addResource(name, calendar, True, reCreate=True)
- finally:
- stream.close()
-
- # Do commit outside of the loop for better performance
- if do_commit:
- self._db_commit()
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/put_addressbook_common.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/put_addressbook_common.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/put_addressbook_common.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -25,7 +25,9 @@
from twisted.internet import reactor
from twisted.python.failure import Failure
-from twisted.internet.defer import Deferred, inlineCallbacks, succeed
+from txdav.common.icommondatastore import ReservationError
+
+from twisted.internet.defer import Deferred, inlineCallbacks
from twisted.internet.defer import returnValue
from twext.web2 import responsecode
from twext.web2.dav import davxml
@@ -42,7 +44,6 @@
from twistedcaldav.config import config
from twistedcaldav.carddavxml import NoUIDConflict, carddav_namespace
from twistedcaldav.vcard import Component
-from twistedcaldav.vcardindex import ReservationError
from twext.python.log import Logger
log = Logger()
@@ -149,6 +150,8 @@
self.access = None
+
+ @inlineCallbacks
def fullValidation(self):
"""
Do full validation of source and destination vcard data.
@@ -191,7 +194,7 @@
else:
# Get UID from original resource
self.source_index = self.sourceparent.index()
- self.uid = self.source_index.resourceUIDForName(self.source.name())
+ self.uid = yield self.source_index.resourceUIDForName(self.source.name())
if self.uid is None:
log.err("Source vcard does not have a UID: %s" % self.source.name())
raise HTTPError(ErrorResponse(responsecode.FORBIDDEN, (carddav_namespace, "valid-addressbook-object-resource")))
@@ -207,7 +210,7 @@
raise HTTPError(ErrorResponse(responsecode.FORBIDDEN, (carddav_namespace, "max-resource-size")))
# Check access
- return succeed(None)
+ returnValue(None)
def validResourceName(self):
"""
@@ -271,6 +274,8 @@
return result, message
+
+ @inlineCallbacks
def noUIDConflict(self, uid):
"""
Check that the UID of the new vcard object conforms to the requirements of
@@ -305,14 +310,15 @@
else:
# Cannot overwrite a resource with different UID
if self.destination.exists():
- olduid = index.resourceUIDForName(self.destination.name())
+ olduid = yield index.resourceUIDForName(self.destination.name())
if olduid != uid:
rname = self.destination.name()
result = False
message = "Cannot overwrite vcard resource %s with different UID %s" % (rname, olduid)
- return result, message, rname
+ returnValue((result, message, rname))
+
@inlineCallbacks
def checkQuota(self):
"""
@@ -436,7 +442,7 @@
# UID conflict check - note we do this after reserving the UID to avoid a race condition where two requests
# try to write the same vcard data to two different resource URIs.
- result, message, rname = self.noUIDConflict(self.uid)
+ result, message, rname = yield self.noUIDConflict(self.uid)
if not result:
log.err(message)
raise HTTPError(ErrorResponse(responsecode.FORBIDDEN,
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/put_common.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/put_common.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/put_common.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -46,6 +46,8 @@
from twext.python.log import Logger
from twext.web2.dav.http import ErrorResponse
+from txdav.common.icommondatastore import ReservationError
+
from twistedcaldav.config import config
from twistedcaldav.caldavxml import ScheduleTag, NoUIDConflict
from twistedcaldav.caldavxml import NumberOfRecurrencesWithinLimits
@@ -57,7 +59,6 @@
from twistedcaldav.datafilters.peruserdata import PerUserDataFilter
from twistedcaldav.ical import Component, Property
-from twistedcaldav.index import ReservationError
from twistedcaldav.instance import TooManyInstancesError,\
InvalidOverriddenInstanceError
from twistedcaldav.memcachelock import MemcacheLock, MemcacheLockTimeoutError
@@ -280,7 +281,7 @@
else:
# Get UID from original resource
self.source_index = self.sourceparent.index()
- self.uid = self.source_index.resourceUIDForName(self.source.name())
+ self.uid = yield self.source_index.resourceUIDForName(self.source.name())
if self.uid is None:
log.err("Source calendar does not have a UID: %s" % self.source)
raise HTTPError(ErrorResponse(responsecode.FORBIDDEN, (caldav_namespace, "valid-calendar-object-resource")))
@@ -573,6 +574,7 @@
returnValue(new_has_private_comments)
+ @inlineCallbacks
def noUIDConflict(self, uid):
"""
Check that the UID of the new calendar object conforms to the requirements of
@@ -586,7 +588,7 @@
result = True
message = ""
rname = ""
-
+
# Adjust for a move into same calendar collection
oldname = None
if self.sourceparent and (self.sourceparent == self.destinationparent) and self.deletesource:
@@ -606,16 +608,15 @@
else:
# Cannot overwrite a resource with different UID
if self.destination.exists():
- olduid = index.resourceUIDForName(self.destination.name())
+ olduid = yield index.resourceUIDForName(self.destination.name())
if olduid != uid:
rname = self.destination.name()
result = False
message = "Cannot overwrite calendar resource %s with different UID %s" % (rname, olduid)
- return result, message, rname
+ returnValue((result, message, rname))
-
@inlineCallbacks
def doImplicitScheduling(self):
@@ -698,12 +699,14 @@
@inlineCallbacks
def mergePerUserData(self):
-
if self.calendar:
accessUID = (yield self.destination.resourceOwnerPrincipal(self.request))
accessUID = accessUID.principalUID() if accessUID else ""
- oldCal = self.destination.iCalendar() if self.destination.exists() and self.destinationcal else None
-
+ if self.destination.exists() and self.destinationcal:
+ oldCal = self.destination.iCalendar()
+ else:
+ oldCal = None
+
# Duplicate before we do the merge because someone else may "own" the calendar object
# and we should not change it. This is not ideal as we may duplicate it unnecessarily
# but we currently have no api to let the caller tell us whether it cares about the
@@ -830,7 +833,7 @@
# UID conflict check - note we do this after reserving the UID to avoid a race condition where two requests
# try to write the same calendar data to two different resource URIs.
if not self.isiTIP:
- result, message, rname = self.noUIDConflict(self.uid)
+ result, message, rname = yield self.noUIDConflict(self.uid)
if not result:
log.err(message)
raise HTTPError(ErrorResponse(responsecode.FORBIDDEN,
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/report_calendar_query.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/report_calendar_query.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/report_calendar_query.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -38,7 +38,7 @@
NumberOfRecurrencesWithinLimits
from twistedcaldav.config import config
from twistedcaldav.customxml import TwistedCalendarAccessProperty
-from twistedcaldav.index import IndexedSearchException
+from txdav.common.icommondatastore import IndexedSearchException
from twistedcaldav.instance import TooManyInstancesError
from twistedcaldav.method import report_common
from twistedcaldav.query import calendarqueryfilter
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/report_common.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/report_common.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/report_common.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -64,9 +64,11 @@
from twistedcaldav.dateops import clipPeriod, normalizePeriodList, timeRangesOverlap
from twistedcaldav.ical import Component, Property, iCalendarProductID
from twistedcaldav.instance import InstanceList
-from twistedcaldav.index import IndexedSearchException
+
from twistedcaldav.query import calendarqueryfilter
+from txdav.common.icommondatastore import IndexedSearchException
+
log = Logger()
COLLECTION_TYPE_REGULAR = "collection"
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/resource.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/resource.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/resource.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -1,4 +1,4 @@
-# -*- test-case-name: twistedcaldav.test.test_resource -*-
+# -*- test-case-name: twistedcaldav.test.test_resource,twistedcaldav.test.test_wrapping -*-
##
# Copyright (c) 2005-2010 Apple Inc. All rights reserved.
#
@@ -72,19 +72,17 @@
from twistedcaldav.extensions import DAVResource, DAVPrincipalResource,\
PropertyNotFoundError, DAVResourceWithChildrenMixin
from twistedcaldav.ical import Component
-from twistedcaldav.ical import Component as iComponent
-from twistedcaldav.ical import Property as iProperty
+
from twistedcaldav.ical import allowedComponents
from twistedcaldav.icaldav import ICalDAVResource, ICalendarPrincipalResource
-from twistedcaldav.index import SyncTokenValidException, Index
from twistedcaldav.linkresource import LinkResource
from twistedcaldav.notify import getPubSubConfiguration, getPubSubPath,\
getPubSubXMPPURI, getPubSubHeartbeatURI, getPubSubAPSConfiguration
from twistedcaldav.sharing import SharedCollectionMixin, SharedHomeMixin
from twistedcaldav.vcard import Component as vComponent
-from twistedcaldav.vcardindex import AddressBookIndex
-from txdav.common.icommondatastore import InternalDataStoreError
+from txdav.common.icommondatastore import InternalDataStoreError, \
+ SyncTokenValidException
##
# Sharing Conts
@@ -273,6 +271,7 @@
self._transactionError = True
+ @inlineCallbacks
def renderHTTP(self, request, transaction=None):
"""
Override C{renderHTTP} to commit the transaction when the resource is
@@ -283,17 +282,15 @@
@param transaction: optional transaction to use instead of associated transaction
@type transaction: L{txdav.caldav.idav.ITransaction}
"""
- d = maybeDeferred(super(CalDAVResource, self).renderHTTP, request)
- def succeeded(result, transaction=None):
- if transaction is None:
- transaction = self._associatedTransaction
- if transaction is not None:
- if self._transactionError:
- transaction.abort()
- else:
- transaction.commit()
- return result
- return d.addCallback(succeeded, transaction=transaction)
+ result = yield super(CalDAVResource, self).renderHTTP(request)
+ if transaction is None:
+ transaction = self._associatedTransaction
+ if transaction is not None:
+ if self._transactionError:
+ yield transaction.abort()
+ else:
+ yield transaction.commit()
+ returnValue(result)
# Begin transitional new-store resource interface:
@@ -839,6 +836,7 @@
else:
returnValue(None)
+
@inlineCallbacks
def resourceOwnerPrincipal(self, request):
"""
@@ -850,13 +848,16 @@
if isVirt:
returnValue(self._shareePrincipal)
else:
- parent = (yield self.locateParent(request, request.urlForResource(self)))
+ parent = (yield self.locateParent(
+ request, request.urlForResource(self)
+ ))
if parent and isinstance(parent, CalDAVResource):
result = (yield parent.resourceOwnerPrincipal(request))
returnValue(result)
else:
returnValue(None)
+
def isOwner(self, request, adminprincipals=False, readprincipals=False):
"""
Determine whether the DAV:owner of this resource matches the currently authorized principal
@@ -1110,35 +1111,19 @@
returnValue(False)
- def iCalendar(self, name=None):
- """
- See L{ICalDAVResource.iCalendar}.
- This implementation returns the an object created from the data returned
- by L{iCalendarText} when given the same arguments.
-
- Note that L{iCalendarText} by default calls this method, which creates
- an infinite loop. A subclass must override one of both of these
- methods.
- """
-
- try:
- calendar_data = self.iCalendarText(name)
- except InternalDataStoreError:
- return None
-
- if calendar_data is None: return None
-
- try:
- return iComponent.fromString(calendar_data)
- except ValueError:
- return None
-
@inlineCallbacks
def iCalendarForUser(self, request, name=None):
-
- caldata = self.iCalendar(name)
-
+ if name is not None:
+ # FIXME: this is really the caller's job; why am I looking up sub-
+ # resources?
+ returnValue(
+ (yield (yield request.locateChildResource(self, name)
+ ).iCalendarForUser(request))
+ )
+
+ caldata = self.iCalendar()
+
accessUID = (yield self.resourceOwnerPrincipal(request))
if accessUID is None:
accessUID = ""
@@ -1147,6 +1132,7 @@
returnValue(PerUserDataFilter(accessUID).filter(caldata))
+
def iCalendarAddressDoNormalization(self, ical):
"""
Normalize calendar user addresses in the supplied iCalendar object into their
@@ -1458,124 +1444,13 @@
return fail(NotImplementedError())
- def createSpecialCollection(self, resourceType=None):
- #
- # Create the collection once we know it is safe to do so
- #
- def onCollection(status):
- if status != responsecode.CREATED:
- raise HTTPError(status)
+ def iCalendarRolledup(self):
+ """
+ Only implemented by calendar collections; see storebridge.
+ """
+
- self.writeDeadProperty(resourceType)
- return status
- def onError(f):
- try:
- rmdir(self.fp)
- except Exception, e:
- log.err("Unable to clean up after failed MKCOL (special resource type: %s): %s" % (e, resourceType,))
- return f
-
- d = mkcollection(self.fp)
- if resourceType is not None:
- d.addCallback(onCollection)
- d.addErrback(onError)
- return d
-
- @inlineCallbacks
- def iCalendarRolledup(self, request):
- if self.isPseudoCalendarCollection():
-
-
-# FIXME: move cache implementation!
- # Determine the cache key
-# isvirt = self.isVirtualShare()
-# if isvirt:
-# principal = (yield self.resourceOwnerPrincipal(request))
-# if principal:
-# cacheKey = principal.principalUID()
-# else:
-# cacheKey = "unknown"
-# else:
-# isowner = (yield self.isOwner(request, adminprincipals=True, readprincipals=True))
-# cacheKey = "owner" if isowner else "notowner"
-
- # Now check for a cached .ics
-# rolled = self.fp.child(".subscriptions")
-# if not rolled.exists():
-# try:
-# rolled.makedirs()
-# except IOError, e:
-# self.log_error("Unable to create internet calendar subscription cache directory: %s because of: %s" % (rolled.path, e,))
-# raise HTTPError(ErrorResponse(responsecode.INTERNAL_SERVER_ERROR))
-# cached = rolled.child(cacheKey)
-# if cached.exists():
-# try:
-# cachedData = cached.open().read()
-# except IOError, e:
-# self.log_error("Unable to open or read internet calendar subscription cache file: %s because of: %s" % (cached.path, e,))
-# else:
-# # Check the cache token
-# token, data = cachedData.split("\r\n", 1)
-# if token == self.getSyncToken():
-# returnValue(data)
-
- # Generate a monolithic calendar
- calendar = iComponent("VCALENDAR")
- calendar.addProperty(iProperty("VERSION", "2.0"))
-
- # Do some optimisation of access control calculation by determining any inherited ACLs outside of
- # the child resource loop and supply those to the checkPrivileges on each child.
- filteredaces = (yield self.inheritedACEsforChildren(request))
-
- tzids = set()
- isowner = (yield self.isOwner(request, adminprincipals=True, readprincipals=True))
- accessPrincipal = (yield self.resourceOwnerPrincipal(request))
-
- for name, uid, type in self.index().bruteForceSearch(): #@UnusedVariable
- try:
- child = yield request.locateChildResource(self, name)
- except TypeError:
- child = None
-
- if child is not None:
- # Check privileges of child - skip if access denied
- try:
- yield child.checkPrivileges(request, (davxml.Read(),), inherited_aces=filteredaces)
- except AccessDeniedError:
- continue
-
- # Get the access filtered view of the data
- caldata = child.iCalendarTextFiltered(isowner, accessPrincipal.principalUID() if accessPrincipal else "")
- try:
- subcalendar = iComponent.fromString(caldata)
- except ValueError:
- continue
- assert subcalendar.name() == "VCALENDAR"
-
- for component in subcalendar.subcomponents():
-
- # Only insert VTIMEZONEs once
- if component.name() == "VTIMEZONE":
- tzid = component.propertyValue("TZID")
- if tzid in tzids:
- continue
- tzids.add(tzid)
-
- calendar.addComponent(component)
-
- # Cache the data
- data = str(calendar)
- data = self.getSyncToken() + "\r\n" + data
-# try:
-# cached.open(mode='w').write(data)
-# except IOError, e:
-# self.log_error("Unable to open or write internet calendar subscription cache file: %s because of: %s" % (cached.path, e,))
-
- returnValue(calendar)
-
- raise HTTPError(ErrorResponse(responsecode.BAD_REQUEST))
-
def iCalendarTextFiltered(self, isowner, accessUID=None):
try:
access = self.readDeadProperty(TwistedCalendarAccessProperty)
@@ -1588,23 +1463,16 @@
caldata = PerUserDataFilter(accessUID).filter(caldata)
return str(caldata)
- def iCalendarText(self, name=None):
- if self.isPseudoCalendarCollection():
- if name is None:
- return str(self.iCalendar())
- calendar_resource = self.getChild(name)
- return calendar_resource.iCalendarText()
+ def iCalendarText(self):
+ # storebridge handles this method
+ raise NotImplementedError()
- elif self.isCollection():
- return None
- else:
- if name is not None:
- raise AssertionError("name must be None for non-collection calendar resource")
+ def iCalendar(self):
+ # storebridge handles this method
+ raise NotImplementedError()
- # FIXME: StoreBridge handles this case
- raise NotImplementedError
def createAddressBook(self, request):
"""
@@ -1704,18 +1572,6 @@
return super(CalDAVResource, self).supportedPrivileges(request)
- def index(self):
- """
- Obtains the index for a calendar collection resource.
- @return: the index object for this resource.
- @raise AssertionError: if this resource is not a calendar collection
- resource.
- """
- if self.isAddressBookCollection():
- return AddressBookIndex(self)
- else:
- return Index(self)
-
##
# Quota
##
@@ -1828,9 +1684,6 @@
def isCalendarCollection(self):
return False
- def isPseudoCalendarCollection(self):
- return False
-
def isAddressBookCollection(self):
return False
@@ -2096,35 +1949,51 @@
class CommonHomeResource(SharedHomeMixin, CalDAVResource):
"""
- Calendar home collection resource.
+ Logic common to Calendar and Addressbook home resources.
"""
- def __init__(self, parent, name, transaction):
- """
- """
-
+ def __init__(self, parent, name, transaction, home):
self.parent = parent
self.name = name
self.associateWithTransaction(transaction)
self._provisionedChildren = {}
self._provisionedLinks = {}
self._setupProvisions()
-
- self._newStoreHome, created = self.makeNewStore()
+ self._newStoreHome = home
CalDAVResource.__init__(self)
from twistedcaldav.storebridge import _NewStorePropertiesWrapper
self._dead_properties = _NewStorePropertiesWrapper(
self._newStoreHome.properties()
)
+
+
+ @classmethod
+ @inlineCallbacks
+ def createHomeResource(cls, parent, name, transaction):
+ home, created = yield cls.homeFromTransaction(
+ transaction, name)
+ resource = cls(parent, name, transaction, home)
if created:
- self.postCreateHome()
+ yield resource.postCreateHome()
+ returnValue(resource)
+
+ @classmethod
+ def homeFromTransaction(cls, transaction, uid):
+ """
+ Create or retrieve an appropriate back-end-home object from a
+ transaction and a home UID.
+
+ @return: a L{Deferred} which fires a 2-tuple of C{(created, home)}
+ where C{created} is a boolean indicating whether this call created
+ the home in the back-end, and C{home} is the home object itself.
+ """
+ raise NotImplementedError("Subclasses must implement.")
+
+
def _setupProvisions(self):
pass
- def makeNewStore(self):
- raise NotImplementedError
-
def postCreateHome(self):
pass
@@ -2184,33 +2053,34 @@
def canShare(self):
raise NotImplementedError
+
+ @inlineCallbacks
def makeChild(self, name):
-
# Try built-in children first
if name in self._provisionedChildren:
cls = self._provisionedChildren[name]
from twistedcaldav.notifications import NotificationCollectionResource
if cls is NotificationCollectionResource:
- return self.createNotificationsCollection()
- child = self._provisionedChildren[name](self)
+ returnValue((yield self.createNotificationsCollection()))
+ child = yield self._provisionedChildren[name](self)
self.propagateTransaction(child)
self.putChild(name, child)
- return child
-
+ returnValue(child)
+
# Try built-in links next
if name in self._provisionedLinks:
child = LinkResource(self, self._provisionedLinks[name])
self.putChild(name, child)
- return child
-
+ returnValue(child)
+
# Try shares next
if self.canShare():
- child = self.provisionShare(name)
+ child = yield self.provisionShare(name)
if child:
- return child
+ returnValue(child)
# Do normal child types
- return self.makeRegularChild(name)
+ returnValue((yield self.makeRegularChild(name)))
def createNotificationsCollection(self):
@@ -2386,14 +2256,26 @@
class CalendarHomeResource(CommonHomeResource):
"""
- Calendar home collection resource.
+ Calendar home collection classmethod.
"""
+ @classmethod
+ @inlineCallbacks
+ def homeFromTransaction(cls, transaction, uid):
+ storeHome = yield transaction.calendarHomeWithUID(uid)
+ if storeHome is not None:
+ created = False
+ else:
+ storeHome = yield transaction.calendarHomeWithUID(uid, create=True)
+ created = True
+ returnValue((storeHome, created))
+
+
def _setupProvisions(self):
# Cache children which must be of a specific type
from twistedcaldav.storebridge import StoreScheduleInboxResource
- self._provisionedChildren["inbox"] = StoreScheduleInboxResource
+ self._provisionedChildren["inbox"] = StoreScheduleInboxResource.maybeCreateInbox
from twistedcaldav.schedule import ScheduleOutboxResource
self._provisionedChildren["outbox"] = ScheduleOutboxResource
@@ -2410,32 +2292,24 @@
from twistedcaldav.notifications import NotificationCollectionResource
self._provisionedChildren["notification"] = NotificationCollectionResource
- def makeNewStore(self):
- storeHome = self._associatedTransaction.calendarHomeWithUID(self.name)
- if storeHome is not None:
- created = False
- else:
- storeHome = self._associatedTransaction.calendarHomeWithUID(
- self.name, create=True
- )
- created = True
- return storeHome, created
-
+ @inlineCallbacks
def postCreateHome(self):
# This is a bit of a hack. Really we ought to be always generating
# this URL live from a back-end method that tells us what the
# default calendar is.
- inbox = self.getChild("inbox")
+ inbox = yield self.getChild("inbox")
childURL = joinURL(self.url(), "calendar")
inbox.processFreeBusyCalendar(childURL, True)
+
def canShare(self):
return config.Sharing.Enabled and config.Sharing.Calendars.Enabled and self.exists()
- def makeRegularChild(self, name):
- newCalendar = self._newStoreHome.calendarWithName(name)
+ @inlineCallbacks
+ def makeRegularChild(self, name):
+ newCalendar = yield self._newStoreHome.calendarWithName(name)
if newCalendar is None:
# Local imports.due to circular dependency between modules.
from twistedcaldav.storebridge import (
@@ -2452,8 +2326,9 @@
principalCollections=self.principalCollections()
)
self.propagateTransaction(similar)
- return similar
+ returnValue(similar)
+
def defaultAccessControlList(self):
myPrincipal = self.principalForRecord()
@@ -2510,7 +2385,19 @@
"""
Address book home collection resource.
"""
-
+
+ @classmethod
+ @inlineCallbacks
+ def homeFromTransaction(cls, transaction, uid):
+ storeHome = yield transaction.addressbookHomeWithUID(uid)
+ if storeHome is not None:
+ created = False
+ else:
+ storeHome = yield transaction.addressbookHomeWithUID(uid, create=True)
+ created = True
+ returnValue((storeHome, created))
+
+
def _setupProvisions(self):
# Cache children which must be of a specific type
@@ -2527,6 +2414,7 @@
def canShare(self):
return config.Sharing.Enabled and config.Sharing.AddressBooks.Enabled and self.exists()
+ @inlineCallbacks
def makeRegularChild(self, name):
# Check for public/global path
@@ -2543,7 +2431,7 @@
mainCls = GlobalAddressBookCollectionResource
protoCls = ProtoGlobalAddressBookCollectionResource
- newAddressBook = self._newStoreHome.addressbookWithName(name)
+ newAddressBook = yield self._newStoreHome.addressbookWithName(name)
if newAddressBook is None:
# Local imports.due to circular dependency between modules.
similar = protoCls(
@@ -2557,7 +2445,7 @@
principalCollections=self.principalCollections()
)
self.propagateTransaction(similar)
- return similar
+ returnValue(similar)
class GlobalAddressBookResource (ReadOnlyResourceMixIn, CalDAVResource):
@@ -2566,7 +2454,7 @@
"""
def resourceType(self):
- return davxml.ResourceType.sharedaddressbook
+ return davxml.ResourceType.sharedaddressbook #@UndefinedVariable
def defaultAccessControlList(self):
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/scheduling/implicit.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/scheduling/implicit.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/scheduling/implicit.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -383,7 +383,7 @@
# Get owner's calendar-home
calendar_owner_principal = (yield self.resource.resourceOwnerPrincipal(self.request))
- calendar_home = calendar_owner_principal.calendarHome(self.request)
+ calendar_home = yield calendar_owner_principal.calendarHome(self.request)
check_parent_uri = parentForURL(check_uri)[:-1] if check_uri else None
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/scheduling/processing.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/scheduling/processing.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/scheduling/processing.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -20,6 +20,7 @@
from vobject.icalendar import dateTimeToString, utc
+from twisted.python.log import err as log_traceback
from twext.python.log import Logger
from twisted.internet import reactor
@@ -94,6 +95,7 @@
# We attempt to recover from this. That involves trying to re-write the attendee data
# to match that of the organizer assuming we have the organizer's full data available, then
# we try the processing operation again.
+ log_traceback()
log.error("ImplicitProcessing - originator '%s' to recipient '%s' with UID: '%s' - exception raised will try to fix: %s" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid, e))
result = (yield self.doImplicitAttendeeEventFix(e))
if result:
@@ -101,6 +103,7 @@
try:
result = (yield self.doImplicitAttendee())
except Exception, e:
+ log_traceback()
log.error("ImplicitProcessing - originator '%s' to recipient '%s' with UID: '%s' - exception raised after fix: %s" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid, e))
raise ImplicitProcessorException("5.1;Service unavailable")
else:
@@ -422,7 +425,7 @@
@param calendar: calendar data to examine
@type calendar: L{Component}
-
+
@return: L{Component} for the new calendar data to write
"""
@@ -441,7 +444,7 @@
# Just try again to get the lock
reactor.callLater(2.0, self.sendAttendeeAutoReply, *(calendar, resource, partstat))
else:
- txn = resource.inNewTransaction(self.request)
+ txn = yield resource.inNewTransaction(self.request)
try:
# Send out a reply
log.debug("ImplicitProcessing - recipient '%s' processing UID: '%s' - auto-reply: %s" % (self.recipient.cuaddr, self.uid, partstat))
@@ -449,9 +452,9 @@
scheduler = ImplicitScheduler()
yield scheduler.sendAttendeeReply(self.request, resource, calendar, self.recipient)
except:
- txn.abort()
+ yield txn.abort()
else:
- txn.commit()
+ yield txn.commit()
finally:
yield lock.clean()
@@ -647,6 +650,7 @@
returnValue(newchild)
+
@inlineCallbacks
def deleteCalendarResource(self, collURL, collection, name):
"""
@@ -659,7 +663,7 @@
@param name: the resource name to write into, or {None} to write a new resource.
@type name: C{str}
"""
- delchild = collection.getChild(name)
+ delchild = yield collection.getChild(name)
childURL = joinURL(collURL, name)
self.request._rememberResource(delchild, childURL)
yield delchild.storeRemove(self.request, False, childURL)
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/scheduling/utils.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/scheduling/utils.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/scheduling/utils.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -14,7 +14,7 @@
# limitations under the License.
##
-from twisted.internet.defer import inlineCallbacks, returnValue, succeed
+from twisted.internet.defer import inlineCallbacks, returnValue
from twistedcaldav.method import report_common
from twext.web2.dav.util import joinURL
@@ -23,7 +23,7 @@
"""
Get a copy of the event for a principal.
"""
-
+
result = {
"resource": None,
"resource_name": None,
@@ -33,33 +33,38 @@
if principal and principal.locallyHosted():
# Get principal's calendar-home
- calendar_home = principal.calendarHome(request)
-
- # FIXME: because of the URL->resource request mapping thing, we have to force the request
- # to recognize this resource
+ calendar_home = yield principal.calendarHome(request)
+
+ # FIXME: because of the URL->resource request mapping thing, we have to
+ # force the request to recognize this resource.
request._rememberResource(calendar_home, calendar_home.url())
- # Run a UID query against the UID
+ # Run a UID query against the UID.
+ @inlineCallbacks
def queryCalendarCollection(collection, uri):
if not allow_shared:
if collection.isVirtualShare():
- return succeed(True)
+ returnValue(True)
- rname = collection.index().resourceNameForUID(uid)
+ rname = yield collection.index().resourceNameForUID(uid)
if rname:
- resource = collection.getChild(rname)
+ resource = yield collection.getChild(rname)
request._rememberResource(resource, joinURL(uri, rname))
result["resource"] = resource
result["resource_name"] = rname
result["calendar_collection"] = collection
result["calendar_collection_uri"] = uri
- return succeed(False)
+ returnValue(False)
else:
- return succeed(True)
-
- # NB We are by-passing privilege checking here. That should be OK as the data found is not
- # exposed to the user.
- yield report_common.applyToCalendarCollections(calendar_home, request, calendar_home.url(), "infinity", queryCalendarCollection, None)
+ returnValue(True)
+ # NB We are by-passing privilege checking here. That should be OK as
+ # the data found is not exposed to the user.
+ yield report_common.applyToCalendarCollections(
+ calendar_home, request, calendar_home.url(),
+ "infinity", queryCalendarCollection, None
+ )
+
returnValue((result["resource"], result["resource_name"], result["calendar_collection"], result["calendar_collection_uri"],))
+
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/sharing.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/sharing.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/sharing.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -174,9 +174,9 @@
# Get the home collection
if self.isCalendarCollection():
- home = principal.calendarHome(request)
+ home = yield principal.calendarHome(request)
elif self.isAddressBookCollection():
- home = principal.addressBookHome(request)
+ home = yield principal.addressBookHome(request)
else:
raise HTTPError(ErrorResponse(
responsecode.FORBIDDEN,
@@ -218,15 +218,16 @@
""" Return True if this is a shared calendar collection """
return hasattr(self, "_isVirtualShare")
+ @inlineCallbacks
def removeVirtualShare(self, request):
""" Return True if this is a shared calendar collection """
# Remove from sharee's calendar/address book home
if self.isCalendarCollection():
- shareeHome = self._shareePrincipal.calendarHome(request)
+ shareeHome = yield self._shareePrincipal.calendarHome(request)
elif self.isAddressBookCollection():
- shareeHome = self._shareePrincipal.addressBookHome(request)
- return shareeHome.removeShare(request, self._share)
+ shareeHome = yield self._shareePrincipal.addressBookHome(request)
+ returnValue((yield shareeHome.removeShare(request, self._share)))
def resourceType(self):
superObject = super(SharedCollectionMixin, self)
@@ -495,9 +496,9 @@
sharee = self.principalForCalendarUserAddress(record.userid)
if sharee:
if self.isCalendarCollection():
- shareeHome = sharee.calendarHome(request)
+ shareeHome = yield sharee.calendarHome(request)
elif self.isAddressBookCollection():
- shareeHome = sharee.addressBookHome(request)
+ shareeHome = yield sharee.addressBookHome(request)
yield shareeHome.removeShareByUID(request, record.inviteuid)
# If current user state is accepted then we send an invite with the new state, otherwise
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/storebridge.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/storebridge.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/storebridge.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -35,7 +35,7 @@
from twext.web2.dav import davxml
from twext.web2.dav.element.base import dav_namespace
from twext.web2.dav.http import ErrorResponse, ResponseQueue
-from twext.web2.dav.resource import TwistedACLInheritable
+from twext.web2.dav.resource import TwistedACLInheritable, AccessDeniedError
from twext.web2.dav.util import parentForURL, allDataFromStream, joinURL, \
davXMLFromStream
from twext.web2.http import HTTPError, StatusResponse, Response
@@ -250,13 +250,15 @@
self._initializeWithCalendar(calendar, home)
+ @inlineCallbacks
def makeChild(self, name):
"""
- Create a L{CalendarObjectResource} or L{ProtoCalendarObjectResource} based on a
- path object.
+ Create a L{CalendarObjectResource} or L{ProtoCalendarObjectResource}
+ based on a calendar object name.
"""
- newStoreObject = self._newStoreCalendar.calendarObjectWithName(name)
+ cal = self._newStoreCalendar
+ newStoreObject = yield cal.calendarObjectWithName(name)
if newStoreObject is not None:
similar = CalendarObjectResource(
@@ -264,20 +266,15 @@
principalCollections=self._principalCollections
)
else:
- # FIXME: creation in http_PUT should talk to a specific resource
- # type; this is the domain of StoreCalendarObjectResource.
- # similar = ProtoCalendarObjectFile(self._newStoreCalendar, path)
similar = ProtoCalendarObjectResource(
- self._newStoreCalendar,
- name,
+ cal, name,
principalCollections=self._principalCollections
)
- # FIXME: tests should be failing without this line.
- # Specifically, http_PUT won't be committing its transaction properly.
self.propagateTransaction(similar)
- return similar
+ returnValue(similar)
+
def listChildren(self):
"""
@return: a sequence of the names of all known children of this resource.
@@ -298,19 +295,26 @@
def __init__(self, *a, **kw):
super(StoreScheduleInboxResource, self).__init__(*a, **kw)
self.parent.propagateTransaction(self)
+
+
+ @classmethod
+ @inlineCallbacks
+ def maybeCreateInbox(cls, *a, **kw):
+ self = cls(*a, **kw)
home = self.parent._newStoreHome
- storage = home.calendarWithName("inbox")
+ storage = yield home.calendarWithName("inbox")
if storage is None:
# raise RuntimeError("backend should be handling this for us")
# FIXME: spurious error, sanity check, should not be needed;
# unfortunately, user09's calendar home does not have an inbox, so
# this is a temporary workaround.
home.createCalendarWithName("inbox")
- storage = home.calendarWithName("inbox")
+ storage = yield home.calendarWithName("inbox")
self._initializeWithCalendar(
storage,
self.parent._newStoreHome
)
+ returnValue(self)
def name(self):
@@ -712,6 +716,62 @@
return True
+ @inlineCallbacks
+ def iCalendarRolledup(self, request):
+ # FIXME: uncached: implement cache in the storage layer
+
+ # Generate a monolithic calendar
+ calendar = vcomponent.VComponent("VCALENDAR")
+ calendar.addProperty(vcomponent.VProperty("VERSION", "2.0"))
+
+ # Do some optimisation of access control calculation by determining any
+ # inherited ACLs outside of the child resource loop and supply those to
+ # the checkPrivileges on each child.
+ filteredaces = (yield self.inheritedACEsforChildren(request))
+
+ tzids = set()
+ isowner = (yield self.isOwner(request, adminprincipals=True, readprincipals=True))
+ accessPrincipal = (yield self.resourceOwnerPrincipal(request))
+
+ for name, uid, type in self.index().bruteForceSearch(): #@UnusedVariable
+ try:
+ child = yield request.locateChildResource(self, name)
+ except TypeError:
+ child = None
+
+ if child is not None:
+ # Check privileges of child - skip if access denied
+ try:
+ yield child.checkPrivileges(request, (davxml.Read(),), inherited_aces=filteredaces)
+ except AccessDeniedError:
+ continue
+
+ # Get the access filtered view of the data
+ caldata = child.iCalendarTextFiltered(isowner, accessPrincipal.principalUID() if accessPrincipal else "")
+ try:
+ subcalendar = vcomponent.VComponent.fromString(caldata)
+ except ValueError:
+ continue
+ assert subcalendar.name() == "VCALENDAR"
+
+ for component in subcalendar.subcomponents():
+
+ # Only insert VTIMEZONEs once
+ if component.name() == "VTIMEZONE":
+ tzid = component.propertyValue("TZID")
+ if tzid in tzids:
+ continue
+ tzids.add(tzid)
+
+ calendar.addComponent(component)
+
+ # Cache the data
+ data = str(calendar)
+ data = self.getSyncToken() + "\r\n" + data
+
+ returnValue(calendar)
+
+
@requiresPermissions(fromParent=[davxml.Unbind()])
@inlineCallbacks
def http_DELETE(self, request):
@@ -912,20 +972,19 @@
return self.createCalendarCollection()
+ @inlineCallbacks
def createCalendarCollection(self):
"""
Override C{createCalendarCollection} to actually do the work.
"""
- d = succeed(CREATED)
-
self._newStoreParentHome.createCalendarWithName(self._name)
- newStoreCalendar = self._newStoreParentHome.calendarWithName(
+ newStoreCalendar = yield self._newStoreParentHome.calendarWithName(
self._name
)
CalendarCollectionResource.transform(
self, newStoreCalendar, self._newStoreParentHome
)
- return d
+ returnValue(CREATED)
def exists(self):
@@ -967,17 +1026,17 @@
self._initializeWithObject(calendarObject)
+ @inlineCallbacks
def inNewTransaction(self, request):
"""
Implicit auto-replies need to span multiple transactions. Clean out
the given request's resource-lookup mapping, transaction, and re-look-
- up my calendar object in a new transaction.
+ up this L{CalendarObjectResource}'s calendar object in a new
+ transaction.
- @return: the new transaction so it can be committed.
+ @return: a Deferred which fires with the new transaction, so it can be
+ committed.
"""
- # FIXME: private names from 'file' implementation; maybe there should
- # be a public way to do this? or maybe we should just have a real
- # queue.
objectName = self._newStoreObject.name()
calendar = self._newStoreObject.calendar()
calendarName = calendar.name()
@@ -985,14 +1044,14 @@
homeUID = ownerHome.uid()
txn = ownerHome.transaction().store().newTransaction(
"new transaction for " + self._newStoreObject.name())
- newObject = (txn.calendarHomeWithUID(homeUID)
- .calendarWithName(calendarName)
- .calendarObjectWithName(objectName))
+ newObject = ((yield (yield (yield txn.calendarHomeWithUID(homeUID))
+ .calendarWithName(calendarName))
+ .calendarObjectWithName(objectName)))
request._newStoreTransaction = txn
request._resourcesByURL.clear()
request._urlsByResource.clear()
self._initializeWithObject(newObject)
- return txn
+ returnValue(txn)
def isCollection(self):
@@ -1009,11 +1068,14 @@
return succeed(len(self._newStoreObject.iCalendarText()))
- def iCalendarText(self, ignored=None):
- assert ignored is None, "This is a calendar object, not a calendar"
+ def iCalendarText(self):
return self._newStoreObject.iCalendarText()
+ def iCalendar(self):
+ return self._newStoreObject.component()
+
+
def text(self):
return self.iCalendarText()
@@ -1190,7 +1252,12 @@
self._newStoreParentCalendar.createCalendarObjectWithName(
self.name(), component
)
- CalendarObjectResource.transform(self, self._newStoreParentCalendar.calendarObjectWithName(self.name()))
+ CalendarObjectResource.transform(
+ self,
+ (yield self._newStoreParentCalendar.calendarObjectWithName(
+ self.name()
+ ))
+ )
returnValue(CREATED)
@@ -1518,20 +1585,19 @@
return self.createAddressBookCollection()
+ @inlineCallbacks
def createAddressBookCollection(self):
"""
Override C{createAddressBookCollection} to actually do the work.
"""
- d = succeed(CREATED)
-
self._newStoreParentHome.createAddressBookWithName(self._name)
- newStoreAddressBook = self._newStoreParentHome.addressbookWithName(
+ newStoreAddressBook = yield self._newStoreParentHome.addressbookWithName(
self._name
)
AddressBookCollectionResource.transform(
self, newStoreAddressBook, self._newStoreParentHome
)
- return d
+ returnValue(CREATED)
def exists(self):
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_addressbookmultiget.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_addressbookmultiget.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_addressbookmultiget.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -27,7 +27,11 @@
from twistedcaldav import carddavxml
from twistedcaldav import vcard
-from twistedcaldav.index import db_basename
+
+# FIXME: remove this, we should not be importing this module, we should be
+# testing the public API. See comments below about cheating.
+from txdav.carddav.datastore.index_file import db_basename
+
from twistedcaldav.config import config
from twistedcaldav.test.util import AddressBookHomeTestCase
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_addressbookquery.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_addressbookquery.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_addressbookquery.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -25,8 +25,11 @@
from twext.web2.dav.util import davXMLFromStream
from twext.web2.test.test_server import SimpleRequest
+# FIXME: remove this, we should not be importing this module, we should be
+# testing the public API. See comments below about cheating.
+from txdav.carddav.datastore.index_file import db_basename
+
from twistedcaldav import carddavxml, vcard
-from twistedcaldav.index import db_basename
from twistedcaldav.config import config
from twistedcaldav.test.util import AddressBookHomeTestCase
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_calendarquery.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_calendarquery.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_calendarquery.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -34,8 +34,7 @@
from twistedcaldav.config import config
from twistedcaldav.test.util import HomeTestCase
from twisted.internet.defer import inlineCallbacks, returnValue
-from txdav.common.datastore.test.util import buildStore
-from txdav.caldav.datastore.test.common import StubNotifierFactory
+from txdav.common.datastore.test.util import buildStore, StubNotifierFactory
@inlineCallbacks
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_extensions.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_extensions.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_extensions.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -21,11 +21,11 @@
from twext.web2.http_headers import MimeType
from twext.web2.static import MetaDataMixin
-from twisted.internet.defer import inlineCallbacks, succeed
+from twisted.internet.defer import inlineCallbacks, Deferred, succeed
from twisted.trial.unittest import TestCase
from twisted.web.microdom import parseString
-from twistedcaldav.extensions import DAVFile
+from twistedcaldav.extensions import DAVFile, DAVResourceWithChildrenMixin
from xml.etree.cElementTree import XML
@@ -184,3 +184,30 @@
yield self.doDirectoryTest([nonASCIIFilename], addUnicodeChild,
[nonASCIIFilename.encode("utf-8")])
+
+
+class ChildTraversalTests(TestCase):
+ def test_makeChildDeferred(self):
+ """
+ If L{DAVResourceWithChildrenMixin.makeChild} returns a L{Deferred},
+ L{DAVResourceWithChildrenMixin.locateChild} will return a L{Deferred}.
+ """
+ class FakeChild(object):
+ def __init__(self, name):
+ self.name = name
+ class SmellsLikeDAVResource(object):
+ def __init__(self, **kw):
+ pass
+ class ResourceWithCheese(DAVResourceWithChildrenMixin,
+ SmellsLikeDAVResource):
+ def makeChild(self, name):
+ return succeed(FakeChild(name))
+ d = ResourceWithCheese().locateChild(None, ['cheese', 'burger'])
+ self.assertIsInstance(d, Deferred)
+ x = []
+ d.addCallback(x.append)
+ self.assertEquals(len(x), 1)
+ [result] = x
+ self.assertEquals(len(result), 2)
+ self.assertEquals(result[0].name, 'cheese')
+ self.assertEquals(result[1], ['burger'])
Deleted: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_index.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_index.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_index.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -1,935 +0,0 @@
-##
-# Copyright (c) 2010 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
-
-from twisted.internet import reactor
-from twisted.internet.task import deferLater
-
-from twistedcaldav import caldavxml
-from twistedcaldav.caldavxml import TimeRange
-from twistedcaldav.ical import Component
-from twistedcaldav.index import Index
-from twistedcaldav.index import ReservationError, MemcachedUIDReserver
-from twistedcaldav.instance import InvalidOverriddenInstanceError
-from twistedcaldav.query import calendarqueryfilter
-from twistedcaldav.test.util import InMemoryMemcacheProtocol
-import twistedcaldav.test.util
-
-import datetime
-import os
-
-
-class MinimalResourceReplacement(object):
- """
- Provide the minimal set of attributes and methods from CalDAVFile required
- by L{Index}.
- """
-
- def __init__(self, filePath):
- self.fp = filePath
-
-
- def isCalendarCollection(self):
- return True
-
-
- def getChild(self, name):
- # FIXME: this should really return something with a child method
- return self.fp.child(name)
-
-
- def initSyncToken(self):
- pass
-
-
-
-class SQLIndexTests (twistedcaldav.test.util.TestCase):
- """
- Test abstract SQL DB class
- """
-
- def setUp(self):
- super(SQLIndexTests, self).setUp()
- self.site.resource.isCalendarCollection = lambda: True
- self.indexDirPath = self.site.resource.fp
- # FIXME: since this resource lies about isCalendarCollection, it doesn't
- # have all the associated backend machinery to actually get children.
- self.db = Index(MinimalResourceReplacement(self.indexDirPath))
-
-
- def test_reserve_uid_ok(self):
- uid = "test-test-test"
- d = self.db.isReservedUID(uid)
- d.addCallback(self.assertFalse)
- d.addCallback(lambda _: self.db.reserveUID(uid))
- d.addCallback(lambda _: self.db.isReservedUID(uid))
- d.addCallback(self.assertTrue)
- d.addCallback(lambda _: self.db.unreserveUID(uid))
- d.addCallback(lambda _: self.db.isReservedUID(uid))
- d.addCallback(self.assertFalse)
-
- return d
-
-
- def test_reserve_uid_twice(self):
- uid = "test-test-test"
- d = self.db.reserveUID(uid)
- d.addCallback(lambda _: self.db.isReservedUID(uid))
- d.addCallback(self.assertTrue)
- d.addCallback(lambda _:
- self.assertFailure(self.db.reserveUID(uid),
- ReservationError))
- return d
-
-
- def test_unreserve_unreserved(self):
- uid = "test-test-test"
- return self.assertFailure(self.db.unreserveUID(uid),
- ReservationError)
-
-
- def test_reserve_uid_timeout(self):
- # WARNING: This test is fundamentally flawed and will fail
- # intermittently because it uses the real clock.
- uid = "test-test-test"
- from twistedcaldav.config import config
- old_timeout = config.UIDReservationTimeOut
- config.UIDReservationTimeOut = 1
-
- def _finally():
- config.UIDReservationTimeOut = old_timeout
-
- d = self.db.isReservedUID(uid)
- d.addCallback(self.assertFalse)
- d.addCallback(lambda _: self.db.reserveUID(uid))
- d.addCallback(lambda _: self.db.isReservedUID(uid))
- d.addCallback(self.assertTrue)
- d.addCallback(lambda _: deferLater(reactor, 2, lambda: None))
- d.addCallback(lambda _: self.db.isReservedUID(uid))
- d.addCallback(self.assertFalse)
- self.addCleanup(_finally)
-
- return d
-
-
- def test_index(self):
- data = (
- (
- "#1.1 Simple component",
- "1.1",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-1.1
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-END:VEVENT
-END:VCALENDAR
-""",
- False,
- True,
- ),
- (
- "#2.1 Recurring component",
- "2.1",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-2.1
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-RRULE:FREQ=WEEKLY;COUNT=2
-END:VEVENT
-END:VCALENDAR
-""",
- False,
- True,
- ),
- (
- "#2.2 Recurring component with override",
- "2.2",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-2.2
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-RRULE:FREQ=WEEKLY;COUNT=2
-END:VEVENT
-BEGIN:VEVENT
-UID:12345-67890-2.2
-RECURRENCE-ID:20080608T120000Z
-DTSTART:20080608T120000Z
-DTEND:20080608T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-END:VEVENT
-END:VCALENDAR
-""",
- False,
- True,
- ),
- (
- "#2.3 Recurring component with broken override - new",
- "2.3",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-2.3
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-RRULE:FREQ=WEEKLY;COUNT=2
-END:VEVENT
-BEGIN:VEVENT
-UID:12345-67890-2.3
-RECURRENCE-ID:20080609T120000Z
-DTSTART:20080608T120000Z
-DTEND:20080608T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-END:VEVENT
-END:VCALENDAR
-""",
- False,
- False,
- ),
- (
- "#2.4 Recurring component with broken override - existing",
- "2.4",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-2.4
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-RRULE:FREQ=WEEKLY;COUNT=2
-END:VEVENT
-BEGIN:VEVENT
-UID:12345-67890-2.4
-RECURRENCE-ID:20080609T120000Z
-DTSTART:20080608T120000Z
-DTEND:20080608T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-END:VEVENT
-END:VCALENDAR
-""",
- True,
- True,
- ),
- )
-
- for description, name, calendar_txt, reCreate, ok in data:
- calendar = Component.fromString(calendar_txt)
- if ok:
- f = open(os.path.join(self.indexDirPath.path, name), "w")
- f.write(calendar_txt)
- del f
-
- self.db.addResource(name, calendar, reCreate=reCreate)
- self.assertTrue(self.db.resourceExists(name), msg=description)
- else:
- self.assertRaises(InvalidOverriddenInstanceError, self.db.addResource, name, calendar)
- self.assertFalse(self.db.resourceExists(name), msg=description)
-
- self.db._db_recreate()
- for description, name, calendar_txt, reCreate, ok in data:
- if ok:
- self.assertTrue(self.db.resourceExists(name), msg=description)
- else:
- self.assertFalse(self.db.resourceExists(name), msg=description)
-
- self.db.testAndUpdateIndex(datetime.date(2020, 1, 1))
- for description, name, calendar_txt, reCreate, ok in data:
- if ok:
- self.assertTrue(self.db.resourceExists(name), msg=description)
- else:
- self.assertFalse(self.db.resourceExists(name), msg=description)
-
- def test_index_timespan(self):
- data = (
- (
- "#1.1 Simple component - busy",
- "1.1",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-1.1
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-END:VEVENT
-END:VCALENDAR
-""",
- "20080601T000000Z", "20080602T000000Z",
- "mailto:user1 at example.com",
- (('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),),
- ),
- (
- "#1.2 Simple component - transparent",
- "1.2",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-1.2
-DTSTART:20080602T120000Z
-DTEND:20080602T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-TRANSP:TRANSPARENT
-END:VEVENT
-END:VCALENDAR
-""",
- "20080602T000000Z", "20080603T000000Z",
- "mailto:user1 at example.com",
- (('N', "2008-06-02 12:00:00+00:00", "2008-06-02 13:00:00+00:00", 'B', 'T'),),
- ),
- (
- "#1.3 Simple component - canceled",
- "1.3",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-1.3
-DTSTART:20080603T120000Z
-DTEND:20080603T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-STATUS:CANCELLED
-END:VEVENT
-END:VCALENDAR
-""",
- "20080603T000000Z", "20080604T000000Z",
- "mailto:user1 at example.com",
- (('N', "2008-06-03 12:00:00+00:00", "2008-06-03 13:00:00+00:00", 'F', 'F'),),
- ),
- (
- "#1.4 Simple component - tentative",
- "1.4",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-1.4
-DTSTART:20080604T120000Z
-DTEND:20080604T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-STATUS:TENTATIVE
-END:VEVENT
-END:VCALENDAR
-""",
- "20080604T000000Z", "20080605T000000Z",
- "mailto:user1 at example.com",
- (('N', "2008-06-04 12:00:00+00:00", "2008-06-04 13:00:00+00:00", 'T', 'F'),),
- ),
- (
- "#2.1 Recurring component - busy",
- "2.1",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-2.1
-DTSTART:20080605T120000Z
-DTEND:20080605T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-RRULE:FREQ=DAILY;COUNT=2
-END:VEVENT
-END:VCALENDAR
-""",
- "20080605T000000Z", "20080607T000000Z",
- "mailto:user1 at example.com",
- (
- ('N', "2008-06-05 12:00:00+00:00", "2008-06-05 13:00:00+00:00", 'B', 'F'),
- ('N', "2008-06-06 12:00:00+00:00", "2008-06-06 13:00:00+00:00", 'B', 'F'),
- ),
- ),
- (
- "#2.2 Recurring component - busy",
- "2.2",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-2.2
-DTSTART:20080607T120000Z
-DTEND:20080607T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-RRULE:FREQ=DAILY;COUNT=2
-END:VEVENT
-BEGIN:VEVENT
-UID:12345-67890-2.2
-RECURRENCE-ID:20080608T120000Z
-DTSTART:20080608T140000Z
-DTEND:20080608T150000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-TRANSP:TRANSPARENT
-END:VEVENT
-END:VCALENDAR
-""",
- "20080607T000000Z", "20080609T000000Z",
- "mailto:user1 at example.com",
- (
- ('N', "2008-06-07 12:00:00+00:00", "2008-06-07 13:00:00+00:00", 'B', 'F'),
- ('N', "2008-06-08 14:00:00+00:00", "2008-06-08 15:00:00+00:00", 'B', 'T'),
- ),
- ),
- )
-
- for description, name, calendar_txt, trstart, trend, organizer, instances in data:
- calendar = Component.fromString(calendar_txt)
-
- f = open(os.path.join(self.indexDirPath.path, name), "w")
- f.write(calendar_txt)
- del f
-
- self.db.addResource(name, calendar)
- self.assertTrue(self.db.resourceExists(name), msg=description)
-
- # Create fake filter element to match time-range
- filter = caldavxml.Filter(
- caldavxml.ComponentFilter(
- caldavxml.ComponentFilter(
- TimeRange(
- start=trstart,
- end=trend,
- ),
- name=("VEVENT", "VFREEBUSY", "VAVAILABILITY"),
- ),
- name="VCALENDAR",
- )
- )
- filter = calendarqueryfilter.Filter(filter)
-
- resources = self.db.indexedSearch(filter, fbtype=True)
- index_results = set()
- for _ignore_name, _ignore_uid, type, test_organizer, float, start, end, fbtype, transp in resources:
- self.assertEqual(test_organizer, organizer, msg=description)
- index_results.add((float, start, end, fbtype, transp,))
-
- self.assertEqual(set(instances), index_results, msg=description)
-
- def test_index_timespan_per_user(self):
- data = (
- (
- "#1.1 Single per-user non-recurring component",
- "1.1",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-1.1
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-END:VEVENT
-BEGIN:X-CALENDARSERVER-PERUSER
-UID:12345-67890-1.1
-X-CALENDARSERVER-PERUSER-UID:user01
-BEGIN:X-CALENDARSERVER-PERINSTANCE
-BEGIN:VALARM
-ACTION:DISPLAY
-DESCRIPTION:Test
-TRIGGER;RELATED=START:-PT10M
-END:VALARM
-TRANSP:TRANSPARENT
-END:X-CALENDARSERVER-PERINSTANCE
-END:X-CALENDARSERVER-PERUSER
-END:VCALENDAR
-""",
- "20080601T000000Z", "20080602T000000Z",
- "mailto:user1 at example.com",
- (
- (
- "user01",
- (('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'T'),),
- ),
- (
- "user02",
- (('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),),
- ),
- ),
- ),
- (
- "#1.2 Two per-user non-recurring component",
- "1.2",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-1.2
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-END:VEVENT
-BEGIN:X-CALENDARSERVER-PERUSER
-UID:12345-67890-1.2
-X-CALENDARSERVER-PERUSER-UID:user01
-BEGIN:X-CALENDARSERVER-PERINSTANCE
-BEGIN:VALARM
-ACTION:DISPLAY
-DESCRIPTION:Test
-TRIGGER;RELATED=START:-PT10M
-END:VALARM
-TRANSP:TRANSPARENT
-END:X-CALENDARSERVER-PERINSTANCE
-END:X-CALENDARSERVER-PERUSER
-BEGIN:X-CALENDARSERVER-PERUSER
-UID:12345-67890-1.2
-X-CALENDARSERVER-PERUSER-UID:user02
-BEGIN:X-CALENDARSERVER-PERINSTANCE
-BEGIN:VALARM
-ACTION:DISPLAY
-DESCRIPTION:Test
-TRIGGER;RELATED=START:-PT10M
-END:VALARM
-END:X-CALENDARSERVER-PERINSTANCE
-END:X-CALENDARSERVER-PERUSER
-END:VCALENDAR
-""",
- "20080601T000000Z", "20080602T000000Z",
- "mailto:user1 at example.com",
- (
- (
- "user01",
- (('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'T'),),
- ),
- (
- "user02",
- (('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),),
- ),
- (
- "user03",
- (('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),),
- ),
- ),
- ),
- (
- "#2.1 Single per-user simple recurring component",
- "2.1",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-1.1
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-RRULE:FREQ=DAILY;COUNT=10
-END:VEVENT
-BEGIN:X-CALENDARSERVER-PERUSER
-UID:12345-67890-1.1
-X-CALENDARSERVER-PERUSER-UID:user01
-BEGIN:X-CALENDARSERVER-PERINSTANCE
-BEGIN:VALARM
-ACTION:DISPLAY
-DESCRIPTION:Test
-TRIGGER;RELATED=START:-PT10M
-END:VALARM
-TRANSP:TRANSPARENT
-END:X-CALENDARSERVER-PERINSTANCE
-END:X-CALENDARSERVER-PERUSER
-END:VCALENDAR
-""",
- "20080601T000000Z", "20080603T000000Z",
- "mailto:user1 at example.com",
- (
- (
- "user01",
- (
- ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'T'),
- ('N', "2008-06-02 12:00:00+00:00", "2008-06-02 13:00:00+00:00", 'B', 'T'),
- ),
- ),
- (
- "user02",
- (
- ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),
- ('N', "2008-06-02 12:00:00+00:00", "2008-06-02 13:00:00+00:00", 'B', 'F'),
- ),
- ),
- ),
- ),
- (
- "#2.2 Two per-user simple recurring component",
- "2.2",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-1.2
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-RRULE:FREQ=DAILY;COUNT=10
-END:VEVENT
-BEGIN:X-CALENDARSERVER-PERUSER
-UID:12345-67890-1.2
-X-CALENDARSERVER-PERUSER-UID:user01
-BEGIN:X-CALENDARSERVER-PERINSTANCE
-BEGIN:VALARM
-ACTION:DISPLAY
-DESCRIPTION:Test
-TRIGGER;RELATED=START:-PT10M
-END:VALARM
-TRANSP:TRANSPARENT
-END:X-CALENDARSERVER-PERINSTANCE
-END:X-CALENDARSERVER-PERUSER
-BEGIN:X-CALENDARSERVER-PERUSER
-UID:12345-67890-1.2
-X-CALENDARSERVER-PERUSER-UID:user02
-BEGIN:X-CALENDARSERVER-PERINSTANCE
-BEGIN:VALARM
-ACTION:DISPLAY
-DESCRIPTION:Test
-TRIGGER;RELATED=START:-PT10M
-END:VALARM
-END:X-CALENDARSERVER-PERINSTANCE
-END:X-CALENDARSERVER-PERUSER
-END:VCALENDAR
-""",
- "20080601T000000Z", "20080603T000000Z",
- "mailto:user1 at example.com",
- (
- (
- "user01",
- (
- ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'T'),
- ('N', "2008-06-02 12:00:00+00:00", "2008-06-02 13:00:00+00:00", 'B', 'T'),
- ),
- ),
- (
- "user02",
- (
- ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),
- ('N', "2008-06-02 12:00:00+00:00", "2008-06-02 13:00:00+00:00", 'B', 'F'),
- ),
- ),
- (
- "user03",
- (
- ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),
- ('N', "2008-06-02 12:00:00+00:00", "2008-06-02 13:00:00+00:00", 'B', 'F'),
- ),
- ),
- ),
- ),
- (
- "#3.1 Single per-user complex recurring component",
- "3.1",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-1.1
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-RRULE:FREQ=DAILY;COUNT=10
-END:VEVENT
-BEGIN:VEVENT
-UID:12345-67890-1.1
-RECURRENCE-ID:20080602T120000Z
-DTSTART:20080602T130000Z
-DTEND:20080602T140000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-END:VEVENT
-BEGIN:X-CALENDARSERVER-PERUSER
-UID:12345-67890-1.1
-X-CALENDARSERVER-PERUSER-UID:user01
-BEGIN:X-CALENDARSERVER-PERINSTANCE
-BEGIN:VALARM
-ACTION:DISPLAY
-DESCRIPTION:Test
-TRIGGER;RELATED=START:-PT10M
-END:VALARM
-TRANSP:TRANSPARENT
-END:X-CALENDARSERVER-PERINSTANCE
-BEGIN:X-CALENDARSERVER-PERINSTANCE
-RECURRENCE-ID:20080602T120000Z
-TRANSP:OPAQUE
-END:X-CALENDARSERVER-PERINSTANCE
-END:X-CALENDARSERVER-PERUSER
-END:VCALENDAR
-""",
- "20080601T000000Z", "20080604T000000Z",
- "mailto:user1 at example.com",
- (
- (
- "user01",
- (
- ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'T'),
- ('N', "2008-06-02 13:00:00+00:00", "2008-06-02 14:00:00+00:00", 'B', 'F'),
- ('N', "2008-06-03 12:00:00+00:00", "2008-06-03 13:00:00+00:00", 'B', 'T'),
- ),
- ),
- (
- "user02",
- (
- ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),
- ('N', "2008-06-02 13:00:00+00:00", "2008-06-02 14:00:00+00:00", 'B', 'F'),
- ('N', "2008-06-03 12:00:00+00:00", "2008-06-03 13:00:00+00:00", 'B', 'F'),
- ),
- ),
- ),
- ),
- (
- "#3.2 Two per-user complex recurring component",
- "3.2",
- """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-1.2
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-RRULE:FREQ=DAILY;COUNT=10
-END:VEVENT
-BEGIN:VEVENT
-UID:12345-67890-1.2
-RECURRENCE-ID:20080602T120000Z
-DTSTART:20080602T130000Z
-DTEND:20080602T140000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-END:VEVENT
-BEGIN:X-CALENDARSERVER-PERUSER
-UID:12345-67890-1.2
-X-CALENDARSERVER-PERUSER-UID:user01
-BEGIN:X-CALENDARSERVER-PERINSTANCE
-BEGIN:VALARM
-ACTION:DISPLAY
-DESCRIPTION:Test
-TRIGGER;RELATED=START:-PT10M
-END:VALARM
-TRANSP:TRANSPARENT
-END:X-CALENDARSERVER-PERINSTANCE
-BEGIN:X-CALENDARSERVER-PERINSTANCE
-RECURRENCE-ID:20080602T120000Z
-TRANSP:OPAQUE
-END:X-CALENDARSERVER-PERINSTANCE
-END:X-CALENDARSERVER-PERUSER
-BEGIN:X-CALENDARSERVER-PERUSER
-UID:12345-67890-1.2
-X-CALENDARSERVER-PERUSER-UID:user02
-BEGIN:X-CALENDARSERVER-PERINSTANCE
-BEGIN:VALARM
-ACTION:DISPLAY
-DESCRIPTION:Test
-TRIGGER;RELATED=START:-PT10M
-END:VALARM
-END:X-CALENDARSERVER-PERINSTANCE
-BEGIN:X-CALENDARSERVER-PERINSTANCE
-RECURRENCE-ID:20080603T120000Z
-TRANSP:TRANSPARENT
-END:X-CALENDARSERVER-PERINSTANCE
-END:X-CALENDARSERVER-PERUSER
-END:VCALENDAR
-""",
- "20080601T000000Z", "20080604T000000Z",
- "mailto:user1 at example.com",
- (
- (
- "user01",
- (
- ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'T'),
- ('N', "2008-06-02 13:00:00+00:00", "2008-06-02 14:00:00+00:00", 'B', 'F'),
- ('N', "2008-06-03 12:00:00+00:00", "2008-06-03 13:00:00+00:00", 'B', 'T'),
- ),
- ),
- (
- "user02",
- (
- ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),
- ('N', "2008-06-02 13:00:00+00:00", "2008-06-02 14:00:00+00:00", 'B', 'F'),
- ('N', "2008-06-03 12:00:00+00:00", "2008-06-03 13:00:00+00:00", 'B', 'T'),
- ),
- ),
- (
- "user03",
- (
- ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),
- ('N', "2008-06-02 13:00:00+00:00", "2008-06-02 14:00:00+00:00", 'B', 'F'),
- ('N', "2008-06-03 12:00:00+00:00", "2008-06-03 13:00:00+00:00", 'B', 'F'),
- ),
- ),
- ),
- ),
- )
-
- for description, name, calendar_txt, trstart, trend, organizer, peruserinstances in data:
- calendar = Component.fromString(calendar_txt)
-
- f = open(os.path.join(self.indexDirPath.path, name), "w")
- f.write(calendar_txt)
- del f
-
- self.db.addResource(name, calendar)
- self.assertTrue(self.db.resourceExists(name), msg=description)
-
- # Create fake filter element to match time-range
- filter = caldavxml.Filter(
- caldavxml.ComponentFilter(
- caldavxml.ComponentFilter(
- TimeRange(
- start=trstart,
- end=trend,
- ),
- name=("VEVENT", "VFREEBUSY", "VAVAILABILITY"),
- ),
- name="VCALENDAR",
- )
- )
- filter = calendarqueryfilter.Filter(filter)
-
- for useruid, instances in peruserinstances:
- resources = self.db.indexedSearch(filter, useruid=useruid, fbtype=True)
- index_results = set()
- for _ignore_name, _ignore_uid, type, test_organizer, float, start, end, fbtype, transp in resources:
- self.assertEqual(test_organizer, organizer, msg=description)
- index_results.add((str(float), str(start), str(end), str(fbtype), str(transp),))
-
- self.assertEqual(set(instances), index_results, msg="%s, user:%s" % (description, useruid,))
-
- self.db.deleteResource(name)
-
- def test_index_revisions(self):
- data1 = """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-1.1
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-END:VEVENT
-END:VCALENDAR
-"""
- data2 = """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-2.1
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-RRULE:FREQ=WEEKLY;COUNT=2
-END:VEVENT
-END:VCALENDAR
-"""
- data3 = """BEGIN:VCALENDAR
-VERSION:2.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-BEGIN:VEVENT
-UID:12345-67890-2.3
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ORGANIZER;CN="User 01":mailto:user1 at example.com
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE:mailto:user2 at example.com
-RRULE:FREQ=WEEKLY;COUNT=2
-END:VEVENT
-END:VCALENDAR
-"""
-
- calendar = Component.fromString(data1)
- self.db.addResource("data1.ics", calendar)
- calendar = Component.fromString(data2)
- self.db.addResource("data2.ics", calendar)
- calendar = Component.fromString(data3)
- self.db.addResource("data3.ics", calendar)
- self.db.deleteResource("data3.ics")
-
- tests = (
- (0, (["data1.ics", "data2.ics",], [],)),
- (1, (["data2.ics",], ["data3.ics",],)),
- (2, ([], ["data3.ics",],)),
- (3, ([], ["data3.ics",],)),
- (4, ([], [],)),
- (5, ([], [],)),
- )
-
- for revision, results in tests:
- self.assertEquals(self.db.whatchanged(revision), results, "Mismatched results for whatchanged with revision %d" % (revision,))
-
-class MemcacheTests(SQLIndexTests):
- def setUp(self):
- super(MemcacheTests, self).setUp()
- self.memcache = InMemoryMemcacheProtocol()
- self.db.reserver = MemcachedUIDReserver(self.db, self.memcache)
-
-
- def tearDown(self):
- for _ignore_k, v in self.memcache._timeouts.iteritems():
- if v.active():
- v.cancel()
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_sharing.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_sharing.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_sharing.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -25,8 +25,7 @@
from twistedcaldav.config import config
from twistedcaldav.test.util import HomeTestCase, norequest
from twistedcaldav.resource import CalDAVResource
-from txdav.common.datastore.test.util import buildStore
-from txdav.caldav.datastore.test.common import StubNotifierFactory
+from txdav.common.datastore.test.util import buildStore, StubNotifierFactory
sharedOwnerType = davxml.ResourceType.sharedownercalendar #@UndefinedVariable
regularCalendarType = davxml.ResourceType.calendar #@UndefinedVariable
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_sql.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_sql.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_sql.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -17,7 +17,7 @@
from twistedcaldav.sql import AbstractSQLDatabase
import twistedcaldav.test.util
-from twistedcaldav.sql import db_prefix
+
from threading import Thread
import time
import os
Deleted: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_vcardindex.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_vcardindex.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_vcardindex.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -1,209 +0,0 @@
-##
-# Copyright (c) 2010 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
-
-from twisted.internet import reactor
-from twisted.internet.task import deferLater
-
-from twistedcaldav.test.util import InMemoryMemcacheProtocol
-from twistedcaldav.vcard import Component
-from twistedcaldav.vcardindex import AddressBookIndex, MemcachedUIDReserver, ReservationError
-import twistedcaldav.test.util
-
-import os
-
-class MinimalResourceReplacement(object):
- """
- Provide the minimal set of attributes and methods from CalDAVFile required
- by L{Index}.
- """
-
- def __init__(self, filePath):
- self.fp = filePath
-
-
- def isAddressBookCollection(self):
- return True
-
-
- def getChild(self, name):
- # FIXME: this should really return something with a child method
- return self.fp.child(name)
-
-
- def initSyncToken(self):
- pass
-
-
-
-class SQLIndexTests (twistedcaldav.test.util.TestCase):
- """
- Test abstract SQL DB class
- """
-
- def setUp(self):
- super(SQLIndexTests, self).setUp()
- self.site.resource.isAddressBookCollection = lambda: True
- self.indexDirPath = self.site.resource.fp
- # FIXME: since this resource lies about isCalendarCollection, it doesn't
- # have all the associated backend machinery to actually get children.
- self.db = AddressBookIndex(MinimalResourceReplacement(self.indexDirPath))
-
-
- def test_reserve_uid_ok(self):
- uid = "test-test-test"
- d = self.db.isReservedUID(uid)
- d.addCallback(self.assertFalse)
- d.addCallback(lambda _: self.db.reserveUID(uid))
- d.addCallback(lambda _: self.db.isReservedUID(uid))
- d.addCallback(self.assertTrue)
- d.addCallback(lambda _: self.db.unreserveUID(uid))
- d.addCallback(lambda _: self.db.isReservedUID(uid))
- d.addCallback(self.assertFalse)
-
- return d
-
-
- def test_reserve_uid_twice(self):
- uid = "test-test-test"
- d = self.db.reserveUID(uid)
- d.addCallback(lambda _: self.db.isReservedUID(uid))
- d.addCallback(self.assertTrue)
- d.addCallback(lambda _:
- self.assertFailure(self.db.reserveUID(uid),
- ReservationError))
- return d
-
-
- def test_unreserve_unreserved(self):
- uid = "test-test-test"
- return self.assertFailure(self.db.unreserveUID(uid),
- ReservationError)
-
-
- def test_reserve_uid_timeout(self):
- # WARNING: This test is fundamentally flawed and will fail
- # intermittently because it uses the real clock.
- uid = "test-test-test"
- from twistedcaldav.config import config
- old_timeout = config.UIDReservationTimeOut
- config.UIDReservationTimeOut = 1
-
- def _finally():
- config.UIDReservationTimeOut = old_timeout
-
- d = self.db.isReservedUID(uid)
- d.addCallback(self.assertFalse)
- d.addCallback(lambda _: self.db.reserveUID(uid))
- d.addCallback(lambda _: self.db.isReservedUID(uid))
- d.addCallback(self.assertTrue)
- d.addCallback(lambda _: deferLater(reactor, 2, lambda: None))
- d.addCallback(lambda _: self.db.isReservedUID(uid))
- d.addCallback(self.assertFalse)
- self.addCleanup(_finally)
-
- return d
-
-
- def test_index(self):
- data = (
- (
- "#1.1 Simple component",
- "1.1",
- """BEGIN:VCARD
-VERSION:3.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-UID:12345-67890-1.1
-FN:Cyrus Daboo
-N:Daboo;Cyrus
-EMAIL;TYPE=INTERNET,PREF:cyrus at example.com
-END:VCARD
-""",
- ),
- )
-
- for description, name, vcard_txt in data:
- calendar = Component.fromString(vcard_txt)
- f = open(os.path.join(self.site.resource.fp.path, name), "w")
- f.write(vcard_txt)
- del f
-
- self.db.addResource(name, calendar)
- self.assertTrue(self.db.resourceExists(name), msg=description)
-
- self.db._db_recreate()
- for description, name, vcard_txt in data:
- self.assertTrue(self.db.resourceExists(name), msg=description)
-
- def test_index_revisions(self):
- data1 = """BEGIN:VCARD
-VERSION:3.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-UID:12345-67890-1-1.1
-FN:Cyrus Daboo
-N:Daboo;Cyrus
-EMAIL;TYPE=INTERNET,PREF:cyrus at example.com
-END:VCARD
-"""
- data2 = """BEGIN:VCARD
-VERSION:3.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-UID:12345-67890-2-1.1
-FN:Wilfredo Sanchez
-N:Sanchez;Wilfredo
-EMAIL;TYPE=INTERNET,PREF:wsanchez at example.com
-END:VCARD
-"""
- data3 = """BEGIN:VCARD
-VERSION:3.0
-PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
-UID:12345-67890-3-1.1
-FN:Bruce Gaya
-N:Gaya;Bruce
-EMAIL;TYPE=INTERNET,PREF:bruce at example.com
-END:VCARD
-"""
-
- vcard = Component.fromString(data1)
- self.db.addResource("data1.vcf", vcard)
- vcard = Component.fromString(data2)
- self.db.addResource("data2.vcf", vcard)
- vcard = Component.fromString(data3)
- self.db.addResource("data3.vcf", vcard)
- self.db.deleteResource("data3.vcf")
-
- tests = (
- (0, (["data1.vcf", "data2.vcf",], [],)),
- (1, (["data2.vcf",], ["data3.vcf",],)),
- (2, ([], ["data3.vcf",],)),
- (3, ([], ["data3.vcf",],)),
- (4, ([], [],)),
- (5, ([], [],)),
- )
-
- for revision, results in tests:
- self.assertEquals(self.db.whatchanged(revision), results, "Mismatched results for whatchanged with revision %d" % (revision,))
-
-class MemcacheTests(SQLIndexTests):
- def setUp(self):
- super(MemcacheTests, self).setUp()
- self.memcache = InMemoryMemcacheProtocol()
- self.db.reserver = MemcachedUIDReserver(self.db, self.memcache)
-
-
- def tearDown(self):
- for _ignore_k, v in self.memcache._timeouts.iteritems():
- if v.active():
- v.cancel()
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_wrapping.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_wrapping.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_wrapping.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -41,15 +41,18 @@
from txdav.carddav.datastore.test.test_file import vcard4_text
-from txdav.common.datastore.test.util import buildStore
-from txdav.caldav.datastore.test.common import StubNotifierFactory, \
- assertProvides
+from txdav.common.datastore.test.util import buildStore, assertProvides,\
+ StubNotifierFactory
+
+
from txdav.caldav.icalendarstore import ICalendarHome
from txdav.carddav.iaddressbookstore import IAddressBookHome
class FakeChanRequest(object):
+ code = 'request-not-finished'
+
def writeHeaders(self, code, headers):
self.code = code
self.headers = headers
@@ -89,6 +92,7 @@
self.setupCalendars()
+ @inlineCallbacks
def populateOneObject(self, objectName, objectText):
"""
Populate one calendar object in the test user's calendar.
@@ -107,12 +111,13 @@
except:
pass
txn = self.calendarCollection._newStore.newTransaction()
- home = txn.calendarHomeWithUID(uid, True)
- cal = home.calendarWithName("calendar")
+ home = yield txn.calendarHomeWithUID(uid, True)
+ cal = yield home.calendarWithName("calendar")
cal.createCalendarObjectWithName(objectName, VComponent.fromString(objectText))
- txn.commit()
+ yield txn.commit()
+ @inlineCallbacks
def populateOneAddressBookObject(self, objectName, objectText):
"""
Populate one addressbook object in the test user's addressbook.
@@ -131,13 +136,13 @@
except:
pass
txn = self.addressbookCollection._newStore.newTransaction()
- home = txn.addressbookHomeWithUID(uid, True)
- adbk = home.addressbookWithName("addressbook")
+ home = yield txn.addressbookHomeWithUID(uid, True)
+ adbk = yield home.addressbookWithName("addressbook")
if adbk is None:
- home.createAddressBookWithName("addressbook")
- adbk = home.addressbookWithName("addressbook")
+ yield home.createAddressBookWithName("addressbook")
+ adbk = yield home.addressbookWithName("addressbook")
adbk.createAddressBookObjectWithName(objectName, VCComponent.fromString(objectText))
- txn.commit()
+ yield txn.commit()
requestUnderTest = None
@@ -169,7 +174,7 @@
an associated transaction. Commit that transaction to bring the
filesystem into a consistent state.
"""
- self.requestUnderTest._newStoreTransaction.commit()
+ return self.requestUnderTest._newStoreTransaction.commit()
def requestForPath(self, path):
@@ -241,7 +246,7 @@
L{CalendarHome} via C{newTransaction().calendarHomeWithUID}.
"""
calDavFile = yield self.getResource("calendars/users/wsanchez/")
- self.commit()
+ yield self.commit()
assertProvides(self, ICalendarHome, calDavFile._newStoreHome)
@@ -256,7 +261,7 @@
dropBoxResource = yield self.getResource(
"calendars/users/wsanchez/dropbox"
)
- self.commit()
+ yield self.commit()
self.assertIsInstance(dropBoxResource, DropboxCollection)
dropboxHomeType = davxml.ResourceType.dropboxhome #@UndefinedVariable
self.assertEquals(dropBoxResource.resourceType(),
@@ -275,7 +280,7 @@
regularCalendarType = davxml.ResourceType.calendar #@UndefinedVariable
self.assertEquals(calDavFile.resourceType(),
regularCalendarType)
- self.commit()
+ yield self.commit()
@inlineCallbacks
@@ -290,7 +295,7 @@
calDavFile = yield self.getResource("calendars/users/wsanchez/frobozz")
self.assertIsInstance(calDavFile, ProtoCalendarCollectionResource)
calDavFile.createCalendarCollection()
- self.commit()
+ yield self.commit()
@inlineCallbacks
@@ -308,7 +313,7 @@
self.assertIdentical(
getattr(calDavFile, "_newStoreCalendar", None), None
)
- self.commit()
+ yield self.commit()
@inlineCallbacks
@@ -336,17 +341,21 @@
@inlineCallbacks
def test_lookupCalendarObject(self):
"""
- When a L{CalDAVResource} representing an existing calendar object is looked
- up on a L{CalDAVResource} representing a calendar collection, a parallel
- L{CalendarObject} will be created (with a matching FilePath).
+ When a L{CalDAVResource} representing an existing calendar object is
+ looked up on a L{CalDAVResource} representing a calendar collection, a
+ parallel L{CalendarObject} will be created. Its principal collections
+ and transaction should match.
"""
- self.populateOneObject("1.ics", event4_text)
+ yield self.populateOneObject("1.ics", event4_text)
+ calendarHome = yield self.getResource("calendars/users/wsanchez")
calDavFileCalendar = yield self.getResource(
"calendars/users/wsanchez/calendar/1.ics"
)
- self.commit()
+ yield self.commit()
self.assertEquals(calDavFileCalendar._principalCollections,
frozenset([self.principalsResource]))
+ self.assertEquals(calDavFileCalendar._associatedTransaction,
+ calendarHome._associatedTransaction)
@inlineCallbacks
@@ -359,7 +368,7 @@
calDavFileCalendar = yield self.getResource(
"calendars/users/wsanchez/calendar/xyzzy.ics"
)
- self.commit()
+ yield self.commit()
self.assertEquals(calDavFileCalendar._principalCollections,
frozenset([self.principalsResource]))
@@ -380,7 +389,7 @@
via C{newTransaction().addressbookHomeWithUID}.
"""
calDavFile = yield self.getResource("addressbooks/users/wsanchez/")
- self.commit()
+ yield self.commit()
assertProvides(self, IAddressBookHome, calDavFile._newStoreHome)
@@ -392,7 +401,7 @@
create a corresponding L{AddressBook} via C{AddressBookHome.addressbookWithName}.
"""
calDavFile = yield self.getResource("addressbooks/users/wsanchez/addressbook")
- self.commit()
+ yield self.commit()
self.assertEquals(calDavFile._principalCollections,
frozenset([self.principalsResource]))
@@ -409,7 +418,7 @@
calDavFile = yield self.getResource("addressbooks/users/wsanchez/frobozz")
self.assertIsInstance(calDavFile, ProtoAddressBookCollectionResource)
calDavFile.createAddressBookCollection()
- self.commit()
+ yield self.commit()
self.assertEquals(calDavFile._principalCollections,
frozenset([self.principalsResource]))
@@ -421,11 +430,11 @@
up on a L{CalDAVResource} representing a addressbook collection, a parallel
L{AddressBookObject} will be created (with a matching FilePath).
"""
- self.populateOneAddressBookObject("1.vcf", vcard4_text)
+ yield self.populateOneAddressBookObject("1.vcf", vcard4_text)
calDavFileAddressBook = yield self.getResource(
"addressbooks/users/wsanchez/addressbook/1.vcf"
)
- self.commit()
+ yield self.commit()
self.assertEquals(calDavFileAddressBook._principalCollections,
frozenset([self.principalsResource]))
@@ -440,7 +449,7 @@
calDavFileAddressBook = yield self.getResource(
"addressbooks/users/wsanchez/addressbook/xyzzy.ics"
)
- self.commit()
+ yield self.commit()
self.assertEquals(calDavFileAddressBook._principalCollections,
frozenset([self.principalsResource]))
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/util.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/util.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/util.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -352,7 +352,7 @@
"""
if not self.committed:
self.committed = True
- self.site.resource._associatedTransaction.commit()
+ return self.site.resource._associatedTransaction.commit()
@inlineCallbacks
@@ -361,10 +361,10 @@
Refresh the user resource positioned at the root of this site, to give
it a new transaction.
"""
- self.noRenderCommit()
+ yield self.noRenderCommit()
if request is None:
request = norequest()
- users = self.homeProvisioner.getChild("users")
+ users = yield self.homeProvisioner.getChild("users")
user, ignored = (yield users.locateChild(request, ["wsanchez"]))
@@ -387,7 +387,7 @@
Override C{send} in order to refresh the 'user' resource each time, to
get a new transaction to associate with the calendar home.
"""
- self.noRenderCommit()
+ yield self.noRenderCommit()
yield self._refreshRoot(request)
result = (yield super(HomeTestCase, self).send(request))
self.committed = True
@@ -424,12 +424,13 @@
self.directoryService, "/addressbooks/",
_newStore
)
-
+
+ @inlineCallbacks
def _defer(user):
# Commit the transaction
- self.site.resource._associatedTransaction.commit()
+ yield self.site.resource._associatedTransaction.commit()
self.docroot = user._newStoreHome._path.path
-
+
return self._refreshRoot().addCallback(_defer)
@inlineCallbacks
Deleted: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/vcardindex.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/vcardindex.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/vcardindex.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -1,701 +0,0 @@
-##
-# Copyright (c) 2005-2009 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
-
-"""
-CardDAV Index.
-
-This API is considered private to static.py and is therefore subject to
-change.
-"""
-
-__all__ = [
- "AddressBookIndex",
-]
-
-import datetime
-import os
-import time
-import hashlib
-
-try:
- import sqlite3 as sqlite
-except ImportError:
- from pysqlite2 import dbapi2 as sqlite
-
-from twisted.internet.defer import maybeDeferred
-
-from twistedcaldav import carddavxml
-from twistedcaldav.index import SyncTokenValidException
-from twistedcaldav.query import addressbookquery
-from twistedcaldav.sql import AbstractSQLDatabase
-from twistedcaldav.sql import db_prefix
-from twistedcaldav.vcard import Component
-
-from twext.python.log import Logger, LoggingMixIn
-from twistedcaldav.config import config
-from twistedcaldav.memcachepool import CachePoolUserMixIn
-
-log = Logger()
-
-db_basename = db_prefix + "sqlite"
-schema_version = "2"
-
-class ReservationError(LookupError):
- """
- Attempt to reserve a UID which is already reserved or to unreverse a UID
- which is not reserved.
- """
-
-def wrapInDeferred(f):
- def _(*args, **kwargs):
- return maybeDeferred(f, *args, **kwargs)
-
- return _
-
-
-class MemcachedUIDReserver(CachePoolUserMixIn, LoggingMixIn):
- def __init__(self, index, cachePool=None):
- self.index = index
- self._cachePool = cachePool
-
- def _key(self, uid):
- return 'reservation:%s' % (
- hashlib.md5('%s:%s' % (uid,
- self.index.resource.fp.path)).hexdigest())
-
- def reserveUID(self, uid):
- uid = uid.encode('utf-8')
- self.log_debug("Reserving UID %r @ %r" % (
- uid,
- self.index.resource.fp.path))
-
- def _handleFalse(result):
- if result is False:
- raise ReservationError(
- "UID %s already reserved for address book collection %s."
- % (uid, self.index.resource)
- )
-
- d = self.getCachePool().add(self._key(uid),
- 'reserved',
- expireTime=config.UIDReservationTimeOut)
- d.addCallback(_handleFalse)
- return d
-
-
- def unreserveUID(self, uid):
- uid = uid.encode('utf-8')
- self.log_debug("Unreserving UID %r @ %r" % (
- uid,
- self.index.resource.fp.path))
-
- def _handleFalse(result):
- if result is False:
- raise ReservationError(
- "UID %s is not reserved for address book collection %s."
- % (uid, self.index.resource)
- )
-
- d =self.getCachePool().delete(self._key(uid))
- d.addCallback(_handleFalse)
- return d
-
-
- def isReservedUID(self, uid):
- uid = uid.encode('utf-8')
- self.log_debug("Is reserved UID %r @ %r" % (
- uid,
- self.index.resource.fp.path))
-
- def _checkValue((flags, value)):
- if value is None:
- return False
- else:
- return True
-
- d = self.getCachePool().get(self._key(uid))
- d.addCallback(_checkValue)
- return d
-
-
-
-class SQLUIDReserver(object):
- def __init__(self, index):
- self.index = index
-
- @wrapInDeferred
- def reserveUID(self, uid):
- """
- Reserve a UID for this index's resource.
- @param uid: the UID to reserve
- @raise ReservationError: if C{uid} is already reserved
- """
-
- try:
- self.index._db_execute("insert into RESERVED (UID, TIME) values (:1, :2)", uid, datetime.datetime.now())
- self.index._db_commit()
- except sqlite.IntegrityError:
- self.index._db_rollback()
- raise ReservationError(
- "UID %s already reserved for address book collection %s."
- % (uid, self.index.resource)
- )
- except sqlite.Error, e:
- log.err("Unable to reserve UID: %s", (e,))
- self.index._db_rollback()
- raise
-
- def unreserveUID(self, uid):
- """
- Unreserve a UID for this index's resource.
- @param uid: the UID to reserve
- @raise ReservationError: if C{uid} is not reserved
- """
-
- def _cb(result):
- if result == False:
- raise ReservationError(
- "UID %s is not reserved for address book collection %s."
- % (uid, self.index.resource)
- )
- else:
- try:
- self.index._db_execute(
- "delete from RESERVED where UID = :1", uid)
- self.index._db_commit()
- except sqlite.Error, e:
- log.err("Unable to unreserve UID: %s", (e,))
- self.index._db_rollback()
- raise
-
- d = self.isReservedUID(uid)
- d.addCallback(_cb)
- return d
-
-
- @wrapInDeferred
- def isReservedUID(self, uid):
- """
- Check to see whether a UID is reserved.
- @param uid: the UID to check
- @return: True if C{uid} is reserved, False otherwise.
- """
-
- rowiter = self.index._db_execute("select UID, TIME from RESERVED where UID = :1", uid)
- for uid, attime in rowiter:
- # Double check that the time is within a reasonable period of now
- # otherwise we probably have a stale reservation
- tm = time.strptime(attime[:19], "%Y-%m-%d %H:%M:%S")
- dt = datetime.datetime(year=tm.tm_year, month=tm.tm_mon, day=tm.tm_mday, hour=tm.tm_hour, minute=tm.tm_min, second = tm.tm_sec)
- if datetime.datetime.now() - dt > datetime.timedelta(seconds=config.UIDReservationTimeOut):
- try:
- self.index._db_execute("delete from RESERVED where UID = :1", uid)
- self.index._db_commit()
- except sqlite.Error, e:
- log.err("Unable to unreserve UID: %s", (e,))
- self.index._db_rollback()
- raise
- return False
- else:
- return True
-
- return False
-
-class AddressBookIndex(AbstractSQLDatabase):
- """
- AddressBook collection index abstract base class that defines the apis for the index.
- """
-
- def __init__(self, resource):
- """
- @param resource: the L{CalDAVResource} resource to
- index. C{resource} must be an addressbook collection (ie.
- C{resource.isAddressBookCollection()} returns C{True}.)
- """
- assert resource.isAddressBookCollection(), "non-addressbook collection resource %s has no index." % (resource,)
- self.resource = resource
- db_filename = os.path.join(self.resource.fp.path, db_basename)
- super(AddressBookIndex, self).__init__(db_filename, False)
-
- if (
- hasattr(config, "Memcached") and
- config.Memcached.Pools.Default.ClientEnabled
- ):
- self.reserver = MemcachedUIDReserver(self)
- else:
- self.reserver = SQLUIDReserver(self)
-
- def create(self):
- """
- Create the index and initialize it.
- """
- self._db()
-
- def recreate(self):
- """
- Delete the database and re-create it
- """
- try:
- os.remove(self.dbpath)
- except OSError:
- pass
- self.create()
-
- #
- # A dict of sets. The dict keys are address book collection paths,
- # and the sets contains reserved UIDs for each path.
- #
-
- def reserveUID(self, uid):
- return self.reserver.reserveUID(uid)
-
- def unreserveUID(self, uid):
- return self.reserver.unreserveUID(uid)
-
- def isReservedUID(self, uid):
- return self.reserver.isReservedUID(uid)
-
- def isAllowedUID(self, uid, *names):
- """
- Checks to see whether to allow an operation which would add the
- specified UID to the index. Specifically, the operation may not
- violate the constraint that UIDs must be unique.
- @param uid: the UID to check
- @param names: the names of resources being replaced or deleted by the
- operation; UIDs associated with these resources are not checked.
- @return: True if the UID is not in the index and is not reserved,
- False otherwise.
- """
- rname = self.resourceNameForUID(uid)
- return (rname is None or rname in names)
-
- def resourceNamesForUID(self, uid):
- """
- Looks up the names of the resources with the given UID.
- @param uid: the UID of the resources to look up.
- @return: a list of resource names
- """
- names = self._db_values_for_sql("select NAME from RESOURCE where UID = :1", uid)
-
- #
- # Check that each name exists as a child of self.resource. If not, the
- # resource record is stale.
- #
- resources = []
- for name in names:
- name_utf8 = name.encode("utf-8")
- if name is not None and self.resource.getChild(name_utf8) is None:
- # Clean up
- log.err("Stale resource record found for child %s with UID %s in %s" % (name, uid, self.resource))
- self._delete_from_db(name, uid, False)
- self._db_commit()
- else:
- resources.append(name_utf8)
-
- return resources
-
- def resourceNameForUID(self, uid):
- """
- Looks up the name of the resource with the given UID.
- @param uid: the UID of the resource to look up.
- @return: If the resource is found, its name; C{None} otherwise.
- """
- result = None
-
- for name in self.resourceNamesForUID(uid):
- assert result is None, "More than one resource with UID %s in address book collection %r" % (uid, self)
- result = name
-
- return result
-
- def resourceUIDForName(self, name):
- """
- Looks up the UID of the resource with the given name.
- @param name: the name of the resource to look up.
- @return: If the resource is found, the UID of the resource; C{None}
- otherwise.
- """
- uid = self._db_value_for_sql("select UID from RESOURCE where NAME = :1", name)
-
- return uid
-
- def addResource(self, name, vcard, fast=False):
- """
- Adding or updating an existing resource.
- To check for an update we attempt to get an existing UID
- for the resource name. If present, then the index entries for
- that UID are removed. After that the new index entries are added.
- @param name: the name of the resource to add.
- @param vCard: a L{Component} object representing the resource
- contents.
- @param fast: if C{True} do not do commit, otherwise do commit.
- """
- oldUID = self.resourceUIDForName(name)
- if oldUID is not None:
- self._delete_from_db(name, oldUID, False)
- self._add_to_db(name, vcard)
- if not fast:
- self._db_commit()
-
- def deleteResource(self, name):
- """
- Remove this resource from the index.
- @param name: the name of the resource to add.
- @param uid: the UID of the vcard component in the resource.
- """
- uid = self.resourceUIDForName(name)
- if uid is not None:
- self._delete_from_db(name, uid)
- self._db_commit()
-
- def resourceExists(self, name):
- """
- Determines whether the specified resource name exists in the index.
- @param name: the name of the resource to test
- @return: True if the resource exists, False if not
- """
- uid = self._db_value_for_sql("select UID from RESOURCE where NAME = :1", name)
- return uid is not None
-
- def resourcesExist(self, names):
- """
- Determines whether the specified resource name exists in the index.
- @param names: a C{list} containing the names of the resources to test
- @return: a C{list} of all names that exist
- """
- statement = "select NAME from RESOURCE where NAME in ("
- for ctr, ignore_name in enumerate(names):
- if ctr != 0:
- statement += ", "
- statement += ":%s" % (ctr,)
- statement += ")"
- results = self._db_values_for_sql(statement, *names)
- return results
-
- def whatchanged(self, revision):
-
- results = [(name.encode("utf-8"), deleted) for name, deleted in self._db_execute("select NAME, DELETED from REVISIONS where REVISION > :1", revision)]
- results.sort(key=lambda x:x[1])
-
- changed = []
- deleted = []
- for name, wasdeleted in results:
- if name:
- if wasdeleted == 'Y':
- if revision:
- deleted.append(name)
- else:
- changed.append(name)
- else:
- raise SyncTokenValidException
-
- return changed, deleted,
-
- def lastRevision(self):
- return self._db_value_for_sql(
- "select REVISION from REVISION_SEQUENCE"
- )
-
- def bumpRevision(self, fast=False):
- self._db_execute(
- """
- update REVISION_SEQUENCE set REVISION = REVISION + 1
- """,
- )
- if not fast:
- self._db_commit()
- return self._db_value_for_sql(
- """
- select REVISION from REVISION_SEQUENCE
- """,
- )
-
- def searchValid(self, filter):
- if isinstance(filter, carddavxml.Filter):
- qualifiers = addressbookquery.sqladdressbookquery(filter)
- else:
- qualifiers = None
-
- return qualifiers is not None
-
- def search(self, filter):
- """
- Finds resources matching the given qualifiers.
- @param filter: the L{Filter} for the addressbook-query to execute.
- @return: an interable iterable of tuples for each resource matching the
- given C{qualifiers}. The tuples are C{(name, uid, type)}, where
- C{name} is the resource name, C{uid} is the resource UID, and
- C{type} is the resource iCalendar component type.x
- """
- # FIXME: Don't forget to use maximum_future_expansion_duration when we
- # start caching...
-
- # Make sure we have a proper Filter element and get the partial SQL statement to use.
- if isinstance(filter, carddavxml.Filter):
- qualifiers = addressbookquery.sqladdressbookquery(filter)
- else:
- qualifiers = None
- if qualifiers is not None:
- rowiter = self._db_execute("select DISTINCT RESOURCE.NAME, RESOURCE.UID" + qualifiers[0], *qualifiers[1])
- else:
- rowiter = self._db_execute("select NAME, UID from RESOURCE")
-
- for row in rowiter:
- name = row[0]
- if self.resource.getChild(name.encode("utf-8")):
- yield row
- else:
- log.err("vCard resource %s is missing from %s. Removing from index."
- % (name, self.resource))
- self.deleteResource(name, None)
-
- def bruteForceSearch(self):
- """
- List the whole index and tests for existence, updating the index
- @return: all resources in the index
- """
- # List all resources
- rowiter = self._db_execute("select NAME, UID from RESOURCE")
-
- # Check result for missing resources:
-
- for row in rowiter:
- name = row[0]
- if self.resource.getChild(name.encode("utf-8")):
- yield row
- else:
- log.err("AddressBook resource %s is missing from %s. Removing from index."
- % (name, self.resource))
- self.deleteResource(name)
-
-
- def _db_version(self):
- """
- @return: the schema version assigned to this index.
- """
- return schema_version
-
- def _db_type(self):
- """
- @return: the collection type assigned to this index.
- """
- return "AddressBook"
-
- def _db_init_data_tables(self, q):
- """
- Initialise the underlying database tables.
- @param q: a database cursor to use.
- """
-
- # Create database where the RESOURCE table has unique UID column.
- self._db_init_data_tables_base(q, True)
-
- def _db_init_data_tables_base(self, q, uidunique):
- """
- Initialise the underlying database tables.
- @param q: a database cursor to use.
- """
- #
- # RESOURCE table is the primary index table
- # NAME: Last URI component (eg. <uid>.ics, RESOURCE primary key)
- # UID: iCalendar UID (may or may not be unique)
- #
- q.execute(
- """
- create table RESOURCE (
- NAME text unique,
- UID text unique
- )
- """
- )
-
- #
- # REVISIONS table tracks changes
- # NAME: Last URI component (eg. <uid>.ics, RESOURCE primary key)
- # REVISION: revision number
- # WASDELETED: Y if revision deleted, N if added or changed
- #
- q.execute(
- """
- create table REVISION_SEQUENCE (
- REVISION integer
- )
- """
- )
- q.execute(
- """
- insert into REVISION_SEQUENCE (REVISION) values (0)
- """
- )
- q.execute(
- """
- create table REVISIONS (
- NAME text unique,
- REVISION integer default 0,
- DELETED text(1) default "N"
- )
- """
- )
- q.execute(
- """
- create index REVISION on REVISIONS (REVISION)
- """
- )
-
- #
- # RESERVED table tracks reserved UIDs
- # UID: The UID being reserved
- # TIME: When the reservation was made
- #
- q.execute(
- """
- create table RESERVED (
- UID text unique,
- TIME date
- )
- """
- )
-
- def _db_recreate(self, do_commit=True):
- """
- Re-create the database tables from existing address book data.
- """
-
- #
- # Populate the DB with data from already existing resources.
- # This allows for index recovery if the DB file gets
- # deleted.
- #
- fp = self.resource.fp
- for name in fp.listdir():
- if name.startswith("."):
- continue
-
- try:
- stream = fp.child(name).open()
- except (IOError, OSError), e:
- log.err("Unable to open resource %s: %s" % (name, e))
- continue
-
- try:
- # FIXME: This is blocking I/O
- try:
- vcard = Component.fromStream(stream)
- vcard.validForCardDAV()
- except ValueError:
- log.err("Non-addressbook resource: %s" % (name,))
- else:
- #log.msg("Indexing resource: %s" % (name,))
- self.addResource(name, vcard, True)
- finally:
- stream.close()
-
- # Do commit outside of the loop for better performance
- if do_commit:
- self._db_commit()
-
- def _db_can_upgrade(self, old_version):
- """
- Can we do an in-place upgrade
- """
-
- # v2 is a minor change
- return True
-
- def _db_upgrade_data_tables(self, q, old_version):
- """
- Upgrade the data from an older version of the DB.
- """
-
- # When going to version 2+ all we need to do is add revision table and index
- if old_version < 2:
- q.execute(
- """
- create table REVISION_SEQUENCE (
- REVISION integer
- )
- """
- )
- q.execute(
- """
- insert into REVISION_SEQUENCE (REVISION) values (0)
- """
- )
- q.execute(
- """
- create table REVISIONS (
- NAME text unique,
- REVISION integer default 0,
- CREATEDREVISION integer default 0,
- WASDELETED text(1) default "N"
- )
- """
- )
- q.execute(
- """
- create index REVISION on REVISIONS (REVISION)
- """
- )
-
- self._db_execute(
- """
- insert into REVISIONS (NAME)
- select NAME from RESOURCE
- """
- )
-
-
- def _add_to_db(self, name, vcard, cursor = None):
- """
- Records the given address book resource in the index with the given name.
- Resource names and UIDs must both be unique; only one resource name may
- be associated with any given UID and vice versa.
- NB This method does not commit the changes to the db - the caller
- MUST take care of that
- @param name: the name of the resource to add.
- @param vcard: a L{AddressBook} object representing the resource
- contents.
- """
- uid = vcard.resourceUID()
-
- self._db_execute(
- """
- insert into RESOURCE (NAME, UID)
- values (:1, :2)
- """, name, uid,
- )
-
- self._db_execute(
- """
- insert or replace into REVISIONS (NAME, REVISION, DELETED)
- values (:1, :2, :3)
- """, name, self.bumpRevision(fast=True), 'N',
- )
-
- def _delete_from_db(self, name, uid, dorevision=True):
- """
- Deletes the specified entry from all dbs.
- @param name: the name of the resource to delete.
- @param uid: the uid of the resource to delete.
- """
- self._db_execute("delete from RESOURCE where NAME = :1", name)
- if dorevision:
- self._db_execute(
- """
- update REVISIONS SET REVISION = :1, DELETED = :2
- where NAME = :3
- """, self.bumpRevision(fast=True), 'Y', name
- )
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/test/base.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/test/base.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/test/base.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -38,10 +38,6 @@
class PropertyStoreTest(unittest.TestCase):
# Subclass must define self.propertyStore in setUp().
- def _preTest(self):
- self.addCleanup(self._postTest)
- def _postTest(self):
- pass
def _changed(self, store):
store.flush()
def _abort(self, store):
@@ -49,14 +45,11 @@
def test_interface(self):
try:
- self._preTest()
verifyObject(IPropertyStore, self.propertyStore)
except BrokenMethodImplementation, e:
self.fail(e)
def test_set_get_contains(self):
-
- self._preTest()
name = propertyName("test")
value = propertyValue("Hello, World!")
@@ -75,8 +68,6 @@
def test_delete_get_contains(self):
- self._preTest()
-
# Test with commit after change
name = propertyName("test")
value = propertyValue("Hello, World!")
@@ -104,8 +95,6 @@
def test_peruser(self):
- self._preTest()
-
name = propertyName("test")
value1 = propertyValue("Hello, World1!")
value2 = propertyValue("Hello, World2!")
@@ -140,8 +129,6 @@
def test_peruser_shadow(self):
- self._preTest()
-
name = propertyName("shadow")
self.propertyStore1.setSpecialProperties((name,), ())
@@ -181,8 +168,6 @@
def test_peruser_global(self):
- self._preTest()
-
name = propertyName("global")
self.propertyStore1.setSpecialProperties((), (name,))
@@ -215,8 +200,6 @@
def test_iteration(self):
- self._preTest()
-
value = propertyValue("Hello, World!")
names = set(propertyName(str(i)) for i in (1,2,3,4))
@@ -229,7 +212,6 @@
def test_delete_none(self):
- self._preTest()
def doDelete():
del self.propertyStore[propertyName("xyzzy")]
@@ -237,7 +219,6 @@
def test_keyInPropertyName(self):
- self._preTest()
def doGet():
self.propertyStore["xyzzy"]
@@ -258,8 +239,6 @@
def test_flush(self):
- self._preTest()
-
name = propertyName("test")
value = propertyValue("Hello, World!")
@@ -286,9 +265,6 @@
self.assertEquals(len(self.propertyStore), 0)
def test_abort(self):
-
- self._preTest()
-
name = propertyName("test")
value = propertyValue("Hello, World!")
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/test/test_sql.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/test/test_sql.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/test/test_sql.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -19,12 +19,10 @@
L{txdav.caldav.datastore.test.common}.
"""
-from twisted.internet.defer import inlineCallbacks
+from twisted.internet.defer import inlineCallbacks, returnValue
-from txdav.caldav.datastore.test.common import StubNotifierFactory
+from txdav.common.datastore.test.util import buildStore, StubNotifierFactory
-from txdav.common.datastore.test.util import buildStore
-
from txdav.base.propertystore.base import PropertyName
from txdav.base.propertystore.test import base
@@ -38,22 +36,30 @@
class PropertyStoreTest(base.PropertyStoreTest):
- def _preTest(self):
+
+ @inlineCallbacks
+ def setUp(self):
+ self.notifierFactory = StubNotifierFactory()
+ self.store = yield buildStore(self, self.notifierFactory)
self._txn = self.store.newTransaction()
self.propertyStore = self.propertyStore1 = PropertyStore(
"user01", self._txn, 1
)
self.propertyStore2 = PropertyStore("user01", self._txn, 1)
self.propertyStore2._setPerUserUID("user02")
-
- self.addCleanup(self._postTest)
- def _postTest(self):
+
+ @inlineCallbacks
+ def tearDown(self):
if hasattr(self, "_txn"):
- self._txn.commit()
+ result = yield self._txn.commit()
delattr(self, "_txn")
+ else:
+ result = None
self.propertyStore = self.propertyStore1 = self.propertyStore2 = None
+ returnValue(result)
+
def _changed(self, store):
if hasattr(self, "_txn"):
self._txn.commit()
@@ -73,6 +79,7 @@
self.propertyStore2._shadowableKeys = store._shadowableKeys
self.propertyStore2._globalKeys = store._globalKeys
+
def _abort(self, store):
if hasattr(self, "_txn"):
self._txn.abort()
@@ -93,11 +100,8 @@
self.propertyStore2._shadowableKeys = store._shadowableKeys
self.propertyStore2._globalKeys = store._globalKeys
- @inlineCallbacks
- def setUp(self):
- self.notifierFactory = StubNotifierFactory()
- self.store = yield buildStore(self, self.notifierFactory)
+
if PropertyStore is None:
PropertyStoreTest.skip = importErrorMessage
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/file.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/file.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/file.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -44,13 +44,14 @@
from twistedcaldav import caldavxml, customxml
from twistedcaldav.caldavxml import ScheduleCalendarTransp, Opaque
-from twistedcaldav.index import Index as OldIndex, IndexSchedule as OldInboxIndex
from twistedcaldav.sharing import InvitesDatabase
from txdav.caldav.icalendarstore import IAttachment
from txdav.caldav.icalendarstore import ICalendar, ICalendarObject
from txdav.caldav.icalendarstore import ICalendarHome
+from txdav.caldav.datastore.index_file import Index as OldIndex,\
+ IndexSchedule as OldInboxIndex
from txdav.caldav.datastore.util import (
validateCalendarComponent, dropboxIDFromCalendarObject
)
Copied: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/index_file.py (from rev 6368, CalendarServer/branches/users/glyph/more-deferreds-6/txdav/caldav/datastore/index_file.py)
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/index_file.py (rev 0)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/index_file.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -0,0 +1,1148 @@
+# -*- test-case-name: twistedcaldav.test.test_index -*-
+##
+# Copyright (c) 2005-2010 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+CalDAV Index.
+
+This API is considered private to static.py and is therefore subject to
+change.
+"""
+
+__all__ = [
+ "db_basename",
+ "ReservationError",
+ "MemcachedUIDReserver",
+ "Index",
+ "IndexSchedule",
+ "IndexedSearchException",
+]
+
+import datetime
+import time
+import hashlib
+
+try:
+ import sqlite3 as sqlite
+except ImportError:
+ from pysqlite2 import dbapi2 as sqlite
+
+from vobject.icalendar import utc
+
+from twisted.internet.defer import maybeDeferred, succeed
+
+from twext.python.log import Logger, LoggingMixIn
+
+from txdav.common.icommondatastore import SyncTokenValidException,\
+ ReservationError, IndexedSearchException
+
+from twistedcaldav.ical import Component
+from twistedcaldav.query import calendarquery, calendarqueryfilter
+from twistedcaldav.sql import AbstractSQLDatabase
+from twistedcaldav.sql import db_prefix
+from twistedcaldav.instance import InvalidOverriddenInstanceError
+from twistedcaldav.config import config
+from twistedcaldav.memcachepool import CachePoolUserMixIn
+
+log = Logger()
+
+db_basename = db_prefix + "sqlite"
+schema_version = "10"
+collection_types = {"Calendar": "Regular Calendar Collection", "iTIP": "iTIP Calendar Collection"}
+
+icalfbtype_to_indexfbtype = {
+ "FREE" : 'F',
+ "BUSY" : 'B',
+ "BUSY-UNAVAILABLE": 'U',
+ "BUSY-TENTATIVE" : 'T',
+}
+indexfbtype_to_icalfbtype = dict([(v, k) for k,v in icalfbtype_to_indexfbtype.iteritems()])
+
+#
+# Duration into the future through which recurrences are expanded in the index
+# by default. This is a caching parameter which affects the size of the index;
+# it does not affect search results beyond this period, but it may affect
+# performance of such a search.
+#
+default_future_expansion_duration = datetime.timedelta(days=365*1)
+
+#
+# Maximum duration into the future through which recurrences are expanded in the
+# index. This is a caching parameter which affects the size of the index; it
+# does not affect search results beyond this period, but it may affect
+# performance of such a search.
+#
+# When a search is performed on a time span that goes beyond that which is
+# expanded in the index, we have to open each resource which may have data in
+# that time period. In order to avoid doing that multiple times, we want to
+# cache those results. However, we don't necessarily want to cache all
+# occurrences into some obscenely far-in-the-future date, so we cap the caching
+# period. Searches beyond this period will always be relatively expensive for
+# resources with occurrences beyond this period.
+#
+maximum_future_expansion_duration = datetime.timedelta(days=365*5)
+
+
+class AbstractCalendarIndex(AbstractSQLDatabase, LoggingMixIn):
+ """
+ Calendar collection index abstract base class that defines the apis for the index.
+ This will be subclassed for the two types of index behaviour we need: one for
+ regular calendar collections, one for schedule calendar collections.
+ """
+
+ def __init__(self, resource):
+ """
+ @param resource: the L{CalDAVResource} resource to
+ index. C{resource} must be a calendar collection (ie.
+ C{resource.isPseudoCalendarCollection()} returns C{True}.)
+ """
+ self.resource = resource
+ db_filename = self.resource.fp.child(db_basename).path
+ super(AbstractCalendarIndex, self).__init__(db_filename, False)
+
+ def create(self):
+ """
+ Create the index and initialize it.
+ """
+ self._db()
+
+ def reserveUID(self, uid):
+ """
+ Reserve a UID for this index's resource.
+ @param uid: the UID to reserve
+ @raise ReservationError: if C{uid} is already reserved
+ """
+ raise NotImplementedError
+
+ def unreserveUID(self, uid):
+ """
+ Unreserve a UID for this index's resource.
+ @param uid: the UID to reserve
+ @raise ReservationError: if C{uid} is not reserved
+ """
+ raise NotImplementedError
+
+ def isReservedUID(self, uid):
+ """
+ Check to see whether a UID is reserved.
+ @param uid: the UID to check
+ @return: True if C{uid} is reserved, False otherwise.
+ """
+ raise NotImplementedError
+
+ def isAllowedUID(self, uid, *names):
+ """
+ Checks to see whether to allow an operation with adds the the specified
+ UID is allowed to the index. Specifically, the operation may not
+ violate the constraint that UIDs must be unique, and the UID must not
+ be reserved.
+ @param uid: the UID to check
+ @param names: the names of resources being replaced or deleted by the
+ operation; UIDs associated with these resources are not checked.
+ @return: True if the UID is not in the index and is not reserved,
+ False otherwise.
+ """
+ raise NotImplementedError
+
+ def resourceNamesForUID(self, uid):
+ """
+ Looks up the names of the resources with the given UID.
+ @param uid: the UID of the resources to look up.
+ @return: a list of resource names
+ """
+ names = self._db_values_for_sql("select NAME from RESOURCE where UID = :1", uid)
+
+ #
+ # Check that each name exists as a child of self.resource. If not, the
+ # resource record is stale.
+ #
+ resources = []
+ for name in names:
+ name_utf8 = name.encode("utf-8")
+ if name is not None and self.resource.getChild(name_utf8) is None:
+ # Clean up
+ log.err("Stale resource record found for child %s with UID %s in %s" % (name, uid, self.resource))
+ self._delete_from_db(name, uid, False)
+ self._db_commit()
+ else:
+ resources.append(name_utf8)
+
+ return resources
+
+ def resourceNameForUID(self, uid):
+ """
+ Looks up the name of the resource with the given UID.
+ @param uid: the UID of the resource to look up.
+ @return: If the resource is found, its name; C{None} otherwise.
+ """
+ result = None
+
+ for name in self.resourceNamesForUID(uid):
+ assert result is None, "More than one resource with UID %s in calendar collection %r" % (uid, self)
+ result = name
+
+ return result
+
+ def resourceUIDForName(self, name):
+ """
+ Looks up the UID of the resource with the given name.
+ @param name: the name of the resource to look up.
+ @return: If the resource is found, the UID of the resource; C{None}
+ otherwise.
+ """
+ uid = self._db_value_for_sql("select UID from RESOURCE where NAME = :1", name)
+
+ return uid
+
+ def addResource(self, name, calendar, fast=False, reCreate=False):
+ """
+ Adding or updating an existing resource.
+ To check for an update we attempt to get an existing UID
+ for the resource name. If present, then the index entries for
+ that UID are removed. After that the new index entries are added.
+ @param name: the name of the resource to add.
+ @param calendar: a L{Calendar} object representing the resource
+ contents.
+ @param fast: if C{True} do not do commit, otherwise do commit.
+ """
+ oldUID = self.resourceUIDForName(name)
+ if oldUID is not None:
+ self._delete_from_db(name, oldUID, False)
+ self._add_to_db(name, calendar, reCreate=reCreate)
+ if not fast:
+ self._db_commit()
+
+ def deleteResource(self, name):
+ """
+ Remove this resource from the index.
+ @param name: the name of the resource to add.
+ @param uid: the UID of the calendar component in the resource.
+ """
+ uid = self.resourceUIDForName(name)
+ if uid is not None:
+ self._delete_from_db(name, uid)
+ self._db_commit()
+
+ def resourceExists(self, name):
+ """
+ Determines whether the specified resource name exists in the index.
+ @param name: the name of the resource to test
+ @return: True if the resource exists, False if not
+ """
+ uid = self._db_value_for_sql("select UID from RESOURCE where NAME = :1", name)
+ return uid is not None
+
+ def resourcesExist(self, names):
+ """
+ Determines whether the specified resource name exists in the index.
+ @param names: a C{list} containing the names of the resources to test
+ @return: a C{list} of all names that exist
+ """
+ statement = "select NAME from RESOURCE where NAME in ("
+ for ctr in (item[0] for item in enumerate(names)):
+ if ctr != 0:
+ statement += ", "
+ statement += ":%s" % (ctr,)
+ statement += ")"
+ results = self._db_values_for_sql(statement, *names)
+ return results
+
+
+ def testAndUpdateIndex(self, minDate):
+ # Find out if the index is expanded far enough
+ names = self.notExpandedBeyond(minDate)
+ # Actually expand recurrence max
+ for name in names:
+ self.log_info("Search falls outside range of index for %s %s" % (name, minDate))
+ self.reExpandResource(name, minDate)
+
+ def whatchanged(self, revision):
+
+ results = [(name.encode("utf-8"), deleted) for name, deleted in self._db_execute("select NAME, DELETED from REVISIONS where REVISION > :1", revision)]
+ results.sort(key=lambda x:x[1])
+
+ changed = []
+ deleted = []
+ for name, wasdeleted in results:
+ if name:
+ if wasdeleted == 'Y':
+ if revision:
+ deleted.append(name)
+ else:
+ changed.append(name)
+ else:
+ raise SyncTokenValidException
+
+ return changed, deleted,
+
+ def lastRevision(self):
+ return self._db_value_for_sql(
+ "select REVISION from REVISION_SEQUENCE"
+ )
+
+ def bumpRevision(self, fast=False):
+ self._db_execute(
+ """
+ update REVISION_SEQUENCE set REVISION = REVISION + 1
+ """,
+ )
+ self._db_commit()
+ return self._db_value_for_sql(
+ """
+ select REVISION from REVISION_SEQUENCE
+ """,
+ )
+
+ def indexedSearch(self, filter, useruid="", fbtype=False):
+ """
+ Finds resources matching the given qualifiers.
+ @param filter: the L{Filter} for the calendar-query to execute.
+ @return: an iterable of tuples for each resource matching the
+ given C{qualifiers}. The tuples are C{(name, uid, type)}, where
+ C{name} is the resource name, C{uid} is the resource UID, and
+ C{type} is the resource iCalendar component type.
+ """
+
+ # Make sure we have a proper Filter element and get the partial SQL
+ # statement to use.
+ if isinstance(filter, calendarqueryfilter.Filter):
+ if fbtype:
+ # Lookup the useruid - try the empty (default) one if needed
+ dbuseruid = self._db_value_for_sql(
+ "select PERUSERID from PERUSER where USERUID == :1",
+ useruid,
+ )
+ else:
+ dbuseruid = ""
+
+ qualifiers = calendarquery.sqlcalendarquery(filter, None, dbuseruid)
+ if qualifiers is not None:
+ # Determine how far we need to extend the current expansion of
+ # events. If we have an open-ended time-range we will expand one
+ # year past the start. That should catch bounded recurrences - unbounded
+ # will have been indexed with an "infinite" value always included.
+ maxDate, isStartDate = filter.getmaxtimerange()
+ if maxDate:
+ maxDate = maxDate.date()
+ if isStartDate:
+ maxDate += datetime.timedelta(days=365)
+ self.testAndUpdateIndex(maxDate)
+ else:
+ # We cannot handler this filter in an indexed search
+ raise IndexedSearchException()
+
+ else:
+ qualifiers = None
+
+ # Perform the search
+ if qualifiers is None:
+ rowiter = self._db_execute("select NAME, UID, TYPE from RESOURCE")
+ else:
+ if fbtype:
+ # Lookup the useruid - try the empty (default) one if needed
+ dbuseruid = self._db_value_for_sql(
+ "select PERUSERID from PERUSER where USERUID == :1",
+ useruid,
+ )
+
+ # For a free-busy time-range query we return all instances
+ rowiter = self._db_execute(
+ "select DISTINCT RESOURCE.NAME, RESOURCE.UID, RESOURCE.TYPE, RESOURCE.ORGANIZER, TIMESPAN.FLOAT, TIMESPAN.START, TIMESPAN.END, TIMESPAN.FBTYPE, TIMESPAN.TRANSPARENT, TRANSPARENCY.TRANSPARENT" +
+ qualifiers[0],
+ *qualifiers[1]
+ )
+ else:
+ rowiter = self._db_execute("select DISTINCT RESOURCE.NAME, RESOURCE.UID, RESOURCE.TYPE" + qualifiers[0], *qualifiers[1])
+
+ # Check result for missing resources
+
+ for row in rowiter:
+ name = row[0]
+ if self.resource.getChild(name.encode("utf-8")):
+ if fbtype:
+ row = list(row)
+ if row[9]:
+ row[8] = row[9]
+ del row[9]
+ yield row
+ else:
+ log.err("Calendar resource %s is missing from %s. Removing from index."
+ % (name, self.resource))
+ self.deleteResource(name)
+
+ def bruteForceSearch(self):
+ """
+ List the whole index and tests for existence, updating the index
+ @return: all resources in the index
+ """
+ # List all resources
+ rowiter = self._db_execute("select NAME, UID, TYPE from RESOURCE")
+
+ # Check result for missing resources:
+
+ for row in rowiter:
+ name = row[0]
+ if self.resource.getChild(name.encode("utf-8")):
+ yield row
+ else:
+ log.err("Calendar resource %s is missing from %s. Removing from index."
+ % (name, self.resource))
+ self.deleteResource(name)
+
+
+ def _db_version(self):
+ """
+ @return: the schema version assigned to this index.
+ """
+ return schema_version
+
+ def _add_to_db(self, name, calendar, cursor=None, expand_until=None, reCreate=False):
+ """
+ Records the given calendar resource in the index with the given name.
+ Resource names and UIDs must both be unique; only one resource name may
+ be associated with any given UID and vice versa.
+ NB This method does not commit the changes to the db - the caller
+ MUST take care of that
+ @param name: the name of the resource to add.
+ @param calendar: a L{Calendar} object representing the resource
+ contents.
+ """
+ raise NotImplementedError
+
+ def _delete_from_db(self, name, uid, dorevision=True):
+ """
+ Deletes the specified entry from all dbs.
+ @param name: the name of the resource to delete.
+ @param uid: the uid of the resource to delete.
+ """
+ raise NotImplementedError
+
+class CalendarIndex (AbstractCalendarIndex):
+ """
+ Calendar index - abstract class for indexer that indexes calendar objects in a collection.
+ """
+
+ def __init__(self, resource):
+ """
+ @param resource: the L{CalDAVResource} resource to
+ index.
+ """
+ super(CalendarIndex, self).__init__(resource)
+
+ def _db_init_data_tables_base(self, q, uidunique):
+ """
+ Initialise the underlying database tables.
+ @param q: a database cursor to use.
+ """
+ #
+ # RESOURCE table is the primary index table
+ # NAME: Last URI component (eg. <uid>.ics, RESOURCE primary key)
+ # UID: iCalendar UID (may or may not be unique)
+ # TYPE: iCalendar component type
+ # RECURRANCE_MAX: Highest date of recurrence expansion
+ # ORGANIZER: cu-address of the Organizer of the event
+ #
+ q.execute(
+ """
+ create table RESOURCE (
+ RESOURCEID integer primary key autoincrement,
+ NAME text unique,
+ UID text%s,
+ TYPE text,
+ RECURRANCE_MAX date,
+ ORGANIZER text
+ )
+ """ % (" unique" if uidunique else "",)
+ )
+
+ #
+ # TIMESPAN table tracks (expanded) time spans for resources
+ # NAME: Related resource (RESOURCE foreign key)
+ # FLOAT: 'Y' if start/end are floating, 'N' otherwise
+ # START: Start date
+ # END: End date
+ # FBTYPE: FBTYPE value:
+ # '?' - unknown
+ # 'F' - free
+ # 'B' - busy
+ # 'U' - busy-unavailable
+ # 'T' - busy-tentative
+ # TRANSPARENT: Y if transparent, N if opaque (default non-per-user value)
+ #
+ q.execute(
+ """
+ create table TIMESPAN (
+ INSTANCEID integer primary key autoincrement,
+ RESOURCEID integer,
+ FLOAT text(1),
+ START date,
+ END date,
+ FBTYPE text(1),
+ TRANSPARENT text(1)
+ )
+ """
+ )
+ q.execute(
+ """
+ create index STARTENDFLOAT on TIMESPAN (START, END, FLOAT)
+ """
+ )
+
+ #
+ # PERUSER table tracks per-user ids
+ # PERUSERID: autoincrement primary key
+ # UID: User ID used in calendar data
+ #
+ q.execute(
+ """
+ create table PERUSER (
+ PERUSERID integer primary key autoincrement,
+ USERUID text
+ )
+ """
+ )
+ q.execute(
+ """
+ create index PERUSER_UID on PERUSER (USERUID)
+ """
+ )
+
+ #
+ # TRANSPARENCY table tracks per-user per-instance transparency
+ # PERUSERID: user id key
+ # INSTANCEID: instance id key
+ # TRANSPARENT: Y if transparent, N if opaque
+ #
+ q.execute(
+ """
+ create table TRANSPARENCY (
+ PERUSERID integer,
+ INSTANCEID integer,
+ TRANSPARENT text(1)
+ )
+ """
+ )
+
+ #
+ # REVISIONS table tracks changes
+ # NAME: Last URI component (eg. <uid>.ics, RESOURCE primary key)
+ # REVISION: revision number
+ # WASDELETED: Y if revision deleted, N if added or changed
+ #
+ q.execute(
+ """
+ create table REVISION_SEQUENCE (
+ REVISION integer
+ )
+ """
+ )
+ q.execute(
+ """
+ insert into REVISION_SEQUENCE (REVISION) values (0)
+ """
+ )
+ q.execute(
+ """
+ create table REVISIONS (
+ NAME text unique,
+ REVISION integer,
+ DELETED text(1)
+ )
+ """
+ )
+ q.execute(
+ """
+ create index REVISION on REVISIONS (REVISION)
+ """
+ )
+
+ if uidunique:
+ #
+ # RESERVED table tracks reserved UIDs
+ # UID: The UID being reserved
+ # TIME: When the reservation was made
+ #
+ q.execute(
+ """
+ create table RESERVED (
+ UID text unique,
+ TIME date
+ )
+ """
+ )
+
+ # Cascading triggers to help on delete
+ q.execute(
+ """
+ create trigger resourceDelete after delete on RESOURCE
+ for each row
+ begin
+ delete from TIMESPAN where TIMESPAN.RESOURCEID = OLD.RESOURCEID;
+ end
+ """
+ )
+ q.execute(
+ """
+ create trigger timespanDelete after delete on TIMESPAN
+ for each row
+ begin
+ delete from TRANSPARENCY where INSTANCEID = OLD.INSTANCEID;
+ end
+ """
+ )
+
+ def _db_can_upgrade(self, old_version):
+ """
+ Can we do an in-place upgrade
+ """
+
+ # v10 is a big change - no upgrade possible
+ return False
+
+ def _db_upgrade_data_tables(self, q, old_version):
+ """
+ Upgrade the data from an older version of the DB.
+ """
+
+ # v10 is a big change - no upgrade possible
+ pass
+
+ def notExpandedBeyond(self, minDate):
+ """
+ Gives all resources which have not been expanded beyond a given date
+ in the index
+ """
+ return self._db_values_for_sql("select NAME from RESOURCE where RECURRANCE_MAX < :1", minDate)
+
+ def reExpandResource(self, name, expand_until):
+ """
+ Given a resource name, remove it from the database and re-add it
+ with a longer expansion.
+ """
+ calendar = self.resource.getChild(name).iCalendar()
+ self._add_to_db(name, calendar, expand_until=expand_until, reCreate=True)
+ self._db_commit()
+
+ def _add_to_db(self, name, calendar, cursor = None, expand_until=None, reCreate=False):
+ """
+ Records the given calendar resource in the index with the given name.
+ Resource names and UIDs must both be unique; only one resource name may
+ be associated with any given UID and vice versa.
+ NB This method does not commit the changes to the db - the caller
+ MUST take care of that
+ @param name: the name of the resource to add.
+ @param calendar: a L{Calendar} object representing the resource
+ contents.
+ """
+ uid = calendar.resourceUID()
+ organizer = calendar.getOrganizer()
+ if not organizer:
+ organizer = ""
+
+ # Decide how far to expand based on the component
+ master = calendar.masterComponent()
+ if master is None or not calendar.isRecurring() and not calendar.isRecurringUnbounded():
+ # When there is no master we have a set of overridden components - index them all.
+ # When there is one instance - index it.
+ # When bounded - index all.
+ expand = datetime.datetime(2100, 1, 1, 0, 0, 0, tzinfo=utc)
+ else:
+ if expand_until:
+ expand = expand_until
+ else:
+ expand = datetime.date.today() + default_future_expansion_duration
+
+ if expand > (datetime.date.today() + maximum_future_expansion_duration):
+ raise IndexedSearchException
+
+ try:
+ instances = calendar.expandTimeRanges(expand, ignoreInvalidInstances=reCreate)
+ except InvalidOverriddenInstanceError, e:
+ log.err("Invalid instance %s when indexing %s in %s" % (e.rid, name, self.resource,))
+ raise
+
+ self._delete_from_db(name, uid, False)
+
+ # Add RESOURCE item
+ self._db_execute(
+ """
+ insert into RESOURCE (NAME, UID, TYPE, RECURRANCE_MAX, ORGANIZER)
+ values (:1, :2, :3, :4, :5)
+ """, name, uid, calendar.resourceType(), instances.limit, organizer
+ )
+ resourceid = self.lastrowid
+
+ # Get a set of all referenced per-user UIDs and map those to entries already
+ # in the DB and add new ones as needed
+ useruids = calendar.allPerUserUIDs()
+ useruids.add("")
+ useruidmap = {}
+ for useruid in useruids:
+ peruserid = self._db_value_for_sql(
+ "select PERUSERID from PERUSER where USERUID = :1",
+ useruid
+ )
+ if peruserid is None:
+ self._db_execute(
+ """
+ insert into PERUSER (USERUID)
+ values (:1)
+ """, useruid
+ )
+ peruserid = self.lastrowid
+ useruidmap[useruid] = peruserid
+
+ for key in instances:
+ instance = instances[key]
+ start = instance.start.replace(tzinfo=utc)
+ end = instance.end.replace(tzinfo=utc)
+ float = 'Y' if instance.start.tzinfo is None else 'N'
+ transp = 'T' if instance.component.propertyValue("TRANSP") == "TRANSPARENT" else 'F'
+ self._db_execute(
+ """
+ insert into TIMESPAN (RESOURCEID, FLOAT, START, END, FBTYPE, TRANSPARENT)
+ values (:1, :2, :3, :4, :5, :6)
+ """,
+ resourceid,
+ float,
+ start,
+ end,
+ icalfbtype_to_indexfbtype.get(instance.component.getFBType(), 'F'),
+ transp
+ )
+ instanceid = self.lastrowid
+ peruserdata = calendar.perUserTransparency(instance.rid)
+ for useruid, transp in peruserdata:
+ peruserid = useruidmap[useruid]
+ self._db_execute(
+ """
+ insert into TRANSPARENCY (PERUSERID, INSTANCEID, TRANSPARENT)
+ values (:1, :2, :3)
+ """, peruserid, instanceid, 'T' if transp else 'F'
+ )
+
+
+ # Special - for unbounded recurrence we insert a value for "infinity"
+ # that will allow an open-ended time-range to always match it.
+ if calendar.isRecurringUnbounded():
+ start = datetime.datetime(2100, 1, 1, 0, 0, 0, tzinfo=utc)
+ end = datetime.datetime(2100, 1, 1, 1, 0, 0, tzinfo=utc)
+ float = 'N'
+ self._db_execute(
+ """
+ insert into TIMESPAN (RESOURCEID, FLOAT, START, END, FBTYPE, TRANSPARENT)
+ values (:1, :2, :3, :4, :5, :6)
+ """, resourceid, float, start, end, '?', '?'
+ )
+ instanceid = self.lastrowid
+ peruserdata = calendar.perUserTransparency(None)
+ for useruid, transp in peruserdata:
+ peruserid = useruidmap[useruid]
+ self._db_execute(
+ """
+ insert into TRANSPARENCY (PERUSERID, INSTANCEID, TRANSPARENT)
+ values (:1, :2, :3)
+ """, peruserid, instanceid, 'T' if transp else 'F'
+ )
+
+ self._db_execute(
+ """
+ insert or replace into REVISIONS (NAME, REVISION, DELETED)
+ values (:1, :2, :3)
+ """, name, self.bumpRevision(fast=True), 'N',
+ )
+
+ def _delete_from_db(self, name, uid, dorevision=True):
+ """
+ Deletes the specified entry from all dbs.
+ @param name: the name of the resource to delete.
+ @param uid: the uid of the resource to delete.
+ """
+ self._db_execute("delete from RESOURCE where NAME = :1", name)
+ if dorevision:
+ self._db_execute(
+ """
+ update REVISIONS SET REVISION = :1, DELETED = :2
+ where NAME = :3
+ """, self.bumpRevision(fast=True), 'Y', name
+ )
+
+
+def wrapInDeferred(f):
+ def _(*args, **kwargs):
+ return maybeDeferred(f, *args, **kwargs)
+
+ return _
+
+
+class MemcachedUIDReserver(CachePoolUserMixIn, LoggingMixIn):
+ def __init__(self, index, cachePool=None):
+ self.index = index
+ self._cachePool = cachePool
+
+ def _key(self, uid):
+ return 'reservation:%s' % (
+ hashlib.md5('%s:%s' % (uid,
+ self.index.resource.fp.path)).hexdigest())
+
+ def reserveUID(self, uid):
+ uid = uid.encode('utf-8')
+ self.log_debug("Reserving UID %r @ %r" % (
+ uid,
+ self.index.resource.fp.path))
+
+ def _handleFalse(result):
+ if result is False:
+ raise ReservationError(
+ "UID %s already reserved for calendar collection %s."
+ % (uid, self.index.resource)
+ )
+
+ d = self.getCachePool().add(self._key(uid),
+ 'reserved',
+ expireTime=config.UIDReservationTimeOut)
+ d.addCallback(_handleFalse)
+ return d
+
+
+ def unreserveUID(self, uid):
+ uid = uid.encode('utf-8')
+ self.log_debug("Unreserving UID %r @ %r" % (
+ uid,
+ self.index.resource.fp.path))
+
+ def _handleFalse(result):
+ if result is False:
+ raise ReservationError(
+ "UID %s is not reserved for calendar collection %s."
+ % (uid, self.index.resource)
+ )
+
+ d =self.getCachePool().delete(self._key(uid))
+ d.addCallback(_handleFalse)
+ return d
+
+
+ def isReservedUID(self, uid):
+ uid = uid.encode('utf-8')
+ self.log_debug("Is reserved UID %r @ %r" % (
+ uid,
+ self.index.resource.fp.path))
+
+ def _checkValue((flags, value)):
+ if value is None:
+ return False
+ else:
+ return True
+
+ d = self.getCachePool().get(self._key(uid))
+ d.addCallback(_checkValue)
+ return d
+
+
+
+class SQLUIDReserver(object):
+ def __init__(self, index):
+ self.index = index
+
+ @wrapInDeferred
+ def reserveUID(self, uid):
+ """
+ Reserve a UID for this index's resource.
+ @param uid: the UID to reserve
+ @raise ReservationError: if C{uid} is already reserved
+ """
+
+ try:
+ self.index._db_execute("insert into RESERVED (UID, TIME) values (:1, :2)", uid, datetime.datetime.now())
+ self.index._db_commit()
+ except sqlite.IntegrityError:
+ self.index._db_rollback()
+ raise ReservationError(
+ "UID %s already reserved for calendar collection %s."
+ % (uid, self.index.resource)
+ )
+ except sqlite.Error, e:
+ log.err("Unable to reserve UID: %s", (e,))
+ self.index._db_rollback()
+ raise
+
+ def unreserveUID(self, uid):
+ """
+ Unreserve a UID for this index's resource.
+ @param uid: the UID to reserve
+ @raise ReservationError: if C{uid} is not reserved
+ """
+
+ def _cb(result):
+ if result == False:
+ raise ReservationError(
+ "UID %s is not reserved for calendar collection %s."
+ % (uid, self.index.resource)
+ )
+ else:
+ try:
+ self.index._db_execute(
+ "delete from RESERVED where UID = :1", uid)
+ self.index._db_commit()
+ except sqlite.Error, e:
+ log.err("Unable to unreserve UID: %s", (e,))
+ self.index._db_rollback()
+ raise
+
+ d = self.isReservedUID(uid)
+ d.addCallback(_cb)
+ return d
+
+
+ @wrapInDeferred
+ def isReservedUID(self, uid):
+ """
+ Check to see whether a UID is reserved.
+ @param uid: the UID to check
+ @return: True if C{uid} is reserved, False otherwise.
+ """
+
+ rowiter = self.index._db_execute("select UID, TIME from RESERVED where UID = :1", uid)
+ for uid, attime in rowiter:
+ # Double check that the time is within a reasonable period of now
+ # otherwise we probably have a stale reservation
+ tm = time.strptime(attime[:19], "%Y-%m-%d %H:%M:%S")
+ dt = datetime.datetime(year=tm.tm_year, month=tm.tm_mon, day=tm.tm_mday, hour=tm.tm_hour, minute=tm.tm_min, second = tm.tm_sec)
+ if datetime.datetime.now() - dt > datetime.timedelta(seconds=config.UIDReservationTimeOut):
+ try:
+ self.index._db_execute("delete from RESERVED where UID = :1", uid)
+ self.index._db_commit()
+ except sqlite.Error, e:
+ log.err("Unable to unreserve UID: %s", (e,))
+ self.index._db_rollback()
+ raise
+ return False
+ else:
+ return True
+
+ return False
+
+
+
+class Index (CalendarIndex):
+ """
+ Calendar collection index - regular collection that enforces CalDAV UID uniqueness requirement.
+ """
+
+ def __init__(self, resource):
+ """
+ @param resource: the L{CalDAVResource} resource to
+ index. C{resource} must be a calendar collection (i.e.
+ C{resource.isPseudoCalendarCollection()} returns C{True}.)
+ """
+ assert resource.isCalendarCollection(), "non-calendar collection resource %s has no index." % (resource,)
+ super(Index, self).__init__(resource)
+
+ if (
+ hasattr(config, "Memcached") and
+ config.Memcached.Pools.Default.ClientEnabled
+ ):
+ self.reserver = MemcachedUIDReserver(self)
+ else:
+ self.reserver = SQLUIDReserver(self)
+
+ #
+ # A dict of sets. The dict keys are calendar collection paths,
+ # and the sets contains reserved UIDs for each path.
+ #
+
+ def reserveUID(self, uid):
+ return self.reserver.reserveUID(uid)
+
+
+ def unreserveUID(self, uid):
+ return self.reserver.unreserveUID(uid)
+
+
+ def isReservedUID(self, uid):
+ return self.reserver.isReservedUID(uid)
+
+
+ def isAllowedUID(self, uid, *names):
+ """
+ Checks to see whether to allow an operation which would add the
+ specified UID to the index. Specifically, the operation may not
+ violate the constraint that UIDs must be unique.
+ @param uid: the UID to check
+ @param names: the names of resources being replaced or deleted by the
+ operation; UIDs associated with these resources are not checked.
+ @return: True if the UID is not in the index and is not reserved,
+ False otherwise.
+ """
+ rname = self.resourceNameForUID(uid)
+ return (rname is None or rname in names)
+
+ def _db_type(self):
+ """
+ @return: the collection type assigned to this index.
+ """
+ return collection_types["Calendar"]
+
+ def _db_init_data_tables(self, q):
+ """
+ Initialise the underlying database tables.
+ @param q: a database cursor to use.
+ """
+
+ # Create database where the RESOURCE table has unique UID column.
+ self._db_init_data_tables_base(q, True)
+
+ def _db_recreate(self, do_commit=True):
+ """
+ Re-create the database tables from existing calendar data.
+ """
+
+ #
+ # Populate the DB with data from already existing resources.
+ # This allows for index recovery if the DB file gets
+ # deleted.
+ #
+ fp = self.resource.fp
+ for name in fp.listdir():
+ if name.startswith("."):
+ continue
+
+ try:
+ stream = fp.child(name).open()
+ except (IOError, OSError), e:
+ log.err("Unable to open resource %s: %s" % (name, e))
+ continue
+
+ # FIXME: This is blocking I/O
+ try:
+ calendar = Component.fromStream(stream)
+ calendar.validateForCalDAV()
+ except ValueError:
+ log.err("Non-calendar resource: %s" % (name,))
+ else:
+ #log.msg("Indexing resource: %s" % (name,))
+ self.addResource(name, calendar, True, reCreate=True)
+ finally:
+ stream.close()
+
+ # Do commit outside of the loop for better performance
+ if do_commit:
+ self._db_commit()
+
+class IndexSchedule (CalendarIndex):
+ """
+ Schedule collection index - does not require UID uniqueness.
+ """
+
+ def reserveUID(self, uid): #@UnusedVariable
+ """
+ Reserve a UID for this index's resource.
+ @param uid: the UID to reserve
+ @raise ReservationError: if C{uid} is already reserved
+ """
+
+ # iTIP does not require unique UIDs
+ return succeed(None)
+
+ def unreserveUID(self, uid): #@UnusedVariable
+ """
+ Unreserve a UID for this index's resource.
+ @param uid: the UID to reserve
+ @raise ReservationError: if C{uid} is not reserved
+ """
+
+ # iTIP does not require unique UIDs
+ return succeed(None)
+
+ def isReservedUID(self, uid): #@UnusedVariable
+ """
+ Check to see whether a UID is reserved.
+ @param uid: the UID to check
+ @return: True if C{uid} is reserved, False otherwise.
+ """
+
+ # iTIP does not require unique UIDs
+ return succeed(False)
+
+ def isAllowedUID(self, uid, *names): #@UnusedVariable
+ """
+ Checks to see whether to allow an operation with adds the the specified
+ UID is allowed to the index. Specifically, the operation may not
+ violate the constraint that UIDs must be unique, and the UID must not
+ be reserved.
+ @param uid: the UID to check
+ @param names: the names of resources being replaced or deleted by the
+ operation; UIDs associated with these resources are not checked.
+ @return: True if the UID is not in the index and is not reserved,
+ False otherwise.
+ """
+
+ # iTIP does not require unique UIDs
+ return True
+
+ def _db_type(self):
+ """
+ @return: the collection type assigned to this index.
+ """
+ return collection_types["iTIP"]
+
+ def _db_init_data_tables(self, q):
+ """
+ Initialise the underlying database tables.
+ @param q: a database cursor to use.
+ """
+
+ # Create database where the RESOURCE table has a UID column that is not unique.
+ self._db_init_data_tables_base(q, False)
+
+ def _db_recreate(self, do_commit=True):
+ """
+ Re-create the database tables from existing calendar data.
+ """
+
+ #
+ # Populate the DB with data from already existing resources.
+ # This allows for index recovery if the DB file gets
+ # deleted.
+ #
+ fp = self.resource.fp
+ for name in fp.listdir():
+ if name.startswith("."):
+ continue
+
+ try:
+ stream = fp.child(name).open()
+ except (IOError, OSError), e:
+ log.err("Unable to open resource %s: %s" % (name, e))
+ continue
+
+ # FIXME: This is blocking I/O
+ try:
+ calendar = Component.fromStream(stream)
+ calendar.validCalendarForCalDAV()
+ calendar.validateComponentsForCalDAV(True)
+ except ValueError:
+ log.err("Non-calendar resource: %s" % (name,))
+ else:
+ #log.msg("Indexing resource: %s" % (name,))
+ self.addResource(name, calendar, True, reCreate=True)
+ finally:
+ stream.close()
+
+ # Do commit outside of the loop for better performance
+ if do_commit:
+ self._db_commit()
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/scheduling.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/scheduling.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/scheduling.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -20,6 +20,7 @@
from twisted.python.util import FancyEqMixin
from twisted.python.components import proxyForInterface
+from twisted.internet.defer import inlineCallbacks, returnValue
@@ -39,16 +40,17 @@
self._transaction = transaction
+ @inlineCallbacks
def calendarHomeWithUID(self, uid, create=False):
# FIXME: 'create' flag
- newHome = super(ImplicitTransaction, self
+ newHome = yield super(ImplicitTransaction, self
).calendarHomeWithUID(uid, create)
# return ImplicitCalendarHome(newHome, self)
if newHome is None:
- return None
+ returnValue(None)
else:
# FIXME: relay transaction
- return ImplicitCalendarHome(newHome, None)
+ returnValue(ImplicitCalendarHome(newHome, None))
@@ -78,16 +80,18 @@
def createCalendarWithName(self, name):
self._calendarHome.createCalendarWithName(name)
+
def removeCalendarWithName(self, name):
self._calendarHome.removeCalendarWithName(name)
+ @inlineCallbacks
def calendarWithName(self, name):
- calendar = self._calendarHome.calendarWithName(name)
+ calendar = yield self._calendarHome.calendarWithName(name)
if calendar is not None:
- return ImplicitCalendar(self, calendar)
+ returnValue(ImplicitCalendar(self, calendar))
else:
- return None
+ returnValue(None)
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/sql.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/sql.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/sql.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -33,7 +33,7 @@
from twistedcaldav import caldavxml, customxml
from twistedcaldav.caldavxml import ScheduleCalendarTransp, Opaque
from twistedcaldav.dateops import normalizeForIndex, datetimeMktime
-from twistedcaldav.index import IndexedSearchException
+from txdav.common.icommondatastore import IndexedSearchException
from twistedcaldav.instance import InvalidOverriddenInstanceError
from txdav.caldav.datastore.util import validateCalendarComponent,\
@@ -93,6 +93,8 @@
Opaque())
self.createCalendarWithName("inbox")
+
+
class Calendar(CommonHomeChild):
"""
File-based implementation of L{ICalendar}.
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/common.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/common.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/common.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -19,11 +19,7 @@
Tests for common calendar store API functions.
"""
-from zope.interface.verify import verifyObject
-from zope.interface.exceptions import (
- BrokenMethodImplementation, DoesNotImplement)
-
-from twisted.internet.defer import Deferred, inlineCallbacks
+from twisted.internet.defer import Deferred, inlineCallbacks, returnValue
from twisted.internet.protocol import Protocol
from txdav.idav import IPropertyStore, IDataStore, AlreadyFinishedError
@@ -36,6 +32,7 @@
from txdav.common.icommondatastore import NoSuchObjectResourceError
from txdav.common.icommondatastore import ObjectResourceNameAlreadyExistsError
from txdav.common.inotifications import INotificationObject
+from txdav.common.datastore.test.util import CommonCommonTests
from txdav.caldav.icalendarstore import (
ICalendarObject, ICalendarHome,
@@ -47,7 +44,6 @@
from twext.web2.dav.element.base import WebDAVUnknownElement
from twext.python.vcomponent import VComponent
-from twistedcaldav.notify import Notifier
from twistedcaldav.customxml import InviteNotification, InviteSummary
storePath = FilePath(__file__).parent().child("calendar_store")
@@ -146,24 +142,8 @@
-def assertProvides(testCase, interface, provider):
+class CommonTests(CommonCommonTests):
"""
- Verify that C{provider} properly provides C{interface}
-
- @type interface: L{zope.interface.Interface}
- @type provider: C{provider}
- """
- try:
- verifyObject(interface, provider)
- except BrokenMethodImplementation, e:
- testCase.fail(e)
- except DoesNotImplement, e:
- testCase.fail("%r does not provide %s.%s" %
- (provider, interface.__module__, interface.getName()))
-
-
-class CommonTests(object):
- """
Tests for common functionality of interfaces defined in
L{txdav.caldav.icalendarstore}.
"""
@@ -194,74 +174,37 @@
raise NotImplementedError()
- lastTransaction = None
- savedStore = None
-
- def transactionUnderTest(self):
- """
- Create a transaction from C{storeUnderTest} and save it as
- C[lastTransaction}. Also makes sure to use the same store, saving the
- value from C{storeUnderTest}.
- """
- if self.lastTransaction is not None:
- return self.lastTransaction
- if self.savedStore is None:
- self.savedStore = self.storeUnderTest()
- txn = self.lastTransaction = self.savedStore.newTransaction(self.id())
- return txn
-
-
- def commit(self):
- """
- Commit the last transaction created from C{transactionUnderTest}, and
- clear it.
- """
- self.lastTransaction.commit()
- self.lastTransaction = None
-
-
- def abort(self):
- """
- Abort the last transaction created from C[transactionUnderTest}, and
- clear it.
- """
- self.lastTransaction.abort()
- self.lastTransaction = None
-
-
- def setUp(self):
- self.notifierFactory = StubNotifierFactory()
-
-
- def tearDown(self):
- if self.lastTransaction is not None:
- self.commit()
-
-
+ @inlineCallbacks
def homeUnderTest(self):
"""
Get the calendar home detailed by C{requirements['home1']}.
"""
- return self.transactionUnderTest().calendarHomeWithUID("home1")
+ returnValue(
+ (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+ )
+ @inlineCallbacks
def calendarUnderTest(self):
"""
Get the calendar detailed by C{requirements['home1']['calendar_1']}.
"""
- return self.homeUnderTest().calendarWithName("calendar_1")
+ returnValue((yield
+ (yield self.homeUnderTest()).calendarWithName("calendar_1"))
+ )
+ @inlineCallbacks
def calendarObjectUnderTest(self):
"""
Get the calendar detailed by
C{requirements['home1']['calendar_1']['1.ics']}.
"""
- return self.calendarUnderTest().calendarObjectWithName("1.ics")
+ returnValue(
+ (yield (yield self.calendarUnderTest())
+ .calendarObjectWithName("1.ics")))
- assertProvides = assertProvides
-
def test_calendarStoreProvides(self):
"""
The calendar store provides L{IDataStore} and its required attributes.
@@ -281,28 +224,33 @@
self.assertProvides(ICalendarTransaction, txn)
+ @inlineCallbacks
def test_homeProvides(self):
"""
The calendar homes generated by the calendar store provide
L{ICalendarHome} and its required attributes.
"""
- self.assertProvides(ICalendarHome, self.homeUnderTest())
+ self.assertProvides(ICalendarHome, (yield self.homeUnderTest()))
+ @inlineCallbacks
def test_calendarProvides(self):
"""
The calendars generated by the calendar store provide L{ICalendar} and
its required attributes.
"""
- self.assertProvides(ICalendar, self.calendarUnderTest())
+ self.assertProvides(ICalendar, (yield self.calendarUnderTest()))
+ @inlineCallbacks
def test_calendarObjectProvides(self):
"""
The calendar objects generated by the calendar store provide
L{ICalendarObject} and its required attributes.
"""
- self.assertProvides(ICalendarObject, self.calendarObjectUnderTest())
+ self.assertProvides(
+ ICalendarObject, (yield self.calendarObjectUnderTest())
+ )
def notificationUnderTest(self):
@@ -362,144 +310,160 @@
self.assertIdentical(collection, notification.notificationCollection())
+ @inlineCallbacks
def test_notifierID(self):
- home = self.homeUnderTest()
+ home = yield self.homeUnderTest()
self.assertEquals(home.notifierID(), "CalDAV|home1")
- calendar = home.calendarWithName("calendar_1")
+ calendar = yield home.calendarWithName("calendar_1")
self.assertEquals(calendar.notifierID(), "CalDAV|home1")
self.assertEquals(calendar.notifierID(label="collection"), "CalDAV|home1/calendar_1")
+ @inlineCallbacks
def test_calendarHomeWithUID_exists(self):
"""
Finding an existing calendar home by UID results in an object that
provides L{ICalendarHome} and has a C{uid()} method that returns the
same value that was passed in.
"""
- calendarHome = (self.transactionUnderTest()
+ calendarHome = (yield self.transactionUnderTest()
.calendarHomeWithUID("home1"))
self.assertEquals(calendarHome.uid(), "home1")
self.assertProvides(ICalendarHome, calendarHome)
+ @inlineCallbacks
def test_calendarHomeWithUID_absent(self):
"""
L{ICommonStoreTransaction.calendarHomeWithUID} should return C{None}
when asked for a non-existent calendar home.
"""
txn = self.transactionUnderTest()
- self.assertEquals(txn.calendarHomeWithUID("xyzzy"), None)
+ self.assertEquals((yield txn.calendarHomeWithUID("xyzzy")), None)
+ @inlineCallbacks
def test_calendarWithName_exists(self):
"""
L{ICalendarHome.calendarWithName} returns an L{ICalendar} provider,
whose name matches the one passed in.
"""
- home = self.homeUnderTest()
+ home = yield self.homeUnderTest()
for name in home1_calendarNames:
- calendar = home.calendarWithName(name)
+ calendar = yield home.calendarWithName(name)
if calendar is None:
self.fail("calendar %r didn't exist" % (name,))
self.assertProvides(ICalendar, calendar)
self.assertEquals(calendar.name(), name)
+ @inlineCallbacks
def test_calendarRename(self):
"""
L{ICalendar.rename} changes the name of the L{ICalendar}.
"""
- home = self.homeUnderTest()
- calendar = home.calendarWithName("calendar_1")
+ home = yield self.homeUnderTest()
+ calendar = yield home.calendarWithName("calendar_1")
calendar.rename("some_other_name")
+ @inlineCallbacks
def positiveAssertions():
self.assertEquals(calendar.name(), "some_other_name")
- self.assertEquals(calendar, home.calendarWithName("some_other_name"))
- self.assertEquals(None, home.calendarWithName("calendar_1"))
- positiveAssertions()
- self.commit()
- home = self.homeUnderTest()
- calendar = home.calendarWithName("some_other_name")
- positiveAssertions()
+ self.assertEquals(
+ calendar, (yield home.calendarWithName("some_other_name")))
+ self.assertEquals(
+ None, (yield home.calendarWithName("calendar_1")))
+ yield positiveAssertions()
+ yield self.commit()
+ home = yield self.homeUnderTest()
+ calendar = yield home.calendarWithName("some_other_name")
+ yield positiveAssertions()
# FIXME: revert
# FIXME: test for multiple renames
# FIXME: test for conflicting renames (a->b, c->a in the same txn)
+ @inlineCallbacks
def test_calendarWithName_absent(self):
"""
L{ICalendarHome.calendarWithName} returns C{None} for calendars which
do not exist.
"""
- self.assertEquals(self.homeUnderTest().calendarWithName("xyzzy"),
- None)
+ home = yield self.homeUnderTest()
+ calendar = yield home.calendarWithName("xyzzy")
+ self.assertEquals(calendar, None)
+ @inlineCallbacks
def test_createCalendarWithName_absent(self):
"""
L{ICalendarHome.createCalendarWithName} creates a new L{ICalendar} that
can be retrieved with L{ICalendarHome.calendarWithName}.
"""
- home = self.homeUnderTest()
+ home = yield self.homeUnderTest()
name = "new"
- self.assertIdentical(home.calendarWithName(name), None)
+ self.assertIdentical((yield home.calendarWithName(name)), None)
home.createCalendarWithName(name)
- self.assertNotIdentical(home.calendarWithName(name), None)
+ self.assertNotIdentical((yield home.calendarWithName(name)), None)
+ @inlineCallbacks
def checkProperties():
- calendarProperties = home.calendarWithName(name).properties()
+ calendarProperties = (
+ yield home.calendarWithName(name)).properties()
self.assertEquals(
calendarProperties[
PropertyName.fromString(davxml.ResourceType.sname())
],
davxml.ResourceType.calendar #@UndefinedVariable
)
- checkProperties()
+ yield checkProperties()
- self.commit()
+ yield self.commit()
# Make sure notification fired after commit
self.assertEquals(self.notifierFactory.history,
[("update", "CalDAV|home1")])
# Make sure it's available in a new transaction; i.e. test the commit.
- home = self.homeUnderTest()
- self.assertNotIdentical(home.calendarWithName(name), None)
+ home = yield self.homeUnderTest()
+ self.assertNotIdentical((yield home.calendarWithName(name)), None)
# Sanity check: are the properties actually persisted? Check in
# subsequent transaction.
- checkProperties()
+ yield checkProperties()
# FIXME: no independent testing of the property store's persistence
# right now
+ @inlineCallbacks
def test_createCalendarWithName_exists(self):
"""
L{ICalendarHome.createCalendarWithName} raises
L{CalendarAlreadyExistsError} when the name conflicts with an already-
existing
"""
+ home = yield self.homeUnderTest()
for name in home1_calendarNames:
self.assertRaises(
HomeChildNameAlreadyExistsError,
- self.homeUnderTest().createCalendarWithName, name
+ home.createCalendarWithName, name
)
+ @inlineCallbacks
def test_removeCalendarWithName_exists(self):
"""
L{ICalendarHome.removeCalendarWithName} removes a calendar that already
exists.
"""
- home = self.homeUnderTest()
+ home = yield self.homeUnderTest()
# FIXME: test transactions
for name in home1_calendarNames:
- self.assertNotIdentical(home.calendarWithName(name), None)
+ self.assertNotIdentical((yield home.calendarWithName(name)), None)
home.removeCalendarWithName(name)
- self.assertEquals(home.calendarWithName(name), None)
+ self.assertEquals((yield home.calendarWithName(name)), None)
- self.commit()
+ yield self.commit()
# Make sure notification fired after commit
self.assertEquals(
@@ -515,22 +479,24 @@
)
+ @inlineCallbacks
def test_removeCalendarWithName_absent(self):
"""
Attempt to remove an non-existing calendar should raise.
"""
- home = self.homeUnderTest()
+ home = yield self.homeUnderTest()
self.assertRaises(NoSuchHomeChildError,
home.removeCalendarWithName, "xyzzy")
+ @inlineCallbacks
def test_calendarObjects(self):
"""
L{ICalendar.calendarObjects} will enumerate the calendar objects present
in the filesystem, in name order, but skip those with hidden names.
"""
- calendar1 = self.calendarUnderTest()
- calendarObjects = list(calendar1.calendarObjects())
+ calendar1 = yield self.calendarUnderTest()
+ calendarObjects = list((yield calendar1.calendarObjects()))
for calendarObject in calendarObjects:
self.assertProvides(ICalendarObject, calendarObject)
@@ -551,50 +517,54 @@
removed by L{Calendar.removeCalendarObjectWithName} in the same
transaction, even if it has not yet been committed.
"""
- calendar1 = self.calendarUnderTest()
+ calendar1 = yield self.calendarUnderTest()
calendar1.removeCalendarObjectWithName("2.ics")
calendarObjects = list(calendar1.calendarObjects())
self.assertEquals(set(o.name() for o in calendarObjects),
set(calendar1_objectNames) - set(["2.ics"]))
+ @inlineCallbacks
def test_ownerCalendarHome(self):
"""
L{ICalendar.ownerCalendarHome} should match the home UID.
"""
self.assertEquals(
- self.calendarUnderTest().ownerCalendarHome().uid(),
- self.homeUnderTest().uid()
+ (yield self.calendarUnderTest()).ownerCalendarHome().uid(),
+ (yield self.homeUnderTest()).uid()
)
+ @inlineCallbacks
def test_calendarObjectWithName_exists(self):
"""
L{ICalendar.calendarObjectWithName} returns an L{ICalendarObject}
provider for calendars which already exist.
"""
- calendar1 = self.calendarUnderTest()
+ calendar1 = yield self.calendarUnderTest()
for name in calendar1_objectNames:
- calendarObject = calendar1.calendarObjectWithName(name)
+ calendarObject = yield calendar1.calendarObjectWithName(name)
self.assertProvides(ICalendarObject, calendarObject)
self.assertEquals(calendarObject.name(), name)
# FIXME: add more tests based on CommonTests.requirements
+ @inlineCallbacks
def test_calendarObjectWithName_absent(self):
"""
L{ICalendar.calendarObjectWithName} returns C{None} for calendars which
don't exist.
"""
- calendar1 = self.calendarUnderTest()
- self.assertEquals(calendar1.calendarObjectWithName("xyzzy"), None)
+ calendar1 = yield self.calendarUnderTest()
+ self.assertEquals((yield calendar1.calendarObjectWithName("xyzzy")), None)
+ @inlineCallbacks
def test_removeCalendarObjectWithUID_exists(self):
"""
Remove an existing calendar object.
"""
- calendar = self.calendarUnderTest()
+ calendar = yield self.calendarUnderTest()
for name in calendar1_objectNames:
uid = (u'uid' + name.rstrip(".ics"))
self.assertNotIdentical(calendar.calendarObjectWithUID(uid),
@@ -610,7 +580,7 @@
)
# Make sure notifications are fired after commit
- self.commit()
+ yield self.commit()
self.assertEquals(
self.notifierFactory.history,
[
@@ -623,11 +593,12 @@
]
)
+ @inlineCallbacks
def test_removeCalendarObjectWithName_exists(self):
"""
Remove an existing calendar object.
"""
- calendar = self.calendarUnderTest()
+ calendar = yield self.calendarUnderTest()
for name in calendar1_objectNames:
self.assertNotIdentical(
calendar.calendarObjectWithName(name), None
@@ -638,37 +609,44 @@
)
+ @inlineCallbacks
def test_removeCalendarObjectWithName_absent(self):
"""
Attempt to remove an non-existing calendar object should raise.
"""
- calendar = self.calendarUnderTest()
+ calendar = yield self.calendarUnderTest()
self.assertRaises(
NoSuchObjectResourceError,
calendar.removeCalendarObjectWithName, "xyzzy"
)
+ @inlineCallbacks
def test_calendarName(self):
"""
L{Calendar.name} reflects the name of the calendar.
"""
- self.assertEquals(self.calendarUnderTest().name(), "calendar_1")
+ self.assertEquals((yield self.calendarUnderTest()).name(), "calendar_1")
+ @inlineCallbacks
def test_calendarObjectName(self):
"""
L{ICalendarObject.name} reflects the name of the calendar object.
"""
- self.assertEquals(self.calendarObjectUnderTest().name(), "1.ics")
+ self.assertEquals(
+ (yield self.calendarObjectUnderTest()).name(),
+ "1.ics"
+ )
+ @inlineCallbacks
def test_component(self):
"""
L{ICalendarObject.component} returns a L{VComponent} describing the
calendar data underlying that calendar object.
"""
- component = self.calendarObjectUnderTest().component()
+ component = (yield self.calendarObjectUnderTest()).component()
self.failUnless(
isinstance(component, VComponent),
@@ -680,24 +658,28 @@
self.assertEquals(component.resourceUID(), "uid1")
+ @inlineCallbacks
def test_iCalendarText(self):
"""
L{ICalendarObject.iCalendarText} returns a C{str} describing the same
data provided by L{ICalendarObject.component}.
"""
- text = self.calendarObjectUnderTest().iCalendarText()
+ text = (yield self.calendarObjectUnderTest()).iCalendarText()
self.assertIsInstance(text, str)
self.failUnless(text.startswith("BEGIN:VCALENDAR\r\n"))
self.assertIn("\r\nUID:uid1\r\n", text)
self.failUnless(text.endswith("\r\nEND:VCALENDAR\r\n"))
+ @inlineCallbacks
def test_calendarObjectUID(self):
"""
L{ICalendarObject.uid} returns a C{str} describing the C{UID} property
of the calendar object's component.
"""
- self.assertEquals(self.calendarObjectUnderTest().uid(), "uid1")
+ self.assertEquals(
+ (yield self.calendarObjectUnderTest()).uid(), "uid1"
+ )
def test_organizer(self):
@@ -707,20 +689,23 @@
component.
"""
self.assertEquals(
- self.calendarObjectUnderTest().organizer(),
+ (yield self.calendarObjectUnderTest()).organizer(),
"mailto:wsanchez at apple.com"
)
+ @inlineCallbacks
def test_calendarObjectWithUID_absent(self):
"""
L{ICalendar.calendarObjectWithUID} returns C{None} for calendars which
don't exist.
"""
- calendar1 = self.calendarUnderTest()
- self.assertEquals(calendar1.calendarObjectWithUID("xyzzy"), None)
+ calendar1 = yield self.calendarUnderTest()
+ self.assertEquals((yield calendar1.calendarObjectWithUID("xyzzy")),
+ None)
+ @inlineCallbacks
def test_calendars(self):
"""
L{ICalendarHome.calendars} returns an iterable of L{ICalendar}
@@ -729,13 +714,13 @@
"""
# Add a dot directory to make sure we don't find it
# self.home1._path.child(".foo").createDirectory()
- home = self.homeUnderTest()
- calendars = list(home.calendars())
+ home = yield self.homeUnderTest()
+ calendars = list((yield home.calendars()))
for calendar in calendars:
self.assertProvides(ICalendar, calendar)
self.assertEquals(calendar,
- home.calendarWithName(calendar.name()))
+ (yield home.calendarWithName(calendar.name())))
self.assertEquals(
set(c.name() for c in calendars),
@@ -743,24 +728,26 @@
)
+ @inlineCallbacks
def test_calendarsAfterAddCalendar(self):
"""
L{ICalendarHome.calendars} includes calendars recently added with
L{ICalendarHome.createCalendarWithName}.
"""
- home = self.homeUnderTest()
+ home = yield self.homeUnderTest()
before = set(x.name() for x in home.calendars())
home.createCalendarWithName("new-name")
after = set(x.name() for x in home.calendars())
self.assertEquals(before | set(['new-name']), after)
+ @inlineCallbacks
def test_createCalendarObjectWithName_absent(self):
"""
L{ICalendar.createCalendarObjectWithName} creates a new
L{ICalendarObject}.
"""
- calendar1 = self.calendarUnderTest()
+ calendar1 = yield self.calendarUnderTest()
name = "4.ics"
self.assertIdentical(calendar1.calendarObjectWithName(name), None)
component = VComponent.fromString(event4_text)
@@ -769,7 +756,7 @@
calendarObject = calendar1.calendarObjectWithName(name)
self.assertEquals(calendarObject.component(), component)
- self.commit()
+ yield self.commit()
# Make sure notifications fire after commit
self.assertEquals(
@@ -787,7 +774,7 @@
L{CalendarObjectNameAlreadyExistsError} if a calendar object with the
given name already exists in that calendar.
"""
- cal = self.calendarUnderTest()
+ cal = yield self.calendarUnderTest()
comp = VComponent.fromString(event4_text)
self.assertRaises(
ObjectResourceNameAlreadyExistsError,
@@ -796,6 +783,7 @@
)
+ @inlineCallbacks
def test_createCalendarObjectWithName_invalid(self):
"""
L{ICalendar.createCalendarObjectWithName} raises
@@ -804,17 +792,17 @@
"""
self.assertRaises(
InvalidObjectResourceError,
- self.calendarUnderTest().createCalendarObjectWithName,
+ (yield self.calendarUnderTest()).createCalendarObjectWithName,
"new", VComponent.fromString(event4notCalDAV_text)
)
-
+ @inlineCallbacks
def test_setComponent_invalid(self):
"""
L{ICalendarObject.setComponent} raises L{InvalidICalendarDataError} if
presented with invalid iCalendar text.
"""
- calendarObject = self.calendarObjectUnderTest()
+ calendarObject = yield self.calendarObjectUnderTest()
self.assertRaises(
InvalidObjectResourceError,
calendarObject.setComponent,
@@ -822,20 +810,22 @@
)
+ @inlineCallbacks
def test_setComponent_uidchanged(self):
"""
L{ICalendarObject.setComponent} raises L{InvalidCalendarComponentError}
when given a L{VComponent} whose UID does not match its existing UID.
"""
- calendar1 = self.calendarUnderTest()
+ calendar1 = yield self.calendarUnderTest()
component = VComponent.fromString(event4_text)
- calendarObject = calendar1.calendarObjectWithName("1.ics")
+ calendarObject = yield calendar1.calendarObjectWithName("1.ics")
self.assertRaises(
InvalidObjectResourceError,
calendarObject.setComponent, component
)
+ @inlineCallbacks
def test_calendarHomeWithUID_create(self):
"""
L{ICommonStoreTransaction.calendarHomeWithUID} with C{create=True}
@@ -843,25 +833,27 @@
"""
txn = self.transactionUnderTest()
noHomeUID = "xyzzy"
- calendarHome = txn.calendarHomeWithUID(
+ calendarHome = yield txn.calendarHomeWithUID(
noHomeUID,
create=True
)
+ @inlineCallbacks
def readOtherTxn():
otherTxn = self.savedStore.newTransaction(self.id() + "other txn")
self.addCleanup(otherTxn.commit)
- return otherTxn.calendarHomeWithUID(noHomeUID)
+ returnValue((yield otherTxn.calendarHomeWithUID(noHomeUID)))
self.assertProvides(ICalendarHome, calendarHome)
# Default calendar should be automatically created.
self.assertProvides(ICalendar,
- calendarHome.calendarWithName("calendar"))
+ (yield calendarHome.calendarWithName("calendar")))
# A concurrent transaction shouldn't be able to read it yet:
- self.assertIdentical(readOtherTxn(), None)
- self.commit()
+ self.assertIdentical((yield readOtherTxn()), None)
+ yield self.commit()
# But once it's committed, other transactions should see it.
- self.assertProvides(ICalendarHome, readOtherTxn())
+ self.assertProvides(ICalendarHome, (yield readOtherTxn()))
+ @inlineCallbacks
def test_setComponent(self):
"""
L{CalendarObject.setComponent} changes the result of
@@ -869,8 +861,8 @@
"""
component = VComponent.fromString(event1modified_text)
- calendar1 = self.calendarUnderTest()
- calendarObject = calendar1.calendarObjectWithName("1.ics")
+ calendar1 = yield self.calendarUnderTest()
+ calendarObject = yield calendar1.calendarObjectWithName("1.ics")
oldComponent = calendarObject.component()
self.assertNotEqual(component, oldComponent)
calendarObject.setComponent(component)
@@ -880,7 +872,7 @@
calendarObject = calendar1.calendarObjectWithName("1.ics")
self.assertEquals(calendarObject.component(), component)
- self.commit()
+ yield self.commit()
# Make sure notification fired after commit
self.assertEquals(
@@ -901,40 +893,45 @@
self.assertProvides(IPropertyStore, properties)
+ @inlineCallbacks
def test_homeProperties(self):
"""
L{ICalendarHome.properties} returns a property store.
"""
- self.checkPropertiesMethod(self.homeUnderTest())
+ self.checkPropertiesMethod((yield self.homeUnderTest()))
+ @inlineCallbacks
def test_calendarProperties(self):
"""
L{ICalendar.properties} returns a property store.
"""
- self.checkPropertiesMethod(self.calendarUnderTest())
+ self.checkPropertiesMethod((yield self.calendarUnderTest()))
+ @inlineCallbacks
def test_calendarObjectProperties(self):
"""
L{ICalendarObject.properties} returns a property store.
"""
- self.checkPropertiesMethod(self.calendarObjectUnderTest())
+ self.checkPropertiesMethod((yield self.calendarObjectUnderTest()))
+ @inlineCallbacks
def test_newCalendarObjectProperties(self):
"""
L{ICalendarObject.properties} returns an empty property store for a
calendar object which has been created but not committed.
"""
- calendar = self.calendarUnderTest()
+ calendar = yield self.calendarUnderTest()
calendar.createCalendarObjectWithName(
"4.ics", VComponent.fromString(event4_text)
)
- newEvent = calendar.calendarObjectWithName("4.ics")
+ newEvent = yield calendar.calendarObjectWithName("4.ics")
self.assertEquals(newEvent.properties().items(), [])
+ @inlineCallbacks
def test_setComponentPreservesProperties(self):
"""
L{ICalendarObject.setComponent} preserves properties.
@@ -948,15 +945,15 @@
propertyContent.name = propertyName.name
propertyContent.namespace = propertyName.namespace
- self.calendarObjectUnderTest().properties()[
+ (yield self.calendarObjectUnderTest()).properties()[
propertyName] = propertyContent
- self.commit()
+ yield self.commit()
# Sanity check; are properties even readable in a separate transaction?
# Should probably be a separate test.
self.assertEquals(
- self.calendarObjectUnderTest().properties()[propertyName],
+ (yield self.calendarObjectUnderTest()).properties()[propertyName],
propertyContent)
- obj = self.calendarObjectUnderTest()
+ obj = yield self.calendarObjectUnderTest()
event1_text = obj.iCalendarText()
event1_text_withDifferentSubject = event1_text.replace(
"SUMMARY:CalDAV protocol updates",
@@ -969,9 +966,9 @@
# Putting everything into a separate transaction to account for any
# caching that may take place.
- self.commit()
+ yield self.commit()
self.assertEquals(
- self.calendarObjectUnderTest().properties()[propertyName],
+ (yield self.calendarObjectUnderTest()).properties()[propertyName],
propertyContent
)
@@ -1014,20 +1011,23 @@
END:VCALENDAR
""".strip().split("\n"))
+
+ @inlineCallbacks
def test_dropboxID(self):
"""
L{ICalendarObject.dropboxID} should synthesize its dropbox from the X
-APPLE-DROPBOX property, if available.
"""
- cal = self.calendarUnderTest()
+ cal = yield self.calendarUnderTest()
cal.createCalendarObjectWithName("drop.ics", VComponent.fromString(
self.eventWithDropbox
)
)
- obj = cal.calendarObjectWithName("drop.ics")
+ obj = yield cal.calendarObjectWithName("drop.ics")
self.assertEquals(obj.dropboxID(), "some-dropbox-id")
+ @inlineCallbacks
def test_indexByDropboxProperty(self):
"""
L{ICalendarHome.calendarObjectWithDropboxID} will return a calendar
@@ -1035,15 +1035,15 @@
-APPLE-DROPBOX} property URI.
"""
objName = "with-dropbox.ics"
- cal = self.calendarUnderTest()
+ cal = yield self.calendarUnderTest()
cal.createCalendarObjectWithName(
objName, VComponent.fromString(
self.eventWithDropbox
)
)
- self.commit()
- home = self.homeUnderTest()
- cal = self.calendarUnderTest()
+ yield self.commit()
+ home = yield self.homeUnderTest()
+ cal = yield self.calendarUnderTest()
fromName = cal.calendarObjectWithName(objName)
fromDropbox = home.calendarObjectWithDropboxID("some-dropbox-id")
self.assertEquals(fromName, fromDropbox)
@@ -1054,12 +1054,12 @@
"""
Common logic for attachment-creation tests.
"""
- obj = self.calendarObjectUnderTest()
+ obj = yield self.calendarObjectUnderTest()
t = obj.createAttachmentWithName("new.attachment", MimeType("text", "x-fixture"))
t.write("new attachment")
t.write(" text")
t.loseConnection()
- obj = refresh(obj)
+ obj = yield refresh(obj)
class CaptureProtocol(Protocol):
buf = ''
def dataReceived(self, data):
@@ -1098,9 +1098,11 @@
L{IAttachment} object that can be retrieved by
L{ICalendarObject.attachmentWithName} in subsequent transactions.
"""
+ @inlineCallbacks
def refresh(obj):
- self.commit()
- return self.calendarObjectUnderTest()
+ yield self.commit()
+ result = yield self.calendarObjectUnderTest()
+ returnValue(result)
return self.createAttachmentTest(refresh)
@@ -1109,10 +1111,11 @@
L{ICalendarObject.removeAttachmentWithName} will remove the calendar
object with the given name.
"""
+ @inlineCallbacks
def deleteIt(ignored):
- obj = self.calendarObjectUnderTest()
+ obj = yield self.calendarObjectUnderTest()
obj.removeAttachmentWithName("new.attachment")
- obj = refresh(obj)
+ obj = yield refresh(obj)
self.assertIdentical(
None, obj.attachmentWithName("new.attachment")
)
@@ -1125,64 +1128,74 @@
L{ICalendarObject.removeAttachmentWithName} will remove the calendar
object with the given name. (After commit, it will still be gone.)
"""
+ @inlineCallbacks
def refresh(obj):
- self.commit()
- return self.calendarObjectUnderTest()
+ yield self.commit()
+ result = yield self.calendarObjectUnderTest()
+ returnValue(result)
return self.test_removeAttachmentWithName(refresh)
+ @inlineCallbacks
def test_noDropboxCalendar(self):
"""
L{ICalendarObject.createAttachmentWithName} may create a directory
named 'dropbox', but this should not be seen as a calendar by
L{ICalendarHome.calendarWithName} or L{ICalendarHome.calendars}.
"""
- obj = self.calendarObjectUnderTest()
- t = obj.createAttachmentWithName("new.attachment", MimeType("text", "plain"))
+ obj = yield self.calendarObjectUnderTest()
+ t = obj.createAttachmentWithName("new.attachment",
+ MimeType("text", "plain"))
t.write("new attachment text")
t.loseConnection()
- self.commit()
- self.assertEquals(self.homeUnderTest().calendarWithName("dropbox"),
- None)
+ yield self.commit()
+ home = (yield self.homeUnderTest())
+ calendars = (yield home.calendars())
+ self.assertEquals((yield home.calendarWithName("dropbox")), None)
self.assertEquals(
- set([n.name() for n in self.homeUnderTest().calendars()]),
+ set([n.name() for n in calendars]),
set(home1_calendarNames))
+ @inlineCallbacks
def test_finishedOnCommit(self):
"""
Calling L{ITransaction.abort} or L{ITransaction.commit} after
L{ITransaction.commit} has already been called raises an
L{AlreadyFinishedError}.
"""
- self.calendarObjectUnderTest()
+ yield self.calendarObjectUnderTest()
txn = self.lastTransaction
- self.commit()
+ yield self.commit()
self.assertRaises(AlreadyFinishedError, txn.commit)
self.assertRaises(AlreadyFinishedError, txn.abort)
+ @inlineCallbacks
def test_dontLeakCalendars(self):
"""
Calendars in one user's calendar home should not show up in another
user's calendar home.
"""
- home2 = self.transactionUnderTest().calendarHomeWithUID(
+ home2 = yield self.transactionUnderTest().calendarHomeWithUID(
"home2", create=True)
- self.assertIdentical(home2.calendarWithName("calendar_1"), None)
+ self.assertIdentical(
+ (yield home2.calendarWithName("calendar_1")), None)
+ @inlineCallbacks
def test_dontLeakObjects(self):
"""
Calendar objects in one user's calendar should not show up in another
user's via uid or name queries.
"""
- home1 = self.homeUnderTest()
- home2 = self.transactionUnderTest().calendarHomeWithUID(
+ home1 = yield self.homeUnderTest()
+ home2 = yield self.transactionUnderTest().calendarHomeWithUID(
"home2", create=True)
- calendar1 = home1.calendarWithName("calendar_1")
- calendar2 = home2.calendarWithName("calendar")
- objects = list(home2.calendarWithName("calendar").calendarObjects())
+ calendar1 = yield home1.calendarWithName("calendar_1")
+ calendar2 = yield home2.calendarWithName("calendar")
+ objects = list(
+ (yield (yield home2.calendarWithName("calendar")).calendarObjects()))
self.assertEquals(objects, [])
for resourceName in self.requirements['home1']['calendar_1'].keys():
obj = calendar1.calendarObjectWithName(resourceName)
@@ -1192,6 +1205,7 @@
calendar2.calendarObjectWithUID(obj.uid()), None)
+ @inlineCallbacks
def test_eachCalendarHome(self):
"""
L{ICalendarTransaction.eachCalendarHome} returns an iterator that
@@ -1201,8 +1215,10 @@
additionalUIDs = set('alpha-uid home2 home3 beta-uid'.split())
txn = self.transactionUnderTest()
for name in additionalUIDs:
- txn.calendarHomeWithUID(name, create=True)
- self.commit()
+ # maybe it's not actually necessary to yield (i.e. wait) for each
+ # one? commit() should wait for all of them.
+ yield txn.calendarHomeWithUID(name, create=True)
+ yield self.commit()
foundUIDs = set([])
lastTxn = None
for txn, home in self.storeUnderTest().eachCalendarHome():
@@ -1219,18 +1235,3 @@
-class StubNotifierFactory(object):
-
- """ For testing push notifications without an XMPP server """
-
- def __init__(self):
- self.reset()
-
- def newNotifier(self, label="default", id=None, prefix=None):
- return Notifier(self, label=label, id=id, prefix=prefix)
-
- def send(self, op, id):
- self.history.append((op, id))
-
- def reset(self):
- self.history = []
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_file.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_file.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_file.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -22,9 +22,11 @@
# deleted and replaced with either implementation-specific methods on
# FileStorageTests, or implementation-agnostic methods on CommonTests.
-from twext.python.filepath import CachingFilePath as FilePath
from twisted.trial import unittest
+from twisted.internet.defer import inlineCallbacks
+from twext.python.filepath import CachingFilePath as FilePath
+
from twext.python.vcomponent import VComponent
from txdav.common.icommondatastore import HomeChildNameNotAllowedError
@@ -63,21 +65,22 @@
storePath.copyTo(calendarPath)
test.calendarStore = CalendarStore(storeRootPath, test.notifierFactory)
- test.txn = test.calendarStore.newTransaction()
+ test.txn = test.calendarStore.newTransaction(test.id() + "(old)")
assert test.calendarStore is not None, "No calendar store?"
+ at inlineCallbacks
def setUpHome1(test):
setUpCalendarStore(test)
- test.home1 = test.txn.calendarHomeWithUID("home1")
+ test.home1 = yield test.txn.calendarHomeWithUID("home1")
assert test.home1 is not None, "No calendar home?"
-
+ at inlineCallbacks
def setUpCalendar1(test):
- setUpHome1(test)
- test.calendar1 = test.home1.calendarWithName("calendar_1")
+ yield setUpHome1(test)
+ test.calendar1 = yield test.home1.calendarWithName("calendar_1")
assert test.calendar1 is not None, "No calendar?"
@@ -109,7 +112,7 @@
notifierFactory = None
def setUp(self):
- setUpHome1(self)
+ return setUpHome1(self)
def test_init(self):
@@ -167,7 +170,7 @@
notifierFactory = None
def setUp(self):
- setUpCalendar1(self)
+ return setUpCalendar1(self)
def test_init(self):
@@ -185,27 +188,28 @@
)
+ @inlineCallbacks
def test_useIndexImmediately(self):
"""
L{Calendar._index} is usable in the same transaction it is created, with
a temporary filename.
"""
self.home1.createCalendarWithName("calendar2")
- calendar = self.home1.calendarWithName("calendar2")
+ calendar = yield self.home1.calendarWithName("calendar2")
index = calendar._index
- self.assertEquals(set(index.calendarObjects()),
- set(calendar.calendarObjects()))
- self.txn.commit()
+ yield self.assertEquals(set(index.calendarObjects()),
+ set(calendar.calendarObjects()))
+ yield self.txn.commit()
self.txn = self.calendarStore.newTransaction()
- self.home1 = self.txn.calendarHomeWithUID("home1")
- calendar = self.home1.calendarWithName("calendar2")
+ self.home1 = yield self.txn.calendarHomeWithUID("home1")
+ calendar = yield self.home1.calendarWithName("calendar2")
# FIXME: we should be curating our own index here, but in order to fix
# that the code in the old implicit scheduler needs to change. This
# test would be more effective if there were actually some objects in
# this list.
index = calendar._index
- self.assertEquals(set(index.calendarObjects()),
- set(calendar.calendarObjects()))
+ self.assertEquals(set((yield index.calendarObjects())),
+ set((yield calendar.calendarObjects())))
def test_calendarObjectWithName_dot(self):
@@ -264,6 +268,7 @@
)
+ @inlineCallbacks
def test_removeCalendarObject_delayedEffect(self):
"""
Removing a calendar object should not immediately remove the underlying
@@ -271,7 +276,7 @@
"""
self.calendar1.removeCalendarObjectWithName("2.ics")
self.failUnless(self.calendar1._path.child("2.ics").exists())
- self.txn.commit()
+ yield self.txn.commit()
self.failIf(self.calendar1._path.child("2.ics").exists())
@@ -289,16 +294,22 @@
)
+ counter = 0
+ @inlineCallbacks
def _refresh(self):
"""
Re-read the (committed) home1 and calendar1 objects in a new
transaction.
"""
- self.txn = self.calendarStore.newTransaction()
- self.home1 = self.txn.calendarHomeWithUID("home1")
- self.calendar1 = self.home1.calendarWithName("calendar_1")
+ self.counter += 1
+ self.txn = self.calendarStore.newTransaction(
+ self.id() + " (old #" + str(self.counter) + ")"
+ )
+ self.home1 = yield self.txn.calendarHomeWithUID("home1")
+ self.calendar1 = yield self.home1.calendarWithName("calendar_1")
+ @inlineCallbacks
def test_undoCreateCalendarObject(self):
"""
If a calendar object is created as part of a transaction, it will be
@@ -306,19 +317,22 @@
"""
# Make sure that the calendar home is actually committed; rolling back
# calendar home creation will remove the whole directory.
- self.txn.commit()
- self._refresh()
+ yield self.txn.commit()
+ yield self._refresh()
self.calendar1.createCalendarObjectWithName(
"sample.ics",
VComponent.fromString(event4_text)
)
- self._refresh()
+ yield self.txn.abort()
+ yield self._refresh()
self.assertIdentical(
- self.calendar1.calendarObjectWithName("sample.ics"),
+ (yield self.calendar1.calendarObjectWithName("sample.ics")),
None
)
+ yield self.txn.commit()
+ @inlineCallbacks
def doThenUndo(self):
"""
Commit the current transaction, but add an operation that will cause it
@@ -330,43 +344,46 @@
raise RuntimeError("oops")
self.txn.addOperation(fail, "dummy failing operation")
self.assertRaises(RuntimeError, self.txn.commit)
- self._refresh()
+ yield self._refresh()
+ @inlineCallbacks
def test_undoModifyCalendarObject(self):
"""
If an existing calendar object is modified as part of a transaction, it
should be restored to its previous status if the transaction aborts.
"""
- originalComponent = self.calendar1.calendarObjectWithName(
+ originalComponent = yield self.calendar1.calendarObjectWithName(
"1.ics").component()
- self.calendar1.calendarObjectWithName("1.ics").setComponent(
+ (yield self.calendar1.calendarObjectWithName("1.ics")).setComponent(
VComponent.fromString(event1modified_text)
)
# Sanity check.
self.assertEquals(
- self.calendar1.calendarObjectWithName("1.ics").component(),
+ (yield self.calendar1.calendarObjectWithName("1.ics")).component(),
VComponent.fromString(event1modified_text)
)
- self.doThenUndo()
+ yield self.doThenUndo()
self.assertEquals(
- self.calendar1.calendarObjectWithName("1.ics").component(),
+ (yield self.calendar1.calendarObjectWithName("1.ics")).component(),
originalComponent
)
+ @inlineCallbacks
def test_modifyCalendarObjectCaches(self):
"""
Modifying a calendar object should cache the modified component in
memory, to avoid unnecessary parsing round-trips.
"""
+ self.addCleanup(self.txn.commit)
modifiedComponent = VComponent.fromString(event1modified_text)
self.calendar1.calendarObjectWithName("1.ics").setComponent(
modifiedComponent
)
self.assertIdentical(
modifiedComponent,
- self.calendar1.calendarObjectWithName("1.ics").component()
+ (yield self.calendar1.calendarObjectWithName("1.ics")).component()
)
@@ -410,9 +427,10 @@
class CalendarObjectTest(unittest.TestCase):
notifierFactory = None
+ @inlineCallbacks
def setUp(self):
- setUpCalendar1(self)
- self.object1 = self.calendar1.calendarObjectWithName("1.ics")
+ yield setUpCalendar1(self)
+ self.object1 = yield self.calendar1.calendarObjectWithName("1.ics")
def test_init(self):
@@ -464,10 +482,11 @@
self.storeRootPath)
+ @inlineCallbacks
def test_calendarObjectsWithDotFile(self):
"""
Adding a dotfile to the calendar home should not increase
"""
- self.homeUnderTest()._path.child(".foo").createDirectory()
+ (yield self.homeUnderTest())._path.child(".foo").createDirectory()
self.test_calendarObjects()
Copied: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_index_file.py (from rev 6368, CalendarServer/branches/users/glyph/more-deferreds-6/txdav/caldav/datastore/test/test_index_file.py)
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_index_file.py (rev 0)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_index_file.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -0,0 +1,936 @@
+##
+# Copyright (c) 2010 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twisted.internet import reactor
+from twisted.internet.task import deferLater
+
+from txdav.caldav.datastore.index_file import Index, MemcachedUIDReserver
+from txdav.common.icommondatastore import ReservationError
+
+from twistedcaldav import caldavxml
+from twistedcaldav.caldavxml import TimeRange
+from twistedcaldav.ical import Component
+from twistedcaldav.instance import InvalidOverriddenInstanceError
+from twistedcaldav.query import calendarqueryfilter
+from twistedcaldav.test.util import InMemoryMemcacheProtocol
+import twistedcaldav.test.util
+
+import datetime
+import os
+
+
+class MinimalResourceReplacement(object):
+ """
+ Provide the minimal set of attributes and methods from CalDAVFile required
+ by L{Index}.
+ """
+
+ def __init__(self, filePath):
+ self.fp = filePath
+
+
+ def isCalendarCollection(self):
+ return True
+
+
+ def getChild(self, name):
+ # FIXME: this should really return something with a child method
+ return self.fp.child(name)
+
+
+ def initSyncToken(self):
+ pass
+
+
+
+class SQLIndexTests (twistedcaldav.test.util.TestCase):
+ """
+ Test abstract SQL DB class
+ """
+
+ def setUp(self):
+ super(SQLIndexTests, self).setUp()
+ self.site.resource.isCalendarCollection = lambda: True
+ self.indexDirPath = self.site.resource.fp
+ # FIXME: since this resource lies about isCalendarCollection, it doesn't
+ # have all the associated backend machinery to actually get children.
+ self.db = Index(MinimalResourceReplacement(self.indexDirPath))
+
+
+ def test_reserve_uid_ok(self):
+ uid = "test-test-test"
+ d = self.db.isReservedUID(uid)
+ d.addCallback(self.assertFalse)
+ d.addCallback(lambda _: self.db.reserveUID(uid))
+ d.addCallback(lambda _: self.db.isReservedUID(uid))
+ d.addCallback(self.assertTrue)
+ d.addCallback(lambda _: self.db.unreserveUID(uid))
+ d.addCallback(lambda _: self.db.isReservedUID(uid))
+ d.addCallback(self.assertFalse)
+
+ return d
+
+
+ def test_reserve_uid_twice(self):
+ uid = "test-test-test"
+ d = self.db.reserveUID(uid)
+ d.addCallback(lambda _: self.db.isReservedUID(uid))
+ d.addCallback(self.assertTrue)
+ d.addCallback(lambda _:
+ self.assertFailure(self.db.reserveUID(uid),
+ ReservationError))
+ return d
+
+
+ def test_unreserve_unreserved(self):
+ uid = "test-test-test"
+ return self.assertFailure(self.db.unreserveUID(uid),
+ ReservationError)
+
+
+ def test_reserve_uid_timeout(self):
+ # WARNING: This test is fundamentally flawed and will fail
+ # intermittently because it uses the real clock.
+ uid = "test-test-test"
+ from twistedcaldav.config import config
+ old_timeout = config.UIDReservationTimeOut
+ config.UIDReservationTimeOut = 1
+
+ def _finally():
+ config.UIDReservationTimeOut = old_timeout
+
+ d = self.db.isReservedUID(uid)
+ d.addCallback(self.assertFalse)
+ d.addCallback(lambda _: self.db.reserveUID(uid))
+ d.addCallback(lambda _: self.db.isReservedUID(uid))
+ d.addCallback(self.assertTrue)
+ d.addCallback(lambda _: deferLater(reactor, 2, lambda: None))
+ d.addCallback(lambda _: self.db.isReservedUID(uid))
+ d.addCallback(self.assertFalse)
+ self.addCleanup(_finally)
+
+ return d
+
+
+ def test_index(self):
+ data = (
+ (
+ "#1.1 Simple component",
+ "1.1",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1.1
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+ False,
+ True,
+ ),
+ (
+ "#2.1 Recurring component",
+ "2.1",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-2.1
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=WEEKLY;COUNT=2
+END:VEVENT
+END:VCALENDAR
+""",
+ False,
+ True,
+ ),
+ (
+ "#2.2 Recurring component with override",
+ "2.2",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-2.2
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=WEEKLY;COUNT=2
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-2.2
+RECURRENCE-ID:20080608T120000Z
+DTSTART:20080608T120000Z
+DTEND:20080608T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+ False,
+ True,
+ ),
+ (
+ "#2.3 Recurring component with broken override - new",
+ "2.3",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-2.3
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=WEEKLY;COUNT=2
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-2.3
+RECURRENCE-ID:20080609T120000Z
+DTSTART:20080608T120000Z
+DTEND:20080608T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+ False,
+ False,
+ ),
+ (
+ "#2.4 Recurring component with broken override - existing",
+ "2.4",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-2.4
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=WEEKLY;COUNT=2
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-2.4
+RECURRENCE-ID:20080609T120000Z
+DTSTART:20080608T120000Z
+DTEND:20080608T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+ True,
+ True,
+ ),
+ )
+
+ for description, name, calendar_txt, reCreate, ok in data:
+ calendar = Component.fromString(calendar_txt)
+ if ok:
+ f = open(os.path.join(self.indexDirPath.path, name), "w")
+ f.write(calendar_txt)
+ del f
+
+ self.db.addResource(name, calendar, reCreate=reCreate)
+ self.assertTrue(self.db.resourceExists(name), msg=description)
+ else:
+ self.assertRaises(InvalidOverriddenInstanceError, self.db.addResource, name, calendar)
+ self.assertFalse(self.db.resourceExists(name), msg=description)
+
+ self.db._db_recreate()
+ for description, name, calendar_txt, reCreate, ok in data:
+ if ok:
+ self.assertTrue(self.db.resourceExists(name), msg=description)
+ else:
+ self.assertFalse(self.db.resourceExists(name), msg=description)
+
+ self.db.testAndUpdateIndex(datetime.date(2020, 1, 1))
+ for description, name, calendar_txt, reCreate, ok in data:
+ if ok:
+ self.assertTrue(self.db.resourceExists(name), msg=description)
+ else:
+ self.assertFalse(self.db.resourceExists(name), msg=description)
+
+ def test_index_timespan(self):
+ data = (
+ (
+ "#1.1 Simple component - busy",
+ "1.1",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1.1
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+ "20080601T000000Z", "20080602T000000Z",
+ "mailto:user1 at example.com",
+ (('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),),
+ ),
+ (
+ "#1.2 Simple component - transparent",
+ "1.2",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1.2
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+TRANSP:TRANSPARENT
+END:VEVENT
+END:VCALENDAR
+""",
+ "20080602T000000Z", "20080603T000000Z",
+ "mailto:user1 at example.com",
+ (('N', "2008-06-02 12:00:00+00:00", "2008-06-02 13:00:00+00:00", 'B', 'T'),),
+ ),
+ (
+ "#1.3 Simple component - canceled",
+ "1.3",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1.3
+DTSTART:20080603T120000Z
+DTEND:20080603T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+STATUS:CANCELLED
+END:VEVENT
+END:VCALENDAR
+""",
+ "20080603T000000Z", "20080604T000000Z",
+ "mailto:user1 at example.com",
+ (('N', "2008-06-03 12:00:00+00:00", "2008-06-03 13:00:00+00:00", 'F', 'F'),),
+ ),
+ (
+ "#1.4 Simple component - tentative",
+ "1.4",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1.4
+DTSTART:20080604T120000Z
+DTEND:20080604T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+STATUS:TENTATIVE
+END:VEVENT
+END:VCALENDAR
+""",
+ "20080604T000000Z", "20080605T000000Z",
+ "mailto:user1 at example.com",
+ (('N', "2008-06-04 12:00:00+00:00", "2008-06-04 13:00:00+00:00", 'T', 'F'),),
+ ),
+ (
+ "#2.1 Recurring component - busy",
+ "2.1",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-2.1
+DTSTART:20080605T120000Z
+DTEND:20080605T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=DAILY;COUNT=2
+END:VEVENT
+END:VCALENDAR
+""",
+ "20080605T000000Z", "20080607T000000Z",
+ "mailto:user1 at example.com",
+ (
+ ('N', "2008-06-05 12:00:00+00:00", "2008-06-05 13:00:00+00:00", 'B', 'F'),
+ ('N', "2008-06-06 12:00:00+00:00", "2008-06-06 13:00:00+00:00", 'B', 'F'),
+ ),
+ ),
+ (
+ "#2.2 Recurring component - busy",
+ "2.2",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-2.2
+DTSTART:20080607T120000Z
+DTEND:20080607T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=DAILY;COUNT=2
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-2.2
+RECURRENCE-ID:20080608T120000Z
+DTSTART:20080608T140000Z
+DTEND:20080608T150000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+TRANSP:TRANSPARENT
+END:VEVENT
+END:VCALENDAR
+""",
+ "20080607T000000Z", "20080609T000000Z",
+ "mailto:user1 at example.com",
+ (
+ ('N', "2008-06-07 12:00:00+00:00", "2008-06-07 13:00:00+00:00", 'B', 'F'),
+ ('N', "2008-06-08 14:00:00+00:00", "2008-06-08 15:00:00+00:00", 'B', 'T'),
+ ),
+ ),
+ )
+
+ for description, name, calendar_txt, trstart, trend, organizer, instances in data:
+ calendar = Component.fromString(calendar_txt)
+
+ f = open(os.path.join(self.indexDirPath.path, name), "w")
+ f.write(calendar_txt)
+ del f
+
+ self.db.addResource(name, calendar)
+ self.assertTrue(self.db.resourceExists(name), msg=description)
+
+ # Create fake filter element to match time-range
+ filter = caldavxml.Filter(
+ caldavxml.ComponentFilter(
+ caldavxml.ComponentFilter(
+ TimeRange(
+ start=trstart,
+ end=trend,
+ ),
+ name=("VEVENT", "VFREEBUSY", "VAVAILABILITY"),
+ ),
+ name="VCALENDAR",
+ )
+ )
+ filter = calendarqueryfilter.Filter(filter)
+
+ resources = self.db.indexedSearch(filter, fbtype=True)
+ index_results = set()
+ for _ignore_name, _ignore_uid, type, test_organizer, float, start, end, fbtype, transp in resources:
+ self.assertEqual(test_organizer, organizer, msg=description)
+ index_results.add((float, start, end, fbtype, transp,))
+
+ self.assertEqual(set(instances), index_results, msg=description)
+
+ def test_index_timespan_per_user(self):
+ data = (
+ (
+ "#1.1 Single per-user non-recurring component",
+ "1.1",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1.1
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+BEGIN:X-CALENDARSERVER-PERUSER
+UID:12345-67890-1.1
+X-CALENDARSERVER-PERUSER-UID:user01
+BEGIN:X-CALENDARSERVER-PERINSTANCE
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+TRANSP:TRANSPARENT
+END:X-CALENDARSERVER-PERINSTANCE
+END:X-CALENDARSERVER-PERUSER
+END:VCALENDAR
+""",
+ "20080601T000000Z", "20080602T000000Z",
+ "mailto:user1 at example.com",
+ (
+ (
+ "user01",
+ (('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'T'),),
+ ),
+ (
+ "user02",
+ (('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),),
+ ),
+ ),
+ ),
+ (
+ "#1.2 Two per-user non-recurring component",
+ "1.2",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1.2
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+BEGIN:X-CALENDARSERVER-PERUSER
+UID:12345-67890-1.2
+X-CALENDARSERVER-PERUSER-UID:user01
+BEGIN:X-CALENDARSERVER-PERINSTANCE
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+TRANSP:TRANSPARENT
+END:X-CALENDARSERVER-PERINSTANCE
+END:X-CALENDARSERVER-PERUSER
+BEGIN:X-CALENDARSERVER-PERUSER
+UID:12345-67890-1.2
+X-CALENDARSERVER-PERUSER-UID:user02
+BEGIN:X-CALENDARSERVER-PERINSTANCE
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:X-CALENDARSERVER-PERINSTANCE
+END:X-CALENDARSERVER-PERUSER
+END:VCALENDAR
+""",
+ "20080601T000000Z", "20080602T000000Z",
+ "mailto:user1 at example.com",
+ (
+ (
+ "user01",
+ (('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'T'),),
+ ),
+ (
+ "user02",
+ (('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),),
+ ),
+ (
+ "user03",
+ (('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),),
+ ),
+ ),
+ ),
+ (
+ "#2.1 Single per-user simple recurring component",
+ "2.1",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1.1
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=DAILY;COUNT=10
+END:VEVENT
+BEGIN:X-CALENDARSERVER-PERUSER
+UID:12345-67890-1.1
+X-CALENDARSERVER-PERUSER-UID:user01
+BEGIN:X-CALENDARSERVER-PERINSTANCE
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+TRANSP:TRANSPARENT
+END:X-CALENDARSERVER-PERINSTANCE
+END:X-CALENDARSERVER-PERUSER
+END:VCALENDAR
+""",
+ "20080601T000000Z", "20080603T000000Z",
+ "mailto:user1 at example.com",
+ (
+ (
+ "user01",
+ (
+ ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'T'),
+ ('N', "2008-06-02 12:00:00+00:00", "2008-06-02 13:00:00+00:00", 'B', 'T'),
+ ),
+ ),
+ (
+ "user02",
+ (
+ ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),
+ ('N', "2008-06-02 12:00:00+00:00", "2008-06-02 13:00:00+00:00", 'B', 'F'),
+ ),
+ ),
+ ),
+ ),
+ (
+ "#2.2 Two per-user simple recurring component",
+ "2.2",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1.2
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=DAILY;COUNT=10
+END:VEVENT
+BEGIN:X-CALENDARSERVER-PERUSER
+UID:12345-67890-1.2
+X-CALENDARSERVER-PERUSER-UID:user01
+BEGIN:X-CALENDARSERVER-PERINSTANCE
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+TRANSP:TRANSPARENT
+END:X-CALENDARSERVER-PERINSTANCE
+END:X-CALENDARSERVER-PERUSER
+BEGIN:X-CALENDARSERVER-PERUSER
+UID:12345-67890-1.2
+X-CALENDARSERVER-PERUSER-UID:user02
+BEGIN:X-CALENDARSERVER-PERINSTANCE
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:X-CALENDARSERVER-PERINSTANCE
+END:X-CALENDARSERVER-PERUSER
+END:VCALENDAR
+""",
+ "20080601T000000Z", "20080603T000000Z",
+ "mailto:user1 at example.com",
+ (
+ (
+ "user01",
+ (
+ ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'T'),
+ ('N', "2008-06-02 12:00:00+00:00", "2008-06-02 13:00:00+00:00", 'B', 'T'),
+ ),
+ ),
+ (
+ "user02",
+ (
+ ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),
+ ('N', "2008-06-02 12:00:00+00:00", "2008-06-02 13:00:00+00:00", 'B', 'F'),
+ ),
+ ),
+ (
+ "user03",
+ (
+ ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),
+ ('N', "2008-06-02 12:00:00+00:00", "2008-06-02 13:00:00+00:00", 'B', 'F'),
+ ),
+ ),
+ ),
+ ),
+ (
+ "#3.1 Single per-user complex recurring component",
+ "3.1",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1.1
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=DAILY;COUNT=10
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-1.1
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T130000Z
+DTEND:20080602T140000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+BEGIN:X-CALENDARSERVER-PERUSER
+UID:12345-67890-1.1
+X-CALENDARSERVER-PERUSER-UID:user01
+BEGIN:X-CALENDARSERVER-PERINSTANCE
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+TRANSP:TRANSPARENT
+END:X-CALENDARSERVER-PERINSTANCE
+BEGIN:X-CALENDARSERVER-PERINSTANCE
+RECURRENCE-ID:20080602T120000Z
+TRANSP:OPAQUE
+END:X-CALENDARSERVER-PERINSTANCE
+END:X-CALENDARSERVER-PERUSER
+END:VCALENDAR
+""",
+ "20080601T000000Z", "20080604T000000Z",
+ "mailto:user1 at example.com",
+ (
+ (
+ "user01",
+ (
+ ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'T'),
+ ('N', "2008-06-02 13:00:00+00:00", "2008-06-02 14:00:00+00:00", 'B', 'F'),
+ ('N', "2008-06-03 12:00:00+00:00", "2008-06-03 13:00:00+00:00", 'B', 'T'),
+ ),
+ ),
+ (
+ "user02",
+ (
+ ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),
+ ('N', "2008-06-02 13:00:00+00:00", "2008-06-02 14:00:00+00:00", 'B', 'F'),
+ ('N', "2008-06-03 12:00:00+00:00", "2008-06-03 13:00:00+00:00", 'B', 'F'),
+ ),
+ ),
+ ),
+ ),
+ (
+ "#3.2 Two per-user complex recurring component",
+ "3.2",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1.2
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=DAILY;COUNT=10
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-1.2
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T130000Z
+DTEND:20080602T140000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+BEGIN:X-CALENDARSERVER-PERUSER
+UID:12345-67890-1.2
+X-CALENDARSERVER-PERUSER-UID:user01
+BEGIN:X-CALENDARSERVER-PERINSTANCE
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+TRANSP:TRANSPARENT
+END:X-CALENDARSERVER-PERINSTANCE
+BEGIN:X-CALENDARSERVER-PERINSTANCE
+RECURRENCE-ID:20080602T120000Z
+TRANSP:OPAQUE
+END:X-CALENDARSERVER-PERINSTANCE
+END:X-CALENDARSERVER-PERUSER
+BEGIN:X-CALENDARSERVER-PERUSER
+UID:12345-67890-1.2
+X-CALENDARSERVER-PERUSER-UID:user02
+BEGIN:X-CALENDARSERVER-PERINSTANCE
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:X-CALENDARSERVER-PERINSTANCE
+BEGIN:X-CALENDARSERVER-PERINSTANCE
+RECURRENCE-ID:20080603T120000Z
+TRANSP:TRANSPARENT
+END:X-CALENDARSERVER-PERINSTANCE
+END:X-CALENDARSERVER-PERUSER
+END:VCALENDAR
+""",
+ "20080601T000000Z", "20080604T000000Z",
+ "mailto:user1 at example.com",
+ (
+ (
+ "user01",
+ (
+ ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'T'),
+ ('N', "2008-06-02 13:00:00+00:00", "2008-06-02 14:00:00+00:00", 'B', 'F'),
+ ('N', "2008-06-03 12:00:00+00:00", "2008-06-03 13:00:00+00:00", 'B', 'T'),
+ ),
+ ),
+ (
+ "user02",
+ (
+ ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),
+ ('N', "2008-06-02 13:00:00+00:00", "2008-06-02 14:00:00+00:00", 'B', 'F'),
+ ('N', "2008-06-03 12:00:00+00:00", "2008-06-03 13:00:00+00:00", 'B', 'T'),
+ ),
+ ),
+ (
+ "user03",
+ (
+ ('N', "2008-06-01 12:00:00+00:00", "2008-06-01 13:00:00+00:00", 'B', 'F'),
+ ('N', "2008-06-02 13:00:00+00:00", "2008-06-02 14:00:00+00:00", 'B', 'F'),
+ ('N', "2008-06-03 12:00:00+00:00", "2008-06-03 13:00:00+00:00", 'B', 'F'),
+ ),
+ ),
+ ),
+ ),
+ )
+
+ for description, name, calendar_txt, trstart, trend, organizer, peruserinstances in data:
+ calendar = Component.fromString(calendar_txt)
+
+ f = open(os.path.join(self.indexDirPath.path, name), "w")
+ f.write(calendar_txt)
+ del f
+
+ self.db.addResource(name, calendar)
+ self.assertTrue(self.db.resourceExists(name), msg=description)
+
+ # Create fake filter element to match time-range
+ filter = caldavxml.Filter(
+ caldavxml.ComponentFilter(
+ caldavxml.ComponentFilter(
+ TimeRange(
+ start=trstart,
+ end=trend,
+ ),
+ name=("VEVENT", "VFREEBUSY", "VAVAILABILITY"),
+ ),
+ name="VCALENDAR",
+ )
+ )
+ filter = calendarqueryfilter.Filter(filter)
+
+ for useruid, instances in peruserinstances:
+ resources = self.db.indexedSearch(filter, useruid=useruid, fbtype=True)
+ index_results = set()
+ for _ignore_name, _ignore_uid, type, test_organizer, float, start, end, fbtype, transp in resources:
+ self.assertEqual(test_organizer, organizer, msg=description)
+ index_results.add((str(float), str(start), str(end), str(fbtype), str(transp),))
+
+ self.assertEqual(set(instances), index_results, msg="%s, user:%s" % (description, useruid,))
+
+ self.db.deleteResource(name)
+
+ def test_index_revisions(self):
+ data1 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1.1
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+"""
+ data2 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-2.1
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=WEEKLY;COUNT=2
+END:VEVENT
+END:VCALENDAR
+"""
+ data3 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-2.3
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=WEEKLY;COUNT=2
+END:VEVENT
+END:VCALENDAR
+"""
+
+ calendar = Component.fromString(data1)
+ self.db.addResource("data1.ics", calendar)
+ calendar = Component.fromString(data2)
+ self.db.addResource("data2.ics", calendar)
+ calendar = Component.fromString(data3)
+ self.db.addResource("data3.ics", calendar)
+ self.db.deleteResource("data3.ics")
+
+ tests = (
+ (0, (["data1.ics", "data2.ics",], [],)),
+ (1, (["data2.ics",], ["data3.ics",],)),
+ (2, ([], ["data3.ics",],)),
+ (3, ([], ["data3.ics",],)),
+ (4, ([], [],)),
+ (5, ([], [],)),
+ )
+
+ for revision, results in tests:
+ self.assertEquals(self.db.whatchanged(revision), results, "Mismatched results for whatchanged with revision %d" % (revision,))
+
+class MemcacheTests(SQLIndexTests):
+ def setUp(self):
+ super(MemcacheTests, self).setUp()
+ self.memcache = InMemoryMemcacheProtocol()
+ self.db.reserver = MemcachedUIDReserver(self.db, self.memcache)
+
+
+ def tearDown(self):
+ for _ignore_k, v in self.memcache._timeouts.iteritems():
+ if v.active():
+ v.cancel()
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_sql.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_sql.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_sql.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -47,11 +47,12 @@
def setUp(self):
super(CalendarSQLStorageTests, self).setUp()
self._sqlCalendarStore = yield buildStore(self, self.notifierFactory)
- self.populate()
+ yield self.populate()
+ @inlineCallbacks
def populate(self):
- populateCalendarsFrom(self.requirements, self.storeUnderTest())
+ yield populateCalendarsFrom(self.requirements, self.storeUnderTest())
self.notifierFactory.reset()
@@ -111,32 +112,37 @@
L{ICalendarObject.attachmentWithName}.
"""
yield self.createAttachmentTest(lambda x: x)
- attachmentRoot = self.calendarObjectUnderTest()._txn._store.attachmentsPath
+ attachmentRoot = (yield self.calendarObjectUnderTest())._txn._store.attachmentsPath
attachmentPath = attachmentRoot.child("ho").child("me").child("home1")
- attachmentPath = attachmentPath.child(self.calendarObjectUnderTest().uid()).child("new.attachment")
+ attachmentPath = attachmentPath.child(
+ (yield self.calendarObjectUnderTest()).uid()).child(
+ "new.attachment")
self.assertTrue(attachmentPath.isfile())
+
+ @inlineCallbacks
def test_migrateCalendarFromFile(self):
"""
C{_migrateCalendar()} can migrate a file-backed calendar to a database-
backed calendar.
"""
- fromCalendar = self.fileTransaction().calendarHomeWithUID(
- "home1").calendarWithName("calendar_1")
- toHome = self.transactionUnderTest().calendarHomeWithUID(
+ fromCalendar = yield (yield self.fileTransaction().calendarHomeWithUID(
+ "home1")).calendarWithName("calendar_1")
+ toHome = yield self.transactionUnderTest().calendarHomeWithUID(
"new-home", create=True)
- toCalendar = toHome.calendarWithName("calendar")
+ toCalendar = yield toHome.calendarWithName("calendar")
_migrateCalendar(fromCalendar, toCalendar, lambda x: x.component())
self.assertCalendarsSimilar(fromCalendar, toCalendar)
+ @inlineCallbacks
def test_migrateHomeFromFile(self):
"""
L{migrateHome} will migrate an L{ICalendarHome} provider from one
backend to another; in this specific case, from the file-based backend
to the SQL-based backend.
"""
- fromHome = self.fileTransaction().calendarHomeWithUID("home1")
+ fromHome = yield self.fileTransaction().calendarHomeWithUID("home1")
builtinProperties = [PropertyName.fromElement(ResourceType)]
@@ -145,10 +151,10 @@
key = PropertyName.fromElement(GETContentLanguage)
fromHome.properties()[key] = GETContentLanguage("C")
- fromHome.calendarWithName("calendar_1").properties()[key] = (
+ (yield fromHome.calendarWithName("calendar_1")).properties()[key] = (
GETContentLanguage("pig-latin")
)
- toHome = self.transactionUnderTest().calendarHomeWithUID(
+ toHome = yield self.transactionUnderTest().calendarHomeWithUID(
"new-home", create=True
)
migrateHome(fromHome, toHome, lambda x: x.component())
@@ -157,7 +163,7 @@
if self.requirements['home1'][k] is not None]))
for c in fromHome.calendars():
self.assertPropertiesSimilar(
- c, toHome.calendarWithName(c.name()),
+ c, (yield toHome.calendarWithName(c.name())),
builtinProperties
)
self.assertPropertiesSimilar(fromHome, toHome, builtinProperties)
@@ -250,8 +256,8 @@
home1 = txn1.homeWithUID(ECALENDARTYPE, "uid1", create=True)
home2 = txn2.homeWithUID(ECALENDARTYPE, "uid1", create=True)
- adbk1 = home1.calendarWithName("calendar")
- adbk2 = home2.calendarWithName("calendar")
+ adbk1 = yield home1.calendarWithName("calendar")
+ adbk2 = yield home2.calendarWithName("calendar")
def _defer1():
adbk1.createObjectResourceWithName("1.ics", VComponent.fromString(
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/util.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/util.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/util.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -192,7 +192,7 @@
if name == "outbox":
continue
outHome.createCalendarWithName(name)
- outCalendar = outHome.calendarWithName(name)
+ outCalendar = yield outHome.calendarWithName(name)
try:
yield _migrateCalendar(calendar, outCalendar, getComponent)
except InternalDataStoreError:
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/icalendarstore.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/icalendarstore.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/icalendarstore.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -57,8 +57,8 @@
If C{create} is C{True}, create the calendar home if it doesn't
already exist.
- @return: an L{ICalendarHome} or C{None} if no such calendar
- home exists.
+ @return: a L{Deferred} which fires with L{ICalendarHome} or C{None} if
+ no such calendar home exists.
"""
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/file.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/file.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/file.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -33,7 +33,7 @@
from twistedcaldav.sharing import InvitesDatabase
from twistedcaldav.vcard import Component as VComponent, InvalidVCardDataError
-from twistedcaldav.vcardindex import AddressBookIndex as OldIndex
+from txdav.carddav.datastore.index_file import AddressBookIndex as OldIndex
from txdav.carddav.datastore.util import validateAddressBookComponent
from txdav.carddav.iaddressbookstore import IAddressBook, IAddressBookObject
Copied: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/index_file.py (from rev 6368, CalendarServer/branches/users/glyph/more-deferreds-6/txdav/carddav/datastore/index_file.py)
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/index_file.py (rev 0)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/index_file.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -0,0 +1,696 @@
+##
+# Copyright (c) 2005-2009 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+CardDAV Index.
+
+This API is considered private to static.py and is therefore subject to
+change.
+"""
+
+__all__ = [
+ "AddressBookIndex",
+]
+
+import datetime
+import os
+import time
+import hashlib
+
+try:
+ import sqlite3 as sqlite
+except ImportError:
+ from pysqlite2 import dbapi2 as sqlite
+
+from twisted.internet.defer import maybeDeferred
+
+from twistedcaldav import carddavxml
+from txdav.common.icommondatastore import SyncTokenValidException,\
+ ReservationError
+from twistedcaldav.query import addressbookquery
+from twistedcaldav.sql import AbstractSQLDatabase
+from twistedcaldav.sql import db_prefix
+from twistedcaldav.vcard import Component
+
+from twext.python.log import Logger, LoggingMixIn
+from twistedcaldav.config import config
+from twistedcaldav.memcachepool import CachePoolUserMixIn
+
+log = Logger()
+
+db_basename = db_prefix + "sqlite"
+schema_version = "2"
+
+def wrapInDeferred(f):
+ def _(*args, **kwargs):
+ return maybeDeferred(f, *args, **kwargs)
+
+ return _
+
+
+class MemcachedUIDReserver(CachePoolUserMixIn, LoggingMixIn):
+ def __init__(self, index, cachePool=None):
+ self.index = index
+ self._cachePool = cachePool
+
+ def _key(self, uid):
+ return 'reservation:%s' % (
+ hashlib.md5('%s:%s' % (uid,
+ self.index.resource.fp.path)).hexdigest())
+
+ def reserveUID(self, uid):
+ uid = uid.encode('utf-8')
+ self.log_debug("Reserving UID %r @ %r" % (
+ uid,
+ self.index.resource.fp.path))
+
+ def _handleFalse(result):
+ if result is False:
+ raise ReservationError(
+ "UID %s already reserved for address book collection %s."
+ % (uid, self.index.resource)
+ )
+
+ d = self.getCachePool().add(self._key(uid),
+ 'reserved',
+ expireTime=config.UIDReservationTimeOut)
+ d.addCallback(_handleFalse)
+ return d
+
+
+ def unreserveUID(self, uid):
+ uid = uid.encode('utf-8')
+ self.log_debug("Unreserving UID %r @ %r" % (
+ uid,
+ self.index.resource.fp.path))
+
+ def _handleFalse(result):
+ if result is False:
+ raise ReservationError(
+ "UID %s is not reserved for address book collection %s."
+ % (uid, self.index.resource)
+ )
+
+ d =self.getCachePool().delete(self._key(uid))
+ d.addCallback(_handleFalse)
+ return d
+
+
+ def isReservedUID(self, uid):
+ uid = uid.encode('utf-8')
+ self.log_debug("Is reserved UID %r @ %r" % (
+ uid,
+ self.index.resource.fp.path))
+
+ def _checkValue((flags, value)):
+ if value is None:
+ return False
+ else:
+ return True
+
+ d = self.getCachePool().get(self._key(uid))
+ d.addCallback(_checkValue)
+ return d
+
+
+
+class SQLUIDReserver(object):
+ def __init__(self, index):
+ self.index = index
+
+ @wrapInDeferred
+ def reserveUID(self, uid):
+ """
+ Reserve a UID for this index's resource.
+ @param uid: the UID to reserve
+ @raise ReservationError: if C{uid} is already reserved
+ """
+
+ try:
+ self.index._db_execute("insert into RESERVED (UID, TIME) values (:1, :2)", uid, datetime.datetime.now())
+ self.index._db_commit()
+ except sqlite.IntegrityError:
+ self.index._db_rollback()
+ raise ReservationError(
+ "UID %s already reserved for address book collection %s."
+ % (uid, self.index.resource)
+ )
+ except sqlite.Error, e:
+ log.err("Unable to reserve UID: %s", (e,))
+ self.index._db_rollback()
+ raise
+
+ def unreserveUID(self, uid):
+ """
+ Unreserve a UID for this index's resource.
+ @param uid: the UID to reserve
+ @raise ReservationError: if C{uid} is not reserved
+ """
+
+ def _cb(result):
+ if result == False:
+ raise ReservationError(
+ "UID %s is not reserved for address book collection %s."
+ % (uid, self.index.resource)
+ )
+ else:
+ try:
+ self.index._db_execute(
+ "delete from RESERVED where UID = :1", uid)
+ self.index._db_commit()
+ except sqlite.Error, e:
+ log.err("Unable to unreserve UID: %s", (e,))
+ self.index._db_rollback()
+ raise
+
+ d = self.isReservedUID(uid)
+ d.addCallback(_cb)
+ return d
+
+
+ @wrapInDeferred
+ def isReservedUID(self, uid):
+ """
+ Check to see whether a UID is reserved.
+ @param uid: the UID to check
+ @return: True if C{uid} is reserved, False otherwise.
+ """
+
+ rowiter = self.index._db_execute("select UID, TIME from RESERVED where UID = :1", uid)
+ for uid, attime in rowiter:
+ # Double check that the time is within a reasonable period of now
+ # otherwise we probably have a stale reservation
+ tm = time.strptime(attime[:19], "%Y-%m-%d %H:%M:%S")
+ dt = datetime.datetime(year=tm.tm_year, month=tm.tm_mon, day=tm.tm_mday, hour=tm.tm_hour, minute=tm.tm_min, second = tm.tm_sec)
+ if datetime.datetime.now() - dt > datetime.timedelta(seconds=config.UIDReservationTimeOut):
+ try:
+ self.index._db_execute("delete from RESERVED where UID = :1", uid)
+ self.index._db_commit()
+ except sqlite.Error, e:
+ log.err("Unable to unreserve UID: %s", (e,))
+ self.index._db_rollback()
+ raise
+ return False
+ else:
+ return True
+
+ return False
+
+class AddressBookIndex(AbstractSQLDatabase):
+ """
+ AddressBook collection index abstract base class that defines the apis for the index.
+ """
+
+ def __init__(self, resource):
+ """
+ @param resource: the L{CalDAVResource} resource to
+ index. C{resource} must be an addressbook collection (ie.
+ C{resource.isAddressBookCollection()} returns C{True}.)
+ """
+ assert resource.isAddressBookCollection(), "non-addressbook collection resource %s has no index." % (resource,)
+ self.resource = resource
+ db_filename = os.path.join(self.resource.fp.path, db_basename)
+ super(AddressBookIndex, self).__init__(db_filename, False)
+
+ if (
+ hasattr(config, "Memcached") and
+ config.Memcached.Pools.Default.ClientEnabled
+ ):
+ self.reserver = MemcachedUIDReserver(self)
+ else:
+ self.reserver = SQLUIDReserver(self)
+
+ def create(self):
+ """
+ Create the index and initialize it.
+ """
+ self._db()
+
+ def recreate(self):
+ """
+ Delete the database and re-create it
+ """
+ try:
+ os.remove(self.dbpath)
+ except OSError:
+ pass
+ self.create()
+
+ #
+ # A dict of sets. The dict keys are address book collection paths,
+ # and the sets contains reserved UIDs for each path.
+ #
+
+ def reserveUID(self, uid):
+ return self.reserver.reserveUID(uid)
+
+ def unreserveUID(self, uid):
+ return self.reserver.unreserveUID(uid)
+
+ def isReservedUID(self, uid):
+ return self.reserver.isReservedUID(uid)
+
+ def isAllowedUID(self, uid, *names):
+ """
+ Checks to see whether to allow an operation which would add the
+ specified UID to the index. Specifically, the operation may not
+ violate the constraint that UIDs must be unique.
+ @param uid: the UID to check
+ @param names: the names of resources being replaced or deleted by the
+ operation; UIDs associated with these resources are not checked.
+ @return: True if the UID is not in the index and is not reserved,
+ False otherwise.
+ """
+ rname = self.resourceNameForUID(uid)
+ return (rname is None or rname in names)
+
+ def resourceNamesForUID(self, uid):
+ """
+ Looks up the names of the resources with the given UID.
+ @param uid: the UID of the resources to look up.
+ @return: a list of resource names
+ """
+ names = self._db_values_for_sql("select NAME from RESOURCE where UID = :1", uid)
+
+ #
+ # Check that each name exists as a child of self.resource. If not, the
+ # resource record is stale.
+ #
+ resources = []
+ for name in names:
+ name_utf8 = name.encode("utf-8")
+ if name is not None and self.resource.getChild(name_utf8) is None:
+ # Clean up
+ log.err("Stale resource record found for child %s with UID %s in %s" % (name, uid, self.resource))
+ self._delete_from_db(name, uid, False)
+ self._db_commit()
+ else:
+ resources.append(name_utf8)
+
+ return resources
+
+ def resourceNameForUID(self, uid):
+ """
+ Looks up the name of the resource with the given UID.
+ @param uid: the UID of the resource to look up.
+ @return: If the resource is found, its name; C{None} otherwise.
+ """
+ result = None
+
+ for name in self.resourceNamesForUID(uid):
+ assert result is None, "More than one resource with UID %s in address book collection %r" % (uid, self)
+ result = name
+
+ return result
+
+ def resourceUIDForName(self, name):
+ """
+ Looks up the UID of the resource with the given name.
+ @param name: the name of the resource to look up.
+ @return: If the resource is found, the UID of the resource; C{None}
+ otherwise.
+ """
+ uid = self._db_value_for_sql("select UID from RESOURCE where NAME = :1", name)
+
+ return uid
+
+ def addResource(self, name, vcard, fast=False):
+ """
+ Adding or updating an existing resource.
+ To check for an update we attempt to get an existing UID
+ for the resource name. If present, then the index entries for
+ that UID are removed. After that the new index entries are added.
+ @param name: the name of the resource to add.
+ @param vCard: a L{Component} object representing the resource
+ contents.
+ @param fast: if C{True} do not do commit, otherwise do commit.
+ """
+ oldUID = self.resourceUIDForName(name)
+ if oldUID is not None:
+ self._delete_from_db(name, oldUID, False)
+ self._add_to_db(name, vcard)
+ if not fast:
+ self._db_commit()
+
+ def deleteResource(self, name):
+ """
+ Remove this resource from the index.
+ @param name: the name of the resource to add.
+ @param uid: the UID of the vcard component in the resource.
+ """
+ uid = self.resourceUIDForName(name)
+ if uid is not None:
+ self._delete_from_db(name, uid)
+ self._db_commit()
+
+ def resourceExists(self, name):
+ """
+ Determines whether the specified resource name exists in the index.
+ @param name: the name of the resource to test
+ @return: True if the resource exists, False if not
+ """
+ uid = self._db_value_for_sql("select UID from RESOURCE where NAME = :1", name)
+ return uid is not None
+
+ def resourcesExist(self, names):
+ """
+ Determines whether the specified resource name exists in the index.
+ @param names: a C{list} containing the names of the resources to test
+ @return: a C{list} of all names that exist
+ """
+ statement = "select NAME from RESOURCE where NAME in ("
+ for ctr, ignore_name in enumerate(names):
+ if ctr != 0:
+ statement += ", "
+ statement += ":%s" % (ctr,)
+ statement += ")"
+ results = self._db_values_for_sql(statement, *names)
+ return results
+
+ def whatchanged(self, revision):
+
+ results = [(name.encode("utf-8"), deleted) for name, deleted in self._db_execute("select NAME, DELETED from REVISIONS where REVISION > :1", revision)]
+ results.sort(key=lambda x:x[1])
+
+ changed = []
+ deleted = []
+ for name, wasdeleted in results:
+ if name:
+ if wasdeleted == 'Y':
+ if revision:
+ deleted.append(name)
+ else:
+ changed.append(name)
+ else:
+ raise SyncTokenValidException
+
+ return changed, deleted,
+
+ def lastRevision(self):
+ return self._db_value_for_sql(
+ "select REVISION from REVISION_SEQUENCE"
+ )
+
+ def bumpRevision(self, fast=False):
+ self._db_execute(
+ """
+ update REVISION_SEQUENCE set REVISION = REVISION + 1
+ """,
+ )
+ if not fast:
+ self._db_commit()
+ return self._db_value_for_sql(
+ """
+ select REVISION from REVISION_SEQUENCE
+ """,
+ )
+
+ def searchValid(self, filter):
+ if isinstance(filter, carddavxml.Filter):
+ qualifiers = addressbookquery.sqladdressbookquery(filter)
+ else:
+ qualifiers = None
+
+ return qualifiers is not None
+
+ def search(self, filter):
+ """
+ Finds resources matching the given qualifiers.
+ @param filter: the L{Filter} for the addressbook-query to execute.
+ @return: an interable iterable of tuples for each resource matching the
+ given C{qualifiers}. The tuples are C{(name, uid, type)}, where
+ C{name} is the resource name, C{uid} is the resource UID, and
+ C{type} is the resource iCalendar component type.x
+ """
+ # FIXME: Don't forget to use maximum_future_expansion_duration when we
+ # start caching...
+
+ # Make sure we have a proper Filter element and get the partial SQL statement to use.
+ if isinstance(filter, carddavxml.Filter):
+ qualifiers = addressbookquery.sqladdressbookquery(filter)
+ else:
+ qualifiers = None
+ if qualifiers is not None:
+ rowiter = self._db_execute("select DISTINCT RESOURCE.NAME, RESOURCE.UID" + qualifiers[0], *qualifiers[1])
+ else:
+ rowiter = self._db_execute("select NAME, UID from RESOURCE")
+
+ for row in rowiter:
+ name = row[0]
+ if self.resource.getChild(name.encode("utf-8")):
+ yield row
+ else:
+ log.err("vCard resource %s is missing from %s. Removing from index."
+ % (name, self.resource))
+ self.deleteResource(name, None)
+
+ def bruteForceSearch(self):
+ """
+ List the whole index and tests for existence, updating the index
+ @return: all resources in the index
+ """
+ # List all resources
+ rowiter = self._db_execute("select NAME, UID from RESOURCE")
+
+ # Check result for missing resources:
+
+ for row in rowiter:
+ name = row[0]
+ if self.resource.getChild(name.encode("utf-8")):
+ yield row
+ else:
+ log.err("AddressBook resource %s is missing from %s. Removing from index."
+ % (name, self.resource))
+ self.deleteResource(name)
+
+
+ def _db_version(self):
+ """
+ @return: the schema version assigned to this index.
+ """
+ return schema_version
+
+ def _db_type(self):
+ """
+ @return: the collection type assigned to this index.
+ """
+ return "AddressBook"
+
+ def _db_init_data_tables(self, q):
+ """
+ Initialise the underlying database tables.
+ @param q: a database cursor to use.
+ """
+
+ # Create database where the RESOURCE table has unique UID column.
+ self._db_init_data_tables_base(q, True)
+
+ def _db_init_data_tables_base(self, q, uidunique):
+ """
+ Initialise the underlying database tables.
+ @param q: a database cursor to use.
+ """
+ #
+ # RESOURCE table is the primary index table
+ # NAME: Last URI component (eg. <uid>.ics, RESOURCE primary key)
+ # UID: iCalendar UID (may or may not be unique)
+ #
+ q.execute(
+ """
+ create table RESOURCE (
+ NAME text unique,
+ UID text unique
+ )
+ """
+ )
+
+ #
+ # REVISIONS table tracks changes
+ # NAME: Last URI component (eg. <uid>.ics, RESOURCE primary key)
+ # REVISION: revision number
+ # WASDELETED: Y if revision deleted, N if added or changed
+ #
+ q.execute(
+ """
+ create table REVISION_SEQUENCE (
+ REVISION integer
+ )
+ """
+ )
+ q.execute(
+ """
+ insert into REVISION_SEQUENCE (REVISION) values (0)
+ """
+ )
+ q.execute(
+ """
+ create table REVISIONS (
+ NAME text unique,
+ REVISION integer default 0,
+ DELETED text(1) default "N"
+ )
+ """
+ )
+ q.execute(
+ """
+ create index REVISION on REVISIONS (REVISION)
+ """
+ )
+
+ #
+ # RESERVED table tracks reserved UIDs
+ # UID: The UID being reserved
+ # TIME: When the reservation was made
+ #
+ q.execute(
+ """
+ create table RESERVED (
+ UID text unique,
+ TIME date
+ )
+ """
+ )
+
+ def _db_recreate(self, do_commit=True):
+ """
+ Re-create the database tables from existing address book data.
+ """
+
+ #
+ # Populate the DB with data from already existing resources.
+ # This allows for index recovery if the DB file gets
+ # deleted.
+ #
+ fp = self.resource.fp
+ for name in fp.listdir():
+ if name.startswith("."):
+ continue
+
+ try:
+ stream = fp.child(name).open()
+ except (IOError, OSError), e:
+ log.err("Unable to open resource %s: %s" % (name, e))
+ continue
+
+ try:
+ # FIXME: This is blocking I/O
+ try:
+ vcard = Component.fromStream(stream)
+ vcard.validForCardDAV()
+ except ValueError:
+ log.err("Non-addressbook resource: %s" % (name,))
+ else:
+ #log.msg("Indexing resource: %s" % (name,))
+ self.addResource(name, vcard, True)
+ finally:
+ stream.close()
+
+ # Do commit outside of the loop for better performance
+ if do_commit:
+ self._db_commit()
+
+ def _db_can_upgrade(self, old_version):
+ """
+ Can we do an in-place upgrade
+ """
+
+ # v2 is a minor change
+ return True
+
+ def _db_upgrade_data_tables(self, q, old_version):
+ """
+ Upgrade the data from an older version of the DB.
+ """
+
+ # When going to version 2+ all we need to do is add revision table and index
+ if old_version < 2:
+ q.execute(
+ """
+ create table REVISION_SEQUENCE (
+ REVISION integer
+ )
+ """
+ )
+ q.execute(
+ """
+ insert into REVISION_SEQUENCE (REVISION) values (0)
+ """
+ )
+ q.execute(
+ """
+ create table REVISIONS (
+ NAME text unique,
+ REVISION integer default 0,
+ CREATEDREVISION integer default 0,
+ WASDELETED text(1) default "N"
+ )
+ """
+ )
+ q.execute(
+ """
+ create index REVISION on REVISIONS (REVISION)
+ """
+ )
+
+ self._db_execute(
+ """
+ insert into REVISIONS (NAME)
+ select NAME from RESOURCE
+ """
+ )
+
+
+ def _add_to_db(self, name, vcard, cursor = None):
+ """
+ Records the given address book resource in the index with the given name.
+ Resource names and UIDs must both be unique; only one resource name may
+ be associated with any given UID and vice versa.
+ NB This method does not commit the changes to the db - the caller
+ MUST take care of that
+ @param name: the name of the resource to add.
+ @param vcard: a L{AddressBook} object representing the resource
+ contents.
+ """
+ uid = vcard.resourceUID()
+
+ self._db_execute(
+ """
+ insert into RESOURCE (NAME, UID)
+ values (:1, :2)
+ """, name, uid,
+ )
+
+ self._db_execute(
+ """
+ insert or replace into REVISIONS (NAME, REVISION, DELETED)
+ values (:1, :2, :3)
+ """, name, self.bumpRevision(fast=True), 'N',
+ )
+
+ def _delete_from_db(self, name, uid, dorevision=True):
+ """
+ Deletes the specified entry from all dbs.
+ @param name: the name of the resource to delete.
+ @param uid: the uid of the resource to delete.
+ """
+ self._db_execute("delete from RESOURCE where NAME = :1", name)
+ if dorevision:
+ self._db_execute(
+ """
+ update REVISIONS SET REVISION = :1, DELETED = :2
+ where NAME = :3
+ """, self.bumpRevision(fast=True), 'Y', name
+ )
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/common.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/common.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/common.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -17,12 +17,8 @@
"""
Tests for common addressbook store API functions.
"""
+from twisted.internet.defer import inlineCallbacks, returnValue
-from zope.interface.verify import verifyObject
-from zope.interface.exceptions import (
- BrokenMethodImplementation, DoesNotImplement
-)
-
from txdav.idav import IPropertyStore, IDataStore
from txdav.base.propertystore.base import PropertyName
@@ -38,8 +34,10 @@
IAddressBookObject, IAddressBookHome,
IAddressBook, IAddressBookTransaction
)
+
+from txdav.common.datastore.test.util import CommonCommonTests
+
from twistedcaldav.vcard import Component as VComponent
-from twistedcaldav.notify import Notifier
from twext.python.filepath import CachingFilePath as FilePath
from twext.web2.dav import davxml
@@ -103,25 +101,8 @@
)
-def assertProvides(testCase, interface, provider):
+class CommonTests(CommonCommonTests):
"""
- Verify that C{provider} properly provides C{interface}
-
- @type interface: L{zope.interface.Interface}
- @type provider: C{provider}
- """
- try:
- verifyObject(interface, provider)
- except BrokenMethodImplementation, e:
- testCase.fail(e)
- except DoesNotImplement, e:
- testCase.fail("%r does not provide %s.%s" %
- (provider, interface.__module__, interface.getName()))
-
-
-
-class CommonTests(object):
- """
Tests for common functionality of interfaces defined in
L{txdav.carddav.iaddressbookstore}.
"""
@@ -152,49 +133,6 @@
raise NotImplementedError()
- lastTransaction = None
- savedStore = None
-
- def transactionUnderTest(self):
- """
- Create a transaction from C{storeUnderTest} and save it as
- C[lastTransaction}. Also makes sure to use the same store, saving the
- value from C{storeUnderTest}.
- """
- if self.lastTransaction is not None:
- return self.lastTransaction
- if self.savedStore is None:
- self.savedStore = self.storeUnderTest()
- txn = self.lastTransaction = self.savedStore.newTransaction(self.id())
- return txn
-
-
- def commit(self):
- """
- Commit the last transaction created from C{transactionUnderTest}, and
- clear it.
- """
- self.lastTransaction.commit()
- self.lastTransaction = None
-
-
- def abort(self):
- """
- Abort the last transaction created from C[transactionUnderTest}, and
- clear it.
- """
- self.lastTransaction.abort()
- self.lastTransaction = None
-
- def setUp(self):
- self.notifierFactory = StubNotifierFactory()
-
- def tearDown(self):
- if self.lastTransaction is not None:
- self.commit()
-
-
-
def homeUnderTest(self):
"""
Get the addressbook home detailed by C{requirements['home1']}.
@@ -202,24 +140,25 @@
return self.transactionUnderTest().addressbookHomeWithUID("home1")
+ @inlineCallbacks
def addressbookUnderTest(self):
"""
Get the addressbook detailed by C{requirements['home1']['addressbook_1']}.
"""
- return self.homeUnderTest().addressbookWithName("addressbook_1")
+ returnValue((yield (yield self.homeUnderTest())
+ .addressbookWithName("addressbook_1")))
+ @inlineCallbacks
def addressbookObjectUnderTest(self):
"""
Get the addressbook detailed by
C{requirements['home1']['addressbook_1']['1.vcf']}.
"""
- return self.addressbookUnderTest().addressbookObjectWithName("1.vcf")
+ returnValue((yield self.addressbookUnderTest())
+ .addressbookObjectWithName("1.vcf"))
- assertProvides = assertProvides
-
-
def test_addressbookStoreProvides(self):
"""
The addressbook store provides L{IAddressBookStore} and its required
@@ -247,50 +186,58 @@
self.assertProvides(IAddressBookHome, self.homeUnderTest())
+ @inlineCallbacks
def test_addressbookProvides(self):
"""
The addressbooks generated by the addressbook store provide L{IAddressBook} and
its required attributes.
"""
- self.assertProvides(IAddressBook, self.addressbookUnderTest())
+ self.assertProvides(IAddressBook, (yield self.addressbookUnderTest()))
+ @inlineCallbacks
def test_addressbookObjectProvides(self):
"""
The addressbook objects generated by the addressbook store provide
L{IAddressBookObject} and its required attributes.
"""
- self.assertProvides(IAddressBookObject, self.addressbookObjectUnderTest())
+ self.assertProvides(IAddressBookObject,
+ (yield self.addressbookObjectUnderTest()))
+
+ @inlineCallbacks
def test_notifierID(self):
home = self.homeUnderTest()
self.assertEquals(home.notifierID(), "CardDAV|home1")
- addressbook = home.addressbookWithName("addressbook_1")
+ addressbook = yield home.addressbookWithName("addressbook_1")
self.assertEquals(addressbook.notifierID(), "CardDAV|home1")
self.assertEquals(addressbook.notifierID(label="collection"), "CardDAV|home1/addressbook_1")
+ @inlineCallbacks
def test_addressbookHomeWithUID_exists(self):
"""
Finding an existing addressbook home by UID results in an object that
provides L{IAddressBookHome} and has a C{uid()} method that returns the
same value that was passed in.
"""
- addressbookHome = (self.transactionUnderTest()
- .addressbookHomeWithUID("home1"))
+ addressbookHome = (yield self.transactionUnderTest()
+ .addressbookHomeWithUID("home1"))
self.assertEquals(addressbookHome.uid(), "home1")
self.assertProvides(IAddressBookHome, addressbookHome)
+ @inlineCallbacks
def test_addressbookHomeWithUID_absent(self):
"""
L{IAddressBookStoreTransaction.addressbookHomeWithUID} should return C{None}
when asked for a non-existent addressbook home.
"""
txn = self.transactionUnderTest()
- self.assertEquals(txn.addressbookHomeWithUID("xyzzy"), None)
+ self.assertEquals((yield txn.addressbookHomeWithUID("xyzzy")), None)
+ @inlineCallbacks
def test_addressbookWithName_exists(self):
"""
L{IAddressBookHome.addressbookWithName} returns an L{IAddressBook} provider,
@@ -298,43 +245,48 @@
"""
home = self.homeUnderTest()
for name in home1_addressbookNames:
- addressbook = home.addressbookWithName(name)
+ addressbook = yield home.addressbookWithName(name)
if addressbook is None:
self.fail("addressbook %r didn't exist" % (name,))
self.assertProvides(IAddressBook, addressbook)
self.assertEquals(addressbook.name(), name)
+ @inlineCallbacks
def test_addressbookRename(self):
"""
L{IAddressBook.rename} changes the name of the L{IAddressBook}.
"""
home = self.homeUnderTest()
- addressbook = home.addressbookWithName("addressbook_1")
+ addressbook = yield home.addressbookWithName("addressbook_1")
addressbook.rename("some_other_name")
+ @inlineCallbacks
def positiveAssertions():
self.assertEquals(addressbook.name(), "some_other_name")
- self.assertEquals(addressbook, home.addressbookWithName("some_other_name"))
- self.assertEquals(None, home.addressbookWithName("addressbook_1"))
+ self.assertEquals(addressbook, (yield home.addressbookWithName("some_other_name")))
+ self.assertEquals(None, (yield home.addressbookWithName("addressbook_1")))
+ yield positiveAssertions()
+ yield self.commit()
+ home = yield self.homeUnderTest()
+ addressbook = yield home.addressbookWithName("some_other_name")
positiveAssertions()
- self.commit()
- home = self.homeUnderTest()
- addressbook = home.addressbookWithName("some_other_name")
- positiveAssertions()
# FIXME: revert
# FIXME: test for multiple renames
# FIXME: test for conflicting renames (a->b, c->a in the same txn)
+ @inlineCallbacks
def test_addressbookWithName_absent(self):
"""
L{IAddressBookHome.addressbookWithName} returns C{None} for addressbooks which
do not exist.
"""
- self.assertEquals(self.homeUnderTest().addressbookWithName("xyzzy"),
- None)
+ self.assertEquals(
+ (yield (yield self.homeUnderTest()).addressbookWithName("xyzzy")),
+ None)
+ @inlineCallbacks
def test_createAddressBookWithName_absent(self):
"""
L{IAddressBookHome.createAddressBookWithName} creates a new L{IAddressBook} that
@@ -342,11 +294,11 @@
"""
home = self.homeUnderTest()
name = "new"
- self.assertIdentical(home.addressbookWithName(name), None)
+ self.assertIdentical((yield home.addressbookWithName(name)), None)
home.createAddressBookWithName(name)
- self.assertNotIdentical(home.addressbookWithName(name), None)
+ self.assertNotIdentical((yield home.addressbookWithName(name)), None)
def checkProperties():
- addressbookProperties = home.addressbookWithName(name).properties()
+ addressbookProperties = (yield home.addressbookWithName(name)).properties()
addressbookType = davxml.ResourceType.addressbook #@UndefinedVariable
self.assertEquals(
addressbookProperties[
@@ -355,7 +307,7 @@
addressbookType
)
checkProperties()
- self.commit()
+ yield self.commit()
# Make sure notification fired after commit
self.assertEquals(self.notifierFactory.history,
@@ -363,7 +315,7 @@
# Make sure it's available in a new transaction; i.e. test the commit.
home = self.homeUnderTest()
- self.assertNotIdentical(home.addressbookWithName(name), None)
+ self.assertNotIdentical((yield home.addressbookWithName(name)), None)
# FIXME: These two lines aren't in the calendar common tests:
# home = self.addressbookStore.newTransaction().addressbookHomeWithUID(
@@ -387,6 +339,7 @@
)
+ @inlineCallbacks
def test_removeAddressBookWithName_exists(self):
"""
L{IAddressBookHome.removeAddressBookWithName} removes a addressbook that already
@@ -395,11 +348,11 @@
home = self.homeUnderTest()
# FIXME: test transactions
for name in home1_addressbookNames:
- self.assertNotIdentical(home.addressbookWithName(name), None)
+ self.assertNotIdentical((yield home.addressbookWithName(name)), None)
home.removeAddressBookWithName(name)
- self.assertEquals(home.addressbookWithName(name), None)
+ self.assertEquals((yield home.addressbookWithName(name)), None)
- self.commit()
+ yield self.commit()
# Make sure notification fired after commit
self.assertEquals(
@@ -424,12 +377,13 @@
home.removeAddressBookWithName, "xyzzy")
+ @inlineCallbacks
def test_addressbookObjects(self):
"""
L{IAddressBook.addressbookObjects} will enumerate the addressbook objects present
in the filesystem, in name order, but skip those with hidden names.
"""
- addressbook1 = self.addressbookUnderTest()
+ addressbook1 = yield self.addressbookUnderTest()
addressbookObjects = list(addressbook1.addressbookObjects())
for addressbookObject in addressbookObjects:
@@ -445,35 +399,38 @@
)
+ @inlineCallbacks
def test_addressbookObjectsWithRemovedObject(self):
"""
L{IAddressBook.addressbookObjects} skips those objects which have been
removed by L{AddressBook.removeAddressBookObjectWithName} in the same
transaction, even if it has not yet been committed.
"""
- addressbook1 = self.addressbookUnderTest()
+ addressbook1 = yield self.addressbookUnderTest()
addressbook1.removeAddressBookObjectWithName("2.vcf")
addressbookObjects = list(addressbook1.addressbookObjects())
self.assertEquals(set(o.name() for o in addressbookObjects),
set(addressbook1_objectNames) - set(["2.vcf"]))
+ @inlineCallbacks
def test_ownerAddressBookHome(self):
"""
L{IAddressBook.ownerAddressBookHome} should match the home UID.
"""
self.assertEquals(
- self.addressbookUnderTest().ownerAddressBookHome().uid(),
- self.homeUnderTest().uid()
+ (yield self.addressbookUnderTest()).ownerAddressBookHome().uid(),
+ (yield self.homeUnderTest()).uid()
)
+ @inlineCallbacks
def test_addressbookObjectWithName_exists(self):
"""
L{IAddressBook.addressbookObjectWithName} returns an L{IAddressBookObject}
provider for addressbooks which already exist.
"""
- addressbook1 = self.addressbookUnderTest()
+ addressbook1 = yield self.addressbookUnderTest()
for name in addressbook1_objectNames:
addressbookObject = addressbook1.addressbookObjectWithName(name)
self.assertProvides(IAddressBookObject, addressbookObject)
@@ -481,20 +438,22 @@
# FIXME: add more tests based on CommonTests.requirements
+ @inlineCallbacks
def test_addressbookObjectWithName_absent(self):
"""
L{IAddressBook.addressbookObjectWithName} returns C{None} for addressbooks which
don't exist.
"""
- addressbook1 = self.addressbookUnderTest()
+ addressbook1 = yield self.addressbookUnderTest()
self.assertEquals(addressbook1.addressbookObjectWithName("xyzzy"), None)
+ @inlineCallbacks
def test_removeAddressBookObjectWithUID_exists(self):
"""
Remove an existing addressbook object.
"""
- addressbook = self.addressbookUnderTest()
+ addressbook = yield self.addressbookUnderTest()
for name in addressbook1_objectNames:
uid = (u'uid' + name.rstrip(".vcf"))
self.assertNotIdentical(addressbook.addressbookObjectWithUID(uid),
@@ -510,11 +469,12 @@
)
+ @inlineCallbacks
def test_removeAddressBookObjectWithName_exists(self):
"""
Remove an existing addressbook object.
"""
- addressbook = self.addressbookUnderTest()
+ addressbook = yield self.addressbookUnderTest()
for name in addressbook1_objectNames:
self.assertNotIdentical(
addressbook.addressbookObjectWithName(name), None
@@ -525,7 +485,7 @@
)
# Make sure notifications are fired after commit
- self.commit()
+ yield self.commit()
self.assertEquals(
self.notifierFactory.history,
[
@@ -539,37 +499,43 @@
)
+ @inlineCallbacks
def test_removeAddressBookObjectWithName_absent(self):
"""
Attempt to remove an non-existing addressbook object should raise.
"""
- addressbook = self.addressbookUnderTest()
+ addressbook = yield self.addressbookUnderTest()
self.assertRaises(
NoSuchObjectResourceError,
addressbook.removeAddressBookObjectWithName, "xyzzy"
)
+ @inlineCallbacks
def test_addressbookName(self):
"""
L{AddressBook.name} reflects the name of the addressbook.
"""
- self.assertEquals(self.addressbookUnderTest().name(), "addressbook_1")
+ self.assertEquals((yield self.addressbookUnderTest()).name(), "addressbook_1")
+ @inlineCallbacks
def test_addressbookObjectName(self):
"""
L{IAddressBookObject.name} reflects the name of the addressbook object.
"""
- self.assertEquals(self.addressbookObjectUnderTest().name(), "1.vcf")
+ self.assertEquals(
+ (yield self.addressbookObjectUnderTest()).name(),
+ "1.vcf")
+ @inlineCallbacks
def test_component(self):
"""
L{IAddressBookObject.component} returns a L{VComponent} describing the
addressbook data underlying that addressbook object.
"""
- component = self.addressbookObjectUnderTest().component()
+ component = (yield self.addressbookObjectUnderTest()).component()
self.failUnless(
isinstance(component, VComponent),
@@ -580,35 +546,39 @@
self.assertEquals(component.resourceUID(), "uid1")
+ @inlineCallbacks
def test_iAddressBookText(self):
"""
L{IAddressBookObject.iAddressBookText} returns a C{str} describing the same
data provided by L{IAddressBookObject.component}.
"""
- text = self.addressbookObjectUnderTest().vCardText()
+ text = (yield self.addressbookObjectUnderTest()).vCardText()
self.assertIsInstance(text, str)
self.failUnless(text.startswith("BEGIN:VCARD\r\n"))
self.assertIn("\r\nUID:uid1\r\n", text)
self.failUnless(text.endswith("\r\nEND:VCARD\r\n"))
+ @inlineCallbacks
def test_addressbookObjectUID(self):
"""
L{IAddressBookObject.uid} returns a C{str} describing the C{UID} property
of the addressbook object's component.
"""
- self.assertEquals(self.addressbookObjectUnderTest().uid(), "uid1")
+ self.assertEquals((yield self.addressbookObjectUnderTest()).uid(), "uid1")
+ @inlineCallbacks
def test_addressbookObjectWithUID_absent(self):
"""
L{IAddressBook.addressbookObjectWithUID} returns C{None} for addressbooks which
don't exist.
"""
- addressbook1 = self.addressbookUnderTest()
+ addressbook1 = yield self.addressbookUnderTest()
self.assertEquals(addressbook1.addressbookObjectWithUID("xyzzy"), None)
+ @inlineCallbacks
def test_addressbooks(self):
"""
L{IAddressBookHome.addressbooks} returns an iterable of L{IAddressBook}
@@ -617,13 +587,15 @@
"""
# Add a dot directory to make sure we don't find it
# self.home1._path.child(".foo").createDirectory()
- home = self.homeUnderTest()
- addressbooks = list(home.addressbooks())
+ home = yield self.homeUnderTest()
+ addressbooks = list((yield home.addressbooks()))
for addressbook in addressbooks:
self.assertProvides(IAddressBook, addressbook)
- self.assertEquals(addressbook,
- home.addressbookWithName(addressbook.name()))
+ self.assertEquals(
+ addressbook,
+ (yield home.addressbookWithName(addressbook.name()))
+ )
self.assertEquals(
set(c.name() for c in addressbooks),
@@ -643,12 +615,13 @@
self.assertEquals(before | set(['new-name']), after)
+ @inlineCallbacks
def test_createAddressBookObjectWithName_absent(self):
"""
L{IAddressBook.createAddressBookObjectWithName} creates a new
L{IAddressBookObject}.
"""
- addressbook1 = self.addressbookUnderTest()
+ addressbook1 = yield self.addressbookUnderTest()
name = "4.vcf"
self.assertIdentical(addressbook1.addressbookObjectWithName(name), None)
component = VComponent.fromString(vcard4_text)
@@ -657,7 +630,7 @@
addressbookObject = addressbook1.addressbookObjectWithName(name)
self.assertEquals(addressbookObject.component(), component)
- self.commit()
+ yield self.commit()
# Make sure notifications fire after commit
self.assertEquals(
@@ -669,6 +642,7 @@
)
+ @inlineCallbacks
def test_createAddressBookObjectWithName_exists(self):
"""
L{IAddressBook.createAddressBookObjectWithName} raises
@@ -677,11 +651,12 @@
"""
self.assertRaises(
ObjectResourceNameAlreadyExistsError,
- self.addressbookUnderTest().createAddressBookObjectWithName,
+ (yield self.addressbookUnderTest()).createAddressBookObjectWithName,
"1.vcf", VComponent.fromString(vcard4_text)
)
+ @inlineCallbacks
def test_createAddressBookObjectWithName_invalid(self):
"""
L{IAddressBook.createAddressBookObjectWithName} raises
@@ -690,17 +665,18 @@
"""
self.assertRaises(
InvalidObjectResourceError,
- self.addressbookUnderTest().createAddressBookObjectWithName,
+ (yield self.addressbookUnderTest()).createAddressBookObjectWithName,
"new", VComponent.fromString(vcard4notCardDAV_text)
)
+ @inlineCallbacks
def test_setComponent_invalid(self):
"""
L{IAddressBookObject.setComponent} raises L{InvalidIAddressBookDataError} if
presented with invalid iAddressBook text.
"""
- addressbookObject = self.addressbookObjectUnderTest()
+ addressbookObject = (yield self.addressbookObjectUnderTest())
self.assertRaises(
InvalidObjectResourceError,
addressbookObject.setComponent,
@@ -708,12 +684,13 @@
)
+ @inlineCallbacks
def test_setComponent_uidchanged(self):
"""
L{IAddressBookObject.setComponent} raises L{InvalidAddressBookComponentError}
when given a L{VComponent} whose UID does not match its existing UID.
"""
- addressbook1 = self.addressbookUnderTest()
+ addressbook1 = yield self.addressbookUnderTest()
component = VComponent.fromString(vcard4_text)
addressbookObject = addressbook1.addressbookObjectWithName("1.vcf")
self.assertRaises(
@@ -722,6 +699,7 @@
)
+ @inlineCallbacks
def test_addressbookHomeWithUID_create(self):
"""
L{IAddressBookStoreTransaction.addressbookHomeWithUID} with C{create=True}
@@ -729,22 +707,24 @@
"""
txn = self.transactionUnderTest()
noHomeUID = "xyzzy"
- addressbookHome = txn.addressbookHomeWithUID(
+ addressbookHome = yield txn.addressbookHomeWithUID(
noHomeUID,
create=True
)
+ @inlineCallbacks
def readOtherTxn():
otherTxn = self.savedStore.newTransaction()
self.addCleanup(otherTxn.commit)
- return otherTxn.addressbookHomeWithUID(noHomeUID)
+ returnValue((yield otherTxn.addressbookHomeWithUID(noHomeUID)))
self.assertProvides(IAddressBookHome, addressbookHome)
- # A concurrent transaction shouldn't be able to read it yet:
- self.assertIdentical(readOtherTxn(), None)
- self.commit()
+ # A concurrent tnransaction shouldn't be able to read it yet:
+ self.assertIdentical((yield readOtherTxn()), None)
+ yield self.commit()
# But once it's committed, other transactions should see it.
- self.assertProvides(IAddressBookHome, readOtherTxn())
+ self.assertProvides(IAddressBookHome, (yield readOtherTxn()))
+ @inlineCallbacks
def test_setComponent(self):
"""
L{AddressBookObject.setComponent} changes the result of
@@ -752,7 +732,7 @@
"""
component = VComponent.fromString(vcard1modified_text)
- addressbook1 = self.addressbookUnderTest()
+ addressbook1 = yield self.addressbookUnderTest()
addressbookObject = addressbook1.addressbookObjectWithName("1.vcf")
oldComponent = addressbookObject.component()
self.assertNotEqual(component, oldComponent)
@@ -763,7 +743,7 @@
addressbookObject = addressbook1.addressbookObjectWithName("1.vcf")
self.assertEquals(addressbookObject.component(), component)
- self.commit()
+ yield self.commit()
# Make sure notification fired after commit
self.assertEquals(
@@ -790,26 +770,29 @@
self.checkPropertiesMethod(self.homeUnderTest())
+ @inlineCallbacks
def test_addressbookProperties(self):
"""
L{IAddressBook.properties} returns a property store.
"""
- self.checkPropertiesMethod(self.addressbookUnderTest())
+ self.checkPropertiesMethod((yield self.addressbookUnderTest()))
+ @inlineCallbacks
def test_addressbookObjectProperties(self):
"""
L{IAddressBookObject.properties} returns a property store.
"""
- self.checkPropertiesMethod(self.addressbookObjectUnderTest())
+ self.checkPropertiesMethod((yield self.addressbookObjectUnderTest()))
+ @inlineCallbacks
def test_newAddressBookObjectProperties(self):
"""
L{IAddressBookObject.properties} returns an empty property store for a
addressbook object which has been created but not committed.
"""
- addressbook = self.addressbookUnderTest()
+ addressbook = yield self.addressbookUnderTest()
addressbook.createAddressBookObjectWithName(
"4.vcf", VComponent.fromString(vcard4_text)
)
@@ -817,6 +800,7 @@
self.assertEquals(newEvent.properties().items(), [])
+ @inlineCallbacks
def test_setComponentPreservesProperties(self):
"""
L{IAddressBookObject.setComponent} preserves properties.
@@ -830,15 +814,17 @@
propertyContent.name = propertyName.name
propertyContent.namespace = propertyName.namespace
- self.addressbookObjectUnderTest().properties()[
+ (yield self.addressbookObjectUnderTest()).properties()[
propertyName] = propertyContent
- self.commit()
+ yield self.commit()
# Sanity check; are properties even readable in a separate transaction?
# Should probably be a separate test.
self.assertEquals(
- self.addressbookObjectUnderTest().properties()[propertyName],
+ (yield self.addressbookObjectUnderTest()).properties()[
+ propertyName
+ ],
propertyContent)
- obj = self.addressbookObjectUnderTest()
+ obj = yield self.addressbookObjectUnderTest()
vcard1_text = obj.vCardText()
vcard1_text_withDifferentNote = vcard1_text.replace(
"NOTE:CardDAV protocol updates",
@@ -851,34 +837,37 @@
# Putting everything into a separate transaction to account for any
# caching that may take place.
- self.commit()
+ yield self.commit()
self.assertEquals(
- self.addressbookObjectUnderTest().properties()[propertyName],
+ (yield self.addressbookObjectUnderTest()).properties()[propertyName],
propertyContent
)
+ @inlineCallbacks
def test_dontLeakAddressbooks(self):
"""
Addressbooks in one user's addressbook home should not show up in another
user's addressbook home.
"""
- home2 = self.transactionUnderTest().addressbookHomeWithUID(
- "home2", create=True)
- self.assertIdentical(home2.addressbookWithName("addressbook_1"), None)
+ home2 = yield self.transactionUnderTest().addressbookHomeWithUID(
+ "home2", create=True
+ )
+ self.assertIdentical((yield home2.addressbookWithName("addressbook_1")), None)
+ @inlineCallbacks
def test_dontLeakObjects(self):
"""
Addressbook objects in one user's addressbook should not show up in another
user's via uid or name queries.
"""
home1 = self.homeUnderTest()
- home2 = self.transactionUnderTest().addressbookHomeWithUID(
+ home2 = yield self.transactionUnderTest().addressbookHomeWithUID(
"home2", create=True)
- addressbook1 = home1.addressbookWithName("addressbook_1")
- addressbook2 = home2.addressbookWithName("addressbook")
- objects = list(home2.addressbookWithName("addressbook").addressbookObjects())
+ addressbook1 = yield home1.addressbookWithName("addressbook_1")
+ addressbook2 = yield home2.addressbookWithName("addressbook")
+ objects = list((yield (yield home2.addressbookWithName("addressbook")).addressbookObjects()))
self.assertEquals(objects, [])
for resourceName in self.requirements['home1']['addressbook_1'].keys():
obj = addressbook1.addressbookObjectWithName(resourceName)
@@ -888,6 +877,7 @@
addressbook2.addressbookObjectWithUID(obj.uid()), None)
+ @inlineCallbacks
def test_eachAddressbookHome(self):
"""
L{IAddressbookTransaction.eachAddressbookHome} returns an iterator that
@@ -897,8 +887,8 @@
additionalUIDs = set('alpha-uid home2 home3 beta-uid'.split())
txn = self.transactionUnderTest()
for name in additionalUIDs:
- txn.addressbookHomeWithUID(name, create=True)
- self.commit()
+ yield txn.addressbookHomeWithUID(name, create=True)
+ yield self.commit()
foundUIDs = set([])
lastTxn = None
for txn, home in self.storeUnderTest().eachAddressbookHome():
@@ -915,18 +905,3 @@
-class StubNotifierFactory(object):
-
- """ For testing push notifications without an XMPP server """
-
- def __init__(self):
- self.reset()
-
- def newNotifier(self, label="default", id=None, prefix=None):
- return Notifier(self, label=label, id=id, prefix=prefix)
-
- def send(self, op, id):
- self.history.append((op, id))
-
- def reset(self):
- self.history = []
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_file.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_file.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_file.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -18,9 +18,11 @@
File addressbook store tests.
"""
-from twext.python.filepath import CachingFilePath as FilePath
+from twisted.internet.defer import inlineCallbacks
from twisted.trial import unittest
+from twext.python.filepath import CachingFilePath as FilePath
+
from twistedcaldav.vcard import Component as VComponent
from txdav.common.icommondatastore import HomeChildNameNotAllowedError
@@ -33,8 +35,10 @@
from txdav.carddav.datastore.file import AddressBook, AddressBookObject
from txdav.carddav.datastore.test.common import (
- CommonTests, vcard4_text, vcard1modified_text, StubNotifierFactory)
+ CommonTests, vcard4_text, vcard1modified_text)
+from txdav.common.datastore.test.util import StubNotifierFactory
+
storePath = FilePath(__file__).parent().child("addressbook_store")
def _todo(f, why):
@@ -59,21 +63,23 @@
test.notifierFactory = StubNotifierFactory()
test.addressbookStore = AddressBookStore(storeRootPath, test.notifierFactory)
- test.txn = test.addressbookStore.newTransaction()
+ test.txn = test.addressbookStore.newTransaction(test.id() + " (old)")
assert test.addressbookStore is not None, "No addressbook store?"
+ at inlineCallbacks
def setUpHome1(test):
setUpAddressBookStore(test)
- test.home1 = test.txn.addressbookHomeWithUID("home1")
+ test.home1 = yield test.txn.addressbookHomeWithUID("home1")
assert test.home1 is not None, "No addressbook home?"
+ at inlineCallbacks
def setUpAddressBook1(test):
- setUpHome1(test)
- test.addressbook1 = test.home1.addressbookWithName("addressbook_1")
+ yield setUpHome1(test)
+ test.addressbook1 = yield test.home1.addressbookWithName("addressbook_1")
assert test.addressbook1 is not None, "No addressbook?"
@@ -87,13 +93,15 @@
setUpAddressBookStore(self)
+ @inlineCallbacks
def test_addressbookHomeWithUID_dot(self):
"""
Filenames starting with "." are reserved by this
implementation, so no UIDs may start with ".".
"""
self.assertEquals(
- self.addressbookStore.newTransaction().addressbookHomeWithUID("xyzzy"),
+ (yield self.addressbookStore.newTransaction(self.id()
+ ).addressbookHomeWithUID(".xyzzy")),
None
)
@@ -102,7 +110,7 @@
class AddressBookHomeTest(unittest.TestCase):
def setUp(self):
- setUpHome1(self)
+ return setUpHome1(self)
def test_init(self):
@@ -176,6 +184,7 @@
)
+ @inlineCallbacks
def test_useIndexImmediately(self):
"""
L{AddressBook._index} is usable in the same transaction it is created, with
@@ -186,10 +195,10 @@
index = addressbook._index
self.assertEquals(set(index.addressbookObjects()),
set(addressbook.addressbookObjects()))
- self.txn.commit()
- self.txn = self.addressbookStore.newTransaction()
- self.home1 = self.txn.addressbookHomeWithUID("home1")
- addressbook = self.home1.addressbookWithName("addressbook2")
+ yield self.txn.commit()
+ self.txn = self.addressbookStore.newTransaction(self.id())
+ self.home1 = yield self.txn.addressbookHomeWithUID("home1")
+ addressbook = yield self.home1.addressbookWithName("addressbook2")
# FIXME: we should be curating our own index here, but in order to fix
# that the code in the old implicit scheduler needs to change. This
# test would be more effective if there were actually some objects in
@@ -280,16 +289,18 @@
)
+ @inlineCallbacks
def _refresh(self):
"""
Re-read the (committed) home1 and addressbook1 objects in a new
transaction.
"""
- self.txn = self.addressbookStore.newTransaction()
- self.home1 = self.txn.addressbookHomeWithUID("home1")
- self.addressbook1 = self.home1.addressbookWithName("addressbook_1")
+ self.txn = self.addressbookStore.newTransaction(self.id())
+ self.home1 = yield self.txn.addressbookHomeWithUID("home1")
+ self.addressbook1 = yield self.home1.addressbookWithName("addressbook_1")
+ @inlineCallbacks
def test_undoCreateAddressBookObject(self):
"""
If a addressbook object is created as part of a transaction, it will be
@@ -297,19 +308,21 @@
"""
# Make sure that the addressbook home is actually committed; rolling back
# addressbook home creation will remove the whole directory.
- self.txn.commit()
- self._refresh()
+ yield self.txn.commit()
+ yield self._refresh()
self.addressbook1.createAddressBookObjectWithName(
"sample.vcf",
VComponent.fromString(vcard4_text)
)
- self._refresh()
+ yield self.txn.abort()
+ yield self._refresh()
self.assertIdentical(
- self.addressbook1.addressbookObjectWithName("sample.vcf"),
+ (yield self.addressbook1.addressbookObjectWithName("sample.vcf")),
None
)
+ @inlineCallbacks
def doThenUndo(self):
"""
Commit the current transaction, but add an operation that will cause it
@@ -321,17 +334,18 @@
raise RuntimeError("oops")
self.txn.addOperation(fail, "dummy failing operation")
self.assertRaises(RuntimeError, self.txn.commit)
- self._refresh()
+ yield self._refresh()
+ @inlineCallbacks
def test_undoModifyAddressBookObject(self):
"""
If an existing addressbook object is modified as part of a transaction, it
should be restored to its previous status if the transaction aborts.
"""
- originalComponent = self.addressbook1.addressbookObjectWithName(
+ originalComponent = yield self.addressbook1.addressbookObjectWithName(
"1.vcf").component()
- self.addressbook1.addressbookObjectWithName("1.vcf").setComponent(
+ (yield self.addressbook1.addressbookObjectWithName("1.vcf")).setComponent(
VComponent.fromString(vcard1modified_text)
)
# Sanity check.
@@ -339,7 +353,7 @@
self.addressbook1.addressbookObjectWithName("1.vcf").component(),
VComponent.fromString(vcard1modified_text)
)
- self.doThenUndo()
+ yield self.doThenUndo()
self.assertEquals(
self.addressbook1.addressbookObjectWithName("1.vcf").component(),
originalComponent
@@ -359,6 +373,7 @@
modifiedComponent,
self.addressbook1.addressbookObjectWithName("1.vcf").component()
)
+ self.txn.commit()
@featureUnimplemented
@@ -420,7 +435,7 @@
)
-class FileStorageTests(unittest.TestCase, CommonTests):
+class FileStorageTests(CommonTests, unittest.TestCase):
"""
File storage tests.
"""
@@ -447,7 +462,8 @@
def test_addressbookObjectsWithDotFile(self):
"""
- Adding a dotfile to the addressbook home should not increase
+ Adding a dotfile to the addressbook home should not create a new
+ addressbook object.
"""
self.homeUnderTest()._path.child(".foo").createDirectory()
self.test_addressbookObjects()
Copied: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_index_file.py (from rev 6368, CalendarServer/branches/users/glyph/more-deferreds-6/txdav/carddav/datastore/test/test_index_file.py)
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_index_file.py (rev 0)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_index_file.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -0,0 +1,212 @@
+##
+# Copyright (c) 2010 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twisted.internet import reactor
+from twisted.internet.task import deferLater
+
+from txdav.common.icommondatastore import ReservationError
+from txdav.carddav.datastore.index_file import AddressBookIndex,\
+ MemcachedUIDReserver
+
+from twistedcaldav.test.util import InMemoryMemcacheProtocol
+from twistedcaldav.vcard import Component
+import twistedcaldav.test.util
+
+import os
+
+class MinimalResourceReplacement(object):
+ """
+ Provide the minimal set of attributes and methods from CalDAVFile required
+ by L{Index}.
+ """
+
+ def __init__(self, filePath):
+ self.fp = filePath
+
+
+ def isAddressBookCollection(self):
+ return True
+
+
+ def getChild(self, name):
+ # FIXME: this should really return something with a child method
+ return self.fp.child(name)
+
+
+ def initSyncToken(self):
+ pass
+
+
+
+class SQLIndexTests (twistedcaldav.test.util.TestCase):
+ """
+ Test abstract SQL DB class
+ """
+
+ def setUp(self):
+ super(SQLIndexTests, self).setUp()
+ self.site.resource.isAddressBookCollection = lambda: True
+ self.indexDirPath = self.site.resource.fp
+ # FIXME: since this resource lies about isCalendarCollection, it doesn't
+ # have all the associated backend machinery to actually get children.
+ self.db = AddressBookIndex(MinimalResourceReplacement(self.indexDirPath))
+
+
+ def test_reserve_uid_ok(self):
+ uid = "test-test-test"
+ d = self.db.isReservedUID(uid)
+ d.addCallback(self.assertFalse)
+ d.addCallback(lambda _: self.db.reserveUID(uid))
+ d.addCallback(lambda _: self.db.isReservedUID(uid))
+ d.addCallback(self.assertTrue)
+ d.addCallback(lambda _: self.db.unreserveUID(uid))
+ d.addCallback(lambda _: self.db.isReservedUID(uid))
+ d.addCallback(self.assertFalse)
+
+ return d
+
+
+ def test_reserve_uid_twice(self):
+ uid = "test-test-test"
+ d = self.db.reserveUID(uid)
+ d.addCallback(lambda _: self.db.isReservedUID(uid))
+ d.addCallback(self.assertTrue)
+ d.addCallback(lambda _:
+ self.assertFailure(self.db.reserveUID(uid),
+ ReservationError))
+ return d
+
+
+ def test_unreserve_unreserved(self):
+ uid = "test-test-test"
+ return self.assertFailure(self.db.unreserveUID(uid),
+ ReservationError)
+
+
+ def test_reserve_uid_timeout(self):
+ # WARNING: This test is fundamentally flawed and will fail
+ # intermittently because it uses the real clock.
+ uid = "test-test-test"
+ from twistedcaldav.config import config
+ old_timeout = config.UIDReservationTimeOut
+ config.UIDReservationTimeOut = 1
+
+ def _finally():
+ config.UIDReservationTimeOut = old_timeout
+
+ d = self.db.isReservedUID(uid)
+ d.addCallback(self.assertFalse)
+ d.addCallback(lambda _: self.db.reserveUID(uid))
+ d.addCallback(lambda _: self.db.isReservedUID(uid))
+ d.addCallback(self.assertTrue)
+ d.addCallback(lambda _: deferLater(reactor, 2, lambda: None))
+ d.addCallback(lambda _: self.db.isReservedUID(uid))
+ d.addCallback(self.assertFalse)
+ self.addCleanup(_finally)
+
+ return d
+
+
+ def test_index(self):
+ data = (
+ (
+ "#1.1 Simple component",
+ "1.1",
+ """BEGIN:VCARD
+VERSION:3.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+UID:12345-67890-1.1
+FN:Cyrus Daboo
+N:Daboo;Cyrus
+EMAIL;TYPE=INTERNET,PREF:cyrus at example.com
+END:VCARD
+""",
+ ),
+ )
+
+ for description, name, vcard_txt in data:
+ calendar = Component.fromString(vcard_txt)
+ f = open(os.path.join(self.site.resource.fp.path, name), "w")
+ f.write(vcard_txt)
+ del f
+
+ self.db.addResource(name, calendar)
+ self.assertTrue(self.db.resourceExists(name), msg=description)
+
+ self.db._db_recreate()
+ for description, name, vcard_txt in data:
+ self.assertTrue(self.db.resourceExists(name), msg=description)
+
+ def test_index_revisions(self):
+ data1 = """BEGIN:VCARD
+VERSION:3.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+UID:12345-67890-1-1.1
+FN:Cyrus Daboo
+N:Daboo;Cyrus
+EMAIL;TYPE=INTERNET,PREF:cyrus at example.com
+END:VCARD
+"""
+ data2 = """BEGIN:VCARD
+VERSION:3.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+UID:12345-67890-2-1.1
+FN:Wilfredo Sanchez
+N:Sanchez;Wilfredo
+EMAIL;TYPE=INTERNET,PREF:wsanchez at example.com
+END:VCARD
+"""
+ data3 = """BEGIN:VCARD
+VERSION:3.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+UID:12345-67890-3-1.1
+FN:Bruce Gaya
+N:Gaya;Bruce
+EMAIL;TYPE=INTERNET,PREF:bruce at example.com
+END:VCARD
+"""
+
+ vcard = Component.fromString(data1)
+ self.db.addResource("data1.vcf", vcard)
+ vcard = Component.fromString(data2)
+ self.db.addResource("data2.vcf", vcard)
+ vcard = Component.fromString(data3)
+ self.db.addResource("data3.vcf", vcard)
+ self.db.deleteResource("data3.vcf")
+
+ tests = (
+ (0, (["data1.vcf", "data2.vcf",], [],)),
+ (1, (["data2.vcf",], ["data3.vcf",],)),
+ (2, ([], ["data3.vcf",],)),
+ (3, ([], ["data3.vcf",],)),
+ (4, ([], [],)),
+ (5, ([], [],)),
+ )
+
+ for revision, results in tests:
+ self.assertEquals(self.db.whatchanged(revision), results, "Mismatched results for whatchanged with revision %d" % (revision,))
+
+class MemcacheTests(SQLIndexTests):
+ def setUp(self):
+ super(MemcacheTests, self).setUp()
+ self.memcache = InMemoryMemcacheProtocol()
+ self.db.reserver = MemcachedUIDReserver(self.db, self.memcache)
+
+
+ def tearDown(self):
+ for _ignore_k, v in self.memcache._timeouts.iteritems():
+ if v.active():
+ v.cancel()
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_sql.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_sql.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_sql.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -45,14 +45,15 @@
def setUp(self):
super(AddressBookSQLStorageTests, self).setUp()
self._sqlStore = yield buildStore(self, self.notifierFactory)
- self.populate()
+ yield self.populate()
+ @inlineCallbacks
def populate(self):
populateTxn = self.storeUnderTest().newTransaction()
for homeUID in self.requirements:
addressbooks = self.requirements[homeUID]
if addressbooks is not None:
- home = populateTxn.addressbookHomeWithUID(homeUID, True)
+ home = yield populateTxn.addressbookHomeWithUID(homeUID, True)
# We don't want the default addressbook to appear unless it's
# explicitly listed.
home.removeAddressBookWithName("addressbook")
@@ -67,7 +68,7 @@
objectName, VCard.fromString(objData)
)
- populateTxn.commit()
+ yield populateTxn.commit()
self.notifierFactory.reset()
@@ -121,14 +122,15 @@
return txn
+ @inlineCallbacks
def test_migrateAddressbookFromFile(self):
"""
C{_migrateAddressbook()} can migrate a file-backed addressbook to a
database- backed addressbook.
"""
- fromAddressbook = self.fileTransaction().addressbookHomeWithUID(
+ fromAddressbook = yield self.fileTransaction().addressbookHomeWithUID(
"home1").addressbookWithName("addressbook_1")
- toHome = self.transactionUnderTest().addressbookHomeWithUID(
+ toHome = yield self.transactionUnderTest().addressbookHomeWithUID(
"new-home", create=True)
toAddressbook = toHome.addressbookWithName("addressbook")
_migrateAddressbook(fromAddressbook, toAddressbook,
@@ -136,13 +138,14 @@
self.assertAddressbooksSimilar(fromAddressbook, toAddressbook)
+ @inlineCallbacks
def test_migrateHomeFromFile(self):
"""
L{migrateHome} will migrate an L{IAddressbookHome} provider from one
backend to another; in this specific case, from the file-based backend
to the SQL-based backend.
"""
- fromHome = self.fileTransaction().addressbookHomeWithUID("home1")
+ fromHome = yield self.fileTransaction().addressbookHomeWithUID("home1")
builtinProperties = [PropertyName.fromElement(ResourceType)]
@@ -151,10 +154,11 @@
key = PropertyName.fromElement(GETContentLanguage)
fromHome.properties()[key] = GETContentLanguage("C")
- fromHome.addressbookWithName("addressbook_1").properties()[key] = (
+ (yield fromHome.addressbookWithName("addressbook_1")).properties()[
+ key] = (
GETContentLanguage("pig-latin")
)
- toHome = self.transactionUnderTest().addressbookHomeWithUID(
+ toHome = yield self.transactionUnderTest().addressbookHomeWithUID(
"new-home", create=True
)
migrateHome(fromHome, toHome, lambda x: x.component())
@@ -203,19 +207,19 @@
# existing data in the table
home_uid2 = txn3.homeWithUID(EADDRESSBOOKTYPE, "uid2", create=True)
self.assertNotEqual(home_uid2, None)
- txn3.commit()
+ yield txn3.commit()
home_uid1_1 = txn1.homeWithUID(EADDRESSBOOKTYPE, "uid1", create=True)
def _defer_home_uid1_2():
home_uid1_2 = txn2.homeWithUID(EADDRESSBOOKTYPE, "uid1", create=True)
- txn2.commit()
+ txn2.commit() # FIXME: CONCURRENT
return home_uid1_2
d1 = deferToThread(_defer_home_uid1_2)
def _pause_home_uid1_1():
time.sleep(1)
- txn1.commit()
+ txn1.commit() # FIXME: CONCURRENT
d2 = deferToThread(_pause_home_uid1_1)
# Verify that we can still get to the existing home - i.e. the lock
@@ -223,7 +227,7 @@
txn4 = addressbookStore3.newTransaction()
home_uid2 = txn4.homeWithUID(EADDRESSBOOKTYPE, "uid2", create=True)
self.assertNotEqual(home_uid2, None)
- txn4.commit()
+ yield txn4.commit()
# Now do the concurrent provision attempt
yield d2
@@ -247,7 +251,7 @@
txn = addressbookStore1.newTransaction()
home = txn.homeWithUID(EADDRESSBOOKTYPE, "uid1", create=True)
self.assertNotEqual(home, None)
- txn.commit()
+ yield txn.commit()
txn1 = addressbookStore1.newTransaction()
txn2 = addressbookStore2.newTransaction()
@@ -257,7 +261,7 @@
adbk1 = home1.addressbookWithName("addressbook")
adbk2 = home2.addressbookWithName("addressbook")
-
+
def _defer1():
adbk1.createObjectResourceWithName("1.vcf", VCard.fromString(
"""BEGIN:VCARD
@@ -273,7 +277,7 @@
END:VCARD
""".replace("\n", "\r\n")
))
- txn1.commit()
+ txn1.commit() # FIXME: CONCURRENT
d1 = deferToThread(_defer1)
def _defer2():
@@ -291,7 +295,7 @@
END:VCARD
""".replace("\n", "\r\n")
))
- txn2.commit()
+ txn2.commit() # FIXME: CONCURRENT
d2 = deferToThread(_defer2)
yield d1
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/iaddressbookstore.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/iaddressbookstore.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/iaddressbookstore.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -44,8 +44,8 @@
If C{create} is C{True}, create the addressbook home if it doesn't
already exist.
- @return: an L{IAddressBookHome} or C{None} if no such addressbook
- home exists.
+ @return: a L{Deferred} which fires with an L{IAddressBookHome} or
+ C{None} if no such addressbook home exists.
"""
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_legacy.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_legacy.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_legacy.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -33,7 +33,6 @@
from twistedcaldav import carddavxml
from twistedcaldav.config import config
from twistedcaldav.dateops import normalizeForIndex
-from twistedcaldav.index import IndexedSearchException, ReservationError
from twistedcaldav.memcachepool import CachePoolUserMixIn
from twistedcaldav.notifications import NotificationRecord
from twistedcaldav.query import calendarqueryfilter, calendarquery, \
@@ -41,6 +40,8 @@
from twistedcaldav.query.sqlgenerator import sqlgenerator
from twistedcaldav.sharing import Invite
+from txdav.common.icommondatastore import IndexedSearchException, \
+ ReservationError
from txdav.common.datastore.sql_tables import \
_BIND_MODE_OWN, _BIND_MODE_READ, _BIND_MODE_WRITE, _BIND_MODE_DIRECT, \
_BIND_STATUS_INVITED, _BIND_STATUS_ACCEPTED, _BIND_STATUS_DECLINED, _BIND_STATUS_INVALID,\
@@ -349,7 +350,9 @@
def _getHomeWithUID(self, uid):
return self._txn.calendarHomeWithUID(uid, create=True)
-
+
+
+
class SQLLegacyAddressBookInvites(SQLLegacyInvites):
"""
Emulator for the implicit interface specified by
@@ -483,9 +486,6 @@
def addOrUpdateRecord(self, record):
-# print '*** SHARING***: Adding or updating this record:'
-# import pprint
-# pprint.pprint(record.__dict__)
# record.hosturl -> /.../__uids__/<uid>/<name>
splithost = record.hosturl.split('/')
ownerUID = splithost[3]
@@ -580,6 +580,8 @@
)
+
+
class SQLLegacyCalendarShares(SQLLegacyShares):
"""
Emulator for the implicit interface specified by
@@ -593,9 +595,12 @@
super(SQLLegacyCalendarShares, self).__init__(home)
+
def _getHomeWithUID(self, uid):
return self._txn.calendarHomeWithUID(uid, create=True)
-
+
+
+
class SQLLegacyAddressBookShares(SQLLegacyShares):
"""
Emulator for the implicit interface specified by
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/test/test_util.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/test/test_util.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/test/test_util.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -27,9 +27,9 @@
from txdav.common.datastore.util import UpgradeToDatabaseService
from txdav.common.datastore.file import CommonDataStore
from txdav.common.datastore.test.util import theStoreBuilder, \
- populateCalendarsFrom
-from txdav.caldav.datastore.test.common import StubNotifierFactory, CommonTests
-from twisted.internet.defer import inlineCallbacks, Deferred
+ populateCalendarsFrom, StubNotifierFactory
+from txdav.caldav.datastore.test.common import CommonTests
+from twisted.internet.defer import inlineCallbacks, Deferred, returnValue
class HomeMigrationTests(TestCase):
@@ -63,7 +63,7 @@
)
self.upgrader.setServiceParent(self.topService)
requirements = CommonTests.requirements
- populateCalendarsFrom(requirements, fileStore)
+ yield populateCalendarsFrom(requirements, fileStore)
self.filesPath.child("calendars").child(
"__uids__").child("ho").child("me").child("home1").child(
".some-extra-data").setContent("some extra data")
@@ -82,7 +82,7 @@
self.addCleanup(txn.commit)
for uid in CommonTests.requirements:
if CommonTests.requirements[uid] is not None:
- self.assertNotIdentical(None, txn.calendarHomeWithUID(uid))
+ self.assertNotIdentical(None, (yield txn.calendarHomeWithUID(uid)))
# Un-migrated data should be preserved.
self.assertEquals(self.filesPath.child("calendars-migrated").child(
"__uids__").child("ho").child("me").child("home1").child(
@@ -98,17 +98,17 @@
homes.
"""
startTxn = self.sqlStore.newTransaction("populate empty sample")
- startTxn.calendarHomeWithUID("home1", create=True)
- startTxn.commit()
+ yield startTxn.calendarHomeWithUID("home1", create=True)
+ yield startTxn.commit()
self.topService.startService()
yield self.subStarted
vrfyTxn = self.sqlStore.newTransaction("verify sample still empty")
self.addCleanup(vrfyTxn.commit)
- home = vrfyTxn.calendarHomeWithUID("home1")
+ home = yield vrfyTxn.calendarHomeWithUID("home1")
# The default calendar is still there.
- self.assertNotIdentical(None, home.calendarWithName("calendar"))
+ self.assertNotIdentical(None, (yield home.calendarWithName("calendar")))
# The migrated calendar isn't.
- self.assertIdentical(None, home.calendarWithName("calendar_1"))
+ self.assertIdentical(None, (yield home.calendarWithName("calendar_1")))
@inlineCallbacks
@@ -122,12 +122,15 @@
def maybeCommit():
if not committed:
committed.append(True)
- txn.commit()
+ return txn.commit()
self.addCleanup(maybeCommit)
+ @inlineCallbacks
def getSampleObj():
- return txn.calendarHomeWithUID("home1").calendarWithName(
- "calendar_1").calendarObjectWithName("1.ics")
- inObject = getSampleObj()
+ home = (yield txn.calendarHomeWithUID("home1"))
+ calendar = (yield home.calendarWithName("calendar_1"))
+ object = (yield calendar.calendarObjectWithName("1.ics"))
+ returnValue(object)
+ inObject = yield getSampleObj()
someAttachmentName = "some-attachment"
someAttachmentType = MimeType.fromString("application/x-custom-type")
transport = inObject.createAttachmentWithName(
@@ -141,7 +144,7 @@
yield self.subStarted
committed = []
txn = self.sqlStore.newTransaction()
- outObject = getSampleObj()
+ outObject = yield getSampleObj()
outAttachment = outObject.attachmentWithName(someAttachmentName)
allDone = Deferred()
class SimpleProto(Protocol):
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/test/util.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/test/util.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/test/util.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -20,12 +20,15 @@
"""
import gc
+from zope.interface.verify import verifyObject
+from zope.interface.exceptions import BrokenMethodImplementation,\
+ DoesNotImplement
from twext.python.filepath import CachingFilePath
from twext.python.vcomponent import VComponent
from twisted.internet import reactor
-from twisted.internet.defer import Deferred, succeed
+from twisted.internet.defer import Deferred, succeed, inlineCallbacks
from twisted.internet.task import deferLater
from twisted.python import log
@@ -33,6 +36,7 @@
from txdav.base.datastore.subpostgres import PostgresService,\
DiagnosticConnectionWrapper
from txdav.common.icommondatastore import NoSuchHomeChildError
+from twistedcaldav.notify import Notifier
def allInstancesOf(cls):
@@ -146,6 +150,7 @@
buildStore = theStoreBuilder.buildStore
+ at inlineCallbacks
def populateCalendarsFrom(requirements, store):
"""
Populate C{store} from C{requirements}.
@@ -159,7 +164,7 @@
for homeUID in requirements:
calendars = requirements[homeUID]
if calendars is not None:
- home = populateTxn.calendarHomeWithUID(homeUID, True)
+ home = yield populateTxn.calendarHomeWithUID(homeUID, True)
# We don't want the default calendar or inbox to appear unless it's
# explicitly listed.
try:
@@ -171,11 +176,98 @@
calendarObjNames = calendars[calendarName]
if calendarObjNames is not None:
home.createCalendarWithName(calendarName)
- calendar = home.calendarWithName(calendarName)
+ calendar = yield home.calendarWithName(calendarName)
for objectName in calendarObjNames:
objData = calendarObjNames[objectName]
calendar.createCalendarObjectWithName(
objectName, VComponent.fromString(objData)
)
- populateTxn.commit()
+ yield populateTxn.commit()
+
+def assertProvides(testCase, interface, provider):
+ """
+ Verify that C{provider} properly provides C{interface}
+
+ @type interface: L{zope.interface.Interface}
+ @type provider: C{provider}
+ """
+ try:
+ verifyObject(interface, provider)
+ except BrokenMethodImplementation, e:
+ testCase.fail(e)
+ except DoesNotImplement, e:
+ testCase.fail("%r does not provide %s.%s" %
+ (provider, interface.__module__, interface.getName()))
+
+
+
+class CommonCommonTests(object):
+ """
+ Common utility functionality for file/store combination tests.
+ """
+
+ lastTransaction = None
+ savedStore = None
+ assertProvides = assertProvides
+
+ def transactionUnderTest(self):
+ """
+ Create a transaction from C{storeUnderTest} and save it as
+ C[lastTransaction}. Also makes sure to use the same store, saving the
+ value from C{storeUnderTest}.
+ """
+ if self.lastTransaction is not None:
+ return self.lastTransaction
+ if self.savedStore is None:
+ self.savedStore = self.storeUnderTest()
+ self.counter += 1
+ txn = self.lastTransaction = self.savedStore.newTransaction(self.id() + " #" + str(self.counter))
+ return txn
+
+
+ def commit(self):
+ """
+ Commit the last transaction created from C{transactionUnderTest}, and
+ clear it.
+ """
+ result = self.lastTransaction.commit()
+ self.lastTransaction = None
+ return result
+
+
+ def abort(self):
+ """
+ Abort the last transaction created from C[transactionUnderTest}, and
+ clear it.
+ """
+ result = self.lastTransaction.abort()
+ self.lastTransaction = None
+ return result
+
+ def setUp(self):
+ self.counter = 0
+ self.notifierFactory = StubNotifierFactory()
+
+ def tearDown(self):
+ if self.lastTransaction is not None:
+ return self.commit()
+
+
+
+class StubNotifierFactory(object):
+ """
+ For testing push notifications without an XMPP server.
+ """
+
+ def __init__(self):
+ self.reset()
+
+ def newNotifier(self, label="default", id=None, prefix=None):
+ return Notifier(self, label=label, id=id, prefix=prefix)
+
+ def send(self, op, id):
+ self.history.append((op, id))
+
+ def reset(self):
+ self.history = []
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/util.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/util.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/util.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -104,13 +104,13 @@
self.log_warn(
"%s home %r already existed not migrating" % (
homeType, uid))
- sqlTxn.abort()
- fileTxn.commit()
+ yield sqlTxn.abort()
+ yield fileTxn.commit()
continue
- sqlHome = homeGetter(uid, create=True)
+ sqlHome = yield homeGetter(uid, create=True)
yield migrateFunc(fileHome, sqlHome)
- fileTxn.commit()
- sqlTxn.commit()
+ yield fileTxn.commit()
+ yield sqlTxn.commit()
# FIXME: need a public remove...HomeWithUID() for de-
# provisioning
storePath = self.fileStore._path
Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/icommondatastore.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/icommondatastore.py 2010-09-24 06:06:13 UTC (rev 6369)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/icommondatastore.py 2010-09-24 06:10:56 UTC (rev 6370)
@@ -108,6 +108,20 @@
Uh, oh.
"""
+# Indexing / sync tokens
+
+class ReservationError(LookupError):
+ """
+ Attempt to reserve a UID which is already reserved or to unreserve a UID
+ which is not reserved.
+ """
+
+class IndexedSearchException(ValueError):
+ pass
+
+class SyncTokenValidException(ValueError):
+ pass
+
#
# Interfaces
#
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/calendarserver-changes/attachments/20100923/74a81f26/attachment-0001.html>
More information about the calendarserver-changes
mailing list