[CalendarServer-changes] [12144] CalendarServer/branches/users/cdaboo/cross-pod-sharing
source_changes at macosforge.org
source_changes at macosforge.org
Wed Mar 12 11:17:29 PDT 2014
Revision: 12144
http://trac.calendarserver.org//changeset/12144
Author: cdaboo at apple.com
Date: 2013-12-19 10:28:54 -0800 (Thu, 19 Dec 2013)
Log Message:
-----------
Get rid of sql_legacy, moving search directly onto store objects - will make cross-pod search api cleaner.
Modified Paths:
--------------
CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/dbinspect.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/purge.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/calverify/accounts.xml
CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/purge/accounts.xml
CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_export.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_purge.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/syntax.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/test/test_sqlsyntax.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/opendirectorybacker.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/accounts.xml
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/augments.xml
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/test_directory.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_addressbook_query.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_calendar_query.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_common.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_multiget_common.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/storebridge.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_calendarquery.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_sharing.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_xml.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/index_file.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/scheduling/freebusy.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/sql.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_index_file.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_sql.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/index_file.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/sql.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/test/test_sql_sharing.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql.py
Added Paths:
-----------
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/__init__.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/builder.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/filter.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/generator.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/__init__.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/test_filter.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/__init__.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/builder.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/filter.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/__init__.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/test_filter.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/__init__.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/expression.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/filegenerator.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/generator.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/__init__.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_expression.py
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_generator.py
Removed Paths:
-------------
CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/query/
CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql_legacy.py
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/dbinspect.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/dbinspect.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/dbinspect.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -37,8 +37,8 @@
from twistedcaldav.datafilters.peruserdata import PerUserDataFilter
from twistedcaldav.directory import calendaruserproxy
from twistedcaldav.directory.directory import DirectoryService
-from twistedcaldav.query import calendarqueryfilter
from twistedcaldav.stdconfig import DEFAULT_CONFIG_FILE
+from txdav.caldav.datastore.query.filter import Filter
from txdav.common.datastore.sql_tables import schema, _BIND_MODE_OWN
from uuid import UUID
import os
@@ -757,10 +757,10 @@
name="VCALENDAR",
)
)
- filter = calendarqueryfilter.Filter(filter)
+ filter = Filter(filter)
filter.settimezone(None)
- matches = yield calendar._index.indexedSearch(filter, useruid=uid, fbtype=False)
+ matches = yield calendar.search(filter, useruid=uid, fbtype=False)
if matches is None:
returnValue(None)
for name, _ignore_uid, _ignore_type in matches:
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/purge.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/purge.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/purge.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -30,10 +30,9 @@
from twisted.internet.defer import inlineCallbacks, returnValue
from twistedcaldav import caldavxml
-from twistedcaldav.caldavxml import TimeRange
from twistedcaldav.directory.directory import DirectoryRecord
-from twistedcaldav.query import calendarqueryfilter
+from txdav.caldav.datastore.query.filter import Filter
from txdav.xml import element as davxml
@@ -817,13 +816,13 @@
query_filter = caldavxml.Filter(
caldavxml.ComponentFilter(
caldavxml.ComponentFilter(
- TimeRange(start=whenString,),
+ caldavxml.TimeRange(start=whenString,),
name=("VEVENT",),
),
name="VCALENDAR",
)
)
- query_filter = calendarqueryfilter.Filter(query_filter)
+ query_filter = Filter(query_filter)
count = 0
txn = self.store.newTransaction()
@@ -844,7 +843,7 @@
childNames.append(childName)
else:
# events matching filter
- for childName, _ignore_childUid, _ignore_childType in (yield calendar._index.indexedSearch(query_filter)):
+ for childName, _ignore_childUid, _ignore_childType in (yield calendar.search(query_filter)):
childNames.append(childName)
yield txn.commit()
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/calverify/accounts.xml
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/calverify/accounts.xml 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/calverify/accounts.xml 2013-12-19 18:28:54 UTC (rev 12144)
@@ -47,4 +47,11 @@
<name>Example User4</name>
<email-address>example4 at example.com</email-address>
</user>
+ <user>
+ <uid>home1</uid>
+ <guid>home1</guid>
+ <password>home1</password>
+ <name>Home 1</name>
+ <email-address>home1 at example.com</email-address>
+ </user>
</accounts>
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/purge/accounts.xml
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/purge/accounts.xml 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/purge/accounts.xml 2013-12-19 18:28:54 UTC (rev 12144)
@@ -26,4 +26,25 @@
<name>Example User</name>
<email-address>example at example.com</email-address>
</user>
+ <user>
+ <uid>example2</uid>
+ <guid>37DB0C90-4DB1-4932-BC69-3DAB66F374F5</guid>
+ <password>example2</password>
+ <name>Example User 2</name>
+ <email-address>example2 at example.com</email-address>
+ </user>
+ <user>
+ <uid>home1</uid>
+ <guid>home1</guid>
+ <password>home1</password>
+ <name>Home 1</name>
+ <email-address>home1 at example.com</email-address>
+ </user>
+ <user>
+ <uid>home2</uid>
+ <guid>home2</guid>
+ <password>home2</password>
+ <name>Home 2</name>
+ <email-address>home2 at example.com</email-address>
+ </user>
</accounts>
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_export.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_export.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_export.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -277,7 +277,7 @@
"""
yield populateCalendarsFrom(
{
- "home1": {
+ "user01": {
"calendar1": {
"valentines-day.ics": (valentines, {})
}
@@ -291,7 +291,7 @@
io = StringIO()
yield exportToFile(
- [(yield self.txn().calendarHomeWithUID("home1"))
+ [(yield self.txn().calendarHomeWithUID("user01"))
.calendarWithName("calendar1")], io
)
self.assertEquals(Component.fromString(io.getvalue()),
@@ -306,7 +306,7 @@
"""
yield populateCalendarsFrom(
{
- "home1": {
+ "user01": {
"calendar1": {
"valentines-day.ics": (valentines, {}),
"new-years-day.ics": (newYears, {})
@@ -324,7 +324,7 @@
io = StringIO()
yield exportToFile(
- [(yield self.txn().calendarHomeWithUID("home1"))
+ [(yield self.txn().calendarHomeWithUID("user01"))
.calendarWithName("calendar1")], io
)
self.assertEquals(Component.fromString(io.getvalue()),
@@ -342,7 +342,7 @@
"""
yield populateCalendarsFrom(
{
- "home1": {
+ "user01": {
"calendar1": {
"1.ics": (one, {}), # EST
"2.ics": (another, {}), # EST
@@ -354,7 +354,7 @@
io = StringIO()
yield exportToFile(
- [(yield self.txn().calendarHomeWithUID("home1"))
+ [(yield self.txn().calendarHomeWithUID("user01"))
.calendarWithName("calendar1")], io
)
result = Component.fromString(io.getvalue())
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_purge.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_purge.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_purge.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -806,16 +806,6 @@
@inlineCallbacks
def setUp(self):
- self.patch(config.DirectoryService.params, "xmlFile",
- os.path.join(
- os.path.dirname(__file__), "purge", "accounts.xml"
- )
- )
- self.patch(config.ResourceService.params, "xmlFile",
- os.path.join(
- os.path.dirname(__file__), "purge", "resources.xml"
- )
- )
yield super(PurgePrincipalTests, self).setUp()
txn = self._sqlCalendarStore.newTransaction()
@@ -850,6 +840,20 @@
(yield txn.commit())
+ def configure(self):
+ super(PurgePrincipalTests, self).configure()
+ self.patch(config.DirectoryService.params, "xmlFile",
+ os.path.join(
+ os.path.dirname(__file__), "purge", "accounts.xml"
+ )
+ )
+ self.patch(config.ResourceService.params, "xmlFile",
+ os.path.join(
+ os.path.dirname(__file__), "purge", "resources.xml"
+ )
+ )
+
+
@inlineCallbacks
def populate(self):
yield populateCalendarsFrom(self.requirements, self.storeUnderTest())
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/syntax.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/syntax.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/syntax.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -347,29 +347,63 @@
@param other: a constant parameter or sub-select
@type other: L{Parameter} or L{Select}
"""
+ return self._commonIn('in', other)
+
+
+ def NotIn(self, other):
+ """
+ We support two forms of the SQL "NOT IN" syntax: one where a list of values is supplied, the other where
+ a sub-select is used to provide a set of values.
+
+ @param other: a constant parameter or sub-select
+ @type other: L{Parameter} or L{Select}
+ """
+ return self._commonIn('not in', other)
+
+
+ def _commonIn(self, op, other):
+ """
+ We support two forms of the SQL "NOT IN" syntax: one where a list of values is supplied, the other where
+ a sub-select is used to provide a set of values.
+
+ @param other: a constant parameter or sub-select
+ @type other: L{Parameter} or L{Select}
+ """
if isinstance(other, Parameter):
if other.count is None:
- raise DALError("IN expression needs an explicit count of parameters")
- return CompoundComparison(self, 'in', Constant(other))
+ raise DALError("{} expression needs an explicit count of parameters".format(op.upper()))
+ return CompoundComparison(self, op, Constant(other))
else:
# Can't be Select.__contains__ because __contains__ gets __nonzero__
- # called on its result by the 'in' syntax.
- return CompoundComparison(self, 'in', other)
+ # called on its result by the 'not in' syntax.
+ return CompoundComparison(self, op, other)
def StartsWith(self, other):
return CompoundComparison(self, "like", CompoundComparison(Constant(other), '||', Constant('%')))
+ def NotStartsWith(self, other):
+ return CompoundComparison(self, "not like", CompoundComparison(Constant(other), '||', Constant('%')))
+
+
def EndsWith(self, other):
return CompoundComparison(self, "like", CompoundComparison(Constant('%'), '||', Constant(other)))
+ def NotEndsWith(self, other):
+ return CompoundComparison(self, "not like", CompoundComparison(Constant('%'), '||', Constant(other)))
+
+
def Contains(self, other):
return CompoundComparison(self, "like", CompoundComparison(Constant('%'), '||', CompoundComparison(Constant(other), '||', Constant('%'))))
+ def NotContains(self, other):
+ return CompoundComparison(self, "not like", CompoundComparison(Constant('%'), '||', CompoundComparison(Constant(other), '||', Constant('%'))))
+
+
class FunctionInvocation(ExpressionSyntax):
def __init__(self, function, *args):
self.function = function
@@ -555,7 +589,7 @@
"""
Create a L{Join}, representing a join between two tables.
"""
- if on is None:
+ if on is None and not type:
type = 'cross'
return Join(self, type, otherTableSyntax, on)
@@ -697,13 +731,16 @@
def subSQL(self, queryGenerator, allTables):
stmt = SQLFragment()
stmt.append(self.leftSide.subSQL(queryGenerator, allTables))
- stmt.text += ' '
- if self.type:
- stmt.text += self.type
+ if self.type == ',':
+ stmt.text += ', '
+ else:
stmt.text += ' '
- stmt.text += 'join '
+ if self.type:
+ stmt.text += self.type
+ stmt.text += ' '
+ stmt.text += 'join '
stmt.append(self.rightSide.subSQL(queryGenerator, allTables))
- if self.type != 'cross':
+ if self.type not in ('cross', ','):
stmt.text += ' on '
stmt.append(self.on.subSQL(queryGenerator, allTables))
return stmt
@@ -882,6 +919,26 @@
+class Not(Comparison):
+ """
+ A L{NotColumn} is a logical NOT of an expression.
+ """
+ def __init__(self, a):
+ # 'op' and 'b' are always None for this comparison type
+ super(Not, self).__init__(a, None, None)
+
+
+ def subSQL(self, queryGenerator, allTables):
+ sqls = SQLFragment()
+ sqls.text += "not "
+ result = self.a.subSQL(queryGenerator, allTables)
+ if isinstance(self.a, CompoundComparison) and self.a.op in ('or', 'and'):
+ result = _inParens(result)
+ sqls.append(result)
+ return sqls
+
+
+
class NullComparison(Comparison):
"""
A L{NullComparison} is a comparison of a column or expression with None.
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/test/test_sqlsyntax.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/test/test_sqlsyntax.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/test/test_sqlsyntax.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -26,7 +26,7 @@
Savepoint, RollbackToSavepoint, ReleaseSavepoint, SavepointAction,
Union, Intersect, Except, SetExpression, DALError,
ResultAliasSyntax, Count, QueryGenerator, ALL_COLUMNS,
- DatabaseLock, DatabaseUnlock)
+ DatabaseLock, DatabaseUnlock, Not)
from twext.enterprise.dal.syntax import FixedPlaceholder, NumericPlaceholder
from twext.enterprise.dal.syntax import Function
from twext.enterprise.dal.syntax import SchemaSyntax
@@ -362,6 +362,17 @@
)
+ def test_commaJoin(self):
+ """
+ A join with no clause specified will generate a cross join. (This is an
+ explicit synonym for an implicit join: i.e. 'select * from FOO, BAR'.)
+ """
+ self.assertEquals(
+ Select(From=self.schema.FOO.join(self.schema.BOZ, type=",")).toSQL(),
+ SQLFragment("select * from FOO, BOZ")
+ )
+
+
def test_crossJoin(self):
"""
A join with no clause specified will generate a cross join. (This is an
@@ -963,6 +974,73 @@
)
+ def test_not(self):
+ """
+ Test for the string starts with comparison.
+ (Note that this should be updated to use different techniques
+ as necessary in different databases.)
+ """
+ self.assertEquals(
+ Select([
+ self.schema.TEXTUAL.MYTEXT],
+ From=self.schema.TEXTUAL,
+ Where=Not(self.schema.TEXTUAL.MYTEXT.StartsWith("test")),
+ ).toSQL(),
+ SQLFragment(
+ "select MYTEXT from TEXTUAL where not MYTEXT like (? || ?)",
+ ["test", "%"]
+ )
+ )
+
+ self.assertEquals(
+ Select([
+ self.schema.TEXTUAL.MYTEXT],
+ From=self.schema.TEXTUAL,
+ Where=Not(self.schema.TEXTUAL.MYTEXT == "test"),
+ ).toSQL(),
+ SQLFragment(
+ "select MYTEXT from TEXTUAL where not MYTEXT = ?",
+ ["test"]
+ )
+ )
+
+ self.assertEquals(
+ Select([
+ self.schema.TEXTUAL.MYTEXT],
+ From=self.schema.TEXTUAL,
+ Where=Not((self.schema.TEXTUAL.MYTEXT == "test1").And(self.schema.TEXTUAL.MYTEXT != "test2")),
+ ).toSQL(),
+ SQLFragment(
+ "select MYTEXT from TEXTUAL where not (MYTEXT = ? and MYTEXT != ?)",
+ ["test1", "test2"]
+ )
+ )
+
+ self.assertEquals(
+ Select([
+ self.schema.TEXTUAL.MYTEXT],
+ From=self.schema.TEXTUAL,
+ Where=Not((self.schema.TEXTUAL.MYTEXT == "test1")).And(self.schema.TEXTUAL.MYTEXT != "test2"),
+ ).toSQL(),
+ SQLFragment(
+ "select MYTEXT from TEXTUAL where not MYTEXT = ? and MYTEXT != ?",
+ ["test1", "test2"]
+ )
+ )
+
+ self.assertEquals(
+ Select([
+ self.schema.TEXTUAL.MYTEXT],
+ From=self.schema.TEXTUAL,
+ Where=Not(self.schema.TEXTUAL.MYTEXT.StartsWith("foo").And(self.schema.TEXTUAL.MYTEXT.NotEndsWith("bar"))),
+ ).toSQL(),
+ SQLFragment(
+ "select MYTEXT from TEXTUAL where not (MYTEXT like (? || ?) and MYTEXT not like (? || ?))",
+ ["foo", "%", "%", "bar"]
+ )
+ )
+
+
def test_insert(self):
"""
L{Insert.toSQL} generates an 'insert' statement with all the relevant
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/opendirectorybacker.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/opendirectorybacker.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/opendirectorybacker.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -23,46 +23,45 @@
"OpenDirectoryBackingService", "VCardRecord",
]
-import traceback
-import hashlib
+from calendarserver.platform.darwin.od import opendirectory, dsattributes, dsquery
-import os
-import sys
-import time
-
-from os import listdir
-from os.path import join, abspath
-from tempfile import mkstemp, gettempdir
-from random import random
-
-from pycalendar.vcard.n import N
-from pycalendar.vcard.adr import Adr
from pycalendar.datetime import DateTime
+from pycalendar.vcard.adr import Adr
+from pycalendar.vcard.n import N
-from socket import getfqdn
-from twisted.internet import reactor
-from twisted.internet.defer import inlineCallbacks, returnValue, deferredGenerator, succeed
from twext.python.filepath import CachingFilePath as FilePath
-from txdav.xml import element as davxml
-from txdav.xml.base import twisted_dav_namespace, dav_namespace, parse_date, twisted_private_namespace
from twext.web2.dav.resource import DAVPropertyMixIn
from twext.web2.dav.util import joinURL
from twext.web2.http_headers import MimeType, generateContentType, ETag
+from twisted.internet import reactor
+from twisted.internet.defer import inlineCallbacks, returnValue, deferredGenerator, succeed
from twistedcaldav import customxml, carddavxml
-from twistedcaldav.customxml import calendarserver_namespace
from twistedcaldav.config import config
+from twistedcaldav.customxml import calendarserver_namespace
from twistedcaldav.directory.directory import DirectoryService, DirectoryRecord
from twistedcaldav.memcachelock import MemcacheLock, MemcacheLockTimeoutError
-from twistedcaldav.query import addressbookqueryfilter
from twistedcaldav.vcard import Component, Property, vCardProductID
+from txdav.carddav.datastore.query.filter import IsNotDefined, ParameterFilter, \
+ TextMatch
+from txdav.xml import element as davxml
+from txdav.xml.base import twisted_dav_namespace, dav_namespace, parse_date, twisted_private_namespace
+
+from os import listdir
+from os.path import join, abspath
+from random import random
+from socket import getfqdn
+from tempfile import mkstemp, gettempdir
from xmlrpclib import datetime
+import hashlib
+import os
+import sys
+import time
+import traceback
-from calendarserver.platform.darwin.od import opendirectory, dsattributes, dsquery
-
class OpenDirectoryBackingService(DirectoryService):
"""
Open Directory implementation of L{IDirectoryService}.
@@ -830,11 +829,11 @@
if not constant and not allAttrStrings:
return (False, [], [])
- if propFilter.qualifier and isinstance(propFilter.qualifier, addressbookqueryfilter.IsNotDefined):
+ if propFilter.qualifier and isinstance(propFilter.qualifier, IsNotDefined):
return definedExpression(False, filterAllOf, propFilter.filter_name, constant, queryAttributes, allAttrStrings)
- paramFilterElements = [paramFilterElement for paramFilterElement in propFilter.filters if isinstance(paramFilterElement, addressbookqueryfilter.ParameterFilter)]
- textMatchElements = [textMatchElement for textMatchElement in propFilter.filters if isinstance(textMatchElement, addressbookqueryfilter.TextMatch)]
+ paramFilterElements = [paramFilterElement for paramFilterElement in propFilter.filters if isinstance(paramFilterElement, ParameterFilter)]
+ textMatchElements = [textMatchElement for textMatchElement in propFilter.filters if isinstance(textMatchElement, TextMatch)]
propFilterAllOf = propFilter.propfilter_test == "allof"
# handle parameter filter elements
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/accounts.xml
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/accounts.xml 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/accounts.xml 2013-12-19 18:28:54 UTC (rev 12144)
@@ -139,7 +139,7 @@
<member type="users">delegateviagroup</member>
</members>
</group>
- <user repeat="2">
+ <user repeat="100">
<uid>user%02d</uid>
<guid>user%02d</guid>
<password>%02duser</password>
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/augments.xml
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/augments.xml 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/augments.xml 2013-12-19 18:28:54 UTC (rev 12144)
@@ -57,7 +57,7 @@
<enable-calendar>false</enable-calendar>
<enable-addressbook>false</enable-addressbook>
</record>
- <record repeat="2">
+ <record repeat="100">
<uid>user%02d</uid>
<enable>true</enable>
<enable-calendar>true</enable-calendar>
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/test_directory.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/test_directory.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/test_directory.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -999,8 +999,9 @@
Exercise the default recordsMatchingTokens implementation
"""
records = list((yield self.directoryService.recordsMatchingTokens(["Use", "01"])))
- self.assertEquals(len(records), 1)
- self.assertEquals(records[0].shortNames[0], "user01")
+ self.assertNotEquals(len(records), 0)
+ shorts = [record.shortNames[0] for record in records]
+ self.assertTrue("user01" in shorts)
records = list((yield self.directoryService.recordsMatchingTokens(['"quotey"'],
context=self.directoryService.searchContext_attendee)))
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_addressbook_query.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_addressbook_query.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_addressbook_query.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -27,7 +27,6 @@
from twext.python.log import Logger
from twext.web2 import responsecode
-from txdav.xml import element as davxml
from twext.web2.dav.http import ErrorResponse, MultiStatusResponse
from twext.web2.dav.method.report import NumberOfMatchesWithinLimits
from twext.web2.dav.util import joinURL
@@ -37,9 +36,11 @@
from twistedcaldav.config import config
from twistedcaldav.carddavxml import carddav_namespace, NResults
from twistedcaldav.method import report_common
-from twistedcaldav.query import addressbookqueryfilter
-from txdav.common.icommondatastore import ConcurrentModification
+from txdav.carddav.datastore.query.filter import Filter
+from txdav.common.icommondatastore import ConcurrentModification, \
+ IndexedSearchException
+from txdav.xml import element as davxml
log = Logger()
@@ -62,7 +63,7 @@
responses = []
xmlfilter = addressbook_query.filter
- filter = addressbookqueryfilter.Filter(xmlfilter)
+ filter = Filter(xmlfilter)
query = addressbook_query.props
limit = addressbook_query.limit
@@ -209,7 +210,7 @@
carddavxml.TextMatch.fromString(resource_name[:-4]),
name="UID", # attributes
), ])
- vCardFilter = addressbookqueryfilter.Filter(vCardFilter)
+ vCardFilter = Filter(vCardFilter)
directoryAddressBookLock, limited[0] = (yield directory.cacheVCardsForAddressBookQuery(vCardFilter, query, max_number_of_results[0]))
@@ -230,11 +231,13 @@
# Check for disabled access
if filteredaces is not None:
- # See whether the filter is valid for an index only query
- index_query_ok = addrresource.index().searchValid(filter)
-
- # Get list of children that match the search and have read access
- names = [name for name, ignore_uid in (yield addrresource.index().search(filter))] #@UnusedVariable
+ index_query_ok = True
+ try:
+ # Get list of children that match the search and have read access
+ names = [name for name, ignore_uid in (yield addrresource.search(filter))] #@UnusedVariable
+ except IndexedSearchException:
+ names = yield addrresource.listChildren()
+ index_query_ok = False
if not names:
return
@@ -277,7 +280,7 @@
carddavxml.TextMatch.fromString(resource_name[:-4]),
name="UID", # attributes
), ])
- vCardFilter = addressbookqueryfilter.Filter(vCardFilter)
+ vCardFilter = Filter(vCardFilter)
yield maybeDeferred(queryDirectoryBackedAddressBook, parent, vCardFilter)
handled = True
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_calendar_query.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_calendar_query.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_calendar_query.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -20,8 +20,7 @@
__all__ = ["report_urn_ietf_params_xml_ns_caldav_calendar_query"]
-from twisted.internet.defer import inlineCallbacks, returnValue, \
- maybeDeferred
+from twisted.internet.defer import inlineCallbacks, returnValue
from twext.python.log import Logger
from twext.web2 import responsecode
@@ -39,8 +38,8 @@
ConcurrentModification
from twistedcaldav.instance import TooManyInstancesError
from twistedcaldav.method import report_common
-from twistedcaldav.query import calendarqueryfilter
+from txdav.caldav.datastore.query.filter import Filter
from txdav.caldav.icalendarstore import TimeRangeLowerLimit, TimeRangeUpperLimit
from txdav.xml import element as davxml
@@ -66,7 +65,7 @@
responses = []
xmlfilter = calendar_query.filter
- filter = calendarqueryfilter.Filter(xmlfilter)
+ filter = Filter(xmlfilter)
props = calendar_query.props
assert props is not None
@@ -190,13 +189,11 @@
if filteredaces is not None:
index_query_ok = True
try:
- # Get list of children that match the search and have read
- # access
- records = yield maybeDeferred(calresource.index().indexedSearch, filter)
+ # Get list of children that match the search and have read access
+ names = [name for name, ignore_uid, ignore_type in (yield calresource.search(filter))]
except IndexedSearchException:
- records = yield maybeDeferred(calresource.index().bruteForceSearch)
+ names = yield calresource.listChildren()
index_query_ok = False
- names = [name for name, ignore_uid, ignore_type in records]
if not names:
returnValue(True)
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_common.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_common.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_common.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -36,17 +36,16 @@
except ImportError:
from md5 import new as md5
-from twisted.internet.defer import inlineCallbacks, returnValue, maybeDeferred
+from twisted.internet.defer import inlineCallbacks, returnValue
from twisted.python.failure import Failure
from twext.web2 import responsecode
-from txdav.xml import element
from twext.web2.dav.http import statusForFailure
from twext.web2.dav.method.propfind import propertyName
from twext.web2.dav.method.report import NumberOfMatchesWithinLimits
from twext.web2.dav.method.report import max_number_of_matches
from twext.web2.dav.resource import AccessDeniedError
-from twext.web2.http import HTTPError
+from twext.web2.http import HTTPError, StatusResponse
from twext.python.log import Logger
@@ -65,9 +64,9 @@
from twistedcaldav.instance import InstanceList
from twistedcaldav.memcacher import Memcacher
-from twistedcaldav.query import calendarqueryfilter
-
+from txdav.caldav.datastore.query.filter import Filter
from txdav.common.icommondatastore import IndexedSearchException
+from txdav.xml import element
from pycalendar.duration import Duration
from pycalendar.datetime import DateTime
@@ -583,17 +582,18 @@
name="VCALENDAR",
)
)
- filter = calendarqueryfilter.Filter(filter)
+ filter = Filter(filter)
tzinfo = filter.settimezone(tz)
try:
- resources = yield maybeDeferred(calresource.index().indexedSearch,
- filter, useruid=useruid, fbtype=True
- )
+ resources = yield calresource.search(filter, useruid=useruid, fbtype=True)
if caching:
yield FBCacheEntry.makeCacheEntry(calresource, useruid, cache_timerange, resources)
except IndexedSearchException:
- resources = yield maybeDeferred(calresource.index().bruteForceSearch)
+ raise HTTPError(StatusResponse(
+ responsecode.INTERNAL_SERVER_ERROR,
+ "Failed freebusy query"
+ ))
else:
# Log extended item
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_multiget_common.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_multiget_common.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_multiget_common.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -24,8 +24,6 @@
from twext.python.log import Logger
from twext.web2 import responsecode
-from txdav.xml import element as davxml
-from txdav.xml.base import dav_namespace
from twext.web2.dav.http import ErrorResponse, MultiStatusResponse
from twext.web2.dav.resource import AccessDeniedError
from twext.web2.http import HTTPError, StatusResponse
@@ -37,11 +35,14 @@
from twistedcaldav.carddavxml import carddav_namespace
from twistedcaldav.config import config
from twistedcaldav.method import report_common
-from txdav.common.icommondatastore import ConcurrentModification
from twistedcaldav.method.report_common import COLLECTION_TYPE_CALENDAR, \
COLLECTION_TYPE_ADDRESSBOOK
-from twistedcaldav.query import addressbookqueryfilter
+from txdav.carddav.datastore.query.filter import Filter
+from txdav.common.icommondatastore import ConcurrentModification
+from txdav.xml import element as davxml
+from txdav.xml.base import dav_namespace
+
log = Logger()
@inlineCallbacks
@@ -265,7 +266,7 @@
returnValue(None)
addressBookFilter = carddavxml.Filter(*vCardFilters)
- addressBookFilter = addressbookqueryfilter.Filter(addressBookFilter)
+ addressBookFilter = Filter(addressBookFilter)
if self.directory.cacheQuery:
# add vcards to directory address book and run "normal case" below
limit = config.DirectoryAddressBook.MaxQueryResults
@@ -333,11 +334,11 @@
parent = (yield child.locateParent(request, resource_uri))
if collection_type == COLLECTION_TYPE_CALENDAR:
- if not parent.isCalendarCollection() or not (yield parent.index().resourceExists(name)):
+ if not parent.isCalendarCollection() or not (yield parent.resourceExists(name)):
responses.append(davxml.StatusResponse(href, davxml.Status.fromResponseCode(responsecode.FORBIDDEN)))
continue
elif collection_type == COLLECTION_TYPE_ADDRESSBOOK:
- if not parent.isAddressBookCollection() or not (yield parent.index().resourceExists(name)):
+ if not parent.isAddressBookCollection() or not (yield parent.resourceExists(name)):
responses.append(davxml.StatusResponse(href, davxml.Status.fromResponseCode(responsecode.FORBIDDEN)))
continue
@@ -367,11 +368,11 @@
parent = (yield self.locateParent(request, resource_uri))
if collection_type == COLLECTION_TYPE_CALENDAR:
- if not parent.isPseudoCalendarCollection() or not (yield parent.index().resourceExists(name)):
+ if not parent.isPseudoCalendarCollection() or not (yield parent.resourceExists(name)):
responses.append(davxml.StatusResponse(href, davxml.Status.fromResponseCode(responsecode.FORBIDDEN)))
continue
elif collection_type == COLLECTION_TYPE_ADDRESSBOOK:
- if not parent.isAddressBookCollection() or not (yield parent.index().resourceExists(name)):
+ if not parent.isAddressBookCollection() or not (yield parent.resourceExists(name)):
responses.append(davxml.StatusResponse(href, davxml.Status.fromResponseCode(responsecode.FORBIDDEN)))
continue
child = self
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/storebridge.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/storebridge.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/storebridge.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -286,13 +286,6 @@
return self._parentResource
- def index(self):
- """
- Retrieve the new-style index wrapper.
- """
- return self._newStoreObject.retrieveOldIndex()
-
-
def exists(self):
# FIXME: tests
return self._newStoreObject is not None
@@ -347,6 +340,18 @@
return self._newStoreObject.countObjectResources()
+ @inlineCallbacks
+ def resourceExists(self, name):
+ """
+ Indicate whether a resource with the specified name exists.
+
+ @return: C{True} if it exists
+ @rtype: C{bool}
+ """
+ allNames = yield self._newStoreObject.listObjectResources()
+ returnValue(name in allNames)
+
+
def name(self):
return self._name
@@ -970,6 +975,10 @@
)
+ def search(self, filter, **kwargs):
+ return self._newStoreObject.search(filter, **kwargs)
+
+
def notifierID(self):
return "%s/%s" % self._newStoreObject.notifierID()
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_calendarquery.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_calendarquery.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_calendarquery.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -27,7 +27,6 @@
from twistedcaldav import caldavxml
from twistedcaldav import ical
-from twistedcaldav.query import calendarqueryfilter
from twistedcaldav.config import config
from twistedcaldav.test.util import StoreTestCase, SimpleStoreRequest
from twisted.internet.defer import inlineCallbacks, returnValue
@@ -36,6 +35,7 @@
from twistedcaldav.ical import Component
from txdav.caldav.icalendarstore import ComponentUpdateState
from twistedcaldav.directory.directory import DirectoryService
+from txdav.caldav.datastore.query.filter import TimeRange
@inlineCallbacks
@@ -167,7 +167,7 @@
cal = property.calendar()
instances = cal.expandTimeRanges(query_timerange.end)
vevents = [x for x in cal.subcomponents() if x.name() == "VEVENT"]
- if not calendarqueryfilter.TimeRange(query_timerange).matchinstance(vevents[0], instances):
+ if not TimeRange(query_timerange).matchinstance(vevents[0], instances):
self.fail("REPORT property %r returned calendar %s outside of request time range %r"
% (property, property.calendar, query_timerange))
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_sharing.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_sharing.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_sharing.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -30,6 +30,7 @@
from twistedcaldav.test.test_cache import StubResponseCacheResource
from twistedcaldav.test.util import norequest, StoreTestCase, SimpleStoreRequest
+from txdav.caldav.datastore.test.util import buildDirectory
from txdav.common.datastore.sql_tables import _BIND_MODE_DIRECT
from txdav.xml import element as davxml
from txdav.xml.parser import WebDAVDocument
@@ -738,6 +739,8 @@
home is at /. Return the name of the newly shared calendar in the
sharee's home.
"""
+
+ self._sqlCalendarStore._directoryService = buildDirectory(homes=("wiki-testing",))
wcreate = self._sqlCalendarStore.newTransaction("create wiki")
yield wcreate.calendarHomeWithUID("wiki-testing", create=True)
yield wcreate.commit()
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_xml.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_xml.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_xml.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -17,12 +17,15 @@
import os
from twisted.trial.unittest import SkipTest
+
from twistedcaldav.ical import Component
-from twistedcaldav.query import calendarqueryfilter
import twistedcaldav.test.util
from twistedcaldav.caldavxml import ComponentFilter, PropertyFilter, TextMatch, \
Filter, TimeRange
+from txdav.caldav.datastore.query.filter import Filter as storeFilter
+from txdav.caldav.datastore.query.filter import ComponentFilter as storeComponentFilter
+
class XML (twistedcaldav.test.util.TestCase):
"""
XML tests
@@ -46,7 +49,7 @@
else:
no = ""
- if has != calendarqueryfilter.ComponentFilter(
+ if has != storeComponentFilter(
ComponentFilter(
ComponentFilter(
name=component_name
@@ -70,7 +73,7 @@
else:
no = ""
- if has != calendarqueryfilter.ComponentFilter(
+ if has != storeComponentFilter(
ComponentFilter(
ComponentFilter(
PropertyFilter(
@@ -106,7 +109,7 @@
else:
no = ""
- if has != calendarqueryfilter.ComponentFilter(
+ if has != storeComponentFilter(
ComponentFilter(
ComponentFilter(
PropertyFilter(
@@ -148,7 +151,7 @@
else:
no = ""
- if has != calendarqueryfilter.Filter(
+ if has != storeFilter(
Filter(
ComponentFilter(
ComponentFilter(
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/index_file.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/index_file.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/index_file.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -15,6 +15,7 @@
# limitations under the License.
##
+
"""
CalDAV Index.
@@ -43,12 +44,14 @@
from twext.python.log import Logger
+from txdav.caldav.datastore.query.builder import buildExpression
+from txdav.caldav.datastore.query.filter import Filter
+from txdav.common.datastore.query.filegenerator import sqllitegenerator
from txdav.common.icommondatastore import SyncTokenValidException, \
ReservationError, IndexedSearchException
from twistedcaldav.dateops import pyCalendarTodatetime
from twistedcaldav.ical import Component
-from twistedcaldav.query import calendarquery, calendarqueryfilter
from twistedcaldav.sql import AbstractSQLDatabase
from twistedcaldav.sql import db_prefix
from twistedcaldav.instance import InvalidOverriddenInstanceError
@@ -320,7 +323,7 @@
# Make sure we have a proper Filter element and get the partial SQL
# statement to use.
- if isinstance(filter, calendarqueryfilter.Filter):
+ if isinstance(filter, Filter):
if fbtype:
# Lookup the useruid - try the empty (default) one if needed
dbuseruid = self._db_value_for_sql(
@@ -330,7 +333,7 @@
else:
dbuseruid = ""
- qualifiers = calendarquery.sqlcalendarquery(filter, None, dbuseruid, fbtype)
+ qualifiers = sqlcalendarquery(filter, None, dbuseruid, fbtype)
if qualifiers is not None:
# Determine how far we need to extend the current expansion of
# events. If we have an open-ended time-range we will expand one
@@ -437,6 +440,24 @@
+def sqlcalendarquery(filter, calendarid=None, userid=None, freebusy=False):
+ """
+ Convert the supplied calendar-query into a partial SQL statement.
+
+ @param filter: the L{Filter} for the calendar-query to convert.
+ @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the partial SQL statement,
+ and the C{list} is the list of argument substitutions to use with the SQL API execute method.
+ Or return C{None} if it is not possible to create an SQL query to fully match the calendar-query.
+ """
+ try:
+ expression = buildExpression(filter, sqllitegenerator.FIELDS)
+ sql = sqllitegenerator(expression, calendarid, userid, freebusy)
+ return sql.generate()
+ except ValueError:
+ return None
+
+
+
class CalendarIndex (AbstractCalendarIndex):
"""
Calendar index - abstract class for indexer that indexes calendar objects in a collection.
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/__init__.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/__init__.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/builder.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/builder.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/builder.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,227 @@
+##
+# Copyright (c) 2006-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twistedcaldav.dateops import floatoffset, pyCalendarTodatetime
+
+from txdav.caldav.datastore.query.filter import ComponentFilter, PropertyFilter, TextMatch, TimeRange
+from txdav.common.datastore.query import expression
+
+
+"""
+SQL statement generator from query expressions.
+"""
+
+__all__ = [
+ "buildExpression",
+]
+
+
+
+# SQL Index column (field) names
+
+def buildExpression(filter, fields):
+ """
+ Convert the supplied calendar-query into an expression tree.
+
+ @param filter: the L{Filter} for the calendar-query to convert.
+ @return: a L{baseExpression} for the expression tree.
+ """
+
+ # Lets assume we have a valid filter from the outset.
+
+ # Top-level filter contains exactly one comp-filter element
+ assert filter.child is not None
+ vcalfilter = filter.child
+ assert isinstance(vcalfilter, ComponentFilter)
+ assert vcalfilter.filter_name == "VCALENDAR"
+
+ if len(vcalfilter.filters) > 0:
+ # Determine logical expression grouping
+ logical = expression.andExpression if vcalfilter.filter_test == "allof" else expression.orExpression
+
+ # Only comp-filters are handled
+ for _ignore in [x for x in vcalfilter.filters if not isinstance(x, ComponentFilter)]:
+ raise ValueError
+
+ return compfilterListExpression(vcalfilter.filters, fields, logical)
+ else:
+ return expression.allExpression()
+
+
+
+def compfilterListExpression(compfilters, fields, logical):
+ """
+ Create an expression for a list of comp-filter elements.
+
+ @param compfilters: the C{list} of L{ComponentFilter} elements.
+ @return: a L{baseExpression} for the expression tree.
+ """
+
+ if len(compfilters) == 1:
+ return compfilterExpression(compfilters[0], fields)
+ else:
+ return logical([compfilterExpression(c, fields) for c in compfilters])
+
+
+
+def compfilterExpression(compfilter, fields):
+ """
+ Create an expression for a single comp-filter element.
+
+ @param compfilter: the L{ComponentFilter} element.
+ @return: a L{baseExpression} for the expression tree.
+ """
+
+ # Handle is-not-defined case
+ if not compfilter.defined:
+ # Test for TYPE != <<component-type name>>
+ return expression.isnotExpression(fields["TYPE"], compfilter.filter_name, True)
+
+ # Determine logical expression grouping
+ logical = expression.andExpression if compfilter.filter_test == "allof" else expression.orExpression
+
+ expressions = []
+ if isinstance(compfilter.filter_name, str):
+ expressions.append(expression.isExpression(fields["TYPE"], compfilter.filter_name, True))
+ else:
+ expressions.append(expression.inExpression(fields["TYPE"], compfilter.filter_name, True))
+
+ # Handle time-range
+ if compfilter.qualifier and isinstance(compfilter.qualifier, TimeRange):
+ start, end, startfloat, endfloat = getTimerangeArguments(compfilter.qualifier)
+ expressions.append(expression.timerangeExpression(start, end, startfloat, endfloat))
+
+ # Handle properties - we can only do UID right now
+ props = []
+ for p in [x for x in compfilter.filters if isinstance(x, PropertyFilter)]:
+ props.append(propfilterExpression(p, fields))
+ if len(props) > 1:
+ propsExpression = logical(props)
+ elif len(props) == 1:
+ propsExpression = props[0]
+ else:
+ propsExpression = None
+
+ # Handle embedded components - we do not right now as our Index does not handle them
+ comps = []
+ for _ignore in [x for x in compfilter.filters if isinstance(x, ComponentFilter)]:
+ raise ValueError
+ if len(comps) > 1:
+ compsExpression = logical(comps)
+ elif len(comps) == 1:
+ compsExpression = comps[0]
+ else:
+ compsExpression = None
+
+ # Now build compound expression
+ if ((propsExpression is not None) and (compsExpression is not None)):
+ expressions.append(logical([propsExpression, compsExpression]))
+ elif propsExpression is not None:
+ expressions.append(propsExpression)
+ elif compsExpression is not None:
+ expressions.append(compsExpression)
+
+ # Now build return expression
+ return expression.andExpression(expressions)
+
+
+
+def propfilterExpression(propfilter, fields):
+ """
+ Create an expression for a single prop-filter element.
+
+ @param propfilter: the L{PropertyFilter} element.
+ @return: a L{baseExpression} for the expression tree.
+ """
+
+ # Only handle UID right now
+ if propfilter.filter_name != "UID":
+ raise ValueError
+
+ # Handle is-not-defined case
+ if not propfilter.defined:
+ # Test for <<field>> != "*"
+ return expression.isExpression(fields["UID"], "", True)
+
+ # Determine logical expression grouping
+ logical = expression.andExpression if propfilter.filter_test == "allof" else expression.orExpression
+
+ # Handle time-range - we cannot do this with our Index right now
+ if propfilter.qualifier and isinstance(propfilter.qualifier, TimeRange):
+ raise ValueError
+
+ # Handle text-match
+ tm = None
+ if propfilter.qualifier and isinstance(propfilter.qualifier, TextMatch):
+ if propfilter.qualifier.match_type == "equals":
+ tm = expression.isnotExpression if propfilter.qualifier.negate else expression.isExpression
+ elif propfilter.qualifier.match_type == "contains":
+ tm = expression.notcontainsExpression if propfilter.qualifier.negate else expression.containsExpression
+ elif propfilter.qualifier.match_type == "starts-with":
+ tm = expression.notstartswithExpression if propfilter.qualifier.negate else expression.startswithExpression
+ elif propfilter.qualifier.match_type == "ends-with":
+ tm = expression.notendswithExpression if propfilter.qualifier.negate else expression.endswithExpression
+ tm = tm(fields[propfilter.filter_name], propfilter.qualifier.text, propfilter.qualifier.caseless)
+
+ # Handle embedded parameters - we do not right now as our Index does not handle them
+ params = []
+ for _ignore in propfilter.filters:
+ raise ValueError
+ if len(params) > 1:
+ paramsExpression = logical(params)
+ elif len(params) == 1:
+ paramsExpression = params[0]
+ else:
+ paramsExpression = None
+
+ # Now build return expression
+ if (tm is not None) and (paramsExpression is not None):
+ return logical([tm, paramsExpression])
+ elif tm is not None:
+ return tm
+ elif paramsExpression is not None:
+ return paramsExpression
+ else:
+ return None
+
+
+
+def getTimerangeArguments(timerange):
+ """
+ Get start/end and floating start/end (adjusted for timezone offset) values from the
+ supplied time-range test.
+
+ @param timerange: the L{TimeRange} used in the query.
+ @return: C{tuple} of C{str} for start, end, startfloat, endfloat
+ """
+
+ # Start/end in UTC
+ start = timerange.start
+ end = timerange.end
+
+ # Get timezone
+ tzinfo = timerange.tzinfo
+
+ # Now force to floating UTC
+ startfloat = floatoffset(start, tzinfo) if start else None
+ endfloat = floatoffset(end, tzinfo) if end else None
+
+ return (
+ pyCalendarTodatetime(start) if start else None,
+ pyCalendarTodatetime(end) if end else None,
+ pyCalendarTodatetime(startfloat) if startfloat else None,
+ pyCalendarTodatetime(endfloat) if endfloat else None,
+ )
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/filter.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/filter.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/filter.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,754 @@
+##
+# Copyright (c) 2011-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+Object model of CALDAV:filter element used in an addressbook-query.
+"""
+
+__all__ = [
+ "Filter",
+]
+
+from twext.python.log import Logger
+
+from twistedcaldav.caldavxml import caldav_namespace, CalDAVTimeZoneElement
+from twistedcaldav.dateops import timeRangesOverlap
+from twistedcaldav.ical import Component, Property
+
+from pycalendar.datetime import DateTime
+from pycalendar.timezone import Timezone
+
+log = Logger()
+
+
+class FilterBase(object):
+ """
+ Determines which matching components are returned.
+ """
+
+ def __init__(self, xml_element):
+ self.xmlelement = xml_element
+
+
+ def match(self, item, access=None):
+ raise NotImplementedError
+
+
+ def valid(self, level=0):
+ raise NotImplementedError
+
+
+
+class Filter(FilterBase):
+ """
+ Determines which matching components are returned.
+ """
+
+ def __init__(self, xml_element):
+
+ super(Filter, self).__init__(xml_element)
+
+ # One comp-filter element must be present
+ if len(xml_element.children) != 1 or xml_element.children[0].qname() != (caldav_namespace, "comp-filter"):
+ raise ValueError("Invalid CALDAV:filter element: %s" % (xml_element,))
+
+ self.child = ComponentFilter(xml_element.children[0])
+
+
+ def match(self, component, access=None):
+ """
+ Returns True if the given calendar component matches this filter, False
+ otherwise.
+ """
+
+ # We only care about certain access restrictions.
+ if access not in (Component.ACCESS_CONFIDENTIAL, Component.ACCESS_RESTRICTED):
+ access = None
+
+ # We need to prepare ourselves for a time-range query by pre-calculating
+ # the set of instances up to the latest time-range limit. That way we can
+ # avoid having to do some form of recurrence expansion for each query sub-part.
+ maxend, isStartTime = self.getmaxtimerange()
+ if maxend:
+ if isStartTime:
+ if component.isRecurringUnbounded():
+ # Unbounded recurrence is always within a start-only time-range
+ instances = None
+ else:
+ # Expand the instances up to infinity
+ instances = component.expandTimeRanges(DateTime(2100, 1, 1, 0, 0, 0, tzid=Timezone(utc=True)), ignoreInvalidInstances=True)
+ else:
+ instances = component.expandTimeRanges(maxend, ignoreInvalidInstances=True)
+ else:
+ instances = None
+ self.child.setInstances(instances)
+
+ # <filter> contains exactly one <comp-filter>
+ return self.child.match(component, access)
+
+
+ def valid(self):
+ """
+ Indicate whether this filter element's structure is valid wrt iCalendar
+ data object model.
+
+ @return: True if valid, False otherwise
+ """
+
+ # Must have one child element for VCALENDAR
+ return self.child.valid(0)
+
+
+ def settimezone(self, tzelement):
+ """
+ Set the default timezone to use with this query.
+ @param calendar: a L{Component} for the VCALENDAR containing the one
+ VTIMEZONE that we want
+ @return: the L{Timezone} derived from the VTIMEZONE or utc.
+ """
+
+ if tzelement is None:
+ tz = None
+ elif isinstance(tzelement, CalDAVTimeZoneElement):
+ tz = tzelement.gettimezone()
+ elif isinstance(tzelement, Component):
+ tz = tzelement.gettimezone()
+ if tz is None:
+ tz = Timezone(utc=True)
+ self.child.settzinfo(tz)
+ return tz
+
+
+ def getmaxtimerange(self):
+ """
+ Get the date farthest into the future in any time-range elements
+ """
+
+ return self.child.getmaxtimerange(None, False)
+
+
+ def getmintimerange(self):
+ """
+ Get the date farthest into the past in any time-range elements. That is either
+ the start date, or if start is not present, the end date.
+ """
+
+ return self.child.getmintimerange(None, False)
+
+
+
+class FilterChildBase(FilterBase):
+ """
+ CalDAV filter element.
+ """
+
+ def __init__(self, xml_element):
+
+ super(FilterChildBase, self).__init__(xml_element)
+
+ qualifier = None
+ filters = []
+
+ for child in xml_element.children:
+ qname = child.qname()
+
+ if qname in (
+ (caldav_namespace, "is-not-defined"),
+ (caldav_namespace, "time-range"),
+ (caldav_namespace, "text-match"),
+ ):
+ if qualifier is not None:
+ raise ValueError("Only one of CalDAV:time-range, CalDAV:text-match allowed")
+
+ if qname == (caldav_namespace, "is-not-defined"):
+ qualifier = IsNotDefined(child)
+ elif qname == (caldav_namespace, "time-range"):
+ qualifier = TimeRange(child)
+ elif qname == (caldav_namespace, "text-match"):
+ qualifier = TextMatch(child)
+
+ elif qname == (caldav_namespace, "comp-filter"):
+ filters.append(ComponentFilter(child))
+ elif qname == (caldav_namespace, "prop-filter"):
+ filters.append(PropertyFilter(child))
+ elif qname == (caldav_namespace, "param-filter"):
+ filters.append(ParameterFilter(child))
+ else:
+ raise ValueError("Unknown child element: %s" % (qname,))
+
+ if qualifier and isinstance(qualifier, IsNotDefined) and (len(filters) != 0):
+ raise ValueError("No other tests allowed when CalDAV:is-not-defined is present")
+
+ self.qualifier = qualifier
+ self.filters = filters
+ self.filter_name = xml_element.attributes["name"]
+ if isinstance(self.filter_name, unicode):
+ self.filter_name = self.filter_name.encode("utf-8")
+ self.defined = not self.qualifier or not isinstance(qualifier, IsNotDefined)
+
+ filter_test = xml_element.attributes.get("test", "allof")
+ if filter_test not in ("anyof", "allof"):
+ raise ValueError("Test must be only one of anyof, allof")
+ self.filter_test = filter_test
+
+
+ def match(self, item, access=None):
+ """
+ Returns True if the given calendar item (either a component, property or parameter value)
+ matches this filter, False otherwise.
+ """
+
+ # Always return True for the is-not-defined case as the result of this will
+ # be negated by the caller
+ if not self.defined:
+ return True
+
+ if self.qualifier and not self.qualifier.match(item, access):
+ return False
+
+ if len(self.filters) > 0:
+ allof = self.filter_test == "allof"
+ for filter in self.filters:
+ if allof != filter._match(item, access):
+ return not allof
+ return allof
+ else:
+ return True
+
+
+
+class ComponentFilter (FilterChildBase):
+ """
+ Limits a search to only the chosen component types.
+ """
+
+ def match(self, item, access):
+ """
+ Returns True if the given calendar item (which is a component)
+ matches this filter, False otherwise.
+ This specialization uses the instance matching option of the time-range filter
+ to minimize instance expansion.
+ """
+
+ # Always return True for the is-not-defined case as the result of this will
+ # be negated by the caller
+ if not self.defined:
+ return True
+
+ if self.qualifier and not self.qualifier.matchinstance(item, self.instances):
+ return False
+
+ if len(self.filters) > 0:
+ allof = self.filter_test == "allof"
+ for filter in self.filters:
+ if allof != filter._match(item, access):
+ return not allof
+ return allof
+ else:
+ return True
+
+
+ def _match(self, component, access):
+ # At least one subcomponent must match (or is-not-defined is set)
+ for subcomponent in component.subcomponents():
+ # If access restrictions are in force, restrict matching to specific components only.
+ # In particular do not match VALARM.
+ if access and subcomponent.name() not in ("VEVENT", "VTODO", "VJOURNAL", "VFREEBUSY", "VTIMEZONE",):
+ continue
+
+ # Try to match the component name
+ if isinstance(self.filter_name, str):
+ if subcomponent.name() != self.filter_name:
+ continue
+ else:
+ if subcomponent.name() not in self.filter_name:
+ continue
+ if self.match(subcomponent, access):
+ break
+ else:
+ return not self.defined
+ return self.defined
+
+
+ def setInstances(self, instances):
+ """
+ Give the list of instances to each comp-filter element.
+ @param instances: the list of instances.
+ """
+ self.instances = instances
+ for compfilter in [x for x in self.filters if isinstance(x, ComponentFilter)]:
+ compfilter.setInstances(instances)
+
+
+ def valid(self, level):
+ """
+ Indicate whether this filter element's structure is valid wrt iCalendar
+ data object model.
+
+ @param level: the nesting level of this filter element, 0 being the top comp-filter.
+ @return: True if valid, False otherwise
+ """
+
+ # Check for time-range
+ timerange = self.qualifier and isinstance(self.qualifier, TimeRange)
+
+ if level == 0:
+ # Must have VCALENDAR at the top
+ if (self.filter_name != "VCALENDAR") or timerange:
+ log.info("Top-level comp-filter must be VCALENDAR, instead: %s" % (self.filter_name,))
+ return False
+ elif level == 1:
+ # Disallow VCALENDAR, VALARM, STANDARD, DAYLIGHT, AVAILABLE at the top, everything else is OK
+ if self.filter_name in ("VCALENDAR", "VALARM", "STANDARD", "DAYLIGHT", "AVAILABLE"):
+ log.info("comp-filter wrong component type: %s" % (self.filter_name,))
+ return False
+
+ # time-range only on VEVENT, VTODO, VJOURNAL, VFREEBUSY, VAVAILABILITY
+ if timerange and self.filter_name not in ("VEVENT", "VTODO", "VJOURNAL", "VFREEBUSY", "VAVAILABILITY"):
+ log.info("time-range cannot be used with component %s" % (self.filter_name,))
+ return False
+ elif level == 2:
+ # Disallow VCALENDAR, VTIMEZONE, VEVENT, VTODO, VJOURNAL, VFREEBUSY, VAVAILABILITY at the top, everything else is OK
+ if (self.filter_name in ("VCALENDAR", "VTIMEZONE", "VEVENT", "VTODO", "VJOURNAL", "VFREEBUSY", "VAVAILABILITY")):
+ log.info("comp-filter wrong sub-component type: %s" % (self.filter_name,))
+ return False
+
+ # time-range only on VALARM, AVAILABLE
+ if timerange and self.filter_name not in ("VALARM", "AVAILABLE",):
+ log.info("time-range cannot be used with sub-component %s" % (self.filter_name,))
+ return False
+ else:
+ # Disallow all standard iCal components anywhere else
+ if (self.filter_name in ("VCALENDAR", "VTIMEZONE", "VEVENT", "VTODO", "VJOURNAL", "VFREEBUSY", "VALARM", "STANDARD", "DAYLIGHT", "AVAILABLE")) or timerange:
+ log.info("comp-filter wrong standard component type: %s" % (self.filter_name,))
+ return False
+
+ # Test each property
+ for propfilter in [x for x in self.filters if isinstance(x, PropertyFilter)]:
+ if not propfilter.valid():
+ return False
+
+ # Test each component
+ for compfilter in [x for x in self.filters if isinstance(x, ComponentFilter)]:
+ if not compfilter.valid(level + 1):
+ return False
+
+ # Test the time-range
+ if timerange:
+ if not self.qualifier.valid():
+ return False
+
+ return True
+
+
+ def settzinfo(self, tzinfo):
+ """
+ Set the default timezone to use with this query.
+ @param tzinfo: a L{Timezone} to use.
+ """
+
+ # Give tzinfo to any TimeRange we have
+ if isinstance(self.qualifier, TimeRange):
+ self.qualifier.settzinfo(tzinfo)
+
+ # Pass down to sub components/properties
+ for x in self.filters:
+ x.settzinfo(tzinfo)
+
+
+ def getmaxtimerange(self, currentMaximum, currentIsStartTime):
+ """
+ Get the date farthest into the future in any time-range elements
+
+ @param currentMaximum: current future value to compare with
+ @type currentMaximum: L{DateTime}
+ """
+
+ # Give tzinfo to any TimeRange we have
+ isStartTime = False
+ if isinstance(self.qualifier, TimeRange):
+ isStartTime = self.qualifier.end is None
+ compareWith = self.qualifier.start if isStartTime else self.qualifier.end
+ if currentMaximum is None or currentMaximum < compareWith:
+ currentMaximum = compareWith
+ currentIsStartTime = isStartTime
+
+ # Pass down to sub components/properties
+ for x in self.filters:
+ currentMaximum, currentIsStartTime = x.getmaxtimerange(currentMaximum, currentIsStartTime)
+
+ return currentMaximum, currentIsStartTime
+
+
+ def getmintimerange(self, currentMinimum, currentIsEndTime):
+ """
+ Get the date farthest into the past in any time-range elements. That is either
+ the start date, or if start is not present, the end date.
+ """
+
+ # Give tzinfo to any TimeRange we have
+ isEndTime = False
+ if isinstance(self.qualifier, TimeRange):
+ isEndTime = self.qualifier.start is None
+ compareWith = self.qualifier.end if isEndTime else self.qualifier.start
+ if currentMinimum is None or currentMinimum > compareWith:
+ currentMinimum = compareWith
+ currentIsEndTime = isEndTime
+
+ # Pass down to sub components/properties
+ for x in self.filters:
+ currentMinimum, currentIsEndTime = x.getmintimerange(currentMinimum, currentIsEndTime)
+
+ return currentMinimum, currentIsEndTime
+
+
+
+class PropertyFilter (FilterChildBase):
+ """
+ Limits a search to specific properties.
+ """
+
+ def _match(self, component, access):
+ # When access restriction is in force, we need to only allow matches against the properties
+ # allowed by the access restriction level.
+ if access:
+ allowedProperties = Component.confidentialPropertiesMap.get(component.name(), None)
+ if allowedProperties and access == Component.ACCESS_RESTRICTED:
+ allowedProperties += Component.extraRestrictedProperties
+ else:
+ allowedProperties = None
+
+ # At least one property must match (or is-not-defined is set)
+ for property in component.properties():
+ # Apply access restrictions, if any.
+ if allowedProperties is not None and property.name().upper() not in allowedProperties:
+ continue
+ if property.name().upper() == self.filter_name.upper() and self.match(property, access):
+ break
+ else:
+ return not self.defined
+ return self.defined
+
+
+ def valid(self):
+ """
+ Indicate whether this filter element's structure is valid wrt iCalendar
+ data object model.
+
+ @return: True if valid, False otherwise
+ """
+
+ # Check for time-range
+ timerange = self.qualifier and isinstance(self.qualifier, TimeRange)
+
+ # time-range only on COMPLETED, CREATED, DTSTAMP, LAST-MODIFIED
+ if timerange and self.filter_name.upper() not in ("COMPLETED", "CREATED", "DTSTAMP", "LAST-MODIFIED"):
+ log.info("time-range cannot be used with property %s" % (self.filter_name,))
+ return False
+
+ # Test the time-range
+ if timerange:
+ if not self.qualifier.valid():
+ return False
+
+ # No other tests
+ return True
+
+
+ def settzinfo(self, tzinfo):
+ """
+ Set the default timezone to use with this query.
+ @param tzinfo: a L{Timezone} to use.
+ """
+
+ # Give tzinfo to any TimeRange we have
+ if isinstance(self.qualifier, TimeRange):
+ self.qualifier.settzinfo(tzinfo)
+
+
+ def getmaxtimerange(self, currentMaximum, currentIsStartTime):
+ """
+ Get the date farthest into the future in any time-range elements
+
+ @param currentMaximum: current future value to compare with
+ @type currentMaximum: L{DateTime}
+ """
+
+ # Give tzinfo to any TimeRange we have
+ isStartTime = False
+ if isinstance(self.qualifier, TimeRange):
+ isStartTime = self.qualifier.end is None
+ compareWith = self.qualifier.start if isStartTime else self.qualifier.end
+ if currentMaximum is None or currentMaximum < compareWith:
+ currentMaximum = compareWith
+ currentIsStartTime = isStartTime
+
+ return currentMaximum, currentIsStartTime
+
+
+ def getmintimerange(self, currentMinimum, currentIsEndTime):
+ """
+ Get the date farthest into the past in any time-range elements. That is either
+ the start date, or if start is not present, the end date.
+ """
+
+ # Give tzinfo to any TimeRange we have
+ isEndTime = False
+ if isinstance(self.qualifier, TimeRange):
+ isEndTime = self.qualifier.start is None
+ compareWith = self.qualifier.end if isEndTime else self.qualifier.start
+ if currentMinimum is None or currentMinimum > compareWith:
+ currentMinimum = compareWith
+ currentIsEndTime = isEndTime
+
+ return currentMinimum, currentIsEndTime
+
+
+
+class ParameterFilter (FilterChildBase):
+ """
+ Limits a search to specific parameters.
+ """
+
+ def _match(self, property, access):
+
+ # At least one parameter must match (or is-not-defined is set)
+ result = not self.defined
+ for parameterName in property.parameterNames():
+ if parameterName.upper() == self.filter_name.upper() and self.match([property.parameterValue(parameterName)], access):
+ result = self.defined
+ break
+
+ return result
+
+
+
+class IsNotDefined (FilterBase):
+ """
+ Specifies that the named iCalendar item does not exist.
+ """
+
+ def match(self, component, access=None):
+ # Oddly, this needs always to return True so that it appears there is
+ # a match - but we then "negate" the result if is-not-defined is set.
+ # Actually this method should never be called as we special case the
+ # is-not-defined option.
+ return True
+
+
+
+class TextMatch (FilterBase):
+ """
+ Specifies a substring match on a property or parameter value.
+ (CalDAV-access-09, section 9.6.4)
+ """
+ def __init__(self, xml_element):
+
+ super(TextMatch, self).__init__(xml_element)
+
+ self.text = str(xml_element)
+ if "caseless" in xml_element.attributes:
+ caseless = xml_element.attributes["caseless"]
+ if caseless == "yes":
+ self.caseless = True
+ elif caseless == "no":
+ self.caseless = False
+ else:
+ self.caseless = True
+
+ if "negate-condition" in xml_element.attributes:
+ negate = xml_element.attributes["negate-condition"]
+ if negate == "yes":
+ self.negate = True
+ elif negate == "no":
+ self.negate = False
+ else:
+ self.negate = False
+
+ if "match-type" in xml_element.attributes:
+ self.match_type = xml_element.attributes["match-type"]
+ if self.match_type not in (
+ "equals",
+ "contains",
+ "starts-with",
+ "ends-with",
+ ):
+ self.match_type = "contains"
+ else:
+ self.match_type = "contains"
+
+
+ def match(self, item, access):
+ """
+ Match the text for the item.
+ If the item is a property, then match the property value,
+ otherwise it may be a list of parameter values - try to match anyone of those
+ """
+ if item is None:
+ return False
+
+ if isinstance(item, Property):
+ values = [item.strvalue()]
+ else:
+ values = item
+
+ test = unicode(self.text, "utf-8")
+ if self.caseless:
+ test = test.lower()
+
+ def _textCompare(s):
+ if self.caseless:
+ s = s.lower()
+
+ if self.match_type == "equals":
+ return s == test
+ elif self.match_type == "contains":
+ return s.find(test) != -1
+ elif self.match_type == "starts-with":
+ return s.startswith(test)
+ elif self.match_type == "ends-with":
+ return s.endswith(test)
+ else:
+ return False
+
+ for value in values:
+ # NB Its possible that we have a text list value which appears as a Python list,
+ # so we need to check for that and iterate over the list.
+ if isinstance(value, list):
+ for subvalue in value:
+ if _textCompare(unicode(subvalue, "utf-8")):
+ return not self.negate
+ else:
+ if _textCompare(unicode(value, "utf-8")):
+ return not self.negate
+
+ return self.negate
+
+
+
+class TimeRange (FilterBase):
+ """
+ Specifies a time for testing components against.
+ """
+
+ def __init__(self, xml_element):
+
+ super(TimeRange, self).__init__(xml_element)
+
+ # One of start or end must be present
+ if "start" not in xml_element.attributes and "end" not in xml_element.attributes:
+ raise ValueError("One of 'start' or 'end' must be present in CALDAV:time-range")
+
+ self.start = DateTime.parseText(xml_element.attributes["start"]) if "start" in xml_element.attributes else None
+ self.end = DateTime.parseText(xml_element.attributes["end"]) if "end" in xml_element.attributes else None
+ self.tzinfo = None
+
+
+ def settzinfo(self, tzinfo):
+ """
+ Set the default timezone to use with this query.
+ @param tzinfo: a L{Timezone} to use.
+ """
+
+ # Give tzinfo to any TimeRange we have
+ self.tzinfo = tzinfo
+
+
+ def valid(self, level=0):
+ """
+ Indicate whether the time-range is valid (must be date-time in UTC).
+
+ @return: True if valid, False otherwise
+ """
+
+ if self.start is not None and self.start.isDateOnly():
+ log.info("start attribute in <time-range> is not a date-time: %s" % (self.start,))
+ return False
+ if self.end is not None and self.end.isDateOnly():
+ log.info("end attribute in <time-range> is not a date-time: %s" % (self.end,))
+ return False
+ if self.start is not None and not self.start.utc():
+ log.info("start attribute in <time-range> is not UTC: %s" % (self.start,))
+ return False
+ if self.end is not None and not self.end.utc():
+ log.info("end attribute in <time-range> is not UTC: %s" % (self.end,))
+ return False
+
+ # No other tests
+ return True
+
+
+ def match(self, property, access=None):
+ """
+ NB This is only called when doing a time-range match on a property.
+ """
+ if property is None:
+ return False
+ else:
+ return property.containsTimeRange(self.start, self.end, self.tzinfo)
+
+
+ def matchinstance(self, component, instances):
+ """
+ Test whether this time-range element causes a match to the specified component
+ using the specified set of instances to determine the expanded time ranges.
+ @param component: the L{Component} to test.
+ @param instances: the list of expanded instances.
+ @return: True if the time-range query matches, False otherwise.
+ """
+ if component is None:
+ return False
+
+ assert instances is not None or self.end is None, "Failure to expand instance for time-range filter: %r" % (self,)
+
+ # Special case open-ended unbounded
+ if instances is None:
+ if component.getRecurrenceIDUTC() is None:
+ return True
+ else:
+ # See if the overridden component's start is past the start
+ start, _ignore_end = component.getEffectiveStartEnd()
+ if start is None:
+ return True
+ else:
+ return start >= self.start
+
+ # Handle alarms as a special case
+ alarms = (component.name() == "VALARM")
+ if alarms:
+ testcomponent = component._parent
+ else:
+ testcomponent = component
+
+ for key in instances:
+ instance = instances[key]
+
+ # First make sure components match
+ if not testcomponent.same(instance.component):
+ continue
+
+ if alarms:
+ # Get all the alarm triggers for this instance and test each one
+ triggers = instance.getAlarmTriggers()
+ for trigger in triggers:
+ if timeRangesOverlap(trigger, None, self.start, self.end, self.tzinfo):
+ return True
+ else:
+ # Regular instance overlap test
+ if timeRangesOverlap(instance.start, instance.end, self.start, self.end, self.tzinfo):
+ return True
+
+ return False
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/generator.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/generator.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/generator.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,207 @@
+##
+# Copyright (c) 2006-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.enterprise.dal.syntax import Select
+
+from txdav.common.datastore.query import expression
+from txdav.common.datastore.query.generator import SQLQueryGenerator
+from txdav.common.datastore.sql_tables import schema
+
+"""
+SQL statement generator from query expressions.
+"""
+
+__all__ = [
+ "CalDAVSQLQueryGenerator",
+]
+
+class CalDAVSQLQueryGenerator(SQLQueryGenerator):
+
+ _timerange = schema.TIME_RANGE
+ _transparency = schema.TRANSPARENCY
+
+ def __init__(self, expr, collection, whereid, userid=None, freebusy=False):
+ """
+
+ @param expr: the query expression object model
+ @type expr: L{expression}
+ @param collection: the resource targeted by the query
+ @type collection: L{CommonHomeChild}
+ @param userid: user for whom query is being done - query will be scoped to that user's privileges and their transparency
+ @type userid: C{str}
+ @param freebusy: whether or not a freebusy query is being done - if it is, additional time range and transparency information is returned
+ @type freebusy: C{bool}
+ """
+ super(CalDAVSQLQueryGenerator, self).__init__(expr, collection, whereid)
+ self.userid = userid if userid else ""
+ self.freebusy = freebusy
+ self.usedtimerange = False
+
+
+ def generate(self):
+ """
+ Generate the actual SQL statement from the passed in expression tree.
+
+ @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the partial SQL statement,
+ and the C{list} is the list of argument substitutions to use with the SQL API execute method.
+ """
+
+ # Init state
+ self.arguments = {}
+ self.argcount = 0
+ obj = self.collection._objectSchema
+
+ columns = [obj.RESOURCE_NAME, obj.ICALENDAR_UID, obj.ICALENDAR_TYPE]
+ if self.freebusy:
+ columns.extend([
+ obj.ORGANIZER,
+ self._timerange.FLOATING,
+ self._timerange.START_DATE,
+ self._timerange.END_DATE,
+ self._timerange.FBTYPE,
+ self._timerange.TRANSPARENT,
+ self._transparency.TRANSPARENT,
+ ])
+
+ # For SQL data DB we need to restrict the query to just the targeted calendar resource-id if provided
+ if self.whereid:
+
+ test = expression.isExpression(obj.CALENDAR_RESOURCE_ID, self.whereid, True)
+
+ # Since timerange expression already have the calendar resource-id test in them, do not
+ # add the additional term to those. When the additional term is added, add it as the first
+ # component in the AND expression to hopefully get the DB to use its index first
+
+ # Top-level timerange expression already has calendar resource-id restriction in it
+ if isinstance(self.expression, expression.timerangeExpression):
+ pass
+
+ # Top-level OR - check each component
+ elif isinstance(self.expression, expression.orExpression):
+
+ def _hasTopLevelTimerange(testexpr):
+ if isinstance(testexpr, expression.timerangeExpression):
+ return True
+ elif isinstance(testexpr, expression.andExpression):
+ return any([isinstance(expr, expression.timerangeExpression) for expr in testexpr.expressions])
+ else:
+ return False
+
+ hasTimerange = any([_hasTopLevelTimerange(expr) for expr in self.expression.expressions])
+
+ if hasTimerange:
+ # timerange expression forces a join on calendarid
+ pass
+ else:
+ # AND the whole thing with calendarid
+ self.expression = test.andWith(self.expression)
+
+ # Top-level AND - only add additional expression if timerange not present
+ elif isinstance(self.expression, expression.andExpression):
+ hasTimerange = any([isinstance(expr, expression.timerangeExpression) for expr in self.expression.expressions])
+ if not hasTimerange:
+ # AND the whole thing
+ self.expression = test.andWith(self.expression)
+
+ # Just use the id test
+ elif isinstance(self.expression, expression.allExpression):
+ self.expression = test
+
+ # Just AND the entire thing
+ else:
+ self.expression = test.andWith(self.expression)
+
+ # Generate ' where ...' partial statement
+ where = self.generateExpression(self.expression)
+
+ if self.usedtimerange:
+ where = where.And(self._timerange.CALENDAR_OBJECT_RESOURCE_ID == obj.RESOURCE_ID).And(self._timerange.CALENDAR_RESOURCE_ID == self.whereid)
+
+ # Set of tables depends on use of timespan and fb use
+ if self.usedtimerange:
+ if self.freebusy:
+ tables = obj.join(
+ self._timerange.join(
+ self._transparency,
+ on=(self._timerange.INSTANCE_ID == self._transparency.TIME_RANGE_INSTANCE_ID).And(self._transparency.USER_ID == self.userid),
+ type="left outer"
+ ),
+ type=","
+ )
+ else:
+ tables = obj.join(self._timerange, type=",")
+ else:
+ tables = obj
+
+ select = Select(
+ columns,
+ From=tables,
+ Where=where,
+ Distinct=True,
+ )
+
+ return select, self.arguments, self.usedtimerange
+
+
+ def generateExpression(self, expr):
+ """
+ Generate an expression and all it's subexpressions.
+
+ @param expr: the L{baseExpression} derived class to write out.
+ """
+
+ # Generate based on each type of expression we might encounter
+ partial = None
+
+ # time-range
+ if isinstance(expr, expression.timerangeExpression):
+ if expr.start and expr.end:
+ partial = (
+ (self._timerange.FLOATING == False).And(self._timerange.START_DATE < expr.end).And(self._timerange.END_DATE > expr.start)
+ ).Or(
+ (self._timerange.FLOATING == True).And(self._timerange.START_DATE < expr.endfloat).And(self._timerange.END_DATE > expr.startfloat)
+ )
+ elif expr.start and expr.end is None:
+ partial = (
+ (self._timerange.FLOATING == False).And(self._timerange.END_DATE > expr.start)
+ ).Or(
+ (self._timerange.FLOATING == True).And(self._timerange.END_DATE > expr.startfloat)
+ )
+ elif not expr.start and expr.end:
+ partial = (
+ (self._timerange.FLOATING == False).And(self._timerange.START_DATE < expr.end)
+ ).Or(
+ (self._timerange.FLOATING == True).And(self._timerange.START_DATE < expr.endfloat)
+ )
+ self.usedtimerange = True
+
+ else:
+ partial = super(CalDAVSQLQueryGenerator, self).generateExpression(expr)
+
+ return partial
+
+
+ def addArgument(self, arg):
+ """
+
+ @param arg: the C{str} of the argument to add
+ """
+
+ # Append argument to the list and add the appropriate substitution string to the output stream.
+ self.argcount += 1
+ argname = "arg{}".format(self.argcount)
+ self.arguments[argname] = arg
+ return argname
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/__init__.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/__init__.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/test_filter.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/test_filter.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/test_filter.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,220 @@
+##
+# Copyright (c) 2011-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from pycalendar.timezone import Timezone
+
+from twext.enterprise.dal.syntax import SQLFragment, Parameter
+
+from twistedcaldav.test.util import TestCase
+from twistedcaldav import caldavxml
+from twistedcaldav.timezones import TimezoneCache
+
+from txdav.caldav.datastore.index_file import sqlcalendarquery
+from txdav.caldav.datastore.query.builder import buildExpression
+from txdav.caldav.datastore.query.filter import Filter
+from txdav.caldav.datastore.query.generator import CalDAVSQLQueryGenerator
+from txdav.common.datastore.sql_tables import schema
+
+from dateutil.tz import tzutc
+import datetime
+
+class TestQueryFilter(TestCase):
+
+ _objectSchema = schema.CALENDAR_OBJECT
+ _queryFields = {
+ "UID": _objectSchema.UID,
+ "TYPE": _objectSchema.ICALENDAR_TYPE,
+ }
+
+ def setUp(self):
+ super(TestQueryFilter, self).setUp()
+ TimezoneCache.create()
+
+
+ def test_query(self):
+ """
+ Basic query test - no time range
+ """
+
+ filter = caldavxml.Filter(
+ caldavxml.ComponentFilter(
+ *[caldavxml.ComponentFilter(
+ **{"name":("VEVENT", "VFREEBUSY", "VAVAILABILITY")}
+ )],
+ **{"name": "VCALENDAR"}
+ )
+ )
+ filter = Filter(filter)
+ filter.child.settzinfo(Timezone(tzid="America/New_York"))
+
+ expression = buildExpression(filter, self._queryFields)
+ sql = CalDAVSQLQueryGenerator(expression, self, 1234)
+ select, args, usedtimerange = sql.generate()
+
+ self.assertEqual(select.toSQL(), SQLFragment(
+ "select distinct RESOURCE_NAME, ICALENDAR_UID, ICALENDAR_TYPE from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ? and ICALENDAR_TYPE in (?, ?, ?)",
+ [1234, Parameter('arg1', 3)]
+ ))
+ self.assertEqual(args, {"arg1": ("VEVENT", "VFREEBUSY", "VAVAILABILITY")})
+ self.assertEqual(usedtimerange, False)
+
+
+ def test_query_timerange(self):
+ """
+ Basic query test - with time range
+ """
+
+ filter = caldavxml.Filter(
+ caldavxml.ComponentFilter(
+ *[caldavxml.ComponentFilter(
+ *[caldavxml.TimeRange(**{"start":"20060605T160000Z", "end":"20060605T170000Z"})],
+ **{"name":("VEVENT", "VFREEBUSY", "VAVAILABILITY")}
+ )],
+ **{"name": "VCALENDAR"}
+ )
+ )
+ filter = Filter(filter)
+ filter.child.settzinfo(Timezone(tzid="America/New_York"))
+
+ expression = buildExpression(filter, self._queryFields)
+ sql = CalDAVSQLQueryGenerator(expression, self, 1234)
+ select, args, usedtimerange = sql.generate()
+
+ self.assertEqual(select.toSQL(), SQLFragment(
+ "select distinct RESOURCE_NAME, ICALENDAR_UID, ICALENDAR_TYPE from CALENDAR_OBJECT, TIME_RANGE where ICALENDAR_TYPE in (?, ?, ?) and (FLOATING = ? and START_DATE < ? and END_DATE > ? or FLOATING = ? and START_DATE < ? and END_DATE > ?) and CALENDAR_OBJECT_RESOURCE_ID = RESOURCE_ID and TIME_RANGE.CALENDAR_RESOURCE_ID = ?",
+ [Parameter('arg1', 3), False, datetime.datetime(2006, 6, 5, 17, 0, tzinfo=tzutc()), datetime.datetime(2006, 6, 5, 16, 0, tzinfo=tzutc()), True, datetime.datetime(2006, 6, 5, 13, 0, tzinfo=tzutc()), datetime.datetime(2006, 6, 5, 12, 0, tzinfo=tzutc()), 1234]
+ ))
+ self.assertEqual(args, {"arg1": ("VEVENT", "VFREEBUSY", "VAVAILABILITY")})
+ self.assertEqual(usedtimerange, True)
+
+
+ def test_query_freebusy(self):
+ """
+ Basic query test - with time range
+ """
+
+ filter = caldavxml.Filter(
+ caldavxml.ComponentFilter(
+ *[caldavxml.ComponentFilter(
+ *[caldavxml.TimeRange(**{"start":"20060605T160000Z", "end":"20060605T170000Z"})],
+ **{"name":("VEVENT", "VFREEBUSY", "VAVAILABILITY")}
+ )],
+ **{"name": "VCALENDAR"}
+ )
+ )
+ filter = Filter(filter)
+ filter.child.settzinfo(Timezone(tzid="America/New_York"))
+
+ expression = buildExpression(filter, self._queryFields)
+ sql = CalDAVSQLQueryGenerator(expression, self, 1234, "user01", True)
+ select, args, usedtimerange = sql.generate()
+
+ self.assertEqual(select.toSQL(), SQLFragment(
+ "select distinct RESOURCE_NAME, ICALENDAR_UID, ICALENDAR_TYPE, ORGANIZER, FLOATING, START_DATE, END_DATE, FBTYPE, TIME_RANGE.TRANSPARENT, TRANSPARENCY.TRANSPARENT from CALENDAR_OBJECT, TIME_RANGE left outer join TRANSPARENCY on INSTANCE_ID = TIME_RANGE_INSTANCE_ID and USER_ID = ? where ICALENDAR_TYPE in (?, ?, ?) and (FLOATING = ? and START_DATE < ? and END_DATE > ? or FLOATING = ? and START_DATE < ? and END_DATE > ?) and CALENDAR_OBJECT_RESOURCE_ID = RESOURCE_ID and TIME_RANGE.CALENDAR_RESOURCE_ID = ?",
+ ['user01', Parameter('arg1', 3), False, datetime.datetime(2006, 6, 5, 17, 0, tzinfo=tzutc()), datetime.datetime(2006, 6, 5, 16, 0, tzinfo=tzutc()), True, datetime.datetime(2006, 6, 5, 13, 0, tzinfo=tzutc()), datetime.datetime(2006, 6, 5, 12, 0, tzinfo=tzutc()), 1234]
+ ))
+ self.assertEqual(args, {"arg1": ("VEVENT", "VFREEBUSY", "VAVAILABILITY")})
+ self.assertEqual(usedtimerange, True)
+
+
+ def test_query_not_extended(self):
+ """
+ Query test - two terms not anyof
+ """
+
+ filter = caldavxml.Filter(
+ caldavxml.ComponentFilter(
+ *[
+ caldavxml.ComponentFilter(
+ **{"name":("VEVENT")}
+ ),
+ caldavxml.ComponentFilter(
+ **{"name":("VTODO")}
+ ),
+ ],
+ **{"name": "VCALENDAR"}
+ )
+ )
+ filter = Filter(filter)
+ filter.child.settzinfo(Timezone(tzid="America/New_York"))
+
+ expression = buildExpression(filter, self._queryFields)
+ sql = CalDAVSQLQueryGenerator(expression, self, 1234)
+ select, args, usedtimerange = sql.generate()
+
+ self.assertEqual(select.toSQL(), SQLFragment(
+ "select distinct RESOURCE_NAME, ICALENDAR_UID, ICALENDAR_TYPE from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ? and ICALENDAR_TYPE = ? and ICALENDAR_TYPE = ?",
+ [1234, "VEVENT", "VTODO"]
+ ))
+ self.assertEqual(args, {})
+ self.assertEqual(usedtimerange, False)
+
+
+ def test_query_extended(self):
+ """
+ Extended query test - two terms with anyof
+ """
+
+ filter = caldavxml.Filter(
+ caldavxml.ComponentFilter(
+ *[
+ caldavxml.ComponentFilter(
+ *[caldavxml.TimeRange(**{"start":"20060605T160000Z", })],
+ **{"name":("VEVENT")}
+ ),
+ caldavxml.ComponentFilter(
+ **{"name":("VTODO")}
+ ),
+ ],
+ **{"name": "VCALENDAR", "test": "anyof"}
+ )
+ )
+ filter = Filter(filter)
+ filter.child.settzinfo(Timezone(tzid="America/New_York"))
+
+ expression = buildExpression(filter, self._queryFields)
+ sql = CalDAVSQLQueryGenerator(expression, self, 1234)
+ select, args, usedtimerange = sql.generate()
+
+ self.assertEqual(select.toSQL(), SQLFragment(
+ "select distinct RESOURCE_NAME, ICALENDAR_UID, ICALENDAR_TYPE from CALENDAR_OBJECT, TIME_RANGE where (ICALENDAR_TYPE = ? and (FLOATING = ? and END_DATE > ? or FLOATING = ? and END_DATE > ?) or ICALENDAR_TYPE = ?) and CALENDAR_OBJECT_RESOURCE_ID = RESOURCE_ID and TIME_RANGE.CALENDAR_RESOURCE_ID = ?",
+ ['VEVENT', False, datetime.datetime(2006, 6, 5, 16, 0, tzinfo=tzutc()), True, datetime.datetime(2006, 6, 5, 12, 0, tzinfo=tzutc()), 'VTODO', 1234]
+ ))
+ self.assertEqual(args, {})
+ self.assertEqual(usedtimerange, True)
+
+
+ def test_sqllite_query(self):
+ """
+ Basic query test - single term.
+ Only UID can be queried via sql.
+ """
+
+ filter = caldavxml.Filter(
+ caldavxml.ComponentFilter(
+ *[caldavxml.ComponentFilter(
+ **{"name":("VEVENT", "VFREEBUSY", "VAVAILABILITY")}
+ )],
+ **{"name": "VCALENDAR"}
+ )
+ )
+ filter = Filter(filter)
+ sql, args = sqlcalendarquery(filter, 1234)
+
+ self.assertTrue(sql.find("RESOURCE") != -1)
+ self.assertTrue(sql.find("TIMESPAN") == -1)
+ self.assertTrue(sql.find("TRANSPARENCY") == -1)
+ self.assertTrue("VEVENT" in args)
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/scheduling/freebusy.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/scheduling/freebusy.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/scheduling/freebusy.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -32,10 +32,11 @@
from twistedcaldav.ical import Component, Property, iCalendarProductID
from twistedcaldav.instance import InstanceList
from twistedcaldav.memcacher import Memcacher
-from twistedcaldav.query import calendarqueryfilter
+from txdav.caldav.datastore.query.filter import Filter
from txdav.caldav.icalendarstore import QueryMaxResources
-from txdav.common.icommondatastore import IndexedSearchException
+from txdav.common.icommondatastore import IndexedSearchException, \
+ InternalDataStoreError
import uuid
@@ -282,23 +283,23 @@
# Create fake filter element to match time-range
filter = caldavxml.Filter(
- caldavxml.ComponentFilter(
- caldavxml.ComponentFilter(
- cache_timerange if caching else timerange,
- name=("VEVENT", "VFREEBUSY", "VAVAILABILITY"),
- ),
- name="VCALENDAR",
- )
- )
- filter = calendarqueryfilter.Filter(filter)
+ caldavxml.ComponentFilter(
+ caldavxml.ComponentFilter(
+ cache_timerange if caching else timerange,
+ name=("VEVENT", "VFREEBUSY", "VAVAILABILITY"),
+ ),
+ name="VCALENDAR",
+ )
+ )
+ filter = Filter(filter)
tzinfo = filter.settimezone(tz)
try:
- resources = yield calresource._index.indexedSearch(filter, useruid=attendee_uid, fbtype=True)
+ resources = yield calresource.search(filter, useruid=attendee_uid, fbtype=True)
if caching:
yield FBCacheEntry.makeCacheEntry(calresource, attendee_uid, cache_timerange, resources)
except IndexedSearchException:
- resources = yield calresource._index.bruteForceSearch()
+ raise InternalDataStoreError("Invalid indexedSearch query")
else:
# Log extended item
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/sql.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/sql.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/sql.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -57,6 +57,9 @@
from twistedcaldav.memcacher import Memcacher
from txdav.base.propertystore.base import PropertyName
+from txdav.caldav.datastore.query.builder import buildExpression
+from txdav.caldav.datastore.query.filter import Filter
+from txdav.caldav.datastore.query.generator import CalDAVSQLQueryGenerator
from txdav.caldav.datastore.scheduling.icalsplitter import iCalSplitter
from txdav.caldav.datastore.scheduling.implicit import ImplicitScheduler
from txdav.caldav.datastore.util import AttachmentRetrievalTransport, \
@@ -72,12 +75,11 @@
AttendeeAllowedError, InvalidPerUserDataMerge, ComponentUpdateState, \
ValidOrganizerError, ShareeAllowedError, ComponentRemoveState, \
InvalidDefaultCalendar, \
- InvalidAttachmentOperation, DuplicatePrivateCommentsError
+ InvalidAttachmentOperation, DuplicatePrivateCommentsError, \
+ TimeRangeUpperLimit, TimeRangeLowerLimit
from txdav.caldav.icalendarstore import QuotaExceeded
from txdav.common.datastore.sql import CommonHome, CommonHomeChild, \
CommonObjectResource, ECALENDARTYPE
-from txdav.common.datastore.sql_legacy import PostgresLegacyIndexEmulator, \
- PostgresLegacyInboxIndexEmulator
from txdav.common.datastore.sql_tables import _ATTACHMENTS_MODE_NONE, \
_ATTACHMENTS_MODE_WRITE, schema, _BIND_MODE_OWN, \
_ATTACHMENTS_MODE_READ, _TRANSP_OPAQUE, _TRANSP_TRANSPARENT
@@ -87,7 +89,8 @@
InvalidObjectResourceError, ObjectResourceNameAlreadyExistsError, \
ObjectResourceNameNotAllowedError, TooManyObjectResourcesError, \
InvalidUIDError, UIDExistsError, UIDExistsElsewhereError, \
- InvalidResourceMove, InvalidComponentForStoreError
+ InvalidResourceMove, InvalidComponentForStoreError, \
+ NoSuchObjectResourceError
from txdav.xml import element
from txdav.idav import ChangeCategory
@@ -949,6 +952,12 @@
_objectSchema = schema.CALENDAR_OBJECT
_timeRangeSchema = schema.TIME_RANGE
+ # Mapping of iCalendar property name to DB column name
+ _queryFields = {
+ "UID": _objectSchema.UID,
+ "TYPE": _objectSchema.ICALENDAR_TYPE,
+ }
+
_supportedComponents = None
def __init__(self, *args, **kw):
@@ -956,10 +965,6 @@
Initialize a calendar pointing at a record in a database.
"""
super(Calendar, self).__init__(*args, **kw)
- if self.isInbox():
- self._index = PostgresLegacyInboxIndexEmulator(self)
- else:
- self._index = PostgresLegacyIndexEmulator(self)
self._transp = _TRANSP_OPAQUE
@@ -1335,6 +1340,189 @@
@inlineCallbacks
+ def search(self, filter, useruid=None, fbtype=False):
+ """
+ Finds resources matching the given qualifiers.
+ @param filter: the L{Filter} for the calendar-query to execute.
+ @return: an iterable of tuples for each resource matching the
+ given C{qualifiers}. The tuples are C{(name, uid)}, where
+ C{name} is the resource name, C{uid} is the resource UID.
+ """
+
+ # Make sure we have a proper Filter element and get the partial SQL statement to use.
+ sql_stmt = self._sqlquery(filter, useruid, fbtype)
+
+ # No result means it is too complex for us
+ if sql_stmt is None:
+ raise IndexedSearchException()
+ sql_stmt, args, usedtimerange = sql_stmt
+
+ # Check for time-range re-expand
+ if usedtimerange is not None:
+
+ today = DateTime.getToday()
+
+ # Determine how far we need to extend the current expansion of
+ # events. If we have an open-ended time-range we will expand
+ # one year past the start. That should catch bounded
+ # recurrences - unbounded will have been indexed with an
+ # "infinite" value always included.
+ maxDate, isStartDate = filter.getmaxtimerange()
+ if maxDate:
+ maxDate = maxDate.duplicate()
+ maxDate.offsetDay(1)
+ maxDate.setDateOnly(True)
+ upperLimit = today + Duration(days=config.FreeBusyIndexExpandMaxDays)
+ if maxDate > upperLimit:
+ raise TimeRangeUpperLimit(upperLimit)
+ if isStartDate:
+ maxDate += Duration(days=365)
+
+ # Determine if the start date is too early for the restricted range we
+ # are applying. If it is today or later we don't need to worry about truncation
+ # in the past.
+ minDate, _ignore_isEndDate = filter.getmintimerange()
+ if minDate >= today:
+ minDate = None
+ if minDate is not None and config.FreeBusyIndexLowerLimitDays:
+ truncateLowerLimit = today - Duration(days=config.FreeBusyIndexLowerLimitDays)
+ if minDate < truncateLowerLimit:
+ raise TimeRangeLowerLimit(truncateLowerLimit)
+
+ if maxDate is not None or minDate is not None:
+ yield self.testAndUpdateIndex(minDate, maxDate)
+
+ rowiter = yield sql_stmt.on(self._txn, **args)
+
+ # Check result for missing resources
+ results = []
+ for row in rowiter:
+ if fbtype:
+ row = list(row)
+ row[4] = 'Y' if row[4] else 'N'
+ row[7] = indexfbtype_to_icalfbtype[row[7]]
+ if row[9] is not None:
+ row[8] = row[9]
+ row[8] = 'T' if row[8] else 'F'
+ del row[9]
+ results.append(row)
+
+ returnValue(results)
+
+
+ def _sqlquery(self, filter, useruid, fbtype):
+ """
+ Convert the supplied addressbook-query into a partial SQL statement.
+
+ @param filter: the L{Filter} for the addressbook-query to convert.
+ @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the partial SQL statement,
+ and the C{list} is the list of argument substitutions to use with the SQL API execute method.
+ Or return C{None} if it is not possible to create an SQL query to fully match the addressbook-query.
+ """
+
+ if not isinstance(filter, Filter):
+ return None
+
+ try:
+ expression = buildExpression(filter, self._queryFields)
+ sql = CalDAVSQLQueryGenerator(expression, self, self.id(), useruid, fbtype)
+ return sql.generate()
+ except ValueError:
+ return None
+
+
+ @classproperty
+ def _notExpandedWithinQuery(cls): #@NoSelf
+ """
+ Query to find resources that need to be re-expanded
+ """
+ co = schema.CALENDAR_OBJECT
+ return Select(
+ [co.RESOURCE_NAME],
+ From=co,
+ Where=((co.RECURRANCE_MIN > Parameter("minDate"))
+ .Or(co.RECURRANCE_MAX < Parameter("maxDate")))
+ .And(co.CALENDAR_RESOURCE_ID == Parameter("resourceID"))
+ )
+
+
+ @inlineCallbacks
+ def notExpandedWithin(self, minDate, maxDate):
+ """
+ Gives all resources which have not been expanded beyond a given date
+ in the database. (Unused; see above L{postgresqlgenerator}.
+ """
+ returnValue([row[0] for row in (
+ yield self._notExpandedWithinQuery.on(
+ self._txn,
+ minDate=pyCalendarTodatetime(normalizeForIndex(minDate)) if minDate is not None else None,
+ maxDate=pyCalendarTodatetime(normalizeForIndex(maxDate)),
+ resourceID=self._resourceID))]
+ )
+
+
+ @inlineCallbacks
+ def reExpandResource(self, name, expand_start, expand_end):
+ """
+ Given a resource name, remove it from the database and re-add it
+ with a longer expansion.
+ """
+ obj = yield self.calendarObjectWithName(name)
+
+ # Use a new transaction to do this update quickly without locking the row for too long. However, the original
+ # transaction may have the row locked, so use wait=False and if that fails, fall back to using the original txn.
+
+ newTxn = obj.transaction().store().newTransaction()
+ try:
+ yield obj.lock(wait=False, txn=newTxn)
+ except NoSuchObjectResourceError:
+ yield newTxn.commit()
+ returnValue(None)
+ except:
+ yield newTxn.abort()
+ newTxn = None
+
+ # Now do the re-expand using the appropriate transaction
+ try:
+ doExpand = False
+ if newTxn is None:
+ doExpand = True
+ else:
+ # We repeat this check because the resource may have been re-expanded by someone else
+ rmin, rmax = (yield obj.recurrenceMinMax(txn=newTxn))
+
+ # If the resource is not fully expanded, see if within the required range or not.
+ # Note that expand_start could be None if no lower limit is applied, but expand_end will
+ # never be None
+ if rmax is not None and rmax < expand_end:
+ doExpand = True
+ if rmin is not None and expand_start is not None and rmin > expand_start:
+ doExpand = True
+
+ if doExpand:
+ yield obj.updateDatabase(
+ (yield obj.component()),
+ expand_until=expand_end,
+ reCreate=True,
+ txn=newTxn,
+ )
+ finally:
+ if newTxn is not None:
+ yield newTxn.commit()
+
+
+ @inlineCallbacks
+ def testAndUpdateIndex(self, minDate, maxDate):
+ # Find out if the index is expanded far enough
+ names = yield self.notExpandedWithin(minDate, maxDate)
+
+ # Actually expand recurrence max
+ for name in names:
+ self.log.info("Search falls outside range of index for %s %s to %s" % (name, minDate, maxDate))
+ yield self.reExpandResource(name, minDate, maxDate)
+
+
+ @inlineCallbacks
def splitCollectionByComponentTypes(self):
"""
If the calendar contains iCalendar data with different component types, then split it into separate collections
@@ -2179,7 +2367,10 @@
"""
if isinstance(component, str) or isinstance(component, unicode):
- component = self._componentClass.fromString(component)
+ try:
+ component = self._componentClass.fromString(component)
+ except InvalidICalendarDataError as e:
+ raise InvalidComponentForStoreError(str(e))
return self._setComponentInternal(component, inserting, ComponentUpdateState.NORMAL, smart_merge)
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_index_file.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_index_file.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_index_file.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -19,6 +19,7 @@
from twisted.internet.task import deferLater
from txdav.caldav.datastore.index_file import Index, MemcachedUIDReserver
+from txdav.caldav.datastore.query.filter import Filter
from txdav.common.icommondatastore import ReservationError, \
InternalDataStoreError
@@ -26,7 +27,6 @@
from twistedcaldav.caldavxml import TimeRange
from twistedcaldav.ical import Component, InvalidICalendarDataError
from twistedcaldav.instance import InvalidOverriddenInstanceError
-from twistedcaldav.query import calendarqueryfilter
from twistedcaldav.test.util import InMemoryMemcacheProtocol
import twistedcaldav.test.util
@@ -480,7 +480,7 @@
name="VCALENDAR",
)
)
- filter = calendarqueryfilter.Filter(filter)
+ filter = Filter(filter)
resources = yield self.db.indexedSearch(filter)
index_results = set()
@@ -666,7 +666,7 @@
name="VCALENDAR",
)
)
- filter = calendarqueryfilter.Filter(filter)
+ filter = Filter(filter)
resources = yield self.db.indexedSearch(filter, fbtype=True)
index_results = set()
@@ -1073,7 +1073,7 @@
name="VCALENDAR",
)
)
- filter = calendarqueryfilter.Filter(filter)
+ filter = Filter(filter)
for useruid, instances in peruserinstances:
resources = yield self.db.indexedSearch(filter, useruid=useruid, fbtype=True)
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_sql.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_sql.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_sql.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -13,14 +13,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
##
-from txdav.caldav.datastore.scheduling.processing import ImplicitProcessor
-from txdav.caldav.datastore.scheduling.cuaddress import RemoteCalendarUser, \
- LocalCalendarUser
-from txdav.caldav.datastore.scheduling.caldav.scheduler import CalDAVScheduler
-from txdav.caldav.datastore.scheduling.scheduler import ScheduleResponseQueue
-from twext.web2 import responsecode
-from txdav.caldav.datastore.scheduling.itip import iTIPRequestStatus
-from twistedcaldav.instance import InvalidOverriddenInstanceError
"""
Tests for txdav.caldav.datastore.postgres, mostly based on
@@ -33,6 +25,7 @@
from twext.enterprise.dal.syntax import Select, Parameter, Insert, Delete, \
Update
from twext.python.vcomponent import VComponent
+from twext.web2 import responsecode
from twext.web2.http_headers import MimeType
from twext.web2.stream import MemoryStream
@@ -47,9 +40,16 @@
from twistedcaldav.config import config
from twistedcaldav.dateops import datetimeMktime
from twistedcaldav.ical import Component, normalize_iCalStr, diff_iCalStrs
-from twistedcaldav.query import calendarqueryfilter
+from twistedcaldav.instance import InvalidOverriddenInstanceError
from txdav.base.propertystore.base import PropertyName
+from txdav.caldav.datastore.query.filter import Filter
+from txdav.caldav.datastore.scheduling.caldav.scheduler import CalDAVScheduler
+from txdav.caldav.datastore.scheduling.cuaddress import RemoteCalendarUser, \
+ LocalCalendarUser
+from txdav.caldav.datastore.scheduling.itip import iTIPRequestStatus
+from txdav.caldav.datastore.scheduling.processing import ImplicitProcessor
+from txdav.caldav.datastore.scheduling.scheduler import ScheduleResponseQueue
from txdav.caldav.datastore.test.common import CommonTests as CalendarCommonTests, \
test_event_text
from txdav.caldav.datastore.test.test_file import setUpCalendarStore
@@ -57,14 +57,13 @@
from txdav.caldav.datastore.util import _migrateCalendar, migrateHome
from txdav.caldav.icalendarstore import ComponentUpdateState, InvalidDefaultCalendar
from txdav.common.datastore.sql import ECALENDARTYPE, CommonObjectResource
-from txdav.common.datastore.sql_legacy import PostgresLegacyIndexEmulator
from txdav.common.datastore.sql_tables import schema, _BIND_MODE_DIRECT, \
_BIND_STATUS_ACCEPTED
from txdav.common.datastore.test.util import populateCalendarsFrom, \
CommonCommonTests
from txdav.common.icommondatastore import NoSuchObjectResourceError
-from txdav.xml.rfc2518 import GETContentLanguage, ResourceType
from txdav.idav import ChangeCategory
+from txdav.xml.rfc2518 import GETContentLanguage, ResourceType
import datetime
@@ -407,10 +406,10 @@
name="VCALENDAR",
)
)
- filter = calendarqueryfilter.Filter(filter)
+ filter = Filter(filter)
filter.settimezone(None)
- results = yield toCalendar._index.indexedSearch(filter, 'user01', True)
+ results = yield toCalendar.search(filter, 'user01', True)
self.assertEquals(len(results), 1)
_ignore_name, uid, _ignore_type, _ignore_organizer, _ignore_float, _ignore_start, _ignore_end, _ignore_fbtype, transp = results[0]
self.assertEquals(uid, "uid4")
@@ -1369,7 +1368,7 @@
@inlineCallbacks
def test_notExpandedWithin(self):
"""
- Test PostgresLegacyIndexEmulator.notExpandedWithin to make sure it returns the correct
+ Test Calendar.notExpandedWithin to make sure it returns the correct
result based on the ranges passed in.
"""
@@ -1378,7 +1377,6 @@
# Create the index on a new calendar
home = yield self.homeUnderTest()
newcalendar = yield home.createCalendarWithName("index_testing")
- index = PostgresLegacyIndexEmulator(newcalendar)
# Create the calendar object to use for testing
nowYear = self.nowYear["now"]
@@ -1406,37 +1404,37 @@
# Fully within range
testMin = DateTime(nowYear, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
testMax = DateTime(nowYear + 1, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
- result = yield index.notExpandedWithin(testMin, testMax)
+ result = yield newcalendar.notExpandedWithin(testMin, testMax)
self.assertEqual(result, [])
# Upper bound exceeded
testMin = DateTime(nowYear, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
testMax = DateTime(nowYear + 5, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
- result = yield index.notExpandedWithin(testMin, testMax)
+ result = yield newcalendar.notExpandedWithin(testMin, testMax)
self.assertEqual(result, ["indexing.ics"])
# Lower bound exceeded
testMin = DateTime(nowYear - 5, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
testMax = DateTime(nowYear + 1, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
- result = yield index.notExpandedWithin(testMin, testMax)
+ result = yield newcalendar.notExpandedWithin(testMin, testMax)
self.assertEqual(result, ["indexing.ics"])
# Lower and upper bounds exceeded
testMin = DateTime(nowYear - 5, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
testMax = DateTime(nowYear + 5, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
- result = yield index.notExpandedWithin(testMin, testMax)
+ result = yield newcalendar.notExpandedWithin(testMin, testMax)
self.assertEqual(result, ["indexing.ics"])
# Lower none within range
testMin = None
testMax = DateTime(nowYear + 1, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
- result = yield index.notExpandedWithin(testMin, testMax)
+ result = yield newcalendar.notExpandedWithin(testMin, testMax)
self.assertEqual(result, [])
# Lower none and upper bounds exceeded
testMin = None
testMax = DateTime(nowYear + 5, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
- result = yield index.notExpandedWithin(testMin, testMax)
+ result = yield newcalendar.notExpandedWithin(testMin, testMax)
self.assertEqual(result, ["indexing.ics"])
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/index_file.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/index_file.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/index_file.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -37,10 +37,6 @@
from twisted.internet.defer import maybeDeferred
-from twistedcaldav import carddavxml
-from txdav.common.icommondatastore import SyncTokenValidException, \
- ReservationError
-from twistedcaldav.query import addressbookquery
from twistedcaldav.sql import AbstractSQLDatabase
from twistedcaldav.sql import db_prefix
from twistedcaldav.vcard import Component
@@ -49,6 +45,12 @@
from twistedcaldav.config import config
from twistedcaldav.memcachepool import CachePoolUserMixIn
+from txdav.carddav.datastore.query.builder import buildExpression
+from txdav.carddav.datastore.query.filter import Filter
+from txdav.common.datastore.query.filegenerator import sqllitegenerator
+from txdav.common.icommondatastore import SyncTokenValidException, \
+ ReservationError
+
log = Logger()
db_basename = db_prefix + "sqlite"
@@ -218,6 +220,24 @@
+def sqladdressbookquery(filter, addressbookid=None):
+ """
+ Convert the supplied addressbook-query into a partial SQL statement.
+
+ @param filter: the L{Filter} for the addressbook-query to convert.
+ @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the partial SQL statement,
+ and the C{list} is the list of argument substitutions to use with the SQL API execute method.
+ Or return C{None} if it is not possible to create an SQL query to fully match the addressbook-query.
+ """
+ try:
+ expression = buildExpression(filter, sqllitegenerator.FIELDS)
+ sql = sqllitegenerator(expression, addressbookid, None)
+ return sql.generate()
+ except ValueError:
+ return None
+
+
+
class AddressBookIndex(AbstractSQLDatabase):
"""
AddressBook collection index abstract base class that defines the apis for the index.
@@ -445,8 +465,8 @@
def searchValid(self, filter):
- if isinstance(filter, carddavxml.Filter):
- qualifiers = addressbookquery.sqladdressbookquery(filter)
+ if isinstance(filter, Filter):
+ qualifiers = sqladdressbookquery(filter)
else:
qualifiers = None
@@ -466,8 +486,8 @@
# start caching...
# Make sure we have a proper Filter element and get the partial SQL statement to use.
- if isinstance(filter, carddavxml.Filter):
- qualifiers = addressbookquery.sqladdressbookquery(filter)
+ if isinstance(filter, Filter):
+ qualifiers = sqladdressbookquery(filter)
else:
qualifiers = None
if qualifiers is not None:
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/__init__.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/__init__.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/builder.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/builder.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/builder.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,107 @@
+##
+# Copyright (c) 2006-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from txdav.common.datastore.query import expression
+from txdav.carddav.datastore.query.filter import TextMatch
+
+"""
+SQL statement generator from query expressions.
+"""
+
+__all__ = [
+ "buildExpression",
+]
+
+
+
+# SQL Index column (field) names
+
+def buildExpression(filter, fields):
+ """
+ Convert the supplied addressbook-query into an expression tree.
+
+ @param filter: the L{Filter} for the addressbook-query to convert.
+ @return: a L{baseExpression} for the expression tree.
+ """
+ # Lets assume we have a valid filter from the outset.
+
+ # Top-level filter contains zero or more prop-filter element
+ if len(filter.children) > 0:
+ return propfilterListExpression(filter.children, fields)
+ else:
+ return expression.allExpression()
+
+
+
+def propfilterListExpression(propfilters, fields):
+ """
+ Create an expression for a list of prop-filter elements.
+
+ @param propfilters: the C{list} of L{ComponentFilter} elements.
+ @return: a L{baseExpression} for the expression tree.
+ """
+
+ if len(propfilters) == 1:
+ return propfilterExpression(propfilters[0], fields)
+ else:
+ return expression.orExpression([propfilterExpression(c, fields) for c in propfilters])
+
+
+
+def propfilterExpression(propfilter, fields):
+ """
+ Create an expression for a single prop-filter element.
+
+ @param propfilter: the L{PropertyFilter} element.
+ @return: a L{baseExpression} for the expression tree.
+ """
+
+ # Only handle UID right now
+ if propfilter.filter_name != "UID":
+ raise ValueError
+
+ # Handle is-not-defined case
+ if not propfilter.defined:
+ # Test for <<field>> != "*"
+ return expression.isExpression(fields["UID"], "", True)
+
+ # Handle embedded parameters/text-match
+ params = []
+ for filter in propfilter.filters:
+ if isinstance(filter, TextMatch):
+ if filter.match_type == "equals":
+ tm = expression.isnotExpression if filter.negate else expression.isExpression
+ elif filter.match_type == "contains":
+ tm = expression.notcontainsExpression if filter.negate else expression.containsExpression
+ elif filter.match_type == "starts-with":
+ tm = expression.notstartswithExpression if filter.negate else expression.startswithExpression
+ elif filter.match_type == "ends-with":
+ tm = expression.notendswithExpression if filter.negate else expression.endswithExpression
+ params.append(tm(fields[propfilter.filter_name], str(filter.text), True))
+ else:
+ # No embedded parameters - not right now as our Index does not handle them
+ raise ValueError
+
+ # Now build return expression
+ if len(params) > 1:
+ if propfilter.propfilter_test == "anyof":
+ return expression.orExpression(params)
+ else:
+ return expression.andExpression(params)
+ elif len(params) == 1:
+ return params[0]
+ else:
+ return None
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/filter.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/filter.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/filter.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,314 @@
+##
+# Copyright (c) 2011-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+Object model of CARDDAV:filter element used in an addressbook-query.
+"""
+
+__all__ = [
+ "Filter",
+]
+
+from twext.python.log import Logger
+
+from twistedcaldav.carddavxml import carddav_namespace
+from twistedcaldav.vcard import Property
+
+log = Logger()
+
+class FilterBase(object):
+ """
+ Determines which matching components are returned.
+ """
+
+ def __init__(self, xml_element):
+ self.xmlelement = xml_element
+
+
+ def match(self, item, access=None):
+ raise NotImplementedError
+
+
+ def valid(self, level=0):
+ raise NotImplementedError
+
+
+
+class Filter(FilterBase):
+ """
+ Determines which matching components are returned.
+ """
+
+ def __init__(self, xml_element):
+
+ super(Filter, self).__init__(xml_element)
+
+ filter_test = xml_element.attributes.get("test", "anyof")
+ if filter_test not in ("anyof", "allof"):
+ raise ValueError("Test must be only one of anyof, allof")
+
+ self.filter_test = filter_test
+
+ self.children = [PropertyFilter(child) for child in xml_element.children]
+
+
+ def match(self, vcard):
+ """
+ Returns True if the given address property matches this filter, False
+ otherwise. Empty element means always match.
+ """
+
+ if len(self.children) > 0:
+ allof = self.filter_test == "allof"
+ for propfilter in self.children:
+ if allof != propfilter._match(vcard):
+ return not allof
+ return allof
+ else:
+ return True
+
+
+ def valid(self):
+ """
+ Indicate whether this filter element's structure is valid wrt vCard
+ data object model.
+
+ @return: True if valid, False otherwise
+ """
+
+ # Test each property
+ for propfilter in self.children:
+ if not propfilter.valid():
+ return False
+ else:
+ return True
+
+
+
+class FilterChildBase(FilterBase):
+ """
+ CardDAV filter element.
+ """
+
+ def __init__(self, xml_element):
+
+ super(FilterChildBase, self).__init__(xml_element)
+
+ qualifier = None
+ filters = []
+
+ for child in xml_element.children:
+ qname = child.qname()
+
+ if qname in (
+ (carddav_namespace, "is-not-defined"),
+ ):
+ if qualifier is not None:
+ raise ValueError("Only one of CardDAV:is-not-defined allowed")
+ qualifier = IsNotDefined(child)
+
+ elif qname == (carddav_namespace, "text-match"):
+ filters.append(TextMatch(child))
+
+ elif qname == (carddav_namespace, "param-filter"):
+ filters.append(ParameterFilter(child))
+ else:
+ raise ValueError("Unknown child element: %s" % (qname,))
+
+ if qualifier and isinstance(qualifier, IsNotDefined) and (len(filters) != 0):
+ raise ValueError("No other tests allowed when CardDAV:is-not-defined is present")
+
+ if xml_element.qname() == (carddav_namespace, "prop-filter"):
+ propfilter_test = xml_element.attributes.get("test", "anyof")
+ if propfilter_test not in ("anyof", "allof"):
+ raise ValueError("Test must be only one of anyof, allof")
+ else:
+ propfilter_test = "anyof"
+
+ self.propfilter_test = propfilter_test
+ self.qualifier = qualifier
+ self.filters = filters
+ self.filter_name = xml_element.attributes["name"]
+ if isinstance(self.filter_name, unicode):
+ self.filter_name = self.filter_name.encode("utf-8")
+ self.defined = not self.qualifier or not isinstance(qualifier, IsNotDefined)
+
+
+ def match(self, item):
+ """
+ Returns True if the given address book item (either a property or parameter value)
+ matches this filter, False otherwise.
+ """
+
+ # Always return True for the is-not-defined case as the result of this will
+ # be negated by the caller
+ if not self.defined:
+ return True
+
+ if self.qualifier and not self.qualifier.match(item):
+ return False
+
+ if len(self.filters) > 0:
+ allof = self.propfilter_test == "allof"
+ for filter in self.filters:
+ if allof != filter._match(item):
+ return not allof
+ return allof
+ else:
+ return True
+
+
+
+class PropertyFilter (FilterChildBase):
+ """
+ Limits a search to specific properties.
+ """
+
+ def _match(self, vcard):
+ # At least one property must match (or is-not-defined is set)
+ for property in vcard.properties():
+ if property.name().upper() == self.filter_name.upper() and self.match(property):
+ break
+ else:
+ return not self.defined
+ return self.defined
+
+
+ def valid(self):
+ """
+ Indicate whether this filter element's structure is valid wrt vCard
+ data object model.
+
+ @return: True if valid, False otherwise
+ """
+
+ # No tests
+ return True
+
+
+
+class ParameterFilter (FilterChildBase):
+ """
+ Limits a search to specific parameters.
+ """
+
+ def _match(self, property):
+
+ # At least one parameter must match (or is-not-defined is set)
+ result = not self.defined
+ for parameterName in property.parameterNames():
+ if parameterName.upper() == self.filter_name.upper() and self.match([property.parameterValues(parameterName)]):
+ result = self.defined
+ break
+
+ return result
+
+
+
+class IsNotDefined (FilterBase):
+ """
+ Specifies that the named iCalendar item does not exist.
+ """
+
+ def match(self, component, access=None):
+ # Oddly, this needs always to return True so that it appears there is
+ # a match - but we then "negate" the result if is-not-defined is set.
+ # Actually this method should never be called as we special case the
+ # is-not-defined option.
+ return True
+
+
+
+class TextMatch (FilterBase):
+ """
+ Specifies a substring match on a property or parameter value.
+ """
+ def __init__(self, xml_element):
+
+ super(TextMatch, self).__init__(xml_element)
+
+ self.text = str(xml_element)
+
+ if "collation" in xml_element.attributes:
+ self.collation = xml_element.attributes["collation"]
+ else:
+ self.collation = "i;unicode-casemap"
+
+ if "negate-condition" in xml_element.attributes:
+ self.negate = xml_element.attributes["negate-condition"]
+ if self.negate not in ("yes", "no"):
+ self.negate = "no"
+ self.negate = {"yes": True, "no": False}[self.negate]
+ else:
+ self.negate = False
+
+ if "match-type" in xml_element.attributes:
+ self.match_type = xml_element.attributes["match-type"]
+ if self.match_type not in (
+ "equals",
+ "contains",
+ "starts-with",
+ "ends-with",
+ ):
+ self.match_type = "contains"
+ else:
+ self.match_type = "contains"
+
+
+ def _match(self, item):
+ """
+ Match the text for the item.
+ If the item is a property, then match the property value,
+ otherwise it may be a list of parameter values - try to match anyone of those
+ """
+ if item is None:
+ return False
+
+ if isinstance(item, Property):
+ values = [item.strvalue()]
+ else:
+ values = item
+
+ test = unicode(self.text, "utf-8").lower()
+
+
+ def _textCompare(s):
+ # Currently ignores the collation and does caseless matching
+ s = s.lower()
+
+ if self.match_type == "equals":
+ return s == test
+ elif self.match_type == "contains":
+ return s.find(test) != -1
+ elif self.match_type == "starts-with":
+ return s.startswith(test)
+ elif self.match_type == "ends-with":
+ return s.endswith(test)
+ else:
+ return False
+
+ for value in values:
+ # NB Its possible that we have a text list value which appears as a Python list,
+ # so we need to check for that and iterate over the list.
+ if isinstance(value, list):
+ for subvalue in value:
+ if _textCompare(unicode(subvalue, "utf-8")):
+ return not self.negate
+ else:
+ if _textCompare(unicode(value, "utf-8")):
+ return not self.negate
+
+ return self.negate
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/__init__.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/__init__.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/test_filter.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/test_filter.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/test_filter.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,74 @@
+##
+# Copyright (c) 2011-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.enterprise.dal.syntax import SQLFragment
+
+from twisted.trial.unittest import TestCase
+
+from twistedcaldav import carddavxml
+
+from txdav.carddav.datastore.query.filter import Filter
+from txdav.common.datastore.sql_tables import schema
+from txdav.carddav.datastore.query.builder import buildExpression
+from txdav.common.datastore.query.generator import SQLQueryGenerator
+from txdav.carddav.datastore.index_file import sqladdressbookquery
+
+class TestQueryFilter(TestCase):
+
+ _objectSchema = schema.ADDRESSBOOK_OBJECT
+ _queryFields = {
+ "UID": _objectSchema.UID
+ }
+
+ def test_query(self):
+ """
+ Basic query test - single term.
+ Only UID can be queried via sql.
+ """
+
+ filter = carddavxml.Filter(
+ *[carddavxml.PropertyFilter(
+ carddavxml.TextMatch.fromString("Example"),
+ **{"name":"UID"}
+ )]
+ )
+ filter = Filter(filter)
+
+ expression = buildExpression(filter, self._queryFields)
+ sql = SQLQueryGenerator(expression, self, 1234)
+ select, args = sql.generate()
+
+ self.assertEqual(select.toSQL(), SQLFragment("select distinct RESOURCE_NAME, VCARD_UID from ADDRESSBOOK_OBJECT where ADDRESSBOOK_HOME_RESOURCE_ID = ? and VCARD_UID like (? || (? || ?))", [1234, "%", "Example", "%"]))
+ self.assertEqual(args, {})
+
+
+ def test_sqllite_query(self):
+ """
+ Basic query test - single term.
+ Only UID can be queried via sql.
+ """
+
+ filter = carddavxml.Filter(
+ *[carddavxml.PropertyFilter(
+ carddavxml.TextMatch.fromString("Example"),
+ **{"name":"UID"}
+ )]
+ )
+ filter = Filter(filter)
+ sql, args = sqladdressbookquery(filter, 1234)
+
+ self.assertEqual(sql, " from RESOURCE where RESOURCE.UID GLOB :1")
+ self.assertEqual(args, ["*Example*"])
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/sql.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/sql.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/sql.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -15,6 +15,7 @@
# limitations under the License.
# #
+
"""
SQL backend for CardDAV storage.
"""
@@ -46,12 +47,14 @@
from txdav.base.propertystore.base import PropertyName
from txdav.base.propertystore.sql import PropertyStore
+from txdav.carddav.datastore.query.builder import buildExpression
+from txdav.carddav.datastore.query.filter import Filter
from txdav.carddav.iaddressbookstore import IAddressBookHome, IAddressBook, \
IAddressBookObject, GroupWithUnsharedAddressNotAllowedError, \
KindChangeNotAllowedError
+from txdav.common.datastore.query.generator import SQLQueryGenerator
from txdav.common.datastore.sql import CommonHome, CommonHomeChild, \
CommonObjectResource, EADDRESSBOOKTYPE, SharingMixIn, SharingInvitation
-from txdav.common.datastore.sql_legacy import PostgresLegacyABIndexEmulator
from txdav.common.datastore.sql_tables import _ABO_KIND_PERSON, \
_ABO_KIND_GROUP, _ABO_KIND_RESOURCE, _ABO_KIND_LOCATION, schema, \
_BIND_MODE_OWN, _BIND_MODE_WRITE, _BIND_STATUS_ACCEPTED, \
@@ -60,7 +63,8 @@
from txdav.common.icommondatastore import InternalDataStoreError, \
InvalidUIDError, UIDExistsError, ObjectResourceTooBigError, \
InvalidObjectResourceError, InvalidComponentForStoreError, \
- AllRetriesFailed, ObjectResourceNameAlreadyExistsError
+ AllRetriesFailed, ObjectResourceNameAlreadyExistsError, \
+ IndexedSearchException
from txdav.xml import element
from zope.interface.declarations import implements
@@ -439,7 +443,12 @@
_revisionsSchema = schema.ADDRESSBOOK_OBJECT_REVISIONS
_objectSchema = schema.ADDRESSBOOK_OBJECT
+ # Mapping of vCard property name to DB column name
+ _queryFields = {
+ "UID": _objectSchema.UID,
+ }
+
@classmethod
@inlineCallbacks
def _getDBDataIndirect(cls, home, name, resourceID, externalID):
@@ -515,7 +524,6 @@
def __init__(self, home, name, resourceID, mode, status, revision=0, message=None, ownerHome=None, ownerName=None, externalID=None):
ownerName = ownerHome.addressbook().name() if ownerHome else None
super(AddressBook, self).__init__(home, name, resourceID, mode, status, revision=revision, message=message, ownerHome=ownerHome, ownerName=ownerName, externalID=externalID)
- self._index = PostgresLegacyABIndexEmulator(self)
def __repr__(self):
@@ -788,6 +796,50 @@
returnValue((yield super(AddressBook, self).bumpModified()))
+ @inlineCallbacks
+ def search(self, filter):
+ """
+ Finds resources matching the given qualifiers.
+ @param filter: the L{Filter} for the addressbook-query to execute.
+ @return: an iterable of tuples for each resource matching the
+ given C{qualifiers}. The tuples are C{(name, uid)}, where
+ C{name} is the resource name, C{uid} is the resource UID.
+ """
+
+ # Make sure we have a proper Filter element and get the partial SQL statement to use.
+ sql_stmt = self._sqlquery(filter)
+
+ # No result means it is too complex for us
+ if sql_stmt is None:
+ raise IndexedSearchException()
+
+ sql_stmt, args = sql_stmt
+ rowiter = yield sql_stmt.on(self._txn, **args)
+
+ returnValue(list(rowiter))
+
+
+ def _sqlquery(self, filter):
+ """
+ Convert the supplied addressbook-query into a partial SQL statement.
+
+ @param filter: the L{Filter} for the addressbook-query to convert.
+ @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the partial SQL statement,
+ and the C{list} is the list of argument substitutions to use with the SQL API execute method.
+ Or return C{None} if it is not possible to create an SQL query to fully match the addressbook-query.
+ """
+
+ if not isinstance(filter, Filter):
+ return None
+
+ try:
+ expression = buildExpression(filter, self._queryFields)
+ sql = SQLQueryGenerator(expression, self, self.id())
+ return sql.generate()
+ except ValueError:
+ return None
+
+
@classmethod
@inlineCallbacks
def listObjects(cls, home):
@@ -1989,6 +2041,10 @@
if isinstance(component, str) or isinstance(component, unicode):
component = self._componentClass.fromString(component)
+ try:
+ component = self._componentClass.fromString(component)
+ except InvalidVCardDataError as e:
+ raise InvalidComponentForStoreError(str(e))
self._componentChanged = False
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/test/test_sql_sharing.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/test/test_sql_sharing.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/test/test_sql_sharing.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -1178,13 +1178,15 @@
otherHome = yield self.addressbookHomeUnderTest(name="user02")
for depth in ("1", "infinity",):
- changed, deleted = yield otherHome.resourceNamesSinceRevision(otherAB._bindRevision - 1, depth)
+ changed, deleted, invalid = yield otherHome.resourceNamesSinceRevision(otherAB._bindRevision - 1, depth)
self.assertNotEqual(len(changed), 0)
self.assertEqual(len(deleted), 0)
+ self.assertEqual(len(invalid), 0)
- changed, deleted = yield otherHome.resourceNamesSinceRevision(otherAB._bindRevision, depth)
+ changed, deleted, invalid = yield otherHome.resourceNamesSinceRevision(otherAB._bindRevision, depth)
self.assertEqual(len(changed), 0)
self.assertEqual(len(deleted), 0)
+ self.assertEqual(len(invalid), 0)
@inlineCallbacks
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/__init__.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/__init__.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/expression.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/expression.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/expression.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,382 @@
+##
+# Copyright (c) 2006-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+Query Expression Elements. These are used to build a 'generic' query
+expression tree that can then be used by different query language
+generators to produce the actual query syntax required (SQL, xpath
+etc).
+"""
+
+__all__ = [
+ "allExpression",
+ "notExpression",
+ "andExpression",
+ "orExpression",
+ "timerangeExpression",
+ "textcompareExpression",
+ "containsExpression",
+ "notcontainsExpression",
+ "isExpression",
+ "isnotExpression",
+ "startswithExpression",
+ "notstartswithExpression",
+ "endswithExpression",
+ "notendswithExpression",
+ "inExpression",
+ "notinExpression",
+]
+
+class baseExpression(object):
+ """
+ The base class for all types of expression.
+ """
+
+ def __init__(self):
+ pass
+
+
+ def multi(self):
+ """
+ Indicate whether this expression is composed of multiple sub-expressions.
+
+ @return: C{True} if this expressions contains multiple sub-expressions,
+ C{False} otherwise.
+ """
+
+ return False
+
+
+ def _collapsedExpression(self):
+ return self
+
+
+ def andWith(self, other):
+ if isinstance(other, andExpression):
+ return andExpression((self._collapsedExpression(),) + tuple(other.expressions))
+ else:
+ return andExpression((self._collapsedExpression(), other._collapsedExpression(),))
+
+
+ def orWith(self, other):
+ if isinstance(other, orExpression):
+ return orExpression((self._collapsedExpression(),) + tuple(other.expressions))
+ else:
+ return orExpression((self._collapsedExpression(), other._collapsedExpression(),))
+
+
+
+class allExpression(baseExpression):
+ """
+ Match everything.
+ """
+
+ def __init__(self):
+ pass
+
+
+
+class logicExpression(baseExpression):
+ """
+ An expression representing a logical operation (boolean).
+ """
+
+ def __init__(self, expressions):
+ self.expressions = expressions
+
+
+ def __str__(self):
+ """
+ Generate a suitable text descriptor of this expression.
+
+ @return: a C{str} of the text for this expression.
+ """
+
+ result = ""
+ for e in self.expressions:
+ if len(result) != 0:
+ result += " " + self.operator() + " "
+ result += str(e)
+ if len(result):
+ result = "(" + result + ")"
+ return result
+
+
+ def multi(self):
+ """
+ Indicate whether this expression is composed of multiple expressions.
+
+ @return: C{True} if this expressions contains multiple sub-expressions,
+ C{False} otherwise.
+ """
+
+ return True
+
+
+ def _collapsedExpression(self):
+ if self.multi() and len(self.expressions) == 1:
+ return self.expressions[0]._collapsedExpression()
+ else:
+ return self
+
+
+
+class notExpression(logicExpression):
+ """
+ Logical NOT operation.
+ """
+
+ def __init__(self, expression):
+ super(notExpression, self).__init__([expression])
+
+
+ def operator(self):
+ return "NOT"
+
+
+ def __str__(self):
+ result = self.operator() + " " + str(self.expressions[0])
+ return result
+
+
+ def multi(self):
+ """
+ Indicate whether this expression is composed of multiple expressions.
+
+ @return: C{True} if this expressions contains multiple sub-expressions,
+ C{False} otherwise.
+ """
+
+ return False
+
+
+
+class andExpression(logicExpression):
+ """
+ Logical AND operation.
+ """
+
+ def __init__(self, expressions):
+ super(andExpression, self).__init__(expressions)
+
+
+ def operator(self):
+ return "AND"
+
+
+ def andWith(self, other):
+ self.expressions = tuple(self.expressions) + (other._collapsedExpression(),)
+ return self
+
+
+
+class orExpression(logicExpression):
+ """
+ Logical OR operation.
+ """
+
+ def __init__(self, expressions):
+ super(orExpression, self).__init__(expressions)
+
+
+ def operator(self):
+ return "OR"
+
+
+ def orWith(self, other):
+ self.expressions = tuple(self.expressions) + (other._collapsedExpression(),)
+ return self
+
+
+
+class timerangeExpression(baseExpression):
+ """
+ CalDAV time-range comparison expression.
+ """
+
+ def __init__(self, start, end, startfloat, endfloat):
+ self.start = start
+ self.end = end
+ self.startfloat = startfloat
+ self.endfloat = endfloat
+
+
+ def __str__(self):
+ return "timerange(" + str(self.start) + ", " + str(self.end) + ")"
+
+
+
+class textcompareExpression(baseExpression):
+ """
+ Base class for text comparison expressions.
+ """
+
+ def __init__(self, field, text, caseless):
+ self.field = field
+ self.text = text
+ self.caseless = caseless
+
+
+ def __str__(self):
+ return self.operator() + "(" + self.field + ", " + self.text + ", " + str(self.caseless) + ")"
+
+
+
+class containsExpression(textcompareExpression):
+ """
+ Text CONTAINS (sub-string match) expression.
+ """
+
+ def __init__(self, field, text, caseless):
+ super(containsExpression, self).__init__(field, text, caseless)
+
+
+ def operator(self):
+ return "contains"
+
+
+
+class notcontainsExpression(textcompareExpression):
+ """
+ Text NOT CONTAINS (sub-string match) expression.
+ """
+
+ def __init__(self, field, text, caseless):
+ super(notcontainsExpression, self).__init__(field, text, caseless)
+
+
+ def operator(self):
+ return "does not contain"
+
+
+
+class isExpression(textcompareExpression):
+ """
+ Text IS (exact string match) expression.
+ """
+
+ def __init__(self, field, text, caseless):
+ super(isExpression, self).__init__(field, text, caseless)
+
+
+ def operator(self):
+ return "is"
+
+
+
+class isnotExpression(textcompareExpression):
+ """
+ Text IS NOT (exact string match) expression.
+ """
+
+ def __init__(self, field, text, caseless):
+ super(isnotExpression, self).__init__(field, text, caseless)
+
+
+ def operator(self):
+ return "is not"
+
+
+
+class startswithExpression(textcompareExpression):
+ """
+ Text STARTSWITH (sub-string match) expression.
+ """
+
+ def __init__(self, field, text, caseless):
+ super(startswithExpression, self).__init__(field, text, caseless)
+
+
+ def operator(self):
+ return "starts with"
+
+
+
+class notstartswithExpression(textcompareExpression):
+ """
+ Text NOT STARTSWITH (sub-string match) expression.
+ """
+
+ def __init__(self, field, text, caseless):
+ super(notstartswithExpression, self).__init__(field, text, caseless)
+
+
+ def operator(self):
+ return "does not start with"
+
+
+
+class endswithExpression(textcompareExpression):
+ """
+ Text STARTSWITH (sub-string match) expression.
+ """
+
+ def __init__(self, field, text, caseless):
+ super(endswithExpression, self).__init__(field, text, caseless)
+
+
+ def operator(self):
+ return "ends with"
+
+
+
+class notendswithExpression(textcompareExpression):
+ """
+ Text NOT STARTSWITH (sub-string match) expression.
+ """
+
+ def __init__(self, field, text, caseless):
+ super(notendswithExpression, self).__init__(field, text, caseless)
+
+
+ def operator(self):
+ return "does not end with"
+
+
+
+class inExpression(textcompareExpression):
+ """
+ Text IN (exact string match to one of the supplied items) expression.
+ """
+
+ def __init__(self, field, text_list, caseless):
+ super(inExpression, self).__init__(field, text_list, caseless)
+
+
+ def operator(self):
+ return "in"
+
+
+ def __str__(self):
+ return self.operator() + "(" + self.field + ", " + str(self.text) + ", " + str(self.caseless) + ")"
+
+
+
+class notinExpression(textcompareExpression):
+ """
+ Text NOT IN (exact string match to none of the supplied items) expression.
+ """
+
+ def __init__(self, field, text, caseless):
+ super(notinExpression, self).__init__(field, text, caseless)
+
+
+ def operator(self):
+ return "not in"
+
+
+ def __str__(self):
+ return self.operator() + "(" + self.field + ", " + str(self.text) + ", " + str(self.caseless) + ")"
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/filegenerator.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/filegenerator.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/filegenerator.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,322 @@
+##
+# Copyright (c) 2006-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from txdav.common.datastore.query import expression
+
+"""
+SQLLite statement generator from query expressions.
+"""
+
+__all__ = [
+ "sqllitegenerator",
+]
+
+import cStringIO as StringIO
+
+class sqllitegenerator(object):
+
+ FROM = " from "
+ WHERE = " where "
+ RESOURCEDB = "RESOURCE"
+ TIMESPANDB = "TIMESPAN"
+ TRANSPARENCYDB = "TRANSPARENCY"
+ PERUSERDB = "PERUSER"
+ NOTOP = "NOT "
+ ANDOP = " AND "
+ OROP = " OR "
+ CONTAINSOP = " GLOB "
+ NOTCONTAINSOP = " NOT GLOB "
+ ISOP = " == "
+ ISNOTOP = " != "
+ STARTSWITHOP = " GLOB "
+ NOTSTARTSWITHOP = " NOT GLOB "
+ ENDSWITHOP = " GLOB "
+ NOTENDSWITHOP = " NOT GLOB "
+ INOP = " IN "
+ NOTINOP = " NOT IN "
+
+ FIELDS = {
+ "TYPE": "RESOURCE.TYPE",
+ "UID": "RESOURCE.UID",
+ }
+
+ TIMESPANTEST = "((TIMESPAN.FLOAT == 'N' AND TIMESPAN.START < %s AND TIMESPAN.END > %s) OR (TIMESPAN.FLOAT == 'Y' AND TIMESPAN.START < %s AND TIMESPAN.END > %s))"
+ TIMESPANTEST_NOEND = "((TIMESPAN.FLOAT == 'N' AND TIMESPAN.END > %s) OR (TIMESPAN.FLOAT == 'Y' AND TIMESPAN.END > %s))"
+ TIMESPANTEST_NOSTART = "((TIMESPAN.FLOAT == 'N' AND TIMESPAN.START < %s) OR (TIMESPAN.FLOAT == 'Y' AND TIMESPAN.START < %s))"
+ TIMESPANTEST_TAIL_PIECE = " AND TIMESPAN.RESOURCEID == RESOURCE.RESOURCEID"
+ TIMESPANTEST_JOIN_ON_PIECE = "TIMESPAN.INSTANCEID == TRANSPARENCY.INSTANCEID AND TRANSPARENCY.PERUSERID == %s"
+
+ def __init__(self, expr, calendarid, userid, freebusy=False):
+ """
+
+ @param expr: the query expression object model
+ @type expr: L{Filter}
+ @param calendarid: resource ID - not used for file-based per-calendar indexes
+ @type calendarid: C{int}
+ @param userid: user for whom query is being done - query will be scoped to that user's privileges and their transparency
+ @type userid: C{str}
+ @param freebusy: whether or not a freebusy query is being done - if it is, additional time range and transparency information is returned
+ @type freebusy: C{bool}
+ """
+ self.expression = expr
+ self.calendarid = calendarid
+ self.userid = userid if userid else ""
+ self.freebusy = freebusy
+ self.usedtimespan = False
+
+
+ def generate(self):
+ """
+ Generate the actual SQL 'where ...' expression from the passed in expression tree.
+
+ @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the partial SQL statement,
+ and the C{list} is the list of argument substitutions to use with the SQL API execute method.
+ """
+
+ # Init state
+ self.sout = StringIO.StringIO()
+ self.arguments = []
+ self.substitutions = []
+ self.usedtimespan = False
+
+ # Generate ' where ...' partial statement
+ self.generateExpression(self.expression)
+
+ # Prefix with ' from ...' partial statement
+ select = self.FROM + self.RESOURCEDB
+ if self.usedtimespan:
+
+ # Free busy needs transparency join
+ if self.freebusy:
+ self.frontArgument(self.userid)
+ select += ", %s LEFT OUTER JOIN %s ON (%s)" % (
+ self.TIMESPANDB,
+ self.TRANSPARENCYDB,
+ self.TIMESPANTEST_JOIN_ON_PIECE
+ )
+ else:
+ select += ", %s" % (
+ self.TIMESPANDB,
+ )
+ select += self.WHERE
+ if self.usedtimespan:
+ select += "("
+ select += self.sout.getvalue()
+ if self.usedtimespan:
+ if self.calendarid:
+ self.setArgument(self.calendarid)
+ select += ")%s" % (self.TIMESPANTEST_TAIL_PIECE,)
+
+ select = select % tuple(self.substitutions)
+
+ return select, self.arguments
+
+
+ def generateExpression(self, expr):
+ """
+ Generate an expression and all it's subexpressions.
+
+ @param expr: the L{baseExpression} derived class to write out.
+ @return: C{True} if the TIMESPAN table is used, C{False} otherwise.
+ """
+
+ # Generate based on each type of expression we might encounter
+
+ # ALL
+ if isinstance(expr, expression.allExpression):
+ # Wipe out the ' where ...' clause so everything is matched
+ self.sout.truncate(0)
+ self.arguments = []
+ self.substitutions = []
+ self.usedtimespan = False
+
+ # NOT
+ elif isinstance(expr, expression.notExpression):
+ self.sout.write(self.NOTOP)
+ self.generateSubExpression(expr.expressions[0])
+
+ # AND
+ elif isinstance(expr, expression.andExpression):
+ first = True
+ for e in expr.expressions:
+ if first:
+ first = False
+ else:
+ self.sout.write(self.ANDOP)
+ self.generateSubExpression(e)
+
+ # OR
+ elif isinstance(expr, expression.orExpression):
+ first = True
+ for e in expr.expressions:
+ if first:
+ first = False
+ else:
+ self.sout.write(self.OROP)
+ self.generateSubExpression(e)
+
+ # time-range
+ elif isinstance(expr, expression.timerangeExpression):
+ if expr.start and expr.end:
+ self.setArgument(expr.end)
+ self.setArgument(expr.start)
+ self.setArgument(expr.endfloat)
+ self.setArgument(expr.startfloat)
+ test = self.TIMESPANTEST
+ elif expr.start and expr.end is None:
+ self.setArgument(expr.start)
+ self.setArgument(expr.startfloat)
+ test = self.TIMESPANTEST_NOEND
+ elif not expr.start and expr.end:
+ self.setArgument(expr.end)
+ self.setArgument(expr.endfloat)
+ test = self.TIMESPANTEST_NOSTART
+
+ self.sout.write(test)
+ self.usedtimespan = True
+
+ # CONTAINS
+ elif isinstance(expr, expression.containsExpression):
+ self.sout.write(expr.field)
+ self.sout.write(self.CONTAINSOP)
+ self.addArgument(self.containsArgument(expr.text))
+
+ # NOT CONTAINS
+ elif isinstance(expr, expression.notcontainsExpression):
+ self.sout.write(expr.field)
+ self.sout.write(self.NOTCONTAINSOP)
+ self.addArgument(self.containsArgument(expr.text))
+
+ # IS
+ elif isinstance(expr, expression.isExpression):
+ self.sout.write(expr.field)
+ self.sout.write(self.ISOP)
+ self.addArgument(expr.text)
+
+ # IS NOT
+ elif isinstance(expr, expression.isnotExpression):
+ self.sout.write(expr.field)
+ self.sout.write(self.ISNOTOP)
+ self.addArgument(expr.text)
+
+ # STARTSWITH
+ elif isinstance(expr, expression.startswithExpression):
+ self.sout.write(expr.field)
+ self.sout.write(self.STARTSWITHOP)
+ self.addArgument(self.startswithArgument(expr.text))
+
+ # NOT STARTSWITH
+ elif isinstance(expr, expression.notstartswithExpression):
+ self.sout.write(expr.field)
+ self.sout.write(self.NOTSTARTSWITHOP)
+ self.addArgument(self.startswithArgument(expr.text))
+
+ # ENDSWITH
+ elif isinstance(expr, expression.endswithExpression):
+ self.sout.write(expr.field)
+ self.sout.write(self.ENDSWITHOP)
+ self.addArgument(self.endswithArgument(expr.text))
+
+ # NOT ENDSWITH
+ elif isinstance(expr, expression.notendswithExpression):
+ self.sout.write(expr.field)
+ self.sout.write(self.NOTENDSWITHOP)
+ self.addArgument(self.endswithArgument(expr.text))
+
+ # IN
+ elif isinstance(expr, expression.inExpression):
+ self.sout.write(expr.field)
+ self.sout.write(self.INOP)
+ self.sout.write("(")
+ for count, item in enumerate(expr.text):
+ if count != 0:
+ self.sout.write(", ")
+ self.addArgument(item)
+ self.sout.write(")")
+
+ # NOT IN
+ elif isinstance(expr, expression.notinExpression):
+ self.sout.write(expr.field)
+ self.sout.write(self.NOTINOP)
+ self.sout.write("(")
+ for count, item in enumerate(expr.text):
+ if count != 0:
+ self.sout.write(", ")
+ self.addArgument(item)
+ self.sout.write(")")
+
+
+ def generateSubExpression(self, expression):
+ """
+ Generate an SQL expression possibly in parenthesis if its a compound expression.
+
+ @param expression: the L{baseExpression} to write out.
+ @return: C{True} if the TIMESPAN table is used, C{False} otherwise.
+ """
+
+ if expression.multi():
+ self.sout.write("(")
+ self.generateExpression(expression)
+ if expression.multi():
+ self.sout.write(")")
+
+
+ def addArgument(self, arg):
+ """
+
+ @param arg: the C{str} of the argument to add
+ """
+
+ # Append argument to the list and add the appropriate substitution string to the output stream.
+ self.arguments.append(arg)
+ self.substitutions.append(":" + str(len(self.arguments)))
+ self.sout.write("%s")
+
+
+ def setArgument(self, arg):
+ """
+
+ @param arg: the C{str} of the argument to add
+ @return: C{str} for argument substitution text
+ """
+
+ # Append argument to the list and add the appropriate substitution string to the output stream.
+ self.arguments.append(arg)
+ self.substitutions.append(":" + str(len(self.arguments)))
+
+
+ def frontArgument(self, arg):
+ """
+
+ @param arg: the C{str} of the argument to add
+ @return: C{str} for argument substitution text
+ """
+
+ # Append argument to the list and add the appropriate substitution string to the output stream.
+ self.arguments.insert(0, arg)
+ self.substitutions.append(":" + str(len(self.arguments)))
+
+
+ def containsArgument(self, arg):
+ return "*%s*" % (arg,)
+
+
+ def startswithArgument(self, arg):
+ return "%s*" % (arg,)
+
+
+ def endswithArgument(self, arg):
+ return "*%s" % (arg,)
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/generator.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/generator.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/generator.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,164 @@
+##
+# Copyright (c) 2006-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.enterprise.dal.syntax import Select, Parameter, Not
+from txdav.common.datastore.query import expression
+
+"""
+SQL statement generator from query expressions.
+"""
+
+__all__ = [
+ "SQLQueryGenerator",
+]
+
+class SQLQueryGenerator(object):
+
+ def __init__(self, expr, collection, whereid):
+ """
+
+ @param expr: the query expression object model
+ @type expr: L{expression}
+ @param collection: the resource targeted by the query
+ @type collection: L{CommonHomeChild}
+ """
+ self.expression = expr
+ self.collection = collection
+ self.whereid = whereid
+
+
+ def generate(self):
+ """
+ Generate the actual SQL statement from the passed in expression tree.
+
+ @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the partial SQL statement,
+ and the C{list} is the list of argument substitutions to use with the SQL API execute method.
+ """
+
+ # Init state
+ self.arguments = {}
+ self.argcount = 0
+ obj = self.collection._objectSchema
+
+ columns = [obj.RESOURCE_NAME, obj.UID]
+
+ # For SQL data DB we need to restrict the query to just the targeted collection resource-id if provided
+ if self.whereid:
+ # AND the whole thing
+ test = expression.isExpression(obj.PARENT_RESOURCE_ID, self.whereid, True)
+ self.expression = test if isinstance(self.expression, expression.allExpression) else test.andWith(self.expression)
+
+ # Generate ' where ...' partial statement
+ where = self.generateExpression(self.expression)
+
+ select = Select(
+ columns,
+ From=obj,
+ Where=where,
+ Distinct=True,
+ )
+
+ return select, self.arguments
+
+
+ def generateExpression(self, expr):
+ """
+ Generate an expression and all it's subexpressions.
+
+ @param expr: the L{baseExpression} derived class to write out.
+ """
+
+ # Generate based on each type of expression we might encounter
+ partial = None
+
+ # ALL
+ if isinstance(expr, expression.allExpression):
+ # Everything is matched
+ partial = None
+ self.arguments = {}
+
+ # NOT
+ elif isinstance(expr, expression.notExpression):
+ partial = Not(self.generateExpression(expr.expressions[0]))
+
+ # AND
+ elif isinstance(expr, expression.andExpression):
+ for e in expr.expressions:
+ next = self.generateExpression(e)
+ partial = partial.And(next) if partial is not None else next
+
+ # OR
+ elif isinstance(expr, expression.orExpression):
+ for e in expr.expressions:
+ next = self.generateExpression(e)
+ partial = partial.Or(next) if partial is not None else next
+
+ # CONTAINS
+ elif isinstance(expr, expression.containsExpression):
+ partial = expr.field.Contains(expr.text)
+
+ # NOT CONTAINS
+ elif isinstance(expr, expression.notcontainsExpression):
+ partial = expr.field.NotContains(expr.text)
+
+ # IS
+ elif isinstance(expr, expression.isExpression):
+ partial = expr.field == expr.text
+
+ # IS NOT
+ elif isinstance(expr, expression.isnotExpression):
+ partial = expr.field != expr.text
+
+ # STARTSWITH
+ elif isinstance(expr, expression.startswithExpression):
+ partial = expr.field.StartsWith(expr.text)
+
+ # NOT STARTSWITH
+ elif isinstance(expr, expression.notstartswithExpression):
+ partial = expr.field.NotStartsWith(expr.text)
+
+ # ENDSWITH
+ elif isinstance(expr, expression.endswithExpression):
+ partial = expr.field.EndsWith(expr.text)
+
+ # NOT ENDSWITH
+ elif isinstance(expr, expression.notendswithExpression):
+ partial = expr.field.NotEndsWith(expr.text)
+
+ # IN
+ elif isinstance(expr, expression.inExpression):
+ argname = self.addArgument(expr.text)
+ partial = expr.field.In(Parameter(argname, len(expr.text)))
+
+ # NOT IN
+ elif isinstance(expr, expression.notinExpression):
+ argname = self.addArgument(expr.text)
+ partial = expr.field.NotIn(Parameter(argname, len(expr.text)))
+
+ return partial
+
+
+ def addArgument(self, arg):
+ """
+
+ @param arg: the C{str} of the argument to add
+ """
+
+ # Append argument to the list and add the appropriate substitution string to the output stream.
+ self.argcount += 1
+ argname = "arg{}".format(self.argcount)
+ self.arguments[argname] = arg
+ return argname
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/__init__.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/__init__.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_expression.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_expression.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_expression.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,167 @@
+##
+# Copyright (c) 2011-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from txdav.common.datastore.query import expression
+from twisted.trial.unittest import TestCase
+
+class Tests(TestCase):
+
+ def test_andWith(self):
+
+ tests = (
+ (
+ expression.isExpression("A", "1", True),
+ expression.isExpression("B", "2", True),
+ "(is(A, 1, True) AND is(B, 2, True))"
+ ),
+ (
+ expression.isExpression("A", "1", True),
+ expression.andExpression((
+ expression.isExpression("B", "2", True),
+ )),
+ "(is(A, 1, True) AND is(B, 2, True))"
+ ),
+ (
+ expression.isExpression("A", "1", True),
+ expression.andExpression((
+ expression.isExpression("B", "2", True),
+ expression.isExpression("C", "3", True),
+ )),
+ "(is(A, 1, True) AND is(B, 2, True) AND is(C, 3, True))"
+ ),
+ (
+ expression.isExpression("A", "1", True),
+ expression.orExpression((
+ expression.isExpression("B", "2", True),
+ )),
+ "(is(A, 1, True) AND is(B, 2, True))"
+ ),
+ (
+ expression.isExpression("A", "1", True),
+ expression.orExpression((
+ expression.isExpression("B", "2", True),
+ expression.isExpression("C", "3", True),
+ )),
+ "(is(A, 1, True) AND (is(B, 2, True) OR is(C, 3, True)))"
+ ),
+ (
+ expression.andExpression((
+ expression.isExpression("A", "1", True),
+ )),
+ expression.isExpression("B", "2", True),
+ "(is(A, 1, True) AND is(B, 2, True))"
+ ),
+ (
+ expression.andExpression((
+ expression.isExpression("A", "1", True),
+ expression.isExpression("B", "2", True),
+ )),
+ expression.isExpression("C", "3", True),
+ "(is(A, 1, True) AND is(B, 2, True) AND is(C, 3, True))"
+ ),
+ (
+ expression.orExpression((
+ expression.isExpression("A", "1", True),
+ )),
+ expression.isExpression("B", "2", True),
+ "(is(A, 1, True) AND is(B, 2, True))"
+ ),
+ (
+ expression.orExpression((
+ expression.isExpression("A", "1", True),
+ expression.isExpression("B", "2", True),
+ )),
+ expression.isExpression("C", "3", True),
+ "((is(A, 1, True) OR is(B, 2, True)) AND is(C, 3, True))"
+ ),
+ )
+
+ for expr1, expr2, result in tests:
+ self.assertEqual(str(expr1.andWith(expr2)), result, msg="Failed on %s" % (result,))
+
+
+ def test_orWith(self):
+
+ tests = (
+ (
+ expression.isExpression("A", "1", True),
+ expression.isExpression("B", "2", True),
+ "(is(A, 1, True) OR is(B, 2, True))"
+ ),
+ (
+ expression.isExpression("A", "1", True),
+ expression.andExpression((
+ expression.isExpression("B", "2", True),
+ )),
+ "(is(A, 1, True) OR is(B, 2, True))"
+ ),
+ (
+ expression.isExpression("A", "1", True),
+ expression.andExpression((
+ expression.isExpression("B", "2", True),
+ expression.isExpression("C", "3", True),
+ )),
+ "(is(A, 1, True) OR (is(B, 2, True) AND is(C, 3, True)))"
+ ),
+ (
+ expression.isExpression("A", "1", True),
+ expression.orExpression((
+ expression.isExpression("B", "2", True),
+ )),
+ "(is(A, 1, True) OR is(B, 2, True))"
+ ),
+ (
+ expression.isExpression("A", "1", True),
+ expression.orExpression((
+ expression.isExpression("B", "2", True),
+ expression.isExpression("C", "3", True),
+ )),
+ "(is(A, 1, True) OR is(B, 2, True) OR is(C, 3, True))"
+ ),
+ (
+ expression.andExpression((
+ expression.isExpression("A", "1", True),
+ )),
+ expression.isExpression("B", "2", True),
+ "(is(A, 1, True) OR is(B, 2, True))"
+ ),
+ (
+ expression.andExpression((
+ expression.isExpression("A", "1", True),
+ expression.isExpression("B", "2", True),
+ )),
+ expression.isExpression("C", "3", True),
+ "((is(A, 1, True) AND is(B, 2, True)) OR is(C, 3, True))"
+ ),
+ (
+ expression.orExpression((
+ expression.isExpression("A", "1", True),
+ )),
+ expression.isExpression("B", "2", True),
+ "(is(A, 1, True) OR is(B, 2, True))"
+ ),
+ (
+ expression.orExpression((
+ expression.isExpression("A", "1", True),
+ expression.isExpression("B", "2", True),
+ )),
+ expression.isExpression("C", "3", True),
+ "(is(A, 1, True) OR is(B, 2, True) OR is(C, 3, True))"
+ ),
+ )
+
+ for expr1, expr2, result in tests:
+ self.assertEqual(str(expr1.orWith(expr2)), result, msg="Failed on %s" % (result,))
Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_generator.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_generator.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_generator.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,127 @@
+##
+# Copyright (c) 2012-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.enterprise.dal.syntax import SQLFragment, Parameter
+from txdav.common.datastore.query.generator import SQLQueryGenerator
+from txdav.common.datastore.query import expression
+
+"""
+Tests for L{txdav.common.datastore.sql}.
+"""
+
+from twisted.trial.unittest import TestCase
+
+from txdav.common.datastore.sql_tables import schema
+
+class SQLQueryGeneratorTests(TestCase):
+ """
+ Tests for shared functionality in L{txdav.common.datastore.sql}.
+ """
+
+ class FakeHomeChild(object):
+ _objectSchema = schema.CALENDAR_OBJECT
+
+ def id(self):
+ return 1234
+
+
+ def test_all_query(self):
+
+ expr = expression.allExpression()
+ resource = self.FakeHomeChild()
+ select, args = SQLQueryGenerator(expr, resource, resource.id()).generate()
+ self.assertEqual(select.toSQL(), SQLFragment("select distinct RESOURCE_NAME, ICALENDAR_UID from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ?", [1234]))
+ self.assertEqual(args, {})
+
+
+ def test_uid_query(self):
+
+ resource = self.FakeHomeChild()
+ obj = resource._objectSchema
+ expr = expression.isExpression(obj.UID, 5678, False)
+ select, args = SQLQueryGenerator(expr, resource, resource.id()).generate()
+ self.assertEqual(select.toSQL(), SQLFragment("select distinct RESOURCE_NAME, ICALENDAR_UID from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ? and ICALENDAR_UID = ?", [1234, 5678]))
+ self.assertEqual(args, {})
+
+
+ def test_or_query(self):
+
+ resource = self.FakeHomeChild()
+ obj = resource._objectSchema
+ expr = expression.orExpression((
+ expression.isExpression(obj.UID, 5678, False),
+ expression.isnotExpression(obj.RESOURCE_NAME, "foobar.ics", False),
+ ))
+ select, args = SQLQueryGenerator(expr, resource, resource.id()).generate()
+ self.assertEqual(
+ select.toSQL(),
+ SQLFragment(
+ "select distinct RESOURCE_NAME, ICALENDAR_UID from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ? and (ICALENDAR_UID = ? or RESOURCE_NAME != ?)",
+ [1234, 5678, "foobar.ics"]
+ )
+ )
+ self.assertEqual(args, {})
+
+
+ def test_and_query(self):
+
+ resource = self.FakeHomeChild()
+ obj = resource._objectSchema
+ expr = expression.andExpression((
+ expression.isExpression(obj.UID, 5678, False),
+ expression.isnotExpression(obj.RESOURCE_NAME, "foobar.ics", False),
+ ))
+ select, args = SQLQueryGenerator(expr, resource, resource.id()).generate()
+ self.assertEqual(
+ select.toSQL(),
+ SQLFragment(
+ "select distinct RESOURCE_NAME, ICALENDAR_UID from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ? and ICALENDAR_UID = ? and RESOURCE_NAME != ?",
+ [1234, 5678, "foobar.ics"]
+ )
+ )
+ self.assertEqual(args, {})
+
+
+ def test_not_query(self):
+
+ resource = self.FakeHomeChild()
+ obj = resource._objectSchema
+ expr = expression.notExpression(expression.isExpression(obj.UID, 5678, False))
+ select, args = SQLQueryGenerator(expr, resource, resource.id()).generate()
+ self.assertEqual(
+ select.toSQL(),
+ SQLFragment(
+ "select distinct RESOURCE_NAME, ICALENDAR_UID from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ? and not ICALENDAR_UID = ?",
+ [1234, 5678]
+ )
+ )
+ self.assertEqual(args, {})
+
+
+ def test_in_query(self):
+
+ resource = self.FakeHomeChild()
+ obj = resource._objectSchema
+ expr = expression.inExpression(obj.RESOURCE_NAME, ["1.ics", "2.ics", "3.ics"], False)
+ select, args = SQLQueryGenerator(expr, resource, resource.id()).generate()
+ self.assertEqual(
+ select.toSQL(),
+ SQLFragment(
+ "select distinct RESOURCE_NAME, ICALENDAR_UID from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ? and RESOURCE_NAME in (?, ?, ?)",
+ [1234, Parameter('arg1', 3)]
+ )
+ )
+ self.assertEqual(args, {"arg1": ["1.ics", "2.ics", "3.ics"]})
Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -67,7 +67,8 @@
_BIND_MODE_INDIRECT, _HOME_STATUS_NORMAL, _HOME_STATUS_EXTERNAL
from txdav.common.datastore.sql_tables import schema, splitSQLString
from txdav.common.icommondatastore import ConcurrentModification, \
- RecordNotAllowedError, ExternalShareFailed, ShareNotAllowed
+ RecordNotAllowedError, ExternalShareFailed, ShareNotAllowed, \
+ IndexedSearchException
from txdav.common.icommondatastore import HomeChildNameNotAllowedError, \
HomeChildNameAlreadyExistsError, NoSuchHomeChildError, \
ObjectResourceNameNotAllowedError, ObjectResourceNameAlreadyExistsError, \
@@ -3224,8 +3225,8 @@
ownerView = yield self.ownerView()
if self.direct():
yield ownerView.removeShare(self)
- if not ownerView.external():
- yield self._removeExternalInvite(ownerView)
+ if ownerView.external():
+ yield self._removeExternalInvite()
else:
yield self.declineShare()
@@ -4129,9 +4130,7 @@
else:
self._notifiers = None
- self._index = None # Derived classes need to set this
-
def memoMe(self, key, memo): #@UnusedVariable
"""
Add this object to the memo dictionary in whatever fashion is appropriate.
@@ -4356,10 +4355,6 @@
return self._txn.store().directoryService()
- def retrieveOldIndex(self):
- return self._index
-
-
def __repr__(self):
return "<%s: %s>" % (self.__class__.__name__, self._resourceID)
@@ -4803,6 +4798,21 @@
return super(CommonHomeChild, self).resourceNamesSinceRevision(revision)
+ def search(self, filter):
+ """
+ Do a query of the contents of this collection.
+
+ @param filter: the query filter to use
+ @type filter: L{Filter}
+
+ @return: the names of the matching resources
+ @rtype: C{list}
+ """
+
+ # This implementation raises - sub-classes override to do the actual query
+ raise IndexedSearchException()
+
+
@inlineCallbacks
def _loadPropertyStore(self, props=None):
if props is None:
Deleted: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql_legacy.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql_legacy.py 2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql_legacy.py 2013-12-19 18:28:54 UTC (rev 12144)
@@ -1,886 +0,0 @@
-# -*- test-case-name: twistedcaldav.test.test_sharing,twistedcaldav.test.test_calendarquery -*-
-##
-# Copyright (c) 2010-2013 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
-
-
-"""
-PostgreSQL data store.
-"""
-
-import StringIO
-
-
-from twisted.python import hashlib
-from twisted.internet.defer import succeed, inlineCallbacks, returnValue
-
-from twistedcaldav.config import config
-from twistedcaldav.dateops import normalizeForIndex, pyCalendarTodatetime
-from twistedcaldav.memcachepool import CachePoolUserMixIn
-from twistedcaldav.query import \
- calendarqueryfilter, calendarquery, addressbookquery, expression, \
- addressbookqueryfilter
-from twistedcaldav.query.sqlgenerator import sqlgenerator
-
-from txdav.caldav.icalendarstore import TimeRangeLowerLimit, TimeRangeUpperLimit
-from txdav.common.icommondatastore import IndexedSearchException, \
- ReservationError, NoSuchObjectResourceError
-
-from txdav.common.datastore.sql_tables import schema
-from twext.enterprise.dal.syntax import Parameter, Select
-from twext.python.clsprop import classproperty
-from twext.python.log import Logger
-
-from pycalendar.datetime import DateTime
-from pycalendar.duration import Duration
-
-log = Logger()
-
-indexfbtype_to_icalfbtype = {
- 0: '?',
- 1: 'F',
- 2: 'B',
- 3: 'U',
- 4: 'T',
-}
-
-class MemcachedUIDReserver(CachePoolUserMixIn):
- log = Logger()
-
- def __init__(self, index, cachePool=None):
- self.index = index
- self._cachePool = cachePool
-
-
- def _key(self, uid):
- return 'reservation:%s' % (
- hashlib.md5('%s:%s' % (uid,
- self.index.resource._resourceID)).hexdigest())
-
-
- def reserveUID(self, uid):
- self.log.debug("Reserving UID %r @ %r" % (
- uid,
- self.index.resource))
-
- def _handleFalse(result):
- if result is False:
- raise ReservationError(
- "UID %s already reserved for calendar collection %s."
- % (uid, self.index.resource._name)
- )
-
- d = self.getCachePool().add(self._key(uid),
- 'reserved',
- expireTime=config.UIDReservationTimeOut)
- d.addCallback(_handleFalse)
- return d
-
-
- def unreserveUID(self, uid):
- self.log.debug("Unreserving UID %r @ %r" % (
- uid,
- self.index.resource))
-
- def _handleFalse(result):
- if result is False:
- raise ReservationError(
- "UID %s is not reserved for calendar collection %s."
- % (uid, self.index.resource._resourceID)
- )
-
- d = self.getCachePool().delete(self._key(uid))
- d.addCallback(_handleFalse)
- return d
-
-
- def isReservedUID(self, uid):
- self.log.debug("Is reserved UID %r @ %r" % (
- uid,
- self.index.resource))
-
- def _checkValue((flags, value)):
- if value is None:
- return False
- else:
- return True
-
- d = self.getCachePool().get(self._key(uid))
- d.addCallback(_checkValue)
- return d
-
-
-
-class DummyUIDReserver(object):
- log = Logger()
-
- def __init__(self, index):
- self.index = index
- self.reservations = set()
-
-
- def _key(self, uid):
- return 'reservation:%s' % (
- hashlib.md5('%s:%s' % (uid,
- self.index.resource._resourceID)).hexdigest())
-
-
- def reserveUID(self, uid):
- self.log.debug("Reserving UID %r @ %r" % (
- uid,
- self.index.resource))
-
- key = self._key(uid)
- if key in self.reservations:
- raise ReservationError(
- "UID %s already reserved for calendar collection %s."
- % (uid, self.index.resource._name)
- )
- self.reservations.add(key)
- return succeed(None)
-
-
- def unreserveUID(self, uid):
- self.log.debug("Unreserving UID %r @ %r" % (
- uid,
- self.index.resource))
-
- key = self._key(uid)
- if key in self.reservations:
- self.reservations.remove(key)
- return succeed(None)
-
-
- def isReservedUID(self, uid):
- self.log.debug("Is reserved UID %r @ %r" % (
- uid,
- self.index.resource))
- key = self._key(uid)
- return succeed(key in self.reservations)
-
-
-
-class RealSQLBehaviorMixin(object):
- """
- Class attributes for 'real' SQL behavior; avoid idiosyncracies of SQLite,
- use standard SQL constructions, and depend on the full schema in
- sql_schema/current.sql rather than the partial one in twistedcaldav which depends
- on the placement of the database in the filesystem for some information.
- """
-
- ISOP = " = "
- STARTSWITHOP = ENDSWITHOP = CONTAINSOP = " LIKE "
- NOTSTARTSWITHOP = NOTENDSWITHOP = NOTCONTAINSOP = " NOT LIKE "
-
- def containsArgument(self, arg):
- return "%%%s%%" % (arg,)
-
-
- def startswithArgument(self, arg):
- return "%s%%" % (arg,)
-
-
- def endswithArgument(self, arg):
- return "%%%s" % (arg,)
-
-
-
-class CalDAVSQLBehaviorMixin(RealSQLBehaviorMixin):
- """
- Query generator for CalDAV indexed searches.
- """
-
- FIELDS = {
- "TYPE": "CALENDAR_OBJECT.ICALENDAR_TYPE",
- "UID": "CALENDAR_OBJECT.ICALENDAR_UID",
- }
- RESOURCEDB = "CALENDAR_OBJECT"
- TIMESPANDB = "TIME_RANGE"
-
- TIMESPANTEST = "((TIME_RANGE.FLOATING = FALSE AND TIME_RANGE.START_DATE < %s AND TIME_RANGE.END_DATE > %s) OR (TIME_RANGE.FLOATING = TRUE AND TIME_RANGE.START_DATE < %s AND TIME_RANGE.END_DATE > %s))"
- TIMESPANTEST_NOEND = "((TIME_RANGE.FLOATING = FALSE AND TIME_RANGE.END_DATE > %s) OR (TIME_RANGE.FLOATING = TRUE AND TIME_RANGE.END_DATE > %s))"
- TIMESPANTEST_NOSTART = "((TIME_RANGE.FLOATING = FALSE AND TIME_RANGE.START_DATE < %s) OR (TIME_RANGE.FLOATING = TRUE AND TIME_RANGE.START_DATE < %s))"
- TIMESPANTEST_TAIL_PIECE = " AND TIME_RANGE.CALENDAR_OBJECT_RESOURCE_ID = CALENDAR_OBJECT.RESOURCE_ID AND TIME_RANGE.CALENDAR_RESOURCE_ID = %s"
- TIMESPANTEST_JOIN_ON_PIECE = "TIME_RANGE.INSTANCE_ID = TRANSPARENCY.TIME_RANGE_INSTANCE_ID AND TRANSPARENCY.USER_ID = %s"
-
- def generate(self):
- """
- Generate the actual SQL 'where ...' expression from the passed in
- expression tree.
-
- @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the
- partial SQL statement, and the C{list} is the list of argument
- substitutions to use with the SQL API execute method.
- """
-
- # Init state
- self.sout = StringIO.StringIO()
- self.arguments = []
- self.substitutions = []
- self.usedtimespan = False
-
- # For SQL data DB we need to restrict the query to just the targeted calendar resource-id if provided
- if self.calendarid:
-
- test = expression.isExpression("CALENDAR_OBJECT.CALENDAR_RESOURCE_ID", str(self.calendarid), True)
-
- # Since timerange expression already have the calendar resource-id test in them, do not
- # add the additional term to those. When the additional term is added, add it as the first
- # component in the AND expression to hopefully get the DB to use its index first
-
- # Top-level timerange expression already has calendar resource-id restriction in it
- if isinstance(self.expression, expression.timerangeExpression):
- pass
-
- # Top-level OR - check each component
- elif isinstance(self.expression, expression.orExpression):
-
- def _hasTopLevelTimerange(testexpr):
- if isinstance(testexpr, expression.timerangeExpression):
- return True
- elif isinstance(testexpr, expression.andExpression):
- return any([isinstance(expr, expression.timerangeExpression) for expr in testexpr.expressions])
- else:
- return False
-
- hasTimerange = any([_hasTopLevelTimerange(expr) for expr in self.expression.expressions])
-
- if hasTimerange:
- # timerange expression forces a join on calendarid
- pass
- else:
- # AND the whole thing with calendarid
- self.expression = test.andWith(self.expression)
-
- # Top-level AND - only add additional expression if timerange not present
- elif isinstance(self.expression, expression.andExpression):
- hasTimerange = any([isinstance(expr, expression.timerangeExpression) for expr in self.expression.expressions])
- if not hasTimerange:
- # AND the whole thing
- self.expression = test.andWith(self.expression)
-
- # Just AND the entire thing
- else:
- self.expression = test.andWith(self.expression)
-
- # Generate ' where ...' partial statement
- self.generateExpression(self.expression)
-
- # Prefix with ' from ...' partial statement
- select = self.FROM + self.RESOURCEDB
- if self.usedtimespan:
-
- # Free busy needs transparency join
- if self.freebusy:
- self.frontArgument(self.userid)
- select += ", %s LEFT OUTER JOIN %s ON (%s)" % (
- self.TIMESPANDB,
- self.TRANSPARENCYDB,
- self.TIMESPANTEST_JOIN_ON_PIECE
- )
- else:
- select += ", %s" % (
- self.TIMESPANDB,
- )
- select += self.WHERE
- if self.usedtimespan:
- select += "("
- select += self.sout.getvalue()
- if self.usedtimespan:
- if self.calendarid:
- self.setArgument(self.calendarid)
- select += ")%s" % (self.TIMESPANTEST_TAIL_PIECE,)
-
- select = select % tuple(self.substitutions)
-
- return select, self.arguments
-
-
-
-class FormatParamStyleMixin(object):
- """
- Mixin for overriding methods on sqlgenerator that generate arguments
- according to format/pyformat rules rather than the base class's 'numeric'
- rules.
- """
-
- def addArgument(self, arg):
- self.arguments.append(arg)
- self.substitutions.append("%s")
- self.sout.write("%s")
-
-
- def setArgument(self, arg):
- self.arguments.append(arg)
- self.substitutions.append("%s")
-
-
- def frontArgument(self, arg):
- self.arguments.insert(0, arg)
- self.substitutions.insert(0, "%s")
-
-
-
-class postgresqlgenerator(FormatParamStyleMixin, CalDAVSQLBehaviorMixin,
- sqlgenerator):
- """
- Query generator for PostgreSQL indexed searches.
- """
-
-
-
-def fixbools(sqltext):
- return sqltext.replace("TRUE", "1").replace("FALSE", "0")
-
-
-
-class oraclesqlgenerator(CalDAVSQLBehaviorMixin, sqlgenerator):
- """
- Query generator for Oracle indexed searches.
- """
- TIMESPANTEST = fixbools(CalDAVSQLBehaviorMixin.TIMESPANTEST)
- TIMESPANTEST_NOEND = fixbools(CalDAVSQLBehaviorMixin.TIMESPANTEST_NOEND)
- TIMESPANTEST_NOSTART = fixbools(CalDAVSQLBehaviorMixin.TIMESPANTEST_NOSTART)
- TIMESPANTEST_TAIL_PIECE = fixbools(
- CalDAVSQLBehaviorMixin.TIMESPANTEST_TAIL_PIECE)
- TIMESPANTEST_JOIN_ON_PIECE = fixbools(
- CalDAVSQLBehaviorMixin.TIMESPANTEST_JOIN_ON_PIECE)
-
-
-
-class LegacyIndexHelper(object):
- log = Logger()
-
- @inlineCallbacks
- def isAllowedUID(self, uid, *names):
- """
- Checks to see whether to allow an operation which would add the
- specified UID to the index. Specifically, the operation may not
- violate the constraint that UIDs must be unique.
- @param uid: the UID to check
- @param names: the names of resources being replaced or deleted by the
- operation; UIDs associated with these resources are not checked.
- @return: True if the UID is not in the index and is not reserved,
- False otherwise.
- """
- rname = yield self.resourceNameForUID(uid)
- returnValue(rname is None or rname in names)
-
-
- def reserveUID(self, uid):
- return self.reserver.reserveUID(uid)
-
-
- def unreserveUID(self, uid):
- return self.reserver.unreserveUID(uid)
-
-
- def isReservedUID(self, uid):
- return self.reserver.isReservedUID(uid)
-
-
-
-class PostgresLegacyIndexEmulator(LegacyIndexHelper):
- """
- Emulator for L{twistedcaldv.index.Index} and
- L{twistedcaldv.index.IndexSchedule}.
- """
-
- def __init__(self, calendar):
- self.resource = self.calendar = calendar
- if (
- hasattr(config, "Memcached") and
- config.Memcached.Pools.Default.ClientEnabled
- ):
- self.reserver = MemcachedUIDReserver(self)
- else:
- # This is only used with unit tests
- self.reserver = DummyUIDReserver(self)
-
- _objectSchema = schema.CALENDAR_OBJECT
-
- @property
- def _txn(self):
- return self.calendar._txn
-
-
- @inlineCallbacks
- def isAllowedUID(self, uid, *names):
- """
- Checks to see whether to allow an operation which would add the
- specified UID to the index. Specifically, the operation may not
- violate the constraint that UIDs must be unique.
- @param uid: the UID to check
- @param names: the names of resources being replaced or deleted by the
- operation; UIDs associated with these resources are not checked.
- @return: True if the UID is not in the index and is not reserved,
- False otherwise.
- """
- rname = yield self.resourceNameForUID(uid)
- returnValue(rname is None or rname in names)
-
-
- @inlineCallbacks
- def resourceUIDForName(self, name):
- uid = yield self.calendar.resourceUIDForName(name)
- returnValue(uid)
-
-
- @inlineCallbacks
- def resourceNameForUID(self, uid):
- name = yield self.calendar.resourceNameForUID(uid)
- returnValue(name)
-
-
- @classproperty
- def _notExpandedWithinQuery(cls): #@NoSelf
- """
- DAL query to satisfy L{PostgresLegacyIndexEmulator.notExpandedBeyond}.
- """
- co = schema.CALENDAR_OBJECT
- return Select(
- [co.RESOURCE_NAME],
- From=co,
- Where=((co.RECURRANCE_MIN > Parameter("minDate"))
- .Or(co.RECURRANCE_MAX < Parameter("maxDate")))
- .And(co.CALENDAR_RESOURCE_ID == Parameter("resourceID"))
- )
-
-
- @inlineCallbacks
- def notExpandedWithin(self, minDate, maxDate):
- """
- Gives all resources which have not been expanded beyond a given date
- in the database. (Unused; see above L{postgresqlgenerator}.
- """
- returnValue([row[0] for row in (
- yield self._notExpandedWithinQuery.on(
- self._txn,
- minDate=pyCalendarTodatetime(normalizeForIndex(minDate)) if minDate is not None else None,
- maxDate=pyCalendarTodatetime(normalizeForIndex(maxDate)),
- resourceID=self.calendar._resourceID))]
- )
-
-
- @inlineCallbacks
- def reExpandResource(self, name, expand_start, expand_end):
- """
- Given a resource name, remove it from the database and re-add it
- with a longer expansion.
- """
- obj = yield self.calendar.calendarObjectWithName(name)
-
- # Use a new transaction to do this update quickly without locking the row for too long. However, the original
- # transaction may have the row locked, so use wait=False and if that fails, fall back to using the original txn.
-
- newTxn = obj.transaction().store().newTransaction()
- try:
- yield obj.lock(wait=False, txn=newTxn)
- except NoSuchObjectResourceError:
- yield newTxn.commit()
- returnValue(None)
- except:
- yield newTxn.abort()
- newTxn = None
-
- # Now do the re-expand using the appropriate transaction
- try:
- doExpand = False
- if newTxn is None:
- doExpand = True
- else:
- # We repeat this check because the resource may have been re-expanded by someone else
- rmin, rmax = (yield obj.recurrenceMinMax(txn=newTxn))
-
- # If the resource is not fully expanded, see if within the required range or not.
- # Note that expand_start could be None if no lower limit is applied, but expand_end will
- # never be None
- if rmax is not None and rmax < expand_end:
- doExpand = True
- if rmin is not None and expand_start is not None and rmin > expand_start:
- doExpand = True
-
- if doExpand:
- yield obj.updateDatabase(
- (yield obj.component()),
- expand_until=expand_end,
- reCreate=True,
- txn=newTxn,
- )
- finally:
- if newTxn is not None:
- yield newTxn.commit()
-
-
- @inlineCallbacks
- def testAndUpdateIndex(self, minDate, maxDate):
- # Find out if the index is expanded far enough
- names = yield self.notExpandedWithin(minDate, maxDate)
-
- # Actually expand recurrence max
- for name in names:
- self.log.info("Search falls outside range of index for %s %s to %s" %
- (name, minDate, maxDate))
- yield self.reExpandResource(name, minDate, maxDate)
-
-
- @inlineCallbacks
- def indexedSearch(self, filter, useruid='', fbtype=False):
- """
- Finds resources matching the given qualifiers.
-
- @param filter: the L{Filter} for the calendar-query to execute.
-
- @return: a L{Deferred} which fires with an iterable of tuples for each
- resource matching the given C{qualifiers}. The tuples are C{(name,
- uid, type)}, where C{name} is the resource name, C{uid} is the
- resource UID, and C{type} is the resource iCalendar component type.
- """
- # Detect which style of parameter-generation we're using. Naming is a
- # little off here, because the reason we're using the numeric one is
- # that it happens to be used by the oracle binding that we're using,
- # whereas the postgres binding happens to use the 'pyformat' (e.g. %s)
- # parameter style.
- if self.calendar._txn.paramstyle == 'numeric':
- generator = oraclesqlgenerator
- else:
- generator = postgresqlgenerator
- # Make sure we have a proper Filter element and get the partial SQL
- # statement to use.
- if isinstance(filter, calendarqueryfilter.Filter):
- qualifiers = calendarquery.sqlcalendarquery(
- filter, self.calendar._resourceID, useruid, fbtype,
- generator=generator
- )
- if qualifiers is not None:
-
- today = DateTime.getToday()
-
- # Determine how far we need to extend the current expansion of
- # events. If we have an open-ended time-range we will expand
- # one year past the start. That should catch bounded
- # recurrences - unbounded will have been indexed with an
- # "infinite" value always included.
- maxDate, isStartDate = filter.getmaxtimerange()
- if maxDate:
- maxDate = maxDate.duplicate()
- maxDate.offsetDay(1)
- maxDate.setDateOnly(True)
- upperLimit = today + Duration(days=config.FreeBusyIndexExpandMaxDays)
- if maxDate > upperLimit:
- raise TimeRangeUpperLimit(upperLimit)
- if isStartDate:
- maxDate += Duration(days=365)
-
- # Determine if the start date is too early for the restricted range we
- # are applying. If it is today or later we don't need to worry about truncation
- # in the past.
- minDate, _ignore_isEndDate = filter.getmintimerange()
- if minDate >= today:
- minDate = None
- if minDate is not None and config.FreeBusyIndexLowerLimitDays:
- truncateLowerLimit = today - Duration(days=config.FreeBusyIndexLowerLimitDays)
- if minDate < truncateLowerLimit:
- raise TimeRangeLowerLimit(truncateLowerLimit)
-
- if maxDate is not None or minDate is not None:
- yield self.testAndUpdateIndex(minDate, maxDate)
-
- else:
- # We cannot handle this filter in an indexed search
- raise IndexedSearchException()
- else:
- qualifiers = None
-
- # Perform the search
- if qualifiers is None:
- rowiter = yield self.bruteForceSearch()
- else:
- if fbtype:
- # For a free-busy time-range query we return all instances
- rowiter = yield self._txn.execSQL(
- """
- select DISTINCT
- CALENDAR_OBJECT.RESOURCE_NAME,
- CALENDAR_OBJECT.ICALENDAR_UID,
- CALENDAR_OBJECT.ICALENDAR_TYPE,
- CALENDAR_OBJECT.ORGANIZER,
- TIME_RANGE.FLOATING, TIME_RANGE.START_DATE,
- TIME_RANGE.END_DATE, TIME_RANGE.FBTYPE,
- TIME_RANGE.TRANSPARENT, TRANSPARENCY.TRANSPARENT
- """ +
- qualifiers[0],
- qualifiers[1]
- )
- else:
- rowiter = yield self._txn.execSQL(
- """
- select
- DISTINCT CALENDAR_OBJECT.RESOURCE_NAME,
- CALENDAR_OBJECT.ICALENDAR_UID,
- CALENDAR_OBJECT.ICALENDAR_TYPE
- """ +
- qualifiers[0],
- qualifiers[1]
- )
-
- # Check result for missing resources
-
- results = []
- for row in rowiter:
- if fbtype:
- row = list(row)
- row[4] = 'Y' if row[4] else 'N'
- row[7] = indexfbtype_to_icalfbtype[row[7]]
- if row[9] is not None:
- row[8] = row[9]
- row[8] = 'T' if row[8] else 'F'
- del row[9]
- results.append(row)
- returnValue(results)
-
-
- @classproperty
- def _bruteForceQuery(cls): #@NoSelf
- """
- DAL query for all C{CALENDAR_OBJECT} rows in the calendar represented by
- this index.
- """
- obj = cls._objectSchema
- return Select(
- [obj.RESOURCE_NAME, obj.ICALENDAR_UID, obj.ICALENDAR_TYPE],
- From=obj, Where=obj.PARENT_RESOURCE_ID == Parameter("resourceID")
- )
-
-
- def bruteForceSearch(self):
- return self._bruteForceQuery.on(
- self._txn, resourceID=self.resource._resourceID)
-
-
- @inlineCallbacks
- def resourcesExist(self, names):
- returnValue(list(set(names).intersection(
- set((yield self.calendar.listCalendarObjects())))))
-
-
- @classproperty
- def _resourceExistsQuery(cls): #@NoSelf
- """
- DAL query to determine whether a calendar object exists in the
- collection represented by this index.
- """
- obj = cls._objectSchema
- return Select(
- [obj.RESOURCE_NAME], From=obj,
- Where=(obj.RESOURCE_NAME == Parameter("name"))
- .And(obj.PARENT_RESOURCE_ID == Parameter("resourceID"))
- )
-
-
- @inlineCallbacks
- def resourceExists(self, name):
- returnValue((bool(
- (yield self._resourceExistsQuery.on(
- self._txn, name=name, resourceID=self.resource._resourceID))
- )))
-
-
-
-class PostgresLegacyInboxIndexEmulator(PostgresLegacyIndexEmulator):
- """
- UIDs need not be unique in the 'inbox' calendar, so override those
- behaviors intended to ensure that.
- """
-
- def isAllowedUID(self):
- return succeed(True)
-
-
- def reserveUID(self, uid):
- return succeed(None)
-
-
- def unreserveUID(self, uid):
- return succeed(None)
-
-
- def isReservedUID(self, uid):
- return succeed(False)
-
-
-
-# CARDDAV
-
-class CardDAVSQLBehaviorMixin(RealSQLBehaviorMixin):
- """
- Query generator for CardDAV indexed searches.
- """
-
- FIELDS = {
- "UID": "ADDRESSBOOK_OBJECT.VCARD_UID",
- }
- RESOURCEDB = "ADDRESSBOOK_OBJECT"
-
- def generate(self):
- """
- Generate the actual SQL 'where ...' expression from the passed in
- expression tree.
-
- @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the
- partial SQL statement, and the C{list} is the list of argument
- substitutions to use with the SQL API execute method.
- """
-
- # Init state
- self.sout = StringIO.StringIO()
- self.arguments = []
- self.substitutions = []
-
- # For SQL data DB we need to restrict the query to just the targeted calendar resource-id if provided
- if self.calendarid:
-
- # AND the whole thing
- test = expression.isExpression("ADDRESSBOOK_OBJECT.ADDRESSBOOK_HOME_RESOURCE_ID", str(self.calendarid), True)
- self.expression = test.andWith(self.expression)
-
- # Generate ' where ...' partial statement
- self.sout.write(self.WHERE)
- self.generateExpression(self.expression)
-
- # Prefix with ' from ...' partial statement
- select = self.FROM + self.RESOURCEDB
- select += self.sout.getvalue()
-
- select = select % tuple(self.substitutions)
-
- return select, self.arguments
-
-
-
-class postgresqladbkgenerator(FormatParamStyleMixin, CardDAVSQLBehaviorMixin, sqlgenerator):
- """
- Query generator for PostgreSQL indexed searches.
- """
-
-
-
-class oraclesqladbkgenerator(CardDAVSQLBehaviorMixin, sqlgenerator):
- """
- Query generator for Oracle indexed searches.
- """
-
-
-
-class PostgresLegacyABIndexEmulator(LegacyIndexHelper):
- """
- Emulator for L{twistedcaldv.index.Index} and
- L{twistedcaldv.index.IndexSchedule}.
- """
-
- _objectSchema = schema.ADDRESSBOOK_OBJECT
-
- def __init__(self, addressbook):
- self.resource = self.addressbook = addressbook
- if (
- hasattr(config, "Memcached") and
- config.Memcached.Pools.Default.ClientEnabled
- ):
- self.reserver = MemcachedUIDReserver(self)
- else:
- # This is only used with unit tests
- self.reserver = DummyUIDReserver(self)
-
-
- @property
- def _txn(self):
- return self.addressbook._txn
-
-
- @inlineCallbacks
- def resourceUIDForName(self, name):
- obj = yield self.addressbook.addressbookObjectWithName(name)
- if obj is None:
- returnValue(None)
- returnValue(obj.uid())
-
-
- @inlineCallbacks
- def resourceNameForUID(self, uid):
- obj = yield self.addressbook.addressbookObjectWithUID(uid)
- if obj is None:
- returnValue(None)
- returnValue(obj.name())
-
-
- def searchValid(self, filter):
- if isinstance(filter, addressbookqueryfilter.Filter):
- qualifiers = addressbookquery.sqladdressbookquery(filter)
- else:
- qualifiers = None
-
- return qualifiers is not None
-
-
- @inlineCallbacks
- def search(self, filter):
- """
- Finds resources matching the given qualifiers.
- @param filter: the L{Filter} for the addressbook-query to execute.
- @return: an iterable of tuples for each resource matching the
- given C{qualifiers}. The tuples are C{(name, uid, type)}, where
- C{name} is the resource name, C{uid} is the resource UID, and
- C{type} is the resource iCalendar component type.x
- """
- if self.addressbook._txn.paramstyle == 'numeric':
- generator = oraclesqladbkgenerator
- else:
- generator = postgresqladbkgenerator
- # Make sure we have a proper Filter element and get the partial SQL statement to use.
- if isinstance(filter, addressbookqueryfilter.Filter):
- qualifiers = addressbookquery.sqladdressbookquery(
- filter, self.addressbook._resourceID, generator=generator)
- else:
- qualifiers = None
- if qualifiers is not None:
- rowiter = yield self._txn.execSQL(
- "select DISTINCT ADDRESSBOOK_OBJECT.RESOURCE_NAME, ADDRESSBOOK_OBJECT.VCARD_UID" +
- qualifiers[0],
- qualifiers[1]
- )
- else:
- rowiter = yield Select(
- [self._objectSchema.RESOURCE_NAME,
- self._objectSchema.VCARD_UID],
- From=self._objectSchema,
- Where=self._objectSchema.ADDRESSBOOK_HOME_RESOURCE_ID ==
- self.addressbook._resourceID
- ).on(self.addressbook._txn)
-
- returnValue(list(rowiter))
-
-
- def indexedSearch(self, filter, useruid='', fbtype=False):
- """
- Always raise L{IndexedSearchException}, since these indexes are not
- fully implemented yet.
- """
- raise IndexedSearchException()
-
-
- @inlineCallbacks
- def resourcesExist(self, names):
- returnValue(list(set(names).intersection(
- set((yield self.addressbook.listAddressBookObjects())))))
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.macosforge.org/pipermail/calendarserver-changes/attachments/20140312/6df01b86/attachment.html>
More information about the calendarserver-changes
mailing list