[CalendarServer-changes] [12144] CalendarServer/branches/users/cdaboo/cross-pod-sharing

source_changes at macosforge.org source_changes at macosforge.org
Wed Mar 12 11:17:29 PDT 2014


Revision: 12144
          http://trac.calendarserver.org//changeset/12144
Author:   cdaboo at apple.com
Date:     2013-12-19 10:28:54 -0800 (Thu, 19 Dec 2013)
Log Message:
-----------
Get rid of sql_legacy, moving search directly onto store objects - will make cross-pod search api cleaner.

Modified Paths:
--------------
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/dbinspect.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/purge.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/calverify/accounts.xml
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/purge/accounts.xml
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_export.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_purge.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/syntax.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/test/test_sqlsyntax.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/opendirectorybacker.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/accounts.xml
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/augments.xml
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/test_directory.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_addressbook_query.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_calendar_query.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_common.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_multiget_common.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/storebridge.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_calendarquery.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_sharing.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_xml.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/index_file.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/scheduling/freebusy.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/sql.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_index_file.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_sql.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/index_file.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/sql.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/test/test_sql_sharing.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql.py

Added Paths:
-----------
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/__init__.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/builder.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/filter.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/generator.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/__init__.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/test_filter.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/__init__.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/builder.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/filter.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/__init__.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/test_filter.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/__init__.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/expression.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/filegenerator.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/generator.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/__init__.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_expression.py
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_generator.py

Removed Paths:
-------------
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/query/
    CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql_legacy.py

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/dbinspect.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/dbinspect.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/dbinspect.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -37,8 +37,8 @@
 from twistedcaldav.datafilters.peruserdata import PerUserDataFilter
 from twistedcaldav.directory import calendaruserproxy
 from twistedcaldav.directory.directory import DirectoryService
-from twistedcaldav.query import calendarqueryfilter
 from twistedcaldav.stdconfig import DEFAULT_CONFIG_FILE
+from txdav.caldav.datastore.query.filter import Filter
 from txdav.common.datastore.sql_tables import schema, _BIND_MODE_OWN
 from uuid import UUID
 import os
@@ -757,10 +757,10 @@
                           name="VCALENDAR",
                        )
                   )
-        filter = calendarqueryfilter.Filter(filter)
+        filter = Filter(filter)
         filter.settimezone(None)
 
-        matches = yield calendar._index.indexedSearch(filter, useruid=uid, fbtype=False)
+        matches = yield calendar.search(filter, useruid=uid, fbtype=False)
         if matches is None:
             returnValue(None)
         for name, _ignore_uid, _ignore_type in matches:

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/purge.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/purge.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/purge.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -30,10 +30,9 @@
 from twisted.internet.defer import inlineCallbacks, returnValue
 
 from twistedcaldav import caldavxml
-from twistedcaldav.caldavxml import TimeRange
 from twistedcaldav.directory.directory import DirectoryRecord
-from twistedcaldav.query import calendarqueryfilter
 
+from txdav.caldav.datastore.query.filter import Filter
 from txdav.xml import element as davxml
 
 
@@ -817,13 +816,13 @@
         query_filter = caldavxml.Filter(
               caldavxml.ComponentFilter(
                   caldavxml.ComponentFilter(
-                      TimeRange(start=whenString,),
+                      caldavxml.TimeRange(start=whenString,),
                       name=("VEVENT",),
                   ),
                   name="VCALENDAR",
                )
           )
-        query_filter = calendarqueryfilter.Filter(query_filter)
+        query_filter = Filter(query_filter)
 
         count = 0
         txn = self.store.newTransaction()
@@ -844,7 +843,7 @@
                     childNames.append(childName)
             else:
                 # events matching filter
-                for childName, _ignore_childUid, _ignore_childType in (yield calendar._index.indexedSearch(query_filter)):
+                for childName, _ignore_childUid, _ignore_childType in (yield calendar.search(query_filter)):
                     childNames.append(childName)
             yield txn.commit()
 

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/calverify/accounts.xml
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/calverify/accounts.xml	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/calverify/accounts.xml	2013-12-19 18:28:54 UTC (rev 12144)
@@ -47,4 +47,11 @@
     <name>Example User4</name>
     <email-address>example4 at example.com</email-address>
   </user>
+  <user>
+    <uid>home1</uid>
+    <guid>home1</guid>
+    <password>home1</password>
+    <name>Home 1</name>
+    <email-address>home1 at example.com</email-address>
+  </user>
 </accounts>

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/purge/accounts.xml
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/purge/accounts.xml	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/purge/accounts.xml	2013-12-19 18:28:54 UTC (rev 12144)
@@ -26,4 +26,25 @@
     <name>Example User</name>
     <email-address>example at example.com</email-address>
   </user>
+  <user>
+    <uid>example2</uid>
+    <guid>37DB0C90-4DB1-4932-BC69-3DAB66F374F5</guid>
+    <password>example2</password>
+    <name>Example User 2</name>
+    <email-address>example2 at example.com</email-address>
+  </user>
+  <user>
+    <uid>home1</uid>
+    <guid>home1</guid>
+    <password>home1</password>
+    <name>Home 1</name>
+    <email-address>home1 at example.com</email-address>
+  </user>
+  <user>
+    <uid>home2</uid>
+    <guid>home2</guid>
+    <password>home2</password>
+    <name>Home 2</name>
+    <email-address>home2 at example.com</email-address>
+  </user>
 </accounts>

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_export.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_export.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_export.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -277,7 +277,7 @@
         """
         yield populateCalendarsFrom(
             {
-                "home1": {
+                "user01": {
                     "calendar1": {
                         "valentines-day.ics": (valentines, {})
                     }
@@ -291,7 +291,7 @@
 
         io = StringIO()
         yield exportToFile(
-            [(yield self.txn().calendarHomeWithUID("home1"))
+            [(yield self.txn().calendarHomeWithUID("user01"))
               .calendarWithName("calendar1")], io
         )
         self.assertEquals(Component.fromString(io.getvalue()),
@@ -306,7 +306,7 @@
         """
         yield populateCalendarsFrom(
             {
-                "home1": {
+                "user01": {
                     "calendar1": {
                         "valentines-day.ics": (valentines, {}),
                         "new-years-day.ics": (newYears, {})
@@ -324,7 +324,7 @@
 
         io = StringIO()
         yield exportToFile(
-            [(yield self.txn().calendarHomeWithUID("home1"))
+            [(yield self.txn().calendarHomeWithUID("user01"))
               .calendarWithName("calendar1")], io
         )
         self.assertEquals(Component.fromString(io.getvalue()),
@@ -342,7 +342,7 @@
         """
         yield populateCalendarsFrom(
             {
-                "home1": {
+                "user01": {
                     "calendar1": {
                         "1.ics": (one, {}), # EST
                         "2.ics": (another, {}), # EST
@@ -354,7 +354,7 @@
 
         io = StringIO()
         yield exportToFile(
-            [(yield self.txn().calendarHomeWithUID("home1"))
+            [(yield self.txn().calendarHomeWithUID("user01"))
               .calendarWithName("calendar1")], io
         )
         result = Component.fromString(io.getvalue())

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_purge.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_purge.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/calendarserver/tools/test/test_purge.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -806,16 +806,6 @@
 
     @inlineCallbacks
     def setUp(self):
-        self.patch(config.DirectoryService.params, "xmlFile",
-            os.path.join(
-                os.path.dirname(__file__), "purge", "accounts.xml"
-            )
-        )
-        self.patch(config.ResourceService.params, "xmlFile",
-            os.path.join(
-                os.path.dirname(__file__), "purge", "resources.xml"
-            )
-        )
         yield super(PurgePrincipalTests, self).setUp()
 
         txn = self._sqlCalendarStore.newTransaction()
@@ -850,6 +840,20 @@
         (yield txn.commit())
 
 
+    def configure(self):
+        super(PurgePrincipalTests, self).configure()
+        self.patch(config.DirectoryService.params, "xmlFile",
+            os.path.join(
+                os.path.dirname(__file__), "purge", "accounts.xml"
+            )
+        )
+        self.patch(config.ResourceService.params, "xmlFile",
+            os.path.join(
+                os.path.dirname(__file__), "purge", "resources.xml"
+            )
+        )
+
+
     @inlineCallbacks
     def populate(self):
         yield populateCalendarsFrom(self.requirements, self.storeUnderTest())

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/syntax.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/syntax.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/syntax.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -347,29 +347,63 @@
         @param other: a constant parameter or sub-select
         @type other: L{Parameter} or L{Select}
         """
+        return self._commonIn('in', other)
+
+
+    def NotIn(self, other):
+        """
+        We support two forms of the SQL "NOT IN" syntax: one where a list of values is supplied, the other where
+        a sub-select is used to provide a set of values.
+
+        @param other: a constant parameter or sub-select
+        @type other: L{Parameter} or L{Select}
+        """
+        return self._commonIn('not in', other)
+
+
+    def _commonIn(self, op, other):
+        """
+        We support two forms of the SQL "NOT IN" syntax: one where a list of values is supplied, the other where
+        a sub-select is used to provide a set of values.
+
+        @param other: a constant parameter or sub-select
+        @type other: L{Parameter} or L{Select}
+        """
         if isinstance(other, Parameter):
             if other.count is None:
-                raise DALError("IN expression needs an explicit count of parameters")
-            return CompoundComparison(self, 'in', Constant(other))
+                raise DALError("{} expression needs an explicit count of parameters".format(op.upper()))
+            return CompoundComparison(self, op, Constant(other))
         else:
             # Can't be Select.__contains__ because __contains__ gets __nonzero__
-            # called on its result by the 'in' syntax.
-            return CompoundComparison(self, 'in', other)
+            # called on its result by the 'not in' syntax.
+            return CompoundComparison(self, op, other)
 
 
     def StartsWith(self, other):
         return CompoundComparison(self, "like", CompoundComparison(Constant(other), '||', Constant('%')))
 
 
+    def NotStartsWith(self, other):
+        return CompoundComparison(self, "not like", CompoundComparison(Constant(other), '||', Constant('%')))
+
+
     def EndsWith(self, other):
         return CompoundComparison(self, "like", CompoundComparison(Constant('%'), '||', Constant(other)))
 
 
+    def NotEndsWith(self, other):
+        return CompoundComparison(self, "not like", CompoundComparison(Constant('%'), '||', Constant(other)))
+
+
     def Contains(self, other):
         return CompoundComparison(self, "like", CompoundComparison(Constant('%'), '||', CompoundComparison(Constant(other), '||', Constant('%'))))
 
 
+    def NotContains(self, other):
+        return CompoundComparison(self, "not like", CompoundComparison(Constant('%'), '||', CompoundComparison(Constant(other), '||', Constant('%'))))
 
+
+
 class FunctionInvocation(ExpressionSyntax):
     def __init__(self, function, *args):
         self.function = function
@@ -555,7 +589,7 @@
         """
         Create a L{Join}, representing a join between two tables.
         """
-        if on is None:
+        if on is None and not type:
             type = 'cross'
         return Join(self, type, otherTableSyntax, on)
 
@@ -697,13 +731,16 @@
     def subSQL(self, queryGenerator, allTables):
         stmt = SQLFragment()
         stmt.append(self.leftSide.subSQL(queryGenerator, allTables))
-        stmt.text += ' '
-        if self.type:
-            stmt.text += self.type
+        if self.type == ',':
+            stmt.text += ', '
+        else:
             stmt.text += ' '
-        stmt.text += 'join '
+            if self.type:
+                stmt.text += self.type
+                stmt.text += ' '
+            stmt.text += 'join '
         stmt.append(self.rightSide.subSQL(queryGenerator, allTables))
-        if self.type != 'cross':
+        if self.type not in ('cross', ','):
             stmt.text += ' on '
             stmt.append(self.on.subSQL(queryGenerator, allTables))
         return stmt
@@ -882,6 +919,26 @@
 
 
 
+class Not(Comparison):
+    """
+    A L{NotColumn} is a logical NOT of an expression.
+    """
+    def __init__(self, a):
+        # 'op' and 'b' are always None for this comparison type
+        super(Not, self).__init__(a, None, None)
+
+
+    def subSQL(self, queryGenerator, allTables):
+        sqls = SQLFragment()
+        sqls.text += "not "
+        result = self.a.subSQL(queryGenerator, allTables)
+        if isinstance(self.a, CompoundComparison) and self.a.op in ('or', 'and'):
+            result = _inParens(result)
+        sqls.append(result)
+        return sqls
+
+
+
 class NullComparison(Comparison):
     """
     A L{NullComparison} is a comparison of a column or expression with None.

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/test/test_sqlsyntax.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/test/test_sqlsyntax.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twext/enterprise/dal/test/test_sqlsyntax.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -26,7 +26,7 @@
     Savepoint, RollbackToSavepoint, ReleaseSavepoint, SavepointAction,
     Union, Intersect, Except, SetExpression, DALError,
     ResultAliasSyntax, Count, QueryGenerator, ALL_COLUMNS,
-    DatabaseLock, DatabaseUnlock)
+    DatabaseLock, DatabaseUnlock, Not)
 from twext.enterprise.dal.syntax import FixedPlaceholder, NumericPlaceholder
 from twext.enterprise.dal.syntax import Function
 from twext.enterprise.dal.syntax import SchemaSyntax
@@ -362,6 +362,17 @@
         )
 
 
+    def test_commaJoin(self):
+        """
+        A join with no clause specified will generate a cross join.  (This is an
+        explicit synonym for an implicit join: i.e. 'select * from FOO, BAR'.)
+        """
+        self.assertEquals(
+            Select(From=self.schema.FOO.join(self.schema.BOZ, type=",")).toSQL(),
+            SQLFragment("select * from FOO, BOZ")
+        )
+
+
     def test_crossJoin(self):
         """
         A join with no clause specified will generate a cross join.  (This is an
@@ -963,6 +974,73 @@
         )
 
 
+    def test_not(self):
+        """
+        Test for the string starts with comparison.
+        (Note that this should be updated to use different techniques
+        as necessary in different databases.)
+        """
+        self.assertEquals(
+            Select([
+                self.schema.TEXTUAL.MYTEXT],
+                From=self.schema.TEXTUAL,
+                Where=Not(self.schema.TEXTUAL.MYTEXT.StartsWith("test")),
+            ).toSQL(),
+            SQLFragment(
+                "select MYTEXT from TEXTUAL where not MYTEXT like (? || ?)",
+                ["test", "%"]
+            )
+        )
+
+        self.assertEquals(
+            Select([
+                self.schema.TEXTUAL.MYTEXT],
+                From=self.schema.TEXTUAL,
+                Where=Not(self.schema.TEXTUAL.MYTEXT == "test"),
+            ).toSQL(),
+            SQLFragment(
+                "select MYTEXT from TEXTUAL where not MYTEXT = ?",
+                ["test"]
+            )
+        )
+
+        self.assertEquals(
+            Select([
+                self.schema.TEXTUAL.MYTEXT],
+                From=self.schema.TEXTUAL,
+                Where=Not((self.schema.TEXTUAL.MYTEXT == "test1").And(self.schema.TEXTUAL.MYTEXT != "test2")),
+            ).toSQL(),
+            SQLFragment(
+                "select MYTEXT from TEXTUAL where not (MYTEXT = ? and MYTEXT != ?)",
+                ["test1", "test2"]
+            )
+        )
+
+        self.assertEquals(
+            Select([
+                self.schema.TEXTUAL.MYTEXT],
+                From=self.schema.TEXTUAL,
+                Where=Not((self.schema.TEXTUAL.MYTEXT == "test1")).And(self.schema.TEXTUAL.MYTEXT != "test2"),
+            ).toSQL(),
+            SQLFragment(
+                "select MYTEXT from TEXTUAL where not MYTEXT = ? and MYTEXT != ?",
+                ["test1", "test2"]
+            )
+        )
+
+        self.assertEquals(
+            Select([
+                self.schema.TEXTUAL.MYTEXT],
+                From=self.schema.TEXTUAL,
+                Where=Not(self.schema.TEXTUAL.MYTEXT.StartsWith("foo").And(self.schema.TEXTUAL.MYTEXT.NotEndsWith("bar"))),
+            ).toSQL(),
+            SQLFragment(
+                "select MYTEXT from TEXTUAL where not (MYTEXT like (? || ?) and MYTEXT not like (? || ?))",
+                ["foo", "%", "%", "bar"]
+            )
+        )
+
+
     def test_insert(self):
         """
         L{Insert.toSQL} generates an 'insert' statement with all the relevant

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/opendirectorybacker.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/opendirectorybacker.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/opendirectorybacker.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -23,46 +23,45 @@
     "OpenDirectoryBackingService", "VCardRecord",
 ]
 
-import traceback
-import hashlib
+from calendarserver.platform.darwin.od import opendirectory, dsattributes, dsquery
 
-import os
-import sys
-import time
-
-from os import listdir
-from os.path import join, abspath
-from tempfile import mkstemp, gettempdir
-from random import random
-
-from pycalendar.vcard.n import N
-from pycalendar.vcard.adr import Adr
 from pycalendar.datetime import DateTime
+from pycalendar.vcard.adr import Adr
+from pycalendar.vcard.n import N
 
-from socket import getfqdn
 
-from twisted.internet import reactor
-from twisted.internet.defer import inlineCallbacks, returnValue, deferredGenerator, succeed
 from twext.python.filepath import CachingFilePath as FilePath
-from txdav.xml import element as davxml
-from txdav.xml.base import twisted_dav_namespace, dav_namespace, parse_date, twisted_private_namespace
 from twext.web2.dav.resource import DAVPropertyMixIn
 from twext.web2.dav.util import joinURL
 from twext.web2.http_headers import MimeType, generateContentType, ETag
 
+from twisted.internet import reactor
+from twisted.internet.defer import inlineCallbacks, returnValue, deferredGenerator, succeed
 
 from twistedcaldav import customxml, carddavxml
-from twistedcaldav.customxml import calendarserver_namespace
 from twistedcaldav.config import config
+from twistedcaldav.customxml import calendarserver_namespace
 from twistedcaldav.directory.directory import DirectoryService, DirectoryRecord
 from twistedcaldav.memcachelock import MemcacheLock, MemcacheLockTimeoutError
-from twistedcaldav.query import addressbookqueryfilter
 from twistedcaldav.vcard import Component, Property, vCardProductID
 
+from txdav.carddav.datastore.query.filter import IsNotDefined, ParameterFilter, \
+    TextMatch
+from txdav.xml import element as davxml
+from txdav.xml.base import twisted_dav_namespace, dav_namespace, parse_date, twisted_private_namespace
+
+from os import listdir
+from os.path import join, abspath
+from random import random
+from socket import getfqdn
+from tempfile import mkstemp, gettempdir
 from xmlrpclib import datetime
+import hashlib
+import os
+import sys
+import time
+import traceback
 
-from calendarserver.platform.darwin.od import opendirectory, dsattributes, dsquery
-
 class OpenDirectoryBackingService(DirectoryService):
     """
     Open Directory implementation of L{IDirectoryService}.
@@ -830,11 +829,11 @@
                 if not constant and not allAttrStrings:
                     return (False, [], [])
 
-                if propFilter.qualifier and isinstance(propFilter.qualifier, addressbookqueryfilter.IsNotDefined):
+                if propFilter.qualifier and isinstance(propFilter.qualifier, IsNotDefined):
                     return definedExpression(False, filterAllOf, propFilter.filter_name, constant, queryAttributes, allAttrStrings)
 
-                paramFilterElements = [paramFilterElement for paramFilterElement in propFilter.filters if isinstance(paramFilterElement, addressbookqueryfilter.ParameterFilter)]
-                textMatchElements = [textMatchElement for textMatchElement in propFilter.filters if isinstance(textMatchElement, addressbookqueryfilter.TextMatch)]
+                paramFilterElements = [paramFilterElement for paramFilterElement in propFilter.filters if isinstance(paramFilterElement, ParameterFilter)]
+                textMatchElements = [textMatchElement for textMatchElement in propFilter.filters if isinstance(textMatchElement, TextMatch)]
                 propFilterAllOf = propFilter.propfilter_test == "allof"
 
                 # handle parameter filter elements

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/accounts.xml
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/accounts.xml	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/accounts.xml	2013-12-19 18:28:54 UTC (rev 12144)
@@ -139,7 +139,7 @@
       <member type="users">delegateviagroup</member>
     </members>
   </group>
-  <user repeat="2">
+  <user repeat="100">
     <uid>user%02d</uid>
     <guid>user%02d</guid>
     <password>%02duser</password>

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/augments.xml
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/augments.xml	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/augments.xml	2013-12-19 18:28:54 UTC (rev 12144)
@@ -57,7 +57,7 @@
     <enable-calendar>false</enable-calendar>
     <enable-addressbook>false</enable-addressbook>
   </record>
-  <record repeat="2">
+  <record repeat="100">
     <uid>user%02d</uid>
     <enable>true</enable>
     <enable-calendar>true</enable-calendar>

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/test_directory.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/test_directory.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/directory/test/test_directory.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -999,8 +999,9 @@
         Exercise the default recordsMatchingTokens implementation
         """
         records = list((yield self.directoryService.recordsMatchingTokens(["Use", "01"])))
-        self.assertEquals(len(records), 1)
-        self.assertEquals(records[0].shortNames[0], "user01")
+        self.assertNotEquals(len(records), 0)
+        shorts = [record.shortNames[0] for record in records]
+        self.assertTrue("user01" in shorts)
 
         records = list((yield self.directoryService.recordsMatchingTokens(['"quotey"'],
             context=self.directoryService.searchContext_attendee)))

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_addressbook_query.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_addressbook_query.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_addressbook_query.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -27,7 +27,6 @@
 
 from twext.python.log import Logger
 from twext.web2 import responsecode
-from txdav.xml import element as davxml
 from twext.web2.dav.http import ErrorResponse, MultiStatusResponse
 from twext.web2.dav.method.report import NumberOfMatchesWithinLimits
 from twext.web2.dav.util import joinURL
@@ -37,9 +36,11 @@
 from twistedcaldav.config import config
 from twistedcaldav.carddavxml import carddav_namespace, NResults
 from twistedcaldav.method import report_common
-from twistedcaldav.query import addressbookqueryfilter
 
-from txdav.common.icommondatastore import ConcurrentModification
+from txdav.carddav.datastore.query.filter import Filter
+from txdav.common.icommondatastore import ConcurrentModification, \
+    IndexedSearchException
+from txdav.xml import element as davxml
 
 log = Logger()
 
@@ -62,7 +63,7 @@
     responses = []
 
     xmlfilter = addressbook_query.filter
-    filter = addressbookqueryfilter.Filter(xmlfilter)
+    filter = Filter(xmlfilter)
     query = addressbook_query.props
     limit = addressbook_query.limit
 
@@ -209,7 +210,7 @@
                                                         carddavxml.TextMatch.fromString(resource_name[:-4]),
                                                         name="UID", # attributes
                                                         ), ])
-                            vCardFilter = addressbookqueryfilter.Filter(vCardFilter)
+                            vCardFilter = Filter(vCardFilter)
 
                             directoryAddressBookLock, limited[0] = (yield  directory.cacheVCardsForAddressBookQuery(vCardFilter, query, max_number_of_results[0]))
 
@@ -230,11 +231,13 @@
 
                     # Check for disabled access
                     if filteredaces is not None:
-                        # See whether the filter is valid for an index only query
-                        index_query_ok = addrresource.index().searchValid(filter)
-
-                        # Get list of children that match the search and have read access
-                        names = [name for name, ignore_uid in (yield addrresource.index().search(filter))] #@UnusedVariable
+                        index_query_ok = True
+                        try:
+                            # Get list of children that match the search and have read access
+                            names = [name for name, ignore_uid in (yield addrresource.search(filter))] #@UnusedVariable
+                        except IndexedSearchException:
+                            names = yield addrresource.listChildren()
+                            index_query_ok = False
                         if not names:
                             return
 
@@ -277,7 +280,7 @@
                                                     carddavxml.TextMatch.fromString(resource_name[:-4]),
                                                     name="UID", # attributes
                                                     ), ])
-                        vCardFilter = addressbookqueryfilter.Filter(vCardFilter)
+                        vCardFilter = Filter(vCardFilter)
 
                         yield  maybeDeferred(queryDirectoryBackedAddressBook, parent, vCardFilter)
                         handled = True

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_calendar_query.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_calendar_query.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_calendar_query.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -20,8 +20,7 @@
 
 __all__ = ["report_urn_ietf_params_xml_ns_caldav_calendar_query"]
 
-from twisted.internet.defer import inlineCallbacks, returnValue, \
-    maybeDeferred
+from twisted.internet.defer import inlineCallbacks, returnValue
 
 from twext.python.log import Logger
 from twext.web2 import responsecode
@@ -39,8 +38,8 @@
     ConcurrentModification
 from twistedcaldav.instance import TooManyInstancesError
 from twistedcaldav.method import report_common
-from twistedcaldav.query import calendarqueryfilter
 
+from txdav.caldav.datastore.query.filter import Filter
 from txdav.caldav.icalendarstore import TimeRangeLowerLimit, TimeRangeUpperLimit
 from txdav.xml import element as davxml
 
@@ -66,7 +65,7 @@
     responses = []
 
     xmlfilter = calendar_query.filter
-    filter = calendarqueryfilter.Filter(xmlfilter)
+    filter = Filter(xmlfilter)
     props = calendar_query.props
 
     assert props is not None
@@ -190,13 +189,11 @@
             if filteredaces is not None:
                 index_query_ok = True
                 try:
-                    # Get list of children that match the search and have read
-                    # access
-                    records = yield maybeDeferred(calresource.index().indexedSearch, filter)
+                    # Get list of children that match the search and have read access
+                    names = [name for name, ignore_uid, ignore_type in (yield calresource.search(filter))]
                 except IndexedSearchException:
-                    records = yield maybeDeferred(calresource.index().bruteForceSearch)
+                    names = yield calresource.listChildren()
                     index_query_ok = False
-                names = [name for name, ignore_uid, ignore_type in records]
 
                 if not names:
                     returnValue(True)

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_common.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_common.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_common.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -36,17 +36,16 @@
 except ImportError:
     from md5 import new as md5
 
-from twisted.internet.defer import inlineCallbacks, returnValue, maybeDeferred
+from twisted.internet.defer import inlineCallbacks, returnValue
 from twisted.python.failure import Failure
 from twext.web2 import responsecode
 
-from txdav.xml import element
 from twext.web2.dav.http import statusForFailure
 from twext.web2.dav.method.propfind import propertyName
 from twext.web2.dav.method.report import NumberOfMatchesWithinLimits
 from twext.web2.dav.method.report import max_number_of_matches
 from twext.web2.dav.resource import AccessDeniedError
-from twext.web2.http import HTTPError
+from twext.web2.http import HTTPError, StatusResponse
 
 from twext.python.log import Logger
 
@@ -65,9 +64,9 @@
 from twistedcaldav.instance import InstanceList
 from twistedcaldav.memcacher import Memcacher
 
-from twistedcaldav.query import calendarqueryfilter
-
+from txdav.caldav.datastore.query.filter import Filter
 from txdav.common.icommondatastore import IndexedSearchException
+from txdav.xml import element
 
 from pycalendar.duration import Duration
 from pycalendar.datetime import DateTime
@@ -583,17 +582,18 @@
                           name="VCALENDAR",
                        )
                   )
-        filter = calendarqueryfilter.Filter(filter)
+        filter = Filter(filter)
         tzinfo = filter.settimezone(tz)
 
         try:
-            resources = yield maybeDeferred(calresource.index().indexedSearch,
-                filter, useruid=useruid, fbtype=True
-            )
+            resources = yield calresource.search(filter, useruid=useruid, fbtype=True)
             if caching:
                 yield FBCacheEntry.makeCacheEntry(calresource, useruid, cache_timerange, resources)
         except IndexedSearchException:
-            resources = yield maybeDeferred(calresource.index().bruteForceSearch)
+            raise HTTPError(StatusResponse(
+                responsecode.INTERNAL_SERVER_ERROR,
+                "Failed freebusy query"
+            ))
 
     else:
         # Log extended item

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_multiget_common.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_multiget_common.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/method/report_multiget_common.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -24,8 +24,6 @@
 
 from twext.python.log import Logger
 from twext.web2 import responsecode
-from txdav.xml import element as davxml
-from txdav.xml.base import dav_namespace
 from twext.web2.dav.http import ErrorResponse, MultiStatusResponse
 from twext.web2.dav.resource import AccessDeniedError
 from twext.web2.http import HTTPError, StatusResponse
@@ -37,11 +35,14 @@
 from twistedcaldav.carddavxml import carddav_namespace
 from twistedcaldav.config import config
 from twistedcaldav.method import report_common
-from txdav.common.icommondatastore import ConcurrentModification
 from twistedcaldav.method.report_common import COLLECTION_TYPE_CALENDAR, \
     COLLECTION_TYPE_ADDRESSBOOK
-from twistedcaldav.query import addressbookqueryfilter
 
+from txdav.carddav.datastore.query.filter import Filter
+from txdav.common.icommondatastore import ConcurrentModification
+from txdav.xml import element as davxml
+from txdav.xml.base import dav_namespace
+
 log = Logger()
 
 @inlineCallbacks
@@ -265,7 +266,7 @@
                     returnValue(None)
 
                 addressBookFilter = carddavxml.Filter(*vCardFilters)
-                addressBookFilter = addressbookqueryfilter.Filter(addressBookFilter)
+                addressBookFilter = Filter(addressBookFilter)
                 if self.directory.cacheQuery:
                     # add vcards to directory address book and run "normal case" below
                     limit = config.DirectoryAddressBook.MaxQueryResults
@@ -333,11 +334,11 @@
                     parent = (yield child.locateParent(request, resource_uri))
 
                     if collection_type == COLLECTION_TYPE_CALENDAR:
-                        if not parent.isCalendarCollection() or not (yield parent.index().resourceExists(name)):
+                        if not parent.isCalendarCollection() or not (yield parent.resourceExists(name)):
                             responses.append(davxml.StatusResponse(href, davxml.Status.fromResponseCode(responsecode.FORBIDDEN)))
                             continue
                     elif collection_type == COLLECTION_TYPE_ADDRESSBOOK:
-                        if not parent.isAddressBookCollection() or not (yield parent.index().resourceExists(name)):
+                        if not parent.isAddressBookCollection() or not (yield parent.resourceExists(name)):
                             responses.append(davxml.StatusResponse(href, davxml.Status.fromResponseCode(responsecode.FORBIDDEN)))
                             continue
 
@@ -367,11 +368,11 @@
                     parent = (yield self.locateParent(request, resource_uri))
 
                     if collection_type == COLLECTION_TYPE_CALENDAR:
-                        if not parent.isPseudoCalendarCollection() or not (yield parent.index().resourceExists(name)):
+                        if not parent.isPseudoCalendarCollection() or not (yield parent.resourceExists(name)):
                             responses.append(davxml.StatusResponse(href, davxml.Status.fromResponseCode(responsecode.FORBIDDEN)))
                             continue
                     elif collection_type == COLLECTION_TYPE_ADDRESSBOOK:
-                        if not parent.isAddressBookCollection() or not (yield parent.index().resourceExists(name)):
+                        if not parent.isAddressBookCollection() or not (yield parent.resourceExists(name)):
                             responses.append(davxml.StatusResponse(href, davxml.Status.fromResponseCode(responsecode.FORBIDDEN)))
                             continue
                     child = self

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/storebridge.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/storebridge.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/storebridge.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -286,13 +286,6 @@
         return self._parentResource
 
 
-    def index(self):
-        """
-        Retrieve the new-style index wrapper.
-        """
-        return self._newStoreObject.retrieveOldIndex()
-
-
     def exists(self):
         # FIXME: tests
         return self._newStoreObject is not None
@@ -347,6 +340,18 @@
         return self._newStoreObject.countObjectResources()
 
 
+    @inlineCallbacks
+    def resourceExists(self, name):
+        """
+        Indicate whether a resource with the specified name exists.
+
+        @return: C{True} if it exists
+        @rtype: C{bool}
+        """
+        allNames = yield self._newStoreObject.listObjectResources()
+        returnValue(name in allNames)
+
+
     def name(self):
         return self._name
 
@@ -970,6 +975,10 @@
                     )
 
 
+    def search(self, filter, **kwargs):
+        return self._newStoreObject.search(filter, **kwargs)
+
+
     def notifierID(self):
         return "%s/%s" % self._newStoreObject.notifierID()
 

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_calendarquery.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_calendarquery.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_calendarquery.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -27,7 +27,6 @@
 from twistedcaldav import caldavxml
 from twistedcaldav import ical
 
-from twistedcaldav.query import calendarqueryfilter
 from twistedcaldav.config import config
 from twistedcaldav.test.util import StoreTestCase, SimpleStoreRequest
 from twisted.internet.defer import inlineCallbacks, returnValue
@@ -36,6 +35,7 @@
 from twistedcaldav.ical import Component
 from txdav.caldav.icalendarstore import ComponentUpdateState
 from twistedcaldav.directory.directory import DirectoryService
+from txdav.caldav.datastore.query.filter import TimeRange
 
 
 @inlineCallbacks
@@ -167,7 +167,7 @@
                             cal = property.calendar()
                             instances = cal.expandTimeRanges(query_timerange.end)
                             vevents = [x for x in cal.subcomponents() if x.name() == "VEVENT"]
-                            if not calendarqueryfilter.TimeRange(query_timerange).matchinstance(vevents[0], instances):
+                            if not TimeRange(query_timerange).matchinstance(vevents[0], instances):
                                 self.fail("REPORT property %r returned calendar %s outside of request time range %r"
                                           % (property, property.calendar, query_timerange))
 

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_sharing.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_sharing.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_sharing.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -30,6 +30,7 @@
 from twistedcaldav.test.test_cache import StubResponseCacheResource
 from twistedcaldav.test.util import norequest, StoreTestCase, SimpleStoreRequest
 
+from txdav.caldav.datastore.test.util import buildDirectory
 from txdav.common.datastore.sql_tables import _BIND_MODE_DIRECT
 from txdav.xml import element as davxml
 from txdav.xml.parser import WebDAVDocument
@@ -738,6 +739,8 @@
         home is at /.  Return the name of the newly shared calendar in the
         sharee's home.
         """
+
+        self._sqlCalendarStore._directoryService = buildDirectory(homes=("wiki-testing",))
         wcreate = self._sqlCalendarStore.newTransaction("create wiki")
         yield wcreate.calendarHomeWithUID("wiki-testing", create=True)
         yield wcreate.commit()

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_xml.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_xml.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/twistedcaldav/test/test_xml.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -17,12 +17,15 @@
 import os
 
 from twisted.trial.unittest import SkipTest
+
 from twistedcaldav.ical import Component
-from twistedcaldav.query import calendarqueryfilter
 import twistedcaldav.test.util
 from twistedcaldav.caldavxml import ComponentFilter, PropertyFilter, TextMatch, \
     Filter, TimeRange
 
+from txdav.caldav.datastore.query.filter import Filter as storeFilter
+from txdav.caldav.datastore.query.filter import ComponentFilter as storeComponentFilter
+
 class XML (twistedcaldav.test.util.TestCase):
     """
     XML tests
@@ -46,7 +49,7 @@
             else:
                 no = ""
 
-            if has != calendarqueryfilter.ComponentFilter(
+            if has != storeComponentFilter(
                 ComponentFilter(
                     ComponentFilter(
                         name=component_name
@@ -70,7 +73,7 @@
             else:
                 no = ""
 
-            if has != calendarqueryfilter.ComponentFilter(
+            if has != storeComponentFilter(
                 ComponentFilter(
                     ComponentFilter(
                         PropertyFilter(
@@ -106,7 +109,7 @@
             else:
                 no = ""
 
-            if has != calendarqueryfilter.ComponentFilter(
+            if has != storeComponentFilter(
                 ComponentFilter(
                     ComponentFilter(
                         PropertyFilter(
@@ -148,7 +151,7 @@
             else:
                 no = ""
 
-            if has != calendarqueryfilter.Filter(
+            if has != storeFilter(
                 Filter(
                     ComponentFilter(
                         ComponentFilter(

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/index_file.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/index_file.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/index_file.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -15,6 +15,7 @@
 # limitations under the License.
 ##
 
+
 """
 CalDAV Index.
 
@@ -43,12 +44,14 @@
 
 from twext.python.log import Logger
 
+from txdav.caldav.datastore.query.builder import buildExpression
+from txdav.caldav.datastore.query.filter import Filter
+from txdav.common.datastore.query.filegenerator import sqllitegenerator
 from txdav.common.icommondatastore import SyncTokenValidException, \
     ReservationError, IndexedSearchException
 
 from twistedcaldav.dateops import pyCalendarTodatetime
 from twistedcaldav.ical import Component
-from twistedcaldav.query import calendarquery, calendarqueryfilter
 from twistedcaldav.sql import AbstractSQLDatabase
 from twistedcaldav.sql import db_prefix
 from twistedcaldav.instance import InvalidOverriddenInstanceError
@@ -320,7 +323,7 @@
 
         # Make sure we have a proper Filter element and get the partial SQL
         # statement to use.
-        if isinstance(filter, calendarqueryfilter.Filter):
+        if isinstance(filter, Filter):
             if fbtype:
                 # Lookup the useruid - try the empty (default) one if needed
                 dbuseruid = self._db_value_for_sql(
@@ -330,7 +333,7 @@
             else:
                 dbuseruid = ""
 
-            qualifiers = calendarquery.sqlcalendarquery(filter, None, dbuseruid, fbtype)
+            qualifiers = sqlcalendarquery(filter, None, dbuseruid, fbtype)
             if qualifiers is not None:
                 # Determine how far we need to extend the current expansion of
                 # events. If we have an open-ended time-range we will expand one
@@ -437,6 +440,24 @@
 
 
 
+def sqlcalendarquery(filter, calendarid=None, userid=None, freebusy=False):
+    """
+    Convert the supplied calendar-query into a partial SQL statement.
+
+    @param filter: the L{Filter} for the calendar-query to convert.
+    @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the partial SQL statement,
+            and the C{list} is the list of argument substitutions to use with the SQL API execute method.
+            Or return C{None} if it is not possible to create an SQL query to fully match the calendar-query.
+    """
+    try:
+        expression = buildExpression(filter, sqllitegenerator.FIELDS)
+        sql = sqllitegenerator(expression, calendarid, userid, freebusy)
+        return sql.generate()
+    except ValueError:
+        return None
+
+
+
 class CalendarIndex (AbstractCalendarIndex):
     """
     Calendar index - abstract class for indexer that indexes calendar objects in a collection.

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/__init__.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/__init__.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/builder.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/builder.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/builder.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,227 @@
+##
+# Copyright (c) 2006-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twistedcaldav.dateops import floatoffset, pyCalendarTodatetime
+
+from txdav.caldav.datastore.query.filter import ComponentFilter, PropertyFilter, TextMatch, TimeRange
+from txdav.common.datastore.query import expression
+
+
+"""
+SQL statement generator from query expressions.
+"""
+
+__all__ = [
+    "buildExpression",
+]
+
+
+
+# SQL Index column (field) names
+
+def buildExpression(filter, fields):
+    """
+    Convert the supplied calendar-query into an expression tree.
+
+    @param filter: the L{Filter} for the calendar-query to convert.
+    @return: a L{baseExpression} for the expression tree.
+    """
+
+    # Lets assume we have a valid filter from the outset.
+
+    # Top-level filter contains exactly one comp-filter element
+    assert filter.child is not None
+    vcalfilter = filter.child
+    assert isinstance(vcalfilter, ComponentFilter)
+    assert vcalfilter.filter_name == "VCALENDAR"
+
+    if len(vcalfilter.filters) > 0:
+        # Determine logical expression grouping
+        logical = expression.andExpression if vcalfilter.filter_test == "allof" else expression.orExpression
+
+        # Only comp-filters are handled
+        for _ignore in [x for x in vcalfilter.filters if not isinstance(x, ComponentFilter)]:
+            raise ValueError
+
+        return compfilterListExpression(vcalfilter.filters, fields, logical)
+    else:
+        return expression.allExpression()
+
+
+
+def compfilterListExpression(compfilters, fields, logical):
+    """
+    Create an expression for a list of comp-filter elements.
+
+    @param compfilters: the C{list} of L{ComponentFilter} elements.
+    @return: a L{baseExpression} for the expression tree.
+    """
+
+    if len(compfilters) == 1:
+        return compfilterExpression(compfilters[0], fields)
+    else:
+        return logical([compfilterExpression(c, fields) for c in compfilters])
+
+
+
+def compfilterExpression(compfilter, fields):
+    """
+    Create an expression for a single comp-filter element.
+
+    @param compfilter: the L{ComponentFilter} element.
+    @return: a L{baseExpression} for the expression tree.
+    """
+
+    # Handle is-not-defined case
+    if not compfilter.defined:
+        # Test for TYPE != <<component-type name>>
+        return expression.isnotExpression(fields["TYPE"], compfilter.filter_name, True)
+
+    # Determine logical expression grouping
+    logical = expression.andExpression if compfilter.filter_test == "allof" else expression.orExpression
+
+    expressions = []
+    if isinstance(compfilter.filter_name, str):
+        expressions.append(expression.isExpression(fields["TYPE"], compfilter.filter_name, True))
+    else:
+        expressions.append(expression.inExpression(fields["TYPE"], compfilter.filter_name, True))
+
+    # Handle time-range
+    if compfilter.qualifier and isinstance(compfilter.qualifier, TimeRange):
+        start, end, startfloat, endfloat = getTimerangeArguments(compfilter.qualifier)
+        expressions.append(expression.timerangeExpression(start, end, startfloat, endfloat))
+
+    # Handle properties - we can only do UID right now
+    props = []
+    for p in [x for x in compfilter.filters if isinstance(x, PropertyFilter)]:
+        props.append(propfilterExpression(p, fields))
+    if len(props) > 1:
+        propsExpression = logical(props)
+    elif len(props) == 1:
+        propsExpression = props[0]
+    else:
+        propsExpression = None
+
+    # Handle embedded components - we do not right now as our Index does not handle them
+    comps = []
+    for _ignore in [x for x in compfilter.filters if isinstance(x, ComponentFilter)]:
+        raise ValueError
+    if len(comps) > 1:
+        compsExpression = logical(comps)
+    elif len(comps) == 1:
+        compsExpression = comps[0]
+    else:
+        compsExpression = None
+
+    # Now build compound expression
+    if ((propsExpression is not None) and (compsExpression is not None)):
+        expressions.append(logical([propsExpression, compsExpression]))
+    elif propsExpression is not None:
+        expressions.append(propsExpression)
+    elif compsExpression is not None:
+        expressions.append(compsExpression)
+
+    # Now build return expression
+    return expression.andExpression(expressions)
+
+
+
+def propfilterExpression(propfilter, fields):
+    """
+    Create an expression for a single prop-filter element.
+
+    @param propfilter: the L{PropertyFilter} element.
+    @return: a L{baseExpression} for the expression tree.
+    """
+
+    # Only handle UID right now
+    if propfilter.filter_name != "UID":
+        raise ValueError
+
+    # Handle is-not-defined case
+    if not propfilter.defined:
+        # Test for <<field>> != "*"
+        return expression.isExpression(fields["UID"], "", True)
+
+    # Determine logical expression grouping
+    logical = expression.andExpression if propfilter.filter_test == "allof" else expression.orExpression
+
+    # Handle time-range - we cannot do this with our Index right now
+    if propfilter.qualifier and isinstance(propfilter.qualifier, TimeRange):
+        raise ValueError
+
+    # Handle text-match
+    tm = None
+    if propfilter.qualifier and isinstance(propfilter.qualifier, TextMatch):
+        if propfilter.qualifier.match_type == "equals":
+            tm = expression.isnotExpression if propfilter.qualifier.negate else expression.isExpression
+        elif propfilter.qualifier.match_type == "contains":
+            tm = expression.notcontainsExpression if propfilter.qualifier.negate else expression.containsExpression
+        elif propfilter.qualifier.match_type == "starts-with":
+            tm = expression.notstartswithExpression if propfilter.qualifier.negate else expression.startswithExpression
+        elif propfilter.qualifier.match_type == "ends-with":
+            tm = expression.notendswithExpression if propfilter.qualifier.negate else expression.endswithExpression
+        tm = tm(fields[propfilter.filter_name], propfilter.qualifier.text, propfilter.qualifier.caseless)
+
+    # Handle embedded parameters - we do not right now as our Index does not handle them
+    params = []
+    for _ignore in propfilter.filters:
+        raise ValueError
+    if len(params) > 1:
+        paramsExpression = logical(params)
+    elif len(params) == 1:
+        paramsExpression = params[0]
+    else:
+        paramsExpression = None
+
+    # Now build return expression
+    if (tm is not None) and (paramsExpression is not None):
+        return logical([tm, paramsExpression])
+    elif tm is not None:
+        return tm
+    elif paramsExpression is not None:
+        return paramsExpression
+    else:
+        return None
+
+
+
+def getTimerangeArguments(timerange):
+    """
+    Get start/end and floating start/end (adjusted for timezone offset) values from the
+    supplied time-range test.
+
+    @param timerange: the L{TimeRange} used in the query.
+    @return: C{tuple} of C{str} for start, end, startfloat, endfloat
+    """
+
+    # Start/end in UTC
+    start = timerange.start
+    end = timerange.end
+
+    # Get timezone
+    tzinfo = timerange.tzinfo
+
+    # Now force to floating UTC
+    startfloat = floatoffset(start, tzinfo) if start else None
+    endfloat = floatoffset(end, tzinfo) if end else None
+
+    return (
+        pyCalendarTodatetime(start) if start else None,
+        pyCalendarTodatetime(end) if end else None,
+        pyCalendarTodatetime(startfloat) if startfloat else None,
+        pyCalendarTodatetime(endfloat) if endfloat else None,
+    )

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/filter.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/filter.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/filter.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,754 @@
+##
+# Copyright (c) 2011-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+Object model of CALDAV:filter element used in an addressbook-query.
+"""
+
+__all__ = [
+    "Filter",
+]
+
+from twext.python.log import Logger
+
+from twistedcaldav.caldavxml import caldav_namespace, CalDAVTimeZoneElement
+from twistedcaldav.dateops import timeRangesOverlap
+from twistedcaldav.ical import Component, Property
+
+from pycalendar.datetime import DateTime
+from pycalendar.timezone import Timezone
+
+log = Logger()
+
+
+class FilterBase(object):
+    """
+    Determines which matching components are returned.
+    """
+
+    def __init__(self, xml_element):
+        self.xmlelement = xml_element
+
+
+    def match(self, item, access=None):
+        raise NotImplementedError
+
+
+    def valid(self, level=0):
+        raise NotImplementedError
+
+
+
+class Filter(FilterBase):
+    """
+    Determines which matching components are returned.
+    """
+
+    def __init__(self, xml_element):
+
+        super(Filter, self).__init__(xml_element)
+
+        # One comp-filter element must be present
+        if len(xml_element.children) != 1 or xml_element.children[0].qname() != (caldav_namespace, "comp-filter"):
+            raise ValueError("Invalid CALDAV:filter element: %s" % (xml_element,))
+
+        self.child = ComponentFilter(xml_element.children[0])
+
+
+    def match(self, component, access=None):
+        """
+        Returns True if the given calendar component matches this filter, False
+        otherwise.
+        """
+
+        # We only care about certain access restrictions.
+        if access not in (Component.ACCESS_CONFIDENTIAL, Component.ACCESS_RESTRICTED):
+            access = None
+
+        # We need to prepare ourselves for a time-range query by pre-calculating
+        # the set of instances up to the latest time-range limit. That way we can
+        # avoid having to do some form of recurrence expansion for each query sub-part.
+        maxend, isStartTime = self.getmaxtimerange()
+        if maxend:
+            if isStartTime:
+                if component.isRecurringUnbounded():
+                    # Unbounded recurrence is always within a start-only time-range
+                    instances = None
+                else:
+                    # Expand the instances up to infinity
+                    instances = component.expandTimeRanges(DateTime(2100, 1, 1, 0, 0, 0, tzid=Timezone(utc=True)), ignoreInvalidInstances=True)
+            else:
+                instances = component.expandTimeRanges(maxend, ignoreInvalidInstances=True)
+        else:
+            instances = None
+        self.child.setInstances(instances)
+
+        # <filter> contains exactly one <comp-filter>
+        return self.child.match(component, access)
+
+
+    def valid(self):
+        """
+        Indicate whether this filter element's structure is valid wrt iCalendar
+        data object model.
+
+        @return: True if valid, False otherwise
+        """
+
+        # Must have one child element for VCALENDAR
+        return self.child.valid(0)
+
+
+    def settimezone(self, tzelement):
+        """
+        Set the default timezone to use with this query.
+        @param calendar: a L{Component} for the VCALENDAR containing the one
+            VTIMEZONE that we want
+        @return: the L{Timezone} derived from the VTIMEZONE or utc.
+        """
+
+        if tzelement is None:
+            tz = None
+        elif isinstance(tzelement, CalDAVTimeZoneElement):
+            tz = tzelement.gettimezone()
+        elif isinstance(tzelement, Component):
+            tz = tzelement.gettimezone()
+        if tz is None:
+            tz = Timezone(utc=True)
+        self.child.settzinfo(tz)
+        return tz
+
+
+    def getmaxtimerange(self):
+        """
+        Get the date farthest into the future in any time-range elements
+        """
+
+        return self.child.getmaxtimerange(None, False)
+
+
+    def getmintimerange(self):
+        """
+        Get the date farthest into the past in any time-range elements. That is either
+        the start date, or if start is not present, the end date.
+        """
+
+        return self.child.getmintimerange(None, False)
+
+
+
+class FilterChildBase(FilterBase):
+    """
+    CalDAV filter element.
+    """
+
+    def __init__(self, xml_element):
+
+        super(FilterChildBase, self).__init__(xml_element)
+
+        qualifier = None
+        filters = []
+
+        for child in xml_element.children:
+            qname = child.qname()
+
+            if qname in (
+                (caldav_namespace, "is-not-defined"),
+                (caldav_namespace, "time-range"),
+                (caldav_namespace, "text-match"),
+            ):
+                if qualifier is not None:
+                    raise ValueError("Only one of CalDAV:time-range, CalDAV:text-match allowed")
+
+                if qname == (caldav_namespace, "is-not-defined"):
+                    qualifier = IsNotDefined(child)
+                elif qname == (caldav_namespace, "time-range"):
+                    qualifier = TimeRange(child)
+                elif qname == (caldav_namespace, "text-match"):
+                    qualifier = TextMatch(child)
+
+            elif qname == (caldav_namespace, "comp-filter"):
+                filters.append(ComponentFilter(child))
+            elif qname == (caldav_namespace, "prop-filter"):
+                filters.append(PropertyFilter(child))
+            elif qname == (caldav_namespace, "param-filter"):
+                filters.append(ParameterFilter(child))
+            else:
+                raise ValueError("Unknown child element: %s" % (qname,))
+
+        if qualifier and isinstance(qualifier, IsNotDefined) and (len(filters) != 0):
+            raise ValueError("No other tests allowed when CalDAV:is-not-defined is present")
+
+        self.qualifier = qualifier
+        self.filters = filters
+        self.filter_name = xml_element.attributes["name"]
+        if isinstance(self.filter_name, unicode):
+            self.filter_name = self.filter_name.encode("utf-8")
+        self.defined = not self.qualifier or not isinstance(qualifier, IsNotDefined)
+
+        filter_test = xml_element.attributes.get("test", "allof")
+        if filter_test not in ("anyof", "allof"):
+            raise ValueError("Test must be only one of anyof, allof")
+        self.filter_test = filter_test
+
+
+    def match(self, item, access=None):
+        """
+        Returns True if the given calendar item (either a component, property or parameter value)
+        matches this filter, False otherwise.
+        """
+
+        # Always return True for the is-not-defined case as the result of this will
+        # be negated by the caller
+        if not self.defined:
+            return True
+
+        if self.qualifier and not self.qualifier.match(item, access):
+            return False
+
+        if len(self.filters) > 0:
+            allof = self.filter_test == "allof"
+            for filter in self.filters:
+                if allof != filter._match(item, access):
+                    return not allof
+            return allof
+        else:
+            return True
+
+
+
+class ComponentFilter (FilterChildBase):
+    """
+    Limits a search to only the chosen component types.
+    """
+
+    def match(self, item, access):
+        """
+        Returns True if the given calendar item (which is a component)
+        matches this filter, False otherwise.
+        This specialization uses the instance matching option of the time-range filter
+        to minimize instance expansion.
+        """
+
+        # Always return True for the is-not-defined case as the result of this will
+        # be negated by the caller
+        if not self.defined:
+            return True
+
+        if self.qualifier and not self.qualifier.matchinstance(item, self.instances):
+            return False
+
+        if len(self.filters) > 0:
+            allof = self.filter_test == "allof"
+            for filter in self.filters:
+                if allof != filter._match(item, access):
+                    return not allof
+            return allof
+        else:
+            return True
+
+
+    def _match(self, component, access):
+        # At least one subcomponent must match (or is-not-defined is set)
+        for subcomponent in component.subcomponents():
+            # If access restrictions are in force, restrict matching to specific components only.
+            # In particular do not match VALARM.
+            if access and subcomponent.name() not in ("VEVENT", "VTODO", "VJOURNAL", "VFREEBUSY", "VTIMEZONE",):
+                continue
+
+            # Try to match the component name
+            if isinstance(self.filter_name, str):
+                if subcomponent.name() != self.filter_name:
+                    continue
+            else:
+                if subcomponent.name() not in self.filter_name:
+                    continue
+            if self.match(subcomponent, access):
+                break
+        else:
+            return not self.defined
+        return self.defined
+
+
+    def setInstances(self, instances):
+        """
+        Give the list of instances to each comp-filter element.
+        @param instances: the list of instances.
+        """
+        self.instances = instances
+        for compfilter in [x for x in self.filters if isinstance(x, ComponentFilter)]:
+            compfilter.setInstances(instances)
+
+
+    def valid(self, level):
+        """
+        Indicate whether this filter element's structure is valid wrt iCalendar
+        data object model.
+
+        @param level: the nesting level of this filter element, 0 being the top comp-filter.
+        @return:      True if valid, False otherwise
+        """
+
+        # Check for time-range
+        timerange = self.qualifier and isinstance(self.qualifier, TimeRange)
+
+        if level == 0:
+            # Must have VCALENDAR at the top
+            if (self.filter_name != "VCALENDAR") or timerange:
+                log.info("Top-level comp-filter must be VCALENDAR, instead: %s" % (self.filter_name,))
+                return False
+        elif level == 1:
+            # Disallow VCALENDAR, VALARM, STANDARD, DAYLIGHT, AVAILABLE at the top, everything else is OK
+            if self.filter_name in ("VCALENDAR", "VALARM", "STANDARD", "DAYLIGHT", "AVAILABLE"):
+                log.info("comp-filter wrong component type: %s" % (self.filter_name,))
+                return False
+
+            # time-range only on VEVENT, VTODO, VJOURNAL, VFREEBUSY, VAVAILABILITY
+            if timerange and self.filter_name not in ("VEVENT", "VTODO", "VJOURNAL", "VFREEBUSY", "VAVAILABILITY"):
+                log.info("time-range cannot be used with component %s" % (self.filter_name,))
+                return False
+        elif level == 2:
+            # Disallow VCALENDAR, VTIMEZONE, VEVENT, VTODO, VJOURNAL, VFREEBUSY, VAVAILABILITY at the top, everything else is OK
+            if (self.filter_name in ("VCALENDAR", "VTIMEZONE", "VEVENT", "VTODO", "VJOURNAL", "VFREEBUSY", "VAVAILABILITY")):
+                log.info("comp-filter wrong sub-component type: %s" % (self.filter_name,))
+                return False
+
+            # time-range only on VALARM, AVAILABLE
+            if timerange and self.filter_name not in ("VALARM", "AVAILABLE",):
+                log.info("time-range cannot be used with sub-component %s" % (self.filter_name,))
+                return False
+        else:
+            # Disallow all standard iCal components anywhere else
+            if (self.filter_name in ("VCALENDAR", "VTIMEZONE", "VEVENT", "VTODO", "VJOURNAL", "VFREEBUSY", "VALARM", "STANDARD", "DAYLIGHT", "AVAILABLE")) or timerange:
+                log.info("comp-filter wrong standard component type: %s" % (self.filter_name,))
+                return False
+
+        # Test each property
+        for propfilter in [x for x in self.filters if isinstance(x, PropertyFilter)]:
+            if not propfilter.valid():
+                return False
+
+        # Test each component
+        for compfilter in [x for x in self.filters if isinstance(x, ComponentFilter)]:
+            if not compfilter.valid(level + 1):
+                return False
+
+        # Test the time-range
+        if timerange:
+            if not self.qualifier.valid():
+                return False
+
+        return True
+
+
+    def settzinfo(self, tzinfo):
+        """
+        Set the default timezone to use with this query.
+        @param tzinfo: a L{Timezone} to use.
+        """
+
+        # Give tzinfo to any TimeRange we have
+        if isinstance(self.qualifier, TimeRange):
+            self.qualifier.settzinfo(tzinfo)
+
+        # Pass down to sub components/properties
+        for x in self.filters:
+            x.settzinfo(tzinfo)
+
+
+    def getmaxtimerange(self, currentMaximum, currentIsStartTime):
+        """
+        Get the date farthest into the future in any time-range elements
+
+        @param currentMaximum: current future value to compare with
+        @type currentMaximum: L{DateTime}
+        """
+
+        # Give tzinfo to any TimeRange we have
+        isStartTime = False
+        if isinstance(self.qualifier, TimeRange):
+            isStartTime = self.qualifier.end is None
+            compareWith = self.qualifier.start if isStartTime else self.qualifier.end
+            if currentMaximum is None or currentMaximum < compareWith:
+                currentMaximum = compareWith
+                currentIsStartTime = isStartTime
+
+        # Pass down to sub components/properties
+        for x in self.filters:
+            currentMaximum, currentIsStartTime = x.getmaxtimerange(currentMaximum, currentIsStartTime)
+
+        return currentMaximum, currentIsStartTime
+
+
+    def getmintimerange(self, currentMinimum, currentIsEndTime):
+        """
+        Get the date farthest into the past in any time-range elements. That is either
+        the start date, or if start is not present, the end date.
+        """
+
+        # Give tzinfo to any TimeRange we have
+        isEndTime = False
+        if isinstance(self.qualifier, TimeRange):
+            isEndTime = self.qualifier.start is None
+            compareWith = self.qualifier.end if isEndTime else self.qualifier.start
+            if currentMinimum is None or currentMinimum > compareWith:
+                currentMinimum = compareWith
+                currentIsEndTime = isEndTime
+
+        # Pass down to sub components/properties
+        for x in self.filters:
+            currentMinimum, currentIsEndTime = x.getmintimerange(currentMinimum, currentIsEndTime)
+
+        return currentMinimum, currentIsEndTime
+
+
+
+class PropertyFilter (FilterChildBase):
+    """
+    Limits a search to specific properties.
+    """
+
+    def _match(self, component, access):
+        # When access restriction is in force, we need to only allow matches against the properties
+        # allowed by the access restriction level.
+        if access:
+            allowedProperties = Component.confidentialPropertiesMap.get(component.name(), None)
+            if allowedProperties and access == Component.ACCESS_RESTRICTED:
+                allowedProperties += Component.extraRestrictedProperties
+        else:
+            allowedProperties = None
+
+        # At least one property must match (or is-not-defined is set)
+        for property in component.properties():
+            # Apply access restrictions, if any.
+            if allowedProperties is not None and property.name().upper() not in allowedProperties:
+                continue
+            if property.name().upper() == self.filter_name.upper() and self.match(property, access):
+                break
+        else:
+            return not self.defined
+        return self.defined
+
+
+    def valid(self):
+        """
+        Indicate whether this filter element's structure is valid wrt iCalendar
+        data object model.
+
+        @return:      True if valid, False otherwise
+        """
+
+        # Check for time-range
+        timerange = self.qualifier and isinstance(self.qualifier, TimeRange)
+
+        # time-range only on COMPLETED, CREATED, DTSTAMP, LAST-MODIFIED
+        if timerange and self.filter_name.upper() not in ("COMPLETED", "CREATED", "DTSTAMP", "LAST-MODIFIED"):
+            log.info("time-range cannot be used with property %s" % (self.filter_name,))
+            return False
+
+        # Test the time-range
+        if timerange:
+            if not self.qualifier.valid():
+                return False
+
+        # No other tests
+        return True
+
+
+    def settzinfo(self, tzinfo):
+        """
+        Set the default timezone to use with this query.
+        @param tzinfo: a L{Timezone} to use.
+        """
+
+        # Give tzinfo to any TimeRange we have
+        if isinstance(self.qualifier, TimeRange):
+            self.qualifier.settzinfo(tzinfo)
+
+
+    def getmaxtimerange(self, currentMaximum, currentIsStartTime):
+        """
+        Get the date farthest into the future in any time-range elements
+
+        @param currentMaximum: current future value to compare with
+        @type currentMaximum: L{DateTime}
+        """
+
+        # Give tzinfo to any TimeRange we have
+        isStartTime = False
+        if isinstance(self.qualifier, TimeRange):
+            isStartTime = self.qualifier.end is None
+            compareWith = self.qualifier.start if isStartTime else self.qualifier.end
+            if currentMaximum is None or currentMaximum < compareWith:
+                currentMaximum = compareWith
+                currentIsStartTime = isStartTime
+
+        return currentMaximum, currentIsStartTime
+
+
+    def getmintimerange(self, currentMinimum, currentIsEndTime):
+        """
+        Get the date farthest into the past in any time-range elements. That is either
+        the start date, or if start is not present, the end date.
+        """
+
+        # Give tzinfo to any TimeRange we have
+        isEndTime = False
+        if isinstance(self.qualifier, TimeRange):
+            isEndTime = self.qualifier.start is None
+            compareWith = self.qualifier.end if isEndTime else self.qualifier.start
+            if currentMinimum is None or currentMinimum > compareWith:
+                currentMinimum = compareWith
+                currentIsEndTime = isEndTime
+
+        return currentMinimum, currentIsEndTime
+
+
+
+class ParameterFilter (FilterChildBase):
+    """
+    Limits a search to specific parameters.
+    """
+
+    def _match(self, property, access):
+
+        # At least one parameter must match (or is-not-defined is set)
+        result = not self.defined
+        for parameterName in property.parameterNames():
+            if parameterName.upper() == self.filter_name.upper() and self.match([property.parameterValue(parameterName)], access):
+                result = self.defined
+                break
+
+        return result
+
+
+
+class IsNotDefined (FilterBase):
+    """
+    Specifies that the named iCalendar item does not exist.
+    """
+
+    def match(self, component, access=None):
+        # Oddly, this needs always to return True so that it appears there is
+        # a match - but we then "negate" the result if is-not-defined is set.
+        # Actually this method should never be called as we special case the
+        # is-not-defined option.
+        return True
+
+
+
+class TextMatch (FilterBase):
+    """
+    Specifies a substring match on a property or parameter value.
+    (CalDAV-access-09, section 9.6.4)
+    """
+    def __init__(self, xml_element):
+
+        super(TextMatch, self).__init__(xml_element)
+
+        self.text = str(xml_element)
+        if "caseless" in xml_element.attributes:
+            caseless = xml_element.attributes["caseless"]
+            if caseless == "yes":
+                self.caseless = True
+            elif caseless == "no":
+                self.caseless = False
+        else:
+            self.caseless = True
+
+        if "negate-condition" in xml_element.attributes:
+            negate = xml_element.attributes["negate-condition"]
+            if negate == "yes":
+                self.negate = True
+            elif negate == "no":
+                self.negate = False
+        else:
+            self.negate = False
+
+        if "match-type" in xml_element.attributes:
+            self.match_type = xml_element.attributes["match-type"]
+            if self.match_type not in (
+                "equals",
+                "contains",
+                "starts-with",
+                "ends-with",
+            ):
+                self.match_type = "contains"
+        else:
+            self.match_type = "contains"
+
+
+    def match(self, item, access):
+        """
+        Match the text for the item.
+        If the item is a property, then match the property value,
+        otherwise it may be a list of parameter values - try to match anyone of those
+        """
+        if item is None:
+            return False
+
+        if isinstance(item, Property):
+            values = [item.strvalue()]
+        else:
+            values = item
+
+        test = unicode(self.text, "utf-8")
+        if self.caseless:
+            test = test.lower()
+
+        def _textCompare(s):
+            if self.caseless:
+                s = s.lower()
+
+            if self.match_type == "equals":
+                return s == test
+            elif self.match_type == "contains":
+                return s.find(test) != -1
+            elif self.match_type == "starts-with":
+                return s.startswith(test)
+            elif self.match_type == "ends-with":
+                return s.endswith(test)
+            else:
+                return False
+
+        for value in values:
+            # NB Its possible that we have a text list value which appears as a Python list,
+            # so we need to check for that and iterate over the list.
+            if isinstance(value, list):
+                for subvalue in value:
+                    if _textCompare(unicode(subvalue, "utf-8")):
+                        return not self.negate
+            else:
+                if _textCompare(unicode(value, "utf-8")):
+                    return not self.negate
+
+        return self.negate
+
+
+
+class TimeRange (FilterBase):
+    """
+    Specifies a time for testing components against.
+    """
+
+    def __init__(self, xml_element):
+
+        super(TimeRange, self).__init__(xml_element)
+
+        # One of start or end must be present
+        if "start" not in xml_element.attributes and "end" not in xml_element.attributes:
+            raise ValueError("One of 'start' or 'end' must be present in CALDAV:time-range")
+
+        self.start = DateTime.parseText(xml_element.attributes["start"]) if "start" in xml_element.attributes else None
+        self.end = DateTime.parseText(xml_element.attributes["end"]) if "end" in xml_element.attributes else None
+        self.tzinfo = None
+
+
+    def settzinfo(self, tzinfo):
+        """
+        Set the default timezone to use with this query.
+        @param tzinfo: a L{Timezone} to use.
+        """
+
+        # Give tzinfo to any TimeRange we have
+        self.tzinfo = tzinfo
+
+
+    def valid(self, level=0):
+        """
+        Indicate whether the time-range is valid (must be date-time in UTC).
+
+        @return:      True if valid, False otherwise
+        """
+
+        if self.start is not None and self.start.isDateOnly():
+            log.info("start attribute in <time-range> is not a date-time: %s" % (self.start,))
+            return False
+        if self.end is not None and self.end.isDateOnly():
+            log.info("end attribute in <time-range> is not a date-time: %s" % (self.end,))
+            return False
+        if self.start is not None and not self.start.utc():
+            log.info("start attribute in <time-range> is not UTC: %s" % (self.start,))
+            return False
+        if self.end is not None and not self.end.utc():
+            log.info("end attribute in <time-range> is not UTC: %s" % (self.end,))
+            return False
+
+        # No other tests
+        return True
+
+
+    def match(self, property, access=None):
+        """
+        NB This is only called when doing a time-range match on a property.
+        """
+        if property is None:
+            return False
+        else:
+            return property.containsTimeRange(self.start, self.end, self.tzinfo)
+
+
+    def matchinstance(self, component, instances):
+        """
+        Test whether this time-range element causes a match to the specified component
+        using the specified set of instances to determine the expanded time ranges.
+        @param component: the L{Component} to test.
+        @param instances: the list of expanded instances.
+        @return: True if the time-range query matches, False otherwise.
+        """
+        if component is None:
+            return False
+
+        assert instances is not None or self.end is None, "Failure to expand instance for time-range filter: %r" % (self,)
+
+        # Special case open-ended unbounded
+        if instances is None:
+            if component.getRecurrenceIDUTC() is None:
+                return True
+            else:
+                # See if the overridden component's start is past the start
+                start, _ignore_end = component.getEffectiveStartEnd()
+                if start is None:
+                    return True
+                else:
+                    return start >= self.start
+
+        # Handle alarms as a special case
+        alarms = (component.name() == "VALARM")
+        if alarms:
+            testcomponent = component._parent
+        else:
+            testcomponent = component
+
+        for key in instances:
+            instance = instances[key]
+
+            # First make sure components match
+            if not testcomponent.same(instance.component):
+                continue
+
+            if alarms:
+                # Get all the alarm triggers for this instance and test each one
+                triggers = instance.getAlarmTriggers()
+                for trigger in triggers:
+                    if timeRangesOverlap(trigger, None, self.start, self.end, self.tzinfo):
+                        return True
+            else:
+                # Regular instance overlap test
+                if timeRangesOverlap(instance.start, instance.end, self.start, self.end, self.tzinfo):
+                    return True
+
+        return False

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/generator.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/generator.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/generator.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,207 @@
+##
+# Copyright (c) 2006-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.enterprise.dal.syntax import Select
+
+from txdav.common.datastore.query import expression
+from txdav.common.datastore.query.generator import SQLQueryGenerator
+from txdav.common.datastore.sql_tables import schema
+
+"""
+SQL statement generator from query expressions.
+"""
+
+__all__ = [
+    "CalDAVSQLQueryGenerator",
+]
+
+class CalDAVSQLQueryGenerator(SQLQueryGenerator):
+
+    _timerange = schema.TIME_RANGE
+    _transparency = schema.TRANSPARENCY
+
+    def __init__(self, expr, collection, whereid, userid=None, freebusy=False):
+        """
+
+        @param expr: the query expression object model
+        @type expr: L{expression}
+        @param collection: the resource targeted by the query
+        @type collection: L{CommonHomeChild}
+        @param userid: user for whom query is being done - query will be scoped to that user's privileges and their transparency
+        @type userid: C{str}
+        @param freebusy: whether or not a freebusy query is being done - if it is, additional time range and transparency information is returned
+        @type freebusy: C{bool}
+        """
+        super(CalDAVSQLQueryGenerator, self).__init__(expr, collection, whereid)
+        self.userid = userid if userid else ""
+        self.freebusy = freebusy
+        self.usedtimerange = False
+
+
+    def generate(self):
+        """
+        Generate the actual SQL statement from the passed in expression tree.
+
+        @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the partial SQL statement,
+            and the C{list} is the list of argument substitutions to use with the SQL API execute method.
+        """
+
+        # Init state
+        self.arguments = {}
+        self.argcount = 0
+        obj = self.collection._objectSchema
+
+        columns = [obj.RESOURCE_NAME, obj.ICALENDAR_UID, obj.ICALENDAR_TYPE]
+        if self.freebusy:
+            columns.extend([
+                obj.ORGANIZER,
+                self._timerange.FLOATING,
+                self._timerange.START_DATE,
+                self._timerange.END_DATE,
+                self._timerange.FBTYPE,
+                self._timerange.TRANSPARENT,
+                self._transparency.TRANSPARENT,
+            ])
+
+        # For SQL data DB we need to restrict the query to just the targeted calendar resource-id if provided
+        if self.whereid:
+
+            test = expression.isExpression(obj.CALENDAR_RESOURCE_ID, self.whereid, True)
+
+            # Since timerange expression already have the calendar resource-id test in them, do not
+            # add the additional term to those. When the additional term is added, add it as the first
+            # component in the AND expression to hopefully get the DB to use its index first
+
+            # Top-level timerange expression already has calendar resource-id restriction in it
+            if isinstance(self.expression, expression.timerangeExpression):
+                pass
+
+            # Top-level OR - check each component
+            elif isinstance(self.expression, expression.orExpression):
+
+                def _hasTopLevelTimerange(testexpr):
+                    if isinstance(testexpr, expression.timerangeExpression):
+                        return True
+                    elif isinstance(testexpr, expression.andExpression):
+                        return any([isinstance(expr, expression.timerangeExpression) for expr in testexpr.expressions])
+                    else:
+                        return False
+
+                hasTimerange = any([_hasTopLevelTimerange(expr) for expr in self.expression.expressions])
+
+                if hasTimerange:
+                    # timerange expression forces a join on calendarid
+                    pass
+                else:
+                    # AND the whole thing with calendarid
+                    self.expression = test.andWith(self.expression)
+
+            # Top-level AND - only add additional expression if timerange not present
+            elif isinstance(self.expression, expression.andExpression):
+                hasTimerange = any([isinstance(expr, expression.timerangeExpression) for expr in self.expression.expressions])
+                if not hasTimerange:
+                    # AND the whole thing
+                    self.expression = test.andWith(self.expression)
+
+            # Just use the id test
+            elif isinstance(self.expression, expression.allExpression):
+                self.expression = test
+
+            # Just AND the entire thing
+            else:
+                self.expression = test.andWith(self.expression)
+
+        # Generate ' where ...' partial statement
+        where = self.generateExpression(self.expression)
+
+        if self.usedtimerange:
+            where = where.And(self._timerange.CALENDAR_OBJECT_RESOURCE_ID == obj.RESOURCE_ID).And(self._timerange.CALENDAR_RESOURCE_ID == self.whereid)
+
+        # Set of tables depends on use of timespan and fb use
+        if self.usedtimerange:
+            if self.freebusy:
+                tables = obj.join(
+                    self._timerange.join(
+                        self._transparency,
+                        on=(self._timerange.INSTANCE_ID == self._transparency.TIME_RANGE_INSTANCE_ID).And(self._transparency.USER_ID == self.userid),
+                        type="left outer"
+                    ),
+                    type=","
+                )
+            else:
+                tables = obj.join(self._timerange, type=",")
+        else:
+            tables = obj
+
+        select = Select(
+            columns,
+            From=tables,
+            Where=where,
+            Distinct=True,
+        )
+
+        return select, self.arguments, self.usedtimerange
+
+
+    def generateExpression(self, expr):
+        """
+        Generate an expression and all it's subexpressions.
+
+        @param expr: the L{baseExpression} derived class to write out.
+        """
+
+        # Generate based on each type of expression we might encounter
+        partial = None
+
+        # time-range
+        if isinstance(expr, expression.timerangeExpression):
+            if expr.start and expr.end:
+                partial = (
+                    (self._timerange.FLOATING == False).And(self._timerange.START_DATE < expr.end).And(self._timerange.END_DATE > expr.start)
+                ).Or(
+                    (self._timerange.FLOATING == True).And(self._timerange.START_DATE < expr.endfloat).And(self._timerange.END_DATE > expr.startfloat)
+                )
+            elif expr.start and expr.end is None:
+                partial = (
+                    (self._timerange.FLOATING == False).And(self._timerange.END_DATE > expr.start)
+                ).Or(
+                    (self._timerange.FLOATING == True).And(self._timerange.END_DATE > expr.startfloat)
+                )
+            elif not expr.start and expr.end:
+                partial = (
+                    (self._timerange.FLOATING == False).And(self._timerange.START_DATE < expr.end)
+                ).Or(
+                    (self._timerange.FLOATING == True).And(self._timerange.START_DATE < expr.endfloat)
+                )
+            self.usedtimerange = True
+
+        else:
+            partial = super(CalDAVSQLQueryGenerator, self).generateExpression(expr)
+
+        return partial
+
+
+    def addArgument(self, arg):
+        """
+
+        @param arg: the C{str} of the argument to add
+        """
+
+        # Append argument to the list and add the appropriate substitution string to the output stream.
+        self.argcount += 1
+        argname = "arg{}".format(self.argcount)
+        self.arguments[argname] = arg
+        return argname

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/__init__.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/__init__.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/test_filter.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/test_filter.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/query/test/test_filter.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,220 @@
+##
+# Copyright (c) 2011-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from pycalendar.timezone import Timezone
+
+from twext.enterprise.dal.syntax import SQLFragment, Parameter
+
+from twistedcaldav.test.util import TestCase
+from twistedcaldav import caldavxml
+from twistedcaldav.timezones import TimezoneCache
+
+from txdav.caldav.datastore.index_file import sqlcalendarquery
+from txdav.caldav.datastore.query.builder import buildExpression
+from txdav.caldav.datastore.query.filter import Filter
+from txdav.caldav.datastore.query.generator import CalDAVSQLQueryGenerator
+from txdav.common.datastore.sql_tables import schema
+
+from dateutil.tz import tzutc
+import datetime
+
+class TestQueryFilter(TestCase):
+
+    _objectSchema = schema.CALENDAR_OBJECT
+    _queryFields = {
+        "UID": _objectSchema.UID,
+        "TYPE": _objectSchema.ICALENDAR_TYPE,
+    }
+
+    def setUp(self):
+        super(TestQueryFilter, self).setUp()
+        TimezoneCache.create()
+
+
+    def test_query(self):
+        """
+        Basic query test - no time range
+        """
+
+        filter = caldavxml.Filter(
+            caldavxml.ComponentFilter(
+                *[caldavxml.ComponentFilter(
+                    **{"name":("VEVENT", "VFREEBUSY", "VAVAILABILITY")}
+                )],
+                **{"name": "VCALENDAR"}
+            )
+        )
+        filter = Filter(filter)
+        filter.child.settzinfo(Timezone(tzid="America/New_York"))
+
+        expression = buildExpression(filter, self._queryFields)
+        sql = CalDAVSQLQueryGenerator(expression, self, 1234)
+        select, args, usedtimerange = sql.generate()
+
+        self.assertEqual(select.toSQL(), SQLFragment(
+            "select distinct RESOURCE_NAME, ICALENDAR_UID, ICALENDAR_TYPE from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ? and ICALENDAR_TYPE in (?, ?, ?)",
+            [1234, Parameter('arg1', 3)]
+        ))
+        self.assertEqual(args, {"arg1": ("VEVENT", "VFREEBUSY", "VAVAILABILITY")})
+        self.assertEqual(usedtimerange, False)
+
+
+    def test_query_timerange(self):
+        """
+        Basic query test - with time range
+        """
+
+        filter = caldavxml.Filter(
+            caldavxml.ComponentFilter(
+                *[caldavxml.ComponentFilter(
+                    *[caldavxml.TimeRange(**{"start":"20060605T160000Z", "end":"20060605T170000Z"})],
+                    **{"name":("VEVENT", "VFREEBUSY", "VAVAILABILITY")}
+                )],
+                **{"name": "VCALENDAR"}
+            )
+        )
+        filter = Filter(filter)
+        filter.child.settzinfo(Timezone(tzid="America/New_York"))
+
+        expression = buildExpression(filter, self._queryFields)
+        sql = CalDAVSQLQueryGenerator(expression, self, 1234)
+        select, args, usedtimerange = sql.generate()
+
+        self.assertEqual(select.toSQL(), SQLFragment(
+            "select distinct RESOURCE_NAME, ICALENDAR_UID, ICALENDAR_TYPE from CALENDAR_OBJECT, TIME_RANGE where ICALENDAR_TYPE in (?, ?, ?) and (FLOATING = ? and START_DATE < ? and END_DATE > ? or FLOATING = ? and START_DATE < ? and END_DATE > ?) and CALENDAR_OBJECT_RESOURCE_ID = RESOURCE_ID and TIME_RANGE.CALENDAR_RESOURCE_ID = ?",
+            [Parameter('arg1', 3), False, datetime.datetime(2006, 6, 5, 17, 0, tzinfo=tzutc()), datetime.datetime(2006, 6, 5, 16, 0, tzinfo=tzutc()), True, datetime.datetime(2006, 6, 5, 13, 0, tzinfo=tzutc()), datetime.datetime(2006, 6, 5, 12, 0, tzinfo=tzutc()), 1234]
+        ))
+        self.assertEqual(args, {"arg1": ("VEVENT", "VFREEBUSY", "VAVAILABILITY")})
+        self.assertEqual(usedtimerange, True)
+
+
+    def test_query_freebusy(self):
+        """
+        Basic query test - with time range
+        """
+
+        filter = caldavxml.Filter(
+            caldavxml.ComponentFilter(
+                *[caldavxml.ComponentFilter(
+                    *[caldavxml.TimeRange(**{"start":"20060605T160000Z", "end":"20060605T170000Z"})],
+                    **{"name":("VEVENT", "VFREEBUSY", "VAVAILABILITY")}
+                )],
+                **{"name": "VCALENDAR"}
+            )
+        )
+        filter = Filter(filter)
+        filter.child.settzinfo(Timezone(tzid="America/New_York"))
+
+        expression = buildExpression(filter, self._queryFields)
+        sql = CalDAVSQLQueryGenerator(expression, self, 1234, "user01", True)
+        select, args, usedtimerange = sql.generate()
+
+        self.assertEqual(select.toSQL(), SQLFragment(
+            "select distinct RESOURCE_NAME, ICALENDAR_UID, ICALENDAR_TYPE, ORGANIZER, FLOATING, START_DATE, END_DATE, FBTYPE, TIME_RANGE.TRANSPARENT, TRANSPARENCY.TRANSPARENT from CALENDAR_OBJECT, TIME_RANGE left outer join TRANSPARENCY on INSTANCE_ID = TIME_RANGE_INSTANCE_ID and USER_ID = ? where ICALENDAR_TYPE in (?, ?, ?) and (FLOATING = ? and START_DATE < ? and END_DATE > ? or FLOATING = ? and START_DATE < ? and END_DATE > ?) and CALENDAR_OBJECT_RESOURCE_ID = RESOURCE_ID and TIME_RANGE.CALENDAR_RESOURCE_ID = ?",
+            ['user01', Parameter('arg1', 3), False, datetime.datetime(2006, 6, 5, 17, 0, tzinfo=tzutc()), datetime.datetime(2006, 6, 5, 16, 0, tzinfo=tzutc()), True, datetime.datetime(2006, 6, 5, 13, 0, tzinfo=tzutc()), datetime.datetime(2006, 6, 5, 12, 0, tzinfo=tzutc()), 1234]
+        ))
+        self.assertEqual(args, {"arg1": ("VEVENT", "VFREEBUSY", "VAVAILABILITY")})
+        self.assertEqual(usedtimerange, True)
+
+
+    def test_query_not_extended(self):
+        """
+        Query test - two terms not anyof
+        """
+
+        filter = caldavxml.Filter(
+            caldavxml.ComponentFilter(
+                *[
+                    caldavxml.ComponentFilter(
+                        **{"name":("VEVENT")}
+                    ),
+                    caldavxml.ComponentFilter(
+                        **{"name":("VTODO")}
+                    ),
+                ],
+                **{"name": "VCALENDAR"}
+            )
+        )
+        filter = Filter(filter)
+        filter.child.settzinfo(Timezone(tzid="America/New_York"))
+
+        expression = buildExpression(filter, self._queryFields)
+        sql = CalDAVSQLQueryGenerator(expression, self, 1234)
+        select, args, usedtimerange = sql.generate()
+
+        self.assertEqual(select.toSQL(), SQLFragment(
+            "select distinct RESOURCE_NAME, ICALENDAR_UID, ICALENDAR_TYPE from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ? and ICALENDAR_TYPE = ? and ICALENDAR_TYPE = ?",
+            [1234, "VEVENT", "VTODO"]
+        ))
+        self.assertEqual(args, {})
+        self.assertEqual(usedtimerange, False)
+
+
+    def test_query_extended(self):
+        """
+        Extended query test - two terms with anyof
+        """
+
+        filter = caldavxml.Filter(
+            caldavxml.ComponentFilter(
+                *[
+                    caldavxml.ComponentFilter(
+                        *[caldavxml.TimeRange(**{"start":"20060605T160000Z", })],
+                        **{"name":("VEVENT")}
+                    ),
+                    caldavxml.ComponentFilter(
+                        **{"name":("VTODO")}
+                    ),
+                ],
+                **{"name": "VCALENDAR", "test": "anyof"}
+            )
+        )
+        filter = Filter(filter)
+        filter.child.settzinfo(Timezone(tzid="America/New_York"))
+
+        expression = buildExpression(filter, self._queryFields)
+        sql = CalDAVSQLQueryGenerator(expression, self, 1234)
+        select, args, usedtimerange = sql.generate()
+
+        self.assertEqual(select.toSQL(), SQLFragment(
+            "select distinct RESOURCE_NAME, ICALENDAR_UID, ICALENDAR_TYPE from CALENDAR_OBJECT, TIME_RANGE where (ICALENDAR_TYPE = ? and (FLOATING = ? and END_DATE > ? or FLOATING = ? and END_DATE > ?) or ICALENDAR_TYPE = ?) and CALENDAR_OBJECT_RESOURCE_ID = RESOURCE_ID and TIME_RANGE.CALENDAR_RESOURCE_ID = ?",
+            ['VEVENT', False, datetime.datetime(2006, 6, 5, 16, 0, tzinfo=tzutc()), True, datetime.datetime(2006, 6, 5, 12, 0, tzinfo=tzutc()), 'VTODO', 1234]
+        ))
+        self.assertEqual(args, {})
+        self.assertEqual(usedtimerange, True)
+
+
+    def test_sqllite_query(self):
+        """
+        Basic query test - single term.
+        Only UID can be queried via sql.
+        """
+
+        filter = caldavxml.Filter(
+            caldavxml.ComponentFilter(
+                *[caldavxml.ComponentFilter(
+                    **{"name":("VEVENT", "VFREEBUSY", "VAVAILABILITY")}
+                )],
+                **{"name": "VCALENDAR"}
+            )
+        )
+        filter = Filter(filter)
+        sql, args = sqlcalendarquery(filter, 1234)
+
+        self.assertTrue(sql.find("RESOURCE") != -1)
+        self.assertTrue(sql.find("TIMESPAN") == -1)
+        self.assertTrue(sql.find("TRANSPARENCY") == -1)
+        self.assertTrue("VEVENT" in args)

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/scheduling/freebusy.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/scheduling/freebusy.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/scheduling/freebusy.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -32,10 +32,11 @@
 from twistedcaldav.ical import Component, Property, iCalendarProductID
 from twistedcaldav.instance import InstanceList
 from twistedcaldav.memcacher import Memcacher
-from twistedcaldav.query import calendarqueryfilter
 
+from txdav.caldav.datastore.query.filter import Filter
 from txdav.caldav.icalendarstore import QueryMaxResources
-from txdav.common.icommondatastore import IndexedSearchException
+from txdav.common.icommondatastore import IndexedSearchException, \
+    InternalDataStoreError
 
 import uuid
 
@@ -282,23 +283,23 @@
 
         # Create fake filter element to match time-range
         filter = caldavxml.Filter(
-                      caldavxml.ComponentFilter(
-                          caldavxml.ComponentFilter(
-                              cache_timerange if caching else timerange,
-                              name=("VEVENT", "VFREEBUSY", "VAVAILABILITY"),
-                          ),
-                          name="VCALENDAR",
-                       )
-                  )
-        filter = calendarqueryfilter.Filter(filter)
+            caldavxml.ComponentFilter(
+                caldavxml.ComponentFilter(
+                    cache_timerange if caching else timerange,
+                    name=("VEVENT", "VFREEBUSY", "VAVAILABILITY"),
+                ),
+                name="VCALENDAR",
+            )
+        )
+        filter = Filter(filter)
         tzinfo = filter.settimezone(tz)
 
         try:
-            resources = yield calresource._index.indexedSearch(filter, useruid=attendee_uid, fbtype=True)
+            resources = yield calresource.search(filter, useruid=attendee_uid, fbtype=True)
             if caching:
                 yield FBCacheEntry.makeCacheEntry(calresource, attendee_uid, cache_timerange, resources)
         except IndexedSearchException:
-            resources = yield calresource._index.bruteForceSearch()
+            raise InternalDataStoreError("Invalid indexedSearch query")
 
     else:
         # Log extended item

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/sql.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/sql.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/sql.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -57,6 +57,9 @@
 from twistedcaldav.memcacher import Memcacher
 
 from txdav.base.propertystore.base import PropertyName
+from txdav.caldav.datastore.query.builder import buildExpression
+from txdav.caldav.datastore.query.filter import Filter
+from txdav.caldav.datastore.query.generator import CalDAVSQLQueryGenerator
 from txdav.caldav.datastore.scheduling.icalsplitter import iCalSplitter
 from txdav.caldav.datastore.scheduling.implicit import ImplicitScheduler
 from txdav.caldav.datastore.util import AttachmentRetrievalTransport, \
@@ -72,12 +75,11 @@
     AttendeeAllowedError, InvalidPerUserDataMerge, ComponentUpdateState, \
     ValidOrganizerError, ShareeAllowedError, ComponentRemoveState, \
     InvalidDefaultCalendar, \
-    InvalidAttachmentOperation, DuplicatePrivateCommentsError
+    InvalidAttachmentOperation, DuplicatePrivateCommentsError, \
+    TimeRangeUpperLimit, TimeRangeLowerLimit
 from txdav.caldav.icalendarstore import QuotaExceeded
 from txdav.common.datastore.sql import CommonHome, CommonHomeChild, \
     CommonObjectResource, ECALENDARTYPE
-from txdav.common.datastore.sql_legacy import PostgresLegacyIndexEmulator, \
-    PostgresLegacyInboxIndexEmulator
 from txdav.common.datastore.sql_tables import _ATTACHMENTS_MODE_NONE, \
     _ATTACHMENTS_MODE_WRITE, schema, _BIND_MODE_OWN, \
     _ATTACHMENTS_MODE_READ, _TRANSP_OPAQUE, _TRANSP_TRANSPARENT
@@ -87,7 +89,8 @@
     InvalidObjectResourceError, ObjectResourceNameAlreadyExistsError, \
     ObjectResourceNameNotAllowedError, TooManyObjectResourcesError, \
     InvalidUIDError, UIDExistsError, UIDExistsElsewhereError, \
-    InvalidResourceMove, InvalidComponentForStoreError
+    InvalidResourceMove, InvalidComponentForStoreError, \
+    NoSuchObjectResourceError
 from txdav.xml import element
 
 from txdav.idav import ChangeCategory
@@ -949,6 +952,12 @@
     _objectSchema = schema.CALENDAR_OBJECT
     _timeRangeSchema = schema.TIME_RANGE
 
+    # Mapping of iCalendar property name to DB column name
+    _queryFields = {
+        "UID": _objectSchema.UID,
+        "TYPE": _objectSchema.ICALENDAR_TYPE,
+    }
+
     _supportedComponents = None
 
     def __init__(self, *args, **kw):
@@ -956,10 +965,6 @@
         Initialize a calendar pointing at a record in a database.
         """
         super(Calendar, self).__init__(*args, **kw)
-        if self.isInbox():
-            self._index = PostgresLegacyInboxIndexEmulator(self)
-        else:
-            self._index = PostgresLegacyIndexEmulator(self)
         self._transp = _TRANSP_OPAQUE
 
 
@@ -1335,6 +1340,189 @@
 
 
     @inlineCallbacks
+    def search(self, filter, useruid=None, fbtype=False):
+        """
+        Finds resources matching the given qualifiers.
+        @param filter: the L{Filter} for the calendar-query to execute.
+        @return: an iterable of tuples for each resource matching the
+            given C{qualifiers}. The tuples are C{(name, uid)}, where
+            C{name} is the resource name, C{uid} is the resource UID.
+        """
+
+        # Make sure we have a proper Filter element and get the partial SQL statement to use.
+        sql_stmt = self._sqlquery(filter, useruid, fbtype)
+
+        # No result means it is too complex for us
+        if sql_stmt is None:
+            raise IndexedSearchException()
+        sql_stmt, args, usedtimerange = sql_stmt
+
+        # Check for time-range re-expand
+        if usedtimerange is not None:
+
+            today = DateTime.getToday()
+
+            # Determine how far we need to extend the current expansion of
+            # events. If we have an open-ended time-range we will expand
+            # one year past the start. That should catch bounded
+            # recurrences - unbounded will have been indexed with an
+            # "infinite" value always included.
+            maxDate, isStartDate = filter.getmaxtimerange()
+            if maxDate:
+                maxDate = maxDate.duplicate()
+                maxDate.offsetDay(1)
+                maxDate.setDateOnly(True)
+                upperLimit = today + Duration(days=config.FreeBusyIndexExpandMaxDays)
+                if maxDate > upperLimit:
+                    raise TimeRangeUpperLimit(upperLimit)
+                if isStartDate:
+                    maxDate += Duration(days=365)
+
+            # Determine if the start date is too early for the restricted range we
+            # are applying. If it is today or later we don't need to worry about truncation
+            # in the past.
+            minDate, _ignore_isEndDate = filter.getmintimerange()
+            if minDate >= today:
+                minDate = None
+            if minDate is not None and config.FreeBusyIndexLowerLimitDays:
+                truncateLowerLimit = today - Duration(days=config.FreeBusyIndexLowerLimitDays)
+                if minDate < truncateLowerLimit:
+                    raise TimeRangeLowerLimit(truncateLowerLimit)
+
+            if maxDate is not None or minDate is not None:
+                yield self.testAndUpdateIndex(minDate, maxDate)
+
+        rowiter = yield sql_stmt.on(self._txn, **args)
+
+        # Check result for missing resources
+        results = []
+        for row in rowiter:
+            if fbtype:
+                row = list(row)
+                row[4] = 'Y' if row[4] else 'N'
+                row[7] = indexfbtype_to_icalfbtype[row[7]]
+                if row[9] is not None:
+                    row[8] = row[9]
+                row[8] = 'T' if row[8] else 'F'
+                del row[9]
+            results.append(row)
+
+        returnValue(results)
+
+
+    def _sqlquery(self, filter, useruid, fbtype):
+        """
+        Convert the supplied addressbook-query into a partial SQL statement.
+
+        @param filter: the L{Filter} for the addressbook-query to convert.
+        @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the partial SQL statement,
+                and the C{list} is the list of argument substitutions to use with the SQL API execute method.
+                Or return C{None} if it is not possible to create an SQL query to fully match the addressbook-query.
+        """
+
+        if not isinstance(filter, Filter):
+            return None
+
+        try:
+            expression = buildExpression(filter, self._queryFields)
+            sql = CalDAVSQLQueryGenerator(expression, self, self.id(), useruid, fbtype)
+            return sql.generate()
+        except ValueError:
+            return None
+
+
+    @classproperty
+    def _notExpandedWithinQuery(cls): #@NoSelf
+        """
+        Query to find resources that need to be re-expanded
+        """
+        co = schema.CALENDAR_OBJECT
+        return Select(
+            [co.RESOURCE_NAME],
+            From=co,
+            Where=((co.RECURRANCE_MIN > Parameter("minDate"))
+                .Or(co.RECURRANCE_MAX < Parameter("maxDate")))
+                .And(co.CALENDAR_RESOURCE_ID == Parameter("resourceID"))
+        )
+
+
+    @inlineCallbacks
+    def notExpandedWithin(self, minDate, maxDate):
+        """
+        Gives all resources which have not been expanded beyond a given date
+        in the database.  (Unused; see above L{postgresqlgenerator}.
+        """
+        returnValue([row[0] for row in (
+            yield self._notExpandedWithinQuery.on(
+                self._txn,
+                minDate=pyCalendarTodatetime(normalizeForIndex(minDate)) if minDate is not None else None,
+                maxDate=pyCalendarTodatetime(normalizeForIndex(maxDate)),
+                resourceID=self._resourceID))]
+        )
+
+
+    @inlineCallbacks
+    def reExpandResource(self, name, expand_start, expand_end):
+        """
+        Given a resource name, remove it from the database and re-add it
+        with a longer expansion.
+        """
+        obj = yield self.calendarObjectWithName(name)
+
+        # Use a new transaction to do this update quickly without locking the row for too long. However, the original
+        # transaction may have the row locked, so use wait=False and if that fails, fall back to using the original txn.
+
+        newTxn = obj.transaction().store().newTransaction()
+        try:
+            yield obj.lock(wait=False, txn=newTxn)
+        except NoSuchObjectResourceError:
+            yield newTxn.commit()
+            returnValue(None)
+        except:
+            yield newTxn.abort()
+            newTxn = None
+
+        # Now do the re-expand using the appropriate transaction
+        try:
+            doExpand = False
+            if newTxn is None:
+                doExpand = True
+            else:
+                # We repeat this check because the resource may have been re-expanded by someone else
+                rmin, rmax = (yield obj.recurrenceMinMax(txn=newTxn))
+
+                # If the resource is not fully expanded, see if within the required range or not.
+                # Note that expand_start could be None if no lower limit is applied, but expand_end will
+                # never be None
+                if rmax is not None and rmax < expand_end:
+                    doExpand = True
+                if rmin is not None and expand_start is not None and rmin > expand_start:
+                    doExpand = True
+
+            if doExpand:
+                yield obj.updateDatabase(
+                    (yield obj.component()),
+                    expand_until=expand_end,
+                    reCreate=True,
+                    txn=newTxn,
+                )
+        finally:
+            if newTxn is not None:
+                yield newTxn.commit()
+
+
+    @inlineCallbacks
+    def testAndUpdateIndex(self, minDate, maxDate):
+        # Find out if the index is expanded far enough
+        names = yield self.notExpandedWithin(minDate, maxDate)
+
+        # Actually expand recurrence max
+        for name in names:
+            self.log.info("Search falls outside range of index for %s %s to %s" % (name, minDate, maxDate))
+            yield self.reExpandResource(name, minDate, maxDate)
+
+
+    @inlineCallbacks
     def splitCollectionByComponentTypes(self):
         """
         If the calendar contains iCalendar data with different component types, then split it into separate collections
@@ -2179,7 +2367,10 @@
         """
 
         if isinstance(component, str) or isinstance(component, unicode):
-            component = self._componentClass.fromString(component)
+            try:
+                component = self._componentClass.fromString(component)
+            except InvalidICalendarDataError as e:
+                raise InvalidComponentForStoreError(str(e))
         return self._setComponentInternal(component, inserting, ComponentUpdateState.NORMAL, smart_merge)
 
 

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_index_file.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_index_file.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_index_file.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -19,6 +19,7 @@
 from twisted.internet.task import deferLater
 
 from txdav.caldav.datastore.index_file import Index, MemcachedUIDReserver
+from txdav.caldav.datastore.query.filter import Filter
 from txdav.common.icommondatastore import ReservationError, \
     InternalDataStoreError
 
@@ -26,7 +27,6 @@
 from twistedcaldav.caldavxml import TimeRange
 from twistedcaldav.ical import Component, InvalidICalendarDataError
 from twistedcaldav.instance import InvalidOverriddenInstanceError
-from twistedcaldav.query import calendarqueryfilter
 from twistedcaldav.test.util import InMemoryMemcacheProtocol
 import twistedcaldav.test.util
 
@@ -480,7 +480,7 @@
                       name="VCALENDAR",
                    )
               )
-            filter = calendarqueryfilter.Filter(filter)
+            filter = Filter(filter)
 
             resources = yield self.db.indexedSearch(filter)
             index_results = set()
@@ -666,7 +666,7 @@
                       name="VCALENDAR",
                    )
               )
-            filter = calendarqueryfilter.Filter(filter)
+            filter = Filter(filter)
 
             resources = yield self.db.indexedSearch(filter, fbtype=True)
             index_results = set()
@@ -1073,7 +1073,7 @@
                       name="VCALENDAR",
                    )
               )
-            filter = calendarqueryfilter.Filter(filter)
+            filter = Filter(filter)
 
             for useruid, instances in peruserinstances:
                 resources = yield self.db.indexedSearch(filter, useruid=useruid, fbtype=True)

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_sql.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_sql.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/caldav/datastore/test/test_sql.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -13,14 +13,6 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 ##
-from txdav.caldav.datastore.scheduling.processing import ImplicitProcessor
-from txdav.caldav.datastore.scheduling.cuaddress import RemoteCalendarUser, \
-    LocalCalendarUser
-from txdav.caldav.datastore.scheduling.caldav.scheduler import CalDAVScheduler
-from txdav.caldav.datastore.scheduling.scheduler import ScheduleResponseQueue
-from twext.web2 import responsecode
-from txdav.caldav.datastore.scheduling.itip import iTIPRequestStatus
-from twistedcaldav.instance import InvalidOverriddenInstanceError
 
 """
 Tests for txdav.caldav.datastore.postgres, mostly based on
@@ -33,6 +25,7 @@
 from twext.enterprise.dal.syntax import Select, Parameter, Insert, Delete, \
     Update
 from twext.python.vcomponent import VComponent
+from twext.web2 import responsecode
 from twext.web2.http_headers import MimeType
 from twext.web2.stream import MemoryStream
 
@@ -47,9 +40,16 @@
 from twistedcaldav.config import config
 from twistedcaldav.dateops import datetimeMktime
 from twistedcaldav.ical import Component, normalize_iCalStr, diff_iCalStrs
-from twistedcaldav.query import calendarqueryfilter
+from twistedcaldav.instance import InvalidOverriddenInstanceError
 
 from txdav.base.propertystore.base import PropertyName
+from txdav.caldav.datastore.query.filter import Filter
+from txdav.caldav.datastore.scheduling.caldav.scheduler import CalDAVScheduler
+from txdav.caldav.datastore.scheduling.cuaddress import RemoteCalendarUser, \
+    LocalCalendarUser
+from txdav.caldav.datastore.scheduling.itip import iTIPRequestStatus
+from txdav.caldav.datastore.scheduling.processing import ImplicitProcessor
+from txdav.caldav.datastore.scheduling.scheduler import ScheduleResponseQueue
 from txdav.caldav.datastore.test.common import CommonTests as CalendarCommonTests, \
     test_event_text
 from txdav.caldav.datastore.test.test_file import setUpCalendarStore
@@ -57,14 +57,13 @@
 from txdav.caldav.datastore.util import _migrateCalendar, migrateHome
 from txdav.caldav.icalendarstore import ComponentUpdateState, InvalidDefaultCalendar
 from txdav.common.datastore.sql import ECALENDARTYPE, CommonObjectResource
-from txdav.common.datastore.sql_legacy import PostgresLegacyIndexEmulator
 from txdav.common.datastore.sql_tables import schema, _BIND_MODE_DIRECT, \
     _BIND_STATUS_ACCEPTED
 from txdav.common.datastore.test.util import populateCalendarsFrom, \
     CommonCommonTests
 from txdav.common.icommondatastore import NoSuchObjectResourceError
-from txdav.xml.rfc2518 import GETContentLanguage, ResourceType
 from txdav.idav import ChangeCategory
+from txdav.xml.rfc2518 import GETContentLanguage, ResourceType
 
 import datetime
 
@@ -407,10 +406,10 @@
                           name="VCALENDAR",
                        )
                   )
-        filter = calendarqueryfilter.Filter(filter)
+        filter = Filter(filter)
         filter.settimezone(None)
 
-        results = yield toCalendar._index.indexedSearch(filter, 'user01', True)
+        results = yield toCalendar.search(filter, 'user01', True)
         self.assertEquals(len(results), 1)
         _ignore_name, uid, _ignore_type, _ignore_organizer, _ignore_float, _ignore_start, _ignore_end, _ignore_fbtype, transp = results[0]
         self.assertEquals(uid, "uid4")
@@ -1369,7 +1368,7 @@
     @inlineCallbacks
     def test_notExpandedWithin(self):
         """
-        Test PostgresLegacyIndexEmulator.notExpandedWithin to make sure it returns the correct
+        Test Calendar.notExpandedWithin to make sure it returns the correct
         result based on the ranges passed in.
         """
 
@@ -1378,7 +1377,6 @@
         # Create the index on a new calendar
         home = yield self.homeUnderTest()
         newcalendar = yield home.createCalendarWithName("index_testing")
-        index = PostgresLegacyIndexEmulator(newcalendar)
 
         # Create the calendar object to use for testing
         nowYear = self.nowYear["now"]
@@ -1406,37 +1404,37 @@
         # Fully within range
         testMin = DateTime(nowYear, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
         testMax = DateTime(nowYear + 1, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
-        result = yield index.notExpandedWithin(testMin, testMax)
+        result = yield newcalendar.notExpandedWithin(testMin, testMax)
         self.assertEqual(result, [])
 
         # Upper bound exceeded
         testMin = DateTime(nowYear, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
         testMax = DateTime(nowYear + 5, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
-        result = yield index.notExpandedWithin(testMin, testMax)
+        result = yield newcalendar.notExpandedWithin(testMin, testMax)
         self.assertEqual(result, ["indexing.ics"])
 
         # Lower bound exceeded
         testMin = DateTime(nowYear - 5, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
         testMax = DateTime(nowYear + 1, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
-        result = yield index.notExpandedWithin(testMin, testMax)
+        result = yield newcalendar.notExpandedWithin(testMin, testMax)
         self.assertEqual(result, ["indexing.ics"])
 
         # Lower and upper bounds exceeded
         testMin = DateTime(nowYear - 5, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
         testMax = DateTime(nowYear + 5, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
-        result = yield index.notExpandedWithin(testMin, testMax)
+        result = yield newcalendar.notExpandedWithin(testMin, testMax)
         self.assertEqual(result, ["indexing.ics"])
 
         # Lower none within range
         testMin = None
         testMax = DateTime(nowYear + 1, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
-        result = yield index.notExpandedWithin(testMin, testMax)
+        result = yield newcalendar.notExpandedWithin(testMin, testMax)
         self.assertEqual(result, [])
 
         # Lower none and upper bounds exceeded
         testMin = None
         testMax = DateTime(nowYear + 5, 1, 1, 0, 0, 0, tzid=Timezone(utc=True))
-        result = yield index.notExpandedWithin(testMin, testMax)
+        result = yield newcalendar.notExpandedWithin(testMin, testMax)
         self.assertEqual(result, ["indexing.ics"])
 
 

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/index_file.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/index_file.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/index_file.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -37,10 +37,6 @@
 
 from twisted.internet.defer import maybeDeferred
 
-from twistedcaldav import carddavxml
-from txdav.common.icommondatastore import SyncTokenValidException, \
-    ReservationError
-from twistedcaldav.query import addressbookquery
 from twistedcaldav.sql import AbstractSQLDatabase
 from twistedcaldav.sql import db_prefix
 from twistedcaldav.vcard import Component
@@ -49,6 +45,12 @@
 from twistedcaldav.config import config
 from twistedcaldav.memcachepool import CachePoolUserMixIn
 
+from txdav.carddav.datastore.query.builder import buildExpression
+from txdav.carddav.datastore.query.filter import Filter
+from txdav.common.datastore.query.filegenerator import sqllitegenerator
+from txdav.common.icommondatastore import SyncTokenValidException, \
+    ReservationError
+
 log = Logger()
 
 db_basename = db_prefix + "sqlite"
@@ -218,6 +220,24 @@
 
 
 
+def sqladdressbookquery(filter, addressbookid=None):
+    """
+    Convert the supplied addressbook-query into a partial SQL statement.
+
+    @param filter: the L{Filter} for the addressbook-query to convert.
+    @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the partial SQL statement,
+            and the C{list} is the list of argument substitutions to use with the SQL API execute method.
+            Or return C{None} if it is not possible to create an SQL query to fully match the addressbook-query.
+    """
+    try:
+        expression = buildExpression(filter, sqllitegenerator.FIELDS)
+        sql = sqllitegenerator(expression, addressbookid, None)
+        return sql.generate()
+    except ValueError:
+        return None
+
+
+
 class AddressBookIndex(AbstractSQLDatabase):
     """
     AddressBook collection index abstract base class that defines the apis for the index.
@@ -445,8 +465,8 @@
 
 
     def searchValid(self, filter):
-        if isinstance(filter, carddavxml.Filter):
-            qualifiers = addressbookquery.sqladdressbookquery(filter)
+        if isinstance(filter, Filter):
+            qualifiers = sqladdressbookquery(filter)
         else:
             qualifiers = None
 
@@ -466,8 +486,8 @@
         # start caching...
 
         # Make sure we have a proper Filter element and get the partial SQL statement to use.
-        if isinstance(filter, carddavxml.Filter):
-            qualifiers = addressbookquery.sqladdressbookquery(filter)
+        if isinstance(filter, Filter):
+            qualifiers = sqladdressbookquery(filter)
         else:
             qualifiers = None
         if qualifiers is not None:

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/__init__.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/__init__.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/builder.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/builder.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/builder.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,107 @@
+##
+# Copyright (c) 2006-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from txdav.common.datastore.query import expression
+from txdav.carddav.datastore.query.filter import TextMatch
+
+"""
+SQL statement generator from query expressions.
+"""
+
+__all__ = [
+    "buildExpression",
+]
+
+
+
+# SQL Index column (field) names
+
+def buildExpression(filter, fields):
+    """
+    Convert the supplied addressbook-query into an expression tree.
+
+    @param filter: the L{Filter} for the addressbook-query to convert.
+    @return: a L{baseExpression} for the expression tree.
+    """
+    # Lets assume we have a valid filter from the outset.
+
+    # Top-level filter contains zero or more prop-filter element
+    if len(filter.children) > 0:
+        return propfilterListExpression(filter.children, fields)
+    else:
+        return expression.allExpression()
+
+
+
+def propfilterListExpression(propfilters, fields):
+    """
+    Create an expression for a list of prop-filter elements.
+
+    @param propfilters: the C{list} of L{ComponentFilter} elements.
+    @return: a L{baseExpression} for the expression tree.
+    """
+
+    if len(propfilters) == 1:
+        return propfilterExpression(propfilters[0], fields)
+    else:
+        return expression.orExpression([propfilterExpression(c, fields) for c in propfilters])
+
+
+
+def propfilterExpression(propfilter, fields):
+    """
+    Create an expression for a single prop-filter element.
+
+    @param propfilter: the L{PropertyFilter} element.
+    @return: a L{baseExpression} for the expression tree.
+    """
+
+    # Only handle UID right now
+    if propfilter.filter_name != "UID":
+        raise ValueError
+
+    # Handle is-not-defined case
+    if not propfilter.defined:
+        # Test for <<field>> != "*"
+        return expression.isExpression(fields["UID"], "", True)
+
+    # Handle embedded parameters/text-match
+    params = []
+    for filter in propfilter.filters:
+        if isinstance(filter, TextMatch):
+            if filter.match_type == "equals":
+                tm = expression.isnotExpression if filter.negate else expression.isExpression
+            elif filter.match_type == "contains":
+                tm = expression.notcontainsExpression if filter.negate else expression.containsExpression
+            elif filter.match_type == "starts-with":
+                tm = expression.notstartswithExpression if filter.negate else expression.startswithExpression
+            elif filter.match_type == "ends-with":
+                tm = expression.notendswithExpression if filter.negate else expression.endswithExpression
+            params.append(tm(fields[propfilter.filter_name], str(filter.text), True))
+        else:
+            # No embedded parameters - not right now as our Index does not handle them
+            raise ValueError
+
+    # Now build return expression
+    if len(params) > 1:
+        if propfilter.propfilter_test == "anyof":
+            return expression.orExpression(params)
+        else:
+            return expression.andExpression(params)
+    elif len(params) == 1:
+        return params[0]
+    else:
+        return None

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/filter.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/filter.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/filter.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,314 @@
+##
+# Copyright (c) 2011-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+Object model of CARDDAV:filter element used in an addressbook-query.
+"""
+
+__all__ = [
+    "Filter",
+]
+
+from twext.python.log import Logger
+
+from twistedcaldav.carddavxml import carddav_namespace
+from twistedcaldav.vcard import Property
+
+log = Logger()
+
+class FilterBase(object):
+    """
+    Determines which matching components are returned.
+    """
+
+    def __init__(self, xml_element):
+        self.xmlelement = xml_element
+
+
+    def match(self, item, access=None):
+        raise NotImplementedError
+
+
+    def valid(self, level=0):
+        raise NotImplementedError
+
+
+
+class Filter(FilterBase):
+    """
+    Determines which matching components are returned.
+    """
+
+    def __init__(self, xml_element):
+
+        super(Filter, self).__init__(xml_element)
+
+        filter_test = xml_element.attributes.get("test", "anyof")
+        if filter_test not in ("anyof", "allof"):
+            raise ValueError("Test must be only one of anyof, allof")
+
+        self.filter_test = filter_test
+
+        self.children = [PropertyFilter(child) for child in xml_element.children]
+
+
+    def match(self, vcard):
+        """
+        Returns True if the given address property matches this filter, False
+        otherwise. Empty element means always match.
+        """
+
+        if len(self.children) > 0:
+            allof = self.filter_test == "allof"
+            for propfilter in self.children:
+                if allof != propfilter._match(vcard):
+                    return not allof
+            return allof
+        else:
+            return True
+
+
+    def valid(self):
+        """
+        Indicate whether this filter element's structure is valid wrt vCard
+        data object model.
+
+        @return: True if valid, False otherwise
+        """
+
+        # Test each property
+        for propfilter in self.children:
+            if not propfilter.valid():
+                return False
+        else:
+            return True
+
+
+
+class FilterChildBase(FilterBase):
+    """
+    CardDAV filter element.
+    """
+
+    def __init__(self, xml_element):
+
+        super(FilterChildBase, self).__init__(xml_element)
+
+        qualifier = None
+        filters = []
+
+        for child in xml_element.children:
+            qname = child.qname()
+
+            if qname in (
+                (carddav_namespace, "is-not-defined"),
+            ):
+                if qualifier is not None:
+                    raise ValueError("Only one of CardDAV:is-not-defined allowed")
+                qualifier = IsNotDefined(child)
+
+            elif qname == (carddav_namespace, "text-match"):
+                filters.append(TextMatch(child))
+
+            elif qname == (carddav_namespace, "param-filter"):
+                filters.append(ParameterFilter(child))
+            else:
+                raise ValueError("Unknown child element: %s" % (qname,))
+
+        if qualifier and isinstance(qualifier, IsNotDefined) and (len(filters) != 0):
+            raise ValueError("No other tests allowed when CardDAV:is-not-defined is present")
+
+        if xml_element.qname() == (carddav_namespace, "prop-filter"):
+            propfilter_test = xml_element.attributes.get("test", "anyof")
+            if propfilter_test not in ("anyof", "allof"):
+                raise ValueError("Test must be only one of anyof, allof")
+        else:
+            propfilter_test = "anyof"
+
+        self.propfilter_test = propfilter_test
+        self.qualifier = qualifier
+        self.filters = filters
+        self.filter_name = xml_element.attributes["name"]
+        if isinstance(self.filter_name, unicode):
+            self.filter_name = self.filter_name.encode("utf-8")
+        self.defined = not self.qualifier or not isinstance(qualifier, IsNotDefined)
+
+
+    def match(self, item):
+        """
+        Returns True if the given address book item (either a property or parameter value)
+        matches this filter, False otherwise.
+        """
+
+        # Always return True for the is-not-defined case as the result of this will
+        # be negated by the caller
+        if not self.defined:
+            return True
+
+        if self.qualifier and not self.qualifier.match(item):
+            return False
+
+        if len(self.filters) > 0:
+            allof = self.propfilter_test == "allof"
+            for filter in self.filters:
+                if allof != filter._match(item):
+                    return not allof
+            return allof
+        else:
+            return True
+
+
+
+class PropertyFilter (FilterChildBase):
+    """
+    Limits a search to specific properties.
+    """
+
+    def _match(self, vcard):
+        # At least one property must match (or is-not-defined is set)
+        for property in vcard.properties():
+            if property.name().upper() == self.filter_name.upper() and self.match(property):
+                break
+        else:
+            return not self.defined
+        return self.defined
+
+
+    def valid(self):
+        """
+        Indicate whether this filter element's structure is valid wrt vCard
+        data object model.
+
+        @return:      True if valid, False otherwise
+        """
+
+        # No tests
+        return True
+
+
+
+class ParameterFilter (FilterChildBase):
+    """
+    Limits a search to specific parameters.
+    """
+
+    def _match(self, property):
+
+        # At least one parameter must match (or is-not-defined is set)
+        result = not self.defined
+        for parameterName in property.parameterNames():
+            if parameterName.upper() == self.filter_name.upper() and self.match([property.parameterValues(parameterName)]):
+                result = self.defined
+                break
+
+        return result
+
+
+
+class IsNotDefined (FilterBase):
+    """
+    Specifies that the named iCalendar item does not exist.
+    """
+
+    def match(self, component, access=None):
+        # Oddly, this needs always to return True so that it appears there is
+        # a match - but we then "negate" the result if is-not-defined is set.
+        # Actually this method should never be called as we special case the
+        # is-not-defined option.
+        return True
+
+
+
+class TextMatch (FilterBase):
+    """
+    Specifies a substring match on a property or parameter value.
+    """
+    def __init__(self, xml_element):
+
+        super(TextMatch, self).__init__(xml_element)
+
+        self.text = str(xml_element)
+
+        if "collation" in xml_element.attributes:
+            self.collation = xml_element.attributes["collation"]
+        else:
+            self.collation = "i;unicode-casemap"
+
+        if "negate-condition" in xml_element.attributes:
+            self.negate = xml_element.attributes["negate-condition"]
+            if self.negate not in ("yes", "no"):
+                self.negate = "no"
+            self.negate = {"yes": True, "no": False}[self.negate]
+        else:
+            self.negate = False
+
+        if "match-type" in xml_element.attributes:
+            self.match_type = xml_element.attributes["match-type"]
+            if self.match_type not in (
+                "equals",
+                "contains",
+                "starts-with",
+                "ends-with",
+            ):
+                self.match_type = "contains"
+        else:
+            self.match_type = "contains"
+
+
+    def _match(self, item):
+        """
+        Match the text for the item.
+        If the item is a property, then match the property value,
+        otherwise it may be a list of parameter values - try to match anyone of those
+        """
+        if item is None:
+            return False
+
+        if isinstance(item, Property):
+            values = [item.strvalue()]
+        else:
+            values = item
+
+        test = unicode(self.text, "utf-8").lower()
+
+
+        def _textCompare(s):
+            # Currently ignores the collation and does caseless matching
+            s = s.lower()
+
+            if self.match_type == "equals":
+                return s == test
+            elif self.match_type == "contains":
+                return s.find(test) != -1
+            elif self.match_type == "starts-with":
+                return s.startswith(test)
+            elif self.match_type == "ends-with":
+                return s.endswith(test)
+            else:
+                return False
+
+        for value in values:
+            # NB Its possible that we have a text list value which appears as a Python list,
+            # so we need to check for that and iterate over the list.
+            if isinstance(value, list):
+                for subvalue in value:
+                    if _textCompare(unicode(subvalue, "utf-8")):
+                        return not self.negate
+            else:
+                if _textCompare(unicode(value, "utf-8")):
+                    return not self.negate
+
+        return self.negate

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/__init__.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/__init__.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/test_filter.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/test_filter.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/query/test/test_filter.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,74 @@
+##
+# Copyright (c) 2011-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.enterprise.dal.syntax import SQLFragment
+
+from twisted.trial.unittest import TestCase
+
+from twistedcaldav import carddavxml
+
+from txdav.carddav.datastore.query.filter import Filter
+from txdav.common.datastore.sql_tables import schema
+from txdav.carddav.datastore.query.builder import buildExpression
+from txdav.common.datastore.query.generator import SQLQueryGenerator
+from txdav.carddav.datastore.index_file import sqladdressbookquery
+
+class TestQueryFilter(TestCase):
+
+    _objectSchema = schema.ADDRESSBOOK_OBJECT
+    _queryFields = {
+        "UID": _objectSchema.UID
+    }
+
+    def test_query(self):
+        """
+        Basic query test - single term.
+        Only UID can be queried via sql.
+        """
+
+        filter = carddavxml.Filter(
+            *[carddavxml.PropertyFilter(
+                carddavxml.TextMatch.fromString("Example"),
+                **{"name":"UID"}
+            )]
+        )
+        filter = Filter(filter)
+
+        expression = buildExpression(filter, self._queryFields)
+        sql = SQLQueryGenerator(expression, self, 1234)
+        select, args = sql.generate()
+
+        self.assertEqual(select.toSQL(), SQLFragment("select distinct RESOURCE_NAME, VCARD_UID from ADDRESSBOOK_OBJECT where ADDRESSBOOK_HOME_RESOURCE_ID = ? and VCARD_UID like (? || (? || ?))", [1234, "%", "Example", "%"]))
+        self.assertEqual(args, {})
+
+
+    def test_sqllite_query(self):
+        """
+        Basic query test - single term.
+        Only UID can be queried via sql.
+        """
+
+        filter = carddavxml.Filter(
+            *[carddavxml.PropertyFilter(
+                carddavxml.TextMatch.fromString("Example"),
+                **{"name":"UID"}
+            )]
+        )
+        filter = Filter(filter)
+        sql, args = sqladdressbookquery(filter, 1234)
+
+        self.assertEqual(sql, " from RESOURCE where RESOURCE.UID GLOB :1")
+        self.assertEqual(args, ["*Example*"])

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/sql.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/sql.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/sql.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -15,6 +15,7 @@
 # limitations under the License.
 # #
 
+
 """
 SQL backend for CardDAV storage.
 """
@@ -46,12 +47,14 @@
 
 from txdav.base.propertystore.base import PropertyName
 from txdav.base.propertystore.sql import PropertyStore
+from txdav.carddav.datastore.query.builder import buildExpression
+from txdav.carddav.datastore.query.filter import Filter
 from txdav.carddav.iaddressbookstore import IAddressBookHome, IAddressBook, \
     IAddressBookObject, GroupWithUnsharedAddressNotAllowedError, \
     KindChangeNotAllowedError
+from txdav.common.datastore.query.generator import SQLQueryGenerator
 from txdav.common.datastore.sql import CommonHome, CommonHomeChild, \
     CommonObjectResource, EADDRESSBOOKTYPE, SharingMixIn, SharingInvitation
-from txdav.common.datastore.sql_legacy import PostgresLegacyABIndexEmulator
 from txdav.common.datastore.sql_tables import _ABO_KIND_PERSON, \
     _ABO_KIND_GROUP, _ABO_KIND_RESOURCE, _ABO_KIND_LOCATION, schema, \
     _BIND_MODE_OWN, _BIND_MODE_WRITE, _BIND_STATUS_ACCEPTED, \
@@ -60,7 +63,8 @@
 from txdav.common.icommondatastore import InternalDataStoreError, \
     InvalidUIDError, UIDExistsError, ObjectResourceTooBigError, \
     InvalidObjectResourceError, InvalidComponentForStoreError, \
-    AllRetriesFailed, ObjectResourceNameAlreadyExistsError
+    AllRetriesFailed, ObjectResourceNameAlreadyExistsError, \
+    IndexedSearchException
 from txdav.xml import element
 
 from zope.interface.declarations import implements
@@ -439,7 +443,12 @@
     _revisionsSchema = schema.ADDRESSBOOK_OBJECT_REVISIONS
     _objectSchema = schema.ADDRESSBOOK_OBJECT
 
+    # Mapping of vCard property name to DB column name
+    _queryFields = {
+        "UID": _objectSchema.UID,
+    }
 
+
     @classmethod
     @inlineCallbacks
     def _getDBDataIndirect(cls, home, name, resourceID, externalID):
@@ -515,7 +524,6 @@
     def __init__(self, home, name, resourceID, mode, status, revision=0, message=None, ownerHome=None, ownerName=None, externalID=None):
         ownerName = ownerHome.addressbook().name() if ownerHome else None
         super(AddressBook, self).__init__(home, name, resourceID, mode, status, revision=revision, message=message, ownerHome=ownerHome, ownerName=ownerName, externalID=externalID)
-        self._index = PostgresLegacyABIndexEmulator(self)
 
 
     def __repr__(self):
@@ -788,6 +796,50 @@
             returnValue((yield super(AddressBook, self).bumpModified()))
 
 
+    @inlineCallbacks
+    def search(self, filter):
+        """
+        Finds resources matching the given qualifiers.
+        @param filter: the L{Filter} for the addressbook-query to execute.
+        @return: an iterable of tuples for each resource matching the
+            given C{qualifiers}. The tuples are C{(name, uid)}, where
+            C{name} is the resource name, C{uid} is the resource UID.
+        """
+
+        # Make sure we have a proper Filter element and get the partial SQL statement to use.
+        sql_stmt = self._sqlquery(filter)
+
+        # No result means it is too complex for us
+        if sql_stmt is None:
+            raise IndexedSearchException()
+
+        sql_stmt, args = sql_stmt
+        rowiter = yield sql_stmt.on(self._txn, **args)
+
+        returnValue(list(rowiter))
+
+
+    def _sqlquery(self, filter):
+        """
+        Convert the supplied addressbook-query into a partial SQL statement.
+
+        @param filter: the L{Filter} for the addressbook-query to convert.
+        @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the partial SQL statement,
+                and the C{list} is the list of argument substitutions to use with the SQL API execute method.
+                Or return C{None} if it is not possible to create an SQL query to fully match the addressbook-query.
+        """
+
+        if not isinstance(filter, Filter):
+            return None
+
+        try:
+            expression = buildExpression(filter, self._queryFields)
+            sql = SQLQueryGenerator(expression, self, self.id())
+            return sql.generate()
+        except ValueError:
+            return None
+
+
     @classmethod
     @inlineCallbacks
     def listObjects(cls, home):
@@ -1989,6 +2041,10 @@
 
         if isinstance(component, str) or isinstance(component, unicode):
             component = self._componentClass.fromString(component)
+            try:
+                component = self._componentClass.fromString(component)
+            except InvalidVCardDataError as e:
+                raise InvalidComponentForStoreError(str(e))
 
         self._componentChanged = False
 

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/test/test_sql_sharing.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/test/test_sql_sharing.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/carddav/datastore/test/test_sql_sharing.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -1178,13 +1178,15 @@
 
         otherHome = yield self.addressbookHomeUnderTest(name="user02")
         for depth in ("1", "infinity",):
-            changed, deleted = yield otherHome.resourceNamesSinceRevision(otherAB._bindRevision - 1, depth)
+            changed, deleted, invalid = yield otherHome.resourceNamesSinceRevision(otherAB._bindRevision - 1, depth)
             self.assertNotEqual(len(changed), 0)
             self.assertEqual(len(deleted), 0)
+            self.assertEqual(len(invalid), 0)
 
-            changed, deleted = yield otherHome.resourceNamesSinceRevision(otherAB._bindRevision, depth)
+            changed, deleted, invalid = yield otherHome.resourceNamesSinceRevision(otherAB._bindRevision, depth)
             self.assertEqual(len(changed), 0)
             self.assertEqual(len(deleted), 0)
+            self.assertEqual(len(invalid), 0)
 
 
     @inlineCallbacks

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/__init__.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/__init__.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/expression.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/expression.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/expression.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,382 @@
+##
+# Copyright (c) 2006-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+Query Expression Elements. These are used to build a 'generic' query
+expression tree that can then be used by different query language
+generators to produce the actual query syntax required (SQL, xpath
+etc).
+"""
+
+__all__ = [
+    "allExpression",
+    "notExpression",
+    "andExpression",
+    "orExpression",
+    "timerangeExpression",
+    "textcompareExpression",
+    "containsExpression",
+    "notcontainsExpression",
+    "isExpression",
+    "isnotExpression",
+    "startswithExpression",
+    "notstartswithExpression",
+    "endswithExpression",
+    "notendswithExpression",
+    "inExpression",
+    "notinExpression",
+]
+
+class baseExpression(object):
+    """
+    The base class for all types of expression.
+    """
+
+    def __init__(self):
+        pass
+
+
+    def multi(self):
+        """
+        Indicate whether this expression is composed of multiple sub-expressions.
+
+        @return: C{True} if this expressions contains multiple sub-expressions,
+            C{False} otherwise.
+        """
+
+        return False
+
+
+    def _collapsedExpression(self):
+        return self
+
+
+    def andWith(self, other):
+        if isinstance(other, andExpression):
+            return andExpression((self._collapsedExpression(),) + tuple(other.expressions))
+        else:
+            return andExpression((self._collapsedExpression(), other._collapsedExpression(),))
+
+
+    def orWith(self, other):
+        if isinstance(other, orExpression):
+            return orExpression((self._collapsedExpression(),) + tuple(other.expressions))
+        else:
+            return orExpression((self._collapsedExpression(), other._collapsedExpression(),))
+
+
+
+class allExpression(baseExpression):
+    """
+    Match everything.
+    """
+
+    def __init__(self):
+        pass
+
+
+
+class logicExpression(baseExpression):
+    """
+    An expression representing a logical operation (boolean).
+    """
+
+    def __init__(self, expressions):
+        self.expressions = expressions
+
+
+    def __str__(self):
+        """
+        Generate a suitable text descriptor of this expression.
+
+        @return: a C{str} of the text for this expression.
+        """
+
+        result = ""
+        for e in self.expressions:
+            if len(result) != 0:
+                result += " " + self.operator() + " "
+            result += str(e)
+        if len(result):
+            result = "(" + result + ")"
+        return result
+
+
+    def multi(self):
+        """
+        Indicate whether this expression is composed of multiple expressions.
+
+        @return: C{True} if this expressions contains multiple sub-expressions,
+            C{False} otherwise.
+        """
+
+        return True
+
+
+    def _collapsedExpression(self):
+        if self.multi() and len(self.expressions) == 1:
+            return self.expressions[0]._collapsedExpression()
+        else:
+            return self
+
+
+
+class notExpression(logicExpression):
+    """
+    Logical NOT operation.
+    """
+
+    def __init__(self, expression):
+        super(notExpression, self).__init__([expression])
+
+
+    def operator(self):
+        return "NOT"
+
+
+    def __str__(self):
+        result = self.operator() + " " + str(self.expressions[0])
+        return result
+
+
+    def multi(self):
+        """
+        Indicate whether this expression is composed of multiple expressions.
+
+        @return: C{True} if this expressions contains multiple sub-expressions,
+            C{False} otherwise.
+        """
+
+        return False
+
+
+
+class andExpression(logicExpression):
+    """
+    Logical AND operation.
+    """
+
+    def __init__(self, expressions):
+        super(andExpression, self).__init__(expressions)
+
+
+    def operator(self):
+        return "AND"
+
+
+    def andWith(self, other):
+        self.expressions = tuple(self.expressions) + (other._collapsedExpression(),)
+        return self
+
+
+
+class orExpression(logicExpression):
+    """
+    Logical OR operation.
+    """
+
+    def __init__(self, expressions):
+        super(orExpression, self).__init__(expressions)
+
+
+    def operator(self):
+        return "OR"
+
+
+    def orWith(self, other):
+        self.expressions = tuple(self.expressions) + (other._collapsedExpression(),)
+        return self
+
+
+
+class timerangeExpression(baseExpression):
+    """
+    CalDAV time-range comparison expression.
+    """
+
+    def __init__(self, start, end, startfloat, endfloat):
+        self.start = start
+        self.end = end
+        self.startfloat = startfloat
+        self.endfloat = endfloat
+
+
+    def __str__(self):
+        return "timerange(" + str(self.start) + ", " + str(self.end) + ")"
+
+
+
+class textcompareExpression(baseExpression):
+    """
+    Base class for text comparison expressions.
+    """
+
+    def __init__(self, field, text, caseless):
+        self.field = field
+        self.text = text
+        self.caseless = caseless
+
+
+    def __str__(self):
+        return self.operator() + "(" + self.field + ", " + self.text + ", " + str(self.caseless) + ")"
+
+
+
+class containsExpression(textcompareExpression):
+    """
+    Text CONTAINS (sub-string match) expression.
+    """
+
+    def __init__(self, field, text, caseless):
+        super(containsExpression, self).__init__(field, text, caseless)
+
+
+    def operator(self):
+        return "contains"
+
+
+
+class notcontainsExpression(textcompareExpression):
+    """
+    Text NOT CONTAINS (sub-string match) expression.
+    """
+
+    def __init__(self, field, text, caseless):
+        super(notcontainsExpression, self).__init__(field, text, caseless)
+
+
+    def operator(self):
+        return "does not contain"
+
+
+
+class isExpression(textcompareExpression):
+    """
+    Text IS (exact string match) expression.
+    """
+
+    def __init__(self, field, text, caseless):
+        super(isExpression, self).__init__(field, text, caseless)
+
+
+    def operator(self):
+        return "is"
+
+
+
+class isnotExpression(textcompareExpression):
+    """
+    Text IS NOT (exact string match) expression.
+    """
+
+    def __init__(self, field, text, caseless):
+        super(isnotExpression, self).__init__(field, text, caseless)
+
+
+    def operator(self):
+        return "is not"
+
+
+
+class startswithExpression(textcompareExpression):
+    """
+    Text STARTSWITH (sub-string match) expression.
+    """
+
+    def __init__(self, field, text, caseless):
+        super(startswithExpression, self).__init__(field, text, caseless)
+
+
+    def operator(self):
+        return "starts with"
+
+
+
+class notstartswithExpression(textcompareExpression):
+    """
+    Text NOT STARTSWITH (sub-string match) expression.
+    """
+
+    def __init__(self, field, text, caseless):
+        super(notstartswithExpression, self).__init__(field, text, caseless)
+
+
+    def operator(self):
+        return "does not start with"
+
+
+
+class endswithExpression(textcompareExpression):
+    """
+    Text STARTSWITH (sub-string match) expression.
+    """
+
+    def __init__(self, field, text, caseless):
+        super(endswithExpression, self).__init__(field, text, caseless)
+
+
+    def operator(self):
+        return "ends with"
+
+
+
+class notendswithExpression(textcompareExpression):
+    """
+    Text NOT STARTSWITH (sub-string match) expression.
+    """
+
+    def __init__(self, field, text, caseless):
+        super(notendswithExpression, self).__init__(field, text, caseless)
+
+
+    def operator(self):
+        return "does not end with"
+
+
+
+class inExpression(textcompareExpression):
+    """
+    Text IN (exact string match to one of the supplied items) expression.
+    """
+
+    def __init__(self, field, text_list, caseless):
+        super(inExpression, self).__init__(field, text_list, caseless)
+
+
+    def operator(self):
+        return "in"
+
+
+    def __str__(self):
+        return self.operator() + "(" + self.field + ", " + str(self.text) + ", " + str(self.caseless) + ")"
+
+
+
+class notinExpression(textcompareExpression):
+    """
+    Text NOT IN (exact string match to none of the supplied items) expression.
+    """
+
+    def __init__(self, field, text, caseless):
+        super(notinExpression, self).__init__(field, text, caseless)
+
+
+    def operator(self):
+        return "not in"
+
+
+    def __str__(self):
+        return self.operator() + "(" + self.field + ", " + str(self.text) + ", " + str(self.caseless) + ")"

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/filegenerator.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/filegenerator.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/filegenerator.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,322 @@
+##
+# Copyright (c) 2006-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from txdav.common.datastore.query import expression
+
+"""
+SQLLite statement generator from query expressions.
+"""
+
+__all__ = [
+    "sqllitegenerator",
+]
+
+import cStringIO as StringIO
+
+class sqllitegenerator(object):
+
+    FROM = " from "
+    WHERE = " where "
+    RESOURCEDB = "RESOURCE"
+    TIMESPANDB = "TIMESPAN"
+    TRANSPARENCYDB = "TRANSPARENCY"
+    PERUSERDB = "PERUSER"
+    NOTOP = "NOT "
+    ANDOP = " AND "
+    OROP = " OR "
+    CONTAINSOP = " GLOB "
+    NOTCONTAINSOP = " NOT GLOB "
+    ISOP = " == "
+    ISNOTOP = " != "
+    STARTSWITHOP = " GLOB "
+    NOTSTARTSWITHOP = " NOT GLOB "
+    ENDSWITHOP = " GLOB "
+    NOTENDSWITHOP = " NOT GLOB "
+    INOP = " IN "
+    NOTINOP = " NOT IN "
+
+    FIELDS = {
+        "TYPE": "RESOURCE.TYPE",
+        "UID": "RESOURCE.UID",
+    }
+
+    TIMESPANTEST = "((TIMESPAN.FLOAT == 'N' AND TIMESPAN.START < %s AND TIMESPAN.END > %s) OR (TIMESPAN.FLOAT == 'Y' AND TIMESPAN.START < %s AND TIMESPAN.END > %s))"
+    TIMESPANTEST_NOEND = "((TIMESPAN.FLOAT == 'N' AND TIMESPAN.END > %s) OR (TIMESPAN.FLOAT == 'Y' AND TIMESPAN.END > %s))"
+    TIMESPANTEST_NOSTART = "((TIMESPAN.FLOAT == 'N' AND TIMESPAN.START < %s) OR (TIMESPAN.FLOAT == 'Y' AND TIMESPAN.START < %s))"
+    TIMESPANTEST_TAIL_PIECE = " AND TIMESPAN.RESOURCEID == RESOURCE.RESOURCEID"
+    TIMESPANTEST_JOIN_ON_PIECE = "TIMESPAN.INSTANCEID == TRANSPARENCY.INSTANCEID AND TRANSPARENCY.PERUSERID == %s"
+
+    def __init__(self, expr, calendarid, userid, freebusy=False):
+        """
+
+        @param expr: the query expression object model
+        @type expr: L{Filter}
+        @param calendarid: resource ID - not used for file-based per-calendar indexes
+        @type calendarid: C{int}
+        @param userid: user for whom query is being done - query will be scoped to that user's privileges and their transparency
+        @type userid: C{str}
+        @param freebusy: whether or not a freebusy query is being done - if it is, additional time range and transparency information is returned
+        @type freebusy: C{bool}
+        """
+        self.expression = expr
+        self.calendarid = calendarid
+        self.userid = userid if userid else ""
+        self.freebusy = freebusy
+        self.usedtimespan = False
+
+
+    def generate(self):
+        """
+        Generate the actual SQL 'where ...' expression from the passed in expression tree.
+
+        @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the partial SQL statement,
+            and the C{list} is the list of argument substitutions to use with the SQL API execute method.
+        """
+
+        # Init state
+        self.sout = StringIO.StringIO()
+        self.arguments = []
+        self.substitutions = []
+        self.usedtimespan = False
+
+        # Generate ' where ...' partial statement
+        self.generateExpression(self.expression)
+
+        # Prefix with ' from ...' partial statement
+        select = self.FROM + self.RESOURCEDB
+        if self.usedtimespan:
+
+            # Free busy needs transparency join
+            if self.freebusy:
+                self.frontArgument(self.userid)
+                select += ", %s LEFT OUTER JOIN %s ON (%s)" % (
+                    self.TIMESPANDB,
+                    self.TRANSPARENCYDB,
+                    self.TIMESPANTEST_JOIN_ON_PIECE
+                )
+            else:
+                select += ", %s" % (
+                    self.TIMESPANDB,
+                )
+        select += self.WHERE
+        if self.usedtimespan:
+            select += "("
+        select += self.sout.getvalue()
+        if self.usedtimespan:
+            if self.calendarid:
+                self.setArgument(self.calendarid)
+            select += ")%s" % (self.TIMESPANTEST_TAIL_PIECE,)
+
+        select = select % tuple(self.substitutions)
+
+        return select, self.arguments
+
+
+    def generateExpression(self, expr):
+        """
+        Generate an expression and all it's subexpressions.
+
+        @param expr: the L{baseExpression} derived class to write out.
+        @return: C{True} if the TIMESPAN table is used, C{False} otherwise.
+        """
+
+        # Generate based on each type of expression we might encounter
+
+        # ALL
+        if isinstance(expr, expression.allExpression):
+            # Wipe out the ' where ...' clause so everything is matched
+            self.sout.truncate(0)
+            self.arguments = []
+            self.substitutions = []
+            self.usedtimespan = False
+
+        # NOT
+        elif isinstance(expr, expression.notExpression):
+            self.sout.write(self.NOTOP)
+            self.generateSubExpression(expr.expressions[0])
+
+        # AND
+        elif isinstance(expr, expression.andExpression):
+            first = True
+            for e in expr.expressions:
+                if first:
+                    first = False
+                else:
+                    self.sout.write(self.ANDOP)
+                self.generateSubExpression(e)
+
+        # OR
+        elif isinstance(expr, expression.orExpression):
+            first = True
+            for e in expr.expressions:
+                if first:
+                    first = False
+                else:
+                    self.sout.write(self.OROP)
+                self.generateSubExpression(e)
+
+        # time-range
+        elif isinstance(expr, expression.timerangeExpression):
+            if expr.start and expr.end:
+                self.setArgument(expr.end)
+                self.setArgument(expr.start)
+                self.setArgument(expr.endfloat)
+                self.setArgument(expr.startfloat)
+                test = self.TIMESPANTEST
+            elif expr.start and expr.end is None:
+                self.setArgument(expr.start)
+                self.setArgument(expr.startfloat)
+                test = self.TIMESPANTEST_NOEND
+            elif not expr.start and expr.end:
+                self.setArgument(expr.end)
+                self.setArgument(expr.endfloat)
+                test = self.TIMESPANTEST_NOSTART
+
+            self.sout.write(test)
+            self.usedtimespan = True
+
+        # CONTAINS
+        elif isinstance(expr, expression.containsExpression):
+            self.sout.write(expr.field)
+            self.sout.write(self.CONTAINSOP)
+            self.addArgument(self.containsArgument(expr.text))
+
+        # NOT CONTAINS
+        elif isinstance(expr, expression.notcontainsExpression):
+            self.sout.write(expr.field)
+            self.sout.write(self.NOTCONTAINSOP)
+            self.addArgument(self.containsArgument(expr.text))
+
+        # IS
+        elif isinstance(expr, expression.isExpression):
+            self.sout.write(expr.field)
+            self.sout.write(self.ISOP)
+            self.addArgument(expr.text)
+
+        # IS NOT
+        elif isinstance(expr, expression.isnotExpression):
+            self.sout.write(expr.field)
+            self.sout.write(self.ISNOTOP)
+            self.addArgument(expr.text)
+
+        # STARTSWITH
+        elif isinstance(expr, expression.startswithExpression):
+            self.sout.write(expr.field)
+            self.sout.write(self.STARTSWITHOP)
+            self.addArgument(self.startswithArgument(expr.text))
+
+        # NOT STARTSWITH
+        elif isinstance(expr, expression.notstartswithExpression):
+            self.sout.write(expr.field)
+            self.sout.write(self.NOTSTARTSWITHOP)
+            self.addArgument(self.startswithArgument(expr.text))
+
+        # ENDSWITH
+        elif isinstance(expr, expression.endswithExpression):
+            self.sout.write(expr.field)
+            self.sout.write(self.ENDSWITHOP)
+            self.addArgument(self.endswithArgument(expr.text))
+
+        # NOT ENDSWITH
+        elif isinstance(expr, expression.notendswithExpression):
+            self.sout.write(expr.field)
+            self.sout.write(self.NOTENDSWITHOP)
+            self.addArgument(self.endswithArgument(expr.text))
+
+        # IN
+        elif isinstance(expr, expression.inExpression):
+            self.sout.write(expr.field)
+            self.sout.write(self.INOP)
+            self.sout.write("(")
+            for count, item in enumerate(expr.text):
+                if count != 0:
+                    self.sout.write(", ")
+                self.addArgument(item)
+            self.sout.write(")")
+
+        # NOT IN
+        elif isinstance(expr, expression.notinExpression):
+            self.sout.write(expr.field)
+            self.sout.write(self.NOTINOP)
+            self.sout.write("(")
+            for count, item in enumerate(expr.text):
+                if count != 0:
+                    self.sout.write(", ")
+                self.addArgument(item)
+            self.sout.write(")")
+
+
+    def generateSubExpression(self, expression):
+        """
+        Generate an SQL expression possibly in parenthesis if its a compound expression.
+
+        @param expression: the L{baseExpression} to write out.
+        @return: C{True} if the TIMESPAN table is used, C{False} otherwise.
+        """
+
+        if expression.multi():
+            self.sout.write("(")
+        self.generateExpression(expression)
+        if expression.multi():
+            self.sout.write(")")
+
+
+    def addArgument(self, arg):
+        """
+
+        @param arg: the C{str} of the argument to add
+        """
+
+        # Append argument to the list and add the appropriate substitution string to the output stream.
+        self.arguments.append(arg)
+        self.substitutions.append(":" + str(len(self.arguments)))
+        self.sout.write("%s")
+
+
+    def setArgument(self, arg):
+        """
+
+        @param arg: the C{str} of the argument to add
+        @return: C{str} for argument substitution text
+        """
+
+        # Append argument to the list and add the appropriate substitution string to the output stream.
+        self.arguments.append(arg)
+        self.substitutions.append(":" + str(len(self.arguments)))
+
+
+    def frontArgument(self, arg):
+        """
+
+        @param arg: the C{str} of the argument to add
+        @return: C{str} for argument substitution text
+        """
+
+        # Append argument to the list and add the appropriate substitution string to the output stream.
+        self.arguments.insert(0, arg)
+        self.substitutions.append(":" + str(len(self.arguments)))
+
+
+    def containsArgument(self, arg):
+        return "*%s*" % (arg,)
+
+
+    def startswithArgument(self, arg):
+        return "%s*" % (arg,)
+
+
+    def endswithArgument(self, arg):
+        return "*%s" % (arg,)

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/generator.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/generator.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/generator.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,164 @@
+##
+# Copyright (c) 2006-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.enterprise.dal.syntax import Select, Parameter, Not
+from txdav.common.datastore.query import expression
+
+"""
+SQL statement generator from query expressions.
+"""
+
+__all__ = [
+    "SQLQueryGenerator",
+]
+
+class SQLQueryGenerator(object):
+
+    def __init__(self, expr, collection, whereid):
+        """
+
+        @param expr: the query expression object model
+        @type expr: L{expression}
+        @param collection: the resource targeted by the query
+        @type collection: L{CommonHomeChild}
+        """
+        self.expression = expr
+        self.collection = collection
+        self.whereid = whereid
+
+
+    def generate(self):
+        """
+        Generate the actual SQL statement from the passed in expression tree.
+
+        @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the partial SQL statement,
+            and the C{list} is the list of argument substitutions to use with the SQL API execute method.
+        """
+
+        # Init state
+        self.arguments = {}
+        self.argcount = 0
+        obj = self.collection._objectSchema
+
+        columns = [obj.RESOURCE_NAME, obj.UID]
+
+        # For SQL data DB we need to restrict the query to just the targeted collection resource-id if provided
+        if self.whereid:
+            # AND the whole thing
+            test = expression.isExpression(obj.PARENT_RESOURCE_ID, self.whereid, True)
+            self.expression = test if isinstance(self.expression, expression.allExpression) else test.andWith(self.expression)
+
+        # Generate ' where ...' partial statement
+        where = self.generateExpression(self.expression)
+
+        select = Select(
+            columns,
+            From=obj,
+            Where=where,
+            Distinct=True,
+        )
+
+        return select, self.arguments
+
+
+    def generateExpression(self, expr):
+        """
+        Generate an expression and all it's subexpressions.
+
+        @param expr: the L{baseExpression} derived class to write out.
+        """
+
+        # Generate based on each type of expression we might encounter
+        partial = None
+
+        # ALL
+        if isinstance(expr, expression.allExpression):
+            # Everything is matched
+            partial = None
+            self.arguments = {}
+
+        # NOT
+        elif isinstance(expr, expression.notExpression):
+            partial = Not(self.generateExpression(expr.expressions[0]))
+
+        # AND
+        elif isinstance(expr, expression.andExpression):
+            for e in expr.expressions:
+                next = self.generateExpression(e)
+                partial = partial.And(next) if partial is not None else next
+
+        # OR
+        elif isinstance(expr, expression.orExpression):
+            for e in expr.expressions:
+                next = self.generateExpression(e)
+                partial = partial.Or(next) if partial is not None else next
+
+        # CONTAINS
+        elif isinstance(expr, expression.containsExpression):
+            partial = expr.field.Contains(expr.text)
+
+        # NOT CONTAINS
+        elif isinstance(expr, expression.notcontainsExpression):
+            partial = expr.field.NotContains(expr.text)
+
+        # IS
+        elif isinstance(expr, expression.isExpression):
+            partial = expr.field == expr.text
+
+        # IS NOT
+        elif isinstance(expr, expression.isnotExpression):
+            partial = expr.field != expr.text
+
+        # STARTSWITH
+        elif isinstance(expr, expression.startswithExpression):
+            partial = expr.field.StartsWith(expr.text)
+
+        # NOT STARTSWITH
+        elif isinstance(expr, expression.notstartswithExpression):
+            partial = expr.field.NotStartsWith(expr.text)
+
+        # ENDSWITH
+        elif isinstance(expr, expression.endswithExpression):
+            partial = expr.field.EndsWith(expr.text)
+
+        # NOT ENDSWITH
+        elif isinstance(expr, expression.notendswithExpression):
+            partial = expr.field.NotEndsWith(expr.text)
+
+        # IN
+        elif isinstance(expr, expression.inExpression):
+            argname = self.addArgument(expr.text)
+            partial = expr.field.In(Parameter(argname, len(expr.text)))
+
+        # NOT IN
+        elif isinstance(expr, expression.notinExpression):
+            argname = self.addArgument(expr.text)
+            partial = expr.field.NotIn(Parameter(argname, len(expr.text)))
+
+        return partial
+
+
+    def addArgument(self, arg):
+        """
+
+        @param arg: the C{str} of the argument to add
+        """
+
+        # Append argument to the list and add the appropriate substitution string to the output stream.
+        self.argcount += 1
+        argname = "arg{}".format(self.argcount)
+        self.arguments[argname] = arg
+        return argname

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/__init__.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/__init__.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_expression.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_expression.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_expression.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,167 @@
+##
+# Copyright (c) 2011-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from txdav.common.datastore.query import expression
+from twisted.trial.unittest import TestCase
+
+class Tests(TestCase):
+
+    def test_andWith(self):
+
+        tests = (
+            (
+                expression.isExpression("A", "1", True),
+                expression.isExpression("B", "2", True),
+                "(is(A, 1, True) AND is(B, 2, True))"
+            ),
+            (
+                expression.isExpression("A", "1", True),
+                expression.andExpression((
+                    expression.isExpression("B", "2", True),
+                )),
+                "(is(A, 1, True) AND is(B, 2, True))"
+            ),
+            (
+                expression.isExpression("A", "1", True),
+                expression.andExpression((
+                    expression.isExpression("B", "2", True),
+                    expression.isExpression("C", "3", True),
+                )),
+                "(is(A, 1, True) AND is(B, 2, True) AND is(C, 3, True))"
+            ),
+            (
+                expression.isExpression("A", "1", True),
+                expression.orExpression((
+                    expression.isExpression("B", "2", True),
+                )),
+                "(is(A, 1, True) AND is(B, 2, True))"
+            ),
+            (
+                expression.isExpression("A", "1", True),
+                expression.orExpression((
+                    expression.isExpression("B", "2", True),
+                    expression.isExpression("C", "3", True),
+                )),
+                "(is(A, 1, True) AND (is(B, 2, True) OR is(C, 3, True)))"
+            ),
+            (
+                expression.andExpression((
+                    expression.isExpression("A", "1", True),
+                )),
+                expression.isExpression("B", "2", True),
+                "(is(A, 1, True) AND is(B, 2, True))"
+            ),
+            (
+                expression.andExpression((
+                    expression.isExpression("A", "1", True),
+                    expression.isExpression("B", "2", True),
+                )),
+                expression.isExpression("C", "3", True),
+                "(is(A, 1, True) AND is(B, 2, True) AND is(C, 3, True))"
+            ),
+            (
+                expression.orExpression((
+                    expression.isExpression("A", "1", True),
+                )),
+                expression.isExpression("B", "2", True),
+                "(is(A, 1, True) AND is(B, 2, True))"
+            ),
+            (
+                expression.orExpression((
+                    expression.isExpression("A", "1", True),
+                    expression.isExpression("B", "2", True),
+                )),
+                expression.isExpression("C", "3", True),
+                "((is(A, 1, True) OR is(B, 2, True)) AND is(C, 3, True))"
+            ),
+        )
+
+        for expr1, expr2, result in tests:
+            self.assertEqual(str(expr1.andWith(expr2)), result, msg="Failed on %s" % (result,))
+
+
+    def test_orWith(self):
+
+        tests = (
+            (
+                expression.isExpression("A", "1", True),
+                expression.isExpression("B", "2", True),
+                "(is(A, 1, True) OR is(B, 2, True))"
+            ),
+            (
+                expression.isExpression("A", "1", True),
+                expression.andExpression((
+                    expression.isExpression("B", "2", True),
+                )),
+                "(is(A, 1, True) OR is(B, 2, True))"
+            ),
+            (
+                expression.isExpression("A", "1", True),
+                expression.andExpression((
+                    expression.isExpression("B", "2", True),
+                    expression.isExpression("C", "3", True),
+                )),
+                "(is(A, 1, True) OR (is(B, 2, True) AND is(C, 3, True)))"
+            ),
+            (
+                expression.isExpression("A", "1", True),
+                expression.orExpression((
+                    expression.isExpression("B", "2", True),
+                )),
+                "(is(A, 1, True) OR is(B, 2, True))"
+            ),
+            (
+                expression.isExpression("A", "1", True),
+                expression.orExpression((
+                    expression.isExpression("B", "2", True),
+                    expression.isExpression("C", "3", True),
+                )),
+                "(is(A, 1, True) OR is(B, 2, True) OR is(C, 3, True))"
+            ),
+            (
+                expression.andExpression((
+                    expression.isExpression("A", "1", True),
+                )),
+                expression.isExpression("B", "2", True),
+                "(is(A, 1, True) OR is(B, 2, True))"
+            ),
+            (
+                expression.andExpression((
+                    expression.isExpression("A", "1", True),
+                    expression.isExpression("B", "2", True),
+                )),
+                expression.isExpression("C", "3", True),
+                "((is(A, 1, True) AND is(B, 2, True)) OR is(C, 3, True))"
+            ),
+            (
+                expression.orExpression((
+                    expression.isExpression("A", "1", True),
+                )),
+                expression.isExpression("B", "2", True),
+                "(is(A, 1, True) OR is(B, 2, True))"
+            ),
+            (
+                expression.orExpression((
+                    expression.isExpression("A", "1", True),
+                    expression.isExpression("B", "2", True),
+                )),
+                expression.isExpression("C", "3", True),
+                "(is(A, 1, True) OR is(B, 2, True) OR is(C, 3, True))"
+            ),
+        )
+
+        for expr1, expr2, result in tests:
+            self.assertEqual(str(expr1.orWith(expr2)), result, msg="Failed on %s" % (result,))

Added: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_generator.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_generator.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/query/test/test_generator.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -0,0 +1,127 @@
+##
+# Copyright (c) 2012-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.enterprise.dal.syntax import SQLFragment, Parameter
+from txdav.common.datastore.query.generator import SQLQueryGenerator
+from txdav.common.datastore.query import expression
+
+"""
+Tests for L{txdav.common.datastore.sql}.
+"""
+
+from twisted.trial.unittest import TestCase
+
+from txdav.common.datastore.sql_tables import schema
+
+class SQLQueryGeneratorTests(TestCase):
+    """
+    Tests for shared functionality in L{txdav.common.datastore.sql}.
+    """
+
+    class FakeHomeChild(object):
+        _objectSchema = schema.CALENDAR_OBJECT
+
+        def id(self):
+            return 1234
+
+
+    def test_all_query(self):
+
+        expr = expression.allExpression()
+        resource = self.FakeHomeChild()
+        select, args = SQLQueryGenerator(expr, resource, resource.id()).generate()
+        self.assertEqual(select.toSQL(), SQLFragment("select distinct RESOURCE_NAME, ICALENDAR_UID from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ?", [1234]))
+        self.assertEqual(args, {})
+
+
+    def test_uid_query(self):
+
+        resource = self.FakeHomeChild()
+        obj = resource._objectSchema
+        expr = expression.isExpression(obj.UID, 5678, False)
+        select, args = SQLQueryGenerator(expr, resource, resource.id()).generate()
+        self.assertEqual(select.toSQL(), SQLFragment("select distinct RESOURCE_NAME, ICALENDAR_UID from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ? and ICALENDAR_UID = ?", [1234, 5678]))
+        self.assertEqual(args, {})
+
+
+    def test_or_query(self):
+
+        resource = self.FakeHomeChild()
+        obj = resource._objectSchema
+        expr = expression.orExpression((
+            expression.isExpression(obj.UID, 5678, False),
+            expression.isnotExpression(obj.RESOURCE_NAME, "foobar.ics", False),
+        ))
+        select, args = SQLQueryGenerator(expr, resource, resource.id()).generate()
+        self.assertEqual(
+            select.toSQL(),
+            SQLFragment(
+                "select distinct RESOURCE_NAME, ICALENDAR_UID from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ? and (ICALENDAR_UID = ? or RESOURCE_NAME != ?)",
+                [1234, 5678, "foobar.ics"]
+            )
+        )
+        self.assertEqual(args, {})
+
+
+    def test_and_query(self):
+
+        resource = self.FakeHomeChild()
+        obj = resource._objectSchema
+        expr = expression.andExpression((
+            expression.isExpression(obj.UID, 5678, False),
+            expression.isnotExpression(obj.RESOURCE_NAME, "foobar.ics", False),
+        ))
+        select, args = SQLQueryGenerator(expr, resource, resource.id()).generate()
+        self.assertEqual(
+            select.toSQL(),
+            SQLFragment(
+                "select distinct RESOURCE_NAME, ICALENDAR_UID from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ? and ICALENDAR_UID = ? and RESOURCE_NAME != ?",
+                [1234, 5678, "foobar.ics"]
+            )
+        )
+        self.assertEqual(args, {})
+
+
+    def test_not_query(self):
+
+        resource = self.FakeHomeChild()
+        obj = resource._objectSchema
+        expr = expression.notExpression(expression.isExpression(obj.UID, 5678, False))
+        select, args = SQLQueryGenerator(expr, resource, resource.id()).generate()
+        self.assertEqual(
+            select.toSQL(),
+            SQLFragment(
+                "select distinct RESOURCE_NAME, ICALENDAR_UID from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ? and not ICALENDAR_UID = ?",
+                [1234, 5678]
+            )
+        )
+        self.assertEqual(args, {})
+
+
+    def test_in_query(self):
+
+        resource = self.FakeHomeChild()
+        obj = resource._objectSchema
+        expr = expression.inExpression(obj.RESOURCE_NAME, ["1.ics", "2.ics", "3.ics"], False)
+        select, args = SQLQueryGenerator(expr, resource, resource.id()).generate()
+        self.assertEqual(
+            select.toSQL(),
+            SQLFragment(
+                "select distinct RESOURCE_NAME, ICALENDAR_UID from CALENDAR_OBJECT where CALENDAR_RESOURCE_ID = ? and RESOURCE_NAME in (?, ?, ?)",
+                [1234, Parameter('arg1', 3)]
+            )
+        )
+        self.assertEqual(args, {"arg1": ["1.ics", "2.ics", "3.ics"]})

Modified: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -67,7 +67,8 @@
     _BIND_MODE_INDIRECT, _HOME_STATUS_NORMAL, _HOME_STATUS_EXTERNAL
 from txdav.common.datastore.sql_tables import schema, splitSQLString
 from txdav.common.icommondatastore import ConcurrentModification, \
-    RecordNotAllowedError, ExternalShareFailed, ShareNotAllowed
+    RecordNotAllowedError, ExternalShareFailed, ShareNotAllowed, \
+    IndexedSearchException
 from txdav.common.icommondatastore import HomeChildNameNotAllowedError, \
     HomeChildNameAlreadyExistsError, NoSuchHomeChildError, \
     ObjectResourceNameNotAllowedError, ObjectResourceNameAlreadyExistsError, \
@@ -3224,8 +3225,8 @@
         ownerView = yield self.ownerView()
         if self.direct():
             yield ownerView.removeShare(self)
-            if not ownerView.external():
-                yield self._removeExternalInvite(ownerView)
+            if ownerView.external():
+                yield self._removeExternalInvite()
         else:
             yield self.declineShare()
 
@@ -4129,9 +4130,7 @@
         else:
             self._notifiers = None
 
-        self._index = None  # Derived classes need to set this
 
-
     def memoMe(self, key, memo): #@UnusedVariable
         """
         Add this object to the memo dictionary in whatever fashion is appropriate.
@@ -4356,10 +4355,6 @@
         return self._txn.store().directoryService()
 
 
-    def retrieveOldIndex(self):
-        return self._index
-
-
     def __repr__(self):
         return "<%s: %s>" % (self.__class__.__name__, self._resourceID)
 
@@ -4803,6 +4798,21 @@
         return super(CommonHomeChild, self).resourceNamesSinceRevision(revision)
 
 
+    def search(self, filter):
+        """
+        Do a query of the contents of this collection.
+
+        @param filter: the query filter to use
+        @type filter: L{Filter}
+
+        @return: the names of the matching resources
+        @rtype: C{list}
+        """
+
+        # This implementation raises - sub-classes override to do the actual query
+        raise IndexedSearchException()
+
+
     @inlineCallbacks
     def _loadPropertyStore(self, props=None):
         if props is None:

Deleted: CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql_legacy.py
===================================================================
--- CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql_legacy.py	2013-12-19 06:18:02 UTC (rev 12143)
+++ CalendarServer/branches/users/cdaboo/cross-pod-sharing/txdav/common/datastore/sql_legacy.py	2013-12-19 18:28:54 UTC (rev 12144)
@@ -1,886 +0,0 @@
-# -*- test-case-name: twistedcaldav.test.test_sharing,twistedcaldav.test.test_calendarquery -*-
-##
-# Copyright (c) 2010-2013 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
-
-
-"""
-PostgreSQL data store.
-"""
-
-import StringIO
-
-
-from twisted.python import hashlib
-from twisted.internet.defer import succeed, inlineCallbacks, returnValue
-
-from twistedcaldav.config import config
-from twistedcaldav.dateops import normalizeForIndex, pyCalendarTodatetime
-from twistedcaldav.memcachepool import CachePoolUserMixIn
-from twistedcaldav.query import \
-    calendarqueryfilter, calendarquery, addressbookquery, expression, \
-    addressbookqueryfilter
-from twistedcaldav.query.sqlgenerator import sqlgenerator
-
-from txdav.caldav.icalendarstore import TimeRangeLowerLimit, TimeRangeUpperLimit
-from txdav.common.icommondatastore import IndexedSearchException, \
-    ReservationError, NoSuchObjectResourceError
-
-from txdav.common.datastore.sql_tables import schema
-from twext.enterprise.dal.syntax import Parameter, Select
-from twext.python.clsprop import classproperty
-from twext.python.log import Logger
-
-from pycalendar.datetime import DateTime
-from pycalendar.duration import Duration
-
-log = Logger()
-
-indexfbtype_to_icalfbtype = {
-    0: '?',
-    1: 'F',
-    2: 'B',
-    3: 'U',
-    4: 'T',
-}
-
-class MemcachedUIDReserver(CachePoolUserMixIn):
-    log = Logger()
-
-    def __init__(self, index, cachePool=None):
-        self.index = index
-        self._cachePool = cachePool
-
-
-    def _key(self, uid):
-        return 'reservation:%s' % (
-            hashlib.md5('%s:%s' % (uid,
-                                   self.index.resource._resourceID)).hexdigest())
-
-
-    def reserveUID(self, uid):
-        self.log.debug("Reserving UID %r @ %r" % (
-                uid,
-                self.index.resource))
-
-        def _handleFalse(result):
-            if result is False:
-                raise ReservationError(
-                    "UID %s already reserved for calendar collection %s."
-                    % (uid, self.index.resource._name)
-                    )
-
-        d = self.getCachePool().add(self._key(uid),
-                                    'reserved',
-                                    expireTime=config.UIDReservationTimeOut)
-        d.addCallback(_handleFalse)
-        return d
-
-
-    def unreserveUID(self, uid):
-        self.log.debug("Unreserving UID %r @ %r" % (
-                uid,
-                self.index.resource))
-
-        def _handleFalse(result):
-            if result is False:
-                raise ReservationError(
-                    "UID %s is not reserved for calendar collection %s."
-                    % (uid, self.index.resource._resourceID)
-                    )
-
-        d = self.getCachePool().delete(self._key(uid))
-        d.addCallback(_handleFalse)
-        return d
-
-
-    def isReservedUID(self, uid):
-        self.log.debug("Is reserved UID %r @ %r" % (
-                uid,
-                self.index.resource))
-
-        def _checkValue((flags, value)):
-            if value is None:
-                return False
-            else:
-                return True
-
-        d = self.getCachePool().get(self._key(uid))
-        d.addCallback(_checkValue)
-        return d
-
-
-
-class DummyUIDReserver(object):
-    log = Logger()
-
-    def __init__(self, index):
-        self.index = index
-        self.reservations = set()
-
-
-    def _key(self, uid):
-        return 'reservation:%s' % (
-            hashlib.md5('%s:%s' % (uid,
-                                   self.index.resource._resourceID)).hexdigest())
-
-
-    def reserveUID(self, uid):
-        self.log.debug("Reserving UID %r @ %r" % (
-                uid,
-                self.index.resource))
-
-        key = self._key(uid)
-        if key in self.reservations:
-            raise ReservationError(
-                "UID %s already reserved for calendar collection %s."
-                % (uid, self.index.resource._name)
-                )
-        self.reservations.add(key)
-        return succeed(None)
-
-
-    def unreserveUID(self, uid):
-        self.log.debug("Unreserving UID %r @ %r" % (
-                uid,
-                self.index.resource))
-
-        key = self._key(uid)
-        if key in self.reservations:
-            self.reservations.remove(key)
-        return succeed(None)
-
-
-    def isReservedUID(self, uid):
-        self.log.debug("Is reserved UID %r @ %r" % (
-                uid,
-                self.index.resource))
-        key = self._key(uid)
-        return succeed(key in self.reservations)
-
-
-
-class RealSQLBehaviorMixin(object):
-    """
-    Class attributes for 'real' SQL behavior; avoid idiosyncracies of SQLite,
-    use standard SQL constructions, and depend on the full schema in
-    sql_schema/current.sql rather than the partial one in twistedcaldav which depends
-    on the placement of the database in the filesystem for some information.
-    """
-
-    ISOP = " = "
-    STARTSWITHOP = ENDSWITHOP = CONTAINSOP = " LIKE "
-    NOTSTARTSWITHOP = NOTENDSWITHOP = NOTCONTAINSOP = " NOT LIKE "
-
-    def containsArgument(self, arg):
-        return "%%%s%%" % (arg,)
-
-
-    def startswithArgument(self, arg):
-        return "%s%%" % (arg,)
-
-
-    def endswithArgument(self, arg):
-        return "%%%s" % (arg,)
-
-
-
-class CalDAVSQLBehaviorMixin(RealSQLBehaviorMixin):
-    """
-    Query generator for CalDAV indexed searches.
-    """
-
-    FIELDS = {
-        "TYPE": "CALENDAR_OBJECT.ICALENDAR_TYPE",
-        "UID": "CALENDAR_OBJECT.ICALENDAR_UID",
-    }
-    RESOURCEDB = "CALENDAR_OBJECT"
-    TIMESPANDB = "TIME_RANGE"
-
-    TIMESPANTEST = "((TIME_RANGE.FLOATING = FALSE AND TIME_RANGE.START_DATE < %s AND TIME_RANGE.END_DATE > %s) OR (TIME_RANGE.FLOATING = TRUE AND TIME_RANGE.START_DATE < %s AND TIME_RANGE.END_DATE > %s))"
-    TIMESPANTEST_NOEND = "((TIME_RANGE.FLOATING = FALSE AND TIME_RANGE.END_DATE > %s) OR (TIME_RANGE.FLOATING = TRUE AND TIME_RANGE.END_DATE > %s))"
-    TIMESPANTEST_NOSTART = "((TIME_RANGE.FLOATING = FALSE AND TIME_RANGE.START_DATE < %s) OR (TIME_RANGE.FLOATING = TRUE AND TIME_RANGE.START_DATE < %s))"
-    TIMESPANTEST_TAIL_PIECE = " AND TIME_RANGE.CALENDAR_OBJECT_RESOURCE_ID = CALENDAR_OBJECT.RESOURCE_ID AND TIME_RANGE.CALENDAR_RESOURCE_ID = %s"
-    TIMESPANTEST_JOIN_ON_PIECE = "TIME_RANGE.INSTANCE_ID = TRANSPARENCY.TIME_RANGE_INSTANCE_ID AND TRANSPARENCY.USER_ID = %s"
-
-    def generate(self):
-        """
-        Generate the actual SQL 'where ...' expression from the passed in
-        expression tree.
-
-        @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the
-            partial SQL statement, and the C{list} is the list of argument
-            substitutions to use with the SQL API execute method.
-        """
-
-        # Init state
-        self.sout = StringIO.StringIO()
-        self.arguments = []
-        self.substitutions = []
-        self.usedtimespan = False
-
-        # For SQL data DB we need to restrict the query to just the targeted calendar resource-id if provided
-        if self.calendarid:
-
-            test = expression.isExpression("CALENDAR_OBJECT.CALENDAR_RESOURCE_ID", str(self.calendarid), True)
-
-            # Since timerange expression already have the calendar resource-id test in them, do not
-            # add the additional term to those. When the additional term is added, add it as the first
-            # component in the AND expression to hopefully get the DB to use its index first
-
-            # Top-level timerange expression already has calendar resource-id restriction in it
-            if isinstance(self.expression, expression.timerangeExpression):
-                pass
-
-            # Top-level OR - check each component
-            elif isinstance(self.expression, expression.orExpression):
-
-                def _hasTopLevelTimerange(testexpr):
-                    if isinstance(testexpr, expression.timerangeExpression):
-                        return True
-                    elif isinstance(testexpr, expression.andExpression):
-                        return any([isinstance(expr, expression.timerangeExpression) for expr in testexpr.expressions])
-                    else:
-                        return False
-
-                hasTimerange = any([_hasTopLevelTimerange(expr) for expr in self.expression.expressions])
-
-                if hasTimerange:
-                    # timerange expression forces a join on calendarid
-                    pass
-                else:
-                    # AND the whole thing with calendarid
-                    self.expression = test.andWith(self.expression)
-
-            # Top-level AND - only add additional expression if timerange not present
-            elif isinstance(self.expression, expression.andExpression):
-                hasTimerange = any([isinstance(expr, expression.timerangeExpression) for expr in self.expression.expressions])
-                if not hasTimerange:
-                    # AND the whole thing
-                    self.expression = test.andWith(self.expression)
-
-            # Just AND the entire thing
-            else:
-                self.expression = test.andWith(self.expression)
-
-        # Generate ' where ...' partial statement
-        self.generateExpression(self.expression)
-
-        # Prefix with ' from ...' partial statement
-        select = self.FROM + self.RESOURCEDB
-        if self.usedtimespan:
-
-            # Free busy needs transparency join
-            if self.freebusy:
-                self.frontArgument(self.userid)
-                select += ", %s LEFT OUTER JOIN %s ON (%s)" % (
-                    self.TIMESPANDB,
-                    self.TRANSPARENCYDB,
-                    self.TIMESPANTEST_JOIN_ON_PIECE
-                )
-            else:
-                select += ", %s" % (
-                    self.TIMESPANDB,
-                )
-        select += self.WHERE
-        if self.usedtimespan:
-            select += "("
-        select += self.sout.getvalue()
-        if self.usedtimespan:
-            if self.calendarid:
-                self.setArgument(self.calendarid)
-            select += ")%s" % (self.TIMESPANTEST_TAIL_PIECE,)
-
-        select = select % tuple(self.substitutions)
-
-        return select, self.arguments
-
-
-
-class FormatParamStyleMixin(object):
-    """
-    Mixin for overriding methods on sqlgenerator that generate arguments
-    according to format/pyformat rules rather than the base class's 'numeric'
-    rules.
-    """
-
-    def addArgument(self, arg):
-        self.arguments.append(arg)
-        self.substitutions.append("%s")
-        self.sout.write("%s")
-
-
-    def setArgument(self, arg):
-        self.arguments.append(arg)
-        self.substitutions.append("%s")
-
-
-    def frontArgument(self, arg):
-        self.arguments.insert(0, arg)
-        self.substitutions.insert(0, "%s")
-
-
-
-class postgresqlgenerator(FormatParamStyleMixin, CalDAVSQLBehaviorMixin,
-                          sqlgenerator):
-    """
-    Query generator for PostgreSQL indexed searches.
-    """
-
-
-
-def fixbools(sqltext):
-    return sqltext.replace("TRUE", "1").replace("FALSE", "0")
-
-
-
-class oraclesqlgenerator(CalDAVSQLBehaviorMixin, sqlgenerator):
-    """
-    Query generator for Oracle indexed searches.
-    """
-    TIMESPANTEST = fixbools(CalDAVSQLBehaviorMixin.TIMESPANTEST)
-    TIMESPANTEST_NOEND = fixbools(CalDAVSQLBehaviorMixin.TIMESPANTEST_NOEND)
-    TIMESPANTEST_NOSTART = fixbools(CalDAVSQLBehaviorMixin.TIMESPANTEST_NOSTART)
-    TIMESPANTEST_TAIL_PIECE = fixbools(
-        CalDAVSQLBehaviorMixin.TIMESPANTEST_TAIL_PIECE)
-    TIMESPANTEST_JOIN_ON_PIECE = fixbools(
-        CalDAVSQLBehaviorMixin.TIMESPANTEST_JOIN_ON_PIECE)
-
-
-
-class LegacyIndexHelper(object):
-    log = Logger()
-
-    @inlineCallbacks
-    def isAllowedUID(self, uid, *names):
-        """
-        Checks to see whether to allow an operation which would add the
-        specified UID to the index.  Specifically, the operation may not
-        violate the constraint that UIDs must be unique.
-        @param uid: the UID to check
-        @param names: the names of resources being replaced or deleted by the
-            operation; UIDs associated with these resources are not checked.
-        @return: True if the UID is not in the index and is not reserved,
-            False otherwise.
-        """
-        rname = yield self.resourceNameForUID(uid)
-        returnValue(rname is None or rname in names)
-
-
-    def reserveUID(self, uid):
-        return self.reserver.reserveUID(uid)
-
-
-    def unreserveUID(self, uid):
-        return self.reserver.unreserveUID(uid)
-
-
-    def isReservedUID(self, uid):
-        return self.reserver.isReservedUID(uid)
-
-
-
-class PostgresLegacyIndexEmulator(LegacyIndexHelper):
-    """
-    Emulator for L{twistedcaldv.index.Index} and
-    L{twistedcaldv.index.IndexSchedule}.
-    """
-
-    def __init__(self, calendar):
-        self.resource = self.calendar = calendar
-        if (
-            hasattr(config, "Memcached") and
-            config.Memcached.Pools.Default.ClientEnabled
-        ):
-            self.reserver = MemcachedUIDReserver(self)
-        else:
-            # This is only used with unit tests
-            self.reserver = DummyUIDReserver(self)
-
-    _objectSchema = schema.CALENDAR_OBJECT
-
-    @property
-    def _txn(self):
-        return self.calendar._txn
-
-
-    @inlineCallbacks
-    def isAllowedUID(self, uid, *names):
-        """
-        Checks to see whether to allow an operation which would add the
-        specified UID to the index.  Specifically, the operation may not
-        violate the constraint that UIDs must be unique.
-        @param uid: the UID to check
-        @param names: the names of resources being replaced or deleted by the
-            operation; UIDs associated with these resources are not checked.
-        @return: True if the UID is not in the index and is not reserved,
-            False otherwise.
-        """
-        rname = yield self.resourceNameForUID(uid)
-        returnValue(rname is None or rname in names)
-
-
-    @inlineCallbacks
-    def resourceUIDForName(self, name):
-        uid = yield self.calendar.resourceUIDForName(name)
-        returnValue(uid)
-
-
-    @inlineCallbacks
-    def resourceNameForUID(self, uid):
-        name = yield self.calendar.resourceNameForUID(uid)
-        returnValue(name)
-
-
-    @classproperty
-    def _notExpandedWithinQuery(cls): #@NoSelf
-        """
-        DAL query to satisfy L{PostgresLegacyIndexEmulator.notExpandedBeyond}.
-        """
-        co = schema.CALENDAR_OBJECT
-        return Select(
-            [co.RESOURCE_NAME],
-            From=co,
-            Where=((co.RECURRANCE_MIN > Parameter("minDate"))
-                .Or(co.RECURRANCE_MAX < Parameter("maxDate")))
-                .And(co.CALENDAR_RESOURCE_ID == Parameter("resourceID"))
-        )
-
-
-    @inlineCallbacks
-    def notExpandedWithin(self, minDate, maxDate):
-        """
-        Gives all resources which have not been expanded beyond a given date
-        in the database.  (Unused; see above L{postgresqlgenerator}.
-        """
-        returnValue([row[0] for row in (
-            yield self._notExpandedWithinQuery.on(
-                self._txn,
-                minDate=pyCalendarTodatetime(normalizeForIndex(minDate)) if minDate is not None else None,
-                maxDate=pyCalendarTodatetime(normalizeForIndex(maxDate)),
-                resourceID=self.calendar._resourceID))]
-        )
-
-
-    @inlineCallbacks
-    def reExpandResource(self, name, expand_start, expand_end):
-        """
-        Given a resource name, remove it from the database and re-add it
-        with a longer expansion.
-        """
-        obj = yield self.calendar.calendarObjectWithName(name)
-
-        # Use a new transaction to do this update quickly without locking the row for too long. However, the original
-        # transaction may have the row locked, so use wait=False and if that fails, fall back to using the original txn.
-
-        newTxn = obj.transaction().store().newTransaction()
-        try:
-            yield obj.lock(wait=False, txn=newTxn)
-        except NoSuchObjectResourceError:
-            yield newTxn.commit()
-            returnValue(None)
-        except:
-            yield newTxn.abort()
-            newTxn = None
-
-        # Now do the re-expand using the appropriate transaction
-        try:
-            doExpand = False
-            if newTxn is None:
-                doExpand = True
-            else:
-                # We repeat this check because the resource may have been re-expanded by someone else
-                rmin, rmax = (yield obj.recurrenceMinMax(txn=newTxn))
-
-                # If the resource is not fully expanded, see if within the required range or not.
-                # Note that expand_start could be None if no lower limit is applied, but expand_end will
-                # never be None
-                if rmax is not None and rmax < expand_end:
-                    doExpand = True
-                if rmin is not None and expand_start is not None and rmin > expand_start:
-                    doExpand = True
-
-            if doExpand:
-                yield obj.updateDatabase(
-                    (yield obj.component()),
-                    expand_until=expand_end,
-                    reCreate=True,
-                    txn=newTxn,
-                )
-        finally:
-            if newTxn is not None:
-                yield newTxn.commit()
-
-
-    @inlineCallbacks
-    def testAndUpdateIndex(self, minDate, maxDate):
-        # Find out if the index is expanded far enough
-        names = yield self.notExpandedWithin(minDate, maxDate)
-
-        # Actually expand recurrence max
-        for name in names:
-            self.log.info("Search falls outside range of index for %s %s to %s" %
-                          (name, minDate, maxDate))
-            yield self.reExpandResource(name, minDate, maxDate)
-
-
-    @inlineCallbacks
-    def indexedSearch(self, filter, useruid='', fbtype=False):
-        """
-        Finds resources matching the given qualifiers.
-
-        @param filter: the L{Filter} for the calendar-query to execute.
-
-        @return: a L{Deferred} which fires with an iterable of tuples for each
-            resource matching the given C{qualifiers}. The tuples are C{(name,
-            uid, type)}, where C{name} is the resource name, C{uid} is the
-            resource UID, and C{type} is the resource iCalendar component type.
-        """
-        # Detect which style of parameter-generation we're using.  Naming is a
-        # little off here, because the reason we're using the numeric one is
-        # that it happens to be used by the oracle binding that we're using,
-        # whereas the postgres binding happens to use the 'pyformat' (e.g. %s)
-        # parameter style.
-        if self.calendar._txn.paramstyle == 'numeric':
-            generator = oraclesqlgenerator
-        else:
-            generator = postgresqlgenerator
-        # Make sure we have a proper Filter element and get the partial SQL
-        # statement to use.
-        if isinstance(filter, calendarqueryfilter.Filter):
-            qualifiers = calendarquery.sqlcalendarquery(
-                filter, self.calendar._resourceID, useruid, fbtype,
-                generator=generator
-            )
-            if qualifiers is not None:
-
-                today = DateTime.getToday()
-
-                # Determine how far we need to extend the current expansion of
-                # events. If we have an open-ended time-range we will expand
-                # one year past the start. That should catch bounded
-                # recurrences - unbounded will have been indexed with an
-                # "infinite" value always included.
-                maxDate, isStartDate = filter.getmaxtimerange()
-                if maxDate:
-                    maxDate = maxDate.duplicate()
-                    maxDate.offsetDay(1)
-                    maxDate.setDateOnly(True)
-                    upperLimit = today + Duration(days=config.FreeBusyIndexExpandMaxDays)
-                    if maxDate > upperLimit:
-                        raise TimeRangeUpperLimit(upperLimit)
-                    if isStartDate:
-                        maxDate += Duration(days=365)
-
-                # Determine if the start date is too early for the restricted range we
-                # are applying. If it is today or later we don't need to worry about truncation
-                # in the past.
-                minDate, _ignore_isEndDate = filter.getmintimerange()
-                if minDate >= today:
-                    minDate = None
-                if minDate is not None and config.FreeBusyIndexLowerLimitDays:
-                    truncateLowerLimit = today - Duration(days=config.FreeBusyIndexLowerLimitDays)
-                    if minDate < truncateLowerLimit:
-                        raise TimeRangeLowerLimit(truncateLowerLimit)
-
-                if maxDate is not None or minDate is not None:
-                    yield self.testAndUpdateIndex(minDate, maxDate)
-
-            else:
-                # We cannot handle this filter in an indexed search
-                raise IndexedSearchException()
-        else:
-            qualifiers = None
-
-        # Perform the search
-        if qualifiers is None:
-            rowiter = yield self.bruteForceSearch()
-        else:
-            if fbtype:
-                # For a free-busy time-range query we return all instances
-                rowiter = yield self._txn.execSQL(
-                    """
-                    select DISTINCT
-                        CALENDAR_OBJECT.RESOURCE_NAME,
-                        CALENDAR_OBJECT.ICALENDAR_UID,
-                        CALENDAR_OBJECT.ICALENDAR_TYPE,
-                        CALENDAR_OBJECT.ORGANIZER,
-                        TIME_RANGE.FLOATING, TIME_RANGE.START_DATE,
-                        TIME_RANGE.END_DATE, TIME_RANGE.FBTYPE,
-                        TIME_RANGE.TRANSPARENT, TRANSPARENCY.TRANSPARENT
-                    """ +
-                    qualifiers[0],
-                    qualifiers[1]
-                )
-            else:
-                rowiter = yield self._txn.execSQL(
-                    """
-                    select
-                        DISTINCT CALENDAR_OBJECT.RESOURCE_NAME,
-                        CALENDAR_OBJECT.ICALENDAR_UID,
-                        CALENDAR_OBJECT.ICALENDAR_TYPE
-                    """ +
-                    qualifiers[0],
-                    qualifiers[1]
-                )
-
-        # Check result for missing resources
-
-        results = []
-        for row in rowiter:
-            if fbtype:
-                row = list(row)
-                row[4] = 'Y' if row[4] else 'N'
-                row[7] = indexfbtype_to_icalfbtype[row[7]]
-                if row[9] is not None:
-                    row[8] = row[9]
-                row[8] = 'T' if row[8] else 'F'
-                del row[9]
-            results.append(row)
-        returnValue(results)
-
-
-    @classproperty
-    def _bruteForceQuery(cls): #@NoSelf
-        """
-        DAL query for all C{CALENDAR_OBJECT} rows in the calendar represented by
-        this index.
-        """
-        obj = cls._objectSchema
-        return Select(
-            [obj.RESOURCE_NAME, obj.ICALENDAR_UID, obj.ICALENDAR_TYPE],
-            From=obj, Where=obj.PARENT_RESOURCE_ID == Parameter("resourceID")
-        )
-
-
-    def bruteForceSearch(self):
-        return self._bruteForceQuery.on(
-            self._txn, resourceID=self.resource._resourceID)
-
-
-    @inlineCallbacks
-    def resourcesExist(self, names):
-        returnValue(list(set(names).intersection(
-            set((yield self.calendar.listCalendarObjects())))))
-
-
-    @classproperty
-    def _resourceExistsQuery(cls): #@NoSelf
-        """
-        DAL query to determine whether a calendar object exists in the
-        collection represented by this index.
-        """
-        obj = cls._objectSchema
-        return Select(
-            [obj.RESOURCE_NAME], From=obj,
-            Where=(obj.RESOURCE_NAME == Parameter("name"))
-            .And(obj.PARENT_RESOURCE_ID == Parameter("resourceID"))
-        )
-
-
-    @inlineCallbacks
-    def resourceExists(self, name):
-        returnValue((bool(
-            (yield self._resourceExistsQuery.on(
-                self._txn, name=name, resourceID=self.resource._resourceID))
-        )))
-
-
-
-class PostgresLegacyInboxIndexEmulator(PostgresLegacyIndexEmulator):
-    """
-    UIDs need not be unique in the 'inbox' calendar, so override those
-    behaviors intended to ensure that.
-    """
-
-    def isAllowedUID(self):
-        return succeed(True)
-
-
-    def reserveUID(self, uid):
-        return succeed(None)
-
-
-    def unreserveUID(self, uid):
-        return succeed(None)
-
-
-    def isReservedUID(self, uid):
-        return succeed(False)
-
-
-
-# CARDDAV
-
-class CardDAVSQLBehaviorMixin(RealSQLBehaviorMixin):
-    """
-    Query generator for CardDAV indexed searches.
-    """
-
-    FIELDS = {
-        "UID": "ADDRESSBOOK_OBJECT.VCARD_UID",
-    }
-    RESOURCEDB = "ADDRESSBOOK_OBJECT"
-
-    def generate(self):
-        """
-        Generate the actual SQL 'where ...' expression from the passed in
-        expression tree.
-
-        @return: a C{tuple} of (C{str}, C{list}), where the C{str} is the
-            partial SQL statement, and the C{list} is the list of argument
-            substitutions to use with the SQL API execute method.
-        """
-
-        # Init state
-        self.sout = StringIO.StringIO()
-        self.arguments = []
-        self.substitutions = []
-
-        # For SQL data DB we need to restrict the query to just the targeted calendar resource-id if provided
-        if self.calendarid:
-
-            # AND the whole thing
-            test = expression.isExpression("ADDRESSBOOK_OBJECT.ADDRESSBOOK_HOME_RESOURCE_ID", str(self.calendarid), True)
-            self.expression = test.andWith(self.expression)
-
-        # Generate ' where ...' partial statement
-        self.sout.write(self.WHERE)
-        self.generateExpression(self.expression)
-
-        # Prefix with ' from ...' partial statement
-        select = self.FROM + self.RESOURCEDB
-        select += self.sout.getvalue()
-
-        select = select % tuple(self.substitutions)
-
-        return select, self.arguments
-
-
-
-class postgresqladbkgenerator(FormatParamStyleMixin, CardDAVSQLBehaviorMixin, sqlgenerator):
-    """
-    Query generator for PostgreSQL indexed searches.
-    """
-
-
-
-class oraclesqladbkgenerator(CardDAVSQLBehaviorMixin, sqlgenerator):
-    """
-    Query generator for Oracle indexed searches.
-    """
-
-
-
-class PostgresLegacyABIndexEmulator(LegacyIndexHelper):
-    """
-    Emulator for L{twistedcaldv.index.Index} and
-    L{twistedcaldv.index.IndexSchedule}.
-    """
-
-    _objectSchema = schema.ADDRESSBOOK_OBJECT
-
-    def __init__(self, addressbook):
-        self.resource = self.addressbook = addressbook
-        if (
-            hasattr(config, "Memcached") and
-            config.Memcached.Pools.Default.ClientEnabled
-        ):
-            self.reserver = MemcachedUIDReserver(self)
-        else:
-            # This is only used with unit tests
-            self.reserver = DummyUIDReserver(self)
-
-
-    @property
-    def _txn(self):
-        return self.addressbook._txn
-
-
-    @inlineCallbacks
-    def resourceUIDForName(self, name):
-        obj = yield self.addressbook.addressbookObjectWithName(name)
-        if obj is None:
-            returnValue(None)
-        returnValue(obj.uid())
-
-
-    @inlineCallbacks
-    def resourceNameForUID(self, uid):
-        obj = yield self.addressbook.addressbookObjectWithUID(uid)
-        if obj is None:
-            returnValue(None)
-        returnValue(obj.name())
-
-
-    def searchValid(self, filter):
-        if isinstance(filter, addressbookqueryfilter.Filter):
-            qualifiers = addressbookquery.sqladdressbookquery(filter)
-        else:
-            qualifiers = None
-
-        return qualifiers is not None
-
-
-    @inlineCallbacks
-    def search(self, filter):
-        """
-        Finds resources matching the given qualifiers.
-        @param filter: the L{Filter} for the addressbook-query to execute.
-        @return: an iterable of tuples for each resource matching the
-            given C{qualifiers}. The tuples are C{(name, uid, type)}, where
-            C{name} is the resource name, C{uid} is the resource UID, and
-            C{type} is the resource iCalendar component type.x
-        """
-        if self.addressbook._txn.paramstyle == 'numeric':
-            generator = oraclesqladbkgenerator
-        else:
-            generator = postgresqladbkgenerator
-        # Make sure we have a proper Filter element and get the partial SQL statement to use.
-        if isinstance(filter, addressbookqueryfilter.Filter):
-            qualifiers = addressbookquery.sqladdressbookquery(
-                filter, self.addressbook._resourceID, generator=generator)
-        else:
-            qualifiers = None
-        if qualifiers is not None:
-            rowiter = yield self._txn.execSQL(
-                "select DISTINCT ADDRESSBOOK_OBJECT.RESOURCE_NAME, ADDRESSBOOK_OBJECT.VCARD_UID" +
-                qualifiers[0],
-                qualifiers[1]
-            )
-        else:
-            rowiter = yield Select(
-                [self._objectSchema.RESOURCE_NAME,
-                 self._objectSchema.VCARD_UID],
-                From=self._objectSchema,
-                Where=self._objectSchema.ADDRESSBOOK_HOME_RESOURCE_ID ==
-                self.addressbook._resourceID
-            ).on(self.addressbook._txn)
-
-        returnValue(list(rowiter))
-
-
-    def indexedSearch(self, filter, useruid='', fbtype=False):
-        """
-        Always raise L{IndexedSearchException}, since these indexes are not
-        fully implemented yet.
-        """
-        raise IndexedSearchException()
-
-
-    @inlineCallbacks
-    def resourcesExist(self, names):
-        returnValue(list(set(names).intersection(
-            set((yield self.addressbook.listAddressBookObjects())))))
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.macosforge.org/pipermail/calendarserver-changes/attachments/20140312/6df01b86/attachment.html>


More information about the calendarserver-changes mailing list