[CalendarServer-changes] [6396] CalendarServer/branches/users/glyph/more-deferreds-7

source_changes at macosforge.org source_changes at macosforge.org
Mon Oct 4 09:38:44 PDT 2010


Revision: 6396
          http://trac.macosforge.org/projects/calendarserver/changeset/6396
Author:   glyph at apple.com
Date:     2010-10-04 09:38:42 -0700 (Mon, 04 Oct 2010)
Log Message:
-----------
Pull changes from trunk (although not everything from those changes async yet)

Modified Paths:
--------------
    CalendarServer/branches/users/glyph/more-deferreds-7/calendarserver/tap/caldav.py
    CalendarServer/branches/users/glyph/more-deferreds-7/run
    CalendarServer/branches/users/glyph/more-deferreds-7/twext/web2/dav/resource.py
    CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/calendaruserproxy.py
    CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/report_sync_collection.py
    CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/notify.py
    CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/resource.py
    CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/stdconfig.py
    CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/storebridge.py
    CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_notify.py
    CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/upgrade.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/base.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/test/base.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/index_file.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/sql.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_index_file.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/index_file.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/sql.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_index_file.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_legacy.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_schema_v1.sql
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_tables.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/test/util.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/util.py

Added Paths:
-----------
    CalendarServer/branches/users/glyph/more-deferreds-7/contrib/performance/sample-many.sh
    CalendarServer/branches/users/glyph/more-deferreds-7/contrib/tools/anonymous_log.py
    CalendarServer/branches/users/glyph/more-deferreds-7/contrib/tools/sortrecurrences.py

Property Changed:
----------------
    CalendarServer/branches/users/glyph/more-deferreds-7/
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/index_file.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_index_file.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/index_file.py
    CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_index_file.py


Property changes on: CalendarServer/branches/users/glyph/more-deferreds-7
___________________________________________________________________
Modified: svn:mergeinfo
   - /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/generic-sqlstore:6167-6191
/CalendarServer/branches/new-store:5594-5934
/CalendarServer/branches/new-store-no-caldavfile:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
/CalendarServer/branches/users/glyph/sendfdport:5388-5424
/CalendarServer/branches/users/glyph/sql-store:5929-6073
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/branches/users/wsanchez/transations:5515-5593
   + /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/generic-sqlstore:6167-6191
/CalendarServer/branches/new-store:5594-5934
/CalendarServer/branches/new-store-no-caldavfile:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
/CalendarServer/branches/users/glyph/sendfdport:5388-5424
/CalendarServer/branches/users/glyph/sql-store:5929-6073
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/branches/users/wsanchez/transations:5515-5593
/CalendarServer/trunk:6369-6394

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/calendarserver/tap/caldav.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/calendarserver/tap/caldav.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/calendarserver/tap/caldav.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -721,7 +721,8 @@
                 attachmentsRoot = dbRoot.child("attachments")
                 return UpgradeToDatabaseService.wrapService(
                     CachingFilePath(config.DocumentRoot), mainService,
-                    connectionFactory, attachmentsRoot
+                    connectionFactory, attachmentsRoot,
+                    uid=postgresUID, gid=postgresGID
                 )
             if os.getuid() == 0: # Only override if root
                 postgresUID = uid

Copied: CalendarServer/branches/users/glyph/more-deferreds-7/contrib/performance/sample-many.sh (from rev 6394, CalendarServer/trunk/contrib/performance/sample-many.sh)
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/contrib/performance/sample-many.sh	                        (rev 0)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/contrib/performance/sample-many.sh	2010-10-04 16:38:42 UTC (rev 6396)
@@ -0,0 +1,5 @@
+#!/bin/bash
+
+for i in `python -c "for i in range($START, $STOP, $STEP): print i,"`; do
+    ./sample.sh $i ~/Projects/CalendarServer/trunk/data/Logs "$1"
+done

Copied: CalendarServer/branches/users/glyph/more-deferreds-7/contrib/tools/anonymous_log.py (from rev 6394, CalendarServer/trunk/contrib/tools/anonymous_log.py)
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/contrib/tools/anonymous_log.py	                        (rev 0)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/contrib/tools/anonymous_log.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -0,0 +1,163 @@
+#!/usr/bin/env python
+##
+# Copyright (c) 2010 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from gzip import GzipFile
+import getopt
+import os
+import sys
+import traceback
+
+class CalendarServerLogAnalyzer(object):
+    
+    def __init__(self):
+        
+        self.userCtr = 1
+        self.users = {}
+
+        self.guidCtr = 1
+        self.guids = {}
+
+        self.resourceCtr = 1
+        self.resources = {}
+
+    def anonymizeLogFile(self, logFilePath):
+        
+        fpath = os.path.expanduser(logFilePath)
+        if fpath.endswith(".gz"):
+            f = GzipFile(fpath)
+        else:
+            f = open(fpath)
+            
+        try:
+            for line in f:
+                
+                if not line.startswith("Log"):
+                    line = self.anonymizeLine(line)
+                print line,
+        
+        except Exception, e:
+            print "Exception: %s for %s" % (e, line,)
+            raise
+
+    def anonymizeLine(self, line):
+
+        
+        startPos = line.find("- ")
+        endPos = line.find(" [")
+        userid = line[startPos+2:endPos]
+        
+        if userid != "-":
+            if userid not in self.users:
+                self.users[userid] = "user%05d" % (self.userCtr,)
+                self.userCtr += 1
+            line = line[:startPos+2] + self.users[userid] + line[endPos:]
+            endPos = line.find(" [")
+        
+        startPos = endPos + 1
+    
+        startPos = line.find(']', startPos + 21) + 3
+        endPos = line.find(' ', startPos)
+        if line[startPos] != '?':
+            
+            startPos = endPos + 1
+            endPos = line.find(" HTTP/", startPos)
+            uri = line[startPos:endPos]
+            
+            splits = uri.split("/")
+            if len(splits) >= 4:
+                if splits[1] in ("calendars", "principals"):
+
+                    if splits[3] not in self.guids:
+                        self.guids[splits[3]] = "guid%05d" % (self.guidCtr,)
+                        self.guidCtr += 1
+                    splits[3] = self.guids[splits[3]]
+
+                    if len(splits) > 4:
+                        if splits[4] not in ("", "calendar", "inbox", "outbox", "dropbox"):
+                            if splits[4] not in self.resources:
+                                self.resources[splits[4]] = "resource%d" % (self.resourceCtr,)
+                                self.resourceCtr += 1
+                            splits[4] = self.resources[splits[4]]
+                        
+                    if len(splits) > 5:
+                        for x in range(5, len(splits)):
+                            if splits[x]:
+                                if splits[x] not in self.resources:
+                                    self.resources[splits[x]] = "resource%d%s" % (self.resourceCtr, os.path.splitext(splits[x])[1])
+                                    self.resourceCtr += 1
+                                splits[x] = self.resources[splits[x]]
+                                
+                        
+                    line = line[:startPos] + "/".join(splits) + line[endPos:]
+    
+        return line
+
+def usage(error_msg=None):
+    if error_msg:
+        print error_msg
+
+    print """Usage: anonymous_log [options] [FILE]
+Options:
+    -h            Print this help and exit
+
+Arguments:
+    FILE      File names for the access logs to anonymize
+
+Description:
+    This utility will anonymize the content of an access log.
+
+"""
+
+    if error_msg:
+        raise ValueError(error_msg)
+    else:
+        sys.exit(0)
+
+if __name__ == "__main__":
+
+    try:
+
+        options, args = getopt.getopt(sys.argv[1:], "h", [])
+
+        for option, value in options:
+            if option == "-h":
+                usage()
+            else:
+                usage("Unrecognized option: %s" % (option,))
+
+        # Process arguments
+        if len(args) == 0:
+            args = ("/var/log/caldavd/access.log",)
+
+        pwd = os.getcwd()
+
+        analyzers = []
+        for arg in args:
+            arg = os.path.expanduser(arg)
+            if not arg.startswith("/"):
+                arg = os.path.join(pwd, arg)
+            if arg.endswith("/"):
+                arg = arg[:-1]
+            if not os.path.exists(arg):
+                print "Path does not exist: '%s'. Ignoring." % (arg,)
+                continue
+
+            CalendarServerLogAnalyzer().anonymizeLogFile(arg)
+
+    except Exception, e:
+        sys.exit(str(e))
+        print traceback.print_exc()

Copied: CalendarServer/branches/users/glyph/more-deferreds-7/contrib/tools/sortrecurrences.py (from rev 6394, CalendarServer/trunk/contrib/tools/sortrecurrences.py)
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/contrib/tools/sortrecurrences.py	                        (rev 0)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/contrib/tools/sortrecurrences.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -0,0 +1,80 @@
+#!/usr/bin/env python
+##
+# Copyright (c) 2010 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+import getopt
+import os
+import sys
+import traceback
+import vobject
+
+def usage(error_msg=None):
+    if error_msg:
+        print error_msg
+
+    print """Usage: sortrecurrences FILE
+Options:
+    -h            Print this help and exit
+
+Arguments:
+    FILE      File name for the calendar data to sort
+
+Description:
+    This utility will output a sorted iCalendar component.
+
+"""
+
+    if error_msg:
+        raise ValueError(error_msg)
+    else:
+        sys.exit(0)
+
+if __name__ == "__main__":
+
+    try:
+
+        options, args = getopt.getopt(sys.argv[1:], "h", [])
+
+        for option, value in options:
+            if option == "-h":
+                usage()
+            else:
+                usage("Unrecognized option: %s" % (option,))
+
+        # Process arguments
+        if len(args) != 1:
+            usage("Must have one argument")
+
+        pwd = os.getcwd()
+
+        analyzers = []
+        for arg in args:
+            arg = os.path.expanduser(arg)
+            if not arg.startswith("/"):
+                arg = os.path.join(pwd, arg)
+            if arg.endswith("/"):
+                arg = arg[:-1]
+            if not os.path.exists(arg):
+                print "Path does not exist: '%s'. Ignoring." % (arg,)
+                continue
+
+            cal = vobject.base.readOne(open(arg))
+            cal.contents['vevent'].sort(key=lambda x:str(x.contents.get('recurrence-id', (vobject.base.ContentLine("recurrence-id", {}, ""),)[0].value)))
+            print cal.serialize()
+
+    except Exception, e:
+        sys.exit(str(e))
+        print traceback.print_exc()

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/run
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/run	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/run	2010-10-04 16:38:42 UTC (rev 6396)
@@ -65,7 +65,6 @@
   echo "	-S  Write a pstats object for each process to the given directory when the server is stopped.";
   echo "	-P  Select the twistd plugin name [${plugin_name}]";
   echo "	-R  Twisted Reactor plugin to execute [${reactor}]";
-  echo "	-a  Start in CardDAV mode";
 
   if [ "${1-}" == "-" ]; then
     return 0;
@@ -81,7 +80,6 @@
     case "${option}" in
       '?') usage; ;;
       'h') usage -; exit 0; ;;
-      'a') DAVD=card; plugin_name="carddav"; ;;
       'v')       verbose="-v"; ;;
       'f')   force_setup="true"; ;;
       'k')          kill="true"; ;;

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twext/web2/dav/resource.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twext/web2/dav/resource.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twext/web2/dav/resource.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -224,8 +224,8 @@
                     if self.deadProperties().contains(qname):
                         return self.deadProperties().get(qname)
                     if self.isCollection():
-                        return davxml.ResourceType.collection
-                    return davxml.ResourceType.empty
+                        return davxml.ResourceType.collection #@UndefinedVariable
+                    return davxml.ResourceType.empty #@UndefinedVariable
 
                 if name == "getetag":
                     etag = self.etag()
@@ -270,12 +270,12 @@
                 if name == "supportedlock":
                     return davxml.SupportedLock(
                         davxml.LockEntry(
-                            davxml.LockScope.exclusive,
-                            davxml.LockType.write
+                            davxml.LockScope.exclusive, #@UndefinedVariable
+                            davxml.LockType.write #@UndefinedVariable
                         ),
                         davxml.LockEntry(
-                            davxml.LockScope.shared,
-                            davxml.LockType.write
+                            davxml.LockScope.shared, #@UndefinedVariable
+                            davxml.LockType.write #@UndefinedVariable
                         ),
                     )
 
@@ -720,6 +720,12 @@
             returnValue(None)
 
         # First find all depth 1 children
+        names1= []
+        namesDeep = []
+        if names:
+            for name in names:
+                (names1 if name.rstrip("/").find("/") == -1 else namesDeep).append(name.rstrip("/"))
+
         #children = []
         #yield self.findChildren("1", request, lambda x, y: children.append((x, y)), privileges=None, inherited_aces=None)
 
@@ -727,7 +733,7 @@
         basepath = request.urlForResource(self)
         childnames = list((yield self.listChildren()))
         for childname in childnames:
-            if names and childname not in names:
+            if names1 and childname not in names1:
                 continue
             childpath = joinURL(basepath, urllib.quote(childname))
             child = (yield request.locateChildResource(self, childname))
@@ -769,17 +775,24 @@
                     for resource, url in items[2]:
                         badcallback(resource, url)
 
-        # TODO: Depth: infinity support
         if depth == "infinity":
+            # Split names into child collection groups
+            child_collections = {}
+            for name in namesDeep:
+                collection, name = name.split("/", 1)
+                child_collections.setdefault(collection, []).append(name)
+
             for collection, url in allowed_collections:
-                collection_inherited_aces = (
-                    yield collection.inheritedACEsforChildren(request)
-                )
-                yield collection.findChildrenFaster(
-                    depth, request, okcallback, badcallback,
-                    names, privileges,
-                    inherited_aces=collection_inherited_aces
-                )
+                collection_name = collection.name()
+                if collection_name in child_collections:
+                    collection_inherited_aces = (
+                        yield collection.inheritedACEsforChildren(request)
+                    )
+                    yield collection.findChildrenFaster(
+                        depth, request, okcallback, badcallback,
+                        child_collections[collection_name], privileges,
+                        inherited_aces=collection_inherited_aces
+                    )
                 
         returnValue(None)
 

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/calendaruserproxy.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/calendaruserproxy.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/directory/calendaruserproxy.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -131,9 +131,9 @@
 
     def resourceType(self):
         if self.proxyType == "calendar-proxy-read":
-            return davxml.ResourceType.calendarproxyread
+            return davxml.ResourceType.calendarproxyread #@UndefinedVariable
         elif self.proxyType == "calendar-proxy-write":
-            return davxml.ResourceType.calendarproxywrite
+            return davxml.ResourceType.calendarproxywrite #@UndefinedVariable
         else:
             return super(CalendarUserProxyPrincipalResource, self).resourceType()
 
@@ -365,7 +365,7 @@
     """
 
     schema_version = "4"
-    schema_type    = "ProxyDB"
+    schema_type    = "CALENDARUSERPROXY"
     
     class ProxyDBMemcacher(Memcacher):
         

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/report_sync_collection.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/report_sync_collection.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/method/report_sync_collection.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -29,11 +29,12 @@
 from twext.web2 import responsecode
 from twext.web2.dav import davxml
 from twext.web2.dav.element.base import WebDAVElement
+from twext.web2.dav.element.extensions import SyncCollection
 from twext.web2.dav.http import MultiStatusResponse, statusForFailure
 from twext.web2.dav.method.prop_common import responseForHref
 from twext.web2.dav.method.propfind import propertyName
 from twext.web2.dav.util import joinURL
-from twext.web2.http import HTTPError
+from twext.web2.http import HTTPError, StatusResponse
 
 from twistedcaldav.config import config
 
@@ -46,16 +47,19 @@
     """
     Generate a sync-collection REPORT.
     """
-    if not config.EnableSyncReport or (
-        not self.isPseudoCalendarCollection() and
-        not self.isAddressBookCollection() and
-        not self.isNotificationCollection()
-    ):
+    
+    # These resource support the report
+    if not config.EnableSyncReport or davxml.Report(SyncCollection(),) not in self.supportedReports():
         log.err("sync-collection report is only allowed on calendar/inbox/addressbook/notification collection resources %s" % (self,))
         raise HTTPError(ErrorResponse(responsecode.FORBIDDEN, davxml.SupportedReport()))
    
     responses = []
 
+    depth = request.headers.getHeader("depth", None)
+    if depth not in ("1", "infinity"):
+        log.err("sync-collection report with invalid depth header: %s" % (depth,))
+        raise HTTPError(StatusResponse(responsecode.BAD_REQUEST, "Invalid Depth header value"))
+        
     propertyreq = sync_collection.property.children if sync_collection.property else None 
     
     @inlineCallbacks
@@ -102,14 +106,14 @@
     # the child resource loop and supply those to the checkPrivileges on each child.
     filteredaces = (yield self.inheritedACEsforChildren(request))
 
-    changed, removed, newtoken = yield self.whatchanged(sync_collection.sync_token)
+    changed, removed, newtoken = yield self.whatchanged(sync_collection.sync_token, depth)
 
     # Now determine which valid resources are readable and which are not
     ok_resources = []
     forbidden_resources = []
     if changed:
         yield self.findChildrenFaster(
-            "1",
+            depth,
             request,
             lambda x, y: ok_resources.append((x, y)),
             lambda x, y: forbidden_resources.append((x, y)),

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/notify.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/notify.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/notify.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -955,6 +955,15 @@
             d.addCallback(self.handleRoster)
 
     def allowedInRoster(self, jid):
+        """ Returns True if jid matches any of the patterns in AllowedJIDs,
+            or is our own JID.  False otherwise. """
+
+        # Always allow our own JID (in case multiple servers are sharing it)
+        settings = self.settings
+        if settings is not None:
+            if settings["JID"] == jid:
+                return True
+
         for pattern in self.settings.get("AllowedJIDs", []):
             if fnmatch(jid, pattern):
                 return True

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/resource.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/resource.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/resource.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -371,7 +371,7 @@
                 customxml.PubSubXMPPPushKeyProperty.qname(),
             )
 
-        if config.EnableSyncReport and (self.isPseudoCalendarCollection() or self.isAddressBookCollection()):
+        if config.EnableSyncReport and (davxml.Report(SyncCollection(),) in self.supportedReports()):
             baseProperties += (davxml.SyncToken.qname(),)
             
         if config.EnableAddMember and (self.isCalendarCollection() or self.isAddressBookCollection()):
@@ -540,7 +540,7 @@
             returnValue(customxml.GETCTag.fromString((yield self.getSyncToken())))
 
         elif qname == davxml.SyncToken.qname() and config.EnableSyncReport and (
-            self.isPseudoCalendarCollection() or self.isAddressBookCollection()
+            davxml.Report(SyncCollection(),) in self.supportedReports()
         ):
             returnValue(davxml.SyncToken.fromString((yield self.getSyncToken())))
 
@@ -1318,7 +1318,7 @@
 
 
     @inlineCallbacks
-    def whatchanged(self, client_token):
+    def whatchanged(self, client_token, depth):
         current_token = yield self.getSyncToken()
         current_uuid, current_revision = current_token.split("#", 1)
         current_revision = int(current_revision)
@@ -1339,14 +1339,15 @@
             revision = 0
 
         try:
-            changed, removed = yield self._indexWhatChanged(revision)
+            changed, removed = yield self._indexWhatChanged(revision, depth)
         except SyncTokenValidException:
             raise HTTPError(ErrorResponse(responsecode.FORBIDDEN, (dav_namespace, "valid-sync-token")))
 
         returnValue((changed, removed, current_token))
 
-    def _indexWhatChanged(self, revision):
-        return self.index().whatchanged(revision)
+    def _indexWhatChanged(self, revision, depth):
+        # Now handled directly by newstore
+        raise NotImplementedError
 
     def getSyncToken(self):
         """
@@ -2060,6 +2061,21 @@
         """
         return config.UserQuota if config.UserQuota != 0 else None
 
+    def supportedReports(self):
+        result = super(CommonHomeResource, self).supportedReports()
+        if config.EnableSyncReport:
+            # Allowed on any home
+            result.append(davxml.Report(SyncCollection(),))
+        return result
+
+    def _indexWhatChanged(self, revision, depth):
+        # The newstore implementation supports this directly
+        return self._newStoreHome.resourceNamesSinceToken(revision, depth)
+
+    def getSyncToken(self):
+        # The newstore implementation supports this directly
+        return self._newStoreHome.syncToken()
+
     def canShare(self):
         raise NotImplementedError
 
@@ -2534,6 +2550,9 @@
 # Utilities
 ##
 
+def isCalendarHomeCollectionResource(resource):
+    return isinstance(resource, CalendarHomeResource)
+
 def isCalendarCollectionResource(resource):
     try:
         resource = ICalDAVResource(resource)
@@ -2550,6 +2569,9 @@
     else:
         return resource.isPseudoCalendarCollection()
 
+def isAddressBookHomeCollectionResource(resource):
+    return isinstance(resource, AddressBookHomeResource)
+
 def isAddressBookCollectionResource(resource):
     try:
         resource = ICalDAVResource(resource)

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/stdconfig.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/stdconfig.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/stdconfig.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -1031,12 +1031,12 @@
     cleanDict = copy.deepcopy(configDict)
 
     def unknown(key):
-        config_key = "ICAL_SERVER_CONFIG_VALIDATION"
+        config_key = "CALENDARSERVER_CONFIG_VALIDATION"
         config_key_value = "loose"
         if config_key in os.environ and os.environ[config_key] == config_key_value:
             pass
         else:
-            log.err("Ignoring unknown configuration option: %r - you can optionally bypass this validation by setting a sys env var %s to '%s'" % (key, config_key, config_key_value))
+            log.err("Ignoring unknown configuration option: %r" % (key,))
             del cleanDict[key]
 
     def deprecated(oldKey, newKey):

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/storebridge.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/storebridge.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/storebridge.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -241,6 +241,10 @@
         return True
 
 
+    def _indexWhatChanged(self, revision, depth):
+        # The newstore implementation supports this directly
+        return self._newStoreCalendar.resourceNamesSinceToken(revision)
+
     @classmethod
     def transform(cls, self, calendar, home):
         """
@@ -529,7 +533,7 @@
     def accessControlList(self, *a, **kw):
         """
         All principals identified as ATTENDEEs on the event for this dropbox
-        may read all its children.
+        may read all its children. Also include proxies of ATTENDEEs.
         """
         d = super(CalendarObjectDropbox, self).accessControlList(*a, **kw)
         def moreACLs(originalACL):
@@ -544,19 +548,35 @@
                     calendarUserAddress
                 )
                 principalURL = principal.principalURL()
-                if othersCanWrite:
-                    privileges = [davxml.Privilege(davxml.All())]
-                else:
-                    privileges = [
-                        davxml.Privilege(davxml.Read()),
-                        davxml.Privilege(davxml.ReadCurrentUserPrivilegeSet())
-                    ]
+                writePrivileges = [
+                    davxml.Privilege(davxml.Read()),
+                    davxml.Privilege(davxml.ReadCurrentUserPrivilegeSet()),
+                    davxml.Privilege(davxml.Write()),
+                ]
+                readPrivileges = [
+                    davxml.Privilege(davxml.Read()),
+                    davxml.Privilege(davxml.ReadCurrentUserPrivilegeSet()),
+                ]
+                privileges = writePrivileges if othersCanWrite else readPrivileges
                 newACEs.append(davxml.ACE(
                     davxml.Principal(davxml.HRef(principalURL)),
                     davxml.Grant(*privileges),
                     davxml.Protected(),
                     TwistedACLInheritable(),
                 ))
+                newACEs.append(davxml.ACE(
+                    davxml.Principal(davxml.HRef(joinURL(principalURL, "calendar-proxy-write/"))),
+                    davxml.Grant(*privileges),
+                    davxml.Protected(),
+                    TwistedACLInheritable(),
+                ))
+                newACEs.append(davxml.ACE(
+                    davxml.Principal(davxml.HRef(joinURL(principalURL, "calendar-proxy-read/"))),
+                    davxml.Grant(*readPrivileges),
+                    davxml.Protected(),
+                    TwistedACLInheritable(),
+                ))
+
             return davxml.ACL(*tuple(newACEs + originalACEs))
         d.addCallback(moreACLs)
         return d
@@ -1329,6 +1349,10 @@
         return True
 
 
+    def _indexWhatChanged(self, revision, depth):
+        # The newstore implementation supports this directly
+        return self._newStoreAddressBook.resourceNamesSinceToken(revision)
+
     @classmethod
     def transform(cls, self, addressbook, home):
         """
@@ -1923,7 +1947,7 @@
     def getSyncToken(self):
         return self._newStoreNotifications.syncToken()
 
-    def _indexWhatChanged(self, revision):
+    def _indexWhatChanged(self, revision, depth):
         return self._newStoreNotifications.resourceNamesSinceToken(revision)
 
     def addNotification(self, request, uid, xmltype, xmldata):

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_notify.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_notify.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/test/test_notify.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -533,6 +533,7 @@
 
 
 
+
 class XMPPNotificationFactoryTests(TestCase):
 
     def test_sendPresence(self):
@@ -605,3 +606,16 @@
         self.assertEquals(conf, None)
         conf = getPubSubAPSConfiguration("UnknownPrefix|foo", enabledConfig)
         self.assertEquals(conf, None)
+
+    def test_allowedInRoster(self):
+        """
+        Our own JID is implicitly included in AllowedJIDs
+        """
+        settings = {
+            "JID" : "test1 at example.com",
+            "AllowedJIDs" : ["test2 at example.com"]
+        }
+        notifier = XMPPNotifier(settings, heartbeat=False)
+        self.assertTrue(notifier.allowedInRoster("test1 at example.com"))
+        self.assertTrue(notifier.allowedInRoster("test2 at example.com"))
+        self.assertFalse(notifier.allowedInRoster("test3 at example.com"))

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/upgrade.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/upgrade.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/twistedcaldav/upgrade.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -35,7 +35,7 @@
 from twistedcaldav.ical import Component
 from twistedcaldav import caldavxml
 
-from twisted.internet.defer import inlineCallbacks, succeed, returnValue
+from twisted.internet.defer import inlineCallbacks, succeed
 
 from calendarserver.tools.util import getDirectory
 from calendarserver.tools.resources import migrateResources
@@ -465,6 +465,17 @@
 
 def upgrade_to_2(config):
     #
+    # Rename proxy DB
+    #
+    oldFilename = "calendaruserproxy.sqlite"
+    newFilename = "proxies.sqlite"
+
+    oldDbPath = os.path.join(config.DataRoot, oldFilename)
+    newDbPath = os.path.join(config.DataRoot, newFilename)
+    if os.path.exists(oldDbPath) and not os.path.exists(newDbPath):
+        os.rename(oldDbPath, newDbPath)
+
+    #
     # Migrates locations and resources from OD
     #
 

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/base.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/base.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/base.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -179,7 +179,7 @@
 
     def keys(self):
         
-        userkeys = self._keys_uid(self._peruser)
+        userkeys = list(self._keys_uid(self._peruser))
         if self._defaultuser != self._peruser:
             defaultkeys = self._keys_uid(self._defaultuser)
             for key in defaultkeys:

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/test/base.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/test/base.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/base/propertystore/test/base.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -275,7 +275,20 @@
         self.assertEquals(self.propertyStore.get(name, None), None)
         self.assertEquals(len(self.propertyStore), 0)
 
+    def test_peruser_keys(self):
 
+        name = propertyName("shadow")
+
+        self.propertyStore1.setSpecialProperties((name,), ())
+        self.propertyStore2.setSpecialProperties((name,), ())
+
+        value1 = propertyValue("Hello, World1!")
+
+        self.propertyStore1[name] = value1
+        self._changed(self.propertyStore1)
+
+        self.failUnless(name in self.propertyStore2.keys())
+
 def propertyName(name):
     return PropertyName("http://calendarserver.org/ns/test/", name)
 

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/index_file.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/index_file.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/index_file.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -269,7 +269,7 @@
             self.log_info("Search falls outside range of index for %s %s" % (name, minDate))
             self.reExpandResource(name, minDate)
 
-    def whatchanged(self, revision):
+    def whatchanged(self, revision, depth):
 
         results = [(name.encode("utf-8"), deleted) for name, deleted in self._db_execute("select NAME, DELETED from REVISIONS where REVISION > :1", revision)]
         results.sort(key=lambda x:x[1])


Property changes on: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/index_file.py
___________________________________________________________________
Added: svn:mergeinfo
   + /CalendarServer/branches/config-separation/txdav/caldav/datastore/index_file.py:4379-4443
/CalendarServer/branches/egg-info-351/txdav/caldav/datastore/index_file.py:4589-4625
/CalendarServer/branches/generic-sqlstore/txdav/caldav/datastore/index_file.py:6167-6191
/CalendarServer/branches/new-store/txdav/caldav/datastore/index_file.py:5594-5934
/CalendarServer/branches/new-store-no-caldavfile/txdav/caldav/datastore/index_file.py:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2/txdav/caldav/datastore/index_file.py:5936-5981
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/caldav/datastore/index_file.py:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/caldav/datastore/index_file.py:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/caldav/datastore/index_file.py:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/txdav/caldav/datastore/index_file.py:4465-4957
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/caldav/datastore/index_file.py:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/caldav/datastore/index_file.py:5188-5440
/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/caldav/datastore/index_file.py:4971-5080
/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/caldav/datastore/index_file.py:6322-6368
/CalendarServer/branches/users/glyph/sendfdport/txdav/caldav/datastore/index_file.py:5388-5424
/CalendarServer/branches/users/glyph/sql-store/txdav/caldav/datastore/index_file.py:5929-6073
/CalendarServer/branches/users/glyph/use-system-twisted/txdav/caldav/datastore/index_file.py:5084-5149
/CalendarServer/branches/users/sagen/locations-resources/txdav/caldav/datastore/index_file.py:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2/txdav/caldav/datastore/index_file.py:5052-5061
/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/caldav/datastore/index_file.py:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/caldav/datastore/index_file.py:4068-4075
/CalendarServer/branches/users/sagen/resources-2/txdav/caldav/datastore/index_file.py:5084-5093
/CalendarServer/branches/users/wsanchez/transations/txdav/caldav/datastore/index_file.py:5515-5593
/CalendarServer/trunk/twistedcaldav/index.py:6322-6394
/CalendarServer/trunk/txdav/caldav/datastore/index_file.py:6369-6394

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/sql.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/sql.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/sql.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -63,13 +63,15 @@
     implements(ICalendarHome)
 
     def __init__(self, transaction, ownerUID, resourceID, notifier):
-        super(CalendarHome, self).__init__(transaction, ownerUID, resourceID, notifier)
 
-        self._shares = SQLLegacyCalendarShares(self)
         self._childClass = Calendar
         self._childTable = CALENDAR_TABLE
         self._bindTable = CALENDAR_BIND_TABLE
+        self._revisionsTable = CALENDAR_OBJECT_REVISIONS_TABLE
 
+        super(CalendarHome, self).__init__(transaction, ownerUID, resourceID, notifier)
+        self._shares = SQLLegacyCalendarShares(self)
+
     createCalendarWithName = CommonHome.createChildWithName
     removeCalendarWithName = CommonHome.removeChildWithName
     calendarWithName = CommonHome.childWithName

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_index_file.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_index_file.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_index_file.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -927,7 +927,8 @@
         )
         
         for revision, results in tests:
-            self.assertEquals(self.db.whatchanged(revision), results, "Mismatched results for whatchanged with revision %d" % (revision,))
+            for depth in ("1", "infinity"):
+                self.assertEquals(self.db.whatchanged(revision, depth), results, "Mismatched results for whatchanged with revision %d" % (revision,))
 
 class MemcacheTests(SQLIndexTests):
     def setUp(self):


Property changes on: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/caldav/datastore/test/test_index_file.py
___________________________________________________________________
Added: svn:mergeinfo
   + /CalendarServer/branches/config-separation/txdav/caldav/datastore/test/test_index_file.py:4379-4443
/CalendarServer/branches/egg-info-351/txdav/caldav/datastore/test/test_index_file.py:4589-4625
/CalendarServer/branches/generic-sqlstore/txdav/caldav/datastore/test/test_index_file.py:6167-6191
/CalendarServer/branches/new-store/txdav/caldav/datastore/test/test_index_file.py:5594-5934
/CalendarServer/branches/new-store-no-caldavfile/txdav/caldav/datastore/test/test_index_file.py:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2/txdav/caldav/datastore/test/test_index_file.py:5936-5981
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/caldav/datastore/test/test_index_file.py:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/caldav/datastore/test/test_index_file.py:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/caldav/datastore/test/test_index_file.py:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/txdav/caldav/datastore/test/test_index_file.py:4465-4957
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/caldav/datastore/test/test_index_file.py:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/caldav/datastore/test/test_index_file.py:5188-5440
/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/caldav/datastore/test/test_index_file.py:4971-5080
/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/caldav/datastore/test/test_index_file.py:6322-6368
/CalendarServer/branches/users/glyph/sendfdport/txdav/caldav/datastore/test/test_index_file.py:5388-5424
/CalendarServer/branches/users/glyph/sql-store/txdav/caldav/datastore/test/test_index_file.py:5929-6073
/CalendarServer/branches/users/glyph/use-system-twisted/txdav/caldav/datastore/test/test_index_file.py:5084-5149
/CalendarServer/branches/users/sagen/locations-resources/txdav/caldav/datastore/test/test_index_file.py:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2/txdav/caldav/datastore/test/test_index_file.py:5052-5061
/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/caldav/datastore/test/test_index_file.py:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/caldav/datastore/test/test_index_file.py:4068-4075
/CalendarServer/branches/users/sagen/resources-2/txdav/caldav/datastore/test/test_index_file.py:5084-5093
/CalendarServer/branches/users/wsanchez/transations/txdav/caldav/datastore/test/test_index_file.py:5515-5593
/CalendarServer/trunk/twistedcaldav/test/test_index.py:6322-6394
/CalendarServer/trunk/txdav/caldav/datastore/test/test_index_file.py:6369-6394

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/index_file.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/index_file.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/index_file.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -380,7 +380,7 @@
         results = self._db_values_for_sql(statement, *names)
         return results
     
-    def whatchanged(self, revision):
+    def whatchanged(self, revision, depth):
 
         results = [(name.encode("utf-8"), deleted) for name, deleted in self._db_execute("select NAME, DELETED from REVISIONS where REVISION > :1", revision)]
         results.sort(key=lambda x:x[1])


Property changes on: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/index_file.py
___________________________________________________________________
Added: svn:mergeinfo
   + /CalendarServer/branches/config-separation/txdav/carddav/datastore/index_file.py:4379-4443
/CalendarServer/branches/egg-info-351/txdav/carddav/datastore/index_file.py:4589-4625
/CalendarServer/branches/generic-sqlstore/txdav/carddav/datastore/index_file.py:6167-6191
/CalendarServer/branches/new-store/txdav/carddav/datastore/index_file.py:5594-5934
/CalendarServer/branches/new-store-no-caldavfile/txdav/carddav/datastore/index_file.py:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2/txdav/carddav/datastore/index_file.py:5936-5981
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/carddav/datastore/index_file.py:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/carddav/datastore/index_file.py:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/carddav/datastore/index_file.py:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/txdav/carddav/datastore/index_file.py:4465-4957
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/carddav/datastore/index_file.py:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/carddav/datastore/index_file.py:5188-5440
/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/carddav/datastore/index_file.py:4971-5080
/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/carddav/datastore/index_file.py:6322-6368
/CalendarServer/branches/users/glyph/sendfdport/txdav/carddav/datastore/index_file.py:5388-5424
/CalendarServer/branches/users/glyph/sql-store/txdav/carddav/datastore/index_file.py:5929-6073
/CalendarServer/branches/users/glyph/use-system-twisted/txdav/carddav/datastore/index_file.py:5084-5149
/CalendarServer/branches/users/sagen/locations-resources/txdav/carddav/datastore/index_file.py:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2/txdav/carddav/datastore/index_file.py:5052-5061
/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/carddav/datastore/index_file.py:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/carddav/datastore/index_file.py:4068-4075
/CalendarServer/branches/users/sagen/resources-2/txdav/carddav/datastore/index_file.py:5084-5093
/CalendarServer/branches/users/wsanchez/transations/txdav/carddav/datastore/index_file.py:5515-5593
/CalendarServer/trunk/twistedcaldav/vcardindex.py:6322-6394
/CalendarServer/trunk/txdav/carddav/datastore/index_file.py:6369-6394

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/sql.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/sql.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/sql.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -49,13 +49,15 @@
     implements(IAddressBookHome)
 
     def __init__(self, transaction, ownerUID, resourceID, notifier):
-        super(AddressBookHome, self).__init__(transaction, ownerUID, resourceID, notifier)
 
-        self._shares = SQLLegacyAddressBookShares(self)
         self._childClass = AddressBook
         self._childTable = ADDRESSBOOK_TABLE
         self._bindTable = ADDRESSBOOK_BIND_TABLE
+        self._revisionsTable = ADDRESSBOOK_OBJECT_REVISIONS_TABLE
 
+        super(AddressBookHome, self).__init__(transaction, ownerUID, resourceID, notifier)
+        self._shares = SQLLegacyAddressBookShares(self)
+
     addressbooks = CommonHome.children
     listAddressbooks = CommonHome.listChildren
     addressbookWithName = CommonHome.childWithName

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_index_file.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_index_file.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_index_file.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -197,7 +197,8 @@
         )
         
         for revision, results in tests:
-            self.assertEquals(self.db.whatchanged(revision), results, "Mismatched results for whatchanged with revision %d" % (revision,))
+            for depth in ("1", "infinity"):
+                self.assertEquals(self.db.whatchanged(revision, depth), results, "Mismatched results for whatchanged with revision %d" % (revision,))
 
 class MemcacheTests(SQLIndexTests):
     def setUp(self):


Property changes on: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/carddav/datastore/test/test_index_file.py
___________________________________________________________________
Added: svn:mergeinfo
   + /CalendarServer/branches/config-separation/txdav/carddav/datastore/test/test_index_file.py:4379-4443
/CalendarServer/branches/egg-info-351/txdav/carddav/datastore/test/test_index_file.py:4589-4625
/CalendarServer/branches/generic-sqlstore/txdav/carddav/datastore/test/test_index_file.py:6167-6191
/CalendarServer/branches/new-store/txdav/carddav/datastore/test/test_index_file.py:5594-5934
/CalendarServer/branches/new-store-no-caldavfile/txdav/carddav/datastore/test/test_index_file.py:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2/txdav/carddav/datastore/test/test_index_file.py:5936-5981
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692/txdav/carddav/datastore/test/test_index_file.py:5693-5702
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627/txdav/carddav/datastore/test/test_index_file.py:3628-3644
/CalendarServer/branches/users/cdaboo/more-sharing-5591/txdav/carddav/datastore/test/test_index_file.py:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464/txdav/carddav/datastore/test/test_index_file.py:4465-4957
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070/txdav/carddav/datastore/test/test_index_file.py:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187/txdav/carddav/datastore/test/test_index_file.py:5188-5440
/CalendarServer/branches/users/glyph/contacts-server-merge/txdav/carddav/datastore/test/test_index_file.py:4971-5080
/CalendarServer/branches/users/glyph/more-deferreds-6/txdav/carddav/datastore/test/test_index_file.py:6322-6368
/CalendarServer/branches/users/glyph/sendfdport/txdav/carddav/datastore/test/test_index_file.py:5388-5424
/CalendarServer/branches/users/glyph/sql-store/txdav/carddav/datastore/test/test_index_file.py:5929-6073
/CalendarServer/branches/users/glyph/use-system-twisted/txdav/carddav/datastore/test/test_index_file.py:5084-5149
/CalendarServer/branches/users/sagen/locations-resources/txdav/carddav/datastore/test/test_index_file.py:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2/txdav/carddav/datastore/test/test_index_file.py:5052-5061
/CalendarServer/branches/users/sagen/resource-delegates-4038/txdav/carddav/datastore/test/test_index_file.py:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066/txdav/carddav/datastore/test/test_index_file.py:4068-4075
/CalendarServer/branches/users/sagen/resources-2/txdav/carddav/datastore/test/test_index_file.py:5084-5093
/CalendarServer/branches/users/wsanchez/transations/txdav/carddav/datastore/test/test_index_file.py:5515-5593
/CalendarServer/trunk/twistedcaldav/test/test_vcardindex.py:6322-6394
/CalendarServer/trunk/txdav/carddav/datastore/test/test_index_file.py:6369-6394

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -290,6 +290,8 @@
     _childClass = None
     _childTable = None
     _bindTable = None
+    _revisionsTable = None
+    _notificationRevisionsTable = NOTIFICATION_OBJECT_REVISIONS_TABLE
 
     def __init__(self, transaction, ownerUID, resourceID, notifier):
         self._txn = transaction
@@ -299,6 +301,12 @@
         self._children = {}
         self._notifier = notifier
 
+        # Needed for REVISION/BIND table join
+        self._revisionBindJoinTable = {}
+        for key, value in self._revisionsTable.iteritems():
+            self._revisionBindJoinTable["REV:%s" % (key,)] = value 
+        for key, value in self._bindTable.iteritems():
+            self._revisionBindJoinTable["BIND:%s" % (key,)] = value 
 
     def __repr__(self):
         return "<%s: %s>" % (self.__class__.__name__, self._resourceID)
@@ -432,6 +440,7 @@
         child = self.childWithName(name)
         if not child:
             raise NoSuchHomeChildError()
+        child._deletedSyncToken()
 
         self._txn.execSQL(
             "delete from %(name)s where %(column_RESOURCE_ID)s = %%s" % self._childTable,
@@ -444,6 +453,52 @@
         child.notifyChanged()
 
 
+    def syncToken(self):
+        revision = self._txn.execSQL(
+            """
+            select max(%(column_REVISION)s) from %(name)s
+            where %(column_HOME_RESOURCE_ID)s = %%s
+            """ % self._revisionsTable,
+            [self._resourceID,]
+        )[0][0]
+        return "%s#%s" % (self._resourceID, revision)
+
+    def resourceNamesSinceToken(self, token, depth):
+        results = [
+            (
+                path if path else (collection if collection else ""),
+                name if name else "",
+                deleted
+            )
+            for path, collection, name, deleted in
+            self._txn.execSQL("""
+                select %(BIND:column_RESOURCE_NAME)s, %(REV:column_COLLECTION_NAME)s, %(REV:column_RESOURCE_NAME)s, %(REV:column_DELETED)s
+                from %(REV:name)s
+                left outer join %(BIND:name)s on (%(REV:name)s.%(REV:column_RESOURCE_ID)s = %(BIND:name)s.%(BIND:column_RESOURCE_ID)s)
+                where %(REV:column_REVISION)s > %%s and %(REV:name)s.%(REV:column_HOME_RESOURCE_ID)s = %%s
+                """ % self._revisionBindJoinTable,
+                [token, self._resourceID],
+            )
+        ]
+        
+        deleted = []
+        deleted_collections = set()
+        for path, name, wasdeleted in results:
+            if wasdeleted:
+                if token:
+                    deleted.append("%s/%s" % (path, name,))
+                if not name:
+                    deleted_collections.add(path)
+        
+        changed = []
+        for path, name, wasdeleted in results:
+            if path not in deleted_collections:
+                changed.append("%s/%s" % (path, name,))
+        
+        changed.sort()
+        deleted.sort()
+        return changed, deleted,
+
     @cached
     def properties(self):
         return PropertyStore(
@@ -550,7 +605,7 @@
         # update memos
         del self._home._children[oldName]
         self._home._children[name] = self
-        self._updateSyncToken()
+        self._renameSyncToken()
 
         self.notifyChanged()
 
@@ -654,22 +709,13 @@
         self.notifyChanged()
 
 
-    def _initSyncToken(self):
-        self._txn.execSQL("""
-            insert into %(name)s
-            (%(column_HOME_RESOURCE_ID)s, %(column_RESOURCE_ID)s, %(column_RESOURCE_NAME)s, %(column_REVISION)s, %(column_DELETED)s)
-            values (%%s, %%s, %%s, nextval('%(sequence)s'), FALSE)
-            """ % self._revisionsTable,
-            [self._home._resourceID, self._resourceID, ""]
-        )
-
     def syncToken(self):
         revision = self._txn.execSQL(
             """
             select %(column_REVISION)s from %(name)s
-            where %(column_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s = %%s
+            where %(column_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s is null
             """ % self._revisionsTable,
-            [self._resourceID, ""]
+            [self._resourceID,]
         )[0][0]
         return "%s#%s" % (self._resourceID, revision,)
 
@@ -678,11 +724,11 @@
 
     def resourceNamesSinceToken(self, token):
         results = [
-            (name.encode("utf-8"), deleted)
+            (name if name else "", deleted)
             for name, deleted in
             self._txn.execSQL("""
                 select %(column_RESOURCE_NAME)s, %(column_DELETED)s from %(name)s
-                where %(column_REVISION)s > %s and %(column_RESOURCE_ID)s = %s
+                where %(column_REVISION)s > %%s and %(column_RESOURCE_ID)s = %%s
                 """ % self._revisionsTable,
                 [token, self._resourceID],
             )
@@ -701,16 +747,65 @@
         
         return changed, deleted,
 
+    def _initSyncToken(self):
+        
+        # Remove any deleted revision entry that uses the same name
+        self._txn.execSQL("""
+            delete from %(name)s
+            where %(column_HOME_RESOURCE_ID)s = %%s and %(column_COLLECTION_NAME)s = %%s
+            """ % self._revisionsTable,
+            [self._home._resourceID, self._name]
+        )
+
+        # Insert new entry
+        self._txn.execSQL("""
+            insert into %(name)s
+            (%(column_HOME_RESOURCE_ID)s, %(column_RESOURCE_ID)s, %(column_COLLECTION_NAME)s, %(column_RESOURCE_NAME)s, %(column_REVISION)s, %(column_DELETED)s)
+            values (%%s, %%s, %%s, null, nextval('%(sequence)s'), FALSE)
+            """ % self._revisionsTable,
+            [self._home._resourceID, self._resourceID, self._name]
+        )
+
     def _updateSyncToken(self):
 
         self._txn.execSQL("""
             update %(name)s
             set (%(column_REVISION)s) = (nextval('%(sequence)s'))
-            where %(column_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s = %%s
+            where %(column_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s is null
             """ % self._revisionsTable,
-            [self._resourceID, ""]
+            [self._resourceID,]
         )
 
+    def _renameSyncToken(self):
+
+        self._txn.execSQL("""
+            update %(name)s
+            set (%(column_REVISION)s, %(column_COLLECTION_NAME)s) = (nextval('%(sequence)s'), %%s)
+            where %(column_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s is null
+            """ % self._revisionsTable,
+            [self._name, self._resourceID,]
+        )
+
+    def _deletedSyncToken(self):
+
+        # Remove all child entries
+        self._txn.execSQL("""
+            delete from %(name)s
+            where %(column_HOME_RESOURCE_ID)s = %%s and %(column_RESOURCE_ID)s = %%s and %(column_COLLECTION_NAME)s is null
+            """ % self._revisionsTable,
+            [self._home._resourceID, self._resourceID,]
+        )
+        
+        # Then adjust collection entry to deleted state
+        self._txn.execSQL("""
+            update %(name)s
+            set (%(column_RESOURCE_ID)s, %(column_REVISION)s, %(column_DELETED)s)
+             = (null, nextval('%(sequence)s'), TRUE)
+            where %(column_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s is null
+            """ % self._revisionsTable,
+            [self._resourceID,]
+        )
+
     def _insertRevision(self, name):
         self._changeRevision("insert", name)
 
@@ -738,9 +833,9 @@
             self._txn.execSQL("""
                 update %(name)s
                 set (%(column_REVISION)s) = (%%s)
-                where %(column_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s = %%s
+                where %(column_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s is null
                 """ % self._revisionsTable,
-                [nextrevision, self._resourceID, ""]
+                [nextrevision, self._resourceID,]
             )
         elif action == "update":
             self._txn.execSQL("""
@@ -753,9 +848,9 @@
             self._txn.execSQL("""
                 update %(name)s
                 set (%(column_REVISION)s) = (%%s)
-                where %(column_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s = %%s
+                where %(column_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s is null
                 """ % self._revisionsTable,
-                [nextrevision, self._resourceID, ""]
+                [nextrevision, self._resourceID,]
             )
         elif action == "insert":
             # Note that an "insert" may happen for a resource that previously existed and then
@@ -788,9 +883,9 @@
             self._txn.execSQL("""
                 update %(name)s
                 set (%(column_REVISION)s) = (%%s)
-                where %(column_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s = %%s
+                where %(column_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s is null
                 """ % self._revisionsTable,
-                [nextrevision, self._resourceID, ""]
+                [nextrevision, self._resourceID,]
             )
 
     @cached
@@ -1050,18 +1145,18 @@
         self._txn.execSQL("""
             insert into %(name)s
             (%(column_HOME_RESOURCE_ID)s, %(column_RESOURCE_NAME)s, %(column_REVISION)s, %(column_DELETED)s)
-            values (%%s, %%s, nextval('%(sequence)s'), FALSE)
+            values (%%s, null, nextval('%(sequence)s'), FALSE)
             """ % self._revisionsTable,
-            [self._resourceID, ""]
+            [self._resourceID,]
         )
 
     def syncToken(self):
         revision = self._txn.execSQL(
             """
             select %(column_REVISION)s from %(name)s
-            where %(column_HOME_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s = %%s
+            where %(column_HOME_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s is null
             """ % self._revisionsTable,
-            [self._resourceID, ""]
+            [self._resourceID,]
         )[0][0]
         return "%s#%s" % (self._resourceID, revision,)
 
@@ -1078,7 +1173,7 @@
 
     def resourceNamesSinceToken(self, token):
         results = [
-            (name.encode("utf-8"), deleted)
+            (name if name else "", deleted)
             for name, deleted in
             self._txn.execSQL("""
                 select %(column_RESOURCE_NAME)s, %(column_DELETED)s from %(name)s
@@ -1106,9 +1201,9 @@
         self._txn.execSQL("""
             update %(name)s
             set (%(column_REVISION)s) = (nextval('%(sequence)s'))
-            where %(column_HOME_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s = %%s
+            where %(column_HOME_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s is null
             """ % self._revisionsTable,
-            [self._resourceID, ""]
+            [self._resourceID,]
         )
 
     def _insertRevision(self, name):
@@ -1138,9 +1233,9 @@
             self._txn.execSQL("""
                 update %(name)s
                 set (%(column_REVISION)s) = (%%s)
-                where %(column_HOME_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s = %%s
+                where %(column_HOME_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s is null
                 """ % self._revisionsTable,
-                [nextrevision, self._resourceID, ""]
+                [nextrevision, self._resourceID]
             )
         elif action == "update":
             self._txn.execSQL("""
@@ -1153,9 +1248,9 @@
             self._txn.execSQL("""
                 update %(name)s
                 set (%(column_REVISION)s) = (%%s)
-                where %(column_HOME_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s = %%s
+                where %(column_HOME_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s is null
                 """ % self._revisionsTable,
-                [nextrevision, self._resourceID, ""]
+                [nextrevision, self._resourceID]
             )
         elif action == "insert":
             # Note that an "insert" may happen for a resource that previously existed and then
@@ -1188,9 +1283,9 @@
             self._txn.execSQL("""
                 update %(name)s
                 set (%(column_REVISION)s) = (%%s)
-                where %(column_HOME_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s = %%s
+                where %(column_HOME_RESOURCE_ID)s = %%s and %(column_RESOURCE_NAME)s is null
                 """ % self._revisionsTable,
-                [nextrevision, self._resourceID, ""]
+                [nextrevision, self._resourceID]
             )
 
     @cached

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_legacy.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_legacy.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_legacy.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -924,30 +924,6 @@
             self.reExpandResource(name, minDate)
 
 
-    def whatchanged(self, revision):
-        results = [
-            (name.encode("utf-8"), deleted)
-            for name, deleted in
-            self._txn.execSQL(
-                """select RESOURCE_NAME, DELETED from CALENDAR_OBJECT_REVISIONS
-                   where REVISION > %s and CALENDAR_RESOURCE_ID = %s""",
-                [revision, self.calendar._resourceID],
-            )
-        ]
-        results.sort(key=lambda x:x[1])
-
-        changed = []
-        deleted = []
-        for name, wasdeleted in results:
-            if name:
-                if wasdeleted:
-                    if revision:
-                        deleted.append(name)
-                else:
-                    changed.append(name)
-        return changed, deleted,
-
-
     @inlineCallbacks
     def indexedSearch(self, filter, useruid='', fbtype=False):
         """
@@ -1187,31 +1163,6 @@
         returnValue(obj.name())
 
 
-    def whatchanged(self, revision):
-
-        results = [
-            (name.encode("utf-8"), deleted)
-            for name, deleted in
-            self._txn.execSQL(
-                """select RESOURCE_NAME, DELETED from ADDRESSBOOK_OBJECT_REVISIONS
-                   where REVISION > %s and ADDRESSBOOK_RESOURCE_ID = %s""",
-                [revision, self.addressbook._resourceID],
-            )
-        ]
-        results.sort(key=lambda x:x[1])
-        
-        changed = []
-        deleted = []
-        for name, wasdeleted in results:
-            if name:
-                if wasdeleted:
-                    if revision:
-                        deleted.append(name)
-                else:
-                    changed.append(name)
-        
-        return changed, deleted,
-
     def searchValid(self, filter):
         if isinstance(filter, carddavxml.Filter):
             qualifiers = addressbookquery.sqladdressbookquery(filter)

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_schema_v1.sql
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_schema_v1.sql	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_schema_v1.sql	2010-10-04 16:38:42 UTC (rev 6396)
@@ -312,8 +312,9 @@
 
 create table CALENDAR_OBJECT_REVISIONS (
   CALENDAR_HOME_RESOURCE_ID integer      not null references CALENDAR_HOME,
-  CALENDAR_RESOURCE_ID      integer      not null references CALENDAR on delete cascade,
-  RESOURCE_NAME             varchar(255) not null,
+  CALENDAR_RESOURCE_ID      integer      references CALENDAR,
+  CALENDAR_NAME             varchar(255) default null,
+  RESOURCE_NAME             varchar(255),
   REVISION                  integer      not null,
   DELETED                   boolean      not null,
 
@@ -327,8 +328,9 @@
 
 create table ADDRESSBOOK_OBJECT_REVISIONS (
   ADDRESSBOOK_HOME_RESOURCE_ID integer      not null references ADDRESSBOOK_HOME,
-  ADDRESSBOOK_RESOURCE_ID      integer      not null references ADDRESSBOOK on delete cascade,
-  RESOURCE_NAME                varchar(255) not null,
+  ADDRESSBOOK_RESOURCE_ID      integer      references ADDRESSBOOK,
+  ADDRESSBOOK_NAME             varchar(255) default null,
+  RESOURCE_NAME                varchar(255),
   REVISION                     integer      not null,
   DELETED                      boolean      not null,
 
@@ -342,7 +344,7 @@
 
 create table NOTIFICATION_OBJECT_REVISIONS (
   NOTIFICATION_HOME_RESOURCE_ID integer      not null references NOTIFICATION_HOME on delete cascade,
-  RESOURCE_NAME                 varchar(255) not null,
+  RESOURCE_NAME                 varchar(255),
   REVISION                      integer      not null,
   DELETED                       boolean      not null,
 

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_tables.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_tables.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/sql_tables.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -80,6 +80,7 @@
     "sequence"                : "REVISION_SEQ",
     "column_HOME_RESOURCE_ID" : "CALENDAR_HOME_RESOURCE_ID",
     "column_RESOURCE_ID"      : "CALENDAR_RESOURCE_ID",
+    "column_COLLECTION_NAME"  : "CALENDAR_NAME",
     "column_RESOURCE_NAME"    : "RESOURCE_NAME",
     "column_REVISION"         : "REVISION",
     "column_DELETED"          : "DELETED",
@@ -90,6 +91,7 @@
     "sequence"                : "REVISION_SEQ",
     "column_HOME_RESOURCE_ID" : "ADDRESSBOOK_HOME_RESOURCE_ID",
     "column_RESOURCE_ID"      : "ADDRESSBOOK_RESOURCE_ID",
+    "column_COLLECTION_NAME"  : "ADDRESSBOOK_NAME",
     "column_RESOURCE_NAME"    : "RESOURCE_NAME",
     "column_REVISION"         : "REVISION",
     "column_DELETED"          : "DELETED",

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/test/util.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/test/util.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/test/util.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -128,6 +128,9 @@
         tables = ['INVITE',
                   'RESOURCE_PROPERTY',
                   'ATTACHMENT',
+                  'NOTIFICATION_OBJECT_REVISIONS',
+                  'ADDRESSBOOK_OBJECT_REVISIONS',
+                  'CALENDAR_OBJECT_REVISIONS',
                   'ADDRESSBOOK_OBJECT',
                   'CALENDAR_OBJECT',
                   'CALENDAR_BIND',

Modified: CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/util.py
===================================================================
--- CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/util.py	2010-10-04 15:08:40 UTC (rev 6395)
+++ CalendarServer/branches/users/glyph/more-deferreds-7/txdav/common/datastore/util.py	2010-10-04 16:38:42 UTC (rev 6396)
@@ -15,6 +15,7 @@
 # limitations under the License.
 ##
 
+import os
 from twext.python.log import LoggingMixIn
 from twisted.application.service import Service
 from txdav.common.datastore.file import CommonDataStore as FileStore, TOPPATHS
@@ -32,7 +33,8 @@
 
 
     @classmethod
-    def wrapService(cls, path, service, connectionFactory, sqlAttachmentsPath):
+    def wrapService(cls, path, service, connectionFactory, sqlAttachmentsPath,
+        uid=None, gid=None):
         """
         Create an L{UpgradeToDatabaseService} if there are still file-based
         calendar or addressbook homes remaining in the given path.
@@ -64,19 +66,26 @@
                     FileStore(path, None, True, True),
                     SqlStore(connectionFactory, None, sqlAttachmentsPath,
                              True, True),
-                    service
+                    service,
+                    sqlAttachmentsPath=sqlAttachmentsPath,
+                    uid=uid,
+                    gid=gid,
                 )
                 return self
         return service
 
 
-    def __init__(self, fileStore, sqlStore, service):
+    def __init__(self, fileStore, sqlStore, service, sqlAttachmentsPath=None,
+        uid=None, gid=None):
         """
         Initialize the service.
         """
         self.wrappedService = service
         self.fileStore = fileStore
         self.sqlStore = sqlStore
+        self.sqlAttachmentsPath = sqlAttachmentsPath
+        self.uid = uid
+        self.gid = gid
 
 
     @inlineCallbacks
@@ -128,6 +137,16 @@
             homesPath = self.fileStore._path.child(homeType)
             if homesPath.isdir():
                 homesPath.remove()
+
+        # Set attachment directory ownership
+        if (self.sqlAttachmentsPath and
+            self.sqlAttachmentsPath.exists() and
+            (self.uid or self.gid)):
+            uid = self.uid or -1
+            gid = self.gid or -1
+            for fp in self.sqlAttachmentsPath.walk():
+                os.chown(fp.path, uid, gid)
+
         self.log_warn(
             "Filesystem upgrade complete, launching database service."
         )
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/calendarserver-changes/attachments/20101004/a121dae0/attachment-0001.html>


More information about the calendarserver-changes mailing list