[CalendarServer-changes] [9806] CalendarServer/branches/users/gaya

source_changes at macosforge.org source_changes at macosforge.org
Tue Sep 11 15:54:02 PDT 2012


Revision: 9806
          http://trac.macosforge.org/projects/calendarserver/changeset/9806
Author:   gaya at apple.com
Date:     2012-09-11 15:54:00 -0700 (Tue, 11 Sep 2012)
Log Message:
-----------
update from trunk

Modified Paths:
--------------
    CalendarServer/branches/users/gaya/directorybacker/calendarserver/push/applepush.py
    CalendarServer/branches/users/gaya/directorybacker/calendarserver/tools/backup_pg.py
    CalendarServer/branches/users/gaya/directorybacker/contrib/migration/calendarmigrator.py
    CalendarServer/branches/users/gaya/directorybacker/contrib/migration/test/test_migrator.py
    CalendarServer/branches/users/gaya/directorybacker/twext/web2/dav/test/test_xattrprops.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/datafilters/peruserdata.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/common.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/principal.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/test/test_calendar.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/util.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/xmlaccountsparser.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/ical.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/method/get.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/method/report_common.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/resource.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/icaldiff.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/imip.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/implicit.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/ischeduleservers.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/itip.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/processing.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/scheduler.py
    CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/test/test_icaldiff.py
    CalendarServer/branches/users/gaya/directorybacker/txdav/caldav/datastore/util.py
    CalendarServer/branches/users/gaya/directorybacker/txdav/xml/__init__.py
    CalendarServer/branches/users/gaya/directorybacker/txdav/xml/parser.py
    CalendarServer/branches/users/gaya/inviteclean/calendarserver/push/applepush.py
    CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/common.py
    CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/principal.py
    CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/test/test_calendar.py
    CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/util.py
    CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/scheduling/imip.py
    CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/scheduling/scheduler.py

Removed Paths:
-------------
    CalendarServer/branches/users/gaya/directorybacker/txdav/xml/draft_sync.py

Property Changed:
----------------
    CalendarServer/branches/users/gaya/directorybacker/
    CalendarServer/branches/users/gaya/inviteclean/


Property changes on: CalendarServer/branches/users/gaya/directorybacker
___________________________________________________________________
Modified: svn:mergeinfo
   - /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/generic-sqlstore:6167-6191
/CalendarServer/branches/new-store:5594-5934
/CalendarServer/branches/new-store-no-caldavfile:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
/CalendarServer/branches/users/cdaboo/batchupload-6699:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
/CalendarServer/branches/users/cdaboo/component-set-fixes:8130-8346
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/implicituidrace:8137-8141
/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/pods:7297-7377
/CalendarServer/branches/users/cdaboo/pycalendar:7085-7206
/CalendarServer/branches/users/cdaboo/pycard:7227-7237
/CalendarServer/branches/users/cdaboo/queued-attendee-refreshes:7740-8287
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
/CalendarServer/branches/users/cdaboo/timezones:7443-7699
/CalendarServer/branches/users/cdaboo/txn-debugging:8730-8743
/CalendarServer/branches/users/glyph/case-insensitive-uid:8772-8805
/CalendarServer/branches/users/glyph/conn-limit:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/dalify:6932-7023
/CalendarServer/branches/users/glyph/db-reconnect:6824-6876
/CalendarServer/branches/users/glyph/deploybuild:7563-7572
/CalendarServer/branches/users/glyph/disable-quota:7718-7727
/CalendarServer/branches/users/glyph/dont-start-postgres:6592-6614
/CalendarServer/branches/users/glyph/imip-and-admin-html:7866-7984
/CalendarServer/branches/users/glyph/ipv6-client:9054-9105
/CalendarServer/branches/users/glyph/linux-tests:6893-6900
/CalendarServer/branches/users/glyph/migrate-merge:8690-8713
/CalendarServer/branches/users/glyph/misc-portability-fixes:7365-7374
/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
/CalendarServer/branches/users/glyph/more-deferreds-7:6369-6445
/CalendarServer/branches/users/glyph/multiget-delete:8321-8330
/CalendarServer/branches/users/glyph/new-export:7444-7485
/CalendarServer/branches/users/glyph/oracle:7106-7155
/CalendarServer/branches/users/glyph/oracle-nulls:7340-7351
/CalendarServer/branches/users/glyph/other-html:8062-8091
/CalendarServer/branches/users/glyph/parallel-sim:8240-8251
/CalendarServer/branches/users/glyph/parallel-upgrade:8376-8400
/CalendarServer/branches/users/glyph/parallel-upgrade_to_1:8571-8583
/CalendarServer/branches/users/glyph/q:9560-9688
/CalendarServer/branches/users/glyph/quota:7604-7637
/CalendarServer/branches/users/glyph/sendfdport:5388-5424
/CalendarServer/branches/users/glyph/shared-pool-fixes:8436-8443
/CalendarServer/branches/users/glyph/shared-pool-take2:8155-8174
/CalendarServer/branches/users/glyph/sharedpool:6490-6550
/CalendarServer/branches/users/glyph/sharing-api:9192-9205
/CalendarServer/branches/users/glyph/skip-lonely-vtimezones:8524-8535
/CalendarServer/branches/users/glyph/sql-store:5929-6073
/CalendarServer/branches/users/glyph/subtransactions:7248-7258
/CalendarServer/branches/users/glyph/table-alias:8651-8664
/CalendarServer/branches/users/glyph/uidexport:7673-7676
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/glyph/uuid-normalize:9268-9296
/CalendarServer/branches/users/glyph/xattrs-from-files:7757-7769
/CalendarServer/branches/users/sagen/applepush:8126-8184
/CalendarServer/branches/users/sagen/inboxitems:7380-7381
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/branches/users/wsanchez/transations:5515-5593
/CalendarServer/trunk:9759-9784
   + /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/generic-sqlstore:6167-6191
/CalendarServer/branches/new-store:5594-5934
/CalendarServer/branches/new-store-no-caldavfile:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
/CalendarServer/branches/users/cdaboo/batchupload-6699:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
/CalendarServer/branches/users/cdaboo/component-set-fixes:8130-8346
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/implicituidrace:8137-8141
/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/pods:7297-7377
/CalendarServer/branches/users/cdaboo/pycalendar:7085-7206
/CalendarServer/branches/users/cdaboo/pycard:7227-7237
/CalendarServer/branches/users/cdaboo/queued-attendee-refreshes:7740-8287
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
/CalendarServer/branches/users/cdaboo/timezones:7443-7699
/CalendarServer/branches/users/cdaboo/txn-debugging:8730-8743
/CalendarServer/branches/users/glyph/case-insensitive-uid:8772-8805
/CalendarServer/branches/users/glyph/conn-limit:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/dalify:6932-7023
/CalendarServer/branches/users/glyph/db-reconnect:6824-6876
/CalendarServer/branches/users/glyph/deploybuild:7563-7572
/CalendarServer/branches/users/glyph/disable-quota:7718-7727
/CalendarServer/branches/users/glyph/dont-start-postgres:6592-6614
/CalendarServer/branches/users/glyph/imip-and-admin-html:7866-7984
/CalendarServer/branches/users/glyph/ipv6-client:9054-9105
/CalendarServer/branches/users/glyph/linux-tests:6893-6900
/CalendarServer/branches/users/glyph/migrate-merge:8690-8713
/CalendarServer/branches/users/glyph/misc-portability-fixes:7365-7374
/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
/CalendarServer/branches/users/glyph/more-deferreds-7:6369-6445
/CalendarServer/branches/users/glyph/multiget-delete:8321-8330
/CalendarServer/branches/users/glyph/new-export:7444-7485
/CalendarServer/branches/users/glyph/oracle:7106-7155
/CalendarServer/branches/users/glyph/oracle-nulls:7340-7351
/CalendarServer/branches/users/glyph/other-html:8062-8091
/CalendarServer/branches/users/glyph/parallel-sim:8240-8251
/CalendarServer/branches/users/glyph/parallel-upgrade:8376-8400
/CalendarServer/branches/users/glyph/parallel-upgrade_to_1:8571-8583
/CalendarServer/branches/users/glyph/q:9560-9688
/CalendarServer/branches/users/glyph/quota:7604-7637
/CalendarServer/branches/users/glyph/sendfdport:5388-5424
/CalendarServer/branches/users/glyph/shared-pool-fixes:8436-8443
/CalendarServer/branches/users/glyph/shared-pool-take2:8155-8174
/CalendarServer/branches/users/glyph/sharedpool:6490-6550
/CalendarServer/branches/users/glyph/sharing-api:9192-9205
/CalendarServer/branches/users/glyph/skip-lonely-vtimezones:8524-8535
/CalendarServer/branches/users/glyph/sql-store:5929-6073
/CalendarServer/branches/users/glyph/subtransactions:7248-7258
/CalendarServer/branches/users/glyph/table-alias:8651-8664
/CalendarServer/branches/users/glyph/uidexport:7673-7676
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/glyph/uuid-normalize:9268-9296
/CalendarServer/branches/users/glyph/xattrs-from-files:7757-7769
/CalendarServer/branches/users/sagen/applepush:8126-8184
/CalendarServer/branches/users/sagen/inboxitems:7380-7381
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/branches/users/wsanchez/transations:5515-5593
/CalendarServer/trunk:9759-9805

Modified: CalendarServer/branches/users/gaya/directorybacker/calendarserver/push/applepush.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/calendarserver/push/applepush.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/calendarserver/push/applepush.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -317,16 +317,16 @@
         @type status: C{int}
         """
         msg = self.STATUS_CODES.get(status, "Unknown status code")
-        self.log_warn("Received APN error %d on identifier %d: %s" % (status, identifier, msg))
+        self.log_info("Received APN error %d on identifier %d: %s" % (status, identifier, msg))
         if status in self.TOKEN_REMOVAL_CODES:
             token = self.history.extractIdentifier(identifier)
             if token is not None:
-                self.log_warn("Removing subscriptions for bad token: %s" %
+                self.log_debug("Removing subscriptions for bad token: %s" %
                     (token,))
                 txn = self.factory.store.newTransaction()
                 subscriptions = (yield txn.apnSubscriptionsByToken(token))
                 for key, modified, uid in subscriptions:
-                    self.log_warn("Removing subscription: %s %s" %
+                    self.log_debug("Removing subscription: %s %s" %
                         (token, key))
                     yield txn.removeAPNSubscription(token, key)
                 yield txn.commit()
@@ -384,13 +384,13 @@
         self.shuttingDown = False
 
     def clientConnectionMade(self):
-        self.log_warn("Connection to APN server made")
+        self.log_info("Connection to APN server made")
         self.service.clientConnectionMade()
         self.delay = 1.0
 
     def clientConnectionLost(self, connector, reason):
         if not self.shuttingDown:
-            self.log_warn("Connection to APN server lost: %s" % (reason,))
+            self.log_info("Connection to APN server lost: %s" % (reason,))
         ReconnectingClientFactory.clientConnectionLost(self, connector, reason)
 
     def clientConnectionFailed(self, connector, reason):
@@ -400,7 +400,7 @@
             reason)
 
     def retry(self, connector=None):
-        self.log_warn("Reconnecting to APN server")
+        self.log_info("Reconnecting to APN server")
         ReconnectingClientFactory.retry(self, connector)
 
     def stopTrying(self):
@@ -470,12 +470,12 @@
             self.scheduler = None
 
     def startService(self):
-        self.log_info("APNProviderService startService")
+        self.log_debug("APNProviderService startService")
         self.factory = APNProviderFactory(self, self.store)
         self.connect(self.factory)
 
     def stopService(self):
-        self.log_info("APNProviderService stopService")
+        self.log_debug("APNProviderService stopService")
         if self.factory is not None:
             self.factory.stopTrying()
         if self.scheduler is not None:

Modified: CalendarServer/branches/users/gaya/directorybacker/calendarserver/tools/backup_pg.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/calendarserver/tools/backup_pg.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/calendarserver/tools/backup_pg.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -21,8 +21,6 @@
 import subprocess
 import sys
 import tarfile
-from tempfile import mkstemp
-from shutil import rmtree
 
 from twistedcaldav.config import config
 from calendarserver.tools.util import loadConfig
@@ -162,38 +160,31 @@
 
     serverRoot = config.ServerRoot
     dataRoot = config.DataRoot
-    docRoot = config.DocumentRoot
-    configRoot = config.ConfigRoot
+    dumpPath = os.path.join(serverRoot, DUMPFILENAME)
 
     if command == "backup":
 
-        fd, tmpPath = mkstemp(suffix=".dbdump")
-
         try:
-            dumpData(tmpPath, verbose=verbose)
+            dumpData(dumpPath, verbose=verbose)
 
             if verbose:
                 print "Creating %s" % (filename,)
             tar = tarfile.open(filename, "w:gz")
+
             if verbose:
-                print "Adding %s" % (dataRoot,)
-            tar.add(dataRoot, "Data")
-            if verbose:
-                print "Adding %s" % (docRoot,)
-            tar.add(docRoot, "Documents")
-            if verbose:
-                print "Adding %s" % (configRoot,)
-            tar.add(configRoot, "Config")
-            if verbose:
-                print "Adding %s" % (tmpPath,)
-            tar.add(tmpPath, DUMPFILENAME)
+                print "Adding %s" % (serverRoot,)
+            tar.add(serverRoot)
+
+            if not dataRoot.startswith(serverRoot):
+                # DataRoot is not contained within ServerRoot (i.e, it's on
+                # another volume)
+                if verbose:
+                    print "Adding %s" % (dataRoot,)
+                tar.add(dataRoot)
+
             tar.close()
 
             if verbose:
-                print "Removing %s" % (tmpPath,)
-            os.remove(tmpPath)
-
-            if verbose:
                 print "Done"
         except BackupError, e:
             error("Failed to dump database; error: %s" % (e,))
@@ -202,32 +193,16 @@
 
         try:
             tar = tarfile.open(filename, "r:gz")
-            os.chdir(serverRoot)
 
-            if os.path.exists(dataRoot):
-                if verbose:
-                    print "Removing old DataRoot: %s" % (dataRoot,)
-                rmtree(dataRoot)
-
-            if os.path.exists(docRoot):
-                if verbose:
-                    print "Removing old DocumentRoot: %s" % (docRoot,)
-                rmtree(docRoot)
-
-            if os.path.exists(configRoot):
-                if verbose:
-                    print "Removing old ConfigRoot: %s" % (configRoot,)
-                rmtree(configRoot)
-
             if verbose:
                 print "Extracting from backup file: %s" % (filename,)
-            tar.extractall()
+            tar.extractall(path="/")
 
-            loadData(DUMPFILENAME, verbose=verbose)
+            loadData(dumpPath, verbose=verbose)
 
             if verbose:
-                print "Cleaning up database dump file: %s" % (DUMPFILENAME,)
-            os.remove(DUMPFILENAME)
+                print "Cleaning up database dump file: %s" % (dumpPath,)
+            os.remove(dumpPath)
 
         except BackupError, e:
             error("Failed to dump database; error: %s" % (e,))

Modified: CalendarServer/branches/users/gaya/directorybacker/contrib/migration/calendarmigrator.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/contrib/migration/calendarmigrator.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/contrib/migration/calendarmigrator.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -375,7 +375,6 @@
     newCalDAVDPlist["EnableCalDAV"] = enableCalDAV
     newCalDAVDPlist["EnableCardDAV"] = enableCardDAV
 
-
     log("Writing %s" % (newConfigFile,))
     writePlist(newCalDAVDPlist, newConfigFile)
 
@@ -483,6 +482,9 @@
     # New DSN value for server-specific Postgres
     combined["DSN"] = "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::"
 
+    # ConfigRoot is now always "Config"
+    combined["ConfigRoot"] = "Config"
+
     return adminChanges
 
 

Modified: CalendarServer/branches/users/gaya/directorybacker/contrib/migration/test/test_migrator.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/contrib/migration/test/test_migrator.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/contrib/migration/test/test_migrator.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -89,6 +89,7 @@
         expected = {
             "BindHTTPPorts": [8008, 8800],
             "BindSSLPorts": [8443, 8843],
+            "ConfigRoot" : "Config",
             "DSN" : "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::",
             "EnableSSL" : True,
             "HTTPPort": 8008,
@@ -127,6 +128,7 @@
         expected = {
             "BindHTTPPorts": [8008, 8800],
             "BindSSLPorts": [8443, 8843],
+            "ConfigRoot" : "Config",
             "DSN" : "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::",
             "EnableSSL" : False,
             "HTTPPort": 8008,
@@ -165,6 +167,7 @@
         expected = {
             "BindHTTPPorts": [8008, 8800],
             "BindSSLPorts": [8443, 8843],
+            "ConfigRoot" : "Config",
             "DSN" : "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::",
             "EnableSSL" : True,
             "HTTPPort": 8008,
@@ -203,6 +206,7 @@
         expected = {
             "BindHTTPPorts": [8008, 8800],
             "BindSSLPorts": [8443, 8843],
+            "ConfigRoot" : "Config",
             "DSN" : "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::",
             "EnableSSL" : True,
             "HTTPPort": 8008,
@@ -241,6 +245,7 @@
         expected = {
             "BindHTTPPorts": [1111, 2222, 4444, 5555, 7777, 8888],
             "BindSSLPorts": [3333, 6666, 9999, 11111],
+            "ConfigRoot" : "Config",
             "DSN" : "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::",
             "EnableSSL" : True,
             "HTTPPort": 8888,
@@ -276,6 +281,7 @@
         expected = {
             "BindHTTPPorts": [8008, 8800],
             "BindSSLPorts": [8443, 8843],
+            "ConfigRoot" : "Config",
             "DSN" : "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::",
             "EnableSSL" : False,
             "HTTPPort": 8008,
@@ -306,6 +312,7 @@
         expected = {
             "BindHTTPPorts": [8008, 8800],
             "BindSSLPorts": [8443, 8843],
+            "ConfigRoot" : "Config",
             "DSN" : "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::",
             "EnableSSL" : True,
             "HTTPPort": 8008,
@@ -327,6 +334,7 @@
         expected = {
             "BindHTTPPorts": [8008, 8800],
             "BindSSLPorts": [8443, 8843],
+            "ConfigRoot" : "Config",
             "DSN" : "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::",
             "EnableSSL" : False,
             "HTTPPort": 8008,
@@ -374,6 +382,7 @@
             },
             "BindHTTPPorts": [8008, 8800],
             "BindSSLPorts": [8443, 8843],
+            "ConfigRoot" : "Config",
             "DSN" : "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::",
             "EnableSSL" : False,
             "HTTPPort": 8008,
@@ -413,6 +422,7 @@
             },
             "BindHTTPPorts": [8008, 8800],
             "BindSSLPorts": [8443, 8843],
+            "ConfigRoot" : "Config",
             "DSN" : "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::",
             "EnableSSL" : False,
             "HTTPPort": 8008,
@@ -465,6 +475,7 @@
             },
             "BindHTTPPorts": [8008, 8800],
             "BindSSLPorts": [8443, 8843],
+            "ConfigRoot" : "Config",
             "DSN" : "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::",
             "EnableSSL" : False,
             "HTTPPort": 8008,
@@ -506,6 +517,7 @@
             },
             "BindHTTPPorts": [8008, 8800],
             "BindSSLPorts": [8443, 8843],
+            "ConfigRoot" : "Config",
             "DSN" : "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::",
             "EnableSSL" : False,
             "HTTPPort": 8008,
@@ -547,6 +559,7 @@
             },
             "BindHTTPPorts": [8008, 8800],
             "BindSSLPorts": [8443, 8843],
+            "ConfigRoot" : "Config",
             "DSN" : "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::",
             "EnableSSL" : False,
             "HTTPPort": 8008,
@@ -582,6 +595,7 @@
             },
             "BindHTTPPorts": [8008, 8800],
             "BindSSLPorts": [8443, 8843],
+            "ConfigRoot" : "Config",
             "DSN" : "/Library/Server/PostgreSQL For Server Services/Socket:caldav:caldav:::",
             "EnableSSL" : False,
             "HTTPPort": 8008,

Modified: CalendarServer/branches/users/gaya/directorybacker/twext/web2/dav/test/test_xattrprops.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twext/web2/dav/test/test_xattrprops.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twext/web2/dav/test/test_xattrprops.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -201,7 +201,7 @@
         self._setValue(document, originalValue)
         self._checkValue(document)
         self.assertEquals(
-            decompress(self._getValue(document)), originalValue)
+            decompress(self._getValue(document)), document.root_element.toxml(pretty=False))
 
 
     def test_getUpgradeCompressedPickle(self):
@@ -214,7 +214,7 @@
         self._setValue(document, compress(dumps(document)))
         self._checkValue(document)
         self.assertEquals(
-            decompress(self._getValue(document)), document.toxml())
+            decompress(self._getValue(document)), document.root_element.toxml(pretty=False))
 
 
     def test_getInvalid(self):
@@ -243,7 +243,7 @@
         document = self._makeValue()
         self.propertyStore.set(document.root_element)
         self.assertEquals(
-            decompress(self._getValue(document)), document.toxml())
+            decompress(self._getValue(document)), document.root_element.toxml(pretty=False))
 
 
     def test_delete(self):
@@ -396,7 +396,7 @@
             document = self._makeValue(uid)
             self.propertyStore.set(document.root_element, uid=uid)
             self.assertEquals(
-                decompress(self._getValue(document, uid)), document.toxml())
+                decompress(self._getValue(document, uid)), document.root_element.toxml(pretty=False))
 
     def test_delete_uids(self):
         """
@@ -415,7 +415,7 @@
                     continue
                 document = self._makeValue(uid)
                 self.assertEquals(
-                    decompress(self._getValue(document, uid)), document.toxml())
+                    decompress(self._getValue(document, uid)), document.root_element.toxml(pretty=False))
         
     def test_contains_uids(self):
         """

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/datafilters/peruserdata.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/datafilters/peruserdata.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/datafilters/peruserdata.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -65,6 +65,7 @@
 
     PERUSER_PROPERTIES    = ("TRANSP",)
     PERUSER_SUBCOMPONENTS = ("VALARM",)
+    IGNORE_X_PROPERTIES   = ("X-CALENDARSERVER-HIDDEN-INSTANCE",)
 
     def __init__(self, uid):
         """
@@ -239,7 +240,7 @@
 
             # Transfer per-user properties from main component to per-instance component
             for property in tuple(component.properties()):
-                if property.name() in PerUserDataFilter.PERUSER_PROPERTIES or property.name().startswith("X-"):
+                if property.name() in PerUserDataFilter.PERUSER_PROPERTIES or property.name().startswith("X-") and property.name() not in PerUserDataFilter.IGNORE_X_PROPERTIES:
                     if self.uid:
                         perinstance_component.addProperty(property)
                     component.removeProperty(property)

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/common.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/common.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/common.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -19,7 +19,7 @@
 from twext.web2.http import HTTPError
 from twext.web2 import responsecode
 from twext.web2.dav.util import joinURL
-from twistedcaldav.directory.util import transactionFromRequest
+from twistedcaldav.directory.util import transactionFromRequest, NotFoundResource
 from twistedcaldav.directory.resource import DirectoryReverseProxyResource
 
 from twext.python.log import Logger
@@ -147,7 +147,9 @@
 
         record = self.directory.recordWithShortName(self.recordType, name)
         if record is None:
-            returnValue((None, []))
+            returnValue(
+                (NotFoundResource(principalCollections=self._parent.principalCollections()), [])
+            )
 
         child = yield self._parent.homeForDirectoryRecord(record, request)
         returnValue((child, segments[1:]))

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/principal.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/principal.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/principal.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -36,6 +36,7 @@
 from twisted.internet.defer import inlineCallbacks, returnValue
 from twisted.internet.defer import succeed
 from twisted.web.template import XMLFile, Element, renderer, tags
+from twistedcaldav.directory.util import NotFoundResource
 
 from twext.web2.auth.digest import DigestedCredentials
 from twext.web2 import responsecode
@@ -46,6 +47,7 @@
 
 from twext.python.log import Logger
 
+
 try:
     from twistedcaldav.authkerb import NegotiateCredentials
     NegotiateCredentials # sigh, pyflakes
@@ -130,6 +132,7 @@
             (origCUAddr,))
 
 
+
 class DirectoryProvisioningResource (
     PermissionsMixIn,
     CalendarPrincipalCollectionResource,
@@ -153,7 +156,7 @@
         child = self.getChild(segments[0])
         if child is not None:
             return (child, segments[1:])
-        return (None, ())
+        return (NotFoundResource(principalCollections=self.principalCollections()),())
 
     def deadProperties(self):
         if not hasattr(self, "_dead_properties"):

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/test/test_calendar.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/test/test_calendar.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/test/test_calendar.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -21,7 +21,7 @@
 
 from twistedcaldav.test.util import TestCase
 from twext.web2.test.test_server import SimpleRequest
-from twistedcaldav.directory.util import transactionFromRequest
+from twistedcaldav.directory.util import transactionFromRequest, NotFoundResource
 
 class ProvisionedCalendars (TestCase):
     """
@@ -55,10 +55,14 @@
 
 
     def test_NonExistentCalendarHome(self):
+        """
+        Requests for missing homes and principals should return
+        NotFoundResources so that we have the opportunity to
+        turn 404s into 401s to protect against user-existence attacks.
+        """
 
         def _response(resource):
-            if resource is not None:
-                self.fail("Incorrect response to GET on non-existent calendar home.")
+            self.assertTrue(isinstance(resource, NotFoundResource))
 
         request = self.oneRequest("/calendars/users/12345/")
         d = request.locateResource(request.uri)

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/util.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/util.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/util.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -22,12 +22,21 @@
 __all__ = [
     "normalizeUUID",
     "uuidFromName",
+    "NotFoundResource",
 ]
 
 from twext.enterprise.ienterprise import AlreadyFinishedError
-
+from twext.python.log import Logger
+from twext.web2 import responsecode
+from twext.web2.auth.wrapper import UnauthorizedResponse
+from twext.web2.dav.resource import DAVResource
+from twext.web2.http import StatusResponse
+from twisted.internet.defer import inlineCallbacks, returnValue
+from txdav.xml import element as davxml
 from uuid import UUID, uuid5
 
+log = Logger()
+
 def uuidFromName(namespace, name):
     """
     Generate a version 5 (SHA-1) UUID from a namespace UUID and a name.
@@ -108,3 +117,31 @@
         yield set(data[:size])
         del data[:size]
 
+
+class NotFoundResource(DAVResource):
+    """
+    In order to prevent unauthenticated discovery of existing users via 401/404
+    response codes, this resource can be returned from locateChild, and it will
+    perform an authentication; if the user is unauthenticated, 404 responses are
+    turned into 401s.
+    """
+
+    @inlineCallbacks
+    def renderHTTP(self, request):
+
+        try:
+            authnUser, authzUser = yield self.authenticate(request)
+        except Exception:
+            authzUser = davxml.Principal(davxml.Unauthenticated())
+
+        # Turn 404 into 401
+        if authzUser == davxml.Principal(davxml.Unauthenticated()):
+            response = (yield UnauthorizedResponse.makeResponse(
+                request.credentialFactories,
+                request.remoteAddr
+            ))
+            returnValue(response)
+        else:
+            response = StatusResponse(responsecode.NOT_FOUND, "Resource not found")
+            returnValue(response)
+

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/xmlaccountsparser.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/xmlaccountsparser.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/directory/xmlaccountsparser.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -23,14 +23,13 @@
     "XMLAccountsParser",
 ]
 
-import xml.dom.minidom
-
 from twext.python.filepath import CachingFilePath as FilePath
 
 from twext.python.log import Logger
 
 from twistedcaldav.directory.directory import DirectoryService
 from twistedcaldav.directory.util import normalizeUUID
+from twistedcaldav.xmlutil import readXML
 
 import re
 import hashlib
@@ -88,15 +87,10 @@
             self.items[recordType] = {}
 
         # Read in XML
-        fd = open(self.xmlFile.path, "r")
-        doc = xml.dom.minidom.parse(fd)
-        fd.close()
-
-        # Verify that top-level element is correct
-        accounts_node = doc._get_documentElement()
-        if accounts_node._get_localName() != ELEMENT_ACCOUNTS:
-            log.error("Ignoring file %r because it is not a repository builder file" % (self.xmlFile,))
-            return
+        try:
+            _ignore_tree, accounts_node = readXML(self.xmlFile.path, ELEMENT_ACCOUNTS)
+        except ValueError, e:
+            log.error("XML parse error for '%s' because: %s" % (self.xmlFile, e,), raiseException=RuntimeError)
         self._parseXML(accounts_node)
 
     def _parseXML(self, node):
@@ -104,8 +98,7 @@
         Parse the XML root node from the accounts configuration document.
         @param node: the L{Node} to parse.
         """
-        if node.hasAttribute(ATTRIBUTE_REALM):
-            self.realm = node.getAttribute(ATTRIBUTE_REALM).encode("utf-8")
+        self.realm = node.get(ATTRIBUTE_REALM, "").encode("utf-8")
 
         def updateMembership(group):
             # Update group membership
@@ -114,20 +107,13 @@
                 if item is not None:
                     item.groups.add(group.shortNames[0])
 
-        for child in node._get_childNodes():
-            child_name = child._get_localName()
-            if child_name is None:
-                continue
-
+        for child in node.getchildren():
             try:
-                recordType = RECORD_TYPES[child_name]
+                recordType = RECORD_TYPES[child.tag]
             except KeyError:
-                raise RuntimeError("Unknown account type: %s" % (child_name,))
+                raise RuntimeError("Unknown account type: %s" % (child.tag,))
 
-            if child.hasAttribute(ATTRIBUTE_REPEAT):
-                repeat = int(child.getAttribute(ATTRIBUTE_REPEAT))
-            else:
-                repeat = 0
+            repeat = int(child.get(ATTRIBUTE_REPEAT, 0))
 
             principal = XMLAccountRecord(recordType)
             principal.parseXML(child)
@@ -249,58 +235,39 @@
         return result
 
     def parseXML(self, node):
-        for child in node._get_childNodes():
-            child_name = child._get_localName()
-            if child_name is None:
-                continue
-            elif child_name == ELEMENT_SHORTNAME:
-                if child.firstChild is not None:
-                    self.shortNames.append(child.firstChild.data.encode("utf-8"))
-            elif child_name == ELEMENT_GUID:
-                if child.firstChild is not None:
-                    self.guid = normalizeUUID(
-                        child.firstChild.data.encode("utf-8")
-                    )
-                    if len(self.guid) < 4:
-                        self.guid += "?" * (4 - len(self.guid))
-            elif child_name == ELEMENT_PASSWORD:
-                if child.firstChild is not None:
-                    self.password = child.firstChild.data.encode("utf-8")
-            elif child_name == ELEMENT_NAME:
-                if child.firstChild is not None:
-                    self.fullName = child.firstChild.data.encode("utf-8")
-            elif child_name == ELEMENT_FIRST_NAME:
-                if child.firstChild is not None:
-                    self.firstName = child.firstChild.data.encode("utf-8")
-            elif child_name == ELEMENT_LAST_NAME:
-                if child.firstChild is not None:
-                    self.lastName = child.firstChild.data.encode("utf-8")
-            elif child_name == ELEMENT_EMAIL_ADDRESS:
-                if child.firstChild is not None:
-                    self.emailAddresses.add(child.firstChild.data.encode("utf-8").lower())
-            elif child_name == ELEMENT_MEMBERS:
+        for child in node.getchildren():
+            if child.tag == ELEMENT_SHORTNAME:
+                self.shortNames.append(child.text.encode("utf-8"))
+            elif child.tag == ELEMENT_GUID:
+                self.guid = normalizeUUID(child.text.encode("utf-8"))
+                if len(self.guid) < 4:
+                    self.guid += "?" * (4 - len(self.guid))
+            elif child.tag == ELEMENT_PASSWORD:
+                self.password = child.text.encode("utf-8")
+            elif child.tag == ELEMENT_NAME:
+                self.fullName = child.text.encode("utf-8")
+            elif child.tag == ELEMENT_FIRST_NAME:
+                self.firstName = child.text.encode("utf-8")
+            elif child.tag == ELEMENT_LAST_NAME:
+                self.lastName = child.text.encode("utf-8")
+            elif child.tag == ELEMENT_EMAIL_ADDRESS:
+                self.emailAddresses.add(child.text.encode("utf-8").lower())
+            elif child.tag == ELEMENT_MEMBERS:
                 self._parseMembers(child, self.members)
-            elif child_name == ELEMENT_EXTRAS:
+            elif child.tag == ELEMENT_EXTRAS:
                 self._parseExtras(child, self.extras)
             else:
-                raise RuntimeError("Unknown account attribute: %s" % (child_name,))
+                raise RuntimeError("Unknown account attribute: %s" % (child.tag,))
 
         if not self.shortNames:
             self.shortNames.append(self.guid)
 
     def _parseMembers(self, node, addto):
-        for child in node._get_childNodes():
-            if child._get_localName() == ELEMENT_MEMBER:
-                if child.hasAttribute(ATTRIBUTE_RECORDTYPE):
-                    recordType = child.getAttribute(ATTRIBUTE_RECORDTYPE).encode("utf-8")
-                else:
-                    recordType = DirectoryService.recordType_users
-                if child.firstChild is not None:
-                    addto.add((recordType, child.firstChild.data.encode("utf-8")))
+        for child in node.getchildren():
+            if child.tag == ELEMENT_MEMBER:
+                recordType = child.get(ATTRIBUTE_RECORDTYPE, DirectoryService.recordType_users)
+                addto.add((recordType, child.text.encode("utf-8")))
 
     def _parseExtras(self, node, addto):
-        for child in node._get_childNodes():
-            key = child._get_localName()
-            if key:
-                value = child.firstChild.data.encode("utf-8")
-                addto[key.encode("utf-8")] = value
+        for child in node.getchildren():
+            addto[child.tag] = child.text.encode("utf-8")

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/ical.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/ical.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/ical.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -325,6 +325,9 @@
     }
     extraRestrictedProperties = ("SUMMARY", "LOCATION",)
 
+    # Hidden instance.
+    HIDDEN_INSTANCE_PROPERTY = "X-CALENDARSERVER-HIDDEN-INSTANCE"
+
     @classmethod
     def allFromString(clazz, string):
         """
@@ -762,7 +765,17 @@
                 return (range == "THISANDFUTURE")
 
         return False
-            
+    
+    def getExdates(self):
+        """
+        Get the set of all EXDATEs in this (master) component.
+        """
+        exdates = set()
+        for property in self.properties("EXDATE"):
+            for exdate in property.value():
+                exdates.add(exdate.getValue())
+        return exdates
+
     def getTriggerDetails(self):
         """
         Return the trigger information for the specified alarm component.

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/method/get.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/method/get.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/method/get.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -32,6 +32,7 @@
 
 from twistedcaldav.config import config
 from twistedcaldav.customxml import calendarserver_namespace
+from twistedcaldav.datafilters.hiddeninstance import HiddenInstanceFilter
 from twistedcaldav.datafilters.privateevents import PrivateEventFilter
 from twistedcaldav.resource import isPseudoCalendarCollectionResource,\
     CalDAVResource
@@ -79,6 +80,9 @@
     
                 caldata = (yield self.iCalendarForUser(request))
     
+                # Filter any attendee hidden instances        
+                caldata = HiddenInstanceFilter().filter(caldata)
+
                 if self.accessMode:
             
                     # Non DAV:owner's have limited access to the data
@@ -86,7 +90,7 @@
                     
                     # Now "filter" the resource calendar data
                     caldata = PrivateEventFilter(self.accessMode, isowner).filter(caldata)
-        
+
                 response = Response()
                 response.stream = MemoryStream(caldata.getTextWithTimezones(includeTimezones=not config.EnableTimezonesByReference))
                 response.headers.setHeader("content-type", MimeType.fromString("text/calendar; charset=utf-8"))

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/method/report_common.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/method/report_common.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/method/report_common.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -56,6 +56,7 @@
 from twistedcaldav.carddavxml import AddressData
 from twistedcaldav.config import config
 from twistedcaldav.datafilters.calendardata import CalendarDataFilter
+from twistedcaldav.datafilters.hiddeninstance import HiddenInstanceFilter
 from twistedcaldav.datafilters.privateevents import PrivateEventFilter
 from twistedcaldav.datafilters.addressdata import AddressDataFilter
 from twistedcaldav.dateops import clipPeriod, normalizePeriodList, timeRangesOverlap,\
@@ -344,7 +345,8 @@
             # Handle private events access restrictions
             if calendar is None:
                 calendar = (yield resource.iCalendarForUser(request))
-            filtered = PrivateEventFilter(resource.accessMode, isowner).filter(calendar)
+            filtered = HiddenInstanceFilter().filter(calendar)
+            filtered = PrivateEventFilter(resource.accessMode, isowner).filter(filtered)
             filtered = CalendarDataFilter(property, timezone).filter(filtered)
             propvalue = CalendarData().fromCalendar(filtered)
             properties_by_status[responsecode.OK].append(propvalue)

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/resource.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/resource.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/resource.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -65,6 +65,7 @@
 from twistedcaldav.carddavxml import carddav_namespace
 from twistedcaldav.config import config
 from twistedcaldav.customxml import calendarserver_namespace
+from twistedcaldav.datafilters.hiddeninstance import HiddenInstanceFilter
 from twistedcaldav.datafilters.peruserdata import PerUserDataFilter
 from twistedcaldav.datafilters.privateevents import PrivateEventFilter
 from twistedcaldav.directory.internal import InternalDirectoryRecord
@@ -1575,11 +1576,11 @@
     def iCalendarFiltered(self, isowner, accessUID=None):
 
         # Now "filter" the resource calendar data
-        caldata = PrivateEventFilter(self.accessMode, isowner).filter(
-            (yield self.iCalendar())
-        )
+        caldata = (yield self.iCalendar())
         if accessUID:
             caldata = PerUserDataFilter(accessUID).filter(caldata)
+        caldata = HiddenInstanceFilter().filter(caldata)
+        caldata = PrivateEventFilter(self.accessMode, isowner).filter(caldata)
         returnValue(caldata)
 
 

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/icaldiff.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/icaldiff.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/icaldiff.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -323,9 +323,17 @@
                 # Attendee may decline by EXDATE'ing an instance - we need to handle that
                 if exdatesnew is None or rid in exdatesnew:
                     # Mark Attendee as DECLINED in the server instance
-                    if self._attendeeDecline(returnCalendar.overriddenComponent(rid)):
+                    overridden = returnCalendar.overriddenComponent(rid)
+                    if self._attendeeDecline(overridden):
                         changeCausesReply = True
                         changedRids.append(rid.getText() if rid else "")
+                        
+                    # When a master component is present we keep the missing override in place but mark it as hidden.
+                    # When no master is present we remove the override,
+                    if exdatesnew is not None:
+                        overridden.replaceProperty(Property(Component.HIDDEN_INSTANCE_PROPERTY, "T"))
+                    else:
+                        returnCalendar.removeComponent(overridden)
                 else:
                     # We used to generate a 403 here - but instead we now ignore this error and let the server data
                     # override the client
@@ -411,10 +419,15 @@
             if not overridden:
                 overridden = returnCalendar.deriveInstance(decline)
                 if overridden:
-                    returnCalendar.addComponent(overridden)
                     if self._attendeeDecline(overridden):
                         changeCausesReply = True
                         changedRids.append(decline.getText() if decline else "")
+                        
+                    # When a master component is present we keep the missing override in place but mark it as hidden.
+                    # When no master is present we remove the override,
+                    if exdatesnew is not None:
+                        overridden.replaceProperty(Property(Component.HIDDEN_INSTANCE_PROPERTY, "T"))
+                        returnCalendar.addComponent(overridden)
                 else:
                     self._logDiffError("attendeeMerge: Unable to override an instance to mark as DECLINED: %s" % (decline,))
                     return False, False, (), None

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/imip.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/imip.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/imip.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -44,6 +44,7 @@
 
 log = Logger()
 
+
 class ScheduleViaIMip(DeliveryService):
     
     @classmethod
@@ -52,12 +53,24 @@
 
     @inlineCallbacks
     def generateSchedulingResponses(self):
+        def failForRecipient(recipient):
+            err = HTTPError(ErrorResponse(
+                responsecode.FORBIDDEN,
+                (caldav_namespace, "recipient-failed"),
+                "iMIP request failed",
+            ))
+            self.responses.add(
+                recipient.cuaddr,
+                Failure(exc_value=err),
+                reqstatus=iTIPRequestStatus.SERVICE_UNAVAILABLE,
+                suppressErrorLog=True
+            )
         
         # Generate an HTTP client request
         try:
             # We do not do freebusy requests via iMIP
             if self.freebusy:
-                raise ValueError("iMIP VFREEBUSY REQUESTs not supported.")
+                raise ValueError("iMIP VFREEBUSY requests not supported.")
 
             method = self.scheduler.calendar.propertyValue("METHOD") 
             if method not in (
@@ -97,17 +110,8 @@
         
                 except Exception, e:
                     # Generated failed response for this recipient
-                    log.err("Could not do server-to-imip request : %s %s" % (self, e))
-                    err = HTTPError(ErrorResponse(
-                        responsecode.FORBIDDEN,
-                        (caldav_namespace, "recipient-failed"),
-                        "iMIP request failed",
-                    ))
-                    self.responses.add(
-                        recipient.cuaddr,
-                        Failure(exc_value=err),
-                        reqstatus=iTIPRequestStatus.SERVICE_UNAVAILABLE
-                    )
+                    log.debug("iMIP request %s failed for recipient %s: %s" % (self, recipient, e))
+                    failForRecipient(recipient)
                 
                 else:
                     self.responses.add(
@@ -118,18 +122,9 @@
 
         except Exception, e:
             # Generated failed responses for each recipient
-            log.err("Could not do server-to-imip request : %s %s" % (self, e))
+            log.debug("iMIP request %s failed: %s" % (self, e))
             for recipient in self.recipients:
-                err = HTTPError(ErrorResponse(
-                    responsecode.FORBIDDEN,
-                    (caldav_namespace, "recipient-failed"),
-                    "iMIP request failed",
-                ))
-                self.responses.add(
-                    recipient.cuaddr,
-                    Failure(exc_value=err),
-                    reqstatus=iTIPRequestStatus.SERVICE_UNAVAILABLE
-                )
+                failForRecipient(recipient)
 
     def postToGateway(self, fromAddr, toAddr, caldata, reactor=None):
         if reactor is None:

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/implicit.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/implicit.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/implicit.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -958,8 +958,14 @@
                 log.debug("Implicit - attendee '%s' is removing cancelled UID: '%s'" % (self.attendee, self.uid))
                 # Nothing else to do
             elif doScheduling:
-                log.debug("Implicit - attendee '%s' is cancelling UID: '%s'" % (self.attendee, self.uid))
-                yield self.scheduleCancelWithOrganizer()
+                # If attendee is already marked as declined in all components - nothing to do
+                attendees = self.calendar.getAttendeeProperties((self.attendee,))
+                if all([attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") == "DECLINED" for attendee in attendees]):
+                    log.debug("Implicit - attendee '%s' is removing fully declined UID: '%s'" % (self.attendee, self.uid))
+                    # Nothing else to do
+                else:
+                    log.debug("Implicit - attendee '%s' is cancelling UID: '%s'" % (self.attendee, self.uid))
+                    yield self.scheduleCancelWithOrganizer()
             else:
                 log.debug("Implicit - attendee '%s' is removing UID without server scheduling: '%s'" % (self.attendee, self.uid))
                 # Nothing else to do

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/ischeduleservers.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/ischeduleservers.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/ischeduleservers.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -20,9 +20,8 @@
 
 from twistedcaldav.config import config, fullServerPath
 from twistedcaldav.scheduling.delivery import DeliveryService
+from twistedcaldav.xmlutil import readXML
 
-import xml.dom.minidom
-
 """
 XML based iSchedule configuration file handling.
 """
@@ -100,15 +99,11 @@
         self.servers = []
         
         # Read in XML
-        fd = open(xmlFile.path, "r")
-        doc = xml.dom.minidom.parse(fd)
-        fd.close()
+        try:
+            _ignore_tree, servers_node = readXML(xmlFile.path, ELEMENT_SERVERS)
+        except ValueError:
+            log.error("Ignoring file %r because it is not a server-to-server config file" % (xmlFile,))
 
-        # Verify that top-level element is correct
-        servers_node = doc._get_documentElement()
-        if servers_node._get_localName() != ELEMENT_SERVERS:
-            log.error("Ignoring file %r because it is not a server-to-server config file" % (self.xmlFile,))
-            return
         self._parseXML(servers_node)
         
     def _parseXML(self, node):
@@ -116,12 +111,8 @@
         Parse the XML root node from the server-to-server configuration document.
         @param node: the L{Node} to parse.
         """
-
-        for child in node._get_childNodes():
-            child_name = child._get_localName()
-            if child_name is None:
-                continue
-            elif child_name == ELEMENT_SERVER:
+        for child in node.getchildren():
+            if child.tag == ELEMENT_SERVER:
                 self.servers.append(IScheduleServerRecord())
                 self.servers[-1].parseXML(child)
                 
@@ -147,49 +138,39 @@
             self._parseDetails()
 
     def parseXML(self, node):
-        for child in node._get_childNodes():
-            child_name = child._get_localName()
-            if child_name is None:
-                continue
-            elif child_name == ELEMENT_URI:
-                if child.firstChild is not None:
-                    self.uri = child.firstChild.data.encode("utf-8")
-            elif child_name == ELEMENT_AUTHENTICATION:
+        for child in node.getchildren():
+            if child.tag == ELEMENT_URI:
+                self.uri = child.text
+            elif child.tag == ELEMENT_AUTHENTICATION:
                 self._parseAuthentication(child)
-            elif child_name == ELEMENT_ALLOW_REQUESTS_FROM:
+            elif child.tag == ELEMENT_ALLOW_REQUESTS_FROM:
                 self.allow_from = True
-            elif child_name == ELEMENT_ALLOW_REQUESTS_TO:
+            elif child.tag == ELEMENT_ALLOW_REQUESTS_TO:
                 self.allow_to = True
-            elif child_name == ELEMENT_DOMAINS:
+            elif child.tag == ELEMENT_DOMAINS:
                 self._parseList(child, ELEMENT_DOMAIN, self.domains)
-            elif child_name == ELEMENT_CLIENT_HOSTS:
+            elif child.tag == ELEMENT_CLIENT_HOSTS:
                 self._parseList(child, ELEMENT_HOST, self.client_hosts)
             else:
-                raise RuntimeError("[%s] Unknown attribute: %s" % (self.__class__, child_name,))
+                raise RuntimeError("[%s] Unknown attribute: %s" % (self.__class__, child.tag,))
         
         self._parseDetails()
 
     def _parseList(self, node, element_name, appendto):
-        for child in node._get_childNodes():
-            if child._get_localName() == element_name:
-                if child.firstChild is not None:
-                    appendto.append(child.firstChild.data.encode("utf-8"))
+        for child in node.getchildren():
+            if child.tag == element_name:
+                appendto.append(child.text)
 
     def _parseAuthentication(self, node):
-        if node.hasAttribute(ATTRIBUTE_TYPE):
-            atype = node.getAttribute(ATTRIBUTE_TYPE).encode("utf-8")
-            if atype != ATTRIBUTE_BASICAUTH:
-                return
-        else:
+        atype = node.getAttribute(ATTRIBUTE_TYPE, "")
+        if atype != ATTRIBUTE_BASICAUTH:
             return
 
-        for child in node._get_childNodes():
-            if child._get_localName() == ELEMENT_USER:
-                if child.firstChild is not None:
-                    user = child.firstChild.data.encode("utf-8")
-            elif child._get_localName() == ELEMENT_PASSWORD:
-                if child.firstChild is not None:
-                    password = child.firstChild.data.encode("utf-8")
+        for child in node.getchildren():
+            if child.tag == ELEMENT_USER:
+                user = child.text
+            elif child.tag == ELEMENT_PASSWORD:
+                password = child.text
         
         self.authentication = ("basic", user, password,)
 

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/itip.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/itip.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/itip.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -47,7 +47,7 @@
 class iTipProcessing(object):
 
     @staticmethod
-    def processNewRequest(itip_message, recipient=None):
+    def processNewRequest(itip_message, recipient=None, creating=False):
         """
         Process a METHOD=REQUEST for a brand new calendar object.
         
@@ -65,6 +65,20 @@
 
         if recipient:
             iTipProcessing.addTranspForNeedsAction(calendar.subcomponents(), recipient)
+                
+            # Check for incoming DECLINED
+            if creating:
+                master = calendar.masterComponent()
+                for component in tuple(calendar.subcomponents()):
+                    if component in ignoredComponents or component is master:
+                        continue
+                    attendee = component.getAttendeeProperty((recipient,))
+                    if attendee and attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") == "DECLINED":
+                        # Mark as hidden if we have a master, otherwise remove
+                        if master is not None:
+                            component.addProperty(Property(Component.HIDDEN_INSTANCE_PROPERTY, "T"))
+                        else:
+                            calendar.removeComponent(component)
 
         return calendar
         
@@ -131,12 +145,12 @@
                 if organizer:
                     organizer.setParameter("SCHEDULE-STATUS", organizer_schedule_status)
                 
-            # Now try to match recurrences
-            for component in new_calendar.subcomponents():
+            # Now try to match recurrences in the new calendar
+            for component in tuple(new_calendar.subcomponents()):
                 if component.name() != "VTIMEZONE" and component.getRecurrenceIDUTC() is not None:
-                    iTipProcessing.transferItems(calendar, master_valarms, private_comments, transps, completeds, organizer_schedule_status, component)
+                    iTipProcessing.transferItems(calendar, master_valarms, private_comments, transps, completeds, organizer_schedule_status, component, recipient)
             
-            # Now try to match recurrences
+            # Now try to match recurrences from the old calendar
             for component in calendar.subcomponents():
                 if component.name() != "VTIMEZONE" and component.getRecurrenceIDUTC() is not None:
                     rid = component.getRecurrenceIDUTC()
@@ -145,7 +159,7 @@
                         new_component = new_calendar.deriveInstance(rid, allowCancelled=allowCancelled)
                         if new_component:
                             new_calendar.addComponent(new_component)
-                            iTipProcessing.transferItems(calendar, master_valarms, private_comments, transps, completeds, organizer_schedule_status, new_component)
+                            iTipProcessing.transferItems(calendar, master_valarms, private_comments, transps, completeds, organizer_schedule_status, new_component, recipient)
             
             # Replace the entire object
             return new_calendar, rids
@@ -162,10 +176,11 @@
                         calendar.addComponent(component)
                 else:
                     component = component.duplicate()
-                    iTipProcessing.transferItems(calendar, master_valarms, private_comments, transps, completeds, organizer_schedule_status, component, remove_matched=True)
-                    calendar.addComponent(component)
-                    if recipient:
-                        iTipProcessing.addTranspForNeedsAction((component,), recipient)
+                    missingDeclined = iTipProcessing.transferItems(calendar, master_valarms, private_comments, transps, completeds, organizer_schedule_status, component, recipient, remove_matched=True)
+                    if not missingDeclined:
+                        calendar.addComponent(component)
+                        if recipient:
+                            iTipProcessing.addTranspForNeedsAction((component,), recipient)
 
             # Write back the modified object
             return calendar, rids
@@ -468,11 +483,13 @@
         return attendee.value(), partstat_changed, private_comment_changed
 
     @staticmethod
-    def transferItems(from_calendar, master_valarms, private_comments, transps, completeds, organizer_schedule_status, to_component, remove_matched=False):
+    def transferItems(from_calendar, master_valarms, private_comments, transps, completeds, organizer_schedule_status, to_component, recipient, remove_matched=False):
         """
         Transfer properties from a calendar to a component by first trying to match the component in the original calendar and
         use the properties from that, or use the values provided as arguments (which have been derived from the original calendar's
         master component).
+        
+        @return: C{True} if an EXDATE match occurred requiring the incoming component to be removed.
         """
 
         rid = to_component.getRecurrenceIDUTC()
@@ -497,7 +514,19 @@
             if remove_matched:
                 from_calendar.removeComponent(matched)
                 
+            # Check for incoming DECLINED
+            attendee = to_component.getAttendeeProperty((recipient,))
+            if attendee and attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") == "DECLINED":
+                # If existing item has HIDDEN property copy that over
+                if matched.hasProperty(Component.HIDDEN_INSTANCE_PROPERTY):
+                    to_component.addProperty(Property(Component.HIDDEN_INSTANCE_PROPERTY, "T"))
+
         else:
+            # Check for incoming DECLINED
+            attendee = to_component.getAttendeeProperty((recipient,))
+            if attendee and attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") == "DECLINED":
+                return True
+                    
             # It is a new override - copy any valarms on the existing master component
             # into the new one.
             [to_component.addComponent(alarm) for alarm in master_valarms]
@@ -508,6 +537,8 @@
                 organizer = to_component.getProperty("ORGANIZER")
                 if organizer:
                     organizer.setParameter("SCHEDULE-STATUS", organizer_schedule_status)
+                
+        return False
     
     @staticmethod
     def addTranspForNeedsAction(components, recipient):

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/processing.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/processing.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/processing.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -460,10 +460,20 @@
 
     @inlineCallbacks
     def doImplicitAttendeeRequest(self):
+        """
+        @return: C{tuple} of (processed, auto-processed, store inbox item, changes)
+        """
 
         # If there is no existing copy, then look for default calendar and copy it here
         if self.new_resource:
             
+            # Check if the incoming data has the recipient declined in all instances. In that case we will not create
+            # a new resource as chances are the recipient previously deleted the resource and we want to keep it deleted.
+            attendees = self.message.getAttendeeProperties((self.recipient.cuaddr,))
+            if all([attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") == "DECLINED" for attendee in attendees]):
+                log.debug("ImplicitProcessing - originator '%s' to recipient '%s' processing METHOD:REQUEST, UID: '%s' - ignoring all declined" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
+                returnValue((True, False, False, None,))
+            
             # Check for default calendar
             default = (yield self.recipient.inbox.defaultCalendar(self.request, self.message.mainType()))
             if default is None:
@@ -471,7 +481,7 @@
                 raise ImplicitProcessorException(iTIPRequestStatus.NO_USER_SUPPORT)
 
             log.debug("ImplicitProcessing - originator '%s' to recipient '%s' processing METHOD:REQUEST, UID: '%s' - new processed" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
-            new_calendar = iTipProcessing.processNewRequest(self.message, self.recipient.cuaddr)
+            new_calendar = iTipProcessing.processNewRequest(self.message, self.recipient.cuaddr, creating=True)
             
             # Handle auto-reply behavior
             if self.recipient.principal.canAutoSchedule():

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/scheduler.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/scheduler.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/scheduler.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -1278,13 +1278,16 @@
         """
         self.location = location
 
-    def add(self, recipient, what, reqstatus=None, calendar=None):
+    def add(self, recipient, what, reqstatus=None, calendar=None, suppressErrorLog=False):
         """
         Add a response.
         @param recipient: the recipient for this response.
         @param what: a status code or a L{Failure} for the given recipient.
         @param status: the iTIP request-status for the given recipient.
         @param calendar: the calendar data for the given recipient response.
+        @param suppressErrorLog: whether to suppress a log message for errors; primarily
+            this is used when trying to process a VFREEBUSY over iMIP, which isn't
+            supported.
         """
         if type(what) is int:
             code    = what
@@ -1297,7 +1300,7 @@
         else:
             raise AssertionError("Unknown data type: %r" % (what,))
 
-        if code > 400: # Error codes only
+        if not suppressErrorLog and code > 400: # Error codes only
             self.log_error("Error during %s for %s: %s" % (self.method, recipient, message))
 
         children = []

Modified: CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/test/test_icaldiff.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/test/test_icaldiff.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/twistedcaldav/scheduling/test/test_icaldiff.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -1738,6 +1738,7 @@
 ATTENDEE:mailto:user1 at example.com
 ATTENDEE;PARTSTAT=DECLINED:mailto:user2 at example.com
 ORGANIZER;CN=User 01:mailto:user1 at example.com
+X-CALENDARSERVER-HIDDEN-INSTANCE:T
 END:VEVENT
 END:VCALENDAR
 """)
@@ -1794,6 +1795,7 @@
 ATTENDEE:mailto:user1 at example.com
 ATTENDEE;PARTSTAT=DECLINED:mailto:user2 at example.com
 ORGANIZER;CN=User 01:mailto:user1 at example.com
+X-CALENDARSERVER-HIDDEN-INSTANCE:T
 END:VEVENT
 BEGIN:VEVENT
 UID:12345-67890
@@ -1803,6 +1805,7 @@
 ATTENDEE:mailto:user1 at example.com
 ATTENDEE;PARTSTAT=DECLINED:mailto:user2 at example.com
 ORGANIZER;CN=User 01:mailto:user1 at example.com
+X-CALENDARSERVER-HIDDEN-INSTANCE:T
 END:VEVENT
 END:VCALENDAR
 """)
@@ -1868,6 +1871,7 @@
 ATTENDEE:mailto:user1 at example.com
 ATTENDEE;PARTSTAT=DECLINED;RSVP=TRUE:mailto:user2 at example.com
 ORGANIZER;CN=User 01:mailto:user1 at example.com
+X-CALENDARSERVER-HIDDEN-INSTANCE:T
 END:VEVENT
 END:VCALENDAR
 """)
@@ -1934,6 +1938,7 @@
 ATTENDEE:mailto:user1 at example.com
 ATTENDEE;PARTSTAT=DECLINED;RSVP=TRUE:mailto:user2 at example.com
 ORGANIZER;CN=User 01:mailto:user1 at example.com
+X-CALENDARSERVER-HIDDEN-INSTANCE:T
 END:VEVENT
 BEGIN:VEVENT
 UID:12345-67890
@@ -1943,6 +1948,7 @@
 ATTENDEE:mailto:user1 at example.com
 ATTENDEE;PARTSTAT=DECLINED;RSVP=TRUE:mailto:user2 at example.com
 ORGANIZER;CN=User 01:mailto:user1 at example.com
+X-CALENDARSERVER-HIDDEN-INSTANCE:T
 END:VEVENT
 END:VCALENDAR
 """)
@@ -1992,15 +1998,6 @@
 PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
 BEGIN:VEVENT
 UID:12345-67890
-RECURRENCE-ID:20080601T120000Z
-DTSTART:20080601T120000Z
-DTEND:20080601T130000Z
-ATTENDEE:mailto:user1 at example.com
-ATTENDEE;PARTSTAT=DECLINED;RSVP=TRUE:mailto:user2 at example.com
-ORGANIZER;CN=User 01:mailto:user1 at example.com
-END:VEVENT
-BEGIN:VEVENT
-UID:12345-67890
 RECURRENCE-ID:20080604T120000Z
 DTSTART:20080604T130000Z
 DTEND:20080604T140000Z

Modified: CalendarServer/branches/users/gaya/directorybacker/txdav/caldav/datastore/util.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/txdav/caldav/datastore/util.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/txdav/caldav/datastore/util.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -36,6 +36,7 @@
 from twext.python.vcomponent import InvalidICalendarDataError
 from twext.python.vcomponent import VComponent
 
+from twistedcaldav.datafilters.hiddeninstance import HiddenInstanceFilter
 from twistedcaldav.datafilters.peruserdata import PerUserDataFilter
 from twistedcaldav.datafilters.privateevents import PrivateEventFilter
 
@@ -406,8 +407,11 @@
         calendar = self.calendar()
         isOwner = asAdmin or (calendar._owned and
                               calendar.ownerCalendarHome().uid() == accessUID)
-        for data_filter in [PrivateEventFilter(self.accessMode, isOwner),
-                       PerUserDataFilter(accessUID)]:
+        for data_filter in [
+            PerUserDataFilter(accessUID),
+            HiddenInstanceFilter(),
+            PrivateEventFilter(self.accessMode, isOwner),
+        ]:
             component = data_filter.filter(component)
         returnValue(component)
 
@@ -501,7 +505,7 @@
                 fixes += 1
                 calprop.setValue(postval)
     for subc in component.subcomponents():
-        count, fixsubc = fixOneCalendarObject(subc)
+        count, _ignore_fixsubc = fixOneCalendarObject(subc)
         fixes += count
     return fixes, component
 

Modified: CalendarServer/branches/users/gaya/directorybacker/txdav/xml/__init__.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/txdav/xml/__init__.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/txdav/xml/__init__.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -33,6 +33,7 @@
 See RFC 3744: http://www.ietf.org/rfc/rfc3744.txt (WebDAV ACLs)
 See RFC 4331: http://www.ietf.org/rfc/rfc4331.txt (WebDAV Quota)
 See RFC 5842: http://www.ietf.org/rfc/rfc5842.txt (WebDAV Bind)
+See RFC 6578: http://www.ietf.org/rfc/rfc6578.txt (WebDAV Sync)
 """
 
 __all__ = [
@@ -45,10 +46,10 @@
 import txdav.xml.rfc3253
 import txdav.xml.rfc3744
 import txdav.xml.rfc4331
+import txdav.xml.rfc5397
 import txdav.xml.rfc5842
-import txdav.xml.rfc5397
 import txdav.xml.rfc5995
-import txdav.xml.draft_sync
+import txdav.xml.rfc6578
 import txdav.xml.extensions
 
 txdav # Shhh pyflakes

Deleted: CalendarServer/branches/users/gaya/directorybacker/txdav/xml/draft_sync.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/txdav/xml/draft_sync.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/txdav/xml/draft_sync.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -1,103 +0,0 @@
-##
-# Copyright (c) 2009-2012 Apple Computer, Inc. All rights reserved.
-#
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to deal
-# in the Software without restriction, including without limitation the rights
-# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-# copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-# 
-# The above copyright notice and this permission notice shall be included in all
-# copies or substantial portions of the Software.
-# 
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-# SOFTWARE.
-##
-
-"""
-draft-daboo-webdav-sync (Collection Synchronization for WebDAV) XML
-Elements
-
-This module provides XML element definitions for use with WebDAV
-Synchronization.
-
-See draft-daboo-webdav-sync: http://tools.ietf.org/html/draft-daboo-webdav-sync
-
-Last draft referenced: -08
-"""
-
-__all__ = []
-
-
-from txdav.xml.base import WebDAVElement, WebDAVTextElement, dav_namespace
-from txdav.xml.element import registerElement, registerElementClass
-from txdav.xml.rfc2518 import MultiStatus
-
-
- at registerElement
- at registerElementClass
-class SyncCollection (WebDAVElement):
-    """
-    DAV report used to retrieve specific calendar component items via
-    their URIs.
-    """
-    name = "sync-collection"
-
-    # To allow for an empty element in a supported-report-set property we need
-    # to relax the child restrictions
-    allowed_children = {
-        (dav_namespace, "sync-token"): (0, 1), # When used in the REPORT this is required
-        (dav_namespace, "sync-level"): (0, 1), # When used in the REPORT this is required
-        (dav_namespace, "prop"      ): (0, 1),
-    }
-
-    def __init__(self, *children, **attributes):
-        super(SyncCollection, self).__init__(*children, **attributes)
-
-        self.property = None
-        self.sync_token = None
-        self.sync_level = None
-
-        for child in self.children:
-            qname = child.qname()
-
-            if qname == (dav_namespace, "sync-token"):
-                self.sync_token = str(child)
-
-            elif qname == (dav_namespace, "sync-level"):
-                self.sync_level = str(child)
-
-            elif qname == (dav_namespace, "prop"):
-                if self.property is not None:
-                    raise ValueError("Only one of DAV:prop allowed")
-                self.property = child
-
-
- at registerElement
- at registerElementClass
-class SyncToken (WebDAVTextElement):
-    """
-    Synchronization token used in report and as a property.
-    """
-    name = "sync-token"
-    hidden = True
-    protected = True
-
-
- at registerElement
- at registerElementClass
-class SyncLevel (WebDAVTextElement):
-    """
-    Synchronization level used in report.
-    """
-    name = "sync-level"
-
-
-# Extend MultiStatus, to add sync-token
-MultiStatus.allowed_children[(dav_namespace, "sync-token")] = (0, 1)

Modified: CalendarServer/branches/users/gaya/directorybacker/txdav/xml/parser.py
===================================================================
--- CalendarServer/branches/users/gaya/directorybacker/txdav/xml/parser.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/directorybacker/txdav/xml/parser.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -32,4 +32,4 @@
     "WebDAVDocument",
 ]
 
-from txdav.xml.parser_sax import WebDAVDocument
+from txdav.xml.parser_etree import WebDAVDocument


Property changes on: CalendarServer/branches/users/gaya/inviteclean
___________________________________________________________________
Modified: svn:mergeinfo
   - /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/generic-sqlstore:6167-6191
/CalendarServer/branches/new-store:5594-5934
/CalendarServer/branches/new-store-no-caldavfile:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
/CalendarServer/branches/users/cdaboo/batchupload-6699:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
/CalendarServer/branches/users/cdaboo/component-set-fixes:8130-8346
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/implicituidrace:8137-8141
/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/pods:7297-7377
/CalendarServer/branches/users/cdaboo/pycalendar:7085-7206
/CalendarServer/branches/users/cdaboo/pycard:7227-7237
/CalendarServer/branches/users/cdaboo/queued-attendee-refreshes:7740-8287
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
/CalendarServer/branches/users/cdaboo/timezones:7443-7699
/CalendarServer/branches/users/cdaboo/txn-debugging:8730-8743
/CalendarServer/branches/users/glyph/case-insensitive-uid:8772-8805
/CalendarServer/branches/users/glyph/conn-limit:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/dalify:6932-7023
/CalendarServer/branches/users/glyph/db-reconnect:6824-6876
/CalendarServer/branches/users/glyph/deploybuild:7563-7572
/CalendarServer/branches/users/glyph/disable-quota:7718-7727
/CalendarServer/branches/users/glyph/dont-start-postgres:6592-6614
/CalendarServer/branches/users/glyph/imip-and-admin-html:7866-7984
/CalendarServer/branches/users/glyph/ipv6-client:9054-9105
/CalendarServer/branches/users/glyph/linux-tests:6893-6900
/CalendarServer/branches/users/glyph/migrate-merge:8690-8713
/CalendarServer/branches/users/glyph/misc-portability-fixes:7365-7374
/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
/CalendarServer/branches/users/glyph/more-deferreds-7:6369-6445
/CalendarServer/branches/users/glyph/multiget-delete:8321-8330
/CalendarServer/branches/users/glyph/new-export:7444-7485
/CalendarServer/branches/users/glyph/oracle:7106-7155
/CalendarServer/branches/users/glyph/oracle-nulls:7340-7351
/CalendarServer/branches/users/glyph/other-html:8062-8091
/CalendarServer/branches/users/glyph/parallel-sim:8240-8251
/CalendarServer/branches/users/glyph/parallel-upgrade:8376-8400
/CalendarServer/branches/users/glyph/parallel-upgrade_to_1:8571-8583
/CalendarServer/branches/users/glyph/q:9560-9688
/CalendarServer/branches/users/glyph/quota:7604-7637
/CalendarServer/branches/users/glyph/sendfdport:5388-5424
/CalendarServer/branches/users/glyph/shared-pool-fixes:8436-8443
/CalendarServer/branches/users/glyph/shared-pool-take2:8155-8174
/CalendarServer/branches/users/glyph/sharedpool:6490-6550
/CalendarServer/branches/users/glyph/sharing-api:9192-9205
/CalendarServer/branches/users/glyph/skip-lonely-vtimezones:8524-8535
/CalendarServer/branches/users/glyph/sql-store:5929-6073
/CalendarServer/branches/users/glyph/subtransactions:7248-7258
/CalendarServer/branches/users/glyph/table-alias:8651-8664
/CalendarServer/branches/users/glyph/uidexport:7673-7676
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/glyph/uuid-normalize:9268-9296
/CalendarServer/branches/users/glyph/xattrs-from-files:7757-7769
/CalendarServer/branches/users/sagen/applepush:8126-8184
/CalendarServer/branches/users/sagen/inboxitems:7380-7381
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/branches/users/wsanchez/transations:5515-5593
/CalendarServer/trunk:9408-9800
   + /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/generic-sqlstore:6167-6191
/CalendarServer/branches/new-store:5594-5934
/CalendarServer/branches/new-store-no-caldavfile:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
/CalendarServer/branches/users/cdaboo/batchupload-6699:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
/CalendarServer/branches/users/cdaboo/component-set-fixes:8130-8346
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/implicituidrace:8137-8141
/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/pods:7297-7377
/CalendarServer/branches/users/cdaboo/pycalendar:7085-7206
/CalendarServer/branches/users/cdaboo/pycard:7227-7237
/CalendarServer/branches/users/cdaboo/queued-attendee-refreshes:7740-8287
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
/CalendarServer/branches/users/cdaboo/timezones:7443-7699
/CalendarServer/branches/users/cdaboo/txn-debugging:8730-8743
/CalendarServer/branches/users/glyph/case-insensitive-uid:8772-8805
/CalendarServer/branches/users/glyph/conn-limit:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/dalify:6932-7023
/CalendarServer/branches/users/glyph/db-reconnect:6824-6876
/CalendarServer/branches/users/glyph/deploybuild:7563-7572
/CalendarServer/branches/users/glyph/disable-quota:7718-7727
/CalendarServer/branches/users/glyph/dont-start-postgres:6592-6614
/CalendarServer/branches/users/glyph/imip-and-admin-html:7866-7984
/CalendarServer/branches/users/glyph/ipv6-client:9054-9105
/CalendarServer/branches/users/glyph/linux-tests:6893-6900
/CalendarServer/branches/users/glyph/migrate-merge:8690-8713
/CalendarServer/branches/users/glyph/misc-portability-fixes:7365-7374
/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
/CalendarServer/branches/users/glyph/more-deferreds-7:6369-6445
/CalendarServer/branches/users/glyph/multiget-delete:8321-8330
/CalendarServer/branches/users/glyph/new-export:7444-7485
/CalendarServer/branches/users/glyph/oracle:7106-7155
/CalendarServer/branches/users/glyph/oracle-nulls:7340-7351
/CalendarServer/branches/users/glyph/other-html:8062-8091
/CalendarServer/branches/users/glyph/parallel-sim:8240-8251
/CalendarServer/branches/users/glyph/parallel-upgrade:8376-8400
/CalendarServer/branches/users/glyph/parallel-upgrade_to_1:8571-8583
/CalendarServer/branches/users/glyph/q:9560-9688
/CalendarServer/branches/users/glyph/quota:7604-7637
/CalendarServer/branches/users/glyph/sendfdport:5388-5424
/CalendarServer/branches/users/glyph/shared-pool-fixes:8436-8443
/CalendarServer/branches/users/glyph/shared-pool-take2:8155-8174
/CalendarServer/branches/users/glyph/sharedpool:6490-6550
/CalendarServer/branches/users/glyph/sharing-api:9192-9205
/CalendarServer/branches/users/glyph/skip-lonely-vtimezones:8524-8535
/CalendarServer/branches/users/glyph/sql-store:5929-6073
/CalendarServer/branches/users/glyph/subtransactions:7248-7258
/CalendarServer/branches/users/glyph/table-alias:8651-8664
/CalendarServer/branches/users/glyph/uidexport:7673-7676
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/glyph/uuid-normalize:9268-9296
/CalendarServer/branches/users/glyph/xattrs-from-files:7757-7769
/CalendarServer/branches/users/sagen/applepush:8126-8184
/CalendarServer/branches/users/sagen/inboxitems:7380-7381
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/branches/users/wsanchez/transations:5515-5593
/CalendarServer/trunk:9408-9805

Modified: CalendarServer/branches/users/gaya/inviteclean/calendarserver/push/applepush.py
===================================================================
--- CalendarServer/branches/users/gaya/inviteclean/calendarserver/push/applepush.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/inviteclean/calendarserver/push/applepush.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -317,16 +317,16 @@
         @type status: C{int}
         """
         msg = self.STATUS_CODES.get(status, "Unknown status code")
-        self.log_warn("Received APN error %d on identifier %d: %s" % (status, identifier, msg))
+        self.log_info("Received APN error %d on identifier %d: %s" % (status, identifier, msg))
         if status in self.TOKEN_REMOVAL_CODES:
             token = self.history.extractIdentifier(identifier)
             if token is not None:
-                self.log_warn("Removing subscriptions for bad token: %s" %
+                self.log_debug("Removing subscriptions for bad token: %s" %
                     (token,))
                 txn = self.factory.store.newTransaction()
                 subscriptions = (yield txn.apnSubscriptionsByToken(token))
                 for key, modified, uid in subscriptions:
-                    self.log_warn("Removing subscription: %s %s" %
+                    self.log_debug("Removing subscription: %s %s" %
                         (token, key))
                     yield txn.removeAPNSubscription(token, key)
                 yield txn.commit()
@@ -384,13 +384,13 @@
         self.shuttingDown = False
 
     def clientConnectionMade(self):
-        self.log_warn("Connection to APN server made")
+        self.log_info("Connection to APN server made")
         self.service.clientConnectionMade()
         self.delay = 1.0
 
     def clientConnectionLost(self, connector, reason):
         if not self.shuttingDown:
-            self.log_warn("Connection to APN server lost: %s" % (reason,))
+            self.log_info("Connection to APN server lost: %s" % (reason,))
         ReconnectingClientFactory.clientConnectionLost(self, connector, reason)
 
     def clientConnectionFailed(self, connector, reason):
@@ -400,7 +400,7 @@
             reason)
 
     def retry(self, connector=None):
-        self.log_warn("Reconnecting to APN server")
+        self.log_info("Reconnecting to APN server")
         ReconnectingClientFactory.retry(self, connector)
 
     def stopTrying(self):
@@ -470,12 +470,12 @@
             self.scheduler = None
 
     def startService(self):
-        self.log_info("APNProviderService startService")
+        self.log_debug("APNProviderService startService")
         self.factory = APNProviderFactory(self, self.store)
         self.connect(self.factory)
 
     def stopService(self):
-        self.log_info("APNProviderService stopService")
+        self.log_debug("APNProviderService stopService")
         if self.factory is not None:
             self.factory.stopTrying()
         if self.scheduler is not None:

Modified: CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/common.py
===================================================================
--- CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/common.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/common.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -19,7 +19,7 @@
 from twext.web2.http import HTTPError
 from twext.web2 import responsecode
 from twext.web2.dav.util import joinURL
-from twistedcaldav.directory.util import transactionFromRequest
+from twistedcaldav.directory.util import transactionFromRequest, NotFoundResource
 from twistedcaldav.directory.resource import DirectoryReverseProxyResource
 
 from twext.python.log import Logger
@@ -147,7 +147,9 @@
 
         record = self.directory.recordWithShortName(self.recordType, name)
         if record is None:
-            returnValue((None, []))
+            returnValue(
+                (NotFoundResource(principalCollections=self._parent.principalCollections()), [])
+            )
 
         child = yield self._parent.homeForDirectoryRecord(record, request)
         returnValue((child, segments[1:]))

Modified: CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/principal.py
===================================================================
--- CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/principal.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/principal.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -36,6 +36,7 @@
 from twisted.internet.defer import inlineCallbacks, returnValue
 from twisted.internet.defer import succeed
 from twisted.web.template import XMLFile, Element, renderer, tags
+from twistedcaldav.directory.util import NotFoundResource
 
 from twext.web2.auth.digest import DigestedCredentials
 from twext.web2 import responsecode
@@ -46,6 +47,7 @@
 
 from twext.python.log import Logger
 
+
 try:
     from twistedcaldav.authkerb import NegotiateCredentials
     NegotiateCredentials # sigh, pyflakes
@@ -130,6 +132,7 @@
             (origCUAddr,))
 
 
+
 class DirectoryProvisioningResource (
     PermissionsMixIn,
     CalendarPrincipalCollectionResource,
@@ -153,7 +156,7 @@
         child = self.getChild(segments[0])
         if child is not None:
             return (child, segments[1:])
-        return (None, ())
+        return (NotFoundResource(principalCollections=self.principalCollections()),())
 
     def deadProperties(self):
         if not hasattr(self, "_dead_properties"):

Modified: CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/test/test_calendar.py
===================================================================
--- CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/test/test_calendar.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/test/test_calendar.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -21,7 +21,7 @@
 
 from twistedcaldav.test.util import TestCase
 from twext.web2.test.test_server import SimpleRequest
-from twistedcaldav.directory.util import transactionFromRequest
+from twistedcaldav.directory.util import transactionFromRequest, NotFoundResource
 
 class ProvisionedCalendars (TestCase):
     """
@@ -55,10 +55,14 @@
 
 
     def test_NonExistentCalendarHome(self):
+        """
+        Requests for missing homes and principals should return
+        NotFoundResources so that we have the opportunity to
+        turn 404s into 401s to protect against user-existence attacks.
+        """
 
         def _response(resource):
-            if resource is not None:
-                self.fail("Incorrect response to GET on non-existent calendar home.")
+            self.assertTrue(isinstance(resource, NotFoundResource))
 
         request = self.oneRequest("/calendars/users/12345/")
         d = request.locateResource(request.uri)

Modified: CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/util.py
===================================================================
--- CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/util.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/directory/util.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -22,12 +22,21 @@
 __all__ = [
     "normalizeUUID",
     "uuidFromName",
+    "NotFoundResource",
 ]
 
 from twext.enterprise.ienterprise import AlreadyFinishedError
-
+from twext.python.log import Logger
+from twext.web2 import responsecode
+from twext.web2.auth.wrapper import UnauthorizedResponse
+from twext.web2.dav.resource import DAVResource
+from twext.web2.http import StatusResponse
+from twisted.internet.defer import inlineCallbacks, returnValue
+from txdav.xml import element as davxml
 from uuid import UUID, uuid5
 
+log = Logger()
+
 def uuidFromName(namespace, name):
     """
     Generate a version 5 (SHA-1) UUID from a namespace UUID and a name.
@@ -109,3 +118,31 @@
         yield set(data[:size])
         del data[:size]
 
+
+class NotFoundResource(DAVResource):
+    """
+    In order to prevent unauthenticated discovery of existing users via 401/404
+    response codes, this resource can be returned from locateChild, and it will
+    perform an authentication; if the user is unauthenticated, 404 responses are
+    turned into 401s.
+    """
+
+    @inlineCallbacks
+    def renderHTTP(self, request):
+
+        try:
+            authnUser, authzUser = yield self.authenticate(request)
+        except Exception:
+            authzUser = davxml.Principal(davxml.Unauthenticated())
+
+        # Turn 404 into 401
+        if authzUser == davxml.Principal(davxml.Unauthenticated()):
+            response = (yield UnauthorizedResponse.makeResponse(
+                request.credentialFactories,
+                request.remoteAddr
+            ))
+            returnValue(response)
+        else:
+            response = StatusResponse(responsecode.NOT_FOUND, "Resource not found")
+            returnValue(response)
+

Modified: CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/scheduling/imip.py
===================================================================
--- CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/scheduling/imip.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/scheduling/imip.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -62,7 +62,8 @@
             self.responses.add(
                 recipient.cuaddr,
                 Failure(exc_value=err),
-                reqstatus=iTIPRequestStatus.SERVICE_UNAVAILABLE
+                reqstatus=iTIPRequestStatus.SERVICE_UNAVAILABLE,
+                suppressErrorLog=True
             )
         
         # Generate an HTTP client request

Modified: CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/scheduling/scheduler.py
===================================================================
--- CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/scheduling/scheduler.py	2012-09-11 19:42:26 UTC (rev 9805)
+++ CalendarServer/branches/users/gaya/inviteclean/twistedcaldav/scheduling/scheduler.py	2012-09-11 22:54:00 UTC (rev 9806)
@@ -1278,13 +1278,16 @@
         """
         self.location = location
 
-    def add(self, recipient, what, reqstatus=None, calendar=None):
+    def add(self, recipient, what, reqstatus=None, calendar=None, suppressErrorLog=False):
         """
         Add a response.
         @param recipient: the recipient for this response.
         @param what: a status code or a L{Failure} for the given recipient.
         @param status: the iTIP request-status for the given recipient.
         @param calendar: the calendar data for the given recipient response.
+        @param suppressErrorLog: whether to suppress a log message for errors; primarily
+            this is used when trying to process a VFREEBUSY over iMIP, which isn't
+            supported.
         """
         if type(what) is int:
             code    = what
@@ -1297,7 +1300,7 @@
         else:
             raise AssertionError("Unknown data type: %r" % (what,))
 
-        if code > 400: # Error codes only
+        if not suppressErrorLog and code > 400: # Error codes only
             self.log_error("Error during %s for %s: %s" % (self.method, recipient, message))
 
         children = []
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/calendarserver-changes/attachments/20120911/63d049b2/attachment-0001.html>


More information about the calendarserver-changes mailing list