[CalendarServer-changes] [11016] CalendarServer/branches/users/cdaboo/store-scheduling

source_changes at macosforge.org source_changes at macosforge.org
Mon Apr 8 13:53:04 PDT 2013


Revision: 11016
          http://trac.calendarserver.org//changeset/11016
Author:   cdaboo at apple.com
Date:     2013-04-08 13:53:03 -0700 (Mon, 08 Apr 2013)
Log Message:
-----------
Checkpoint for initial scheduling in the store work. The twistedcaldav.scheduling module has been pulled into txdav.caldav.datastore
and initial work done to allow setComponent() to do validation and store with basic scheduling.

Modified Paths:
--------------
    CalendarServer/branches/users/cdaboo/store-scheduling/twistedcaldav/upgrade.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/sql.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/common.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/test_index_file.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/test_sql.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/util.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/icalendarstore.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/datastore/sql.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/datastore/sql_schema/current.sql
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/datastore/test/util.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/icommondatastore.py

Added Paths:
-----------
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/__init__.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/addressmapping.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/__init__.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/delivery.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/resource.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/scheduler.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/test/
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/test/__init__.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/test/test_delivery.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/test/test_resource.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/cuaddress.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/delivery.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/icaldiff.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/__init__.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/delivery.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/inbound.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/mailgateway.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/outbound.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/scheduler.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/smtpsender.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/__init__.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_delivery.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_inbound.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_mailgateway.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_outbound.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/implicit.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/__init__.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/delivery.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/dkim.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/localservers.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/remoteservers.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/resource.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/scheduler.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/__init__.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_delivery.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_dkim.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_localservers.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_remoteservers.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_resource.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_utils.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/utils.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/xml.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/itip.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/processing.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/scheduler.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/__init__.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/accounts.xml
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/resources.xml
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_icaldiff.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_implicit.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_itip.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_pocessing.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_utils.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/utils.py
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/calendar_store/ho/me/home_defaults/
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/calendar_store/ho/me/home_defaults/calendar_1/
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/calendar_store/ho/me/home_defaults/calendar_1/1.ics
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/calendar_store/ho/me/home_defaults/calendar_1/3.ics
    CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/test_implicit.py

Modified: CalendarServer/branches/users/cdaboo/store-scheduling/twistedcaldav/upgrade.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/twistedcaldav/upgrade.py	2013-04-08 19:22:13 UTC (rev 11015)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/twistedcaldav/upgrade.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -310,7 +310,7 @@
         errorOccurred.append(True)
 
 
-    def doProxyDatabaseMoveUpgrade(config, uid=-1, gid=-1):
+    def doProxyDatabaseMoveUpgrade(config, uid= -1, gid= -1):
         # See if the new one is already present
         oldFilename = ".db.calendaruserproxy"
         newFilename = "proxies.sqlite"
@@ -349,7 +349,7 @@
         )
 
 
-    def moveCalendarHome(oldHome, newHome, uid=-1, gid=-1):
+    def moveCalendarHome(oldHome, newHome, uid= -1, gid= -1):
         if os.path.exists(newHome):
             # Both old and new homes exist; stop immediately to let the
             # administrator fix it
@@ -865,7 +865,7 @@
 
 
 
-def makeDirsUserGroup(path, uid=-1, gid=-1):
+def makeDirsUserGroup(path, uid= -1, gid= -1):
     parts = path.split("/")
     if parts[0] == "": # absolute path
         parts[0] = "/"
@@ -1089,7 +1089,6 @@
         yield migrateTokensToStore(self.config.DataRoot, self.store)
 
 
-
     @inlineCallbacks
     def processInboxItems(self):
         """
@@ -1259,7 +1258,7 @@
             scheduler = DirectScheduler(request, inboxItem)
             # Process inbox item
             yield scheduler.doSchedulingViaPUT(originator, recipients, calendar,
-                internal_request=False)
+                internal_request=False, noAttendeeRefresh=True)
         else:
             log.warn("Removing invalid inbox item: %s" % (uri,))
 

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/__init__.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/__init__.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/addressmapping.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/addressmapping.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/addressmapping.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,103 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twisted.internet.defer import inlineCallbacks, returnValue
+
+from twext.python.log import Logger
+
+from twistedcaldav.config import config
+from twistedcaldav.memcacher import Memcacher
+
+from txdav.caldav.datastore.scheduling.caldav.delivery import ScheduleViaCalDAV
+from txdav.caldav.datastore.scheduling.delivery import DeliveryService
+from txdav.caldav.datastore.scheduling.imip.delivery import ScheduleViaIMip
+from txdav.caldav.datastore.scheduling.ischedule.delivery import ScheduleViaISchedule
+from txdav.caldav.datastore.scheduling.cuaddress import RemoteCalendarUser, EmailCalendarUser, InvalidCalendarUser, \
+    calendarUserFromPrincipal
+
+__all__ = [
+    "ScheduleAddressMapper",
+    "mapper",
+]
+
+log = Logger()
+
+"""
+Handle mapping a calendar user address to a schedule delivery type.
+"""
+
+class ScheduleAddressMapper(object):
+    """
+    Class that maps a calendar user address into a delivery service type.
+    """
+
+    def __init__(self):
+
+        # We are going to cache mappings whilst running
+        self.cache = Memcacher("ScheduleAddressMapper", no_invalidation=True)
+
+
+    @inlineCallbacks
+    def getCalendarUser(self, cuaddr, principal):
+
+        # If we have a principal always treat the user as local or partitioned
+        if principal:
+            returnValue(calendarUserFromPrincipal(cuaddr, principal))
+
+        # Get the type
+        cuaddr_type = (yield self.getCalendarUserServiceType(cuaddr))
+        if cuaddr_type == DeliveryService.serviceType_caldav:
+            returnValue(InvalidCalendarUser(cuaddr))
+        elif cuaddr_type == DeliveryService.serviceType_ischedule:
+            returnValue(RemoteCalendarUser(cuaddr))
+        elif cuaddr_type == DeliveryService.serviceType_imip:
+            returnValue(EmailCalendarUser(cuaddr))
+        else:
+            returnValue(InvalidCalendarUser(cuaddr))
+
+
+    @inlineCallbacks
+    def getCalendarUserServiceType(self, cuaddr):
+
+        # Try cache first
+        cuaddr_type = (yield self.cache.get(str(cuaddr)))
+        if cuaddr_type is None:
+
+            serviceTypes = (ScheduleViaCalDAV,)
+            if config.Scheduling[DeliveryService.serviceType_ischedule]["Enabled"]:
+                serviceTypes += (ScheduleViaISchedule,)
+            if config.Scheduling[DeliveryService.serviceType_imip]["Enabled"]:
+                serviceTypes += (ScheduleViaIMip,)
+            for service in serviceTypes:
+                matched = (yield service.matchCalendarUserAddress(cuaddr))
+                if matched:
+                    yield self.cache.set(str(cuaddr), service.serviceType())
+                    returnValue(service.serviceType())
+
+        returnValue(cuaddr_type)
+
+
+    def isCalendarUserInMyDomain(self, cuaddr):
+
+        # Check whether it is a possible local address
+        def _gotResult(serviceType):
+            return serviceType == DeliveryService.serviceType_caldav
+
+        d = self.getCalendarUserServiceType(cuaddr)
+        d.addCallback(_gotResult)
+        return d
+
+mapper = ScheduleAddressMapper()

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/__init__.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/__init__.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/delivery.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/delivery.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/delivery.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,275 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.python.log import Logger
+from twext.web2.dav.http import ErrorResponse
+
+from twisted.internet.defer import inlineCallbacks, returnValue, succeed
+from twisted.python.failure import Failure
+from twext.web2 import responsecode
+from twext.web2.http import HTTPError
+
+from twistedcaldav.caldavxml import caldav_namespace
+from twistedcaldav.config import config
+from twistedcaldav.customxml import calendarserver_namespace
+from twistedcaldav.method import report_common
+from twistedcaldav.resource import isCalendarCollectionResource
+
+from txdav.caldav.datastore.scheduling.cuaddress import LocalCalendarUser, RemoteCalendarUser, \
+    PartitionedCalendarUser, OtherServerCalendarUser
+from txdav.caldav.datastore.scheduling.delivery import DeliveryService
+from txdav.caldav.datastore.scheduling.itip import iTIPRequestStatus
+from txdav.caldav.datastore.scheduling.processing import ImplicitProcessor, ImplicitProcessorException
+
+import hashlib
+import uuid
+from txdav.base.propertystore.base import PropertyName
+
+
+"""
+Handles the sending of scheduling messages to the server itself. This will cause
+actual processing of the delivery of the message to the recipient's inbox, via the
+L{ImplicitProcessor} class.
+"""
+
+__all__ = [
+    "ScheduleViaCalDAV",
+]
+
+log = Logger()
+
+class ScheduleViaCalDAV(DeliveryService):
+
+    def __init__(self, scheduler, recipients, responses, freebusy):
+
+        self.scheduler = scheduler
+        self.recipients = recipients
+        self.responses = responses
+        self.freebusy = freebusy
+
+
+    @classmethod
+    def serviceType(cls):
+        return DeliveryService.serviceType_caldav
+
+
+    @classmethod
+    def matchCalendarUserAddress(cls, cuaddr):
+
+        # Check for local address matches first
+        if cuaddr.startswith("mailto:") and config.Scheduling[cls.serviceType()]["EmailDomain"]:
+            addr = cuaddr[7:].split("?")[0]
+            domain = config.Scheduling[cls.serviceType()]["EmailDomain"]
+            _ignore_account, addrDomain = addr.split("@")
+            if addrDomain == domain:
+                return succeed(True)
+
+        elif (cuaddr.startswith("http://") or cuaddr.startswith("https://")) and config.Scheduling[cls.serviceType()]["HTTPDomain"]:
+            splits = cuaddr.split(":")[0][2:].split("?")
+            domain = config.Scheduling[cls.serviceType()]["HTTPDomain"]
+            if splits[0].endswith(domain):
+                return succeed(True)
+
+        elif cuaddr.startswith("/"):
+            # Assume relative HTTP URL - i.e. on this server
+            return succeed(True)
+
+        # Do default match
+        return super(ScheduleViaCalDAV, cls).matchCalendarUserAddress(cuaddr)
+
+
+    @inlineCallbacks
+    def generateSchedulingResponses(self):
+
+        # Extract the ORGANIZER property and UID value from the calendar data for use later
+        organizerProp = self.scheduler.calendar.getOrganizerProperty()
+        uid = self.scheduler.calendar.resourceUID()
+
+        organizerPrincipal = None
+        if type(self.scheduler.organizer) in (LocalCalendarUser, PartitionedCalendarUser, OtherServerCalendarUser,):
+            organizerPrincipal = self.scheduler.organizer.principal.uid()
+
+        for recipient in self.recipients:
+
+            #
+            # Check access controls - we do not do this right now. But if we ever implement access controls to
+            # determine which users can schedule with other users, here is where we would do that test.
+            #
+
+            # Different behavior for free-busy vs regular invite
+            if self.freebusy:
+                # Look for special delegate extended free-busy request
+                event_details = [] if self.scheduler.calendar.getExtendedFreeBusy() else None
+
+                yield self.generateFreeBusyResponse(recipient, self.responses, organizerProp, organizerPrincipal, uid, event_details)
+            else:
+                yield self.generateResponse(recipient, self.responses)
+
+
+    @inlineCallbacks
+    def generateResponse(self, recipient, responses):
+        # Hash the iCalendar data for use as the last path element of the URI path
+        name = "%s-%s.ics" % (hashlib.md5(self.scheduler.calendar.resourceUID()).hexdigest(), str(uuid.uuid4())[:8],)
+
+        # Do implicit scheduling message processing.
+        try:
+            processor = ImplicitProcessor()
+            _ignore_processed, autoprocessed, store_inbox, changes = (yield processor.doImplicitProcessing(
+                self.scheduler.txn,
+                self.scheduler.calendar,
+                self.scheduler.originator,
+                recipient,
+                noAttendeeRefresh=self.scheduler.noAttendeeRefresh,
+            ))
+        except ImplicitProcessorException, e:
+            log.err("Could not store data in Inbox : %s" % (recipient.inbox,))
+            err = HTTPError(ErrorResponse(
+                responsecode.FORBIDDEN,
+                (caldav_namespace, "recipient-permissions"),
+                "Could not store data in inbox",
+            ))
+            responses.add(recipient.cuaddr, Failure(exc_value=err), reqstatus=e.msg)
+            returnValue(False)
+
+        if store_inbox:
+            # Copy calendar to inbox
+            try:
+                yield recipient.inbox.createCalendarObjectWithName(name, self.scheduler.calendar)
+            except Exception as e:
+                # FIXME: Bare except
+                log.err("Could not store data in Inbox : %s %s" % (recipient.inbox, e,))
+                err = HTTPError(ErrorResponse(
+                    responsecode.FORBIDDEN,
+                    (caldav_namespace, "recipient-permissions"),
+                    "Could not store data in inbox",
+                ))
+                responses.add(recipient.cuaddr, Failure(exc_value=err), reqstatus=iTIPRequestStatus.NO_AUTHORITY)
+                returnValue(False)
+            else:
+                # Store CS:schedule-changes property if present
+                if changes:
+                    props = recipient.inbox.properties()
+                    props[PropertyName(*changes.qname())] = changes
+
+        responses.add(recipient.cuaddr, responsecode.OK, reqstatus=iTIPRequestStatus.MESSAGE_DELIVERED)
+        if autoprocessed:
+            if self.scheduler.logItems is not None:
+                self.scheduler.logItems["itip.auto"] = self.scheduler.logItems.get("itip.auto", 0) + 1
+        returnValue(True)
+
+
+    @inlineCallbacks
+    def generateFreeBusyResponse(self, recipient, responses, organizerProp, organizerPrincipal, uid, event_details):
+
+        # Extract the ATTENDEE property matching current recipient from the calendar data
+        cuas = recipient.principal.calendarUserAddresses()
+        attendeeProp = self.scheduler.calendar.getAttendeeProperty(cuas)
+
+        remote = isinstance(self.scheduler.organizer, RemoteCalendarUser)
+
+        try:
+            fbresult = (yield self.generateAttendeeFreeBusyResponse(
+                recipient,
+                organizerProp,
+                organizerPrincipal,
+                uid,
+                attendeeProp,
+                remote,
+                event_details,
+            ))
+        except:
+            log.err("Could not determine free busy information: %s" % (recipient.cuaddr,))
+            err = HTTPError(ErrorResponse(
+                responsecode.FORBIDDEN,
+                (caldav_namespace, "recipient-permissions"),
+                "Could not determine free busy information",
+            ))
+            responses.add(
+                recipient.cuaddr,
+                Failure(exc_value=err),
+                reqstatus=iTIPRequestStatus.NO_AUTHORITY
+            )
+            returnValue(False)
+        else:
+            responses.add(
+                recipient.cuaddr,
+                responsecode.OK,
+                reqstatus=iTIPRequestStatus.SUCCESS,
+                calendar=fbresult
+            )
+            returnValue(True)
+
+
+    @inlineCallbacks
+    def generateAttendeeFreeBusyResponse(self, recipient, organizerProp, organizerPrincipal, uid, attendeeProp, remote, event_details=None):
+
+        # Find the current recipients calendar-free-busy-set
+        fbset = (yield recipient.principal.calendarFreeBusyURIs(self.scheduler.request))
+
+        # First list is BUSY, second BUSY-TENTATIVE, third BUSY-UNAVAILABLE
+        fbinfo = ([], [], [])
+
+        # Process the availability property from the Inbox.
+        has_prop = (yield recipient.inbox.hasProperty((calendarserver_namespace, "calendar-availability"), self.scheduler.request))
+        if has_prop:
+            availability = (yield recipient.inbox.readProperty((calendarserver_namespace, "calendar-availability"), self.scheduler.request))
+            availability = availability.calendar()
+            report_common.processAvailabilityFreeBusy(availability, fbinfo, self.scheduler.timeRange)
+
+        # Check to see if the recipient is the same calendar user as the organizer.
+        # Needed for masked UID stuff.
+        if isinstance(self.scheduler.organizer, LocalCalendarUser):
+            same_calendar_user = self.scheduler.organizer.principal.principalURL() == recipient.principal.principalURL()
+        else:
+            same_calendar_user = False
+
+        # Now process free-busy set calendars
+        matchtotal = 0
+        for calendarResourceURL in fbset:
+            if not calendarResourceURL.endswith('/'):
+                calendarResourceURL += '/'
+            calendarResource = (yield self.scheduler.request.locateResource(calendarResourceURL))
+            if calendarResource is None or not calendarResource.exists() or not isCalendarCollectionResource(calendarResource):
+                # We will ignore missing calendars. If the recipient has failed to
+                # properly manage the free busy set that should not prevent us from working.
+                continue
+
+            matchtotal = (yield report_common.generateFreeBusyInfo(
+                self.scheduler.request,
+                calendarResource,
+                fbinfo,
+                self.scheduler.timeRange,
+                matchtotal,
+                excludeuid=self.scheduler.excludeUID,
+                organizer=self.scheduler.organizer.cuaddr,
+                organizerPrincipal=organizerPrincipal,
+                same_calendar_user=same_calendar_user,
+                servertoserver=remote,
+                event_details=event_details,
+            ))
+
+        # Build VFREEBUSY iTIP reply for this recipient
+        fbresult = report_common.buildFreeBusyResult(
+            fbinfo,
+            self.scheduler.timeRange,
+            organizer=organizerProp,
+            attendee=attendeeProp,
+            uid=uid,
+            method="REPLY",
+            event_details=event_details,
+        )
+
+        returnValue(fbresult)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/resource.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/resource.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/resource.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,511 @@
+# -*- test-case-name: twistedcaldav.directory.test.test_calendar -*-
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+CalDAV scheduling resources.
+"""
+
+__all__ = [
+    "ScheduleInboxResource",
+    "ScheduleOutboxResource",
+    "deliverSchedulePrivilegeSet",
+]
+
+
+from twext.web2 import responsecode
+from txdav.xml import element as davxml
+from txdav.xml.rfc2518 import HRef
+from twext.web2.dav.http import ErrorResponse, MultiStatusResponse
+from twext.web2.dav.resource import davPrivilegeSet
+from twext.web2.dav.util import joinURL, normalizeURL
+from twext.web2.http import HTTPError
+
+from twisted.internet.defer import inlineCallbacks, returnValue, succeed
+
+from twistedcaldav import caldavxml, customxml
+from twistedcaldav.caldavxml import caldav_namespace, Opaque, \
+    CalendarFreeBusySet, ScheduleCalendarTransp
+from twistedcaldav.config import config
+from twistedcaldav.customxml import calendarserver_namespace
+from twistedcaldav.ical import allowedComponents
+from twistedcaldav.resource import CalDAVResource
+from twistedcaldav.resource import isCalendarCollectionResource
+from txdav.caldav.datastore.scheduling.caldav.scheduler import CalDAVScheduler
+
+from txdav.base.propertystore.base import PropertyName
+
+def _schedulePrivilegeSet(deliver):
+    edited = False
+
+    top_supported_privileges = []
+
+    for supported_privilege in davPrivilegeSet.childrenOfType(davxml.SupportedPrivilege):
+        all_privilege = supported_privilege.childOfType(davxml.Privilege)
+        if isinstance(all_privilege.children[0], davxml.All):
+            all_description = supported_privilege.childOfType(davxml.Description)
+            all_supported_privileges = list(supported_privilege.childrenOfType(davxml.SupportedPrivilege))
+            all_supported_privileges.append(
+                davxml.SupportedPrivilege(
+                    davxml.Privilege(caldavxml.ScheduleDeliver() if deliver else caldavxml.ScheduleSend()),
+                    davxml.Description("schedule privileges for current principal", **{"xml:lang": "en"}),
+                ),
+            )
+            if config.Scheduling.CalDAV.OldDraftCompatibility:
+                all_supported_privileges.append(
+                    davxml.SupportedPrivilege(
+                        davxml.Privilege(caldavxml.Schedule()),
+                        davxml.Description("old-style schedule privileges for current principal", **{"xml:lang": "en"}),
+                    ),
+                )
+            top_supported_privileges.append(
+                davxml.SupportedPrivilege(all_privilege, all_description, *all_supported_privileges)
+            )
+            edited = True
+        else:
+            top_supported_privileges.append(supported_privilege)
+
+    assert edited, "Structure of davPrivilegeSet changed in a way that I don't know how to extend for schedulePrivilegeSet"
+
+    return davxml.SupportedPrivilegeSet(*top_supported_privileges)
+
+deliverSchedulePrivilegeSet = _schedulePrivilegeSet(True)
+sendSchedulePrivilegeSet = _schedulePrivilegeSet(False)
+
+class CalendarSchedulingCollectionResource (CalDAVResource):
+    """
+    CalDAV principal resource.
+
+    Extends L{DAVResource} to provide CalDAV scheduling collection
+    functionality.
+    """
+    def __init__(self, parent):
+        """
+        @param parent: the parent resource of this one.
+        """
+        assert parent is not None
+
+        super(CalendarSchedulingCollectionResource, self).__init__(principalCollections=parent.principalCollections())
+
+        self.parent = parent
+
+
+    def isCollection(self):
+        return True
+
+
+    def isCalendarCollection(self):
+        return False
+
+
+    def isPseudoCalendarCollection(self):
+        return True
+
+
+    def supportedReports(self):
+        result = super(CalDAVResource, self).supportedReports()
+        result.append(davxml.Report(caldavxml.CalendarQuery(),))
+        result.append(davxml.Report(caldavxml.CalendarMultiGet(),))
+        # free-busy report not allowed
+        if config.EnableSyncReport:
+            # Only allowed on calendar/inbox/addressbook collections
+            result.append(davxml.Report(davxml.SyncCollection(),))
+        return result
+
+
+
+class ScheduleInboxResource (CalendarSchedulingCollectionResource):
+    """
+    CalDAV schedule Inbox resource.
+
+    Extends L{DAVResource} to provide CalDAV functionality.
+    """
+
+    def liveProperties(self):
+
+        return super(ScheduleInboxResource, self).liveProperties() + (
+            caldavxml.CalendarFreeBusySet.qname(),
+            caldavxml.ScheduleDefaultCalendarURL.qname(),
+            customxml.ScheduleDefaultTasksURL.qname(),
+        )
+
+
+    def resourceType(self):
+        return davxml.ResourceType.scheduleInbox
+
+
+    @inlineCallbacks
+    def readProperty(self, property, request):
+        if type(property) is tuple:
+            qname = property
+        else:
+            qname = property.qname()
+
+        if qname == caldavxml.CalendarFreeBusySet.qname():
+            # Always return at least an empty list
+            if not self.hasDeadProperty(property):
+                top = self.parent.url()
+                values = []
+                for cal in (yield self.parent._newStoreHome.calendars()):
+                    prop = cal.properties().get(PropertyName.fromString(ScheduleCalendarTransp.sname()))
+                    if prop == ScheduleCalendarTransp(Opaque()):
+                        values.append(HRef(joinURL(top, cal.name())))
+                returnValue(CalendarFreeBusySet(*values))
+        elif qname in (caldavxml.ScheduleDefaultCalendarURL.qname(), customxml.ScheduleDefaultTasksURL.qname()):
+            result = (yield self.readDefaultCalendarProperty(request, qname))
+            returnValue(result)
+
+        result = (yield super(ScheduleInboxResource, self).readProperty(property, request))
+        returnValue(result)
+
+
+    @inlineCallbacks
+    def writeProperty(self, property, request):
+        assert isinstance(property, davxml.WebDAVElement)
+
+        # Strictly speaking CS:calendar-availability is a live property in the sense that the
+        # server enforces what can be stored, however it need not actually
+        # exist so we cannot list it in liveProperties on this resource, since its
+        # its presence there means that hasProperty will always return True for it.
+        if property.qname() == customxml.CalendarAvailability.qname():
+            if not property.valid():
+                raise HTTPError(ErrorResponse(
+                    responsecode.CONFLICT,
+                    (caldav_namespace, "valid-calendar-data"),
+                    description="Invalid property"
+                ))
+
+        elif property.qname() == caldavxml.CalendarFreeBusySet.qname():
+            # Verify that the calendars added in the PROPPATCH are valid. We do not check
+            # whether existing items in the property are still valid - only new ones.
+            property.children = [davxml.HRef(normalizeURL(str(href))) for href in property.children]
+            new_calendars = set([str(href) for href in property.children])
+            if not self.hasDeadProperty(property):
+                old_calendars = set()
+            else:
+                old_calendars = set([normalizeURL(str(href)) for href in self.readDeadProperty(property).children])
+            added_calendars = new_calendars.difference(old_calendars)
+            for href in added_calendars:
+                cal = (yield request.locateResource(str(href)))
+                if cal is None or not cal.exists() or not isCalendarCollectionResource(cal):
+                    # Validate that href's point to a valid calendar.
+                    raise HTTPError(ErrorResponse(
+                        responsecode.CONFLICT,
+                        (caldav_namespace, "valid-calendar-url"),
+                        "Invalid URI",
+                    ))
+            for href in tuple(new_calendars):
+                cal = (yield request.locateResource(str(href)))
+                if cal is None or not cal.exists() or not isCalendarCollectionResource(cal):
+                    new_calendars.remove(href)
+            property.children = [davxml.HRef(href) for href in new_calendars]
+
+        elif property.qname() in (caldavxml.ScheduleDefaultCalendarURL.qname(), customxml.ScheduleDefaultTasksURL.qname()):
+            property = (yield self.writeDefaultCalendarProperty(request, property))
+
+        yield super(ScheduleInboxResource, self).writeProperty(property, request)
+
+
+    def processFreeBusyCalendar(self, uri, addit):
+        uri = normalizeURL(uri)
+
+        if not self.hasDeadProperty(caldavxml.CalendarFreeBusySet.qname()):
+            fbset = set()
+        else:
+            fbset = set([normalizeURL(str(href)) for href in self.readDeadProperty(caldavxml.CalendarFreeBusySet.qname()).children])
+        if addit:
+            if uri not in fbset:
+                fbset.add(uri)
+                self.writeDeadProperty(caldavxml.CalendarFreeBusySet(*[davxml.HRef(url) for url in fbset]))
+        else:
+            if uri in fbset:
+                fbset.remove(uri)
+                self.writeDeadProperty(caldavxml.CalendarFreeBusySet(*[davxml.HRef(url) for url in fbset]))
+
+
+    @inlineCallbacks
+    def readDefaultCalendarProperty(self, request, qname):
+        """
+        Read either the default VEVENT or VTODO calendar property. Try to pick one if not present.
+        """
+
+        tasks = qname == customxml.ScheduleDefaultTasksURL.qname()
+
+        # Must have a valid default
+        try:
+            defaultCalendarProperty = self.readDeadProperty(qname)
+        except HTTPError:
+            defaultCalendarProperty = None
+        if defaultCalendarProperty and len(defaultCalendarProperty.children) == 1:
+            defaultCalendar = str(defaultCalendarProperty.children[0])
+            cal = (yield request.locateResource(str(defaultCalendar)))
+            if cal is not None and isCalendarCollectionResource(cal) and cal.exists() and not cal.isShareeCollection():
+                returnValue(defaultCalendarProperty)
+
+        # Default is not valid - we have to try to pick one
+        defaultCalendarProperty = (yield self.pickNewDefaultCalendar(request, tasks=tasks))
+        returnValue(defaultCalendarProperty)
+
+
+    @inlineCallbacks
+    def writeDefaultCalendarProperty(self, request, property):
+        """
+        Write either the default VEVENT or VTODO calendar property, validating and canonicalizing the value
+        """
+        tasks = property.qname() == customxml.ScheduleDefaultTasksURL
+        componentType = "VTODO" if tasks else "VEVENT"
+        prop_to_set = customxml.ScheduleDefaultTasksURL if tasks else caldavxml.ScheduleDefaultCalendarURL
+        error_element = (calendarserver_namespace, "valid-schedule-default-tasks-URL") if tasks else (caldav_namespace, "valid-schedule-default-calendar-URL")
+
+        # Verify that the calendar added in the PROPPATCH is valid.
+        property.children = [davxml.HRef(normalizeURL(str(href))) for href in property.children]
+        new_calendar = [str(href) for href in property.children]
+        cal = None
+        if len(new_calendar) == 1:
+            calURI = str(new_calendar[0])
+            cal = (yield request.locateResource(str(new_calendar[0])))
+
+        # TODO: check that owner of the new calendar is the same as owner of this inbox
+        if cal is None or not cal.exists() or not isCalendarCollectionResource(cal) or \
+            cal.isShareeCollection() or not cal.isSupportedComponent(componentType):
+            # Validate that href's point to a valid calendar.
+            raise HTTPError(ErrorResponse(
+                responsecode.CONFLICT,
+                error_element,
+                "Invalid URI",
+            ))
+        else:
+            # Canonicalize the URL to __uids__ form
+            calURI = (yield cal.canonicalURL(request))
+            property = prop_to_set(davxml.HRef(calURI))
+            returnValue(property)
+
+
+    @inlineCallbacks
+    def pickNewDefaultCalendar(self, request, tasks=False):
+        """
+        First see if default provisioned calendar exists in the calendar home and pick that. Otherwise
+        pick another from the calendar home.
+        """
+
+        componentType = "VTODO" if tasks else "VEVENT"
+        test_name = "tasks" if tasks else "calendar"
+        prop_to_set = customxml.ScheduleDefaultTasksURL if tasks else caldavxml.ScheduleDefaultCalendarURL
+
+        calendarHomeURL = self.parent.url()
+        defaultCalendarURL = joinURL(calendarHomeURL, test_name)
+        defaultCalendar = (yield request.locateResource(defaultCalendarURL))
+        if defaultCalendar is None or not defaultCalendar.exists():
+            # Really, the dead property shouldn't be necessary, and this should
+            # be entirely computed by a back-end method like 'defaultCalendar()'
+
+            @inlineCallbacks
+            def _findDefault():
+                for calendarName in (yield self.parent._newStoreHome.listCalendars()):  # These are only unshared children
+                    if calendarName == "inbox":
+                        continue
+                    calendar = (yield self.parent._newStoreHome.calendarWithName(calendarName))
+                    if not calendar.owned():
+                        continue
+                    if not calendar.isSupportedComponent(componentType):
+                        continue
+                    break
+                else:
+                    calendarName = None
+                returnValue(calendarName)
+
+            foundName = yield _findDefault()
+            if foundName is None:
+                # Create a default and try and get its name again
+                yield self.parent._newStoreHome.ensureDefaultCalendarsExist()
+                foundName = yield _findDefault()
+                if foundName is None:
+                    # Failed to even create a default - bad news...
+                    raise RuntimeError("No valid calendars to use as a default %s calendar." % (componentType,))
+
+            defaultCalendarURL = joinURL(calendarHomeURL, foundName)
+
+        prop = prop_to_set(davxml.HRef(defaultCalendarURL))
+        self.writeDeadProperty(prop)
+        returnValue(prop)
+
+
+    @inlineCallbacks
+    def defaultCalendar(self, request, componentType):
+        """
+        Find the default calendar for the supplied iCalendar component type. If one does
+        not exist, automatically provision it.
+        """
+
+        # Check any default calendar property first - this will create if none exists
+        default = (yield self.readProperty(caldavxml.ScheduleDefaultCalendarURL.qname(), request))
+        if len(default.children) == 1:
+            defaultURL = str(default.children[0])
+            default = (yield request.locateResource(defaultURL))
+        else:
+            default = None
+
+        # Check that default handles the component type
+        if default is not None:
+            if not default.isSupportedComponent(componentType):
+                default = None
+
+        # Must have a default - provision one if not
+        if default is None:
+
+            # Try to find a calendar supporting the required component type. If there are multiple, pick
+            # the one with the oldest created timestamp as that will likely be the initial provision.
+            for calendarName in (yield self.parent._newStoreHome.listCalendars()):  # These are only unshared children
+                if calendarName == "inbox":
+                    continue
+                calendar = (yield self.parent._newStoreHome.calendarWithName(calendarName))
+                if not calendar.isSupportedComponent(componentType):
+                    continue
+                if default is None or calendar.created() < default.created():
+                    default = calendar
+
+            # If none can be found, provision one
+            if default is None:
+                new_name = "%ss" % (componentType.lower()[1:],)
+                default = yield self.parent._newStoreHome.createCalendarWithName(new_name)
+                yield default.setSupportedComponents(componentType.upper())
+
+            # Need L{DAVResource} object to return not new store object
+            default = (yield request.locateResource(joinURL(self.parent.url(), default.name())))
+
+        returnValue(default)
+
+
+    @inlineCallbacks
+    def isDefaultCalendar(self, request, calendar):
+        """
+        Is the supplied calendar one of the possible default calendars.
+        """
+        assert calendar.isCalendarCollection()
+
+        # Not allowed to delete the default calendar
+        for default_prop in (caldavxml.ScheduleDefaultCalendarURL, customxml.ScheduleDefaultTasksURL,):
+            default = (yield self.readProperty(default_prop.qname(), request))
+            if default and len(default.children) == 1:
+                defaultURL = normalizeURL(str(default.children[0]))
+                myURL = (yield calendar.canonicalURL(request))
+                if defaultURL == myURL:
+                    returnValue(default_prop)
+
+        returnValue(None)
+
+
+    ##
+    # ACL
+    ##
+
+    def supportedPrivileges(self, request):
+        return succeed(deliverSchedulePrivilegeSet)
+
+
+    def defaultAccessControlList(self):
+
+        privs = (
+            davxml.Privilege(caldavxml.ScheduleDeliver()),
+        )
+        if config.Scheduling.CalDAV.OldDraftCompatibility:
+            privs += (davxml.Privilege(caldavxml.Schedule()),)
+
+        return davxml.ACL(
+            # CalDAV:schedule-deliver for any authenticated user
+            davxml.ACE(
+                davxml.Principal(davxml.Authenticated()),
+                davxml.Grant(*privs),
+            ),
+        )
+
+
+
+class ScheduleOutboxResource (CalendarSchedulingCollectionResource):
+    """
+    CalDAV schedule Outbox resource.
+
+    Extends L{DAVResource} to provide CalDAV functionality.
+    """
+
+    def resourceType(self):
+        return davxml.ResourceType.scheduleOutbox
+
+
+    def getSupportedComponentSet(self):
+        return caldavxml.SupportedCalendarComponentSet(
+            *[caldavxml.CalendarComponent(name=item) for item in allowedComponents]
+        )
+
+
+    @inlineCallbacks
+    def http_POST(self, request):
+        """
+        The CalDAV POST method.
+
+        This uses a generator function yielding either L{waitForDeferred} objects or L{Response} objects.
+        This allows for code that follows a 'linear' execution pattern rather than having to use nested
+        L{Deferred} callbacks. The logic is easier to follow this way plus we don't run into deep nesting
+        issues which the other approach would have with large numbers of recipients.
+        """
+        # Check authentication and access controls
+        yield self.authorize(request, (caldavxml.ScheduleSend(),))
+
+        # This is a local CALDAV scheduling operation.
+        scheduler = CalDAVScheduler(request, self)
+
+        # Do the POST processing treating
+        result = (yield scheduler.doSchedulingViaPOST(self._associatedTransaction))
+        returnValue(result.response())
+
+
+    ##
+    # ACL
+    ##
+
+    def supportedPrivileges(self, request):
+        return succeed(sendSchedulePrivilegeSet)
+
+
+    def defaultAccessControlList(self):
+        if config.EnableProxyPrincipals:
+            myPrincipal = self.parent.principalForRecord()
+
+            privs = (
+                davxml.Privilege(caldavxml.ScheduleSend()),
+            )
+            if config.Scheduling.CalDAV.OldDraftCompatibility:
+                privs += (davxml.Privilege(caldavxml.Schedule()),)
+
+            return davxml.ACL(
+                # CalDAV:schedule for associated write proxies
+                davxml.ACE(
+                    davxml.Principal(davxml.HRef(joinURL(myPrincipal.principalURL(), "calendar-proxy-write"))),
+                    davxml.Grant(*privs),
+                    davxml.Protected(),
+                ),
+            )
+        else:
+            return super(ScheduleOutboxResource, self).defaultAccessControlList()
+
+
+    def report_urn_ietf_params_xml_ns_caldav_calendar_query(self, request, calendar_query):
+        return succeed(MultiStatusResponse(()))
+
+
+    def report_urn_ietf_params_xml_ns_caldav_calendar_multiget(self, request, multiget):
+        responses = [davxml.StatusResponse(href, davxml.Status.fromResponseCode(responsecode.NOT_FOUND)) for href in multiget.resources]
+        return succeed(MultiStatusResponse((responses)))

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/scheduler.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/scheduler.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/scheduler.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,282 @@
+##
+# Copyright (c) 2012-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.python.log import Logger
+from twext.web2 import responsecode
+from twext.web2.dav.http import ErrorResponse
+from twext.web2.http import HTTPError, StatusResponse
+
+from twisted.internet.defer import inlineCallbacks
+
+from twistedcaldav.caldavxml import caldav_namespace
+from twistedcaldav.config import config
+
+from txdav.caldav.datastore.scheduling import addressmapping
+from txdav.caldav.datastore.scheduling.cuaddress import LocalCalendarUser, \
+    InvalidCalendarUser, calendarUserFromPrincipal, RemoteCalendarUser
+from txdav.caldav.datastore.scheduling.scheduler import Scheduler, ScheduleResponseQueue
+
+from txdav.xml import element as davxml
+
+
+"""
+L{CalDAVScheduler} - handles deliveries for scheduling messages within the CalDAV server.
+"""
+
+__all__ = [
+    "CalDAVScheduler",
+]
+
+
+log = Logger()
+
+class CalDAVScheduler(Scheduler):
+
+    scheduleResponse = ScheduleResponseQueue
+
+    errorResponse = ErrorResponse
+
+    errorElements = {
+        "originator-missing": (caldav_namespace, "originator-specified"),
+        "originator-invalid": (caldav_namespace, "originator-allowed"),
+        "originator-denied": (caldav_namespace, "originator-allowed"),
+        "recipient-missing": (caldav_namespace, "recipient-specified"),
+        "recipient-invalid": (caldav_namespace, "recipient-exists"),
+        "organizer-denied": (caldav_namespace, "organizer-allowed"),
+        "attendee-denied": (caldav_namespace, "attendee-allowed"),
+        "invalid-calendar-data-type": (caldav_namespace, "supported-calendar-data"),
+        "invalid-calendar-data": (caldav_namespace, "valid-calendar-data"),
+        "invalid-scheduling-message": (caldav_namespace, "valid-calendar-data"),
+        "max-recipients": (caldav_namespace, "recipient-limit"),
+    }
+
+    def __init__(self, calendar_home, calendar_collection, resource, **kwargs):
+        super(CalDAVScheduler, self).__init__(calendar_home, calendar_collection, resource, **kwargs)
+        self.doingPOST = False
+
+
+    def doSchedulingViaPOST(self, transaction):
+        """
+        The Scheduling POST operation on an Outbox.
+        """
+        self.doingPOST = True
+        return super(CalDAVScheduler, self).doSchedulingViaPOST(transaction)
+
+
+    def checkAuthorization(self):
+        # Must have an authenticated user
+        if not self.internal_request and self.resource.currentPrincipal(self.request) == davxml.Principal(davxml.Unauthenticated()):
+            log.err("Unauthenticated originators not allowed: %s" % (self.originator,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["originator-denied"],
+                "Invalid originator",
+            ))
+
+
+    def checkOriginator(self):
+        """
+        Check the validity of the Originator header. Extract the corresponding principal.
+        """
+
+        # Verify that Originator is a valid calendar user
+        originatorPrincipal = self.calendar_home.principalForCalendarUserAddress(self.originator)
+        if originatorPrincipal is None:
+            # Local requests MUST have a principal.
+            log.err("Could not find principal for originator: %s" % (self.originator,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["originator-denied"],
+                "No principal for originator",
+            ))
+        else:
+            self.originator = LocalCalendarUser(self.originator, originatorPrincipal)
+
+
+    @inlineCallbacks
+    def checkRecipients(self):
+        """
+        Check the validity of the Recipient header values. Map these into local or
+        remote CalendarUsers.
+        """
+
+        results = []
+        for recipient in self.recipients:
+            # Get the principal resource for this recipient
+            principal = self.calendar_home.principalForCalendarUserAddress(recipient)
+
+            # If no principal we may have a remote recipient but we should check whether
+            # the address is one that ought to be on our server and treat that as a missing
+            # user. Also if server-to-server is not enabled then remote addresses are not allowed.
+            if principal is None:
+                address = (yield addressmapping.mapper.getCalendarUser(recipient, principal))
+                if isinstance(address, InvalidCalendarUser):
+                    log.err("Unknown calendar user address: %s" % (recipient,))
+                results.append(address)
+            else:
+                # Map recipient to their inbox
+                inbox = None
+                if principal.thisServer():
+                    if principal.locallyHosted():
+                        recipient_home = yield self.calendar_home.transaction().calendarHomeWithUID(principal.uid())
+                        if recipient_home:
+                            inbox = (yield recipient_home.calendarWithName("inbox"))
+                    else:
+                        inbox = "dummy"
+
+                if inbox:
+                    results.append(calendarUserFromPrincipal(recipient, principal, inbox))
+                else:
+                    log.err("No schedule inbox for principal: %s" % (principal,))
+                    results.append(InvalidCalendarUser(recipient))
+
+        self.recipients = results
+
+
+    @inlineCallbacks
+    def checkOrganizer(self):
+        """
+        Check the validity of the ORGANIZER value. ORGANIZER must be local.
+        """
+
+        # Verify that the ORGANIZER's cu address maps to a valid user
+        organizer = self.calendar.getOrganizer()
+        if organizer:
+            organizerPrincipal = self.calendar_home.principalForCalendarUserAddress(organizer)
+            if organizerPrincipal:
+                if organizerPrincipal.calendarsEnabled():
+
+                    # Only do this check for a freebusy request. A check for an invite needs
+                    # to be handled later when we know whether a new invite is being added
+                    # (which we reject) vs an update to an existing one (which we allow).
+                    if self.checkForFreeBusy() and not organizerPrincipal.enabledAsOrganizer():
+                        log.err("ORGANIZER not allowed to be an Organizer: %s" % (self.calendar,))
+                        raise HTTPError(self.errorResponse(
+                            responsecode.FORBIDDEN,
+                            self.errorElements["organizer-denied"],
+                            "Organizer cannot schedule",
+                        ))
+
+                    self.organizer = LocalCalendarUser(organizer, organizerPrincipal)
+                else:
+                    log.err("No outbox for ORGANIZER in calendar data: %s" % (self.calendar,))
+                    raise HTTPError(self.errorResponse(
+                        responsecode.FORBIDDEN,
+                        self.errorElements["organizer-denied"],
+                        "Organizer cannot schedule",
+                    ))
+            else:
+                localUser = (yield addressmapping.mapper.isCalendarUserInMyDomain(organizer))
+                if localUser:
+                    log.err("No principal for ORGANIZER in calendar data: %s" % (self.calendar,))
+                    raise HTTPError(self.errorResponse(
+                        responsecode.FORBIDDEN,
+                        self.errorElements["organizer-denied"],
+                        "No principal for organizer",
+                    ))
+                else:
+                    self.organizer = RemoteCalendarUser(organizer)
+        else:
+            log.err("ORGANIZER missing in calendar data: %s" % (self.calendar,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["invalid-scheduling-message"],
+                "Missing organizer",
+            ))
+
+
+    def checkOrganizerAsOriginator(self):
+
+        # Make sure that the ORGANIZER is local
+        if not isinstance(self.organizer, LocalCalendarUser):
+            log.err("ORGANIZER is not local to server in calendar data: %s" % (self.calendar,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["organizer-denied"],
+                "Organizer is not local to server",
+            ))
+
+        # Make sure that the ORGANIZER's Outbox is the request URI
+        if self.doingPOST and self.organizer.principal.uid() != self.calendar_home.uid():
+            log.err("Wrong outbox for ORGANIZER in calendar data: %s" % (self.calendar,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["organizer-denied"],
+                "Outbox does not belong to organizer",
+            ))
+
+
+    def checkAttendeeAsOriginator(self):
+        """
+        Check the validity of the ATTENDEE value as this is the originator of the iTIP message.
+        Only local attendees are allowed for message originating from this server.
+        """
+
+        # Attendee's Outbox MUST be the request URI
+        attendeePrincipal = self.calendar_home.principalForCalendarUserAddress(self.attendee)
+        if attendeePrincipal:
+            if self.doingPOST and attendeePrincipal.uid() != self.calendar_home.uid():
+                log.err("ATTENDEE in calendar data does not match owner of Outbox: %s" % (self.calendar,))
+                raise HTTPError(self.errorResponse(
+                    responsecode.FORBIDDEN,
+                    self.errorElements["attendee-denied"],
+                    "Outbox does not belong to attendee",
+                ))
+        else:
+            log.err("Unknown ATTENDEE in calendar data: %s" % (self.calendar,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["attendee-denied"],
+                "No principal for attendee",
+            ))
+
+
+    def securityChecks(self):
+        """
+        Check that the originator has the appropriate rights to send this type of iTIP message.
+        """
+
+        # Prevent spoofing of ORGANIZER with specific METHODs when local
+        if self.isiTIPRequest:
+            self.checkOrganizerAsOriginator()
+
+        # Prevent spoofing when doing reply-like METHODs
+        else:
+            self.checkAttendeeAsOriginator()
+
+
+    def finalChecks(self):
+        """
+        Final checks before doing the actual scheduling.
+        """
+
+        # With implicit scheduling only certain types of iTIP operations are allowed for POST.
+
+        if self.doingPOST:
+            # Freebusy requests always processed
+            if self.checkForFreeBusy():
+                return
+
+            # COUNTER and DECLINE-COUNTER allowed
+            if self.calendar.propertyValue("METHOD") in ("COUNTER", "DECLINECOUNTER"):
+                return
+
+            # Anything else is not allowed. However, for compatibility we will optionally
+            # return a success response for all attendees.
+            if config.Scheduling.CalDAV.OldDraftCompatibility:
+                self.fakeTheResult = True
+            else:
+                raise HTTPError(StatusResponse(responsecode.BAD_REQUEST, "Invalid iTIP message for implicit scheduling"))

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/test/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/test/__init__.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/test/__init__.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/test/test_delivery.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/test/test_delivery.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/test/test_delivery.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,41 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twisted.internet.defer import inlineCallbacks
+
+from twistedcaldav.config import config
+
+from txdav.caldav.datastore.scheduling.caldav.delivery import ScheduleViaCalDAV
+
+import twistedcaldav.test.util
+
+class CalDAV (twistedcaldav.test.util.TestCase):
+    """
+    twistedcaldav.scheduling.caldav tests
+    """
+
+    @inlineCallbacks
+    def test_matchCalendarUserAddress(self):
+        """
+        Make sure we do an exact comparison on EmailDomain
+        """
+        self.patch(config.Scheduling[ScheduleViaCalDAV.serviceType()], "EmailDomain", "example.com")
+        result = yield ScheduleViaCalDAV.matchCalendarUserAddress("mailto:user at example.com")
+        self.assertTrue(result)
+        result = yield ScheduleViaCalDAV.matchCalendarUserAddress("mailto:user at foo.example.com")
+        self.assertFalse(result)
+        result = yield ScheduleViaCalDAV.matchCalendarUserAddress("mailto:user at xyzexample.com")
+        self.assertFalse(result)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/test/test_resource.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/test/test_resource.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/caldav/test/test_resource.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,457 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.web2 import responsecode, http_headers
+from twext.web2.dav.util import davXMLFromStream
+from twext.web2.http import HTTPError
+from twext.web2.iweb import IResponse
+from twext.web2.stream import MemoryStream
+from twext.web2.test.test_server import SimpleRequest
+from twisted.internet.defer import inlineCallbacks
+from twistedcaldav import caldavxml, customxml
+from twistedcaldav.test.util import HomeTestCase, TestCase
+from txdav.xml import element as davxml
+
+class Properties (HomeTestCase):
+    """
+    CalDAV properties
+    """
+    def test_free_busy_set_prop(self):
+        """
+        Test for PROPFIND on Inbox with missing calendar-free-busy-set property.
+        """
+
+        inbox_uri = "/inbox/"
+
+        def propfind_cb(response):
+            response = IResponse(response)
+
+            if response.code != responsecode.MULTI_STATUS:
+                self.fail("Incorrect response to PROPFIND: %s" % (response.code,))
+
+            def got_xml(doc):
+                if not isinstance(doc.root_element, davxml.MultiStatus):
+                    self.fail("PROPFIND response XML root element is not multistatus: %r" % (doc.root_element,))
+
+                response = doc.root_element.childOfType(davxml.Response)
+                href = response.childOfType(davxml.HRef)
+                self.failUnless(str(href) == inbox_uri)
+
+                for propstat in response.childrenOfType(davxml.PropertyStatus):
+                    status = propstat.childOfType(davxml.Status)
+                    if status.code != responsecode.OK:
+                        self.fail("Unable to read requested properties (%s): %r"
+                                  % (status, propstat.childOfType(davxml.PropertyContainer).toxml()))
+
+                container = propstat.childOfType(davxml.PropertyContainer)
+
+                #
+                # Check CalDAV:calendar-free-busy-set
+                #
+
+                free_busy_set = container.childOfType(caldavxml.CalendarFreeBusySet)
+                if not free_busy_set:
+                    self.fail("Expected CalDAV:calendar-free-busy-set element; but got none.")
+
+                if not free_busy_set.children:
+                    self.fail("Expected non-empty CalDAV:calendar-free-busy-set element.")
+
+            return davXMLFromStream(response.stream).addCallback(got_xml)
+
+        query = davxml.PropertyFind(
+                    davxml.PropertyContainer(
+                        caldavxml.CalendarFreeBusySet(),
+                    ),
+                )
+
+        request = SimpleRequest(
+            self.site,
+            "PROPFIND",
+            inbox_uri,
+            headers=http_headers.Headers({"Depth": "0"}),
+        )
+        request.stream = MemoryStream(query.toxml())
+        return self.send(request, propfind_cb)
+
+
+    @inlineCallbacks
+    def test_free_busy_set_remove_broken(self):
+        """
+        ???
+        """
+
+        request = SimpleRequest(self.site, "GET", "/inbox/")
+        inbox = yield request.locateResource("/inbox/")
+        self.assertTrue(inbox.hasDeadProperty(caldavxml.CalendarFreeBusySet))
+        oldfbset = set(("/calendar",))
+        oldset = caldavxml.CalendarFreeBusySet(*[davxml.HRef(url) for url in oldfbset])
+
+        newfbset = set()
+        newfbset.update(oldfbset)
+        newfbset.add("/calendar-broken")
+        newset = caldavxml.CalendarFreeBusySet(*[davxml.HRef(url) for url in newfbset])
+
+        inbox.writeDeadProperty(newset)
+        changedset = inbox.readDeadProperty(caldavxml.CalendarFreeBusySet)
+        self.assertEqual(tuple(changedset.children), tuple(newset.children))
+
+        yield inbox.writeProperty(newset, request)
+
+        changedset = inbox.readDeadProperty(caldavxml.CalendarFreeBusySet)
+        self.assertEqual(tuple(changedset.children), tuple(oldset.children))
+
+
+    @inlineCallbacks
+    def test_free_busy_set_strip_slash(self):
+        """
+        ???
+        """
+
+        request = SimpleRequest(self.site, "GET", "/inbox/")
+        inbox = yield request.locateResource("/inbox/")
+        self.assertTrue(inbox.hasDeadProperty(caldavxml.CalendarFreeBusySet))
+
+        oldfbset = set(("/calendar/",))
+        oldset = caldavxml.CalendarFreeBusySet(*[davxml.HRef(url) for url in oldfbset])
+        inbox.writeDeadProperty(oldset)
+
+        writefbset = set(("/calendar/",))
+        writeset = caldavxml.CalendarFreeBusySet(*[davxml.HRef(url) for url in writefbset])
+        yield inbox.writeProperty(writeset, request)
+
+        correctfbset = set(("/calendar",))
+        correctset = caldavxml.CalendarFreeBusySet(*[davxml.HRef(url) for url in correctfbset])
+        changedset = inbox.readDeadProperty(caldavxml.CalendarFreeBusySet)
+        self.assertEqual(tuple(changedset.children), tuple(correctset.children))
+
+
+    @inlineCallbacks
+    def test_free_busy_set_strip_slash_remove(self):
+        """
+        ???
+        """
+
+        request = SimpleRequest(self.site, "GET", "/inbox/")
+        inbox = yield request.locateResource("/inbox/")
+        self.assertTrue(inbox.hasDeadProperty(caldavxml.CalendarFreeBusySet))
+
+        oldfbset = set(("/calendar/", "/broken/"))
+        oldset = caldavxml.CalendarFreeBusySet(*[davxml.HRef(url) for url in oldfbset])
+        inbox.writeDeadProperty(oldset)
+
+        writefbset = set(("/calendar/", "/broken/"))
+        writeset = caldavxml.CalendarFreeBusySet(*[davxml.HRef(url) for url in writefbset])
+        yield inbox.writeProperty(writeset, request)
+
+        correctfbset = set(("/calendar",))
+        correctset = caldavxml.CalendarFreeBusySet(*[davxml.HRef(url) for url in correctfbset])
+        changedset = inbox.readDeadProperty(caldavxml.CalendarFreeBusySet)
+        self.assertEqual(tuple(changedset.children), tuple(correctset.children))
+
+
+
+class DefaultCalendar (TestCase):
+
+    def setUp(self):
+        super(DefaultCalendar, self).setUp()
+        self.createStockDirectoryService()
+        self.setupCalendars()
+
+
+    @inlineCallbacks
+    def test_pick_default_vevent_calendar(self):
+        """
+        Test that pickNewDefaultCalendar will choose the correct calendar.
+        """
+
+        request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/")
+        inbox = yield request.locateResource("/calendars/users/wsanchez/inbox")
+
+        # default property initially not present
+        try:
+            inbox.readDeadProperty(caldavxml.ScheduleDefaultCalendarURL)
+        except HTTPError:
+            pass
+        else:
+            self.fail("caldavxml.ScheduleDefaultCalendarURL is not empty")
+
+        yield inbox.pickNewDefaultCalendar(request)
+
+        try:
+            default = inbox.readDeadProperty(caldavxml.ScheduleDefaultCalendarURL)
+        except HTTPError:
+            self.fail("caldavxml.ScheduleDefaultCalendarURL is not present")
+        else:
+            self.assertEqual(str(default.children[0]), "/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/calendar")
+
+        request._newStoreTransaction.abort()
+
+
+    @inlineCallbacks
+    def test_pick_default_vtodo_calendar(self):
+        """
+        Test that pickNewDefaultCalendar will choose the correct tasks calendar.
+        """
+
+        request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/")
+        inbox = yield request.locateResource("/calendars/users/wsanchez/inbox")
+
+        # default property initially not present
+        try:
+            inbox.readDeadProperty(customxml.ScheduleDefaultTasksURL)
+        except HTTPError:
+            pass
+        else:
+            self.fail("customxml.ScheduleDefaultTasksURL is not empty")
+
+        yield inbox.pickNewDefaultCalendar(request, tasks=True)
+
+        try:
+            default = inbox.readDeadProperty(customxml.ScheduleDefaultTasksURL)
+        except HTTPError:
+            self.fail("customxml.ScheduleDefaultTasksURL is not present")
+        else:
+            self.assertEqual(str(default.children[0]), "/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/tasks")
+
+        request._newStoreTransaction.abort()
+
+
+    @inlineCallbacks
+    def test_missing_default_vevent_calendar(self):
+        """
+        Test that pickNewDefaultCalendar will create a missing default calendar.
+        """
+
+        request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/")
+        home = yield request.locateResource("/calendars/users/wsanchez/")
+        inbox = yield request.locateResource("/calendars/users/wsanchez/inbox")
+
+        # default property initially not present
+        try:
+            inbox.readDeadProperty(caldavxml.ScheduleDefaultCalendarURL)
+        except HTTPError:
+            pass
+        else:
+            self.fail("caldavxml.ScheduleDefaultCalendarURL is not empty")
+
+        # Forcibly remove the one we need
+        yield home._newStoreHome.removeChildWithName("calendar")
+        names = [calendarName for calendarName in (yield home._newStoreHome.listCalendars())]
+        self.assertTrue("calendar" not in names)
+
+        yield inbox.pickNewDefaultCalendar(request)
+
+        try:
+            default = inbox.readDeadProperty(caldavxml.ScheduleDefaultCalendarURL)
+        except HTTPError:
+            self.fail("caldavxml.ScheduleDefaultCalendarURL is not present")
+        else:
+            self.assertEqual(str(default.children[0]), "/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/calendar")
+
+        request._newStoreTransaction.abort()
+
+
+    @inlineCallbacks
+    def test_missing_default_vtodo_calendar(self):
+        """
+        Test that pickNewDefaultCalendar will create a missing default tasks calendar.
+        """
+
+        request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/")
+        home = yield request.locateResource("/calendars/users/wsanchez/")
+        inbox = yield request.locateResource("/calendars/users/wsanchez/inbox")
+
+        # default property initially not present
+        try:
+            inbox.readDeadProperty(customxml.ScheduleDefaultTasksURL)
+        except HTTPError:
+            pass
+        else:
+            self.fail("caldavxml.ScheduleDefaultTasksURL is not empty")
+
+        # Forcibly remove the one we need
+        yield home._newStoreHome.removeChildWithName("tasks")
+        names = [calendarName for calendarName in (yield home._newStoreHome.listCalendars())]
+        self.assertTrue("tasks" not in names)
+
+        yield inbox.pickNewDefaultCalendar(request, tasks=True)
+
+        try:
+            default = inbox.readDeadProperty(customxml.ScheduleDefaultTasksURL)
+        except HTTPError:
+            self.fail("caldavxml.ScheduleDefaultTasksURL is not present")
+        else:
+            self.assertEqual(str(default.children[0]), "/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/tasks")
+
+        request._newStoreTransaction.abort()
+
+
+    @inlineCallbacks
+    def test_pick_default_other(self):
+        """
+        Make calendar
+        """
+
+        request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/")
+        inbox = yield request.locateResource("/calendars/users/wsanchez/inbox")
+
+        # default property not present
+        try:
+            inbox.readDeadProperty(caldavxml.ScheduleDefaultCalendarURL)
+        except HTTPError:
+            pass
+        else:
+            self.fail("caldavxml.ScheduleDefaultCalendarURL is not empty")
+
+        # Create a new default calendar
+        newcalendar = yield request.locateResource("/calendars/users/wsanchez/newcalendar")
+        yield newcalendar.createCalendarCollection()
+        inbox.writeDeadProperty(caldavxml.ScheduleDefaultCalendarURL(
+            davxml.HRef("/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/newcalendar")
+        ))
+
+        # Delete the normal calendar
+        calendar = yield request.locateResource("/calendars/users/wsanchez/calendar")
+        yield calendar.storeRemove(request, False, "/calendars/users/wsanchez/calendar")
+
+        inbox.removeDeadProperty(caldavxml.ScheduleDefaultCalendarURL)
+
+        # default property not present
+        try:
+            inbox.readDeadProperty(caldavxml.ScheduleDefaultCalendarURL)
+        except HTTPError:
+            pass
+        else:
+            self.fail("caldavxml.ScheduleDefaultCalendarURL is not empty")
+        request._newStoreTransaction.commit()
+
+        request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/")
+        inbox = yield request.locateResource("/calendars/users/wsanchez/inbox")
+        yield inbox.pickNewDefaultCalendar(request)
+
+        try:
+            default = inbox.readDeadProperty(caldavxml.ScheduleDefaultCalendarURL)
+        except HTTPError:
+            self.fail("caldavxml.ScheduleDefaultCalendarURL is not present")
+        else:
+            self.assertEqual(str(default.children[0]), "/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/newcalendar")
+
+        request._newStoreTransaction.abort()
+
+
+    @inlineCallbacks
+    def test_fix_shared_default(self):
+        """
+        Make calendar
+        """
+
+        request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/")
+        inbox = yield request.locateResource("/calendars/users/wsanchez/inbox")
+
+        # Create a new default calendar
+        newcalendar = yield request.locateResource("/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/newcalendar")
+        yield newcalendar.createCalendarCollection()
+        inbox.writeDeadProperty(caldavxml.ScheduleDefaultCalendarURL(
+            davxml.HRef("/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/newcalendar")
+        ))
+        try:
+            default = yield inbox.readProperty(caldavxml.ScheduleDefaultCalendarURL, request)
+        except HTTPError:
+            self.fail("caldavxml.ScheduleDefaultCalendarURL is not present")
+        else:
+            self.assertEqual(str(default.children[0]), "/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/newcalendar")
+
+        # Force the new calendar to think it is a virtual share
+        newcalendar._isShareeCollection = True
+
+        try:
+            default = yield inbox.readProperty(caldavxml.ScheduleDefaultCalendarURL, request)
+        except HTTPError:
+            self.fail("caldavxml.ScheduleDefaultCalendarURL is not present")
+        else:
+            self.assertEqual(str(default.children[0]), "/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/calendar")
+
+        request._newStoreTransaction.abort()
+
+
+    @inlineCallbacks
+    def test_set_default_vevent_other(self):
+        """
+        Test that the default URL can be set to another VEVENT calendar
+        """
+
+        request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/")
+        inbox = yield request.locateResource("/calendars/users/wsanchez/inbox")
+
+        # default property not present
+        try:
+            inbox.readDeadProperty(caldavxml.ScheduleDefaultCalendarURL)
+        except HTTPError:
+            pass
+        else:
+            self.fail("caldavxml.ScheduleDefaultCalendarURL is not empty")
+
+        # Create a new default calendar
+        newcalendar = yield request.locateResource("/calendars/users/wsanchez/newcalendar")
+        yield newcalendar.createCalendarCollection()
+        yield newcalendar.setSupportedComponents(("VEVENT",))
+        request._newStoreTransaction.commit()
+
+        request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/")
+        inbox = yield request.locateResource("/calendars/users/wsanchez/inbox")
+        yield inbox.writeProperty(caldavxml.ScheduleDefaultCalendarURL(davxml.HRef("/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/newcalendar")), request)
+
+        try:
+            default = inbox.readDeadProperty(caldavxml.ScheduleDefaultCalendarURL)
+        except HTTPError:
+            self.fail("caldavxml.ScheduleDefaultCalendarURL is not present")
+        else:
+            self.assertEqual(str(default.children[0]), "/calendars/__uids__/6423F94A-6B76-4A3A-815B-D52CFD77935D/newcalendar")
+
+        request._newStoreTransaction.commit()
+
+
+    @inlineCallbacks
+    def test_is_default_calendar(self):
+        """
+        Test .isDefaultCalendar() returns the proper class or None.
+        """
+
+        # Create a new non-default calendar
+        request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/")
+        newcalendar = yield request.locateResource("/calendars/users/wsanchez/newcalendar")
+        yield newcalendar.createCalendarCollection()
+        yield newcalendar.setSupportedComponents(("VEVENT",))
+        inbox = yield request.locateResource("/calendars/users/wsanchez/inbox")
+        yield inbox.pickNewDefaultCalendar(request)
+        request._newStoreTransaction.commit()
+
+        request = SimpleRequest(self.site, "GET", "/calendars/users/wsanchez/")
+        inbox = yield request.locateResource("/calendars/users/wsanchez/inbox")
+        calendar = yield request.locateResource("/calendars/users/wsanchez/calendar")
+        newcalendar = yield request.locateResource("/calendars/users/wsanchez/newcalendar")
+        tasks = yield request.locateResource("/calendars/users/wsanchez/tasks")
+
+        result = yield inbox.isDefaultCalendar(request, calendar)
+        self.assertEqual(result, caldavxml.ScheduleDefaultCalendarURL)
+
+        result = yield inbox.isDefaultCalendar(request, newcalendar)
+        self.assertEqual(result, None)
+
+        result = yield inbox.isDefaultCalendar(request, tasks)
+        self.assertEqual(result, customxml.ScheduleDefaultTasksURL)
+
+        request._newStoreTransaction.commit()

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/cuaddress.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/cuaddress.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/cuaddress.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,153 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.python.log import Logger
+
+from txdav.caldav.datastore.scheduling.delivery import DeliveryService
+
+__all__ = [
+    "LocalCalendarUser",
+    "PartitionedCalendarUser",
+    "OtherServerCalendarUser",
+    "RemoteCalendarUser",
+    "EmailCalendarUser",
+    "InvalidCalendarUser",
+    "normalizeCUAddr",
+]
+
+log = Logger()
+
+class CalendarUser(object):
+
+    def __init__(self, cuaddr):
+        self.cuaddr = cuaddr
+        self.serviceType = None
+
+
+
+class LocalCalendarUser(CalendarUser):
+
+    def __init__(self, cuaddr, principal, inbox=None):
+        self.cuaddr = cuaddr
+        self.principal = principal
+        self.inbox = inbox
+        self.serviceType = DeliveryService.serviceType_caldav
+
+
+    def __str__(self):
+        return "Local calendar user: %s" % (self.cuaddr,)
+
+
+
+class PartitionedCalendarUser(CalendarUser):
+
+    def __init__(self, cuaddr, principal):
+        self.cuaddr = cuaddr
+        self.principal = principal
+        self.serviceType = DeliveryService.serviceType_ischedule
+
+
+    def __str__(self):
+        return "Partitioned calendar user: %s" % (self.cuaddr,)
+
+
+
+class OtherServerCalendarUser(CalendarUser):
+
+    def __init__(self, cuaddr, principal):
+        self.cuaddr = cuaddr
+        self.principal = principal
+        self.serviceType = DeliveryService.serviceType_ischedule
+
+
+    def __str__(self):
+        return "Other server calendar user: %s" % (self.cuaddr,)
+
+
+
+class RemoteCalendarUser(CalendarUser):
+
+    def __init__(self, cuaddr):
+        self.cuaddr = cuaddr
+        self.extractDomain()
+        self.serviceType = DeliveryService.serviceType_ischedule
+
+
+    def __str__(self):
+        return "Remote calendar user: %s" % (self.cuaddr,)
+
+
+    def extractDomain(self):
+        if self.cuaddr.startswith("mailto:"):
+            splits = self.cuaddr[7:].split("?")
+            self.domain = splits[0].split("@")[1]
+        elif self.cuaddr.startswith("http://") or self.cuaddr.startswith("https://"):
+            splits = self.cuaddr.split(":")[1][2:].split("/")
+            self.domain = splits[0]
+        else:
+            self.domain = ""
+
+
+
+class EmailCalendarUser(CalendarUser):
+
+    def __init__(self, cuaddr):
+        self.cuaddr = cuaddr
+        self.serviceType = DeliveryService.serviceType_imip
+
+
+    def __str__(self):
+        return "Email/iMIP calendar user: %s" % (self.cuaddr,)
+
+
+
+class InvalidCalendarUser(CalendarUser):
+
+    def __str__(self):
+        return "Invalid calendar user: %s" % (self.cuaddr,)
+
+
+
+def normalizeCUAddr(addr):
+    """
+    Normalize a cuaddr string by lower()ing it if it's a mailto:, or
+    removing trailing slash if it's a URL.
+    @param addr: a cuaddr string to normalize
+    @return: normalized string
+    """
+    lower = addr.lower()
+    if lower.startswith("mailto:"):
+        addr = lower
+    if (addr.startswith("/") or
+        addr.startswith("http:") or
+        addr.startswith("https:")):
+        return addr.rstrip("/")
+    else:
+        return addr
+
+
+
+def calendarUserFromPrincipal(recipient, principal, inbox=None):
+    """
+    Get the appropriate calendar user address class for the provided principal.
+    """
+
+    if principal.locallyHosted():
+        return LocalCalendarUser(recipient, principal, inbox)
+    elif principal.thisServer():
+        return PartitionedCalendarUser(recipient, principal)
+    else:
+        return OtherServerCalendarUser(recipient, principal)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/delivery.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/delivery.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/delivery.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,78 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+import re
+
+from twext.python.log import Logger
+
+from twistedcaldav.config import config
+from twisted.internet.defer import succeed
+
+__all__ = [
+    "DeliveryService",
+]
+
+log = Logger()
+
+class DeliveryService(object):
+    """
+    Abstract base class that defines a delivery method for a scheduling message.
+    """
+
+    # Known types
+
+    serviceType_caldav = 'CalDAV'
+    serviceType_ischedule = 'iSchedule'
+    serviceType_imip = 'iMIP'
+
+    def __init__(self, scheduler, recipients, responses, freebusy):
+
+        self.scheduler = scheduler
+        self.recipients = recipients
+        self.responses = responses
+        self.freebusy = freebusy
+
+
+    @classmethod
+    def serviceType(cls):
+        raise NotImplementedError
+
+
+    @classmethod
+    def matchCalendarUserAddress(cls, cuaddr):
+        """
+        Determine whether the delivery service is able to handle the specified calendar user address.
+
+        @param cuaddr: calendar user address to test
+        @type cuaddr: C{str}
+
+        @return: L{Deferred} with result C{True} or C{False}
+        """
+
+        cuaddr = cuaddr.lower()
+        # Do the pattern match
+        for pattern in config.Scheduling[cls.serviceType()]["AddressPatterns"]:
+            try:
+                if re.match(pattern, cuaddr) is not None:
+                    return succeed(True)
+            except re.error:
+                log.error("Invalid regular expression for Scheduling configuration '%s/LocalAddresses': %s" % (cls.serviceType(), pattern,))
+
+        return succeed(False)
+
+
+    def generateSchedulingResponses(self):
+        raise NotImplementedError

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/icaldiff.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/icaldiff.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/icaldiff.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,818 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from difflib import unified_diff
+
+from pycalendar.datetime import PyCalendarDateTime
+from pycalendar.period import PyCalendarPeriod
+
+from twext.python.log import Logger
+
+from twistedcaldav import accounting
+from twistedcaldav.config import config
+from twistedcaldav.ical import Component, Property
+from txdav.caldav.datastore.scheduling.cuaddress import normalizeCUAddr
+from txdav.caldav.datastore.scheduling.itip import iTipGenerator
+
+"""
+Class that handles diff'ing two calendar objects.
+"""
+
+__all__ = [
+    "iCalDiff",
+]
+
+log = Logger()
+
+class iCalDiff(object):
+
+    def __init__(self, oldcalendar, newcalendar, smart_merge):
+        """
+
+        @param oldcalendar:
+        @type oldcalendar:
+        @param newcalendar:
+        @type newcalendar:
+        """
+
+        self.oldcalendar = oldcalendar
+        self.newcalendar = newcalendar
+        self.smart_merge = smart_merge
+
+
+    def organizerDiff(self):
+        """
+        Diff the two calendars looking for changes that should trigger implicit scheduling if
+        changed by an organizer. Basically any change except for anything related to a VALARM.
+        """
+
+        # If smart merge is needed we have to do this before trying the diff
+        if self.smart_merge:
+            log.debug("organizerDiff: doing smart Organizer diff/merge")
+            self._organizerMerge()
+
+        def duplicateAndNormalize(calendar):
+            calendar = calendar.duplicate()
+            calendar.removeAlarms()
+            calendar.filterProperties(remove=("X-CALENDARSERVER-ACCESS",), do_subcomponents=False)
+            calendar.filterProperties(remove=(
+                "CREATED",
+                "DTSTAMP",
+                "LAST-MODIFIED",
+            ))
+            calendar.removeXProperties(keep_properties=(
+                "X-APPLE-DROPBOX",
+            ))
+            calendar.removePropertyParameters("ATTENDEE", ("RSVP", "SCHEDULE-STATUS", "SCHEDULE-FORCE-SEND",))
+            calendar.normalizeAll()
+            return calendar
+
+        # Normalize components for comparison
+        oldcalendar_norm = duplicateAndNormalize(self.oldcalendar)
+        newcalendar_norm = duplicateAndNormalize(self.newcalendar)
+
+        result = oldcalendar_norm == newcalendar_norm
+        return result
+
+
+    def _organizerMerge(self):
+        """
+        Merge changes to ATTENDEE properties in oldcalendar into newcalendar.
+        """
+        organizer = normalizeCUAddr(self.newcalendar.masterComponent().propertyValue("ORGANIZER"))
+        self._doSmartMerge(organizer, True)
+
+
+    def _doSmartMerge(self, ignore_attendee, is_organizer):
+        """
+        Merge changes to ATTENDEE properties in oldcalendar into newcalendar.
+        """
+
+        old_master = self.oldcalendar.masterComponent()
+        new_master = self.newcalendar.masterComponent()
+
+        # Do master merge first
+        self._tryComponentMerge(old_master, new_master, ignore_attendee, is_organizer)
+
+        # New check the matching components
+        for old_component in self.oldcalendar.subcomponents():
+
+            # Make sure we have an appropriate component
+            if old_component.name() == "VTIMEZONE":
+                continue
+            rid = old_component.getRecurrenceIDUTC()
+            if rid is None:
+                continue
+
+            # Find matching component in new calendar
+            new_component = self.newcalendar.overriddenComponent(rid)
+            if new_component is None:
+                # If the old component was cancelled ignore when an attendee
+                if not is_organizer and old_component.propertyValue("STATUS") == "CANCELLED":
+                    continue
+
+                # Determine whether the instance is still valid in the new calendar
+                new_component = self.newcalendar.deriveInstance(rid)
+                if new_component:
+                    # Derive a new instance from the new calendar and transfer attendee status
+                    self.newcalendar.addComponent(new_component)
+                    self._tryComponentMerge(old_component, new_component, ignore_attendee, is_organizer)
+                else:
+                    # Ignore the old instance as it no longer exists
+                    pass
+            else:
+                self._tryComponentMerge(old_component, new_component, ignore_attendee, is_organizer)
+
+        # Check the new instances not in the old calendar
+        for new_component in self.newcalendar.subcomponents():
+
+            # Make sure we have an appropriate component
+            if new_component.name() == "VTIMEZONE":
+                continue
+            rid = new_component.getRecurrenceIDUTC()
+            if rid is None:
+                continue
+
+            # Find matching component in old calendar
+            old_component = self.oldcalendar.overriddenComponent(rid)
+            if old_component is None:
+                # If the new component is cancelled ignore when an attendee
+                if not is_organizer and new_component.propertyValue("STATUS") == "CANCELLED":
+                    continue
+
+                # Try to derive a new instance in the client and transfer attendee status
+                old_component = self.oldcalendar.deriveInstance(rid)
+                if old_component:
+                    self.oldcalendar.addComponent(old_component)
+                    self._tryComponentMerge(old_component, new_component, ignore_attendee, is_organizer)
+                else:
+                    # Ignore as we have no state for the new instance
+                    pass
+
+
+    def _tryComponentMerge(self, old_comp, new_comp, ignore_attendee_value, is_organizer):
+        if not is_organizer or not self._organizerChangePreventsMerge(old_comp, new_comp):
+            self._transferAttendees(old_comp, new_comp, ignore_attendee_value)
+
+
+    def _organizerChangePreventsMerge(self, old_comp, new_comp):
+        """
+        Check whether a change from an Organizer needs a re-schedule which means that any
+        Attendee state changes on the server are no longer relevant.
+
+        @param old_comp: existing server calendar component
+        @type old_comp: L{Component}
+        @param new_comp: new calendar component
+        @type new_comp: L{Component}
+        @return: C{True} if changes in new component are such that old attendee state is not
+            relevant, C{False} otherwise
+        """
+
+        props_to_test = ("DTSTART", "DTEND", "DURATION", "RRULE", "RDATE", "EXDATE", "RECURRENCE-ID",)
+
+        for prop in props_to_test:
+            # Change => no merge
+            old_props = set(old_comp.properties(prop))
+            new_props = set(new_comp.properties(prop))
+            if old_props.difference(new_props):
+                return True
+
+        return False
+
+
+    def _transferAttendees(self, old_comp, new_comp, ignore_attendee_value):
+        """
+        Transfer Attendee PARTSTAT from old component to new component.
+
+        @param old_comp: existing server calendar component
+        @type old_comp: L{Component}
+        @param new_comp: new calendar component
+        @type new_comp: L{Component}
+        @param ignore_attendee_value: Attendee to ignore
+        @type ignore_attendee_value: C{str}
+        """
+
+        # Create map of ATTENDEEs in old component
+        old_attendees = {}
+        for attendee in old_comp.properties("ATTENDEE"):
+            value = normalizeCUAddr(attendee.value())
+            if value == ignore_attendee_value:
+                continue
+            old_attendees[value] = attendee
+
+        for new_attendee in new_comp.properties("ATTENDEE"):
+
+            # Whenever SCHEDULE-FORCE-SEND is explicitly set by the Organizer we assume the Organizer
+            # is deliberately overwriting PARTSTAT
+            if new_attendee.parameterValue("SCHEDULE-FORCE-SEND", "") == "REQUEST":
+                continue
+
+            # Transfer parameters from any old Attendees found
+            value = normalizeCUAddr(new_attendee.value())
+            old_attendee = old_attendees.get(value)
+            if old_attendee:
+                self._transferParameter(old_attendee, new_attendee, "PARTSTAT")
+                self._transferParameter(old_attendee, new_attendee, "RSVP")
+                self._transferParameter(old_attendee, new_attendee, "SCHEDULE-STATUS")
+
+
+    def _transferParameter(self, old_property, new_property, parameter):
+        paramvalue = old_property.parameterValue(parameter)
+        if paramvalue is None:
+            try:
+                new_property.removeParameter(parameter)
+            except KeyError:
+                pass
+        else:
+            new_property.setParameter(parameter, paramvalue)
+
+
+    def attendeeMerge(self, attendee):
+        """
+        Merge the ATTENDEE specific changes with the organizer's view of the attendee's event.
+        This will remove any attempt by the attendee to change things like the time or location.
+
+        @param attendee: the value of the ATTENDEE property corresponding to the attendee making the change
+        @type attendee: C{str}
+
+        @return: C{tuple} of:
+            C{bool} - change is allowed
+            C{bool} - iTIP reply needs to be sent
+            C{list} - list of RECURRENCE-IDs changed
+            L{Component} - new calendar object to store
+        """
+
+        self.attendee = normalizeCUAddr(attendee)
+
+        returnCalendar = self.oldcalendar.duplicate()
+        returnMaster = returnCalendar.masterComponent()
+
+        changeCausesReply = False
+        changedRids = []
+
+        # First get uid/rid map of components
+        def mapComponents(calendar):
+            map = {}
+            cancelledRids = set()
+            master = None
+            for component in calendar.subcomponents():
+                if component.name() == "VTIMEZONE":
+                    continue
+                name = component.name()
+                uid = component.propertyValue("UID")
+                rid = component.getRecurrenceIDUTC()
+                map[(name, uid, rid,)] = component
+                if component.propertyValue("STATUS") == "CANCELLED" and rid is not None:
+                    cancelledRids.add(rid)
+                if rid is None:
+                    master = component
+
+            # Normalize each master by adding any STATUS:CANCELLED components as EXDATEs
+            exdates = None
+            if master:
+                # Get all EXDATEs in UTC
+                exdates = set()
+                for exdate in master.properties("EXDATE"):
+                    exdates.update([value.getValue().duplicate().adjustToUTC() for value in exdate.value()])
+
+            return exdates, map, master
+
+        exdatesold, mapold, masterold = mapComponents(self.oldcalendar)
+        setold = set(mapold.keys())
+        exdatesnew, mapnew, masternew = mapComponents(self.newcalendar)
+        setnew = set(mapnew.keys())
+
+        # Handle case where iCal breaks events without a master component
+        if masternew is not None and masterold is None:
+            masternewStart = masternew.getStartDateUTC()
+            keynew = (masternew.name(), masternew.propertyValue("UID"), masternewStart)
+            if keynew not in setold:
+                # The DTSTART in the fake master does not match a RECURRENCE-ID in the real data.
+                # We have to do a brute force search for the component that matches based on DTSTART
+                for componentold in self.oldcalendar.subcomponents():
+                    if componentold.name() == "VTIMEZONE":
+                        continue
+                    if masternewStart == componentold.getStartDateUTC():
+                        break
+                else:
+                    # Nothing matches - this has to be treated as an error
+                    self._logDiffError("attendeeMerge: Unable to match fake master component: %s" % (keynew,))
+                    return False, False, (), None
+            else:
+                componentold = self.oldcalendar.overriddenComponent(masternewStart)
+
+            # Take the recurrence ID from component1 and fix map2/set2
+            keynew = (masternew.name(), masternew.propertyValue("UID"), None)
+            componentnew = mapnew[keynew]
+            del mapnew[keynew]
+
+            ridold = componentold.getRecurrenceIDUTC()
+            newkeynew = (masternew.name(), masternew.propertyValue("UID"), ridold)
+            mapnew[newkeynew] = componentnew
+            setnew.remove(keynew)
+            setnew.add(newkeynew)
+
+        # All the components in oldcalendar must be in newcalendar unless they are CANCELLED
+        for key in setold - setnew:
+            _ignore_name, _ignore_uid, rid = key
+            component = mapold[key]
+            if component.propertyValue("STATUS") != "CANCELLED":
+                # Attendee may decline by EXDATE'ing an instance - we need to handle that
+                if exdatesnew is None or rid in exdatesnew:
+                    # Mark Attendee as DECLINED in the server instance
+                    overridden = returnCalendar.overriddenComponent(rid)
+                    if self._attendeeDecline(overridden):
+                        changeCausesReply = True
+                        changedRids.append(rid.getText() if rid else "")
+
+                    # When a master component is present we keep the missing override in place but mark it as hidden.
+                    # When no master is present we now do the same so we can track updates to the override correctly.
+                    overridden.replaceProperty(Property(Component.HIDDEN_INSTANCE_PROPERTY, "T"))
+
+                else:
+                    # We used to generate a 403 here - but instead we now ignore this error and let the server data
+                    # override the client
+                    self._logDiffError("attendeeMerge: Missing uncancelled component from first calendar: %s" % (key,))
+            else:
+                if exdatesnew is not None and rid not in exdatesnew:
+                    # We used to generate a 403 here - but instead we now ignore this error and let the server data
+                    # override the client
+                    self._logDiffError("attendeeMerge: Missing EXDATE for cancelled components from first calendar: %s" % (key,))
+                else:
+                    # Remove the CANCELLED component from the new calendar and add an EXDATE
+                    overridden = returnCalendar.overriddenComponent(rid)
+                    returnCalendar.removeComponent(overridden)
+                    if returnMaster:
+                        # Use the original R-ID value so we preserve the timezone
+                        original_rid = component.propertyValue("RECURRENCE-ID")
+                        returnMaster.addProperty(Property("EXDATE", [original_rid, ]))
+
+        # Derive a new component in the new calendar for each new one in setnew
+        for key in setnew - setold:
+
+            # First check if the attendee's copy is cancelled and properly EXDATE'd
+            # and skip it if so.
+            _ignore_name, _ignore_uid, rid = key
+            componentnew = mapnew[key]
+            if componentnew.propertyValue("STATUS") == "CANCELLED":
+                if exdatesold is None or rid not in exdatesold:
+                    # We used to generate a 403 here - but instead we now ignore this error and let the server data
+                    # override the client
+                    self._logDiffError("attendeeMerge: Cancelled component not found in first calendar (or no EXDATE): %s" % (key,))
+                    setnew.remove(key)
+                else:
+                    # Derive new component with STATUS:CANCELLED and remove EXDATE
+                    newOverride = returnCalendar.deriveInstance(rid, allowCancelled=True)
+                    if newOverride is None:
+                        # We used to generate a 403 here - but instead we now ignore this error and let the server data
+                        # override the client
+                        self._logDiffError("attendeeMerge: Could not derive instance for cancelled component: %s" % (key,))
+                        setnew.remove(key)
+                    else:
+                        returnCalendar.addComponent(newOverride)
+            else:
+                # Derive new component
+                newOverride = returnCalendar.deriveInstance(rid)
+                if newOverride is None:
+                    # We used to generate a 403 here - but instead we now ignore this error and let the server data
+                    # override the client
+                    self._logDiffError("attendeeMerge: Could not derive instance for uncancelled component: %s" % (key,))
+                    setnew.remove(key)
+                else:
+                    returnCalendar.addComponent(newOverride)
+
+        # So now returnCalendar has all the same components as set2. Check changes and do transfers.
+
+        # Make sure the same VCALENDAR properties match
+        if not self._checkVCALENDARProperties(returnCalendar, self.newcalendar):
+            # We used to generate a 403 here - but instead we now ignore this error and let the server data
+            # override the client
+            self._logDiffError("attendeeMerge: VCALENDAR properties do not match")
+
+        # Now we transfer per-Attendee
+        # data from newcalendar into returnCalendar to sync up changes, whilst verifying that other
+        # key properties are unchanged
+        declines = []
+        for key in setnew:
+            _ignore_name, _ignore_uid, rid = key
+            serverData = returnCalendar.overriddenComponent(rid)
+            clientData = mapnew[key]
+
+            allowed, reply = self._transferAttendeeData(serverData, clientData, declines)
+            if not allowed:
+                # We used to generate a 403 here - but instead we now ignore this error and let the server data
+                # override the client
+                self._logDiffError("attendeeMerge: Mismatched calendar objects")
+                #return False, False, (), None
+            changeCausesReply |= reply
+            if reply:
+                changedRids.append(rid.getText() if rid else "")
+
+        # We need to derive instances for any declined using an EXDATE
+        for decline in sorted(declines):
+            overridden = returnCalendar.overriddenComponent(decline)
+            if not overridden:
+                overridden = returnCalendar.deriveInstance(decline)
+                if overridden:
+                    if self._attendeeDecline(overridden):
+                        changeCausesReply = True
+                        changedRids.append(decline.getText() if decline else "")
+
+                    # When a master component is present we keep the missing override in place but mark it as hidden.
+                    # When no master is present we remove the override,
+                    if exdatesnew is not None:
+                        overridden.replaceProperty(Property(Component.HIDDEN_INSTANCE_PROPERTY, "T"))
+                        returnCalendar.addComponent(overridden)
+                else:
+                    self._logDiffError("attendeeMerge: Unable to override an instance to mark as DECLINED: %s" % (decline,))
+                    return False, False, (), None
+
+        return True, changeCausesReply, changedRids, returnCalendar
+
+
+    def _checkVCALENDARProperties(self, serverData, clientData):
+
+        self._transferProperty("X-CALENDARSERVER-ACCESS", serverData, clientData)
+
+        # Get property differences in the VCALENDAR objects
+        propdiff = set(serverData.properties()) ^ set(clientData.properties())
+
+        # Ignore certain properties
+        ignored = ("PRODID", "CALSCALE",)
+        propdiff = set([prop for prop in propdiff if prop.name() not in ignored])
+
+        result = len(propdiff) == 0
+        if not result:
+            log.debug("VCALENDAR properties differ: %s" % (propdiff,))
+        return result
+
+
+    def _transferAttendeeData(self, serverComponent, clientComponent, declines):
+
+        # We are skipping this check now - instead we let the server data override the broken client data
+        # First check validity of date-time related properties and get removed components which are declines
+        self._checkInvalidChanges(serverComponent, clientComponent, declines)
+
+        # Now look for items to transfer from one to the other.
+        # We care about the ATTENDEE's PARTSTAT, TRANSP, VALARMS, X-APPLE-NEEDS-REPLY,
+        # DTSTAMP, LAST-MODIFIED, COMPLETED, and ATTACH's referring to a dropbox
+
+        replyNeeded = False
+
+        # ATTENDEE/PARTSTAT/RSVP
+        serverAttendee = serverComponent.getAttendeeProperty((self.attendee,))
+        clientAttendee = clientComponent.getAttendeeProperty((self.attendee,))
+
+        # Possible case where one ATTENDEE prop is missing - this happens with a "fake" master sometimes
+        if serverAttendee is None or clientAttendee is None:
+            log.err("ATTENDEE for user making an attendee change is missing: %s" % (self.attendee,))
+            return False, False
+
+        if serverAttendee.parameterValue("PARTSTAT", "NEEDS-ACTION") != clientAttendee.parameterValue("PARTSTAT", "NEEDS-ACTION"):
+            serverAttendee.setParameter("PARTSTAT", clientAttendee.parameterValue("PARTSTAT", "NEEDS-ACTION"))
+
+            # If PARTSTAT was changed by the attendee, add a timestamp if needed
+            if config.Scheduling.Options.TimestampAttendeePartStatChanges:
+                serverAttendee.setParameter("X-CALENDARSERVER-DTSTAMP", PyCalendarDateTime.getNowUTC().getText())
+
+            replyNeeded = True
+
+        if serverAttendee.parameterValue("RSVP", "FALSE") != clientAttendee.parameterValue("RSVP", "FALSE"):
+            if clientAttendee.parameterValue("RSVP", "FALSE") == "FALSE":
+                try:
+                    serverAttendee.removeParameter("RSVP")
+                except KeyError:
+                    pass
+            else:
+                serverAttendee.setParameter("RSVP", "TRUE")
+
+        # Transfer these properties from the client data
+        replyNeeded |= self._transferProperty("X-CALENDARSERVER-PRIVATE-COMMENT", serverComponent, clientComponent)
+        self._transferProperty("TRANSP", serverComponent, clientComponent)
+        self._transferProperty("DTSTAMP", serverComponent, clientComponent)
+        self._transferProperty("LAST-MODIFIED", serverComponent, clientComponent)
+        self._transferProperty("COMPLETED", serverComponent, clientComponent)
+        for pname in config.Scheduling.CalDAV.PerAttendeeProperties:
+            self._transferProperty(pname, serverComponent, clientComponent)
+
+        # Dropbox - this now never returns false
+        if config.EnableDropBox:
+            self._transferDropBoxData(serverComponent, clientComponent)
+
+        # Handle VALARMs
+        serverComponent.removeAlarms()
+        for comp in clientComponent.subcomponents():
+            if comp.name() == "VALARM":
+                serverComponent.addComponent(comp)
+
+        return True, replyNeeded
+
+
+    def _transferDropBoxData(self, serverComponent, clientComponent):
+
+        serverDropbox = serverComponent.propertyValue("X-APPLE-DROPBOX")
+        clientDropbox = clientComponent.propertyValue("X-APPLE-DROPBOX")
+
+        # Handle four cases
+        if not clientDropbox:
+            return True
+        elif not serverDropbox:
+            # Attendee not allowed to add a dropbox - ignore this
+            self._logDiffError("Attendee not allowed to add dropbox: %s" % (clientDropbox,))
+            return True
+        else:
+            # Values must be the same - ignore this
+            if serverDropbox != clientDropbox:
+                self._logDiffError("Attendee not allowed to change dropbox from: %s to: %s" % (serverDropbox, clientDropbox,))
+                return True
+
+            # Remove existing ATTACH's from server
+            for attachment in tuple(serverComponent.properties("ATTACH")):
+                valueType = attachment.parameterValue("VALUE")
+                if valueType in (None, "URI"):
+                    dataValue = attachment.value()
+                    if dataValue.find(serverDropbox) != -1:
+                        serverComponent.removeProperty(attachment)
+
+            # Copy new ATTACH's to server
+            for attachment in tuple(clientComponent.properties("ATTACH")):
+                valueType = attachment.parameterValue("VALUE")
+                if valueType in (None, "URI"):
+                    dataValue = attachment.value()
+                    if dataValue.find(serverDropbox) != -1:
+                        serverComponent.addProperty(attachment)
+
+            return True
+
+
+    def _checkInvalidChanges(self, serverComponent, clientComponent, declines):
+
+        # Properties we care about: DTSTART, DTEND, DURATION, RRULE, RDATE, EXDATE
+
+        serverProps = self._getNormalizedDateTimeProperties(serverComponent)
+        clientProps = self._getNormalizedDateTimeProperties(clientComponent)
+
+        # Need to special case EXDATEs as an Attendee can effectively DECLINE by adding an EXDATE
+        if serverProps[:-1] != clientProps[:-1]:
+            invalidChanges = []
+            propNames = ("DTSTART", "DTEND", "DUE", "RRULE", "RDATE", "EXDATE")
+            invalidChanges = [propName for ctr, propName in enumerate(propNames) if serverProps[ctr] != clientProps[ctr]]
+            log.debug("Critical properties do not match: %s" % (", ".join(invalidChanges),))
+            return False
+        elif serverProps[-1] != clientProps[-1]:
+            # Bad if EXDATEs have been removed
+            missing = serverProps[-1] - clientProps[-1]
+            if missing:
+                log.debug("EXDATEs missing: %s" % (", ".join([exdate.getText() for exdate in missing]),))
+                return False
+            declines.extend(clientProps[-1] - serverProps[-1])
+            return True
+        else:
+            return True
+
+
+    def _getNormalizedDateTimeProperties(self, component):
+
+        # Basic time properties
+        if component.name() in ("VEVENT", "VJOURNAL",):
+            dtstart = component.getProperty("DTSTART")
+            dtend = component.getProperty("DTEND")
+            duration = component.getProperty("DURATION")
+
+            timeRange = PyCalendarPeriod(
+                start=dtstart.value()  if dtstart  is not None else None,
+                end=dtend.value()    if dtend    is not None else None,
+                duration=duration.value() if duration is not None else None,
+            )
+            newdue = None
+
+        elif component.name() == "VTODO":
+            dtstart = component.getProperty("DTSTART")
+            duration = component.getProperty("DURATION")
+
+            if dtstart or duration:
+                timeRange = PyCalendarPeriod(
+                    start=dtstart.value()  if dtstart  is not None else None,
+                    duration=duration.value() if duration is not None else None,
+                )
+            else:
+                timeRange = PyCalendarPeriod()
+
+            newdue = component.getProperty("DUE")
+            if newdue is not None:
+                newdue = newdue.value().duplicate().adjustToUTC()
+
+        # Recurrence rules - we need to normalize the order of the value parts
+        newrrules = set()
+        rrules = component.properties("RRULE")
+        for rrule in rrules:
+            indexedTokens = {}
+            indexedTokens.update([valuePart.split("=") for valuePart in rrule.value().getText().split(";")])
+            sortedValue = ";".join(["%s=%s" % (key, value,) for key, value in sorted(indexedTokens.iteritems(), key=lambda x:x[0])])
+            newrrules.add(sortedValue)
+
+        # RDATEs
+        newrdates = set()
+        rdates = component.properties("RDATE")
+        for rdate in rdates:
+            for value in rdate.value():
+                if isinstance(value, PyCalendarDateTime):
+                    value = value.duplicate().adjustToUTC()
+                newrdates.add(value)
+
+        # EXDATEs
+        newexdates = set()
+        exdates = component.properties("EXDATE")
+        for exdate in exdates:
+            newexdates.update([value.getValue().duplicate().adjustToUTC() for value in exdate.value()])
+
+        return timeRange.getStart(), timeRange.getEnd(), newdue, newrrules, newrdates, newexdates
+
+
+    def _transferProperty(self, propName, serverComponent, clientComponent):
+
+        changed = False
+        serverProp = serverComponent.getProperty(propName)
+        clientProp = clientComponent.getProperty(propName)
+        if serverProp != clientProp:
+            if clientProp:
+                serverComponent.replaceProperty(Property(propName, clientProp.value()))
+            else:
+                serverComponent.removeProperty(serverProp)
+            changed = True
+        return changed
+
+
+    def _attendeeDecline(self, component):
+        """
+        Mark attendee as DECLINED in the component.
+
+        @param component:
+        @type component:
+
+        @return: C{bool} indicating whether the PARTSTAT value was in fact changed
+        """
+        attendee = component.getAttendeeProperty((self.attendee,))
+
+        # Possible case where ATTENDEE prop is missing - this happens with a "fake" master sometimes
+        if attendee is None:
+            log.err("ATTENDEE for user making an attendee change is missing: %s" % (self.attendee,))
+            return False
+
+        partstatChanged = attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") != "DECLINED"
+        attendee.setParameter("PARTSTAT", "DECLINED")
+        prop = component.getProperty("X-APPLE-NEEDS-REPLY")
+        if prop:
+            component.removeProperty(prop)
+        component.replaceProperty(Property("TRANSP", "TRANSPARENT"))
+        return partstatChanged
+
+
+    def whatIsDifferent(self):
+        """
+        Compare the two calendar objects in their entirety and return a list of properties
+        and PARTSTAT parameters that are different.
+        """
+
+        # First get uid/rid map of components
+        def mapComponents(calendar):
+            map = {}
+            for component in calendar.subcomponents():
+                if component.name() == "VTIMEZONE":
+                    continue
+                name = component.name()
+                uid = component.propertyValue("UID")
+                rid = component.getRecurrenceIDUTC()
+                map[(name, uid, rid,)] = component
+            return map
+
+        rids = {}
+
+        oldmap = mapComponents(self.oldcalendar)
+        oldset = set(oldmap.keys())
+        newmap = mapComponents(self.newcalendar)
+        newset = set(newmap.keys())
+
+        # Now verify that each component in oldset matches what is in newset
+        for key in (oldset & newset):
+            component1 = oldmap[key]
+            component2 = newmap[key]
+            self._diffComponents(component1, component2, rids)
+
+        # Now verify that each additional component in oldset matches a derived component in newset
+        for key in oldset - newset:
+            oldcomponent = oldmap[key]
+            newcomponent = self.newcalendar.deriveInstance(key[2])
+            if newcomponent is None:
+                continue
+            self._diffComponents(oldcomponent, newcomponent, rids)
+
+        # Now verify that each additional component in oldset matches a derived component in newset
+        for key in newset - oldset:
+            oldcomponent = self.oldcalendar.deriveInstance(key[2])
+            if oldcomponent is None:
+                continue
+            newcomponent = newmap[key]
+            self._diffComponents(oldcomponent, newcomponent, rids)
+
+        return rids
+
+
+    def _componentDuplicateAndNormalize(self, comp):
+        comp = comp.duplicate()
+        comp.normalizePropertyValueLists("EXDATE")
+        comp.removePropertyParameters("ORGANIZER", ("SCHEDULE-STATUS",))
+        comp.removePropertyParameters("ATTENDEE", ("SCHEDULE-STATUS", "SCHEDULE-FORCE-SEND",))
+        comp.removeAlarms()
+        comp.normalizeAll()
+        comp.normalizeAttachments()
+        iTipGenerator.prepareSchedulingMessage(comp, reply=True)
+        return comp
+
+
+    def _diffComponents(self, comp1, comp2, rids):
+
+        assert isinstance(comp1, Component) and isinstance(comp2, Component)
+
+        if comp1.name() != comp2.name():
+            log.debug("Component names are different: '%s' and '%s'" % (comp1.name(), comp2.name()))
+            return
+
+        # Duplicate then normalize for comparison
+        comp1 = self._componentDuplicateAndNormalize(comp1)
+        comp2 = self._componentDuplicateAndNormalize(comp2)
+
+        # Diff all the properties
+        propdiff = set(comp1.properties()) ^ set(comp2.properties())
+        addedChanges = False
+
+        propsChanged = {}
+        for prop in propdiff:
+            if prop.name() in (
+                "TRANSP",
+                "DTSTAMP",
+                "CREATED",
+                "LAST-MODIFIED",
+                "X-CALENDARSERVER-PRIVATE-COMMENT",
+            ):
+                continue
+            propsChanged.setdefault(prop.name(), set())
+            addedChanges = True
+            prop1s = tuple(comp1.properties(prop.name()))
+            prop2s = tuple(comp2.properties(prop.name()))
+            if len(prop1s) == 1 and len(prop2s) == 1:
+                param1s = set(["%s=%s" % (name, prop1s[0].parameterValue(name)) for name in prop1s[0].parameterNames()])
+                param2s = set(["%s=%s" % (name, prop2s[0].parameterValue(name)) for name in prop2s[0].parameterNames()])
+                paramDiffs = param1s ^ param2s
+                propsChanged[prop.name()].update([param.split("=")[0] for param in paramDiffs])
+            if "_TZID" in propsChanged[prop.name()]:
+                propsChanged[prop.name()].remove("_TZID")
+                propsChanged[prop.name()].add("TZID")
+
+        if addedChanges:
+            rid = comp1.getRecurrenceIDUTC()
+            rids[rid.getText() if rid is not None else ""] = propsChanged
+
+
+    def _logDiffError(self, title):
+
+        strcal1 = str(self.oldcalendar)
+        strcal2 = str(self.newcalendar)
+        strdiff = "\n".join(unified_diff(
+            strcal1.split("\n"),
+            strcal2.split("\n"),
+            fromfile='Existing Calendar Object',
+            tofile='New Calendar Object',
+        ))
+
+        logstr = """%s
+
+------ Existing Calendar Data ------
+%s
+------ New Calendar Data ------
+%s
+------ Diff ------
+%s
+""" % (title, strcal1, strcal2, strdiff,)
+
+        loggedUID = self.oldcalendar.resourceUID()
+        if loggedUID:
+            loggedUID = loggedUID.encode("base64")[:-1]
+        else:
+            loggedUID = "Unknown"
+        loggedName = accounting.emitAccounting("Implicit Errors", loggedUID, logstr)
+        if loggedName:
+            log.err("Generating Implicit Error accounting at path: %s" % (loggedName,))

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/__init__.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/__init__.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/delivery.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/delivery.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/delivery.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,123 @@
+# -*- test-case-name: twistedcaldav.scheduling.test.test_imip -*-
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+Handles the sending of scheduling messages via iMIP (mail gateway).
+"""
+
+from twext.python.log import Logger
+from twext.web2 import responsecode
+from twext.web2.dav.http import ErrorResponse
+from twext.web2.http import HTTPError
+from twisted.internet.defer import inlineCallbacks, returnValue
+from twisted.python.failure import Failure
+from twistedcaldav.caldavxml import caldav_namespace
+from twistedcaldav.directory.util import transactionFromRequest
+from txdav.caldav.datastore.scheduling.delivery import DeliveryService
+from txdav.caldav.datastore.scheduling.imip.outbound import IMIPInvitationWork
+from txdav.caldav.datastore.scheduling.itip import iTIPRequestStatus
+
+
+
+__all__ = [
+    "ScheduleViaIMip",
+]
+
+log = Logger()
+
+class ScheduleViaIMip(DeliveryService):
+
+    @classmethod
+    def serviceType(cls):
+        return DeliveryService.serviceType_imip
+
+
+    @inlineCallbacks
+    def generateSchedulingResponses(self):
+        def failForRecipient(recipient):
+            err = HTTPError(ErrorResponse(
+                responsecode.FORBIDDEN,
+                (caldav_namespace, "recipient-failed"),
+                "iMIP request failed",
+            ))
+            self.responses.add(
+                recipient.cuaddr,
+                Failure(exc_value=err),
+                reqstatus=iTIPRequestStatus.SERVICE_UNAVAILABLE,
+                suppressErrorLog=True
+            )
+
+        # Generate an HTTP client request
+        try:
+            # We do not do freebusy requests via iMIP
+            if self.freebusy:
+                raise ValueError("iMIP VFREEBUSY requests not supported.")
+
+            method = self.scheduler.calendar.propertyValue("METHOD")
+            if method not in (
+                "PUBLISH",
+                "REQUEST",
+                "REPLY",
+                "ADD",
+                "CANCEL",
+                "DECLINE_COUNTER",
+            ):
+                log.info("Could not do server-to-imip method: %s" % (method,))
+                for recipient in self.recipients:
+                    err = HTTPError(ErrorResponse(
+                        responsecode.FORBIDDEN,
+                        (caldav_namespace, "recipient-failed"),
+                        "iMIP method not allowed: %s" % (method,),
+                    ))
+                    self.responses.add(
+                        recipient.cuaddr,
+                        Failure(exc_value=err),
+                        reqstatus=iTIPRequestStatus.NO_USER_SUPPORT
+                    )
+                returnValue(None)
+
+            caldata = str(self.scheduler.calendar)
+
+            for recipient in self.recipients:
+                try:
+                    toAddr = str(recipient.cuaddr)
+                    if not toAddr.lower().startswith("mailto:"):
+                        raise ValueError("ATTENDEE address '%s' must be mailto: for iMIP operation." % (toAddr,))
+
+                    fromAddr = str(self.scheduler.originator.cuaddr)
+
+                    txn = transactionFromRequest(self.scheduler.request, self.scheduler.request._newStoreTransaction.store)
+                    log.debug("Submitting iMIP message...  To: '%s', From :'%s'\n%s" % (toAddr, fromAddr, caldata,))
+                    yield txn.enqueue(IMIPInvitationWork, fromAddr=fromAddr, toAddr=toAddr, icalendarText=caldata)
+
+                except Exception, e:
+                    # Generated failed response for this recipient
+                    log.debug("iMIP request %s failed for recipient %s: %s" % (self, recipient, e))
+                    failForRecipient(recipient)
+
+                else:
+                    self.responses.add(
+                        recipient.cuaddr,
+                        responsecode.OK,
+                        reqstatus=iTIPRequestStatus.MESSAGE_SENT
+                    )
+
+        except Exception, e:
+            # Generated failed responses for each recipient
+            log.debug("iMIP request %s failed: %s" % (self, e))
+            for recipient in self.recipients:
+                failForRecipient(recipient)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/inbound.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/inbound.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/inbound.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,637 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+Inbound IMIP mail handling for Calendar Server
+"""
+import datetime
+from calendarserver.tap.util import FakeRequest
+import email.utils
+from twext.enterprise.dal.record import fromTable
+from twext.enterprise.queue import WorkItem
+from twext.python.log import Logger, LoggingMixIn
+from twisted.application import service
+from twisted.internet import protocol, defer, ssl
+from twisted.internet.defer import inlineCallbacks, returnValue, succeed
+from twisted.mail import pop3client, imap4
+from twisted.mail.smtp import messageid
+from twistedcaldav.config import config
+from twistedcaldav.ical import Property, Component
+from txdav.caldav.datastore.scheduling.imip.scheduler import IMIPScheduler
+from txdav.caldav.datastore.scheduling.imip.smtpsender import SMTPSender
+from txdav.caldav.datastore.scheduling.itip import iTIPRequestStatus
+from txdav.common.datastore.sql_tables import schema
+from twext.internet.gaiendpoint import GAIEndpoint
+
+
+log = Logger()
+
+#
+# Monkey patch imap4.log so it doesn't emit useless logging,
+# specifically, "Unhandled unsolicited response" nonsense.
+#
+class IMAPLogger(Logger):
+    def emit(self, level, message, *args, **kwargs):
+        if message.startswith("Unhandled unsolicited response:"):
+            return
+
+        Logger.emit(self, level, message, *args, **kwargs)
+
+imap4.log = IMAPLogger()
+
+
+""" SCHEMA:
+create table IMIP_REPLY_WORK (
+  WORK_ID                       integer primary key default nextval('WORKITEM_SEQ') not null,
+  NOT_BEFORE                    timestamp    default timezone('UTC', CURRENT_TIMESTAMP),
+  ORGANIZER                     varchar(255) not null,
+  ATTENDEE                      varchar(255) not null,
+  ICALENDAR_TEXT                text         not null
+);
+create table IMIP_POLLING_WORK (
+  WORK_ID                       integer primary key default nextval('WORKITEM_SEQ') not null,
+  NOT_BEFORE                    timestamp    default timezone('UTC', CURRENT_TIMESTAMP)
+);
+"""
+
+class IMIPReplyWork(WorkItem, fromTable(schema.IMIP_REPLY_WORK)):
+
+    @inlineCallbacks
+    def doWork(self):
+        rootResource = self.transaction._rootResource
+        calendar = Component.fromString(self.icalendarText)
+        yield injectMessage(self.transaction, rootResource, self.organizer, self.attendee,
+            calendar)
+
+
+
+class IMIPPollingWork(WorkItem, fromTable(schema.IMIP_POLLING_WORK)):
+
+    # FIXME: delete all other polling work items
+    # FIXME: purge all old tokens here
+
+    @inlineCallbacks
+    def doWork(self):
+        mailRetriever = self.transaction._mailRetriever
+        if mailRetriever is not None:
+            try:
+                yield mailRetriever.fetchMail()
+            except Exception, e:
+                log.error("Failed to fetch mail (%s)" % (e,))
+            finally:
+                yield mailRetriever.scheduleNextPoll()
+
+
+
+class MailRetriever(service.Service):
+
+    def __init__(self, store, directory, settings, reactor=None):
+        self.store = store
+        self.settings = settings
+        if reactor is None:
+            from twisted.internet import reactor
+        self.reactor = reactor
+
+        self.mailReceiver = MailReceiver(store, directory)
+        mailType = settings['Type']
+        if mailType.lower().startswith('pop'):
+            self.factory = POP3DownloadFactory
+        else:
+            self.factory = IMAP4DownloadFactory
+
+        contextFactory = None
+        if settings["UseSSL"]:
+            contextFactory = ssl.ClientContextFactory()
+        self.point = GAIEndpoint(self.reactor, settings.Server,
+            settings.Port, contextFactory=contextFactory)
+
+
+    def startService(self):
+        return self.scheduleNextPoll(seconds=0)
+
+
+    def fetchMail(self):
+        return self.point.connect(self.factory(self.settings, self.mailReceiver))
+
+
+    @inlineCallbacks
+    def scheduleNextPoll(self, seconds=None):
+        if seconds is None:
+            seconds = self.settings["PollingSeconds"]
+        txn = self.store.newTransaction()
+        notBefore = datetime.datetime.utcnow() + datetime.timedelta(seconds=seconds)
+        yield txn.enqueue(IMIPPollingWork, notBefore=notBefore)
+        yield txn.commit()
+
+
+
+class MailReceiver(object):
+
+    NO_TOKEN = 0
+    UNKNOWN_TOKEN = 1
+    MALFORMED_TO_ADDRESS = 2
+    NO_ORGANIZER_ADDRESS = 3
+    REPLY_FORWARDED_TO_ORGANIZER = 4
+    INJECTION_SUBMITTED = 5
+
+    # What about purge( ) and lowercase( )
+    def __init__(self, store, directory):
+        self.store = store
+        self.directory = directory
+
+
+    def checkDSN(self, message):
+        # returns (isdsn, action, icalendar attachment)
+
+        report = deliveryStatus = calBody = None
+
+        for part in message.walk():
+            contentType = part.get_content_type()
+            if contentType == "multipart/report":
+                report = part
+                continue
+            elif contentType == "message/delivery-status":
+                deliveryStatus = part
+                continue
+            elif contentType == "message/rfc822":
+                #original = part
+                continue
+            elif contentType == "text/calendar":
+                calBody = part.get_payload(decode=True)
+                continue
+
+        if report is not None and deliveryStatus is not None:
+            # we have what appears to be a dsn
+
+            lines = str(deliveryStatus).split("\n")
+            for line in lines:
+                lower = line.lower()
+                if lower.startswith("action:"):
+                    # found action:
+                    action = lower.split(' ')[1]
+                    break
+            else:
+                action = None
+
+            return True, action, calBody
+
+        else:
+            # not a dsn
+            return False, None, None
+
+
+    def _extractToken(self, text):
+        try:
+            pre, _ignore_post = text.split('@')
+            pre, token = pre.split('+')
+            return token
+        except ValueError:
+            return None
+
+
+    @inlineCallbacks
+    def processDSN(self, calBody, msgId):
+        calendar = Component.fromString(calBody)
+        # Extract the token (from organizer property)
+        organizer = calendar.getOrganizer()
+        token = self._extractToken(organizer)
+        if not token:
+            log.error("Mail gateway can't find token in DSN %s" % (msgId,))
+            return
+
+        txn = self.store.newTransaction()
+        result = (yield txn.imipLookupByToken(token))
+        yield txn.commit()
+        try:
+            # Note the results are returned as utf-8 encoded strings
+            organizer, attendee, _ignore_icaluid = result[0]
+        except:
+            # This isn't a token we recognize
+            log.error("Mail gateway found a token (%s) but didn't "
+                           "recognize it in message %s"
+                           % (token, msgId))
+            returnValue(self.UNKNOWN_TOKEN)
+
+        calendar.removeAllButOneAttendee(attendee)
+        calendar.getOrganizerProperty().setValue(organizer)
+        for comp in calendar.subcomponents():
+            if comp.name() == "VEVENT":
+                comp.addProperty(Property("REQUEST-STATUS",
+                    ["5.1", "Service unavailable"]))
+                break
+        else:
+            # no VEVENT in the calendar body.
+            # TODO: what to do in this case?
+            pass
+
+        log.warn("Mail gateway processing DSN %s" % (msgId,))
+        txn = self.store.newTransaction()
+        yield txn.enqueue(IMIPReplyWork, organizer=organizer, attendee=attendee,
+            icalendarText=str(calendar))
+        yield txn.commit()
+        returnValue(self.INJECTION_SUBMITTED)
+
+
+    @inlineCallbacks
+    def processReply(self, msg):
+        # extract the token from the To header
+        _ignore_name, addr = email.utils.parseaddr(msg['To'])
+        if addr:
+            # addr looks like: server_address+token at example.com
+            token = self._extractToken(addr)
+            if not token:
+                log.error("Mail gateway didn't find a token in message "
+                               "%s (%s)" % (msg['Message-ID'], msg['To']))
+                returnValue(self.NO_TOKEN)
+        else:
+            log.error("Mail gateway couldn't parse To: address (%s) in "
+                           "message %s" % (msg['To'], msg['Message-ID']))
+            returnValue(self.MALFORMED_TO_ADDRESS)
+
+        txn = self.store.newTransaction()
+        result = (yield txn.imipLookupByToken(token))
+        yield txn.commit()
+        try:
+            # Note the results are returned as utf-8 encoded strings
+            organizer, attendee, _ignore_icaluid = result[0]
+        except:
+            # This isn't a token we recognize
+            log.error("Mail gateway found a token (%s) but didn't "
+                           "recognize it in message %s"
+                           % (token, msg['Message-ID']))
+            returnValue(self.UNKNOWN_TOKEN)
+
+        for part in msg.walk():
+            if part.get_content_type() == "text/calendar":
+                calBody = part.get_payload(decode=True)
+                break
+        else:
+            # No icalendar attachment
+            log.warn("Mail gateway didn't find an icalendar attachment "
+                          "in message %s" % (msg['Message-ID'],))
+
+            toAddr = None
+            fromAddr = attendee[7:]
+            if organizer.startswith("mailto:"):
+                toAddr = organizer[7:]
+            elif organizer.startswith("urn:uuid:"):
+                guid = organizer[9:]
+                record = self.directory.recordWithGUID(guid)
+                if record and record.emailAddresses:
+                    toAddr = list(record.emailAddresses)[0]
+
+            if toAddr is None:
+                log.error("Don't have an email address for the organizer; "
+                               "ignoring reply.")
+                returnValue(self.NO_ORGANIZER_ADDRESS)
+
+            settings = config.Scheduling["iMIP"]["Sending"]
+            smtpSender = SMTPSender(settings.Username, settings.Password,
+                settings.UseSSL, settings.Server, settings.Port)
+
+            del msg["From"]
+            msg["From"] = fromAddr
+            del msg["Reply-To"]
+            msg["Reply-To"] = fromAddr
+            del msg["To"]
+            msg["To"] = toAddr
+            log.warn("Mail gateway forwarding reply back to organizer")
+            yield smtpSender.sendMessage(fromAddr, toAddr, messageid(), msg)
+            returnValue(self.REPLY_FORWARDED_TO_ORGANIZER)
+
+        # Process the imip attachment; inject to calendar server
+
+        log.debug(calBody)
+        calendar = Component.fromString(calBody)
+        event = calendar.mainComponent()
+
+        calendar.removeAllButOneAttendee(attendee)
+        organizerProperty = calendar.getOrganizerProperty()
+        if organizerProperty is None:
+            # ORGANIZER is required per rfc2446 section 3.2.3
+            log.warn("Mail gateway didn't find an ORGANIZER in REPLY %s"
+                          % (msg['Message-ID'],))
+            event.addProperty(Property("ORGANIZER", organizer))
+        else:
+            organizerProperty.setValue(organizer)
+
+        if not calendar.getAttendees():
+            # The attendee we're expecting isn't there, so add it back
+            # with a SCHEDULE-STATUS of SERVICE_UNAVAILABLE.
+            # The organizer will then see that the reply was not successful.
+            attendeeProp = Property("ATTENDEE", attendee,
+                params={
+                    "SCHEDULE-STATUS": iTIPRequestStatus.SERVICE_UNAVAILABLE,
+                }
+            )
+            event.addProperty(attendeeProp)
+
+            # TODO: We have talked about sending an email to the reply-to
+            # at this point, to let them know that their reply was missing
+            # the appropriate ATTENDEE.  This will require a new localizable
+            # email template for the message.
+
+        txn = self.store.newTransaction()
+        yield txn.enqueue(IMIPReplyWork, organizer=organizer, attendee=attendee,
+            icalendarText=str(calendar))
+        yield txn.commit()
+        returnValue(self.INJECTION_SUBMITTED)
+
+
+    # returns a deferred
+    def inbound(self, message):
+
+        try:
+            msg = email.message_from_string(message)
+
+            isDSN, action, calBody = self.checkDSN(msg)
+            if isDSN:
+                if action == 'failed' and calBody:
+                    # This is a DSN we can handle
+                    return self.processDSN(calBody, msg['Message-ID'])
+                else:
+                    # It's a DSN without enough to go on
+                    log.error("Mail gateway can't process DSN %s"
+                                   % (msg['Message-ID'],))
+                    return succeed(None)
+
+            log.info("Mail gateway received message %s from %s to %s" %
+                (msg['Message-ID'], msg['From'], msg['To']))
+
+            return self.processReply(msg)
+
+        except Exception, e:
+            # Don't let a failure of any kind stop us
+            log.error("Failed to process message: %s" % (e,))
+        return succeed(None)
+
+
+
+ at inlineCallbacks
+def injectMessage(txn, root, organizer, attendee, calendar):
+
+    request = FakeRequest(root, None, "/", transaction=txn)
+    resource = root.getChild("principals")
+    scheduler = IMIPScheduler(request, resource)
+    scheduler.originator = attendee
+    scheduler.recipients = [organizer, ]
+    scheduler.calendar = calendar
+
+    try:
+        results = (yield scheduler.doScheduling())
+        log.info("Successfully injected iMIP response from %s to %s" %
+            (attendee, organizer))
+    except Exception, e:
+        log.error("Failed to inject iMIP response (%s)" % (e,))
+        raise
+
+    returnValue(results)
+
+
+
+#
+# POP3
+#
+
+class POP3DownloadProtocol(pop3client.POP3Client, LoggingMixIn):
+    allowInsecureLogin = False
+
+    def serverGreeting(self, greeting):
+        self.log_debug("POP servergreeting")
+        pop3client.POP3Client.serverGreeting(self, greeting)
+        login = self.login(self.factory.settings["Username"],
+            self.factory.settings["Password"])
+        login.addCallback(self.cbLoggedIn)
+        login.addErrback(self.cbLoginFailed)
+
+
+    def cbLoginFailed(self, reason):
+        self.log_error("POP3 login failed for %s" %
+            (self.factory.settings["Username"],))
+        return self.quit()
+
+
+    def cbLoggedIn(self, result):
+        self.log_debug("POP loggedin")
+        return self.listSize().addCallback(self.cbGotMessageSizes)
+
+
+    def cbGotMessageSizes(self, sizes):
+        self.log_debug("POP gotmessagesizes")
+        downloads = []
+        for i in range(len(sizes)):
+            downloads.append(self.retrieve(i).addCallback(self.cbDownloaded, i))
+        return defer.DeferredList(downloads).addCallback(self.cbFinished)
+
+
+    def cbDownloaded(self, lines, id):
+        self.log_debug("POP downloaded message %d" % (id,))
+        self.factory.handleMessage("\r\n".join(lines))
+        self.log_debug("POP deleting message %d" % (id,))
+        self.delete(id)
+
+
+    def cbFinished(self, results):
+        self.log_debug("POP finished")
+        return self.quit()
+
+
+
+class POP3DownloadFactory(protocol.ClientFactory, LoggingMixIn):
+    protocol = POP3DownloadProtocol
+
+    def __init__(self, settings, mailReceiver):
+        self.mailReceiver = mailReceiver
+        self.noisy = False
+
+
+    def clientConnectionLost(self, connector, reason):
+        self.connector = connector
+        self.log_debug("POP factory connection lost")
+
+
+    def clientConnectionFailed(self, connector, reason):
+        self.connector = connector
+        self.log_info("POP factory connection failed")
+
+
+    def handleMessage(self, message):
+        self.log_debug("POP factory handle message")
+        self.log_debug(message)
+        return self.mailReceiver.inbound(message)
+
+
+
+#
+# IMAP4
+#
+
+
+class IMAP4DownloadProtocol(imap4.IMAP4Client, LoggingMixIn):
+
+    def serverGreeting(self, capabilities):
+        self.log_debug("IMAP servergreeting")
+        return self.authenticate(self.factory.settings["Password"]
+            ).addCallback(self.cbLoggedIn
+            ).addErrback(self.ebAuthenticateFailed)
+
+
+    def ebLogError(self, error):
+        self.log_error("IMAP Error: %s" % (error,))
+
+
+    def ebAuthenticateFailed(self, reason):
+        self.log_debug("IMAP authenticate failed for %s, trying login" %
+            (self.factory.settings["Username"],))
+        return self.login(self.factory.settings["Username"],
+            self.factory.settings["Password"]
+            ).addCallback(self.cbLoggedIn
+            ).addErrback(self.ebLoginFailed)
+
+
+    def ebLoginFailed(self, reason):
+        self.log_error("IMAP login failed for %s" %
+            (self.factory.settings["Username"],))
+        self.transport.loseConnection()
+
+
+    def cbLoggedIn(self, result):
+        self.log_debug("IMAP logged in [%s]" % (self.state,))
+        self.select("Inbox").addCallback(self.cbInboxSelected)
+
+
+    def cbInboxSelected(self, result):
+        self.log_debug("IMAP Inbox selected [%s]" % (self.state,))
+        allMessages = imap4.MessageSet(1, None)
+        self.fetchUID(allMessages, True).addCallback(self.cbGotUIDs)
+
+
+    def cbGotUIDs(self, results):
+        self.log_debug("IMAP got uids [%s]" % (self.state,))
+        self.messageUIDs = [result['UID'] for result in results.values()]
+        self.messageCount = len(self.messageUIDs)
+        self.log_debug("IMAP Inbox has %d messages" % (self.messageCount,))
+        if self.messageCount:
+            self.fetchNextMessage()
+        else:
+            # No messages; close it out
+            self.close().addCallback(self.cbClosed)
+
+
+    def fetchNextMessage(self):
+        self.log_debug("IMAP in fetchnextmessage [%s]" % (self.state,))
+        if self.messageUIDs:
+            nextUID = self.messageUIDs.pop(0)
+            messageListToFetch = imap4.MessageSet(nextUID)
+            self.log_debug("Downloading message %d of %d (%s)" %
+                (self.messageCount - len(self.messageUIDs), self.messageCount,
+                nextUID))
+            self.fetchMessage(messageListToFetch, True).addCallback(
+                self.cbGotMessage, messageListToFetch).addErrback(
+                    self.ebLogError)
+        else:
+            self.log_debug("Seeing if anything new has arrived")
+            # Go back and see if any more messages have come in
+            self.expunge().addCallback(self.cbInboxSelected)
+
+
+    def cbGotMessage(self, results, messageList):
+        self.log_debug("IMAP in cbGotMessage [%s]" % (self.state,))
+        try:
+            messageData = results.values()[0]['RFC822']
+        except IndexError:
+            # results will be empty unless the "twistedmail-imap-flags-anywhere"
+            # patch from http://twistedmatrix.com/trac/ticket/1105 is applied
+            self.log_error("Skipping empty results -- apply twisted patch!")
+            self.fetchNextMessage()
+            return
+
+        d = self.factory.handleMessage(messageData)
+        if isinstance(d, defer.Deferred):
+            d.addCallback(self.cbFlagDeleted, messageList)
+        else:
+            # No deferred returned, so no need for addCallback( )
+            self.cbFlagDeleted(None, messageList)
+
+
+    def cbFlagDeleted(self, results, messageList):
+        self.addFlags(messageList, ("\\Deleted",),
+            uid=True).addCallback(self.cbMessageDeleted, messageList)
+
+
+    def cbMessageDeleted(self, results, messageList):
+        self.log_debug("IMAP in cbMessageDeleted [%s]" % (self.state,))
+        self.log_debug("Deleted message")
+        self.fetchNextMessage()
+
+
+    def cbClosed(self, results):
+        self.log_debug("IMAP in cbClosed [%s]" % (self.state,))
+        self.log_debug("Mailbox closed")
+        self.logout().addCallback(
+            lambda _: self.transport.loseConnection())
+
+
+    def rawDataReceived(self, data):
+        self.log_debug("RAW RECEIVED: %s" % (data,))
+        imap4.IMAP4Client.rawDataReceived(self, data)
+
+
+    def lineReceived(self, line):
+        self.log_debug("RECEIVED: %s" % (line,))
+        imap4.IMAP4Client.lineReceived(self, line)
+
+
+    def sendLine(self, line):
+        self.log_debug("SENDING: %s" % (line,))
+        imap4.IMAP4Client.sendLine(self, line)
+
+
+
+class IMAP4DownloadFactory(protocol.ClientFactory, LoggingMixIn):
+    protocol = IMAP4DownloadProtocol
+
+    def __init__(self, settings, mailReceiver):
+        self.log_debug("Setting up IMAPFactory")
+
+        self.settings = settings
+        self.mailReceiver = mailReceiver
+        self.noisy = False
+
+
+    def buildProtocol(self, addr):
+        p = protocol.ClientFactory.buildProtocol(self, addr)
+        username = self.settings["Username"]
+        p.registerAuthenticator(imap4.CramMD5ClientAuthenticator(username))
+        p.registerAuthenticator(imap4.LOGINAuthenticator(username))
+        p.registerAuthenticator(imap4.PLAINAuthenticator(username))
+        return p
+
+
+    def handleMessage(self, message):
+        self.log_debug("IMAP factory handle message")
+        self.log_debug(message)
+        return self.mailReceiver.inbound(message)
+
+
+    def clientConnectionLost(self, connector, reason):
+        self.connector = connector
+        self.log_debug("IMAP factory connection lost")
+
+
+    def clientConnectionFailed(self, connector, reason):
+        self.connector = connector
+        self.log_warn("IMAP factory connection failed")

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/mailgateway.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/mailgateway.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/mailgateway.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,236 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+SQLite implementation of mail token database (deprecated).  This only exists
+now in order to migrate tokens from sqlite to the new store.
+"""
+
+import datetime
+import os
+import uuid
+
+from twext.python.log import Logger, LoggingMixIn
+from twistedcaldav.sql import AbstractSQLDatabase
+from twisted.internet.defer import inlineCallbacks
+
+log = Logger()
+
+
+class MailGatewayTokensDatabase(AbstractSQLDatabase, LoggingMixIn):
+    """
+    A database to maintain "plus-address" tokens for IMIP requests.
+
+    SCHEMA:
+
+    Token Database:
+
+    ROW: TOKEN, ORGANIZER, ATTENDEE, ICALUID, DATESTAMP
+
+    """
+
+    dbType = "MAILGATEWAYTOKENS"
+    dbFilename = "mailgatewaytokens.sqlite"
+    dbFormatVersion = "1"
+
+
+    def __init__(self, path):
+        if path != ":memory:":
+            path = os.path.join(path, MailGatewayTokensDatabase.dbFilename)
+        super(MailGatewayTokensDatabase, self).__init__(path, True)
+
+
+    def createToken(self, organizer, attendee, icaluid, token=None):
+        if token is None:
+            token = str(uuid.uuid4())
+        self._db_execute(
+            """
+            insert into TOKENS (TOKEN, ORGANIZER, ATTENDEE, ICALUID, DATESTAMP)
+            values (:1, :2, :3, :4, :5)
+            """, token, organizer, attendee, icaluid, datetime.date.today()
+        )
+        self._db_commit()
+        return token
+
+
+    def lookupByToken(self, token):
+        results = list(
+            self._db_execute(
+                """
+                select ORGANIZER, ATTENDEE, ICALUID from TOKENS
+                where TOKEN = :1
+                """, token
+            )
+        )
+
+        if len(results) != 1:
+            return None
+
+        return results[0]
+
+
+    def getToken(self, organizer, attendee, icaluid):
+        token = self._db_value_for_sql(
+            """
+            select TOKEN from TOKENS
+            where ORGANIZER = :1 and ATTENDEE = :2 and ICALUID = :3
+            """, organizer, attendee, icaluid
+        )
+        if token is not None:
+            # update the datestamp on the token to keep it from being purged
+            self._db_execute(
+                """
+                update TOKENS set DATESTAMP = :1 WHERE TOKEN = :2
+                """, datetime.date.today(), token
+            )
+            return str(token)
+        else:
+            return None
+
+
+    def getAllTokens(self):
+        results = list(
+            self._db_execute(
+                """
+                select TOKEN, ORGANIZER, ATTENDEE, ICALUID from TOKENS
+                """
+            )
+        )
+        return results
+
+
+    def deleteToken(self, token):
+        self._db_execute(
+            """
+            delete from TOKENS where TOKEN = :1
+            """, token
+        )
+        self._db_commit()
+
+
+    def purgeOldTokens(self, before):
+        self._db_execute(
+            """
+            delete from TOKENS where DATESTAMP < :1
+            """, before
+        )
+        self._db_commit()
+
+
+    def lowercase(self):
+        """
+        Lowercase mailto: addresses (and uppercase urn:uuid: addresses!) so
+        they can be located via normalized names.
+        """
+        rows = self._db_execute(
+            """
+            select ORGANIZER, ATTENDEE from TOKENS
+            """
+        )
+        for row in rows:
+            organizer = row[0]
+            attendee = row[1]
+            if organizer.lower().startswith("mailto:"):
+                self._db_execute(
+                    """
+                    update TOKENS set ORGANIZER = :1 WHERE ORGANIZER = :2
+                    """, organizer.lower(), organizer
+                )
+            else:
+                from txdav.base.datastore.util import normalizeUUIDOrNot
+                self._db_execute(
+                    """
+                    update TOKENS set ORGANIZER = :1 WHERE ORGANIZER = :2
+                    """, normalizeUUIDOrNot(organizer), organizer
+                )
+            # ATTENDEEs are always mailto: so unconditionally lower().
+            self._db_execute(
+                """
+                update TOKENS set ATTENDEE = :1 WHERE ATTENDEE = :2
+                """, attendee.lower(), attendee
+            )
+        self._db_commit()
+
+
+    def _db_version(self):
+        """
+        @return: the schema version assigned to this index.
+        """
+        return MailGatewayTokensDatabase.dbFormatVersion
+
+
+    def _db_type(self):
+        """
+        @return: the collection type assigned to this index.
+        """
+        return MailGatewayTokensDatabase.dbType
+
+
+    def _db_init_data_tables(self, q):
+        """
+        Initialise the underlying database tables.
+        @param q:           a database cursor to use.
+        """
+
+        #
+        # TOKENS table
+        #
+        q.execute(
+            """
+            create table TOKENS (
+                TOKEN       text,
+                ORGANIZER   text,
+                ATTENDEE    text,
+                ICALUID     text,
+                DATESTAMP   date
+            )
+            """
+        )
+        q.execute(
+            """
+            create index TOKENSINDEX on TOKENS (TOKEN)
+            """
+        )
+
+
+    def _db_upgrade_data_tables(self, q, old_version):
+        """
+        Upgrade the data from an older version of the DB.
+        @param q: a database cursor to use.
+        @param old_version: existing DB's version number
+        @type old_version: str
+        """
+        pass
+
+
+
+ at inlineCallbacks
+def migrateTokensToStore(path, store):
+    """
+    Copy all the tokens from the sqlite db into the new store.
+
+    @param path: Filesystem path to directory containing the sqlite db file.
+    @type path: C{str}
+
+    @param store: The store to copy tokens into
+    @type store: L{CommonDataStore}
+    """
+    oldDB = MailGatewayTokensDatabase(path)
+    txn = store.newTransaction()
+    for token, organizer, attendee, icaluid in oldDB.getAllTokens():
+        yield txn.imipCreateToken(organizer, attendee, icaluid, token=token)
+    yield txn.commit()
+    os.remove(oldDB.dbpath)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/outbound.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/outbound.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/outbound.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,738 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+Outbound IMIP mail handling for Calendar Server
+"""
+
+from __future__ import with_statement
+
+from cStringIO import StringIO
+import os
+
+from email.mime.multipart import MIMEMultipart
+from email.mime.text import MIMEText
+import email.utils
+from pycalendar.datetime import PyCalendarDateTime
+from pycalendar.duration import PyCalendarDuration
+from twext.enterprise.dal.record import fromTable
+from twext.enterprise.queue import WorkItem
+from twext.python.log import Logger, LoggingMixIn
+from twisted.internet.defer import inlineCallbacks, returnValue
+from twisted.mail.smtp import messageid, rfc822date
+from twisted.web.microdom import Text as DOMText, Element as DOMElement
+from twisted.web.microdom import parseString
+from twisted.web.template import XMLString, TEMPLATE_NAMESPACE, Element, renderer, flattenString, tags
+from twistedcaldav.config import config
+from twistedcaldav.ical import Component
+from twistedcaldav.localization import translationTo, _, getLanguage
+from txdav.caldav.datastore.scheduling.cuaddress import normalizeCUAddr
+from txdav.caldav.datastore.scheduling.imip.smtpsender import SMTPSender
+from txdav.common.datastore.sql_tables import schema
+
+
+
+log = Logger()
+
+
+""" SCHEMA:
+create sequence WORKITEM_SEQ;
+
+create table IMIP_INVITATION_WORK (
+  WORK_ID         integer primary key default nextval('WORKITEM_SEQ') not null,
+  NOT_BEFORE      timestamp    default timezone('UTC', CURRENT_TIMESTAMP),
+  FROM_ADDR       varchar(255) not null,
+  TO_ADDR         varchar(255) not null,
+  ICALENDAR_TEXT  text         not null
+);
+"""
+
+
+class IMIPInvitationWork(WorkItem, fromTable(schema.IMIP_INVITATION_WORK)):
+    """
+    Sends outbound IMIP messages
+    """
+
+    mailSender = None
+
+    @classmethod
+    def getMailSender(cls):
+        """
+        Instantiate and return a singleton MailSender object
+        @return: a MailSender
+        """
+        if cls.mailSender is None:
+            if config.Scheduling.iMIP.Enabled:
+                settings = config.Scheduling.iMIP.Sending
+                smtpSender = SMTPSender(settings.Username, settings.Password,
+                    settings.UseSSL, settings.Server, settings.Port)
+                cls.mailSender = MailSender(settings.Address,
+                    settings.SuppressionDays, smtpSender, getLanguage(config))
+        return cls.mailSender
+
+
+    @inlineCallbacks
+    def doWork(self):
+        """
+        Send an outbound IMIP message
+        """
+        mailSender = self.getMailSender()
+        if mailSender is not None:
+            calendar = Component.fromString(self.icalendarText)
+            yield mailSender.outbound(self.transaction,
+                self.fromAddr, self.toAddr, calendar)
+
+#
+# Templates
+#
+
+plainCancelTemplate = u"""%(subject)s
+
+%(orgLabel)s: %(plainOrganizer)s
+%(dateLabel)s: %(dateInfo)s %(recurrenceInfo)s
+%(timeLabel)s: %(timeInfo)s %(durationInfo)s
+"""
+
+plainInviteTemplate = u"""%(subject)s
+
+%(orgLabel)s: %(plainOrganizer)s
+%(locLabel)s: %(location)s
+%(dateLabel)s: %(dateInfo)s %(recurrenceInfo)s
+%(timeLabel)s: %(timeInfo)s %(durationInfo)s
+%(descLabel)s: %(description)s
+%(urlLabel)s: %(url)s
+%(attLabel)s: %(plainAttendees)s
+"""
+
+
+htmlCancelTemplate = u"""<html>
+    <body><div>
+
+    <h1>%(subject)s</h1>
+    <p>
+    <h3>%(orgLabel)s:</h3> %(htmlOrganizer)s
+    </p>
+    <p>
+    <h3>%(dateLabel)s:</h3> %(dateInfo)s %(recurrenceInfo)s
+    </p>
+    <p>
+    <h3>%(timeLabel)s:</h3> %(timeInfo)s %(durationInfo)s
+    </p>
+    """.encode("utf-8")
+
+
+htmlInviteTemplate = u"""<html>
+    <body><div>
+    <p>%(inviteLabel)s</p>
+
+    <h1>%(summary)s</h1>
+    <p>
+    <h3>%(orgLabel)s:</h3> %(htmlOrganizer)s
+    </p>
+    <p>
+    <h3>%(locLabel)s:</h3> %(location)s
+    </p>
+    <p>
+    <h3>%(dateLabel)s:</h3> %(dateInfo)s %(recurrenceInfo)s
+    </p>
+    <p>
+    <h3>%(timeLabel)s:</h3> %(timeInfo)s %(durationInfo)s
+    </p>
+    <p>
+    <h3>%(descLabel)s:</h3> %(description)s
+    </p>
+    <p>
+    <h3>%(urlLabel)s:</h3> <a href="%(url)s">%(url)s</a>
+    </p>
+    <p>
+    <h3>%(attLabel)s:</h3> %(htmlAttendees)s
+    </p>
+    """.encode("utf-8")
+
+def _visit(document, node):
+    if isinstance(node, DOMText):
+        idx = node.parentNode.childNodes.index(node)
+        splitted = node.data.split("%(")
+        firstTextNode = document.createTextNode(splitted[0])
+        firstTextNode.parentNode = node.parentNode
+        replacements = [firstTextNode]
+        for moreText in splitted[1:]:
+            slotName, extra = moreText.split(')', 1)
+            extra = extra[1:]
+            slotElement = document.createElement('t:slot')
+            slotElement.setAttribute("name", slotName)
+            slotElement.parentNode = node.parentNode
+            textNode = document.createTextNode(extra)
+            textNode.parentNode = node.parentNode
+            replacements.append(slotElement)
+            replacements.append(textNode)
+        node.parentNode.childNodes[idx:idx + 1] = replacements
+
+    elif isinstance(node, DOMElement):
+        for attrName, attrVal in node.attributes.items():
+            if '%(' in attrVal:
+                del node.attributes[attrName]
+                elem = document.createElement('t:attr')
+                elem.setAttribute('name', attrName)
+                textNode = document.createTextNode(attrVal)
+                elem.appendChild(textNode)
+                node.appendChild(elem)
+
+
+
+def _walk(document, n):
+    _visit(document, n)
+    for subn in n.childNodes:
+        _walk(document, subn)
+
+
+
+def _fixup(data, rendererName):
+    document = parseString(data, beExtremelyLenient=True)
+    document.documentElement.setAttribute(
+        "xmlns:t", TEMPLATE_NAMESPACE
+    )
+    document.doctype = (
+        'html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" '
+        '"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"'
+    )
+    document.documentElement.setAttribute(
+        "t:render", rendererName
+    )
+    _walk(document, document)
+    result = document.toxml()
+    return result
+
+
+
+class StringFormatTemplateLoader(object):
+    """
+    Loader for twisted.web.template that converts a template with %()s slots.
+    """
+    def __init__(self, fileFactory, rendererName):
+        """
+        @param fileFactory: a 1-argument callable which returns a file-like
+            object that contains the %()s-format template.
+
+        @param rendererName: the name of the renderer.
+
+        @type rendererName: C{str}
+        """
+        self.fileFactory = fileFactory
+        self.rendererName = rendererName
+
+
+    def load(self):
+        html = _fixup(self.fileFactory().read(), self.rendererName)
+        return XMLString(html).load()
+
+
+
+def localizedLabels(language, canceled, inviteState):
+    """
+    Generate localized labels for an email in the given language.
+
+    @param language: a 2-letter language code
+
+    @type language: C{str}
+
+    @return: a 2-tuple of (subjectFormatString, labelDict), where the first is a
+        format string for use in the subject, and the latter is a dictionary
+        with labels suitable for filling out HTML and plain-text templates.  All
+        values are C{str}s.
+    """
+    with translationTo(language):
+        if canceled:
+            subjectFormatString = _("Event canceled: %(summary)s")
+        elif inviteState == "new":
+            subjectFormatString = _("Event invitation: %(summary)s")
+        elif inviteState == "update":
+            subjectFormatString = _("Event update: %(summary)s")
+        else:
+            subjectFormatString = _("Event reply: %(summary)s")
+
+        if canceled:
+            inviteLabel = _("Event Canceled")
+        else:
+            if inviteState == "new":
+                inviteLabel = _("Event Invitation")
+            elif inviteState == "update":
+                inviteLabel = _("Event Update")
+            else:
+                inviteLabel = _("Event Reply")
+
+        labels = dict(
+            dateLabel=_("Date"),
+            timeLabel=_("Time"),
+            durationLabel=_("Duration"),
+            recurrenceLabel=_("Occurs"),
+            descLabel=_("Description"),
+            urlLabel=_("URL"),
+            orgLabel=_("Organizer"),
+            attLabel=_("Attendees"),
+            locLabel=_("Location"),
+            inviteLabel=inviteLabel,
+        )
+
+        # The translations we get back from gettext are utf-8 encoded
+        # strings, so convert to unicode
+        for key in labels.keys():
+            if isinstance(labels[key], str):
+                labels[key] = labels[key].decode("utf-8")
+
+    return subjectFormatString.decode("utf-8"), labels
+
+
+
+class MailSender(LoggingMixIn):
+    """
+    Generates outbound IMIP messages and sends them.
+    """
+
+    def __init__(self, address, suppressionDays, smtpSender, language):
+        self.address = address
+        self.suppressionDays = suppressionDays
+        self.smtpSender = smtpSender
+        self.language = language
+
+
+    @inlineCallbacks
+    def outbound(self, txn, originator, recipient, calendar, onlyAfter=None):
+        """
+        Generates and sends an outbound IMIP message.
+
+        @param txn: the transaction to use for looking up/creating tokens
+        @type txn: L{CommonStoreTransaction}
+        """
+
+        if onlyAfter is None:
+            duration = PyCalendarDuration(days=self.suppressionDays)
+            onlyAfter = PyCalendarDateTime.getNowUTC() - duration
+
+        component = calendar.masterComponent()
+        if component is None:
+            component = calendar.mainComponent(True)
+        icaluid = component.propertyValue("UID")
+        method = calendar.propertyValue("METHOD")
+
+        # Clean up the attendee list which is purely used within the human
+        # readable email message (not modifying the calendar body)
+        attendees = []
+        for attendeeProp in calendar.getAllAttendeeProperties():
+            cutype = attendeeProp.parameterValue("CUTYPE", "INDIVIDUAL")
+            if cutype == "INDIVIDUAL":
+                cn = attendeeProp.parameterValue("CN", None)
+                if cn is not None:
+                    cn = cn.decode("utf-8")
+                cuaddr = normalizeCUAddr(attendeeProp.value())
+                if cuaddr.startswith("mailto:"):
+                    mailto = cuaddr[7:]
+                    if not cn:
+                        cn = mailto
+                else:
+                    emailAddress = attendeeProp.parameterValue("EMAIL", None)
+                    if emailAddress:
+                        mailto = emailAddress
+                    else:
+                        mailto = None
+
+                if cn or mailto:
+                    attendees.append((cn, mailto))
+
+        toAddr = recipient
+        if not recipient.lower().startswith("mailto:"):
+            raise ValueError("ATTENDEE address '%s' must be mailto: for iMIP "
+                             "operation." % (recipient,))
+        recipient = recipient[7:]
+
+        if method != "REPLY":
+            # Invites and cancellations:
+
+            # Reuse or generate a token based on originator, toAddr, and
+            # event uid
+            token = (yield txn.imipGetToken(originator, toAddr.lower(), icaluid))
+            if token is None:
+
+                # Because in the past the originator was sometimes in mailto:
+                # form, lookup an existing token by mailto: as well
+                organizerProperty = calendar.getOrganizerProperty()
+                organizerEmailAddress = organizerProperty.parameterValue("EMAIL", None)
+                if organizerEmailAddress is not None:
+                    token = (yield txn.imipGetToken("mailto:%s" % (organizerEmailAddress.lower(),), toAddr.lower(), icaluid))
+
+            if token is None:
+                token = (yield txn.imipCreateToken(originator, toAddr.lower(), icaluid))
+                self.log_debug("Mail gateway created token %s for %s "
+                               "(originator), %s (recipient) and %s (icaluid)"
+                               % (token, originator, toAddr, icaluid))
+                inviteState = "new"
+
+            else:
+                self.log_debug("Mail gateway reusing token %s for %s "
+                               "(originator), %s (recipient) and %s (icaluid)"
+                               % (token, originator, toAddr, icaluid))
+                inviteState = "update"
+
+            fullServerAddress = self.address
+            _ignore_name, serverAddress = email.utils.parseaddr(fullServerAddress)
+            pre, post = serverAddress.split('@')
+            addressWithToken = "%s+%s@%s" % (pre, token, post)
+
+            organizerProperty = calendar.getOrganizerProperty()
+            organizerEmailAddress = organizerProperty.parameterValue("EMAIL",
+                                                                     None)
+            organizerValue = organizerProperty.value()
+            organizerProperty.setValue("mailto:%s" % (addressWithToken,))
+
+            # If the organizer is also an attendee, update that attendee value
+            # to match
+            organizerAttendeeProperty = calendar.getAttendeeProperty(
+                [organizerValue])
+            if organizerAttendeeProperty is not None:
+                organizerAttendeeProperty.setValue("mailto:%s" %
+                                                   (addressWithToken,))
+
+            # The email's From will include the originator's real name email
+            # address if available.  Otherwise it will be the server's email
+            # address (without # + addressing)
+            if organizerEmailAddress:
+                orgEmail = fromAddr = organizerEmailAddress
+            else:
+                fromAddr = serverAddress
+                orgEmail = None
+            cn = calendar.getOrganizerProperty().parameterValue('CN', None)
+            if cn is None:
+                cn = u'Calendar Server'
+                orgCN = orgEmail
+            else:
+                orgCN = cn = cn.decode("utf-8")
+
+            # a unicode cn (rather than an encode string value) means the
+            # from address will get properly encoded per rfc2047 within the
+            # MIMEMultipart in generateEmail
+            formattedFrom = "%s <%s>" % (cn, fromAddr)
+
+            # Reply-to address will be the server+token address
+
+        else: # REPLY
+            inviteState = "reply"
+
+            # Look up the attendee property corresponding to the originator
+            # of this reply
+            originatorAttendeeProperty = calendar.getAttendeeProperty(
+                [originator])
+            formattedFrom = fromAddr = originator = ""
+            if originatorAttendeeProperty:
+                originatorAttendeeEmailAddress = (
+                    originatorAttendeeProperty.parameterValue("EMAIL", None)
+                )
+                if originatorAttendeeEmailAddress:
+                    formattedFrom = fromAddr = originator = (
+                        originatorAttendeeEmailAddress
+                    )
+
+            organizerMailto = str(calendar.getOrganizer())
+            if not organizerMailto.lower().startswith("mailto:"):
+                raise ValueError("ORGANIZER address '%s' must be mailto: "
+                                 "for REPLY." % (organizerMailto,))
+            orgEmail = organizerMailto[7:]
+
+            orgCN = calendar.getOrganizerProperty().parameterValue('CN', None)
+            addressWithToken = formattedFrom
+
+        # At the point we've created the token in the db, which we always
+        # want to do, but if this message is for an event completely in
+        # the past we don't want to actually send an email.
+        if not calendar.hasInstancesAfter(onlyAfter):
+            self.log_debug("Skipping IMIP message for old event")
+            returnValue(True)
+
+        # Now prevent any "internal" CUAs from being exposed by converting
+        # to mailto: if we have one
+        for attendeeProp in calendar.getAllAttendeeProperties():
+            cutype = attendeeProp.parameterValue('CUTYPE', None)
+            if cutype == "INDIVIDUAL":
+                cuaddr = normalizeCUAddr(attendeeProp.value())
+                if not cuaddr.startswith("mailto:"):
+                    emailAddress = attendeeProp.parameterValue("EMAIL", None)
+                    if emailAddress:
+                        attendeeProp.setValue("mailto:%s" % (emailAddress,))
+
+        msgId, message = self.generateEmail(inviteState, calendar, orgEmail,
+            orgCN, attendees, formattedFrom, addressWithToken, recipient,
+            language=self.language)
+
+        try:
+            success = (yield self.smtpSender.sendMessage(fromAddr, toAddr,
+                msgId, message))
+            returnValue(success)
+        except Exception, e:
+            self.log_error("Failed to send IMIP message (%s)" % (str(e),))
+            returnValue(False)
+
+
+    def generateEmail(self, inviteState, calendar, orgEmail, orgCN,
+                      attendees, fromAddress, replyToAddress, toAddress,
+                      language='en'):
+        """
+        Generate MIME text containing an iMIP invitation, cancellation, update
+        or reply.
+
+        @param inviteState: 'new', 'update', or 'reply'.
+
+        @type inviteState: C{str}
+
+        @param calendar: the iCalendar component to attach to the email.
+
+        @type calendar: L{twistedcaldav.ical.Component}
+
+        @param orgEmail: The email for the organizer, in C{localhost at domain}
+            format, or C{None} if the organizer has no email address.
+
+        @type orgEmail: C{str} or C{NoneType}
+
+        @param orgCN: Common name / display name for the organizer.
+
+        @type orgCN: C{unicode}
+
+        @param attendees: A C{list} of 2-C{tuple}s of (common name, email
+            address) similar to (orgEmail, orgCN).
+
+        @param fromAddress: the address to use in the C{From:} header of the
+            email.
+
+        @type fromAddress: C{str}
+
+        @param replyToAddress: the address to use in the C{Reply-To} header.
+
+        @type replyToAddress: C{str}
+
+        @param toAddress: the address to use in the C{To} header.
+
+        @type toAddress: C{str}
+
+        @param language: a 2-letter language code describing the target
+            language that the email should be generated in.
+
+        @type language: C{str}
+
+        @return: a 2-tuple of C{str}s: (message ID, message text).  The message
+            ID is the value of the C{Message-ID} header, and the message text is
+            the full MIME message, ready for transport over SMTP.
+        """
+
+        details = self.getEventDetails(calendar, language=language)
+        canceled = (calendar.propertyValue("METHOD") == "CANCEL")
+
+        subjectFormat, labels = localizedLabels(language, canceled, inviteState)
+        details.update(labels)
+
+        details['subject'] = subjectFormat % {'summary' : details['summary']}
+
+        plainText = self.renderPlainText(details, (orgCN, orgEmail),
+                                         attendees, canceled)
+
+        htmlText = self.renderHTML(details, (orgCN, orgEmail),
+                                              attendees, canceled)
+
+        msg = MIMEMultipart()
+        msg["From"] = fromAddress
+        msg["Subject"] = details['subject']
+        msg["Reply-To"] = replyToAddress
+        msg["To"] = toAddress
+        msg["Date"] = rfc822date()
+        msgId = messageid()
+        msg["Message-ID"] = msgId
+
+        msgAlt = MIMEMultipart("alternative")
+        msg.attach(msgAlt)
+
+        # plain version
+        msgPlain = MIMEText(plainText, "plain", "UTF-8")
+        msgAlt.attach(msgPlain)
+
+        # html version
+        msgHtmlRelated = MIMEMultipart("related", type="text/html")
+        msgAlt.attach(msgHtmlRelated)
+
+        msgHtml = MIMEText(htmlText, "html", "UTF-8")
+        msgHtmlRelated.attach(msgHtml)
+
+        calendarText = str(calendar)
+        # the icalendar attachment
+        self.log_debug("Mail gateway sending calendar body: %s"
+                       % (calendarText,))
+        msgIcal = MIMEText(calendarText, "calendar", "UTF-8")
+        method = calendar.propertyValue("METHOD").lower()
+        msgIcal.set_param("method", method)
+        msgIcal.add_header("Content-ID", "<invitation.ics>")
+        msgIcal.add_header("Content-Disposition",
+            "inline;filename=invitation.ics")
+        msg.attach(msgIcal)
+
+        return msgId, msg.as_string()
+
+
+    def renderPlainText(self, details, (orgCN, orgEmail), attendees, canceled):
+        """
+        Render text/plain message part based on invitation details and a flag
+        indicating whether the message is a cancellation.
+
+        @return: UTF-8 encoded text.
+
+        @rtype: C{str}
+        """
+        plainAttendeeList = []
+        for cn, mailto in attendees:
+            if cn:
+                plainAttendeeList.append(cn if not mailto else
+                    "%s <%s>" % (cn, mailto))
+            elif mailto:
+                plainAttendeeList.append("<%s>" % (mailto,))
+
+        details['plainAttendees'] = ", ".join(plainAttendeeList)
+
+        details['plainOrganizer'] = (orgCN if not orgEmail else
+            "%s <%s>" % (orgCN, orgEmail))
+
+        # plain text version
+        if canceled:
+            plainTemplate = plainCancelTemplate
+        else:
+            plainTemplate = plainInviteTemplate
+
+        return (plainTemplate % details).encode("UTF-8")
+
+
+    def renderHTML(self, details, organizer, attendees, canceled):
+        """
+        Render HTML message part based on invitation details and a flag
+        indicating whether the message is a cancellation.
+
+        @return: html text (C{str}, representing utf-8 encoded bytes)).
+        """
+        orgCN, orgEmail = organizer
+
+        # TODO: htmlAttendees needs to be a separate element with a separate
+        # template fragment.  Luckily that fragment is the same regardless
+        # of the rest of the template.
+        htmlAttendees = []
+        first = True
+        for cn, mailto in attendees:
+            if not first:
+                htmlAttendees.append(u", ")
+            else:
+                first = False
+
+            if mailto:
+                if not cn:
+                    cn = mailto
+                htmlAttendees.append(
+                    tags.a(href="mailto:%s" % (mailto,))(cn)
+                )
+            else:
+                htmlAttendees.append(cn)
+
+        details['htmlAttendees'] = htmlAttendees
+
+        # TODO: htmlOrganizer is also some HTML that requires additional
+        # template stuff, and once again, it's just a 'mailto:'.
+        # tags.a(href="mailto:"+email)[cn]
+        if orgEmail:
+            details['htmlOrganizer'] = tags.a(href="mailto:%s" % (orgEmail,))(
+                orgCN)
+        else:
+            details['htmlOrganizer'] = orgCN
+
+        templateDir = config.Scheduling.iMIP.MailTemplatesDirectory.rstrip("/")
+        templateName = "cancel.html" if canceled else "invite.html"
+        templatePath = os.path.join(templateDir, templateName)
+
+        if not os.path.exists(templatePath):
+            # Fall back to built-in simple templates:
+            if canceled:
+                htmlTemplate = htmlCancelTemplate
+            else:
+                htmlTemplate = htmlInviteTemplate
+        else: # HTML template file exists
+
+            with open(templatePath) as templateFile:
+                htmlTemplate = templateFile.read()
+
+        class EmailElement(Element):
+            loader = StringFormatTemplateLoader(lambda : StringIO(htmlTemplate),
+                                                "email")
+
+            @renderer
+            def email(self, request, tag):
+                return tag.fillSlots(**details)
+
+        textCollector = []
+        flattenString(None, EmailElement()).addCallback(textCollector.append)
+        htmlText = textCollector[0]
+        return htmlText
+
+
+    def getEventDetails(self, calendar, language='en'):
+        """
+        Create a dictionary mapping slot names - specifically: summary,
+        description, location, dateInfo, timeInfo, durationInfo, recurrenceInfo,
+        url - with localized string values that should be placed into the HTML
+        and plain-text templates.
+
+        @param calendar: a L{Component} upon which to base the language.
+        @type calendar: L{Component}
+
+        @param language: a 2-letter language code.
+        @type language: C{str}
+
+        @return: a mapping from template slot name to localized text.
+        @rtype: a C{dict} mapping C{bytes} to C{unicode}.
+        """
+
+        # Get the most appropriate component
+        component = calendar.masterComponent()
+        if component is None:
+            component = calendar.mainComponent(True)
+
+        results = {}
+
+        dtStart = component.propertyValue('DTSTART')
+        results['month'] = dtStart.getMonth()
+        results['day'] = dtStart.getDay()
+
+        for propertyToResult in ['summary', 'description', 'location', 'url']:
+            result = component.propertyValue(propertyToResult.upper())
+            if result is None:
+                result = u""
+            else:
+                result = result.decode('utf-8')
+            results[propertyToResult] = result
+
+        with translationTo(language) as trans:
+            results['dateInfo'] = trans.date(component).decode('utf-8')
+            results['timeInfo'], duration = (x.decode('utf-8') for x in trans.time(component))
+            results['durationInfo'] = u"(%s)" % (duration,) if duration else u""
+
+            for propertyName in ('RRULE', 'RDATE', 'EXRULE', 'EXDATE',
+                                 'RECURRENCE-ID'):
+                if component.hasProperty(propertyName):
+                    results['recurrenceInfo'] = _("(Repeating)").decode('utf-8')
+                    break
+            else:
+                results['recurrenceInfo'] = u""
+
+        return results

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/scheduler.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/scheduler.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/scheduler.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,78 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.python.log import Logger
+from twext.web2.dav.http import ErrorResponse
+from twisted.internet.defer import succeed
+from twistedcaldav.caldavxml import caldav_namespace
+from txdav.caldav.datastore.scheduling.cuaddress import RemoteCalendarUser
+from txdav.caldav.datastore.scheduling.scheduler import RemoteScheduler
+from txdav.caldav.datastore.scheduling.scheduler import ScheduleResponseQueue
+
+
+"""
+L{IMIPScheduler} - handles deliveries for scheduling messages retrieved via
+mail
+"""
+
+__all__ = [
+    "IMIPScheduler",
+]
+
+log = Logger()
+
+class IMIPScheduler(RemoteScheduler):
+
+    scheduleResponse = ScheduleResponseQueue
+
+    errorResponse = ErrorResponse
+
+    errorElements = {
+        "originator-missing": (caldav_namespace, "originator-specified"),
+        "originator-invalid": (caldav_namespace, "originator-allowed"),
+        "originator-denied": (caldav_namespace, "originator-allowed"),
+        "recipient-missing": (caldav_namespace, "recipient-specified"),
+        "recipient-invalid": (caldav_namespace, "recipient-exists"),
+        "organizer-denied": (caldav_namespace, "organizer-allowed"),
+        "attendee-denied": (caldav_namespace, "attendee-allowed"),
+        "invalid-calendar-data-type": (caldav_namespace, "supported-calendar-data"),
+        "invalid-calendar-data": (caldav_namespace, "valid-calendar-data"),
+        "invalid-scheduling-message": (caldav_namespace, "valid-calendar-data"),
+        "max-recipients": (caldav_namespace, "recipient-limit"),
+    }
+
+    def checkAuthorization(self):
+        pass
+
+
+    def checkOriginator(self):
+        """
+        The originator always comes out of the tokens db
+        """
+        self.originator = RemoteCalendarUser(self.originator)
+        return succeed(None)
+
+
+    def checkOrganizerAsOriginator(self):
+        pass
+
+
+    def checkAttendeeAsOriginator(self):
+        pass
+
+
+    def securityChecks(self):
+        pass

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/smtpsender.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/smtpsender.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/smtpsender.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,76 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+SMTP sending utility
+"""
+
+from cStringIO import StringIO
+
+from twext.internet.adaptendpoint import connect
+from twext.internet.gaiendpoint import GAIEndpoint
+from twext.python.log import Logger
+from twisted.internet import defer, ssl, reactor as _reactor
+from twisted.mail.smtp import ESMTPSenderFactory
+
+log = Logger()
+
+class SMTPSender(object):
+
+    def __init__(self, username, password, useSSL, server, port):
+        self.username = username
+        self.password = password
+        self.useSSL = useSSL
+        self.server = server
+        self.port = port
+
+
+    def sendMessage(self, fromAddr, toAddr, msgId, message):
+
+        log.debug("Sending: %s" % (message,))
+        def _success(result, msgId, fromAddr, toAddr):
+            log.info("Sent IMIP message %s from %s to %s" %
+                (msgId, fromAddr, toAddr))
+            return True
+
+        def _failure(failure, msgId, fromAddr, toAddr):
+            log.error("Failed to send IMIP message %s from %s "
+                           "to %s (Reason: %s)" %
+                           (msgId, fromAddr, toAddr,
+                            failure.getErrorMessage()))
+            return False
+
+        deferred = defer.Deferred()
+
+        if self.useSSL:
+            contextFactory = ssl.ClientContextFactory()
+        else:
+            contextFactory = None
+
+        factory = ESMTPSenderFactory(
+            self.username, self.password,
+            fromAddr, toAddr,
+            # per http://trac.calendarserver.org/ticket/416 ...
+            StringIO(message.replace("\r\n", "\n")), deferred,
+            contextFactory=contextFactory,
+            requireAuthentication=False,
+            requireTransportSecurity=self.useSSL)
+
+        connect(GAIEndpoint(_reactor, self.server, self.port),
+                factory)
+        deferred.addCallback(_success, msgId, fromAddr, toAddr)
+        deferred.addErrback(_failure, msgId, fromAddr, toAddr)
+        return deferred

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/__init__.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/__init__.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_delivery.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_delivery.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_delivery.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,76 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.web2 import responsecode
+from twisted.internet.defer import inlineCallbacks
+from twistedcaldav.config import config
+from twistedcaldav.ical import Component
+
+from txdav.caldav.datastore.scheduling.cuaddress import RemoteCalendarUser
+from txdav.caldav.datastore.scheduling.imip.delivery import ScheduleViaIMip
+from txdav.caldav.datastore.scheduling.itip import iTIPRequestStatus
+from txdav.caldav.datastore.scheduling.scheduler import ScheduleResponseQueue
+
+import twistedcaldav.test.util
+
+class iMIPProcessing (twistedcaldav.test.util.TestCase):
+    """
+    iCalendar support tests
+    """
+
+    class FakeSchedule(object):
+
+        def __init__(self, calendar):
+            self.calendar = calendar
+
+
+    @inlineCallbacks
+    def test_no_freebusy(self):
+
+        data = """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REQUEST
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VFREEBUSY
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VFREEBUSY
+END:VCALENDAR
+"""
+
+        scheduler = iMIPProcessing.FakeSchedule(Component.fromString(data))
+        recipients = (RemoteCalendarUser("mailto:user1 at example.com"),)
+        responses = ScheduleResponseQueue("REQUEST", responsecode.OK)
+
+        delivery = ScheduleViaIMip(scheduler, recipients, responses, True)
+        yield delivery.generateSchedulingResponses()
+
+        self.assertEqual(len(responses.responses), 1)
+        self.assertEqual(str(responses.responses[0].children[1]), iTIPRequestStatus.SERVICE_UNAVAILABLE)
+
+
+    @inlineCallbacks
+    def test_matchCalendarUserAddress(self):
+        # iMIP not sensitive to case:
+        self.patch(config.Scheduling[ScheduleViaIMip.serviceType()], "AddressPatterns", ["mailto:.*"])
+        result = yield ScheduleViaIMip.matchCalendarUserAddress("mailto:user at xyzexample.com")
+        self.assertTrue(result)
+        result = ScheduleViaIMip.matchCalendarUserAddress("MAILTO:user at xyzexample.com")
+        self.assertTrue(result)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_inbound.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_inbound.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_inbound.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,365 @@
+##
+# Copyright (c) 2008-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+
+import email
+from twisted.internet.defer import inlineCallbacks
+from twisted.python.modules import getModule
+from twistedcaldav.ical import Component
+from twistedcaldav.scheduling.imip.inbound import MailReceiver
+from twistedcaldav.scheduling.imip.inbound import MailRetriever
+from twistedcaldav.scheduling.imip.inbound import injectMessage
+from twistedcaldav.scheduling.imip.inbound import IMIPReplyWork
+from twistedcaldav.scheduling.itip import iTIPRequestStatus
+from twistedcaldav.test.util import TestCase
+from twistedcaldav.test.util import xmlFile
+from txdav.common.datastore.test.util import buildStore
+from calendarserver.tap.util import getRootResource
+from twistedcaldav.config import config, ConfigDict
+
+
+class InboundTests(TestCase):
+
+    @inlineCallbacks
+    def setUp(self):
+        super(InboundTests, self).setUp()
+
+        self.store = yield buildStore(self, None)
+        self.patch(config.DirectoryService.params, "xmlFile", xmlFile)
+        self.root = getRootResource(config, self.store)
+        self.directory = self.root.getDirectory()
+        self.receiver = MailReceiver(self.store, self.directory)
+        self.retriever = MailRetriever(self.store, self.directory,
+            ConfigDict({
+                "Type" : "pop",
+                "UseSSL" : False,
+                "Server" : "example.com",
+                "Port" : 123,
+            })
+        )
+
+        def decorateTransaction(txn):
+            txn._rootResource = self.root
+            txn._mailRetriever = self.retriever
+
+        self.store.callWithNewTransactions(decorateTransaction)
+        module = getModule(__name__)
+        self.dataPath = module.filePath.sibling("data")
+
+
+    def dataFile(self, name):
+        """
+        Get the contents of a given data file from the 'data/mail' test
+        fixtures directory.
+        """
+        return self.dataPath.child(name).getContent()
+
+
+    def test_checkDSNFailure(self):
+
+        data = {
+            'good_reply' : (False, None, None),
+            'dsn_failure_no_original' : (True, 'failed', None),
+            'dsn_failure_no_ics' : (True, 'failed', None),
+            'dsn_failure_with_ics' : (True, 'failed', '''BEGIN:VCALENDAR
+VERSION:2.0
+CALSCALE:GREGORIAN
+METHOD:REQUEST
+PRODID:-//example Inc.//iCal 3.0//EN
+BEGIN:VTIMEZONE
+TZID:US/Pacific
+BEGIN:STANDARD
+DTSTART:20071104T020000
+RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
+TZNAME:PST
+TZOFFSETFROM:-0700
+TZOFFSETTO:-0800
+END:STANDARD
+BEGIN:DAYLIGHT
+DTSTART:20070311T020000
+RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
+TZNAME:PDT
+TZOFFSETFROM:-0800
+TZOFFSETTO:-0700
+END:DAYLIGHT
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:1E71F9C8-AEDA-48EB-98D0-76E898F6BB5C
+DTSTART;TZID=US/Pacific:20080812T094500
+DTEND;TZID=US/Pacific:20080812T104500
+ATTENDEE;CUTYPE=INDIVIDUAL;CN=User 01;PARTSTAT=ACCEPTED:mailto:user01 at exam
+ ple.com
+ATTENDEE;CUTYPE=INDIVIDUAL;RSVP=TRUE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-A
+ CTION;CN=nonexistant at example.com:mailto:nonexistant at example.com
+CREATED:20080812T191857Z
+DTSTAMP:20080812T191932Z
+ORGANIZER;CN=User 01:mailto:xyzzy+8e16b897-d544-4217-88e9-a363d08
+ 46f6c at example.com
+SEQUENCE:2
+SUMMARY:New Event
+TRANSP:OPAQUE
+END:VEVENT
+END:VCALENDAR
+'''),
+        }
+
+        for filename, expected in data.iteritems():
+            msg = email.message_from_string(self.dataFile(filename))
+            self.assertEquals(self.receiver.checkDSN(msg), expected)
+
+
+    @inlineCallbacks
+    def test_processDSN(self):
+
+        template = """BEGIN:VCALENDAR
+VERSION:2.0
+CALSCALE:GREGORIAN
+METHOD:REQUEST
+PRODID:-//example Inc.//iCal 3.0//EN
+BEGIN:VTIMEZONE
+TZID:US/Pacific
+BEGIN:DAYLIGHT
+DTSTART:20070311T020000
+RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
+TZNAME:PDT
+TZOFFSETFROM:-0800
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:20071104T020000
+RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
+TZNAME:PST
+TZOFFSETFROM:-0700
+TZOFFSETTO:-0800
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:1E71F9C8-AEDA-48EB-98D0-76E898F6BB5C
+DTSTART;TZID=US/Pacific:20080812T094500
+DTEND;TZID=US/Pacific:20080812T104500
+ATTENDEE;CUTYPE=INDIVIDUAL;CN=User 01;PARTSTAT=ACCEPTED:mailto:user01 at exam
+ ple.com
+ATTENDEE;CUTYPE=INDIVIDUAL;RSVP=TRUE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-A
+ CTION;CN=nonexistant at example.com:mailto:nonexistant at example.com
+CREATED:20080812T191857Z
+DTSTAMP:20080812T191932Z
+ORGANIZER;CN=User 01:mailto:xyzzy+%s at example.com
+SEQUENCE:2
+SUMMARY:New Event
+TRANSP:OPAQUE
+END:VEVENT
+END:VCALENDAR
+"""
+
+        # Make sure an unknown token is not processed
+        calBody = template % "bogus_token"
+        self.assertEquals(
+            (yield self.receiver.processDSN(calBody, "xyzzy")),
+            MailReceiver.UNKNOWN_TOKEN
+        )
+
+        # Make sure a known token *is* processed
+        txn = self.store.newTransaction()
+        token = (yield txn.imipCreateToken(
+            "urn:uuid:5A985493-EE2C-4665-94CF-4DFEA3A89500",
+            "mailto:user02 at example.com",
+            "1E71F9C8-AEDA-48EB-98D0-76E898F6BB5C"
+        ))
+        yield txn.commit()
+        calBody = template % token
+        result = (yield self.receiver.processDSN(calBody, "xyzzy"))
+        self.assertEquals(result, MailReceiver.INJECTION_SUBMITTED)
+
+
+    @inlineCallbacks
+    def test_processReply(self):
+        msg = email.message_from_string(self.dataFile('good_reply'))
+
+        # Make sure an unknown token is not processed
+        result = (yield self.receiver.processReply(msg))
+        self.assertEquals(result, MailReceiver.UNKNOWN_TOKEN)
+
+        # Make sure a known token *is* processed
+        txn = self.store.newTransaction()
+        yield txn.imipCreateToken(
+            "urn:uuid:5A985493-EE2C-4665-94CF-4DFEA3A89500",
+            "mailto:xyzzy at example.com",
+            "1E71F9C8-AEDA-48EB-98D0-76E898F6BB5C",
+            token="d7cdf68d-8b73-4df1-ad3b-f08002fb285f"
+        )
+        yield txn.commit()
+
+        result = (yield self.receiver.processReply(msg))
+        self.assertEquals(result, MailReceiver.INJECTION_SUBMITTED)
+
+
+    def test_processReplyMissingOrganizer(self):
+        msg = email.message_from_string(self.dataFile('reply_missing_organizer'))
+
+        # stick the token in the database first
+        txn = self.store.newTransaction()
+        yield txn.imipCreateToken(
+            "urn:uuid:5A985493-EE2C-4665-94CF-4DFEA3A89500",
+            "mailto:xyzzy at example.com",
+            "1E71F9C8-AEDA-48EB-98D0-76E898F6BB5C",
+            token="d7cdf68d-8b73-4df1-ad3b-f08002fb285f"
+        )
+        yield txn.commit()
+
+        result = (yield self.receiver.processReply(msg))
+        organizer, _ignore_attendee, calendar = result
+        organizerProp = calendar.mainComponent().getOrganizerProperty()
+        self.assertTrue(organizerProp is not None)
+        self.assertEquals(organizer,
+                          "urn:uuid:5A985493-EE2C-4665-94CF-4DFEA3A89500")
+
+
+    def test_processReplyMissingAttendee(self):
+        msg = email.message_from_string(self.dataFile('reply_missing_attendee'))
+
+        txn = self.store.newTransaction()
+        yield txn.imipCreateToken(
+            "urn:uuid:5A985493-EE2C-4665-94CF-4DFEA3A89500",
+            "mailto:xyzzy at example.com",
+            "1E71F9C8-AEDA-48EB-98D0-76E898F6BB5C",
+            token="d7cdf68d-8b73-4df1-ad3b-f08002fb285f"
+        )
+        yield txn.commit()
+
+        result = (yield self.receiver.processReply(msg))
+        _ignore_organizer, attendee, calendar = result
+
+        # Since the expected attendee was missing, the reply processor should
+        # have added an attendee back in with a "5.1;Service unavailable"
+        # schedule-status
+        attendeeProp = calendar.mainComponent().getAttendeeProperty([attendee])
+        self.assertEquals(attendeeProp.parameterValue("SCHEDULE-STATUS"),
+                          iTIPRequestStatus.SERVICE_UNAVAILABLE)
+
+
+    def test_processReplyMissingAttachment(self):
+
+        msg = email.message_from_string(
+            self.dataFile('reply_missing_attachment')
+        )
+
+        # stick the token in the database first
+        txn = self.store.newTransaction()
+        yield txn.imipCreateToken(
+            "urn:uuid:5A985493-EE2C-4665-94CF-4DFEA3A89500",
+            "mailto:xyzzy at example.com",
+            "1E71F9C8-AEDA-48EB-98D0-76E898F6BB5C",
+            token="d7cdf68d-8b73-4df1-ad3b-f08002fb285f"
+        )
+        yield txn.commit()
+
+        result = (yield self.receiver.processReply(msg))
+        self.assertEquals(result, MailReceiver.REPLY_FORWARDED_TO_ORGANIZER)
+
+
+    @inlineCallbacks
+    def test_injectMessage(self):
+
+        calendar = Component.fromString("""BEGIN:VCALENDAR
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+VERSION:2.0
+METHOD:REPLY
+BEGIN:VEVENT
+UID:12345-67890
+DTSTAMP:20130208T120000Z
+DTSTART:20180601T120000Z
+DTEND:20180601T130000Z
+ORGANIZER:urn:uuid:user01
+ATTENDEE:mailto:xyzzy at example.com;PARTSTAT=ACCEPTED
+END:VEVENT
+END:VCALENDAR
+""")
+
+        txn = self.store.newTransaction()
+        result = (yield injectMessage(
+                txn,
+                self.root,
+                "urn:uuid:user01",
+                "mailto:xyzzy at example.com",
+                calendar
+            )
+        )
+        yield txn.commit()
+        self.assertEquals(
+            "1.2;Scheduling message has been delivered",
+            result.responses[0].children[1].toString()
+        )
+
+
+    @inlineCallbacks
+    def test_injectMessageWithError(self):
+
+        calendar = Component.fromString("""BEGIN:VCALENDAR
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+VERSION:2.0
+METHOD:REPLY
+BEGIN:VEVENT
+UID:12345-67890
+DTSTAMP:20130208T120000Z
+DTSTART:20180601T120000Z
+DTEND:20180601T130000Z
+ORGANIZER:urn:uuid:unknown_user
+ATTENDEE:mailto:xyzzy at example.com;PARTSTAT=ACCEPTED
+END:VEVENT
+END:VCALENDAR
+""")
+
+        txn = self.store.newTransaction()
+        result = (yield injectMessage(
+                txn,
+                self.root,
+                "urn:uuid:unknown_user",
+                "mailto:xyzzy at example.com",
+                calendar
+            )
+        )
+        yield txn.commit()
+        self.assertEquals(
+            "3.7;Invalid Calendar User",
+            result.responses[0].children[1].toString()
+        )
+
+
+    @inlineCallbacks
+    def test_work(self):
+
+        calendar = """BEGIN:VCALENDAR
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+VERSION:2.0
+METHOD:REPLY
+BEGIN:VEVENT
+UID:12345-67890
+DTSTAMP:20130208T120000Z
+DTSTART:20180601T120000Z
+DTEND:20180601T130000Z
+ORGANIZER:urn:uuid:user01
+ATTENDEE:mailto:xyzzy at example.com;PARTSTAT=ACCEPTED
+END:VEVENT
+END:VCALENDAR
+"""
+        txn = self.store.newTransaction()
+        wp = (yield txn.enqueue(IMIPReplyWork,
+            organizer="urn:uuid:user01",
+            attendee="mailto:xyzzy at example.com",
+            icalendarText=calendar
+        ))
+        yield txn.commit()
+        yield wp.whenExecuted()

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_mailgateway.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_mailgateway.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_mailgateway.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,55 @@
+##
+# Copyright (c) 2008-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+import os
+
+from twisted.internet.defer import inlineCallbacks
+from twistedcaldav.test.util import TestCase
+from twistedcaldav.test.util import xmlFile
+from txdav.common.datastore.test.util import buildStore
+from calendarserver.tap.util import getRootResource
+from twistedcaldav.config import config
+from txdav.caldav.datastore.scheduling.imip.mailgateway import MailGatewayTokensDatabase
+from txdav.caldav.datastore.scheduling.imip.mailgateway import migrateTokensToStore
+
+
+class MailGatewayTokenDBTests(TestCase):
+
+    @inlineCallbacks
+    def setUp(self):
+        super(MailGatewayTokenDBTests, self).setUp()
+
+        self.store = yield buildStore(self, None)
+        self.patch(config.DirectoryService.params, "xmlFile", xmlFile)
+        self.root = getRootResource(config, self.store)
+        self.directory = self.root.getDirectory()
+
+
+    @inlineCallbacks
+    def test_migrate(self):
+        self.path = self.mktemp()
+        os.mkdir(self.path)
+        oldDB = MailGatewayTokensDatabase(self.path)
+        oldDB.createToken("urn:uuid:user01", "mailto:attendee at example.com",
+            "icaluid1", token="token1")
+        yield migrateTokensToStore(self.path, self.store)
+        txn = self.store.newTransaction()
+        results = yield (txn.imipLookupByToken("token1"))
+        organizer, attendee, icaluid = results[0]
+        yield txn.commit()
+        self.assertEquals(organizer, "urn:uuid:user01")
+        self.assertEquals(attendee, "mailto:attendee at example.com")
+        self.assertEquals(icaluid, "icaluid1")

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_outbound.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_outbound.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/imip/test/test_outbound.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,565 @@
+##
+# Copyright (c) 2008-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+from __future__ import print_function
+
+from cStringIO import StringIO
+import os
+
+import email
+from pycalendar.datetime import PyCalendarDateTime
+from twisted.internet.defer import inlineCallbacks, succeed
+from twisted.web.template import Element, renderer, flattenString
+from twistedcaldav.config import config
+from twistedcaldav.directory import augment
+from twistedcaldav.directory.xmlfile import XMLDirectoryService
+from twistedcaldav.ical import Component
+from twistedcaldav.scheduling.imip.outbound import IMIPInvitationWork
+from twistedcaldav.scheduling.imip.outbound import MailSender
+from twistedcaldav.scheduling.imip.outbound import StringFormatTemplateLoader
+from twistedcaldav.test.util import TestCase, xmlFile, augmentsFile
+from txdav.common.datastore.test.util import buildStore
+
+initialInviteText = u"""BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REQUEST
+BEGIN:VEVENT
+UID:CFDD5E46-4F74-478A-9311-B3FF905449C3
+DTSTART:20200325T154500Z
+DTEND:20200325T164500Z
+ATTENDEE;CN=Th\xe9 Attendee;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;RSVP=TRU
+ E:mailto:attendee at example.com
+ATTENDEE;CN=Th\xe9 Organizer;CUTYPE=INDIVIDUAL;EMAIL=organizer at example.com;P
+ ARTSTAT=ACCEPTED:urn:uuid:C3B38B00-4166-11DD-B22C-A07C87E02F6A
+ATTENDEE;CN=An Attendee without CUTYPE;EMAIL=nocutype at example.com;PARTSTAT=A
+ CCEPTED:urn:uuid:4DB528DC-3E60-44FA-9546-2A00FCDCFFAB
+ATTENDEE;EMAIL=nocn at example.com;PARTSTAT=ACCEPTED:urn:uuid:A592CF8B-4FC8-4E4
+ F-B543-B2F29A7EEB0B
+ORGANIZER;CN=Th\xe9 Organizer;EMAIL=organizer at example.com:urn:uuid:C3B38B00-
+ 4166-11DD-B22C-A07C87E02F6A
+SUMMARY:t\xe9sting outbound( )
+DESCRIPTION:awesome description with "<" and "&"
+END:VEVENT
+END:VCALENDAR
+"""
+
+ORGANIZER = "urn:uuid:C3B38B00-4166-11DD-B22C-A07C87E02F6A"
+ATTENDEE = "mailto:attendee at example.com"
+ICALUID = "CFDD5E46-4F74-478A-9311-B3FF905449C3"
+
+class DummySMTPSender(object):
+
+    def __init__(self):
+        self.reset()
+        self.shouldSucceed = True
+
+
+    def reset(self):
+        self.sendMessageCalled = False
+        self.fromAddr = None
+        self.toAddr = None
+        self.msgId = None
+        self.message = None
+
+
+    def sendMessage(self, fromAddr, toAddr, msgId, message):
+        self.sendMessageCalled = True
+        self.fromAddr = fromAddr
+        self.toAddr = toAddr
+        self.msgId = msgId
+        self.message = message
+        return succeed(self.shouldSucceed)
+
+
+
+class OutboundTests(TestCase):
+
+    @inlineCallbacks
+    def setUp(self):
+        self.store = yield buildStore(self, None)
+        self.directory = XMLDirectoryService(
+            {
+                'xmlFile' : xmlFile,
+                'augmentService' :
+                    augment.AugmentXMLDB(xmlFiles=(augmentsFile.path,)),
+            }
+        )
+        self.sender = MailSender("server at example.com", 7, DummySMTPSender(),
+            language="en")
+
+        def _getSender(ignored):
+            return self.sender
+        self.patch(IMIPInvitationWork, "getMailSender", _getSender)
+
+        self.wp = None
+        self.store.queuer.callWithNewProposals(self._proposalCallback)
+
+
+    def _proposalCallback(self, wp):
+        self.wp = wp
+
+
+    @inlineCallbacks
+    def test_work(self):
+        txn = self.store.newTransaction()
+        wp = (yield txn.enqueue(IMIPInvitationWork,
+            fromAddr=ORGANIZER,
+            toAddr=ATTENDEE,
+            icalendarText=initialInviteText.replace("\n", "\r\n"),
+        ))
+        self.assertEquals(wp, self.wp)
+        yield txn.commit()
+        yield wp.whenExecuted()
+
+        txn = self.store.newTransaction()
+        token = (yield txn.imipGetToken(
+            ORGANIZER,
+            ATTENDEE,
+            ICALUID
+        ))
+        self.assertTrue(token)
+        organizer, attendee, icaluid = (yield txn.imipLookupByToken(token))[0]
+        yield txn.commit()
+        self.assertEquals(organizer, ORGANIZER)
+        self.assertEquals(attendee, ATTENDEE)
+        self.assertEquals(icaluid, ICALUID)
+
+
+    @inlineCallbacks
+    def test_workFailure(self):
+        self.sender.smtpSender.shouldSucceed = False
+
+        txn = self.store.newTransaction()
+        wp = (yield txn.enqueue(IMIPInvitationWork,
+            fromAddr=ORGANIZER,
+            toAddr=ATTENDEE,
+            icalendarText=initialInviteText.replace("\n", "\r\n"),
+        ))
+        yield txn.commit()
+        yield wp.whenExecuted()
+        # Verify a new work proposal was not created
+        self.assertEquals(wp, self.wp)
+
+
+    def _interceptEmail(self, inviteState, calendar, orgEmail, orgCn,
+        attendees, fromAddress, replyToAddress, toAddress, language="en"):
+        self.inviteState = inviteState
+        self.calendar = calendar
+        self.orgEmail = orgEmail
+        self.orgCn = orgCn
+        self.attendees = attendees
+        self.fromAddress = fromAddress
+        self.replyToAddress = replyToAddress
+        self.toAddress = toAddress
+        self.language = language
+        self.results = self._actualGenerateEmail(inviteState, calendar,
+            orgEmail, orgCn, attendees, fromAddress, replyToAddress, toAddress,
+            language=language)
+        return self.results
+
+
+    @inlineCallbacks
+    def test_outbound(self):
+        """
+        Make sure outbound( ) stores tokens properly so they can be looked up
+        """
+
+        config.Scheduling.iMIP.Sending.Address = "server at example.com"
+        self.patch(config.Localization, "LocalesDirectory", os.path.join(os.path.dirname(__file__), "locales"))
+        self._actualGenerateEmail = self.sender.generateEmail
+        self.patch(self.sender, "generateEmail", self._interceptEmail)
+
+        data = (
+            # Initial invite
+            (
+                initialInviteText,
+                "CFDD5E46-4F74-478A-9311-B3FF905449C3",
+                "urn:uuid:C3B38B00-4166-11DD-B22C-A07C87E02F6A",
+                "mailto:attendee at example.com",
+                "new",
+                "organizer at example.com",
+                u"Th\xe9 Organizer",
+                [
+                    (u'Th\xe9 Attendee', u'attendee at example.com'),
+                    (u'Th\xe9 Organizer', u'organizer at example.com'),
+                    (u'An Attendee without CUTYPE', u'nocutype at example.com'),
+                    (None, u'nocn at example.com'),
+                ],
+                u"Th\xe9 Organizer <organizer at example.com>",
+                "=?utf-8?q?Th=C3=A9_Organizer_=3Corganizer=40example=2Ecom=3E?=",
+                "attendee at example.com",
+            ),
+
+            # Update
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REQUEST
+BEGIN:VEVENT
+UID:CFDD5E46-4F74-478A-9311-B3FF905449C3
+DTSTART:20100325T154500Z
+DTEND:20100325T164500Z
+ATTENDEE;CN=The Attendee;CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:
+ mailto:attendee at example.com
+ATTENDEE;CN=The Organizer;CUTYPE=INDIVIDUAL;EMAIL=organizer at example.com;PAR
+ TSTAT=ACCEPTED:urn:uuid:C3B38B00-4166-11DD-B22C-A07C87E02F6A
+ORGANIZER;CN=The Organizer;EMAIL=organizer at example.com:urn:uuid:C3B38B00-41
+ 66-11DD-B22C-A07C87E02F6A
+SUMMARY:testing outbound( ) *update*
+END:VEVENT
+END:VCALENDAR
+""",
+                "CFDD5E46-4F74-478A-9311-B3FF905449C3",
+                "urn:uuid:C3B38B00-4166-11DD-B22C-A07C87E02F6A",
+                "mailto:attendee at example.com",
+                "update",
+                "organizer at example.com",
+                "The Organizer",
+                [
+                    (u'The Attendee', u'attendee at example.com'),
+                    (u'The Organizer', u'organizer at example.com')
+                ],
+                "The Organizer <organizer at example.com>",
+                "The Organizer <organizer at example.com>",
+                "attendee at example.com",
+            ),
+
+            # Reply
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REPLY
+BEGIN:VEVENT
+UID:DFDD5E46-4F74-478A-9311-B3FF905449C4
+DTSTART:20100325T154500Z
+DTEND:20100325T164500Z
+ATTENDEE;CN=The Attendee;CUTYPE=INDIVIDUAL;EMAIL=attendee at example.com;PARTST
+ AT=ACCEPTED:urn:uuid:C3B38B00-4166-11DD-B22C-A07C87E02F6A
+ORGANIZER;CN=The Organizer;EMAIL=organizer at example.com:mailto:organizer at exam
+ ple.com
+SUMMARY:testing outbound( ) *reply*
+END:VEVENT
+END:VCALENDAR
+""",
+                None,
+                "urn:uuid:C3B38B00-4166-11DD-B22C-A07C87E02F6A",
+                "mailto:organizer at example.com",
+                "reply",
+                "organizer at example.com",
+                "The Organizer",
+                [
+                    (u'The Attendee', u'attendee at example.com'),
+                ],
+                "attendee at example.com",
+                "attendee at example.com",
+                "organizer at example.com",
+            ),
+
+        )
+        for (inputCalendar, UID, inputOriginator, inputRecipient, inviteState,
+            outputOrganizerEmail, outputOrganizerName, outputAttendeeList,
+            outputFrom, encodedFrom, outputRecipient) in data:
+
+            txn = self.store.newTransaction()
+            yield self.sender.outbound(
+                txn,
+                inputOriginator,
+                inputRecipient,
+                Component.fromString(inputCalendar.replace("\n", "\r\n")),
+                onlyAfter=PyCalendarDateTime(2010, 1, 1, 0, 0, 0)
+            )
+            yield txn.commit()
+
+            msg = email.message_from_string(self.sender.smtpSender.message)
+            self.assertEquals(msg["From"], encodedFrom)
+            self.assertEquals(self.inviteState, inviteState)
+            self.assertEquals(self.orgEmail, outputOrganizerEmail)
+            self.assertEquals(self.orgCn, outputOrganizerName)
+            self.assertEquals(self.attendees, outputAttendeeList)
+            self.assertEquals(self.fromAddress, outputFrom)
+            self.assertEquals(self.toAddress, outputRecipient)
+
+            if UID: # The organizer is local, and server is sending to remote
+                    # attendee
+                txn = self.store.newTransaction()
+                token = (yield txn.imipGetToken(inputOriginator, inputRecipient,
+                    UID))
+                yield txn.commit()
+                self.assertNotEquals(token, None)
+                self.assertEquals(msg["Reply-To"],
+                    "server+%s at example.com" % (token,))
+
+                # Make sure attendee property for organizer exists and matches
+                # the CUA of the organizer property
+                orgValue = self.calendar.getOrganizerProperty().value()
+                self.assertEquals(
+                    orgValue,
+                    self.calendar.getAttendeeProperty([orgValue]).value()
+                )
+
+            else: # Reply only -- the attendee is local, and server is sending reply to remote organizer
+
+                self.assertEquals(msg["Reply-To"], self.fromAddress)
+
+            # Check that we don't send any messages for events completely in
+            # the past.
+            self.sender.smtpSender.reset()
+            txn = self.store.newTransaction()
+            yield self.sender.outbound(
+                txn,
+                inputOriginator,
+                inputRecipient,
+                Component.fromString(inputCalendar.replace("\n", "\r\n")),
+                onlyAfter=PyCalendarDateTime(2021, 1, 1, 0, 0, 0)
+            )
+            yield txn.commit()
+            self.assertFalse(self.sender.smtpSender.sendMessageCalled)
+
+
+    @inlineCallbacks
+    def test_tokens(self):
+        txn = self.store.newTransaction()
+        token = (yield txn.imipLookupByToken("xyzzy"))
+        yield txn.commit()
+        self.assertEquals(token, [])
+
+        txn = self.store.newTransaction()
+        token1 = (yield txn.imipCreateToken("organizer", "attendee", "icaluid"))
+        yield txn.commit()
+
+        txn = self.store.newTransaction()
+        token2 = (yield txn.imipGetToken("organizer", "attendee", "icaluid"))
+        yield txn.commit()
+        self.assertEquals(token1, token2)
+
+        txn = self.store.newTransaction()
+        self.assertEquals((yield txn.imipLookupByToken(token1)),
+            [["organizer", "attendee", "icaluid"]])
+        yield txn.commit()
+
+        txn = self.store.newTransaction()
+        yield txn.imipRemoveToken(token1)
+        yield txn.commit()
+
+        txn = self.store.newTransaction()
+        self.assertEquals((yield txn.imipLookupByToken(token1)), [])
+        yield txn.commit()
+
+
+    @inlineCallbacks
+    def test_mailtoTokens(self):
+        """
+        Make sure old mailto tokens are still honored
+        """
+
+        organizerEmail = "mailto:organizer at example.com"
+
+        # Explictly store a token with mailto: CUA for organizer
+        # (something that doesn't happen any more, but did in the past)
+        txn = self.store.newTransaction()
+        origToken = (yield txn.imipCreateToken(organizerEmail,
+            "mailto:attendee at example.com",
+            "CFDD5E46-4F74-478A-9311-B3FF905449C3"
+            )
+        )
+        yield txn.commit()
+
+        inputCalendar = initialInviteText
+        UID = "CFDD5E46-4F74-478A-9311-B3FF905449C3"
+        inputOriginator = "urn:uuid:C3B38B00-4166-11DD-B22C-A07C87E02F6A"
+        inputRecipient = "mailto:attendee at example.com"
+
+        txn = self.store.newTransaction()
+        yield self.sender.outbound(txn, inputOriginator, inputRecipient,
+            Component.fromString(inputCalendar.replace("\n", "\r\n")),
+            onlyAfter=PyCalendarDateTime(2010, 1, 1, 0, 0, 0))
+
+        # Verify we didn't create a new token...
+        txn = self.store.newTransaction()
+        token = (yield txn.imipGetToken(inputOriginator, inputRecipient, UID))
+        yield txn.commit()
+        self.assertEquals(token, None)
+
+        # But instead kept the old one...
+        txn = self.store.newTransaction()
+        token = (yield txn.imipGetToken(organizerEmail, inputRecipient, UID))
+        yield txn.commit()
+        self.assertEquals(token, origToken)
+
+
+    def generateSampleEmail(self):
+        """
+        Invoke L{MailHandler.generateEmail} and parse the result.
+        """
+        calendar = Component.fromString(initialInviteText)
+        msgID, msgTxt = self.sender.generateEmail(
+            inviteState='new',
+            calendar=calendar,
+            orgEmail=u"user01 at localhost",
+            orgCN=u"User Z\xe9ro One",
+            attendees=[(u"Us\xe9r One", "user01 at localhost"),
+                       (u"User 2", "user02 at localhost")],
+            fromAddress="user01 at localhost",
+            replyToAddress="imip-system at localhost",
+            toAddress="user03 at localhost",
+        )
+        message = email.message_from_string(msgTxt)
+        return msgID, message
+
+
+    def test_generateEmail(self):
+        """
+        L{MailHandler.generateEmail} generates a MIME-formatted email with a
+        text/plain part, a text/html part, and a text/calendar part.
+        """
+        msgID, message = self.generateSampleEmail()
+        self.assertEquals(message['Message-ID'], msgID)
+        expectedTypes = set(["text/plain", "text/html", "text/calendar"])
+        actualTypes = set([
+            part.get_content_type() for part in message.walk()
+            if part.get_content_type().startswith("text/")
+        ])
+        self.assertEquals(actualTypes, expectedTypes)
+
+
+    def test_emailEncoding(self):
+        """
+        L{MailHandler.generateEmail} will preserve any non-ASCII characters
+        present in the fields that it formats in the message body.
+        """
+        _ignore_msgID, message = self.generateSampleEmail()
+        textPart = partByType(message, "text/plain")
+        htmlPart = partByType(message, "text/html")
+
+        plainText = textPart.get_payload(decode=True).decode(
+            textPart.get_content_charset()
+        )
+        htmlText = htmlPart.get_payload(decode=True).decode(
+            htmlPart.get_content_charset()
+        )
+
+        self.assertIn(u"Us\u00e9r One", plainText)
+        self.assertIn(u'<a href="mailto:user01 at localhost">Us\u00e9r One</a>',
+                      htmlText)
+
+        # The same assertion, but with the organizer's form.
+        self.assertIn(
+            u'<a href="mailto:user01 at localhost">User Z\u00e9ro One</a>',
+            htmlText)
+
+
+    def test_emailQuoting(self):
+        """
+        L{MailHandler.generateEmail} will HTML-quote all relevant fields in the
+        HTML part, but not the text/plain part.
+        """
+        _ignore_msgID, message = self.generateSampleEmail()
+        htmlPart = partByType(message, "text/html").get_payload(decode=True)
+        plainPart = partByType(message, "text/plain").get_payload(decode=True)
+        expectedPlain = 'awesome description with "<" and "&"'
+        expectedHTML = expectedPlain.replace("&", "&amp;").replace("<", "&lt;")
+
+        self.assertIn(expectedPlain, plainPart)
+        self.assertIn(expectedHTML, htmlPart)
+
+
+    def test_stringFormatTemplateLoader(self):
+        """
+        L{StringFormatTemplateLoader.load} will convert a template with
+        C{%(x)s}-format slots by converting it to a template with C{<t:slot
+        name="x" />} slots, and a renderer on the document element named
+        according to the constructor argument.
+        """
+        class StubElement(Element):
+            loader = StringFormatTemplateLoader(
+                lambda : StringIO(
+                    "<test><alpha>%(slot1)s</alpha>%(other)s</test>"
+                ),
+                "testRenderHere"
+            )
+
+            @renderer
+            def testRenderHere(self, request, tag):
+                return tag.fillSlots(slot1="hello",
+                                     other="world")
+        result = []
+        flattenString(None, StubElement()).addCallback(result.append)
+        self.assertEquals(result,
+                          ["<test><alpha>hello</alpha>world</test>"])
+
+
+    def test_templateLoaderWithAttributes(self):
+        """
+        L{StringFormatTemplateLoader.load} will convert a template with
+        C{%(x)s}-format slots inside attributes into t:attr elements containing
+        t:slot slots.
+        """
+        class StubElement(Element):
+            loader = StringFormatTemplateLoader(
+                lambda : StringIO(
+                    '<test><alpha beta="before %(slot1)s after">inner</alpha>'
+                    '%(other)s</test>'
+                ),
+                "testRenderHere"
+            )
+
+            @renderer
+            def testRenderHere(self, request, tag):
+                return tag.fillSlots(slot1="hello",
+                                     other="world")
+        result = []
+        flattenString(None, StubElement()).addCallback(result.append)
+        self.assertEquals(result,
+                          ['<test><alpha beta="before hello after">'
+                           'inner</alpha>world</test>'])
+
+
+    def test_templateLoaderTagSoup(self):
+        """
+        L{StringFormatTemplateLoader.load} will convert a template with
+        C{%(x)s}-format slots into t:slot slots, and render a well-formed output
+        document, even if the input is malformed (i.e. missing necessary closing
+        tags).
+        """
+        class StubElement(Element):
+            loader = StringFormatTemplateLoader(
+                lambda : StringIO(
+                    '<test><alpha beta="before %(slot1)s after">inner</alpha>'
+                    '%(other)s'
+                ),
+                "testRenderHere"
+            )
+
+            @renderer
+            def testRenderHere(self, request, tag):
+                return tag.fillSlots(slot1="hello",
+                                     other="world")
+        result = []
+        flattenString(None, StubElement()).addCallback(result.append)
+        self.assertEquals(result,
+                          ['<test><alpha beta="before hello after">'
+                           'inner</alpha>world</test>'])
+
+
+
+def partByType(message, contentType):
+    """
+    Retrieve a MIME part from an L{email.message.Message} based on a content
+    type.
+    """
+    for part in message.walk():
+        if part.get_content_type() == contentType:
+            return part
+    raise KeyError(contentType)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/implicit.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/implicit.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/implicit.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,1243 @@
+#
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.python.log import Logger
+from twext.web2.dav.http import ErrorResponse
+
+from twisted.internet.defer import inlineCallbacks, returnValue
+from twext.web2 import responsecode
+from twext.web2.http import HTTPError
+
+from twistedcaldav import caldavxml
+from twistedcaldav.caldavxml import caldav_namespace
+from twistedcaldav.config import config
+from twistedcaldav.ical import Property
+
+from txdav.caldav.datastore.scheduling import addressmapping
+from txdav.caldav.datastore.scheduling.caldav.scheduler import CalDAVScheduler
+from txdav.caldav.datastore.scheduling.cuaddress import InvalidCalendarUser, \
+    LocalCalendarUser, PartitionedCalendarUser, OtherServerCalendarUser, \
+    normalizeCUAddr
+from txdav.caldav.datastore.scheduling.icaldiff import iCalDiff
+from txdav.caldav.datastore.scheduling.itip import iTipGenerator, iTIPRequestStatus
+from txdav.caldav.datastore.scheduling.utils import getCalendarObjectForPrincipals
+
+__all__ = [
+    "ImplicitScheduler",
+]
+
+log = Logger()
+
+# TODO:
+#
+# Handle the case where a PUT removes the ORGANIZER property. That should be equivalent to cancelling the entire meeting.
+# Support Schedule-Reply header
+#
+
+class ImplicitScheduler(object):
+
+    # Return Status codes
+    STATUS_OK = 0
+    STATUS_ORPHANED_CANCELLED_EVENT = 1
+    STATUS_ORPHANED_EVENT = 2
+
+    def __init__(self):
+
+        self.return_status = ImplicitScheduler.STATUS_OK
+        self.logItems = {}
+
+
+    @inlineCallbacks
+    def testImplicitSchedulingPUT(self, parent, resource, calendar, internal_request=False):
+        """
+        Determine whether a store operation is a valid scheduling operation.
+
+        @param parent: the parent (calendar) store object
+        @type parent: L{txdav.caldav.datastore.sql.Calendar}
+        @param resource: the store object - will be C{None} if creating a new one
+        @type resource: L{txdav.caldav.datastore.sql.CalendarObject}
+        @param calendar: the calendar data to store
+        @type calendar: L{twistedcaldav.ical.Component}
+        @param internal_request: whether or not the store originated from within the server itself
+        @type internal_request: C{bool}
+        """
+
+        self.parent = parent
+        self.resource = resource
+        self.calendar = calendar
+        self.internal_request = internal_request
+
+        self.calendar_home = self.parent.viewerHome()
+        self.calendar_owner = self.calendar_home.uid()
+
+        existing_resource = resource is not None
+        is_scheduling_object = (yield self.checkSchedulingObjectResource(resource))
+        existing_type = "schedule" if is_scheduling_object else "calendar"
+        new_type = "schedule" if (yield self.checkImplicitState()) else "calendar"
+
+        # If the types do not currently match, re-check the stored one. We need this to work around the possibility
+        # that data exists using the older algorithm of determining a scheduling object resource, and that could be
+        # wrong.
+        if existing_type != new_type and existing_resource:
+            resource.isScheduleObject = None
+            is_scheduling_object = (yield self.checkSchedulingObjectResource(resource))
+            existing_type = "schedule" if is_scheduling_object else "calendar"
+
+        if existing_type == "calendar":
+            self.action = "create" if new_type == "schedule" else "none"
+        else:
+            self.action = "modify" if new_type == "schedule" else "remove"
+
+        # Cannot create new resource with existing UID
+        if not existing_resource or self.action == "create":
+            yield self.hasCalendarResourceUIDSomewhereElse(resource, new_type)
+
+        # If action is remove we actually need to get state from the existing scheduling object resource
+        if self.action == "remove":
+            # Also make sure that we return the new calendar being written rather than the old one
+            # when the implicit action is executed
+            self.return_calendar = calendar
+            self.calendar = (yield resource.componentForUser(self.calendar_owner))
+            yield self.checkImplicitState()
+
+        # Once we have collected sufficient information from the calendar data, check validity of organizer and attendees
+        self.checkValidOrganizer()
+
+        # Attendees are not allowed to overwrite one type with another
+        if (
+            not self.internal_request and
+            self.state == "attendee" and
+            (existing_type != new_type) and
+            existing_resource
+        ):
+            raise HTTPError(ErrorResponse(
+                responsecode.FORBIDDEN,
+                (caldav_namespace, "valid-attendee-change"),
+                "Cannot change scheduling object mode",
+            ))
+
+        # Organizer events must have a master component
+        if self.state == "organizer" and self.calendar.masterComponent() is None:
+            raise HTTPError(ErrorResponse(
+                responsecode.FORBIDDEN,
+                (caldav_namespace, "organizer-allowed"),
+                "Organizer cannot schedule without a master component.",
+            ))
+
+        returnValue((self.action != "none", new_type == "schedule",))
+
+
+    @inlineCallbacks
+    def testImplicitSchedulingDELETE(self, parent, resource, calendar, internal_request=False):
+        """
+        Determine whether a store operation is a valid scheduling operation.
+
+        @param parent: the parent (calendar) store object
+        @type parent: L{txdav.caldav.datastore.sql.Calendar}
+        @param resource: the store object
+        @type resource: L{txdav.caldav.datastore.sql.CalendarObject}
+        @param calendar: the calendar data being removed
+        @type calendar: L{twistedcaldav.ical.Component}
+        @param internal_request: whether or not the store originated from within the server itself
+        @type internal_request: C{bool}
+        """
+
+        self.parent = parent
+        self.resource = resource
+        self.calendar = calendar
+        self.internal_request = internal_request
+
+        self.calendar_home = self.parent.viewerHome()
+        self.calendar_owner = self.calendar_home.uid()
+
+        yield self.checkImplicitState()
+
+        is_scheduling_object = (yield self.checkSchedulingObjectResource(resource))
+        resource_type = "schedule" if is_scheduling_object else "calendar"
+        self.action = "remove" if resource_type == "schedule" else "none"
+
+        returnValue((self.action != "none", False,))
+
+
+    @inlineCallbacks
+    def testAttendeeEvent(self, parent, resource, calendar):
+        """
+        Test the existing resource to see if it is an Attendee scheduling object resource.
+
+        @param parent: the parent (calendar) store object
+        @type parent: L{txdav.caldav.datastore.sql.Calendar}
+        @param resource: the store object
+        @type resource: L{txdav.caldav.datastore.sql.CalendarObject}
+        @param calendar: the calendar data being tested
+        @type calendar: L{twistedcaldav.ical.Component}
+        """
+
+        self.parent = parent
+        self.resource = resource
+        self.calendar = calendar
+        self.internal_request = False
+        self.action = "modify"
+
+        self.calendar_home = self.parent.viewerHome()
+        self.calendar_owner = self.calendar_home.uid()
+
+        is_scheduling_object = (yield self.checkSchedulingObjectResource(resource))
+        if not is_scheduling_object:
+            returnValue(False)
+
+        yield self.checkImplicitState()
+
+        returnValue(self.state in ("attendee", "attendee-missing",))
+
+
+    def checkValidOrganizer(self):
+        """
+        Make sure the ORGANIZER is allowed to do certain scheduling operations.
+        """
+
+        # Check to see whether the organizer principal is enabled for scheduling. If not, do not allow them
+        # to create new scheduling resources.
+        if self.action == "create":
+            if self.organizerPrincipal and not self.organizerPrincipal.enabledAsOrganizer():
+                log.err("ORGANIZER not allowed to be an Organizer: %s" % (self.organizer,))
+                raise HTTPError(ErrorResponse(
+                    responsecode.FORBIDDEN,
+                    (caldav_namespace, "organizer-allowed"),
+                    "Organizer cannot schedule",
+                ))
+
+
+    @inlineCallbacks
+    def checkSchedulingObjectResource(self, resource):
+
+        if resource is not None:
+            implicit = resource.isScheduleObject
+            if implicit is not None:
+                returnValue(implicit)
+            else:
+                calendar = (yield resource.componentForUser(self.calendar_owner))
+                # Get the ORGANIZER and verify it is the same for all components
+                try:
+                    organizer = calendar.validOrganizerForScheduling()
+                except ValueError:
+                    # We have different ORGANIZERs in the same iCalendar object - this is an error
+                    returnValue(False)
+
+                # Any ORGANIZER => a scheduling object resource
+                returnValue(organizer is not None)
+
+        returnValue(False)
+
+
+    @inlineCallbacks
+    def checkImplicitState(self):
+        # Get some useful information from the calendar
+        yield self.extractCalendarData()
+
+        # Determine what type of scheduling this is: Organizer triggered or Attendee triggered
+        organizer_scheduling = (yield self.isOrganizerScheduling())
+        if organizer_scheduling:
+            self.state = "organizer"
+        elif self.isAttendeeScheduling():
+            self.state = "attendee"
+        elif self.organizer:
+            # There is an ORGANIZER that is not this user but no ATTENDEE property for
+            # the user.
+            self.state = "attendee-missing"
+        else:
+            self.state = None
+
+        returnValue(self.state is not None)
+
+
+    @inlineCallbacks
+    def doImplicitScheduling(self, do_smart_merge=False):
+        """
+        Do implicit scheduling operation based on the data already set by call to checkImplicitScheduling.
+
+        @param do_smart_merge: if True, merge attendee data on disk with new data being stored,
+            else overwrite data on disk.
+        @return: a new calendar object modified with scheduling information,
+            or C{None} if nothing happened or C{int} if some other state occurs
+        """
+
+        # Setup some parameters
+        self.do_smart_merge = do_smart_merge
+        self.except_attendees = ()
+        self.only_refresh_attendees = None
+
+        # Determine what type of scheduling this is: Organizer triggered or Attendee triggered
+        if self.state == "organizer":
+            yield self.doImplicitOrganizer()
+        elif self.state == "attendee":
+            yield self.doImplicitAttendee()
+        elif self.state == "attendee-missing":
+            yield self.doImplicitMissingAttendee()
+        else:
+            returnValue(None)
+
+        if self.return_status:
+            returnValue(self.return_status)
+        else:
+            returnValue(self.return_calendar if hasattr(self, "return_calendar") else self.calendar)
+
+
+    @inlineCallbacks
+    def refreshAllAttendeesExceptSome(self, request, resource, except_attendees=(), only_attendees=None):
+        """
+        Refresh the iCalendar data for all attendees except the one specified in attendees.
+        """
+
+        self.request = request
+        self.resource = resource
+        self.calendar = (yield self.resource.iCalendarForUser(self.request))
+        self.state = "organizer"
+        self.action = "modify"
+
+        self.calendar_owner = None
+        self.internal_request = True
+        self.except_attendees = except_attendees
+        self.only_refresh_attendees = only_attendees
+        self.changed_rids = None
+        self.reinvites = None
+
+        # Get some useful information from the calendar
+        yield self.extractCalendarData()
+        self.organizerPrincipal = self.home.principalForCalendarUserAddress(self.organizer)
+        self.organizerAddress = (yield addressmapping.mapper.getCalendarUser(self.organizer, self.organizerPrincipal))
+
+        # Originator is the organizer in this case
+        self.originatorPrincipal = self.organizerPrincipal
+        self.originator = self.organizer
+
+        # We want to suppress chatty iMIP messages when other attendees reply
+        self.request.suppressRefresh = False
+
+        for attendee in self.calendar.getAllAttendeeProperties():
+            if attendee.parameterValue("PARTSTAT", "NEEDS-ACTION").upper() == "NEEDS-ACTION":
+                self.request.suppressRefresh = True
+
+        if hasattr(self.request, "doing_attendee_refresh"):
+            self.request.doing_attendee_refresh += 1
+        else:
+            self.request.doing_attendee_refresh = 1
+        try:
+            refreshCount = (yield self.processRequests())
+        finally:
+            self.request.doing_attendee_refresh -= 1
+            if self.request.doing_attendee_refresh == 0:
+                delattr(self.request, "doing_attendee_refresh")
+
+        if refreshCount:
+            self.logItems["itip.refreshes"] = refreshCount
+
+
+    @inlineCallbacks
+    def sendAttendeeReply(self, request, resource, calendar, attendee):
+
+        self.request = request
+        self.resource = resource
+        self.calendar = calendar
+        self.action = "modify"
+        self.state = "attendee"
+
+        self.calendar_owner = None
+        self.internal_request = True
+        self.changed_rids = None
+
+        # Get some useful information from the calendar
+        yield self.extractCalendarData()
+
+        self.originator = self.attendee = attendee.principal.canonicalCalendarUserAddress()
+        self.attendeePrincipal = attendee.principal
+
+        result = (yield self.scheduleWithOrganizer())
+
+        returnValue(result)
+
+
+    @inlineCallbacks
+    def extractCalendarData(self):
+
+        # Get the originator who is the owner of the calendar resource being modified
+        self.originatorPrincipal = None
+        self.originator = ""
+        self.originatorPrincipal = self.calendar_home.principalForUID(self.calendar_owner)
+
+        # Pick the canonical CUA:
+        self.originator = self.originatorPrincipal.canonicalCalendarUserAddress()
+
+        # Get the ORGANIZER and verify it is the same for all components
+        try:
+            self.organizer = self.calendar.validOrganizerForScheduling()
+        except ValueError:
+            # We have different ORGANIZERs in the same iCalendar object - this is an error
+            log.error("Only one ORGANIZER is allowed in an iCalendar object:\n%s" % (self.calendar,))
+            raise HTTPError(ErrorResponse(
+                responsecode.FORBIDDEN,
+                (caldav_namespace, "single-organizer"),
+                "Only one organizer allowed in scheduling object resource",
+            ))
+
+        # Coerce any local with SCHEDULE-AGENT=CLIENT
+        yield self.coerceAttendeeScheduleAgent()
+
+        # Get the ATTENDEEs
+        self.attendeesByInstance = self.calendar.getAttendeesByInstance(True, onlyScheduleAgentServer=True)
+        self.instances = set(self.calendar.getComponentInstances())
+        self.attendees = set()
+        for attendee, _ignore in self.attendeesByInstance:
+            self.attendees.add(attendee)
+
+        # Some other useful things
+        self.uid = self.calendar.resourceUID()
+
+
+    @inlineCallbacks
+    def hasCalendarResourceUIDSomewhereElse(self, check_resource, mode):
+        """
+        See if a calendar component with a matching UID exists anywhere in the calendar home of the
+        current recipient owner and is not the resource being targeted.
+        """
+
+        # Don't care in some cases
+        if self.internal_request or self.action == "remove":
+            returnValue(None)
+
+        # Check for matching resource somewhere else in the home
+        foundElsewhere = (yield self.calendar_home.hasCalendarResourceUIDSomewhereElse(self.uid, check_resource, mode))
+        if foundElsewhere:
+            log.debug("Implicit - found component with same UID in a different collection: %s" % (check_resource,))
+            raise HTTPError(ErrorResponse(
+                responsecode.FORBIDDEN,
+                (caldav_namespace, "unique-scheduling-object-resource"),
+                "Cannot duplicate scheduling object resource",
+            ))
+
+
+    @inlineCallbacks
+    def isOrganizerScheduling(self):
+        """
+        Test whether this is a scheduling operation by an organizer
+        """
+
+        # First must have organizer property
+        if not self.organizer:
+            returnValue(False)
+
+        # Organizer must map to a valid principal
+        self.organizerPrincipal = self.calendar_home.principalForCalendarUserAddress(self.organizer)
+        self.organizerAddress = (yield addressmapping.mapper.getCalendarUser(self.organizer, self.organizerPrincipal))
+        if not self.organizerPrincipal:
+            returnValue(False)
+
+        # Organizer must be the owner of the calendar resource
+        if self.calendar_owner != self.organizerPrincipal.uid():
+            returnValue(False)
+
+        returnValue(True)
+
+
+    def isAttendeeScheduling(self):
+
+        # First must have organizer property
+        if not self.organizer:
+            return False
+
+        # Check to see whether any attendee is the owner
+        for attendee in self.attendees:
+            attendeePrincipal = self.calendar_home.principalForCalendarUserAddress(attendee)
+            if attendeePrincipal and attendeePrincipal.uid() == self.calendar_owner:
+                self.attendee = attendee
+                self.attendeePrincipal = attendeePrincipal
+                return True
+
+        return False
+
+
+    def makeScheduler(self):
+        """
+        Convenience method which we can override in unit tests to make testing easier.
+        """
+        return CalDAVScheduler(self.calendar_home, self.parent, self.resource, logItems=self.logItems)
+
+
+    @inlineCallbacks
+    def doImplicitOrganizer(self):
+
+        self.oldcalendar = None
+        self.changed_rids = None
+        self.cancelledAttendees = ()
+        self.reinvites = None
+        self.needs_action_rids = None
+
+        self.needs_sequence_change = False
+
+        # Check for a delete
+        if self.action == "remove":
+
+            log.debug("Implicit - organizer '%s' is removing UID: '%s'" % (self.organizer, self.uid))
+            self.oldcalendar = self.calendar
+
+            # Cancel all attendees
+            self.cancelledAttendees = [(attendee, None) for attendee in self.attendees]
+
+            # CANCEL always bumps sequence
+            self.needs_sequence_change = True
+
+        # Check for a new resource or an update
+        elif self.action == "modify":
+
+            # Read in existing data
+            self.oldcalendar = (yield self.resource.componentForUser(self.calendar_owner))
+            self.oldAttendeesByInstance = self.oldcalendar.getAttendeesByInstance(True, onlyScheduleAgentServer=True)
+            self.oldInstances = set(self.oldcalendar.getComponentInstances())
+            self.coerceAttendeesPartstatOnModify()
+
+            # Don't allow any SEQUENCE to decrease
+            if self.oldcalendar:
+                self.calendar.sequenceInSync(self.oldcalendar)
+
+            # Significant change
+            no_change, self.changed_rids, self.needs_action_rids, reinvites, recurrence_reschedule = self.isOrganizerChangeInsignificant()
+            if no_change:
+                if reinvites:
+                    log.debug("Implicit - organizer '%s' is re-inviting UID: '%s', attendees: %s" % (self.organizer, self.uid, ", ".join(reinvites)))
+                    self.reinvites = reinvites
+                else:
+                    # Nothing to do
+                    log.debug("Implicit - organizer '%s' is modifying UID: '%s' but change is not significant" % (self.organizer, self.uid))
+                    returnValue(None)
+            else:
+                log.debug("Implicit - organizer '%s' is modifying UID: '%s'" % (self.organizer, self.uid))
+
+                for rid in self.needs_action_rids:
+                    comp = self.calendar.overriddenComponent(rid)
+                    if comp is None:
+                        comp = self.calendar.deriveInstance(rid)
+                        self.calendar.addComponent(comp)
+
+                    for attendee in comp.getAllAttendeeProperties():
+                        if attendee.hasParameter("PARTSTAT"):
+                            cuaddr = attendee.value()
+
+                            if cuaddr in self.organizerPrincipal.calendarUserAddresses():
+                                # If the attendee is the organizer then do not update
+                                # the PARTSTAT to NEEDS-ACTION.
+                                # The organizer is automatically ACCEPTED to the event.
+                                continue
+
+                            attendee.setParameter("PARTSTAT", "NEEDS-ACTION")
+
+                # Check for removed attendees
+                if not recurrence_reschedule:
+                    self.findRemovedAttendees()
+                else:
+                    self.findRemovedAttendeesOnRecurrenceChange()
+
+                # For now we always bump the sequence number on modifications because we cannot track DTSTAMP on
+                # the Attendee side. But we check the old and the new and only bump if the client did not already do it.
+                self.needs_sequence_change = self.calendar.needsiTIPSequenceChange(self.oldcalendar)
+
+        elif self.action == "create":
+            log.debug("Implicit - organizer '%s' is creating UID: '%s'" % (self.organizer, self.uid))
+            self.coerceAttendeesPartstatOnCreate()
+
+        # Always set RSVP=TRUE for any NEEDS-ACTION
+        for attendee in self.calendar.getAllAttendeeProperties():
+            if attendee.parameterValue("PARTSTAT", "NEEDS-ACTION").upper() == "NEEDS-ACTION":
+                attendee.setParameter("RSVP", "TRUE")
+
+        if self.needs_sequence_change:
+            self.calendar.bumpiTIPInfo(oldcalendar=self.oldcalendar, doSequence=True)
+
+        yield self.scheduleWithAttendees()
+
+        # Always clear SCHEDULE-FORCE-SEND from all attendees after scheduling
+        for attendee in self.calendar.getAllAttendeeProperties():
+            try:
+                attendee.removeParameter("SCHEDULE-FORCE-SEND")
+            except KeyError:
+                pass
+
+
+    def isOrganizerChangeInsignificant(self):
+
+        rids = None
+        date_changed_rids = None
+        reinvites = None
+        recurrence_reschedule = False
+        differ = iCalDiff(self.oldcalendar, self.calendar, self.do_smart_merge)
+        no_change = differ.organizerDiff()
+        if not no_change:
+            # ORGANIZER change is absolutely not allowed!
+            diffs = differ.whatIsDifferent()
+            rids = set()
+            date_changed_rids = set()
+            checkOrganizerValue = False
+            for rid, props in diffs.iteritems():
+                if "ORGANIZER" in props:
+                    checkOrganizerValue = True
+                rids.add(rid)
+
+                if any([testprop in props for testprop in (
+                    "DTSTART",
+                    "DTEND",
+                    "DURATION",
+                    "DUE",
+                    "RECURRENCE-ID",
+                )]):
+                    date_changed_rids.add(rid)
+
+                # Check to see whether a change to R-ID's happened
+                if rid == "":
+
+                    if "DTSTART" in props and self.calendar.masterComponent().hasProperty("RRULE"):
+                        # DTSTART change with RRULE present is always a reschedule
+                        recurrence_reschedule = True
+
+                    elif "RRULE" in props:
+
+                        # Need to see if the RRULE change is a simple truncation or expansion - i.e. a change to
+                        # COUNT or UNTIL only. If so we don't need to treat this as a complete re-schedule.
+
+                        # Start off assuming they are different
+                        recurrence_reschedule = True
+
+                        # Get each RRULE (can be only one in the master)
+                        oldrrule = tuple(self.oldcalendar.masterComponent().properties("RRULE"))
+                        oldrrule = oldrrule[0].value() if len(oldrrule) else None
+                        newrrule = tuple(self.calendar.masterComponent().properties("RRULE"))
+                        newrrule = newrrule[0].value() if len(newrrule) else None
+
+                        if newrrule is not None and oldrrule is not None:
+
+                            # Normalize the rrules by removing COUNT/UNTIL and then compare
+                            oldrrule = oldrrule.duplicate()
+                            newrrule = newrrule.duplicate()
+
+                            oldrrule.setUseUntil(False)
+                            oldrrule.setUntil(None)
+                            oldrrule.setUseCount(False)
+                            oldrrule.setCount(0)
+
+                            newrrule.setUseUntil(False)
+                            newrrule.setUntil(None)
+                            newrrule.setUseCount(False)
+                            newrrule.setCount(0)
+
+                            # If they are equal we have a simple change - no overall reschedule
+                            if newrrule == oldrrule:
+                                recurrence_reschedule = False
+
+            if checkOrganizerValue:
+                oldOrganizer = self.oldcalendar.getOrganizer()
+                newOrganizer = self.calendar.getOrganizer()
+                if oldOrganizer != newOrganizer:
+                    log.error("Cannot change ORGANIZER: UID:%s" % (self.uid,))
+                    raise HTTPError(ErrorResponse(
+                        responsecode.FORBIDDEN,
+                        (caldav_namespace, "valid-organizer-change"),
+                        "Organizer cannot be changed",
+                    ))
+        else:
+            # Special case of SCHEDULE-FORCE-SEND added to attendees and no other change
+            reinvites = set()
+            for attendee in self.calendar.getAllAttendeeProperties():
+                try:
+                    if attendee.parameterValue("SCHEDULE-FORCE-SEND", "").upper() == "REQUEST":
+                        reinvites.add(attendee.value())
+                except KeyError:
+                    pass
+
+        return no_change, rids, date_changed_rids, reinvites, recurrence_reschedule
+
+
+    def findRemovedAttendees(self):
+        """
+        Look for attendees that have been removed from any instances. Save those off
+        as users that need to be sent a cancel.
+
+        This method does not handle a full recurrence change (one where the RRULE pattern
+        changes or the associated DTSTART changes). For the full change we will have another
+        method to handle that.
+        """
+
+        # Several possibilities for when CANCELs need to be sent:
+        #
+        # Remove ATTENDEE property
+        # Add EXDATE
+        # Remove overridden component
+        # Remove RDATE
+        # Truncate RRULE
+        # Change RRULE
+
+        # TODO: the later three will be ignored for now.
+
+        mappedOld = set(self.oldAttendeesByInstance)
+        mappedNew = set(self.attendeesByInstance)
+
+        # Get missing instances
+        removedInstances = self.oldInstances - self.instances
+        addedInstances = self.instances - self.oldInstances
+
+        # Also look for new EXDATEs
+        oldexdates = set()
+        for property in self.oldcalendar.masterComponent().properties("EXDATE"):
+            oldexdates.update([value.getValue() for value in property.value()])
+        newexdates = set()
+        for property in self.calendar.masterComponent().properties("EXDATE"):
+            newexdates.update([value.getValue() for value in property.value()])
+
+        addedexdates = newexdates - oldexdates
+
+        # Now figure out the attendees that need to be sent CANCELs
+        self.cancelledAttendees = set()
+
+        for item in mappedOld:
+            if item not in mappedNew:
+
+                # Several possibilities:
+                #
+                # 1. removed from master component - always a CANCEL
+                # 2. removed from overridden component - always a CANCEL
+                # 3. removed overridden component - only CANCEL if not in master or exdate added
+
+                new_attendee, rid = item
+
+                # 1. & 2.
+                if rid is None or rid not in removedInstances:
+                    self.cancelledAttendees.add(item)
+                else:
+                    # 3.
+                    if (new_attendee, None) not in mappedNew or rid in addedexdates:
+                        self.cancelledAttendees.add(item)
+
+        master_attendees = self.oldcalendar.masterComponent().getAttendeesByInstance(onlyScheduleAgentServer=True)
+        for attendee, _ignore in master_attendees:
+            for exdate in addedexdates:
+                # Don't remove the master attendee's when an EXDATE is added for a removed overridden component
+                # as the set of attendees in the override may be different from the master set, but the override
+                # will have been accounted for by the previous attendee/instance logic.
+                if exdate not in removedInstances:
+                    self.cancelledAttendees.add((attendee, exdate,))
+
+        # For overridden instances added, check whether any attendees were removed from the master
+        for attendee, _ignore in master_attendees:
+            for rid in addedInstances:
+                if (attendee, rid) not in mappedNew and rid not in oldexdates:
+                    self.cancelledAttendees.add((attendee, rid,))
+
+
+    def findRemovedAttendeesOnRecurrenceChange(self):
+        """
+        Look for attendees that have been removed during a change to the overall recurrence.
+
+        This is a special case to try and minimize the number of cancels sent to just those
+        attendees actually removed. The basic policy is this:
+
+        1) If an attendee is present in the master component of the new event, they never
+        receive a CANCEL as they will always receive a REQUEST with the entire new event
+        data. i.e., they will see an event "replacement" rather than a cancel+new request.
+
+        2) For all attendees in the old event, not in the new master, send a cancel of
+        the master or each override they appear in. That happens even if they appear in an
+        override in the new calendar, since in all likelihood there is no guaranteed exact
+        mapping between old and new instances.
+        """
+
+        self.cancelledAttendees = set()
+        new_master_attendees = set([attendee for attendee, _ignore in self.calendar.masterComponent().getAttendeesByInstance(onlyScheduleAgentServer=True)])
+        for attendee, rid in self.oldAttendeesByInstance:
+            if attendee not in new_master_attendees:
+                self.cancelledAttendees.add((attendee, rid,))
+
+
+    def coerceAttendeesPartstatOnCreate(self):
+        """
+        Make sure any attendees handled by the server start off with PARTSTAT=NEEDS-ACTION as
+        we do not allow the organizer to forcibly set PARTSTAT to anything else.
+        """
+        for attendee in self.calendar.getAllAttendeeProperties():
+            # Don't adjust ORGANIZER's ATTENDEE
+            if attendee.value() in self.organizerPrincipal.calendarUserAddresses():
+                continue
+            if attendee.parameterValue("SCHEDULE-AGENT", "SERVER").upper() == "SERVER" and attendee.hasParameter("PARTSTAT"):
+                attendee.setParameter("PARTSTAT", "NEEDS-ACTION")
+
+
+    def coerceAttendeesPartstatOnModify(self):
+        """
+        Make sure that the organizer does not change attendees' PARTSTAT to anything
+        other than NEEDS-ACTION for those attendees handled by the server.
+        """
+
+        # Get the set of Rids in each calendar
+        newRids = set(self.calendar.getComponentInstances())
+        oldRids = set(self.oldcalendar.getComponentInstances())
+
+        # Test/fix ones that are the same
+        for rid in (newRids & oldRids):
+            self.compareAttendeePartstats(self.oldcalendar.overriddenComponent(rid), self.calendar.overriddenComponent(rid))
+
+        # Test/fix ones added
+        for rid in (newRids - oldRids):
+            # Compare the new one to the old master
+            self.compareAttendeePartstats(self.oldcalendar.overriddenComponent(None), self.calendar.overriddenComponent(rid))
+
+        # For removals, we ignore ones that are no longer valid
+        valid_old_rids = self.calendar.validInstances(oldRids - newRids)
+
+        # Test/fix ones removed
+        for rid in valid_old_rids:
+            # Compare the old one to a derived instance, and if there is a change
+            # add the derived instance to the new data
+            newcomp = self.calendar.deriveInstance(rid)
+            changed = self.compareAttendeePartstats(
+                self.oldcalendar.overriddenComponent(rid),
+                newcomp,
+            )
+            if changed:
+                self.calendar.addComponent(newcomp)
+
+
+    def compareAttendeePartstats(self, old_component, new_component):
+        """
+        Compare two components, old and new, and make sure the Organizer has not changed the PARTSTATs
+        in the new one to anything other than NEEDS-ACTION. If there is a change, undo it.
+        """
+
+        old_attendees = dict([(normalizeCUAddr(attendee.value()), attendee) for attendee in old_component.getAllAttendeeProperties()])
+        new_attendees = dict([(normalizeCUAddr(attendee.value()), attendee) for attendee in new_component.getAllAttendeeProperties()])
+
+        changed = False
+        for cuaddr, newattendee in new_attendees.items():
+            # Don't adjust ORGANIZER's ATTENDEE
+            if newattendee.value() in self.organizerPrincipal.calendarUserAddresses():
+                continue
+            new_partstat = newattendee.parameterValue("PARTSTAT", "NEEDS-ACTION").upper()
+            if newattendee.parameterValue("SCHEDULE-AGENT", "SERVER").upper() == "SERVER" and new_partstat != "NEEDS-ACTION":
+                old_attendee = old_attendees.get(cuaddr)
+                old_partstat = old_attendee.parameterValue("PARTSTAT", "NEEDS-ACTION").upper() if old_attendee else "NEEDS-ACTION"
+                if old_attendee is None or old_partstat != new_partstat:
+                    newattendee.setParameter("PARTSTAT", old_partstat)
+                    changed = True
+
+        return changed
+
+
+    @inlineCallbacks
+    def coerceAttendeeScheduleAgent(self):
+        """
+        Do not allow SCHEDULE-AGENT=CLIENT/NONE for attendees hosted by this server. Coerce to
+        SCHEDULE-AGENT=SERVER.
+        """
+
+        coerced = {}
+        for attendee in self.calendar.getAllAttendeeProperties():
+            if attendee.parameterValue("SCHEDULE-AGENT", "SERVER").upper() == "CLIENT":
+                cuaddr = attendee.value()
+                if cuaddr not in coerced:
+                    attendeePrincipal = self.resource.principalForCalendarUserAddress(cuaddr)
+                    attendeeAddress = (yield addressmapping.mapper.getCalendarUser(cuaddr, attendeePrincipal))
+                    local_attendee = type(attendeeAddress) in (LocalCalendarUser, PartitionedCalendarUser, OtherServerCalendarUser,)
+                    coerced[cuaddr] = local_attendee
+                if coerced[cuaddr]:
+                    attendee.removeParameter("SCHEDULE-AGENT")
+
+
+    @inlineCallbacks
+    def scheduleWithAttendees(self):
+
+        # First process cancelled attendees
+        total = (yield self.processCancels())
+
+        # Process regular requests next
+        if self.action in ("create", "modify",):
+            total += (yield self.processRequests())
+
+        self.logItems["itip.requests"] = total
+
+
+    @inlineCallbacks
+    def processCancels(self):
+
+        # TODO: a better policy here is to aggregate by attendees with the same set of instances
+        # being cancelled, but for now we will do one scheduling message per attendee.
+
+        # Do one per attendee
+        aggregated = {}
+        for attendee, rid in self.cancelledAttendees:
+            aggregated.setdefault(attendee, []).append(rid)
+
+        count = 0
+        for attendee, rids in aggregated.iteritems():
+
+            # Don't send message back to the ORGANIZER
+            if attendee in self.organizerPrincipal.calendarUserAddresses():
+                continue
+
+            # Generate an iTIP CANCEL message for this attendee, cancelling
+            # each instance or the whole
+
+            if None in rids:
+                # One big CANCEL will do
+                itipmsg = iTipGenerator.generateCancel(self.oldcalendar, (attendee,), None, self.action == "remove")
+            else:
+                # Multiple CANCELs
+                itipmsg = iTipGenerator.generateCancel(self.oldcalendar, (attendee,), rids)
+
+            # Send scheduling message
+            if itipmsg:
+                # This is a local CALDAV scheduling operation.
+                scheduler = self.makeScheduler()
+
+                # Do the PUT processing
+                log.info("Implicit CANCEL - organizer: '%s' to attendee: '%s', UID: '%s', RIDs: '%s'" % (self.organizer, attendee, self.uid, rids))
+                response = (yield scheduler.doSchedulingViaPUT(self.originator, (attendee,), itipmsg, internal_request=True))
+                self.handleSchedulingResponse(response, True)
+
+                count += 1
+
+        returnValue(count)
+
+
+    @inlineCallbacks
+    def processRequests(self):
+
+        # TODO: a better policy here is to aggregate by attendees with the same set of instances
+        # being requested, but for now we will do one scheduling message per attendee.
+
+        # Do one per attendee
+        count = 0
+        for attendee in self.attendees:
+
+            # Don't send message back to the ORGANIZER
+            if attendee in self.organizerPrincipal.calendarUserAddresses():
+                continue
+
+            # Don't send message to specified attendees
+            if attendee in self.except_attendees:
+                continue
+
+            # Only send to specified attendees
+            if self.only_refresh_attendees is not None and attendee not in self.only_refresh_attendees:
+                continue
+
+            # If SCHEDULE-FORCE-SEND only change, only send message to those Attendees
+            if self.reinvites and attendee in self.reinvites:
+                continue
+
+            itipmsg = iTipGenerator.generateAttendeeRequest(self.calendar, (attendee,), self.changed_rids)
+
+            # Send scheduling message
+            if itipmsg is not None:
+                # This is a local CALDAV scheduling operation.
+                scheduler = self.makeScheduler()
+
+                # Do the PUT processing
+                log.info("Implicit REQUEST - organizer: '%s' to attendee: '%s', UID: '%s'" % (self.organizer, attendee, self.uid,))
+                response = (yield scheduler.doSchedulingViaPUT(self.originator, (attendee,), itipmsg, internal_request=True))
+                self.handleSchedulingResponse(response, True)
+
+                count += 1
+
+        returnValue(count)
+
+
+    def handleSchedulingResponse(self, response, is_organizer):
+
+        # Map each recipient in the response to a status code
+        responses = {}
+        for item in response.responses:
+            assert isinstance(item, caldavxml.Response), "Wrong element in response"
+            recipient = str(item.children[0].children[0])
+            status = str(item.children[1])
+            responses[recipient] = status
+
+            # Now apply to each ATTENDEE/ORGANIZER in the original data
+            self.calendar.setParameterToValueForPropertyWithValue(
+                "SCHEDULE-STATUS",
+                status.split(";")[0],
+                "ATTENDEE" if is_organizer else "ORGANIZER",
+                recipient)
+
+
+    @inlineCallbacks
+    def doImplicitAttendee(self):
+
+        # Check SCHEDULE-AGENT
+        doScheduling = self.checkOrganizerScheduleAgent()
+
+        if self.action == "remove":
+            if self.calendar.hasPropertyValueInAllComponents(Property("STATUS", "CANCELLED")):
+                log.debug("Implicit - attendee '%s' is removing cancelled UID: '%s'" % (self.attendee, self.uid))
+                # Nothing else to do
+            elif doScheduling:
+                # If attendee is already marked as declined in all components - nothing to do
+                attendees = self.calendar.getAttendeeProperties((self.attendee,))
+                if all([attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") == "DECLINED" for attendee in attendees]):
+                    log.debug("Implicit - attendee '%s' is removing fully declined UID: '%s'" % (self.attendee, self.uid))
+                    # Nothing else to do
+                else:
+                    log.debug("Implicit - attendee '%s' is cancelling UID: '%s'" % (self.attendee, self.uid))
+                    yield self.scheduleCancelWithOrganizer()
+            else:
+                log.debug("Implicit - attendee '%s' is removing UID without server scheduling: '%s'" % (self.attendee, self.uid))
+                # Nothing else to do
+            returnValue(None)
+
+        else:
+            # Make sure ORGANIZER is not changed
+            if self.resource is not None:
+                self.oldcalendar = (yield self.resource.componentForUser(self.calendar_owner))
+                oldOrganizer = self.oldcalendar.getOrganizer()
+                newOrganizer = self.calendar.getOrganizer()
+                if oldOrganizer != newOrganizer:
+                    log.error("Cannot change ORGANIZER: UID:%s" % (self.uid,))
+                    raise HTTPError(ErrorResponse(
+                        responsecode.FORBIDDEN,
+                        (caldav_namespace, "valid-attendee-change"),
+                        "Cannot change organizer",
+                    ))
+            else:
+                self.oldcalendar = None
+
+            # Get the ORGANIZER's current copy of the calendar object
+            yield self.getOrganizersCopy()
+            if self.organizer_calendar:
+
+                # If Organizer copy exists we cannot allow SCHEDULE-AGENT=CLIENT or NONE
+                if not doScheduling:
+                    log.error("Attendee '%s' is not allowed to change SCHEDULE-AGENT on organizer: UID:%s" % (self.attendeePrincipal, self.uid,))
+                    raise HTTPError(ErrorResponse(
+                        responsecode.FORBIDDEN,
+                        (caldav_namespace, "valid-attendee-change"),
+                        "Cannot alter organizer",
+                    ))
+
+                # Determine whether the current change is allowed
+                changeAllowed, doITipReply, changedRids, newCalendar = self.isAttendeeChangeInsignificant()
+                if changeAllowed:
+                    self.return_calendar = self.calendar = newCalendar
+
+                if not changeAllowed:
+                    if self.calendar.hasPropertyValueInAllComponents(Property("STATUS", "CANCELLED")):
+                        log.debug("Attendee '%s' is creating CANCELLED event for mismatched UID: '%s' - removing entire event" % (self.attendee, self.uid,))
+                        self.return_status = ImplicitScheduler.STATUS_ORPHANED_EVENT
+                        returnValue(None)
+                    else:
+                        log.error("Attendee '%s' is not allowed to make an unauthorized change to an organized event: UID:%s" % (self.attendeePrincipal, self.uid,))
+                        raise HTTPError(ErrorResponse(
+                            responsecode.FORBIDDEN,
+                            (caldav_namespace, "valid-attendee-change"),
+                            "Attendee changes are not allowed",
+                        ))
+
+                # Check that the return calendar actually has any components left - this can happen if a cancelled
+                # component is removed and replaced by another cancelled or invalid one
+                if self.calendar.mainType() is None:
+                    log.debug("Attendee '%s' is replacing CANCELLED event: '%s' - removing entire event" % (self.attendee, self.uid,))
+                    self.return_status = ImplicitScheduler.STATUS_ORPHANED_EVENT
+                    returnValue(None)
+
+                if not doITipReply:
+                    log.debug("Implicit - attendee '%s' is updating UID: '%s' but change is not significant" % (self.attendee, self.uid))
+                    returnValue(self.return_calendar)
+                log.debug("Attendee '%s' is allowed to update UID: '%s' with local organizer '%s'" % (self.attendee, self.uid, self.organizer))
+
+            elif isinstance(self.organizerAddress, LocalCalendarUser):
+                # If Organizer copy does not exists we cannot allow SCHEDULE-AGENT=SERVER
+                if doScheduling:
+                    # Check to see whether all instances are CANCELLED
+                    if self.calendar.hasPropertyValueInAllComponents(Property("STATUS", "CANCELLED")):
+                        log.debug("Attendee '%s' is creating CANCELLED event for missing UID: '%s' - removing entire event" % (self.attendee, self.uid,))
+                        self.return_status = ImplicitScheduler.STATUS_ORPHANED_CANCELLED_EVENT
+                        returnValue(None)
+                    else:
+                        # Check to see whether existing event is SCHEDULE-AGENT=CLIENT/NONE
+                        if self.oldcalendar:
+                            oldScheduling = self.oldcalendar.getOrganizerScheduleAgent()
+                            if not oldScheduling:
+                                log.error("Attendee '%s' is not allowed to set SCHEDULE-AGENT=SERVER on organizer: UID:%s" % (self.attendeePrincipal, self.uid,))
+                                raise HTTPError(ErrorResponse(
+                                    responsecode.FORBIDDEN,
+                                    (caldav_namespace, "valid-attendee-change"),
+                                    "Attendee cannot change organizer state",
+                                ))
+
+                        log.debug("Attendee '%s' is not allowed to update UID: '%s' - missing organizer copy - removing entire event" % (self.attendee, self.uid,))
+                        self.return_status = ImplicitScheduler.STATUS_ORPHANED_EVENT
+                        returnValue(None)
+                else:
+                    log.debug("Implicit - attendee '%s' is modifying UID without server scheduling: '%s'" % (self.attendee, self.uid))
+                    # Nothing else to do
+                    returnValue(None)
+
+            elif isinstance(self.organizerAddress, InvalidCalendarUser):
+                # We will allow the attendee to do anything in this case, but we will mark the organizer
+                # with an schedule-status error
+                log.debug("Attendee '%s' is allowed to update UID: '%s' with invalid organizer '%s'" % (self.attendee, self.uid, self.organizer))
+                if doScheduling:
+                    self.calendar.setParameterToValueForPropertyWithValue(
+                        "SCHEDULE-STATUS",
+                        iTIPRequestStatus.NO_USER_SUPPORT_CODE,
+                        "ORGANIZER",
+                        self.organizer)
+                returnValue(None)
+
+            else:
+                # We have a remote Organizer of some kind. For now we will allow the Attendee
+                # to make any change they like as we cannot verify what is reasonable. In reality
+                # we ought to be comparing the Attendee changes against the attendee's own copy
+                # and restrict changes based on that when the organizer's copy is not available.
+                log.debug("Attendee '%s' is allowed to update UID: '%s' with remote organizer '%s'" % (self.attendee, self.uid, self.organizer))
+                changedRids = None
+
+            if doScheduling:
+                log.debug("Implicit - attendee '%s' is updating UID: '%s'" % (self.attendee, self.uid))
+                yield self.scheduleWithOrganizer(changedRids)
+            else:
+                log.debug("Implicit - attendee '%s' is updating UID without server scheduling: '%s'" % (self.attendee, self.uid))
+                # Nothing else to do
+
+
+    @inlineCallbacks
+    def doImplicitMissingAttendee(self):
+
+        if self.action == "remove":
+            # Nothing else to do
+            log.debug("Implicit - missing attendee is removing UID without server scheduling: '%s'" % (self.uid,))
+
+        else:
+            # We will allow the attendee to do anything in this case, but we will mark the organizer
+            # with an schedule-status error and schedule-agent none
+            log.debug("Missing attendee is allowed to update UID: '%s' with invalid organizer '%s'" % (self.uid, self.organizer))
+
+            # Make sure ORGANIZER is not changed if originally SCHEDULE-AGENT=SERVER
+            if self.resource.exists():
+                self.oldcalendar = (yield self.resource.iCalendarForUser(self.request))
+                oldOrganizer = self.oldcalendar.getOrganizer()
+                newOrganizer = self.calendar.getOrganizer()
+                if oldOrganizer != newOrganizer and self.oldcalendar.getOrganizerScheduleAgent():
+                    log.error("Cannot change ORGANIZER: UID:%s" % (self.uid,))
+                    raise HTTPError(ErrorResponse(
+                        responsecode.FORBIDDEN,
+                        (caldav_namespace, "valid-attendee-change"),
+                        "Cannot change organizer",
+                    ))
+
+            # Check SCHEDULE-AGENT and coerce SERVER to NONE
+            if self.calendar.getOrganizerScheduleAgent():
+                self.calendar.setParameterToValueForPropertyWithValue("SCHEDULE-AGENT", "NONE", "ORGANIZER", None)
+                self.calendar.setParameterToValueForPropertyWithValue("SCHEDULE-STATUS", iTIPRequestStatus.NO_USER_SUPPORT_CODE, "ORGANIZER", None)
+
+
+    def checkOrganizerScheduleAgent(self):
+
+        is_server = self.calendar.getOrganizerScheduleAgent()
+        local_organizer = type(self.organizerAddress) in (LocalCalendarUser, PartitionedCalendarUser, OtherServerCalendarUser,)
+
+        if config.Scheduling.iMIP.Enabled and self.organizerAddress.cuaddr.lower().startswith("mailto:"):
+            return is_server
+
+        if not config.Scheduling.iSchedule.Enabled and not local_organizer and is_server:
+            # Coerce ORGANIZER to SCHEDULE-AGENT=NONE
+            log.debug("Attendee '%s' is not allowed to use SCHEDULE-AGENT=SERVER on organizer: UID:%s" % (self.attendeePrincipal, self.uid,))
+            self.calendar.setParameterToValueForPropertyWithValue("SCHEDULE-AGENT", "NONE", "ORGANIZER", None)
+            self.calendar.setParameterToValueForPropertyWithValue("SCHEDULE-STATUS", iTIPRequestStatus.NO_USER_SUPPORT_CODE, "ORGANIZER", None)
+            is_server = False
+
+        return is_server
+
+
+    @inlineCallbacks
+    def getOrganizersCopy(self):
+        """
+        Get the Organizer's copy of the event being processed.
+
+        NB it is possible that the Organizer is not hosted on this server
+        so the result here will be None. In that case we have to trust that
+        the attendee does the right thing about changing the details in the event.
+        """
+
+        self.organizer_calendar = None
+        calendar_resource = (yield getCalendarObjectForPrincipals(self.calendar_home.transaction(), self.organizerPrincipal, self.uid))
+        if calendar_resource is not None:
+            self.organizer_calendar = (yield calendar_resource.componentForUser(self.organizerPrincipal.uid()))
+        elif type(self.organizerAddress) in (PartitionedCalendarUser, OtherServerCalendarUser,):
+            # For partitioning where the organizer is on a different node, we will assume that the attendee's copy
+            # of the event is up to date and "authoritative". So we pretend that is the organizer copy
+            self.organizer_calendar = self.oldcalendar
+
+
+    def isAttendeeChangeInsignificant(self):
+        """
+        Check whether the change is significant (PARTSTAT) or allowed
+        (attendee can only change their property, alarms, TRANSP, and
+        instances. Raise an exception if it is not allowed.
+        """
+
+        oldcalendar = self.oldcalendar
+        if oldcalendar is None:
+            oldcalendar = self.organizer_calendar
+            oldcalendar.attendeesView((self.attendee,), onlyScheduleAgentServer=True)
+            if oldcalendar.mainType() is None:
+                log.debug("Implicit - attendee '%s' cannot use an event they are not an attendee of, UID: '%s'" % (self.attendee, self.uid))
+                raise HTTPError(ErrorResponse(
+                    responsecode.FORBIDDEN,
+                    (caldav_namespace, "valid-attendee-change"),
+                    "Cannot use an event when not listed as an attendee in the organizer's copy",
+                ))
+        differ = iCalDiff(oldcalendar, self.calendar, self.do_smart_merge)
+        return differ.attendeeMerge(self.attendee)
+
+
+    def scheduleWithOrganizer(self, changedRids=None):
+
+        self.logItems["itip.reply"] = "reply"
+
+        itipmsg = iTipGenerator.generateAttendeeReply(self.calendar, self.attendee, changedRids=changedRids)
+
+        # Send scheduling message
+        return self.sendToOrganizer("REPLY", itipmsg)
+
+
+    def scheduleCancelWithOrganizer(self):
+
+        self.logItems["itip.reply"] = "cancel"
+
+        itipmsg = iTipGenerator.generateAttendeeReply(self.calendar, self.attendee, force_decline=True)
+
+        # Send scheduling message
+        return self.sendToOrganizer("CANCEL", itipmsg)
+
+
+    def sendToOrganizer(self, action, itipmsg):
+
+        # Send scheduling message
+
+        # This is a local CALDAV scheduling operation.
+        scheduler = self.makeScheduler()
+
+        # Do the PUT processing
+        def _gotResponse(response):
+            self.handleSchedulingResponse(response, False)
+
+        log.info("Implicit %s - attendee: '%s' to organizer: '%s', UID: '%s'" % (action, self.attendee, self.organizer, self.uid,))
+        d = scheduler.doSchedulingViaPUT(self.originator, (self.organizer,), itipmsg, internal_request=True)
+        d.addCallback(_gotResponse)
+        return d


Property changes on: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/implicit.py
___________________________________________________________________
Added: svn:executable
   + *

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/__init__.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/__init__.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/delivery.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/delivery.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/delivery.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,516 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from StringIO import StringIO
+
+from calendarserver.version import version
+
+from twext.internet.gaiendpoint import GAIEndpoint
+from twext.python.log import Logger
+from twext.web2 import responsecode
+from twext.web2.client.http import ClientRequest
+from twext.web2.client.http import HTTPClientProtocol
+from twext.web2.dav.http import ErrorResponse
+from twext.web2.dav.util import davXMLFromStream, joinURL, allDataFromStream
+from twext.web2.http import HTTPError
+from twext.web2.http_headers import Headers
+from twext.web2.http_headers import MimeType
+from twext.web2.stream import MemoryStream
+
+from twisted.internet.defer import inlineCallbacks, DeferredList, returnValue
+from twisted.internet.protocol import Factory
+from twisted.python.failure import Failure
+
+from twistedcaldav.accounting import accountingEnabledForCategory, emitAccounting
+from twistedcaldav.client.pool import _configuredClientContextFactory
+from twistedcaldav.config import config
+from twistedcaldav.ical import normalizeCUAddress, Component
+from txdav.caldav.datastore.scheduling.cuaddress import PartitionedCalendarUser, RemoteCalendarUser, \
+    OtherServerCalendarUser
+from txdav.caldav.datastore.scheduling.delivery import DeliveryService
+from txdav.caldav.datastore.scheduling.ischedule.dkim import DKIMRequest, DKIMUtils
+from txdav.caldav.datastore.scheduling.ischedule.remoteservers import IScheduleServerRecord
+from txdav.caldav.datastore.scheduling.ischedule.remoteservers import IScheduleServers
+from txdav.caldav.datastore.scheduling.ischedule.utils import lookupServerViaSRV
+from twistedcaldav.scheduling.ischedule.xml import ScheduleResponse, Response, \
+    RequestStatus, Recipient, ischedule_namespace, CalendarData, \
+    ResponseDescription, Error
+from txdav.caldav.datastore.scheduling.itip import iTIPRequestStatus
+from twistedcaldav.util import utf8String, normalizationLookup
+
+from urlparse import urlsplit
+
+"""
+Handles the sending of iSchedule scheduling messages. Used for both cross-domain scheduling,
+as well as internal partitioning or podding.
+"""
+
+__all__ = [
+    "ScheduleViaISchedule",
+]
+
+log = Logger()
+
+
+
+class ScheduleViaISchedule(DeliveryService):
+
+    domainServerMap = {}
+
+    @classmethod
+    def serviceType(cls):
+        return DeliveryService.serviceType_ischedule
+
+
+    @classmethod
+    @inlineCallbacks
+    def matchCalendarUserAddress(cls, cuaddr):
+
+        # TODO: here is where we would attempt service discovery based on the cuaddr.
+
+        # Only handle mailtos:
+        if cuaddr.lower().startswith("mailto:"):
+            _ignore_local, domain = cuaddr[7:].split("@", 1)
+            server = (yield cls.serverForDomain(domain))
+            returnValue(server is not None)
+
+        # Do default match
+        result = (yield super(ScheduleViaISchedule, cls).matchCalendarUserAddress(cuaddr))
+        returnValue(result)
+
+
+    @classmethod
+    @inlineCallbacks
+    def serverForDomain(cls, domain):
+        if domain not in cls.domainServerMap:
+
+            # First check built-in list of remote servers
+            servermgr = IScheduleServers()
+            server = servermgr.mapDomain(domain)
+            if server is not None:
+                cls.domainServerMap[domain] = server
+            else:
+                # Lookup domain
+                result = (yield lookupServerViaSRV(domain))
+                if result is None:
+                    # Lookup domain
+                    result = (yield lookupServerViaSRV(domain, service="_ischedule"))
+                    if result is None:
+                        cls.domainServerMap[domain] = None
+                    else:
+                        # Create the iSchedule server record for this server
+                        cls.domainServerMap[domain] = IScheduleServerRecord(uri="http://%s:%s/.well-known/ischedule" % result)
+                else:
+                    # Create the iSchedule server record for this server
+                    cls.domainServerMap[domain] = IScheduleServerRecord(uri="https://%s:%s/.well-known/ischedule" % result)
+
+        returnValue(cls.domainServerMap[domain])
+
+
+    @inlineCallbacks
+    def generateSchedulingResponses(self, refreshOnly=False):
+        """
+        Generate scheduling responses for remote recipients.
+        """
+
+        # Group recipients by server so that we can do a single request with multiple recipients
+        # to each different server.
+        groups = {}
+        for recipient in self.recipients:
+            if isinstance(recipient, RemoteCalendarUser):
+                # Map the recipient's domain to a server
+                server = (yield self.serverForDomain(recipient.domain))
+            elif isinstance(recipient, PartitionedCalendarUser):
+                server = self._getServerForPartitionedUser(recipient)
+            elif isinstance(recipient, OtherServerCalendarUser):
+                server = self._getServerForOtherServerUser(recipient)
+            else:
+                assert False, "Incorrect calendar user address class"
+            if not server:
+                # Cannot do server-to-server for this recipient.
+                err = HTTPError(ErrorResponse(
+                    responsecode.NOT_FOUND,
+                    (ischedule_namespace, "recipient-allowed"),
+                    "No server for recipient",
+                ))
+                self.responses.add(recipient.cuaddr, Failure(exc_value=err), reqstatus=iTIPRequestStatus.NO_USER_SUPPORT)
+
+                # Process next recipient
+                continue
+
+            if not server.allow_to:
+                # Cannot do server-to-server outgoing requests for this server.
+                err = HTTPError(ErrorResponse(
+                    responsecode.NOT_FOUND,
+                    (ischedule_namespace, "recipient-allowed"),
+                    "Cannot send to recipient's server",
+                ))
+                self.responses.add(recipient.cuaddr, Failure(exc_value=err), reqstatus=iTIPRequestStatus.SERVICE_UNAVAILABLE)
+
+                # Process next recipient
+                continue
+
+            groups.setdefault(server, []).append(recipient)
+
+        if len(groups) == 0:
+            returnValue(None)
+
+        # Now we process each server: let's use a DeferredList to aggregate all the Deferred's
+        # we will generate for each request. That way we can have parallel requests in progress
+        # rather than serialize them.
+        deferreds = []
+        for server, recipients in groups.iteritems():
+            requestor = IScheduleRequest(self.scheduler, server, recipients, self.responses, refreshOnly)
+            deferreds.append(requestor.doRequest())
+
+        yield DeferredList(deferreds)
+
+
+    def _getServerForPartitionedUser(self, recipient):
+
+        if not hasattr(self, "partitionedServers"):
+            self.partitionedServers = {}
+
+        partition = recipient.principal.partitionURI()
+        if partition not in self.partitionedServers:
+            self.partitionedServers[partition] = IScheduleServerRecord(uri=joinURL(partition, "/ischedule"))
+            self.partitionedServers[partition].unNormalizeAddresses = False
+            self.partitionedServers[partition].moreHeaders.append(recipient.principal.server().secretHeader())
+
+        return self.partitionedServers[partition]
+
+
+    def _getServerForOtherServerUser(self, recipient):
+
+        if not hasattr(self, "otherServers"):
+            self.otherServers = {}
+
+        serverURI = recipient.principal.serverURI()
+        if serverURI not in self.otherServers:
+            self.otherServers[serverURI] = IScheduleServerRecord(uri=joinURL(serverURI, "/ischedule"))
+            self.otherServers[serverURI].unNormalizeAddresses = not recipient.principal.server().isImplicit
+            self.otherServers[serverURI].moreHeaders.append(recipient.principal.server().secretHeader())
+
+        return self.otherServers[serverURI]
+
+
+
+class IScheduleRequest(object):
+
+    def __init__(self, scheduler, server, recipients, responses, refreshOnly=False):
+
+        self.scheduler = scheduler
+        self.server = server
+        self.recipients = recipients
+        self.responses = responses
+        self.refreshOnly = refreshOnly
+        self.headers = None
+        self.data = None
+
+
+    @inlineCallbacks
+    def doRequest(self):
+
+        # Generate an HTTP client request
+        try:
+            if not hasattr(self.scheduler.request, "extendedLogItems"):
+                self.scheduler.request.extendedLogItems = {}
+            if "itip.ischedule" not in self.scheduler.request.extendedLogItems:
+                self.scheduler.request.extendedLogItems["itip.ischedule"] = 0
+            self.scheduler.request.extendedLogItems["itip.ischedule"] += 1
+
+            # Loop over at most 3 redirects
+            ssl, host, port, path = self.server.details()
+            for _ignore in xrange(3):
+                self._prepareRequest(host, port)
+                response = (yield self._processRequest(ssl, host, port, path))
+                if response.code not in (responsecode.MOVED_PERMANENTLY, responsecode.TEMPORARY_REDIRECT,):
+                    break
+                if response.code == responsecode.MOVED_PERMANENTLY:
+                    self.server.redirect(response.headers.getRawHeaders("location")[0])
+                    ssl, host, port, path = self.server.details()
+                else:
+                    scheme, netloc, path, _ignore_query, _ignore_fragment = urlsplit(response.headers.getRawHeaders("location")[0])
+                    ssl = scheme.lower() == "https"
+                    host = netloc.split(":")
+                    if ":" in netloc:
+                        host, port = netloc.split(":")
+                        port = int(port)
+                    else:
+                        host = netloc
+                        port = 443 if ssl else 80
+            else:
+                raise ValueError("Too many redirects")
+
+            if accountingEnabledForCategory("iSchedule"):
+                self.loggedResponse = yield self.logResponse(response)
+                emitAccounting("iSchedule", "", self.loggedRequest + "\n" + self.loggedResponse, "POST")
+
+            if response.code in (responsecode.OK,):
+                xml = (yield davXMLFromStream(response.stream))
+                self._parseResponse(xml)
+            else:
+                raise ValueError("Incorrect server response status code: %s" % (response.code,))
+
+        except Exception, e:
+            # Generated failed responses for each recipient
+            log.err("Could not do server-to-server request : %s %s" % (self, e))
+            for recipient in self.recipients:
+                err = HTTPError(ErrorResponse(
+                    responsecode.FORBIDDEN,
+                    (ischedule_namespace, "recipient-failed"),
+                    "Server-to-server request failed",
+                ))
+                self.responses.add(recipient.cuaddr, Failure(exc_value=err), reqstatus=iTIPRequestStatus.SERVICE_UNAVAILABLE)
+
+
+    @inlineCallbacks
+    def logRequest(self, request):
+        """
+        Log an HTTP request.
+        """
+
+        iostr = StringIO()
+        iostr.write(">>>> Request start\n\n")
+        if hasattr(request, "clientproto"):
+            protocol = "HTTP/%d.%d" % (request.clientproto[0], request.clientproto[1],)
+        else:
+            protocol = "HTTP/1.1"
+        iostr.write("%s %s %s\n" % (request.method, request.uri, protocol,))
+        for name, valuelist in request.headers.getAllRawHeaders():
+            for value in valuelist:
+                # Do not log authorization details
+                if name not in ("Authorization",):
+                    iostr.write("%s: %s\n" % (name, value))
+                else:
+                    iostr.write("%s: xxxxxxxxx\n" % (name,))
+        iostr.write("\n")
+
+        # We need to play a trick with the request stream as we can only read it once. So we
+        # read it, store the value in a MemoryStream, and replace the request's stream with that,
+        # so the data can be read again.
+        data = (yield allDataFromStream(request.stream))
+        iostr.write(data)
+        request.stream = MemoryStream(data if data is not None else "")
+        request.stream.doStartReading = None
+
+        iostr.write("\n\n>>>> Request end\n")
+        returnValue(iostr.getvalue())
+
+
+    @inlineCallbacks
+    def logResponse(self, response):
+        """
+        Log an HTTP request.
+        """
+        iostr = StringIO()
+        iostr.write(">>>> Response start\n\n")
+        code_message = responsecode.RESPONSES.get(response.code, "Unknown Status")
+        iostr.write("HTTP/1.1 %s %s\n" % (response.code, code_message,))
+        for name, valuelist in response.headers.getAllRawHeaders():
+            for value in valuelist:
+                # Do not log authorization details
+                if name not in ("WWW-Authenticate",):
+                    iostr.write("%s: %s\n" % (name, value))
+                else:
+                    iostr.write("%s: xxxxxxxxx\n" % (name,))
+        iostr.write("\n")
+
+        # We need to play a trick with the response stream to ensure we don't mess it up. So we
+        # read it, store the value in a MemoryStream, and replace the response's stream with that,
+        # so the data can be read again.
+        data = (yield allDataFromStream(response.stream))
+        iostr.write(data)
+        response.stream = MemoryStream(data if data is not None else "")
+        response.stream.doStartReading = None
+
+        iostr.write("\n\n>>>> Response end\n")
+        returnValue(iostr.getvalue())
+
+
+    def _prepareRequest(self, host, port):
+        """
+        Setup the request for sending. We might need to do this several times
+        whilst following redirects.
+        """
+
+        component, method = self._prepareData()
+        self._prepareHeaders(host, port, component, method)
+
+
+    def _prepareHeaders(self, host, port, component, method):
+        """
+        Always generate a new set of headers because the Host may varying during redirects,
+        or we may need to dump DKIM added headers during a redirect.
+        """
+        self.sign_headers = []
+
+        self.headers = Headers()
+        self.headers.setHeader("Host", utf8String(host + ":%s" % (port,)))
+
+        # The Originator must be the ORGANIZER (for a request) or ATTENDEE (for a reply)
+        originator = self.scheduler.organizer.cuaddr if self.scheduler.isiTIPRequest else self.scheduler.attendee
+        originator = normalizeCUAddress(originator, normalizationLookup, self.scheduler.resource.principalForCalendarUserAddress, toUUID=False)
+        self.headers.addRawHeader("Originator", utf8String(originator))
+        self.sign_headers.append("Originator")
+
+        for recipient in self.recipients:
+            self.headers.addRawHeader("Recipient", utf8String(recipient.cuaddr))
+
+        # Only one Recipient header as they get concatenated in ischedule-relaxed canonicalization
+        self.sign_headers.append("Recipient")
+
+        self._doAuthentication()
+
+        self.headers.setHeader("Content-Type", MimeType(
+            "text", "calendar",
+            params={
+                "charset": "utf-8",
+                "component": component,
+                "method": method,
+            }
+        ))
+        self.sign_headers.append("Content-Type")
+
+        self.headers.setHeader("User-Agent", "CalendarServer/%s" % (version,))
+        self.sign_headers.append("User-Agent")
+
+        # Add any additional headers
+        for name, value in self.server.moreHeaders:
+            self.headers.addRawHeader(name, value)
+
+        if self.refreshOnly:
+            self.headers.addRawHeader("X-CALENDARSERVER-ITIP-REFRESHONLY", "T")
+
+
+    def _doAuthentication(self):
+        if self.server.authentication and self.server.authentication[0] == "basic":
+            self.headers.setHeader(
+                "Authorization",
+                ('Basic', ("%s:%s" % (self.server.authentication[1], self.server.authentication[2],)).encode('base64')[:-1])
+            )
+            self.sign_headers.append("Authorization")
+
+
+    def _prepareData(self):
+        """
+        Prepare data via normalization etc. Only need to do this once even when
+        redirects occur.
+        """
+
+        if self.data is None:
+            # Need to remap cuaddrs from urn:uuid
+            if self.server.unNormalizeAddresses and self.scheduler.method == "PUT":
+                normalizedCalendar = self.scheduler.calendar.duplicate()
+                normalizedCalendar.normalizeCalendarUserAddresses(
+                    normalizationLookup,
+                    self.scheduler.resource.principalForCalendarUserAddress,
+                    toUUID=False)
+            else:
+                normalizedCalendar = self.scheduler.calendar
+
+            # For VFREEBUSY we need to strip out ATTENDEEs that do not match the recipient list
+            if self.scheduler.isfreebusy:
+                normalizedCalendar.removeAllButTheseAttendees([recipient.cuaddr for recipient in self.recipients])
+
+            component = normalizedCalendar.mainType()
+            method = normalizedCalendar.propertyValue("METHOD")
+            self.data = str(normalizedCalendar)
+            return component, method
+        else:
+            cal = Component.fromString(self.data)
+            component = cal.mainType()
+            method = cal.propertyValue("METHOD")
+            return component, method
+
+
+    @inlineCallbacks
+    def _processRequest(self, ssl, host, port, path):
+        from twisted.internet import reactor
+        f = Factory()
+        f.protocol = HTTPClientProtocol
+        if ssl:
+            ep = GAIEndpoint(reactor, host, port,
+                             _configuredClientContextFactory())
+        else:
+            ep = GAIEndpoint(reactor, host, port)
+        proto = (yield ep.connect(f))
+
+        if config.Scheduling.iSchedule.DKIM.Enabled:
+            domain, selector, key_file, algorithm, useDNSKey, useHTTPKey, usePrivateExchangeKey, expire = DKIMUtils.getConfiguration(config)
+            request = DKIMRequest(
+                "POST",
+                path,
+                self.headers,
+                self.data,
+                domain,
+                selector,
+                key_file,
+                algorithm,
+                self.sign_headers,
+                useDNSKey,
+                useHTTPKey,
+                usePrivateExchangeKey,
+                expire,
+            )
+            yield request.sign()
+        else:
+            request = ClientRequest("POST", path, self.headers, self.data)
+
+        if accountingEnabledForCategory("iSchedule"):
+            self.loggedRequest = yield self.logRequest(request)
+
+        response = (yield proto.submitRequest(request))
+
+        returnValue(response)
+
+
+    def _parseResponse(self, xml):
+
+        # Check for correct root element
+        schedule_response = xml.root_element
+        if not isinstance(schedule_response, ScheduleResponse) or not schedule_response.children:
+            raise HTTPError(responsecode.BAD_REQUEST)
+
+        # Parse each response - do this twice: once looking for errors that will
+        # result in all recipients shown as failures; the second loop adds all the
+        # valid responses to the actual result.
+        for response in schedule_response.children:
+            if not isinstance(response, Response) or not response.children:
+                raise HTTPError(responsecode.BAD_REQUEST)
+            recipient = response.childOfType(Recipient)
+            request_status = response.childOfType(RequestStatus)
+            if not recipient or not request_status:
+                raise HTTPError(responsecode.BAD_REQUEST)
+        for response in schedule_response.children:
+            recipient = str(response.childOfType(Recipient))
+            request_status = str(response.childOfType(RequestStatus))
+            calendar_data = response.childOfType(CalendarData)
+            if calendar_data:
+                calendar_data = str(calendar_data)
+            error = response.childOfType(Error)
+            if error:
+                error = error.children
+            desc = response.childOfType(ResponseDescription)
+            if desc:
+                desc = str(desc)
+            self.responses.clone(
+                recipient,
+                request_status,
+                calendar_data,
+                error,
+                desc,
+            )

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/dkim.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/dkim.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/dkim.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,1017 @@
+##
+# Copyright (c) 2012-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.python.log import Logger
+from twext.web2.client.http import ClientRequest
+from twext.web2.dav.util import allDataFromStream, joinURL
+from twext.web2.http import Response
+from twext.web2.http_headers import MimeType
+from twext.web2.stream import MemoryStream
+
+from twisted.internet.defer import inlineCallbacks, returnValue, succeed
+from twistedcaldav.client.geturl import getURL
+from twistedcaldav.config import ConfigurationError
+from twistedcaldav.simpleresource import SimpleResource, SimpleDataResource
+from txdav.caldav.datastore.scheduling.ischedule.utils import lookupDataViaTXT, \
+    lookupServerViaSRV
+
+from Crypto.Hash import SHA, SHA256
+from Crypto.PublicKey import RSA
+from Crypto.Signature import PKCS1_v1_5
+
+import base64
+import hashlib
+import os
+import textwrap
+import time
+import uuid
+
+"""
+DKIM HTTP message generation and validation,
+"""
+
+log = Logger()
+
+# DKIM/iSchedule Constants
+RSA1 = "rsa-sha1"
+RSA256 = "rsa-sha256"
+Q_DNS = "dns/txt"
+Q_HTTP = "http/well-known"
+Q_PRIVATE = "private-exchange"
+
+KEY_SERVICE_TYPE = "ischedule"
+
+# Headers
+DKIM_SIGNATURE = "DKIM-Signature"
+ISCHEDULE_VERSION = "iSchedule-Version"
+ISCHEDULE_VERSION_VALUE = "1.0"
+ISCHEDULE_MESSAGE_ID = "iSchedule-Message-ID"
+ISCHEDULE_CAPABILITIES = "iSchedule-Capabilities"
+
+
+class DKIMUtils(object):
+    """
+    Some useful functions.
+    """
+
+    @staticmethod
+    def validConfiguration(config):
+        if config.Scheduling.iSchedule.DKIM.Enabled:
+
+            if not config.Scheduling.iSchedule.DKIM.Domain and not config.ServerHostName:
+                msg = "DKIM: No domain specified"
+                log.error(msg)
+                raise ConfigurationError(msg)
+
+            if not config.Scheduling.iSchedule.DKIM.KeySelector:
+                msg = "DKIM: No selector specified"
+                log.error(msg)
+                raise ConfigurationError(msg)
+
+            if config.Scheduling.iSchedule.DKIM.SignatureAlgorithm not in (RSA1, RSA256):
+                msg = "DKIM: Invalid algorithm: %s" % (config.Scheduling.iSchedule.SignatureAlgorithm,)
+                log.error(msg)
+                raise ConfigurationError(msg)
+
+            try:
+                with open(config.Scheduling.iSchedule.DKIM.PrivateKeyFile) as f:
+                    key_data = f.read()
+            except IOError, e:
+                msg = "DKIM: Cannot read private key file: %s %s" % (config.Scheduling.iSchedule.DKIM.PrivateKeyFile, e,)
+                log.error(msg)
+                raise ConfigurationError(msg)
+            try:
+                RSA.importKey(key_data)
+            except:
+                msg = "DKIM: Invalid private key file: %s" % (config.Scheduling.iSchedule.DKIM.PrivateKeyFile,)
+                log.error(msg)
+                raise ConfigurationError(msg)
+
+            try:
+                with open(config.Scheduling.iSchedule.DKIM.PublicKeyFile) as f:
+                    key_data = f.read()
+            except IOError, e:
+                msg = "DKIM: Cannot read public key file: %s %s" % (config.Scheduling.iSchedule.DKIM.PublicKeyFile, e,)
+                log.error(msg)
+                raise ConfigurationError(msg)
+            try:
+                RSA.importKey(key_data)
+            except:
+                msg = "DKIM: Invalid public key file: %s" % (config.Scheduling.iSchedule.DKIM.PublicKeyFile,)
+                log.error(msg)
+                raise ConfigurationError(msg)
+
+            if config.Scheduling.iSchedule.DKIM.PrivateExchanges:
+                if not os.path.exists(config.Scheduling.iSchedule.DKIM.PrivateExchanges):
+                    try:
+                        os.makedirs(config.Scheduling.iSchedule.DKIM.PrivateExchanges)
+                    except IOError, e:
+                        msg = "DKIM: Cannot create public key private exchange directory: %s" % (config.Scheduling.iSchedule.DKIM.PrivateExchanges,)
+                        log.error(msg)
+                        raise ConfigurationError(msg)
+                if not os.path.isdir(config.Scheduling.iSchedule.DKIM.PrivateExchanges):
+                    msg = "DKIM: Invalid public key private exchange directory: %s" % (config.Scheduling.iSchedule.DKIM.PrivateExchanges,)
+                    log.error(msg)
+                    raise ConfigurationError(msg)
+                PublicKeyLookup_PrivateExchange.directory = config.Scheduling.iSchedule.DKIM.PrivateExchanges
+
+            log.info("DKIM: Enabled")
+        else:
+            log.info("DKIM: Disabled")
+
+
+    @staticmethod
+    def getConfiguration(config):
+        """
+        Return a tuple of the parameters derived from the config that are used to initialize the DKIMRequest.
+
+        @param config: configuration to look at
+        @type config: L{Config}
+        """
+
+        domain = config.Scheduling.iSchedule.DKIM.Domain if config.Scheduling.iSchedule.DKIM.Domain else config.ServerHostName
+        selector = config.Scheduling.iSchedule.DKIM.KeySelector
+        key_file = config.Scheduling.iSchedule.DKIM.PrivateKeyFile
+        algorithm = config.Scheduling.iSchedule.DKIM.SignatureAlgorithm
+        useDNSKey = config.Scheduling.iSchedule.DKIM.UseDNSKey
+        useHTTPKey = config.Scheduling.iSchedule.DKIM.UseHTTPKey
+        usePrivateExchangeKey = config.Scheduling.iSchedule.DKIM.UsePrivateExchangeKey
+        expire = config.Scheduling.iSchedule.DKIM.ExpireSeconds
+
+        return domain, selector, key_file, algorithm, useDNSKey, useHTTPKey, usePrivateExchangeKey, expire
+
+
+    @staticmethod
+    def hashlib_method(algorithm):
+        """
+        Return hashlib function for DKIM algorithm.
+        """
+        return {
+            RSA1  : hashlib.sha1,
+            RSA256: hashlib.sha256,
+        }[algorithm]
+
+
+    @staticmethod
+    def hash_name(algorithm):
+        """
+        Return RSA hash name for DKIM algorithm.
+        """
+        return {
+            RSA1  : "SHA-1",
+            RSA256: "SHA-256",
+        }[algorithm]
+
+
+    @staticmethod
+    def hash_func(algorithm):
+        """
+        Return RSA hash name for DKIM algorithm.
+        """
+        return {
+            RSA1  : SHA,
+            RSA256: SHA256,
+        }[algorithm]
+
+
+    @staticmethod
+    def extractTags(data):
+        """
+        Split a DKIM tag list into a dict, removing unneeded whitespace.
+        """
+        # Extract tags from the data
+        splits = [item.strip() for item in data.split(";")]
+        dkim_tags = {}
+        for item in splits:
+            try:
+                name, value = item.split("=", 1)
+                dkim_tags[name.strip()] = value.strip()
+            except ValueError:
+                pass
+        return dkim_tags
+
+
+    @staticmethod
+    def canonicalizeHeader(name, value, dkim_tags=None, method="ischedule-relaxed"):
+
+        if method == "relaxed":
+            return DKIMUtils.relaxedHeader(name, value, dkim_tags)
+        elif method == "ischedule-relaxed":
+            return DKIMUtils.ischeduleHeader(name, value, dkim_tags)
+        else:
+            assert "Invalid header canonicalization method: %s" % (method,)
+
+
+    @staticmethod
+    def relaxedHeader(name, value, dkim_tags=None):
+        """
+        Canonicalize the header using "relaxed" method. Optionally remove the b= value from
+        any DKIM-Signature present.
+
+        FIXME: this needs to be smarter about where valid WSP can occur in a header. Right now it will
+        blindly collapse all runs of SP/HTAB into a single SP. That could be wrong if a legitimate sequence of
+        SP/HTAB occurs in a header value.
+
+        @param name: header name
+        @type name: C{str}
+        @param value: header value
+        @type value: C{str}
+        @param dkim_tags: the extracted DKIM tags, or C{None} if no removal needed
+        @type dkim_tags: C{dict} or C{None}
+        """
+
+        # Special case DKIM-Signature: remove the b= value for signature
+        name = name.lower()
+        if dkim_tags is not None and name == DKIM_SIGNATURE.lower():
+            value = DKIMUtils.canonicalizeDKIMHeaderFields(value, dkim_tags)
+
+        # Basic relaxed behavior
+        value = " ".join(value.split())
+
+        crlf = "" if name == DKIM_SIGNATURE.lower() else "\r\n"
+        return "%s:%s%s" % (name, value, crlf)
+
+
+    @staticmethod
+    def ischeduleHeader(name, value, dkim_tags=None):
+        """
+        Canonicalize the header using "ischedule-relaxed" method. Optionally remove the b= value from
+        any DKIM-Signature present.
+
+        FIXME: this needs to be smarter about where valid WSP can occur in a header. Right now it will
+        blindly collapse all runs of SP/HTAB into a single SP. That could be wrong if a legitimate sequence of
+        SP/HTAB occurs in a header value.
+
+        @param name: header name
+        @type name: C{str}
+        @param value: header value
+        @type value: C{str}
+        @param dkim_tags: the extracted DKIM tags, or C{None} if no removal needed
+        @type dkim_tags: C{dict} or C{None}
+        """
+
+        # Special case DKIM-Signature: remove the b= value for signature
+        name = name.lower()
+        if dkim_tags is not None and name == DKIM_SIGNATURE.lower():
+            value = DKIMUtils.canonicalizeDKIMHeaderFields(value, dkim_tags)
+
+        # Basic relaxed behavior
+        value = " ".join(value.split())
+        value = value.replace(" ,", ",")
+        value = value.replace(", ", ",")
+
+        crlf = "" if name == DKIM_SIGNATURE.lower() else "\r\n"
+        return "%s:%s%s" % (name, value, crlf)
+
+
+    @staticmethod
+    def canonicalizeDKIMHeaderFields(value, dkim_tags):
+        """
+        DKIM-Signature b= value needs to be stripped.
+
+        @param value: header value to process
+        @type value: C{str}
+        """
+
+        pos = value.find(dkim_tags["b"])
+        value = value[:pos] + value[pos + len(dkim_tags["b"]):]
+        value = " ".join(value.split())
+        return value
+
+
+    @staticmethod
+    def canonicalizeBody(data):
+        if not data.endswith("\r\n"):
+            data += "\r\n"
+        return data
+
+
+    @staticmethod
+    def sign(data, privkey, hashfunc):
+        h = hashfunc.new(data)
+        signer = PKCS1_v1_5.new(privkey)
+        return base64.b64encode(signer.sign(h))
+
+
+    @staticmethod
+    def verify(data, signature, pubkey, hashfunc):
+        h = hashfunc.new(data)
+        verifier = PKCS1_v1_5.new(pubkey)
+        if not verifier.verify(h, base64.b64decode(signature)):
+            raise ValueError()
+
+
+
+class DKIMRequest(ClientRequest):
+    """
+    A ClientRequest that optionally creates a DKIM signature.
+    """
+
+    keys = {}
+
+    def __init__(
+        self,
+        method,
+        uri,
+        headers,
+        stream,
+        domain,
+        selector,
+        key_file,
+        algorithm,
+        sign_headers,
+        useDNSKey,
+        useHTTPKey,
+        usePrivateExchangeKey,
+        expire,
+    ):
+        """
+        Create a DKIM request, which is a regular client request with the additional information needed to sign the message.
+
+        @param method: HTTP method to use
+        @type method: C{str}
+        @param uri: request-URI
+        @type uri: C{str}
+        @param headers: request headers
+        @type headers: L{http_headers}
+        @param stream: body data
+        @type stream: L{Stream}
+        @param domain: the signing domain
+        @type domain: C{str}
+        @param selector: the signing key selector
+        @type selector: C{str}
+        @param key_file: path to a private key file
+        @type key_file: C{str}
+        @param algorithm: the signing algorithm to use
+        @type algorithm: C{str}
+        @param sign_headers: list of header names to sign - to "over sign" a header append a "+" to the name
+        @type sign_headers: C{tuple}
+        @param useDNSKey: whether or not to add DNS TXT lookup as a key lookup option
+        @type useDNSKey: C{bool}
+        @param useHTTPKey: whether or not to add HTTP .well-known as a key lookup option
+        @type useHTTPKey: C{bool}
+        @param usePrivateExchangeKey: whether or not to add private-exchange as a key lookup option
+        @type usePrivateExchangeKey: C{bool}
+        @param expire: number of seconds to expiration of signature
+        @type expire: C{int}
+        """
+        super(DKIMRequest, self).__init__(method, uri, headers, stream)
+        self.domain = domain
+        self.selector = selector
+        self.algorithm = algorithm
+        self.key_file = key_file
+        self.sign_headers = sign_headers
+        self.time = str(int(time.time()))
+        self.expire = str(int(time.time() + expire))
+
+        assert self.domain
+        assert self.selector
+        assert self.algorithm in (RSA1, RSA256,)
+        assert useDNSKey or useHTTPKey or usePrivateExchangeKey
+
+        self.hash_method = DKIMUtils.hashlib_method(self.algorithm)
+        self.hash_name = DKIMUtils.hash_name(self.algorithm)
+        self.hash_func = DKIMUtils.hash_func(self.algorithm)
+
+        self.keyMethods = []
+        if usePrivateExchangeKey:
+            self.keyMethods.append(Q_PRIVATE)
+        if useHTTPKey:
+            self.keyMethods.append(Q_HTTP)
+        if useDNSKey:
+            self.keyMethods.append(Q_DNS)
+
+        self.message_id = str(uuid.uuid4())
+
+
+    @inlineCallbacks
+    def sign(self):
+        """
+        Generate the DKIM headers by signing the request. This should only be called once on the request and there must
+        be no changes to the request (no headers, no body change) after it is called.
+        """
+
+        # Get the headers and the DKIM-Signature tags
+        headers, dkim_tags = (yield self.signatureHeaders())
+
+        # Sign the hash
+        signature = self.generateSignature(headers)
+
+        # Complete the header
+        dkim_tags[-1] = ("b", signature,)
+        dkim_header = "; ".join(["%s=%s" % item for item in dkim_tags])
+        self.headers.addRawHeader(DKIM_SIGNATURE, dkim_header)
+
+        log.debug("DKIM: Generated header: DKIM-Signature:%s" % (dkim_header,))
+        log.debug("DKIM: Signed headers:\n%s" % (headers,))
+
+        returnValue(signature)
+
+
+    @inlineCallbacks
+    def bodyHash(self):
+        """
+        Generate the hash of the request body data.
+        """
+
+        # We need to play a trick with the request stream as we can only read it once. So we
+        # read it, store the value in a MemoryStream, and replace the request's stream with that,
+        # so the data can be read again.
+        data = (yield allDataFromStream(self.stream))
+        self.stream = MemoryStream(data if data is not None else "")
+        self.stream.doStartReading = None
+
+        returnValue(base64.b64encode(self.hash_method(DKIMUtils.canonicalizeBody(data)).digest()))
+
+
+    @inlineCallbacks
+    def signatureHeaders(self):
+        """
+        Generate the headers that are going to be signed as well as the DKIM-Signature tags.
+        """
+
+        # Make sure we have the required iSchedule headers
+        self.headers.addRawHeader(ISCHEDULE_VERSION, ISCHEDULE_VERSION_VALUE)
+        self.headers.addRawHeader(ISCHEDULE_MESSAGE_ID, self.message_id)
+        self.sign_headers += (ISCHEDULE_VERSION, ISCHEDULE_MESSAGE_ID,)
+
+        # Need Cache-Control
+        self.headers.setRawHeaders("Cache-Control", ("no-cache", "no-transform",))
+
+        # Figure out all the existing headers to sign
+        headers = []
+        sign_headers = []
+        raw = dict([(name.lower(), values) for name, values in self.headers.getAllRawHeaders()])
+        for name in self.sign_headers:
+            # ischedule-relaxed canonicalization requires headers with the same name concatenated
+            # with a comma in between
+            value = ",".join(raw.get(name.lower(), ()))
+            headers.append(DKIMUtils.canonicalizeHeader(name, value))
+            sign_headers.append(name)
+
+        # Generate the DKIM header tags we care about
+        dkim_tags = []
+        dkim_tags.append(("v", "1",))
+        dkim_tags.append(("d", self.domain,))
+        dkim_tags.append(("s", self.selector,))
+        dkim_tags.append(("t", self.time,))
+        dkim_tags.append(("x", self.expire,))
+        dkim_tags.append(("a", self.algorithm,))
+        dkim_tags.append(("q", ":".join(self.keyMethods),))
+        dkim_tags.append(("c", "ischedule-relaxed/simple",))
+        dkim_tags.append(("h", ":".join(sign_headers),))
+        dkim_tags.append(("bh", (yield self.bodyHash()),))
+        dkim_tags.append(("b", "",))
+        dkim_header = "; ".join(["%s=%s" % item for item in dkim_tags])
+
+        headers.append(DKIMUtils.canonicalizeHeader(DKIM_SIGNATURE, dkim_header))
+        headers = "".join(headers)
+
+        returnValue((headers, dkim_tags,))
+
+
+    def generateSignature(self, headers):
+        # Sign the hash
+        if self.key_file not in self.keys:
+            self.keys[self.key_file] = RSA.importKey(open(self.key_file).read())
+        return DKIMUtils.sign(headers, self.keys[self.key_file], self.hash_func)
+
+
+
+class DKIMMissingError(Exception):
+    """
+    Used to indicate that the DKIM-Signature header is not present when
+    attempting verification.
+    """
+    pass
+
+
+
+class DKIMVerificationError(Exception):
+    """
+    Used to indicate a DKIM verification error.
+    """
+    pass
+
+
+
+class DKIMVerifier(object):
+    """
+    Class used to verify an DKIM-signed HTTP request.
+    """
+
+    def __init__(self, request, key_lookup=None, protocol_debug=False):
+        """
+        @param request: The HTTP request to process
+        @type request: L{twext.server.Request}
+        """
+        self.request = request
+        self._debug = protocol_debug
+        self.dkim_tags = {}
+
+        # Prefer private exchange over HTTP over DNS when multiple are present
+        self.key_lookup_methods = (
+            PublicKeyLookup_PrivateExchange,
+            PublicKeyLookup_HTTP_WellKnown,
+            PublicKeyLookup_DNSTXT,
+        ) if key_lookup is None else key_lookup
+
+        self.time = int(time.time())
+
+
+    @inlineCallbacks
+    def verify(self):
+        """
+        @raise: DKIMVerificationError
+        """
+
+        # Check presence of DKIM header
+        self.processDKIMHeader()
+
+        # Extract the set of canonicalized headers being signed
+        headers = self.extractSignedHeaders()
+        log.debug("DKIM: Signed headers:\n%s" % (headers,))
+
+        # Locate the public key
+        pubkey = (yield self.locatePublicKey())
+        if pubkey is None:
+            raise DKIMVerificationError("No public key to verify the DKIM signature")
+
+        # Do header verification
+        try:
+            DKIMUtils.verify(headers, self.dkim_tags["_b"], pubkey, self.hash_func)
+        except ValueError:
+            msg = "Could not verify signature"
+            _debug_msg = """
+DKIM-Signature:%s
+
+Headers to evaluate:
+%s
+
+Public key used:
+%s
+""" % (self.request.headers.getRawHeaders(DKIM_SIGNATURE)[0], headers, pubkey._original_data,)
+            log.debug("DKIM: %s:%s" % (msg, _debug_msg,))
+            if self._debug:
+                msg = "%s:%s" % (msg, _debug_msg,)
+            raise DKIMVerificationError(msg)
+
+        # Do body validation
+        data = (yield allDataFromStream(self.request.stream))
+        self.request.stream = MemoryStream(data if data is not None else "")
+        self.request.stream.doStartReading = None
+        body = DKIMUtils.canonicalizeBody(data)
+        bh = base64.b64encode(self.hash_method(body).digest())
+        if bh != self.dkim_tags["_bh"]:
+            msg = "Could not verify the DKIM body hash"
+            _debug_msg = """
+DKIM-Signature:%s
+
+Hash Method: %s
+
+Base64 encoded body:
+%s
+""" % (self.request.headers.getRawHeaders(DKIM_SIGNATURE), self.hash_method.__name__, base64.b64encode(body),)
+            log.debug("DKIM: %s:%s" % (msg, _debug_msg,))
+            if self._debug:
+                msg = "%s:%s" % (msg, _debug_msg,)
+            raise DKIMVerificationError(msg)
+
+
+    def processDKIMHeader(self):
+        """
+        Extract the DKIM-Signature header and process the tags.
+
+        @raise: DKIMVerificationError
+        """
+
+        # Check presence of header
+        dkim = self.request.headers.getRawHeaders(DKIM_SIGNATURE)
+        if dkim is None:
+            msg = "No DKIM-Signature header present in the request"
+            log.debug("DKIM: " + msg)
+            raise DKIMMissingError(msg)
+        if len(dkim) != 1:
+            # TODO: This might need to be changed if we ever support forwarding of iSchedule messages - the forwarder
+            # might also sign the message and add its own header
+            msg = "Only one DKIM-Signature allowed in the request"
+            log.debug("DKIM: " + msg)
+            raise DKIMVerificationError(msg)
+        dkim = dkim[0]
+        log.debug("DKIM: Found header: DKIM-Signature:%s" % (dkim,))
+
+        # Extract tags from the header
+        self.dkim_tags = DKIMUtils.extractTags(dkim)
+
+        # Verify validity of tags
+        required_tags = ("v", "a", "b", "bh", "c", "d", "h", "s",)
+        for tag in required_tags:
+            if tag not in self.dkim_tags:
+                msg = "Missing DKIM-Signature tag: %s" % (tag,)
+                log.debug("DKIM: " + msg)
+                raise DKIMVerificationError(msg)
+
+        check_values = {
+            "v": ("1",),
+            "a": (RSA1, RSA256,),
+            "c": ("ischedule-relaxed", "ischedule-relaxed/simple",),
+            "q": (Q_DNS, Q_HTTP, Q_PRIVATE,),
+        }
+        for tag, values in check_values.items():
+            if tag not in required_tags and tag not in self.dkim_tags:
+                pass
+
+            # Handle some structured values
+            if tag == "q":
+                test = self.dkim_tags[tag].split(":")
+            else:
+                test = (self.dkim_tags[tag],)
+            for item in test:
+                if item not in values:
+                    msg = "Tag: %s has incorrect value: %s" % (tag, self.dkim_tags[tag],)
+                    log.debug("DKIM: " + msg)
+                    raise DKIMVerificationError(msg)
+
+        # Check time stamp
+        if "t" in self.dkim_tags and self.time:
+            diff_time = self.time - int(self.dkim_tags["t"])
+            if diff_time < -360:
+                msg = "Signature time too far in the future: %d seconds" % (diff_time,)
+                log.debug("DKIM: " + msg)
+                raise DKIMVerificationError(msg)
+
+        # Check expiration
+        if "x" in self.dkim_tags and self.time:
+            diff_time = self.time - int(self.dkim_tags["x"])
+            if diff_time > 0:
+                msg = "Signature expired: %d seconds" % (diff_time,)
+                log.debug("DKIM: " + msg)
+                raise DKIMVerificationError(msg)
+
+        # Base64 encoded tags might include WSP which we need to ignore
+        for tag in ("b", "bh",):
+            self.dkim_tags["_%s" % (tag,)] = "".join(self.dkim_tags[tag].split())
+
+        # Some useful bits
+        self.hash_method = DKIMUtils.hashlib_method(self.dkim_tags["a"])
+        self.hash_func = DKIMUtils.hash_func(self.dkim_tags["a"])
+        self.key_methods = self.dkim_tags["q"].split(":")
+
+
+    def extractSignedHeaders(self):
+        """
+        Extract the set of headers from the request that are supposed to be signed. Canonicalize them
+        and return the expected signed data.
+        """
+
+        # Extract all the expected signed headers taking into account multiple occurrences of a header
+        # which get concatenated with a single comma in between.
+        header_list = [hdr.strip() for hdr in self.dkim_tags["h"].split(":")]
+
+        headers = []
+        for header in header_list:
+            actual_headers = self.request.headers.getRawHeaders(header)
+            if actual_headers:
+                headers.append((header, ",".join(actual_headers),))
+
+        # DKIM-Signature is always included at the end
+        headers.append((DKIM_SIGNATURE, self.request.headers.getRawHeaders(DKIM_SIGNATURE)[0],))
+
+        # Now canonicalize the values
+        return "".join([DKIMUtils.canonicalizeHeader(name, value, dkim_tags=self.dkim_tags) for name, value in headers])
+
+
+    @inlineCallbacks
+    def locatePublicKey(self):
+        """
+        Try to lookup the public key matching the signature.
+        """
+
+        for lookup in self.key_lookup_methods:
+            if lookup.method in self.key_methods or lookup.method == "*":
+                pubkey = (yield lookup(self.dkim_tags).getPublicKey())
+                if pubkey is not None:
+                    returnValue(pubkey)
+        else:
+            returnValue(None)
+
+
+
+class PublicKeyLookup(object):
+    """
+    Abstract base class for public key lookup methods.
+
+    The L{method} attribute indicated the DKIM q= lookup method that the class will support, or if set to "*",
+    the class will handle any q= value.
+    """
+
+    keyCache = {}
+    method = None
+
+    def __init__(self, dkim_tags):
+        self.dkim_tags = dkim_tags
+
+
+    @inlineCallbacks
+    def getPublicKey(self, useCache=True):
+        """
+        Get key from cache or directly do query.
+
+        @param useCache: whether or not to use the cache
+        @type useCache: C{bool}
+        """
+        key = self._getSelectorKey()
+        if key not in PublicKeyLookup.keyCache or not useCache:
+            pubkeys = (yield self._lookupKeys())
+            PublicKeyLookup.keyCache[key] = pubkeys
+
+        returnValue(self._selectKey())
+
+
+    def _getSelectorKey(self):
+        """
+        Get a token used to uniquely identify the key being looked up. Token format will
+        depend on the lookup method.
+        """
+        raise NotImplementedError
+
+
+    def _lookupKeys(self):
+        """
+        Do the key lookup using the actual lookup method. Return a C{list} of C{dict}
+        that contains the key tag-list. Return a L{Deferred}.
+        """
+        raise NotImplementedError
+
+
+    def _selectKey(self):
+        """
+        Select a specific key from the list that best matches the DKIM-Signature tags
+        """
+
+        pubkeys = PublicKeyLookup.keyCache.get(self._getSelectorKey(), [])
+        for pkey in pubkeys:
+            # Check validity
+            if pkey.get("v", "DKIM1") != "DKIM1":
+                continue
+
+            # Check key type
+            if pkey.get("k", "rsa") != "rsa":
+                continue
+
+            # Check valid hash algorithms
+            hashes = set([hash.strip() for hash in pkey.get("h", "sha1:sha256").split(":")])
+            if self.dkim_tags["a"][4:] not in hashes:
+                continue
+
+            # Service type
+            if pkey.get("s", KEY_SERVICE_TYPE) not in ("*", KEY_SERVICE_TYPE,):
+                continue
+
+            # Non-revoked key
+            if len(pkey.get("p", "")) == 0:
+                continue
+
+            return self._makeKey(pkey)
+
+        log.debug("DKIM: No valid public key: %s %s" % (self._getSelectorKey(), pubkeys,))
+        return None
+
+
+    def _makeKey(self, pkey):
+        """
+        Turn the key tag list into an actual RSA public key object
+
+        @param pkey: key tag list
+        @type pkey: C{list}
+        """
+        key_data = """-----BEGIN PUBLIC KEY-----
+%s
+-----END PUBLIC KEY-----
+""" % ("\n".join(textwrap.wrap(pkey["p"], 64)),)
+
+        try:
+            key = RSA.importKey(key_data)
+            key._original_data = key_data
+            return key
+        except:
+            log.debug("DKIM: Unable to make public key:\n%s" % (key_data,))
+            return None
+
+
+    def flushCache(self):
+        PublicKeyLookup.keyCache = {}
+
+
+
+class PublicKeyLookup_DNSTXT(PublicKeyLookup):
+
+    method = Q_DNS
+
+    def _getSelectorKey(self):
+        """
+        Get a token used to uniquely identify the key being looked up. Token format will
+        depend on the lookup method.
+        """
+        return "%s._domainkey.%s" % (self.dkim_tags["s"], self.dkim_tags["d"],)
+
+
+    @inlineCallbacks
+    def _lookupKeys(self):
+        """
+        Do the key lookup using the actual lookup method.
+        """
+        log.debug("DKIM: TXT lookup: %s" % (self._getSelectorKey(),))
+        data = (yield lookupDataViaTXT(self._getSelectorKey()))
+        log.debug("DKIM: TXT lookup results: %s\n%s" % (self._getSelectorKey(), "\n".join(data),))
+        returnValue(tuple([DKIMUtils.extractTags(line) for line in data]))
+
+
+
+class PublicKeyLookup_HTTP_WellKnown(PublicKeyLookup):
+
+    method = Q_HTTP
+
+    def _getSelectorKey(self):
+        """
+        Get a token used to uniquely identify the key being looked up. Token format will
+        depend on the lookup method.
+        """
+
+        host = ".".join(self.dkim_tags["d"].split(".")[-2:])
+        return "https://%s/.well-known/domainkey/%s/%s" % (host, self.dkim_tags["d"], self.dkim_tags["s"],)
+
+
+    @inlineCallbacks
+    def _getURI(self):
+        """
+        Determine the well-known URI for the public key service.
+        """
+
+        # First we do an SRV lookup for _domainkey to get the public key server host/port
+        result = (yield lookupServerViaSRV(self.dkim_tags["d"], service="_domainkey"))
+        if result is None:
+            log.debug("DKIM: SRV _domainkey failed on: %s trying domain directly" % (self.dkim_tags["d"],))
+            host = self.dkim_tags["d"]
+            port = ""
+            scheme = "https"
+        else:
+            host, port = result
+            scheme = "http" if port in (80, 8008, 8080,) else "https"
+            if port == 80 and scheme == "http" or port == 443 and scheme == "https":
+                port = ""
+            else:
+                port = ":%s" % (port,)
+
+        returnValue("%s://%s%s/.well-known/domainkey/%s/%s" % (scheme, host, port, self.dkim_tags["d"], self.dkim_tags["s"],))
+
+
+    @inlineCallbacks
+    def _lookupKeys(self):
+        """
+        Do the key lookup using the actual lookup method.
+        """
+
+        # First we do an SRV lookup for _domainkey to get the public key server URI
+        uri = (yield self._getURI())
+
+        log.debug("DKIM: HTTP/.well-known lookup: %s" % (uri,))
+        response = (yield getURL(uri))
+        if response is None or response.code / 100 != 2:
+            log.debug("DKIM: Failed http/well-known lookup: %s %s" % (uri, response,))
+            returnValue(())
+
+        ct = response.headers.getRawHeaders("content-type", ("bogus/type",))[0]
+        ct = ct.split(";", 1)
+        ct = ct[0].strip()
+        if ct not in ("text/plain",):
+            log.debug("DKIM: Failed http/well-known lookup: wrong content-type returned %s %s" % (uri, ct,))
+            returnValue(())
+
+        log.debug("DKIM: HTTP/.well-known lookup results: %s\n%s" % (uri, response.data,))
+        returnValue(tuple([DKIMUtils.extractTags(line) for line in response.data.splitlines()]))
+
+
+
+class PublicKeyLookup_PrivateExchange(PublicKeyLookup):
+
+    method = Q_PRIVATE
+    directory = None
+
+    def _getSelectorKey(self):
+        """
+        Get a token used to uniquely identify the key being looked up. Token format will
+        depend on the lookup method.
+        """
+        return "%s#%s" % (self.dkim_tags["d"], self.dkim_tags["s"],)
+
+
+    def _lookupKeys(self):
+        """
+        Key information is stored in a file, one record per line.
+        """
+
+        # Check validity of paths
+        if PublicKeyLookup_PrivateExchange.directory is None:
+            log.debug("DKIM: Failed private-exchange lookup: no directory configured")
+            return succeed(())
+        keyfile = os.path.join(PublicKeyLookup_PrivateExchange.directory, self._getSelectorKey())
+        if not os.path.exists(keyfile):
+            log.debug("DKIM: Failed private-exchange lookup: no path %s" % (keyfile,))
+            return succeed(())
+
+        # Now read the data
+        log.debug("DKIM: Private exchange lookup: %s" % (keyfile,))
+        try:
+            with open(keyfile) as f:
+                keys = f.read()
+        except IOError, e:
+            log.debug("DKIM: Failed private-exchange lookup: could not read %s %s" % (keyfile, e,))
+            return succeed(())
+
+        log.debug("DKIM: Private exchange lookup results: %s\n%s" % (keyfile, keys))
+        return succeed(tuple([DKIMUtils.extractTags(line) for line in keys.splitlines()]))
+
+
+
+class DomainKeyResource (SimpleResource):
+    """
+    Domainkey well-known resource.
+    """
+
+    def __init__(self, domain, selector, pubkeyfile):
+        """
+        """
+        assert domain
+        assert selector
+
+        SimpleResource.__init__(self, principalCollections=None, isdir=True, defaultACL=SimpleResource.allReadACL)
+        self.makeKeyData(domain, selector, pubkeyfile)
+        self.domain = domain
+        self.selector = selector
+
+
+    def makeKeyData(self, domain, selector, pubkeyfile):
+        """
+        Check that a valid key exists, create the TXT record format data and make the needed child resources.
+        """
+
+        # Get data from file
+        try:
+            with open(pubkeyfile) as f:
+                key_data = f.read()
+        except IOError, e:
+            log.error("DKIM: Unable to open the public key file: %s because of %s" % (pubkeyfile, e,))
+            raise
+
+        # Make sure we can parse a valid public key
+        try:
+            RSA.importKey(key_data)
+        except:
+            log.error("DKIM: Invalid public key file: %s" % (pubkeyfile,))
+            raise
+
+        # Make the TXT record
+        key_data = "".join(key_data.strip().splitlines()[1:-1])
+        txt_data = "v=DKIM1; s=ischedule; p=%s\n" % (key_data,)
+
+        # Setup resource hierarchy
+        domainResource = SimpleResource(principalCollections=None, isdir=True, defaultACL=SimpleResource.allReadACL)
+        self.putChild(domain, domainResource)
+
+        selectorResource = SimpleDataResource(principalCollections=None, content_type=MimeType.fromString("text/plain"), data=txt_data, defaultACL=SimpleResource.allReadACL)
+        domainResource.putChild(selector, selectorResource)
+
+
+    def contentType(self):
+        return MimeType.fromString("text/html; charset=utf-8")
+
+
+    def render(self, request):
+        output = """<html>
+<head>
+<title>DomainKey Resource</title>
+</head>
+<body>
+<h1>DomainKey Resource.</h1>
+<a href="%s">Domain: %s<br>
+Selector: %s</a>
+</body
+</html>""" % (joinURL(request.uri, self.domain, self.selector), self.domain, self.selector,)
+
+        response = Response(200, {}, output)
+        response.headers.setHeader("content-type", MimeType("text", "html"))
+        return response

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/localservers.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/localservers.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/localservers.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,324 @@
+##
+# Copyright (c) 2011-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.python.log import Logger
+from twisted.internet.abstract import isIPAddress
+from twistedcaldav.client.pool import installPool
+from twistedcaldav.config import config, fullServerPath
+from twistedcaldav.xmlutil import readXML
+
+from txdav.caldav.datastore.scheduling.ischedule.utils import getIPsFromHost
+import socket
+import urlparse
+
+"""
+XML based server configuration file handling.
+
+This is used in an environment where more than one server is being used within a single domain. i.e., all
+the principals across the whole domain need to be able to directly schedule each other and know of each others
+existence. A common scenario would be a production server and a development/test server.
+
+Each server is identified by an id and url. The id is used when assigning principals to a specific server. Each
+server can also support multiple partitions, and each of those is identified by an id and url, with the id also
+being used to assign principals to a specific partition.
+
+These servers support the concept of "partitioning" and "podding".
+
+A "partitioned" service is one that spreads its
+users out across multiple stores and does reverse proxying of incoming requests to the appropriate partitioned host.
+All servers within the same partition have to be running the same version of the software etc.
+
+A "podded" service is one where different groups of users are hosted on different servers, which may be of
+different versions etc. A "pod" may itself be "partitioned", but the partitioning is "invisible" to the outside world.
+"""
+
+__all__ = [
+    "Servers",
+]
+
+log = Logger()
+
+SERVER_SECRET_HEADER = "X-CALENDARSERVER-ISCHEDULE"
+
+class ServersDB(object):
+    """
+    Represents the set of servers within the same domain.
+    """
+
+    def __init__(self):
+
+        self._servers = {}
+        self._xmlFile = None
+        self._thisServer = None
+
+
+    def load(self, xmlFile=None, ignoreIPLookupFailures=False):
+        if self._xmlFile is None or xmlFile is not None:
+            self._servers = {}
+            if xmlFile:
+                self._xmlFile = xmlFile
+            else:
+                self._xmlFile = fullServerPath(
+                    config.ConfigRoot,
+                    config.Servers.ConfigFile
+                )
+        self._servers = ServersParser.parse(self._xmlFile, ignoreIPLookupFailures=ignoreIPLookupFailures)
+        for server in self._servers.values():
+            if server.thisServer:
+                self._thisServer = server
+                break
+        else:
+            raise ValueError("No server in %s matches this server." % (self._xmlFile,))
+
+
+    def clear(self):
+        self._servers = {}
+        self._xmlFile = None
+        self._thisServer = None
+
+
+    def getServerById(self, id):
+        return self._servers.get(id)
+
+
+    def getServerURIById(self, id):
+        try:
+            return self._servers[id].uri
+        except KeyError:
+            return None
+
+
+    def getThisServer(self):
+        return self._thisServer
+
+Servers = ServersDB()   # Global server DB
+
+
+
+class Server(object):
+    """
+    Represents a server which may itself be partitioned.
+    """
+
+    def __init__(self):
+        self.id = None
+        self.uri = None
+        self.thisServer = False
+        self.ips = set()
+        self.allowed_from_ips = set()
+        self.shared_secret = None
+        self.partitions = {}
+        self.partitions_ips = set()
+        self.isImplicit = True
+
+
+    def check(self, ignoreIPLookupFailures=False):
+        # Check whether this matches the current server
+        parsed_uri = urlparse.urlparse(self.uri)
+        if parsed_uri.hostname == config.ServerHostName:
+            if parsed_uri.scheme == "http":
+                if config.HTTPPort:
+                    self.thisServer = parsed_uri.port in (config.HTTPPort,) + tuple(config.BindHTTPPorts)
+            elif parsed_uri.scheme == "https":
+                if config.SSLPort:
+                    self.thisServer = parsed_uri.port in (config.SSLPort,) + tuple(config.BindSSLPorts)
+
+        # Need to cache IP addresses
+        try:
+            ips = getIPsFromHost(parsed_uri.hostname)
+        except socket.gaierror, e:
+            msg = "Unable to lookup ip-addr for server '%s': %s" % (parsed_uri.hostname, str(e))
+            log.error(msg)
+            if ignoreIPLookupFailures:
+                ips = ()
+            else:
+                raise ValueError(msg)
+        self.ips = set(ips)
+
+        actual_ips = set()
+        for item in self.allowed_from_ips:
+            if not isIPAddress(item):
+                try:
+                    ips = getIPsFromHost(item)
+                except socket.gaierror, e:
+                    msg = "Unable to lookup ip-addr for allowed-from '%s': %s" % (item, str(e))
+                    log.error(msg)
+                    if not ignoreIPLookupFailures:
+                        raise ValueError(msg)
+                else:
+                    actual_ips.update(ips)
+            else:
+                actual_ips.add(item)
+        self.allowed_from_ips = actual_ips
+
+        for uri in self.partitions.values():
+            parsed_uri = urlparse.urlparse(uri)
+            try:
+                ips = getIPsFromHost(parsed_uri.hostname)
+            except socket.gaierror, e:
+                msg = "Unable to lookup ip-addr for partition '%s': %s" % (parsed_uri.hostname, str(e))
+                log.error(msg)
+                if ignoreIPLookupFailures:
+                    ips = ()
+                else:
+                    raise ValueError(msg)
+            self.partitions_ips.update(ips)
+
+
+    def checkThisIP(self, ip):
+        """
+        Check that the passed in IP address corresponds to this server or one of its partitions.
+        """
+        return (ip in self.ips) or (ip in self.partitions_ips)
+
+
+    def hasAllowedFromIP(self):
+        return len(self.allowed_from_ips) > 0
+
+
+    def checkAllowedFromIP(self, ip):
+        return ip in self.allowed_from_ips
+
+
+    def checkSharedSecret(self, request):
+
+        # Get header from the request
+        request_secret = request.headers.getRawHeaders(SERVER_SECRET_HEADER)
+
+        if request_secret is not None and self.shared_secret is None:
+            log.error("iSchedule request included unexpected %s header" % (SERVER_SECRET_HEADER,))
+            return False
+        elif request_secret is None and self.shared_secret is not None:
+            log.error("iSchedule request did not include required %s header" % (SERVER_SECRET_HEADER,))
+            return False
+        elif (request_secret[0] if request_secret else None) != self.shared_secret:
+            log.error("iSchedule request %s header did not match" % (SERVER_SECRET_HEADER,))
+            return False
+        else:
+            return True
+
+
+    def secretHeader(self):
+        """
+        Return a tuple of header name, header value
+        """
+        return (SERVER_SECRET_HEADER, self.shared_secret,)
+
+
+    def addPartition(self, id, uri):
+        self.partitions[id] = uri
+
+
+    def getPartitionURIForId(self, id):
+        return self.partitions.get(id)
+
+
+    def isPartitioned(self):
+        return len(self.partitions) != 0
+
+
+    def installReverseProxies(self, ownUID, maxClients):
+
+        for partition, url in self.partitions.iteritems():
+            if partition != ownUID:
+                installPool(
+                    partition,
+                    url,
+                    maxClients,
+                )
+
+
+
+ELEMENT_SERVERS = "servers"
+ELEMENT_SERVER = "server"
+ELEMENT_ID = "id"
+ELEMENT_URI = "uri"
+ELEMENT_ALLOWED_FROM = "allowed-from"
+ELEMENT_SHARED_SECRET = "shared-secret"
+ELEMENT_PARTITIONS = "partitions"
+ELEMENT_PARTITION = "partition"
+ATTR_IMPLICIT = "implicit"
+ATTR_VALUE_YES = "yes"
+ATTR_VALUE_NO = "no"
+
+class ServersParser(object):
+    """
+    Servers configuration file parser.
+    """
+    @staticmethod
+    def parse(xmlFile, ignoreIPLookupFailures=False):
+
+        results = {}
+
+        # Read in XML
+        try:
+            _ignore_tree, servers_node = readXML(xmlFile, ELEMENT_SERVERS)
+        except ValueError, e:
+            log.error("XML parse error for '%s' because: %s" % (xmlFile, e,), raiseException=RuntimeError)
+
+        for child in servers_node.getchildren():
+
+            if child.tag != ELEMENT_SERVER:
+                log.error("Unknown server type: '%s' in servers file: '%s'" % (child.tag, xmlFile,), raiseException=RuntimeError)
+
+            server = Server()
+            server.isImplicit = child.get(ATTR_IMPLICIT, ATTR_VALUE_YES) == ATTR_VALUE_YES
+
+            for node in child.getchildren():
+                if node.tag == ELEMENT_ID:
+                    server.id = node.text
+                elif node.tag == ELEMENT_URI:
+                    server.uri = node.text
+                elif node.tag == ELEMENT_ALLOWED_FROM:
+                    server.allowed_from_ips.add(node.text)
+                elif node.tag == ELEMENT_SHARED_SECRET:
+                    server.shared_secret = node.text
+                elif node.tag == ELEMENT_PARTITIONS:
+                    ServersParser._parsePartition(xmlFile, node, server)
+                else:
+                    log.error("Invalid element '%s' in servers file: '%s'" % (node.tag, xmlFile,), raiseException=RuntimeError)
+
+            if server.id is None or server.uri is None:
+                log.error("Invalid partition '%s' in servers file: '%s'" % (child.tag, xmlFile,), raiseException=RuntimeError)
+
+            server.check(ignoreIPLookupFailures=ignoreIPLookupFailures)
+            results[server.id] = server
+
+        return results
+
+
+    @staticmethod
+    def _parsePartition(xmlFile, partitions, server):
+
+        for child in partitions.getchildren():
+
+            if child.tag != ELEMENT_PARTITION:
+                log.error("Unknown partition type: '%s' in servers file: '%s'" % (child.tag, xmlFile,), raiseException=RuntimeError)
+
+            id = None
+            uri = None
+            for node in child.getchildren():
+                if node.tag == ELEMENT_ID:
+                    id = node.text
+                elif node.tag == ELEMENT_URI:
+                    uri = node.text
+                else:
+                    log.error("Invalid element '%s' in augment file: '%s'" % (node.tag, xmlFile,), raiseException=RuntimeError)
+
+            if id is None or uri is None:
+                log.error("Invalid partition '%s' in servers file: '%s'" % (child.tag, xmlFile,), raiseException=RuntimeError)
+
+            server.addPartition(id, uri)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/remoteservers.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/remoteservers.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/remoteservers.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,227 @@
+##
+# Copyright (c) 2006-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.python.filepath import CachingFilePath as FilePath
+
+from twext.python.log import Logger
+
+from twistedcaldav.config import config, fullServerPath
+from twistedcaldav import xmlutil
+
+"""
+XML based iSchedule configuration file handling. This is for handling of remote servers. The localservers.py module
+handles servers that are local (partitioned or podded).
+"""
+
+__all__ = [
+    "IScheduleServers",
+]
+
+log = Logger()
+
+
+
+class IScheduleServers(object):
+
+    _fileInfo = None
+    _xmlFile = None
+    _servers = None
+    _domainMap = None
+
+    def __init__(self):
+
+        self._loadConfig()
+
+
+    def _loadConfig(self):
+        if config.Scheduling.iSchedule.RemoteServers:
+            if IScheduleServers._servers is None:
+                IScheduleServers._xmlFile = FilePath(
+                    fullServerPath(
+                        config.ConfigRoot,
+                        config.Scheduling.iSchedule.RemoteServers,
+                    )
+                )
+            if IScheduleServers._xmlFile.exists():
+                IScheduleServers._xmlFile.restat()
+                fileInfo = (IScheduleServers._xmlFile.getmtime(), IScheduleServers._xmlFile.getsize())
+                if fileInfo != IScheduleServers._fileInfo:
+                    parser = IScheduleServersParser(IScheduleServers._xmlFile)
+                    IScheduleServers._servers = parser.servers
+                    self._mapDomains()
+                    IScheduleServers._fileInfo = fileInfo
+            else:
+                IScheduleServers._servers = ()
+                IScheduleServers._domainMap = {}
+
+        else:
+            IScheduleServers._servers = ()
+            IScheduleServers._domainMap = {}
+
+
+    def _mapDomains(self):
+        IScheduleServers._domainMap = {}
+        for server in IScheduleServers._servers:
+            for domain in server.domains:
+                IScheduleServers._domainMap[domain] = server
+
+
+    def mapDomain(self, domain):
+        """
+        Map a calendar user address domain to a suitable server that can
+        handle server-to-server requests for that user.
+        """
+        return IScheduleServers._domainMap.get(domain)
+
+ELEMENT_SERVERS = "servers"
+ELEMENT_SERVER = "server"
+ELEMENT_URI = "uri"
+ELEMENT_AUTHENTICATION = "authentication"
+ATTRIBUTE_TYPE = "type"
+ATTRIBUTE_BASICAUTH = "basic"
+ELEMENT_USER = "user"
+ELEMENT_PASSWORD = "password"
+ELEMENT_ALLOW_REQUESTS_FROM = "allow-requests-from"
+ELEMENT_ALLOW_REQUESTS_TO = "allow-requests-to"
+ELEMENT_DOMAINS = "domains"
+ELEMENT_DOMAIN = "domain"
+ELEMENT_CLIENT_HOSTS = "hosts"
+ELEMENT_HOST = "host"
+
+
+
+class IScheduleServersParser(object):
+    """
+    Server-to-server configuration file parser.
+    """
+    def __repr__(self):
+        return "<%s %r>" % (self.__class__.__name__, self.xmlFile)
+
+
+    def __init__(self, xmlFile):
+
+        self.servers = []
+
+        # Read in XML
+        _ignore_etree, servers_node = xmlutil.readXML(xmlFile.path, ELEMENT_SERVERS)
+        self._parseXML(servers_node)
+
+
+    def _parseXML(self, node):
+        """
+        Parse the XML root node from the server-to-server configuration document.
+        @param node: the L{Node} to parse.
+        """
+
+        for child in node.getchildren():
+            if child.tag == ELEMENT_SERVER:
+                self.servers.append(IScheduleServerRecord())
+                self.servers[-1].parseXML(child)
+
+
+
+class IScheduleServerRecord (object):
+    """
+    Contains server-to-server details.
+    """
+    def __init__(self, uri=None):
+        """
+        @param recordType: record type for directory entry.
+        """
+        self.uri = ""
+        self.authentication = None
+        self.allow_from = False
+        self.allow_to = True
+        self.domains = []
+        self.client_hosts = []
+        self.unNormalizeAddresses = True
+        self.moreHeaders = []
+
+        if uri:
+            self.uri = uri
+            self._parseDetails()
+
+
+    def details(self):
+        return (self.ssl, self.host, self.port, self.path,)
+
+
+    def redirect(self, location):
+        """
+        Permanent redirect for the lifetime of this record.
+        """
+        self.uri = location
+        self._parseDetails()
+
+
+    def parseXML(self, node):
+        for child in node.getchildren():
+            if child.tag == ELEMENT_URI:
+                self.uri = child.text
+            elif child.tag == ELEMENT_AUTHENTICATION:
+                self._parseAuthentication(child)
+            elif child.tag == ELEMENT_ALLOW_REQUESTS_FROM:
+                self.allow_from = True
+            elif child.tag == ELEMENT_ALLOW_REQUESTS_TO:
+                self.allow_to = True
+            elif child.tag == ELEMENT_DOMAINS:
+                self._parseList(child, ELEMENT_DOMAIN, self.domains)
+            elif child.tag == ELEMENT_CLIENT_HOSTS:
+                self._parseList(child, ELEMENT_HOST, self.client_hosts)
+            else:
+                raise RuntimeError("[%s] Unknown attribute: %s" % (self.__class__, child.tag,))
+
+        self._parseDetails()
+
+
+    def _parseList(self, node, element_name, appendto):
+        for child in node.getchildren():
+            if child.tag == element_name:
+                appendto.append(child.text)
+
+
+    def _parseAuthentication(self, node):
+        if node.get(ATTRIBUTE_TYPE) != ATTRIBUTE_BASICAUTH:
+            return
+
+        for child in node.getchildren():
+            if child.tag == ELEMENT_USER:
+                user = child.text
+            elif child.tag == ELEMENT_PASSWORD:
+                password = child.text
+
+        self.authentication = ("basic", user, password,)
+
+
+    def _parseDetails(self):
+        # Extract scheme, host, port and path
+        if self.uri.startswith("http://"):
+            self.ssl = False
+            rest = self.uri[7:]
+        elif self.uri.startswith("https://"):
+            self.ssl = True
+            rest = self.uri[8:]
+
+        splits = rest.split("/", 1)
+        hostport = splits[0].split(":")
+        self.host = hostport[0]
+        if len(hostport) > 1:
+            self.port = int(hostport[1])
+        else:
+            self.port = {False: 80, True: 443}[self.ssl]
+        self.path = "/"
+        if len(splits) > 1:
+            self.path += splits[1]

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/resource.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/resource.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/resource.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,256 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from pycalendar.datetime import PyCalendarDateTime
+from pycalendar.timezone import PyCalendarTimezone
+
+from twext.web2 import responsecode
+from twext.web2.dav.noneprops import NonePropertyStore
+from twext.web2.http import Response, HTTPError, StatusResponse, XMLResponse
+from twext.web2.http_headers import MimeType
+
+from twisted.internet.defer import succeed, returnValue, inlineCallbacks
+
+from twistedcaldav import caldavxml
+from twistedcaldav.config import config
+from twistedcaldav.directory.util import transactionFromRequest
+from twistedcaldav.extensions import DAVResource, \
+    DAVResourceWithoutChildrenMixin
+from twistedcaldav.resource import ReadOnlyNoCopyResourceMixIn
+
+from txdav.caldav.datastore.scheduling.caldav.resource import deliverSchedulePrivilegeSet
+from txdav.caldav.datastore.scheduling.ischedule.dkim import ISCHEDULE_CAPABILITIES
+from txdav.caldav.datastore.scheduling.ischedule.scheduler import IScheduleScheduler
+from txdav.xml import element as davxml
+import txdav.caldav.datastore.scheduling.ischedule.xml  as ischedulexml
+
+__all__ = [
+    "IScheduleInboxResource",
+]
+
+class IScheduleInboxResource (ReadOnlyNoCopyResourceMixIn, DAVResourceWithoutChildrenMixin, DAVResource):
+    """
+    iSchedule Inbox resource.
+
+    Extends L{DAVResource} to provide iSchedule inbox functionality.
+    """
+
+    def __init__(self, parent, store):
+        """
+        @param parent: the parent resource of this one.
+        """
+        assert parent is not None
+
+        DAVResource.__init__(self, principalCollections=parent.principalCollections())
+
+        self.parent = parent
+        self._newStore = store
+
+
+    def deadProperties(self):
+        if not hasattr(self, "_dead_properties"):
+            self._dead_properties = NonePropertyStore(self)
+        return self._dead_properties
+
+
+    def etag(self):
+        return succeed(None)
+
+
+    def checkPreconditions(self, request):
+        return None
+
+
+    def resourceType(self):
+        return davxml.ResourceType.ischeduleinbox
+
+
+    def contentType(self):
+        return MimeType.fromString("text/html; charset=utf-8")
+
+
+    def isCollection(self):
+        return False
+
+
+    def isCalendarCollection(self):
+        return False
+
+
+    def isPseudoCalendarCollection(self):
+        return False
+
+
+    def principalForCalendarUserAddress(self, address):
+        for principalCollection in self.principalCollections():
+            principal = principalCollection.principalForCalendarUserAddress(address)
+            if principal is not None:
+                return principal
+        return None
+
+
+    def render(self, request):
+        output = """<html>
+<head>
+<title>Server To Server Inbox Resource</title>
+</head>
+<body>
+<h1>Server To Server Inbox Resource.</h1>
+</body
+</html>"""
+
+        response = Response(200, {}, output)
+        response.headers.setHeader("content-type", MimeType("text", "html"))
+        return response
+
+
+    def http_GET(self, request):
+        """
+        The iSchedule GET method.
+        """
+
+        if not request.args:
+            # Do normal GET behavior
+            return self.render(request)
+
+        action = request.args.get("action", ("",))
+        if len(action) != 1:
+            raise HTTPError(StatusResponse(
+                responsecode.BAD_REQUEST,
+                "Invalid action parameter",
+            ))
+        action = action[0]
+
+        action = {
+            "capabilities"  : self.doCapabilities,
+        }.get(action, None)
+
+        if action is None:
+            raise HTTPError(StatusResponse(
+                responsecode.BAD_REQUEST,
+                "Unknown action action parameter",
+            ))
+
+        return action(request)
+
+
+    def doCapabilities(self, request):
+        """
+        Return a list of all timezones known to the server.
+        """
+
+        # Determine min/max date-time for iSchedule
+        now = PyCalendarDateTime.getNowUTC()
+        minDateTime = PyCalendarDateTime(now.getYear(), 1, 1, 0, 0, 0, PyCalendarTimezone(utc=True))
+        minDateTime.offsetYear(-1)
+        maxDateTime = PyCalendarDateTime(now.getYear(), 1, 1, 0, 0, 0, PyCalendarTimezone(utc=True))
+        maxDateTime.offsetYear(10)
+
+        result = ischedulexml.QueryResult(
+
+            ischedulexml.Capabilities(
+                ischedulexml.Version.fromString(config.Scheduling.iSchedule.SerialNumber),
+                ischedulexml.Versions(
+                    ischedulexml.Version.fromString("1.0"),
+                ),
+                ischedulexml.SchedulingMessages(
+                    ischedulexml.Component(
+                        ischedulexml.Method(name="REQUEST"),
+                        ischedulexml.Method(name="CANCEL"),
+                        ischedulexml.Method(name="REPLY"),
+                        name="VEVENT"
+                    ),
+                    ischedulexml.Component(
+                        ischedulexml.Method(name="REQUEST"),
+                        ischedulexml.Method(name="CANCEL"),
+                        ischedulexml.Method(name="REPLY"),
+                        name="VTODO"
+                    ),
+                    ischedulexml.Component(
+                        ischedulexml.Method(name="REQUEST"),
+                        name="VFREEBUSY"
+                    ),
+                ),
+                ischedulexml.CalendarDataTypes(
+                    ischedulexml.CalendarDataType(**{
+                            "content-type": "text/calendar",
+                            "version": "2.0",
+                    }),
+                ),
+                ischedulexml.Attachments(
+                    ischedulexml.External(),
+                ),
+                ischedulexml.MaxContentLength.fromString(config.MaxResourceSize),
+                ischedulexml.MinDateTime.fromString(minDateTime.getText()),
+                ischedulexml.MaxDateTime.fromString(maxDateTime.getText()),
+                ischedulexml.MaxInstances.fromString(config.MaxAllowedInstances),
+                ischedulexml.MaxRecipients.fromString(config.MaxAttendeesPerInstance),
+                ischedulexml.Administrator.fromString(request.unparseURL(params="", querystring="", fragment="")),
+            ),
+        )
+        response = XMLResponse(responsecode.OK, result)
+        response.headers.addRawHeader(ISCHEDULE_CAPABILITIES, str(config.Scheduling.iSchedule.SerialNumber))
+        return response
+
+
+    @inlineCallbacks
+    def http_POST(self, request):
+        """
+        The server-to-server POST method.
+        """
+
+        # This is a server-to-server scheduling operation.
+        scheduler = IScheduleScheduler(request, self)
+
+        # Need a transaction to work with
+        txn = transactionFromRequest(request, self._newStore)
+        request._newStoreTransaction = txn
+
+        # Do the POST processing treating this as a non-local schedule
+        try:
+            result = (yield scheduler.doSchedulingViaPOST(txn, use_request_headers=True))
+        except Exception, e:
+            yield txn.abort()
+            raise e
+        else:
+            yield txn.commit()
+        response = result.response()
+        response.headers.addRawHeader(ISCHEDULE_CAPABILITIES, str(config.Scheduling.iSchedule.SerialNumber))
+        returnValue(response)
+
+    ##
+    # ACL
+    ##
+
+
+    def supportedPrivileges(self, request):
+        return succeed(deliverSchedulePrivilegeSet)
+
+
+    def defaultAccessControlList(self):
+        privs = (
+            davxml.Privilege(davxml.Read()),
+            davxml.Privilege(caldavxml.ScheduleDeliver()),
+        )
+
+        return davxml.ACL(
+            # DAV:Read, CalDAV:schedule-deliver for all principals (includes anonymous)
+            davxml.ACE(
+                davxml.Principal(davxml.All()),
+                davxml.Grant(*privs),
+                davxml.Protected(),
+            ),
+        )

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/scheduler.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/scheduler.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/scheduler.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,455 @@
+
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.python.log import Logger
+from twext.web2 import responsecode
+from twext.web2.http import HTTPError, Response
+from twisted.internet.abstract import isIPAddress
+from twisted.internet.defer import inlineCallbacks, returnValue
+from txdav.caldav.datastore.scheduling import addressmapping
+from txdav.caldav.datastore.scheduling.cuaddress import RemoteCalendarUser
+from txdav.caldav.datastore.scheduling.cuaddress import calendarUserFromPrincipal
+from txdav.caldav.datastore.scheduling.ischedule.remoteservers import IScheduleServers
+from txdav.caldav.datastore.scheduling.scheduler import RemoteScheduler, \
+    ScheduleResponseQueue
+import txdav.caldav.datastore.scheduling.ischedule.xml as ixml
+from txdav.caldav.datastore.scheduling.ischedule.localservers import Servers
+from twistedcaldav.util import normalizationLookup
+from txdav.xml import element as davxml
+import itertools
+import re
+import socket
+import urlparse
+from twistedcaldav.config import config
+from txdav.caldav.datastore.scheduling.ischedule.dkim import DKIMVerifier, \
+    DKIMVerificationError, DKIMMissingError
+from twext.web2.http_headers import MimeType
+from twistedcaldav.scheduling.ischedule.xml import ischedule_namespace
+from txdav.xml.base import WebDAVUnknownElement
+from twistedcaldav.scheduling.ischedule.utils import getIPsFromHost
+from twistedcaldav.scheduling.ischedule import xml
+from twistedcaldav.ical import normalizeCUAddress
+
+"""
+L{IScheduleScheduler} - handles deliveries for scheduling messages being POSTed to the iSchedule inbox.
+"""
+
+__all__ = [
+    "IScheduleScheduler",
+]
+
+
+log = Logger()
+
+class ErrorResponse(Response):
+    """
+    A L{Response} object which contains a status code and a L{element.Error}
+    element.
+    Renders itself as a DAV:error XML document.
+    """
+    error = None
+    unregistered = True     # base class is already registered
+
+    def __init__(self, code, error, description=None):
+        """
+        @param code: a response code.
+        @param error: an L{WebDAVElement} identifying the error, or a
+            tuple C{(namespace, name)} with which to create an empty element
+            denoting the error.  (The latter is useful in the case of
+            preconditions and postconditions, not all of which have defined
+            XML element classes.)
+        @param description: an optional string that, if present, will get
+            wrapped in a (twisted_dav_namespace, error-description) element.
+        """
+        if type(error) is tuple:
+            xml_namespace, xml_name = error
+            error = WebDAVUnknownElement()
+            error.namespace = xml_namespace
+            error.name = xml_name
+
+        self.description = description
+        if self.description:
+            output = ixml.Error(error, ixml.ResponseDescription(self.description)).toxml()
+        else:
+            output = ixml.Error(error).toxml()
+
+        Response.__init__(self, code=code, stream=output)
+
+        self.headers.setHeader("content-type", MimeType("text", "xml"))
+
+        self.error = error
+
+
+    def __repr__(self):
+        return "<%s %s %s>" % (self.__class__.__name__, self.code, self.error.sname())
+
+
+
+class IScheduleResponseQueue (ScheduleResponseQueue):
+    """
+    Stores a list of (typically error) responses for use in a
+    L{ScheduleResponse}.
+    """
+
+    schedule_response_element = xml.ScheduleResponse
+    response_element = xml.Response
+    recipient_element = xml.Recipient
+    recipient_uses_href = False
+    request_status_element = xml.RequestStatus
+    error_element = xml.Error
+    response_description_element = xml.ResponseDescription
+    calendar_data_element = xml.CalendarData
+
+
+
+class IScheduleScheduler(RemoteScheduler):
+
+    scheduleResponse = IScheduleResponseQueue
+
+    errorResponse = ErrorResponse
+
+    errorElements = {
+        "originator-missing": (ischedule_namespace, "originator-missing"),
+        "originator-invalid": (ischedule_namespace, "originator-invalid"),
+        "originator-denied": (ischedule_namespace, "originator-denied"),
+        "recipient-missing": (ischedule_namespace, "recipient-missing"),
+        "recipient-invalid": (ischedule_namespace, "recipient-invalid"),
+        "organizer-denied": (ischedule_namespace, "organizer-denied"),
+        "attendee-denied": (ischedule_namespace, "attendee-denied"),
+        "invalid-calendar-data-type": (ischedule_namespace, "invalid-calendar-data-type"),
+        "invalid-calendar-data": (ischedule_namespace, "invalid-calendar-data"),
+        "invalid-scheduling-message": (ischedule_namespace, "invalid-scheduling-message"),
+        "max-recipients": (ischedule_namespace, "max-recipients"),
+    }
+
+    @inlineCallbacks
+    def doSchedulingViaPOST(self, transaction, use_request_headers=False):
+        """
+        Carry out iSchedule specific processing.
+        """
+
+        self.verified = False
+        if config.Scheduling.iSchedule.DKIM.Enabled:
+            verifier = DKIMVerifier(self.request, protocol_debug=config.Scheduling.iSchedule.DKIM.ProtocolDebug)
+            try:
+                yield verifier.verify()
+                self.verified = True
+
+            except DKIMMissingError:
+                # Carry on processing, but we will do extra checks on the originator as we would
+                # when DKIM is not enabled, so that any local policy via remoteservers.xml can be used.
+                pass
+
+            except DKIMVerificationError, e:
+                # If DKIM is enabled and there was a DKIM header present, then fail
+                msg = "Failed to verify DKIM signature"
+                _debug_msg = str(e)
+                log.debug("%s:%s" % (msg, _debug_msg,))
+                if config.Scheduling.iSchedule.DKIM.ProtocolDebug:
+                    msg = "%s:%s" % (msg, _debug_msg,)
+                raise HTTPError(self.errorResponse(
+                    responsecode.FORBIDDEN,
+                    (ischedule_namespace, "verification-failed"),
+                    msg,
+                ))
+
+        result = (yield super(IScheduleScheduler, self).doSchedulingViaPOST(transaction, use_request_headers))
+        returnValue(result)
+
+
+    def loadFromRequestHeaders(self):
+        """
+        Load Originator and Recipient from request headers.
+        """
+        super(IScheduleScheduler, self).loadFromRequestHeaders()
+
+        if self.request.headers.getRawHeaders('x-calendarserver-itip-refreshonly', ("F"))[0] == "T":
+            self.request.doing_attendee_refresh = 1
+
+
+    def preProcessCalendarData(self):
+        """
+        For data coming in from outside we need to normalize the calendar user addresses so that later iTIP
+        processing will match calendar users against those in stored calendar data. Only do that for invites
+        not freebusy.
+        """
+
+        if not self.checkForFreeBusy():
+            # Need to normalize the calendar data and recipient values to keep those in sync,
+            # as we might later try to match them
+            self.calendar.normalizeCalendarUserAddresses(normalizationLookup, self.resource.principalForCalendarUserAddress)
+
+
+    def loadRecipientsFromRequestHeaders(self):
+        """
+        Need to normalize the calendar data and recipient values to keep those in sync,
+        as we might later try to match them
+        """
+        super(IScheduleScheduler, self).loadRecipientsFromRequestHeaders()
+        self.recipients = [normalizeCUAddress(recipient, normalizationLookup, self.resource.principalForCalendarUserAddress) for recipient in self.recipients]
+
+
+    def checkAuthorization(self):
+        # Must have an unauthenticated user
+        if self.resource.currentPrincipal(self.request) != davxml.Principal(davxml.Unauthenticated()):
+            log.err("Authenticated originators not allowed: %s" % (self.originator,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["originator-denied"],
+                "Authentication not allowed",
+            ))
+
+
+    @inlineCallbacks
+    def checkOriginator(self):
+        """
+        Check the validity of the Originator header.
+        """
+
+        # For remote requests we do not allow the originator to be a local user or one within our domain.
+        originatorPrincipal = self.resource.principalForCalendarUserAddress(self.originator)
+        localUser = (yield addressmapping.mapper.isCalendarUserInMyDomain(self.originator))
+        if originatorPrincipal or localUser:
+            if originatorPrincipal.locallyHosted():
+                log.err("Cannot use originator that is on this server: %s" % (self.originator,))
+                raise HTTPError(self.errorResponse(
+                    responsecode.FORBIDDEN,
+                    self.errorElements["originator-denied"],
+                    "Originator cannot be local to server",
+                ))
+            else:
+                self.originator = calendarUserFromPrincipal(self.originator, originatorPrincipal)
+                self._validAlternateServer(originatorPrincipal)
+        else:
+            self.originator = RemoteCalendarUser(self.originator)
+            self._validiScheduleServer()
+
+
+    def _validiScheduleServer(self):
+        """
+        Check the validity of the iSchedule host.
+        """
+
+        # Check for DKIM verification first and treat as valid
+        if self.verified:
+            return
+
+        # We will only accept originator in known domains.
+        servermgr = IScheduleServers()
+        server = servermgr.mapDomain(self.originator.domain)
+        if not server or not server.allow_from:
+            log.err("Originator not on recognized server: %s" % (self.originator,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["originator-denied"],
+                "Originator not recognized by server",
+            ))
+        else:
+            # Get the request IP and map to hostname.
+            clientip = self.request.remoteAddr.host
+
+            # First compare as dotted IP
+            matched = False
+            compare_with = (server.host,) + tuple(server.client_hosts)
+            if clientip in compare_with:
+                matched = True
+            else:
+                # Now do hostname lookup
+                try:
+                    host, aliases, _ignore_ips = socket.gethostbyaddr(clientip)
+                    for host in itertools.chain((host,), aliases):
+                        # Try simple match first
+                        if host in compare_with:
+                            matched = True
+                            break
+
+                        # Try pattern match next
+                        for pattern in compare_with:
+                            try:
+                                if re.match(pattern, host) is not None:
+                                    matched = True
+                                    break
+                            except re.error:
+                                log.debug("Invalid regular expression for ServerToServer white list for server domain %s: %s" % (self.originator.domain, pattern,))
+                        else:
+                            continue
+                        break
+                except socket.herror, e:
+                    log.debug("iSchedule cannot lookup client ip '%s': %s" % (clientip, str(e),))
+
+            if not matched:
+                log.err("Originator not on allowed server: %s" % (self.originator,))
+                raise HTTPError(self.errorResponse(
+                    responsecode.FORBIDDEN,
+                    self.errorElements["originator-denied"],
+                    "Originator not allowed to send to this server",
+                ))
+
+
+    def _validAlternateServer(self, principal):
+        """
+        Check the validity of the partitioned host.
+        """
+
+        # Extract expected host/port. This will be the partitionURI, or if no partitions,
+        # the serverURI
+        expected_uri = principal.partitionURI()
+        if expected_uri is None:
+            expected_uri = principal.serverURI()
+        expected_uri = urlparse.urlparse(expected_uri)
+
+        # Get the request IP and map to hostname.
+        clientip = self.request.remoteAddr.host
+
+        # Check against this server (or any of its partitions). We need this because an external iTIP message
+        # may be addressed to users on different partitions, and the node receiving the iTIP message will need to
+        # forward it to the partition nodes, thus the client ip seen by the partitions will in fact be the initial
+        # receiving node.
+        matched = False
+        if Servers.getThisServer().checkThisIP(clientip):
+            matched = True
+
+        # Checked allowed IPs - if any were defined we only check against them, we do not
+        # go on to check the expected server host ip
+        elif Servers.getThisServer().hasAllowedFromIP():
+            matched = Servers.getThisServer().checkAllowedFromIP(clientip)
+            if not matched:
+                log.error("Invalid iSchedule connection from client: %s" % (clientip,))
+
+        # Next compare as dotted IP
+        elif isIPAddress(expected_uri.hostname):
+            if clientip == expected_uri.hostname:
+                matched = True
+        else:
+            # Now do expected hostname -> IP lookup
+            try:
+                # So now try the lookup of the expected host
+                for ip in getIPsFromHost(expected_uri.hostname):
+                    if ip == clientip:
+                        matched = True
+                        break
+            except socket.herror, e:
+                log.debug("iSchedule cannot lookup client ip '%s': %s" % (clientip, str(e),))
+
+        # Check possible shared secret
+        if matched and not Servers.getThisServer().checkSharedSecret(self.request):
+            log.err("Invalid iSchedule shared secret")
+            matched = False
+
+        if not matched:
+            log.err("Originator not on allowed server: %s" % (self.originator,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["originator-denied"],
+                "Originator not allowed to send to this server",
+            ))
+
+
+    @inlineCallbacks
+    def checkOrganizerAsOriginator(self):
+        """
+        Check the validity of the ORGANIZER value. ORGANIZER must not be local.
+        """
+
+        # Verify that the ORGANIZER's cu address does not map to a valid user
+        organizer = self.calendar.getOrganizer()
+        if organizer:
+            organizerPrincipal = self.resource.principalForCalendarUserAddress(organizer)
+            if organizerPrincipal:
+                if organizerPrincipal.locallyHosted():
+                    log.err("Invalid ORGANIZER in calendar data: %s" % (self.calendar,))
+                    raise HTTPError(self.errorResponse(
+                        responsecode.FORBIDDEN,
+                        self.errorElements["organizer-denied"],
+                        "Organizer is not local to server",
+                    ))
+                else:
+                    # Check that the origin server is the correct partition
+                    self.organizer = calendarUserFromPrincipal(organizer, organizerPrincipal)
+                    self._validAlternateServer(self.organizer.principal)
+            else:
+                localUser = (yield addressmapping.mapper.isCalendarUserInMyDomain(organizer))
+                if localUser:
+                    log.err("Unsupported ORGANIZER in calendar data: %s" % (self.calendar,))
+                    raise HTTPError(self.errorResponse(
+                        responsecode.FORBIDDEN,
+                        self.errorElements["organizer-denied"],
+                        "Organizer not allowed to be originator",
+                    ))
+                else:
+                    self.organizer = RemoteCalendarUser(organizer)
+        else:
+            log.err("ORGANIZER missing in calendar data: %s" % (self.calendar,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["organizer-denied"],
+                "No organizer in calendar data",
+            ))
+
+
+    @inlineCallbacks
+    def checkAttendeeAsOriginator(self):
+        """
+        Check the validity of the ATTENDEE value as this is the originator of the iTIP message.
+        Only local attendees are allowed for message originating from this server.
+        """
+
+        # Attendee cannot be local.
+        attendeePrincipal = self.resource.principalForCalendarUserAddress(self.attendee)
+        if attendeePrincipal:
+            if attendeePrincipal.locallyHosted():
+                log.err("Invalid ATTENDEE in calendar data: %s" % (self.calendar,))
+                raise HTTPError(self.errorResponse(
+                    responsecode.FORBIDDEN,
+                    self.errorElements["attendee-denied"],
+                    "Local attendee cannot send to this server",
+                ))
+            else:
+                self._validAlternateServer(attendeePrincipal)
+        else:
+            localUser = (yield addressmapping.mapper.isCalendarUserInMyDomain(self.attendee))
+            if localUser:
+                log.err("Unknown ATTENDEE in calendar data: %s" % (self.calendar,))
+                raise HTTPError(self.errorResponse(
+                    responsecode.FORBIDDEN,
+                    self.errorElements["attendee-denied"],
+                    "Attendee not allowed to schedule",
+                ))
+
+        # TODO: in this case we should check that the ORGANIZER is the sole recipient.
+
+
+    @inlineCallbacks
+    def securityChecks(self):
+        """
+        Check that the originator has the appropriate rights to send this type of iTIP message.
+        """
+
+        # Prevent spoofing of ORGANIZER with specific METHODs when local
+        if self.calendar.propertyValue("METHOD") in ("PUBLISH", "REQUEST", "ADD", "CANCEL", "DECLINECOUNTER"):
+            yield self.checkOrganizerAsOriginator()
+
+        # Prevent spoofing when doing reply-like METHODs
+        elif self.calendar.propertyValue("METHOD") in ("REPLY", "COUNTER", "REFRESH"):
+            yield self.checkAttendeeAsOriginator()
+
+        else:
+            log.err("Unknown iTIP METHOD for security checks: %s" % (self.calendar.propertyValue("METHOD"),))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["invalid-scheduling-message"],
+                "Unknown iTIP method",
+            ))

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/__init__.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/__init__.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_delivery.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_delivery.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_delivery.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,62 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+import twistedcaldav.test.util
+from twisted.internet.defer import inlineCallbacks
+from twisted.python.modules import getModule
+from twistedcaldav.config import config
+from twistedcaldav.scheduling.ischedule import utils
+from twisted.names import client
+from txdav.caldav.datastore.scheduling.ischedule.delivery import ScheduleViaISchedule
+
+class CalDAV (twistedcaldav.test.util.TestCase):
+    """
+    twistedcaldav.scheduling.caldav tests
+    """
+
+    def tearDown(self):
+        """
+        By setting the resolver to None, it will be recreated next time a name
+        lookup is done.
+        """
+        client.theResolver = None
+        utils.DebugResolver = None
+
+
+    @inlineCallbacks
+    def test_matchCalendarUserAddress(self):
+        """
+        Make sure we do an exact comparison on EmailDomain
+        """
+
+        self.patch(config.Scheduling.iSchedule, "RemoteServers", "")
+
+        # Only mailtos:
+        result = yield ScheduleViaISchedule.matchCalendarUserAddress("http://example.com/principal/user")
+        self.assertFalse(result)
+
+        # Need to setup a fake resolver
+        module = getModule(__name__)
+        dataPath = module.filePath.sibling("data")
+        bindPath = dataPath.child("db.example.com")
+        self.patch(config.Scheduling.iSchedule, "DNSDebug", bindPath.path)
+        utils.DebugResolver = None
+        utils._initResolver()
+
+        result = yield ScheduleViaISchedule.matchCalendarUserAddress("mailto:user at example.com")
+        self.assertTrue(result)
+        result = yield ScheduleViaISchedule.matchCalendarUserAddress("mailto:user at example.org")
+        self.assertFalse(result)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_dkim.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_dkim.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_dkim.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,842 @@
+##
+# Copyright (c) 2012-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from Crypto.PublicKey import RSA
+
+from twext.web2.http_headers import Headers, MimeType
+from twext.web2.stream import MemoryStream
+
+from twisted.internet.defer import inlineCallbacks, succeed
+from twisted.names import client
+from twisted.python.modules import getModule
+
+from twistedcaldav.config import config
+from txdav.caldav.datastore.scheduling.ischedule import utils
+from txdav.caldav.datastore.scheduling.ischedule.dkim import DKIMRequest, DKIMVerifier, \
+    DKIMVerificationError, DKIMUtils, PublicKeyLookup_DNSTXT, \
+    PublicKeyLookup_HTTP_WellKnown, PublicKeyLookup_PrivateExchange
+
+import base64
+import hashlib
+import os
+import time
+import twistedcaldav.test.util
+
+class TestDKIMBase (twistedcaldav.test.util.TestCase):
+    """
+    DKIM support tests
+    """
+
+    class PublicKeyLookup_Testing(PublicKeyLookup_HTTP_WellKnown):
+
+        keys = []
+
+        def _lookupKeys(self):
+            """
+            Do the key lookup using the actual lookup method.
+            """
+            return succeed(self.keys)
+
+
+    def setUp(self):
+        super(TestDKIMBase, self).setUp()
+
+        self.private_keyfile = self.mktemp()
+        f = open(self.private_keyfile, "w")
+        f.write("""-----BEGIN RSA PRIVATE KEY-----
+MIIEogIBAAKCAQEAw7bJxD1k5VSA5AqdfmJ7vj99oKQ4qYtSeJ5HiK6W40dzC++k
+LweUWLzeUErgXwcJlyOC6rqVVPBfSJq4l7yPdVqpWUo6s2jnUsSWOfhpre22yc4B
+K0QY2Euc3R+gT59eM0mtJPtWaQw5BmQ2GrV6f0OUiKi17jEPasKcxf1qZrWU0+Ik
+D2DhUCuRrNb/baUkuIkxoit6M7k7s5X9swT1hE/Eso0gS79FSti1fkDeoPZ296Gu
+5uYWdpaLl03Nr0w65Gbw+2v79AcwOyvbZD6y9xYGLWubic0dUeWuhUipZdmQf8Bd
+t7cZVgjQX/giQQqqLDFhfNFwapUZDhS7TCtujQIDAQABAoIBADfFuzHFHR+NOT3D
+GKaPghvxE+fXZJ5MKbBdypzUxAL4tXxNSkhsrIWtLN1MuSvbYYxEfmZNzYhrB3w1
+Oy1ieq9CqsfbM2c1GdaoVvcmJ1d9Sn2vyv19ZmcdBRKulIycKcgL0t+bEEDXTtjX
+beOmm8XwiD95dH7wVChkVTDGyq+BxtSY6wav9y15zWnBH7+BAeq3OnKaNIQB0iTI
+UA41jWocKYI18/6D5gQTDSoYvKB7saFVGw9IgmmHA/3rYztcHCxUoE15x7wWuwtF
+vzQanEt/QwEEFMibNTjvfIUPoeIeQH7MzcD56AL9u/cs8LNeSbappWE7BneQ0ll3
+CfTsAQECgYEA/eoDkpPMWxuoSrZ1oXLxeImEAB2EHNs4UV9dmcUkhNeYZP0rv7pL
+4jpkNHTRvFeeovy5khXhykb9BUYDuZy6rcELlFxpCKYRw3d+PPWM+wfqmJp+fIN7
+Z4F1Kpznt0F2e+9LXF1Qi5bM2dHy1maxEjaBUIOIoczbjJJDmNN8zR0CgYEAxVJg
+2VCpjaRoJtaZYeserkVgB8SFffBnm/8XQv8uTbKrz104t9nFyezbINmKrQs3cxT3
+1+PiVbLJpPRcik129x4xIlz3zapsMqwXL97Lz92vXm/nELRnV8d+F9SxVzlijRDL
+rvl3X3Vayq2zKb6euBOwOu8UnQO3xJkTtLPtHDECgYAptxuVJkEJqtaQR7+1oZu4
+UOdl2XOOBhoPjFplW/Uu+fiohst8OVAkP7GcyKB4j/CZGGoobP3mbJk/F4yfHvew
+eim72x7Kc/YxJd2QiEr8JwXMwn0LWdKZY7RrJtIO0mtz2xGHgDEubb0EADEkNkTb
+GCdQoft9kZl0U8dVQVGcpQKBgHsvjIre0ps8slDc1HDO6h597Q+sXnJbLTO0Mv9+
+c5fKHXydhBUy/UmsdrixVuPlBr7vrjK3b8t0jHJQo50r80MfNClxxLo+1MFlsiwO
+eUrR6POaBLTnC0U/o7aY8AW2K5JJk/8uepm7l+zEN/+to0Tj9bc1HrdPZOB1eFnt
+oe9hAoGAEwwDhNrmSlZjmZMT8WehCdyS7zQgI8OCLAlu9KTiwFzoSDcnhVbS4nd4
+iblllHCLZ2Q/rHSH3cQor94kxePm+b3KH9ZwAgInMModuSPcScrR5/vsORZCtJEO
+CAXnxZHhrExMGIIa7KV33W5v7Hstl7SnPWKFgCvlBH2QoMTjoUE=
+-----END RSA PRIVATE KEY-----
+""")
+        f.close()
+
+        pkey_data = """MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAw7bJxD1k5VSA5AqdfmJ7
+vj99oKQ4qYtSeJ5HiK6W40dzC++kLweUWLzeUErgXwcJlyOC6rqVVPBfSJq4l7yP
+dVqpWUo6s2jnUsSWOfhpre22yc4BK0QY2Euc3R+gT59eM0mtJPtWaQw5BmQ2GrV6
+f0OUiKi17jEPasKcxf1qZrWU0+IkD2DhUCuRrNb/baUkuIkxoit6M7k7s5X9swT1
+hE/Eso0gS79FSti1fkDeoPZ296Gu5uYWdpaLl03Nr0w65Gbw+2v79AcwOyvbZD6y
+9xYGLWubic0dUeWuhUipZdmQf8Bdt7cZVgjQX/giQQqqLDFhfNFwapUZDhS7TCtu
+jQIDAQAB
+"""
+        self.public_keyfile = self.mktemp()
+        f = open(self.public_keyfile, "w")
+        f.write("""-----BEGIN PUBLIC KEY-----
+%s-----END PUBLIC KEY-----
+""" % (pkey_data,))
+        f.close()
+        self.public_key_data = pkey_data.replace("\n", "")
+
+
+
+class TestDKIMRequest (TestDKIMBase):
+    """
+    L{DKIMRequest} support tests.
+    """
+
+    @inlineCallbacks
+    def test_body_hash(self):
+
+        data = "Hello World!"
+        for algorithm, hash_method in (
+            ("rsa-sha1", hashlib.sha1,),
+            ("rsa-sha256", hashlib.sha256,),
+        ):
+            stream = str(data)
+            headers = Headers()
+            headers.addRawHeader("Originator", "mailto:user01 at example.com")
+            headers.addRawHeader("Recipient", "mailto:user02 at example.com")
+            headers.setHeader("Content-Type", MimeType("text", "calendar", **{"component": "VEVENT", "charset": "utf-8"}))
+            request = DKIMRequest("POST", "/", headers, stream, "example.com", "dkim", "/tmp/key", algorithm, ("Originator", "Recipient", "Content-Type",), True, True, True, 3600)
+            hash = base64.b64encode(hash_method(DKIMUtils.canonicalizeBody(data)).digest())
+            result = (yield request.bodyHash())
+            self.assertEqual(result, hash)
+
+
+    def test_generateSignature(self):
+
+        data = "Hello World!"
+
+        for algorithm, hash_method in (
+            ("rsa-sha1", hashlib.sha1,),
+            ("rsa-sha256", hashlib.sha256,),
+        ):
+            stream = MemoryStream(data)
+            headers = Headers()
+            headers.addRawHeader("Originator", "mailto:user01 at example.com")
+            headers.addRawHeader("Recipient", "mailto:user02 at example.com")
+            headers.setHeader("Content-Type", MimeType("text", "calendar", **{"component": "VEVENT", "charset": "utf-8"}))
+            request = DKIMRequest("POST", "/", headers, stream, "example.com", "dkim", self.private_keyfile, algorithm, ("Originator", "Recipient", "Content-Type",), True, True, True, 3600)
+
+            # Manually create what should be the correct thing to sign
+            bodyhash = base64.b64encode(hash_method(data).digest())
+            sign_this = """originator:mailto:user01 at example.com
+recipient:mailto:user02 at example.com
+content-type:%s
+ischedule-version:1.0
+dkim-signature:v=1; d=example.com; s=dkim; t=%s; x=%s; a=%s; q=dns/txt:http/well-known; c=ischedule-relaxed/simple; h=Originator:Recipient; bh=%s; b=""".replace("\n", "\r\n") % (headers.getRawHeaders("Content-Type")[0], str(int(time.time())), str(int(time.time() + 3600)), algorithm, bodyhash)
+
+            result = request.generateSignature(sign_this)
+
+            key = RSA.importKey(open(self.private_keyfile).read())
+            signature = DKIMUtils.sign(sign_this, key, DKIMUtils.hash_func(algorithm))
+
+            self.assertEqual(result, signature)
+
+
+    @inlineCallbacks
+    def test_signatureHeaders(self):
+
+        data = "Hello World!"
+
+        for algorithm, hash_method in (
+            ("rsa-sha1", hashlib.sha1,),
+            ("rsa-sha256", hashlib.sha256,),
+        ):
+            stream = MemoryStream(data)
+            headers = Headers()
+            headers.addRawHeader("Originator", "mailto:user01 at example.com")
+            headers.addRawHeader("Recipient", "mailto:user02 at example.com")
+            headers.setHeader("Content-Type", MimeType("text", "calendar", **{"component": "VEVENT", "charset": "utf-8"}))
+            request = DKIMRequest("POST", "/", headers, stream, "example.com", "dkim", self.private_keyfile, algorithm, ("Originator", "Recipient", "Content-Type",), True, True, True, 3600)
+            result, _ignore_tags = (yield request.signatureHeaders())
+
+            # Manually create what should be the correct thing to sign
+            bodyhash = base64.b64encode(hash_method(DKIMUtils.canonicalizeBody(data)).digest())
+            sign_this = """originator:mailto:user01 at example.com
+recipient:mailto:user02 at example.com
+content-type:%s
+ischedule-version:1.0
+ischedule-message-id:%s
+dkim-signature:v=1; d=example.com; s=dkim; t=%s; x=%s; a=%s; q=private-exchange:http/well-known:dns/txt; c=ischedule-relaxed/simple; h=Originator:Recipient:Content-Type:iSchedule-Version:iSchedule-Message-ID; bh=%s; b=""".replace("\n", "\r\n") % (headers.getRawHeaders("Content-Type")[0], request.message_id, request.time, request.expire, algorithm, bodyhash)
+
+            self.assertEqual(result, sign_this)
+
+
+    @inlineCallbacks
+    def test_sign(self):
+
+        data = "Hello World!"
+        for algorithm, hash_method in (
+            ("rsa-sha1", hashlib.sha1,),
+            ("rsa-sha256", hashlib.sha256,),
+        ):
+            stream = MemoryStream(data)
+            headers = Headers()
+            headers.addRawHeader("Originator", "mailto:user01 at example.com")
+            headers.addRawHeader("Recipient", "mailto:user02 at example.com")
+            headers.setHeader("Content-Type", MimeType("text", "calendar", **{"component": "VEVENT", "charset": "utf-8"}))
+            request = DKIMRequest("POST", "/", headers, stream, "example.com", "dkim", self.private_keyfile, algorithm, ("Originator", "Recipient", "Content-Type",), True, True, True, 3600)
+            result = (yield request.sign())
+
+            # Manually create what should be the correct thing to sign and make sure signatures match
+            bodyhash = base64.b64encode(hash_method(DKIMUtils.canonicalizeBody(data)).digest())
+            sign_this = """originator:mailto:user01 at example.com
+recipient:mailto:user02 at example.com
+content-type:%s
+ischedule-version:1.0
+ischedule-message-id:%s
+dkim-signature:v=1; d=example.com; s=dkim; t=%s; x=%s; a=%s; q=private-exchange:http/well-known:dns/txt; c=ischedule-relaxed/simple; h=Originator:Recipient:Content-Type:iSchedule-Version:iSchedule-Message-ID; bh=%s; b=""".replace("\n", "\r\n") % (headers.getRawHeaders("Content-Type")[0], request.message_id, request.time, request.expire, algorithm, bodyhash)
+            key = RSA.importKey(open(self.private_keyfile).read())
+            signature = DKIMUtils.sign(sign_this, key, DKIMUtils.hash_func(algorithm))
+
+            self.assertEqual(result, signature)
+
+            # Make sure header is updated in the request
+            updated_header = "v=1; d=example.com; s=dkim; t=%s; x=%s; a=%s; q=private-exchange:http/well-known:dns/txt; c=ischedule-relaxed/simple; h=Originator:Recipient:Content-Type:iSchedule-Version:iSchedule-Message-ID; bh=%s; b=%s" % (request.time, request.expire, algorithm, bodyhash, signature,)
+            self.assertEqual(request.headers.getRawHeaders("DKIM-Signature")[0], updated_header)
+
+            # Try to verify result using public key
+            pubkey = RSA.importKey(open(self.public_keyfile).read())
+            self.assertEqual(DKIMUtils.verify(sign_this, result, pubkey, DKIMUtils.hash_func(algorithm)), None)
+
+
+
+class TestDKIMVerifier (TestDKIMBase):
+    """
+    L{DKIMVerifier} support tests.
+    """
+
+    class StubRequest(object):
+
+        def __init__(self, method, uri, headers, body):
+            self.method = method
+            self.uri = uri
+            self.headers = Headers()
+            for name, value in headers:
+                self.headers.addRawHeader(name, value)
+            self.stream = MemoryStream(body)
+
+
+    def test_valid_dkim_headers(self):
+        """
+        L{DKIMVerifier.processDKIMHeader} correctly validates DKIM-Signature headers.
+        """
+
+        data = (
+            # Bogus
+            ((("DKIM-Signature", "v=1"),), False,),
+
+            # More than one
+            ((
+                ("DKIM-Signature", "v=1; d=example.com; s=dkim; t=1234; a=rsa-sha1; q=dns/txt:http/well-known; c=ischedule-relaxed/simple; h=Originator:Recipient; bh=abc; b=def"),
+                ("DKIM-Signature", "v=1; d=example.com; s=dkim; t=1234; a=rsa-sha256; q=dns/txt:http/well-known; c=ischedule-relaxed/simple; h=Originator:Recipient; bh=abc; b=def"),
+            ), False,),
+
+            # Valid
+            ((("DKIM-Signature", "v=1; d=example.com; s=dkim; t=1234; a=rsa-sha1; q=dns/txt:http/well-known; c=ischedule-relaxed/simple; h=Originator:Recipient; bh=abc; b=def"),), True,),
+            ((("DKIM-Signature", "v=1; d=example.com; s=dkim; t=1234; a=rsa-sha256; q=dns/txt; c=ischedule-relaxed; h=Originator:Recipient; bh=abc; b=def"),), True,),
+            ((("DKIM-Signature", "v=1; d=example.com; s=dkim; t=1234; x=%d; a=rsa-sha256; q=dns/txt; c=ischedule-relaxed; h=Originator:Recipient; bh=abc; b=def" % (int(time.time() + 30),)),), True,),
+
+            # Invalid
+            ((("DKIM-Signature", "v=2; d=example.com; s=dkim; t=1234; a=rsa-sha1; q=dns/txt:http/well-known; c=ischedule-relaxed/simple; h=Originator:Recipient; bh=abc; b=def"),), False,),
+            ((("DKIM-Signature", "v=1; d=example.com; s=dkim; t=1234; a=rsa-sha512; q=dns/txt:http/well-known; c=ischedule-relaxed/simple; h=Originator:Recipient; bh=abc; b=def"),), False,),
+            ((("DKIM-Signature", "v=1; d=example.com; s=dkim; t=1234; a=rsa-sha1; q=dns/txt:http/well-known; c=ischedule-relaxed/relaxed; h=Originator:Recipient; bh=abc; b=def"),), False,),
+            ((("DKIM-Signature", "v=1; d=example.com; t=1234; a=rsa-sha1; q=dns/txt:http/well-known; c=ischedule-relaxed/simple; h=Originator:Recipient; bh=abc; b=def"),), False,),
+            ((("DKIM-Signature", "v=1; d=example.com; s=dkim; t=1234; x=%d; a=rsa-sha256; q=dns/txt; c=ischedule-relaxed; h=Originator:Recipient; bh=abc; b=def" % (int(time.time() - 30),)),), False,),
+            ((("DKIM-Signature", "v=1; d=example.com; s=dkim; t=1234; x=%d; a=rsa-sha256; q=dns/txt; c=ischedule-relaxed; h=Originator:Recipient; bh=abc; b=def" % (int(time.time() - 30),)),), False,),
+            ((("DKIM-Signature", "v=1; d=example.com; s=dkim; t=1234; x=%d; a=rsa-sha256; q=dns/txt; c=ischedule-relaxed; h=Originator:Recipient; bh=abc; b=def" % (int(time.time() - 30),)),), False,),
+            ((("DKIM-Signature", "v=1; d=example.com; s=dkim; t=1234; x=%d; a=rsa-sha256; q=dns/txt; c=ischedule-relaxed; h=Originator:Recipient; bh=abc; b=def" % (int(time.time() - 30),)),), False,),
+        )
+
+        for headers, result in data:
+            request = self.StubRequest("POST", "/", headers, "")
+            verifier = DKIMVerifier(request)
+            if result:
+                verifier.processDKIMHeader()
+            else:
+                self.assertRaises(DKIMVerificationError, verifier.processDKIMHeader)
+
+
+    def test_canonicalize_header(self):
+        """
+        L{DKIMVerifier.canonicalizeHeader} correctly canonicalizes headers.
+        """
+
+        data = (
+            ("Content-Type", " text/calendar  ; charset =  \"utf-8\"  ", "content-type:text/calendar ; charset = \"utf-8\"\r\n"),
+            ("Originator", "  mailto:user01 at example.com  ", "originator:mailto:user01 at example.com\r\n"),
+            ("Recipient", "  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t  ", "recipient:mailto:user02 at example.com,mailto:user03 at example.com\r\n"),
+            ("iSchedule-Version", " 1.0 ", "ischedule-version:1.0\r\n"),
+            (
+                "DKIM-Signature",
+                "  v=1;\t\t d=example.com; s = dkim; t\t=\t1234; a=rsa-sha1; \t\tq=dns/txt:http/well-known\t\t; c=ischedule-relaxed/simple; h=Originator:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=a b  c; b=d ef",
+                "dkim-signature:v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known ; c=ischedule-relaxed/simple; h=Originator:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=a b c; b=",
+            ),
+            (
+                "DKIM-Signature",
+                "  v=1;\t\t d=example.com; s = dkim; t\t=\t1234; a=rsa-sha1; \t\tq=dns/txt:http/well-known\t\t; b= def ; c=ischedule-relaxed/simple; h=Originator:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=a\t bc",
+                "dkim-signature:v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known ; b= ; c=ischedule-relaxed/simple; h=Originator:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=a bc",
+            ),
+        )
+
+        for name, value, result in data:
+            request = self.StubRequest("POST", "/", ((name, value,),), "")
+            verifier = DKIMVerifier(request)
+            if name == "DKIM-Signature":
+                verifier.processDKIMHeader()
+            canonicalized = DKIMUtils.canonicalizeHeader(name, value, verifier.dkim_tags if name == "DKIM-Signature" else None)
+            self.assertEqual(canonicalized, result)
+
+
+    def test_extract_headers(self):
+        """
+        L{DKIMVerifier.extractSignedHeaders} correctly extracts canonicalizes headers.
+        """
+
+        data = (
+            # Count on Recipient
+            ("""Host:example.com
+Content-Type: text/calendar  ; charset =  "utf-8"
+Originator:  mailto:user01 at example.com
+Recipient:  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t
+iSchedule-Version: 1.0
+DKIM-Signature:  v=1;\t\t d=example.com; s = dkim; t\t=\t1234; a=rsa-sha1; \t\tq=dns/txt:http/well-known\t\t; c=ischedule-relaxed/simple; h=Content-Type:Originator:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b=def
+Cache-Control:no-cache
+Connection:close
+""",
+            """content-type:text/calendar ; charset = "utf-8"
+originator:mailto:user01 at example.com
+recipient:mailto:user02 at example.com,mailto:user03 at example.com
+ischedule-version:1.0
+dkim-signature:v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known ; c=ischedule-relaxed/simple; h=Content-Type:Originator:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="""
+            ),
+            # Exact count on Recipient
+            ("""Host:example.com
+Content-Type: text/calendar  ; charset =  "utf-8"
+Originator:  mailto:user01 at example.com
+Recipient:  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t
+Recipient:\t\t  mailto:user04 at example.com
+iSchedule-Version: 1.0
+DKIM-Signature:  v=1;\t\t d=example.com; s = dkim; t\t=\t1234; a=rsa-sha1; \t\tq=dns/txt:http/well-known\t\t; c=ischedule-relaxed/simple; h=Content-Type:Originator:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b=def
+Cache-Control:no-cache
+Connection:close
+""",
+            """content-type:text/calendar ; charset = "utf-8"
+originator:mailto:user01 at example.com
+recipient:mailto:user02 at example.com,mailto:user03 at example.com,mailto:user04 at example.com
+ischedule-version:1.0
+dkim-signature:v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known ; c=ischedule-relaxed/simple; h=Content-Type:Originator:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="""
+            ),
+            # Re-ordered Content-Type
+            ("""Host:example.com
+iSchedule-Version: 1.0
+Originator:  mailto:user01 at example.com
+Recipient:  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t
+DKIM-Signature:  v=1;\t\t d=example.com; s = dkim; t\t=\t1234; a=rsa-sha1; \t\tq=dns/txt:http/well-known\t\t; c=ischedule-relaxed/simple; h=Content-Type:Originator:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b=def
+Content-Type: text/calendar  ; charset =  "utf-8"
+Cache-Control:no-cache
+Connection:close
+""",
+            """content-type:text/calendar ; charset = "utf-8"
+originator:mailto:user01 at example.com
+recipient:mailto:user02 at example.com,mailto:user03 at example.com
+ischedule-version:1.0
+dkim-signature:v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known ; c=ischedule-relaxed/simple; h=Content-Type:Originator:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="""
+            ),
+        )
+
+        for hdrs, result in data:
+            headers = [hdr.split(":", 1) for hdr in hdrs.splitlines()]
+            request = self.StubRequest("POST", "/", headers, "")
+            verifier = DKIMVerifier(request)
+            verifier.processDKIMHeader()
+            extracted = verifier.extractSignedHeaders()
+            self.assertEqual(extracted, result.replace("\n", "\r\n"))
+
+
+    def test_locate_public_key(self):
+        """
+        L{DKIMVerifier.locatePublicKey} correctly finds key matching headers.
+        """
+
+        data = (
+            # Valid
+            ("""Host:example.com
+Content-Type: text/calendar  ; charset =  "utf-8"
+Originator:  mailto:user01 at example.com
+Recipient:  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t
+DKIM-Signature:  v=1;\t\t d=example.com; s = dkim; t\t=\t1234; a=rsa-sha1; \t\tq=dns/txt:http/well-known\t\t; c=ischedule-relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b=def
+Cache-Control:no-cache
+Connection:close
+""",
+            [DKIMUtils.extractTags("v=DKIM1; p=%s" % (self.public_key_data,))],
+            True,
+            ),
+            # Invalid - no method
+            ("""Host:example.com
+Content-Type: text/calendar  ; charset =  "utf-8"
+Originator:  mailto:user01 at example.com
+Recipient:  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t
+DKIM-Signature:  v=1;\t\t d=example.com; s = dkim; t\t=\t1234; a=rsa-sha1; \t\tq=dns/txt\t\t; c=ischedule-relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b=def
+Cache-Control:no-cache
+Connection:close
+""",
+            [DKIMUtils.extractTags("v=DKIM1; p=%s" % (self.public_key_data,))],
+            False,
+            ),
+            # Invalid - wrong algorithm
+            ("""Host:example.com
+Content-Type: text/calendar  ; charset =  "utf-8"
+Originator:  mailto:user01 at example.com
+Recipient:  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t
+DKIM-Signature:  v=1;\t\t d=example.com; s = dkim; t\t=\t1234; a=rsa-sha1; \t\tq=dns/txt:http/well-known\t\t; c=ischedule-relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b=def
+Cache-Control:no-cache
+Connection:close
+""",
+            [DKIMUtils.extractTags("v=DKIM1; h=sha-1; p=%s" % (self.public_key_data,))],
+            False,
+            ),
+        )
+
+        for hdrs, keys, result in data:
+            headers = [hdr.split(":", 1) for hdr in hdrs.splitlines()]
+            request = self.StubRequest("POST", "/", headers, "")
+            TestPublicKeyLookup.PublicKeyLookup_Testing.keys = keys
+            verifier = DKIMVerifier(request, key_lookup=(TestPublicKeyLookup.PublicKeyLookup_Testing,))
+            verifier.processDKIMHeader()
+            pkey = (yield verifier.locatePublicKey())
+            if result:
+                self.assertNotEqual(pkey, None)
+            else:
+                self.assertEqual(pkey, None)
+
+
+    @inlineCallbacks
+    def test_verify(self):
+        """
+        L{DKIMVerifier.verify} correctly finds key matching headers.
+        """
+
+        @inlineCallbacks
+        def _verify(hdrs, body, keys, result, sign_headers=("Originator", "Recipient", "Content-Type",), manipulate_request=None):
+            for algorithm in ("rsa-sha1", "rsa-sha256",):
+                # Create signature
+                stream = MemoryStream(body)
+                headers = Headers()
+                for name, value in [hdr.split(":", 1) for hdr in hdrs.splitlines()]:
+                    headers.addRawHeader(name, value)
+                request = DKIMRequest("POST", "/", headers, stream, "example.com", "dkim", self.private_keyfile, algorithm, sign_headers, True, True, True, 3600)
+                yield request.sign()
+
+                # Possibly munge the request after the signature is done
+                if manipulate_request is not None:
+                    manipulate_request(request)
+
+                # Verify signature
+                TestPublicKeyLookup.PublicKeyLookup_Testing.keys = keys
+                verifier = DKIMVerifier(request, key_lookup=(TestPublicKeyLookup.PublicKeyLookup_Testing,))
+                TestPublicKeyLookup.PublicKeyLookup_Testing({}).flushCache()
+                try:
+                    yield verifier.verify()
+                except Exception, e:
+                    if result:
+                        self.fail("DKIMVerifier:verify failed: %s" % (e,))
+                else:
+                    if not result:
+                        self.fail("DKIMVerifier:verify did not fail")
+
+        # Valid
+        yield _verify(
+            """Host:example.com
+Content-Type: text/calendar  ; charset =  "utf-8"
+Originator:  mailto:user01 at example.com
+Recipient:  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t
+Cache-Control:no-cache
+Connection:close
+""",
+            """BEGIN:DATA
+END:DATA
+""",
+            [DKIMUtils.extractTags("v=DKIM1; p=%s" % (self.public_key_data,))],
+            True,
+        )
+
+        # Invalid - key revoked
+        yield _verify(
+            """Host:example.com
+Content-Type: text/calendar  ; charset =  "utf-8"
+Originator:  mailto:user01 at example.com
+Recipient:  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t
+Cache-Control:no-cache
+Connection:close
+""",
+            """BEGIN:DATA
+END:DATA
+""",
+            [DKIMUtils.extractTags("v=DKIM1; p=")],
+            False,
+        )
+
+        # Invalid - missing header
+        yield _verify(
+            """Host:example.com
+Content-Type: text/calendar  ; charset =  "utf-8"
+Originator:  mailto:user01 at example.com
+Recipient:  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t
+Cache-Control:no-cache
+Connection:close
+""",
+            """BEGIN:DATA
+END:DATA
+""",
+            [DKIMUtils.extractTags("v=DKIM1; p=%s" % (self.public_key_data,))],
+            False,
+            manipulate_request=lambda request: request.headers.removeHeader("Originator")
+        )
+
+        # Invalid - changed header
+        yield _verify(
+            """Host:example.com
+Content-Type: text/calendar  ; charset =  "utf-8"
+Originator:  mailto:user01 at example.com
+Recipient:  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t
+Cache-Control:no-cache
+Connection:close
+""",
+            """BEGIN:DATA
+END:DATA
+""",
+            [DKIMUtils.extractTags("v=DKIM1; p=%s" % (self.public_key_data,))],
+            False,
+            manipulate_request=lambda request: request.headers.setRawHeaders("Originator", ("mailto:user04 at example.com",))
+        )
+
+        # Invalid - changed body
+        yield _verify(
+            """Host:example.com
+Content-Type: text/calendar  ; charset =  "utf-8"
+Originator:  mailto:user01 at example.com
+Recipient:  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t
+Cache-Control:no-cache
+Connection:close
+""",
+            """BEGIN:DATA
+END:DATA
+""",
+            [DKIMUtils.extractTags("v=DKIM1; p=%s" % (self.public_key_data,))],
+            False,
+            manipulate_request=lambda request: setattr(request, "stream", MemoryStream("BEGIN:DATA\n")),
+        )
+
+        # Invalid - extra header
+        yield _verify(
+            """Host:example.com
+Content-Type: text/calendar  ; charset =  "utf-8"
+Originator:  mailto:user01 at example.com
+Recipient:  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t
+Cache-Control:no-cache
+Connection:close
+""",
+            """BEGIN:DATA
+END:DATA
+""",
+            [DKIMUtils.extractTags("v=DKIM1; p=%s" % (self.public_key_data,))],
+            False,
+            manipulate_request=lambda request: request.headers.getRawHeaders("Recipient").insert(0, "mailto:user04 at example.com"),
+        )
+
+        # Valid - header
+        yield _verify(
+            """Host:example.com
+Content-Type: text/calendar  ; charset =  "utf-8"
+Originator:  mailto:user01 at example.com
+Recipient:  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t
+Cache-Control:no-cache
+Connection:close
+""",
+            """BEGIN:DATA
+END:DATA
+""",
+            [DKIMUtils.extractTags("v=DKIM1; p=%s" % (self.public_key_data,))],
+            True,
+            sign_headers=("Originator", "Recipient", "Content-Type",),
+        )
+
+        # Invalid - over sign header extra header
+        yield _verify(
+            """Host:example.com
+Content-Type: text/calendar  ; charset =  "utf-8"
+Originator:  mailto:user01 at example.com
+Recipient:  mailto:user02 at example.com  ,\t mailto:user03 at example.com\t\t
+Cache-Control:no-cache
+Connection:close
+""",
+            """BEGIN:DATA
+END:DATA
+""",
+            [DKIMUtils.extractTags("v=DKIM1; p=%s" % (self.public_key_data,))],
+            False,
+            sign_headers=("Originator", "Recipient", "Content-Type",),
+            manipulate_request=lambda request: request.headers.addRawHeader("Recipient", ("mailto:user04 at example.com",))
+        )
+
+
+
+class TestPublicKeyLookup (TestDKIMBase):
+    """
+    L{PublicKeyLookup} support tests.
+    """
+
+    def tearDown(self):
+        """
+        By setting the resolver to None, it will be recreated next time a name
+        lookup is done.
+        """
+        client.theResolver = None
+        utils.DebugResolver = None
+
+
+    def test_selector_key(self):
+
+        for lookup, d, result in (
+            (PublicKeyLookup_DNSTXT, "example.com", "dkim._domainkey.example.com"),
+            (PublicKeyLookup_DNSTXT, "calendar.example.com", "dkim._domainkey.calendar.example.com"),
+            (PublicKeyLookup_HTTP_WellKnown, "example.com", "https://example.com/.well-known/domainkey/example.com/dkim"),
+            (PublicKeyLookup_HTTP_WellKnown, "calendar.example.com", "https://example.com/.well-known/domainkey/calendar.example.com/dkim"),
+            (PublicKeyLookup_PrivateExchange, "example.com", "example.com#dkim"),
+            (PublicKeyLookup_PrivateExchange, "calendar.example.com", "calendar.example.com#dkim"),
+        ):
+            dkim = "v=1; d=%s; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b=" % (d,)
+            tester = lookup(DKIMUtils.extractTags(dkim))
+            self.assertEqual(tester._getSelectorKey(), result)
+
+
+    @inlineCallbacks
+    def test_get_key(self):
+
+        # Valid
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = TestPublicKeyLookup.PublicKeyLookup_Testing(DKIMUtils.extractTags(dkim))
+        lookup.flushCache()
+        lookup.keys = [DKIMUtils.extractTags("v=DKIM1; p=%s" % (self.public_key_data,))]
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is not None)
+
+        # Valid with more tags
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = TestPublicKeyLookup.PublicKeyLookup_Testing(DKIMUtils.extractTags(dkim))
+        lookup.flushCache()
+        lookup.keys = [DKIMUtils.extractTags("v=DKIM1; k = rsa ; h=  sha1 : sha256  ; s=ischedule ; p=%s" % (self.public_key_data,))]
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is not None)
+
+        # Invalid - key type
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = TestPublicKeyLookup.PublicKeyLookup_Testing(DKIMUtils.extractTags(dkim))
+        lookup.flushCache()
+        lookup.keys = [DKIMUtils.extractTags("v=DKIM1; k=dsa ; p=%s" % (self.public_key_data,))]
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is None)
+
+        # Invalid - hash
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = TestPublicKeyLookup.PublicKeyLookup_Testing(DKIMUtils.extractTags(dkim))
+        lookup.flushCache()
+        lookup.keys = [DKIMUtils.extractTags("v=DKIM1; k=rsa ; h=sha512 ; p=%s" % (self.public_key_data,))]
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is None)
+
+        # Invalid - service
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = TestPublicKeyLookup.PublicKeyLookup_Testing(DKIMUtils.extractTags(dkim))
+        lookup.flushCache()
+        lookup.keys = [DKIMUtils.extractTags("v=DKIM1; k=rsa ; s=email ; p=%s" % (self.public_key_data,))]
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is None)
+
+        # Invalid - revoked
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = TestPublicKeyLookup.PublicKeyLookup_Testing(DKIMUtils.extractTags(dkim))
+        lookup.flushCache()
+        lookup.keys = [DKIMUtils.extractTags("v=DKIM1; k=rsa ; s=email ; p=")]
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is None)
+
+        # Multiple valid
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = TestPublicKeyLookup.PublicKeyLookup_Testing(DKIMUtils.extractTags(dkim))
+        lookup.flushCache()
+        lookup.keys = [
+            DKIMUtils.extractTags("v=DKIM1; p=%s" % (self.public_key_data,)),
+            DKIMUtils.extractTags("v=DKIM1; k = rsa ; h=  sha1 : sha256  ; s=ischedule ; p=%s" % (self.public_key_data,)),
+            DKIMUtils.extractTags("v=DKIM1; k = rsa ; h=  sha1 : sha256  ; s=* ; p=%s" % (self.public_key_data,)),
+        ]
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is not None)
+
+        # Multiple - some valid, some invalid
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = TestPublicKeyLookup.PublicKeyLookup_Testing(DKIMUtils.extractTags(dkim))
+        lookup.flushCache()
+        lookup.keys = [
+            DKIMUtils.extractTags("v=DKIM1; k=rsa ; s=email ; p="),
+            DKIMUtils.extractTags("v=DKIM1; p=%s" % (self.public_key_data,)),
+            DKIMUtils.extractTags("v=DKIM1; k = rsa ; h=  sha1 : sha256  ; s=ischedule ; p=%s" % (self.public_key_data,)),
+            DKIMUtils.extractTags("v=DKIM1; k = rsa ; h=  sha1 : sha256  ; s=* ; p=%s" % (self.public_key_data,)),
+        ]
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is not None)
+
+        # Multiple - invalid
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = TestPublicKeyLookup.PublicKeyLookup_Testing(DKIMUtils.extractTags(dkim))
+        lookup.flushCache()
+        lookup.keys = [
+            DKIMUtils.extractTags("v=DKIM1; k=rsa ; s=email ; p="),
+            DKIMUtils.extractTags("v=DKIM1; k=rsa ; s=email ; p="),
+        ]
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is None)
+
+
+    @inlineCallbacks
+    def test_cached_key(self):
+
+        # Create cache entry
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = TestPublicKeyLookup.PublicKeyLookup_Testing(DKIMUtils.extractTags(dkim))
+        lookup.flushCache()
+        lookup.keys = [DKIMUtils.extractTags("v=DKIM1; p=%s" % (self.public_key_data,))]
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is not None)
+
+        # Cache valid
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = TestPublicKeyLookup.PublicKeyLookup_Testing(DKIMUtils.extractTags(dkim))
+        lookup.keys = []
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is not None)
+
+        # Cache invalid
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = TestPublicKeyLookup.PublicKeyLookup_Testing(DKIMUtils.extractTags(dkim))
+        lookup.flushCache()
+        lookup.keys = []
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is None)
+
+
+    @inlineCallbacks
+    def test_TXT_key(self):
+
+        # Need to setup a fake resolver
+        module = getModule(__name__)
+        dataPath = module.filePath.sibling("data")
+        bindPath = dataPath.child("db.example.com")
+        self.patch(config.Scheduling.iSchedule, "DNSDebug", bindPath.path)
+        utils.DebugResolver = None
+        utils._initResolver()
+
+        for d, s, result in (
+            ("example.com", "_ischedule", True),
+            ("example.com", "_revoked", False),
+            ("example.com", "dkim", False),
+            ("calendar.example.com", "_ischedule", False),
+            ("example.org", "_ischedule", False),
+        ):
+            dkim = "v=1; d=%s; s = %s; t = 1234; a=rsa-sha1; q=dns/txt ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b=" % (d, s,)
+            tester = PublicKeyLookup_DNSTXT(DKIMUtils.extractTags(dkim))
+            pkey = yield tester.getPublicKey(False)
+            self.assertEqual(pkey is not None, result)
+
+
+    @inlineCallbacks
+    def test_HTTP_URI_key(self):
+
+        # Need to setup a fake resolver
+        module = getModule(__name__)
+        dataPath = module.filePath.sibling("data")
+        bindPath = dataPath.child("db.example.com")
+        self.patch(config.Scheduling.iSchedule, "DNSDebug", bindPath.path)
+        utils.DebugResolver = None
+        utils._initResolver()
+
+        for d, s, result in (
+            ("example.com", "_ischedule", "https://key.example.com:8443/.well-known/domainkey/example.com/_ischedule"),
+            ("www.example.com", "_ischedule", "http://key.example.com/.well-known/domainkey/www.example.com/_ischedule"),
+            ("example.org", "_ischedule", "https://example.org/.well-known/domainkey/example.org/_ischedule"),
+        ):
+            dkim = "v=1; d=%s; s = %s; t = 1234; a=rsa-sha1; q=http/well-known ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b=" % (d, s,)
+            tester = PublicKeyLookup_HTTP_WellKnown(DKIMUtils.extractTags(dkim))
+            uri = (yield tester._getURI())
+            self.assertEqual(uri, result)
+
+
+    @inlineCallbacks
+    def test_private_exchange(self):
+
+        keydir = self.mktemp()
+        PublicKeyLookup_PrivateExchange.directory = keydir
+        os.mkdir(keydir)
+        keyfile = os.path.join(keydir, "example.com#dkim")
+        with open(keyfile, "w") as f:
+            f.write("""v=DKIM1; p=%s
+""" % (self.public_key_data,))
+
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = PublicKeyLookup_PrivateExchange(DKIMUtils.extractTags(dkim))
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is not None)
+
+        dkim = "v=1; d=example.com; s = dkim2; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = PublicKeyLookup_PrivateExchange(DKIMUtils.extractTags(dkim))
+        lookup.flushCache()
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is None)
+
+        with open(keyfile, "w") as f:
+            f.write("""v=DKIM1; s=email; p=%s
+""" % (self.public_key_data,))
+
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = PublicKeyLookup_PrivateExchange(DKIMUtils.extractTags(dkim))
+        lookup.flushCache()
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is None)
+
+        with open(keyfile, "w") as f:
+            f.write("""v=DKIM1; s=email; p=%s
+v=DKIM1; s=ischedule; p=%s
+""" % (self.public_key_data, self.public_key_data,))
+
+        dkim = "v=1; d=example.com; s = dkim; t = 1234; a=rsa-sha1; q=dns/txt:http/well-known:private-exchange ; http=UE9TVDov; c=relaxed/simple; h=Content-Type:Originator:Recipient:Recipient:iSchedule-Version:iSchedule-Message-ID; bh=abc; b="
+        lookup = PublicKeyLookup_PrivateExchange(DKIMUtils.extractTags(dkim))
+        lookup.flushCache()
+        pubkey = (yield lookup.getPublicKey())
+        self.assertTrue(pubkey is not None)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_localservers.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_localservers.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_localservers.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,184 @@
+##
+# Copyright (c) 2009-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.web2.test.test_server import SimpleRequest
+from twistedcaldav.config import config
+from txdav.caldav.datastore.scheduling.ischedule.localservers import Servers, SERVER_SECRET_HEADER
+from twistedcaldav.test.util import TestCase
+import StringIO as StringIO
+
+class ServerTests(TestCase):
+
+    data1 = """<?xml version="1.0" encoding="utf-8"?>
+<servers>
+  <server>
+    <id>00001</id>
+    <uri>http://caldav1.example.com:8008</uri>
+    <allowed-from>127.0.0.1</allowed-from>
+    <shared-secret>foobar</shared-secret>
+  </server>
+  <server>
+    <id>00002</id>
+    <uri>https://caldav2.example.com:8843</uri>
+    <partitions>
+        <partition>
+            <id>A</id>
+            <uri>https://machine1.example.com:8443</uri>
+        </partition>
+        <partition>
+            <id>B</id>
+            <uri>https://machine2.example.com:8443</uri>
+        </partition>
+    </partitions>
+  </server>
+</servers>
+"""
+
+    data2 = """<?xml version="1.0" encoding="utf-8"?>
+<servers>
+  <server>
+    <id>00001</id>
+    <uri>http://caldav1.example.com:8008</uri>
+    <allowed-from>localhost</allowed-from>
+    <shared-secret>foobar</shared-secret>
+  </server>
+  <server>
+    <id>00002</id>
+    <uri>https://caldav2.example.com:8843</uri>
+    <partitions>
+        <partition>
+            <id>A</id>
+            <uri>https://machine1.example.com:8443</uri>
+        </partition>
+        <partition>
+            <id>B</id>
+            <uri>https://machine2.example.com:8443</uri>
+        </partition>
+    </partitions>
+  </server>
+</servers>
+"""
+
+    def _setupServers(self, data=data1):
+        self.patch(config, "ServerHostName", "caldav1.example.com")
+        self.patch(config, "HTTPPort", 8008)
+
+        xmlFile = StringIO.StringIO(data)
+        servers = Servers
+        servers.load(xmlFile, ignoreIPLookupFailures=True)
+
+        return servers
+
+
+    def test_read_ok(self):
+
+        servers = self._setupServers()
+
+        self.assertTrue(servers.getServerById("00001") is not None)
+        self.assertTrue(servers.getServerById("00002") is not None)
+
+        self.assertEqual(servers.getServerById("00001").uri, "http://caldav1.example.com:8008")
+        self.assertEqual(servers.getServerById("00002").uri, "https://caldav2.example.com:8843")
+
+        self.assertEqual(servers.getServerById("00001").allowed_from_ips, set(("127.0.0.1",)))
+        self.assertEqual(servers.getServerById("00002").allowed_from_ips, set())
+
+        self.assertEqual(servers.getServerById("00001").shared_secret, "foobar")
+        self.assertEqual(servers.getServerById("00002").shared_secret, None)
+
+        self.assertEqual(len(servers.getServerById("00001").partitions), 0)
+        self.assertEqual(len(servers.getServerById("00002").partitions), 2)
+
+        self.assertEqual(servers.getServerById("00002").getPartitionURIForId("A"), "https://machine1.example.com:8443")
+        self.assertEqual(servers.getServerById("00002").getPartitionURIForId("B"), "https://machine2.example.com:8443")
+
+
+    def test_this_server(self):
+
+        servers = self._setupServers()
+
+        self.assertTrue(servers.getServerById("00001").thisServer)
+        self.assertFalse(servers.getServerById("00002").thisServer)
+
+        self.patch(config, "ServerHostName", "caldav2.example.com")
+        self.patch(config, "SSLPort", 8443)
+        self.patch(config, "BindSSLPorts", [8843])
+
+        xmlFile = StringIO.StringIO(ServerTests.data1)
+        servers = Servers
+        servers.load(xmlFile, ignoreIPLookupFailures=True)
+
+        self.assertFalse(servers.getServerById("00001").thisServer)
+        self.assertTrue(servers.getServerById("00002").thisServer)
+
+
+    def test_check_is_partitioned(self):
+
+        servers = self._setupServers()
+
+        self.assertFalse(servers.getServerById("00001").isPartitioned())
+        self.assertTrue(servers.getServerById("00002").isPartitioned())
+
+
+    def test_check_this_ip(self):
+
+        servers = self._setupServers()
+        servers.getServerById("00001").ips = set(("127.0.0.2",))
+        servers.getServerById("00002").ips = set(("127.0.0.3",))
+
+        self.assertTrue(servers.getServerById("00001").checkThisIP("127.0.0.2"))
+        self.assertFalse(servers.getServerById("00001").checkThisIP("127.0.0.3"))
+
+
+    def test_check_allowed_from(self):
+
+        for servers in (self._setupServers(), self._setupServers(data=self.data2),):
+            self.assertTrue(servers.getServerById("00001").hasAllowedFromIP())
+            self.assertFalse(servers.getServerById("00002").hasAllowedFromIP())
+
+            self.assertTrue(servers.getServerById("00001").checkAllowedFromIP("127.0.0.1"))
+            self.assertFalse(servers.getServerById("00001").checkAllowedFromIP("127.0.0.2"))
+            self.assertFalse(servers.getServerById("00001").checkAllowedFromIP("127.0.0.3"))
+            self.assertFalse(servers.getServerById("00002").checkAllowedFromIP("127.0.0.1"))
+            self.assertFalse(servers.getServerById("00002").checkAllowedFromIP("127.0.0.2"))
+            self.assertFalse(servers.getServerById("00002").checkAllowedFromIP("127.0.0.3"))
+
+
+    def test_check_shared_secret(self):
+
+        servers = self._setupServers()
+
+        request = SimpleRequest(None, "POST", "/ischedule")
+        request.headers.addRawHeader(SERVER_SECRET_HEADER, "foobar")
+        self.assertTrue(servers.getServerById("00001").checkSharedSecret(request))
+
+        request = SimpleRequest(None, "POST", "/ischedule")
+        request.headers.addRawHeader(SERVER_SECRET_HEADER, "foobar1")
+        self.assertFalse(servers.getServerById("00001").checkSharedSecret(request))
+
+        request = SimpleRequest(None, "POST", "/ischedule")
+        self.assertFalse(servers.getServerById("00001").checkSharedSecret(request))
+
+        request = SimpleRequest(None, "POST", "/ischedule")
+        request.headers.addRawHeader(SERVER_SECRET_HEADER, "foobar")
+        self.assertFalse(servers.getServerById("00002").checkSharedSecret(request))
+
+        request = SimpleRequest(None, "POST", "/ischedule")
+        request.headers.addRawHeader(SERVER_SECRET_HEADER, "foobar1")
+        self.assertFalse(servers.getServerById("00002").checkSharedSecret(request))
+
+        request = SimpleRequest(None, "POST", "/ischedule")
+        self.assertTrue(servers.getServerById("00002").checkSharedSecret(request))

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_remoteservers.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_remoteservers.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_remoteservers.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,49 @@
+##
+# Copyright (c) 2012-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twisted.python.filepath import FilePath
+from txdav.caldav.datastore.scheduling.ischedule.remoteservers import IScheduleServersParser
+import twistedcaldav.test.util
+
+class Test_IScheduleServersParser(twistedcaldav.test.util.TestCase):
+    """
+    Test L{IScheduleServersParser} implementation.
+    """
+
+    def test_readXML(self):
+
+        fp = FilePath(self.mktemp())
+        fp.open("w").write(
+"""<?xml version="1.0" encoding="utf-8"?>
+<!DOCTYPE servers SYSTEM "servertoserver.dtd">
+<servers>
+  <server>
+    <uri>https://localhost:8543/inbox</uri>
+    <allow-requests-from/>
+    <allow-requests-to/>
+    <domains>
+        <domain>example.org</domain>
+    </domains>
+    <hosts>
+        <host>127.0.0.1</host>
+    </hosts>
+  </server>
+</servers>
+"""
+)
+
+        parser = IScheduleServersParser(fp)
+        self.assertEqual(len(parser.servers), 1)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_resource.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_resource.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_resource.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,75 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.web2 import http_headers, responsecode
+from twext.web2.test.test_server import SimpleRequest
+from twisted.internet.defer import inlineCallbacks
+from twistedcaldav.config import config
+from twistedcaldav.memcachelock import MemcacheLock
+from txdav.caldav.datastore.scheduling.ischedule.resource import IScheduleInboxResource
+from twistedcaldav.test.util import TestCase
+
+class iSchedulePOST (TestCase):
+
+    def setUp(self):
+        super(iSchedulePOST, self).setUp()
+        self.createStockDirectoryService()
+        self.setupCalendars()
+        self.site.resource.putChild(
+            "ischedule", IScheduleInboxResource(self.site.resource,
+                                                self.createDataStore()))
+
+
+    @inlineCallbacks
+    def test_deadlock(self):
+        """
+        Make calendar
+        """
+
+        request = SimpleRequest(
+            self.site,
+            "POST",
+            "/ischedule",
+            headers=http_headers.Headers(rawHeaders={
+                "Originator": ("mailto:wsanchez at example.com",),
+                "Recipient": ("mailto:cdaboo at example.com",),
+            }),
+            content="""BEGIN:VCALENDAR
+CALSCALE:GREGORIAN
+PRODID:-//Example Inc.//Example Calendar//EN
+VERSION:2.0
+BEGIN:VEVENT
+DTSTAMP:20051222T205953Z
+CREATED:20060101T150000Z
+DTSTART:20060101T100000Z
+DURATION:PT1H
+SUMMARY:event 1
+UID:deadlocked
+ORGANIZER:mailto:wsanchez at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:wsanchez at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:cdaboo at example.com
+END:VEVENT
+END:VCALENDAR
+""".replace("\n", "\r\n")
+        )
+
+        # Lock the UID here to force a deadlock - but adjust the timeout so the test does not wait too long
+        self.patch(config.Scheduling.Options, "UIDLockTimeoutSeconds", 1)
+        lock = MemcacheLock("ImplicitUIDLock", "deadlocked", timeout=60, expire_time=60)
+        yield lock.acquire()
+
+        response = (yield self.send(request))
+        self.assertEqual(response.code, responsecode.CONFLICT)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_utils.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_utils.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/test/test_utils.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,116 @@
+##
+# Copyright (c) 2012-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twisted.internet.defer import inlineCallbacks
+from twistedcaldav.config import config
+from twistedcaldav.scheduling.ischedule import utils
+from twistedcaldav.test.util import TestCase
+from twisted.python.modules import getModule
+from twisted.names.authority import BindAuthority
+from twisted.names import client
+from twisted.names.test.test_client import FakeResolver
+
+class LookupService (TestCase):
+
+
+    def setUp(self):
+        """
+        Replace the resolver with a FakeResolver
+        """
+        client.theResolver = FakeResolver()
+
+
+    def tearDown(self):
+        """
+        By setting the resolver to None, it will be recreated next time a name
+        lookup is done.
+        """
+        client.theResolver = None
+        utils.DebugResolver = None
+
+
+    def test_initResolver(self):
+        """
+        Test L{lookupServerViaSRV} with a local Bind find
+        """
+
+        # Default resolver
+        utils.DebugResolver = None
+        utils._initResolver()
+        self.assertNotEqual(utils.DebugResolver, None)
+        self.assertFalse(isinstance(utils.DebugResolver, BindAuthority))
+
+        # Patch config for Bind resolver
+        for zonefile in ("db.example.com", "db.two.zones",):
+            module = getModule(__name__)
+            dataPath = module.filePath.sibling("data")
+            bindPath = dataPath.child(zonefile)
+            self.patch(config.Scheduling.iSchedule, "DNSDebug", bindPath.path)
+            utils.DebugResolver = None
+            utils._initResolver()
+            self.assertNotEqual(utils.DebugResolver, None)
+            self.assertTrue(isinstance(utils.DebugResolver, BindAuthority))
+
+
+    @inlineCallbacks
+    def test_lookupServerViaSRV(self):
+        """
+        Test L{lookupServerViaSRV} with a local Bind find
+        """
+
+        # Patch config
+        for zonefile, checks in (
+            ("db.example.com", (("example.com", "example.com", 8443,),),),
+            ("db.two.zones", (
+                ("example.com", "example.com", 8443,),
+                ("example.org", "example.org", 8543,),
+            ),),
+        ):
+            module = getModule(__name__)
+            dataPath = module.filePath.sibling("data")
+            bindPath = dataPath.child(zonefile)
+            self.patch(config.Scheduling.iSchedule, "DNSDebug", bindPath.path)
+            utils.DebugResolver = None
+
+            for domain, result_host, result_port in checks:
+                host, port = (yield utils.lookupServerViaSRV(domain))
+                self.assertEqual(host, result_host)
+                self.assertEqual(port, result_port)
+
+
+    @inlineCallbacks
+    def test_lookupDataViaTXT(self):
+        """
+        Test L{lookupDataViaTXT} with a local Bind find
+        """
+
+        # Patch config
+        for zonefile, checks in (
+            ("db.example.com", (("example.com", "_ischedule._domainkey", "v=DKIM1; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDjUfDqd8ICAL0dyq2KdjKN6LS8O/Y4yMxOxgATqtSIMi7baKXEs1w5Wj9efOC2nU+aqyhP2/J6AzfFJfSB+GV5gcIT+LAC4btJKPGjPUyXcQFJV4a73y0jIgCTBzWxdaP6qD9P9rzYlvMPcdrrKiKoAOtI3JZqAAdZudOmGlc4QQIDAQAB"),),),
+            ("db.two.zones", (
+                ("example.com", "_ischedule._domainkey", "v=DKIM1; p="),
+                ("example.org", "_ischedule2._domainkey", "v=DKIM1; s=ischedule; p="),
+            )),
+        ):
+            module = getModule(__name__)
+            dataPath = module.filePath.sibling("data")
+            bindPath = dataPath.child(zonefile)
+            self.patch(config.Scheduling.iSchedule, "DNSDebug", bindPath.path)
+            utils.DebugResolver = None
+
+            for domain, prefix, result in checks:
+                texts = (yield utils.lookupDataViaTXT(domain, prefix))
+                self.assertEqual(texts, [result])

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/utils.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/utils.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/utils.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,182 @@
+##
+# Copyright (c) 2012-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.python.log import Logger
+
+from twisted.internet.defer import inlineCallbacks, returnValue
+from twisted.names import dns
+from twisted.names.authority import BindAuthority
+from twisted.names.client import getResolver
+from twisted.names.error import DomainError, AuthoritativeDomainError
+
+from twistedcaldav.config import config
+
+import socket
+
+log = Logger()
+
+DebugResolver = None
+
+
+def getIPsFromHost(host):
+    """
+    Map a hostname to an IPv4 or IPv6 address.
+
+    @param host: the hostname
+    @type host: C{str}
+
+    @return: a C{set} of IPs
+    """
+    ips = set()
+    for family in (socket.AF_INET, socket.AF_INET6):
+        results = socket.getaddrinfo(host, None, family, socket.SOCK_STREAM)
+        for _ignore_family, _ignore_socktype, _ignore_proto, _ignore_canonname, sockaddr in results:
+            ips.add(sockaddr[0])
+
+    return ips
+
+
+
+ at inlineCallbacks
+def lookupServerViaSRV(domain, service="_ischedules"):
+
+    _initResolver()
+
+    lookup = "%s._tcp.%s" % (service, domain,)
+    log.debug("DNS SRV: lookup: %s" % (lookup,))
+    try:
+        answers = (yield DebugResolver.lookupService(lookup))[0]
+    except (DomainError, AuthoritativeDomainError), e:
+        log.debug("DNS SRV: lookup failed: %s" % (e,))
+        returnValue(None)
+
+    if len(answers) == 1 and answers[0].type == dns.SRV \
+                         and answers[0].payload \
+                         and answers[0].payload.target == dns.Name('.'):
+        # decidedly not available
+        log.debug("DNS SRV: disabled: %s" % (lookup,))
+        returnValue(None)
+
+    servers = []
+    for a in answers:
+
+        if a.type != dns.SRV or not a.payload:
+            continue
+
+        servers.append((a.payload.priority, a.payload.weight, str(a.payload.target), a.payload.port))
+
+    log.debug("DNS SRV: lookup results: %s\n%s" % (lookup, servers,))
+
+
+    def _serverCmp(a, b):
+        if a[0] != b[0]:
+            return cmp(a[0], b[0])
+        else:
+            return cmp(a[1], b[1])
+
+    servers.sort(_serverCmp)
+    minPriority = servers[0][0]
+
+    weightIndex = zip(xrange(len(servers)), [x[1] for x in servers if x[0] == minPriority])
+    weightSum = reduce(lambda x, y: (None, x[1] + y[1]), weightIndex, (None, 0))[1]
+
+    for index, weight in weightIndex:
+        weightSum -= weight
+        if weightSum <= 0:
+            chosen = servers[index]
+            _ignore_p, _ignore_w, host, port = chosen
+            host = host.rstrip(".")
+            break
+    else:
+        log.debug("DNS SRV: unable to determine best record to use: %s" % (lookup,))
+        returnValue(None)
+
+    log.debug("DNS SRV: lookup chosen service: %s %s %s" % (lookup, host, port,))
+    returnValue((host, port,))
+
+
+
+ at inlineCallbacks
+def lookupDataViaTXT(domain, prefix=""):
+
+    _initResolver()
+
+    lookup = "%s.%s" % (prefix, domain,) if prefix else domain
+    log.debug("DNS TXT: lookup: %s" % (lookup,))
+    try:
+        answers = (yield DebugResolver.lookupText(lookup))[0]
+    except (DomainError, AuthoritativeDomainError), e:
+        log.debug("DNS TXT: lookup failed: %s" % (e,))
+        answers = ()
+
+    results = []
+    for a in answers:
+
+        if a.type != dns.TXT or not a.payload:
+            continue
+
+        results.append("".join(a.payload.data))
+
+    log.debug("DNS TXT: lookup results: %s\n%s" % (lookup, "\n".join(results),))
+    returnValue(results)
+
+
+
+class FakeBindAuthority(BindAuthority):
+
+    @inlineCallbacks
+    def _lookup(self, name, cls, type, timeout=None):
+        log.debug("DNS FakeBindAuthority: lookup: %s %s %s" % (name, cls, type,))
+        result = yield BindAuthority._lookup(self, name, cls, type, timeout)
+        log.debug("DNS FakeBindAuthority: lookup results: %s %s %s\n%s" % (name, cls, type, result[0]))
+        returnValue(result)
+
+
+    def stripComments(self, lines):
+        """
+        Work around a bug in the base implementation that causes parsing of TXT RRs with
+        a ; in the RDATA to fail because the ; is treated as the start of a comment. Here
+        we simply ignore all comments.
+        """
+        return [
+            (a.find(';') == -1 or "TXT" in a) and a or a[:a.find(';')] for a in [
+                b.strip() for b in lines
+            ]
+        ]
+
+
+    def parseLines(self, lines):
+        """
+        Work around a bug in the base implementation that causes parsing of TXT RRs with
+        spaces in the RDATA to be broken into multiple fragments and for quotes around the
+        data to not be removed.
+        """
+        for line in lines:
+            if line[3] == "TXT":
+                line[4] = " ".join(line[4:])[1:-1]
+                del line[5:]
+
+        BindAuthority.parseLines(self, lines)
+
+
+
+def _initResolver():
+    global DebugResolver
+    if DebugResolver is None:
+        if config.Scheduling.iSchedule.DNSDebug:
+            DebugResolver = FakeBindAuthority(config.Scheduling.iSchedule.DNSDebug)
+        else:
+            DebugResolver = getResolver()

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/xml.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/xml.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/ischedule/xml.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,289 @@
+##
+# Copyright (c) 2011-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+This module provides XML definitions for use with Timezone Standard Service.
+"""
+
+from twistedcaldav.config import config
+from twistedcaldav.ical import Component as iComponent
+from txdav.xml.element import PCDATAElement, WebDAVElement, WebDAVEmptyElement, WebDAVTextElement
+from txdav.xml.element import registerElement
+
+
+##
+# iSchedule XML Definitions
+##
+
+ischedule_namespace = "urn:ietf:params:xml:ns:ischedule"
+
+
+ at registerElement
+class QueryResult (WebDAVElement):
+    namespace = ischedule_namespace
+    name = "query-result"
+
+    allowed_children = {
+        (ischedule_namespace, "capability-set"): (0, None),
+    }
+
+
+
+ at registerElement
+class Capabilities (WebDAVElement):
+    namespace = ischedule_namespace
+    name = "capabilities"
+
+    allowed_children = {
+        (ischedule_namespace, "serial-number"): (1, 1),
+        (ischedule_namespace, "versions"): (1, 1),
+        (ischedule_namespace, "scheduling-messages"): (1, 1),
+        (ischedule_namespace, "calendar-data-types"): (1, 1),
+        (ischedule_namespace, "attachments"): (1, 1),
+        (ischedule_namespace, "supported-recipient-uri-scheme-set"): (1, 1),
+        (ischedule_namespace, "max-content-length"): (1, 1),
+        (ischedule_namespace, "min-date-time"): (1, 1),
+        (ischedule_namespace, "max-date-time"): (1, 1),
+        (ischedule_namespace, "max-instances"): (1, 1),
+        (ischedule_namespace, "max-recipients"): (1, 1),
+        (ischedule_namespace, "administrator"): (1, 1),
+    }
+
+
+
+ at registerElement
+class SerialNumber (WebDAVTextElement):
+    namespace = ischedule_namespace
+    name = "serial-number"
+
+
+
+ at registerElement
+class Versions (WebDAVElement):
+    namespace = ischedule_namespace
+    name = "versions"
+
+    allowed_children = {
+        (ischedule_namespace, "version"): (1, None),
+    }
+
+
+
+ at registerElement
+class Version (WebDAVTextElement):
+    namespace = ischedule_namespace
+    name = "version"
+
+
+
+ at registerElement
+class SchedulingMessages (WebDAVElement):
+    namespace = ischedule_namespace
+    name = "scheduling-messages"
+
+    allowed_children = {
+        (ischedule_namespace, "component"): (1, None),
+    }
+
+
+
+ at registerElement
+class Component (WebDAVElement):
+    namespace = ischedule_namespace
+    name = "component"
+
+    allowed_children = {
+        (ischedule_namespace, "method"): (0, None),
+    }
+    allowed_attributes = {"name": True}
+
+
+
+ at registerElement
+class Method (WebDAVEmptyElement):
+    namespace = ischedule_namespace
+    name = "method"
+
+    allowed_attributes = {"name": True}
+
+
+
+ at registerElement
+class CalendarDataTypes (WebDAVElement):
+    namespace = ischedule_namespace
+    name = "calendar-data-types"
+
+    allowed_children = {
+        (ischedule_namespace, "calendar-data-type"): (1, None),
+    }
+
+
+
+ at registerElement
+class CalendarDataType (WebDAVTextElement):
+    namespace = ischedule_namespace
+    name = "calendar-data-type"
+
+    allowed_attributes = {
+        "content-type": True,
+        "version": True,
+    }
+
+
+
+ at registerElement
+class Attachments (WebDAVElement):
+    namespace = ischedule_namespace
+    name = "attachments"
+
+    allowed_children = {
+        (ischedule_namespace, "inline"): (0, 1),
+        (ischedule_namespace, "external"): (0, 1),
+    }
+
+
+
+ at registerElement
+class Inline (WebDAVEmptyElement):
+    namespace = ischedule_namespace
+    name = "inline"
+
+
+
+ at registerElement
+class External (WebDAVEmptyElement):
+    namespace = ischedule_namespace
+    name = "external"
+
+
+
+ at registerElement
+class MaxContentLength (WebDAVTextElement):
+    namespace = ischedule_namespace
+    name = "max-content-length"
+
+
+
+ at registerElement
+class MinDateTime (WebDAVTextElement):
+    namespace = ischedule_namespace
+    name = "min-date-time"
+
+
+
+ at registerElement
+class MaxDateTime (WebDAVTextElement):
+    namespace = ischedule_namespace
+    name = "max-date-time"
+
+
+
+ at registerElement
+class MaxInstances (WebDAVTextElement):
+    namespace = ischedule_namespace
+    name = "max-instances"
+
+
+
+ at registerElement
+class MaxRecipients (WebDAVTextElement):
+    namespace = ischedule_namespace
+    name = "max-recipients"
+
+
+
+ at registerElement
+class Administrator (WebDAVTextElement):
+    namespace = ischedule_namespace
+    name = "administrator"
+
+
+
+ at registerElement
+class ScheduleResponse (WebDAVTextElement):
+    namespace = ischedule_namespace
+    name = "schedule-response"
+
+    allowed_children = {
+        (ischedule_namespace, "response"): (0, None),
+    }
+
+
+
+ at registerElement
+class Response (WebDAVElement):
+    namespace = ischedule_namespace
+    name = "response"
+
+    allowed_children = {
+        (ischedule_namespace, "recipient"): (1, 1),
+        (ischedule_namespace, "request-status"): (1, 1),
+        (ischedule_namespace, "calendar-data"): (0, 1),
+        (ischedule_namespace, "error"): (0, 1),
+        (ischedule_namespace, "response-description"): (0, 1),
+    }
+
+
+
+ at registerElement
+class Recipient (WebDAVTextElement):
+    namespace = ischedule_namespace
+    name = "recipient"
+
+
+
+ at registerElement
+class RequestStatus (WebDAVTextElement):
+    namespace = ischedule_namespace
+    name = "request-status"
+
+
+
+ at registerElement
+class CalendarData (WebDAVTextElement):
+    namespace = ischedule_namespace
+    name = "calendar-data"
+
+
+    @classmethod
+    def fromCalendar(clazz, calendar):
+        if isinstance(calendar, str):
+            if not calendar:
+                raise ValueError("Missing calendar data")
+            return clazz(PCDATAElement(calendar))
+        elif isinstance(calendar, iComponent):
+            assert calendar.name() == "VCALENDAR", "Not a calendar: %r" % (calendar,)
+            return clazz(PCDATAElement(calendar.getTextWithTimezones(includeTimezones=not config.EnableTimezonesByReference)))
+        else:
+            raise ValueError("Not a calendar: %s" % (calendar,))
+
+    fromTextData = fromCalendar
+
+
+
+ at registerElement
+class Error (WebDAVElement):
+    namespace = ischedule_namespace
+    name = "error"
+
+    allowed_children = {WebDAVElement: (0, None)}
+
+
+
+ at registerElement
+class ResponseDescription (WebDAVTextElement):
+    namespace = ischedule_namespace
+    name = "response-description"

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/itip.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/itip.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/itip.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,910 @@
+##
+# Copyright (c) 2006-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+iTIP (RFC5546) scheduling message processing and generation.
+
+
+This is currently used for handling auto-replies to schedule requests arriving
+in an inbox. It is called in a delayed fashion via reactor.callLater.
+
+We assume that all the components/calendars we deal with have been determined
+as being 'valid for CalDAV/iTIP', i.e. they contain UIDs, single component
+types, etc.
+"""
+
+
+from twext.python.log import Logger
+
+from twistedcaldav.config import config
+from twistedcaldav.ical import Property, iCalendarProductID, Component, \
+    ignoredComponents
+
+from pycalendar.datetime import PyCalendarDateTime
+
+log = Logger()
+
+__all__ = [
+    "iTipProcessing",
+    "iTipGenerator",
+]
+
+class iTipProcessing(object):
+
+    @staticmethod
+    def processNewRequest(itip_message, recipient=None, creating=False):
+        """
+        Process a METHOD=REQUEST for a brand new calendar object.
+
+        @param itip_message: the iTIP message calendar object to process.
+        @type itip_message:
+
+        @return: calendar object ready to save
+        """
+        assert itip_message.propertyValue("METHOD") == "REQUEST", "iTIP message must have METHOD:REQUEST"
+
+        calendar = itip_message.duplicate()
+        method = calendar.getProperty("METHOD")
+        if method:
+            calendar.removeProperty(method)
+
+        if recipient:
+            iTipProcessing.addTranspForNeedsAction(calendar.subcomponents(), recipient)
+
+            # Check for incoming DECLINED
+            if creating:
+                master = calendar.masterComponent()
+                for component in tuple(calendar.subcomponents()):
+                    if component in ignoredComponents or component is master:
+                        continue
+                    attendee = component.getAttendeeProperty((recipient,))
+                    if attendee and attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") == "DECLINED":
+                        # Mark as hidden if we have a master, otherwise remove
+                        if master is not None:
+                            component.addProperty(Property(Component.HIDDEN_INSTANCE_PROPERTY, "T"))
+                        else:
+                            calendar.removeComponent(component)
+
+        return calendar
+
+
+    @staticmethod
+    def processRequest(itip_message, calendar, recipient):
+        """
+        Process a METHOD=REQUEST. We need to merge per-attendee properties such as TRANPS, COMPLETED etc
+        with the data coming from the organizer.
+
+        @param itip_message: the iTIP message calendar object to process.
+        @type itip_message:
+        @param calendar: the calendar object to apply the REQUEST to
+        @type calendar:
+
+        @return: a C{tuple} of:
+            calendar object ready to save, or C{None} (request should be ignored)
+            a C{set} of recurrences that changed, or C{None}
+        """
+
+        # Check sequencing
+        if not iTipProcessing.sequenceComparison(itip_message, calendar):
+            # Ignore out of sequence message
+            return None, None
+
+        # Merge Organizer data with Attendee's own changes (VALARMs, Comment only for now).
+        from twistedcaldav.scheduling.icaldiff import iCalDiff
+        rids = iCalDiff(calendar, itip_message, False).whatIsDifferent()
+
+        # Different behavior depending on whether a master component is present or not
+        # Here we cache per-attendee data from the master that we need to use in any new
+        # overridden components that the organizer added
+        current_master = calendar.masterComponent()
+        if current_master:
+            master_valarms = [comp for comp in current_master.subcomponents() if comp.name() == "VALARM"]
+            private_comments = current_master.properties("X-CALENDARSERVER-PRIVATE-COMMENT")
+            transps = current_master.properties("TRANSP")
+            completeds = current_master.properties("COMPLETED")
+            organizer = current_master.getProperty("ORGANIZER")
+            organizer_schedule_status = organizer.parameterValue("SCHEDULE-STATUS", None) if organizer else None
+            attendee = current_master.getAttendeeProperty((recipient,))
+            attendee_dtstamp = attendee.parameterValue("X-CALENDARSERVER-DTSTAMP") if attendee else None
+            other_props = {}
+            for pname in config.Scheduling.CalDAV.PerAttendeeProperties:
+                props = tuple(current_master.properties(pname))
+                if props:
+                    other_props[pname] = props
+        else:
+            master_valarms = ()
+            private_comments = ()
+            transps = ()
+            completeds = ()
+            organizer_schedule_status = None
+            attendee_dtstamp = None
+            other_props = {}
+
+        if itip_message.masterComponent() is not None:
+
+            # Get a new calendar object first
+            new_calendar = iTipProcessing.processNewRequest(itip_message, recipient)
+
+            # Copy over master alarms, comments
+            master_component = new_calendar.masterComponent()
+            for alarm in master_valarms:
+                master_component.addComponent(alarm)
+            for comment in private_comments:
+                master_component.addProperty(comment)
+            for transp in transps:
+                master_component.replaceProperty(transp)
+            for completed in completeds:
+                master_component.replaceProperty(completed)
+            if organizer_schedule_status:
+                organizer = master_component.getProperty("ORGANIZER")
+                if organizer:
+                    organizer.setParameter("SCHEDULE-STATUS", organizer_schedule_status)
+            if attendee_dtstamp:
+                attendee = master_component.getAttendeeProperty((recipient,))
+                if attendee:
+                    attendee.setParameter("X-CALENDARSERVER-DTSTAMP", attendee_dtstamp)
+            for props in other_props.values():
+                [master_component.replaceProperty(prop) for prop in props]
+
+            # Now try to match recurrences in the new calendar
+            for component in tuple(new_calendar.subcomponents()):
+                if component.name() != "VTIMEZONE" and component.getRecurrenceIDUTC() is not None:
+                    iTipProcessing.transferItems(calendar, master_valarms, private_comments, transps, completeds, organizer_schedule_status, attendee_dtstamp, other_props, component, recipient)
+
+            # Now try to match recurrences from the old calendar
+            for component in calendar.subcomponents():
+                if component.name() != "VTIMEZONE" and component.getRecurrenceIDUTC() is not None:
+                    rid = component.getRecurrenceIDUTC()
+                    if new_calendar.overriddenComponent(rid) is None:
+                        allowCancelled = component.propertyValue("STATUS") == "CANCELLED"
+                        new_component = new_calendar.deriveInstance(rid, allowCancelled=allowCancelled)
+                        if new_component:
+                            new_calendar.addComponent(new_component)
+                            iTipProcessing.transferItems(calendar, master_valarms, private_comments, transps, completeds, organizer_schedule_status, attendee_dtstamp, other_props, new_component, recipient)
+
+            # Replace the entire object
+            return new_calendar, rids
+
+        else:
+            # Need existing tzids
+            tzids = calendar.timezones()
+
+            # Update existing instances
+            for component in itip_message.subcomponents():
+                if component.name() == "VTIMEZONE":
+                    # May need to add a new VTIMEZONE
+                    if component.propertyValue("TZID") not in tzids:
+                        calendar.addComponent(component)
+                else:
+                    component = component.duplicate()
+                    missingDeclined = iTipProcessing.transferItems(calendar, master_valarms, private_comments, transps, completeds, organizer_schedule_status, attendee_dtstamp, other_props, component, recipient, remove_matched=True)
+                    if not missingDeclined:
+                        calendar.addComponent(component)
+                        if recipient:
+                            iTipProcessing.addTranspForNeedsAction((component,), recipient)
+
+            # Write back the modified object
+            return calendar, rids
+
+
+    @staticmethod
+    def processCancel(itip_message, calendar, autoprocessing=False):
+        """
+        Process a METHOD=CANCEL.
+
+        TODO: Yes, I am going to ignore RANGE= on RECURRENCE-ID for now...
+
+        @param itip_message: the iTIP message calendar object to process.
+        @type itip_message:
+        @param calendar: the calendar object to apply the CANCEL to
+        @type calendar:
+
+        @return: C{tuple} of:
+            C{bool} : C{True} if processed, C{False} if scheduling message should be ignored
+            C{bool} : C{True} if calendar object should be deleted, C{False} otherwise
+            C{set}  : set of Recurrence-IDs for cancelled instances, or C{None} if all cancelled
+        """
+
+        assert itip_message.propertyValue("METHOD") == "CANCEL", "iTIP message must have METHOD:CANCEL"
+        assert itip_message.resourceUID() == calendar.resourceUID(), "UIDs must be the same to process iTIP message"
+
+        # Check sequencing
+        if not iTipProcessing.sequenceComparison(itip_message, calendar):
+            # Ignore out of sequence message
+            return False, False, None
+
+        # Check to see if this is a cancel of the entire event
+        if itip_message.masterComponent() is not None:
+            if autoprocessing:
+                # Delete the entire event off the auto-processed calendar
+                return True, True, None
+            else:
+                # Cancel every instance in the existing event and sync over SEQUENCE
+                calendar.replacePropertyInAllComponents(Property("STATUS", "CANCELLED"))
+                newseq = itip_message.masterComponent().propertyValue("SEQUENCE")
+                calendar.replacePropertyInAllComponents(Property("SEQUENCE", newseq))
+                return True, False, None
+
+        # iTIP CANCEL can contain multiple components being cancelled in the RECURRENCE-ID case.
+        # So we need to iterate over each iTIP component.
+
+        # Get the existing calendar master object if it exists
+        calendar_master = calendar.masterComponent()
+        exdates = []
+        rids = set()
+
+        # Look at each component in the iTIP message
+        for component in itip_message.subcomponents():
+            if component.name() == "VTIMEZONE":
+                continue
+
+            # Extract RECURRENCE-ID value from component
+            rid = component.getRecurrenceIDUTC()
+            rids.add(rid)
+
+            # Get the one that matches in the calendar
+            overridden = calendar.overriddenComponent(rid)
+
+            if overridden:
+                # We are cancelling an overridden component.
+
+                if autoprocessing:
+                    # Exclude the cancelled instance
+                    exdates.append(component.getRecurrenceIDUTC())
+
+                    # Remove the existing component.
+                    calendar.removeComponent(overridden)
+                else:
+                    # Existing component is cancelled.
+                    overridden.replaceProperty(Property("STATUS", "CANCELLED"))
+                    newseq = component.propertyValue("SEQUENCE")
+                    overridden.replaceProperty(Property("SEQUENCE", newseq))
+
+            elif calendar_master:
+                # We are trying to CANCEL a non-overridden instance.
+
+                if autoprocessing:
+                    # Exclude the cancelled instance
+                    exdates.append(component.getRecurrenceIDUTC())
+                else:
+                    # Derive a new component and cancel it.
+                    overridden = calendar.deriveInstance(rid)
+                    if overridden:
+                        overridden.replaceProperty(Property("STATUS", "CANCELLED"))
+                        calendar.addComponent(overridden)
+                        newseq = component.propertyValue("SEQUENCE")
+                        overridden.replacePropertyInAllComponents(Property("SEQUENCE", newseq))
+
+        # If we have any EXDATEs lets add them to the existing calendar object.
+        if exdates and calendar_master:
+            calendar_master.addProperty(Property("EXDATE", exdates))
+
+        # See if there are still components in the calendar - we might have deleted the last overridden instance
+        # in which case the calendar object is empty (except for VTIMEZONEs).
+        if calendar.mainType() is None:
+            # Delete the now empty calendar object
+            return True, True, None
+        else:
+            return True, False, rids
+
+
+    @staticmethod
+    def processReply(itip_message, calendar):
+        """
+        Process a METHOD=REPLY.
+
+        TODO: Yes, I am going to ignore RANGE= on RECURRENCE-ID for now...
+        TODO: We have no way to track SEQUENCE/DTSTAMP on a per-attendee basis to correctly serialize out-of-order
+              replies.
+
+        @param itip_message: the iTIP message calendar object to process.
+        @type itip_message:
+        @param calendar: the calendar object to apply the REPLY to
+        @type calendar:
+
+        @return: a C{tuple} of:
+            C{True} if processed, C{False} if scheduling message should be ignored
+            C{tuple} of change info
+        """
+
+        assert itip_message.propertyValue("METHOD") == "REPLY", "iTIP message must have METHOD:REPLY"
+        assert itip_message.resourceUID() == calendar.resourceUID(), "UIDs must be the same to process iTIP message"
+
+        # Take each component in the reply and update the corresponding component
+        # in the organizer's copy (possibly generating new ones) so that the ATTENDEE
+        # PARTSTATs match up.
+
+        # Do the master first
+        old_master = calendar.masterComponent()
+        new_master = itip_message.masterComponent()
+        attendees = set()
+        rids = set()
+        if new_master is not None and old_master is not None:
+            attendee, partstat, private_comment = iTipProcessing.updateAttendeeData(new_master, old_master)
+            if attendee:
+                attendees.add(attendee)
+                if partstat or private_comment:
+                    rids.add(("", partstat, private_comment,))
+
+        # Now do all overridden ones (sort by RECURRENCE-ID)
+        sortedComponents = []
+        for itip_component in itip_message.subcomponents():
+
+            # Make sure we have an appropriate component
+            if itip_component.name() == "VTIMEZONE":
+                continue
+            rid = itip_component.getRecurrenceIDUTC()
+            if rid is None:
+                continue
+            sortedComponents.append((rid, itip_component,))
+
+        sortedComponents.sort(key=lambda x: x[0])
+
+        for rid, itip_component in sortedComponents:
+            # Find matching component in organizer's copy
+            match_component = calendar.overriddenComponent(rid)
+            if match_component is None:
+                # Attendee is overriding an instance themselves - we need to create a derived one
+                # for the Organizer
+                match_component = calendar.deriveInstance(rid)
+                if match_component:
+                    calendar.addComponent(match_component)
+                else:
+                    log.error("Ignoring instance: %s in iTIP REPLY for: %s" % (rid, itip_message.resourceUID()))
+                    continue
+
+            attendee, partstat, private_comment = iTipProcessing.updateAttendeeData(itip_component, match_component)
+            if attendee:
+                attendees.add(attendee)
+                if rids is not None and (partstat or private_comment):
+                    rids.add((rid.getText(), partstat, private_comment,))
+
+        # Check for an invalid instance by itself
+        len_attendees = len(attendees)
+        if len_attendees == 0:
+            return False, None
+        elif len_attendees == 1:
+            return True, (attendees.pop(), rids)
+        else:
+            log.error("ATTENDEE property in a REPLY must be the same in all components\n%s" % (str(itip_message),))
+            return False, None
+
+
+    @staticmethod
+    def updateAttendeeData(from_component, to_component):
+        """
+        Copy the PARTSTAT of the Attendee in the from_component to the matching ATTENDEE
+        in the to_component. Ignore if no match found. Also update the private comments.
+
+        @param from_component:
+        @type from_component:
+        @param to_component:
+        @type to_component:
+        """
+
+        # Track what changed
+        partstat_changed = False
+        private_comment_changed = False
+
+        # Get REQUEST-STATUS as we need to write that into the saved ATTENDEE property
+        reqstatus = tuple(from_component.properties("REQUEST-STATUS"))
+        if reqstatus:
+            reqstatus = ",".join(status.value()[0] for status in reqstatus)
+        else:
+            reqstatus = "2.0"
+
+        # Get attendee in from_component - there MUST be only one
+        attendees = tuple(from_component.properties("ATTENDEE"))
+        if len(attendees) != 1:
+            log.error("There must be one and only one ATTENDEE property in a REPLY\n%s" % (str(from_component),))
+            return None, False, False
+
+        attendee = attendees[0]
+        partstat = attendee.parameterValue("PARTSTAT", "NEEDS-ACTION")
+
+        # Now find matching ATTENDEE in to_component
+        existing_attendee = to_component.getAttendeeProperty((attendee.value(),))
+        if existing_attendee:
+            oldpartstat = existing_attendee.parameterValue("PARTSTAT", "NEEDS-ACTION")
+            existing_attendee.setParameter("PARTSTAT", partstat)
+            existing_attendee.setParameter("SCHEDULE-STATUS", reqstatus)
+            partstat_changed = (oldpartstat != partstat)
+
+            # Always delete RSVP on PARTSTAT change
+            if partstat_changed:
+                try:
+                    existing_attendee.removeParameter("RSVP")
+                except KeyError:
+                    pass
+
+            # Handle attendee comments
+            if config.Scheduling.CalDAV.get("EnablePrivateComments", True):
+                # Look for X-CALENDARSERVER-PRIVATE-COMMENT property in iTIP component (State 1 in spec)
+                attendee_comment = tuple(from_component.properties("X-CALENDARSERVER-PRIVATE-COMMENT"))
+                attendee_comment = attendee_comment[0] if len(attendee_comment) else None
+
+                # Look for matching X-CALENDARSERVER-ATTENDEE-COMMENT property in existing data (State 2 in spec)
+                private_comments = tuple(to_component.properties("X-CALENDARSERVER-ATTENDEE-COMMENT"))
+                for comment in private_comments:
+                    attendeeref = comment.parameterValue("X-CALENDARSERVER-ATTENDEE-REF")
+                    if attendeeref == attendee.value():
+                        private_comment = comment
+                        break
+                else:
+                    private_comment = None
+            else:
+                attendee_comment = None
+                private_comment = None
+
+            # Now do update logic
+            if attendee_comment is None and private_comment is None:
+                # Nothing to do
+                pass
+
+            elif attendee_comment is None and private_comment is not None:
+                # Remove all property parameters
+                private_comment.removeAllParameters()
+
+                # Add default parameters
+                private_comment.setParameter("X-CALENDARSERVER-ATTENDEE-REF", attendee.value())
+                private_comment.setParameter("X-CALENDARSERVER-DTSTAMP", PyCalendarDateTime.getNowUTC().getText())
+
+                # Set value empty
+                private_comment.setValue("")
+
+                private_comment_changed = True
+
+            elif attendee_comment is not None and private_comment is None:
+
+                # Add new property
+                private_comment = Property(
+                    "X-CALENDARSERVER-ATTENDEE-COMMENT",
+                    attendee_comment.value(),
+                    params={
+                        "X-CALENDARSERVER-ATTENDEE-REF": attendee.value(),
+                        "X-CALENDARSERVER-DTSTAMP": PyCalendarDateTime.getNowUTC().getText(),
+                    }
+                )
+                to_component.addProperty(private_comment)
+
+                private_comment_changed = True
+
+            else:
+                # Only change if different
+                if private_comment.value() != attendee_comment.value():
+                    # Remove all property parameters
+                    private_comment.removeAllParameters()
+
+                    # Add default parameters
+                    private_comment.setParameter("X-CALENDARSERVER-ATTENDEE-REF", attendee.value())
+                    private_comment.setParameter("X-CALENDARSERVER-DTSTAMP", PyCalendarDateTime.getNowUTC().getText())
+
+                    # Set new value
+                    private_comment.setValue(attendee_comment.value())
+
+                    private_comment_changed = True
+
+        return attendee.value(), partstat_changed, private_comment_changed
+
+
+    @staticmethod
+    def transferItems(from_calendar, master_valarms, private_comments, transps, completeds, organizer_schedule_status, attendee_dtstamp, other_props, to_component, recipient, remove_matched=False):
+        """
+        Transfer properties from a calendar to a component by first trying to match the component in the original calendar and
+        use the properties from that, or use the values provided as arguments (which have been derived from the original calendar's
+        master component).
+
+        @return: C{True} if an EXDATE match occurred requiring the incoming component to be removed.
+        """
+
+        rid = to_component.getRecurrenceIDUTC()
+
+        # Is there a matching component
+        matched = from_calendar.overriddenComponent(rid)
+        if matched:
+            # Copy over VALARMs from existing component
+            [to_component.addComponent(comp) for comp in matched.subcomponents() if comp.name() == "VALARM"]
+            [to_component.addProperty(prop) for prop in matched.properties("X-CALENDARSERVER-ATTENDEE-COMMENT")]
+            [to_component.replaceProperty(prop) for prop in matched.properties("TRANSP")]
+            [to_component.replaceProperty(prop) for prop in matched.properties("COMPLETED")]
+
+            organizer = matched.getProperty("ORGANIZER")
+            organizer_schedule_status = organizer.parameterValue("SCHEDULE-STATUS", None) if organizer else None
+            if organizer_schedule_status:
+                organizer = to_component.getProperty("ORGANIZER")
+                if organizer:
+                    organizer.setParameter("SCHEDULE-STATUS", organizer_schedule_status)
+
+            # Remove the old one
+            if remove_matched:
+                from_calendar.removeComponent(matched)
+
+            # Check for incoming DECLINED
+            attendee = to_component.getAttendeeProperty((recipient,))
+            if attendee and attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") == "DECLINED":
+                # If existing item has HIDDEN property copy that over
+                if matched.hasProperty(Component.HIDDEN_INSTANCE_PROPERTY):
+                    to_component.addProperty(Property(Component.HIDDEN_INSTANCE_PROPERTY, "T"))
+
+            if attendee and attendee_dtstamp:
+                attendee.setParameter("X-CALENDARSERVER-DTSTAMP", attendee_dtstamp)
+
+            for pname in config.Scheduling.CalDAV.PerAttendeeProperties:
+                [to_component.replaceProperty(prop) for prop in matched.properties(pname)]
+        else:
+            # Check for incoming DECLINED
+            attendee = to_component.getAttendeeProperty((recipient,))
+            if attendee and attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") == "DECLINED":
+                return True
+
+            # It is a new override - copy any valarms on the existing master component
+            # into the new one.
+            [to_component.addComponent(alarm) for alarm in master_valarms]
+            [to_component.addProperty(comment) for comment in private_comments]
+            [to_component.replaceProperty(transp) for transp in transps]
+            [to_component.replaceProperty(completed) for completed in completeds]
+
+            if organizer_schedule_status:
+                organizer = to_component.getProperty("ORGANIZER")
+                if organizer:
+                    organizer.setParameter("SCHEDULE-STATUS", organizer_schedule_status)
+            if attendee_dtstamp:
+                attendee = to_component.getAttendeeProperty((recipient,))
+                if attendee:
+                    attendee.setParameter("X-CALENDARSERVER-DTSTAMP", attendee_dtstamp)
+
+            for props in other_props.values():
+                [to_component.replaceProperty(prop) for prop in props]
+
+        return False
+
+
+    @staticmethod
+    def addTranspForNeedsAction(components, recipient):
+        # For each component where the ATTENDEE property of the recipient has PARTSTAT
+        # NEEDS-ACTION we add TRANSP:TRANSPARENT for VEVENTs
+        for component in components:
+            if component.name() != "VEVENT":
+                continue
+            attendee = component.getAttendeeProperty((recipient,))
+            if attendee and attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") == "NEEDS-ACTION":
+                component.replaceProperty(Property("TRANSP", "TRANSPARENT"))
+
+
+    @staticmethod
+    def sequenceComparison(itip, calendar):
+        """
+        Do appropriate itip message sequencing based by comparison with existing calendar data.
+
+        @return: C{True} if the itip message is new and should be processed, C{False}
+            if no processing is needed
+        @rtype: C{bool}
+        """
+
+        # Master component comparison trumps all else
+        itip_master = itip.masterComponent()
+        cal_master = calendar.masterComponent()
+
+        # If master component exists, compare all in iTIP and update if any are new
+        if cal_master:
+            for itip_component in itip.subcomponents():
+                if itip_component.name() in ignoredComponents:
+                    continue
+                cal_component = calendar.overriddenComponent(itip_component.getRecurrenceIDUTC())
+                if cal_component is None:
+                    cal_component = cal_master
+
+                # TODO: No DTSTAMP comparison because we do not track DTSTAMPs
+                # Treat components the same as meaning so an update - in theory no harm in doing that
+                if Component.compareComponentsForITIP(itip_component, cal_component, use_dtstamp=False) >= 0:
+                    return True
+
+            return False
+
+        elif itip_master:
+
+            # Do comparison of each appropriate component if any one is new, process the itip
+            for cal_component in calendar.subcomponents():
+                if cal_component.name() in ignoredComponents:
+                    continue
+                itip_component = itip.overriddenComponent(cal_component.getRecurrenceIDUTC())
+                if itip_component is None:
+                    itip_component = itip_master
+
+                # TODO: No DTSTAMP comparison because we do not track DTSTAMPs
+                # Treat components the same as meaning so an update - in theory no harm in doing that
+                if Component.compareComponentsForITIP(itip_component, cal_component, use_dtstamp=False) >= 0:
+                    return True
+
+            return False
+
+        else:
+            # Do comparison of each matching component if any one is new, process the entire itip.
+            # There is a race condition here, similar to REPLY, where we could reinstate an instance
+            # that has been removed. Not much we can do about it without additional tracking.
+
+            cal_rids = set()
+            for cal_component in calendar.subcomponents():
+                if cal_component.name() in ignoredComponents:
+                    continue
+                cal_rids.add(cal_component.getRecurrenceIDUTC())
+            itip_rids = set()
+            for itip_component in itip.subcomponents():
+                if itip_component.name() in ignoredComponents:
+                    continue
+                itip_rids.add(itip_component.getRecurrenceIDUTC())
+
+            # Compare ones that match
+            for rid in cal_rids & itip_rids:
+                cal_component = calendar.overriddenComponent(rid)
+                itip_component = itip.overriddenComponent(rid)
+
+                # TODO: No DTSTAMP comparison because we do not track DTSTAMPs
+                # Treat components the same as meaning so an update - in theory no harm in doing that
+                if Component.compareComponentsForITIP(itip_component, cal_component, use_dtstamp=False) >= 0:
+                    return True
+
+            # If there are others in one set and not the other - always process, else no process
+            return len(cal_rids ^ itip_rids) > 0
+
+
+
+class iTipGenerator(object):
+    """
+    This assumes that DTSTAMP and SEQUENCE are already at their new values in the original calendar
+    data passed in to each generateXXX() call.
+    """
+
+    @staticmethod
+    def generateCancel(original, attendees, instances=None, full_cancel=False):
+        """
+        This assumes that SEQUENCE is not already at its new value in the original calendar data. This
+        is because the component passed in is the one that originally contained the attendee that is
+        being removed.
+        """
+
+        itip = Component("VCALENDAR")
+        itip.addProperty(Property("VERSION", "2.0"))
+        itip.addProperty(Property("PRODID", iCalendarProductID))
+        itip.addProperty(Property("METHOD", "CANCEL"))
+
+        if instances is None:
+            instances = (None,)
+
+        tzids = set()
+        added = False
+        for instance_rid in instances:
+
+            # Create a new component matching the type of the original
+            comp = Component(original.mainType())
+
+            # Use the master component when the instance is None
+            if not instance_rid:
+                instance = original.masterComponent()
+                assert instance is not None, "Need a master component"
+            else:
+                instance = original.overriddenComponent(instance_rid)
+                if instance is None:
+                    instance = original.deriveInstance(instance_rid)
+
+                # If the instance to be cancelled did not exist in the original, then
+                # do nothing
+                if instance is None:
+                    continue
+
+            # Add some required properties extracted from the original
+            comp.addProperty(Property("DTSTAMP", instance.propertyValue("DTSTAMP")))
+            comp.addProperty(Property("UID", instance.propertyValue("UID")))
+            seq = instance.propertyValue("SEQUENCE")
+            seq = int(seq) + 1 if seq else 1
+            comp.addProperty(Property("SEQUENCE", seq))
+            comp.addProperty(instance.getOrganizerProperty())
+            if instance_rid:
+                comp.addProperty(Property("RECURRENCE-ID", instance_rid.duplicate().adjustToUTC()))
+
+            def addProperties(propname):
+                for icalproperty in instance.properties(propname):
+                    comp.addProperty(icalproperty)
+
+            addProperties("SUMMARY")
+            addProperties("DTSTART")
+            addProperties("DTEND")
+            addProperties("DURATION")
+            if not instance_rid:
+                addProperties("RRULE")
+                addProperties("RDATE")
+                addProperties("EXDATE")
+
+            # Extract the matching attendee property
+            for attendee in attendees:
+                if full_cancel:
+                    attendeeProp = original.getAttendeeProperty((attendee,))
+                else:
+                    attendeeProp = instance.getAttendeeProperty((attendee,))
+                assert attendeeProp is not None, "Must have matching ATTENDEE property"
+                comp.addProperty(attendeeProp)
+
+            tzids.update(comp.timezoneIDs())
+
+            itip.addComponent(comp)
+            added = True
+
+        if added:
+            # Now include any referenced tzids
+            for comp in original.subcomponents():
+                if comp.name() == "VTIMEZONE":
+                    tzid = comp.propertyValue("TZID")
+                    if tzid in tzids:
+                        itip.addComponent(comp)
+
+            # Strip out unwanted bits
+            iTipGenerator.prepareSchedulingMessage(itip)
+
+            return itip
+        else:
+            return None
+
+
+    @staticmethod
+    def generateAttendeeRequest(original, attendees, filter_rids):
+        """
+        This assumes that SEQUENCE is already at its new value in the original calendar data.
+        """
+
+        # Start with a copy of the original as we may have to modify bits of it
+        itip = original.duplicate()
+        itip.replaceProperty(Property("PRODID", iCalendarProductID))
+        itip.addProperty(Property("METHOD", "REQUEST"))
+
+        # Now filter out components that do not contain every attendee
+        itip.attendeesView(attendees, onlyScheduleAgentServer=True)
+
+        # Now filter out components except the ones specified
+        if itip.filterComponents(filter_rids):
+            # Strip out unwanted bits
+            iTipGenerator.prepareSchedulingMessage(itip)
+            return itip
+
+        else:
+            return None
+
+
+    @staticmethod
+    def generateAttendeeReply(original, attendee, changedRids=None, force_decline=False):
+
+        # Start with a copy of the original as we may have to modify bits of it
+        itip = original.duplicate()
+        itip.replaceProperty(Property("PRODID", iCalendarProductID))
+        itip.addProperty(Property("METHOD", "REPLY"))
+
+        # Now filter out components except the ones specified
+        itip.filterComponents(changedRids)
+
+        # Force update to DTSTAMP everywhere so reply sequencing will work
+        itip.replacePropertyInAllComponents(Property("DTSTAMP", PyCalendarDateTime.getNowUTC()))
+
+        # Remove all attendees except the one we want
+        itip.removeAllButOneAttendee(attendee)
+
+        # Remove all components which are missing the attendee
+        for component in itip.subcomponents():
+            if component.name() in ignoredComponents:
+                continue
+            if not component.getAttendeeProperty((attendee,)):
+                itip.removeComponent(component)
+
+        # No alarms
+        itip.removeAlarms()
+
+        # Remove all but essential properties
+        itip.filterProperties(keep=(
+            "UID",
+            "RECURRENCE-ID",
+            "SEQUENCE",
+            "STATUS",
+            "DTSTAMP",
+            "DTSTART",
+            "DTEND",
+            "DURATION",
+            "RRULE",
+            "RDATE",
+            "EXDATE",
+            "ORGANIZER",
+            "ATTENDEE",
+            "X-CALENDARSERVER-PRIVATE-COMMENT",
+            "SUMMARY",
+            "LOCATION",
+            "DESCRIPTION",
+        ))
+
+        # Now set each ATTENDEE's PARTSTAT to DECLINED
+        if force_decline:
+            attendeeProps = itip.getAttendeeProperties((attendee,))
+            assert attendeeProps, "Must have some matching ATTENDEEs"
+            for attendeeProp in attendeeProps:
+                attendeeProp.setParameter("PARTSTAT", "DECLINED")
+
+        # Add REQUEST-STATUS to each top-level component
+        itip.addPropertyToAllComponents(Property("REQUEST-STATUS", ["2.0", "Success", ]))
+
+        # Strip out unwanted bits
+        iTipGenerator.prepareSchedulingMessage(itip, reply=True)
+
+        return itip
+
+
+    @staticmethod
+    def prepareSchedulingMessage(itip, reply=False):
+        """
+        Remove properties and parameters that should not be sent in an iTIP message
+        """
+
+        # All X- components go away
+        itip.removeXComponents()
+
+        # Alarms
+        itip.removeAlarms()
+
+        # Top-level properties - remove all X-
+        itip.removeXProperties(do_subcomponents=False)
+
+        # Component properties - remove all X- except for those specified
+        if not reply:
+            # Organizer properties that need to go to the Attendees
+            keep_properties = ("X-APPLE-DROPBOX",)
+        else:
+            # Attendee properties that need to go to the Organizer
+            keep_properties = ("X-CALENDARSERVER-PRIVATE-COMMENT",)
+        itip.removeXProperties(keep_properties=keep_properties)
+
+        # Property Parameters
+        itip.removePropertyParameters("ATTENDEE", ("SCHEDULE-AGENT", "SCHEDULE-STATUS", "SCHEDULE-FORCE-SEND", "X-CALENDARSERVER-DTSTAMP",))
+        itip.removePropertyParameters("ORGANIZER", ("SCHEDULE-AGENT", "SCHEDULE-STATUS", "SCHEDULE-FORCE-SEND",))
+
+
+
+class iTIPRequestStatus(object):
+    """
+    String constants for various iTIP status codes we use.
+    """
+
+    MESSAGE_PENDING_CODE = "1.0"
+    MESSAGE_SENT_CODE = "1.1"
+    MESSAGE_DELIVERED_CODE = "1.2"
+
+    SUCCESS_CODE = "2.0"
+
+    INVALID_CALENDAR_USER_CODE = "3.7"
+    NO_AUTHORITY_CODE = "3.8"
+
+    BAD_REQUEST_CODE = "5.0"
+    SERVICE_UNAVAILABLE_CODE = "5.1"
+    INVALID_SERVICE_CODE = "5.2"
+    NO_USER_SUPPORT_CODE = "5.3"
+
+    MESSAGE_PENDING = MESSAGE_PENDING_CODE + ";Scheduling message send is pending"
+    MESSAGE_SENT = MESSAGE_SENT_CODE + ";Scheduling message has been sent"
+    MESSAGE_DELIVERED = MESSAGE_DELIVERED_CODE + ";Scheduling message has been delivered"
+
+    SUCCESS = SUCCESS_CODE + ";Success"
+
+    INVALID_CALENDAR_USER = INVALID_CALENDAR_USER_CODE + ";Invalid Calendar User"
+    NO_AUTHORITY = NO_AUTHORITY_CODE + ";No authority"
+
+    BAD_REQUEST = BAD_REQUEST_CODE + ";Service cannot handle request"
+    SERVICE_UNAVAILABLE = SERVICE_UNAVAILABLE_CODE + ";Service unavailable"
+    INVALID_SERVICE = INVALID_SERVICE_CODE + ";Invalid calendar service"
+    NO_USER_SUPPORT = NO_USER_SUPPORT_CODE + ";No scheduling support for user"

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/processing.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/processing.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/processing.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,1054 @@
+#
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from pycalendar.datetime import PyCalendarDateTime
+from pycalendar.duration import PyCalendarDuration
+from pycalendar.timezone import PyCalendarTimezone
+
+from twext.python.log import Logger
+from twext.web2.dav.method.report import NumberOfMatchesWithinLimits
+from twext.web2.http import HTTPError
+
+from twisted.internet import reactor
+from twisted.internet.defer import inlineCallbacks, returnValue
+from twisted.python.log import err as log_traceback
+
+from twistedcaldav import customxml, caldavxml
+from twistedcaldav.caldavxml import caldav_namespace
+from twistedcaldav.config import config
+from twistedcaldav.ical import Property
+from twistedcaldav.instance import InvalidOverriddenInstanceError
+from twistedcaldav.memcachelock import MemcacheLock, MemcacheLockTimeoutError
+from twistedcaldav.memcacher import Memcacher
+from twistedcaldav.method import report_common
+
+from txdav.caldav.datastore.scheduling.cuaddress import normalizeCUAddr
+from txdav.caldav.datastore.scheduling.itip import iTipProcessing, iTIPRequestStatus
+from txdav.caldav.datastore.scheduling.utils import getCalendarObjectForPrincipals
+
+import collections
+import hashlib
+import uuid
+
+"""
+CalDAV implicit processing.
+
+This module handles the processing of scheduling messages being delivered to a calendar user's inbox.
+It determines who is scheduling (organizer or attendee) and applies the scheduling message changes
+to the recipient's calendar data as well as depositing the scheduling message in the inbox. For users
+who have an auto-accept option on, it will also handle the automatic response. Also, refreshes of other
+attendees (when one attendee replies) are triggered from here.
+"""
+
+__all__ = [
+    "ImplicitProcessor",
+    "ImplicitProcessorException",
+]
+
+log = Logger()
+
+class ImplicitProcessorException(Exception):
+
+    def __init__(self, msg):
+        self.msg = msg
+
+
+
+class ImplicitProcessor(object):
+
+    def __init__(self):
+        pass
+
+
+    @inlineCallbacks
+    def doImplicitProcessing(self, txn, message, originator, recipient, noAttendeeRefresh=False):
+        """
+        Do implicit processing of a scheduling message, and possibly also auto-process it
+        if the recipient has auto-accept on.
+
+        @param message:
+        @type message:
+        @param originator:
+        @type originator:
+        @param recipient:
+        @type recipient:
+
+        @return: a C{tuple} of (C{bool}, C{bool}) indicating whether the message was processed, and if it was whether
+            auto-processing has taken place.
+        """
+
+        self.txn = txn
+        self.message = message
+        self.originator = originator
+        self.recipient = recipient
+        self.noAttendeeRefresh = noAttendeeRefresh
+
+        # TODO: for now going to assume that the originator is local - i.e. the scheduling message sent
+        # represents the actual organizer's view.
+
+        # First see whether this is the organizer or attendee sending the message
+        self.extractCalendarData()
+
+        if self.isOrganizerReceivingMessage():
+            result = (yield self.doImplicitOrganizer())
+        elif self.isAttendeeReceivingMessage():
+            try:
+                result = (yield self.doImplicitAttendee())
+            except ImplicitProcessorException:
+                # These we always pass up
+                raise
+            except Exception, e:
+                # We attempt to recover from this. That involves trying to re-write the attendee data
+                # to match that of the organizer assuming we have the organizer's full data available, then
+                # we try the processing operation again.
+                log_traceback()
+                log.error("ImplicitProcessing - originator '%s' to recipient '%s' with UID: '%s' - exception raised will try to fix: %s" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid, e))
+                result = (yield self.doImplicitAttendeeEventFix(e))
+                if result:
+                    log.error("ImplicitProcessing - originator '%s' to recipient '%s' with UID: '%s' - restored organizer's copy" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
+                    try:
+                        result = (yield self.doImplicitAttendee())
+                    except Exception, e:
+                        log_traceback()
+                        log.error("ImplicitProcessing - originator '%s' to recipient '%s' with UID: '%s' - exception raised after fix: %s" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid, e))
+                        raise ImplicitProcessorException("5.1;Service unavailable")
+                else:
+                    log.error("ImplicitProcessing - originator '%s' to recipient '%s' with UID: '%s' - could not fix" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
+                    raise ImplicitProcessorException("5.1;Service unavailable")
+        else:
+            log.error("METHOD:%s not supported for implicit scheduling." % (self.method,))
+            raise ImplicitProcessorException("3.14;Unsupported capability")
+
+        returnValue(result)
+
+
+    def extractCalendarData(self):
+
+        # Some other useful things
+        self.method = self.message.propertyValue("METHOD")
+        self.uid = self.message.resourceUID()
+
+
+    def isOrganizerReceivingMessage(self):
+        return self.method in ("REPLY", "REFRESH")
+
+
+    def isAttendeeReceivingMessage(self):
+        return self.method in ("REQUEST", "ADD", "CANCEL")
+
+
+    @inlineCallbacks
+    def getRecipientsCopy(self):
+        """
+        Get the Recipient's copy of the event being processed.
+        """
+
+        self.recipient_calendar = None
+        self.recipient_calendar_resource = None
+        calendar_resource = (yield getCalendarObjectForPrincipals(self.txn, self.recipient.principal, self.uid))
+        if calendar_resource:
+            self.recipient_calendar = (yield calendar_resource.componentForUser(self.recipient.principal.uid()))
+            self.recipient_calendar_resource = calendar_resource
+
+
+    @inlineCallbacks
+    def doImplicitOrganizer(self):
+
+        # Locate the organizer's copy of the event.
+        yield self.getRecipientsCopy()
+        if self.recipient_calendar is None:
+            log.debug("ImplicitProcessing - originator '%s' to recipient '%s' ignoring UID: '%s' - organizer has no copy" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
+            returnValue((True, True, False, None,))
+
+        # Handle new items differently than existing ones.
+        if self.method == "REPLY":
+            result = (yield self.doImplicitOrganizerUpdate())
+        elif self.method == "REFRESH":
+            # With implicit we ignore refreshes.
+            # TODO: for iMIP etc we do need to handle them
+            result = (True, True, False, None,)
+
+        returnValue(result)
+
+
+    @inlineCallbacks
+    def doImplicitOrganizerUpdate(self):
+
+        # Check to see if this is a valid reply
+        result, processed = iTipProcessing.processReply(self.message, self.recipient_calendar)
+        if result:
+
+            # Let the store know that no time-range info has changed
+            self.recipient_calendar.noInstanceIndexing = True
+
+            # Update the organizer's copy of the event
+            log.debug("ImplicitProcessing - originator '%s' to recipient '%s' processing METHOD:REPLY, UID: '%s' - updating event" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
+            self.organizer_calendar_resource = (yield self.writeCalendarResource(None, self.recipient_calendar_resource, self.recipient_calendar))
+
+            organizer = self.recipient_calendar.getOrganizer()
+
+            # Build the schedule-changes XML element
+            attendeeReplying, rids = processed
+            partstatChanged = False
+            reply_details = (customxml.Attendee.fromString(attendeeReplying),)
+
+            for rid, partstatChanged, privateCommentChanged in sorted(rids):
+                recurrence = []
+                if rid == "":
+                    recurrence.append(customxml.Master())
+                else:
+                    recurrence.append(customxml.RecurrenceID.fromString(rid))
+                changes = []
+                if partstatChanged:
+                    changes.append(customxml.ChangedProperty(customxml.ChangedParameter(name="PARTSTAT"), name="ATTENDEE"))
+                    partstatChanged = True
+                if privateCommentChanged:
+                    changes.append(customxml.ChangedProperty(name="X-CALENDARSERVER-PRIVATE-COMMENT"))
+                recurrence.append(customxml.Changes(*changes))
+                reply_details += (customxml.Recurrence(*recurrence),)
+
+            changes = customxml.ScheduleChanges(
+                customxml.DTStamp(),
+                customxml.Action(
+                    customxml.Reply(*reply_details),
+                ),
+            )
+
+            # Only update other attendees when the partstat was changed by the reply,
+            # and only if the request does not indicate we should skip attendee refresh
+            # (e.g. inbox item processing during migration from non-implicit server)
+            if partstatChanged and not self.noAttendeeRefresh:
+                yield self.queueAttendeeUpdate((attendeeReplying, organizer,))
+
+            result = (True, False, True, changes,)
+
+        else:
+            # Ignore scheduling message
+            result = (True, True, False, None,)
+
+        returnValue(result)
+
+
+    @inlineCallbacks
+    def queueAttendeeUpdate(self, exclude_attendees):
+        """
+        Queue up an update to attendees and use a memcache lock to ensure we don't update too frequently.
+
+        @param exclude_attendees: list of attendees who should not be refreshed (e.g., the one that triggeed the refresh)
+        @type exclude_attendees: C{list}
+        """
+
+        # When doing auto-processing of replies, only refresh attendees when the last auto-accept is done.
+        # Note that when we do this we also need to refresh the attendee that is generating the reply because they
+        # are no longer up to date with changes of other auto-accept attendees.
+        if hasattr(self.txn, "auto_reply_processing_count") and self.txn.auto_reply_processing_count > 1:
+            self.txn.auto_reply_suppressed = True
+            returnValue(None)
+        if hasattr(self.txn, "auto_reply_suppressed"):
+            exclude_attendees = ()
+
+        self.uid = self.recipient_calendar.resourceUID()
+
+        # Check for batched refreshes
+        if config.Scheduling.Options.AttendeeRefreshBatch:
+
+            # Need to lock whilst manipulating the batch list
+            lock = MemcacheLock(
+                "BatchRefreshUIDLock",
+                self.uid,
+                timeout=config.Scheduling.Options.UIDLockTimeoutSeconds,
+                expire_time=config.Scheduling.Options.UIDLockExpirySeconds,
+            )
+            try:
+                yield lock.acquire()
+            except MemcacheLockTimeoutError:
+                # If we could not lock then just fail the refresh - not sure what else to do
+                returnValue(None)
+
+            try:
+                # Get all attendees to refresh
+                allAttendees = sorted(list(self.recipient_calendar.getAllUniqueAttendees()))
+                allAttendees = filter(lambda x: x not in exclude_attendees, allAttendees)
+
+                if allAttendees:
+                    # See if there is already a pending refresh and merge current attendees into that list,
+                    # otherwise just mark all attendees as pending
+                    cache = Memcacher("BatchRefreshAttendees", pickle=True)
+                    pendingAttendees = yield cache.get(self.uid)
+                    firstTime = False
+                    if pendingAttendees:
+                        for attendee in allAttendees:
+                            if attendee not in pendingAttendees:
+                                pendingAttendees.append(attendee)
+                    else:
+                        firstTime = True
+                        pendingAttendees = allAttendees
+                    yield cache.set(self.uid, pendingAttendees)
+
+                    # Now start the first batch off
+                    if firstTime:
+                        self._enqueueBatchRefresh()
+            finally:
+                yield lock.clean()
+
+        else:
+            yield self._doRefresh(self.organizer_calendar_resource, exclude_attendees)
+
+
+    @inlineCallbacks
+    def _doRefresh(self, organizer_resource, exclude_attendees=(), only_attendees=None):
+        """
+        Do a refresh of attendees.
+
+        @param organizer_resource: the resource for the organizer's calendar data
+        @type organizer_resource: L{DAVResource}
+        @param exclude_attendees: list of attendees to not refresh
+        @type exclude_attendees: C{tuple}
+        @param only_attendees: list of attendees to refresh (C{None} - refresh all)
+        @type only_attendees: C{tuple}
+        """
+        log.debug("ImplicitProcessing - refreshing UID: '%s', Attendees: %s" % (self.uid, ", ".join(only_attendees) if only_attendees else "all"))
+        from twistedcaldav.scheduling.implicit import ImplicitScheduler
+        scheduler = ImplicitScheduler()
+        yield scheduler.refreshAllAttendeesExceptSome(
+            self.request,
+            organizer_resource,
+            exclude_attendees,
+            only_attendees=only_attendees,
+        )
+
+
+    @inlineCallbacks
+    def _doDelayedRefresh(self, attendeesToProcess):
+        """
+        Do an attendee refresh that has been delayed until after processing of the request that called it. That
+        requires that we create a new transaction to work with.
+
+        @param attendeesToProcess: list of attendees to refresh.
+        @type attendeesToProcess: C{list}
+        """
+
+        # We need to get the UID lock for implicit processing whilst we send the auto-reply
+        # as the Organizer processing will attempt to write out data to other attendees to
+        # refresh them. To prevent a race we need a lock.
+        uidlock = MemcacheLock(
+            "ImplicitUIDLock",
+            self.uid,
+            timeout=config.Scheduling.Options.UIDLockTimeoutSeconds,
+            expire_time=config.Scheduling.Options.UIDLockExpirySeconds,
+        )
+
+        try:
+            yield uidlock.acquire()
+        except MemcacheLockTimeoutError:
+            # Just try again to get the lock
+            reactor.callLater(2.0, self._doDelayedRefresh, attendeesToProcess)
+        else:
+
+            # inNewTransaction wipes out the remembered resource<-> URL mappings in the
+            # request object but we need to be able to map the actual reply resource to its
+            # URL when doing auto-processing, so we have to sneak that mapping back in here.
+            txn = yield self.organizer_calendar_resource.inNewTransaction(self.request, label="Delayed attendee refresh")
+
+            try:
+                organizer_resource = (yield self.request.locateResource(self.organizer_calendar_resource._url))
+                if organizer_resource.exists():
+                    yield self._doRefresh(organizer_resource, only_attendees=attendeesToProcess)
+                else:
+                    log.debug("ImplicitProcessing - skipping refresh of missing UID: '%s'" % (self.uid,))
+            except Exception, e:
+                log.debug("ImplicitProcessing - refresh exception UID: '%s', %s" % (self.uid, str(e)))
+                yield txn.abort()
+            except:
+                log.debug("ImplicitProcessing - refresh bare exception UID: '%s'" % (self.uid,))
+                yield txn.abort()
+            else:
+                yield txn.commit()
+        finally:
+            yield uidlock.clean()
+
+
+    def _enqueueBatchRefresh(self):
+        """
+        Mostly here to help unit test by being able to stub this out.
+        """
+        reactor.callLater(config.Scheduling.Options.AttendeeRefreshBatchDelaySeconds, self._doBatchRefresh)
+
+
+    @inlineCallbacks
+    def _doBatchRefresh(self):
+        """
+        Do refresh of attendees in batches until the batch list is empty.
+        """
+
+        # Need to lock whilst manipulating the batch list
+        log.debug("ImplicitProcessing - batch refresh for UID: '%s'" % (self.uid,))
+        lock = MemcacheLock(
+            "BatchRefreshUIDLock",
+            self.uid,
+            timeout=config.Scheduling.Options.UIDLockTimeoutSeconds,
+            expire_time=config.Scheduling.Options.UIDLockExpirySeconds,
+        )
+        try:
+            yield lock.acquire()
+        except MemcacheLockTimeoutError:
+            # If we could not lock then just fail the refresh - not sure what else to do
+            returnValue(None)
+
+        try:
+            # Get the batch list
+            cache = Memcacher("BatchRefreshAttendees", pickle=True)
+            pendingAttendees = yield cache.get(self.uid)
+            if pendingAttendees:
+
+                # Get the next batch of attendees to process and update the cache value or remove it if
+                # no more processing is needed
+                attendeesToProcess = pendingAttendees[:config.Scheduling.Options.AttendeeRefreshBatch]
+                pendingAttendees = pendingAttendees[config.Scheduling.Options.AttendeeRefreshBatch:]
+                if pendingAttendees:
+                    yield cache.set(self.uid, pendingAttendees)
+                else:
+                    yield cache.delete(self.uid)
+
+                # Make sure we release this here to avoid potential deadlock when grabbing the ImplicitUIDLock in the next call
+                yield lock.release()
+
+                # Now do the batch refresh
+                yield self._doDelayedRefresh(attendeesToProcess)
+
+                # Queue the next refresh if needed
+                if pendingAttendees:
+                    self._enqueueBatchRefresh()
+            else:
+                yield cache.delete(self.uid)
+                yield lock.release()
+        finally:
+            yield lock.clean()
+
+
+    @inlineCallbacks
+    def doImplicitAttendee(self):
+
+        # Locate the attendee's copy of the event if it exists.
+        yield self.getRecipientsCopy()
+        self.new_resource = self.recipient_calendar is None
+
+        # Handle new items differently than existing ones.
+        if self.new_resource and self.method == "CANCEL":
+            result = (True, True, False, None)
+        else:
+            result = (yield self.doImplicitAttendeeUpdate())
+
+        returnValue(result)
+
+
+    @inlineCallbacks
+    def doImplicitAttendeeUpdate(self):
+
+        # Do security check: ORGANZIER in iTIP MUST match existing resource value
+        if self.recipient_calendar:
+            existing_organizer = self.recipient_calendar.getOrganizer()
+            existing_organizer = normalizeCUAddr(existing_organizer) if existing_organizer else ""
+            new_organizer = normalizeCUAddr(self.message.getOrganizer())
+            new_organizer = normalizeCUAddr(new_organizer) if new_organizer else ""
+            if existing_organizer != new_organizer:
+                log.debug("ImplicitProcessing - originator '%s' to recipient '%s' ignoring UID: '%s' - organizer has no copy" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
+                raise ImplicitProcessorException("5.3;Organizer change not allowed")
+
+        # Different based on method
+        if self.method == "REQUEST":
+            result = (yield self.doImplicitAttendeeRequest())
+        elif self.method == "CANCEL":
+            result = (yield self.doImplicitAttendeeCancel())
+        elif self.method == "ADD":
+            # TODO: implement ADD
+            result = (False, False, False, None)
+        else:
+            # NB We should never get here as we will have rejected unsupported METHODs earlier.
+            result = (True, True, False, None,)
+
+        returnValue(result)
+
+
+    @inlineCallbacks
+    def doImplicitAttendeeRequest(self):
+        """
+        @return: C{tuple} of (processed, auto-processed, store inbox item, changes)
+        """
+
+        # If there is no existing copy, then look for default calendar and copy it here
+        if self.new_resource:
+
+            # Check if the incoming data has the recipient declined in all instances. In that case we will not create
+            # a new resource as chances are the recipient previously deleted the resource and we want to keep it deleted.
+            attendees = self.message.getAttendeeProperties((self.recipient.cuaddr,))
+            if all([attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") == "DECLINED" for attendee in attendees]):
+                log.debug("ImplicitProcessing - originator '%s' to recipient '%s' processing METHOD:REQUEST, UID: '%s' - ignoring all declined" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
+                returnValue((True, False, False, None,))
+
+            # Check for default calendar
+            default = (yield self.recipient.inbox.viewerHome().defaultCalendar(self.message.mainType()))
+            if default is None:
+                log.error("No default calendar for recipient: '%s'." % (self.recipient.cuaddr,))
+                raise ImplicitProcessorException(iTIPRequestStatus.NO_USER_SUPPORT)
+
+            log.debug("ImplicitProcessing - originator '%s' to recipient '%s' processing METHOD:REQUEST, UID: '%s' - new processed" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
+            new_calendar = iTipProcessing.processNewRequest(self.message, self.recipient.cuaddr, creating=True)
+
+            # Handle auto-reply behavior
+            organizer = normalizeCUAddr(self.message.getOrganizer())
+            if self.recipient.principal.canAutoSchedule(organizer=organizer):
+                # auto schedule mode can depend on who the organizer is
+                mode = self.recipient.principal.getAutoScheduleMode(organizer=organizer)
+                send_reply, store_inbox, partstat = (yield self.checkAttendeeAutoReply(new_calendar, mode))
+
+                # Only store inbox item when reply is not sent or always for users
+                store_inbox = store_inbox or self.recipient.principal.getCUType() == "INDIVIDUAL"
+            else:
+                send_reply = False
+                store_inbox = True
+
+            new_resource = (yield self.writeCalendarResource(default, None, new_calendar))
+
+            if send_reply:
+                # Track outstanding auto-reply processing
+                if not hasattr(self.txn, "auto_reply_processing_count"):
+                    self.txn.auto_reply_processing_count = 1
+                else:
+                    self.txn.auto_reply_processing_count += 1
+                reactor.callLater(2.0, self.sendAttendeeAutoReply, *(new_calendar, new_resource, partstat))
+
+            # Build the schedule-changes XML element
+            changes = customxml.ScheduleChanges(
+                customxml.DTStamp(),
+                customxml.Action(
+                    customxml.Create(),
+                ),
+            )
+            result = (True, send_reply, store_inbox, changes,)
+        else:
+            # Processing update to existing event
+            new_calendar, rids = iTipProcessing.processRequest(self.message, self.recipient_calendar, self.recipient.cuaddr)
+            if new_calendar:
+
+                # Handle auto-reply behavior
+                organizer = normalizeCUAddr(self.message.getOrganizer())
+                if self.recipient.principal.canAutoSchedule(organizer=organizer):
+                    # auto schedule mode can depend on who the organizer is
+                    mode = self.recipient.principal.getAutoScheduleMode(organizer=organizer)
+                    send_reply, store_inbox, partstat = (yield self.checkAttendeeAutoReply(new_calendar, mode))
+
+                    # Only store inbox item when reply is not sent or always for users
+                    store_inbox = store_inbox or self.recipient.principal.getCUType() == "INDIVIDUAL"
+                else:
+                    send_reply = False
+                    store_inbox = True
+
+                # Let the store know that no time-range info has changed for a refresh (assuming that
+                # no auto-accept changes were made)
+                if hasattr(self.txn, "doing_attendee_refresh") and not send_reply:
+                    new_calendar.noInstanceIndexing = True
+
+                # Update the attendee's copy of the event
+                log.debug("ImplicitProcessing - originator '%s' to recipient '%s' processing METHOD:REQUEST, UID: '%s' - updating event" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
+                new_resource = (yield self.writeCalendarResource(None, self.recipient_calendar_resource, new_calendar))
+
+                if send_reply:
+                    # Track outstanding auto-reply processing
+                    if not hasattr(self.txn, "auto_reply_processing_count"):
+                        self.txn.auto_reply_processing_count = 1
+                    else:
+                        self.txn.auto_reply_processing_count += 1
+                    reactor.callLater(2.0, self.sendAttendeeAutoReply, *(new_calendar, new_resource, partstat))
+
+                # Build the schedule-changes XML element
+                update_details = []
+                for rid, props_changed in sorted(rids.iteritems(), key=lambda x: x[0]):
+                    recurrence = []
+                    if rid == "":
+                        recurrence.append(customxml.Master())
+                    else:
+                        recurrence.append(customxml.RecurrenceID.fromString(rid))
+                    changes = []
+                    for propName, paramNames in sorted(props_changed.iteritems(), key=lambda x: x[0]):
+                        params = tuple([customxml.ChangedParameter(name=param) for param in paramNames])
+                        changes.append(customxml.ChangedProperty(*params, **{"name": propName}))
+                    recurrence.append(customxml.Changes(*changes))
+                    update_details += (customxml.Recurrence(*recurrence),)
+
+                changes = customxml.ScheduleChanges(
+                    customxml.DTStamp(),
+                    customxml.Action(
+                        customxml.Update(*update_details),
+                    ),
+                )
+
+                # Refresh from another Attendee should not have Inbox item
+                if hasattr(self.txn, "doing_attendee_refresh"):
+                    store_inbox = False
+
+                result = (True, send_reply, store_inbox, changes,)
+
+            else:
+                # Request needs to be ignored
+                log.debug("ImplicitProcessing - originator '%s' to recipient '%s' processing METHOD:REQUEST, UID: '%s' - ignoring" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
+                result = (True, True, False, None,)
+
+        returnValue(result)
+
+
+    @inlineCallbacks
+    def doImplicitAttendeeCancel(self):
+
+        # If there is no existing copy, then ignore
+        if self.recipient_calendar is None:
+            log.debug("ImplicitProcessing - originator '%s' to recipient '%s' ignoring METHOD:CANCEL, UID: '%s' - attendee has no copy" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
+            result = (True, True, None)
+        else:
+            # Need to check for auto-respond attendees. These need to suppress the inbox message
+            # if the cancel is processed. However, if the principal is a user we always force the
+            # inbox item on them even if auto-schedule is true so that they get a notification
+            # of the cancel.
+            organizer = normalizeCUAddr(self.message.getOrganizer())
+            autoprocessed = self.recipient.principal.canAutoSchedule(organizer=organizer)
+            store_inbox = not autoprocessed or self.recipient.principal.getCUType() == "INDIVIDUAL"
+
+            # Check to see if this is a cancel of the entire event
+            processed_message, delete_original, rids = iTipProcessing.processCancel(self.message, self.recipient_calendar, autoprocessing=autoprocessed)
+            if processed_message:
+                if delete_original:
+
+                    # Delete the attendee's copy of the event
+                    log.debug("ImplicitProcessing - originator '%s' to recipient '%s' processing METHOD:CANCEL, UID: '%s' - deleting entire event" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
+                    yield self.deleteCalendarResource(self.recipient_calendar_collection_uri, self.recipient_calendar_collection, self.recipient_calendar_name)
+
+                    # Build the schedule-changes XML element
+                    changes = customxml.ScheduleChanges(
+                        customxml.DTStamp(),
+                        customxml.Action(
+                            customxml.Cancel(),
+                        ),
+                    )
+                    result = (True, autoprocessed, store_inbox, changes,)
+
+                else:
+
+                    # Update the attendee's copy of the event
+                    log.debug("ImplicitProcessing - originator '%s' to recipient '%s' processing METHOD:CANCEL, UID: '%s' - updating event" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
+                    yield self.writeCalendarResource(None, self.recipient_calendar_resource, self.recipient_calendar)
+
+                    # Build the schedule-changes XML element
+                    if rids:
+                        action = customxml.Cancel(
+                            *[customxml.Recurrence(customxml.RecurrenceID.fromString(rid)) for rid in sorted(rids)]
+                        )
+                    else:
+                        action = customxml.Cancel()
+                    changes = customxml.ScheduleChanges(
+                        customxml.DTStamp(),
+                        customxml.Action(action),
+                    )
+                    result = (True, autoprocessed, store_inbox, changes)
+            else:
+                log.debug("ImplicitProcessing - originator '%s' to recipient '%s' processing METHOD:CANCEL, UID: '%s' - ignoring" % (self.originator.cuaddr, self.recipient.cuaddr, self.uid))
+                result = (True, True, False, None)
+
+        returnValue(result)
+
+
+    @inlineCallbacks
+    def sendAttendeeAutoReply(self, calendar, resource, partstat):
+        """
+        Auto-process the calendar option to generate automatic accept/decline status and
+        send a reply if needed.
+
+        @param calendar: calendar data to examine
+        @type calendar: L{Component}
+
+        @return: L{Component} for the new calendar data to write
+        """
+
+        # We need to get the UID lock for implicit processing whilst we send the auto-reply
+        # as the Organizer processing will attempt to write out data to other attendees to
+        # refresh them. To prevent a race we need a lock.
+        lock = MemcacheLock(
+            "ImplicitUIDLock",
+            calendar.resourceUID(),
+            timeout=config.Scheduling.Options.UIDLockTimeoutSeconds,
+            expire_time=config.Scheduling.Options.UIDLockExpirySeconds,
+        )
+
+        # Note that this lock also protects the request, as this request is
+        # being re-used by potentially multiple transactions and should not be
+        # used concurrency (the locateResource cache needs to be cleared each
+        # time, by inNewTransaction). -glyph
+        try:
+            yield lock.acquire()
+        except MemcacheLockTimeoutError:
+            # Just try again to get the lock
+            reactor.callLater(2.0, self.sendAttendeeAutoReply, *(calendar, resource, partstat))
+        else:
+            # inNewTransaction wipes out the remembered resource<-> URL mappings in the
+            # request object but we need to be able to map the actual reply resource to its
+            # URL when doing auto-processing, so we have to sneak that mapping back in here.
+            txn = yield resource.inNewTransaction(self.request, label="Send Attendee auto-reply")
+
+            try:
+                self.request._rememberResource(resource, resource._url)
+                # Send out a reply
+                log.debug("ImplicitProcessing - recipient '%s' processing UID: '%s' - auto-reply: %s" % (self.recipient.cuaddr, self.uid, partstat))
+                from twistedcaldav.scheduling.implicit import ImplicitScheduler
+                scheduler = ImplicitScheduler()
+                yield scheduler.sendAttendeeReply(self.request, resource, calendar, self.recipient)
+            except Exception, e:
+                log.debug("ImplicitProcessing - auto-reply exception UID: '%s', %s" % (self.uid, str(e)))
+                yield txn.abort()
+            except:
+                log.debug("ImplicitProcessing - auto-reply bare exception UID: '%s'" % (self.uid,))
+                yield txn.abort()
+            else:
+                yield txn.commit()
+        finally:
+            # This correctly gets called only after commit or abort is done
+            yield lock.clean()
+
+            # Track outstanding auto-reply processing
+            if hasattr(self.txn, "auto_reply_processing_count"):
+                self.txn.auto_reply_processing_count -= 1
+
+
+    @inlineCallbacks
+    def checkAttendeeAutoReply(self, calendar, automode):
+        """
+        Check whether a reply to the given iTIP message is needed and if so make the
+        appropriate changes to the calendar data. Changes are only made for the case
+        where the PARTSTAT of the attendee is NEEDS-ACTION - i.e., any existing state
+        is left unchanged. This allows, e.g., proxies to decline events that would
+        otherwise have been auto-accepted and those stay declined as non-schedule-change
+        updates are received.
+
+        @param calendar: the iTIP message to process
+        @type calendar: L{Component}
+        @param automode: the auto-schedule mode for the recipient
+        @type automode: C{str}
+
+        @return: C{tuple} of C{bool}, C{bool}, C{str} indicating whether changes were made, whether the inbox item
+            should be added, and the new PARTSTAT.
+        """
+
+        # First ignore the none mode
+        if automode == "none":
+            returnValue((False, True, "",))
+        elif not automode or automode == "default":
+            automode = config.Scheduling.Options.AutoSchedule.DefaultMode
+
+        log.debug("ImplicitProcessing - recipient '%s' processing UID: '%s' - checking for auto-reply with mode: %s" % (self.recipient.cuaddr, self.uid, automode,))
+
+        cuas = self.recipient.principal.calendarUserAddresses()
+
+        # First expand current one to get instances (only go 1 year into the future)
+        default_future_expansion_duration = PyCalendarDuration(days=config.Scheduling.Options.AutoSchedule.FutureFreeBusyDays)
+        expand_max = PyCalendarDateTime.getToday() + default_future_expansion_duration
+        instances = calendar.expandTimeRanges(expand_max, ignoreInvalidInstances=True)
+
+        # We are goin g to ignore auto-accept processing for anything more than a day old (actually use -2 days
+        # to add some slop to account for possible timezone offsets)
+        min_date = PyCalendarDateTime.getToday()
+        min_date.offsetDay(-2)
+        allOld = True
+
+        # Cache the current attendee partstat on the instance object for later use, and
+        # also mark whether the instance time slot would be free
+        for instance in instances.instances.itervalues():
+            attendee = instance.component.getAttendeeProperty(cuas)
+            instance.partstat = attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") if attendee else None
+            instance.free = True
+            instance.active = (instance.end > min_date)
+            if instance.active:
+                allOld = False
+
+        # If every instance is in the past we punt right here so we don't waste time on freebusy lookups etc.
+        # There will be no auto-accept and no inbox item stored (so as not to waste storage on items that will
+        # never be processed).
+        if allOld:
+            returnValue((False, False, "",))
+
+        # Extract UID from primary component as we want to ignore this one if we match it
+        # in any calendars.
+        comp = calendar.mainComponent(allow_multiple=True)
+        uid = comp.propertyValue("UID")
+
+        # Now compare each instance time-range with the index and see if there is an overlap
+        calendars = (yield self._getCalendarsToMatch())
+
+        for calURL in calendars:
+            testcal = (yield self.request.locateResource(calURL))
+
+            # Get the timezone property from the collection, and store in the query filter
+            # for use during the query itself.
+            has_prop = (yield testcal.hasProperty((caldav_namespace, "calendar-timezone"), self.request))
+            if has_prop:
+                tz = (yield testcal.readProperty((caldav_namespace, "calendar-timezone"), self.request))
+                tzinfo = tz.calendar().gettimezone()
+            else:
+                tzinfo = PyCalendarTimezone(utc=True)
+
+            # Now do search for overlapping time-range and set instance.free based
+            # on whether there is an overlap or not
+            for instance in instances.instances.itervalues():
+                if instance.partstat == "NEEDS-ACTION" and instance.free and instance.active:
+                    try:
+                        # First list is BUSY, second BUSY-TENTATIVE, third BUSY-UNAVAILABLE
+                        fbinfo = ([], [], [])
+
+                        def makeTimedUTC(dt):
+                            dt = dt.duplicate()
+                            if dt.isDateOnly():
+                                dt.setDateOnly(False)
+                                dt.setHHMMSS(0, 0, 0)
+                            if dt.floating():
+                                dt.setTimezone(tzinfo)
+                                dt.adjustToUTC()
+                            return dt
+
+                        tr = caldavxml.TimeRange(
+                            start=str(makeTimedUTC(instance.start)),
+                            end=str(makeTimedUTC(instance.end)),
+                        )
+
+                        yield report_common.generateFreeBusyInfo(self.request, testcal, fbinfo, tr, 0, uid, servertoserver=True)
+
+                        # If any fbinfo entries exist we have an overlap
+                        if len(fbinfo[0]) or len(fbinfo[1]) or len(fbinfo[2]):
+                            instance.free = False
+                    except NumberOfMatchesWithinLimits:
+                        instance.free[instance] = False
+                        log.info("Exceeded number of matches whilst trying to find free-time.")
+
+            # If everything is declined we can exit now
+            if not any([instance.free for instance in instances.instances.itervalues()]):
+                break
+
+        # Now adjust the instance.partstat currently set to "NEEDS-ACTION" to the
+        # value determined by auto-accept logic based on instance.free state. However,
+        # ignore any instance in the past - leave them as NEEDS-ACTION.
+        partstat_counts = collections.defaultdict(int)
+        for instance in instances.instances.itervalues():
+            if instance.partstat == "NEEDS-ACTION" and instance.active:
+                if automode == "accept-always":
+                    freePartstat = busyPartstat = "ACCEPTED"
+                elif automode == "decline-always":
+                    freePartstat = busyPartstat = "DECLINED"
+                else:
+                    freePartstat = "ACCEPTED" if automode in ("accept-if-free", "automatic",) else "NEEDS-ACTION"
+                    busyPartstat = "DECLINED" if automode in ("decline-if-busy", "automatic",) else "NEEDS-ACTION"
+                instance.partstat = freePartstat if instance.free else busyPartstat
+            partstat_counts[instance.partstat] += 1
+
+        if len(partstat_counts) == 0:
+            # Nothing to do
+            returnValue((False, False, "",))
+
+        elif len(partstat_counts) == 1:
+            # Do the simple case of all PARTSTATs the same separately
+            # Extract the ATTENDEE property matching current recipient from the calendar data
+            attendeeProps = calendar.getAttendeeProperties(cuas)
+            if not attendeeProps:
+                returnValue((False, False, "",))
+
+            made_changes = False
+            partstat = partstat_counts.keys()[0]
+            for component in calendar.subcomponents():
+                made_changes |= self.resetAttendeePartstat(component, cuas, partstat)
+            store_inbox = partstat == "NEEDS-ACTION"
+
+        else:
+            # Hard case: some accepted, some declined, some needs-action
+            # What we will do is mark any master instance as accepted, then mark each existing
+            # overridden instance as accepted or declined, and generate new overridden instances for
+            # any other declines.
+
+            made_changes = False
+            store_inbox = False
+            partstat = "MIXED RESPONSE"
+
+            # Default state is whichever of free or busy has most instances
+            defaultPartStat = max(partstat_counts.items(), key=lambda x: x[1])[0]
+
+            # See if there is a master component first
+            hadMasterRsvp = False
+            master = calendar.masterComponent()
+            if master:
+                attendee = master.getAttendeeProperty(cuas)
+                if attendee:
+                    hadMasterRsvp = attendee.parameterValue("RSVP", "FALSE") == "TRUE"
+                    if defaultPartStat == "NEEDS-ACTION":
+                        store_inbox = True
+                    made_changes |= self.resetAttendeePartstat(master, cuas, defaultPartStat)
+
+            # Look at expanded instances and change partstat accordingly
+            for instance in sorted(instances.instances.values(), key=lambda x: x.rid):
+
+                overridden = calendar.overriddenComponent(instance.rid)
+                if not overridden and instance.partstat == defaultPartStat:
+                    # Nothing to do as state matches the master
+                    continue
+
+                if overridden:
+                    # Change ATTENDEE property to match new state
+                    if instance.partstat == "NEEDS-ACTION" and instance.active:
+                        store_inbox = True
+                    made_changes |= self.resetAttendeePartstat(overridden, cuas, instance.partstat)
+                else:
+                    # Derive a new overridden component and change partstat. We also need to make sure we restore any RSVP
+                    # value that may have been overwritten by any change to the master itself.
+                    derived = calendar.deriveInstance(instance.rid)
+                    if derived:
+                        attendee = derived.getAttendeeProperty(cuas)
+                        if attendee:
+                            if instance.partstat == "NEEDS-ACTION" and instance.active:
+                                store_inbox = True
+                            self.resetAttendeePartstat(derived, cuas, instance.partstat, hadMasterRsvp)
+                            made_changes = True
+                            calendar.addComponent(derived)
+
+        # Fake a SCHEDULE-STATUS on the ORGANIZER property
+        if made_changes:
+            calendar.setParameterToValueForPropertyWithValue("SCHEDULE-STATUS", iTIPRequestStatus.MESSAGE_DELIVERED_CODE, "ORGANIZER", None)
+
+        returnValue((made_changes, store_inbox, partstat,))
+
+
+    def _getCalendarsToMatch(self):
+        # Determine the set of calendar URIs for a principal need to be searched.
+
+        # Find the current recipients calendar-free-busy-set
+        return self.recipient.principal.calendarFreeBusyURIs(self.request)
+
+
+    @inlineCallbacks
+    def writeCalendarResource(self, collection, resource, calendar):
+        """
+        Write out the calendar resource (iTIP) message to the specified calendar, either over-writing the named
+        resource or by creating a new one.
+
+        @param collection: the L{Calendar} for the calendar collection to store the resource in.
+        @param resource: the L{CalendarObject} for the resource name to write into, or {None} to write a new resource.
+        @param calendar: the L{Component} calendar to write.
+        @return: L{Deferred} -> L{CalDAVResource}
+        """
+
+        # Create a new name if one was not provided
+        if resource is None:
+            name = "%s-%s.ics" % (hashlib.md5(calendar.resourceUID()).hexdigest(), str(uuid.uuid4())[:8],)
+            newchild = (yield collection.createCalendarObjectWithName(name, calendar))
+        else:
+            yield resource.setComponent(calendar)
+            newchild = None
+
+        returnValue(newchild)
+
+
+    @inlineCallbacks
+    def deleteCalendarResource(self, resource):
+        """
+        Delete the calendar resource in the specified calendar.
+
+        @param collURL: the URL of the calendar collection.
+        @type name: C{str}
+        @param collection: the calendar collection to delete the resource from.
+        @type collection: L{CalDAVResource}
+        @param name: the resource name to write into, or {None} to write a new resource.
+        @type name: C{str}
+        """
+
+        yield resource._parent.removeObjectResource(resource)
+
+
+    def resetAttendeePartstat(self, component, cuas, partstat, hadRSVP=False):
+        """
+        Change the PARTSTAT on any ATTENDEE properties that match the list of calendar user
+        addresses on the component passed in. Also adjust the TRANSP property to match the
+        new PARTSTAT value.
+
+        @param component: an iCalendar component to modify
+        @type attendees: L{Component}
+        @param cuas: a list of calendar user addresses to match
+        @type attendees: C{list} or C{tuple}
+        @param partstat: new PARTSTAT to set
+        @type partstat: C{str}
+        @param hadRSVP: indicates whether RSVP should be added when changing to NEEDS-ACTION
+        @type hadRSVP: C{bool}
+
+        @return: C{True} if any change was made, C{False} otherwise
+        """
+
+        madeChanges = False
+        attendee = component.getAttendeeProperty(cuas)
+        if attendee:
+            if attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") != partstat:
+                attendee.setParameter("PARTSTAT", partstat)
+                madeChanges = True
+
+            # Always remove RSVP when a state other than NEEDS-ACTION is set - this
+            # is only an attendee change so madeChanges does not need to be changed
+            try:
+                if attendee.parameterValue("PARTSTAT", "NEEDS-ACTION") != "NEEDS-ACTION":
+                    attendee.removeParameter("RSVP")
+                elif hadRSVP:
+                    attendee.setParameter("RSVP", "TRUE")
+            except KeyError:
+                pass
+
+            # Adjust TRANSP to OPAQUE if PARTSTAT is ACCEPTED, otherwise TRANSPARENT
+            component.replaceProperty(Property("TRANSP", "OPAQUE" if partstat == "ACCEPTED" else "TRANSPARENT"))
+
+        return madeChanges
+
+
+    @inlineCallbacks
+    def doImplicitAttendeeEventFix(self, ex):
+
+        # Only certain types of exception should be handled - ones related to calendar data errors.
+        # All others should result in the scheduling response coming back as a 5.x code
+
+        if type(ex) not in (InvalidOverriddenInstanceError, HTTPError):
+            raise ImplicitProcessorException("5.1;Service unavailable")
+
+        # Check to see whether the originator is hosted on this server
+        if not self.originator.principal:
+            raise ImplicitProcessorException("5.1;Service unavailable")
+
+        # Locate the originator's copy of the event
+        calendar_resource, _ignore_name, _ignore_collection, _ignore_uri = (yield getCalendarObjectForPrincipals(self.request, self.originator.principal, self.uid))
+        if not calendar_resource:
+            raise ImplicitProcessorException("5.1;Service unavailable")
+        originator_calendar = (yield calendar_resource.iCalendarForUser(self.request))
+
+        # Get attendee's view of that
+        originator_calendar.attendeesView((self.recipient.cuaddr,))
+
+        # Locate the attendee's copy of the event if it exists.
+        recipient_resource, recipient_resource_name, recipient_collection, recipient_collection_uri = (yield getCalendarObjectForPrincipals(self.request, self.recipient.principal, self.uid))
+
+        # We only need to fix data that already exists
+        if recipient_resource:
+            if originator_calendar.mainType() != None:
+                yield self.writeCalendarResource(recipient_collection_uri, recipient_collection, recipient_resource_name, originator_calendar)
+            else:
+                yield self.deleteCalendarResource(recipient_collection_uri, recipient_collection, recipient_resource_name)
+
+        returnValue(True)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/scheduler.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/scheduler.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/scheduler.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,885 @@
+
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twisted.internet.defer import inlineCallbacks, returnValue
+from twisted.python.failure import Failure
+
+from twext.python.log import Logger, LoggingMixIn
+from twext.web2 import responsecode
+from twext.web2.http import HTTPError, Response, StatusResponse
+from twext.web2.http_headers import MimeType
+from txdav.xml import element as davxml
+from twext.web2.dav.http import messageForFailure, statusForFailure, \
+    ErrorResponse
+
+from twistedcaldav import caldavxml
+from twistedcaldav.customxml import calendarserver_namespace
+from twistedcaldav.accounting import accountingEnabled, emitAccounting
+from twistedcaldav.config import config
+from twistedcaldav.ical import Component
+from twistedcaldav.memcachelock import MemcacheLock, MemcacheLockTimeoutError
+from txdav.caldav.datastore.scheduling import addressmapping
+from txdav.caldav.datastore.scheduling.caldav.delivery import ScheduleViaCalDAV
+from txdav.caldav.datastore.scheduling.cuaddress import InvalidCalendarUser, \
+    calendarUserFromPrincipal, OtherServerCalendarUser
+from txdav.caldav.datastore.scheduling.cuaddress import LocalCalendarUser
+from txdav.caldav.datastore.scheduling.cuaddress import RemoteCalendarUser
+from txdav.caldav.datastore.scheduling.cuaddress import EmailCalendarUser
+from txdav.caldav.datastore.scheduling.cuaddress import PartitionedCalendarUser
+from txdav.caldav.datastore.scheduling.imip.delivery import ScheduleViaIMip
+from txdav.caldav.datastore.scheduling.ischedule.delivery import ScheduleViaISchedule
+from txdav.caldav.datastore.scheduling.itip import iTIPRequestStatus
+
+"""
+CalDAV/Server-to-Server scheduling behavior.
+
+This module handles the delivery of scheduling messages to organizer and attendees. The basic idea is to first
+confirm the integrity of the incoming scheduling message, check authorization. Appropriate L{DeliveryService}s
+are then used to deliver the message to attendees or organizer. Delivery responses are processed and returned.
+This takes into account partitioning and podding of users by detecting the appropriate host for a calendar
+user and then dispatching the delivery accordingly.
+
+The L{Scheduler} class defines the basic behavior for processing deliveries. Sub-classes are defined for the
+different ways a deliver can be triggered.
+
+L{CalDAVScheduler} - handles deliveries for scheduling messages originating from inside the CalDAV server
+i.e. user PUTs or POSTs.
+
+L{IScheduleScheduler} - handles deliveries for scheduling messages being POSTed to the iSchedule inbox.
+
+L{IMIPScheduler} - handles deliveries for POSTs on the iMIP inbox (coming from the mail gateway).
+
+L{DirectScheduler} - used when doing some internal processing (e.g., inbox item processing during an
+upgrade.
+
+Here is a typical flow of activity for a iTIP between users on the server:
+
+iTIP PUT request
+\
+ \_L{ImplicitScheduler}           - does CalDAV-schedule logic and sends iTIP message
+   \
+    \_L{CalDAVScheduler}          - receives iTIP message
+      \
+       \_L{ScheduleViaCalDAV}     - handles delivery of iTIP message
+         \
+          \_L{ImplicitProcessor}  - dispatches iTIP message (also auto-accept)
+            \
+             \_L{iTipProcessing}  - processes iTIP message
+
+Here is a typical flow of activity for a iTIP between an organizer on the server and an iMIP attendee:
+
+iTIP PUT request
+\
+ \_L{ImplicitScheduler}
+   \
+    \_L{CalDAVScheduler}
+      \
+       \_L{ScheduleViaIMip}
+
+Here is a typical flow of activity for a iTIP between an organizer not on the server and attendee on the server:
+
+iTIP POST on /ischedule
+\
+ \_L{IScheduleScheduler}
+   \
+    \_L{ScheduleViaCalDAV}
+      \
+       \_L{ImplicitProcessor}
+         \
+          \_L{iTipProcessing}
+
+"""
+
+__all__ = [
+    "Scheduler",
+    "RemoteScheduler",
+    "DirectScheduler",
+]
+
+
+log = Logger()
+
+class Scheduler(object):
+
+    scheduleResponse = None
+
+    errorResponse = None # The class used for generating an HTTP XML error response
+
+    errorElements = {
+        "originator-missing": (),
+        "originator-invalid": (),
+        "originator-denied": (),
+        "recipient-missing": (),
+        "recipient-invalid": (),
+        "organizer-denied": (),
+        "attendee-denied": (),
+        "invalid-calendar-data-type": (),
+        "invalid-calendar-data": (),
+        "invalid-scheduling-message": (),
+        "max-recipients": (),
+    }
+
+    def __init__(self, calendar_home, calendar_collection, resource, logItems=None, noAttendeeRefresh=False):
+
+        self.txn = calendar_home.transaction()
+        self.calendar_home = calendar_home
+        self.calendar_collection = calendar_collection
+        self.resource = resource
+        self.logItems = logItems
+        self.noAttendeeRefresh = noAttendeeRefresh
+
+        self.originator = None
+        self.recipients = None
+        self.calendar = None
+        self.organizer = None
+        self.attendee = None
+        self.isiTIPRequest = None
+        self.timeRange = None
+        self.excludeUID = None
+        self.fakeTheResult = False
+        self.method = "Unknown"
+        self.internal_request = False
+
+
+    @inlineCallbacks
+    def doSchedulingViaPOST(self, transaction, use_request_headers=False):
+        """
+        The Scheduling POST operation on an Outbox.
+        """
+
+        self.method = "POST"
+
+        # Load various useful bits doing some basic checks on those
+        yield self.loadCalendarFromRequest()
+
+        if use_request_headers:
+            self.loadFromRequestHeaders()
+        else:
+            yield self.loadFromRequestData()
+
+        if self.logItems is not None:
+            self.logItems["recipients"] = len(self.recipients)
+            self.logItems["cl"] = str(len(str(self.calendar)))
+
+        # Do some extra authorization checks
+        self.checkAuthorization()
+
+        # We might trigger an implicit scheduling operation here that will require consistency
+        # of data for all events with the same UID. So detect this and use a lock
+        if self.calendar.resourceType() != "VFREEBUSY":
+            uid = self.calendar.resourceUID()
+            lock = MemcacheLock(
+                "ImplicitUIDLock",
+                uid,
+                timeout=config.Scheduling.Options.UIDLockTimeoutSeconds,
+                expire_time=config.Scheduling.Options.UIDLockExpirySeconds,
+            )
+
+            try:
+                yield lock.acquire()
+            except MemcacheLockTimeoutError:
+                raise HTTPError(StatusResponse(responsecode.CONFLICT, "UID: %s currently in use on the server." % (uid,)))
+            else:
+                # Release lock after commit or abort
+                transaction.postCommit(lock.clean)
+                transaction.postAbort(lock.clean)
+
+        result = (yield self.doScheduling())
+        returnValue(result)
+
+
+    def doSchedulingViaPUT(self, originator, recipients, calendar, internal_request=False, suppress_refresh=False):
+        """
+        The implicit scheduling PUT operation.
+        """
+
+        self.method = "PUT"
+
+        # Load various useful bits doing some basic checks on those
+        self.originator = originator
+        self.recipients = recipients
+        self.calendar = calendar
+        self.internal_request = internal_request
+        self.suppress_refresh = suppress_refresh
+
+        # Do some extra authorization checks
+        self.checkAuthorization()
+
+        return self.doScheduling()
+
+
+    @inlineCallbacks
+    def doScheduling(self):
+        # Check validity of Originator header.
+        yield self.checkOriginator()
+
+        # Get recipient details.
+        yield self.checkRecipients()
+
+        # Check calendar data.
+        self.checkCalendarData()
+
+        # Check validity of ORGANIZER
+        yield self.checkOrganizer()
+
+        # Do security checks (e.g. spoofing)
+        yield self.securityChecks()
+
+        # Generate accounting information
+        self.doAccounting()
+
+        # Do some final checks after we have gathered all our information
+        self.finalChecks()
+
+        # Do scheduling tasks
+        result = (yield self.generateSchedulingResponse())
+
+        returnValue(result)
+
+
+    @inlineCallbacks
+    def loadFromRequestData(self):
+        yield self.loadOriginatorFromRequestDetails()
+        self.loadRecipientsFromCalendarData()
+
+
+    @inlineCallbacks
+    def loadOriginatorFromRequestDetails(self):
+        # Get the originator who is the authenticated user
+        originatorPrincipal = None
+        originator = ""
+        authz_principal = self.resource.currentPrincipal(self.request).children[0]
+        if isinstance(authz_principal, davxml.HRef):
+            originatorPrincipalURL = str(authz_principal)
+            if originatorPrincipalURL:
+                originatorPrincipal = (yield self.request.locateResource(originatorPrincipalURL))
+                if originatorPrincipal:
+                    # Pick the canonical CUA:
+                    originator = originatorPrincipal.canonicalCalendarUserAddress()
+
+        if not originator:
+            log.err("%s request must have Originator" % (self.method,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["originator-missing"],
+                "Missing originator",
+            ))
+        else:
+            self.originator = originator
+
+
+    def loadRecipientsFromCalendarData(self):
+
+        # Get the ATTENDEEs
+        attendees = list()
+        unique_set = set()
+        for attendee, _ignore in self.calendar.getAttendeesByInstance():
+            if attendee not in unique_set:
+                attendees.append(attendee)
+                unique_set.add(attendee)
+
+        if not attendees:
+            log.err("%s request must have at least one Recipient" % (self.method,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["recipient-missing"],
+                "Must have recipients",
+            ))
+        else:
+            self.recipients = list(attendees)
+
+
+    def loadFromRequestHeaders(self):
+        """
+        Load Originator and Recipient from request headers.
+        """
+        self.loadOriginatorFromRequestHeaders()
+        self.loadRecipientsFromRequestHeaders()
+
+
+    def loadOriginatorFromRequestHeaders(self):
+        # Must have Originator header
+        originator = self.request.headers.getRawHeaders("originator")
+        if originator is None or (len(originator) != 1):
+            log.err("%s request must have Originator header" % (self.method,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["originator-missing"],
+                "Missing originator",
+            ))
+        else:
+            self.originator = originator[0]
+
+
+    def loadRecipientsFromRequestHeaders(self):
+        # Get list of Recipient headers
+        rawRecipients = self.request.headers.getRawHeaders("recipient")
+        if rawRecipients is None or (len(rawRecipients) == 0):
+            log.err("%s request must have at least one Recipient header" % (self.method,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["recipient-missing"],
+                "No recipients",
+            ))
+
+        # Recipient header may be comma separated list
+        self.recipients = []
+        for rawRecipient in rawRecipients:
+            for r in rawRecipient.split(","):
+                r = r.strip()
+                if len(r):
+                    self.recipients.append(r)
+
+
+    @inlineCallbacks
+    def loadCalendarFromRequest(self):
+        # Must be content-type text/calendar
+        contentType = self.request.headers.getHeader("content-type")
+        if contentType is not None and (contentType.mediaType, contentType.mediaSubtype) != ("text", "calendar"):
+            log.err("MIME type %s not allowed in calendar collection" % (contentType,))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["invalid-calendar-data-type"],
+                "Data is not calendar data",
+            ))
+
+        # Parse the calendar object from the HTTP request stream
+        try:
+            self.calendar = (yield Component.fromIStream(self.request.stream))
+
+            self.preProcessCalendarData()
+        except:
+            # FIXME: Bare except
+            log.err("Error while handling %s: %s" % (self.method, Failure(),))
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["invalid-calendar-data"],
+                description="Can't parse calendar data"
+            ))
+
+
+    def preProcessCalendarData(self):
+        """
+        After loading calendar data from the request, do some optional processing of it. This method will be
+        overridden by those schedulers that need to do special things to the data.
+        """
+        pass
+
+
+    def checkAuthorization(self):
+        raise NotImplementedError
+
+
+    def checkOriginator(self):
+        raise NotImplementedError
+
+
+    def checkRecipients(self):
+        raise NotImplementedError
+
+
+    def checkOrganizer(self):
+        raise NotImplementedError
+
+
+    def checkOrganizerAsOriginator(self):
+        raise NotImplementedError
+
+
+    def checkAttendeeAsOriginator(self):
+        raise NotImplementedError
+
+
+    def checkCalendarData(self):
+
+        # Skip all the valid data checks for an internal request as we are going to assume all the internal
+        # request data has been generated properly.
+
+        if not self.internal_request:
+            # Must be a valid calendar
+            try:
+                self.calendar.validCalendarData()
+            except ValueError, e:
+                log.err("%s request calendar component is not valid:%s %s" % (self.method, e, self.calendar,))
+                raise HTTPError(self.errorResponse(
+                    responsecode.FORBIDDEN,
+                    self.errorElements["invalid-calendar-data"],
+                    description="Calendar component is not valid"
+                ))
+
+            # Must have a METHOD
+            if not self.calendar.isValidMethod():
+                log.err("%s request must have valid METHOD property in calendar component: %s" % (self.method, self.calendar,))
+                raise HTTPError(self.errorResponse(
+                    responsecode.FORBIDDEN,
+                    self.errorElements["invalid-scheduling-message"],
+                    description="Must have valid METHOD property"
+                ))
+
+            # Verify iTIP behavior
+            if not self.calendar.isValidITIP():
+                log.err("%s request must have a calendar component that satisfies iTIP requirements: %s" % (self.method, self.calendar,))
+                raise HTTPError(self.errorResponse(
+                    responsecode.FORBIDDEN,
+                    self.errorElements["invalid-scheduling-message"],
+                    description="Must have a calendar component that satisfies iTIP requirements"
+                ))
+
+            # X-CALENDARSERVER-ACCESS is not allowed in Outbox POSTs
+            if self.calendar.hasProperty(Component.ACCESS_PROPERTY):
+                log.err("X-CALENDARSERVER-ACCESS not allowed in a calendar component %s request: %s" % (self.method, self.calendar,))
+                raise HTTPError(self.errorResponse(
+                    responsecode.FORBIDDEN,
+                    (calendarserver_namespace, "no-access-restrictions"),
+                    "Private events cannot be scheduled",
+                ))
+
+        # Determine iTIP method mode
+        if self.calendar.propertyValue("METHOD") in ("PUBLISH", "REQUEST", "ADD", "CANCEL", "DECLINECOUNTER"):
+            self.isiTIPRequest = True
+
+        elif self.calendar.propertyValue("METHOD") in ("REPLY", "COUNTER", "REFRESH"):
+            self.isiTIPRequest = False
+
+            # Verify that there is a single ATTENDEE property
+            attendees = self.calendar.getAttendees()
+
+            # Must have only one
+            if len(attendees) != 1:
+                log.err("Wrong number of ATTENDEEs in calendar data: %s" % (str(self.calendar),))
+                raise HTTPError(self.errorResponse(
+                    responsecode.FORBIDDEN,
+                    self.errorElements["invalid-scheduling-message"],
+                    "Wrong number of attendees",
+                ))
+            self.attendee = attendees[0]
+
+        else:
+            msg = "Unknown iTIP METHOD: %s" % (self.calendar.propertyValue("METHOD"),)
+            log.err(msg)
+            raise HTTPError(self.errorResponse(
+                responsecode.FORBIDDEN,
+                self.errorElements["invalid-scheduling-message"],
+                description=msg
+            ))
+
+
+    def checkForFreeBusy(self):
+        if not hasattr(self, "isfreebusy"):
+            if (self.calendar.propertyValue("METHOD") == "REQUEST") and (self.calendar.mainType() == "VFREEBUSY"):
+                # Extract time range from VFREEBUSY object
+                vfreebusies = [v for v in self.calendar.subcomponents() if v.name() == "VFREEBUSY"]
+                if len(vfreebusies) != 1:
+                    log.err("iTIP data is not valid for a VFREEBUSY request: %s" % (self.calendar,))
+                    raise HTTPError(self.errorResponse(
+                        responsecode.FORBIDDEN,
+                        self.errorElements["invalid-scheduling-message"],
+                        "iTIP data is not valid for a VFREEBUSY request",
+                    ))
+                dtstart = vfreebusies[0].getStartDateUTC()
+                dtend = vfreebusies[0].getEndDateUTC()
+                if dtstart is None or dtend is None:
+                    log.err("VFREEBUSY start/end not valid: %s" % (self.calendar,))
+                    raise HTTPError(self.errorResponse(
+                        responsecode.FORBIDDEN,
+                        self.errorElements["invalid-scheduling-message"],
+                        "VFREEBUSY start/end not valid",
+                    ))
+
+                # Some clients send floating instead of UTC - coerce to UTC
+                if not dtstart.utc() or not dtend.utc():
+                    log.err("VFREEBUSY start or end not UTC: %s" % (self.calendar,))
+                    raise HTTPError(self.errorResponse(
+                        responsecode.FORBIDDEN,
+                        self.errorElements["invalid-scheduling-message"],
+                        "VFREEBUSY start or end not UTC",
+                    ))
+
+                self.timeRange = caldavxml.TimeRange(start=dtstart.getText(), end=dtend.getText())
+                self.timeRange.start = dtstart
+                self.timeRange.end = dtend
+
+                # Look for masked UID
+                self.excludeUID = self.calendar.getMaskUID()
+
+                # Do free busy operation
+                self.isfreebusy = True
+            else:
+                # Do regular invite (fan-out)
+                self.isfreebusy = False
+
+        return self.isfreebusy
+
+
+    def securityChecks(self):
+        raise NotImplementedError
+
+
+    def doAccounting(self):
+        #
+        # Accounting
+        #
+        # Note that we associate logging with the organizer, not the
+        # originator, which is good for looking for why something
+        # shows up in a given principal's calendars, rather than
+        # tracking the activities of a specific user.
+        #
+        if isinstance(self.organizer, LocalCalendarUser):
+            accountingType = "iTIP-VFREEBUSY" if self.calendar.mainType() == "VFREEBUSY" else "iTIP"
+            if accountingEnabled(accountingType, self.organizer.principal):
+                emitAccounting(
+                    accountingType, self.organizer.principal,
+                    "Originator: %s\nRecipients:\n%sMethod:%s\n\n%s"
+                    % (
+                        str(self.originator),
+                        str("".join(["    %s\n" % (recipient,) for recipient in self.recipients])),
+                        str(self.method),
+                        str(self.calendar),
+                    )
+                )
+
+
+    def finalChecks(self):
+        """
+        Final checks before doing the actual scheduling.
+        """
+        pass
+
+
+    @inlineCallbacks
+    def generateSchedulingResponse(self):
+
+        log.info("METHOD: %s, Component: %s" % (self.calendar.propertyValue("METHOD"), self.calendar.mainType(),))
+
+        # For free-busy do immediate determination of iTIP result rather than fan-out
+        freebusy = self.checkForFreeBusy()
+
+        # Prepare for multiple responses
+        responses = self.scheduleResponse(self.method, responsecode.OK)
+
+        # Loop over each recipient and aggregate into lists by service types.
+        caldav_recipients = []
+        partitioned_recipients = []
+        otherserver_recipients = []
+        remote_recipients = []
+        imip_recipients = []
+        for ctr, recipient in enumerate(self.recipients):
+
+            # Check for freebusy limit
+            if freebusy and config.Scheduling.Options.LimitFreeBusyAttendees and ctr >= config.Scheduling.Options.LimitFreeBusyAttendees:
+                err = HTTPError(self.errorResponse(
+                    responsecode.NOT_FOUND,
+                    self.errorElements["max-recipients"],
+                    "Too many attendees",
+                ))
+                responses.add(recipient.cuaddr, Failure(exc_value=err), reqstatus=iTIPRequestStatus.SERVICE_UNAVAILABLE)
+                continue
+
+            if self.fakeTheResult:
+                responses.add(recipient.cuaddr, responsecode.OK, reqstatus=iTIPRequestStatus.SUCCESS if freebusy else iTIPRequestStatus.MESSAGE_DELIVERED)
+
+            elif isinstance(recipient, LocalCalendarUser):
+                caldav_recipients.append(recipient)
+
+            elif isinstance(recipient, PartitionedCalendarUser):
+                partitioned_recipients.append(recipient)
+
+            elif isinstance(recipient, OtherServerCalendarUser):
+                otherserver_recipients.append(recipient)
+
+            elif isinstance(recipient, RemoteCalendarUser):
+                remote_recipients.append(recipient)
+
+            elif isinstance(recipient, EmailCalendarUser):
+                imip_recipients.append(recipient)
+
+            else:
+                err = HTTPError(self.errorResponse(
+                    responsecode.NOT_FOUND,
+                    self.errorElements["recipient-invalid"],
+                    "Unknown recipient",
+                ))
+                responses.add(recipient.cuaddr, Failure(exc_value=err), reqstatus=iTIPRequestStatus.INVALID_CALENDAR_USER)
+
+        # Now process local recipients
+        if caldav_recipients:
+            yield self.generateLocalSchedulingResponses(caldav_recipients, responses, freebusy)
+
+        # Now process partitioned recipients
+        if partitioned_recipients:
+            yield self.generateRemoteSchedulingResponses(partitioned_recipients, responses, freebusy, getattr(self.request, 'doing_attendee_refresh', False))
+
+        # Now process other server recipients
+        if otherserver_recipients:
+            yield self.generateRemoteSchedulingResponses(otherserver_recipients, responses, freebusy, getattr(self.request, 'doing_attendee_refresh', False))
+
+        # To reduce chatter, we suppress certain messages
+        if not self.suppress_refresh:
+
+            # Now process remote recipients
+            if remote_recipients:
+                yield self.generateRemoteSchedulingResponses(remote_recipients, responses, freebusy)
+
+            # Now process iMIP recipients
+            if imip_recipients:
+                yield self.generateIMIPSchedulingResponses(imip_recipients, responses, freebusy)
+
+        # Return with final response if we are done
+        returnValue(responses)
+
+
+    def generateLocalSchedulingResponses(self, recipients, responses, freebusy):
+        """
+        Generate scheduling responses for CalDAV recipients.
+        """
+
+        # Create the scheduler and run it.
+        requestor = ScheduleViaCalDAV(self, recipients, responses, freebusy)
+        return requestor.generateSchedulingResponses()
+
+
+    def generateRemoteSchedulingResponses(self, recipients, responses, freebusy, refreshOnly=False):
+        """
+        Generate scheduling responses for remote recipients.
+        """
+
+        # Create the scheduler and run it.
+        requestor = ScheduleViaISchedule(self, recipients, responses, freebusy)
+        return requestor.generateSchedulingResponses(refreshOnly)
+
+
+    def generateIMIPSchedulingResponses(self, recipients, responses, freebusy):
+        """
+        Generate scheduling responses for iMIP recipients.
+        """
+
+        # Create the scheduler and run it.
+        requestor = ScheduleViaIMip(self, recipients, responses, freebusy)
+        return requestor.generateSchedulingResponses()
+
+
+
+class RemoteScheduler(Scheduler):
+
+    def checkOrganizer(self):
+        """
+        Delay ORGANIZER check until we know what their role is.
+        """
+        pass
+
+
+    @inlineCallbacks
+    def checkRecipients(self):
+        """
+        Check the validity of the Recipient header values. These must all be local as there
+        is no concept of server-to-server relaying.
+        """
+
+        results = []
+        for recipient in self.recipients:
+            # Get the principal resource for this recipient
+            principal = self.resource.principalForCalendarUserAddress(recipient)
+
+            # If no principal we may have a remote recipient but we should check whether
+            # the address is one that ought to be on our server and treat that as a missing
+            # user. Also if server-to-server is not enabled then remote addresses are not allowed.
+            if principal is None:
+                localUser = (yield addressmapping.mapper.isCalendarUserInMyDomain(recipient))
+                if localUser:
+                    log.err("No principal for calendar user address: %s" % (recipient,))
+                else:
+                    log.err("Unknown calendar user address: %s" % (recipient,))
+                results.append(InvalidCalendarUser(recipient))
+            else:
+                # Map recipient to their inbox
+                inboxURL = principal.scheduleInboxURL()
+                inbox = (yield self.request.locateResource(inboxURL)) if principal.locallyHosted() else "dummy"
+
+                if inbox:
+                    results.append(calendarUserFromPrincipal(recipient, principal, inbox, inboxURL))
+                else:
+                    log.err("No schedule inbox for principal: %s" % (principal,))
+                    results.append(InvalidCalendarUser(recipient))
+
+        self.recipients = results
+
+
+
+class DirectScheduler(Scheduler):
+    """ An implicit scheduler meant for use by local processes which don't
+        need to go through all these checks. """
+
+    errorResponse = ErrorResponse
+
+    def checkAuthorization(self):
+        pass
+
+
+    def checkOrganizer(self):
+        pass
+
+
+    def checkOrganizerAsOriginator(self):
+        pass
+
+
+    def checkAttendeeAsOriginator(self):
+        pass
+
+
+    def securityChecks(self):
+        pass
+
+
+    def checkOriginator(self):
+        pass
+
+
+    def checkRecipients(self):
+        pass
+
+
+
+class ScheduleResponseResponse (Response):
+    """
+    ScheduleResponse L{Response} object.
+    Renders itself as a CalDAV:schedule-response XML document.
+    """
+    def __init__(self, schedule_response_element, xml_responses, location=None):
+        """
+        @param xml_responses: an iterable of davxml.Response objects.
+        @param location:      the value of the location header to return in the response,
+            or None.
+        """
+
+        Response.__init__(self, code=responsecode.OK,
+                          stream=schedule_response_element(*xml_responses).toxml())
+
+        self.headers.setHeader("content-type", MimeType("text", "xml"))
+
+        if location is not None:
+            self.headers.setHeader("location", location)
+
+
+
+class ScheduleResponseQueue (LoggingMixIn):
+    """
+    Stores a list of (typically error) responses for use in a
+    L{ScheduleResponse}.
+    """
+
+    schedule_response_element = caldavxml.ScheduleResponse
+    response_element = caldavxml.Response
+    recipient_element = caldavxml.Recipient
+    recipient_uses_href = True
+    request_status_element = caldavxml.RequestStatus
+    error_element = davxml.Error
+    response_description_element = davxml.ResponseDescription
+    calendar_data_element = caldavxml.CalendarData
+
+    def __init__(self, method, success_response):
+        """
+        @param method: the name of the method generating the queue.
+        @param success_response: the response to return in lieu of a
+            L{ScheduleResponse} if no responses are added to this queue.
+        """
+        self.responses = []
+        self.method = method
+        self.success_response = success_response
+        self.location = None
+
+
+    def setLocation(self, location):
+        """
+        @param location:      the value of the location header to return in the response,
+            or None.
+        """
+        self.location = location
+
+
+    def add(self, recipient, what, reqstatus=None, calendar=None, suppressErrorLog=False):
+        """
+        Add a response.
+        @param recipient: the recipient for this response.
+        @param what: a status code or a L{Failure} for the given recipient.
+        @param status: the iTIP request-status for the given recipient.
+        @param calendar: the calendar data for the given recipient response.
+        @param suppressErrorLog: whether to suppress a log message for errors; primarily
+            this is used when trying to process a VFREEBUSY over iMIP, which isn't
+            supported.
+        """
+        if type(what) is int:
+            code = what
+            error = None
+            message = responsecode.RESPONSES[code]
+        elif isinstance(what, Failure):
+            code = statusForFailure(what)
+            error = self.errorForFailure(what)
+            message = messageForFailure(what)
+        else:
+            raise AssertionError("Unknown data type: %r" % (what,))
+
+        if not suppressErrorLog and code > 400: # Error codes only
+            self.log_error("Error during %s for %s: %s" % (self.method, recipient, message))
+
+        children = []
+        children.append(self.recipient_element(davxml.HRef.fromString(recipient)) if self.recipient_uses_href else self.recipient_element.fromString(recipient))
+        children.append(self.request_status_element(reqstatus))
+        if calendar is not None:
+            children.append(self.calendar_data_element.fromCalendar(calendar))
+        if error is not None:
+            children.append(error)
+        if message is not None:
+            children.append(self.response_description_element(message))
+        self.responses.append(self.response_element(*children))
+
+
+    def errorForFailure(self, failure):
+        if failure.check(HTTPError) and isinstance(failure.value.response, ErrorResponse):
+            return self.error_element(failure.value.response.error)
+        else:
+            return None
+
+
+    def clone(self, recipient, request_status, calendar_data, error, desc):
+        """
+        Add a response cloned from existing data.
+        @param clone: the response to clone.
+        """
+
+        children = []
+        children.append(self.recipient_element(davxml.HRef.fromString(recipient)) if self.recipient_uses_href else self.recipient_element.fromString(recipient))
+        children.append(self.request_status_element.fromString(request_status))
+        if calendar_data is not None:
+            children.append(self.calendar_data_element.fromCalendar(calendar_data))
+        if error is not None:
+            children.append(self.error_element(*error))
+        if desc is not None:
+            children.append(self.response_description_element.fromString(desc))
+        self.responses.append(self.response_element(*children))
+
+
+    def response(self):
+        """
+        Generate a L{ScheduleResponseResponse} with the responses contained in the
+        queue or, if no such responses, return the C{success_response} provided
+        to L{__init__}.
+        @return: the response.
+        """
+        if self.responses:
+            return ScheduleResponseResponse(self.schedule_response_element, self.responses, self.location)
+        else:
+            return self.success_response

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/__init__.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/__init__.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/__init__.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,15 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/accounts.xml
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/accounts.xml	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/accounts.xml	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,32 @@
+<?xml version="1.0" encoding="utf-8"?>
+
+<!--
+Copyright (c) 2013 Apple Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+ -->
+
+<!DOCTYPE accounts SYSTEM "../../../conf/auth/accounts.dtd">
+
+<accounts realm="Test">
+  <user repeat="99">
+    <uid>user%02d</uid>
+    <uid>User %02d</uid>
+    <guid>user%02d</guid>
+    <password>user%02d</password>
+    <name>User %02d</name>
+    <first-name>User</first-name>
+    <last-name>%02d</last-name>
+    <email-address>user%02d at example.com</email-address>
+  </user>
+</accounts>

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/resources.xml
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/resources.xml	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/resources.xml	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,4 @@
+<?xml version="1.0" encoding="utf-8"?>
+
+<accounts realm="/Search">
+</accounts>


Property changes on: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/resources.xml
___________________________________________________________________
Added: svn:executable
   + *

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_icaldiff.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_icaldiff.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_icaldiff.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,5082 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from difflib import unified_diff
+
+from twistedcaldav.config import config
+from twistedcaldav.ical import Component
+import twistedcaldav.test.util
+
+from txdav.caldav.datastore.scheduling.icaldiff import iCalDiff
+
+import itertools
+import re
+
+
+class ICalDiff (twistedcaldav.test.util.TestCase):
+    """
+    iCalendar support tests
+    """
+
+    def test_organizer_diff(self):
+
+        data = (
+            (
+                "#1.1 Simple component, no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "#1.2 Simple component, one property change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                False,
+            ),
+            (
+                "#1.3 Simple component, one parameter change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01 Again":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                False,
+            ),
+            (
+                "#1.4 Simple component, no change, re-ordered",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "#1.5 Simple component, no change, re-ordered, added alarm",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "#1.6 Simple component, no change, re-ordered, removed alarm",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "#1.7 Simple component, no change, re-ordered, changed alarm",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT15M
+END:VALARM
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "#2.1 Recurring component, no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REPLY
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REPLY
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "#2.2 Recurring component, one change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REPLY
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REPLY
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user3 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                False,
+            ),
+            (
+                "#2.3 Recurring component, no change, add alarm",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REPLY
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REPLY
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "#2.4 Recurring component, one change, remove component",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REPLY
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REPLY
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                False,
+            ),
+        )
+
+        for description, calendar1, calendar2, result in data:
+            differ = iCalDiff(Component.fromString(calendar1), Component.fromString(calendar2), False)
+            self.assertEqual(differ.organizerDiff(), result, msg=description)
+
+
+    def test_attendee_merge_simple(self):
+
+        data = (
+            (
+                "#1.1 Simple component, no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.2 Simple component, PARTSTAT change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ("",), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXXZ:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.3 Simple component, bad change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T123000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.4 Simple component, valarm change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test for Organizer
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test for Attendee
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test for Attendee
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.5 Simple component, vcalendar props change ok",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test for Organizer
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//EXAMPLE.ORG//NONSGML Version 1//EN
+CALSCALE:GREGORIAN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test for Attendee
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test for Attendee
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.6 Simple component, vcalendar props change bad",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test for Organizer
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+X-WR-CALID:12345
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test for Attendee
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test for Attendee
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.7 Simple component, vtimezone no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VTIMEZONE
+LAST-MODIFIED:20040110T032845Z
+TZID:US-Eastern
+BEGIN:DAYLIGHT
+DTSTART:19900404T010000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4
+TZNAME:EDT
+TZOFFSETFROM:-0500
+TZOFFSETTO:-0400
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:19901026T060000
+RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
+TZNAME:EST
+TZOFFSETFROM:-0400
+TZOFFSETTO:-0500
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;TZID=US-Eastern:20080601T120000
+DTEND;TZID=US-Eastern:20080601T130000
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;TZID=US-Eastern:20080601T120000
+DTEND;TZID=US-Eastern:20080601T130000
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+BEGIN:VTIMEZONE
+LAST-MODIFIED:20040110T032845Z
+TZID:US-Eastern
+BEGIN:DAYLIGHT
+DTSTART:19900404T010000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4
+TZNAME:EDT
+TZOFFSETFROM:-0500
+TZOFFSETTO:-0400
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:19901026T060000
+RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
+TZNAME:EST
+TZOFFSETFROM:-0400
+TZOFFSETTO:-0500
+END:STANDARD
+END:VTIMEZONE
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VTIMEZONE
+TZID:US-Eastern
+LAST-MODIFIED:20040110T032845Z
+BEGIN:DAYLIGHT
+DTSTART:19900404T010000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4
+TZNAME:EDT
+TZOFFSETFROM:-0500
+TZOFFSETTO:-0400
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:19901026T060000
+RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
+TZNAME:EST
+TZOFFSETFROM:-0400
+TZOFFSETTO:-0500
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;TZID=US-Eastern:20080601T120000
+DTEND;TZID=US-Eastern:20080601T130000
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.8 Simple component, vtimezone bad change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VTIMEZONE
+LAST-MODIFIED:20040110T032845Z
+TZID:US-Eastern
+BEGIN:DAYLIGHT
+DTSTART:19900404T010000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4
+TZNAME:EDT
+TZOFFSETFROM:-0500
+TZOFFSETTO:-0400
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:19901026T060000
+RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
+TZNAME:EST
+TZOFFSETFROM:-0400
+TZOFFSETTO:-0500
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;TZID=US-Eastern:20080601T120000
+DTEND;TZID=US-Eastern:20080601T130000
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;TZID=US-Pacific:20080601T120000
+DTEND;TZID=US-Pacific:20080601T130000
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+BEGIN:VTIMEZONE
+LAST-MODIFIED:20040110T032845Z
+TZID:US-Pacific
+BEGIN:DAYLIGHT
+DTSTART:19900404T010000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4
+TZNAME:EDT
+TZOFFSETFROM:-0800
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:19901026T060000
+RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
+TZNAME:EST
+TZOFFSETFROM:-0700
+TZOFFSETTO:-0800
+END:STANDARD
+END:VTIMEZONE
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VTIMEZONE
+TZID:US-Eastern
+LAST-MODIFIED:20040110T032845Z
+BEGIN:DAYLIGHT
+DTSTART:19900404T010000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4
+TZNAME:EDT
+TZOFFSETFROM:-0500
+TZOFFSETTO:-0400
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:19901026T060000
+RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
+TZNAME:EST
+TZOFFSETFROM:-0400
+TZOFFSETTO:-0500
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;TZID=US-Eastern:20080601T120000
+DTEND;TZID=US-Eastern:20080601T130000
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.9 Simple component, vtimezone substitute",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VTIMEZONE
+LAST-MODIFIED:20040110T032845Z
+TZID:US-Eastern
+BEGIN:DAYLIGHT
+DTSTART:19900404T010000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4
+TZNAME:EDT
+TZOFFSETFROM:-0500
+TZOFFSETTO:-0400
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:19901026T060000
+RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
+TZNAME:EST
+TZOFFSETFROM:-0400
+TZOFFSETTO:-0500
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;TZID=US-Eastern:20080601T120000
+DTEND;TZID=US-Eastern:20080601T130000
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VTIMEZONE
+LAST-MODIFIED:20040110T032845Z
+TZID:America/New_York
+BEGIN:DAYLIGHT
+DTSTART:19900404T010000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4
+TZNAME:EDT
+TZOFFSETFROM:-0500
+TZOFFSETTO:-0400
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:19901026T060000
+RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
+TZNAME:EST
+TZOFFSETFROM:-0400
+TZOFFSETTO:-0500
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;TZID=America/New_York:20080601T120000
+DTEND;TZID=America/New_York:20080601T130000
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VTIMEZONE
+TZID:US-Eastern
+LAST-MODIFIED:20040110T032845Z
+BEGIN:DAYLIGHT
+DTSTART:19900404T010000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4
+TZNAME:EDT
+TZOFFSETFROM:-0500
+TZOFFSETTO:-0400
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:19901026T060000
+RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
+TZNAME:EST
+TZOFFSETFROM:-0400
+TZOFFSETTO:-0500
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;TZID=US-Eastern:20080601T120000
+DTEND;TZID=US-Eastern:20080601T130000
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.10 Simple component, vtimezone substitute",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VTIMEZONE
+LAST-MODIFIED:20040110T032845Z
+TZID:US-Eastern
+BEGIN:DAYLIGHT
+DTSTART:19900404T010000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4
+TZNAME:EDT
+TZOFFSETFROM:-0500
+TZOFFSETTO:-0400
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:19901026T060000
+RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
+TZNAME:EST
+TZOFFSETFROM:-0400
+TZOFFSETTO:-0500
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;TZID=US-Eastern:20080601T120000
+DTEND;TZID=US-Eastern:20080601T130000
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VTIMEZONE
+LAST-MODIFIED:20040110T032845Z
+TZID:America/Los_Angeles
+BEGIN:DAYLIGHT
+DTSTART:19900404T010000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4
+TZNAME:EDT
+TZOFFSETFROM:-0800
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:19901026T060000
+RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
+TZNAME:EST
+TZOFFSETFROM:-0700
+TZOFFSETTO:-0800
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;TZID=America/Los_Angeles:20080601T090000
+DTEND;TZID=America/Los_Angeles:20080601T100000
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VTIMEZONE
+TZID:US-Eastern
+LAST-MODIFIED:20040110T032845Z
+BEGIN:DAYLIGHT
+DTSTART:19900404T010000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4
+TZNAME:EDT
+TZOFFSETFROM:-0500
+TZOFFSETTO:-0400
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:19901026T060000
+RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
+TZNAME:EST
+TZOFFSETFROM:-0400
+TZOFFSETTO:-0500
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;TZID=US-Eastern:20080601T120000
+DTEND;TZID=US-Eastern:20080601T130000
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+        )
+
+        for description, calendar1, calendar2, attendee, result in data:
+            differ = iCalDiff(Component.fromString(calendar1), Component.fromString(calendar2), False)
+            diffResult = differ.attendeeMerge(attendee)
+            diffResult = (
+                diffResult[0],
+                diffResult[1],
+                tuple(sorted(diffResult[2])),
+                re.sub(
+                    "X-CALENDARSERVER-DTSTAMP=[^Z]+",
+                    "X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXX",
+                    str(diffResult[3]).replace("\r", "").replace("\n ", "")
+                ) if diffResult[3] else None,
+            )
+            self.assertEqual(diffResult, result, msg="%s: actual result: (%s)" % (description, ", ".join([str(i).replace("\r", "") for i in diffResult]),))
+
+
+    def test_attendee_merge_complex(self):
+
+        data = (
+            (
+                "#1.1 Complex component, no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=DAILY;COUNT=400
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.2 Complex component, alarm change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test for Organizer
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test for Attendee
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=DAILY;COUNT=400
+BEGIN:VALARM
+ACTION:DISPLAY
+DESCRIPTION:Test for Attendee
+TRIGGER;RELATED=START:-PT10M
+END:VALARM
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.3 Complex component, missing override",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=DAILY;COUNT=400
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.4 Complex component, additional override no change ok",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+RECURRENCE-ID:20080604T120000Z
+UID:12345-67890
+DTSTART:20080604T120000Z
+DTEND:20080604T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=DAILY;COUNT=400
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T120000Z
+DTEND:20080604T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.5 Complex component, additional override change ok",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;RSVP=TRUE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+RECURRENCE-ID:20080604T120000Z
+UID:12345-67890
+DTSTART:20080604T120000Z
+DTEND:20080604T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=DECLINED:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ("20080602T120000Z", "20080604T120000Z",), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=DAILY;COUNT=400
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;PARTSTAT=ACCEPTED;RSVP=TRUE;X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXXZ:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T120000Z
+DTEND:20080604T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED;RSVP=TRUE;X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXXZ:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.6 Complex component, additional override bad",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+RECURRENCE-ID:20080604T120000Z
+UID:12345-67890
+DTSTART:20080604T123000Z
+DTEND:20080604T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=DAILY;COUNT=400
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T123000Z
+DTEND:20080602T130000Z
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T120000Z
+DTEND:20080604T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+        )
+
+        for description, calendar1, calendar2, attendee, result in data:
+            differ = iCalDiff(Component.fromString(calendar1), Component.fromString(calendar2), False)
+            diffResult = differ.attendeeMerge(attendee)
+            diffResult = (
+                diffResult[0],
+                diffResult[1],
+                tuple(sorted(diffResult[2])),
+                re.sub(
+                    "X-CALENDARSERVER-DTSTAMP=[^Z]+",
+                    "X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXX",
+                    str(diffResult[3]).replace("\r", "").replace("\n ", "")
+                ) if diffResult[3] else None,
+            )
+            self.assertEqual(diffResult, result, msg="%s: actual result: (%s)" % (description, ", ".join([str(i).replace("\r", "") for i in diffResult]),))
+
+
+    def test_attendee_merge_exdate(self):
+
+        data = (
+            (
+                "#1.1 Single component, one EXDATE",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+EXDATE:20080604T120000Z
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ("20080604T120000Z",), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=DAILY;COUNT=400
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T120000Z
+DTEND:20080604T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+TRANSP:TRANSPARENT
+X-CALENDARSERVER-HIDDEN-INSTANCE:T
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.2 Single component, two EXDATEs",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+EXDATE:20080604T120000Z,20080605T120000Z
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ("20080604T120000Z", "20080605T120000Z",), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=DAILY;COUNT=400
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T120000Z
+DTEND:20080604T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+TRANSP:TRANSPARENT
+X-CALENDARSERVER-HIDDEN-INSTANCE:T
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080605T120000Z
+DTSTART:20080605T120000Z
+DTEND:20080605T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+TRANSP:TRANSPARENT
+X-CALENDARSERVER-HIDDEN-INSTANCE:T
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.3 Two components, one EXDATE",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+EXDATE:20080604T120000Z
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ("20080604T120000Z",), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=DAILY;COUNT=400
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+TRANSP:TRANSPARENT
+X-CALENDARSERVER-HIDDEN-INSTANCE:T
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.4 Two components, two EXDATEs",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+EXDATE:20080604T120000Z
+EXDATE:20080606T120000Z
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ("20080604T120000Z", "20080606T120000Z",), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=DAILY;COUNT=400
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+TRANSP:TRANSPARENT
+X-CALENDARSERVER-HIDDEN-INSTANCE:T
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080606T120000Z
+DTSTART:20080606T120000Z
+DTEND:20080606T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+TRANSP:TRANSPARENT
+X-CALENDARSERVER-HIDDEN-INSTANCE:T
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.5 Two components, no master, one removal",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ("20080601T120000Z",), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+TRANSP:TRANSPARENT
+X-CALENDARSERVER-HIDDEN-INSTANCE:T
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+        )
+
+        for description, calendar1, calendar2, attendee, result in data:
+            differ = iCalDiff(Component.fromString(calendar1), Component.fromString(calendar2), False)
+            diffResult = differ.attendeeMerge(attendee)
+            diffResult = (
+                diffResult[0],
+                diffResult[1],
+                tuple(sorted(diffResult[2])),
+                str(diffResult[3]).replace("\r", "") if diffResult[3] else None,
+            )
+            self.assertEqual(diffResult, result, msg="%s: actual result: (%s)" % (description, ", ".join([str(i).replace("\r", "") for i in diffResult]),))
+
+
+    def test_attendee_merge_cancelled(self):
+
+        data = (
+            (
+                "#1.1 Remove EXDATE add CANCELLED",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+EXDATE:20080604T120000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080604T120000Z
+DTEND:20080604T130000Z
+RECURRENCE-ID:20080604T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+STATUS:CANCELLED
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=DAILY;COUNT=400
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T120000Z
+DTEND:20080604T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+STATUS:CANCELLED
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.2 Removed CANCELLED add EXDATE",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080604T120000Z
+DTEND:20080604T130000Z
+RECURRENCE-ID:20080604T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+STATUS:CANCELLED
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+EXDATE:20080604T120000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+EXDATE:20080604T120000Z
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=DAILY;COUNT=400
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.3 Removed CANCELLED no master",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080604T120000Z
+DTEND:20080604T130000Z
+RECURRENCE-ID:20080604T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+STATUS:CANCELLED
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080605T120000Z
+DTEND:20080605T130000Z
+RECURRENCE-ID:20080605T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080605T120000Z
+DTEND:20080605T130000Z
+RECURRENCE-ID:20080605T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080605T120000Z
+DTSTART:20080605T120000Z
+DTEND:20080605T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.4 Two components, no master, cancelled removed",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+STATUS:CANCELLED
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.5 Add cancelled component, no master",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+STATUS:CANCELLED
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",)
+            ),
+            (
+                "#1.6 Add uncancelled component, no master",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",)
+            ),
+        )
+
+        for description, calendar1, calendar2, attendee, result in data:
+            differ = iCalDiff(Component.fromString(calendar1), Component.fromString(calendar2), False)
+            diffResult = differ.attendeeMerge(attendee)
+            diffResult = (
+                diffResult[0],
+                diffResult[1],
+                tuple(sorted(diffResult[2])),
+                str(diffResult[3]).replace("\r", "") if diffResult[3] else None,
+            )
+            self.assertEqual(diffResult, result, msg="%s: actual result: (%s)" % (description, ", ".join([str(i).replace("\r", "") for i in diffResult]),))
+
+
+    def test_attendee_merge_dropbox(self):
+
+        data = (
+            (
+                "#1.1 Remove dropbox",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTACH;VALUE=URI:http://localhost/calendars/users/dropbox/6073432E-644B-49
+ 65-B6F7-C3F08E70BBF9.dropbox/caldavd.plist
+X-APPLE-DROPBOX:/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3F08E70B
+ BF9.dropbox
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTACH:http://localhost/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3
+ F08E70BBF9.dropbox/caldavd.plist
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+X-APPLE-DROPBOX:/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3F08E70B
+ BF9.dropbox
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.2 Add dropbox",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTACH;VALUE=URI:http://localhost/calendars/users/dropbox/6073432E-644B-49
+ 65-B6F7-C3F08E70BBF9.dropbox/caldavd.plist
+X-APPLE-DROPBOX:/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3F08E70B
+ BF9.dropbox
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",)
+            ),
+            (
+                "#1.3 Different dropbox",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTACH;VALUE=URI:http://localhost/calendars/users/dropbox/6073432E-644B-49
+ 65-B6F7-C3F08E70BBF9.dropbox/caldavd.plist
+X-APPLE-DROPBOX:/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3F08E70B
+ BF9.dropbox
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTACH;VALUE=URI:http://localhost/calendars/users/dropbox/7073432E-644B-49
+ 65-B6F7-C3F08E70BBF9.dropbox/caldavd.plist
+X-APPLE-DROPBOX:/calendars/users/dropbox/7073432E-644B-4965-B6F7-C3F08E70B
+ BF9.dropbox
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTACH:http://localhost/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3
+ F08E70BBF9.dropbox/caldavd.plist
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+X-APPLE-DROPBOX:/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3F08E70B
+ BF9.dropbox
+END:VEVENT
+END:VCALENDAR
+""",)
+            ),
+            (
+                "#1.4 Change dropbox - remove ATTACH",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTACH;VALUE=URI:http://localhost/calendars/users/dropbox/6073432E-644B-49
+ 65-B6F7-C3F08E70BBF9.dropbox/caldavd.plist
+X-APPLE-DROPBOX:/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3F08E70B
+ BF9.dropbox
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+X-APPLE-DROPBOX:/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3F08E70B
+ BF9.dropbox
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+X-APPLE-DROPBOX:/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3F08E70B
+ BF9.dropbox
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.5 Change dropbox - add ATTACH",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTACH;VALUE=URI:http://localhost/calendars/users/dropbox/6073432E-644B-49
+ 65-B6F7-C3F08E70BBF9.dropbox/caldavd.plist
+X-APPLE-DROPBOX:/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3F08E70B
+ BF9.dropbox
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTACH;VALUE=URI:http://localhost/calendars/users/dropbox/6073432E-644B-49
+ 65-B6F7-C3F08E70BBF9.dropbox/caldavd.plist
+ATTACH;VALUE=URI:http://localhost/calendars/users/dropbox/6073432E-644B-49
+ 65-B6F7-C3F08E70BBF9.dropbox/caldavd-2.plist
+X-APPLE-DROPBOX:/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3F08E70B
+ BF9.dropbox
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, False, (), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTACH:http://localhost/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3
+ F08E70BBF9.dropbox/caldavd.plist
+ATTACH:http://localhost/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3
+ F08E70BBF9.dropbox/caldavd-2.plist
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+X-APPLE-DROPBOX:/calendars/users/dropbox/6073432E-644B-4965-B6F7-C3F08E70B
+ BF9.dropbox
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+        )
+
+        self.patch(config, "EnableDropBox", True)
+        for description, calendar1, calendar2, attendee, result in data:
+            differ = iCalDiff(Component.fromString(calendar1), Component.fromString(calendar2), False)
+            diffResult = differ.attendeeMerge(attendee)
+            diffResult = (
+                diffResult[0],
+                diffResult[1],
+                tuple(sorted(diffResult[2])),
+                str(diffResult[3]).replace("\r", "") if diffResult[3] else None,
+            )
+            self.assertEqual(diffResult, result, msg="%s: actual result: (%s)" % (description, ", ".join([str(i).replace("\r", "") for i in diffResult]),))
+
+
+    def test_attendee_merge_fake_master(self):
+
+        data = (
+            (
+                "#1.1 Single overridden component, partstat change - ok",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+RECURRENCE-ID:20080601T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ('20080601T120000Z',), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXXZ:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.2 Single overridden component DTSTART different, partstat change - ok",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T130000Z
+DTEND:20080601T140000Z
+RECURRENCE-ID:20080601T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T130000Z
+DTEND:20080601T140000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ('20080601T120000Z',), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T130000Z
+DTEND:20080601T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXXZ:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#1.3 Single overridden component DTSTART different, partstat change - DTSTART bad",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T130000Z
+DTEND:20080601T140000Z
+RECURRENCE-ID:20080601T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ('20080601T120000Z',), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T130000Z
+DTEND:20080601T140000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXXZ:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",)
+            ),
+            (
+                "#2.1 Two overridden components, partstat change - ok",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+RECURRENCE-ID:20080601T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+RECURRENCE-ID:20080602T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;RSVP=TRUE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ('20080601T120000Z', '20080602T120000Z',), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXXZ:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;RSVP=TRUE;X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXXZ:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#2.2 Two overridden components DTSTART different, partstat change - ok",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T140000Z
+DTEND:20080601T150000Z
+RECURRENCE-ID:20080601T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+RECURRENCE-ID:20080602T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T140000Z
+DTEND:20080601T150000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ('20080601T120000Z', '20080602T120000Z',), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T140000Z
+DTEND:20080601T150000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXXZ:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXXZ:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+            ),
+            (
+                "#2.3 Two overridden components DTSTART different, partstat change - bad DTSTART",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T140000Z
+DTEND:20080601T150000Z
+RECURRENCE-ID:20080601T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+RECURRENCE-ID:20080602T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T150000Z
+DTEND:20080601T160000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;RSVP=TRUE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (False, False, (), None,)
+            ),
+            (
+                "#3.1 Single overridden component, fake master, missing attendee",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+RECURRENCE-ID:20080601T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ('20080601T120000Z',), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXXZ:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",)
+            ),
+            (
+                "#3.2 Single overridden component, fake master, missing attendee in old",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+RECURRENCE-ID:20080601T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ('20080601T120000Z',), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXXZ:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",)
+            ),
+            (
+                "#3.3 Single overridden component, fake master, missing attendee in new",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+RECURRENCE-ID:20080601T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                "mailto:user2 at example.com",
+                (True, True, ('20080601T120000Z',), """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION;RSVP=TRUE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXXZ:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",)
+            ),
+        )
+
+        for description, calendar1, calendar2, attendee, result in data:
+            differ = iCalDiff(Component.fromString(calendar1), Component.fromString(calendar2), False)
+            diffResult = differ.attendeeMerge(attendee)
+            diffResult = (
+                diffResult[0],
+                diffResult[1],
+                tuple(sorted(diffResult[2])),
+                re.sub(
+                    "X-CALENDARSERVER-DTSTAMP=[^Z]+",
+                    "X-CALENDARSERVER-DTSTAMP=XXXXXXXXTXXXXXX",
+                    str(diffResult[3]).replace("\r", "").replace("\n ", "")
+                ) if diffResult[3] else None,
+            )
+            self.assertEqual(diffResult, result, msg="%s: actual result: (%s)" % (description, ", ".join([str(i).replace("\r", "") for i in diffResult]),))
+
+
+    def test_what_is_different(self):
+
+        data1 = (
+            (
+                "#1.1 Simple component, no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                {},
+            ),
+            (
+                "#1.2 Simple component, one property change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test1
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"SUMMARY": set(), }},
+            ),
+            (
+                "#1.3 Simple component, one property change, one addition, one removal",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+LOCATION:The Office
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test1
+DESCRIPTION:Something to do.
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"SUMMARY": set(), "LOCATION": set(), "DESCRIPTION": set(), }},
+            ),
+            (
+                "#1.4 Simple component, add attendee",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"ATTENDEE": set(), }},
+            ),
+            (
+                "#1.5 Simple component, remove attendee",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"ATTENDEE": set(), }},
+            ),
+            (
+                "#1.6 Simple component, attendee PARTSTAT only",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT="ACCEPTED":mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"ATTENDEE": set(), }},
+            ),
+            (
+                "#1.7 Simple component, attendee PARTSTAT and addition",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT="ACCEPTED":mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"ATTENDEE": set(), }},
+            ),
+            (
+                "#1.8 Simple component, attendee RSVP only",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;RSVP=TRUE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"ATTENDEE": set(), }},
+            ),
+            (
+                "#1.9 Simple component, DTSTART/DTEND VALUE",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;VALUE=DATE:20080601
+DTEND;VALUE=DATE:20080601
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"DTEND": set(("VALUE",)), "DTSTART": set(("VALUE",)), }},
+            ),
+            (
+                "#1.10 Simple component, DTSTART/DTEND TZID",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VTIMEZONE
+LAST-MODIFIED:20040110T032845Z
+TZID:US-Eastern
+BEGIN:DAYLIGHT
+DTSTART:19900404T010000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4
+TZNAME:EDT
+TZOFFSETFROM:-0500
+TZOFFSETTO:-0400
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:19901026T060000
+RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
+TZNAME:EST
+TZOFFSETFROM:-0400
+TZOFFSETTO:-0500
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;TZID=US/Eastern:20080601T120000
+DTEND;TZID=US/Eastern:20080601T130000
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VTIMEZONE
+LAST-MODIFIED:20040110T032845Z
+TZID:America/New_York
+BEGIN:DAYLIGHT
+DTSTART:19900404T010000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4
+TZNAME:EDT
+TZOFFSETFROM:-0500
+TZOFFSETTO:-0400
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:19901026T060000
+RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
+TZNAME:EST
+TZOFFSETFROM:-0400
+TZOFFSETTO:-0500
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART;TZID=America/New_York:20080601T120000
+DTEND;TZID=America/New_York:20080601T130000
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"DTEND": set(("TZID",)), "DTSTART": set(("TZID",)), }},
+            ),
+        )
+
+        data2 = (
+            (
+                "#2.1 Simple recurring component, no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                {},
+            ),
+            (
+                "#2.2 Simple component, one property change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test1
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"SUMMARY": set(), }},
+            ),
+            (
+                "#2.3 Simple component, one property change, one addition, one removal",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+LOCATION:The Office
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test1
+DESCRIPTION:Something to do.
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"SUMMARY": set(), "LOCATION": set(), "DESCRIPTION": set(), }},
+            ),
+            (
+                "#2.4 Simple component, add attendee",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"ATTENDEE": set(), }},
+            ),
+            (
+                "#2.5 Simple component, remove attendee",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"ATTENDEE": set(), }},
+            ),
+            (
+                "#2.6 Simple component, attendee PARTSTAT only",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT="ACCEPTED":mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"ATTENDEE": set(), }},
+            ),
+            (
+                "#2.7 Simple component, attendee PARTSTAT and addition",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT="ACCEPTED":mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"ATTENDEE": set(), }},
+            ),
+            (
+                "#2.8 Simple recurring component, property order change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+EXDATE:20080602T120000Z
+EXDATE:20080603T120000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+EXDATE:20080603T120000Z
+EXDATE:20080602T120000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                {},
+            ),
+        )
+
+        data3 = (
+            (
+                "#3.1 Complex recurring component, no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T130000Z
+DTEND:20080602T140000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T130000Z
+DTEND:20080602T140000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                {},
+            ),
+            (
+                "#3.2 Simple component, one property change in instance",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T130000Z
+DTEND:20080602T140000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T130000Z
+DTEND:20080602T140000Z
+SUMMARY:Test1
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                {"20080602T120000Z": {"SUMMARY": set(), }},
+            ),
+            (
+                "#3.3 Simple component, one property change in master",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T130000Z
+DTEND:20080602T140000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test1
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T130000Z
+DTEND:20080602T140000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"SUMMARY": set(), }},
+            ),
+            (
+                "#3.4 Simple component, one property change in master and instance",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T130000Z
+DTEND:20080602T140000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test1
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T130000Z
+DTEND:20080602T140000Z
+SUMMARY:Test2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"SUMMARY": set(), }, "20080602T120000Z": {"SUMMARY": set(), }},
+            ),
+            (
+                "#3.5 Simple component, different property change in master and instance",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T130000Z
+DTEND:20080602T140000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test1
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T130000Z
+DTEND:20080602T140000Z
+SUMMARY:Test
+Description:Instance
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                {"": {"SUMMARY": set()}, "20080602T120000Z": {"Description": set()}},
+            ),
+            (
+                "#3.6 Simple component, instance added no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test
+END:VEVENT
+END:VCALENDAR
+""",
+                {},
+            ),
+            (
+                "#3.7 Simple component, instance added time change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T130000Z
+DTEND:20080602T140000Z
+SUMMARY:Test
+END:VEVENT
+END:VCALENDAR
+""",
+                {"20080602T120000Z": {"DTSTART": set(), "DTEND": set(), }},
+            ),
+            (
+                "#3.8 Simple component, instance removed no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                {},
+            ),
+            (
+                "#3.9 Simple component, instance removed time change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T130000Z
+DTEND:20080602T140000Z
+SUMMARY:Test
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                {"20080602T120000Z": {"DTSTART": set(), "DTEND": set(), }},
+            ),
+        )
+
+        for description, calendar1, calendar2, rids in itertools.chain(data1, data2, data3,):
+            differ = iCalDiff(Component.fromString(calendar1), Component.fromString(calendar2), False)
+            got_rids = differ.whatIsDifferent()
+            self.assertEqual(got_rids, rids, msg="%s expected R-IDs: '%s', got: '%s'" % (description, rids, got_rids,))
+
+
+    def test_organizer_smart_merge(self):
+
+        data1 = (
+            (
+                "#1.1 Simple component, no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+            ),
+            (
+                "#1.2 Simple component, client change only",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+            ),
+            (
+                "#1.3 Simple component, server change only",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+            ),
+            (
+                "#1.4 Simple component, both change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+            ),
+            (
+                "#1.5 Simple component, incompatible change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T130000Z
+DTEND:20080601T140000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T130000Z
+DTEND:20080601T140000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+            ),
+        )
+
+        data2 = (
+            (
+                "#2.1 Simple recurring component, no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+            ),
+            (
+                "#2.2 Simple recurring component, client instance change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080603T120000Z
+DTSTART:20080603T120000Z
+DTEND:20080603T130000Z
+SUMMARY:Test - 3
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080603T120000Z
+DTSTART:20080603T120000Z
+DTEND:20080603T130000Z
+SUMMARY:Test - 3
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+            ),
+            (
+                "#2.3 Simple recurring component, server instance change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080603T120000Z
+DTSTART:20080603T120000Z
+DTEND:20080603T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080603T120000Z
+DTSTART:20080603T120000Z
+DTEND:20080603T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+            ),
+            (
+                "#2.4 Simple recurring component, both instance change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080603T120000Z
+DTSTART:20080603T120000Z
+DTEND:20080603T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T120000Z
+DTEND:20080604T130000Z
+SUMMARY:Test - 4
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T120000Z
+DTEND:20080604T130000Z
+SUMMARY:Test - 4
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080603T120000Z
+DTSTART:20080603T120000Z
+DTEND:20080603T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+            ),
+            (
+                "#2.5 Simple recurring component, both instance incompatible change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080603T120000Z
+DTSTART:20080603T120000Z
+DTEND:20080603T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+SUMMARY:Test - 4
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:COUNT=400;FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080602T120000Z
+DTSTART:20080602T120000Z
+DTEND:20080602T130000Z
+SUMMARY:Test - 2
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080604T120000Z
+DTSTART:20080604T130000Z
+DTEND:20080604T140000Z
+SUMMARY:Test - 4
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080603T120000Z
+DTSTART:20080603T120000Z
+DTEND:20080603T130000Z
+SUMMARY:Test
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+            ),
+        )
+
+        for description, calendar1, calendar2, changed_calendar in itertools.chain(data1, data2,):
+            cal1 = Component.fromString(calendar1)
+            cal2 = Component.fromString(calendar2)
+
+            differ = iCalDiff(cal1, cal2, True)
+            differ.organizerDiff()
+
+            strcal2 = str(cal2)
+            strchanged = str(Component.fromString(changed_calendar))
+            self.assertEqual(strchanged, strcal2, msg="%s mismatch:\n%s" % (description, "\n".join(unified_diff(strchanged.split("\n"), strcal2.split("\n")))))

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_implicit.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_implicit.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_implicit.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,1329 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from pycalendar.datetime import PyCalendarDateTime
+from pycalendar.timezone import PyCalendarTimezone
+
+from twext.web2 import responsecode
+from twext.web2.http import HTTPError
+
+from twisted.internet.defer import succeed, inlineCallbacks
+
+from twistedcaldav.ical import Component
+
+from txdav.caldav.datastore.scheduling.implicit import ImplicitScheduler
+from txdav.caldav.datastore.scheduling.scheduler import ScheduleResponseQueue
+
+import twistedcaldav.test.util
+from txdav.common.datastore.test.util import CommonCommonTests, buildStore, \
+    populateCalendarsFrom
+from twisted.trial.unittest import TestCase
+from twext.python.clsprop import classproperty
+from txdav.caldav.datastore.sql import CalendarPrincipal
+import hashlib
+
+class FakeScheduler(object):
+    """
+    A fake CalDAVScheduler that does nothing except track who messages were sent to.
+    """
+
+    def __init__(self, recipients):
+        self.recipients = recipients
+
+
+    def doSchedulingViaPUT(self, originator, recipients, calendar, internal_request=False):
+        self.recipients.extend(recipients)
+        return succeed(ScheduleResponseQueue("FAKE", responsecode.OK))
+
+
+
+class Implicit (twistedcaldav.test.util.TestCase):
+    """
+    iCalendar support tests
+    """
+
+    def test_removed_attendees(self):
+
+        data = (
+            (
+                "#1.1 Simple component, no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                (),
+            ),
+            (
+                "#1.2 Simple component, one removal",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                (("mailto:user2 at example.com", None),),
+            ),
+            (
+                "#1.3 Simple component, two removals",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                (
+                    ("mailto:user2 at example.com", None),
+                    ("mailto:user3 at example.com", None),
+                ),
+            ),
+            (
+                "#2.1 Simple recurring component, two removals",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+END:VCALENDAR
+""",
+                (
+                    ("mailto:user2 at example.com", None),
+                    ("mailto:user3 at example.com", None),
+                ),
+            ),
+            (
+                "#2.2 Simple recurring component, add exdate",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+EXDATE:20080801T120000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                (
+                    ("mailto:user1 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user2 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user3 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                ),
+            ),
+            (
+                "#2.3 Simple recurring component, add multiple comma exdates",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+EXDATE:20080801T120000Z,20080901T120000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                (
+                    ("mailto:user1 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user2 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user3 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user1 at example.com", PyCalendarDateTime(2008, 9, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user2 at example.com", PyCalendarDateTime(2008, 9, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user3 at example.com", PyCalendarDateTime(2008, 9, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                ),
+            ),
+            (
+                "#2.3 Simple recurring component, add multiple comma/property exdates",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+EXDATE:20080801T120000Z,20080901T120000Z
+EXDATE:20081201T120000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                (
+                    ("mailto:user1 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user2 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user3 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user1 at example.com", PyCalendarDateTime(2008, 9, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user2 at example.com", PyCalendarDateTime(2008, 9, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user3 at example.com", PyCalendarDateTime(2008, 9, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user1 at example.com", PyCalendarDateTime(2008, 12, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user2 at example.com", PyCalendarDateTime(2008, 12, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user3 at example.com", PyCalendarDateTime(2008, 12, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                ),
+            ),
+            (
+                "#3.1 Complex recurring component with same attendees, no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                (),
+            ),
+            (
+                "#3.2 Complex recurring component with same attendees, change master/override",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                (
+                    ("mailto:user3 at example.com", None),
+                    ("mailto:user3 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                ),
+            ),
+            (
+                "#3.3 Complex recurring component with same attendees, change override",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                (
+                    ("mailto:user3 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                ),
+            ),
+            (
+                "#3.4 Complex recurring component with same attendees, change master",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                (
+                    ("mailto:user3 at example.com", None),
+                ),
+            ),
+            (
+                "#3.5 Complex recurring component with same attendees, remove override - no exdate",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+END:VCALENDAR
+""",
+                (),
+            ),
+            (
+                "#3.6 Complex recurring component with same attendees, remove override - exdate",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+EXDATE:20080801T120000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                (
+                    ("mailto:user1 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user2 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user3 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                ),
+            ),
+            (
+                "#4.1 Complex recurring component with different attendees, change master/override",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user4 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                (
+                    ("mailto:user3 at example.com", None),
+                    ("mailto:user4 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                ),
+            ),
+            (
+                "#4.2 Complex recurring component with different attendees, remove override - no exdate",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user4 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+END:VCALENDAR
+""",
+                (
+                    ("mailto:user4 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                ),
+            ),
+            (
+                "#4.3 Complex recurring component with different attendees, remove override - exdate",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user4 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+RRULE:FREQ=MONTHLY
+EXDATE:20080801T120000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                (
+                    ("mailto:user1 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user2 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                    ("mailto:user4 at example.com", PyCalendarDateTime(2008, 8, 1, 12, 0, 0, tzid=PyCalendarTimezone(utc=True))),
+                ),
+            ),
+        )
+
+        for description, calendar1, calendar2, result in data:
+            scheduler = ImplicitScheduler()
+            scheduler.resource = None
+            scheduler.oldcalendar = Component.fromString(calendar1)
+            scheduler.oldAttendeesByInstance = scheduler.oldcalendar.getAttendeesByInstance(True, onlyScheduleAgentServer=True)
+            scheduler.oldInstances = set(scheduler.oldcalendar.getComponentInstances())
+            scheduler.calendar = Component.fromString(calendar2)
+            scheduler.extractCalendarData()
+            scheduler.findRemovedAttendees()
+            self.assertEqual(scheduler.cancelledAttendees, set(result), msg=description)
+
+
+    @inlineCallbacks
+    def test_process_request_excludes_includes(self):
+        """
+        Test that processRequests correctly excludes or includes the specified attendees.
+        """
+
+        data = (
+            ((), None, 3, ("mailto:user2 at example.com", "mailto:user3 at example.com", "mailto:user4 at example.com",),),
+            (("mailto:user2 at example.com",), None, 2, ("mailto:user3 at example.com", "mailto:user4 at example.com",),),
+            ((), ("mailto:user2 at example.com", "mailto:user4 at example.com",) , 2, ("mailto:user2 at example.com", "mailto:user4 at example.com",),),
+        )
+
+        calendar = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+ATTENDEE:mailto:user4 at example.com
+END:VEVENT
+END:VCALENDAR
+"""
+
+        for excludes, includes, result_count, result_set in data:
+            scheduler = ImplicitScheduler()
+            scheduler.resource = None
+            scheduler.request = None
+            scheduler.calendar = Component.fromString(calendar)
+            scheduler.state = "organizer"
+            scheduler.action = "modify"
+            scheduler.calendar_owner = None
+            scheduler.internal_request = True
+            scheduler.except_attendees = excludes
+            scheduler.only_refresh_attendees = includes
+            scheduler.changed_rids = None
+            scheduler.reinvites = None
+
+            # Get some useful information from the calendar
+            yield scheduler.extractCalendarData()
+            scheduler.organizerPrincipal = CalendarPrincipal(scheduler.organizer, scheduler.organizer)
+
+            recipients = []
+
+            def makeFakeScheduler():
+                return FakeScheduler(recipients)
+            scheduler.makeScheduler = makeFakeScheduler
+
+            count = (yield scheduler.processRequests())
+            self.assertEqual(count, result_count)
+            self.assertEqual(len(recipients), result_count)
+            self.assertEqual(set(recipients), set(result_set))
+
+
+
+class ImplicitRequests (CommonCommonTests, TestCase):
+    """
+    Test twistedcaldav.scheduyling.implicit with a Request object.
+    """
+
+    @inlineCallbacks
+    def setUp(self):
+        yield super(ImplicitRequests, self).setUp()
+        self._sqlCalendarStore = yield buildStore(self, self.notifierFactory)
+        yield self.populate()
+
+
+    @inlineCallbacks
+    def populate(self):
+        yield populateCalendarsFrom(self.requirements, self.storeUnderTest())
+        self.notifierFactory.reset()
+
+
+    @classproperty(cache=False)
+    def requirements(cls): #@NoSelf
+        return {
+        "user01": {
+            "calendar_1": {
+            },
+            "inbox": {
+            },
+        },
+        "user02": {
+            "calendar_1": {
+            },
+            "inbox": {
+            },
+        },
+    }
+
+
+    def storeUnderTest(self):
+        """
+        Create and return a L{CalendarStore} for testing.
+        """
+        return self._sqlCalendarStore
+
+
+    @inlineCallbacks
+    def test_testImplicitSchedulingPUT_ScheduleState(self):
+        """
+        Test that checkImplicitState() always returns True for any organizer, valid or not.
+        """
+
+        data = (
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                False,
+            ),
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:bogus at bogus.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE:mailto:bogus at bogus.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+        )
+
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        for calendar, result in data:
+            calendar = Component.fromString(calendar)
+
+            scheduler = ImplicitScheduler()
+            doAction, isScheduleObject = (yield scheduler.testImplicitSchedulingPUT(calendar_collection, None, calendar, False))
+            self.assertEqual(doAction, result)
+            self.assertEqual(isScheduleObject, result)
+
+
+    @inlineCallbacks
+    def test_testImplicitSchedulingPUT_FixScheduleState(self):
+        """
+        Test that testImplicitSchedulingPUT will fix an old cached schedule object state by
+        re-evaluating the calendar data.
+        """
+
+        calendarOld = Component.fromString("""BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 02":mailto:user02 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+
+        calendarNew = Component.fromString("""BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 02":mailto:user02 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calresource = (yield calendar_collection.createCalendarObjectWithName(
+            "1.ics", calendarOld
+        ))
+        calresource.isScheduleObject = False
+
+        scheduler = ImplicitScheduler()
+        try:
+            doAction, isScheduleObject = (yield scheduler.testImplicitSchedulingPUT(calendar_collection, calresource, calendarNew, False))
+        except Exception as e:
+            print e
+            self.fail("Exception must not be raised")
+        self.assertTrue(doAction)
+        self.assertTrue(isScheduleObject)
+
+
+    @inlineCallbacks
+    def test_testImplicitSchedulingPUT_NoChangeScheduleState(self):
+        """
+        Test that testImplicitSchedulingPUT will prevent attendees from changing the
+        schedule object state.
+        """
+
+        calendarOld = Component.fromString("""BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+END:VEVENT
+END:VCALENDAR
+""")
+
+        calendarNew = Component.fromString("""BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 02":mailto:user02 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+""")
+
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calresource = (yield calendar_collection.createCalendarObjectWithName(
+            "1.ics", calendarOld
+        ))
+        calresource.isScheduleObject = False
+
+        scheduler = ImplicitScheduler()
+        try:
+            yield scheduler.testImplicitSchedulingPUT(calendar_collection, calresource, calendarNew, False)
+        except HTTPError:
+            pass
+        except:
+            self.fail("HTTPError exception must be raised")
+        else:
+            self.fail("Exception must be raised")
+
+
+    @inlineCallbacks
+    def test_doImplicitScheduling_NewOrganizerEvent(self):
+        """
+        Test that doImplicitScheduling delivers scheduling messages to attendees.
+        """
+
+        data = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+"""
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar = Component.fromString(data)
+
+        scheduler = ImplicitScheduler()
+        doAction, isScheduleObject = (yield scheduler.testImplicitSchedulingPUT(calendar_collection, None, calendar, False))
+        self.assertEqual(doAction, True)
+        self.assertEqual(isScheduleObject, True)
+        yield scheduler.doImplicitScheduling()
+
+        calendar_collection2 = (yield self.calendarUnderTest(home="user02"))
+        items = (yield calendar_collection2.listCalendarObjects())
+        self.assertEqual(len(items), 1)
+        self.assertTrue(items[0].startswith(hashlib.md5("12345-67890").hexdigest()))
+        inbox2 = (yield self.calendarUnderTest(name="inbox", home="user02"))
+        items = (yield inbox2.listCalendarObjects())
+        self.assertEqual(len(items), 1)
+        self.assertTrue(items[0].startswith(hashlib.md5("12345-67890").hexdigest()))
+
+
+    @inlineCallbacks
+    def test_doImplicitScheduling_UpdateOrganizerEvent(self):
+        """
+        Test that doImplicitScheduling delivers scheduling messages to attendees.
+        """
+
+        data1 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+"""
+        data2 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T130000Z
+DTEND:20080601T140000Z
+ORGANIZER;CN="User 01":mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+"""
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar = Component.fromString(data1)
+        scheduler = ImplicitScheduler()
+        doAction, isScheduleObject = (yield scheduler.testImplicitSchedulingPUT(calendar_collection, None, calendar, False))
+        self.assertEqual(doAction, True)
+        self.assertEqual(isScheduleObject, True)
+        yield scheduler.doImplicitScheduling()
+        yield calendar_collection.createCalendarObjectWithName("test.ics", calendar)
+        yield self.commit()
+
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar_resource = (yield self.calendarObjectUnderTest(name="test.ics", home="user01",))
+        calendar2 = Component.fromString(data2)
+        scheduler = ImplicitScheduler()
+        doAction, isScheduleObject = (yield scheduler.testImplicitSchedulingPUT(calendar_collection, calendar_resource, calendar2, False))
+        self.assertEqual(doAction, True)
+        self.assertEqual(isScheduleObject, True)
+        yield scheduler.doImplicitScheduling()
+        yield self.commit()
+
+        calendar_collection2 = (yield self.calendarUnderTest(home="user02"))
+        items = (yield calendar_collection2.listCalendarObjects())
+        self.assertEqual(len(items), 1)
+        self.assertTrue(items[0].startswith(hashlib.md5("12345-67890").hexdigest()))
+        inbox2 = (yield self.calendarUnderTest(name="inbox", home="user02"))
+        items = (yield inbox2.listCalendarObjects())
+        self.assertEqual(len(items), 2)
+        self.assertTrue(items[0].startswith(hashlib.md5("12345-67890").hexdigest()))
+        self.assertTrue(items[1].startswith(hashlib.md5("12345-67890").hexdigest()))
+
+
+    @inlineCallbacks
+    def test_doImplicitScheduling_DeleteOrganizerEvent(self):
+        """
+        Test that doImplicitScheduling delivers scheduling messages to attendees.
+        """
+
+        data1 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+"""
+        data2 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T130000Z
+DTEND:20080601T140000Z
+ORGANIZER;CN="User 01":mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+"""
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar = Component.fromString(data1)
+        scheduler = ImplicitScheduler()
+        doAction, isScheduleObject = (yield scheduler.testImplicitSchedulingPUT(calendar_collection, None, calendar, False))
+        self.assertEqual(doAction, True)
+        self.assertEqual(isScheduleObject, True)
+        yield scheduler.doImplicitScheduling()
+        yield calendar_collection.createCalendarObjectWithName("test.ics", calendar)
+        yield self.commit()
+
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar_resource = (yield self.calendarObjectUnderTest(name="test.ics", home="user01",))
+        calendar_resource.isScheduleObject = True
+        calendar2 = Component.fromString(data2)
+        scheduler = ImplicitScheduler()
+        doAction, isScheduleObject = (yield scheduler.testImplicitSchedulingDELETE(calendar_collection, calendar_resource, calendar2, False))
+        self.assertEqual(doAction, True)
+        self.assertEqual(isScheduleObject, False)
+        yield scheduler.doImplicitScheduling()
+        yield self.commit()
+
+        calendar_collection2 = (yield self.calendarUnderTest(home="user02"))
+        items = (yield calendar_collection2.listCalendarObjects())
+        self.assertEqual(len(items), 1)
+        self.assertTrue(items[0].startswith(hashlib.md5("12345-67890").hexdigest()))
+        inbox2 = (yield self.calendarUnderTest(name="inbox", home="user02"))
+        items = (yield inbox2.listCalendarObjects())
+        self.assertEqual(len(items), 2)
+        self.assertTrue(items[0].startswith(hashlib.md5("12345-67890").hexdigest()))
+        self.assertTrue(items[1].startswith(hashlib.md5("12345-67890").hexdigest()))
+
+
+    @inlineCallbacks
+    def test_doImplicitScheduling_AttendeeEventNoOrganizerEvent(self):
+        """
+        Test that doImplicitScheduling handles an attendee reply with no organizer event.
+        """
+
+        data = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-no-organizer
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+"""
+        calendar_collection = (yield self.calendarUnderTest(home="user02"))
+        calendar = Component.fromString(data)
+
+        scheduler = ImplicitScheduler()
+        doAction, isScheduleObject = (yield scheduler.testImplicitSchedulingPUT(calendar_collection, None, calendar, False))
+        self.assertEqual(doAction, True)
+        self.assertEqual(isScheduleObject, True)
+        result = (yield scheduler.doImplicitScheduling())
+        self.assertEqual(result, ImplicitScheduler.STATUS_ORPHANED_EVENT)
+
+        inbox1 = (yield self.calendarUnderTest(name="inbox", home="user01"))
+        items = (yield inbox1.listCalendarObjects())
+        self.assertEqual(len(items), 0)
+
+
+    @inlineCallbacks
+    def test_doImplicitScheduling_AttendeeReply(self):
+        """
+        Test that doImplicitScheduling delivers scheduling messages to attendees who can then reply.
+        """
+
+        data1 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+"""
+        data2 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+"""
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar1 = Component.fromString(data1)
+
+        scheduler = ImplicitScheduler()
+        doAction, isScheduleObject = (yield scheduler.testImplicitSchedulingPUT(calendar_collection, None, calendar1, False))
+        self.assertEqual(doAction, True)
+        self.assertEqual(isScheduleObject, True)
+        yield scheduler.doImplicitScheduling()
+        yield calendar_collection.createCalendarObjectWithName("test.ics", calendar1)
+        yield self.commit()
+
+        calendar_collection2 = (yield self.calendarUnderTest(home="user02"))
+        items = (yield calendar_collection2.listCalendarObjects())
+        calendar_resource2 = (yield self.calendarObjectUnderTest(name=items[0], home="user02",))
+        calendar2 = Component.fromString(data2)
+
+        scheduler = ImplicitScheduler()
+        doAction, isScheduleObject = (yield scheduler.testImplicitSchedulingPUT(calendar_collection2, calendar_resource2, calendar2, False))
+        self.assertEqual(doAction, True)
+        self.assertEqual(isScheduleObject, True)
+        yield scheduler.doImplicitScheduling()
+
+        inbox1 = (yield self.calendarUnderTest(name="inbox", home="user01"))
+        items = (yield inbox1.listCalendarObjects())
+        self.assertEqual(len(items), 1)
+        yield self.commit()
+
+        calendar_resource1 = (yield self.calendarObjectUnderTest(name="test.ics", home="user01"))
+        calendar1 = (yield calendar_resource1.component())
+        self.assertTrue("PARTSTAT=ACCEPTED" in str(calendar1))

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_itip.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_itip.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_itip.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,2241 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+from __future__ import print_function
+
+from pycalendar.datetime import PyCalendarDateTime
+from pycalendar.timezone import PyCalendarTimezone
+
+from twistedcaldav.ical import Component
+
+from txdav.caldav.datastore.scheduling.itip import iTipProcessing, iTipGenerator
+
+import os
+
+import twistedcaldav.test.util
+
+class iTIPProcessing (twistedcaldav.test.util.TestCase):
+    """
+    iCalendar support tests
+    """
+
+    def test_processReply(self):
+        """
+        Test iTIPProcessing.processReply
+        """
+
+        data = (
+            (
+                "1.1 Simple Reply - non recurring",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+ORGANIZER:mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHOD:REPLY
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071115T000000Z
+DTSTAMP:20071114T000000Z
+ORGANIZER:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+ORGANIZER:mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "1.2 Simple Reply - recurring no overrides",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+ORGANIZER:mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user02 at example.com
+RRULE:FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHOD:REPLY
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071115T000000Z
+DTSTAMP:20071114T000000Z
+ORGANIZER:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user02 at example.com
+RRULE:FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+ORGANIZER:mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:user02 at example.com
+RRULE:FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "1.3 Simple Reply - recurring with missing master",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071114T000000Z
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+ORGANIZER:mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHOD:REPLY
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071115T000000Z
+DTSTAMP:20071114T000000Z
+ORGANIZER:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user02 at example.com
+RRULE:FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071114T000000Z
+DTSTART:20071115T000000Z
+DTSTAMP:20071114T000000Z
+ORGANIZER:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071114T000000Z
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+ORGANIZER:mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "1.4 Simple Reply - recurring with missing master, invalid override",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071114T000000Z
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+ORGANIZER:mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHOD:REPLY
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+ORGANIZER:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user02 at example.com
+RRULE:FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071115T000000Z
+DTSTART:20071115T000000Z
+DTSTAMP:20071114T000000Z
+ORGANIZER:mailto:user01 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                None,
+                False,
+            ),
+        )
+
+        for title, calendar_txt, itip_txt, changed_txt, expected in data:
+            calendar = Component.fromString(calendar_txt)
+            itip = Component.fromString(itip_txt)
+            if expected:
+                changed = Component.fromString(changed_txt)
+
+            result, _ignore = iTipProcessing.processReply(itip, calendar)
+            self.assertEqual(result, expected, msg="Result mismatch: %s" % (title,))
+            if expected:
+                self.assertEqual(changed, calendar, msg="Calendar mismatch: %s" % (title,))
+
+
+    def test_update_attendee_partstat(self):
+
+        data = (
+            (
+                "#1.1 Simple component, accepted",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True, "mailto:user1 at example.com", (("", True, False),),
+            ),
+            (
+                "#1.2 Simple component, accepted",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True, "mailto:user1 at example.com", (("", True, False),),
+            ),
+            (
+                "#1.3 Simple component, no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=NEEDS-ACTION;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True, "mailto:user1 at example.com", (),
+            ),
+            (
+                "#2.1 Recurring component, change master/override",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080801T123000Z
+DTEND:20080801T133000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080801T123000Z
+DTEND:20080801T133000Z
+ATTENDEE;PARTSTAT=DECLINED;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True, "mailto:user1 at example.com", (("", True, False), ("20080801T120000Z", True, False),),
+            ),
+            (
+                "#2.2 Recurring component, change master only",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080801T123000Z
+DTEND:20080801T133000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080801T123000Z
+DTEND:20080801T133000Z
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True, "mailto:user1 at example.com", (("", True, False),),
+            ),
+            (
+                "#2.3 Recurring component, change override only",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080801T123000Z
+DTEND:20080801T133000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080801T123000Z
+DTEND:20080801T133000Z
+ATTENDEE;PARTSTAT=DECLINED;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True, "mailto:user1 at example.com", (("20080801T120000Z", True, False),),
+            ),
+            (
+                "#3.1 Recurring component, change master/override, new override",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080801T123000Z
+DTEND:20080801T133000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080901T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=TENTATIVE:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080801T123000Z
+DTEND:20080801T133000Z
+ATTENDEE;PARTSTAT=DECLINED;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080901T120000Z
+DTSTART:20080901T120000Z
+DTEND:20080901T130000Z
+ATTENDEE;PARTSTAT=TENTATIVE;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True, "mailto:user1 at example.com", (("", True, False), ("20080801T120000Z", True, False), ("20080901T120000Z", True, False),),
+            ),
+            (
+                "#3.2 Recurring component, change master, new override",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080801T123000Z
+DTEND:20080801T133000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080901T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=TENTATIVE:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080801T123000Z
+DTEND:20080801T133000Z
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080901T120000Z
+DTSTART:20080901T120000Z
+DTEND:20080901T130000Z
+ATTENDEE;PARTSTAT=TENTATIVE;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True, "mailto:user1 at example.com", (("", True, False), ("20080901T120000Z", True, False),),
+            ),
+            (
+                "#3.3 Recurring component, change override, new override",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080801T123000Z
+DTEND:20080801T133000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080901T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=TENTATIVE:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080801T123000Z
+DTEND:20080801T133000Z
+ATTENDEE;PARTSTAT=DECLINED;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080901T120000Z
+DTSTART:20080901T120000Z
+DTEND:20080901T130000Z
+ATTENDEE;PARTSTAT=TENTATIVE;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True, "mailto:user1 at example.com", (("20080801T120000Z", True, False), ("20080901T120000Z", True, False),),
+            ),
+            (
+                "#4.1 Recurring component, invalid override",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:FREQ=MONTHLY
+EXDATE:20080801T120000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+EXDATE:20080801T120000Z
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+END:VCALENDAR
+""",
+                False, "", (),
+            ),
+            (
+                "#5.1 Invalid iTIP",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                False, "", (),
+            ),
+            (
+                "#5.2 Recurring component, different attendees in components",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080801T123000Z
+DTEND:20080801T133000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user1 at example.com
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+RRULE:FREQ=MONTHLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20080801T120000Z
+DTSTART:20080801T123000Z
+DTEND:20080801T133000Z
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED;SCHEDULE-STATUS=2.0:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                False, "", (),
+            ),
+            (
+                "#6.1 REQUEST-STATUS",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user1 at example.com
+REQUEST-STATUS:2.0;Success
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=DECLINED;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True, "mailto:user1 at example.com", (("", True, False),),
+            ),
+            (
+                "#6.2 Multiple REQUEST-STATUS",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=DECLINED:mailto:user1 at example.com
+REQUEST-STATUS:2.1;Success but fallback taken on one or more property values
+REQUEST-STATUS:2.2;Success, invalid property ignored
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=DECLINED;SCHEDULE-STATUS="2.1,2.2":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True, "mailto:user1 at example.com", (("", True, False),),
+            ),
+            (
+                "#6.3 Bad REQUEST-STATUS",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+METHOD:REPLY
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+ORGANIZER;CN="User 01":mailto:user1 at example.com
+ATTENDEE;PARTSTAT=ACCEPTED:mailto:user1 at example.com
+REQUEST-STATUS:2.0\;Success
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ATTENDEE;PARTSTAT=ACCEPTED;SCHEDULE-STATUS=2.0:mailto:user1 at example.com
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER;CN=User 01:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                True, "mailto:user1 at example.com", (("", True, False),),
+            ),
+        )
+
+        for description, calendar_txt, itipmsg_txt, result, success, attendee, rids in data:
+            calendar = Component.fromString(calendar_txt)
+            itipmsg = Component.fromString(itipmsg_txt)
+            reply_success, reply_processed = iTipProcessing.processReply(itipmsg, calendar)
+#            if not description.startswith("#3.1"):
+#                continue
+#            print(description)
+#            print(str(calendar))
+#            print(str(result))
+            self.assertEqual(
+                str(calendar).replace("\r", "").replace("\n ", ""),
+                str(result).replace("\n ", ""),
+                msg=description
+            )
+            self.assertEqual(
+                reply_success,
+                success,
+                msg=description
+            )
+            if success:
+                reply_attendee, reply_rids, = reply_processed
+                self.assertEqual(
+                    reply_attendee,
+                    attendee,
+                    msg=description
+                )
+                self.assertEqual(
+                    tuple(sorted(list(reply_rids), key=lambda x: x[0])),
+                    rids,
+                    msg=description
+                )
+            else:
+                self.assertEqual(
+                    reply_processed,
+                    None,
+                    msg=description
+                )
+
+
+    def test_sequenceComparison(self):
+        """
+        Test iTIPProcessing.sequenceComparison
+        """
+
+        data = (
+            (
+                "1.1 Simple Update - SEQUENCE change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHOD:REQUEST
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071115T000000Z
+DTSTAMP:20071114T000000Z
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "1.2 Simple Update - DTSTAMP change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHOD:REQUEST
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T010000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "1.3 Simple Update - no change",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHOD:REQUEST
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "2.1 Recurrence add changed SEQUENCE instance",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+RRULE:FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHOD:REQUEST
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071115T000000Z
+DTSTAMP:20071114T000000Z
+RRULE:FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T010000Z
+DTSTAMP:20071114T010000Z
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "2.2 Recurrence add changed DTSTAMP instance",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+RRULE:FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHOD:REQUEST
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071115T000000Z
+DTSTAMP:20071114T000000Z
+RRULE:FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T010000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "2.3 Recurrence add unchanged instance",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+RRULE:FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHOD:REQUEST
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071115T000000Z
+DTSTAMP:20071114T000000Z
+RRULE:FREQ=DAILY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T000000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "3.1 Recurrence master/no-master changed SEQUENCE instance",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+RRULE:FREQ=DAILY
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHOD:CANCEL
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T010000Z
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "3.2 Recurrence master/no-master old SEQUENCE instance no prior instance",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+RRULE:FREQ=DAILY
+SEQUENCE:2
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHOD:CANCEL
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T010000Z
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                False,
+            ),
+            (
+                "3.3 Recurrence master/no-master old SEQUENCE instance with prior instance",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+RRULE:FREQ=DAILY
+SEQUENCE:2
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T010000Z
+SEQUENCE:2
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHOD:CANCEL
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T010000Z
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                False,
+            ),
+            (
+                "4.1 Recurrence no-master/master changed SEQUENCE master",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T010000Z
+SEQUENCE:0
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHID:REQUEST
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+RRULE:FREQ=DAILY
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "4.2 Recurrence no-master/master changed DTSTAMP master",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T000000Z
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHID:REQUEST
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T010000Z
+RRULE:FREQ=DAILY
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "4.3 Recurrence no-master/master old SEQUENCE instance",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T010000Z
+SEQUENCE:2
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHID:REQUEST
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+RRULE:FREQ=DAILY
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                False,
+            ),
+            (
+                "4.4 Recurrence no-master/master changed SEQUENCE instance",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T010000Z
+SEQUENCE:0
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHID:REQUEST
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+DTSTAMP:20071114T000000Z
+RRULE:FREQ=DAILY
+SEQUENCE:1
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T010000Z
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "5.1 Recurrence no-masters changed SEQUENCE same instance",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T010000Z
+SEQUENCE:0
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHID:REQUEST
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T010000Z
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "5.2 Recurrence no-masters changed DTSTAMP same instance",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T010000Z
+SEQUENCE:0
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHID:REQUEST
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T020000Z
+SEQUENCE:0
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+            (
+                "5.3 Recurrence no-masters changed SEQUENCE different instances",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071116T000000Z
+DTSTART:20071116T000000Z
+DTSTAMP:20071114T010000Z
+SEQUENCE:0
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHID:REQUEST
+BEGIN:VEVENT
+UID:12345-67890-1
+RECURRENCE-ID:20071117T000000Z
+DTSTART:20071117T000000Z
+DTSTAMP:20071114T010000Z
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                True,
+            ),
+        )
+
+        for title, calendar_txt, itip_txt, expected in data:
+            calendar = Component.fromString(calendar_txt)
+            itip = Component.fromString(itip_txt)
+
+            result = iTipProcessing.sequenceComparison(itip, calendar)
+            self.assertEqual(result, expected, msg="Result mismatch: %s" % (title,))
+
+
+
+class iTIPGenerator (twistedcaldav.test.util.TestCase):
+    """
+    iCalendar support tests
+    """
+    data_dir = os.path.join(os.path.dirname(__file__), "data")
+
+    def test_request(self):
+
+        data = (
+            # Simple component, no Attendees - no filtering
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REQUEST
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                ()
+            ),
+
+            # Simple component, no Attendees - filtering
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-2
+DTSTART:20071114T000000Z
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REQUEST
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+END:VCALENDAR
+""",
+                ("mailto:user01 at example.com",)
+            ),
+
+            # Simple component, with one attendee - filtering match
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REQUEST
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                ("mailto:user2 at example.com",)
+            ),
+
+            # Simple component, with one attendee - no filtering match
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-4
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REQUEST
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+END:VCALENDAR
+""",
+                ("mailto:user3 at example.com",)
+            ),
+
+            # Recurring component with one instance, each with one attendee - filtering match
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REQUEST
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                ("mailto:user2 at example.com",)
+            ),
+
+            # Recurring component with one instance, each with one attendee - no filtering match
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-4
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REQUEST
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+END:VCALENDAR
+""",
+                ("mailto:user3 at example.com",)
+            ),
+
+            # Recurring component with one instance, master with one attendee, instance without attendee - filtering match
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REQUEST
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+EXDATE:20081114T000000Z
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+END:VEVENT
+END:VCALENDAR
+""",
+                ("mailto:user2 at example.com",)
+            ),
+
+            # Recurring component with one instance, master with one attendee, instance without attendee - no filtering match
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-4
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REQUEST
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+END:VCALENDAR
+""",
+                ("mailto:user3 at example.com",)
+            ),
+
+            # Recurring component with one instance, master without attendee, instance with attendee - filtering match
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REQUEST
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                ("mailto:user2 at example.com",)
+            ),
+
+            # Recurring component with one instance, master without attendee, instance with attendee - no filtering match
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-4
+DTSTART:20071114T000000Z
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:REQUEST
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+END:VCALENDAR
+""",
+                ("mailto:user3 at example.com",)
+            ),
+        )
+
+        for original, filtered, attendees in data:
+            component = Component.fromString(original)
+            itipped = iTipGenerator.generateAttendeeRequest(component, attendees, None)
+            itipped = str(itipped).replace("\r", "")
+            itipped = "".join([line for line in itipped.splitlines(True) if not line.startswith("DTSTAMP:")])
+            self.assertEqual(filtered, itipped)
+
+
+    def test_cancel(self):
+
+        data = (
+            # Simple component, with two attendees - cancel one
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+ORGANIZER:mailto:user1 at example.com
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:CANCEL
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-1
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+SEQUENCE:2
+END:VEVENT
+END:VCALENDAR
+""",
+                ("mailto:user2 at example.com",),
+                (None,),
+            ),
+
+            # Simple component, with two attendees - cancel two
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-2
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ATTENDEE:mailto:user3 at example.com
+ORGANIZER:mailto:user1 at example.com
+SEQUENCE:2
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:CANCEL
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-2
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user3 at example.com
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+SEQUENCE:3
+END:VEVENT
+END:VCALENDAR
+""",
+                ("mailto:user3 at example.com", "mailto:user2 at example.com",),
+                (None,)
+            ),
+
+            # Recurring component with no instance, one attendee - cancel instance
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-3
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:CANCEL
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-3
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20081114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+SEQUENCE:2
+END:VEVENT
+END:VCALENDAR
+""",
+                ("mailto:user2 at example.com",),
+                (PyCalendarDateTime(2008, 11, 14, 0, 0, 0, tzid=PyCalendarTimezone(utc=True)),),
+            ),
+
+            # Recurring component with one instance, each with one attendee - cancel instance
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-4
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+SEQUENCE:1
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-4
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20081114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:CANCEL
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-4
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20081114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+SEQUENCE:2
+END:VEVENT
+END:VCALENDAR
+""",
+                ("mailto:user2 at example.com",),
+                (PyCalendarDateTime(2008, 11, 14, 0, 0, 0, tzid=PyCalendarTimezone(utc=True)),),
+            ),
+
+            # Recurring component with one instance, each with one attendee - cancel master
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-5
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+SEQUENCE:1
+END:VEVENT
+BEGIN:VEVENT
+UID:12345-67890-5
+RECURRENCE-ID:20081114T000000Z
+DTSTART:20071114T010000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                """BEGIN:VCALENDAR
+VERSION:2.0
+METHOD:CANCEL
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-5
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=YEARLY
+SEQUENCE:2
+END:VEVENT
+END:VCALENDAR
+""",
+                ("mailto:user2 at example.com",),
+                (None,),
+            ),
+
+            # Recurring component - cancel non-existent instance
+            (
+                """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-4
+DTSTART:20071114T000000Z
+ATTENDEE:mailto:user2 at example.com
+ORGANIZER:mailto:user1 at example.com
+RRULE:FREQ=DAILY;COUNT=10
+SEQUENCE:1
+END:VEVENT
+END:VCALENDAR
+""",
+                "",
+                ("mailto:user2 at example.com",),
+                (PyCalendarDateTime(2008, 12, 14, 0, 0, 0, tzid=PyCalendarTimezone(utc=True)),),
+            ),
+
+        )
+
+        for original, filtered, attendees, instances in data:
+            component = Component.fromString(original)
+            itipped = iTipGenerator.generateCancel(component, attendees, instances)
+            itipped = str(itipped).replace("\r", "") if itipped else ""
+            itipped = "".join([line for line in itipped.splitlines(True) if not line.startswith("DTSTAMP:")])
+            self.assertEqual(filtered, itipped)
+
+
+    def test_missingAttendee(self):
+        """
+        When generating a reply, remove all components that are missing
+        the ATTENDEE
+        """
+
+        original = """BEGIN:VCALENDAR
+VERSION:2.0
+CALSCALE:GREGORIAN
+BEGIN:VTIMEZONE
+TZID:America/Los_Angeles
+BEGIN:DAYLIGHT
+TZOFFSETFROM:-0800
+RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
+DTSTART:20070311T020000
+TZNAME:PDT
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:STANDARD
+TZOFFSETFROM:-0700
+RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
+DTSTART:20071104T020000
+TZNAME:PST
+TZOFFSETTO:-0800
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:04405DDD-C938-46FC-A4CE-8573613BEA39
+DTEND;TZID=America/Los_Angeles:20100826T130000
+TRANSP:TRANSPARENT
+SUMMARY:Missing attendee in master
+DTSTART;TZID=America/Los_Angeles:20100826T130000
+DTSTAMP:20101115T160533Z
+ORGANIZER;CN="The Organizer":mailto:organizer at example.com
+SEQUENCE:0
+END:VEVENT
+BEGIN:VEVENT
+DTEND;TZID=America/Los_Angeles:20101007T120000
+TRANSP:OPAQUE
+UID:04405DDD-C938-46FC-A4CE-8573613BEA39
+DTSTAMP:20101005T213326Z
+X-APPLE-NEEDS-REPLY:TRUE
+SEQUENCE:24
+RECURRENCE-ID;TZID=America/Los_Angeles:20100826T130000
+SUMMARY:Missing attendee in master
+DTSTART;TZID=America/Los_Angeles:20101007T113000
+CREATED:20100820T235846Z
+ORGANIZER;CN="The Organizer":mailto:organizer at example.com
+ATTENDEE;CN="Attendee 1";CUTYPE=INDIVIDUAL;EMAIL="attendee1 at example.com";
+ PARTSTAT=NEEDS-ACTION;ROLE=OPT-PARTICIPANT;RSVP=TRUE:mailto:attendee1 at ex
+ ample.com
+ATTENDEE;CN="Attendee 2";CUTYPE=INDIVIDUAL;EMAIL="attendee2 at example.com";
+ PARTSTAT=NEEDS-ACTION;ROLE=OPT-PARTICIPANT;RSVP=TRUE:mailto:attendee2 at ex
+ ample.com
+ATTENDEE;CN="Missing Attendee";CUTYPE=INDIVIDUAL;EMAIL="missing at example.com";
+ PARTSTAT=NEEDS-ACTION;ROLE=OPT-PARTICIPANT;RSVP=TRUE:mailto:missing at ex
+ ample.com
+END:VEVENT
+END:VCALENDAR
+"""
+
+        filtered = """BEGIN:VCALENDAR
+VERSION:2.0
+CALSCALE:GREGORIAN
+METHOD:REPLY
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VTIMEZONE
+TZID:America/Los_Angeles
+BEGIN:DAYLIGHT
+DTSTART:20070311T020000
+RRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3
+TZNAME:PDT
+TZOFFSETFROM:-0800
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:STANDARD
+DTSTART:20071104T020000
+RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11
+TZNAME:PST
+TZOFFSETFROM:-0700
+TZOFFSETTO:-0800
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:04405DDD-C938-46FC-A4CE-8573613BEA39
+RECURRENCE-ID;TZID=America/Los_Angeles:20100826T130000
+DTSTART;TZID=America/Los_Angeles:20101007T113000
+DTEND;TZID=America/Los_Angeles:20101007T120000
+ATTENDEE;CN=Missing Attendee;CUTYPE=INDIVIDUAL;EMAIL=missing at example.com;P
+ ARTSTAT=DECLINED;ROLE=OPT-PARTICIPANT;RSVP=TRUE:mailto:missing at example.com
+ORGANIZER;CN=The Organizer:mailto:organizer at example.com
+REQUEST-STATUS:2.0;Success
+SEQUENCE:24
+SUMMARY:Missing attendee in master
+END:VEVENT
+END:VCALENDAR
+"""
+        component = Component.fromString(original)
+        itipped = iTipGenerator.generateAttendeeReply(component, "mailto:missing at example.com", force_decline=True)
+        itipped = str(itipped).replace("\r", "")
+        itipped = "".join([line for line in itipped.splitlines(True) if not line.startswith("DTSTAMP:")])
+        self.assertEqual(filtered, itipped)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_pocessing.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_pocessing.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_pocessing.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,110 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.web2.test.test_server import SimpleRequest
+
+from twisted.internet.defer import inlineCallbacks
+
+from twistedcaldav.config import config
+from twistedcaldav.ical import Component
+
+from txdav.caldav.datastore.scheduling.processing import ImplicitProcessor
+
+import twistedcaldav.test.util
+
+class FakeImplicitProcessor(ImplicitProcessor):
+    """
+    A fake ImplicitProcessor that tracks batch refreshes.
+    """
+
+    def __init__(self):
+        self.batches = 0
+
+
+    def _enqueueBatchRefresh(self):
+        self.batches += 1
+
+
+
+class FakePrincipal(object):
+
+    def __init__(self, cuaddr):
+        self.cuaddr = cuaddr
+
+
+    def calendarUserAddresses(self):
+        return (self.cuaddr,)
+
+
+
+class BatchRefresh (twistedcaldav.test.util.TestCase):
+    """
+    iCalendar support tests
+    """
+
+    @inlineCallbacks
+    def test_queueAttendeeUpdate_no_refresh(self):
+
+        self.patch(config.Scheduling.Options, "AttendeeRefreshBatch", 5)
+
+        calendar = Component.fromString("""BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER:urn:uuid:user01
+ATTENDEE:urn:uuid:user01
+ATTENDEE:urn:uuid:user02
+END:VEVENT
+END:VCALENDAR
+""")
+        request = SimpleRequest(self.site, "PUT", "/calendar/1.ics")
+        processor = FakeImplicitProcessor()
+        processor.request = request
+        processor.uid = "12345-67890"
+        processor.recipient_calendar = calendar
+        yield processor.queueAttendeeUpdate(("urn:uuid:user02", "urn:uuid:user01",))
+        self.assertEqual(processor.batches, 0)
+
+
+    @inlineCallbacks
+    def test_queueAttendeeUpdate_with_refresh(self):
+
+        self.patch(config.Scheduling.Options, "AttendeeRefreshBatch", 5)
+
+        calendar = Component.fromString("""BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER:urn:uuid:user01
+ATTENDEE:urn:uuid:user01
+ATTENDEE:urn:uuid:user02
+ATTENDEE:urn:uuid:user03
+END:VEVENT
+END:VCALENDAR
+""")
+        request = SimpleRequest(self.site, "PUT", "/calendar/1.ics")
+        processor = FakeImplicitProcessor()
+        processor.request = request
+        processor.uid = "12345-67890"
+        processor.recipient_calendar = calendar
+        yield processor.queueAttendeeUpdate(("urn:uuid:user02", "urn:uuid:user01",))
+        self.assertEqual(processor.batches, 1)

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_utils.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_utils.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/test/test_utils.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,198 @@
+##
+# Copyright (c) 2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+Tests for calendarserver.tools.purge
+"""
+
+from calendarserver.tap.util import getRootResource
+
+from pycalendar.datetime import PyCalendarDateTime
+
+from twisted.internet.defer import inlineCallbacks
+from twisted.trial import unittest
+
+from twistedcaldav.config import config
+
+from txdav.caldav.datastore.scheduling.utils import getCalendarObjectForPrincipals
+from txdav.caldav.datastore.sql import CalendarPrincipal
+from txdav.common.datastore.test.util import buildStore, populateCalendarsFrom, CommonCommonTests
+
+import os
+
+
+now = PyCalendarDateTime.getToday().getYear()
+
+ORGANIZER_ICS = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//Apple Inc.//iCal 4.0.1//EN
+CALSCALE:GREGORIAN
+BEGIN:VEVENT
+CREATED:20100303T181216Z
+UID:685BC3A1-195A-49B3-926D-388DDACA78A6
+TRANSP:OPAQUE
+SUMMARY:Ancient event
+DTSTART:%(year)s0307T111500Z
+DURATION:PT1H
+DTSTAMP:20100303T181220Z
+ORGANIZER:urn:uuid:user01
+ATTENDEE;PARTSTAT=ACCEPTED:urn:uuid:user01
+ATTENDEE;PARTSTAT=ACCEPTED:urn:uuid:user02
+SEQUENCE:2
+END:VEVENT
+END:VCALENDAR
+""".replace("\n", "\r\n") % {"year": now + 1}
+
+ATTENDEE_ICS = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//Apple Inc.//iCal 4.0.1//EN
+CALSCALE:GREGORIAN
+BEGIN:VEVENT
+CREATED:20100303T181216Z
+UID:685BC3A1-195A-49B3-926D-388DDACA78A6
+TRANSP:OPAQUE
+SUMMARY:Ancient event
+DTSTART:%(year)s0307T111500Z
+DURATION:PT1H
+DTSTAMP:20100303T181220Z
+ORGANIZER:urn:uuid:user01
+ATTENDEE;PARTSTAT=ACCEPTED:urn:uuid:user01
+ATTENDEE;PARTSTAT=ACCEPTED:urn:uuid:user02
+SEQUENCE:2
+END:VEVENT
+END:VCALENDAR
+""".replace("\n", "\r\n") % {"year": now + 1}
+
+
+
+class RecipientCopy(CommonCommonTests, unittest.TestCase):
+    """
+    Tests for deleting events older than a given date
+    """
+
+    metadata = {
+        "accessMode": "PUBLIC",
+        "isScheduleObject": True,
+        "scheduleTag": "abc",
+        "scheduleEtags": (),
+        "hasPrivateComment": False,
+    }
+
+    requirements = {
+        "user01" : {
+            "calendar1" : {
+                "1.ics" : (ORGANIZER_ICS, metadata,),
+            }
+        },
+        "user02" : {
+            "calendar2" : {
+                "2.ics" : (ATTENDEE_ICS, metadata,),
+            },
+            "calendar3" : {
+                "3.ics" : (ATTENDEE_ICS, metadata,),
+            }
+        }
+    }
+
+    @inlineCallbacks
+    def setUp(self):
+
+        yield super(RecipientCopy, self).setUp()
+        self._sqlCalendarStore = yield buildStore(self, self.notifierFactory)
+        yield self.populate()
+
+        self.patch(config.DirectoryService.params, "xmlFile",
+            os.path.join(
+                os.path.dirname(__file__), "accounts.xml"
+            )
+        )
+        self.patch(config.ResourceService.params, "xmlFile",
+            os.path.join(
+                os.path.dirname(__file__), "resources.xml"
+            )
+        )
+        self.rootResource = getRootResource(config, self._sqlCalendarStore)
+        self.directory = self.rootResource.getDirectory()
+
+
+    @inlineCallbacks
+    def populate(self):
+        yield populateCalendarsFrom(self.requirements, self.storeUnderTest())
+        self.notifierFactory.reset()
+
+
+    def storeUnderTest(self):
+        """
+        Create and return a L{CalendarStore} for testing.
+        """
+        return self._sqlCalendarStore
+
+
+    @inlineCallbacks
+    def test_getCalendarObjectForPrincipals(self):
+        """
+        Test that L{twistedcaldav.scheduling.utils.getCalendarObjectForPrincipals} detects and removes
+        resources with duplicate UIDs in the same calendar home.
+        """
+
+        # Check that expected resources are present
+        txn = self.transactionUnderTest()
+        for home_uid, calendar_name, resource_name in (
+            ("user01", "calendar1", "1.ics",),
+            ("user02", "calendar2", "2.ics",),
+            ("user02", "calendar3", "3.ics",),
+        ):
+            resource = (yield self.calendarObjectUnderTest(txn, name=resource_name, calendar_name=calendar_name, home=home_uid))
+            self.assertNotEqual(resource, None)
+        yield self.commit()
+
+        # Look up resource by UID in home where only one exists
+        principal = CalendarPrincipal("user01", ("urn:uuid:user01",))
+        txn = self.transactionUnderTest()
+        resource = (yield getCalendarObjectForPrincipals(txn, principal, "685BC3A1-195A-49B3-926D-388DDACA78A6"))
+        self.assertEqual(resource.name(), "1.ics")
+        self.assertEqual(resource._parentCollection.name(), "calendar1")
+        self.assertEqual(resource._parentCollection.viewerHome().uid(), "user01")
+        yield self.commit()
+
+        # Check that expected resources are still present
+        txn = self.transactionUnderTest()
+        for home_uid, calendar_name, resource_name in (
+            ("user01", "calendar1", "1.ics",),
+            ("user02", "calendar2", "2.ics",),
+            ("user02", "calendar3", "3.ics",),
+        ):
+            resource = (yield self.calendarObjectUnderTest(txn, name=resource_name, calendar_name=calendar_name, home=home_uid))
+            self.assertNotEqual(resource, None)
+        yield self.commit()
+
+        # Look up resource by UID in home where two exists
+        principal = CalendarPrincipal("user02", ("urn:uuid:user02",))
+        txn = self.transactionUnderTest()
+        resource = (yield getCalendarObjectForPrincipals(txn, principal, "685BC3A1-195A-49B3-926D-388DDACA78A6"))
+        self.assertTrue(resource.name() in ("2.ics", "3.ics",))
+        self.assertTrue(resource._parentCollection.name() in ("calendar2", "calendar3",))
+        self.assertEqual(resource._parentCollection.viewerHome().uid(), "user02")
+        yield self.commit()
+
+        # Check that expected resources are still present, but the duplicate missing
+        txn = self.transactionUnderTest()
+        resource = (yield self.calendarObjectUnderTest(txn, name="1.ics", calendar_name="calendar1", home="user01"))
+        self.assertNotEqual(resource, None)
+        resource2 = (yield self.calendarObjectUnderTest(txn, name="2.ics", calendar_name="calendar2", home="user02"))
+        resource3 = (yield self.calendarObjectUnderTest(txn, name="3.ics", calendar_name="calendar3", home="user02"))
+        self.assertTrue((resource2 is not None) ^ (resource3 is not None))
+        yield self.commit()

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/utils.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/utils.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/scheduling/utils.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,47 @@
+#
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twisted.internet.defer import inlineCallbacks, returnValue
+from twext.python.log import Logger
+
+log = Logger()
+
+ at inlineCallbacks
+def getCalendarObjectForPrincipals(txn, principal, uid, allow_shared=False):
+    """
+    Get a copy of the event for a principal.
+
+    NOTE: if more than one resource with the same UID is found, we will delete all but
+    one of them to avoid scheduling problems.
+    """
+
+    if principal and principal.locallyHosted():
+        # Get principal's calendar-home
+        calendar_home = yield txn.calendarHomeWithUID(principal.uid())
+
+        # Get matching newstore objects
+        objectResources = (yield calendar_home.getCalendarResourcesForUID(uid, allow_shared))
+
+        if len(objectResources) > 1:
+            # Delete all but the first one
+            log.debug("Should only have zero or one scheduling object resource with UID '%s' in calendar home: %s" % (uid, calendar_home,))
+            for resource in objectResources[1:]:
+                yield resource._parentCollection.removeObjectResource(resource)
+            objectResources = objectResources[:1]
+
+        returnValue(objectResources[0] if len(objectResources) == 1 else None)
+    else:
+        returnValue(None)

Modified: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/sql.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/sql.py	2013-04-08 19:22:13 UTC (rev 11015)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/sql.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -14,6 +14,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 ##
+from twext.enterprise.locking import NamedLock
 
 """
 SQL backend for CalDAV storage.
@@ -48,6 +49,7 @@
 from twistedcaldav import caldavxml, customxml
 from twistedcaldav.caldavxml import ScheduleCalendarTransp, Opaque
 from twistedcaldav.config import config
+from twistedcaldav.datafilters.peruserdata import PerUserDataFilter
 from twistedcaldav.dateops import normalizeForIndex, datetimeMktime, \
     pyCalendarTodatetime, parseSQLDateToPyCalendar
 from twistedcaldav.ical import Component, InvalidICalendarDataError, Property
@@ -55,14 +57,16 @@
 from twistedcaldav.memcacher import Memcacher
 
 from txdav.base.propertystore.base import PropertyName
-from txdav.caldav.datastore.util import AttachmentRetrievalTransport
+from txdav.caldav.datastore.util import AttachmentRetrievalTransport, \
+    normalizationLookup
 from txdav.caldav.datastore.util import CalendarObjectBase
 from txdav.caldav.datastore.util import StorageTransportBase
-from txdav.caldav.datastore.util import validateCalendarComponent, \
-    dropboxIDFromCalendarObject
+from txdav.caldav.datastore.util import dropboxIDFromCalendarObject
 from txdav.caldav.icalendarstore import ICalendarHome, ICalendar, ICalendarObject, \
     IAttachment, AttachmentStoreFailed, AttachmentStoreValidManagedID, \
-    AttachmentMigrationFailed, AttachmentDropboxNotAllowed
+    AttachmentMigrationFailed, AttachmentDropboxNotAllowed, \
+    TooManyAttendeesError, InvalidComponentTypeError, InvalidCalendarAccessError, \
+    InvalidUIDError, UIDExistsError
 from txdav.caldav.icalendarstore import QuotaExceeded
 from txdav.common.datastore.sql import CommonHome, CommonHomeChild, \
     CommonObjectResource, ECALENDARTYPE
@@ -77,7 +81,8 @@
     _ATTACHMENTS_MODE_READ
 from txdav.common.icommondatastore import IndexedSearchException, \
     InternalDataStoreError, HomeChildNameAlreadyExistsError, \
-    HomeChildNameNotAllowedError
+    HomeChildNameNotAllowedError, ObjectResourceTooBigError, \
+    InvalidObjectResourceError
 from txdav.xml.rfc2518 import ResourceType
 
 from pycalendar.datetime import PyCalendarDateTime
@@ -302,6 +307,58 @@
 
 
 
+class CalendarPrincipal(object):
+
+    def __init__(self, uid, cuaddrs):
+        self.principal_uid = uid
+        self.cuaddrs = cuaddrs
+
+
+    def uid(self):
+        return self.principal_uid
+
+
+    def fullName(self):
+        return "%s %s" % (self.principal_uid[:4], self.principal_uid[4:])
+
+
+    def calendarUserAddresses(self):
+        return self.cuaddrs
+
+
+    def canonicalCalendarUserAddress(self):
+        return [cuaddr for cuaddr in self.cuaddrs if cuaddr.startswith("urn:uuid")][0]
+
+
+    def locallyHosted(self):
+        return True
+
+
+    def thisServer(self):
+        return True
+
+
+    def calendarsEnabled(self):
+        return True
+
+
+    def getCUType(self):
+        return "INDIVIDUAL"
+
+
+    def enabledAsOrganizer(self):
+        return True
+
+
+    def canAutoSchedule(self, organizer):
+        return False
+
+
+    def getAutoScheduleMode(self, organizer):
+        return "auto"
+
+
+
 class CalendarHome(CommonHome):
 
     implements(ICalendarHome)
@@ -331,6 +388,42 @@
         self._childClass = Calendar
         super(CalendarHome, self).__init__(transaction, ownerUID, notifiers)
 
+
+    @classmethod
+    def metadataColumns(cls):
+        """
+        Return a list of column name for retrieval of metadata. This allows
+        different child classes to have their own type specific data, but still make use of the
+        common base logic.
+        """
+
+        # Common behavior is to have created and modified
+
+        return (
+            cls._homeMetaDataSchema.DEFAULT_EVENTS,
+            cls._homeMetaDataSchema.DEFAULT_TASKS,
+            cls._homeMetaDataSchema.CREATED,
+            cls._homeMetaDataSchema.MODIFIED,
+        )
+
+
+    @classmethod
+    def metadataAttributes(cls):
+        """
+        Return a list of attribute names for retrieval of metadata. This allows
+        different child classes to have their own type specific data, but still make use of the
+        common base logic.
+        """
+
+        # Common behavior is to have created and modified
+
+        return (
+            "_default_events",
+            "_default_tasks",
+            "_created",
+            "_modified",
+        )
+
     createCalendarWithName = CommonHome.createChildWithName
     removeCalendarWithName = CommonHome.removeChildWithName
     calendarWithName = CommonHome.childWithName
@@ -407,8 +500,7 @@
         for objectResource in objectResources:
             if ok_object and objectResource._resourceID == ok_object._resourceID:
                 continue
-            matched_mode = ("schedule" if objectResource.isScheduleObject
-                            else "calendar")
+            matched_mode = ("schedule" if objectResource.isScheduleObject else "calendar")
             if mode == "schedule" or matched_mode == "schedule":
                 returnValue(True)
 
@@ -528,7 +620,7 @@
         for calendar in calendars:
 
             # Ignore inbox - also shared calendars are not part of .calendars()
-            if calendar.name() == "inbox":
+            if calendar.isInbox():
                 continue
             split_count = yield calendar.splitCollectionByComponentTypes()
             self.log_warn("  Calendar: '%s', split into %d" % (calendar.name(), split_count + 1,))
@@ -549,7 +641,7 @@
             names = set()
             calendars = yield self.calendars()
             for calendar in calendars:
-                if calendar.name() == "inbox":
+                if calendar.isInbox():
                     continue
                 names.add(calendar.name())
                 result = yield calendar.getSupportedComponents()
@@ -568,6 +660,136 @@
             yield _requireCalendarWithType("VTODO", "tasks")
 
 
+    @inlineCallbacks
+    def pickNewDefaultCalendar(self, tasks=False):
+        """
+        First see if default provisioned calendar exists in the calendar home and pick that. Otherwise
+        pick another from the calendar home.
+        """
+
+        chm = self._homeMetaDataSchema
+        componentType = "VTODO" if tasks else "VEVENT"
+        test_name = "tasks" if tasks else "calendar"
+        attribute_to_test = "_default_tasks" if tasks else "_default_events"
+        column_to_set = chm.DEFAULT_TASKS if tasks else chm.DEFAULT_EVENTS
+
+        defaultCalendar = (yield self.calendarWithName(test_name))
+        if defaultCalendar is None or not defaultCalendar.owned():
+
+            @inlineCallbacks
+            def _findDefault():
+                for calendarName in (yield self.listCalendars()):
+                    calendar = (yield self.calendarWithName(calendarName))
+                    if calendar.isInbox():
+                        continue
+                    if not calendar.owned():
+                        continue
+                    if not calendar.isSupportedComponent(componentType):
+                        continue
+                    break
+                else:
+                    calendar = None
+                returnValue(calendar)
+
+            defaultCalendar = yield _findDefault()
+            if defaultCalendar is None:
+                # Create a default and try and get its name again
+                yield self.ensureDefaultCalendarsExist()
+                defaultCalendar = yield _findDefault()
+                if defaultCalendar is None:
+                    # Failed to even create a default - bad news...
+                    raise RuntimeError("No valid calendars to use as a default %s calendar." % (componentType,))
+
+        setattr(self, attribute_to_test, defaultCalendar._resourceID)
+        yield Update(
+            {column_to_set: defaultCalendar._resourceID},
+            Where=chm.RESOURCE_ID == self._resourceID,
+        ).on(self._txn)
+        yield self.invalidateQueryCache()
+
+        returnValue(defaultCalendar)
+
+
+    @inlineCallbacks
+    def defaultCalendar(self, componentType):
+        """
+        Find the default calendar for the supplied iCalendar component type. If one does
+        not exist, automatically provision it.
+        """
+
+        # Check any default calendar property first - this will create if none exists
+        attribute_to_test = "_default_tasks" if componentType == "VTODO" else "_default_events"
+        defaultID = getattr(self, attribute_to_test)
+        if defaultID:
+            default = (yield self.childWithID(defaultID))
+        else:
+            default = None
+
+        # Check that default handles the component type
+        if default is not None:
+            if not default.isSupportedComponent(componentType):
+                default = None
+
+        # Must have a default - provision one if not
+        if default is None:
+
+            # Try to find a calendar supporting the required component type. If there are multiple, pick
+            # the one with the oldest created timestamp as that will likely be the initial provision.
+            for calendarName in (yield self.listCalendars()):
+                calendar = (yield self.calendarWithName(calendarName))
+                if calendar.isInbox():
+                    continue
+                if not calendar.owned():
+                    continue
+                if not calendar.isSupportedComponent(componentType):
+                    continue
+                if default is None or calendar.created() < default.created():
+                    default = calendar
+
+            # If none can be found, provision one
+            if default is None:
+                new_name = "%ss" % (componentType.lower()[1:],)
+                default = yield self.createCalendarWithName(new_name)
+                yield default.setSupportedComponents(componentType.upper())
+
+            # Update the metadata
+            chm = self._homeMetaDataSchema
+            column_to_set = chm.DEFAULT_TASKS if componentType == "VTODO" else chm.DEFAULT_EVENTS
+            setattr(self, attribute_to_test, default._resourceID)
+            yield Update(
+                {column_to_set: default._resourceID},
+                Where=chm.RESOURCE_ID == self._resourceID,
+            ).on(self._txn)
+            yield self.invalidateQueryCache()
+
+        returnValue(default)
+
+
+    def isDefaultCalendar(self, calendar):
+        """
+        Is the supplied calendar one of the possible default calendars.
+        """
+        # Not allowed to delete the default calendar
+        return calendar._resourceID in (self._default_events, self._default_tasks)
+
+
+    def principalForUID(self, uid):
+        return CalendarPrincipal(uid, ("urn:uuid:%s" % (uid,), "mailto:%s at example.com" % (uid,),))
+
+
+    def principalForCalendarUserAddress(self, cuaddr):
+        if cuaddr.startswith("mailto:"):
+            uid, domain = cuaddr[7:].split('@')
+            if domain != "example.com":
+                return None
+            return CalendarPrincipal(uid, (cuaddr, "urn:uuid:%s" % (uid,)))
+        elif cuaddr.startswith("urn:uuid:"):
+            uid = cuaddr[9:]
+            return CalendarPrincipal(uid, (cuaddr, "mailto:%s at example.com" % (uid,)))
+        else:
+            return None
+
+
 CalendarHome._register(ECALENDARTYPE)
 
 
@@ -672,7 +894,7 @@
         inbox resources need to store Originator, Recipient etc properties.
         Other calendars do not have object resources with properties.
         """
-        return self._name == "inbox"
+        return not self.isInbox()
 
 
     @inlineCallbacks
@@ -708,6 +930,16 @@
             return True
 
 
+    def isInbox(self):
+        """
+        Indicates whether this calendar is an "inbox".
+
+        @return: C{True} if it is an "inbox, C{False} otherwise
+        @rtype: C{bool}
+        """
+        return self.name() == "inbox"
+
+
     def initPropertyStore(self, props):
         # Setup peruser special properties
         props.setSpecialProperties(
@@ -976,6 +1208,10 @@
         self.hasPrivateComment = metadata.get("hasPrivateComment", False)
         self._dropboxID = None
 
+        # Component caching
+        self._cachedComponent = None
+        self._cachedCommponentPerUser = {}
+
     _allColumns = [
         _objectSchema.RESOURCE_ID,
         _objectSchema.RESOURCE_NAME,
@@ -1024,11 +1260,199 @@
         return self._calendar
 
 
+    # Stuff from put_common
     @inlineCallbacks
+    def fullValidation(self, component, inserting, internal):
+        """
+        Do full validation of source and destination calendar data.
+        """
+
+        # Basic validation
+        #TODO: figure out what to do about etag/schedule-tag
+        self.validIfScheduleMatch(False, False, internal)
+
+        # Skip validation on internal requests
+        if not internal:
+
+            # Valid data sizes - do before parsing the data
+            if config.MaxResourceSize:
+                calsize = len(str(component))
+                if calsize > config.MaxResourceSize:
+                    raise ObjectResourceTooBigError()
+
+        # Possible timezone stripping
+        if config.EnableTimezonesByReference:
+            component.stripKnownTimezones()
+
+        # Skip validation on internal requests
+        if not internal:
+
+            # Valid calendar data checks
+            yield self.validCalendarDataCheck(component, inserting)
+
+            # Valid calendar component for check
+            if not self.calendar().isSupportedComponent(component.mainType()):
+                raise InvalidComponentTypeError("Invalid component type %s for calendar: %s" % (component.mainType(), self.calendar(),))
+
+            # Valid attendee list size check
+            yield self.validAttendeeListSizeCheck(component, inserting)
+
+            # Normalize the calendar user addresses once we know we have valid
+            # calendar data
+            component.normalizeCalendarUserAddresses(normalizationLookup, self.calendar().viewerHome().principalForCalendarUserAddress)
+
+        # Check access
+        yield self.validAccess(component, inserting, internal)
+
+
+    def validIfScheduleMatch(self, etag_match, schedule_tag, internal):
+        """
+        Check for If-ScheduleTag-Match header behavior.
+        """
+        # Only when a direct request
+        self.schedule_tag_match = False
+        if not self.calendar().isInbox() and not internal:
+            if schedule_tag:
+                self._validIfScheduleMatch(self.request)
+                self.schedule_tag_match = True
+            elif config.Scheduling.CalDAV.ScheduleTagCompatibility:
+                # Compatibility with old clients. Policy:
+                #
+                # 1. If If-Match header is not present, never do smart merge.
+                # 2. If If-Match is present and the specified ETag is
+                #    considered a "weak" match to the current Schedule-Tag,
+                #    then do smart merge, else reject with a 412.
+                #
+                # Actually by the time we get here the precondition will
+                # already have been tested and found to be OK, so we can just
+                # always do smart merge now if If-Match is present.
+                self.schedule_tag_match = etag_match is not None
+
+
+    def validCalendarDataCheck(self, component, inserting):
+        """
+        Check that the calendar data is valid iCalendar.
+        @return:         tuple: (True/False if the calendar data is valid,
+                                 log message string).
+        """
+
+        # Valid calendar data checks
+        if not isinstance(component, VComponent):
+            raise InvalidObjectResourceError("Wrong type of object: %s" % (type(component),))
+
+        try:
+            component.validCalendarData(validateRecurrences=self._txn._migrating)
+            component.validCalendarForCalDAV(methodAllowed=self.calendar().name() == 'inbox')
+            if self._txn._migrating:
+                component.validOrganizerForScheduling(doFix=True)
+        except InvalidICalendarDataError, e:
+            raise InvalidObjectResourceError(e)
+
+
+    @inlineCallbacks
+    def validAttendeeListSizeCheck(self, component, inserting):
+        """
+        Make sure that the Attendee list length is within bounds. We don't do this check for inbox because we
+        will assume that the limit has been applied on the PUT causing the iTIP message to be created.
+
+        FIXME: The inbox check might not take into account iSchedule stuff from outside. That needs to have
+        the max attendees check applied at the time of delivery.
+        """
+
+        if config.MaxAttendeesPerInstance and not self.calendar().isInbox():
+            uniqueAttendees = set()
+            for attendee in component.getAllAttendeeProperties():
+                uniqueAttendees.add(attendee.value())
+            attendeeListLength = len(uniqueAttendees)
+            if attendeeListLength > config.MaxAttendeesPerInstance:
+
+                # Check to see whether we are increasing the count on an existing resource
+                if not inserting:
+                    oldcalendar = (yield self.componentForUser())
+                    uniqueAttendees = set()
+                    for attendee in oldcalendar.getAllAttendeeProperties():
+                        uniqueAttendees.add(attendee.value())
+                    oldAttendeeListLength = len(uniqueAttendees)
+                else:
+                    oldAttendeeListLength = 0
+
+                if attendeeListLength > oldAttendeeListLength:
+                    raise TooManyAttendeesError("Attendee list size %d is larger than allowed limit %d" % (attendeeListLength, config.MaxAttendeesPerInstance))
+
+
+    def validAccess(self, component, inserting, internal):
+        """
+        Make sure that the X-CALENDARSERVER-ACCESS property is properly dealt with.
+        """
+
+        if component.hasProperty(Component.ACCESS_PROPERTY):
+
+            # Must be a value we know about
+            access = component.accessLevel(default=None)
+            if access is None:
+                raise InvalidCalendarAccessError("Private event access level not allowed")
+
+            # Only DAV:owner is able to set the property to other than PUBLIC
+            if not internal:
+                if self.calendar().viewerHome().uid() != self._txn._authz_uid and access != Component.ACCESS_PUBLIC:
+                    raise InvalidCalendarAccessError("Private event access level change not allowed")
+
+            self.accessMode = access
+        else:
+            # Check whether an access property was present before and write that into the calendar data
+            if not inserting and self.accessMode:
+                old_access = self.accessMode
+                component.addProperty(Property(name=Component.ACCESS_PROPERTY, value=old_access))
+
+
+    @inlineCallbacks
+    def _lockUID(self, component, inserting):
+        """
+        Create a lock on the component's UID and verify, after getting the lock, that the incoming UID
+        meets the requirements of the store.
+        """
+
+        new_uid = component.resourceUID()
+        yield NamedLock.acquire(self._txn, "ImplicitUIDLock:%s" % (hashlib.md5(new_uid).hexdigest(),))
+
+        # UID conflict check - note we do this after reserving the UID to avoid a race condition where two requests
+        # try to write the same calendar data to two different resource URIs.
+        if not self.calendar().isInbox():
+            # Cannot overwrite a resource with different UID
+            if not inserting:
+                if self._uid != new_uid:
+                    raise InvalidUIDError()
+            else:
+                # New UID must be unique for the owner - no need to do this on an overwrite as we can assume
+                # the store is already consistent in this regard
+                elsewhere = (yield self.calendar().ownerHome().hasCalendarResourceUIDSomewhereElse(new_uid, self, "schedule"))
+                if elsewhere:
+                    raise UIDExistsError()
+
+
     def setComponent(self, component, inserting=False):
+        """
+        Public api for storing a component. This will do full data validation checks on the specified component.
+        Scheduling will be done automatically.
+        """
 
-        validateCalendarComponent(self, self._calendar, component, inserting, self._txn._migrating)
+        return self._setComponentInternal(component, inserting, False)
 
+
+    @inlineCallbacks
+    def _setComponentInternal(self, component, inserting=False, internal=True):
+        """
+        Setting the component internally to the store itself. This will bypass a whole bunch of data consistency checks
+        on the assumption that those have been done prior to the component data being provided, provided the flag is set.
+        This should always be treated as an api private to the store.
+        """
+
+        # Handle all validation operations here.
+        yield self.fullValidation(component, inserting, internal)
+
+        # UID lock - this will remain active until the end of the current txn
+        yield self._lockUID(component, inserting)
+
         yield self.updateDatabase(component, inserting=inserting)
 
         if inserting:
@@ -1056,7 +1480,7 @@
         txn = txn if txn is not None else self._txn
 
         # inbox does things slightly differently
-        isInboxItem = self._parentCollection.name() == "inbox"
+        isInboxItem = self.calendar().isInbox()
 
         # In some cases there is no need to remove/rebuild the instance index because we know no time or
         # freebusy related properties have changed (e.g. an attendee reply and refresh). In those cases
@@ -1146,6 +1570,9 @@
         if not reCreate:
             componentText = str(component)
             self._objectText = componentText
+            self._cachedComponent = component
+            self._cachedCommponentPerUser = {}
+
             organizer = component.getOrganizer()
             if not organizer:
                 organizer = ""
@@ -1163,7 +1590,7 @@
             # after setting up other properties as UID at least is needed
             self._attachment = _ATTACHMENTS_MODE_NONE
             if self._dropboxID is None:
-                if self._parentCollection.name() != "inbox":
+                if not isInboxItem:
                     if component.hasPropertyInAnyComponent("X-APPLE-DROPBOX"):
                         self._attachment = _ATTACHMENTS_MODE_WRITE
                         self._dropboxID = (yield self.dropboxID())
@@ -1333,32 +1760,59 @@
         only allowed in good data.
         """
 
-        text = yield self._text()
+        if self._cachedComponent is None:
 
-        try:
-            component = VComponent.fromString(text)
-        except InvalidICalendarDataError, e:
-            # This is a really bad situation, so do raise
-            raise InternalDataStoreError(
-                "Data corruption detected (%s) in id: %s"
-                % (e, self._resourceID)
-            )
+            text = yield self._text()
 
-        # Fix any bogus data we can
-        fixed, unfixed = component.validCalendarData(doFix=True, doRaise=False)
+            try:
+                component = VComponent.fromString(text)
+            except InvalidICalendarDataError, e:
+                # This is a really bad situation, so do raise
+                raise InternalDataStoreError(
+                    "Data corruption detected (%s) in id: %s"
+                    % (e, self._resourceID)
+                )
 
-        if unfixed:
-            self.log_error("Calendar data id=%s had unfixable problems:\n  %s" %
-                           (self._resourceID, "\n  ".join(unfixed),))
+            # Fix any bogus data we can
+            fixed, unfixed = component.validCalendarData(doFix=True, doRaise=False)
 
-        if fixed:
-            self.log_error("Calendar data id=%s had fixable problems:\n  %s" %
-                           (self._resourceID, "\n  ".join(fixed),))
+            if unfixed:
+                self.log_error("Calendar data id=%s had unfixable problems:\n  %s" %
+                               (self._resourceID, "\n  ".join(unfixed),))
 
-        returnValue(component)
+            if fixed:
+                self.log_error("Calendar data id=%s had fixable problems:\n  %s" %
+                               (self._resourceID, "\n  ".join(fixed),))
 
+            self._cachedComponent = component
+            self._cachedCommponentPerUser = {}
 
+        returnValue(self._cachedComponent)
+
+
     @inlineCallbacks
+    def componentForUser(self, user_uuid=None):
+        """
+        Return the iCalendar component filtered for the specified user's per-user data.
+
+        @param user_uuid: the user UUID to filter on
+        @type user_uuid: C{str}
+
+        @return: the filtered calendar component
+        @rtype: L{twistedcaldav.ical.Component}
+        """
+
+        if user_uuid is None:
+            user_uuid = self._parentCollection.viewHome().uid()
+
+        if user_uuid not in self._cachedCommponentPerUser:
+            caldata = yield self.component()
+            filtered = PerUserDataFilter(user_uuid).filter(caldata)
+            self._cachedCommponentPerUser[user_uuid] = filtered
+        returnValue(self._cachedCommponentPerUser[user_uuid])
+
+
+    @inlineCallbacks
     def remove(self):
         # Need to also remove attachments
         if self._dropboxID:

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/calendar_store/ho/me/home_defaults/calendar_1/1.ics
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/calendar_store/ho/me/home_defaults/calendar_1/1.ics	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/calendar_store/ho/me/home_defaults/calendar_1/1.ics	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,32 @@
+BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//Apple Inc.//iCal 4.0.1//EN
+CALSCALE:GREGORIAN
+BEGIN:VTIMEZONE
+TZID:US/Pacific
+BEGIN:DAYLIGHT
+TZOFFSETFROM:-0800
+RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
+DTSTART:20070311T020000
+TZNAME:PDT
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:STANDARD
+TZOFFSETFROM:-0700
+RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
+DTSTART:20071104T020000
+TZNAME:PST
+TZOFFSETTO:-0800
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+DTEND;TZID=US/Pacific:%(now)s0324T124500
+UID:uid2-1
+DTSTAMP:20090326T145447Z
+X-APPLE-DROPBOX:/calendars/__uids__/user01/dropbox/uid1.dropbox
+ATTACH;VALUE=URI:/calendars/__uids__/user01/dropbox/uid1.dropbox/test.txt
+SUMMARY:CalDAV protocol updates
+DTSTART;TZID=US/Pacific:%(now)s0324T121500
+CREATED:20090326T145440Z
+END:VEVENT
+END:VCALENDAR

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/calendar_store/ho/me/home_defaults/calendar_1/3.ics
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/calendar_store/ho/me/home_defaults/calendar_1/3.ics	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/calendar_store/ho/me/home_defaults/calendar_1/3.ics	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,29 @@
+BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//Apple Inc.//iCal 4.0.1//EN
+CALSCALE:GREGORIAN
+BEGIN:VTIMEZONE
+TZID:US/Pacific
+BEGIN:DAYLIGHT
+TZOFFSETFROM:-0800
+RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
+DTSTART:20070311T020000
+TZNAME:PDT
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:STANDARD
+TZOFFSETFROM:-0700
+RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
+DTSTART:20071104T020000
+TZNAME:PST
+TZOFFSETTO:-0800
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VTODO
+UID:uid2-3
+DTSTAMP:20090326T145447Z
+SUMMARY:Do something...
+DUE;TZID=US/Pacific:%(now)s0324T121500
+CREATED:20090326T145440Z
+END:VTODO
+END:VCALENDAR

Modified: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/common.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/common.py	2013-04-08 19:22:13 UTC (rev 11015)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/common.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -66,6 +66,9 @@
 homeNoSplitsRoot = storePath.child("ho").child("me").child("home_no_splits")
 cal1NoSplitsRoot = homeNoSplitsRoot.child("calendar_1")
 
+homeDefaultsRoot = storePath.child("ho").child("me").child("home_defaults")
+cal1DefaultsRoot = homeDefaultsRoot.child("calendar_1")
+
 calendar1_objectNames = [
     "1.ics",
     "2.ics",
@@ -270,6 +273,12 @@
         "home_splits_shared": {
             "calendar_1": {},
         },
+        "home_defaults": {
+            "calendar_1": {
+                "1.ics": (cal1NoSplitsRoot.child("1.ics").getContent(), metadata1),
+                "3.ics": (cal1NoSplitsRoot.child("3.ics").getContent(), metadata3),
+            },
+        },
     }
     md5s = {
         "home1": {
@@ -536,7 +545,6 @@
         self.assertEquals(name, "/CalDAV/example.com/home1/")
 
 
-
     @inlineCallbacks
     def test_displayNameNone(self):
         """

Added: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/test_implicit.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/test_implicit.py	                        (rev 0)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/test_implicit.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -0,0 +1,543 @@
+##
+# Copyright (c) 2005-2013 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+
+from twisted.internet.defer import inlineCallbacks
+
+from twistedcaldav.ical import Component
+
+from txdav.common.datastore.test.util import CommonCommonTests, buildStore, \
+    populateCalendarsFrom
+from twisted.trial.unittest import TestCase
+from twext.python.clsprop import classproperty
+from twistedcaldav.config import config
+from txdav.common.icommondatastore import ObjectResourceTooBigError, \
+    InvalidObjectResourceError
+from txdav.caldav.icalendarstore import InvalidComponentTypeError, \
+    TooManyAttendeesError, InvalidCalendarAccessError, InvalidUIDError, \
+    UIDExistsError
+import sys
+
+class ImplicitRequests (CommonCommonTests, TestCase):
+    """
+    Test twistedcaldav.scheduyling.implicit with a Request object.
+    """
+
+    @inlineCallbacks
+    def setUp(self):
+        yield super(ImplicitRequests, self).setUp()
+        self._sqlCalendarStore = yield buildStore(self, self.notifierFactory)
+        yield self.populate()
+
+
+    @inlineCallbacks
+    def populate(self):
+        yield populateCalendarsFrom(self.requirements, self.storeUnderTest())
+        self.notifierFactory.reset()
+
+
+    @classproperty(cache=False)
+    def requirements(cls): #@NoSelf
+        return {
+        "user01": {
+            "calendar_1": {
+            },
+            "inbox": {
+            },
+        },
+        "user02": {
+            "calendar_1": {
+            },
+            "inbox": {
+            },
+        },
+    }
+
+
+    def storeUnderTest(self):
+        """
+        Create and return a L{CalendarStore} for testing.
+        """
+        return self._sqlCalendarStore
+
+
+    @inlineCallbacks
+    def test_doCreateResource(self):
+        """
+        Test that resource creation works.
+        """
+
+        data1 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE:mailto:user02 at example.com
+END:VEVENT
+END:VCALENDAR
+"""
+
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar1 = Component.fromString(data1)
+        yield calendar_collection.createCalendarObjectWithName("test.ics", calendar1)
+        yield self.commit()
+
+        calendar_resource1 = (yield self.calendarObjectUnderTest(name="test.ics", home="user01",))
+        calendar1 = (yield calendar_resource1.component())
+        self.assertTrue("urn:uuid:user01" in str(calendar1))
+        self.assertTrue("urn:uuid:user02" in str(calendar1))
+        self.assertTrue("CN=" in str(calendar1))
+        yield self.commit()
+
+
+    @inlineCallbacks
+    def test_validation_maxResourceSize(self):
+        """
+        Test that various types of invalid calendar data are rejected when creating a resource.
+        """
+
+        data1 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+END:VEVENT
+END:VCALENDAR
+"""
+
+        data2 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Changed
+END:VEVENT
+END:VCALENDAR
+"""
+
+        self.patch(config, "MaxResourceSize", 100)
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar1 = Component.fromString(data1)
+        try:
+            yield calendar_collection.createCalendarObjectWithName("test.ics", calendar1)
+        except ObjectResourceTooBigError:
+            pass
+        except:
+            self.fail("Wrong exception raised: %s" % (sys.exc_info()[0].__name__,))
+        else:
+            self.fail("Exception not raised")
+        yield self.commit()
+
+        self.patch(config, "MaxResourceSize", 10000)
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar1 = Component.fromString(data1)
+        yield calendar_collection.createCalendarObjectWithName("test.ics", calendar1)
+        yield self.commit()
+
+        self.patch(config, "MaxResourceSize", 100)
+        calendar_resource = (yield self.calendarObjectUnderTest(name="test.ics", home="user01",))
+        calendar2 = Component.fromString(data2)
+        try:
+            yield calendar_resource.setComponent(calendar2)
+        except ObjectResourceTooBigError:
+            pass
+        except:
+            self.fail("Wrong exception raised: %s" % (sys.exc_info()[0].__name__,))
+        else:
+            self.fail("Exception not raised")
+        yield self.commit()
+
+
+    @inlineCallbacks
+    def test_validation_validCalendarDataCheck(self):
+        """
+        Test that various types of invalid calendar data are rejected when creating a resource.
+        """
+
+        data = (
+            "xyz",
+            Component.fromString("""BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:1
+SUMMARY:2
+END:VEVENT
+END:VCALENDAR
+"""),
+
+        Component.fromString("""BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+METHOD:PUBLISH
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+END:VEVENT
+END:VCALENDAR
+"""),
+        )
+
+        for item in data:
+            calendar_collection = (yield self.calendarUnderTest(home="user01"))
+            calendar = item
+            try:
+                yield calendar_collection.createCalendarObjectWithName("test.ics", calendar)
+            except InvalidObjectResourceError:
+                pass
+            except:
+                self.fail("Wrong exception raised: %s" % (sys.exc_info()[0].__name__,))
+            else:
+                self.fail("Exception not raised")
+            yield self.commit()
+
+
+    @inlineCallbacks
+    def test_validation_validSupportedComponentType(self):
+        """
+        Test that resources are restricted by component type.
+        """
+
+        data1 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+END:VEVENT
+END:VCALENDAR
+"""
+
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar_collection.setSupportedComponents("VTODO")
+        calendar = Component.fromString(data1)
+        try:
+            yield calendar_collection.createCalendarObjectWithName("test.ics", calendar)
+        except InvalidComponentTypeError:
+            pass
+        except:
+            self.fail("Wrong exception raised: %s" % (sys.exc_info()[0].__name__,))
+        else:
+            self.fail("Exception not raised")
+        yield self.commit()
+
+
+    @inlineCallbacks
+    def test_validation_validAttendeeListSizeCheck(self):
+        """
+        Test that resource with too many attendees are rejected.
+        """
+
+        data1 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+ORGANIZER;CN="User 01":mailto:user01 at example.com
+ATTENDEE:mailto:user01 at example.com
+ATTENDEE:mailto:user02 at example.com
+ATTENDEE:mailto:user03 at example.com
+ATTENDEE:mailto:user04 at example.com
+ATTENDEE:mailto:user05 at example.com
+END:VEVENT
+END:VCALENDAR
+"""
+
+        self.patch(config, "MaxAttendeesPerInstance", 2)
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar = Component.fromString(data1)
+        try:
+            yield calendar_collection.createCalendarObjectWithName("test.ics", calendar)
+        except TooManyAttendeesError:
+            pass
+        except:
+            self.fail("Wrong exception raised: %s" % (sys.exc_info()[0].__name__,))
+        else:
+            self.fail("Exception not raised")
+        yield self.commit()
+
+
+    @inlineCallbacks
+    def test_validation_validAccess_invalidValue(self):
+        """
+        Test that resource access mode changes are rejected.
+        """
+
+        data1 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+X-CALENDARSERVER-ACCESS:BOGUS
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+END:VEVENT
+END:VCALENDAR
+"""
+
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar = Component.fromString(data1)
+        try:
+            yield calendar_collection.createCalendarObjectWithName("test.ics", calendar)
+        except InvalidCalendarAccessError:
+            pass
+        except:
+            self.fail("Wrong exception raised: %s" % (sys.exc_info()[0].__name__,))
+        else:
+            self.fail("Exception not raised")
+        yield self.commit()
+
+
+    @inlineCallbacks
+    def test_validation_validAccess_authzChangeNotAllowed(self):
+        """
+        Test that resource access mode changes are rejected.
+        """
+
+        data1 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+X-CALENDARSERVER-ACCESS:PRIVATE
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+END:VEVENT
+END:VCALENDAR
+"""
+
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar = Component.fromString(data1)
+        txn = self.transactionUnderTest()
+        txn._authz_uid = "user02"
+        try:
+            yield calendar_collection.createCalendarObjectWithName("test.ics", calendar)
+        except InvalidCalendarAccessError:
+            pass
+        except:
+            self.fail("Wrong exception raised: %s" % (sys.exc_info()[0].__name__,))
+        else:
+            self.fail("Exception not raised")
+        yield self.commit()
+
+        # This one should be OK
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar = Component.fromString(data1)
+        txn = self.transactionUnderTest()
+        txn._authz_uid = "user01"
+        yield calendar_collection.createCalendarObjectWithName("test.ics", calendar)
+        yield self.commit()
+
+        # This one should re-insert access mode
+        data2 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Changed
+END:VEVENT
+END:VCALENDAR
+"""
+
+        calendar_resource = (yield self.calendarObjectUnderTest(name="test.ics", home="user01",))
+        calendar = Component.fromString(data2)
+        txn = self.transactionUnderTest()
+        txn._authz_uid = "user01"
+        yield calendar_resource.setComponent(calendar)
+        yield self.commit()
+
+        calendar_resource = (yield self.calendarObjectUnderTest(name="test.ics", home="user01",))
+        calendar1 = (yield calendar_resource.component())
+        self.assertTrue("X-CALENDARSERVER-ACCESS:PRIVATE" in str(calendar1))
+        self.assertTrue("SUMMARY:Changed" in str(calendar1))
+        yield self.commit()
+
+
+    @inlineCallbacks
+    def test_validation_overwriteUID(self):
+        """
+        Test that a change to a resource UID is not allowed.
+        """
+
+        data1 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+END:VEVENT
+END:VCALENDAR
+"""
+
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar = Component.fromString(data1)
+        yield calendar_collection.createCalendarObjectWithName("test.ics", calendar)
+        yield self.commit()
+
+        # This one should fail
+        data2 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply-1
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+END:VEVENT
+END:VCALENDAR
+"""
+
+        calendar_resource = (yield self.calendarObjectUnderTest(name="test.ics", home="user01",))
+        calendar = Component.fromString(data2)
+        try:
+            yield calendar_resource.setComponent(calendar)
+        except InvalidUIDError:
+            pass
+        except:
+            self.fail("Wrong exception raised: %s" % (sys.exc_info()[0].__name__,))
+        else:
+            self.fail("Exception not raised")
+        yield self.commit()
+
+
+    @inlineCallbacks
+    def test_validation_duplicateUIDSameCalendar(self):
+        """
+        Test that a resource with a duplicate UID in the same calendar is not allowed.
+        """
+
+        data1 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+END:VEVENT
+END:VCALENDAR
+"""
+
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar = Component.fromString(data1)
+        yield calendar_collection.createCalendarObjectWithName("test.ics", calendar)
+        yield self.commit()
+
+        # This one should fail
+        data2 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Changed
+END:VEVENT
+END:VCALENDAR
+"""
+
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar = Component.fromString(data2)
+        try:
+            yield calendar_collection.createCalendarObjectWithName("test2.ics", calendar)
+        except UIDExistsError:
+            pass
+        except:
+            self.fail("Wrong exception raised: %s" % (sys.exc_info()[0].__name__,))
+        else:
+            self.fail("Exception not raised")
+        yield self.commit()
+
+
+    @inlineCallbacks
+    def test_validation_duplicateUIDDifferentCalendar(self):
+        """
+        Test that a resource with a duplicate UID in a different calendar is not allowed.
+        """
+
+        data1 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+END:VEVENT
+END:VCALENDAR
+"""
+
+        calendar_collection = (yield self.calendarUnderTest(home="user01"))
+        calendar = Component.fromString(data1)
+        yield calendar_collection.createCalendarObjectWithName("test.ics", calendar)
+        yield self.commit()
+
+        # This one should fail
+        data2 = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//CALENDARSERVER.ORG//NONSGML Version 1//EN
+BEGIN:VEVENT
+UID:12345-67890-attendee-reply
+DTSTAMP:20080601T120000Z
+DTSTART:20080601T120000Z
+DTEND:20080601T130000Z
+SUMMARY:Changed
+END:VEVENT
+END:VCALENDAR
+"""
+
+        home_collection = (yield self.homeUnderTest(name="user01"))
+        calendar_collection_2 = (yield home_collection.createCalendarWithName("calendar_2"))
+        calendar = Component.fromString(data2)
+        try:
+            yield calendar_collection_2.createCalendarObjectWithName("test2.ics", calendar)
+        except UIDExistsError:
+            pass
+        except:
+            self.fail("Wrong exception raised: %s" % (sys.exc_info()[0].__name__,))
+        else:
+            self.fail("Exception not raised")
+        yield self.commit()

Modified: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/test_index_file.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/test_index_file.py	2013-04-08 19:22:13 UTC (rev 11015)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/test_index_file.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -19,7 +19,7 @@
 from twisted.internet.task import deferLater
 
 from txdav.caldav.datastore.index_file import Index, MemcachedUIDReserver
-from txdav.common.icommondatastore import ReservationError,\
+from txdav.common.icommondatastore import ReservationError, \
     InternalDataStoreError
 
 from twistedcaldav import caldavxml
@@ -59,6 +59,7 @@
         return component
 
 
+
 class MinimalResourceReplacement(object):
     """
     Provide the minimal set of attributes and methods from CalDAVFile required
@@ -467,7 +468,7 @@
             self.assertTrue(self.db.resourceExists(name), msg=description)
 
             # Create fake filter element to match time-range
-            filter =  caldavxml.Filter(
+            filter = caldavxml.Filter(
                   caldavxml.ComponentFilter(
                       caldavxml.ComponentFilter(
                           TimeRange(
@@ -653,7 +654,7 @@
             self.assertTrue(self.db.resourceExists(name), msg=description)
 
             # Create fake filter element to match time-range
-            filter =  caldavxml.Filter(
+            filter = caldavxml.Filter(
                   caldavxml.ComponentFilter(
                       caldavxml.ComponentFilter(
                           TimeRange(
@@ -1060,7 +1061,7 @@
             self.assertTrue(self.db.resourceExists(name), msg=description)
 
             # Create fake filter element to match time-range
-            filter =  caldavxml.Filter(
+            filter = caldavxml.Filter(
                   caldavxml.ComponentFilter(
                       caldavxml.ComponentFilter(
                           TimeRange(
@@ -1080,7 +1081,7 @@
                 for _ignore_name, _ignore_uid, type, test_organizer, float, start, end, fbtype, transp in resources:
                     self.assertEqual(test_organizer, organizer, msg=description)
                     index_results.add((str(float), str(start), str(end), str(fbtype), str(transp),))
-    
+
                 self.assertEqual(set(instances), index_results, msg="%s, user:%s" % (description, useruid,))
 
             self.db.deleteResource(name)
@@ -1141,17 +1142,19 @@
         self.db.deleteResource("data3.ics")
 
         tests = (
-            (0, (["data1.ics", "data2.ics",], [],)),
-            (1, (["data2.ics",], ["data3.ics",],)),
-            (2, ([], ["data3.ics",],)),
-            (3, ([], ["data3.ics",],)),
+            (0, (["data1.ics", "data2.ics", ], [],)),
+            (1, (["data2.ics", ], ["data3.ics", ],)),
+            (2, ([], ["data3.ics", ],)),
+            (3, ([], ["data3.ics", ],)),
             (4, ([], [],)),
             (5, ([], [],)),
         )
-        
+
         for revision, results in tests:
             self.assertEquals(self.db.whatchanged(revision), results, "Mismatched results for whatchanged with revision %d" % (revision,))
 
+
+
 class MemcacheTests(SQLIndexTests):
     def setUp(self):
         super(MemcacheTests, self).setUp()

Modified: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/test_sql.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/test_sql.py	2013-04-08 19:22:13 UTC (rev 11015)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/test/test_sql.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -1111,6 +1111,54 @@
 
 
     @inlineCallbacks
+    def test_defaultCalendar(self):
+        """
+        Make sure a default_events calendar is assigned.
+        """
+
+        home = yield self.transactionUnderTest().calendarHomeWithUID("home_defaults")
+        calendar1 = yield home.calendarWithName("calendar_1")
+        yield calendar1.splitCollectionByComponentTypes()
+        yield self.commit()
+
+        home = yield self.transactionUnderTest().calendarHomeWithUID("home_defaults")
+        self.assertEqual(home._default_events, None)
+        self.assertEqual(home._default_tasks, None)
+
+        default_events = yield home.defaultCalendar("VEVENT")
+        self.assertTrue(default_events is not None)
+        self.assertEqual(home._default_events, default_events._resourceID)
+        self.assertEqual(home._default_tasks, None)
+        yield self.commit()
+
+        home = yield self.transactionUnderTest().calendarHomeWithUID("home_defaults")
+        self.assertEqual(home._default_events, default_events._resourceID)
+        self.assertEqual(home._default_tasks, None)
+
+        default_tasks = yield home.defaultCalendar("VTODO")
+        self.assertTrue(default_tasks is not None)
+        self.assertEqual(home._default_events, default_events._resourceID)
+        self.assertEqual(home._default_tasks, default_tasks._resourceID)
+        yield self.commit()
+
+        home = yield self.transactionUnderTest().calendarHomeWithUID("home_defaults")
+        self.assertEqual(home._default_events, default_events._resourceID)
+        self.assertEqual(home._default_tasks, default_tasks._resourceID)
+        yield home.removeCalendarWithName("todos")
+        yield self.commit()
+
+        home = yield self.transactionUnderTest().calendarHomeWithUID("home_defaults")
+        self.assertEqual(home._default_events, default_events._resourceID)
+        self.assertEqual(home._default_tasks, default_tasks._resourceID)
+
+        default_tasks2 = yield home.defaultCalendar("VTODO")
+        self.assertTrue(default_tasks2 is not None)
+        self.assertEqual(home._default_events, default_events._resourceID)
+        self.assertEqual(home._default_tasks, default_tasks2._resourceID)
+        yield self.commit()
+
+
+    @inlineCallbacks
     def test_resourceLock(self):
         """
         Test CommonObjectResource.lock to make sure it locks, raises on missing resource,
@@ -1149,7 +1197,7 @@
         # FIXME: not sure why, but without this statement here, this portion of the test fails in a funny way.
         # Basically the query in the try block seems to execute twice, failing each time, one of which is caught,
         # and the other not - causing the test to fail. Seems like some state on newTxn is not being initialized?
-        yield self.calendarObjectUnderTest("2.ics", txn=newTxn)
+        yield self.calendarObjectUnderTest(txn=newTxn, name="2.ics")
 
         try:
             yield resource.lock(wait=False, useTxn=newTxn)
@@ -1160,7 +1208,7 @@
         self.assertTrue(resource._locked)
 
         # Test missing resource
-        resource2 = yield self.calendarObjectUnderTest("2.ics")
+        resource2 = yield self.calendarObjectUnderTest(name="2.ics")
         resource2._resourceID = 123456789
         try:
             yield resource2.lock()
@@ -1300,7 +1348,7 @@
 
         # Re-add event with re-indexing
         calendar = yield self.calendarUnderTest()
-        calendarObject = yield self.calendarObjectUnderTest("indexing.ics")
+        calendarObject = yield self.calendarObjectUnderTest(name="indexing.ics")
         yield calendarObject.setComponent(component)
         instances2 = yield calendarObject.instances()
         self.assertNotEqual(
@@ -1311,7 +1359,7 @@
 
         # Re-add event without re-indexing
         calendar = yield self.calendarUnderTest()
-        calendarObject = yield self.calendarObjectUnderTest("indexing.ics")
+        calendarObject = yield self.calendarObjectUnderTest(name="indexing.ics")
         component.noInstanceIndexing = True
         yield calendarObject.setComponent(component)
         instances3 = yield calendarObject.instances()

Modified: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/util.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/util.py	2013-04-08 19:22:13 UTC (rev 11015)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/datastore/util.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -98,6 +98,35 @@
 
 
 
+def normalizationLookup(cuaddr, principalFunction, config):
+    """
+    Lookup function to be passed to ical.normalizeCalendarUserAddresses.
+    Returns a tuple of (Full name, guid, and calendar user address list)
+    for the given cuaddr.  The principalFunction is called to retrieve the
+    principal for the cuaddr.
+    """
+    try:
+        principal = principalFunction(cuaddr)
+    except Exception, e:
+        log.debug("Lookup of %s failed: %s" % (cuaddr, e))
+        principal = None
+
+    if principal is None:
+        return (None, None, None)
+    else:
+        # RFC5545 syntax does not allow backslash escaping in
+        # parameter values. A double-quote is thus not allowed
+        # in a parameter value except as the start/end delimiters.
+        # Single quotes are allowed, so we convert any double-quotes
+        # to single-quotes.
+        return (
+            principal.fullName().replace('"', "'"),
+            principal.uid(),
+            principal.calendarUserAddresses(),
+        )
+
+
+
 @inlineCallbacks
 def dropboxIDFromCalendarObject(calendarObject):
     """

Modified: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/icalendarstore.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/icalendarstore.py	2013-04-08 19:22:13 UTC (rev 11015)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/caldav/icalendarstore.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -20,7 +20,7 @@
 """
 
 from txdav.common.icommondatastore import ICommonTransaction, \
-    IShareableCollection
+    IShareableCollection, CommonStoreError
 from txdav.idav import IDataStoreObject, IDataStore
 
 from twisted.internet.interfaces import ITransport
@@ -56,6 +56,41 @@
 
 
 
+class InvalidComponentTypeError(CommonStoreError):
+    """
+    Invalid object resource component type for collection.
+    """
+
+
+
+class InvalidCalendarAccessError(CommonStoreError):
+    """
+    Invalid access mode in calendar data.
+    """
+
+
+
+class TooManyAttendeesError(CommonStoreError):
+    """
+    Too many attendees in calendar data.
+    """
+
+
+
+class InvalidUIDError(CommonStoreError):
+    """
+    The UID of the component in a store operation does not match the existing value.
+    """
+
+
+
+class UIDExistsError(CommonStoreError):
+    """
+    The UID of the component in a store operation exists in another calendar belonging to the owner.
+    """
+
+
+
 class AttachmentStoreFailed(Exception):
     """
     Unable to store an attachment.

Modified: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/datastore/sql.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/datastore/sql.py	2013-04-08 19:22:13 UTC (rev 11015)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/datastore/sql.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -65,7 +65,7 @@
     HomeChildNameAlreadyExistsError, NoSuchHomeChildError, \
     ObjectResourceNameNotAllowedError, ObjectResourceNameAlreadyExistsError, \
     NoSuchObjectResourceError, AllRetriesFailed, InvalidSubscriptionValues, \
-    InvalidIMIPTokenValues
+    InvalidIMIPTokenValues, TooManyObjectResourcesError
 from txdav.common.inotifications import INotificationCollection, \
     INotificationObject
 
@@ -1454,11 +1454,43 @@
     @classproperty
     def _metaDataQuery(cls): #@NoSelf
         metadata = cls._homeMetaDataSchema
-        return Select([metadata.CREATED, metadata.MODIFIED],
+        return Select(cls.metadataColumns(),
                       From=metadata,
                       Where=metadata.RESOURCE_ID == Parameter("resourceID"))
 
 
+    @classmethod
+    def metadataColumns(cls):
+        """
+        Return a list of column name for retrieval of metadata. This allows
+        different child classes to have their own type specific data, but still make use of the
+        common base logic.
+        """
+
+        # Common behavior is to have created and modified
+
+        return (
+            cls._homeMetaDataSchema.CREATED,
+            cls._homeMetaDataSchema.MODIFIED,
+        )
+
+
+    @classmethod
+    def metadataAttributes(cls):
+        """
+        Return a list of attribute names for retrieval of metadata. This allows
+        different child classes to have their own type specific data, but still make use of the
+        common base logic.
+        """
+
+        # Common behavior is to have created and modified
+
+        return (
+            "_created",
+            "_modified",
+        )
+
+
     @inlineCallbacks
     def initFromStore(self, no_cache=False):
         """
@@ -1491,7 +1523,8 @@
                     # Cache the data
                     yield queryCacher.setAfterCommit(self._txn, cacheKey, data)
 
-            self._created, self._modified = data
+            for attr, value in zip(self.metadataAttributes(), data):
+                setattr(self, attr, value)
 
             yield self._loadPropertyStore()
             returnValue(self)
@@ -1597,6 +1630,14 @@
         return self._txn
 
 
+    @inlineCallbacks
+    def invalidateQueryCache(self):
+        queryCacher = self._txn._queryCacher
+        if queryCacher is not None:
+            cacheKey = queryCacher.keyForHomeMetaData(self._resourceID)
+            yield queryCacher.invalidateAfterCommit(self._txn, cacheKey)
+
+
     def name(self):
         """
         Implement L{IDataStoreObject.name} to return the uid.
@@ -2096,10 +2137,7 @@
 
         try:
             self._modified = (yield self._txn.subtransaction(_bumpModified, retries=0, failureOK=True))[0][0]
-            queryCacher = self._txn._queryCacher
-            if queryCacher is not None:
-                cacheKey = queryCacher.keyForHomeMetaData(self._resourceID)
-                yield queryCacher.invalidateAfterCommit(self._txn, cacheKey)
+            yield self.invalidateQueryCache()
 
         except AllRetriesFailed:
             log.debug("CommonHome.bumpModified failed")
@@ -3640,10 +3678,17 @@
         create the python object using the metadata then create the actual store
         object with setComponent.
         """
-        if name in self._objects:
-            if self._objects[name]:
-                raise ObjectResourceNameAlreadyExistsError()
 
+        # Create => a new resource name
+        if name in self._objects and self._objects[name]:
+            raise ObjectResourceNameAlreadyExistsError()
+
+        # Apply check to the size of the collection
+        if config.MaxResourcesPerCollection:
+            child_count = (yield self.countObjectResources())
+            if child_count >= config.MaxResourcesPerCollection:
+                raise TooManyObjectResourcesError()
+
         objectResource = (
             yield self._objectResourceClass.create(self, name, component,
                                                    metadata)

Modified: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/datastore/sql_schema/current.sql
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/datastore/sql_schema/current.sql	2013-04-08 19:22:13 UTC (rev 11015)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/datastore/sql_schema/current.sql	2013-04-08 20:53:03 UTC (rev 11016)
@@ -62,6 +62,8 @@
 create table CALENDAR_HOME_METADATA (
   RESOURCE_ID      integer      primary key references CALENDAR_HOME on delete cascade, -- implicit index
   QUOTA_USED_BYTES integer      default 0 not null,
+  DEFAULT_EVENTS   integer      default null,
+  DEFAULT_TASKS    integer      default null,
   CREATED          timestamp    default timezone('UTC', CURRENT_TIMESTAMP),
   MODIFIED         timestamp    default timezone('UTC', CURRENT_TIMESTAMP)
 );

Modified: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/datastore/test/util.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/datastore/test/util.py	2013-04-08 19:22:13 UTC (rev 11015)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/datastore/test/util.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -644,12 +644,12 @@
 
 
     @inlineCallbacks
-    def calendarObjectUnderTest(self, name="1.ics", txn=None):
+    def calendarObjectUnderTest(self, txn=None, name="1.ics", calendar_name="calendar_1", home="home1"):
         """
         Get the calendar detailed by
-        C{requirements['home1']['calendar_1'][name]}.
+        C{requirements[home][calendar_name][name]}.
         """
-        returnValue((yield (yield self.calendarUnderTest(txn))
+        returnValue((yield (yield self.calendarUnderTest(txn, name=calendar_name, home=home))
                      .calendarObjectWithName(name)))
 
 

Modified: CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/icommondatastore.py
===================================================================
--- CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/icommondatastore.py	2013-04-08 19:22:13 UTC (rev 11015)
+++ CalendarServer/branches/users/cdaboo/store-scheduling/txdav/common/icommondatastore.py	2013-04-08 20:53:03 UTC (rev 11016)
@@ -49,56 +49,85 @@
     Store generic error.
     """
 
+
+
 class NameNotAllowedError(CommonStoreError):
     """
     Attempt to create an object with a name that is not allowed.
     """
 
+
+
 class HomeChildNameNotAllowedError(NameNotAllowedError):
     """
     Home child name not allowed.
     """
 
+
+
 class ObjectResourceNameNotAllowedError(NameNotAllowedError):
     """
     Object resource name not allowed.
     """
 
+
+
 class AlreadyExistsError(CommonStoreError):
     """
     Attempt to create an object that already exists.
     """
 
+
+
 class HomeChildNameAlreadyExistsError(AlreadyExistsError):
     """
     Home child already exists.
     """
 
+
+
 class ObjectResourceNameAlreadyExistsError(AlreadyExistsError):
     """
     An object resource with the requested name already exists.
     """
 
+
+
 class ObjectResourceUIDAlreadyExistsError(AlreadyExistsError):
     """
     An object resource with the requested UID already exists.
     """
 
+
+
+class TooManyObjectResourcesError(CommonStoreError):
+    """
+    Home child has maximum allowed count of resources.
+    """
+
+
+
 class NotFoundError(CommonStoreError):
     """
     Requested data not found.
     """
 
+
+
 class NoSuchHomeChildError(NotFoundError):
     """
     The requested home child does not exist.
     """
 
+
+
 class NoSuchObjectResourceError(NotFoundError):
     """
     The requested object resource does not exist.
     """
 
+
+
 class ConcurrentModification(NotFoundError):
     """
     Despite being loaded in the current transaction, the object whose data is
@@ -111,54 +140,83 @@
     happen.)
     """
 
+
+
 class InvalidObjectResourceError(CommonStoreError):
     """
     Invalid object resource data.
     """
 
+
+
+class ObjectResourceTooBigError(CommonStoreError):
+    """
+    Object resource data is larger than allowed limit.
+    """
+
+
+
 class InternalDataStoreError(CommonStoreError):
     """
     Uh, oh.
     """
 
+
+
 class AllRetriesFailed(CommonStoreError):
     """
     In a re-tried subtransaction, all attempts failed to produce useful
     progress.  Other exceptions will be logged.
     """
 
+
 # Indexing / sync tokens
 
+
+
 class ReservationError(LookupError):
     """
     Attempt to reserve a UID which is already reserved or to unreserve a UID
     which is not reserved.
     """
 
+
+
 class IndexedSearchException(ValueError):
     pass
 
+
+
 class SyncTokenValidException(ValueError):
     pass
 
+
 # APN Subscriptions
 
+
+
 class InvalidSubscriptionValues(ValueError):
     """
     Invalid APN subscription values passed in.
     """
 
+
 # IMIP Tokens
 
+
+
 class InvalidIMIPTokenValues(ValueError):
     """
     Invalid IMIP token values passed in.
     """
 
+
 #
 # Interfaces
 #
 
+
+
 class ICommonTransaction(ITransaction):
     """
     Transaction functionality shared in common by calendar and addressbook
@@ -270,7 +328,6 @@
         @type token: C{str}
         """
 
-
     def imipGetToken(organizer, attendee, icaluid):
         """
         Returns the token (if any) corresponding to the given organizer, attendee,
@@ -284,7 +341,6 @@
         @type organizer: C{str}
         """
 
-
     def imipRemoveToken(token):
         """
         Removes the entry for the given token.
@@ -293,15 +349,14 @@
         @type token: C{str}
         """
 
-
     def purgeOldIMIPTokens(olderThan):
         """
         Removes all tokens whose access time is before olderThan
         """
-        
 
+
+
 class IShareableCollection(Interface):
     """
     A collection resource which may be shared.
     """
-
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/calendarserver-changes/attachments/20130408/44cdbf88/attachment-0001.html>


More information about the calendarserver-changes mailing list