[CalendarServer-changes] [10191] CalendarServer/trunk

source_changes at macosforge.org source_changes at macosforge.org
Wed Dec 19 07:07:20 PST 2012


Revision: 10191
          http://trac.calendarserver.org//changeset/10191
Author:   cdaboo at apple.com
Date:     2012-12-19 07:07:20 -0800 (Wed, 19 Dec 2012)
Log Message:
-----------
Merge r10180 and r10190 to trunk for attachment temp dir clean-up and attachment removal/purge fixes.

Revision Links:
--------------
    http://trac.calendarserver.org//changeset/10180
    http://trac.calendarserver.org//changeset/10190

Modified Paths:
--------------
    CalendarServer/trunk/calendarserver/tools/test/test_purge_old_events.py
    CalendarServer/trunk/txdav/caldav/datastore/file.py
    CalendarServer/trunk/txdav/caldav/datastore/sql.py
    CalendarServer/trunk/txdav/caldav/datastore/test/common.py
    CalendarServer/trunk/txdav/caldav/datastore/test/test_sql.py
    CalendarServer/trunk/txdav/common/datastore/file.py

Property Changed:
----------------
    CalendarServer/trunk/


Property changes on: CalendarServer/trunk
___________________________________________________________________
Modified: svn:mergeinfo
   - /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/generic-sqlstore:6167-6191
/CalendarServer/branches/new-store:5594-5934
/CalendarServer/branches/new-store-no-caldavfile:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
/CalendarServer/branches/release/CalendarServer-4.3-dev:10188
/CalendarServer/branches/users/cdaboo/batchupload-6699:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
/CalendarServer/branches/users/cdaboo/component-set-fixes:8130-8346
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/implicituidrace:8137-8141
/CalendarServer/branches/users/cdaboo/ischedule-dkim:9747-9979
/CalendarServer/branches/users/cdaboo/managed-attachments:9985-10145
/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/pods:7297-7377
/CalendarServer/branches/users/cdaboo/pycalendar:7085-7206
/CalendarServer/branches/users/cdaboo/pycard:7227-7237
/CalendarServer/branches/users/cdaboo/queued-attendee-refreshes:7740-8287
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
/CalendarServer/branches/users/cdaboo/timezones:7443-7699
/CalendarServer/branches/users/cdaboo/txn-debugging:8730-8743
/CalendarServer/branches/users/glyph/always-abort-txn-on-error:9958-9969
/CalendarServer/branches/users/glyph/case-insensitive-uid:8772-8805
/CalendarServer/branches/users/glyph/conn-limit:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/dalify:6932-7023
/CalendarServer/branches/users/glyph/db-reconnect:6824-6876
/CalendarServer/branches/users/glyph/deploybuild:7563-7572
/CalendarServer/branches/users/glyph/disable-quota:7718-7727
/CalendarServer/branches/users/glyph/dont-start-postgres:6592-6614
/CalendarServer/branches/users/glyph/imip-and-admin-html:7866-7984
/CalendarServer/branches/users/glyph/ipv6-client:9054-9105
/CalendarServer/branches/users/glyph/linux-tests:6893-6900
/CalendarServer/branches/users/glyph/migrate-merge:8690-8713
/CalendarServer/branches/users/glyph/misc-portability-fixes:7365-7374
/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
/CalendarServer/branches/users/glyph/more-deferreds-7:6369-6445
/CalendarServer/branches/users/glyph/multiget-delete:8321-8330
/CalendarServer/branches/users/glyph/new-export:7444-7485
/CalendarServer/branches/users/glyph/one-home-list-api:10048-10073
/CalendarServer/branches/users/glyph/oracle:7106-7155
/CalendarServer/branches/users/glyph/oracle-nulls:7340-7351
/CalendarServer/branches/users/glyph/other-html:8062-8091
/CalendarServer/branches/users/glyph/parallel-sim:8240-8251
/CalendarServer/branches/users/glyph/parallel-upgrade:8376-8400
/CalendarServer/branches/users/glyph/parallel-upgrade_to_1:8571-8583
/CalendarServer/branches/users/glyph/q:9560-9688
/CalendarServer/branches/users/glyph/quota:7604-7637
/CalendarServer/branches/users/glyph/sendfdport:5388-5424
/CalendarServer/branches/users/glyph/shared-pool-fixes:8436-8443
/CalendarServer/branches/users/glyph/shared-pool-take2:8155-8174
/CalendarServer/branches/users/glyph/sharedpool:6490-6550
/CalendarServer/branches/users/glyph/sharing-api:9192-9205
/CalendarServer/branches/users/glyph/skip-lonely-vtimezones:8524-8535
/CalendarServer/branches/users/glyph/sql-store:5929-6073
/CalendarServer/branches/users/glyph/subtransactions:7248-7258
/CalendarServer/branches/users/glyph/table-alias:8651-8664
/CalendarServer/branches/users/glyph/uidexport:7673-7676
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/glyph/uuid-normalize:9268-9296
/CalendarServer/branches/users/glyph/xattrs-from-files:7757-7769
/CalendarServer/branches/users/sagen/applepush:8126-8184
/CalendarServer/branches/users/sagen/inboxitems:7380-7381
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/branches/users/wsanchez/transations:5515-5593
   + /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/generic-sqlstore:6167-6191
/CalendarServer/branches/new-store:5594-5934
/CalendarServer/branches/new-store-no-caldavfile:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
/CalendarServer/branches/release/CalendarServer-4.3-dev:10180-10190
/CalendarServer/branches/users/cdaboo/batchupload-6699:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
/CalendarServer/branches/users/cdaboo/component-set-fixes:8130-8346
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/implicituidrace:8137-8141
/CalendarServer/branches/users/cdaboo/ischedule-dkim:9747-9979
/CalendarServer/branches/users/cdaboo/managed-attachments:9985-10145
/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/pods:7297-7377
/CalendarServer/branches/users/cdaboo/pycalendar:7085-7206
/CalendarServer/branches/users/cdaboo/pycard:7227-7237
/CalendarServer/branches/users/cdaboo/queued-attendee-refreshes:7740-8287
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
/CalendarServer/branches/users/cdaboo/timezones:7443-7699
/CalendarServer/branches/users/cdaboo/txn-debugging:8730-8743
/CalendarServer/branches/users/glyph/always-abort-txn-on-error:9958-9969
/CalendarServer/branches/users/glyph/case-insensitive-uid:8772-8805
/CalendarServer/branches/users/glyph/conn-limit:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/dalify:6932-7023
/CalendarServer/branches/users/glyph/db-reconnect:6824-6876
/CalendarServer/branches/users/glyph/deploybuild:7563-7572
/CalendarServer/branches/users/glyph/disable-quota:7718-7727
/CalendarServer/branches/users/glyph/dont-start-postgres:6592-6614
/CalendarServer/branches/users/glyph/imip-and-admin-html:7866-7984
/CalendarServer/branches/users/glyph/ipv6-client:9054-9105
/CalendarServer/branches/users/glyph/linux-tests:6893-6900
/CalendarServer/branches/users/glyph/migrate-merge:8690-8713
/CalendarServer/branches/users/glyph/misc-portability-fixes:7365-7374
/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
/CalendarServer/branches/users/glyph/more-deferreds-7:6369-6445
/CalendarServer/branches/users/glyph/multiget-delete:8321-8330
/CalendarServer/branches/users/glyph/new-export:7444-7485
/CalendarServer/branches/users/glyph/one-home-list-api:10048-10073
/CalendarServer/branches/users/glyph/oracle:7106-7155
/CalendarServer/branches/users/glyph/oracle-nulls:7340-7351
/CalendarServer/branches/users/glyph/other-html:8062-8091
/CalendarServer/branches/users/glyph/parallel-sim:8240-8251
/CalendarServer/branches/users/glyph/parallel-upgrade:8376-8400
/CalendarServer/branches/users/glyph/parallel-upgrade_to_1:8571-8583
/CalendarServer/branches/users/glyph/q:9560-9688
/CalendarServer/branches/users/glyph/quota:7604-7637
/CalendarServer/branches/users/glyph/sendfdport:5388-5424
/CalendarServer/branches/users/glyph/shared-pool-fixes:8436-8443
/CalendarServer/branches/users/glyph/shared-pool-take2:8155-8174
/CalendarServer/branches/users/glyph/sharedpool:6490-6550
/CalendarServer/branches/users/glyph/sharing-api:9192-9205
/CalendarServer/branches/users/glyph/skip-lonely-vtimezones:8524-8535
/CalendarServer/branches/users/glyph/sql-store:5929-6073
/CalendarServer/branches/users/glyph/subtransactions:7248-7258
/CalendarServer/branches/users/glyph/table-alias:8651-8664
/CalendarServer/branches/users/glyph/uidexport:7673-7676
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/glyph/uuid-normalize:9268-9296
/CalendarServer/branches/users/glyph/xattrs-from-files:7757-7769
/CalendarServer/branches/users/sagen/applepush:8126-8184
/CalendarServer/branches/users/sagen/inboxitems:7380-7381
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/branches/users/wsanchez/transations:5515-5593

Modified: CalendarServer/trunk/calendarserver/tools/test/test_purge_old_events.py
===================================================================
--- CalendarServer/trunk/calendarserver/tools/test/test_purge_old_events.py	2012-12-18 22:06:38 UTC (rev 10190)
+++ CalendarServer/trunk/calendarserver/tools/test/test_purge_old_events.py	2012-12-19 15:07:20 UTC (rev 10191)
@@ -17,9 +17,14 @@
 """
 Tests for calendarserver.tools.purge
 """
+
 from calendarserver.tap.util import getRootResource
 from calendarserver.tools.purge import purgeOldEvents, purgeUID, purgeOrphanedAttachments
 
+from pycalendar.datetime import PyCalendarDateTime
+from pycalendar.timezone import PyCalendarTimezone
+
+from twext.enterprise.dal.syntax import Update
 from twext.web2.http_headers import MimeType
 
 from twisted.internet.defer import inlineCallbacks, returnValue
@@ -28,11 +33,9 @@
 from twistedcaldav.config import config
 from twistedcaldav.vcard import Component as VCardComponent
 
+from txdav.common.datastore.sql_tables import schema
 from txdav.common.datastore.test.util import buildStore, populateCalendarsFrom, CommonCommonTests
 
-from pycalendar.datetime import PyCalendarDateTime
-from pycalendar.timezone import PyCalendarTimezone
-
 import os
 
 
@@ -429,7 +432,7 @@
 
 
     @inlineCallbacks
-    def _addAttachment(self):
+    def _addAttachment(self, orphan=False):
 
         txn = self._sqlCalendarStore.newTransaction()
 
@@ -442,6 +445,15 @@
         t.write("old attachment")
         t.write(" text")
         (yield t.loseConnection())
+
+        if orphan:
+            # Reset dropbox id in calendar_object
+            co = schema.CALENDAR_OBJECT
+            Update(
+                {co.DROPBOX_ID: None, },
+                Where=co.RESOURCE_ID == event._resourceID,
+            ).on(txn)
+
         (yield txn.commit())
 
         returnValue(attachment)
@@ -449,44 +461,48 @@
 
     @inlineCallbacks
     def test_removeOrphanedAttachments(self):
+
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        self.assertEqual(quota, 0)
+
         attachment = (yield self._addAttachment())
-        txn = self._sqlCalendarStore.newTransaction()
         attachmentPath = attachment._path.path
         self.assertTrue(os.path.exists(attachmentPath))
+        (yield self.commit())
 
-        orphans = (yield txn.orphanedAttachments())
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        self.assertNotEqual(quota, 0)
+
+        orphans = (yield self.transactionUnderTest().orphanedAttachments())
         self.assertEquals(len(orphans), 0)
 
-        count = (yield txn.removeOrphanedAttachments(batchSize=100))
+        count = (yield self.transactionUnderTest().removeOrphanedAttachments(batchSize=100))
         self.assertEquals(count, 0)
+        (yield self.commit())
 
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        self.assertNotEqual(quota, 0)
+
         # File still exists
         self.assertTrue(os.path.exists(attachmentPath))
 
         # Delete all old events (including the event containing the attachment)
         cutoff = PyCalendarDateTime(now, 4, 1, 0, 0, 0)
-        count = (yield txn.removeOldEvents(cutoff))
+        count = (yield self.transactionUnderTest().removeOldEvents(cutoff))
+        (yield self.commit())
 
-        # Just look for orphaned attachments but don't delete
-        orphans = (yield txn.orphanedAttachments())
-        self.assertEquals(len(orphans), 1)
-        self.assertEquals(orphans, [["home1", 19, 19, 1]])
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        self.assertEqual(quota, 0)
 
-        # Remove orphaned attachments, should be 1
-        count = (yield txn.removeOrphanedAttachments(batchSize=100))
-        self.assertEquals(count, 1)
+        # Just look for orphaned attachments - none left
+        orphans = (yield self.transactionUnderTest().orphanedAttachments())
+        self.assertEquals(len(orphans), 0)
 
-        # Remove orphaned attachments, shouldn't be any
-        count = (yield txn.removeOrphanedAttachments())
-        self.assertEquals(count, 0)
 
-        # File isn't actually removed until after commit
-        (yield txn.commit())
-
-        # Verify the file itself is gone
-        self.assertFalse(os.path.exists(attachmentPath))
-
-
     @inlineCallbacks
     def test_purgeOldEvents(self):
 
@@ -581,23 +597,46 @@
     @inlineCallbacks
     def test_purgeOrphanedAttachments(self):
 
-        (yield self._addAttachment())
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        self.assertEqual(quota, 0)
 
+        (yield self._addAttachment(orphan=True))
+        (yield self.commit())
+
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        self.assertNotEqual(quota, 0)
+
         # Remove old events first
         total = (yield purgeOldEvents(self._sqlCalendarStore, self.directory,
             self.rootResource, PyCalendarDateTime(now, 4, 1, 0, 0, 0), 2, verbose=False))
         self.assertEquals(total, 4)
 
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        self.assertNotEqual(quota, 0)
+
         # Dry run
         total = (yield purgeOrphanedAttachments(self._sqlCalendarStore, 2,
             dryrun=True, verbose=False))
         self.assertEquals(total, 1)
+        (yield self.commit())
 
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        self.assertNotEqual(quota, 0)
+
         # Actually remove
         total = (yield purgeOrphanedAttachments(self._sqlCalendarStore, 2,
             dryrun=False, verbose=False))
         self.assertEquals(total, 1)
+        (yield self.commit())
 
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quotaAfter = (yield home.quotaUsedBytes())
+        self.assertEqual(quotaAfter, 0)
+
         # There should be no more left
         total = (yield purgeOrphanedAttachments(self._sqlCalendarStore, 2,
             dryrun=False, verbose=False))

Modified: CalendarServer/trunk/txdav/caldav/datastore/file.py
===================================================================
--- CalendarServer/trunk/txdav/caldav/datastore/file.py	2012-12-18 22:06:38 UTC (rev 10190)
+++ CalendarServer/trunk/txdav/caldav/datastore/file.py	2012-12-19 15:07:20 UTC (rev 10191)
@@ -264,7 +264,6 @@
     def resourceType(self):
         return ResourceType.calendar #@UndefinedVariable
 
-
     ownerCalendarHome = CommonHomeChild.ownerHome
     viewerCalendarHome = CommonHomeChild.viewerHome
     calendarObjects = CommonHomeChild.objectResources
@@ -498,6 +497,10 @@
         return component
 
 
+    def remove(self):
+        pass
+
+
     def _text(self):
         if self._objectText is not None:
             return self._objectText
@@ -770,7 +773,22 @@
         self._path = self._attachment._path.temporarySibling()
         self._file = self._path.open("w")
 
+        self._txn.postAbort(self.aborted)
 
+
+    @property
+    def _txn(self):
+        return self._attachment._txn
+
+
+    def aborted(self):
+        """
+        Transaction aborted - clean up temp files.
+        """
+        if self._path.exists():
+            self._path.remove()
+
+
     def write(self, data):
         # FIXME: multiple chunks
         self._file.write(data)
@@ -823,6 +841,11 @@
         self._dropboxPath = dropboxPath
 
 
+    @property
+    def _txn(self):
+        return self._calendarObject._txn
+
+
     def name(self):
         return self._name
 
@@ -898,4 +921,3 @@
             calendarObject._componentType = componentType
 
             yield calendarObject
-

Modified: CalendarServer/trunk/txdav/caldav/datastore/sql.py
===================================================================
--- CalendarServer/trunk/txdav/caldav/datastore/sql.py	2012-12-18 22:06:38 UTC (rev 10190)
+++ CalendarServer/trunk/txdav/caldav/datastore/sql.py	2012-12-19 15:07:20 UTC (rev 10191)
@@ -1125,6 +1125,8 @@
     @inlineCallbacks
     def remove(self):
         # Need to also remove attachments
+        if self._dropboxID:
+            yield DropBoxAttachment.resourceRemoved(self._txn, self._resourceID, self._dropboxID)
         yield ManagedAttachment.resourceRemoved(self._txn, self._resourceID)
         yield super(CalendarObject, self).remove()
 
@@ -1701,7 +1703,9 @@
         self._hash = hashlib.md5()
         self._creating = creating
 
+        self._txn.postAbort(self.aborted)
 
+
     def _temporaryFile(self):
         """
         Returns a (file descriptor, absolute path) tuple for a temporary file within
@@ -1721,6 +1725,14 @@
         return self._attachment._txn
 
 
+    def aborted(self):
+        """
+        Transaction aborted - clean up temp files.
+        """
+        if self._path.exists():
+            self._path.remove()
+
+
     def write(self, data):
         if isinstance(data, buffer):
             data = str(data)
@@ -2050,7 +2062,37 @@
         return attachmentRoot.child(self.name())
 
 
+    @classmethod
     @inlineCallbacks
+    def resourceRemoved(cls, txn, resourceID, dropboxID):
+        """
+        Remove all attachments referencing the specified resource.
+        """
+
+        # See if any other resources still reference this dropbox ID
+        co = schema.CALENDAR_OBJECT
+        rows = (yield Select(
+            [co.RESOURCE_ID, ],
+            From=co,
+            Where=(co.DROPBOX_ID == dropboxID).And(
+                co.RESOURCE_ID != resourceID)
+        ).on(txn))
+
+        if not rows:
+            # Find each attachment with matching dropbox ID
+            att = schema.ATTACHMENT
+            rows = (yield Select(
+                [att.PATH],
+                From=att,
+                Where=(att.DROPBOX_ID == dropboxID)
+            ).on(txn))
+            for name in rows:
+                name = name[0]
+                attachment = yield cls.load(txn, dropboxID, name)
+                yield attachment.remove()
+
+
+    @inlineCallbacks
     def changed(self, contentType, dispositionName, md5, size):
         """
         Dropbox attachments never change their path - ignore dispositionName.

Modified: CalendarServer/trunk/txdav/caldav/datastore/test/common.py
===================================================================
--- CalendarServer/trunk/txdav/caldav/datastore/test/common.py	2012-12-18 22:06:38 UTC (rev 10190)
+++ CalendarServer/trunk/txdav/caldav/datastore/test/common.py	2012-12-19 15:07:20 UTC (rev 10191)
@@ -20,6 +20,7 @@
 """
 
 from StringIO import StringIO
+import os
 
 from twisted.internet.defer import Deferred, inlineCallbacks, returnValue, \
     maybeDeferred
@@ -2035,6 +2036,37 @@
 
 
     @inlineCallbacks
+    def test_attachmentTemporaryFileCleanup(self):
+        """
+        L{IAttachmentStream} object cleans-up its temporary file on txn abort.
+        """
+        obj = yield self.calendarObjectUnderTest()
+        attachment = yield obj.createAttachmentWithName(
+            "new.attachment",
+        )
+        t = attachment.store(MimeType("text", "x-fixture"))
+
+        temp = t._path.path
+
+        yield self.abort()
+
+        self.assertFalse(os.path.exists(temp))
+
+        obj = yield self.calendarObjectUnderTest()
+        attachment = yield obj.createAttachmentWithName(
+            "new.attachment",
+        )
+        t = attachment.store(MimeType("text", "x-fixture"))
+
+        temp = t._path.path
+        os.remove(temp)
+
+        yield self.abort()
+
+        self.assertFalse(os.path.exists(temp))
+
+
+    @inlineCallbacks
     def test_quotaAllowedBytes(self):
         """
         L{ICalendarHome.quotaAllowedBytes} should return the configuration value
@@ -2292,8 +2324,12 @@
         for uid in additionalUIDs:
             yield txn.calendarHomeWithUID(uid, create=True)
         yield self.commit()
+
+
         # try to create a calendar in all of them, then fail.
-        class AnException(Exception): pass
+        class AnException(Exception):
+            pass
+
         caught = []
         @inlineCallbacks
         def toEachCalendarHome(txn, eachHome):
@@ -2315,4 +2351,3 @@
         yield noNewCalendar(caught[0])
         yield noNewCalendar('home2')
         yield noNewCalendar('home3')
-

Modified: CalendarServer/trunk/txdav/caldav/datastore/test/test_sql.py
===================================================================
--- CalendarServer/trunk/txdav/caldav/datastore/test/test_sql.py	2012-12-18 22:06:38 UTC (rev 10190)
+++ CalendarServer/trunk/txdav/caldav/datastore/test/test_sql.py	2012-12-19 15:07:20 UTC (rev 10191)
@@ -19,15 +19,25 @@
 L{txdav.caldav.datastore.test.common}.
 """
 
+from pycalendar.datetime import PyCalendarDateTime
+from pycalendar.timezone import PyCalendarTimezone
+
+from twext.enterprise.dal.syntax import Select, Parameter, Insert
+from twext.python.vcomponent import VComponent
+from twext.web2.http_headers import MimeType
+
 from twisted.internet import reactor
 from twisted.internet.defer import inlineCallbacks, returnValue
 from twisted.internet.task import deferLater
 from twisted.python import hashlib
 from twisted.trial import unittest
 
-from twext.enterprise.dal.syntax import Select, Parameter, Insert
-from twext.python.vcomponent import VComponent
-from txdav.xml.rfc2518 import GETContentLanguage, ResourceType
+from twistedcaldav import caldavxml
+from twistedcaldav.caldavxml import CalendarDescription
+from twistedcaldav.config import config
+from twistedcaldav.dateops import datetimeMktime
+from twistedcaldav.ical import Component
+from twistedcaldav.query import calendarqueryfilter
 
 from txdav.base.propertystore.base import PropertyName
 from txdav.caldav.datastore.test.common import CommonTests as CalendarCommonTests, \
@@ -40,17 +50,10 @@
     _BIND_STATUS_ACCEPTED
 from txdav.common.datastore.test.util import buildStore, populateCalendarsFrom
 from txdav.common.icommondatastore import NoSuchObjectResourceError
+from txdav.xml.rfc2518 import GETContentLanguage, ResourceType
 
-from twistedcaldav import caldavxml
-from twistedcaldav.caldavxml import CalendarDescription
-from twistedcaldav.config import config
-from twistedcaldav.dateops import datetimeMktime
-from twistedcaldav.ical import Component
-from twistedcaldav.query import calendarqueryfilter
-
 import datetime
-from pycalendar.datetime import PyCalendarDateTime
-from pycalendar.timezone import PyCalendarTimezone
+import os
 
 class CalendarSQLStorageTests(CalendarCommonTests, unittest.TestCase):
     """
@@ -1433,3 +1436,200 @@
         obj = (yield self.calendarObjectUnderTest())
         calendarObject = (yield home.objectResourceWithID(obj._resourceID))
         self.assertNotEquals(calendarObject, None)
+
+
+    @inlineCallbacks
+    def test_cleanupAttachments(self):
+        """
+        L{ICalendarObject.remove} will remove an associated calendar
+        attachment.
+        """
+
+        # Create attachment
+        obj = yield self.calendarObjectUnderTest()
+        attachment = yield obj.createAttachmentWithName(
+            "new.attachment",
+        )
+        t = attachment.store(MimeType("text", "x-fixture"))
+        t.write("new attachment")
+        t.write(" text")
+        yield t.loseConnection()
+        apath = attachment._path.path
+        yield self.commit()
+
+        self.assertTrue(os.path.exists(apath))
+
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        yield self.commit()
+        self.assertNotEqual(quota, 0)
+
+        # Remove resource
+        obj = yield self.calendarObjectUnderTest()
+        yield obj.remove()
+        yield self.commit()
+
+        self.assertFalse(os.path.exists(apath))
+
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        yield self.commit()
+        self.assertEqual(quota, 0)
+
+
+    @inlineCallbacks
+    def test_cleanupMultipleAttachments(self):
+        """
+        L{ICalendarObject.remove} will remove all associated calendar
+        attachments.
+        """
+
+        # Create attachment
+        obj = yield self.calendarObjectUnderTest()
+
+        attachment = yield obj.createAttachmentWithName(
+            "new.attachment",
+        )
+        t = attachment.store(MimeType("text", "x-fixture"))
+        t.write("new attachment")
+        t.write(" text")
+        yield t.loseConnection()
+        apath1 = attachment._path.path
+
+        attachment = yield obj.createAttachmentWithName(
+            "new.attachment2",
+        )
+        t = attachment.store(MimeType("text", "x-fixture"))
+        t.write("new attachment 2")
+        t.write(" text")
+        yield t.loseConnection()
+        apath2 = attachment._path.path
+
+        yield self.commit()
+
+        self.assertTrue(os.path.exists(apath1))
+        self.assertTrue(os.path.exists(apath2))
+
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        yield self.commit()
+        self.assertNotEqual(quota, 0)
+
+        # Remove resource
+        obj = yield self.calendarObjectUnderTest()
+        yield obj.remove()
+        yield self.commit()
+
+        self.assertFalse(os.path.exists(apath1))
+        self.assertFalse(os.path.exists(apath2))
+
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        yield self.commit()
+        self.assertEqual(quota, 0)
+
+
+    @inlineCallbacks
+    def test_cleanupAttachmentsOnMultipleResources(self):
+        """
+        L{ICalendarObject.remove} will remove all associated calendar
+        attachments unless used in another resource.
+        """
+
+        # Create attachment
+        obj = yield self.calendarObjectUnderTest()
+
+        attachment = yield obj.createAttachmentWithName(
+            "new.attachment",
+        )
+        t = attachment.store(MimeType("text", "x-fixture"))
+        t.write("new attachment")
+        t.write(" text")
+        yield t.loseConnection()
+        apath = attachment._path.path
+
+        new_component = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//Apple Inc.//iCal 4.0.1//EN
+CALSCALE:GREGORIAN
+BEGIN:VTIMEZONE
+TZID:US/Pacific
+BEGIN:DAYLIGHT
+TZOFFSETFROM:-0800
+RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
+DTSTART:20070311T020000
+TZNAME:PDT
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:STANDARD
+TZOFFSETFROM:-0700
+RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
+DTSTART:20071104T020000
+TZNAME:PST
+TZOFFSETTO:-0800
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+ATTENDEE;CN="Wilfredo Sanchez";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailt
+ o:wsanchez at example.com
+ATTENDEE;CN="Cyrus Daboo";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mailto:cda
+ boo at example.com
+DTEND;TZID=US/Pacific:%(now)s0324T124500
+TRANSP:OPAQUE
+ORGANIZER;CN="Wilfredo Sanchez":mailto:wsanchez at example.com
+UID:uid1-attachmenttest
+DTSTAMP:20090326T145447Z
+LOCATION:Wilfredo's Office
+SEQUENCE:2
+X-APPLE-EWS-BUSYSTATUS:BUSY
+X-APPLE-DROPBOX:/calendars/__uids__/user01/dropbox/FE5CDC6F-7776-4607-83
+ A9-B90FF7ACC8D0.dropbox
+SUMMARY:CalDAV protocol updates
+DTSTART;TZID=US/Pacific:%(now)s0324T121500
+CREATED:20090326T145440Z
+BEGIN:VALARM
+X-WR-ALARMUID:DB39AB67-449C-441C-89D2-D740B5F41A73
+TRIGGER;VALUE=DATE-TIME:%(now)s0324T180009Z
+ACTION:AUDIO
+END:VALARM
+END:VEVENT
+END:VCALENDAR
+""".replace("\n", "\r\n") % {"now": 2012}
+
+        calendar = yield self.calendarUnderTest()
+        yield calendar.createCalendarObjectWithName(
+            "test.ics", VComponent.fromString(new_component)
+        )
+
+        yield self.commit()
+
+        self.assertTrue(os.path.exists(apath))
+
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        yield self.commit()
+        self.assertNotEqual(quota, 0)
+
+        # Remove resource
+        obj = yield self.calendarObjectUnderTest()
+        yield obj.remove()
+        yield self.commit()
+
+        self.assertTrue(os.path.exists(apath))
+
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        yield self.commit()
+        self.assertNotEqual(quota, 0)
+
+        # Remove resource
+        obj = yield self.calendarObjectUnderTest("test.ics")
+        yield obj.remove()
+        yield self.commit()
+
+        self.assertFalse(os.path.exists(apath))
+
+        home = (yield self.transactionUnderTest().calendarHomeWithUID("home1"))
+        quota = (yield home.quotaUsedBytes())
+        yield self.commit()
+        self.assertEqual(quota, 0)

Modified: CalendarServer/trunk/txdav/common/datastore/file.py
===================================================================
--- CalendarServer/trunk/txdav/common/datastore/file.py	2012-12-18 22:06:38 UTC (rev 10190)
+++ CalendarServer/trunk/txdav/common/datastore/file.py	2012-12-19 15:07:20 UTC (rev 10191)
@@ -388,6 +388,7 @@
         self.summary = summary
 
 
+
 class SharedCollectionsDatabase(AbstractSQLDatabase, LoggingMixIn):
 
     db_basename = db_prefix + "shares"
@@ -530,6 +531,8 @@
 
         return SharedCollectionRecord(*[str(item) if type(item) == types.UnicodeType else item for item in row])
 
+
+
 class CommonHome(FileMetaDataMixin, LoggingMixIn):
 
     # All these need to be initialized by derived classes for each store type
@@ -1267,6 +1270,7 @@
                 self._transaction.postCommit(notifier.notify)
             self._transaction.notificationAddedForObject(self)
 
+
     @inlineCallbacks
     def asInvited(self):
         """
@@ -1275,6 +1279,7 @@
         yield None
         returnValue([])
 
+
     @inlineCallbacks
     def asShared(self):
         """
@@ -1283,6 +1288,8 @@
         yield None
         returnValue([])
 
+
+
 class CommonObjectResource(FileMetaDataMixin, LoggingMixIn, FancyEqMixin):
     """
     @ivar _path: The path of the file on disk
@@ -1311,6 +1318,11 @@
         return "<%s: %s>" % (self.__class__.__name__, self._path.path)
 
 
+    @property
+    def _txn(self):
+        return self._transaction
+
+
     def transaction(self):
         return self._transaction
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/calendarserver-changes/attachments/20121219/04862e33/attachment-0001.html>


More information about the calendarserver-changes mailing list