[CalendarServer-changes] [9868] CalendarServer/branches/release/CalendarServer-4.2-dev

source_changes at macosforge.org source_changes at macosforge.org
Thu Sep 27 10:52:04 PDT 2012 

Revision: 9868
          http://trac.calendarserver.org//changeset/9868
Author:   sagen at apple.com
Date:     2012-09-27 10:52:04 -0700 (Thu, 27 Sep 2012)
Log Message:
-----------
Merge r9867 from trunk

Revision Links:
--------------
    http://trac.calendarserver.org//changeset/9867

Modified Paths:
--------------
    CalendarServer/branches/release/CalendarServer-4.2-dev/calendarserver/tap/caldav.py
    CalendarServer/branches/release/CalendarServer-4.2-dev/calendarserver/tap/test/test_caldav.py
    CalendarServer/branches/release/CalendarServer-4.2-dev/twext/internet/tcp.py
    CalendarServer/branches/release/CalendarServer-4.2-dev/twext/web2/channel/http.py
    CalendarServer/branches/release/CalendarServer-4.2-dev/twext/web2/metafd.py
    CalendarServer/branches/release/CalendarServer-4.2-dev/twext/web2/test/test_http.py

Property Changed:
----------------
    CalendarServer/branches/release/CalendarServer-4.2-dev/


Property changes on: CalendarServer/branches/release/CalendarServer-4.2-dev
___________________________________________________________________
Modified: svn:mergeinfo
   - /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/generic-sqlstore:6167-6191
/CalendarServer/branches/new-store:5594-5934
/CalendarServer/branches/new-store-no-caldavfile:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
/CalendarServer/branches/users/cdaboo/batchupload-6699:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
/CalendarServer/branches/users/cdaboo/component-set-fixes:8130-8346
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/implicituidrace:8137-8141
/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/pods:7297-7377
/CalendarServer/branches/users/cdaboo/pycalendar:7085-7206
/CalendarServer/branches/users/cdaboo/pycard:7227-7237
/CalendarServer/branches/users/cdaboo/queued-attendee-refreshes:7740-8287
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
/CalendarServer/branches/users/cdaboo/timezones:7443-7699
/CalendarServer/branches/users/cdaboo/txn-debugging:8730-8743
/CalendarServer/branches/users/glyph/case-insensitive-uid:8772-8805
/CalendarServer/branches/users/glyph/conn-limit:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/dalify:6932-7023
/CalendarServer/branches/users/glyph/db-reconnect:6824-6876
/CalendarServer/branches/users/glyph/deploybuild:7563-7572
/CalendarServer/branches/users/glyph/disable-quota:7718-7727
/CalendarServer/branches/users/glyph/dont-start-postgres:6592-6614
/CalendarServer/branches/users/glyph/imip-and-admin-html:7866-7984
/CalendarServer/branches/users/glyph/ipv6-client:9054-9105
/CalendarServer/branches/users/glyph/linux-tests:6893-6900
/CalendarServer/branches/users/glyph/migrate-merge:8690-8713
/CalendarServer/branches/users/glyph/misc-portability-fixes:7365-7374
/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
/CalendarServer/branches/users/glyph/more-deferreds-7:6369-6445
/CalendarServer/branches/users/glyph/multiget-delete:8321-8330
/CalendarServer/branches/users/glyph/new-export:7444-7485
/CalendarServer/branches/users/glyph/oracle:7106-7155
/CalendarServer/branches/users/glyph/oracle-nulls:7340-7351
/CalendarServer/branches/users/glyph/other-html:8062-8091
/CalendarServer/branches/users/glyph/parallel-sim:8240-8251
/CalendarServer/branches/users/glyph/parallel-upgrade:8376-8400
/CalendarServer/branches/users/glyph/parallel-upgrade_to_1:8571-8583
/CalendarServer/branches/users/glyph/q:9560-9688
/CalendarServer/branches/users/glyph/quota:7604-7637
/CalendarServer/branches/users/glyph/sendfdport:5388-5424
/CalendarServer/branches/users/glyph/shared-pool-fixes:8436-8443
/CalendarServer/branches/users/glyph/shared-pool-take2:8155-8174
/CalendarServer/branches/users/glyph/sharedpool:6490-6550
/CalendarServer/branches/users/glyph/sharing-api:9192-9205
/CalendarServer/branches/users/glyph/skip-lonely-vtimezones:8524-8535
/CalendarServer/branches/users/glyph/sql-store:5929-6073
/CalendarServer/branches/users/glyph/subtransactions:7248-7258
/CalendarServer/branches/users/glyph/table-alias:8651-8664
/CalendarServer/branches/users/glyph/uidexport:7673-7676
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/glyph/uuid-normalize:9268-9296
/CalendarServer/branches/users/glyph/xattrs-from-files:7757-7769
/CalendarServer/branches/users/sagen/applepush:8126-8184
/CalendarServer/branches/users/sagen/inboxitems:7380-7381
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/branches/users/wsanchez/transations:5515-5593
   + /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/generic-sqlstore:6167-6191
/CalendarServer/branches/new-store:5594-5934
/CalendarServer/branches/new-store-no-caldavfile:5911-5935
/CalendarServer/branches/new-store-no-caldavfile-2:5936-5981
/CalendarServer/branches/users/cdaboo/batchupload-6699:6700-7198
/CalendarServer/branches/users/cdaboo/cached-subscription-calendars-5692:5693-5702
/CalendarServer/branches/users/cdaboo/component-set-fixes:8130-8346
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/implicituidrace:8137-8141
/CalendarServer/branches/users/cdaboo/more-sharing-5591:5592-5601
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/pods:7297-7377
/CalendarServer/branches/users/cdaboo/pycalendar:7085-7206
/CalendarServer/branches/users/cdaboo/pycard:7227-7237
/CalendarServer/branches/users/cdaboo/queued-attendee-refreshes:7740-8287
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/cdaboo/shared-calendars-5187:5188-5440
/CalendarServer/branches/users/cdaboo/timezones:7443-7699
/CalendarServer/branches/users/cdaboo/txn-debugging:8730-8743
/CalendarServer/branches/users/glyph/case-insensitive-uid:8772-8805
/CalendarServer/branches/users/glyph/conn-limit:6574-6577
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/dalify:6932-7023
/CalendarServer/branches/users/glyph/db-reconnect:6824-6876
/CalendarServer/branches/users/glyph/deploybuild:7563-7572
/CalendarServer/branches/users/glyph/disable-quota:7718-7727
/CalendarServer/branches/users/glyph/dont-start-postgres:6592-6614
/CalendarServer/branches/users/glyph/imip-and-admin-html:7866-7984
/CalendarServer/branches/users/glyph/ipv6-client:9054-9105
/CalendarServer/branches/users/glyph/linux-tests:6893-6900
/CalendarServer/branches/users/glyph/migrate-merge:8690-8713
/CalendarServer/branches/users/glyph/misc-portability-fixes:7365-7374
/CalendarServer/branches/users/glyph/more-deferreds-6:6322-6368
/CalendarServer/branches/users/glyph/more-deferreds-7:6369-6445
/CalendarServer/branches/users/glyph/multiget-delete:8321-8330
/CalendarServer/branches/users/glyph/new-export:7444-7485
/CalendarServer/branches/users/glyph/oracle:7106-7155
/CalendarServer/branches/users/glyph/oracle-nulls:7340-7351
/CalendarServer/branches/users/glyph/other-html:8062-8091
/CalendarServer/branches/users/glyph/parallel-sim:8240-8251
/CalendarServer/branches/users/glyph/parallel-upgrade:8376-8400
/CalendarServer/branches/users/glyph/parallel-upgrade_to_1:8571-8583
/CalendarServer/branches/users/glyph/q:9560-9688
/CalendarServer/branches/users/glyph/quota:7604-7637
/CalendarServer/branches/users/glyph/sendfdport:5388-5424
/CalendarServer/branches/users/glyph/shared-pool-fixes:8436-8443
/CalendarServer/branches/users/glyph/shared-pool-take2:8155-8174
/CalendarServer/branches/users/glyph/sharedpool:6490-6550
/CalendarServer/branches/users/glyph/sharing-api:9192-9205
/CalendarServer/branches/users/glyph/skip-lonely-vtimezones:8524-8535
/CalendarServer/branches/users/glyph/sql-store:5929-6073
/CalendarServer/branches/users/glyph/subtransactions:7248-7258
/CalendarServer/branches/users/glyph/table-alias:8651-8664
/CalendarServer/branches/users/glyph/uidexport:7673-7676
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/glyph/uuid-normalize:9268-9296
/CalendarServer/branches/users/glyph/xattrs-from-files:7757-7769
/CalendarServer/branches/users/sagen/applepush:8126-8184
/CalendarServer/branches/users/sagen/inboxitems:7380-7381
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/purge_old_events:6735-6746
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/branches/users/wsanchez/transations:5515-5593
/CalendarServer/trunk:9867

Modified: CalendarServer/branches/release/CalendarServer-4.2-dev/calendarserver/tap/caldav.py
===================================================================
--- CalendarServer/branches/release/CalendarServer-4.2-dev/calendarserver/tap/caldav.py	2012-09-27 17:44:33 UTC (rev 9867)
+++ CalendarServer/branches/release/CalendarServer-4.2-dev/calendarserver/tap/caldav.py	2012-09-27 17:52:04 UTC (rev 9868)
@@ -216,7 +216,7 @@
         self.logger = logObserver
 
 
-class ErrorLoggingMultiService(MultiService):
+class ErrorLoggingMultiService(MultiService, object):
     """ Registers a rotating file logger for error logging, if
         config.ErrorLogEnabled is True. """
 
@@ -237,6 +237,12 @@
 
 
 class CalDAVService (ErrorLoggingMultiService):
+
+    # The ConnectionService is a MultiService which bundles all the connection
+    # services together for the purposes of being able to stop them and wait
+    # for all of their connections to close before shutting down.
+    connectionServiceName = "ConnectionService"
+
     def __init__(self, logObserver):
         self.logObserver = logObserver # accesslog observer
         MultiService.__init__(self)
@@ -251,11 +257,11 @@
         Wait for outstanding requests to finish
         @return: a Deferred which fires when all outstanding requests are complete
         """
-        connectionService = self.getServiceNamed("ConnectionService")
+        connectionService = self.getServiceNamed(self.connectionServiceName)
         # Note: removeService() also calls stopService()
         yield self.removeService(connectionService)
         # At this point, all outstanding requests have been responded to
-        yield MultiService.stopService(self)
+        yield super(CalDAVService, self).stopService()
         self.logObserver.stop()
 
 
@@ -870,7 +876,7 @@
         # Bundle the various connection services within a single MultiService
         # that can be stopped before the others for graceful shutdown.
         connectionService = MultiService()
-        connectionService.setName("ConnectionService")
+        connectionService.setName(CalDAVService.connectionServiceName)
         connectionService.setServiceParent(service)
 
         if config.InheritFDs or config.InheritSSLFDs:

Modified: CalendarServer/branches/release/CalendarServer-4.2-dev/calendarserver/tap/test/test_caldav.py
===================================================================
--- CalendarServer/branches/release/CalendarServer-4.2-dev/calendarserver/tap/test/test_caldav.py	2012-09-27 17:44:33 UTC (rev 9867)
+++ CalendarServer/branches/release/CalendarServer-4.2-dev/calendarserver/tap/test/test_caldav.py	2012-09-27 17:52:04 UTC (rev 9868)
@@ -543,7 +543,7 @@
         default TCP and SSL configuration
         """
         # Note: the listeners are bundled within a MultiService named "ConnectionService"
-        service = self.makeService().getServiceNamed("ConnectionService")
+        service = self.makeService().getServiceNamed(CalDAVService.connectionServiceName)
 
         expectedSubServices = dict((
             (MaxAcceptTCPServer, self.config["HTTPPort"]),
@@ -569,7 +569,7 @@
         Test that the configuration of the SSLServer reflect the config file's
         SSL Private Key and SSL Certificate
         """
-        service = self.makeService().getServiceNamed("ConnectionService")
+        service = self.makeService().getServiceNamed(CalDAVService.connectionServiceName)
 
         sslService = None
         for s in service.services:
@@ -598,7 +598,7 @@
         del self.config["SSLPort"]
         self.writeConfig()
 
-        service = self.makeService().getServiceNamed("ConnectionService")
+        service = self.makeService().getServiceNamed(CalDAVService.connectionServiceName)
 
         self.assertNotIn(
             internet.SSLServer,
@@ -613,7 +613,7 @@
         del self.config["HTTPPort"]
         self.writeConfig()
 
-        service = self.makeService().getServiceNamed("ConnectionService")
+        service = self.makeService().getServiceNamed(CalDAVService.connectionServiceName)
 
         self.assertNotIn(
             internet.TCPServer,
@@ -627,7 +627,7 @@
         self.config.BindAddresses = ["127.0.0.1"]
         self.writeConfig()
 
-        service = self.makeService().getServiceNamed("ConnectionService")
+        service = self.makeService().getServiceNamed(CalDAVService.connectionServiceName)
 
         for s in service.services:
             if isinstance(s, (internet.TCPServer, internet.SSLServer)):
@@ -645,7 +645,7 @@
         ]
 
         self.writeConfig()
-        service = self.makeService().getServiceNamed("ConnectionService")
+        service = self.makeService().getServiceNamed(CalDAVService.connectionServiceName)
 
         tcpServers = []
         sslServers = []
@@ -677,7 +677,7 @@
         """
         self.config.ListenBacklog = 1024
         self.writeConfig()
-        service = self.makeService().getServiceNamed("ConnectionService")
+        service = self.makeService().getServiceNamed(CalDAVService.connectionServiceName)
 
         for s in service.services:
             if isinstance(s, (internet.TCPServer, internet.SSLServer)):

Modified: CalendarServer/branches/release/CalendarServer-4.2-dev/twext/internet/tcp.py
===================================================================
--- CalendarServer/branches/release/CalendarServer-4.2-dev/twext/internet/tcp.py	2012-09-27 17:44:33 UTC (rev 9867)
+++ CalendarServer/branches/release/CalendarServer-4.2-dev/twext/internet/tcp.py	2012-09-27 17:52:04 UTC (rev 9868)
@@ -28,6 +28,7 @@
 
 from twisted.application import internet
 from twisted.internet import tcp, ssl
+from twisted.internet.defer import succeed
 
 from twext.python.log import Logger
 
@@ -103,6 +104,24 @@
         transport._startTLS()
         return tcp.Port._preMakeConnection(self, transport)
 
+
+def _allConnectionsClosed(protocolFactory):
+    """
+    Check to see if protocolFactory implements allConnectionsClosed( ) and
+    if so, call it.  Otherwise, return immediately.
+    This allows graceful shutdown by waiting for all requests to be completed.
+
+    @param protocolFactory: (usually) an HTTPFactory implementing allConnectionsClosed
+        which returns a Deferred which fires when all connections are closed.
+    @return: A Deferred firing None when all connections are closed, or immediately
+        if the given factory does not track its connections (e.g.
+        InheritingProtocolFactory)
+    """
+    if hasattr(protocolFactory, "allConnectionsClosed"):
+        return protocolFactory.allConnectionsClosed()
+    return succeed(None)
+
+
 class MaxAcceptTCPServer(internet.TCPServer):
     """
     TCP server which will uses MaxAcceptTCPPorts (and optionally,
@@ -114,8 +133,8 @@
 
     def __init__(self, *args, **kwargs):
         internet.TCPServer.__init__(self, *args, **kwargs)
-        self.httpFactory = self.args[1]
-        self.httpFactory.myServer = self
+        self.protocolFactory = self.args[1]
+        self.protocolFactory.myServer = self
         self.inherit = self.kwargs.get("inherit", False)
         self.backlog = self.kwargs.get("backlog", None)
         self.interface = self.kwargs.get("interface", None)
@@ -138,7 +157,7 @@
         @return: a Deferred which fires when all outstanding requests are complete
         """
         internet.TCPServer.stopService(self)
-        return self.httpFactory.waitForCompletion()
+        return _allConnectionsClosed(self.protocolFactory)
 
 
 class MaxAcceptSSLServer(internet.SSLServer):
@@ -149,8 +168,8 @@
 
     def __init__(self, *args, **kwargs):
         internet.SSLServer.__init__(self, *args, **kwargs)
-        self.httpFactory = self.args[1]
-        self.httpFactory.myServer = self
+        self.protocolFactory = self.args[1]
+        self.protocolFactory.myServer = self
         self.inherit = self.kwargs.get("inherit", False)
         self.backlog = self.kwargs.get("backlog", None)
         self.interface = self.kwargs.get("interface", None)
@@ -173,6 +192,7 @@
         @return: a Deferred which fires when all outstanding requests are complete
         """
         internet.SSLServer.stopService(self)
-        return self.httpFactory.waitForCompletion()
+        # TODO: check for an ICompletionWaiter interface
+        return _allConnectionsClosed(self.protocolFactory)
 
 

Modified: CalendarServer/branches/release/CalendarServer-4.2-dev/twext/web2/channel/http.py
===================================================================
--- CalendarServer/branches/release/CalendarServer-4.2-dev/twext/web2/channel/http.py	2012-09-27 17:44:33 UTC (rev 9867)
+++ CalendarServer/branches/release/CalendarServer-4.2-dev/twext/web2/channel/http.py	2012-09-27 17:52:04 UTC (rev 9868)
@@ -1020,7 +1020,7 @@
         self.protocolArgs = kwargs
         self.protocolArgs['requestFactory'] = requestFactory
         self.connectedChannels = set()
-        self.deferred = None
+        self.allConnectionsClosedDeferred = None
 
 
     def buildProtocol(self, addr):
@@ -1046,30 +1046,30 @@
         """
         Remove a connected channel from the set of currently connected channels
         and decrease the outstanding request count.
-        If someone is waiting for all the requests to be completed, self.deferred
-        will be non-None; fire that callback when the number of outstanding requests
-        hits zero.
+        If someone is waiting for all the requests to be completed,
+        self.allConnectionsClosedDeferred will be non-None; fire that callback
+        when the number of outstanding requests hits zero.
         """
         self.connectedChannels.remove(channel)
 
-        if self.deferred is not None:
+        if self.allConnectionsClosedDeferred is not None:
             if self.outstandingRequests == 0:
-                self.deferred.callback(None)
+                self.allConnectionsClosedDeferred.callback(None)
 
     @property
     def outstandingRequests(self):
         return len(self.connectedChannels)
 
 
-    def waitForCompletion(self):
+    def allConnectionsClosed(self):
         """
         Return a Deferred that will fire when all outstanding requests have completed.
         @return: A Deferred with a result of None
         """
         if self.outstandingRequests == 0:
             return succeed(None)
-        self.deferred = Deferred()
-        return self.deferred
+        self.allConnectionsClosedDeferred = Deferred()
+        return self.allConnectionsClosedDeferred
 
 
 

Modified: CalendarServer/branches/release/CalendarServer-4.2-dev/twext/web2/metafd.py
===================================================================
--- CalendarServer/branches/release/CalendarServer-4.2-dev/twext/web2/metafd.py	2012-09-27 17:44:33 UTC (rev 9867)
+++ CalendarServer/branches/release/CalendarServer-4.2-dev/twext/web2/metafd.py	2012-09-27 17:52:04 UTC (rev 9868)
@@ -23,7 +23,6 @@
 from twisted.application.service import MultiService, Service
 from twisted.internet import reactor
 from twisted.internet.tcp import Server
-from twisted.internet.defer import succeed
 
 log = Logger()
 
@@ -90,7 +89,7 @@
         self.reportingFactory.inheritedPort.stopReading()
 
         # Let any outstanding requests finish
-        return self.reportingFactory.waitForCompletion()
+        return self.reportingFactory.allConnectionsClosed()
 
 
     def createTransport(self, skt, peer, data, protocol):
@@ -289,11 +288,3 @@
     @property
     def outstandingRequests(self):
         return self.limiter.outstandingRequests
-
-    def waitForCompletion(self):
-        """
-        Return a Deferred that will fire when all outstanding requests have completed.
-        @return: A Deferred with a result of None
-        """
-        # TODO: Make this actually wait for outstandingRequests to drop to zero
-        return succeed(None)

Modified: CalendarServer/branches/release/CalendarServer-4.2-dev/twext/web2/test/test_http.py
===================================================================
--- CalendarServer/branches/release/CalendarServer-4.2-dev/twext/web2/test/test_http.py	2012-09-27 17:44:33 UTC (rev 9867)
+++ CalendarServer/branches/release/CalendarServer-4.2-dev/twext/web2/test/test_http.py	2012-09-27 17:52:04 UTC (rev 9868)
@@ -454,19 +454,19 @@
     def _callback(self, result):
         self.callbackFired = True
 
-    def testWaitForCompletionWithoutConnectedChannels(self):
+    def testAllConnectionsClosedWithoutConnectedChannels(self):
         """
-        waitForCompletion( ) should fire right away if no connected channels
+        allConnectionsClosed( ) should fire right away if no connected channels
         """
         self.callbackFired = False
 
         factory = HTTPFactory(None)
-        factory.waitForCompletion().addCallback(self._callback)
+        factory.allConnectionsClosed().addCallback(self._callback)
         self.assertTrue(self.callbackFired)  # now!
 
-    def testWaitForCompletionWithConnectedChannels(self):
+    def testallConnectionsClosedWithConnectedChannels(self):
         """
-        waitForCompletion( ) should only fire after all connected channels
+        allConnectionsClosed( ) should only fire after all connected channels
         have been removed
         """
         self.callbackFired = False
@@ -476,7 +476,7 @@
         factory.addConnectedChannel("B")
         factory.addConnectedChannel("C")
 
-        factory.waitForCompletion().addCallback(self._callback)
+        factory.allConnectionsClosed().addCallback(self._callback)
 
         factory.removeConnectedChannel("A")
         self.assertFalse(self.callbackFired) # wait for it...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/calendarserver-changes/attachments/20120927/08dcc1f4/attachment-0001.html>

More information about the calendarserver-changes mailing list