[CalendarServer-changes] [5298] CalendarServer/branches/users/cdaboo/shared-calendars-5187
source_changes at macosforge.org
source_changes at macosforge.org
Fri Mar 12 12:35:31 PST 2010
Revision: 5298
http://trac.macosforge.org/projects/calendarserver/changeset/5298
Author: cdaboo at apple.com
Date: 2010-03-12 12:35:30 -0800 (Fri, 12 Mar 2010)
Log Message:
-----------
Merge from trunk.
Modified Paths:
--------------
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/sidecar/task.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/caldav.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/test/test_caldav.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/gateway.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/principals.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/test_gateway.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/test_principals.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/run
CalendarServer/branches/users/cdaboo/shared-calendars-5187/setup.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/Makefile.Apple
CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/build.sh
CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/submit
CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/python/filepath.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/web2/resource.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/web2/server.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/directory/augment.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/directory/directory.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/directory/xmlfile.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/ical.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/mail.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/method/delete_common.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/scheduling/processing.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/stdconfig.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/test/test_icalendar.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/txcaldav/icalendarstore.py
Added Paths:
-----------
CalendarServer/branches/users/cdaboo/shared-calendars-5187/bin/calendarserver_purge_events
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/accesslog.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/test/longlines.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/util.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/purge.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/purge/
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/purge/accounts.xml
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/test_purge.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/doc/calendarserver_purge_events.8
CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/python/test/test_filepath.py
Removed Paths:
-------------
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/purge/accounts.xml
CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/util.py
CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/accesslog.py
Property Changed:
----------------
CalendarServer/branches/users/cdaboo/shared-calendars-5187/
CalendarServer/branches/users/cdaboo/shared-calendars-5187/bin/calendarserver_make_partition
CalendarServer/branches/users/cdaboo/shared-calendars-5187/bin/calendarserver_manage_augments
Property changes on: CalendarServer/branches/users/cdaboo/shared-calendars-5187
___________________________________________________________________
Modified: svn:mergeinfo
- /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/trunk:5188-5224
+ /CalendarServer/branches/config-separation:4379-4443
/CalendarServer/branches/egg-info-351:4589-4625
/CalendarServer/branches/users/cdaboo/directory-cache-on-demand-3627:3628-3644
/CalendarServer/branches/users/cdaboo/partition-4464:4465-4957
/CalendarServer/branches/users/cdaboo/relative-config-paths-5070:5071-5105
/CalendarServer/branches/users/glyph/contacts-server-merge:4971-5080
/CalendarServer/branches/users/glyph/use-system-twisted:5084-5149
/CalendarServer/branches/users/sagen/locations-resources:5032-5051
/CalendarServer/branches/users/sagen/locations-resources-2:5052-5061
/CalendarServer/branches/users/sagen/resource-delegates-4038:4040-4067
/CalendarServer/branches/users/sagen/resource-delegates-4066:4068-4075
/CalendarServer/branches/users/sagen/resources-2:5084-5093
/CalendarServer/trunk:5188-5293
Property changes on: CalendarServer/branches/users/cdaboo/shared-calendars-5187/bin/calendarserver_make_partition
___________________________________________________________________
Added: svn:executable
+ *
Property changes on: CalendarServer/branches/users/cdaboo/shared-calendars-5187/bin/calendarserver_manage_augments
___________________________________________________________________
Added: svn:executable
+ *
Copied: CalendarServer/branches/users/cdaboo/shared-calendars-5187/bin/calendarserver_purge_events (from rev 5293, CalendarServer/trunk/bin/calendarserver_purge_events)
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/bin/calendarserver_purge_events (rev 0)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/bin/calendarserver_purge_events 2010-03-12 20:35:30 UTC (rev 5298)
@@ -0,0 +1,44 @@
+#!/usr/bin/env python
+
+##
+# Copyright (c) 2006-2010 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+import sys
+
+#PYTHONPATH
+
+if __name__ == "__main__":
+ if "PYTHONPATH" in globals():
+ sys.path.insert(0, PYTHONPATH)
+ else:
+ from os.path import dirname, abspath, join
+ from subprocess import Popen, PIPE
+
+ home = dirname(dirname(abspath(__file__)))
+ run = join(home, "run")
+
+ child = Popen((run, "-p"), stdout=PIPE)
+ path, stderr = child.communicate()
+
+ path = path.rstrip("\n")
+
+ if child.wait() == 0:
+ sys.path[0:0] = path.split(":")
+
+ sys.argv[1:1] = ["-f", join(home, "conf", "caldavd-dev.plist")]
+
+ from calendarserver.tools.purge import main
+ main()
Copied: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/accesslog.py (from rev 5293, CalendarServer/trunk/calendarserver/accesslog.py)
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/accesslog.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/accesslog.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -0,0 +1,455 @@
+##
+# Copyright (c) 2006-2009 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+Access logs.
+"""
+
+__all__ = [
+ "DirectoryLogWrapperResource",
+ "RotatingFileAccessLoggingObserver",
+ "AMPCommonAccessLoggingObserver",
+ "AMPLoggingFactory",
+]
+
+import datetime
+import os
+import time
+
+from twisted.internet import protocol
+from twisted.protocols import amp
+from twext.web2 import iweb
+from twext.web2.dav import davxml
+from twext.web2.log import BaseCommonAccessLoggingObserver
+from twext.web2.log import LogWrapperResource
+
+from twext.python.log import Logger
+
+from twistedcaldav.config import config
+from twistedcaldav.directory.directory import DirectoryService
+
+log = Logger()
+
+class DirectoryLogWrapperResource(LogWrapperResource):
+
+ def __init__(self, resource, directory):
+ super(DirectoryLogWrapperResource, self).__init__(resource)
+
+ self.directory = directory
+
+ def getDirectory(self):
+ return self.directory
+
+class CommonAccessLoggingObserverExtensions(BaseCommonAccessLoggingObserver):
+ """
+ A base class for our extension to the L{BaseCommonAccessLoggingObserver}
+ """
+
+ def emit(self, eventDict):
+
+ if eventDict.get("interface") is iweb.IRequest:
+
+ if config.GlobalStatsLoggingFrequency is not 0:
+ self.logGlobalHit()
+
+ request = eventDict["request"]
+ response = eventDict["response"]
+ loginfo = eventDict["loginfo"]
+
+ # Try to determine authentication and authorization identifiers
+ uid = "-"
+ if hasattr(request, "authnUser"):
+ if isinstance(request.authnUser.children[0], davxml.HRef):
+ uidn = str(request.authnUser.children[0])
+ uidz = None
+ if hasattr(request, "authzUser") and str(request.authzUser.children[0]) != uidn:
+ uidz = str(request.authzUser.children[0])
+
+ def convertUIDtoShortName(uid):
+ uid = uid.rstrip("/")
+ uid = uid[uid.rfind("/") + 1:]
+ record = request.site.resource.getDirectory().recordWithUID(uid)
+ if record:
+ if record.recordType == DirectoryService.recordType_users:
+ return record.shortNames[0]
+ else:
+ return "(%s)%s" % (record.recordType, record.shortNames[0],)
+ else:
+ return uid
+
+ uidn = convertUIDtoShortName(uidn)
+ if uidz:
+ uidz = convertUIDtoShortName(uidz)
+
+ if uidn and uidz:
+ uid = '"%s as %s"' % (uidn, uidz,)
+ else:
+ uid = uidn
+
+ #
+ # For some methods which basically allow you to tunnel a
+ # custom request (eg. REPORT, POST), the method name
+ # itself doesn't tell you much about what action is being
+ # requested. This allows a method to tack a submethod
+ # attribute to the request, so we can provide a little
+ # more detail here.
+ #
+ if config.EnableExtendedAccessLog and hasattr(request, "submethod"):
+ method = "%s(%s)" % (request.method, request.submethod)
+ else:
+ method = request.method
+
+ # Standard Apache access log fields
+ format = (
+ '%(host)s - %(uid)s [%(date)s]'
+ ' "%(method)s %(uri)s HTTP/%(protocolVersion)s"'
+ ' %(statusCode)s %(bytesSent)d'
+ ' "%(referer)s" "%(userAgent)s"'
+ )
+
+ if config.EnableExtendedAccessLog:
+ formats = [
+ format,
+ # Performance monitoring extensions
+ 'i=%(serverInstance)s t=%(timeSpent).1f or=%(outstandingRequests)s',
+ ]
+ if hasattr(request, "extendedLogItems"):
+ for k, v in request.extendedLogItems.iteritems():
+ k = str(k).replace('"', "%22")
+ v = str(v).replace('"', "%22")
+ if " " in v:
+ v = '"%s"' % (v,)
+ formats.append("%s=%s" % (k, v))
+
+ fwdHeaders = request.headers.getRawHeaders("x-forwarded-for", "")
+ if fwdHeaders:
+ # Limit each x-forwarded-header to 50 in case someone is
+ # trying to overwhelm the logs
+ forwardedFor = ",".join([hdr[:50] for hdr in fwdHeaders])
+ forwardedFor = forwardedFor.replace(" ", "")
+ formats.append("fwd=%(fwd)s")
+ else:
+ forwardedFor = ""
+
+ format = " ".join(formats)
+
+ formatArgs = {
+ "host" : request.remoteAddr.host,
+ "uid" : uid,
+ "date" : self.logDateString(response.headers.getHeader("date", 0)),
+ "method" : method,
+ "uri" : request.uri.replace('"', "%22"),
+ "protocolVersion" : ".".join(str(x) for x in request.clientproto),
+ "statusCode" : response.code,
+ "bytesSent" : loginfo.bytesSent,
+ "referer" : request.headers.getHeader("referer", "-"),
+ "userAgent" : request.headers.getHeader("user-agent", "-"),
+ "serverInstance" : config.LogID,
+ "timeSpent" : (time.time() - request.initTime) * 1000,
+ "outstandingRequests" : request.chanRequest.channel.factory.outstandingRequests,
+ "fwd" : forwardedFor,
+ }
+
+ # sanitize output to mitigate log injection
+ for k,v in formatArgs.items():
+ if not isinstance(v, basestring):
+ continue
+ v = v.replace("\r", "\\r")
+ v = v.replace("\n", "\\n")
+ v = v.replace("\"", "\\\"")
+ formatArgs[k] = v
+
+ self.logMessage(format % formatArgs)
+
+ elif "overloaded" in eventDict:
+ overloaded = eventDict.get("overloaded")
+ format_str = '%s - - [%s] "???" 503 0 "-" "-" [0.0 ms]'
+ format_data = (
+ overloaded.transport.hostname,
+ self.logDateString(time.time()),
+ )
+ if config.EnableExtendedAccessLog:
+ format_str += " [%s %s]"
+ format_data += (
+ overloaded.transport.server.port,
+ overloaded.outstandingRequests,
+ )
+ self.logMessage(format_str % format_data)
+
+class RotatingFileAccessLoggingObserver(CommonAccessLoggingObserverExtensions):
+ """
+ Class to do "apache" style access logging to a rotating log file. The log
+ file is rotated after midnight each day.
+ """
+
+ def __init__(self, logpath):
+ self.logpath = logpath
+ self.globalHitCount = 0
+ self.globalHitHistory = []
+ for i in range(0, config.GlobalStatsLoggingFrequency + 1):
+ self.globalHitHistory.append({"time":int(time.time()), "hits":0})
+
+ def logMessage(self, message, allowrotate=True):
+ """
+ Log a message to the file and possibly rotate if date has changed.
+
+ @param message: C{str} for the message to log.
+ @param allowrotate: C{True} if log rotate allowed, C{False} to log to current file
+ without testing for rotation.
+ """
+
+ if self.shouldRotate() and allowrotate:
+ self.flush()
+ self.rotate()
+ self.f.write(message + "\n")
+
+ def rotateGlobalHitHistoryStats(self):
+ """
+ Roll the global hit history array: push the current stats as
+ the last element; pop the first (oldest) element and reschedule the task.
+ """
+
+ self.globalHitHistory.append({"time":int(time.time()), "hits":self.globalHitCount})
+ del self.globalHitHistory[0]
+ log.debug("rotateGlobalHitHistoryStats: %s" % (self.globalHitHistory,))
+ if config.GlobalStatsLoggingFrequency is not 0:
+ self.reactor.callLater(
+ config.GlobalStatsLoggingPeriod * 60 / config.GlobalStatsLoggingFrequency,
+ self.rotateGlobalHitHistoryStats
+ )
+
+ def start(self):
+ """
+ Start logging. Open the log file and log an "open" message.
+ """
+
+ super(RotatingFileAccessLoggingObserver, self).start()
+ self._open()
+ self.logMessage("Log opened - server start: [%s]." % (datetime.datetime.now().ctime(),))
+
+ # Need a reactor for the callLater() support for rotateGlobalHitHistoryStats()
+ from twisted.internet import reactor
+ self.reactor = reactor
+ self.rotateGlobalHitHistoryStats()
+
+ def stop(self):
+ """
+ Stop logging. Close the log file and log an "open" message.
+ """
+
+ self.logMessage("Log closed - server stop: [%s]." % (datetime.datetime.now().ctime(),), False)
+ super(RotatingFileAccessLoggingObserver, self).stop()
+ self._close()
+
+ def _open(self):
+ """
+ Open the log file.
+ """
+
+ self.f = open(self.logpath, "a", 1)
+ self.lastDate = self.toDate(os.stat(self.logpath)[8])
+
+ def _close(self):
+ """
+ Close the log file.
+ """
+
+ self.f.close()
+
+ def flush(self):
+ """
+ Flush the log file.
+ """
+
+ self.f.flush()
+
+ def shouldRotate(self):
+ """
+ Rotate when the date has changed since last write
+ """
+
+ if config.RotateAccessLog:
+ return self.toDate() > self.lastDate
+ else:
+ return False
+
+ def toDate(self, *args):
+ """
+ Convert a unixtime to (year, month, day) localtime tuple,
+ or return the current (year, month, day) localtime tuple.
+
+ This function primarily exists so you may overload it with
+ gmtime, or some cruft to make unit testing possible.
+ """
+
+ # primarily so this can be unit tested easily
+ return time.localtime(*args)[:3]
+
+ def suffix(self, tupledate):
+ """
+ Return the suffix given a (year, month, day) tuple or unixtime
+ """
+
+ try:
+ return "_".join(map(str, tupledate))
+ except:
+ # try taking a float unixtime
+ return "_".join(map(str, self.toDate(tupledate)))
+
+ def rotate(self):
+ """
+ Rotate the file and create a new one.
+
+ If it's not possible to open new logfile, this will fail silently,
+ and continue logging to old logfile.
+ """
+
+ newpath = "%s.%s" % (self.logpath, self.suffix(self.lastDate))
+ if os.path.exists(newpath):
+ log.msg("Cannot rotate log file to %s because it already exists." % (newpath,))
+ return
+ self.logMessage("Log closed - rotating: [%s]." % (datetime.datetime.now().ctime(),), False)
+ log.msg("Rotating log file to: %s" % (newpath,), system="Logging")
+ self.f.close()
+ os.rename(self.logpath, newpath)
+ self._open()
+ self.logMessage("Log opened - rotated: [%s]." % (datetime.datetime.now().ctime(),), False)
+
+ def logGlobalHit(self):
+ """
+ Increment the service-global hit counter
+ """
+
+ self.globalHitCount += 1
+
+ def getGlobalHits(self):
+ """
+ Return the global hit stats
+ """
+
+ stats = '<?xml version="1.0" encoding="UTF-8"?><plist version="1.0">'
+ stats += "<dict><key>totalHits</key><integer>%d</integer>"
+ stats += "<key>recentHits</key><dict>"
+ stats += "<key>count</key><integer>%d</integer>"
+ stats += "<key>since</key><integer>%d</integer>"
+ stats += "<key>period</key><integer>%d</integer>"
+ stats += "<key>frequency</key><integer>%d</integer>"
+ stats += "</dict></dict></plist>"
+ return stats % (
+ self.globalHitCount,
+ self.globalHitCount - self.globalHitHistory[0]["hits"],
+ self.globalHitHistory[0]["time"],
+ config.GlobalStatsLoggingPeriod,
+ config.GlobalStatsLoggingFrequency
+ )
+
+class LogMessage(amp.Command):
+ arguments = [("message", amp.String())]
+
+class LogGlobalHit(amp.Command):
+ arguments = []
+
+class AMPCommonAccessLoggingObserver(CommonAccessLoggingObserverExtensions):
+ def __init__(self, mode, id):
+ self.mode = mode
+ self.id = id
+ self.protocol = None
+ self._buffer = []
+
+ def flushBuffer(self):
+ if self._buffer:
+ for msg in self._buffer:
+ self.logMessage(msg)
+
+ def start(self):
+ super(AMPCommonAccessLoggingObserver, self).start()
+
+ from twisted.internet import reactor
+
+ def _gotProtocol(proto):
+ self.protocol = proto
+ self.flushBuffer()
+
+ self.client = protocol.ClientCreator(reactor, amp.AMP)
+ if self.mode == "AF_UNIX":
+ d = self.client.connectUNIX(self.id)
+ else:
+ d = self.client.connectTCP("localhost", self.id)
+ d.addCallback(_gotProtocol)
+
+ def stop(self):
+ super(AMPCommonAccessLoggingObserver, self).stop()
+ self.client.disconnect()
+
+ def logMessage(self, message):
+ """
+ Log a message to the remote AMP Protocol
+ """
+ if self.protocol is not None:
+ # XXX: Yeah we're not waiting for anything to happen here.
+ # but we will log an error.
+ if isinstance(message, unicode):
+ message = message.encode("utf-8")
+ d = self.protocol.callRemote(LogMessage, message=message)
+ d.addErrback(log.err)
+ else:
+ self._buffer.append(message)
+
+ def logGlobalHit(self):
+ """
+ Log a server hit via the remote AMP Protocol
+ """
+
+ if self.protocol is not None:
+ d = self.protocol.callRemote(LogGlobalHit)
+ d.addErrback(log.err)
+ else:
+ log.msg("logGlobalHit() only works with an AMP Protocol")
+
+class AMPLoggingProtocol(amp.AMP):
+ """
+ A server side protocol for logging to the given observer.
+ """
+
+ def __init__(self, observer):
+ self.observer = observer
+
+ super(AMPLoggingProtocol, self).__init__()
+
+ def logMessage(self, message):
+ self.observer.logMessage(message)
+ return {}
+
+ LogMessage.responder(logMessage)
+
+ def logGlobalHit(self):
+ self.observer.logGlobalHit()
+ return {}
+
+ LogGlobalHit.responder(logGlobalHit)
+
+class AMPLoggingFactory(protocol.ServerFactory):
+ def __init__(self, observer):
+ self.observer = observer
+
+ def doStart(self):
+ self.observer.start()
+
+ def doStop(self):
+ self.observer.stop()
+
+ def buildProtocol(self, addr):
+ return AMPLoggingProtocol(self.observer)
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/sidecar/task.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/sidecar/task.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/sidecar/task.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -23,6 +23,7 @@
]
import os
+from datetime import date, timedelta
from zope.interface import implements
@@ -31,7 +32,6 @@
from twisted.internet.reactor import callLater
from twisted.plugin import IPlugin
from twisted.python.usage import Options, UsageError
-from twext.web2.http_headers import Headers
from twext.python.log import Logger, LoggingMixIn
from twext.python.log import logLevelForNamespace, setLogLevelForNamespace
@@ -42,52 +42,11 @@
from twistedcaldav.scheduling.cuaddress import LocalCalendarUser
from twistedcaldav.scheduling.scheduler import DirectScheduler
-from calendarserver.util import getRootResource
+from calendarserver.tap.util import getRootResource, FakeRequest
+from calendarserver.tools.purge import purgeOldEvents
log = Logger()
-class FakeRequest(object):
-
- def __init__(self, rootResource, method):
- self.rootResource = rootResource
- self.method = method
- self._resourcesByURL = {}
- self._urlsByResource = {}
- self.headers = Headers()
-
- @inlineCallbacks
- def _getChild(self, resource, segments):
- if not segments:
- returnValue(resource)
-
- child, remaining = (yield resource.locateChild(self, segments))
- returnValue((yield self._getChild(child, remaining)))
-
- @inlineCallbacks
- def locateResource(self, url):
- url = url.strip("/")
- segments = url.split("/")
- resource = (yield self._getChild(self.rootResource, segments))
- if resource:
- self._rememberResource(resource, url)
- returnValue(resource)
-
- def _rememberResource(self, resource, url):
- self._resourcesByURL[url] = resource
- self._urlsByResource[resource] = url
- return resource
-
- def urlForResource(self, resource):
- url = self._urlsByResource.get(resource, None)
- if url is None:
- class NoURLForResourceError(RuntimeError):
- pass
- raise NoURLForResourceError(resource)
- return url
-
- def addResponseFilter(*args, **kwds):
- pass
-
@inlineCallbacks
def processInboxItem(rootResource, directory, inboxFile, inboxItemFile, uuid):
log.debug("Processing inbox item %s" % (inboxItemFile,))
@@ -192,13 +151,31 @@
os.remove(self.taskFile)
+ @inlineCallbacks
+ def task_purgeoldevents(self):
+ with open(self.taskFile) as input:
+ try:
+ value = input.read().strip()
+ days = int(value)
+ except ValueError:
+ log.error("Illegal value for purge days: %s" % (value,))
+ else:
+ cutoff = (date.today() -
+ timedelta(days=days)).strftime("%Y%m%dT000000Z")
+ count = (yield purgeOldEvents(self.service.directory,
+ self.service.root, cutoff))
+ log.info("Purged %d events" % (count,))
+
+ os.remove(self.taskFile)
+
+
class CalDAVTaskService(Service):
def __init__(self, root):
self.root = root
self.directory = root.directory
- self.seconds = 30 # How often to check for new tasks in incomingDir
+ self.seconds = 5 # How often to check for new tasks in incomingDir
self.taskDir = os.path.join(config.DataRoot, "tasks")
# New task files are placed into "incoming"
self.incomingDir = os.path.join(self.taskDir, "incoming")
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/caldav.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/caldav.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/caldav.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -60,9 +60,6 @@
from version import version as getVersion
version = "%s (%s)" % getVersion()
-from twistedcaldav.accesslog import AMPCommonAccessLoggingObserver
-from twistedcaldav.accesslog import AMPLoggingFactory
-from twistedcaldav.accesslog import RotatingFileAccessLoggingObserver
from twistedcaldav.config import ConfigurationError
from twistedcaldav.config import config
from twistedcaldav.directory.principal import DirectoryPrincipalProvisioningResource
@@ -81,10 +78,13 @@
except ImportError:
NegotiateCredentialFactory = None
+from calendarserver.accesslog import AMPCommonAccessLoggingObserver
+from calendarserver.accesslog import AMPLoggingFactory
+from calendarserver.accesslog import RotatingFileAccessLoggingObserver
from calendarserver.provision.root import RootResource
from calendarserver.webadmin.resource import WebAdminResource
from calendarserver.webcal.resource import WebCalendarResource
-from calendarserver.util import getRootResource
+from calendarserver.tap.util import getRootResource
log = Logger()
@@ -784,7 +784,7 @@
for p in xrange(0, config.MultiProcess.ProcessCount):
process = TwistdSlaveProcess(
- config.Twisted.twistd,
+ sys.argv[0],
self.tapname,
options["config"],
p,
@@ -825,7 +825,7 @@
notificationsArgv = [
sys.executable,
- config.Twisted.twistd,
+ sys.argv[0],
]
if config.UserName:
notificationsArgv.extend(("-u", config.UserName))
@@ -847,7 +847,7 @@
mailGatewayArgv = [
sys.executable,
- config.Twisted.twistd,
+ sys.argv[0],
]
if config.UserName:
mailGatewayArgv.extend(("-u", config.UserName))
@@ -864,7 +864,7 @@
self.log_info("Adding task service")
taskArgv = [
sys.executable,
- config.Twisted.twistd,
+ sys.argv[0],
]
if config.UserName:
taskArgv.extend(("-u", config.UserName))
@@ -881,7 +881,7 @@
stats = CalDAVStatisticsServer(logger)
statsService = GroupOwnedUNIXServer(
- gid, config.GlobalStatsSocket, stats, mode=0660
+ gid, config.GlobalStatsSocket, stats, mode=0440
)
statsService.setName("stats")
statsService.setServiceParent(s)
@@ -1076,7 +1076,7 @@
def startProcess(self, name):
if self.protocols.has_key(name):
return
- p = self.protocols[name] = procmon.LoggingProtocol()
+ p = self.protocols[name] = DelayedStartupLoggingProtocol()
p.service = self
p.name = name
args, uid, gid, env = self.processes[name]
@@ -1096,6 +1096,74 @@
childFDs=childFDs)
+class DelayedStartupLineLogger(object):
+ """
+ A line logger that can handle very long lines.
+ """
+
+ MAX_LENGTH = 1024
+ tag = None
+ exceeded = False # Am I in the middle of parsing a long line?
+ _buffer = ''
+
+ def makeConnection(self, transport):
+ """
+ Ignore this IProtocol method, since I don't need a transport.
+ """
+
+
+ def dataReceived(self, data):
+ lines = (self._buffer + data).split("\n")
+ while len(lines) > 1:
+ line = lines.pop(0)
+ if len(line) > self.MAX_LENGTH:
+ self.lineLengthExceeded(line)
+ elif self.exceeded:
+ self.lineLengthExceeded(line)
+ self.exceeded = False
+ else:
+ self.lineReceived(line)
+ lastLine = lines.pop(0)
+ if len(lastLine) > self.MAX_LENGTH:
+ self.lineLengthExceeded(lastLine)
+ self.exceeded = True
+ self._buffer = ''
+ else:
+ self._buffer = lastLine
+
+
+ def lineReceived(self, line):
+ from twisted.python.log import msg
+ msg('[%s] %s' % (self.tag, line))
+
+
+ def lineLengthExceeded(self, line):
+ """
+ A very long line is being received. Log it immediately and forget
+ about buffering it.
+ """
+ for i in range(len(line)/self.MAX_LENGTH):
+ self.lineReceived(line[i*self.MAX_LENGTH:(i+1)*self.MAX_LENGTH]
+ + " (truncated, continued)")
+
+
+
+class DelayedStartupLoggingProtocol(procmon.LoggingProtocol, object):
+ """
+ Logging protocol that handles lines which are too long.
+ """
+
+ def connectionMade(self):
+ """
+ Replace the superclass's output monitoring logic with one that can
+ handle lineLengthExceeded.
+ """
+ super(DelayedStartupLoggingProtocol, self).connectionMade()
+ self.output = DelayedStartupLineLogger()
+ self.output.tag = self.name
+
+
+
def getSSLPassphrase(*ignored):
if not config.SSLPrivateKey:
Copied: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/test/longlines.py (from rev 5293, CalendarServer/trunk/calendarserver/tap/test/longlines.py)
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/test/longlines.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/test/longlines.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -0,0 +1,26 @@
+##
+# Copyright (c) 2007-2010 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+import sys
+
+length = int(sys.argv[1])
+
+data = (("x" * length) +
+ ("y" * length) + "\n" +
+ ("z" + "\n"))
+
+sys.stdout.write(data)
+sys.stdout.flush()
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/test/test_caldav.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/test/test_caldav.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/test/test_caldav.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -14,6 +14,7 @@
# limitations under the License.
##
+import sys
import os
import stat
import grp
@@ -22,16 +23,21 @@
from twisted.trial.unittest import TestCase as BaseTestCase
+from twisted.python.threadable import isInIOThread
+from twisted.internet.reactor import callFromThread
from twisted.python.usage import Options, UsageError
from twisted.python.reflect import namedAny
+from twisted.python import log
from twisted.internet.protocol import ServerFactory
+from twisted.internet.defer import Deferred
from twisted.application.service import IService
from twisted.application import internet
from twext.web2.dav import auth
from twext.web2.log import LogWrapperResource
+from twext.python.filepath import CachingFilePath as FilePath
from twext.python.plistlib import writePlist
from twext.internet.tcp import MaxAcceptTCPServer, MaxAcceptSSLServer
@@ -45,7 +51,9 @@
from twistedcaldav.test.util import TestCase
from calendarserver.tap.caldav import (CalDAVOptions, CalDAVServiceMaker,
- CalDAVService, GroupOwnedUNIXServer)
+ CalDAVService, GroupOwnedUNIXServer,
+ DelayedStartupProcessMonitor,
+ DelayedStartupLineLogger)
# Points to top of source tree.
@@ -220,6 +228,8 @@
self.config.ProcessType = "Slave"
self.config.SSLPrivateKey = pemFile
self.config.SSLCertificate = pemFile
+ self.config.Memcached.Pools.Default.ClientEnabled = False
+ self.config.Memcached.Pools.Default.ServerEnabled = False
self.config.SudoersFile = ""
@@ -330,7 +340,7 @@
self.config["ProcessType"] = "Combined"
self.writeConfig()
svc = self.makeService()
- for serviceName in ["logging", "stats"]:
+ for serviceName in ["logging"]:
socketService = svc.getServiceNamed(serviceName)
self.assertIsInstance(socketService, GroupOwnedUNIXServer)
m = socketService.kwargs.get("mode", 0666)
@@ -339,9 +349,19 @@
"Wrong mode on %s: %s" % (serviceName, oct(m))
)
self.assertEquals(socketService.gid, alternateGroup)
+ for serviceName in ["stats"]:
+ socketService = svc.getServiceNamed(serviceName)
+ self.assertIsInstance(socketService, GroupOwnedUNIXServer)
+ m = socketService.kwargs.get("mode", 0444)
+ self.assertEquals(
+ m, int("440", 8),
+ "Wrong mode on %s: %s" % (serviceName, oct(m))
+ )
+ self.assertEquals(socketService.gid, alternateGroup)
+
class SlaveServiceTest(BaseServiceMakerTests):
"""
Test various configurations of the Slave service
@@ -773,3 +793,84 @@
configuredDirectory = namedAny(self.config.DirectoryService.type)
self.failUnless(isinstance(realDirectory, configuredDirectory))
+
+
+
+class DummyProcessObject(object):
+ """
+ Simple stub for the Process Object API that will run a test script.
+ """
+
+ def __init__(self, scriptname, *args):
+ self.scriptname = scriptname
+ self.args = list(args)
+
+
+ def getCommandLine(self):
+ """
+ Get the command line to invoke this script.
+ """
+ return [sys.executable, FilePath(__file__).sibling(self.scriptname).path] + self.args
+
+
+ def getName(self):
+ """
+ Get a dummy name.
+ """
+ return 'Dummy'
+
+
+
+class DelayedStartupProcessMonitorTests(TestCase):
+ """
+ Test cases for L{DelayedStartupProcessMonitor}.
+ """
+
+ def test_lineAfterLongLine(self):
+ """
+ A "long" line of output from a monitored process (longer than
+ L{LineReceiver.MAX_LENGTH}) should be logged in chunks rather than all
+ at once, to avoid resource exhaustion.
+ """
+ dspm = DelayedStartupProcessMonitor()
+ dspm.addProcessObject(DummyProcessObject(
+ 'longlines.py', str(DelayedStartupLineLogger.MAX_LENGTH)),
+ os.environ)
+ dspm.startService()
+ self.addCleanup(dspm.stopService)
+
+ logged = []
+
+ def tempObserver(event):
+ # Probably won't be a problem, but let's not have any intermittent
+ # test issues that stem from multi-threaded log messages randomly
+ # going off...
+ if not isInIOThread():
+ callFromThread(tempObserver, event)
+ return
+ if event.get('isError'):
+ d.errback()
+ m = event.get('message')[0]
+ if m.startswith('[Dummy] '):
+ logged.append(event)
+ if m == '[Dummy] z':
+ d.callback("done")
+
+ log.addObserver(tempObserver)
+ self.addCleanup(log.removeObserver, tempObserver)
+ d = Deferred()
+ def assertions(result):
+ self.assertEquals(["[Dummy] x",
+ "[Dummy] y",
+ "[Dummy] z"],
+ [''.join(evt['message'])[:len('[Dummy]') + 2]
+ for evt in logged])
+ self.assertEquals([" (truncated, continued)",
+ " (truncated, continued)",
+ "[Dummy] z"],
+ [''.join(evt['message'])[-len(" (truncated, continued)"):]
+ for evt in logged])
+ d.addCallback(assertions)
+ return d
+
+
Copied: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/util.py (from rev 5293, CalendarServer/trunk/calendarserver/tap/util.py)
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/util.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tap/util.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -0,0 +1,480 @@
+# -*- test-case-name: calendarserver.tap.test.test_caldav -*-
+##
+# Copyright (c) 2005-2010 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+__all__ = [
+ "getRootResource",
+ "FakeRequest",
+]
+
+import errno
+import os
+from time import sleep
+
+from twisted.python.reflect import namedClass
+from twisted.internet import reactor
+from twisted.cred.portal import Portal
+from twext.web2.http_headers import Headers
+from twext.web2.dav import auth
+from twext.web2.auth.basic import BasicCredentialFactory
+from twext.web2.static import File as FileResource
+from twext.python.filepath import CachingFilePath as FilePath
+
+from twext.python.log import Logger
+
+from twistedcaldav import memcachepool
+from twistedcaldav.directory import augment, calendaruserproxy
+from twistedcaldav.directory.aggregate import AggregateDirectoryService
+from twistedcaldav.directory.calendaruserproxyloader import XMLCalendarUserProxyLoader
+from twistedcaldav.directory.digest import QopDigestCredentialFactory
+from twistedcaldav.directory.principal import DirectoryPrincipalProvisioningResource
+from twistedcaldav.directory.sudo import SudoDirectoryService
+from twistedcaldav.directory.util import NotFilePath
+from twistedcaldav.directory.wiki import WikiDirectoryService
+from twistedcaldav.notify import installNotificationClient
+from twistedcaldav.resource import CalDAVResource, AuthenticationWrapper
+from twistedcaldav.static import CalendarHomeProvisioningFile
+from twistedcaldav.static import IScheduleInboxFile
+from twistedcaldav.static import TimezoneServiceFile
+from twistedcaldav.static import AddressBookHomeProvisioningFile, DirectoryBackedAddressBookFile
+from twistedcaldav.timezones import TimezoneCache
+from twisted.internet.defer import inlineCallbacks, returnValue
+
+try:
+ from twistedcaldav.authkerb import NegotiateCredentialFactory
+ NegotiateCredentialFactory # pacify pyflakes
+except ImportError:
+ NegotiateCredentialFactory = None
+
+from calendarserver.accesslog import DirectoryLogWrapperResource
+from calendarserver.provision.root import RootResource
+from calendarserver.webadmin.resource import WebAdminResource
+from calendarserver.webcal.resource import WebCalendarResource
+
+log = Logger()
+
+
+
+def getRootResource(config, resources=None):
+ """
+ Set up directory service and resource hierarchy based on config.
+ Return root resource.
+
+ Additional resources can be added to the hierarchy by passing a list of
+ tuples containing: path, resource class, __init__ args list, and optional
+ authentication scheme ("basic" or "digest").
+ """
+
+ #
+ # Default resource classes
+ #
+ rootResourceClass = RootResource
+ principalResourceClass = DirectoryPrincipalProvisioningResource
+ calendarResourceClass = CalendarHomeProvisioningFile
+ iScheduleResourceClass = IScheduleInboxFile
+ timezoneServiceResourceClass = TimezoneServiceFile
+ webCalendarResourceClass = WebCalendarResource
+ webAdminResourceClass = WebAdminResource
+ addressBookResourceClass = AddressBookHomeProvisioningFile
+ directoryBackedAddressBookResourceClass = DirectoryBackedAddressBookFile
+
+ #
+ # Setup the Directory
+ #
+ directories = []
+
+ directoryClass = namedClass(config.DirectoryService.type)
+
+ log.info("Configuring directory service of type: %s"
+ % (config.DirectoryService.type,))
+
+ baseDirectory = directoryClass(config.DirectoryService.params)
+
+ # Wait for the directory to become available
+ while not baseDirectory.isAvailable():
+ sleep(5)
+
+ directories.append(baseDirectory)
+
+ #
+ # Setup the Locations and Resources Service
+ #
+ if config.ResourceService.Enabled:
+ resourceClass = namedClass(config.ResourceService.type)
+
+ log.info("Configuring resource service of type: %s" % (resourceClass,))
+
+ resourceDirectory = resourceClass(config.ResourceService.params)
+ directories.append(resourceDirectory)
+
+ #
+ # Add sudoers directory
+ #
+ sudoDirectory = None
+
+ if config.SudoersFile and os.path.exists(config.SudoersFile):
+ log.info("Configuring SudoDirectoryService with file: %s"
+ % (config.SudoersFile,))
+
+ sudoDirectory = SudoDirectoryService(config.SudoersFile)
+ sudoDirectory.realmName = baseDirectory.realmName
+
+ CalDAVResource.sudoDirectory = sudoDirectory
+ directories.insert(0, sudoDirectory)
+ else:
+ log.info( "Not using SudoDirectoryService; file doesn't exist: %s"
+ % (config.SudoersFile,)
+ )
+
+ #
+ # Add wiki directory service
+ #
+ if config.Authentication.Wiki.Enabled:
+ wikiDirectory = WikiDirectoryService()
+ wikiDirectory.realmName = baseDirectory.realmName
+ directories.append(wikiDirectory)
+
+ directory = AggregateDirectoryService(directories)
+
+ if sudoDirectory:
+ directory.userRecordTypes.insert(0,
+ SudoDirectoryService.recordType_sudoers)
+
+ #
+ # Setup the Augment Service
+ #
+ augmentClass = namedClass(config.AugmentService.type)
+
+ log.info("Configuring augment service of type: %s" % (augmentClass,))
+
+ try:
+ augment.AugmentService = augmentClass(**config.AugmentService.params)
+ except IOError:
+ log.error("Could not start augment service")
+ raise
+
+ #
+ # Setup the PoxyDB Service
+ #
+ proxydbClass = namedClass(config.ProxyDBService.type)
+
+ log.info("Configuring proxydb service of type: %s" % (proxydbClass,))
+
+ try:
+ calendaruserproxy.ProxyDBService = proxydbClass(**config.ProxyDBService.params)
+ except IOError:
+ log.error("Could not start proxydb service")
+ raise
+
+ #
+ # Make sure proxies get initialized
+ #
+ if config.ProxyLoadFromFile:
+ def _doProxyUpdate():
+ loader = XMLCalendarUserProxyLoader(config.ProxyLoadFromFile)
+ return loader.updateProxyDB()
+
+ reactor.addSystemEventTrigger("after", "startup", _doProxyUpdate)
+
+ #
+ # Configure Memcached Client Pool
+ #
+ memcachepool.installPools(
+ config.Memcached.Pools,
+ config.Memcached.MaxClients,
+ )
+
+ #
+ # Configure NotificationClient
+ #
+ if config.Notifications.Enabled:
+ installNotificationClient(
+ config.Notifications.InternalNotificationHost,
+ config.Notifications.InternalNotificationPort,
+ )
+
+ #
+ # Configure the Site and Wrappers
+ #
+ credentialFactories = []
+
+ portal = Portal(auth.DavRealm())
+
+ portal.registerChecker(directory)
+
+ realm = directory.realmName or ""
+
+ log.info("Configuring authentication for realm: %s" % (realm,))
+
+ for scheme, schemeConfig in config.Authentication.iteritems():
+ scheme = scheme.lower()
+
+ credFactory = None
+
+ if schemeConfig["Enabled"]:
+ log.info("Setting up scheme: %s" % (scheme,))
+
+ if scheme == "kerberos":
+ if not NegotiateCredentialFactory:
+ log.info("Kerberos support not available")
+ continue
+
+ try:
+ principal = schemeConfig["ServicePrincipal"]
+ if not principal:
+ credFactory = NegotiateCredentialFactory(
+ type="HTTP",
+ hostname=config.ServerHostName,
+ )
+ else:
+ credFactory = NegotiateCredentialFactory(
+ principal=principal,
+ )
+ except ValueError:
+ log.info("Could not start Kerberos")
+ continue
+
+ elif scheme == "digest":
+ credFactory = QopDigestCredentialFactory(
+ schemeConfig["Algorithm"],
+ schemeConfig["Qop"],
+ realm,
+ )
+
+ elif scheme == "basic":
+ credFactory = BasicCredentialFactory(realm)
+
+ elif scheme == "wiki":
+ pass
+
+ else:
+ log.error("Unknown scheme: %s" % (scheme,))
+
+ if credFactory:
+ credentialFactories.append(credFactory)
+
+
+ #
+ # Setup Resource hierarchy
+ #
+ log.info("Setting up document root at: %s"
+ % (config.DocumentRoot,))
+ log.info("Setting up principal collection: %r"
+ % (principalResourceClass,))
+
+ principalCollection = principalResourceClass("/principals/", directory)
+
+ log.info("Setting up calendar collection: %r" % (calendarResourceClass,))
+
+ calendarCollection = calendarResourceClass(
+ os.path.join(config.DocumentRoot, "calendars"),
+ directory, "/calendars/",
+ )
+
+ log.info("Setting up root resource: %r" % (rootResourceClass,))
+
+ root = rootResourceClass(
+ config.DocumentRoot,
+ principalCollections=(principalCollection,),
+ )
+
+ if config.EnableCardDAV:
+ root.saclService = "addressbook" # XXX this needs to be dealt with
+ # differently if caldav and carddav
+ # are going to be in the same process
+ log.info("Setting up address book collection: %r" % (addressBookResourceClass,))
+
+ addressBookCollection = addressBookResourceClass(
+ os.path.join(config.DocumentRoot, "addressbooks"),
+ directory, "/addressbooks/"
+ )
+
+ directoryPath = os.path.join(config.DocumentRoot, "directory")
+ doBacking = config.DirectoryAddressBook and config.EnableSearchAddressBook
+ if doBacking:
+ log.info("Setting up directory address book: %r" % (directoryBackedAddressBookResourceClass,))
+
+ directoryBackedAddressBookCollection = directoryBackedAddressBookResourceClass(
+ directoryPath,
+ principalCollections=(principalCollection,)
+ )
+ # do this after process is owned by carddav user, not root. XXX
+ # this should be fixed to execute at a different stage of service
+ # startup entirely.
+ reactor.callLater(1.0, directoryBackedAddressBookCollection.provisionDirectory)
+ else:
+ # remove /directory from previous runs that may have created it
+ try:
+ FilePath(directoryPath).remove()
+ log.info("Deleted: %s" % directoryPath)
+ except (OSError, IOError), e:
+ if e.errno != errno.ENOENT:
+ log.error("Could not delete: %s : %r" % (directoryPath, e,))
+
+ root.putChild('addressbooks', addressBookCollection)
+ if doBacking:
+ root.putChild('directory', directoryBackedAddressBookCollection)
+
+ root.putChild("principals", principalCollection)
+ root.putChild("calendars", calendarCollection)
+
+ for name, info in config.Aliases.iteritems():
+ if os.path.sep in name or not info.get("path", None):
+ log.error("Invalid alias: %s" % (name,))
+ continue
+ log.info("Adding alias %s -> %s" % (name, info["path"]))
+ resource = FileResource(info["path"])
+ root.putChild(name, resource)
+
+ # Timezone service is optional
+ if config.EnableTimezoneService:
+ log.info("Setting up time zone service resource: %r"
+ % (timezoneServiceResourceClass,))
+
+ timezoneService = timezoneServiceResourceClass(
+ NotFilePath(isfile=True),
+ root,
+ )
+ root.putChild("timezones", timezoneService)
+
+ # iSchedule service is optional
+ if config.Scheduling.iSchedule.Enabled:
+ log.info("Setting up iSchedule inbox resource: %r"
+ % (iScheduleResourceClass,))
+
+ ischedule = iScheduleResourceClass(
+ NotFilePath(isfile=True),
+ root,
+ )
+ root.putChild("ischedule", ischedule)
+
+ #
+ # WebCal
+ #
+ if config.WebCalendarRoot:
+ log.info("Setting up WebCalendar resource: %s"
+ % (config.WebCalendarRoot,))
+ webCalendar = webCalendarResourceClass(
+ config.WebCalendarRoot,
+ principalCollections=(principalCollection,),
+ )
+ root.putChild("webcal", webCalendar)
+
+ #
+ # WebAdmin
+ #
+ if config.EnableWebAdmin:
+ log.info("Setting up WebAdmin resource")
+ webAdmin = webAdminResourceClass(
+ config.WebCalendarRoot,
+ root,
+ directory,
+ principalCollections=(principalCollection,),
+ )
+ root.putChild("admin", webAdmin)
+
+ #
+ # Configure ancillary data
+ #
+ log.info("Setting up Timezone Cache")
+ TimezoneCache.create()
+
+
+ log.info("Configuring authentication wrapper")
+
+ overrides = { }
+ if resources:
+ for path, cls, args, scheme in resources:
+
+ # putChild doesn't want "/" starting the path
+ root.putChild(path, cls(root, *args))
+
+ # overrides requires "/" prepended
+ path = "/" + path
+
+ if scheme == "basic":
+ overrides[path] = (BasicCredentialFactory(realm),)
+
+ elif scheme == "digest":
+ schemeConfig = config.Authentication.Digest
+ overrides[path] = (QopDigestCredentialFactory(
+ schemeConfig["Algorithm"],
+ schemeConfig["Qop"],
+ realm,
+ ),)
+ log.info("Overriding %s with %s (%s)" % (path, cls, scheme))
+
+ authWrapper = AuthenticationWrapper(
+ root,
+ portal,
+ credentialFactories,
+ (auth.IPrincipal,),
+ overrides=overrides,
+ )
+
+ logWrapper = DirectoryLogWrapperResource(
+ authWrapper,
+ directory,
+ )
+
+ return logWrapper
+
+
+
+
+class FakeRequest(object):
+
+ def __init__(self, rootResource, method, path):
+ self.rootResource = rootResource
+ self.method = method
+ self.path = path
+ self._resourcesByURL = {}
+ self._urlsByResource = {}
+ self.headers = Headers()
+
+ @inlineCallbacks
+ def _getChild(self, resource, segments):
+ if not segments:
+ returnValue(resource)
+
+ child, remaining = (yield resource.locateChild(self, segments))
+ returnValue((yield self._getChild(child, remaining)))
+
+ @inlineCallbacks
+ def locateResource(self, url):
+ url = url.strip("/")
+ segments = url.split("/")
+ resource = (yield self._getChild(self.rootResource, segments))
+ if resource:
+ self._rememberResource(resource, url)
+ returnValue(resource)
+
+ def _rememberResource(self, resource, url):
+ self._resourcesByURL[url] = resource
+ self._urlsByResource[resource] = url
+ return resource
+
+ def urlForResource(self, resource):
+ url = self._urlsByResource.get(resource, None)
+ if url is None:
+ class NoURLForResourceError(RuntimeError):
+ pass
+ raise NoURLForResourceError(resource)
+ return url
+
+ def addResponseFilter(*args, **kwds):
+ pass
+
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/gateway.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/gateway.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/gateway.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -199,22 +199,37 @@
def command_getLocationList(self, command):
respondWithRecordsOfType(self.dir, command, "locations")
+ @inlineCallbacks
def command_createLocation(self, command):
kwargs = {}
for key, info in attrMap.iteritems():
if command.has_key(key):
kwargs[info['attr']] = command[key]
+
try:
- self.dir.createRecord("locations", **kwargs)
+ record = self.dir.createRecord("locations", **kwargs)
except DirectoryError, e:
respondWithError(str(e))
return
+
+ principal = self.dir.principalCollection.principalForRecord(record)
+ (yield principal.setAutoSchedule(command.get('AutoSchedule', True)))
+
+ try:
+ self.dir.updateRecord("locations", **kwargs)
+ except DirectoryError, e:
+ respondWithError(str(e))
+ return
+
respondWithRecordsOfType(self.dir, command, "locations")
def command_getLocationAttributes(self, command):
guid = command['GeneratedUID']
record = self.dir.recordWithGUID(guid)
+ if record is None:
+ respondWithError("Principal not found: %s" % (guid,))
+ return
recordDict = recordToDict(record)
principal = principalForPrincipalID(guid, directory=self.dir)
if principal is None:
@@ -226,6 +241,12 @@
@inlineCallbacks
def command_setLocationAttributes(self, command):
+ # Set autoSchedule prior to the updateRecord so that the right
+ # value ends up in memcached
+ principal = principalForPrincipalID(command['GeneratedUID'],
+ directory=self.dir)
+ (yield principal.setAutoSchedule(command.get('AutoSchedule', False)))
+
kwargs = {}
for key, info in attrMap.iteritems():
if command.has_key(key):
@@ -236,10 +257,6 @@
respondWithError(str(e))
return
- principal = principalForPrincipalID(command['GeneratedUID'],
- directory=self.dir)
- (yield principal.setAutoSchedule(command.get('AutoSchedule', False)))
-
self.command_getLocationAttributes(command)
def command_deleteLocation(self, command):
@@ -259,16 +276,28 @@
def command_getResourceList(self, command):
respondWithRecordsOfType(self.dir, command, "resources")
+ @inlineCallbacks
def command_createResource(self, command):
kwargs = {}
for key, info in attrMap.iteritems():
if command.has_key(key):
kwargs[info['attr']] = command[key]
+
try:
- self.dir.createRecord("resources", **kwargs)
+ record = self.dir.createRecord("resources", **kwargs)
except DirectoryError, e:
respondWithError(str(e))
return
+
+ principal = self.dir.principalCollection.principalForRecord(record)
+ (yield principal.setAutoSchedule(command.get('AutoSchedule', True)))
+
+ try:
+ self.dir.updateRecord("resources", **kwargs)
+ except DirectoryError, e:
+ respondWithError(str(e))
+ return
+
respondWithRecordsOfType(self.dir, command, "resources")
def command_getResourceAttributes(self, command):
@@ -277,6 +306,12 @@
@inlineCallbacks
def command_setResourceAttributes(self, command):
+ # Set autoSchedule prior to the updateRecord so that the right
+ # value ends up in memcached
+ principal = principalForPrincipalID(command['GeneratedUID'],
+ directory=self.dir)
+ (yield principal.setAutoSchedule(command.get('AutoSchedule', False)))
+
kwargs = {}
for key, info in attrMap.iteritems():
if command.has_key(key):
@@ -287,10 +322,6 @@
respondWithError(str(e))
return
- principal = principalForPrincipalID(command['GeneratedUID'],
- directory=self.dir)
- (yield principal.setAutoSchedule(command.get('AutoSchedule', False)))
-
self.command_getResourceAttributes(command)
def command_deleteResource(self, command):
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/principals.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/principals.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/principals.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -84,12 +84,6 @@
sys.exit(0)
def main():
- #
- # Send logging output to stdout
- #
- observer = StandardIOObserver()
- observer.start()
-
try:
(optargs, args) = getopt(
sys.argv[1:], "hf:P:", [
@@ -193,6 +187,13 @@
loadConfig(configFileName)
setLogLevelForNamespace(None, "warn")
+ #
+ # Send logging output to stdout
+ #
+ observer = StandardIOObserver()
+ observer.start()
+
+
# Shed privileges
if config.UserName and config.GroupName and os.getuid() == 0:
uid = getpwnam(config.UserName).pw_uid
@@ -227,7 +228,6 @@
usage("Too many arguments")
try:
- print config.directory
for record in config.directory.listRecords(listPrincipals):
print record
except UnknownRecordTypeError, e:
@@ -374,7 +374,10 @@
def action_addProxy(principal, proxyType, *proxyIDs):
for proxyID in proxyIDs:
proxyPrincipal = principalForPrincipalID(proxyID)
- (yield action_addProxyPrincipal(principal, proxyType, proxyPrincipal))
+ if proxyPrincipal is None:
+ print "Invalid principal ID: %s" % (proxyID,)
+ else:
+ (yield action_addProxyPrincipal(principal, proxyType, proxyPrincipal))
@inlineCallbacks
def action_addProxyPrincipal(principal, proxyType, proxyPrincipal):
@@ -416,7 +419,10 @@
def action_removeProxy(principal, *proxyIDs, **kwargs):
for proxyID in proxyIDs:
proxyPrincipal = principalForPrincipalID(proxyID)
- (yield action_removeProxyPrincipal(principal, proxyPrincipal, **kwargs))
+ if proxyPrincipal is None:
+ print "Invalid principal ID: %s" % (proxyID,)
+ else:
+ (yield action_removeProxyPrincipal(principal, proxyPrincipal, **kwargs))
@inlineCallbacks
def action_removeProxyPrincipal(principal, proxyPrincipal, **kwargs):
@@ -464,6 +470,14 @@
)
(yield principal.setAutoSchedule(autoSchedule))
+ # Invalidate the directory cache by updating this record
+ config.directory.updateRecord(principal.record.recordType,
+ guid=principal.record.guid,
+ shortNames=principal.record.shortNames,
+ fullName=principal.record.fullName,
+ **principal.record.extras
+ )
+
def action_getAutoSchedule(principal):
autoSchedule = principal.getAutoSchedule()
print "Autoschedule for %s is %s" % (
@@ -474,6 +488,8 @@
@inlineCallbacks
def _run(directory, root, optargs, principalIDs):
+ print sys.path
+
print ""
for opt, arg in optargs:
Copied: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/purge.py (from rev 5293, CalendarServer/trunk/calendarserver/tools/purge.py)
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/purge.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/purge.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -0,0 +1,249 @@
+#!/usr/bin/env python
+
+##
+# Copyright (c) 2006-2010 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from pwd import getpwnam
+from twisted.python.util import switchUID
+from twistedcaldav.directory.directory import DirectoryError
+from grp import getgrnam
+from calendarserver.tap.util import FakeRequest
+from calendarserver.tap.util import getRootResource
+from calendarserver.tools.util import loadConfig, setupMemcached, setupNotifications
+from datetime import date, timedelta
+from getopt import getopt, GetoptError
+from twext.python.log import Logger
+from twisted.internet import reactor
+from twisted.internet.defer import inlineCallbacks, returnValue, Deferred
+from twistedcaldav import caldavxml
+from twistedcaldav.caldavxml import TimeRange
+from twistedcaldav.config import config, ConfigurationError
+from twistedcaldav.method.delete_common import DeleteResource
+import os
+import sys
+
+log = Logger()
+
+def usage(e=None):
+
+ name = os.path.basename(sys.argv[0])
+ print "usage: %s [options]" % (name,)
+ print ""
+ print " Remove old events from the calendar server"
+ print ""
+ print "options:"
+ print " -d --days <number>: specify how many days in the past to retain (default=365)"
+ print " -f --config <path>: Specify caldavd.plist configuration path"
+ print " -h --help: print this help and exit"
+ print " -n --dry-run: only calculate how many events to purge"
+ print " -v --verbose: print progress information"
+ print ""
+
+ if e:
+ sys.exit(64)
+ else:
+ sys.exit(0)
+
+
+def main():
+
+ try:
+ (optargs, args) = getopt(
+ sys.argv[1:], "d:f:hnv", [
+ "days=",
+ "dry-run",
+ "config=",
+ "help",
+ "verbose",
+ ],
+ )
+ except GetoptError, e:
+ usage(e)
+
+ #
+ # Get configuration
+ #
+ configFileName = None
+ days = 365
+ dryrun = False
+ verbose = False
+
+ for opt, arg in optargs:
+ if opt in ("-h", "--help"):
+ usage()
+
+ elif opt in ("-d", "--days"):
+ try:
+ days = int(arg)
+ except ValueError, e:
+ print "Invalid value for --days: %s" % (arg,)
+ usage(e)
+
+ elif opt in ("-v", "--verbose"):
+ verbose = True
+
+ elif opt in ("-n", "--dry-run"):
+ dryrun = True
+
+ elif opt in ("-f", "--config"):
+ configFileName = arg
+
+ else:
+ raise NotImplementedError(opt)
+
+ try:
+ loadConfig(configFileName)
+
+ # Shed privileges
+ if config.UserName and config.GroupName and os.getuid() == 0:
+ uid = getpwnam(config.UserName).pw_uid
+ gid = getgrnam(config.GroupName).gr_gid
+ switchUID(uid, uid, gid)
+
+ os.umask(config.umask)
+
+ try:
+ rootResource = getRootResource(config)
+ directory = rootResource.getDirectory()
+ except DirectoryError, e:
+ print "Error: %s" % (e,)
+ return
+ setupMemcached(config)
+ setupNotifications(config)
+ except ConfigurationError, e:
+ print "Error: %s" % (e,)
+ return
+
+ cutoff = (date.today() - timedelta(days=days)).strftime("%Y%m%dT000000Z")
+
+ #
+ # Start the reactor
+ #
+ reactor.callLater(0.1, purgeThenStop, directory, rootResource, cutoff,
+ verbose=verbose, dryrun=dryrun)
+
+ reactor.run()
+
+ at inlineCallbacks
+def purgeThenStop(directory, rootResource, cutoff, verbose=False, dryrun=False):
+ exitCode = 0
+ try:
+ count = (yield purgeOldEvents(directory, rootResource, cutoff,
+ verbose=verbose, dryrun=dryrun))
+ if dryrun:
+ print "Would have purged %d events" % (count,)
+ else:
+ print "Purged %d events" % (count,)
+ except Exception, e:
+ print "Error: %s" % (e,)
+ finally:
+ reactor.stop()
+
+
+ at inlineCallbacks
+def purgeOldEvents(directory, root, date, verbose=False, dryrun=False):
+
+ calendars = root.getChild("calendars")
+ uidsFPath = calendars.fp.child("__uids__")
+
+ if dryrun:
+ print "Dry run"
+
+ if verbose:
+ print "Scanning calendar homes ...",
+
+ records = []
+ if uidsFPath.exists():
+ for firstFPath in uidsFPath.children():
+ if len(firstFPath.basename()) == 2:
+ for secondFPath in firstFPath.children():
+ if len(secondFPath.basename()) == 2:
+ for homeFPath in secondFPath.children():
+ uid = homeFPath.basename()
+ record = directory.recordWithUID(uid)
+ if record is not None:
+ records.append(record)
+ if verbose:
+ print "%d calendar homes found" % (len(records),)
+
+ log.info("Purging events from %d calendar homes" % (len(records),))
+
+ filter = caldavxml.Filter(
+ caldavxml.ComponentFilter(
+ caldavxml.ComponentFilter(
+ TimeRange(start=date,),
+ name=("VEVENT", "VFREEBUSY", "VAVAILABILITY"),
+ ),
+ name="VCALENDAR",
+ )
+ )
+
+ eventCount = 0
+ for record in records:
+ # Get the calendar home
+ principalCollection = directory.principalCollection
+ principal = principalCollection.principalForRecord(record)
+ calendarHome = principal.calendarHome()
+
+ if verbose:
+ print "%s %-15s :" % (record.uid, record.shortNames[0]),
+
+ homeEventCount = 0
+ # For each collection in calendar home...
+ for collName in calendarHome.listChildren():
+ collection = calendarHome.getChild(collName)
+ if collection.isCalendarCollection():
+ # ...use their indexes to figure out which events to purge.
+
+ # First, get the list of all child resources...
+ resources = set(collection.listChildren())
+
+ # ...and ignore those that appear *after* the given cutoff
+ for name, uid, type in collection.index().indexedSearch(filter):
+ if name in resources:
+ resources.remove(name)
+
+ for name in resources:
+ resource = collection.getChild(name)
+ uri = "/calendars/__uids__/%s/%s/%s" % (
+ record.uid,
+ collName,
+ name
+ )
+ try:
+ if not dryrun:
+ (yield deleteResource(root, collection, resource, uri))
+ eventCount += 1
+ homeEventCount += 1
+ except Exception, e:
+ log.error("Failed to purge old event: %s (%s)" %
+ (uri, e))
+
+ if verbose:
+ print "%d events" % (homeEventCount,)
+
+ returnValue(eventCount)
+
+
+def deleteResource(root, collection, resource, uri):
+ request = FakeRequest(root, "DELETE", uri)
+
+ # TODO: this seems hacky, even for a stub request:
+ request._rememberResource(resource, uri)
+
+ deleter = DeleteResource(request, resource, uri,
+ collection, "infinity", allowImplicitSchedule=False)
+ return deleter.run()
Deleted: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/purge/accounts.xml
===================================================================
--- CalendarServer/trunk/calendarserver/tools/test/purge/accounts.xml 2010-03-12 19:42:39 UTC (rev 5293)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/purge/accounts.xml 2010-03-12 20:35:30 UTC (rev 5298)
@@ -1,29 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-
-<!--
-Copyright (c) 2006-2010 Apple Inc. All rights reserved.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
- -->
-
-<!DOCTYPE accounts SYSTEM "../../../conf/auth/accounts.dtd">
-
-<accounts realm="/Search">
- <user>
- <uid>example</uid>
- <guid>6423F94A-6B76-4A3A-815B-D52CFD77935D</guid>
- <password>example</password>
- <name>Example User</name>
- <email-address>example at example.com</email-address>
- </user>
-</accounts>
Copied: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/purge/accounts.xml (from rev 5293, CalendarServer/trunk/calendarserver/tools/test/purge/accounts.xml)
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/purge/accounts.xml (rev 0)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/purge/accounts.xml 2010-03-12 20:35:30 UTC (rev 5298)
@@ -0,0 +1,29 @@
+<?xml version="1.0" encoding="utf-8"?>
+
+<!--
+Copyright (c) 2006-2010 Apple Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+ -->
+
+<!DOCTYPE accounts SYSTEM "../../../conf/auth/accounts.dtd">
+
+<accounts realm="/Search">
+ <user>
+ <uid>example</uid>
+ <guid>6423F94A-6B76-4A3A-815B-D52CFD77935D</guid>
+ <password>example</password>
+ <name>Example User</name>
+ <email-address>example at example.com</email-address>
+ </user>
+</accounts>
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/test_gateway.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/test_gateway.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/test_gateway.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -117,7 +117,7 @@
self.assertEquals(results['result']['Street'], "1 Infinite Loop")
self.assertEquals(results['result']['RealName'], "Created Location 01")
self.assertEquals(results['result']['Comment'], "Test Comment")
- self.assertEquals(results['result']['AutoSchedule'], False)
+ self.assertEquals(results['result']['AutoSchedule'], True)
@inlineCallbacks
def test_getResourceList(self):
@@ -140,8 +140,15 @@
yield self.runCommand(command_createLocation)
directory.flushCaches()
+
+ # This appears to be necessary in order for record.autoSchedule to
+ # reflect the change prior to the directory record expiration
+ augment.AugmentService.refresh()
+
record = directory.recordWithUID("836B1B66-2E9A-4F46-8B1C-3DD6772C20B2")
+
self.assertNotEquals(record, None)
+ self.assertEquals(record.autoSchedule, True)
self.assertEquals(record.extras['comment'], "Test Comment")
self.assertEquals(record.extras['building'], "Test Building")
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/test_principals.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/test_principals.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/test_principals.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -79,6 +79,7 @@
cwd = sourceRoot
deferred = Deferred()
+ e = os.environ
reactor.spawnProcess(CapturingProcessProtocol(deferred, None), python, args, env=os.environ, path=cwd)
output = yield deferred
returnValue(output)
Copied: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/test_purge.py (from rev 5293, CalendarServer/trunk/calendarserver/tools/test/test_purge.py)
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/test_purge.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/tools/test/test_purge.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -0,0 +1,330 @@
+##
+# Copyright (c) 2005-2010 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+import os
+import zlib
+from twistedcaldav.config import config
+from twistedcaldav.test.util import TestCase
+from twisted.internet.defer import inlineCallbacks
+from calendarserver.tap.util import getRootResource
+from calendarserver.tools.purge import purgeOldEvents
+
+resourceAttr = "WebDAV:{DAV:}resourcetype"
+collectionType = zlib.compress("""<?xml version='1.0' encoding='UTF-8'?>
+<resourcetype xmlns='DAV:'>
+ <collection/>
+ <calendar xmlns='urn:ietf:params:xml:ns:caldav'/>
+</resourcetype>
+""")
+
+
+class PurgeOldEventsTestCase(TestCase):
+
+ def setUp(self):
+ super(PurgeOldEventsTestCase, self).setUp()
+
+ config.DirectoryService.params['xmlFile'] = os.path.join(os.path.dirname(__file__), "purge", "accounts.xml")
+ self.rootResource = getRootResource(config)
+ self.directory = self.rootResource.getDirectory()
+
+ @inlineCallbacks
+ def test_purge(self):
+ before = {
+ "calendars" : {
+ "__uids__" : {
+ "64" : {
+ "23" : {
+ "6423F94A-6B76-4A3A-815B-D52CFD77935D" : {
+ "calendar": {
+ "@xattrs" :
+ {
+ resourceAttr : collectionType,
+ },
+ "oneshot.ics": {
+ "@contents" : OLD_ICS,
+ },
+ "endless.ics": {
+ "@contents" : ENDLESS_ICS,
+ },
+ "awhile.ics": {
+ "@contents" : REPEATING_AWHILE_ICS,
+ },
+ "straddling.ics": {
+ "@contents" : STRADDLING_ICS,
+ },
+ "recent.ics": {
+ "@contents" : RECENT_ICS,
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ }
+ self.createHierarchy(before, config.DocumentRoot)
+
+ count = (yield purgeOldEvents(self.directory, self.rootResource,
+ "20100303T000000Z"))
+
+ self.assertEquals(count, 2)
+
+ after = {
+ "calendars" : {
+ "__uids__" : {
+ "64" : {
+ "23" : {
+ "6423F94A-6B76-4A3A-815B-D52CFD77935D" : {
+ "calendar": {
+ ".db.sqlite": {
+ "@contents" : None, # ignore contents
+ },
+ "endless.ics": {
+ "@contents" : ENDLESS_ICS,
+ },
+ "straddling.ics": {
+ "@contents" : STRADDLING_ICS,
+ },
+ "recent.ics": {
+ "@contents" : RECENT_ICS,
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ }
+ self.assertTrue(self.verifyHierarchy(config.DocumentRoot, after))
+
+
+
+
+OLD_ICS = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//Apple Inc.//iCal 4.0.1//EN
+CALSCALE:GREGORIAN
+BEGIN:VTIMEZONE
+TZID:US/Pacific
+BEGIN:STANDARD
+TZOFFSETFROM:-0700
+RRULE:FREQ=YEARLY;UNTIL=20061029T090000Z;BYMONTH=10;BYDAY=-1SU
+DTSTART:19621028T020000
+TZNAME:PST
+TZOFFSETTO:-0800
+END:STANDARD
+BEGIN:DAYLIGHT
+TZOFFSETFROM:-0800
+RRULE:FREQ=YEARLY;UNTIL=20060402T100000Z;BYMONTH=4;BYDAY=1SU
+DTSTART:19870405T020000
+TZNAME:PDT
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:DAYLIGHT
+TZOFFSETFROM:-0800
+RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
+DTSTART:20070311T020000
+TZNAME:PDT
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:STANDARD
+TZOFFSETFROM:-0700
+RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
+DTSTART:20071104T020000
+TZNAME:PST
+TZOFFSETTO:-0800
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+CREATED:20100303T181216Z
+UID:685BC3A1-195A-49B3-926D-388DDACA78A6
+DTEND;TZID=US/Pacific:20000307T151500
+TRANSP:OPAQUE
+SUMMARY:Ancient event
+DTSTART;TZID=US/Pacific:20000307T111500
+DTSTAMP:20100303T181220Z
+SEQUENCE:2
+END:VEVENT
+END:VCALENDAR
+""".replace("\n", "\r\n")
+
+ENDLESS_ICS = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//Apple Inc.//iCal 4.0.1//EN
+CALSCALE:GREGORIAN
+BEGIN:VTIMEZONE
+TZID:US/Pacific
+BEGIN:STANDARD
+TZOFFSETFROM:-0700
+RRULE:FREQ=YEARLY;UNTIL=20061029T090000Z;BYMONTH=10;BYDAY=-1SU
+DTSTART:19621028T020000
+TZNAME:PST
+TZOFFSETTO:-0800
+END:STANDARD
+BEGIN:DAYLIGHT
+TZOFFSETFROM:-0800
+RRULE:FREQ=YEARLY;UNTIL=20060402T100000Z;BYMONTH=4;BYDAY=1SU
+DTSTART:19870405T020000
+TZNAME:PDT
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:DAYLIGHT
+TZOFFSETFROM:-0800
+RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
+DTSTART:20070311T020000
+TZNAME:PDT
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:STANDARD
+TZOFFSETFROM:-0700
+RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
+DTSTART:20071104T020000
+TZNAME:PST
+TZOFFSETTO:-0800
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+CREATED:20100303T194654Z
+UID:9FDE0E4C-1495-4CAF-863B-F7F0FB15FE8C
+DTEND;TZID=US/Pacific:20000308T151500
+RRULE:FREQ=YEARLY;INTERVAL=1
+TRANSP:OPAQUE
+SUMMARY:Ancient Repeating Endless
+DTSTART;TZID=US/Pacific:20000308T111500
+DTSTAMP:20100303T194710Z
+SEQUENCE:4
+END:VEVENT
+END:VCALENDAR
+""".replace("\n", "\r\n")
+
+REPEATING_AWHILE_ICS = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//Apple Inc.//iCal 4.0.1//EN
+CALSCALE:GREGORIAN
+BEGIN:VTIMEZONE
+TZID:US/Pacific
+BEGIN:STANDARD
+TZOFFSETFROM:-0700
+RRULE:FREQ=YEARLY;UNTIL=20061029T090000Z;BYMONTH=10;BYDAY=-1SU
+DTSTART:19621028T020000
+TZNAME:PST
+TZOFFSETTO:-0800
+END:STANDARD
+BEGIN:DAYLIGHT
+TZOFFSETFROM:-0800
+RRULE:FREQ=YEARLY;UNTIL=20060402T100000Z;BYMONTH=4;BYDAY=1SU
+DTSTART:19870405T020000
+TZNAME:PDT
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:DAYLIGHT
+TZOFFSETFROM:-0800
+RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
+DTSTART:20070311T020000
+TZNAME:PDT
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:STANDARD
+TZOFFSETFROM:-0700
+RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
+DTSTART:20071104T020000
+TZNAME:PST
+TZOFFSETTO:-0800
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+CREATED:20100303T194716Z
+UID:76236B32-2BC4-4D78-956B-8D42D4086200
+DTEND;TZID=US/Pacific:20000309T151500
+RRULE:FREQ=YEARLY;INTERVAL=1;COUNT=3
+TRANSP:OPAQUE
+SUMMARY:Ancient Repeat Awhile
+DTSTART;TZID=US/Pacific:20000309T111500
+DTSTAMP:20100303T194747Z
+SEQUENCE:6
+END:VEVENT
+END:VCALENDAR
+""".replace("\n", "\r\n")
+
+STRADDLING_ICS = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//Apple Inc.//iCal 4.0.1//EN
+CALSCALE:GREGORIAN
+BEGIN:VTIMEZONE
+TZID:US/Pacific
+BEGIN:DAYLIGHT
+TZOFFSETFROM:-0800
+RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
+DTSTART:20070311T020000
+TZNAME:PDT
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:STANDARD
+TZOFFSETFROM:-0700
+RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
+DTSTART:20071104T020000
+TZNAME:PST
+TZOFFSETTO:-0800
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+CREATED:20100303T213643Z
+UID:1C219DAD-D374-4822-8C98-ADBA85E253AB
+DTEND;TZID=US/Pacific:20090508T121500
+RRULE:FREQ=MONTHLY;INTERVAL=1;UNTIL=20100509T065959Z
+TRANSP:OPAQUE
+SUMMARY:Straddling cut-off
+DTSTART;TZID=US/Pacific:20090508T111500
+DTSTAMP:20100303T213704Z
+SEQUENCE:5
+END:VEVENT
+END:VCALENDAR
+""".replace("\n", "\r\n")
+
+RECENT_ICS = """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//Apple Inc.//iCal 4.0.1//EN
+CALSCALE:GREGORIAN
+BEGIN:VTIMEZONE
+TZID:US/Pacific
+BEGIN:DAYLIGHT
+TZOFFSETFROM:-0800
+RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
+DTSTART:20070311T020000
+TZNAME:PDT
+TZOFFSETTO:-0700
+END:DAYLIGHT
+BEGIN:STANDARD
+TZOFFSETFROM:-0700
+RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
+DTSTART:20071104T020000
+TZNAME:PST
+TZOFFSETTO:-0800
+END:STANDARD
+END:VTIMEZONE
+BEGIN:VEVENT
+CREATED:20100303T195159Z
+UID:F2F14D94-B944-43D9-8F6F-97F95B2764CA
+DTEND;TZID=US/Pacific:20100304T141500
+TRANSP:OPAQUE
+SUMMARY:Recent
+DTSTART;TZID=US/Pacific:20100304T120000
+DTSTAMP:20100303T195203Z
+SEQUENCE:2
+END:VEVENT
+END:VCALENDAR
+""".replace("\n", "\r\n")
+
Deleted: CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/util.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/util.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/calendarserver/util.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -1,431 +0,0 @@
-# -*- test-case-name: calendarserver.tap.test.test_caldav -*-
-##
-# Copyright (c) 2005-2010 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
-
-__all__ = [
- "getRootResource",
-]
-
-import errno
-import os
-from time import sleep
-
-from twisted.python.reflect import namedClass
-from twisted.internet import reactor
-from twisted.cred.portal import Portal
-from twext.web2.dav import auth
-from twext.web2.auth.basic import BasicCredentialFactory
-from twext.web2.static import File as FileResource
-from twext.python.filepath import CachingFilePath as FilePath
-
-from twext.python.log import Logger
-
-from twistedcaldav import memcachepool
-from twistedcaldav.accesslog import DirectoryLogWrapperResource
-from twistedcaldav.directory import augment, calendaruserproxy
-from twistedcaldav.directory.aggregate import AggregateDirectoryService
-from twistedcaldav.directory.calendaruserproxyloader import XMLCalendarUserProxyLoader
-from twistedcaldav.directory.digest import QopDigestCredentialFactory
-from twistedcaldav.directory.principal import DirectoryPrincipalProvisioningResource
-from twistedcaldav.directory.sudo import SudoDirectoryService
-from twistedcaldav.directory.util import NotFilePath
-from twistedcaldav.directory.wiki import WikiDirectoryService
-from twistedcaldav.notify import installNotificationClient
-from twistedcaldav.resource import CalDAVResource, AuthenticationWrapper
-from twistedcaldav.static import CalendarHomeProvisioningFile
-from twistedcaldav.static import IScheduleInboxFile
-from twistedcaldav.static import TimezoneServiceFile
-from twistedcaldav.static import AddressBookHomeProvisioningFile, DirectoryBackedAddressBookFile
-from twistedcaldav.timezones import TimezoneCache
-
-try:
- from twistedcaldav.authkerb import NegotiateCredentialFactory
- NegotiateCredentialFactory # pacify pyflakes
-except ImportError:
- NegotiateCredentialFactory = None
-
-from calendarserver.provision.root import RootResource
-from calendarserver.webadmin.resource import WebAdminResource
-from calendarserver.webcal.resource import WebCalendarResource
-
-log = Logger()
-
-
-
-def getRootResource(config, resources=None):
- """
- Set up directory service and resource hierarchy based on config.
- Return root resource.
-
- Additional resources can be added to the hierarchy by passing a list of
- tuples containing: path, resource class, __init__ args list, and optional
- authentication scheme ("basic" or "digest").
- """
-
- #
- # Default resource classes
- #
- rootResourceClass = RootResource
- principalResourceClass = DirectoryPrincipalProvisioningResource
- calendarResourceClass = CalendarHomeProvisioningFile
- iScheduleResourceClass = IScheduleInboxFile
- timezoneServiceResourceClass = TimezoneServiceFile
- webCalendarResourceClass = WebCalendarResource
- webAdminResourceClass = WebAdminResource
- addressBookResourceClass = AddressBookHomeProvisioningFile
- directoryBackedAddressBookResourceClass = DirectoryBackedAddressBookFile
-
- #
- # Setup the Directory
- #
- directories = []
-
- directoryClass = namedClass(config.DirectoryService.type)
-
- log.info("Configuring directory service of type: %s"
- % (config.DirectoryService.type,))
-
- baseDirectory = directoryClass(config.DirectoryService.params)
-
- # Wait for the directory to become available
- while not baseDirectory.isAvailable():
- sleep(5)
-
- directories.append(baseDirectory)
-
- #
- # Setup the Locations and Resources Service
- #
- if config.ResourceService.Enabled:
- resourceClass = namedClass(config.ResourceService.type)
-
- log.info("Configuring resource service of type: %s" % (resourceClass,))
-
- resourceDirectory = resourceClass(config.ResourceService.params)
- directories.append(resourceDirectory)
-
- #
- # Add sudoers directory
- #
- sudoDirectory = None
-
- if config.SudoersFile and os.path.exists(config.SudoersFile):
- log.info("Configuring SudoDirectoryService with file: %s"
- % (config.SudoersFile,))
-
- sudoDirectory = SudoDirectoryService(config.SudoersFile)
- sudoDirectory.realmName = baseDirectory.realmName
-
- CalDAVResource.sudoDirectory = sudoDirectory
- directories.insert(0, sudoDirectory)
- else:
- log.info( "Not using SudoDirectoryService; file doesn't exist: %s"
- % (config.SudoersFile,)
- )
-
- #
- # Add wiki directory service
- #
- if config.Authentication.Wiki.Enabled:
- wikiDirectory = WikiDirectoryService()
- wikiDirectory.realmName = baseDirectory.realmName
- directories.append(wikiDirectory)
-
- directory = AggregateDirectoryService(directories)
-
- if sudoDirectory:
- directory.userRecordTypes.insert(0,
- SudoDirectoryService.recordType_sudoers)
-
- #
- # Setup the Augment Service
- #
- augmentClass = namedClass(config.AugmentService.type)
-
- log.info("Configuring augment service of type: %s" % (augmentClass,))
-
- try:
- augment.AugmentService = augmentClass(**config.AugmentService.params)
- except IOError:
- log.error("Could not start augment service")
- raise
-
- #
- # Setup the PoxyDB Service
- #
- proxydbClass = namedClass(config.ProxyDBService.type)
-
- log.info("Configuring proxydb service of type: %s" % (proxydbClass,))
-
- try:
- calendaruserproxy.ProxyDBService = proxydbClass(**config.ProxyDBService.params)
- except IOError:
- log.error("Could not start proxydb service")
- raise
-
- #
- # Make sure proxies get initialized
- #
- if config.ProxyLoadFromFile:
- def _doProxyUpdate():
- loader = XMLCalendarUserProxyLoader(config.ProxyLoadFromFile)
- return loader.updateProxyDB()
-
- reactor.addSystemEventTrigger("after", "startup", _doProxyUpdate)
-
- #
- # Configure Memcached Client Pool
- #
- memcachepool.installPools(
- config.Memcached.Pools,
- config.Memcached.MaxClients,
- )
-
- #
- # Configure NotificationClient
- #
- if config.Notifications.Enabled:
- installNotificationClient(
- config.Notifications.InternalNotificationHost,
- config.Notifications.InternalNotificationPort,
- )
-
- #
- # Configure the Site and Wrappers
- #
- credentialFactories = []
-
- portal = Portal(auth.DavRealm())
-
- portal.registerChecker(directory)
-
- realm = directory.realmName or ""
-
- log.info("Configuring authentication for realm: %s" % (realm,))
-
- for scheme, schemeConfig in config.Authentication.iteritems():
- scheme = scheme.lower()
-
- credFactory = None
-
- if schemeConfig["Enabled"]:
- log.info("Setting up scheme: %s" % (scheme,))
-
- if scheme == "kerberos":
- if not NegotiateCredentialFactory:
- log.info("Kerberos support not available")
- continue
-
- try:
- principal = schemeConfig["ServicePrincipal"]
- if not principal:
- credFactory = NegotiateCredentialFactory(
- type="HTTP",
- hostname=config.ServerHostName,
- )
- else:
- credFactory = NegotiateCredentialFactory(
- principal=principal,
- )
- except ValueError:
- log.info("Could not start Kerberos")
- continue
-
- elif scheme == "digest":
- credFactory = QopDigestCredentialFactory(
- schemeConfig["Algorithm"],
- schemeConfig["Qop"],
- realm,
- )
-
- elif scheme == "basic":
- credFactory = BasicCredentialFactory(realm)
-
- elif scheme == "wiki":
- pass
-
- else:
- log.error("Unknown scheme: %s" % (scheme,))
-
- if credFactory:
- credentialFactories.append(credFactory)
-
-
- #
- # Setup Resource hierarchy
- #
- log.info("Setting up document root at: %s"
- % (config.DocumentRoot,))
- log.info("Setting up principal collection: %r"
- % (principalResourceClass,))
-
- principalCollection = principalResourceClass("/principals/", directory)
-
- log.info("Setting up calendar collection: %r" % (calendarResourceClass,))
-
- calendarCollection = calendarResourceClass(
- os.path.join(config.DocumentRoot, "calendars"),
- directory, "/calendars/",
- )
-
- log.info("Setting up root resource: %r" % (rootResourceClass,))
-
- root = rootResourceClass(
- config.DocumentRoot,
- principalCollections=(principalCollection,),
- )
-
- if config.EnableCardDAV:
- root.saclService = "addressbook" # XXX this needs to be dealt with
- # differently if caldav and carddav
- # are going to be in the same process
- log.info("Setting up address book collection: %r" % (addressBookResourceClass,))
-
- addressBookCollection = addressBookResourceClass(
- os.path.join(config.DocumentRoot, "addressbooks"),
- directory, "/addressbooks/"
- )
-
- directoryPath = os.path.join(config.DocumentRoot, "directory")
- doBacking = config.DirectoryAddressBook and config.EnableSearchAddressBook
- if doBacking:
- log.info("Setting up directory address book: %r" % (directoryBackedAddressBookResourceClass,))
-
- directoryBackedAddressBookCollection = directoryBackedAddressBookResourceClass(
- directoryPath,
- principalCollections=(principalCollection,)
- )
- # do this after process is owned by carddav user, not root. XXX
- # this should be fixed to execute at a different stage of service
- # startup entirely.
- reactor.callLater(1.0, directoryBackedAddressBookCollection.provisionDirectory)
- else:
- # remove /directory from previous runs that may have created it
- try:
- FilePath(directoryPath).remove()
- log.info("Deleted: %s" % directoryPath)
- except (OSError, IOError), e:
- if e.errno != errno.ENOENT:
- log.error("Could not delete: %s : %r" % (directoryPath, e,))
-
- root.putChild('addressbooks', addressBookCollection)
- if doBacking:
- root.putChild('directory', directoryBackedAddressBookCollection)
-
- root.putChild("principals", principalCollection)
- root.putChild("calendars", calendarCollection)
-
- for name, info in config.Aliases.iteritems():
- if os.path.sep in name or not info.get("path", None):
- log.error("Invalid alias: %s" % (name,))
- continue
- log.info("Adding alias %s -> %s" % (name, info["path"]))
- resource = FileResource(info["path"])
- root.putChild(name, resource)
-
- # Timezone service is optional
- if config.EnableTimezoneService:
- log.info("Setting up time zone service resource: %r"
- % (timezoneServiceResourceClass,))
-
- timezoneService = timezoneServiceResourceClass(
- NotFilePath(isfile=True),
- root,
- )
- root.putChild("timezones", timezoneService)
-
- # iSchedule service is optional
- if config.Scheduling.iSchedule.Enabled:
- log.info("Setting up iSchedule inbox resource: %r"
- % (iScheduleResourceClass,))
-
- ischedule = iScheduleResourceClass(
- NotFilePath(isfile=True),
- root,
- )
- root.putChild("ischedule", ischedule)
-
- #
- # WebCal
- #
- if config.WebCalendarRoot:
- log.info("Setting up WebCalendar resource: %s"
- % (config.WebCalendarRoot,))
- webCalendar = webCalendarResourceClass(
- config.WebCalendarRoot,
- principalCollections=(principalCollection,),
- )
- root.putChild("webcal", webCalendar)
-
- #
- # WebAdmin
- #
- if config.EnableWebAdmin:
- log.info("Setting up WebAdmin resource")
- webAdmin = webAdminResourceClass(
- config.WebCalendarRoot,
- root,
- directory,
- principalCollections=(principalCollection,),
- )
- root.putChild("admin", webAdmin)
-
- #
- # Configure ancillary data
- #
- log.info("Setting up Timezone Cache")
- TimezoneCache.create()
-
-
- log.info("Configuring authentication wrapper")
-
- overrides = { }
- if resources:
- for path, cls, args, scheme in resources:
-
- # putChild doesn't want "/" starting the path
- root.putChild(path, cls(root, *args))
-
- # overrides requires "/" prepended
- path = "/" + path
-
- if scheme == "basic":
- overrides[path] = (BasicCredentialFactory(realm),)
-
- elif scheme == "digest":
- schemeConfig = config.Authentication.Digest
- overrides[path] = (QopDigestCredentialFactory(
- schemeConfig["Algorithm"],
- schemeConfig["Qop"],
- realm,
- ),)
- log.info("Overriding %s with %s (%s)" % (path, cls, scheme))
-
- authWrapper = AuthenticationWrapper(
- root,
- portal,
- credentialFactories,
- (auth.IPrincipal,),
- overrides=overrides,
- )
-
- logWrapper = DirectoryLogWrapperResource(
- authWrapper,
- directory,
- )
-
- return logWrapper
-
Copied: CalendarServer/branches/users/cdaboo/shared-calendars-5187/doc/calendarserver_purge_events.8 (from rev 5293, CalendarServer/trunk/doc/calendarserver_purge_events.8)
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/doc/calendarserver_purge_events.8 (rev 0)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/doc/calendarserver_purge_events.8 2010-03-12 20:35:30 UTC (rev 5298)
@@ -0,0 +1,57 @@
+.\"
+.\" Copyright (c) 2006-2010 Apple Inc. All rights reserved.
+.\"
+.\" Licensed under the Apache License, Version 2.0 (the "License");
+.\" you may not use this file except in compliance with the License.
+.\" You may obtain a copy of the License at
+.\"
+.\" http://www.apache.org/licenses/LICENSE-2.0
+.\"
+.\" Unless required by applicable law or agreed to in writing, software
+.\" distributed under the License is distributed on an "AS IS" BASIS,
+.\" WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.\" See the License for the specific language governing permissions and
+.\" limitations under the License.
+.\"
+.\" The following requests are required for all man pages.
+.Dd June 17, 2009
+.Dt CALENDARSERVER_PURGE_EVENTS 8
+.Os
+.Sh NAME
+.Nm calendarserver_purge_events
+.Nd Darwin Calendar Server event clean-up utility
+.Sh SYNOPSIS
+.Nm
+.Op Fl -config Ar file
+.Op Fl -days Ad number
+.Op Fl -dry-run
+.Op Fl -verbose
+.Op Fl -help
+.Sh DESCRIPTION
+.Nm
+is a tool for removing old events from the calendar server. By default, events older than 365 days are removed, but the user can specify the number of days in the past to use as a cut-off. Repeating events that have any occurrences after the cut-off day are not removed.
+.Pp
+.Nm
+should be run as a user with the same priviledges as the Calendar
+Server itself, as it needs to read and write data that belongs to the
+server.
+.Sh OPTIONS
+.Bl -tag -width flag
+.It Fl h, -help
+Display usage information
+.It Fl f, -config Ar FILE
+Use the Calendar Server configuration specified in the given file. Defaults to /etc/caldavd/caldavd.plist.
+.It Fl n, -dry-run
+Calculate and display how many events would be removed, but don't actually remove them.
+.It Fl v, -verbose
+Print progress information including purged events per calendar home.
+.It Fl d, -days Ar NUMBER
+Specify how many days in the past to retain. Defaults to 365 days.
+.El
+.Sh FILES
+.Bl -tag -width flag
+.It /etc/caldavd/caldavd.plist
+The Calendar Server configuration file.
+.El
+.Sh SEE ALSO
+.Xr caldavd 8
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/run
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/run 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/run 2010-03-12 20:35:30 UTC (rev 5298)
@@ -35,7 +35,7 @@
wd="$(cd "$(dirname "$0")" && pwd)";
-. support/build.sh;
+. "${wd}/support/build.sh";
DAVD="cal"
@@ -67,6 +67,7 @@
echo " -S Write a pstats object for each process to the given directory when the server is stopped.";
echo " -P Select the twistd plugin name [${plugin_name}]";
echo " -R Twisted Reactor plugin to execute [${reactor}]";
+ echo " -a Start in CardDAV mode";
if [ "${1-}" == "-" ]; then
return 0;
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/setup.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/setup.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/setup.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -109,11 +109,12 @@
scripts = [
"bin/caldavd",
"bin/calendarserver_export",
- "bin/calendarserver_load_augmentdb",
- "bin/calendarserver_make_partition",
- "bin/calendarserver_manage_augments",
+ #"bin/calendarserver_load_augmentdb",
+ #"bin/calendarserver_make_partition",
+ #"bin/calendarserver_manage_augments",
"bin/calendarserver_manage_principals",
"bin/calendarserver_command_gateway",
+ "bin/calendarserver_purge_events",
"bin/carddavd",
],
data_files = [ ("caldavd", ["conf/caldavd.plist"]),
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/Makefile.Apple
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/Makefile.Apple 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/Makefile.Apple 2010-03-12 20:35:30 UTC (rev 5298)
@@ -39,22 +39,21 @@
# Build
#
-.phony: $(Project) vobject Twisted build setup prep install install-ossfiles buildit
+.phony: $(Project) vobject build setup prep install install-ossfiles buildit
CalDAVTester:: $(BuildDirectory)/CalDAVTester
PyKerberos:: $(BuildDirectory)/PyKerberos
PyOpenDirectory:: $(BuildDirectory)/PyOpenDirectory
PyXML-0.8.4:: $(BuildDirectory)/PyXML-0.8.4
vobject:: $(BuildDirectory)/vobject
-Twisted:: $(BuildDirectory)/Twisted
$(Project):: $(BuildDirectory)/$(Project)
-build:: PyKerberos PyOpenDirectory PyXML-0.8.4 vobject Twisted $(Project)
+build:: PyKerberos PyOpenDirectory PyXML-0.8.4 vobject $(Project)
setup:
$(_v) ./run -g
-prep:: setup CalDAVTester.tgz PyKerberos.tgz PyOpenDirectory.tgz PyXML-0.8.4.tgz vobject.tgz Twisted.tgz
+prep:: setup CalDAVTester.tgz PyKerberos.tgz PyOpenDirectory.tgz PyXML-0.8.4.tgz vobject.tgz
PyKerberos PyOpenDirectory PyXML-0.8.4 vobject $(Project)::
@echo "Building $@..."
@@ -62,15 +61,6 @@
TwistedSubEnvironment = $(Environment) PYTHONPATH="$(DSTROOT)$(PY_HOME)/lib/python"
-Twisted::
- @echo "Building Twisted..."
- $(_v) cd $(BuildDirectory)/Twisted && $(Environment) $(PYTHON) twisted/topfiles/setup.py install $(PY_INSTALL_FLAGS)
- $(_v) cd $(BuildDirectory)/Twisted && $(Environment) $(PYTHON) twisted/runner/topfiles/setup.py build
- $(_v) cd $(BuildDirectory)/Twisted && $(TwistedSubEnvironment) $(PYTHON) twisted/mail/topfiles/setup.py build
- $(_v) cd $(BuildDirectory)/Twisted && $(TwistedSubEnvironment) $(PYTHON) twisted/web/topfiles/setup.py build
- $(_v) cd $(BuildDirectory)/Twisted && $(TwistedSubEnvironment) $(PYTHON) twisted/web2/topfiles/setup.py build
- $(_v) cd $(BuildDirectory)/Twisted && $(TwistedSubEnvironment) $(PYTHON) twisted/words/topfiles/setup.py build
-
install:: build
$(_v) cd $(BuildDirectory)/$(Project) && $(Environment) $(PYTHON) setup.py install \
$(PY_INSTALL_FLAGS) \
@@ -80,19 +70,7 @@
$(_v) cd $(BuildDirectory)/PyOpenDirectory && $(Environment) $(PYTHON) setup.py install $(PY_INSTALL_FLAGS)
$(_v) cd $(BuildDirectory)/PyXML-0.8.4 && $(Environment) $(PYTHON) setup.py install $(PY_INSTALL_FLAGS)
$(_v) cd $(BuildDirectory)/vobject && $(Environment) $(PYTHON) setup.py install $(PY_INSTALL_FLAGS)
- $(_v) cd $(BuildDirectory)/Twisted && $(TwistedSubEnvironment) $(PYTHON) twisted/runner/topfiles/setup.py install $(PY_INSTALL_FLAGS)
- $(_v) cd $(BuildDirectory)/Twisted && $(TwistedSubEnvironment) $(PYTHON) twisted/mail/topfiles/setup.py install $(PY_INSTALL_FLAGS)
- $(_v) cd $(BuildDirectory)/Twisted && $(TwistedSubEnvironment) $(PYTHON) twisted/web/topfiles/setup.py install $(PY_INSTALL_FLAGS)
- $(_v) cd $(BuildDirectory)/Twisted && $(TwistedSubEnvironment) $(PYTHON) twisted/web2/topfiles/setup.py install $(PY_INSTALL_FLAGS)
- $(_v) cd $(BuildDirectory)/Twisted && $(TwistedSubEnvironment) $(PYTHON) twisted/words/topfiles/setup.py install $(PY_INSTALL_FLAGS)
- $(_v) for so in $$(find "$(DSTROOT)$(PY_HOME)/lib" -type f -name '*.so'); do $(STRIP) -Sx "$${so}"; done
- $(_v) echo "Removing plugins:"
- $(_v) find "$(DSTROOT)$(PY_HOME)/lib/python/twisted/plugins/" -type f ! -name 'caldav.*' ! -name 'carddav.*' ! -name 'twisted_reactors.*' ! -name 'twisted_trial.*' ! -name '__init__.*' ! -name 'kqueuereactor.*' -print -exec rm '{}' ';'
- $(_v) rm -fr "$(DSTROOT)$(PY_HOME)/lib/python/twisted/python/zsh"
- $(_v) rm -f "$(DSTROOT)$(PY_HOME)/lib/python/twisted/python/zshcomp.py"
- $(_v) rm -f "$(DSTROOT)$(PY_HOME)/lib/python/twisted/python/zshcomp.py"
- $(_v) rm -f "$(DSTROOT)$(PY_HOME)/lib/python/twisted/python/zshcomp.pyc"
- $(_v) rm -f "$(DSTROOT)$(PY_HOME)/lib/python/twisted/python/_twisted_zsh_stub"
+ $(_v) for so in $$(find "$(DSTROOT)$(PY_HOME)/lib" -type f -name '*.so'); do $(STRIP) -Sx "$${so}"; done
$(_v) $(INSTALL_FILE) "$(Sources)/conf/caldavd-apple.plist" "$(DSTROOT)$(ETCDIR)/caldavd/caldavd.plist"
$(_v) $(INSTALL_FILE) "$(Sources)/conf/caldavd-partitioning-primary.plist" "$(DSTROOT)$(ETCDIR)/caldavd/caldavd-partitioning-primary.plist"
$(_v) $(INSTALL_FILE) "$(Sources)/conf/caldavd-partitioning-secondary.plist" "$(DSTROOT)$(ETCDIR)/caldavd/caldavd-partitioning-secondary.plist"
@@ -111,6 +89,7 @@
$(_v) $(INSTALL_FILE) "$(Sources)/doc/calendarserver_export.8" "$(DSTROOT)$(MANDIR)/man8"
$(_v) $(INSTALL_FILE) "$(Sources)/doc/calendarserver_manage_principals.8" "$(DSTROOT)$(MANDIR)/man8"
$(_v) $(INSTALL_FILE) "$(Sources)/doc/calendarserver_command_gateway.8" "$(DSTROOT)$(MANDIR)/man8"
+ $(_v) $(INSTALL_FILE) "$(Sources)/doc/calendarserver_purge_events.8" "$(DSTROOT)$(MANDIR)/man8"
$(_v) gzip -9 -f "$(DSTROOT)$(MANDIR)/man8/"*.[0-9]
@echo "Installing launchd config..."
$(_v) $(INSTALL_DIRECTORY) "$(DSTROOT)$(NSLOCALDIR)/$(NSLIBRARYSUBDIR)/$(Project)"
@@ -141,7 +120,7 @@
$(BuildDirectory)/$(Project):
@echo "Copying source for $(Project)..."
$(_v) $(MKDIR) -p "$@"
- $(_v) pax -rw bin conf Makefile lib-patches setup.py kqreactor.py memcacheclient.py calendarserver twistedcaldav twext twisted support "$@/"
+ $(_v) pax -rw bin conf Makefile lib-patches setup.py kqreactor.py memcacheclient.py calendarserver twistedcaldav twext txdav txcaldav txcarddav twisted support "$@/"
$(BuildDirectory)/%: %.tgz
@echo "Extracting source for $(notdir $<)..."
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/build.sh
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/build.sh 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/build.sh 2010-03-12 20:35:30 UTC (rev 5298)
@@ -17,7 +17,7 @@
# limitations under the License.
##
-. support/py.sh;
+. "${wd}/support/py.sh";
# Provide a default value: if the variable named by the first argument is
# empty, set it to the default in the second argument.
@@ -491,9 +491,9 @@
"PyGreSQL" "pgdb" "PyGreSQL-4.0" \
"ftp://ftp.pygresql.org/pub/distrib/PyGreSQL.tgz";
- py_dependency -v 10 -r 27606 \
+ py_dependency -v 10 -r 28657 \
"Twisted" "twisted" "Twisted" \
- "svn://svn.twistedmatrix.com/svn/Twisted/tags/releases/twisted-9.0.0";
+ "svn://svn.twistedmatrix.com/svn/Twisted/tags/releases/twisted-10.0.0";
py_dependency \
"dateutil" "dateutil" "python-dateutil-1.4.1" \
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/submit
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/submit 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/support/submit 2010-03-12 20:35:30 UTC (rev 5298)
@@ -174,7 +174,7 @@
merge_flags="";
fi;
- sudo ~rc/bin/buildit "${wc}" CALENDARSERVER_CACHE_DEPS="${CALENDARSERVER_CACHE_DEPS-${wd}/.dependencies}" \
+ sudo buildit "${wc}" CALENDARSERVER_CACHE_DEPS="${CALENDARSERVER_CACHE_DEPS-${wd}/.dependencies}" \
$(file /System/Library/Frameworks/Python.framework/Versions/Current/Python | sed -n -e 's|^.*(for architecture \([^)][^)]*\).*$|-arch \1|p' | sed 's|ppc7400|ppc|') \
${merge_flags};
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/python/filepath.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/python/filepath.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/python/filepath.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -1,3 +1,4 @@
+# -*- test-case-name: twext.python.test.test_filepath -*-
##
# Copyright (c) 2010 Apple Inc. All rights reserved.
#
@@ -19,6 +20,17 @@
calendar server.
"""
+from os import listdir as _listdir
+
+from os.path import (join as _joinpath,
+ basename as _basename,
+ exists as _exists,
+ dirname as _dirname)
+
+from time import sleep as _sleep
+from types import FunctionType, MethodType
+from errno import EINVAL
+
from twisted.python.filepath import FilePath
from stat import S_ISDIR
@@ -29,12 +41,33 @@
aggressive caching policy.
"""
+ _listdir = _listdir # integration points for tests
+ _sleep = _sleep
+
+ BACKOFF_MAX = 5.0 # Maximum time to wait between calls to
+ # listdir()
+
def __init__(self, path, alwaysCreate=False):
super(CachingFilePath, self).__init__(path, alwaysCreate)
self.existsCached = None
self.isDirCached = None
+ @property
+ def siblingExtensionSearch(self):
+ """
+ Dynamically create a version of L{FilePath.siblingExtensionSearch} that
+ uses a pluggable 'listdir' implementation.
+ """
+ return MethodType(FunctionType(
+ FilePath.siblingExtensionSearch.im_func.func_code,
+ {'listdir': self._retryListdir,
+ 'basename': _basename,
+ 'dirname': _dirname,
+ 'joinpath': _joinpath,
+ 'exists': _exists}), self, self.__class__)
+
+
def changed(self):
"""
This path may have changed in the filesystem, so forget all cached
@@ -45,6 +78,32 @@
self.isDirCached = None
+ def _retryListdir(self, pathname):
+ """
+ Implementation of retry logic for C{listdir} and
+ C{siblingExtensionSearch}.
+ """
+ delay = 0.1
+ while True:
+ try:
+ return self._listdir(pathname)
+ except OSError, e:
+ if e.errno == EINVAL:
+ self._sleep(delay)
+ delay = min(self.BACKOFF_MAX, delay * 2.0)
+ else:
+ raise
+ raise RuntimeError("unreachable code.")
+
+
+ def listdir(self):
+ """
+ List the directory which C{self.path} points to, compensating for
+ EINVAL from C{os.listdir}.
+ """
+ return self._retryListdir(self.path)
+
+
def restat(self, reraise=True):
"""
Re-cache stat information.
Copied: CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/python/test/test_filepath.py (from rev 5293, CalendarServer/trunk/twext/python/test/test_filepath.py)
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/python/test/test_filepath.py (rev 0)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/python/test/test_filepath.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -0,0 +1,160 @@
+##
+# Copyright (c) 2010 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+"""
+Tests for specialized behavior of L{CachingFilePath}
+"""
+from errno import EINVAL
+from os.path import join as pathjoin
+
+from twisted.internet.task import Clock
+
+from twisted.trial.unittest import TestCase
+
+from twext.python.filepath import CachingFilePath
+
+# Cheat and pull in the Twisted test cases for FilePath. XXX: Twisteds should
+# provide a supported way of doing this for exported interfaces. Also, it
+# should export IFilePath. --glyph
+
+from twisted.test.test_paths import FilePathTestCase
+
+class BaseVerification(FilePathTestCase):
+ """
+ Make sure that L{CachingFilePath} doesn't break the contracts that
+ L{FilePath} tries to provide.
+ """
+
+ def setUp(self):
+ """
+ Set up the test case to set the base attributes to point at
+ L{AbstractFilePathTestCase}.
+ """
+ FilePathTestCase.setUp(self)
+ self.root = CachingFilePath(self.root.path)
+ self.path = CachingFilePath(self.path.path)
+
+
+
+class EINVALTestCase(TestCase):
+ """
+ Sometimes, L{os.listdir} will raise C{EINVAL}. This is a transient error,
+ and L{CachingFilePath.listdir} should work around it by retrying the
+ C{listdir} operation until it succeeds.
+ """
+
+ def setUp(self):
+ """
+ Create a L{CachingFilePath} for the test to use.
+ """
+ self.cfp = CachingFilePath(self.mktemp())
+ self.clock = Clock()
+ self.cfp._sleep = self.clock.advance
+
+
+ def test_testValidity(self):
+ """
+ If C{listdir} is replaced on a L{CachingFilePath}, we should be able to
+ observe exceptions raised by the replacement. This verifies that the
+ test patching done here is actually testing something.
+ """
+ class CustomException(Exception): "Just for testing."
+ def blowUp(dirname):
+ raise CustomException()
+ self.cfp._listdir = blowUp
+ self.assertRaises(CustomException, self.cfp.listdir)
+ self.assertRaises(CustomException, self.cfp.children)
+
+
+ def test_retryLoop(self):
+ """
+ L{CachingFilePath} should catch C{EINVAL} and respond by retrying the
+ C{listdir} operation until it succeeds.
+ """
+ calls = []
+ def raiseEINVAL(dirname):
+ calls.append(dirname)
+ if len(calls) < 5:
+ raise OSError(EINVAL, "This should be caught by the test.")
+ return ['a', 'b', 'c']
+ self.cfp._listdir = raiseEINVAL
+ self.assertEquals(self.cfp.listdir(), ['a', 'b', 'c'])
+ self.assertEquals(self.cfp.children(), [
+ CachingFilePath(pathjoin(self.cfp.path, 'a')),
+ CachingFilePath(pathjoin(self.cfp.path, 'b')),
+ CachingFilePath(pathjoin(self.cfp.path, 'c')),])
+
+
+ def requireTimePassed(self, filenames):
+ """
+ Create a replacement for listdir() which only fires after a certain
+ amount of time.
+ """
+ self.calls = []
+ def thunk(dirname):
+ now = self.clock.seconds()
+ if now < 20.0:
+ self.calls.append(now)
+ raise OSError(EINVAL, "Not enough time has passed yet.")
+ else:
+ return filenames
+ self.cfp._listdir = thunk
+
+
+ def assertRequiredTimePassed(self):
+ """
+ Assert that calls to the simulated time.sleep() installed by
+ C{requireTimePassed} have been invoked the required number of times.
+ """
+ # Waiting should be growing by *2 each time until the additional wait
+ # exceeds BACKOFF_MAX (5), at which point we should wait for 5s each
+ # time.
+ def cumulative(values):
+ current = 0.0
+ for value in values:
+ current += value
+ yield current
+
+ self.assertEquals(self.calls,
+ list(cumulative(
+ [0.0, 0.1, 0.2, 0.4, 0.8, 1.6, 3.2, 5.0, 5.0])))
+
+
+ def test_backoff(self):
+ """
+ L{CachingFilePath} will wait for an increasing interval up to
+ C{BACKOFF_MAX} between calls to listdir().
+ """
+ self.requireTimePassed(['a', 'b', 'c'])
+ self.assertEquals(self.cfp.listdir(), ['a', 'b', 'c'])
+
+
+ def test_siblingExtensionSearch(self):
+ """
+ L{FilePath.siblingExtensionSearch} is unfortunately not implemented in
+ terms of L{FilePath.listdir}, so we need to verify that it will also
+ retry.
+ """
+ filenames = [self.cfp.basename()+'.a',
+ self.cfp.basename() + '.b',
+ self.cfp.basename() + '.c']
+ siblings = map(self.cfp.sibling, filenames)
+ for sibling in siblings:
+ sibling.touch()
+ self.requireTimePassed(filenames)
+ self.assertEquals(self.cfp.siblingExtensionSearch("*"),
+ siblings[0])
+ self.assertRequiredTimePassed()
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/web2/resource.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/web2/resource.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/web2/resource.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -312,5 +312,8 @@
return x.addCallback(lambda data: self.resource)
return self.resource
+ def getChild(self, name):
+ return self.resource.getChild(name)
+
__all__ = ['RenderMixin', 'Resource', 'PostableResource', 'LeafResource', 'WrapperResource']
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/web2/server.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/web2/server.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/twext/web2/server.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -489,14 +489,6 @@
"URL may not contain a query or fragment: %s" % (url,)
))
- # The caller shouldn't be asking a request on one server to lookup a
- # resource on some other server.
- if (scheme and scheme != self.scheme) or (host and host != self.headers.getHeader("host")):
- raise http.HTTPError(http.StatusResponse(
- responsecode.BAD_GATEWAY,
- "URL is not on this site (%s://%s/): %s" % (scheme, self.headers.getHeader("host"), url)
- ))
-
# Look for cached value
cached = self._resourcesByURL.get(path, None)
if cached is not None:
Deleted: CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/accesslog.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/accesslog.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/accesslog.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -1,445 +0,0 @@
-##
-# Copyright (c) 2006-2009 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
-
-"""
-Access logs.
-"""
-
-__all__ = [
- "DirectoryLogWrapperResource",
- "RotatingFileAccessLoggingObserver",
- "AMPCommonAccessLoggingObserver",
- "AMPLoggingFactory",
-]
-
-import datetime
-import os
-import time
-
-from twisted.internet import protocol
-from twisted.protocols import amp
-from twext.web2 import iweb
-from twext.web2.dav import davxml
-from twext.web2.log import BaseCommonAccessLoggingObserver
-from twext.web2.log import LogWrapperResource
-
-from twext.python.log import Logger
-
-from twistedcaldav.config import config
-from twistedcaldav.directory.directory import DirectoryService
-
-log = Logger()
-
-class DirectoryLogWrapperResource(LogWrapperResource):
-
- def __init__(self, resource, directory):
- super(DirectoryLogWrapperResource, self).__init__(resource)
-
- self.directory = directory
-
- def getDirectory(self):
- return self.directory
-
-class CommonAccessLoggingObserverExtensions(BaseCommonAccessLoggingObserver):
- """
- A base class for our extension to the L{BaseCommonAccessLoggingObserver}
- """
-
- def emit(self, eventDict):
-
- if eventDict.get("interface") is iweb.IRequest:
-
- if config.GlobalStatsLoggingFrequency is not 0:
- self.logGlobalHit()
-
- request = eventDict["request"]
- response = eventDict["response"]
- loginfo = eventDict["loginfo"]
-
- # Try to determine authentication and authorization identifiers
- uid = "-"
- if hasattr(request, "authnUser"):
- if isinstance(request.authnUser.children[0], davxml.HRef):
- uidn = str(request.authnUser.children[0])
- uidz = None
- if hasattr(request, "authzUser") and str(request.authzUser.children[0]) != uidn:
- uidz = str(request.authzUser.children[0])
-
- def convertUIDtoShortName(uid):
- uid = uid.rstrip("/")
- uid = uid[uid.rfind("/") + 1:]
- record = request.site.resource.getDirectory().recordWithUID(uid)
- if record:
- if record.recordType == DirectoryService.recordType_users:
- return record.shortNames[0]
- else:
- return "(%s)%s" % (record.recordType, record.shortNames[0],)
- else:
- return uid
-
- uidn = convertUIDtoShortName(uidn)
- if uidz:
- uidz = convertUIDtoShortName(uidz)
-
- if uidn and uidz:
- uid = '"%s as %s"' % (uidn, uidz,)
- else:
- uid = uidn
-
- #
- # For some methods which basically allow you to tunnel a
- # custom request (eg. REPORT, POST), the method name
- # itself doesn't tell you much about what action is being
- # requested. This allows a method to tack a submethod
- # attribute to the request, so we can provide a little
- # more detail here.
- #
- if config.EnableExtendedAccessLog and hasattr(request, "submethod"):
- method = "%s(%s)" % (request.method, request.submethod)
- else:
- method = request.method
-
- # Standard Apache access log fields
- format = (
- '%(host)s - %(uid)s [%(date)s]'
- ' "%(method)s %(uri)s HTTP/%(protocolVersion)s"'
- ' %(statusCode)s %(bytesSent)d'
- ' "%(referer)s" "%(userAgent)s"'
- )
-
- if config.EnableExtendedAccessLog:
- formats = [
- format,
- # Performance monitoring extensions
- 'i=%(serverInstance)s t=%(timeSpent).1f or=%(outstandingRequests)s',
- ]
- if hasattr(request, "extendedLogItems"):
- for k, v in request.extendedLogItems.iteritems():
- k = str(k).replace('"', "%22")
- v = str(v).replace('"', "%22")
- if " " in v:
- v = '"%s"' % (v,)
- formats.append("%s=%s" % (k, v))
-
- fwdHeaders = request.headers.getRawHeaders("x-forwarded-for", "")
- if fwdHeaders:
- # Limit each x-forwarded-header to 50 in case someone is
- # trying to overwhelm the logs
- forwardedFor = ",".join([hdr[:50] for hdr in fwdHeaders])
- forwardedFor = forwardedFor.replace(" ", "")
- formats.append("fwd=%(fwd)s")
- else:
- forwardedFor = ""
-
- format = " ".join(formats)
-
- formatArgs = {
- "host" : request.remoteAddr.host,
- "uid" : uid,
- "date" : self.logDateString(response.headers.getHeader("date", 0)),
- "method" : method,
- "uri" : request.uri.replace('"', "%22"),
- "protocolVersion" : ".".join(str(x) for x in request.clientproto),
- "statusCode" : response.code,
- "bytesSent" : loginfo.bytesSent,
- "referer" : request.headers.getHeader("referer", "-"),
- "userAgent" : request.headers.getHeader("user-agent", "-"),
- "serverInstance" : config.LogID,
- "timeSpent" : (time.time() - request.initTime) * 1000,
- "outstandingRequests" : request.chanRequest.channel.factory.outstandingRequests,
- "fwd" : forwardedFor,
- }
- self.logMessage(format % formatArgs)
-
- elif "overloaded" in eventDict:
- overloaded = eventDict.get("overloaded")
- format_str = '%s - - [%s] "???" 503 0 "-" "-" [0.0 ms]'
- format_data = (
- overloaded.transport.hostname,
- self.logDateString(time.time()),
- )
- if config.EnableExtendedAccessLog:
- format_str += " [%s %s]"
- format_data += (
- overloaded.transport.server.port,
- overloaded.outstandingRequests,
- )
- self.logMessage(format_str % format_data)
-
-class RotatingFileAccessLoggingObserver(CommonAccessLoggingObserverExtensions):
- """
- Class to do "apache" style access logging to a rotating log file. The log
- file is rotated after midnight each day.
- """
-
- def __init__(self, logpath):
- self.logpath = logpath
- self.globalHitCount = 0
- self.globalHitHistory = []
- for i in range(0, config.GlobalStatsLoggingFrequency + 1):
- self.globalHitHistory.append({"time":int(time.time()), "hits":0})
-
- def logMessage(self, message, allowrotate=True):
- """
- Log a message to the file and possibly rotate if date has changed.
-
- @param message: C{str} for the message to log.
- @param allowrotate: C{True} if log rotate allowed, C{False} to log to current file
- without testing for rotation.
- """
-
- if self.shouldRotate() and allowrotate:
- self.flush()
- self.rotate()
- self.f.write(message + "\n")
-
- def rotateGlobalHitHistoryStats(self):
- """
- Roll the global hit history array: push the current stats as
- the last element; pop the first (oldest) element and reschedule the task.
- """
-
- self.globalHitHistory.append({"time":int(time.time()), "hits":self.globalHitCount})
- del self.globalHitHistory[0]
- log.debug("rotateGlobalHitHistoryStats: %s" % (self.globalHitHistory,))
- if config.GlobalStatsLoggingFrequency is not 0:
- self.reactor.callLater(
- config.GlobalStatsLoggingPeriod * 60 / config.GlobalStatsLoggingFrequency,
- self.rotateGlobalHitHistoryStats
- )
-
- def start(self):
- """
- Start logging. Open the log file and log an "open" message.
- """
-
- super(RotatingFileAccessLoggingObserver, self).start()
- self._open()
- self.logMessage("Log opened - server start: [%s]." % (datetime.datetime.now().ctime(),))
-
- # Need a reactor for the callLater() support for rotateGlobalHitHistoryStats()
- from twisted.internet import reactor
- self.reactor = reactor
- self.rotateGlobalHitHistoryStats()
-
- def stop(self):
- """
- Stop logging. Close the log file and log an "open" message.
- """
-
- self.logMessage("Log closed - server stop: [%s]." % (datetime.datetime.now().ctime(),), False)
- super(RotatingFileAccessLoggingObserver, self).stop()
- self._close()
-
- def _open(self):
- """
- Open the log file.
- """
-
- self.f = open(self.logpath, "a", 1)
- self.lastDate = self.toDate(os.stat(self.logpath)[8])
-
- def _close(self):
- """
- Close the log file.
- """
-
- self.f.close()
-
- def flush(self):
- """
- Flush the log file.
- """
-
- self.f.flush()
-
- def shouldRotate(self):
- """
- Rotate when the date has changed since last write
- """
-
- if config.RotateAccessLog:
- return self.toDate() > self.lastDate
- else:
- return False
-
- def toDate(self, *args):
- """
- Convert a unixtime to (year, month, day) localtime tuple,
- or return the current (year, month, day) localtime tuple.
-
- This function primarily exists so you may overload it with
- gmtime, or some cruft to make unit testing possible.
- """
-
- # primarily so this can be unit tested easily
- return time.localtime(*args)[:3]
-
- def suffix(self, tupledate):
- """
- Return the suffix given a (year, month, day) tuple or unixtime
- """
-
- try:
- return "_".join(map(str, tupledate))
- except:
- # try taking a float unixtime
- return "_".join(map(str, self.toDate(tupledate)))
-
- def rotate(self):
- """
- Rotate the file and create a new one.
-
- If it's not possible to open new logfile, this will fail silently,
- and continue logging to old logfile.
- """
-
- newpath = "%s.%s" % (self.logpath, self.suffix(self.lastDate))
- if os.path.exists(newpath):
- log.msg("Cannot rotate log file to %s because it already exists." % (newpath,))
- return
- self.logMessage("Log closed - rotating: [%s]." % (datetime.datetime.now().ctime(),), False)
- log.msg("Rotating log file to: %s" % (newpath,), system="Logging")
- self.f.close()
- os.rename(self.logpath, newpath)
- self._open()
- self.logMessage("Log opened - rotated: [%s]." % (datetime.datetime.now().ctime(),), False)
-
- def logGlobalHit(self):
- """
- Increment the service-global hit counter
- """
-
- self.globalHitCount += 1
-
- def getGlobalHits(self):
- """
- Return the global hit stats
- """
-
- stats = '<?xml version="1.0" encoding="UTF-8"?><plist version="1.0">'
- stats += "<dict><key>totalHits</key><integer>%d</integer>"
- stats += "<key>recentHits</key><dict>"
- stats += "<key>count</key><integer>%d</integer>"
- stats += "<key>since</key><integer>%d</integer>"
- stats += "<key>period</key><integer>%d</integer>"
- stats += "<key>frequency</key><integer>%d</integer>"
- stats += "</dict></dict></plist>"
- return stats % (
- self.globalHitCount,
- self.globalHitCount - self.globalHitHistory[0]["hits"],
- self.globalHitHistory[0]["time"],
- config.GlobalStatsLoggingPeriod,
- config.GlobalStatsLoggingFrequency
- )
-
-class LogMessage(amp.Command):
- arguments = [("message", amp.String())]
-
-class LogGlobalHit(amp.Command):
- arguments = []
-
-class AMPCommonAccessLoggingObserver(CommonAccessLoggingObserverExtensions):
- def __init__(self, mode, id):
- self.mode = mode
- self.id = id
- self.protocol = None
- self._buffer = []
-
- def flushBuffer(self):
- if self._buffer:
- for msg in self._buffer:
- self.logMessage(msg)
-
- def start(self):
- super(AMPCommonAccessLoggingObserver, self).start()
-
- from twisted.internet import reactor
-
- def _gotProtocol(proto):
- self.protocol = proto
- self.flushBuffer()
-
- self.client = protocol.ClientCreator(reactor, amp.AMP)
- if self.mode == "AF_UNIX":
- d = self.client.connectUNIX(self.id)
- else:
- d = self.client.connectTCP("localhost", self.id)
- d.addCallback(_gotProtocol)
-
- def stop(self):
- super(AMPCommonAccessLoggingObserver, self).stop()
- self.client.disconnect()
-
- def logMessage(self, message):
- """
- Log a message to the remote AMP Protocol
- """
- if self.protocol is not None:
- # XXX: Yeah we're not waiting for anything to happen here.
- # but we will log an error.
- if isinstance(message, unicode):
- message = message.encode("utf-8")
- d = self.protocol.callRemote(LogMessage, message=message)
- d.addErrback(log.err)
- else:
- self._buffer.append(message)
-
- def logGlobalHit(self):
- """
- Log a server hit via the remote AMP Protocol
- """
-
- if self.protocol is not None:
- d = self.protocol.callRemote(LogGlobalHit)
- d.addErrback(log.err)
- else:
- log.msg("logGlobalHit() only works with an AMP Protocol")
-
-class AMPLoggingProtocol(amp.AMP):
- """
- A server side protocol for logging to the given observer.
- """
-
- def __init__(self, observer):
- self.observer = observer
-
- super(AMPLoggingProtocol, self).__init__()
-
- def logMessage(self, message):
- self.observer.logMessage(message)
- return {}
-
- LogMessage.responder(logMessage)
-
- def logGlobalHit(self):
- self.observer.logGlobalHit()
- return {}
-
- LogGlobalHit.responder(logGlobalHit)
-
-class AMPLoggingFactory(protocol.ServerFactory):
- def __init__(self, observer):
- self.observer = observer
-
- def doStart(self):
- self.observer.start()
-
- def doStop(self):
- self.observer.stop()
-
- def buildProtocol(self, addr):
- return AMPLoggingProtocol(self.observer)
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/directory/augment.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/directory/augment.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/directory/augment.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -15,6 +15,9 @@
##
import copy
+import grp
+import os
+import pwd
import time
from twisted.internet.defer import inlineCallbacks, returnValue, succeed
@@ -28,7 +31,6 @@
from twistedcaldav.directory.xmlaugmentsparser import XMLAugmentsParser
from twistedcaldav.xmlutil import newElementTreeWithRoot, addSubElement,\
writeXML, readXML
-import os
log = Logger()
@@ -183,6 +185,22 @@
addSubElement(record, xmlaugmentsparser.ELEMENT_ENABLEADDRESSBOOK, "true")
doDefault = False
writeXML(missedFile, root)
+
+ # Set permissions
+ uid = -1
+ if config.UserName:
+ try:
+ uid = pwd.getpwnam(config.UserName).pw_uid
+ except KeyError:
+ log.error("User not found: %s" % (config.UserName,))
+ gid = -1
+ if config.GroupName:
+ try:
+ gid = grp.getgrnam(config.GroupName).gr_gid
+ except KeyError:
+ log.error("Group not found: %s" % (config.GroupName,))
+ if uid != -1 and gid != -1:
+ os.chown(missedFile, uid, gid)
try:
self.db = self._parseXML()
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/directory/directory.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/directory/directory.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/directory/directory.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -429,7 +429,8 @@
self.enabledForCalendaring = False
else:
- self.enabled = False
+ # Groups are by default always enabled
+ self.enabled = (self.recordType == self.service.recordType_groups)
self.hostedAt = ""
self.enabledForCalendaring = False
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/directory/xmlfile.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/directory/xmlfile.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/directory/xmlfile.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -351,7 +351,9 @@
self._persistRecords(accountsElement)
+ return self.recordWithGUID(guid)
+
def destroyRecord(self, recordType, guid=None):
"""
Remove the record matching guid. In this XML-based implementation,
@@ -407,7 +409,13 @@
self._persistRecords(accountsElement)
+ # Force a cache update - both local and memcached
+ self.queryDirectory([recordType], self.INDEX_TYPE_GUID, guid)
+ for shortName in shortNames:
+ self.queryDirectory([recordType], self.INDEX_TYPE_SHORTNAME, shortName)
+ return self.recordWithGUID(guid)
+
class XMLDirectoryRecord(CachingDirectoryRecord):
"""
XML based implementation implementation of L{IDirectoryRecord}.
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/ical.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/ical.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/ical.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -1320,6 +1320,23 @@
else:
component_rids.add(rid)
+ # Check for mismatch in DTSTART and UNTIL value type
+ # If they're not both date or both date-time, raise error
+ if (subcomponent.hasProperty("DTSTART") and
+ subcomponent.hasProperty("RRULE")):
+ dtType = type(subcomponent.getProperty("DTSTART").value())
+ for rrule in subcomponent.properties("RRULE"):
+ indexedTokens = {}
+ indexedTokens.update([valuePart.split("=")
+ for valuePart in rrule.value().split(";")])
+ until = indexedTokens.get('UNTIL', None)
+ if until:
+ untilType = datetime.date if len(until) == 8 else datetime.datetime
+ if untilType is not dtType:
+ msg = "Calendar resources must have matching type for DTSTART and UNTIL"
+ log.debug(msg)
+ raise InvalidICalendarDataError(msg)
+
timezone_refs.update(subcomponent.timezoneIDs())
#
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/mail.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/mail.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/mail.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -68,7 +68,7 @@
from twistedcaldav.util import AuthorizedHTTPGetter
from twistedcaldav.stdconfig import DEFAULT_CONFIG, DEFAULT_CONFIG_FILE
-from calendarserver.util import getRootResource
+from calendarserver.tap.util import getRootResource
__all__ = [
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/method/delete_common.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/method/delete_common.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/method/delete_common.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -43,7 +43,8 @@
class DeleteResource(object):
- def __init__(self, request, resource, resource_uri, parent, depth, internal_request=False):
+ def __init__(self, request, resource, resource_uri, parent, depth,
+ internal_request=False, allowImplicitSchedule=True):
self.request = request
self.resource = resource
@@ -51,6 +52,7 @@
self.parent = parent
self.depth = depth
self.internal_request = internal_request
+ self.allowImplicitSchedule = allowImplicitSchedule
def validIfScheduleMatch(self):
"""
@@ -139,7 +141,7 @@
scheduler = None
lock = None
- if not self.internal_request:
+ if not self.internal_request and self.allowImplicitSchedule:
# Get data we need for implicit scheduling
calendar = delresource.iCalendar()
scheduler = ImplicitScheduler()
@@ -164,7 +166,7 @@
index.deleteResource(delresource.fp.basename(), newrevision)
# Do scheduling
- if scheduler and not self.internal_request:
+ if scheduler and not self.internal_request and self.allowImplicitSchedule:
yield scheduler.doImplicitScheduling()
except MemcacheLockTimeoutError:
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/scheduling/processing.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/scheduling/processing.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/scheduling/processing.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -432,22 +432,21 @@
lock = MemcacheLock("ImplicitUIDLock", calendar.resourceUID(), timeout=60.0)
try:
- if lock:
- yield lock.acquire()
+ yield lock.acquire()
# Send out a reply
log.debug("ImplicitProcessing - recipient '%s' processing UID: '%s' - auto-reply: %s" % (self.recipient.cuaddr, self.uid, partstat))
from twistedcaldav.scheduling.implicit import ImplicitScheduler
scheduler = ImplicitScheduler()
- scheduler.sendAttendeeReply(self.request, resource, calendar, self.recipient)
+ yield scheduler.sendAttendeeReply(self.request, resource, calendar, self.recipient)
+
except MemcacheLockTimeoutError:
# Just try again to get the lock
reactor.callLater(2.0, self.sendAttendeeAutoReply, *(calendar, resource, partstat))
finally:
- if lock:
- yield lock.clean()
+ yield lock.clean()
@inlineCallbacks
def checkAttendeeAutoReply(self, calendar):
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/stdconfig.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/stdconfig.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/stdconfig.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -53,7 +53,7 @@
DEFAULT_RESOURCE_PARAMS = {
"twistedcaldav.directory.xmlfile.XMLDirectoryService": {
"xmlFile": "resources.xml",
- "cacheTimeout": 30,
+ "cacheTimeout": 1,
"recordTypes" : ("locations", "resources"),
},
}
@@ -489,7 +489,6 @@
# Twisted
"Twisted": {
- "twistd": "/usr/share/caldavd/bin/twistd",
"reactor": "select",
},
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/test/test_icalendar.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/test/test_icalendar.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/twistedcaldav/test/test_icalendar.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -23,7 +23,7 @@
from twisted.trial.unittest import SkipTest
from twistedcaldav.ical import Component, parse_date, parse_datetime,\
- parse_date_or_datetime, parse_duration, Property
+ parse_date_or_datetime, parse_duration, Property, InvalidICalendarDataError
from twistedcaldav.instance import InvalidOverriddenInstanceError
import twistedcaldav.test.util
@@ -3123,3 +3123,100 @@
self.assertEqual(str(ical1), str(ical2), "Failed comparison: %s\n%s" % (title, diff,))
elif changed:
self.fail("Truncation happened when not expected: %s" % (title,))
+
+ def test_mismatched_until(self):
+ invalid = (
+ """BEGIN:VCALENDAR
+VERSION:2.0
+CALSCALE:GREGORIAN
+PRODID:-//Apple Inc.//iCal 3.0//EN
+BEGIN:VTIMEZONE
+TZID:US/Pacific
+BEGIN:STANDARD
+DTSTART:20071104T020000
+RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
+TZNAME:PST
+TZOFFSETFROM:-0700
+TZOFFSETTO:-0800
+END:STANDARD
+BEGIN:DAYLIGHT
+DTSTART:20070311T020000
+RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
+TZNAME:PDT
+TZOFFSETFROM:-0800
+TZOFFSETTO:-0700
+END:DAYLIGHT
+END:VTIMEZONE
+BEGIN:VEVENT
+UID:FB81D520-ED27-4DBA-8894-45B7612A7621
+DTSTART;TZID=US/Pacific:20090705T100000
+DTEND;TZID=US/Pacific:20090730T103000
+CREATED:20090604T225706Z
+DTSTAMP:20090604T230500Z
+RRULE:FREQ=DAILY;INTERVAL=1;UNTIL=20090706
+SEQUENCE:1
+SUMMARY:TEST
+TRANSP:OPAQUE
+END:VEVENT
+BEGIN:VEVENT
+UID:FB81D520-ED27-4DBA-8894-45B7612A7621
+RECURRENCE-ID;TZID=US/Pacific:20090705T100000
+DTSTART;TZID=US/Pacific:20090705T114500
+DTEND;TZID=US/Pacific:20090705T121500
+CREATED:20090604T225706Z
+DTSTAMP:20090604T230504Z
+SEQUENCE:2
+SUMMARY:TEST
+TRANSP:OPAQUE
+END:VEVENT
+END:VCALENDAR
+""",
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//Apple Inc.//iCal 4.0.2//EN
+CALSCALE:GREGORIAN
+BEGIN:VEVENT
+CREATED:20100311T234221Z
+UID:D0151FAD-4739-4B61-96EB-9289FF1F7716
+DTEND;VALUE=DATE:20100316
+RRULE:FREQ=WEEKLY;INTERVAL=1;UNTIL=20110604T225706Z
+TRANSP:TRANSPARENT
+SUMMARY:ALL DAY
+DTSTART;VALUE=DATE:20100315
+DTSTAMP:20100312T002640Z
+SEQUENCE:5
+END:VEVENT
+END:VCALENDAR
+""",
+ )
+
+ valid = (
+ """BEGIN:VCALENDAR
+VERSION:2.0
+PRODID:-//Apple Inc.//iCal 4.0.2//EN
+CALSCALE:GREGORIAN
+BEGIN:VEVENT
+CREATED:20100311T234221Z
+UID:D0151FAD-4739-4B61-96EB-9289FF1F7716
+DTEND;VALUE=DATE:20100316
+RRULE:FREQ=WEEKLY;INTERVAL=1;UNTIL=20110316
+TRANSP:TRANSPARENT
+SUMMARY:ALL DAY
+DTSTART;VALUE=DATE:20100315
+DTSTAMP:20100312T002640Z
+SEQUENCE:5
+END:VEVENT
+END:VCALENDAR
+""",
+ )
+
+
+ for text in invalid:
+ calendar = Component.fromString(text)
+ self.assertRaises(InvalidICalendarDataError, calendar.validateForCalDAV)
+ for text in valid:
+ calendar = Component.fromString(text)
+ try:
+ calendar.validateForCalDAV()
+ except:
+ self.fail("Valid calendar should validate")
Modified: CalendarServer/branches/users/cdaboo/shared-calendars-5187/txcaldav/icalendarstore.py
===================================================================
--- CalendarServer/branches/users/cdaboo/shared-calendars-5187/txcaldav/icalendarstore.py 2010-03-12 20:20:04 UTC (rev 5297)
+++ CalendarServer/branches/users/cdaboo/shared-calendars-5187/txcaldav/icalendarstore.py 2010-03-12 20:35:30 UTC (rev 5298)
@@ -50,6 +50,7 @@
from twext.python.vcomponent import VComponent
from txdav.idav import IPropertyStore
+
#
# Exceptions
#
@@ -119,6 +120,7 @@
Uh, oh.
"""
+
#
# Interfaces
#
@@ -126,6 +128,11 @@
class ICalendarHome(Interface):
"""
Calendar home
+
+ A calendar home belongs to a specific principal and contains the
+ calendars which that principal has direct access to. This
+ includes both calendars owned by the principal as well as
+ calendars that have been shared with and accepts by the principal.
"""
def uid():
"""
@@ -178,9 +185,15 @@
@return: an L{IPropertyStore}.
"""
+
class ICalendar(Interface):
"""
Calendar
+
+ A calendar is a container for calendar objects (events, to-dos,
+ etc.). A calendar belongs to a specific principal but may be
+ shared with other principals, granting them read-only or
+ read/write access.
"""
def ownerCalendarHome():
"""
@@ -292,9 +305,13 @@
@return: an L{IPropertyStore}.
"""
+
class ICalendarObject(Interface):
"""
- Calendar object (event, to-do, etc.).
+ Calendar object
+
+ A calendar object decribes an event, to-do, or other iCalendar
+ object.
"""
def setComponent(component):
"""
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/calendarserver-changes/attachments/20100312/80909aca/attachment-0001.html>
More information about the calendarserver-changes
mailing list