<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[12892] CalendarServer/trunk</title>
</head>
<body>

<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt;  }
#msg dl a { font-weight: bold}
#msg dl a:link    { color:#fc3; }
#msg dl a:active  { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff  {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.calendarserver.org//changeset/12892">12892</a></dd>
<dt>Author</dt> <dd>cdaboo@apple.com</dd>
<dt>Date</dt> <dd>2014-03-13 11:09:35 -0700 (Thu, 13 Mar 2014)</dd>
</dl>

<h3>Log Message</h3>
<pre>Rework stats socket into a bi-directional request/response protocol. Turn the job queue viewer into a more generic
&quot;dashboard&quot; tool that can make use of the new protocol. Add master/child slots as another type of data to view in
the dashboard.</pre>

<h3>Modified Paths</h3>
<ul>
<li><a href="#CalendarServertrunkcalendarserveraccesslogpy">CalendarServer/trunk/calendarserver/accesslog.py</a></li>
<li><a href="#CalendarServertrunkcalendarservertapcaldavpy">CalendarServer/trunk/calendarserver/tap/caldav.py</a></li>
<li><a href="#CalendarServertrunkcontribtoolsreadStatspy">CalendarServer/trunk/contrib/tools/readStats.py</a></li>
<li><a href="#CalendarServertrunktxweb2metafdpy">CalendarServer/trunk/txweb2/metafd.py</a></li>
</ul>

<h3>Added Paths</h3>
<ul>
<li><a href="#CalendarServertrunkbincalendarserver_dashboard">CalendarServer/trunk/bin/calendarserver_dashboard</a></li>
<li><a href="#CalendarServertrunkcalendarserverdashboard_servicepy">CalendarServer/trunk/calendarserver/dashboard_service.py</a></li>
<li><a href="#CalendarServertrunkcalendarservertoolsdashboardpy">CalendarServer/trunk/calendarserver/tools/dashboard.py</a></li>
</ul>

<h3>Removed Paths</h3>
<ul>
<li><a href="#CalendarServertrunkbincalendarserver_monitor_jobs">CalendarServer/trunk/bin/calendarserver_monitor_jobs</a></li>
<li><a href="#CalendarServertrunkcalendarservertoolsjobitemspy">CalendarServer/trunk/calendarserver/tools/jobitems.py</a></li>
</ul>

</div>
<div id="patch">
<h3>Diff</h3>
<a id="CalendarServertrunkbincalendarserver_dashboard"></a>
<div class="addfile"><h4>Added: CalendarServer/trunk/bin/calendarserver_dashboard (0 => 12892)</h4>
<pre class="diff"><span>
<span class="info">--- CalendarServer/trunk/bin/calendarserver_dashboard                                (rev 0)
+++ CalendarServer/trunk/bin/calendarserver_dashboard        2014-03-13 18:09:35 UTC (rev 12892)
</span><span class="lines">@@ -0,0 +1,23 @@
</span><ins>+#!/usr/bin/env python
+
+##
+# Copyright (c) 2006-2014 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the &quot;License&quot;);
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an &quot;AS IS&quot; BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+# This tool can run standalone, so does not need any special PYTHONPATH or preamble handling
+
+if __name__ == &quot;__main__&quot;:
+    from calendarserver.tools.dashboard import main
+    main()
</ins><span class="cx">Property changes on: CalendarServer/trunk/bin/calendarserver_dashboard
</span><span class="cx">___________________________________________________________________
</span></span></pre></div>
<a id="svnexecutable"></a>
<div class="addfile"><h4>Added: svn:executable</h4></div>
<a id="CalendarServertrunkbincalendarserver_monitor_jobs"></a>
<div class="delfile"><h4>Deleted: CalendarServer/trunk/bin/calendarserver_monitor_jobs (12891 => 12892)</h4>
<pre class="diff"><span>
<span class="info">--- CalendarServer/trunk/bin/calendarserver_monitor_jobs        2014-03-13 00:51:31 UTC (rev 12891)
+++ CalendarServer/trunk/bin/calendarserver_monitor_jobs        2014-03-13 18:09:35 UTC (rev 12892)
</span><span class="lines">@@ -1,38 +0,0 @@
</span><del>-#!/usr/bin/env python
-
-##
-# Copyright (c) 2006-2014 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the &quot;License&quot;);
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an &quot;AS IS&quot; BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
-
-import os
-import sys
-
-# In OS X Server context, add to PATH to find Postgres utilities (initdb, pg_ctl)
-if &quot;Server.app&quot; in sys.argv[0]:
-    os.environ[&quot;PATH&quot;] += &quot;:&quot; + os.path.join(os.path.dirname(os.path.dirname(sys.argv[0])), &quot;bin&quot;)
-
-#PYTHONPATH
-
-if __name__ == &quot;__main__&quot;:
-    if &quot;PYTHONPATH&quot; in globals():
-        sys.path.insert(0, PYTHONPATH)
-    else:
-        try:
-            import _calendarserver_preamble
-        except ImportError:
-            sys.exc_clear()
-
-    from calendarserver.tools.jobitems import main
-    main()
</del></span></pre></div>
<a id="CalendarServertrunkcalendarserveraccesslogpy"></a>
<div class="modfile"><h4>Modified: CalendarServer/trunk/calendarserver/accesslog.py (12891 => 12892)</h4>
<pre class="diff"><span>
<span class="info">--- CalendarServer/trunk/calendarserver/accesslog.py        2014-03-13 00:51:31 UTC (rev 12891)
+++ CalendarServer/trunk/calendarserver/accesslog.py        2014-03-13 18:09:35 UTC (rev 12892)
</span><span class="lines">@@ -428,13 +428,13 @@
</span><span class="cx">             self.mergeStats(oneHour, stat)
</span><span class="cx"> 
</span><span class="cx">         printStats = {
</span><del>-            &quot;System&quot;: self.systemStats.items,
-            &quot;Current&quot;: currentStats,
-            &quot;1 Minute&quot;: previousMinute,
-            &quot;5 Minutes&quot;: fiveMinutes,
-            &quot;1 Hour&quot;: oneHour,
</del><ins>+            &quot;system&quot;: self.systemStats.items,
+            &quot;current&quot;: currentStats,
+            &quot;1m&quot;: previousMinute,
+            &quot;5m&quot;: fiveMinutes,
+            &quot;1h&quot;: oneHour,
</ins><span class="cx">         }
</span><del>-        return json.dumps(printStats)
</del><ins>+        return printStats
</ins><span class="cx"> 
</span><span class="cx"> 
</span><span class="cx">     def ensureSequentialStats(self):
</span></span></pre></div>
<a id="CalendarServertrunkcalendarserverdashboard_servicepy"></a>
<div class="addfile"><h4>Added: CalendarServer/trunk/calendarserver/dashboard_service.py (0 => 12892)</h4>
<pre class="diff"><span>
<span class="info">--- CalendarServer/trunk/calendarserver/dashboard_service.py                                (rev 0)
+++ CalendarServer/trunk/calendarserver/dashboard_service.py        2014-03-13 18:09:35 UTC (rev 12892)
</span><span class="lines">@@ -0,0 +1,144 @@
</span><ins>+##
+# Copyright (c) 2012-2014 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the &quot;License&quot;);
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an &quot;AS IS&quot; BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+from twext.enterprise.jobqueue import JobItem
+
+from twisted.internet.defer import inlineCallbacks, succeed, returnValue
+from twisted.internet.protocol import Factory
+from twisted.protocols.basic import LineReceiver
+
+import json
+
+&quot;&quot;&quot;
+Protocol and other items that enable introspection of the server. This will include
+server protocol analysis statistics, job queue load, and other useful information that
+a server admin or developer would like to keep an eye on.
+&quot;&quot;&quot;
+
+class DashboardProtocol (LineReceiver):
+    &quot;&quot;&quot;
+    A protocol that receives a line containing a JSON object representing a request,
+    and returns a line containing a JSON object as a response.
+    &quot;&quot;&quot;
+
+    unknown_cmd = json.dumps({&quot;result&quot;: &quot;unknown command&quot;})
+    bad_cmd = json.dumps({&quot;result&quot;: &quot;bad command&quot;})
+
+    def lineReceived(self, line):
+        &quot;&quot;&quot;
+        Process a request which is expected to be a JSON object.
+
+        @param line: The line which was received with the delimiter removed.
+        @type line: C{bytes}
+        &quot;&quot;&quot;
+
+        # Look for exit
+        if line in (&quot;exit&quot;, &quot;quit&quot;):
+            self.sendLine(&quot;Done&quot;)
+            self.transport.loseConnection()
+            return
+
+        def _write(result):
+            self.sendLine(result)
+
+        try:
+            j = json.loads(line)
+            if isinstance(j, list):
+                self.process_data(j)
+        except (ValueError, KeyError):
+            _write(self.bad_cmd)
+
+
+    @inlineCallbacks
+    def process_data(self, j):
+        results = {}
+        for data in j:
+            if hasattr(self, &quot;data_{}&quot;.format(data)):
+                result = yield getattr(self, &quot;data_{}&quot;.format(data))()
+            elif data.startswith(&quot;stats_&quot;):
+                result = yield self.data_stats()
+                result = result.get(data[6:], &quot;&quot;)
+            else:
+                result = &quot;&quot;
+            results[data] = result
+
+        self.sendLine(json.dumps(results))
+
+
+    def data_stats(self):
+        &quot;&quot;&quot;
+        Return the logging protocol statistics.
+
+        @return: a string containing the JSON result.
+        @rtype: L{str}
+        &quot;&quot;&quot;
+        return succeed(self.factory.logger.observer.getStats())
+
+
+    def data_slots(self):
+        &quot;&quot;&quot;
+        Return the logging protocol statistics.
+
+        @return: a string containing the JSON result.
+        @rtype: L{str}
+        &quot;&quot;&quot;
+        if self.factory.limiter is None:
+            raise ValueError()
+        states = tuple(self.factory.limiter.dispatcher.slavestates)
+        results = []
+        for num, status in states:
+            result = {&quot;slot&quot;: num}
+            result.update(status.items())
+            results.append(result)
+        return succeed({&quot;slots&quot;: results, &quot;overloaded&quot;: self.factory.limiter.overloaded})
+
+
+    def data_jobcount(self):
+        &quot;&quot;&quot;
+        Return a count of job types.
+
+        @return: the JSON result.
+        @rtype: L{int}
+        &quot;&quot;&quot;
+
+        return succeed(JobItem.numberOfWorkTypes())
+
+
+    @inlineCallbacks
+    def data_jobs(self):
+        &quot;&quot;&quot;
+        Return a summary of the job queue.
+
+        @return: a string containing the JSON result.
+        @rtype: L{str}
+        &quot;&quot;&quot;
+
+        txn = self.factory.store.newTransaction()
+        records = (yield JobItem.histogram(txn))
+        yield txn.commit()
+
+        returnValue(records)
+
+
+
+class DashboardServer(Factory):
+
+    protocol = DashboardProtocol
+
+    def __init__(self, logObserver, limiter):
+        self.logger = logObserver
+        self.limiter = limiter
+        self.store = None
</ins></span></pre></div>
<a id="CalendarServertrunkcalendarservertapcaldavpy"></a>
<div class="modfile"><h4>Modified: CalendarServer/trunk/calendarserver/tap/caldav.py (12891 => 12892)</h4>
<pre class="diff"><span>
<span class="info">--- CalendarServer/trunk/calendarserver/tap/caldav.py        2014-03-13 00:51:31 UTC (rev 12891)
+++ CalendarServer/trunk/calendarserver/tap/caldav.py        2014-03-13 18:09:35 UTC (rev 12892)
</span><span class="lines">@@ -46,7 +46,7 @@
</span><span class="cx"> from twisted.internet.endpoints import UNIXClientEndpoint, TCP4ClientEndpoint
</span><span class="cx"> from twisted.internet.process import ProcessExitedAlready
</span><span class="cx"> from twisted.internet.protocol import ProcessProtocol
</span><del>-from twisted.internet.protocol import Protocol, Factory
</del><ins>+from twisted.internet.protocol import Factory
</ins><span class="cx"> from twisted.plugin import IPlugin
</span><span class="cx"> from twisted.protocols.amp import AMP
</span><span class="cx"> from twisted.python.log import FileLogObserver, ILogObserver
</span><span class="lines">@@ -110,6 +110,7 @@
</span><span class="cx"> from calendarserver.accesslog import RotatingFileAccessLoggingObserver
</span><span class="cx"> from calendarserver.controlsocket import ControlSocket
</span><span class="cx"> from calendarserver.controlsocket import ControlSocketConnectingService
</span><ins>+from calendarserver.dashboard_service import DashboardServer
</ins><span class="cx"> from calendarserver.push.amppush import AMPPushMaster, AMPPushForwarder
</span><span class="cx"> from calendarserver.push.applepush import ApplePushNotifierService
</span><span class="cx"> from calendarserver.push.notifier import PushDistributor
</span><span class="lines">@@ -216,24 +217,6 @@
</span><span class="cx"> 
</span><span class="cx"> 
</span><span class="cx"> 
</span><del>-class CalDAVStatisticsProtocol (Protocol):
-
-    def connectionMade(self):
-        stats = self.factory.logger.observer.getStats()
-        self.transport.write(&quot;{}\r\n&quot;.format(stats))
-        self.transport.loseConnection()
-
-
-
-class CalDAVStatisticsServer (Factory):
-
-    protocol = CalDAVStatisticsProtocol
-
-    def __init__(self, logObserver):
-        self.logger = logObserver
-
-
-
</del><span class="cx"> class ErrorLoggingMultiService(MultiService, object):
</span><span class="cx">     &quot;&quot;&quot; Registers a rotating file logger for error logging, if
</span><span class="cx">         config.ErrorLogEnabled is True. &quot;&quot;&quot;
</span><span class="lines">@@ -1834,14 +1817,14 @@
</span><span class="cx">         # Start listening on the stats socket, for administrators to inspect
</span><span class="cx">         # the current stats on the server.
</span><span class="cx">         if config.Stats.EnableUnixStatsSocket:
</span><del>-            stats = CalDAVStatisticsServer(logger)
</del><ins>+            stats = DashboardServer(logger, cl if config.UseMetaFD else None)
</ins><span class="cx">             statsService = GroupOwnedUNIXServer(
</span><span class="cx">                 gid, config.Stats.UnixStatsSocket, stats, mode=0660
</span><span class="cx">             )
</span><span class="cx">             statsService.setName(&quot;unix-stats&quot;)
</span><span class="cx">             statsService.setServiceParent(s)
</span><span class="cx">         if config.Stats.EnableTCPStatsSocket:
</span><del>-            stats = CalDAVStatisticsServer(logger)
</del><ins>+            stats = DashboardServer(logger, cl if config.UseMetaFD else None)
</ins><span class="cx">             statsService = TCPServer(
</span><span class="cx">                 config.Stats.TCPStatsPort, stats, interface=&quot;&quot;
</span><span class="cx">             )
</span><span class="lines">@@ -1888,6 +1871,8 @@
</span><span class="cx">             if store is None:
</span><span class="cx">                 raise StoreNotAvailable()
</span><span class="cx"> 
</span><ins>+            stats.store = store
+
</ins><span class="cx">             from twisted.internet import reactor
</span><span class="cx">             pool = PeerConnectionPool(
</span><span class="cx">                 reactor, store.newTransaction, config.WorkQueue.ampPort
</span></span></pre></div>
<a id="CalendarServertrunkcalendarservertoolsdashboardpy"></a>
<div class="addfile"><h4>Added: CalendarServer/trunk/calendarserver/tools/dashboard.py (0 => 12892)</h4>
<pre class="diff"><span>
<span class="info">--- CalendarServer/trunk/calendarserver/tools/dashboard.py                                (rev 0)
+++ CalendarServer/trunk/calendarserver/tools/dashboard.py        2014-03-13 18:09:35 UTC (rev 12892)
</span><span class="lines">@@ -0,0 +1,538 @@
</span><ins>+##
+# Copyright (c) 2012-2014 Apple Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the &quot;License&quot;);
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an &quot;AS IS&quot; BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+##
+
+&quot;&quot;&quot;
+A curses (or plain text) based dashboard for viewing various aspects of the server as exposed by
+the L{DashboardProtocol} stats socket.
+&quot;&quot;&quot;
+
+from getopt import getopt, GetoptError
+
+import curses
+import json
+import os
+import sched
+import sys
+import time
+import socket
+import errno
+
+def usage(e=None):
+
+    name = os.path.basename(sys.argv[0])
+    print(&quot;usage: %s [options]&quot; % (name,))
+    print(&quot;&quot;)
+    print(&quot;  TODO: describe usage&quot;)
+    print(&quot;&quot;)
+    print(&quot;options:&quot;)
+    print(&quot;  -h --help: print this help and exit&quot;)
+    print(&quot;  -t: text output, not curses&quot;)
+    print(&quot;&quot;)
+
+    if e:
+        sys.exit(64)
+    else:
+        sys.exit(0)
+
+
+BOX_WIDTH = 52
+
+def main():
+
+    try:
+        (optargs, _ignore_args) = getopt(
+            sys.argv[1:], &quot;ht&quot;, [
+                &quot;help&quot;,
+            ],
+        )
+    except GetoptError, e:
+        usage(e)
+
+    #
+    # Get configuration
+    #
+    useCurses = True
+
+    for opt, _ignore_arg in optargs:
+        if opt in (&quot;-h&quot;, &quot;--help&quot;):
+            usage()
+
+        elif opt in (&quot;-t&quot;):
+            useCurses = False
+
+        else:
+            raise NotImplementedError(opt)
+
+    if useCurses:
+        def _wrapped(stdscrn):
+            curses.curs_set(0)
+            curses.use_default_colors()
+            d = Dashboard(stdscrn, True)
+            d.run()
+        curses.wrapper(_wrapped)
+    else:
+        d = Dashboard(None, False)
+        d.run()
+
+
+
+class Dashboard(object):
+    &quot;&quot;&quot;
+    Main dashboard controller. Use Python's L{sched} feature to schedule updates.
+    &quot;&quot;&quot;
+
+    screen = None
+    registered_windows = {}
+
+    def __init__(self, screen, usesCurses):
+        self.screen = screen
+        self.usesCurses = usesCurses
+        self.paused = False
+        self.seconds = 0.1
+        self.sched = sched.scheduler(time.time, time.sleep)
+
+
+    @classmethod
+    def registerWindow(cls, wtype, keypress):
+        &quot;&quot;&quot;
+        Register a window type along with a key press action. This allows the controller to
+        select the appropriate window when its key is pressed, and also provides help information
+        to the L{HelpWindow} for each available window type.
+        &quot;&quot;&quot;
+        cls.registered_windows[keypress] = wtype
+
+
+    def run(self):
+        &quot;&quot;&quot;
+        Create the initial window and run the L{scheduler}.
+        &quot;&quot;&quot;
+        self.windows = []
+        self.displayWindow(None)
+        self.sched.enter(0, 0, self.updateDisplay, ())
+        self.sched.run()
+
+
+    def displayWindow(self, wtype):
+        &quot;&quot;&quot;
+        Display a new window type, clearing out the old one first.
+        &quot;&quot;&quot;
+        if self.windows:
+            for window in self.windows:
+                window.clear()
+            self.windows = []
+
+        if wtype is not None:
+            self.windows.append(wtype(self.usesCurses).makeWindow())
+            self.windows[-1].update()
+        else:
+            top = 0
+            for wtype in filter(lambda x: x.all, self.registered_windows.values()):
+                self.windows.append(wtype(self.usesCurses).makeWindow(top=top))
+                self.windows[-1].update()
+                top += self.windows[-1].nlines
+
+
+    def updateDisplay(self):
+        &quot;&quot;&quot;
+        Periodic update of the current window and check for a key press.
+        &quot;&quot;&quot;
+        try:
+            if not self.paused:
+                for window in filter(lambda x: x.requiresUpdate(), self.windows):
+                    window.update()
+        except Exception as e:
+            print(str(e))
+        if not self.usesCurses:
+            print(&quot;-------------&quot;)
+
+        # Check keystrokes
+        if self.usesCurses:
+            try:
+                c = self.windows[-1].window.getkey()
+            except:
+                c = -1
+            if c == &quot;q&quot;:
+                sys.exit(0)
+            elif c == &quot; &quot;:
+                self.paused = not self.paused
+            elif c == &quot;t&quot;:
+                self.seconds = 1.0 if self.seconds == 0.1 else 0.1
+            elif c == &quot;a&quot;:
+                self.displayWindow(None)
+            elif c in self.registered_windows:
+                self.displayWindow(self.registered_windows[c])
+
+        self.sched.enter(self.seconds, 0, self.updateDisplay, ())
+
+
+
+class BaseWindow(object):
+    &quot;&quot;&quot;
+    Common behavior for window types.
+    &quot;&quot;&quot;
+
+    help = &quot;Not Implemented&quot;
+    all = True
+
+    def __init__(self, usesCurses):
+        self.usesCurses = usesCurses
+
+
+    def makeWindow(self, top=0, left=0):
+        raise NotImplementedError
+
+
+    def _createWindow(self, title, nlines, ncols=BOX_WIDTH, begin_y=0, begin_x=0):
+        &quot;&quot;&quot;
+        Initialize a curses window based on the sizes required.
+        &quot;&quot;&quot;
+        if self.usesCurses:
+            self.window = curses.newwin(nlines, ncols, begin_y, begin_x)
+            self.window.nodelay(1)
+        else:
+            self.window = None
+        self.title = title
+        self.nlines = nlines
+        self.ncols = ncols
+        self.iter = 0
+        self.lastResult = {}
+
+
+    def requiresUpdate(self):
+        &quot;&quot;&quot;
+        Indicates whether a window type has dynamic data that should be refreshed on each update,
+        or whether it is static data (e.g., L{HelpWindow}) that only needs to be drawn once.
+        &quot;&quot;&quot;
+        return True
+
+
+    def clear(self):
+        &quot;&quot;&quot;
+        Clear any drawing done by the current window type.
+        &quot;&quot;&quot;
+        if self.usesCurses:
+            self.window.erase()
+            self.window.refresh()
+
+
+    def update(self):
+        &quot;&quot;&quot;
+        Periodic window update - redraw the window.
+        &quot;&quot;&quot;
+        raise NotImplementedError
+
+
+
+class BaseSocketWindow(BaseWindow):
+    &quot;&quot;&quot;
+    Common behavior for a window that reads from the server's stats socket.
+    &quot;&quot;&quot;
+
+    def __init__(self, usesCurses):
+        super(BaseSocketWindow, self).__init__(usesCurses)
+        self.socket = None
+        self.sockname = (&quot;localhost&quot;, 8100)
+        self.useTCP = True
+
+
+    def readSock(self, item):
+        &quot;&quot;&quot;
+        Open a socket, send the specified request, and retrieve the response. Keep the socket open.
+        &quot;&quot;&quot;
+        try:
+            if self.socket is None:
+                self.socket = socket.socket(socket.AF_INET if self.useTCP else socket.AF_UNIX, socket.SOCK_STREAM)
+                self.socket.connect(self.sockname)
+                self.socket.setblocking(0)
+            self.socket.sendall(json.dumps([item]) + &quot;\r\n&quot;)
+            data = &quot;&quot;
+            while not data.endswith(&quot;\n&quot;):
+                try:
+                    d = self.socket.recv(1024)
+                except socket.error as se:
+                    if se.args[0] != errno.EWOULDBLOCK:
+                        raise
+                    continue
+                if d:
+                    data += d
+                else:
+                    break
+            data = json.loads(data)[item]
+        except socket.error as e:
+            data = {&quot;Failed&quot;: &quot;Unable to read statistics from server: %s %s&quot; % (self.sockname, e)}
+            self.socket = None
+        return data
+
+
+
+class HelpWindow(BaseWindow):
+    &quot;&quot;&quot;
+    Display help for the dashboard.
+    &quot;&quot;&quot;
+
+    help = &quot;display dashboard help&quot;
+    all = False
+
+    def __init__(self, usesCurses):
+        super(HelpWindow, self).__init__(usesCurses)
+        self.help = (
+            &quot;&quot;,
+            &quot;a - all windows&quot;,
+            &quot;  - (space) pause dashboard polling&quot;,
+            &quot;t - toggle update between 0.1 and 1.0 seconds&quot;,
+            &quot;&quot;,
+            &quot;q - exit the dashboard&quot;,
+        )
+
+
+    def makeWindow(self, top=0, left=0):
+        self._createWindow(&quot;Help&quot;, len(self.help) + len(Dashboard.registered_windows) + 2, begin_y=top, begin_x=left)
+        return self
+
+
+    def requiresUpdate(self):
+        return False
+
+
+    def update(self):
+
+        if self.usesCurses:
+            self.window.erase()
+            self.window.border()
+            self.window.addstr(0, 2, &quot;Help for Dashboard&quot;)
+
+        x = 1
+        y = 1
+
+        items = []
+        for keypress, wtype in sorted(Dashboard.registered_windows.items(), key=lambda x: x[0]):
+            items.append(&quot;{} - {}&quot;.format(keypress, wtype.help))
+        items.extend(self.help)
+        for item in items:
+            if self.usesCurses:
+                self.window.addstr(y, x, item)
+            else:
+                print(item)
+            y += 1
+
+        if self.usesCurses:
+            self.window.refresh()
+
+
+
+class WorkWindow(BaseSocketWindow):
+    &quot;&quot;&quot;
+    Display the status of the server's job queue.
+    &quot;&quot;&quot;
+
+    help = &quot;display server jobs&quot;
+
+    def makeWindow(self, top=0, left=0):
+        nlines = self.readSock(&quot;jobcount&quot;)
+        self._createWindow(&quot;Jobs&quot;, nlines + 5, begin_y=top, begin_x=left)
+        return self
+
+
+    def update(self):
+        records = self.readSock(&quot;jobs&quot;)
+        self.iter += 1
+
+        if self.usesCurses:
+            self.window.erase()
+            self.window.border()
+            self.window.addstr(0, 2, self.title + &quot; {} ({})&quot;.format(len(records), self.iter,))
+
+        x = 1
+        y = 1
+        s = &quot; {:&lt;40}{:&gt;8} &quot;.format(&quot;Work Type&quot;, &quot;Count&quot;)
+        if self.usesCurses:
+            self.window.addstr(y, x, s, curses.A_REVERSE)
+        else:
+            print(s)
+        y += 1
+        for work_type, count in sorted(records.items(), key=lambda x: x[0]):
+            changed = work_type in self.lastResult and self.lastResult[work_type] != count
+            s = &quot;{}{:&lt;40}{:&gt;8} &quot;.format(&quot;&gt;&quot; if count else &quot; &quot;, work_type, count)
+            try:
+                if self.usesCurses:
+                    self.window.addstr(y, x, s, curses.A_REVERSE if changed else (curses.A_BOLD if count else curses.A_NORMAL))
+                else:
+                    print(s)
+            except curses.error:
+                pass
+            y += 1
+
+        s = &quot; {:&lt;40}{:&gt;8} &quot;.format(&quot;Total:&quot;, sum(records.values()))
+        if self.usesCurses:
+            self.window.hline(y, x, &quot;-&quot;, BOX_WIDTH - 2)
+            y += 1
+            self.window.addstr(y, x, s)
+        else:
+            print(s)
+        y += 1
+
+        if self.usesCurses:
+            self.window.refresh()
+
+        self.lastResult = records
+
+
+
+class SlotsWindow(BaseSocketWindow):
+    &quot;&quot;&quot;
+    Displays the status of the server's master process worker slave slots.
+    &quot;&quot;&quot;
+
+    help = &quot;display server child slots&quot;
+    FORMAT_WIDTH = 72
+
+    def makeWindow(self, top=0, left=0):
+        slots = self.readSock(&quot;slots&quot;)[&quot;slots&quot;]
+        self._createWindow(&quot;Slots&quot;, len(slots) + 5, self.FORMAT_WIDTH, begin_y=top, begin_x=left)
+        return self
+
+
+    def update(self):
+        data = self.readSock(&quot;slots&quot;)
+        records = data[&quot;slots&quot;]
+        self.iter += 1
+
+        if self.usesCurses:
+            self.window.erase()
+            self.window.border()
+            self.window.addstr(0, 2, self.title + &quot; {} ({})&quot;.format(len(records), self.iter,))
+
+        x = 1
+        y = 1
+        s = &quot; {:&gt;4}{:&gt;8}{:&gt;8}{:&gt;8}{:&gt;8}{:&gt;8}{:&gt;8}{:&gt;8}{:&gt;8} &quot;.format(&quot;Slot&quot;, &quot;unack&quot;, &quot;ack&quot;, &quot;uncls&quot;, &quot;total&quot;, &quot;start&quot;, &quot;strting&quot;, &quot;stopped&quot;, &quot;abd&quot;)
+        if self.usesCurses:
+            self.window.addstr(y, x, s, curses.A_REVERSE)
+        else:
+            print(s)
+        y += 1
+        for record in sorted(records, key=lambda x: x[&quot;slot&quot;]):
+            changed = record[&quot;slot&quot;] in self.lastResult and self.lastResult[record[&quot;slot&quot;]] != record
+            s = &quot; {:&gt;4}{:&gt;8}{:&gt;8}{:&gt;8}{:&gt;8}{:&gt;8}{:&gt;8}{:&gt;8}{:&gt;8} &quot;.format(
+                record[&quot;slot&quot;],
+                record[&quot;unacknowledged&quot;],
+                record[&quot;acknowledged&quot;],
+                record[&quot;unclosed&quot;],
+                record[&quot;total&quot;],
+                record[&quot;started&quot;],
+                record[&quot;starting&quot;],
+                record[&quot;stopped&quot;],
+                record[&quot;abandoned&quot;],
+            )
+            try:
+                count = record[&quot;unacknowledged&quot;] + record[&quot;acknowledged&quot;]
+                if self.usesCurses:
+                    self.window.addstr(y, x, s, curses.A_REVERSE if changed else (curses.A_BOLD if count else curses.A_NORMAL))
+                else:
+                    print(s)
+            except curses.error:
+                pass
+            y += 1
+
+        s = &quot; {:&lt;12}{:&gt;8}{:&gt;16}&quot;.format(
+            &quot;Total:&quot;,
+            sum([record[&quot;unacknowledged&quot;] + record[&quot;acknowledged&quot;] for record in records]),
+            sum([record[&quot;total&quot;] for record in records]),
+        )
+        if self.usesCurses:
+            self.window.hline(y, x, &quot;-&quot;, self.FORMAT_WIDTH - 2)
+            y += 1
+            self.window.addstr(y, x, s)
+            x += len(s) + 4
+            s = &quot;{:&gt;10}&quot;.format(&quot;OVERLOADED&quot; if data[&quot;overloaded&quot;] else &quot;&quot;)
+            self.window.addstr(y, x, s, curses.A_REVERSE if data[&quot;overloaded&quot;] else curses.A_NORMAL)
+        else:
+            if data[&quot;overloaded&quot;]:
+                s += &quot;    OVERLOADED&quot;
+            print(s)
+        y += 1
+
+        if self.usesCurses:
+            self.window.refresh()
+
+        self.lastResult = records
+
+
+
+class SystemWindow(BaseSocketWindow):
+    &quot;&quot;&quot;
+    Displays the system information provided by the server.
+    &quot;&quot;&quot;
+
+    help = &quot;display system details&quot;
+
+    def makeWindow(self, top=0, left=0):
+        slots = self.readSock(&quot;stats&quot;)[&quot;system&quot;]
+        self._createWindow(&quot;System&quot;, len(slots) + 3, begin_y=top, begin_x=left)
+        return self
+
+
+    def update(self):
+        records = self.readSock(&quot;stats&quot;)[&quot;system&quot;]
+        self.iter += 1
+
+        if self.usesCurses:
+            self.window.erase()
+            self.window.border()
+            self.window.addstr(0, 2, self.title + &quot; {} ({})&quot;.format(len(records), self.iter,))
+
+        x = 1
+        y = 1
+        s = &quot; {:&lt;30}{:&gt;18} &quot;.format(&quot;Item&quot;, &quot;Value&quot;)
+        if self.usesCurses:
+            self.window.addstr(y, x, s, curses.A_REVERSE)
+        else:
+            print(s)
+        y += 1
+
+        records[&quot;cpu use&quot;] = &quot;{:.2f}&quot;.format(records[&quot;cpu use&quot;])
+        records[&quot;memory percent&quot;] = &quot;{:.1f}&quot;.format(records[&quot;memory percent&quot;])
+        records[&quot;memory used&quot;] = &quot;{:.2f} GB&quot;.format(records[&quot;memory used&quot;] / (1000.0 * 1000.0 * 1000.0))
+        records[&quot;uptime&quot;] = int(time.time() - records[&quot;start time&quot;])
+        hours, mins = divmod(records[&quot;uptime&quot;] / 60, 60)
+        records[&quot;uptime&quot;] = &quot;{}:{:02d} hours&quot;.format(hours, mins)
+        del records[&quot;start time&quot;]
+
+        for item, value in sorted(records.items(), key=lambda x: x[0]):
+            changed = item in self.lastResult and self.lastResult[item] != value
+            s = &quot; {:&lt;30}{:&gt;18} &quot;.format(item, value)
+            try:
+                if self.usesCurses:
+                    self.window.addstr(y, x, s, curses.A_REVERSE if changed else curses.A_NORMAL)
+                else:
+                    print(s)
+            except curses.error:
+                pass
+            y += 1
+
+        if self.usesCurses:
+            self.window.refresh()
+
+        self.lastResult = records
+
+
+Dashboard.registerWindow(HelpWindow, &quot;h&quot;)
+Dashboard.registerWindow(WorkWindow, &quot;j&quot;)
+Dashboard.registerWindow(SlotsWindow, &quot;c&quot;)
+Dashboard.registerWindow(SystemWindow, &quot;s&quot;)
+
+
+if __name__ == &quot;__main__&quot;:
+    main()
</ins></span></pre></div>
<a id="CalendarServertrunkcalendarservertoolsjobitemspy"></a>
<div class="delfile"><h4>Deleted: CalendarServer/trunk/calendarserver/tools/jobitems.py (12891 => 12892)</h4>
<pre class="diff"><span>
<span class="info">--- CalendarServer/trunk/calendarserver/tools/jobitems.py        2014-03-13 00:51:31 UTC (rev 12891)
+++ CalendarServer/trunk/calendarserver/tools/jobitems.py        2014-03-13 18:09:35 UTC (rev 12892)
</span><span class="lines">@@ -1,293 +0,0 @@
</span><del>-#!/usr/bin/env python
-
-##
-# Copyright (c) 2006-2014 Apple Inc. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the &quot;License&quot;);
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an &quot;AS IS&quot; BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-##
-from __future__ import print_function
-
-from calendarserver.tools.cmdline import utilityMain, WorkerService
-
-from getopt import getopt, GetoptError
-
-from twext.enterprise.jobqueue import JobItem
-
-from twisted.internet.defer import inlineCallbacks, succeed
-
-import curses
-import os
-import sys
-
-useCurses = True
-
-def usage(e=None):
-
-    name = os.path.basename(sys.argv[0])
-    print(&quot;usage: %s [options]&quot; % (name,))
-    print(&quot;&quot;)
-    print(&quot;  TODO: describe usage&quot;)
-    print(&quot;&quot;)
-    print(&quot;options:&quot;)
-    print(&quot;  -h --help: print this help and exit&quot;)
-    print(&quot;  -e --error: send stderr to stdout&quot;)
-    print(&quot;  -f --config &lt;path&gt;: specify caldavd.plist configuration path&quot;)
-    print(&quot;  -t: text output, not curses&quot;)
-    print(&quot;&quot;)
-
-    if e:
-        sys.exit(64)
-    else:
-        sys.exit(0)
-
-
-BOX_WIDTH = 52
-
-def main():
-
-    try:
-        (optargs, _ignore_args) = getopt(
-            sys.argv[1:], &quot;hef:t&quot;, [
-                &quot;help&quot;,
-                &quot;error&quot;,
-                &quot;config=&quot;,
-            ],
-        )
-    except GetoptError, e:
-        usage(e)
-
-    #
-    # Get configuration
-    #
-    global useCurses
-    configFileName = None
-    debug = False
-
-    for opt, arg in optargs:
-        if opt in (&quot;-h&quot;, &quot;--help&quot;):
-            usage()
-
-        if opt in (&quot;-e&quot;, &quot;--error&quot;):
-            debug = True
-
-        elif opt in (&quot;-f&quot;, &quot;--config&quot;):
-            configFileName = arg
-
-        elif opt in (&quot;-t&quot;):
-            useCurses = False
-
-        else:
-            raise NotImplementedError(opt)
-
-    if useCurses:
-        def _wrapped(stdscrn):
-            JobItemMonitorService.screen = stdscrn
-            curses.curs_set(0)
-            curses.use_default_colors()
-            utilityMain(configFileName, JobItemMonitorService, verbose=debug)
-        curses.wrapper(_wrapped)
-    else:
-        utilityMain(configFileName, JobItemMonitorService, verbose=debug)
-
-
-
-class JobItemMonitorService(WorkerService, object):
-
-    screen = None
-
-    def __init__(self, store):
-        super(JobItemMonitorService, self).__init__(store)
-        from twisted.internet import reactor
-        self.reactor = reactor
-        self.paused = False
-        self.seconds = 0.1
-
-
-    @inlineCallbacks
-    def doWork(self):
-        self.window = None
-        yield self.displayJobs()
-        self.reactor.callLater(0, self.updateDisplay)
-
-
-    def postStartService(self):
-        &quot;&quot;&quot;
-        Don't quit right away
-        &quot;&quot;&quot;
-        pass
-
-
-    @inlineCallbacks
-    def displayHelp(self):
-        if self.window is not None:
-            self.window.clear()
-        self.window = HelpWindow(10, BOX_WIDTH, 0, 0, self.store, &quot;Help&quot;)
-        yield self.window.update()
-
-
-    @inlineCallbacks
-    def displayJobs(self):
-        if self.window is not None:
-            self.window.clear()
-        self.window = WorkWindow(JobItem.numberOfWorkTypes() + 5, BOX_WIDTH, 0, 0, self.store, &quot;Jobs&quot;)
-        yield self.window.update()
-
-
-    @inlineCallbacks
-    def updateDisplay(self):
-        try:
-            if not self.paused and self.window.requiresUpdate():
-                yield self.window.update()
-        except Exception as e:
-            print(str(e))
-        if not useCurses:
-            print(&quot;-------------&quot;)
-
-        # Check keystrokes
-        if useCurses:
-            try:
-                c = self.window.window.getkey()
-            except:
-                c = -1
-            if c == &quot;q&quot;:
-                self.reactor.stop()
-            elif c == &quot; &quot;:
-                self.paused = not self.paused
-            elif c == &quot;t&quot;:
-                self.seconds = 1.0 if self.seconds == 0.1 else 1.0
-            elif c == &quot;h&quot;:
-                yield self.displayHelp()
-            elif c == &quot;j&quot;:
-                yield self.displayJobs()
-
-        self.reactor.callLater(self.seconds, self.updateDisplay)
-
-
-
-class BaseWindow(object):
-    def __init__(self, nlines, ncols, begin_y, begin_x, store, title):
-        self.window = curses.newwin(nlines, ncols, begin_y, begin_x) if useCurses else None
-        if useCurses:
-            self.window.nodelay(1)
-        self.ncols = ncols
-        self.store = store
-        self.title = title
-        self.iter = 0
-        self.lastResult = {}
-
-
-    def requiresUpdate(self):
-        return True
-
-
-    def clear(self):
-
-        if useCurses:
-            self.window.erase()
-            self.window.refresh()
-
-
-    def update(self):
-        return succeed(True)
-
-
-
-class HelpWindow(BaseWindow):
-
-    def requiresUpdate(self):
-        return False
-
-
-    def update(self):
-
-        if useCurses:
-            self.window.erase()
-            self.window.border()
-            self.window.addstr(0, 2, &quot;Help for Dashboard&quot;)
-
-        x = 1
-        y = 1
-
-        for title in (
-            &quot;j - display server jobs&quot;,
-            &quot;h - display dashboard help&quot;,
-            &quot;&quot;,
-            &quot;  - (space) pause dashboard polling&quot;,
-            &quot;t - toggle update between 0.1 and 1.0 seconds&quot;,
-            &quot;&quot;,
-            &quot;q - exit the dashboard&quot;,
-        ):
-            if useCurses:
-                self.window.addstr(y, x, title)
-            else:
-                print(title)
-            y += 1
-
-        if useCurses:
-            self.window.refresh()
-
-        return succeed(True)
-
-
-
-class WorkWindow(BaseWindow):
-
-    @inlineCallbacks
-    def update(self):
-        txn = self.store.newTransaction()
-        records = (yield JobItem.histogram(txn))
-        self.iter += 1
-
-        if useCurses:
-            self.window.erase()
-            self.window.border()
-            self.window.addstr(0, 2, self.title + &quot; {} ({})&quot;.format(len(records), self.iter,))
-
-        x = 1
-        y = 1
-        s = &quot; {:&lt;40}{:&gt;8} &quot;.format(&quot;Work Type&quot;, &quot;Count&quot;)
-        if useCurses:
-            self.window.addstr(y, x, s, curses.A_REVERSE)
-        else:
-            print(s)
-        y += 1
-        for work_type, count in sorted(records.items(), key=lambda x: x[0]):
-            changed = work_type in self.lastResult and self.lastResult[work_type] != count
-            s = &quot;{}{:&lt;40}{:&gt;8} &quot;.format(&quot;&gt;&quot; if count else &quot; &quot;, work_type, count)
-            try:
-                if useCurses:
-                    self.window.addstr(y, x, s, curses.A_REVERSE if changed else (curses.A_BOLD if count else curses.A_NORMAL))
-                else:
-                    print(s)
-            except curses.error:
-                pass
-            y += 1
-
-        s = &quot; {:&lt;40}{:&gt;8} &quot;.format(&quot;Total:&quot;, sum(records.values()))
-        if useCurses:
-            self.window.hline(y, x, &quot;-&quot;, BOX_WIDTH - 2)
-            y += 1
-            self.window.addstr(y, x, s)
-        else:
-            print(s)
-        y += 1
-
-        if useCurses:
-            self.window.refresh()
-
-        self.lastResult = records
-        yield txn.commit()
-
-
-if __name__ == &quot;__main__&quot;:
-    main()
</del></span></pre></div>
<a id="CalendarServertrunkcontribtoolsreadStatspy"></a>
<div class="modfile"><h4>Modified: CalendarServer/trunk/contrib/tools/readStats.py (12891 => 12892)</h4>
<pre class="diff"><span>
<span class="info">--- CalendarServer/trunk/contrib/tools/readStats.py        2014-03-13 00:51:31 UTC (rev 12891)
+++ CalendarServer/trunk/contrib/tools/readStats.py        2014-03-13 18:09:35 UTC (rev 12892)
</span><span class="lines">@@ -25,6 +25,7 @@
</span><span class="cx"> import sys
</span><span class="cx"> import tables
</span><span class="cx"> import time
</span><ins>+import errno
</ins><span class="cx"> 
</span><span class="cx"> &quot;&quot;&quot;
</span><span class="cx"> This tool reads data from the server's statistics socket and prints a summary.
</span><span class="lines">@@ -39,18 +40,25 @@
</span><span class="cx">     try:
</span><span class="cx">         s = socket.socket(socket.AF_INET if useTCP else socket.AF_UNIX, socket.SOCK_STREAM)
</span><span class="cx">         s.connect(sockname)
</span><ins>+        s.sendall(json.dumps([&quot;stats&quot;]) + &quot;\r\n&quot;)
+        s.setblocking(0)
</ins><span class="cx">         data = &quot;&quot;
</span><del>-        while True:
-            d = s.recv(1024)
</del><ins>+        while not data.endswith(&quot;\n&quot;):
+            try:
+                d = s.recv(1024)
+            except socket.error as se:
+                if se.args[0] != errno.EWOULDBLOCK:
+                    raise
+                continue
</ins><span class="cx">             if d:
</span><span class="cx">                 data += d
</span><span class="cx">             else:
</span><span class="cx">                 break
</span><span class="cx">         s.close()
</span><del>-        data = json.loads(data)
-    except socket.error:
-        data = {&quot;Failed&quot;: &quot;Unable to read statistics from server: %s&quot; % (sockname,)}
-    data[&quot;Server&quot;] = sockname
</del><ins>+        data = json.loads(data)[&quot;stats&quot;]
+    except socket.error as e:
+        data = {&quot;Failed&quot;: &quot;Unable to read statistics from server: %s %s&quot; % (sockname, e)}
+    data[&quot;server&quot;] = sockname
</ins><span class="cx">     return data
</span><span class="cx"> 
</span><span class="cx"> 
</span><span class="lines">@@ -74,18 +82,18 @@
</span><span class="cx"> def printStat(stats, index, showMethods, topUsers, showAgents):
</span><span class="cx"> 
</span><span class="cx">     print(&quot;- &quot; * 40)
</span><del>-    print(&quot;Server: %s&quot; % (stats[&quot;Server&quot;],))
</del><ins>+    print(&quot;Server: %s&quot; % (stats[&quot;server&quot;],))
</ins><span class="cx">     print(datetime.datetime.now().strftime(&quot;%Y/%m/%d %H:%M:%S&quot;))
</span><del>-    print(&quot;Service Uptime: %s&quot; % (datetime.timedelta(seconds=(int(time.time() - stats[&quot;System&quot;][&quot;start time&quot;]))),))
-    if stats[&quot;System&quot;][&quot;cpu count&quot;] &gt; 0:
</del><ins>+    print(&quot;Service Uptime: %s&quot; % (datetime.timedelta(seconds=(int(time.time() - stats[&quot;system&quot;][&quot;start time&quot;]))),))
+    if stats[&quot;system&quot;][&quot;cpu count&quot;] &gt; 0:
</ins><span class="cx">         print(&quot;Current CPU: %.1f%% (%d CPUs)&quot; % (
</span><del>-            stats[&quot;System&quot;][&quot;cpu use&quot;],
-            stats[&quot;System&quot;][&quot;cpu count&quot;],
</del><ins>+            stats[&quot;system&quot;][&quot;cpu use&quot;],
+            stats[&quot;system&quot;][&quot;cpu count&quot;],
</ins><span class="cx">         ))
</span><span class="cx">         print(&quot;Current Memory Used: %d bytes (%.1f GB) (%.1f%% of total)&quot; % (
</span><del>-            stats[&quot;System&quot;][&quot;memory used&quot;],
-            stats[&quot;System&quot;][&quot;memory used&quot;] / (1024.0 * 1024 * 1024),
-            stats[&quot;System&quot;][&quot;memory percent&quot;],
</del><ins>+            stats[&quot;system&quot;][&quot;memory used&quot;],
+            stats[&quot;system&quot;][&quot;memory used&quot;] / (1024.0 * 1024 * 1024),
+            stats[&quot;system&quot;][&quot;memory percent&quot;],
</ins><span class="cx">         ))
</span><span class="cx">     else:
</span><span class="cx">         print(&quot;Current CPU: Unavailable&quot;)
</span><span class="lines">@@ -112,7 +120,7 @@
</span><span class="cx">     times = []
</span><span class="cx">     for stat in stats:
</span><span class="cx">         try:
</span><del>-            t = str(datetime.timedelta(seconds=int(time.time() - stat[&quot;System&quot;][&quot;start time&quot;])))
</del><ins>+            t = str(datetime.timedelta(seconds=int(time.time() - stat[&quot;system&quot;][&quot;start time&quot;])))
</ins><span class="cx">         except KeyError:
</span><span class="cx">             t = &quot;-&quot;
</span><span class="cx">         times.append(t)
</span><span class="lines">@@ -120,9 +128,9 @@
</span><span class="cx">     cpus = []
</span><span class="cx">     memories = []
</span><span class="cx">     for stat in stats:
</span><del>-        if stat[&quot;System&quot;][&quot;cpu count&quot;] &gt; 0:
-            cpus.append(stat[&quot;System&quot;][&quot;cpu use&quot;])
-            memories.append(stat[&quot;System&quot;][&quot;memory percent&quot;])
</del><ins>+        if stat[&quot;system&quot;][&quot;cpu count&quot;] &gt; 0:
+            cpus.append(stat[&quot;system&quot;][&quot;cpu use&quot;])
+            memories.append(stat[&quot;system&quot;][&quot;memory percent&quot;])
</ins><span class="cx">         else:
</span><span class="cx">             cpus.append(-1)
</span><span class="cx">             memories.append(-1)
</span><span class="lines">@@ -139,7 +147,7 @@
</span><span class="cx"> 
</span><span class="cx"> 
</span><span class="cx"> def serverLabels(stats):
</span><del>-    servers = [stat[&quot;Server&quot;] for stat in stats]
</del><ins>+    servers = [stat[&quot;server&quot;] for stat in stats]
</ins><span class="cx">     if isinstance(servers[0], tuple):
</span><span class="cx">         hosts = set([item[0] for item in servers])
</span><span class="cx">         ports = set([item[1] for item in servers])
</span><span class="lines">@@ -193,7 +201,7 @@
</span><span class="cx">         )
</span><span class="cx">     )
</span><span class="cx"> 
</span><del>-    for key, seconds in ((&quot;Current&quot;, 60,), (&quot;1 Minute&quot;, 60,), (&quot;5 Minutes&quot;, 5 * 60,), (&quot;1 Hour&quot;, 60 * 60,),):
</del><ins>+    for key, seconds in ((&quot;current&quot;, 60,), (&quot;1m&quot;, 60,), (&quot;5m&quot;, 5 * 60,), (&quot;1h&quot;, 60 * 60,),):
</ins><span class="cx"> 
</span><span class="cx">         stat = stats[key]
</span><span class="cx">         table.addRow((
</span><span class="lines">@@ -515,7 +523,7 @@
</span><span class="cx">     topUsers = 0
</span><span class="cx">     showAgents = False
</span><span class="cx"> 
</span><del>-    multimodes = ((&quot;Current&quot;, 60,), (&quot;1 Minute&quot;, 60,), (&quot;5 Minutes&quot;, 5 * 60,), (&quot;1 Hour&quot;, 60 * 60,),)
</del><ins>+    multimodes = ((&quot;current&quot;, 60,), (&quot;1m&quot;, 60,), (&quot;5m&quot;, 5 * 60,), (&quot;1h&quot;, 60 * 60,),)
</ins><span class="cx">     multimode = multimodes[2]
</span><span class="cx"> 
</span><span class="cx">     options, args = getopt.getopt(sys.argv[1:], &quot;hs:t:&quot;, [&quot;tcp=&quot;, &quot;0&quot;, &quot;1&quot;, &quot;5&quot;, &quot;60&quot;, &quot;methods&quot;, &quot;users=&quot;, &quot;agents&quot;])
</span></span></pre></div>
<a id="CalendarServertrunktxweb2metafdpy"></a>
<div class="modfile"><h4>Modified: CalendarServer/trunk/txweb2/metafd.py (12891 => 12892)</h4>
<pre class="diff"><span>
<span class="info">--- CalendarServer/trunk/txweb2/metafd.py        2014-03-13 00:51:31 UTC (rev 12891)
+++ CalendarServer/trunk/txweb2/metafd.py        2014-03-13 18:09:35 UTC (rev 12892)
</span><span class="lines">@@ -223,6 +223,10 @@
</span><span class="cx">         self.stopped = stopped
</span><span class="cx"> 
</span><span class="cx"> 
</span><ins>+    def items(self):
+        return dict([(attr, getattr(self, attr)) for attr in self.showAttributes])
+
+
</ins><span class="cx">     def effective(self):
</span><span class="cx">         &quot;&quot;&quot;
</span><span class="cx">         The current effective load.
</span></span></pre>
</div>
</div>

</body>
</html>