[CalendarServer-users] CalendarServer 3.1 on Ubuntu 11.10 Server
Jim J
jahosefet at gmail.com
Wed Dec 28 11:29:24 PST 2011
Thanks Andre!
The idea works, not so much the implementation though. I'll post what I've
found, perhaps I missed something otherwise it might be something to look
into or at least help others.
I was able to set up the database schema but I couldn't do it using the
bootstrap Python script. The script imports
calendarserver.tools.bootstrapdatabase and runs main(). The
bootstrapdatabase.py file makes some improper (and un-overrideable)
assumptions. First, CONNECTNAME is assumed to be "_postgres" but on Ubuntu
it's just "postgres" (no leading underscore). Maybe this isn't a bad
assumption on Mac OS X Server? Second it assumes the existence of database
user "caldav" and database "caldav". Unfortunately I'd already committed to
'caldavd' (trailing 'd') being the username and group for the daemon so
that didn't work either.
I went into the Python file and changed those. Then it assumed that the SQL
script that it needed to execute was located at
/usr/share/caldavd/lib/python/.../current.sql. However, when I installed
calendarserver using ./run -i that's not where it put the SQL script! It
put it in /usr/local/lib/python2.7/dist-packages/.../current.sql. I ended
up quitting at this point and logged into PostgresSQL as the caldavd user
and manually executed the SQL script.
Finally I was able to get calendarserver fully running *without SSL
encryption*. When I went to enable SSL, I seem to have tripped over some
crazy bug in the twisted framework. I didn't dig too deep but in Twisted
11.0.0 there's a file internet/tcp.py. In the Connection class there's an
"if SSL:" and then a bunch of functions. Inside _startTLS there's a line:
self.fileno = self.socket.fileno
This line asserts. I don't really know why. Something about
OpenSSH.SSL.Connection doesn't have attribute 'fileno'. Anyway, I gave up
at this point, locked the unencrypted calendarserver to localhost and
wrapped it in stunnel4. That seems to work. That line doesn't seem to exist
in Twisted 10.x or even in 11.1.0 so I'm guessing it was a problem.
Thanks for the help.
Jim
On Tue, Dec 27, 2011 at 7:10 PM, Andre LaBranche <dre at apple.com> wrote:
> Forgot to cc the list... See quoted text for answers :)
>
> -dre
>
> Sent from my iPhone
>
> On Dec 27, 2011, at 6:34 PM, Andre LaBranche <dre at apple.com> wrote:
>
> We only provision schema if we spawn Postgres. To use an externally
> managed DB, provision schema with calendarserver_bootstrap_database.
>
> Cheers,
> -dre
>
> Sent from my iPhone
>
> On Dec 27, 2011, at 4:33 PM, Jim J <jahosefet at gmail.com> wrote:
>
> Andre,
>
> Thanks! That's a very useful link actually. I've been trying to figure out
> what all the options are. For some reason it didn't catch my attention
> during my 'grep's. That fixes all the above problems, I can even use -u and
> -g now.
>
> Next problem:
> I created the caldavd user in PostgreSQL and created the caldavd database
> (I'm using -u caldavd -g caldavd again). I noticed this connects to the
> database using the caldavd user. Now I get this error:
>
> 2011-12-27 16:17:47-0500 [-]
> [txdav.common.datastore.util.UpgradeDatabaseSchemaService#warn] Beginning
> database schema check.
> 2011-12-27 16:17:47-0500 [-]
> [txdav.common.datastore.util.UpgradeDatabaseSchemaService#warn] Required
> schema version: 3.
> 2011-12-27 16:17:47-0500 [-] Unhandled error in Deferred:
> 2011-12-27 16:17:47-0500 [-] Unhandled Error
> Traceback (most recent call last):
> File
> "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 388, in
> errback
> self._startRunCallbacks(fail)
> File
> "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 455, in
> _startRunCallbacks
> self._runCallbacks()
> File
> "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 542, in
> _runCallbacks
> current.result = callback(current.result, *args, **kw)
> File
> "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1076, in
> gotResult
> _inlineCallbacks(r, g, deferred)
> --- <exception caught here> ---
> File
> "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1018, in
> _inlineCallbacks
> result = result.throwExceptionIntoGenerator(g)
> File
> "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 350, in
> throwExceptionIntoGenerator
> return g.throw(self.type, self.value, self.tb)
> File
> "/usr/local/lib/python2.7/dist-packages/txdav/common/datastore/util.py",
> line 271, in doUpgrade
> actual_version = yield sqlTxn.schemaVersion()
> File
> "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1018, in
> _inlineCallbacks
> result = result.throwExceptionIntoGenerator(g)
> File
> "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 350, in
> throwExceptionIntoGenerator
> return g.throw(self.type, self.value, self.tb)
> File
> "/usr/local/lib/python2.7/dist-packages/txdav/common/datastore/sql.py",
> line 233, in schemaVersion
> result = yield self._schemaVersion.on(self)
> File
> "/usr/local/lib/python2.7/dist-packages/twext/internet/threadutils.py",
> line 64, in _oneWorkUnit
> result = instruction()
> File
> "/usr/local/lib/python2.7/dist-packages/twext/enterprise/adbapi2.py", line
> 155, in <lambda>
> lambda : self._reallyExecSQL(*args, **kw)
> File
> "/usr/local/lib/python2.7/dist-packages/twext/enterprise/adbapi2.py", line
> 136, in _reallyExecSQL
> result = self._reallyExecSQL(sql, args, raiseOnZeroRowCount)
> File
> "/usr/local/lib/python2.7/dist-packages/twext/enterprise/adbapi2.py", line
> 130, in _reallyExecSQL
> self._cursor.execute(sql, args)
> File
> "/usr/local/lib/python2.7/dist-packages/txdav/base/datastore/dbapiclient.py",
> line 76, in execute
> self.realCursor.execute(sql, args)
> File "/usr/local/lib/python2.7/dist-packages/pgdb.py", line 259,
> in execute
> self.executemany(operation, (params,))
> File "/usr/local/lib/python2.7/dist-packages/pgdb.py", line 289,
> in executemany
> raise DatabaseError("error '%s' in '%s'" % (msg, sql))
> pg.DatabaseError: error 'ERROR: relation "calendarserver" does
> not exist
> LINE 1: select VALUE from CALENDARSERVER where NAME = 'VERSION'
> ^
> ' in 'select VALUE from CALENDARSERVER where NAME = 'VERSION''
>
>
> I would have expected calendarserver to set up its own tables/schema (as
> it appears to be attempting to do). Can I force it to rebuild the database?
>
> Thanks,
>
> Jim
>
> On Tue, Dec 27, 2011 at 1:40 PM, Andre LaBranche <dre at apple.com> wrote:
>
>> Hi,
>>
>> Try setting UseDatabase to Postgres, as described here:
>> http://trac.calendarserver.org/browser/CalendarServer/tags/release/CalendarServer-3.1/twistedcaldav/stdconfig.py
>>
>> You will also need to specify DSN which tells calendar server how to
>> connect to postgres.
>>
>> You can also set ServerEnabled to false for memcached to keep calendar
>> server from spawning memcached.
>>
>> I can provide examples once I get back home to a real machine...
>>
>> Cheers,
>> -dre
>>
>> Sent from my iPhone
>>
>> On Dec 27, 2011, at 1:21 PM, Jim J <jahosefet at gmail.com> wrote:
>>
>> All,
>>
>> I'm quite new to this calendarserver thing. I'm trying to set up 3.1
>> (from tags/releases in Subversion) on a 64-bit Ubuntu 11.10 server
>> installation. It's been a disaster. I was able to run calendarserver from
>> the SVN directory using ./run and then I "installed" it using ./run -i.
>> That's where things went bad. Some issues I've worked around, but I'd like
>> to get good solutions for. Currently I'm stuck trying to figure out what's
>> going on with PostgreSQL. Perhaps you guys can clue me in on how to
>> properly fix the following issues.
>>
>> This is how I'm currently executing caldavd. Once it works I'll put it
>> into init.d and run it as a proper server. The () are because I *was*
>> running with those flags (see part 1) and have given up on them.
>>
>> sudo caldavd -X -L -f <path to caldavd.plist file> (-u caldavd -g caldavd)
>>
>> 1. As I understand it, it is good practice to run Linux services that are
>> exposed to the internet as a restricted user. Given that and existence of
>> the -u and -g flags for caldavd, I figured those flags would be the right
>> way to do things. That doesn't work at all.
>>
>> The first problem is that caldavd wants to write to
>> /var/run/ccs_postgresql_xxxxx which it can't do because /var belongs to
>> root and it attempts to do this after the setuid() call to become the
>> caldavd user. So, I went into caldavd.plist and changed the "DataRoot" to a
>> different directory and gave it to caldavd:caldavd.
>>
>> Then I tried again. This time it failed because it appears that
>> calendarserver attempts to start postgresql whether or not the postgresql
>> service is already running on the local box. To do this, it tries to
>> setuid() again, this time to ROOT! If the entire point of setting a user
>> for a service is to restrict it's privileges, why would one of those privs
>> be to allow setuid to root? That makes no sense.
>>
>> So I gave up, now I'm trying to run it as root to see if the 3.1 release
>> even functions. How do I make this server run as somebody other than root?
>>
>> 2. Running as root (without the -u and -g flags shown above) gets the
>> application started and then it fails attempting to start memcached. I get
>> the error "cannot listen on tcp port". This is because memcachd is a system
>> service and is *already running*. Again, why is calendarserver attempting
>> to start a service that's already running?
>>
>> So I stopped memcachd and tried again.
>>
>> 3. Calendarserver now successfully starts and connects to memcached but
>> it cannot connect to postgresql. This I haven't yet worked around. I get
>> the following error:
>> 2011-12-27 12:43:11-0500 [-] [txdav.base.datastore.subpostgres#error]
>> [Failure instance: Traceback: <class 'pg.InternalError'>: could not connect
>> to server: No such file or directory
>> Is the server running locally and accepting
>> connections on Unix domain socket
>> "/tmp/ccs_postgres_f5e263275ce7de576bd42ca08db4fc56/.s.PGSQL.5432"?
>>
>>
>> /usr/lib/python2.7/dist-packages/twisted/internet/_baseprocess.py:60:maybeCallProcessEnded
>>
>> /usr/local/lib/python2.7/dist-packages/txdav/base/datastore/subpostgres.py:92:processEnded
>>
>> /usr/lib/python2.7/dist-packages/twisted/internet/defer.py:361:callback
>>
>> /usr/lib/python2.7/dist-packages/twisted/internet/defer.py:455:_startRunCallbacks
>> --- <exception caught here> ---
>>
>> /usr/lib/python2.7/dist-packages/twisted/internet/defer.py:542:_runCallbacks
>>
>> /usr/local/lib/python2.7/dist-packages/txdav/base/datastore/subpostgres.py:367:gotReady
>>
>> /usr/local/lib/python2.7/dist-packages/txdav/base/datastore/subpostgres.py:269:ready
>>
>> /usr/local/lib/python2.7/dist-packages/txdav/base/datastore/subpostgres.py:261:produceConnection
>>
>> /usr/local/lib/python2.7/dist-packages/txdav/base/datastore/dbapiclient.py:196:connect
>> /usr/local/lib/python2.7/dist-packages/pgdb.py:482:connect
>> ]
>>
>> Here I'm stuck. Quite clearly calendarserver is expecting to launch the
>> postgresql service (which, again, doesn't make sense to me), but then it
>> can't connect when it does. Well, at least I'm assuming calendarserver
>> successfully launches postgresql because I don't get the setuid() error I
>> got in part 1 when it tries to setuid() to root in order to launch
>> postgresql. I can't figure out how to fix this. Any thoughts?
>>
>> Thanks for the help,
>>
>> Jim
>>
>> _______________________________________________
>> calendarserver-users mailing list
>> calendarserver-users at lists.macosforge.org
>> http://lists.macosforge.org/mailman/listinfo.cgi/calendarserver-users
>>
>>
>
> _______________________________________________
> calendarserver-users mailing list
> calendarserver-users at lists.macosforge.org
> http://lists.macosforge.org/mailman/listinfo.cgi/calendarserver-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/calendarserver-users/attachments/20111228/db6c3d79/attachment-0001.html>
More information about the calendarserver-users
mailing list