On Sunday 10 November 2013 11:59 PM, Rahul Amaram wrote:

On Tuesday 05 November 2013 12:01 AM, Glyph wrote:

...

On Nov 2, 2013, at 9:15 AM, Rahul Amaram <amaramrahul@users.sourceforge.net> wrote:

...

I am trying to run calendarserver 5.0 on Debian Jessie. While postgresql backend seems to work fine, I have been facing issues with files backend (I have user_xattr enabled). Could some feedback be provided on this? I am attaching the error.log for reference. The errors with timestamp 18:31 popped up as soon as I started calendarserver. The errors with timestamp 18:33 were encountered when I tried to authenticate to calendarserver.

Early response would be much appreciated so that I could push the new Debian package to testing.

Hi Rahul,

The filesystem store is known to be incomplete. It exists only to provide a migration path for legacy data to the PostgreSQL database (and to provide a little extra test coverage to allow us to test multiple implementations of a subset of functionality). The right fix here is probably to remove all configuration options which would allow you to attempt to use it in a running server until someone volunteers to maintain it and keep it up to date :-).

Are you trying to use it as the actual backend for the store or are you having problems with it during an upgrade from old data? When reporting a bug like this it's always a good idea to attach your plist :-).

-glyph

Hi, I was trying to use it as an actual backend. I guess that is why it didn't work. So, are there any upgrade instructions on how to migrate from the file store to postgresql? I think that this would be done automatically. Even then, I would like to know the manual instructions for performing migration.

- Rahul.

Any response to this? I am unable to push to the new version to Debian as I need more clarity with the migration process. Request the developers to help me out with this. Regards, Rahul.

On Monday 18 November 2013 11:29 PM, Andre LaBranche wrote:

On Nov 18, 2013, at 9:17 AM, Rahul Amaram <amaramrahul@users.sourceforge.net> wrote:

...

Any response to this? I am unable to push to the new version to Debian as I need more clarity with the migration process. Request the developers to help me out with this. Hi,

When the server starts, if there is any calendar data in the DocumentRoot, that data will be upgraded and imported into the database. This is configured to happen automatically for Server.app; however for open source, please be aware of the following items, which are also useful for controlling how database schema updates are performed:

* The caldavd.plist key FailIfUpgradeNeeded

"FailIfUpgradeNeeded" : True, # Set to True to prevent the server or utility tools # tools from running if the database needs a schema # upgrade.

(and hey, looks like we have a typo... They are just tools, not tools tools :)

The cli tool calendarserver_upgrade:

andre@xomg[trunk/CalendarServer]./python bin/calendarserver_upgrade --help Usage: calendarserver_upgrade [options] [input specifiers]

This tool allows any necessary upgrade to complete, then exits. Options: -s, --status Check database status and exit. -p, --postprocess Perform post-database-import processing. -D, --debug Debug logging. -f, --config= Specify caldavd.plist configuration path. [default: /Applications/Server.app/Contents/ServerRoot/private/etc/caldavd/caldavd-apple.plist] -x, --prefix= Only upgrade homes with the specified GUID prefix - partial upgrade only. [default: ] --help Display this help and exit. -m, --merge Rather than skipping homes that exist on the filesystem but not in the database, merge their data into the existing homes. --version Display Twisted version and exit. -o, --output= Specify output file path (default: '-', meaning stdout).

Unless you set FailIfUpgradeNeeded to false, you need to run calendarserver_upgrade any time the server's schema and data version are newer than existing data (whether it's in the filesystem or the DB).

-dre This is a really old thread for which I needed a clarification. How do we perform database schema upgrade? Currently I am doing it manually by calling each of the upgrade_from_<from>_to_<to>.sql version files. Does calendarserver_upgrade take care of it? If not, is there an easier way of doing it?

Currently for upgrading from 3.2 files to 5.1 db backend, I am just creating a new db. And the migration from 3.2 files backend to 5.1 db seems to work fine. Is there anything else that I should take care? Thanks, Rahul.

On Thursday 06 March 2014 01:13 AM, Andre LaBranche wrote:

On Mar 5, 2014, at 11:34 AM, Rahul Amaram <amaramrahul@users.sourceforge.net <mailto:amaramrahul@users.sourceforge.net>> wrote:

...

How do we perform database schema upgrade? Currently I am doing it manually by calling each of the upgrade_from_<from>_to_<to>.sql version files. Does calendarserver_upgrade take care of it? If not, is there an easier way of doing it?

Yes, calendarserver_upgrade has one job: to apply any necessary schema and data version upgrades. It is intended to be used immediately after upgrading to a newer version, before the service is started.

...

Currently for upgrading from 3.2 files to 5.1 db backend, I am just creating a new db. And the migration from 3.2 files backend to 5.1 db seems to work fine. Is there anything else that I should take care?

That sounds like a workable strategy for this upgrade delta. Going forward, calendarserver_upgrade should be all that is needed.

-dre

Thanks Andre. So, calendarserver_bootstrap_database is run for the first time to setup db. And calendarserver_upgrade should be run for any subsequent upgrades. Also, let me know if you think that there is anything in calendarserver_upgrade which might make it to break in linux? Regards, Rahul.

On Thursday 06 March 2014 01:57 AM, Andre LaBranche wrote:

On Mar 5, 2014, at 12:13 PM, Rahul Amaram <amaramrahul@users.sourceforge.net> wrote:

...

Thanks Andre. So, calendarserver_bootstrap_database is run for the first time to setup db. And calendarserver_upgrade should be run for any subsequent upgrades. Correct.

...

Also, let me know if you think that there is anything in calendarserver_upgrade which might make it to break in linux? It is regularly tested and used in linux. As long as the DB is up and running a current or previous version of the schema, and caldavd.plist is properly configured to access that DB, running calendarserver_upgrade is expected to ensure upgrade to the current schema and data version. There are of course errors that can occur during upgrade, but these are not specific to Linux and generally have more to do with unexpected data in the DB. As always, please report any issues you encounter.

-dre Ok. Also, it would be good if calendarserver_bootstrap_database would work in cases other than that of peer authentication. As I don't use peer authentication, I have to manually run the create user, create db and import schema commands. My understanding is that calendarserver_bootstrap_database cannot currently be used for remote postgresql servers.

Thanks for the quick responses. I have finished packaging and testing calendarserver 5.1 for Debian. It should be in the Debian repos in a couple of days. Regards, Rahul.

On Thursday 06 March 2014 02:13 AM, Fredrik Unger wrote:

...

Thanks for the quick responses. I have finished packaging and testing calendarserver 5.1 for Debian. It should be in the Debian repos in a couple of days.

Thanks Rahul,

I started working a bit on it, but I will wait for this package and see if encounter any problems. I can then help debug or whatever. That is better than doing twice the packaging work.

Yeah. That would be better. You can also join as a co-maintainer for the debian package if you are interested and have time to work on it.

As for issues I went package to package so I have not yet reached the 5x series. I do not have any issues to report, but have not tested the intermediate packages very well.

I have not made any Debian package for 4.x series. This package is meant for users to upgrade from 2.x / 3.x (I have not tested upgrading package from 2.x. Will do this if 5.x could make it to Ubuntu LTS).

If I can help with anything let me know.

Definitely. Regards, Rahul.