Error Upgrading Database (Schema ver 17->ver 28)
In attempting to upgrade the database from version 17 to version 28 (CalendarServer 5.2), I encountered an error. The Calendar Database upgraded just fine, but when calendarserver_upgrade attempted to execute txdav/common/datastore/upgrade/sql/upgrades/addressbook_upgrade_from_1_to_2.py it gave an error. Here's the Traceback: 2014-09-23 01:54:10-0600 Beginning database addressbook data check. 2014-09-23 01:54:10-0600 Required database key ADDRESSBOOK-DATAVERSION: 2. 2014-09-23 01:54:10-0600 Actual database key ADDRESSBOOK-DATAVERSION: 1. 2014-09-23 01:54:10-0600 Starting addressbook data upgrade from version 1 to 2. 2014-09-23 01:54:10-0600 Applying data upgrade: txdav.common.datastore.upgrade.sql.upgrades.addressbook_upgrade_from_1_to_2.doUpgrade 2014-09-23 01:54:10-0600 Step failure [calendarserver.tap.util#error] Step failure Traceback (most recent call last): File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/internet/defer.py", line 422, in errback self._startRunCallbacks(fail) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/internet/defer.py", line 489, in _startRunCallbacks self._runCallbacks() File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/internet/defer.py", line 576, in _runCallbacks current.result = callback(current.result, *args, **kw) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/internet/defer.py", line 1127, in gotResult _inlineCallbacks(r, g, deferred) --- <exception caught here> --- File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/internet/defer.py", line 1069, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/CalDAV/CalendarServer/txdav/common/datastore/upgrade/sql/upgrade.py", line 146, in databaseUpgrade yield self.upgradeVersion(actual_version, required_version, dialect) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/internet/defer.py", line 1069, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/CalDAV/CalendarServer/txdav/common/datastore/upgrade/sql/upgrade.py", line 211, in upgradeVersion yield self.applyUpgrade(fp) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/internet/defer.py", line 1069, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/CalDAV/CalendarServer/txdav/common/datastore/upgrade/sql/upgrade.py", line 360, in applyUpgrade yield doUpgrade(self.sqlStore) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/internet/defer.py", line 1069, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/CalDAV/CalendarServer/txdav/common/datastore/upgrade/sql/upgrades/addressbook_upgrade_from_1_to_2.py", line 40, in doUpgrade yield populateMemberTables(sqlStore) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/internet/defer.py", line 1069, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/CalDAV/CalendarServer/txdav/common/datastore/upgrade/sql/upgrades/addressbook_upgrade_from_1_to_2.py", line 80, in populateMemberTables yield doToEachHomeNotAtVersion(sqlStore, schema.ADDRESSBOOK_HOME, UPGRADE_TO_VERSION, doIt, "Populate Members") File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/internet/defer.py", line 1069, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/CalDAV/CalendarServer/txdav/common/datastore/upgrade/sql/upgrades/util.py", line 173, in doToEachHomeNotAtVersion yield doIt(txn, homeResourceID) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/internet/defer.py", line 1069, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/CalDAV/CalendarServer/txdav/common/datastore/upgrade/sql/upgrades/addressbook_upgrade_from_1_to_2.py", line 62, in doIt home = yield txn.addressbookHomeWithResourceID(homeResourceID) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/internet/defer.py", line 1069, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/CalDAV/CalendarServer/txdav/common/datastore/sql.py", line 626, in homeWithResourceID result = (yield self.homeWithUID(storeType, uid, create)) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/internet/defer.py", line 1069, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/CalDAV/CalendarServer/txdav/common/datastore/sql.py", line 1606, in homeWithUID homeObject = (yield homeObject.initFromStore()) File "/usr/local/CalDAV/Twisted-12.3.0/build/lib.macosx-10.9-x86_64-2.7/twisted/internet/defer.py", line 1071, in _inlineCallbacks result = g.send(result) File "/usr/local/CalDAV/CalendarServer/txdav/carddav/datastore/sql.py", line 126, in initFromStore self._resourceID, self._addressbookPropertyStoreID = result[0] exceptions.ValueError: need more than 1 value to unpack
Hi Jacques, --On September 23, 2014 at 3:00:24 AM -0500 Jacques Distler <distler@golem.ph.utexas.edu> wrote:
In attempting to upgrade the database from version 17 to version 28 (CalendarServer 5.2), I encountered an error.
The Calendar Database upgraded just fine, but when
calendarserver_upgrade
attempted to execute
txdav/common/datastore/upgrade/sql/upgrades/addressbook_upgrade_from_1_to _2.py
it gave an error. Here's the Traceback:
Do you have any memcached (or calendar server) processes running prior to using calendarserver_upgrade? Looks to me like there is still a memcached running with data from the previous server, but the new server version uses a slightly different format for that cached data. -- Cyrus Daboo
On Sep 23, 2014, at 10:28 AM, Cyrus Daboo <cdaboo@apple.com> wrote:
Do you have any memcached (or calendar server) processes running prior to using calendarserver_upgrade? Looks to me like there is still a memcached running with data from the previous server, but the new server version uses a slightly different format for that cached data.
Ah! Indeed, that was the problem. Toggling memcached off and on made the upgrade go through smoothly. Thanks for the quick response! Best, Jacques
On Sep 23, 2014, at 8:37 AM, Jacques Distler <distler@golem.ph.utexas.edu> wrote:
On Sep 23, 2014, at 10:28 AM, Cyrus Daboo <cdaboo@apple.com> wrote:
Do you have any memcached (or calendar server) processes running prior to using calendarserver_upgrade? Looks to me like there is still a memcached running with data from the previous server, but the new server version uses a slightly different format for that cached data.
Ah!
Indeed, that was the problem. Toggling memcached off and on made the upgrade go through smoothly.
Thanks for the quick response!
Hmm... seems like filing a bug in Trac would still be a good idea though. At least going forward for future upgrades, the server could store a key in memcache indicating its data-format version; while it would still be an error, it could fail a lot faster and alert you that you need to restart memcache first :-). -glyph
participants (3)
-
Cyrus Daboo
-
Glyph Lefkowitz
-
Jacques Distler