From sean at rogue-research.com Mon Jan 6 09:24:16 2020 From: sean at rogue-research.com (Sean McBride) Date: Mon, 6 Jan 2020 12:24:16 -0500 Subject: [CalendarServer-users] Help! CalendarServer was disconnected from OpenDirectory for days, and now everything is gone!? Message-ID: <20200106172416.198875884@mail.rogue-research.com> Hi all, I'm running CalendarServer on macOS 10.13.6 with Server.app 5.6.3 and it's been basically set-and-forget. But now I have a big problem. I updated a network switch's firmware, which broke the connection from my Calendar server to my (separate) Open Directory server. For whatever reason, the connection didn't auto-reestablish. This went unnoticed for days during the holidays. Back in the office today, no one could authenticate. Magic reboot didn't help. I had to go to Sys Prefs > Users&Groups > Login Options and un-join and re-join to the OD server. Now I can authenticate to the calendar server, but everything seems to be gone! Looking in the logs, I see things like: 2019-12-27T18:48:28-0500 [caldav-2] [calendarserver.tools.purge#warn] Principal XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX is no longer in the directory; scheduling clean-up at 2020-01-03 23:48:28.984229 2020-01-03T18:48:29-0500 [caldav-1] [calendarserver.tools.purge#warn] Cleaning up future events for principal XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX since they are no longer in directory So has it indeed "helpfully" deleted everything for me? :( I do have backups. Is there a single database file I could restore? Where is it? Thanks for any help!!! Sean From dre at apple.com Mon Jan 6 11:58:39 2020 From: dre at apple.com (Andre LaBranche) Date: Mon, 06 Jan 2020 11:58:39 -0800 Subject: [CalendarServer-users] Help! CalendarServer was disconnected from OpenDirectory for days, and now everything is gone!? In-Reply-To: <20200106172416.198875884@mail.rogue-research.com> References: <20200106172416.198875884@mail.rogue-research.com> Message-ID: Hi, Sorry to hear about this... it sounds like your diagnosis is correct that this is a helpful feature gone awry. The default for macOS Server's Calendar service is for this feature to be enabled, and to delete data associated with users that have been missing from the directory for more than one week. Most of the configurable values are not expressed in the main config file, but rather inherited. Here's a full view of the available knobs: https://github.com/apple/ccs-calendarserver/blob/master/conf/caldavd-stdconfig.plist#L2114 First thing you should do is check the calendarserver_trash tool, a command line tool that might be able to recover data for you. I honestly don't remember if data purged by 'AutomaticPurging' can be recovered by calendarserver_trash, but it's definitely worth a shot. Run it with "-h" to see the options. If you have to restore from backup, it goes something like this: 1) Fish the DB directory out of your backups. By default, it's at "/Library/Server/Calendar and Contacts/Data/Database.xpg" (I think. See below.) 2) Stop the service 3) Move the existing Database.xpg aside (don't delete it, just move or rename it) 4) Copy the Database.xpg directory from your backups into place 5) Start service Caveat: it has been literally years since I touched any of this, but I can say that the database wrapper used by CalendarServer was designed to make this sort of stuff as easy as possible (or at least, far far easier than the equivalent cajoling of Postgres required to do it 'the old fashioned way'.) You might choose instead to restore the entire "/Library/Server/Calendar and Contacts/Data/" directory, which by default includes Database.xpg and also event attachments. To fully comprehend how CalendarServer evaluates relative paths for things in config files, you also need: https://github.com/apple/ccs-calendarserver/blob/master/twistedcaldav/stdconfig.py#L1276 Because the macOS server plist configures an absolute ServerRoot and relative values for DataRoot and DatabaseRoot, that should mean that Database.xpg lives inside "/Library/Server/Calendar and Contacts/Data/". Good luck! -dre > On Jan 6, 2020, at 9:24 AM, Sean McBride wrote: > > Hi all, > > I'm running CalendarServer on macOS 10.13.6 with Server.app 5.6.3 and it's been basically set-and-forget. But now I have a big problem. > > I updated a network switch's firmware, which broke the connection from my Calendar server to my (separate) Open Directory server. For whatever reason, the connection didn't auto-reestablish. This went unnoticed for days during the holidays. > > Back in the office today, no one could authenticate. Magic reboot didn't help. I had to go to Sys Prefs > Users&Groups > Login Options and un-join and re-join to the OD server. Now I can authenticate to the calendar server, but everything seems to be gone! Looking in the logs, I see things like: > > 2019-12-27T18:48:28-0500 [caldav-2] [calendarserver.tools.purge#warn] Principal XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX is no longer in the directory; scheduling clean-up at 2020-01-03 23:48:28.984229 > > 2020-01-03T18:48:29-0500 [caldav-1] [calendarserver.tools.purge#warn] Cleaning up future events for principal XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX since they are no longer in directory > > So has it indeed "helpfully" deleted everything for me? :( > > I do have backups. Is there a single database file I could restore? Where is it? > > Thanks for any help!!! > > Sean > > > _______________________________________________ > calendarserver-users mailing list > calendarserver-users at lists.macosforge.org > https://lists.macosforge.org/mailman/listinfo/calendarserver-users -------------- next part -------------- An HTML attachment was scrubbed... URL: