Thank you for the advice! Setting the BindAddress explicitly to the server's IPv4 address fixed the issue. I also went ahead and made a ticket for this on the website as you asked. For anyone curious, the caldavd.plist file looked like this with the BindAddress set