Calendar Server has a feature for exchanging calendar data via email ("
iMIP"). This can be used to schedule with people who don't have accounts on your server.
In Server.app, there's a checkbox sets the state of this feature: "Enable invitations by email". Turn it off and calendar server should stop trying to ping your email services.