[CalendarServer-users] DCS Production Deployment

Gaurav Jain monkeyfdude at gmail.com
Wed Feb 18 17:33:40 PST 2015


Thank you very much for all your support!

On Wed, Feb 18, 2015 at 4:49 PM, Andre LaBranche <dre at apple.com> wrote:

> Hi Rhodey,
>
> Comments inline.
>
> > On Feb 18, 2015, at 4:18 PM, rhodey <rhodey at anhonesteffort.org> wrote:
> >
> > I've been running DCS in production for the past ~year, at the moment my
> > deployment supports ~800 users. It took *a lot* of reading through doc
> > and code to feel even remotely comfortable with storing real data for
> > real people and I still don't feel all that great about it.
>
> We are slowly compiling actual documentation. I'll take this as feedback
> in support of accelerating this process.
>
> > ...
> >
> > 2. DCS uses the python twisted framework but at the same time it
> > doesn't-- calls to DirectoryService (regardless of the implementation)
> > are blocking 0.o
>
> I believe this is addressed in 6.0+ with the new twext.who stuff that is
> all async, iirc.
>
> > 3. DCS has this weird multi-process, master-slave concept where multiple
> > DCS instances run on the same box-- I just think this is weird and
> > dealing with dead processes has been strange.
>
> The Python GIL means we need a multi process architecture to saturate CPU
> on multi core systems. Technically these processes are all part of the same
> service instance because they all cooperate on doing the work.
>
> > 4. If a user uploads a contact containing a photo and you're using a
> > database for storage (as you should), DCS will store that photo in your
> > database-- yuck! Even with your 1000 users, each with maybe 100 contact
> > photos, things get kinda bloated. So, I modified DCS to strip contact
> > photos from contacts and store them in Amazon S3, then piece them back
> > together on demand.
>
> Cool hack :)
>
> > 5. Someone could DOS my DCS instances via network resource exhaustion
> > very easily and there is very little that I could do to stop it other
> > than launch a crazy number of EC2 instances and burn through money. DCS
> > is not efficient (blocking calls to DirectoryService, etc) and I feel
> > very uncomfortable having any less than than two m3.medium instances
> > running at all times (costly!).
>
> Understood, however this is true of any network service that limits its
> own concurrency (imo this is required of any sane service). Part of the
> challenge is the extremely large range of request cost for legitimate
> CalDAV requests. We are moving to a model where we do more work a
> synchronously in a database work queue, and we now have a notion of cost in
> that implementation, so we are becoming better equipped to address resource
> utilization problems.
>
> > All that being said, DCS is still the best free and open source option
> > out there but I wish I had written my own WebDAV server from scratch
> > using 'dropwizard' instead. The world of high availability services
> > changes so rapidly and many of the ideas in this project are dated--
> > like, even table joins really shouldn't exist anymore.
>
> We're not yet ready for NoSQL but maybe someday :)
>
> Thanks for the feedback. I'm interested to hear anything else you'd like
> to share about your experiences hosting DCS.
>
> -dre
>
> >
> > -- rhodey orbits
> >
> >> On 02/18/2015 01:04 PM, Gaurav Jain wrote:
> >> Hi,
> >>
> >> Thank you very much for CalendarServer
> >>
> >> I have few questions:
> >>
> >> * I used CalendarServer-5.2.2 during development phase.
> >> * Now I want to use it for some real (users) use cases.
> >> * Right now, I do not have any scaling issues.
> >> * I run it on Ubuntu EC2 instance.
> >>
> >> My question is:
> >>
> >> * Has CalendarSever-5.2.2 been used in PROD deployments?
> >> * Since I don't have scaling issues, is it ok to use XMLDirectoryService
> >> for like 1000 accounts?
> >>
> >> Any tip would be useful for PROD deployments. Even for 1000 users, we
> would
> >> like it not to go down at all.
> >>
> >>
> >> Best Regards,
> >> Gaurav Jain
> >>
> >>
> >>
> >> _______________________________________________
> >> calendarserver-users mailing list
> >> calendarserver-users at lists.macosforge.org
> >> https://lists.macosforge.org/mailman/listinfo/calendarserver-users
> > _______________________________________________
> > calendarserver-users mailing list
> > calendarserver-users at lists.macosforge.org
> > https://lists.macosforge.org/mailman/listinfo/calendarserver-users
> _______________________________________________
> calendarserver-users mailing list
> calendarserver-users at lists.macosforge.org
> https://lists.macosforge.org/mailman/listinfo/calendarserver-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.macosforge.org/pipermail/calendarserver-users/attachments/20150218/0100445b/attachment-0001.html>


More information about the calendarserver-users mailing list