February 2013 - calendarserver-dev - lists.macosforge.org

Debian updated package 4.2
by Fredrik Unger 05 Mar '14

05 Mar '14

Hi, I updated my personal package to 4.2. https://tree.se/debian/ There were three minor changes, fixing a few SQL statements for 0.1.4, adding python-psutil as dependency, debianizing PGSOCKETDIR. I will improve on packaging, this was the first attempt. The server starts and http://localhost:8008/ works in the browser. Not tested any further. Not a SAFE upgrade path, and it might eat your cat. It is for testing on a empty server without any important data. With regards to the SQL patch. Looking at trunk some of these SQL statements are OK for 0.1.4, some are not. It is the same problems with 'primary key(' and 'unique(' as in Ticket #759. http://trac.calendarserver.org/ticket/759 Example from current trunk (r10297) is line 186 vs 192. http://trac.calendarserver.org/browser/CalendarServer/trunk/txdav/common/da… Still not a big deal and calendar server does not support 0.1.4, but 0.1.2 http://trac.calendarserver.org/browser/CalendarServer/trunk/support/build.s… Not sure it makes sense to open a bug against the SQL issue as new code got added with 'primary key(' sql. It is after all a sqlparse bug. Better for packagers to patch it I think, as the distribution is using the "wrong" version. To fix it is just to add a space, and the issues is now well known. Thanks /Fred

3 6

Re: [CalendarServer-dev] Exception in recvfd --was: Re: continuing work on FreeBSD ports
by Glyph 27 Feb '13

27 Feb '13

On Feb 27, 2013, at 1:17 PM, Axel Rau <Axel.Rau(a)Chaos1.DE> wrote: > > Am 26.02.2013 um 00:39 schrieb glyph(a)twistedmatrix.com: > >> There are no calls to 'sendmsg()' in this trace, that I can see: >> >> glyph@rem:~/Downloads/2013.02.21$ grep 'sendmsg(' debug.trace >> [Error: 1] >> glyph@rem:~/Downloads/2013.02.21$ grep 'recvmsg(' debug.trace >> 50395: 3.830958519 recvmsg(0x19,0x7fffffffab30,0x0,0x0,0x1010,0x7fffffffa760) = 1 (0x1) >> 50395: 3.945254507 recvmsg(0x1b,0x7fffffffab30,0x0,0x0,0x1010,0x7fffffffa760) = 1 (0x1) >> glyph@rem:~/Downloads/2013.02.21$ > Yes, as explained earlier, my patch prevents sendmsg from seing I ... don't think I understood the earlier explanation. >> Can you include the master process as well? > I think, it's included (50395). Sorry, I meant, can you include a dtruss of the master process as well, rather than instrumenting with a debug patch? At this point it might be more effort than it's worth, since I think I know what's going on. >>> This version hangs after the accept() instead of segfaulting. >> >> Hangs *after* the accept(), not *in* the accept()? > As 50395 continues after the accept(), 'hang' is the wrong word. > I just see no http answer on the client side. Right, and that makes sense, since the FD never properly gets to the worker process (which is where all the actually HTTP traffic happens). > I think we should focus on the original problem This is all related, I'm still concerned about the original problem :). -glyph

1 0

continuing work on FreeBSD ports
by Axel Rau 27 Feb '13

27 Feb '13

Hi, I'm trying to complete Rolands work on caldavports https://github.com/rvanlaar/caldavports Currently I'm using the 4.2 release of dcs with twisted 12.1 and all dependencies are installed through the ports system (pycalendar and py-psutil had to be upgraded to more recent versions). So far the server is running in a fbsd jail with the db on a different host and I have created a rc.script. I'm new to dcs and struggling with the configuration. Questions: 1. Do I still need 2 separate run/config environments in order to run both a caldav and a cardav server (i.e. EnableCalDAV and EnableCardDAV can't both be true)? 2. Is it correct that IPv6 is not (yet) supported? Thanks, Axel --- PGP-Key:29E99DD6 ☀ +49 151 2300 9283 ☀ computing @ chaos claudius

6 29

Re: [CalendarServer-dev] Ldap users are showing, auth fails
by Morgen Sagen 26 Feb '13

26 Feb '13

If you are using LDAP, try disabling Digest auth in caldavd plist and enable Basic auth (and be sure to turn on SSL). On Feb 23, 2013, at 2:58 PM, Marc Roos <M.Roos(a)f1-outsourcing.eu> wrote: > > I have managed to display the users from ldap on > http://192.168.11.5:8008/addressbooks/users/. But I cannot get an > authentication. Is this somewhere else configured? > > Thanks in advance, > Marc > > > > > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -. > F1 Outsourcing Development Sp. z o.o. > Poland > > t: +48 (0)124466845 > f: +48 (0)124466843 > e: marc(a)f1-outsourcing.eu > > > _______________________________________________ > calendarserver-dev mailing list > calendarserver-dev(a)lists.macosforge.org > https://lists.macosforge.org/mailman/listinfo/calendarserver-dev

1 0

Re: [CalendarServer-dev] Different base dn for users , groups and resources
by Andre LaBranche 25 Feb '13

25 Feb '13

On Feb 21, 2013, at 8:01 AM, Marc Roos <M.Roos(a)f1-outsourcing.eu> wrote: > > > Is it possible to configure different base dn's for users and groups? > I have them stored at different locations in the ldap directory and I > dont want to give to much rights to the binddn Hi, Sorry for the late reply. This might work. The effective DN used by Calendar Server for a given record type is derived from the base DN and the rdn for each record type, as configured in caldavd.plist. So for example, if your ldap config has: base : dc=example,dc=com users rdn: ou=People This is (should be) effectively the same as: base : dc=example users rdn: dc=com,ou=People As long as you have at least one common top-level component in the 'path' to your record types, this might be an effective strategy. Let us know how it works out. HTH, -dre > > Thanks in advance, > Marc > > > > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -. > F1 Outsourcing Development Sp. z o.o. > Poland > > t: +48 (0)124466845 > f: +48 (0)124466843 > e: marc(a)f1-outsourcing.eu > > > -----Original Message----- > From: Axel Rau [mailto:Axel.Rau@Chaos1.DE] > Sent: woensdag 20 februari 2013 13:13 > To: calendarserver-dev(a)lists.macosforge.org > Subject: Re: [CalendarServer-dev] Exception in recvfd --was: Re: > continuing work on FreeBSD ports > > > Am 20.02.2013 um 10:51 schrieb Axel Rau: > >>> >>> Like... no ancillary data is being *passed to sendmsg*? >> Yes. My debug print at the beginning of sendmsg always shows 0. >>> Or no data is being received from recvmsg? Does tracing the relevant > processes with something like strace or dtruss confirm this? What > process is being debugged here; master, worker, or both? >> Both, because the unpack error pointed at a platform/compatibility > issue in sendmsg.c, which is used by all parts. >> I will try to get a trace with dtruss. > truss trace of the handling of the http accept attached. >>> >>> The debug messages which you've added seem like they might be missing > some data, since they just truncate after 'return from sendmsg with', > rather than printing a string (even an empty one) after. >> Yes, the code was not finished. should display sendmsg_result, which > was always 1. See attached diff. >>> >>>> Without ancillary data, no fd can be shipped. >>>> The only place where ancillary data could be provided seems to be >>>> twext.internet.sendfdport._SubprocessSocket.doWrite. >>>> I would be happy for any hint where to dig... >>> >>> Well, hrm. This code works fine on other platforms, so, one thing to > check would be to see if previous versions of the FreeBSD kernel have > the same issue. What exact versions of everything are you testing? >> 8.2 and 9.1. Both show same bug. > Both platforms are 64-bit. > > Axel > > > > _______________________________________________ > calendarserver-dev mailing list > calendarserver-dev(a)lists.macosforge.org > https://lists.macosforge.org/mailman/listinfo/calendarserver-dev

1 0

Ldap users are showing, auth fails
by Marc Roos 23 Feb '13

23 Feb '13

I have managed to display the users from ldap on http://192.168.11.5:8008/addressbooks/users/. But I cannot get an authentication. Is this somewhere else configured? Thanks in advance, Marc - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -. F1 Outsourcing Development Sp. z o.o. Poland t: +48 (0)124466845 f: +48 (0)124466843 e: marc(a)f1-outsourcing.eu

1 0

FW: Different base dn for users , groups and resources
by Marc Roos 21 Feb '13

21 Feb '13

Is it possible to configure different base dn's for users and groups? I have them stored at different locations in the ldap directory and I dont want to give to much rights to the binddn Thanks in advance, Marc - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -. F1 Outsourcing Development Sp. z o.o. Poland t: +48 (0)124466845 f: +48 (0)124466843 e: marc(a)f1-outsourcing.eu -----Original Message----- From: Axel Rau [mailto:Axel.Rau@Chaos1.DE] Sent: woensdag 20 februari 2013 13:13 To: calendarserver-dev(a)lists.macosforge.org Subject: Re: [CalendarServer-dev] Exception in recvfd --was: Re: continuing work on FreeBSD ports Am 20.02.2013 um 10:51 schrieb Axel Rau: >> >> Like... no ancillary data is being *passed to sendmsg*? > Yes. My debug print at the beginning of sendmsg always shows 0. >> Or no data is being received from recvmsg? Does tracing the relevant processes with something like strace or dtruss confirm this? What process is being debugged here; master, worker, or both? > Both, because the unpack error pointed at a platform/compatibility issue in sendmsg.c, which is used by all parts. > I will try to get a trace with dtruss. truss trace of the handling of the http accept attached. >> >> The debug messages which you've added seem like they might be missing some data, since they just truncate after 'return from sendmsg with', rather than printing a string (even an empty one) after. > Yes, the code was not finished. should display sendmsg_result, which was always 1. See attached diff. >> >>> Without ancillary data, no fd can be shipped. >>> The only place where ancillary data could be provided seems to be >>> twext.internet.sendfdport._SubprocessSocket.doWrite. >>> I would be happy for any hint where to dig... >> >> Well, hrm. This code works fine on other platforms, so, one thing to check would be to see if previous versions of the FreeBSD kernel have the same issue. What exact versions of everything are you testing? > 8.2 and 9.1. Both show same bug. Both platforms are 64-bit. Axel

1 0

Re: [CalendarServer-dev] twistedcaldav/directory/ldapdirectory.py:1267:getGroups empty argument error
by Dennison Williams 14 Feb '13

14 Feb '13

Bringing this back onto the list in case its useful to someone else. Inline below. On 02/08/2013 06:51 PM, Morgen Sagen wrote: > > On Feb 8, 2013, at 4:45 PM, Dennison Williams <dennison.williams(a)gmail.com> wrote: > >> On 02/08/2013 05:55 PM, Morgen Sagen wrote: >>> That patch is not what you want. getGroups( ) should not be getting passed a set([None]) to begin with, and it's likely due to a configuration problem. >> >> Ok ... got it. >> >>> >>> 1) Do you have proxy assignments, i.e. does proxies.sqlite have anything in the GROUPS table? You should also see the number of proxy assignments logged in the error.log with an "info" level line like: "There are 0 proxies" >> >> In the log there is a line: >> [twistedcaldav.directory.directory.GroupMembershipCacheUpdater#info] >> There are 1 proxies >> >> When I try to browse this with the sqlite utility I get >> >> sqlite /var/lib/caldavd/proxies.sqlite >> Unable to open database "/var/lib/caldavd/proxies.sqlite": file is >> encrypted or is not a database > > Try using "sqlite3"? > >> >>> 2) Does your caldavd.plist have UseExternalProxies set to true? (It shouldn't. Set it to false. Setting it to true is an advanced config where proxy assignments can be controlled via LDAP.) >> >> Its set to false > > Good. > >> >>> 3) [My question from earlier] How does your LDAP schema specify group membership? How have you adjusted your caldavd.plist to match? (If you explain the former, I can help with the latter.) >> >> ldapsearch -x -h ad.meow.com -D 'auth' -w 'also_not_the_real_pass' -b >> 'cn=Users,dc=meow,dc=comm' '(&(objectClass=user)(cn=MeowGroup))' >> >> dn: CN=MeowGroup,CN=Users,DC=meow,DC=com >> objectClass: top >> objectClass: group >> cn: MeowGroup >> member: CN=Meow1,CN=Users,DC=meow,DC=com >> member: CN=Meow2,CN=Users,DC=meow,DC=com >> distinguishedName: CN=MeowGroup,CN=Users,DC=meow,DC=com >> instanceType: 4 >> whenCreated: 20111021025749.0Z >> whenChanged: 20120816194841.0Z >> uSNCreated: 12429 >> uSNChanged: 12429 >> name: Executive >> objectGUID:: n/mb9e13cki32CdHMCjdPA== >> objectSid:: AQUAAAAAAAUVAAAARUxc9755Z7MIG4EGcQQAAA== >> sAMAccountName: MeowGroup >> sAMAccountType: 268435456 >> groupType: -2147483646 >> objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=meow,DC=com >> dSCorePropagationData: 16010101000000.0Z >> >> Attached is my caldavd.plist > > Great, your caldavd.plist appears to match the schema. > > So I am really interested to see what's wrong with your proxy db. If that file is corrupt, you can just move that sqlite db aside and a new one will get created next time you start the server. I have attached the previous proxies.sqlite file. As you can see there is one row in the GROUPS table: sqlite> select * from GROUPS; None#calendar-proxy-write| Indeed deleting this DB and restarting the server caused the server to recreate a new empty DB. After a few restarts I got this working though. Thanks for the help > > >> >> >> >>> >>> ~morgen >>> >>> On Feb 8, 2013, at 2:53 PM, Dennison Williams <dennison.williams(a)gmail.com> wrote: >>> >>>> I think this is a bug, and I have a fix, but I am not sure if it is in >>>> the "right" place. Anyhow this allows calendarserver to at least to >>>> populate the Group Membership Cache in memcache. And start up the >>>> network listeners. >>>> >>>> I am now seeing other issues with starting the group caching service, so >>>> maybe this does not actually fix the problem. >>>> >>>> The two attached files are the patches, directory-4.2-20130208.patch >>>> against the 4.2 branch and directory-main-20130208.patch against the >>>> main development branch (checked out today). >>>> >>>> -Dennison >>>> >>>> On 01/30/2013 11:42 PM, Dennison Williams wrote: >>>>> On 01/30/2013 10:53 PM, Glyph wrote: >>>>>> >>>>>> On Jan 30, 2013, at 8:06 PM, Dennison Williams >>>>>> <dennison.williams(a)gmail.com <mailto:dennison.williams@gmail.com>> wrote: >>>>>> >>>>>>> Tracing this all the way up the stack I see tha the getGroups method >>>>>>> receives a guid value of set([None]), but this is not caught as I think >>>>>>> maybe it should be on line 675 >>>>>>> >>>>>>> if guids is None: >>>>>>> >>>>>>> But because I am not super familar with this application and have >>>>>>> limited familiarity with python I am not sure if this indicates an issue >>>>>>> with my config, my environment, or the code. >>>>>> >>>>>> The LDAP directory is incredibly flexible, and can be coerced to do >>>>>> various insane things by setting up mappings incorrectly. Having a copy >>>>>> of your caldavd.plist would be helpful when diagnosing this error. >>>>>> (Actually having a copy of your entire directory along with that would >>>>>> be even more useful, but I presume that isn't possible ;-)). >>>>> >>>>> Please see attached my caldavd.plist. I could also send a few sections >>>>> from an ldapsearch query if that was helpful >>>>>> >>>>>> -glyph >>>>> >>>> >>>> <directory-4.2-20130208.patch><directory-main-20130208.patch>_______________________________________________ >>>> calendarserver-dev mailing list >>>> calendarserver-dev(a)lists.macosforge.org >>>> https://lists.macosforge.org/mailman/listinfo/calendarserver-dev >>> >> >> <caldavd.plist> >

1 0

twisted.internet.addressalready loaded, cannot load required backport
by Dennison Williams 14 Feb '13

14 Feb '13

Hey all, Running on the 4.2 branch I am getting the following exception (see attached backtrace): exceptions.RuntimeError: twisted.internet.addressalready loaded, cannot load required backport This is repeated over and over again in the error log. I see that this is trigggered when call runtime patching the twisted library. I do not have the system python twisted library installed and I do not have a src install of the twisted lib. Only the one that gets installed when running: run.sh -i / Later on down in the input I also get the following exception (see second backtrace): exceptions.AttributeError: 'module' object has no attribute 'backport' Further on down in the file I get the following the help dump from twistd and the following message: /usr/local/bin/twistd: Unknown command: caldav Any help is appreciated. Sincerely, Dennison Williams

2 1

Re: [CalendarServer-dev] twistedcaldav/directory/ldapdirectory.py:1267:getGroups empty argument error
by Dennison Williams 08 Feb '13

08 Feb '13

I think this is a bug, and I have a fix, but I am not sure if it is in the "right" place. Anyhow this allows calendarserver to at least to populate the Group Membership Cache in memcache. And start up the network listeners. I am now seeing other issues with starting the group caching service, so maybe this does not actually fix the problem. The two attached files are the patches, directory-4.2-20130208.patch against the 4.2 branch and directory-main-20130208.patch against the main development branch (checked out today). -Dennison On 01/30/2013 11:42 PM, Dennison Williams wrote: > On 01/30/2013 10:53 PM, Glyph wrote: >> >> On Jan 30, 2013, at 8:06 PM, Dennison Williams >> <dennison.williams(a)gmail.com <mailto:dennison.williams@gmail.com>> wrote: >> >>> Tracing this all the way up the stack I see tha the getGroups method >>> receives a guid value of set([None]), but this is not caught as I think >>> maybe it should be on line 675 >>> >>> if guids is None: >>> >>> But because I am not super familar with this application and have >>> limited familiarity with python I am not sure if this indicates an issue >>> with my config, my environment, or the code. >> >> The LDAP directory is incredibly flexible, and can be coerced to do >> various insane things by setting up mappings incorrectly. Having a copy >> of your caldavd.plist would be helpful when diagnosing this error. >> (Actually having a copy of your entire directory along with that would >> be even more useful, but I presume that isn't possible ;-)). > > Please see attached my caldavd.plist. I could also send a few sections > from an ldapsearch query if that was helpful >> >> -glyph >

2 1