I have been able to compile libdispatch on Linux with a blocks-enabled clang and ran into some issues. The biggest problem is <unistd.h> in the GNU libc declares a function which takes an argument named "__block" which causes an error when compiling with Clang. This has been reported as a bug to GNU and rejected by drepper@ as a WONTFIX bug. I'll submit some patches soon to address these issues, but see below for a list of what I encountered and the "fix" used to get it to work. Regards, - Mark Build environment: clang version 1.1 (branches/release_27) Target: x86_64-pc-linux-gnu Build Commands: cd ~/src/libdispatch sh ./autogen.sh && CC=clang CFLAGS=-fPIC ./configure && make clean && make Problems: [1] __block used within unistd.h declaration In file included from ./internal.h:35: In file included from ../dispatch/dispatch.h:36: /usr/include/unistd.h:1128:35: error: __block attribute not allowed, only allowed on local variables extern void encrypt (char *__block, int __edflag) __THROW __nonnull ((1)); ^ fix: edit unistd.h and remove the __block identifier see: http://sources.redhat.com/bugzilla/show_bug.cgi?id=11157 [2] two private_extern variables cause compile failures. /usr/bin/ld: .libs/libdispatch_la-queue_kevent.o: relocation R_X86_64_PC32 against symbol `_dispatch_safe_fork' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: final link failed: Bad value fix: remove __private_extern__ from variable declarations [3] clang wants arc4random defined fix: add u_int32_t arc4random(); to source [4] clang doesn't like __unused dispatch_after.c:32:21: error: expected ')' void done(void *arg __unused) { ^ dispatch_after.c:32:10: note: to match this '(' void done(void *arg __unused) { ^ fix: delete this attribute [5] fabs() undefined symbol dispatch_drift.o: In function `__main_block_invoke_': dispatch_drift.c:(.text+0x529): undefined reference to `fabs' fix: add -lm to testing/Makefile.am [6] fgetln() not declared summarize.c:69:8: warning: implicit declaration of function 'fgetln' is invalid in C99 [-Wimplicit-function-declaration] ln = fgetln(stdin, &len); fix: add declaration: char *fgetln(FILE *fp, size_t *lenp); [7] Dispatch Source Read test failed ================================================== [TEST] Dispatch Source Read [PID] 8515 ================================================== Actual: 0x603200 Expected: 0x603200 [PASS] dispatch_get_main_queue Actual: 0x12dd010 Expected: 0x12dd010 [PASS] DISPATCH_SOURCE_TYPE_READ bytes available: 1 bytes read: 512000 ^^^^^ hung here, process 8515 was defunct and I killed it manually /bin/bash: line 5: 8515 Terminated ${dir}$tst FAIL: dispatch_read fix: none as of yet
Oh well time to use a different libc? I hear FreeBSD has a nice one! :-) On Monday, June 7, 2010, Mark Heily <mark@heily.com> wrote:
I have been able to compile libdispatch on Linux with a blocks-enabled clang and ran into some issues. The biggest problem is <unistd.h> in the GNU libc declares a function which takes an argument named "__block" which causes an error when compiling with Clang. This has been reported as a bug to GNU and rejected by drepper@ as a WONTFIX bug.
I'll submit some patches soon to address these issues, but see below for a list of what I encountered and the "fix" used to get it to work.
Regards,
- Mark
Build environment:
clang version 1.1 (branches/release_27) Target: x86_64-pc-linux-gnu
Build Commands:
cd ~/src/libdispatch sh ./autogen.sh && CC=clang CFLAGS=-fPIC ./configure && make clean && make
Problems:
[1] __block used within unistd.h declaration
In file included from ./internal.h:35: In file included from ../dispatch/dispatch.h:36: /usr/include/unistd.h:1128:35: error: __block attribute not allowed, only allowed on local variables extern void encrypt (char *__block, int __edflag) __THROW __nonnull ((1)); ^
fix: edit unistd.h and remove the __block identifier
see: http://sources.redhat.com/bugzilla/show_bug.cgi?id=11157
[2] two private_extern variables cause compile failures.
/usr/bin/ld: .libs/libdispatch_la-queue_kevent.o: relocation R_X86_64_PC32 against symbol `_dispatch_safe_fork' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: final link failed: Bad value
fix: remove __private_extern__ from variable declarations
[3] clang wants arc4random defined
fix: add u_int32_t arc4random(); to source
[4] clang doesn't like __unused
dispatch_after.c:32:21: error: expected ')' void done(void *arg __unused) { ^ dispatch_after.c:32:10: note: to match this '(' void done(void *arg __unused) { ^
fix: delete this attribute
[5] fabs() undefined symbol
dispatch_drift.o: In function `__main_block_invoke_': dispatch_drift.c:(.text+0x529): undefined reference to `fabs'
fix: add -lm to testing/Makefile.am
[6] fgetln() not declared
summarize.c:69:8: warning: implicit declaration of function 'fgetln' is invalid in C99 [-Wimplicit-function-declaration] ln = fgetln(stdin, &len);
fix: add declaration: char *fgetln(FILE *fp, size_t *lenp);
[7] Dispatch Source Read test failed
================================================== [TEST] Dispatch Source Read [PID] 8515 ==================================================
Actual: 0x603200 Expected: 0x603200 [PASS] dispatch_get_main_queue Actual: 0x12dd010 Expected: 0x12dd010 [PASS] DISPATCH_SOURCE_TYPE_READ bytes available: 1 bytes read: 512000 ^^^^^ hung here, process 8515 was defunct and I killed it manually /bin/bash: line 5: 8515 Terminated ${dir}$tst FAIL: dispatch_read
fix: none as of yet
_______________________________________________ libdispatch-dev mailing list libdispatch-dev@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/libdispatch-dev
On 06/08/2010 04:16 AM, David Leimbach wrote:
Oh well time to use a different libc? I hear FreeBSD has a nice one! :-)
No, time to fix clang. See my answer to http://sources.redhat.com/bugzilla/show_bug.cgi?id=11157 Paolo
On Tue, Jun 8, 2010 at 1:06 AM, Paolo Bonzini <bonzini@gnu.org> wrote:
On 06/08/2010 04:16 AM, David Leimbach wrote:
Oh well time to use a different libc? I hear FreeBSD has a nice one! :-)
No, time to fix clang. See my answer to
http://sources.redhat.com/bugzilla/show_bug.cgi?id=11157
Paolo
It seems pretty cut and dry to me. 1. C99 defines that double underscored tokens are reserved identifiers in the implementation of C99. 2. C99 is defined by both the compiler and the standard C libraries defined in C99. 3. unistd.h is not part of C99. Unistd.h is therefore clashing with a C99's totally valid usage of a double underscored and reserved name. Yes, unistd.h is shipping with a libc implementation, but nowhere does it say in C99 that that's an excuse for utilizing the C99 implementation's reserved namespace. One could question whether it is a good idea or not to tell the users of your C99 implementation to have to use extensions to the language via the use of double underscored names, but it really does say in C99 that these are there for ANY use by the implementation. This means that if you see double underscore'd identifiers in a user's code, that this code is at worst "not portable to other compiler environments". Dave
It seems pretty cut and dry to me.
1. C99 defines that double underscored tokens are reserved identifiers in the implementation of C99. 2. C99 is defined by both the compiler and the standard C libraries defined in C99. 3. unistd.h is not part of C99.
You know perfectly that the same thing could have happened with __block was used in stdlib.h (libdispatch is using _GNU_SOURCE, so it is allowing explicitly to declare non-C99 things in stdlib.h). So this is at best a strawman. BTW, from your favorite libc's _ctype.h: static __inline int __istype(__ct_rune_t c, unsigned long _f) { return (!!__maskrune(_c, _f)); } __istype looks like a nice candidate for a new compiler keyword...
Unistd.h is therefore clashing with a C99's totally valid usage of a double underscored and reserved name. Yes, unistd.h is shipping with a libc implementation, but nowhere does it say in C99 that that's an excuse for utilizing the C99 implementation's reserved namespace.
The alternative is not using double underscores and breaking on anyone using "#define block blah". What kind of conflict do you think is more common? User macros or extended keywords?
One could question whether it is a good idea or not to tell the users of your C99 implementation to have to use extensions to the language via the use of double underscored names, but it really does say in C99 that these are there for ANY use by the implementation.
True, but everything IMO is against clang in this circumstance: 1) GCC has followed the standard of using __keyword__, instead leaving __keyword to libc and libstdc++. __complex__, __asm__, __attribute__, you name it. No reason why clang should not have done the same for __block. It wouldn't have clashed with glibc. 2) past experience has suggested a workaround, namely "fixincludes". Currently clang is being sloppy in this respect; it can afford that because it only supports a few targets (basically Linux and Mac OS X). It's a young project, so I don't have anything against that. But sooner or later you will have to deal with the consequences of not controlling the whole C99/POSIX environment.
This means that if you see double underscore'd identifiers in a user's code, that this code is at worst "not portable to other compiler environments".
libc is not user code. Paolo
On Tue, Jun 8, 2010 at 9:12 AM, Paolo Bonzini <bonzini@gnu.org> wrote:
It seems pretty cut and dry to me.
1. C99 defines that double underscored tokens are reserved identifiers in the implementation of C99. 2. C99 is defined by both the compiler and the standard C libraries defined in C99. 3. unistd.h is not part of C99.
You know perfectly that the same thing could have happened with __block was used in stdlib.h (libdispatch is using _GNU_SOURCE, so it is allowing explicitly to declare non-C99 things in stdlib.h). So this is at best a strawman.
This is not a strawman because it is not an argument. I'm telling you that by definition, C99 says double underscore names are for the implementation. It's also not arguable that unistd.h is not part of C99's standard library by definition, and therefore had no business using double underscored names to begin with. If we were talking about stdlib.h, it would actually be a different story. I'm not the one who drew these seemingly arbitrary lines, but they exist all the same. It may very well be that the folks sitting on the committee sometimes are out of touch with reality, and don't realize that C99 compilers don't always ship with exactly their own libc, and that libc libraries often are scoped to handle far more than just the C99 defined standard library, but the point of the standard is try to keep things portable, and to lay down some rules for users and implementors to collaborate and work together. Someone's not following the rules, and it's not the clang or llvm teams.
BTW, from your favorite libc's _ctype.h:
static __inline int __istype(__ct_rune_t c, unsigned long _f) { return (!!__maskrune(_c, _f)); }
__istype looks like a nice candidate for a new compiler keyword...
Unfortunately for your counter-example, ctype.h is part of C99, and therefore a use of double underscored names is valid within that context as far as I can tell.
Unistd.h is therefore clashing with a C99's totally valid usage of a
double underscored and reserved name. Yes, unistd.h is shipping with a libc implementation, but nowhere does it say in C99 that that's an excuse for utilizing the C99 implementation's reserved namespace.
The alternative is not using double underscores and breaking on anyone using "#define block blah". What kind of conflict do you think is more common? User macros or extended keywords?
It's irrelvant what's more common if you're looking at this from the perspective of someone trying to comply with C99. If you're looking at this situation from the perspective of what changes will affect the most users, then glibc still loses most likely because I bet the number of people with Clang -fblock enabled code outnumbers the people maintaining glibc and using the __block definition (which is illegal by C99) from unistd.h. I didn't draw the lines, so if they don't make sense, do not blame me, but they're there to help people collaborate and write portable code. Claiming that Apple and the LLVM team is being a poor collaborator for using __block is hard to maintain when that entire namespace was allotted to them by the C99 standards committee.
One could question whether it is a good idea or not to tell the users of
your C99 implementation to have to use extensions to the language via the use of double underscored names, but it really does say in C99 that these are there for ANY use by the implementation.
True, but everything IMO is against clang in this circumstance:
1) GCC has followed the standard of using __keyword__, instead leaving __keyword to libc and libstdc++. __complex__, __asm__, __attribute__, you name it. No reason why clang should not have done the same for __block. It wouldn't have clashed with glibc.
I disagree with your claim that "everything is against clang", but you make
a valid point all the same. GCC is a de-facto standard due to its ubiquity and longevity, and there's some value in tracking this, as Intel has realized with their compiler toolchain. And you're correct that Clang could have done this for __block, but the primary target platform was *not* glibc for this stuff. Maybe it could easily be __block__ going forward, but don't you think that's going to change a lot of people's libdispatch related code just to make things compatible with unistd.h from glibc? Isn't that worse than fixing glibc's header to comply with C99? As you say below, it's a young implementation, and possibly easier to change that unistd.h. All I know about the lack of desire to make the change in unistd.h for glibc is that Ulrich Drepper doesn't want to do it, and believes incorrectly that the Clang and LLVM develoeprs "stole" something that he thought was his, when it really wasn't. 2) past experience has suggested a workaround, namely "fixincludes".
Currently clang is being sloppy in this respect; it can afford that because it only supports a few targets (basically Linux and Mac OS X). It's a young project, so I don't have anything against that. But sooner or later you will have to deal with the consequences of not controlling the whole C99/POSIX environment.
fixincludes was to deal with non-ANSI headers right? How is that going to help with the __block keyword? I can't find the Block implementation text that used to be at clang.llvm.org anymore as the site has moved stuff around or deleted the file, but my impression is it's a keyword, not a macro.
This means that if
you see double underscore'd identifiers in a user's code, that this code is at worst "not portable to other compiler environments".
libc is not user code.
From one perspective you're correct, however unistd.h is a user of the standard C library code, and should not have used double underscore identifiers per the C99 standard. They're ALL reserved.
Dave
Paolo
On 06/08/2010 07:27 PM, David Leimbach wrote:
This is not a strawman because it is not an argument. I'm telling you that by definition, C99 says double underscore names are for the implementation. It's also not arguable that unistd.h is not part of C99's standard library by definition, and therefore had no business using double underscored names to begin with.
Ok, it is not C99. It's POSIX. Come on. You're "drawing the line" in the most unreasonable place. Anyway.
BTW, from your favorite libc's _ctype.h:
static __inline int __istype(__ct_rune_t c, unsigned long _f) { return (!!__maskrune(_c, _f)); }
__istype looks like a nice candidate for a new compiler keyword...
Unfortunately for your counter-example, ctype.h is part of C99, and therefore a use of double underscored names is valid within that context as far as I can tell.
No, it is _ctype.h with the underscore. Nowhere in that file I read that it is internal use only. Anyway, 4 seconds of grep and here is this from FreeBSD pthread.h: #define pthread_cleanup_push(cleanup_routine, cleanup_arg) \ { \ struct _pthread_cleanup_info __cleanup_info__; \ __pthread_cleanup_push_imp(cleanup_routine, cleanup_arg, \ &__cleanup_info__); \ { #define pthread_cleanup_pop(execute) } \ __pthread_cleanup_pop_imp(execute); \ } Tada, __cleanup_info__. BTW pthread_cleanup_{push,pop} just _cannot_ be implemented without macros of this kind, so pthreads cannot be implemented without either following your interpretation of the C99 standard, or breaking the user's namespace.
Claiming that Apple and the LLVM team is being a poor collaborator for using __block is hard to maintain when that entire namespace was allotted to them by the C99 standards committee.
I never said that. I said that the choice of __block without trailing underscores was poor. Nobody's perfect, GCC did the same for __thread, but guess what, there is a fixincludes for that: /* * __thread is now a keyword. */ fix = { hackname = thread_keyword; files = "pthread.h"; files = "bits/sigthread.h"; select = "([* ])__thread([,)])"; c_fix = format; c_fix_arg = "%1__thr%2"; };
As you say below, it's a young implementation, and possibly easier to change that unistd.h.
No, it's set in stone now. I'm not suggesting changing it. I'm just suggesting that for the first time clang is showing the need for fixincludes.
Maybe it could easily be __block__ going forward, but don't you think that's going to change a lot of people's libdispatch related code just to make things compatible with unistd.h from glibc?
I never proposed that.
Isn't that worse than fixing glibc's header to comply with C99?
No need to say that again. Repeating the same argument sounds like the preferred strategy of people writing to glibc bugzilla, just to see Drepper getting upset. But we're not on glibc bugzilla, and I'm not Drepper.
2) past experience has suggested a workaround, namely "fixincludes". Currently clang is being sloppy in this respect; it can afford that because it only supports a few targets (basically Linux and Mac OS X). It's a young project, so I don't have anything against that. But sooner or later you will have to deal with the consequences of not controlling the whole C99/POSIX environment.
fixincludes was to deal with non-ANSI headers right? How is that going to help with the __block keyword?
Nowadays fixincludes nowadays deals with all sort of problems. It is even fixing old glibc to cope with removed extensions, or backwards-incompatible changes in the compiler (in addition to the above example with __thread, I can count two cases: C99 inline semantics, and removal of lvalue casts). It can run arbitrary substitutions and sed scripts. Here is another example: /* * Incorrect sprintf declaration in X11/Xmu.h */ fix = { hackname = x11_sprintf; files = X11/Xmu.h; files = X11/Xmu/Xmu.h; select = "^extern char \\*\tsprintf\\(\\);$"; c_fix = format; c_fix_arg = "#ifndef __STDC__\n%0\n#endif /* !defined __STDC__ */"; test_text = "extern char *\tsprintf();"; }; Paolo
Well I guess we don't agree at all :-). Have a nice day. On Tue, Jun 8, 2010 at 11:49 AM, Paolo Bonzini <bonzini@gnu.org> wrote:
On 06/08/2010 07:27 PM, David Leimbach wrote:
This is not a strawman because it is not an argument. I'm telling you that by definition, C99 says double underscore names are for the implementation. It's also not arguable that unistd.h is not part of C99's standard library by definition, and therefore had no business using double underscored names to begin with.
Ok, it is not C99. It's POSIX. Come on. You're "drawing the line" in the most unreasonable place. Anyway.
BTW, from your favorite libc's _ctype.h:
static __inline int __istype(__ct_rune_t c, unsigned long _f) { return (!!__maskrune(_c, _f)); }
__istype looks like a nice candidate for a new compiler keyword...
Unfortunately for your counter-example, ctype.h is part of C99, and therefore a use of double underscored names is valid within that context as far as I can tell.
No, it is _ctype.h with the underscore. Nowhere in that file I read that it is internal use only. Anyway, 4 seconds of grep and here is this from FreeBSD pthread.h:
#define pthread_cleanup_push(cleanup_routine, cleanup_arg) \ { \ struct _pthread_cleanup_info __cleanup_info__; \ __pthread_cleanup_push_imp(cleanup_routine, cleanup_arg, \ &__cleanup_info__); \ {
#define pthread_cleanup_pop(execute) } \ __pthread_cleanup_pop_imp(execute); \ }
Tada, __cleanup_info__. BTW pthread_cleanup_{push,pop} just _cannot_ be implemented without macros of this kind, so pthreads cannot be implemented without either following your interpretation of the C99 standard, or breaking the user's namespace.
Claiming that Apple and the LLVM team is being a poor collaborator for
using __block is hard to maintain when that entire namespace was allotted to them by the C99 standards committee.
I never said that. I said that the choice of __block without trailing underscores was poor. Nobody's perfect, GCC did the same for __thread, but guess what, there is a fixincludes for that:
/* * __thread is now a keyword. */ fix = { hackname = thread_keyword; files = "pthread.h"; files = "bits/sigthread.h"; select = "([* ])__thread([,)])"; c_fix = format; c_fix_arg = "%1__thr%2";
};
As you say below, it's a young implementation, and possibly easier to change that unistd.h.
No, it's set in stone now. I'm not suggesting changing it. I'm just suggesting that for the first time clang is showing the need for fixincludes.
Maybe it could easily be __block__ going forward, but don't you think
that's going to change a lot of people's libdispatch related code just to make things compatible with unistd.h from glibc?
I never proposed that.
Isn't that worse
than fixing glibc's header to comply with C99?
No need to say that again. Repeating the same argument sounds like the preferred strategy of people writing to glibc bugzilla, just to see Drepper getting upset. But we're not on glibc bugzilla, and I'm not Drepper.
2) past experience has suggested a workaround, namely "fixincludes".
Currently clang is being sloppy in this respect; it can afford that because it only supports a few targets (basically Linux and Mac OS X). It's a young project, so I don't have anything against that. But sooner or later you will have to deal with the consequences of not controlling the whole C99/POSIX environment.
fixincludes was to deal with non-ANSI headers right? How is that going to help with the __block keyword?
Nowadays fixincludes nowadays deals with all sort of problems. It is even fixing old glibc to cope with removed extensions, or backwards-incompatible changes in the compiler (in addition to the above example with __thread, I can count two cases: C99 inline semantics, and removal of lvalue casts). It can run arbitrary substitutions and sed scripts. Here is another example:
/* * Incorrect sprintf declaration in X11/Xmu.h */ fix = { hackname = x11_sprintf; files = X11/Xmu.h; files = X11/Xmu/Xmu.h; select = "^extern char \\*\tsprintf\\(\\);$";
c_fix = format; c_fix_arg = "#ifndef __STDC__\n%0\n#endif /* !defined __STDC__ */";
test_text = "extern char *\tsprintf();"; };
Paolo
Or we could just come to the conclusion that the time is simply not yet right to add Linux to the supported OS list. From what I can read between the lines on the discussion thread in that bug, there doesn't seem to be much interest in clang, or even in merging block support into whatever branch of gcc linux follows, and the ultimate goal of any porting work we do (or incorporate) is to see the technology adopted more along the lines of what FreeBSD is on schedule to do with their 8.1 release. If there's already this much push-back on a simple header change then that tends to suggest that there will be even more insurmountable obstacles just around the corner as we move on to the stuff that's actually *hard*, like kernel support. :-) My vote would be to focus on platforms who's user communities display an active interest in the GCD API. We're not here to do a "hard sell" on it since that's just a good way to alienate folks who might someday, under different circumstances, be interested in what it has to offer. Regards, - Jordan On Jun 8, 2010, at 1:06 AM, Paolo Bonzini wrote:
On 06/08/2010 04:16 AM, David Leimbach wrote:
Oh well time to use a different libc? I hear FreeBSD has a nice one! :-)
No, time to fix clang. See my answer to
http://sources.redhat.com/bugzilla/show_bug.cgi?id=11157
Paolo _______________________________________________ libdispatch-dev mailing list libdispatch-dev@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/libdispatch-dev
On 06/08/2010 08:40 PM, Jordan K. Hubbard wrote:
Or we could just come to the conclusion that the time is simply not yet right to add Linux to the supported OS list.
I agree, but it's nice to have it build for people who want to experiment. Have you seen the attached patch that fixes libdispatch? Paolo
participants (4)
-
David Leimbach
-
Jordan K. Hubbard
-
Mark Heily
-
Paolo Bonzini