On Oct 17, 2007, at 15:53, Simon Ruderich wrote:
On Sun, Oct 14, 2007 at 10:32:20PM -0500, Ryan Schmidt wrote:
Very interesting and useful! Some random comments:
I hope you will regenerate this occasionally, at least those that are failing right now. How long did it take to generate that list?
It took my MacBook about 2-3 days which is quite long. I guess the whole port tree would need at least a week, maybe (much) more.
I figured it would take quite awhile.
It would help to know the date/time when you encountered each failure, and/or the revision of the portfile that was used.
Yes, this would be indeed very nice.
You may want to add a table of contents at the top so we can jump to specific ports.
Good idea, I will add this in the future.
You may want to sort the ports in some way. It currently seems to be somewhat alphabetical, but with other ports in between (maybe dependencies?).
I thought it's already sorted alphabetical. Maybe the problem is that ports with a uppercase name are sorted in an own group at the top, then all lowercased ports.
Well, build.html currently lists failures in this order: ArpSpyX gnustep-base DesktopManager gnustep-base gnustep-base GNUMail-Aqua HandBrake gnustep-base ID3 gnustep-base qt3-mac NotificationWatcher gnustep-base You see my confusion.
In build.html you have gnome-session, gedit, gnu-crypto, and gnustep-base listed several times. Probably others as well.
Yes, this is a problem with the parser I wrote to get the ports. I hope I can improve it.
I fixed iksemel's fetch failure. Note that you had iksemel listed build.html. You have other fetch failures listed in build.html (DarwinPortsStartup, cook, p5-email-address, etc.).
Thanks, I removed it from build.html. This is also a parser problem, sorry.
Lots of ports complaining about /opt-devel/bin/python2.4 being missing. Aren't those ports in the python portgroup? Wonder what's going on there.
Not sure either. I will look into it.
Some of your failures are suspicious. For example:
php5 does not fail under normal circumstances, so I'd like to know how your build farm is operating.
You report a gettext/libintl build failure for grep that I am unable to reproduce. The portfile includes the --disable-nls flag so I cannot imagine why it would be trying to link against gettext/libintl.
The reported build failure for ufraw ("Image error: /opt-devel/bin/ dcraw is being used by the active dcraw port") is curious since ufraw does not depend on dcraw (not even indirectly). Why was dcraw installed? I would think a build farm would need to build each port in a clean system.
As this was my first test I just executed "sudo port install all" and wrote a small script which parsed the output of this command. I know see how limited this approach was, but I'm already working on a better one.
A big problem for example is that some fetches are not failing after a timeout even if the server is not responding. I'm not sure how to fix this as an own timeout in my buildfarm would stop long downloads.
MacPorts uses curl to download. Curl has options that can be used to consider the download failed if the download speed drops below some threshold for some period of time. If MacPorts is not currently using this option, perhaps it should. It would alleviate this problem. Here's how it's done on the command line (from "man curl"): --connect-timeout <seconds> Maximum time in seconds that you allow the connection to the server to take. This only limits the connection phase, once curl has connected this option is of no more use. See also the -m/--max-time option. -y/--speed-time <time> If a download is slower than speed-limit bytes per second during a speed-time period, the download gets aborted. If speed-time is used, the default speed-limit will be 1 unless set with -y. This option controls transfers and thus will not affect slow connects etc. If this is a concern for you, try the --connect- timeout option. -Y/--speed-limit <speed> If a download is slower than this given speed, in bytes per sec- ond, for speed-time seconds it gets aborted. speed- time is set with -Y and is 30 if not set.
My current idea is to get a list of all ports ("port list") and then build each port after another. Before each new build all other ports are removed ("port uninstall installed"). The output of port is parsed and then based on this saved in a database which is then used to create the html files.
It would also be good if maintainers could mark a port fixed so it's removed from the failure list.
But the best thing would be if the buildfarm is linked to the svn tree and just tries to rebuild new or failed ports so we always know which ones are failing. I already have some ideas for this and will try to integrate it in my new approach I'm currently working on.
Yes, that would be better.
The only problem is that some ports (like gtk2) need a long time to build so maybe they shouldn't be removed after each port build.
What do you think of my ideas?
Some ports do take a very long time, and some ports have very many dependencies. To start with, maybe it would be good to limit your build farm to ports that don't have so many dependencies (including indirect dependencies). Just so it doesn't take forever, and so that we start learning about the easier-to-fix failures.