Is it time to start regression testing yet?

[Jordan K. Hubbard]

I know that the word "packaging" is kind of a dirty word in MacPorts- 
land (perhaps largely due to the fact that certain people just won't  
stop harping about it :-), so maybe it's time for a new(er) topic in  
an old conversation:  Testing.

Since a picture is supposed to be worth a thousand words, let me also  
refer to the picture below for justification as to why we should be  
worrying more about testing.  Ports are accumulating at the fairly  
steady rate of 800-1000 a year, and it's also fair to say that  
individual ports are getting more complex.  What started as a fairly  
simplistic attempt at key/value pairs in Tcl has since grown Groups,  
variants and, in some cases, fairly non-trivial tcl code in individual  
Portfiles, and all of that begs the question:  Given all the  
complexity involved, how many of these almost 6000 ports actually work  
at any given time?  Anyone have an accurate number?   Anyone?   
Beuller? No worries, it was a purely rhetorical question to which I  
already know the answer:  We have no idea, though we certainly hope  
that users will report breakage in a fairly timely fashion so we can  
fix things as they come up, and if there are no users of a port to  
report errors, then who really cares if it's broken?  We then proceed  
to the rather circular argument of justifying the existence of ports  
which don't currently work but are kept around purely on the argument  
that they *might* at some point in the future.

Wouldn't it be easier to simply make the creation and homing of a  
testing framework a bigger priority?  I've said it many times, but  
I'll say it again:  If someone can handle the creation part, I'm  
fairly confident that the "homing" situation can be worked out.

- Jordan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: MacPortsProgress.001.jpg
Type: image/jpg
Size: 63859 bytes
Desc: not available
URL: <http://lists.macosforge.org/pipermail/macports-dev/attachments/20090606/731bbcc2/attachment-0001.jpg>