[MacRuby-devel] testing macruby/hotcocoa apps

[Eloy Duran]

Hi Matt,

Testing is indeed an area I find interesting. But it's not so much  
that I like testing itself, but rather the results that one can  
achieve with it. Be it fixing bugs or rigorously refactoring. Real UI  
testing, like Squish does, is very cumbersome imo, maybe that's why  
that testing framework is called Cucumber? ;-)

But maybe the level of GUI testing you are talking about isn't that  
high as Squish does?
For instance, I wouldn't care how much tweets were visible, because  
that's just an issue of how big a table view should be which is a  
design issue and really annoying to test as you have to keep updating  
your tests after design changes (brittle/annoying). Rather I find it  
more interesting to test if an NSArrayController which is backing a  
NSTableView contains the correct result set _after_ a NSButton was  
pushed. This is behaviour which shouldn't break and thus the ratio of  
the amount of time spent on writing tests vs how usable they are is  
much better.

As you may know, Manfred is talking about testing on the rubyonosx  
conference. We have discussed the realm of UI testing and come to the  
conclusion above. Which also means I'll be adding some helpers to  
Rucola like the push_button helper:

describe "TweetController" do
   before do
     @person = Person.create(:name => "lrz")
     @tweets = Array.new(50) {|i| Tweet.create(:tweetee =>  
@person, :message => "Hey Ninh, this is cool vid number #{i}. I swear  
it's not a Hernandez one!") }
   end

   it "should have assigned all tweets from the specified person to  
the tweets array controller" do
     personSearchField.stringValue = @person.name
     push_button searchButton
     tweetsController.arrangedObjects.should == @tweets
   end
end

The push_button helper is theoretical, but very easy to implement, as  
a control knows its target and the action to call. The other code is  
already possible with the Rucola test case helper. Basically this is a  
regular functional test and a much higher level test than this is not  
interesting in general imo.

(If you're wondering which test framework I use; test/spec. It's imo  
the sweetest balance between Test::Unit and rSpec like frameworks.)

Of course you are free to create such a framework and I would surely  
help out wherever I could with improving testing :-)
However, I must say that I don't think it's what's really needed right  
now. What imo is much much more important to the users of HotCocoa,  
and developers, is a full test suite for HotCocoa itself. I'm sure  
Rich and I will come to this topic at the conference as well, but it  
should be discussed openly as well.

Cheers,
Eloy

On May 12, 2009, at 8:18 AM, Matt Aimonetti wrote:

> This is a very interesting topic and probably Eloy's favorite theme ;)
>
> Overall, the Ruby community strongly believes in testing, as a  
> matter of fact we have so many testing frameworks I did not even  
> test them all: test/unit, rspec, shoulda, bacon, context, cucumber  
> etc....
>
> Doing unit tests on your "models" isn't really a challenge, we  
> probably all know how to do that and we have the tools to do so. The  
> real challenge is to test expected behaviors at the highest level:  
> the GUI.
>
> I talked with few very smart people during RailsConf and before that  
> I talked to some Java friends of mine developing swing apps. It  
> seems that everybody agrees that testing behavio(u)rs at the GUI  
> level is really challenging. The approval testing approach is nice  
> but it's kind of a pain since you need to validate every UI change.
> What would be totally awesome is a solution like Webrat + Cucumber http://cukes.info/ 
>  but for Cocoa apps developed using MacRuby.
>
> Think about it, would it be really nice if we could do something  
> like that:
>
> Given I'm a valid twitter user
> When I enter my credentials
> Then I should see the last 50 tweets sent by my friends
>
> To do this kind of magic, we need to be able to parse the UI, the  
> good thing is that thanks to MacRuby, we have an easy way to do full  
> introspection on all the objects.
> So, in theory, we should be able to walk down the 'tree', starting  
> at the NSApp level, pick a window, a view, an tableview, a row and  
> check on its content after a button was clicked. This way, we can  
> trigger UI components and see resulting behavior.
> We could even imagine some sort of selenium-type DSL to drive our  
> tests in real time :)
>
> Before I get too excited, I'd like to hear what you guys think and  
> what you believe you need to write better code?
>
> - Matt
>
> _______________________________________________
> MacRuby-devel mailing list
> MacRuby-devel at lists.macosforge.org
> http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/macruby-devel/attachments/20090512/f6443faf/attachment.html>