Hi Matt, Testing is indeed an area I find interesting. But it's not so much that I like testing itself, but rather the results that one can achieve with it. Be it fixing bugs or rigorously refactoring. Real UI testing, like Squish does, is very cumbersome imo, maybe that's why that testing framework is called Cucumber? ;-) But maybe the level of GUI testing you are talking about isn't that high as Squish does? For instance, I wouldn't care how much tweets were visible, because that's just an issue of how big a table view should be which is a design issue and really annoying to test as you have to keep updating your tests after design changes (brittle/annoying). Rather I find it more interesting to test if an NSArrayController which is backing a NSTableView contains the correct result set _after_ a NSButton was pushed. This is behaviour which shouldn't break and thus the ratio of the amount of time spent on writing tests vs how usable they are is much better. As you may know, Manfred is talking about testing on the rubyonosx conference. We have discussed the realm of UI testing and come to the conclusion above. Which also means I'll be adding some helpers to Rucola like the push_button helper: describe "TweetController" do before do @person = Person.create(:name => "lrz") @tweets = Array.new(50) {|i| Tweet.create(:tweetee => @person, :message => "Hey Ninh, this is cool vid number #{i}. I swear it's not a Hernandez one!") } end it "should have assigned all tweets from the specified person to the tweets array controller" do personSearchField.stringValue = @person.name push_button searchButton tweetsController.arrangedObjects.should == @tweets end end The push_button helper is theoretical, but very easy to implement, as a control knows its target and the action to call. The other code is already possible with the Rucola test case helper. Basically this is a regular functional test and a much higher level test than this is not interesting in general imo. (If you're wondering which test framework I use; test/spec. It's imo the sweetest balance between Test::Unit and rSpec like frameworks.) Of course you are free to create such a framework and I would surely help out wherever I could with improving testing :-) However, I must say that I don't think it's what's really needed right now. What imo is much much more important to the users of HotCocoa, and developers, is a full test suite for HotCocoa itself. I'm sure Rich and I will come to this topic at the conference as well, but it should be discussed openly as well. Cheers, Eloy On May 12, 2009, at 8:18 AM, Matt Aimonetti wrote:

This is a very interesting topic and probably Eloy's favorite theme ;)

Overall, the Ruby community strongly believes in testing, as a matter of fact we have so many testing frameworks I did not even test them all: test/unit, rspec, shoulda, bacon, context, cucumber etc....

Doing unit tests on your "models" isn't really a challenge, we probably all know how to do that and we have the tools to do so. The real challenge is to test expected behaviors at the highest level: the GUI.

I talked with few very smart people during RailsConf and before that I talked to some Java friends of mine developing swing apps. It seems that everybody agrees that testing behavio(u)rs at the GUI level is really challenging. The approval testing approach is nice but it's kind of a pain since you need to validate every UI change. What would be totally awesome is a solution like Webrat + Cucumber http://cukes.info/ but for Cocoa apps developed using MacRuby.

Think about it, would it be really nice if we could do something like that:

Given I'm a valid twitter user When I enter my credentials Then I should see the last 50 tweets sent by my friends

To do this kind of magic, we need to be able to parse the UI, the good thing is that thanks to MacRuby, we have an easy way to do full introspection on all the objects. So, in theory, we should be able to walk down the 'tree', starting at the NSApp level, pick a window, a view, an tableview, a row and check on its content after a button was clicked. This way, we can trigger UI components and see resulting behavior. We could even imagine some sort of selenium-type DSL to drive our tests in real time :)

Before I get too excited, I'd like to hear what you guys think and what you believe you need to write better code?

- Matt

_______________________________________________ MacRuby-devel mailing list MacRuby-devel@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel

On May 11, 2009, at May 11, 11:18 PM, Matt Aimonetti wrote:

Before I get too excited, I'd like to hear what you guys think and what you believe you need to write better code?

I love the concept, but have often found UI testing tools to be lacking. I think that's because in the past I was always hoping to write *unit*-tests for UIs which near-impossible to do without simply re-asserting the implementation (which makes for a terrible unit- test). Having seen the approach taken by Cucumber I'm more hopeful that this would get some traction. Hell, even to have something that would step through all the little manual scenarios automatically would be a great tool to have. So, yeah, Matt. I'm all for it! —Alex ---- http://alexvollmer.com http://moochbot.com