[MacRuby-devel] [ANN] AXElements: UI Automation with MacRuby

[Mark Rada]

Whether or not an app is supported depends on if it implements the accessibility protocol. All Cocoa applications, including NSStatusBar apps, do implement this by default.

However, sometimes the default implementation is not sufficient or is just buggy. So far, the only places I have had issues is with things in the menu bar; I have logged bugs with Apple, but only in the last few months and so they haven't been fixed yet. However, there is nothing stopping you from overriding the appropriate methods in the NSAccessibility protocol for the class/object that isn't behaving properly. Customizing accessibility is supported by Apple; though you usually will only have to override/implement the protocol if you have a lot of custom UI elements that either directly subclass from NSView or change the default behaviour a lot. I would check things out using the Accessibility Inspector first to see how viable AXElements would be in this case.

The difficult part about menu bar objects is getting the initial reference to the object; in a worst case scenario you would have to scan across the menu bar to find the object (use Accessibility.element_at_position for various coordinates). Normally, though, you can just get the application reference, just  as you would for an app that uses a window. Here is a short example for using the standard menu bar items to futz with the volume slider:

	ui = Accessibility.application_with_bundle_identifier 'com.apple.systemuiserver'
	volume = ui.menu_extra(description: 'system sound volume')

	click volume # this will click the icon so the slider is visible, otherwise it won't work

	15.times { decrement volume.slider }
	15.times { increment volume.slider }


HTH,
	Mark


On 2011-09-29, at 11:24 AM, Brice Ruth wrote:

> This looks very useful. I skimmed through the documentation and didn't see anything for accessing NSStatusBar applications. Is this possible? Or would I need to add some hooks in my application to allow the test to open the windows when launched versus via clicking a menuitem from the NSStatusBar menu?
> 
> Brice Ruth, FCD
> Software Engineer, Madison WI
> 
> 
> On Thu, Sep 29, 2011 at 9:55 AM, Mark Rada <mrada at marketcircle.com> wrote:
> Hey all,
> 
> On behalf of Marketcircle Inc., I am open sourcing a library that I have been working on for a while now. AXElements is an alternative to using Auotmator or the ScriptingBridge frameworks. AXElements provides more generic actions than it's alternatives and should work with almost any Cocoa app without needing to be explicitly supported; you can think of it as Capybara for desktop apps.
> 
> It is meant to be used for automated testing of GUI apps, but can be used for general UI automation. You can combine it with testing libraries like minitest, Rspec, or macbacon; but there is nothing stopping you from combining AXElements with James to voice command some workflows.
> 
> The source is available on github and I have put together some small tutorails in the documentation. I've also made a beta release to rubygems, and will make a proper release soon. To get it now:
> 
>     macgem install AXElements --pre
> 
> And then require 'ax_elements' to get started in macirb or a script.
> 
> Source: http://github.com/Marketcircle/AXElements
> Documentation (including some tutorials): http://rdoc.info/github/Marketcircle/AXElements/master/frames/file/README.markdown
> 
> Cheers,
>        Mark Rada
> 
> _______________________________________________
> MacRuby-devel mailing list
> MacRuby-devel at lists.macosforge.org
> http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
> 
> _______________________________________________
> MacRuby-devel mailing list
> MacRuby-devel at lists.macosforge.org
> http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/macruby-devel/attachments/20110929/2bace6d6/attachment.html>