[MacRuby-devel] [ANN] AXElements: UI Automation with MacRuby

Brice Ruth bdruth at gmail.com
Thu Sep 29 17:40:37 PDT 2011


It appears that there's a known bug in the public NSStatusBar / NSStatusItem
API that prevents these from being selected by the Accessibility API. At
least that's what I'm assuming - see here:

http://openradar.appspot.com/6832098

Granted, this is from 2009, but I'm actually unable to access my application
as you indicated below .. I can access the volume control, but when I try to
find mine, I can't ... is there any way to list all the menu_extra
descriptions that AX can see? That way I can confirm that I'm not just going
after the wrong description.

Thanks!

Brice Ruth, FCD
Software Engineer, Madison WI


On Thu, Sep 29, 2011 at 11:47 AM, Mark Rada <mrada at marketcircle.com> wrote:

> Whether or not an app is supported depends on if it implements the
> accessibility protocol. All Cocoa applications, including NSStatusBar apps,
> do implement this by default.
>
> However, sometimes the default implementation is not sufficient or is just
> buggy. So far, the only places I have had issues is with things in the menu
> bar; I have logged bugs with Apple, but only in the last few months and so
> they haven't been fixed yet. However, there is nothing stopping you from
> overriding the appropriate methods in the NSAccessibility protocol for the
> class/object that isn't behaving properly. Customizing accessibility is
> supported by Apple; though you usually will only have to override/implement
> the protocol if you have a lot of custom UI elements that either directly
> subclass from NSView or change the default behaviour a lot. I would check
> things out using the Accessibility Inspector first to see how viable
> AXElements would be in this case.
>
> The difficult part about menu bar objects is getting the initial reference
> to the object; in a worst case scenario you would have to scan across the
> menu bar to find the object (use Accessibility.element_at_position for
> various coordinates). Normally, though, you can just get the application
> reference, just  as you would for an app that uses a window. Here is a short
> example for using the standard menu bar items to futz with the volume
> slider:
>
> ui = Accessibility.application_with_bundle_identifier
> 'com.apple.systemuiserver'
> volume = ui.menu_extra(description: 'system sound volume')
>
> click volume # this will click the icon so the slider is visible, otherwise
> it won't work
>
> 15.times { decrement volume.slider }
>  15.times { increment volume.slider }
>
>
> HTH,
> Mark
>
>
> On 2011-09-29, at 11:24 AM, Brice Ruth wrote:
>
> This looks very useful. I skimmed through the documentation and didn't see
> anything for accessing NSStatusBar applications. Is this possible? Or would
> I need to add some hooks in my application to allow the test to open the
> windows when launched versus via clicking a menuitem from the NSStatusBar
> menu?
>
> Brice Ruth, FCD
> Software Engineer, Madison WI
>
>
> On Thu, Sep 29, 2011 at 9:55 AM, Mark Rada <mrada at marketcircle.com> wrote:
>
>> Hey all,
>>
>> On behalf of Marketcircle Inc., I am open sourcing a library that I have
>> been working on for a while now. AXElements is an alternative to using
>> Auotmator or the ScriptingBridge frameworks. AXElements provides more
>> generic actions than it's alternatives and should work with almost any Cocoa
>> app without needing to be explicitly supported; you can think of it as
>> Capybara for desktop apps.
>>
>> It is meant to be used for automated testing of GUI apps, but can be used
>> for general UI automation. You can combine it with testing libraries like
>> minitest, Rspec, or macbacon; but there is nothing stopping you from
>> combining AXElements with James to voice command some workflows.
>>
>> The source is available on github and I have put together some small
>> tutorails in the documentation. I've also made a beta release to rubygems,
>> and will make a proper release soon. To get it now:
>>
>>     macgem install AXElements --pre
>>
>> And then require 'ax_elements' to get started in macirb or a script.
>>
>> Source: http://github.com/Marketcircle/AXElements
>> Documentation (including some tutorials):
>> http://rdoc.info/github/Marketcircle/AXElements/master/frames/file/README.markdown
>>
>> Cheers,
>>        Mark Rada
>>
>> _______________________________________________
>> MacRuby-devel mailing list
>> MacRuby-devel at lists.macosforge.org
>> http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
>>
>
> _______________________________________________
> MacRuby-devel mailing list
> MacRuby-devel at lists.macosforge.org
> http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
>
>
>
> _______________________________________________
> MacRuby-devel mailing list
> MacRuby-devel at lists.macosforge.org
> http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/macruby-devel/attachments/20110929/38f27208/attachment.html>


More information about the MacRuby-devel mailing list