[MacRuby-devel] [ANN] AXElements: UI Automation with MacRuby

Mark Rada mrada at marketcircle.com
Thu Sep 29 18:17:20 PDT 2011


Hmm, that bug sounds like it could be the culprit, but I haven't checked. I'm busy tonight, but I'll have time tomorrow to look into it; I should make a note in the AXElements documentation about NSStatusBar things.



On 2011-09-29, at 8:46 PM, Brice Ruth wrote:

> I followed the steps in the radar below to activate VO and had the same results as described. Interesting to see which of my non-Apple menu items are using the private API ;-)
> 
> SwitchResX
> MenuMeters
> 
> And those that are using the public API
> 
> Quicksilver
> pyTivoX
> Jumpcut
> gfxCardStatus
> vmware
> dropbox
> (mine)
> 
> The workaround listed in the radar talks about associating a global shortcut with a NSStatusItem. Does anyone know how to do that? And for AX - can that be accessed then via the global shortcut?
> 
> Thanks!
> 
> Brice Ruth, FCD
> Software Engineer, Madison WI
> 
> 
> On Thu, Sep 29, 2011 at 7:40 PM, Brice Ruth <bdruth at gmail.com> wrote:
> It appears that there's a known bug in the public NSStatusBar / NSStatusItem API that prevents these from being selected by the Accessibility API. At least that's what I'm assuming - see here:
> 
> http://openradar.appspot.com/6832098
> 
> Granted, this is from 2009, but I'm actually unable to access my application as you indicated below .. I can access the volume control, but when I try to find mine, I can't ... is there any way to list all the menu_extra descriptions that AX can see? That way I can confirm that I'm not just going after the wrong description.
> 
> Thanks!
> 
> Brice Ruth, FCD
> Software Engineer, Madison WI
> 
> 
> On Thu, Sep 29, 2011 at 11:47 AM, Mark Rada <mrada at marketcircle.com> wrote:
> Whether or not an app is supported depends on if it implements the accessibility protocol. All Cocoa applications, including NSStatusBar apps, do implement this by default.
> 
> However, sometimes the default implementation is not sufficient or is just buggy. So far, the only places I have had issues is with things in the menu bar; I have logged bugs with Apple, but only in the last few months and so they haven't been fixed yet. However, there is nothing stopping you from overriding the appropriate methods in the NSAccessibility protocol for the class/object that isn't behaving properly. Customizing accessibility is supported by Apple; though you usually will only have to override/implement the protocol if you have a lot of custom UI elements that either directly subclass from NSView or change the default behaviour a lot. I would check things out using the Accessibility Inspector first to see how viable AXElements would be in this case.
> 
> The difficult part about menu bar objects is getting the initial reference to the object; in a worst case scenario you would have to scan across the menu bar to find the object (use Accessibility.element_at_position for various coordinates). Normally, though, you can just get the application reference, just  as you would for an app that uses a window. Here is a short example for using the standard menu bar items to futz with the volume slider:
> 
> 	ui = Accessibility.application_with_bundle_identifier 'com.apple.systemuiserver'
> 	volume = ui.menu_extra(description: 'system sound volume')
> 
> 	click volume # this will click the icon so the slider is visible, otherwise it won't work
> 
> 	15.times { decrement volume.slider }
> 	15.times { increment volume.slider }
> 
> 
> HTH,
> 	Mark
> 
> 
> On 2011-09-29, at 11:24 AM, Brice Ruth wrote:
> 
>> This looks very useful. I skimmed through the documentation and didn't see anything for accessing NSStatusBar applications. Is this possible? Or would I need to add some hooks in my application to allow the test to open the windows when launched versus via clicking a menuitem from the NSStatusBar menu?
>> 
>> Brice Ruth, FCD
>> Software Engineer, Madison WI
>> 
>> 
>> On Thu, Sep 29, 2011 at 9:55 AM, Mark Rada <mrada at marketcircle.com> wrote:
>> Hey all,
>> 
>> On behalf of Marketcircle Inc., I am open sourcing a library that I have been working on for a while now. AXElements is an alternative to using Auotmator or the ScriptingBridge frameworks. AXElements provides more generic actions than it's alternatives and should work with almost any Cocoa app without needing to be explicitly supported; you can think of it as Capybara for desktop apps.
>> 
>> It is meant to be used for automated testing of GUI apps, but can be used for general UI automation. You can combine it with testing libraries like minitest, Rspec, or macbacon; but there is nothing stopping you from combining AXElements with James to voice command some workflows.
>> 
>> The source is available on github and I have put together some small tutorails in the documentation. I've also made a beta release to rubygems, and will make a proper release soon. To get it now:
>> 
>>     macgem install AXElements --pre
>> 
>> And then require 'ax_elements' to get started in macirb or a script.
>> 
>> Source: http://github.com/Marketcircle/AXElements
>> Documentation (including some tutorials): http://rdoc.info/github/Marketcircle/AXElements/master/frames/file/README.markdown
>> 
>> Cheers,
>>        Mark Rada
>> 
>> _______________________________________________
>> MacRuby-devel mailing list
>> MacRuby-devel at lists.macosforge.org
>> http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
>> 
>> _______________________________________________
>> MacRuby-devel mailing list
>> MacRuby-devel at lists.macosforge.org
>> http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
> 
> 
> _______________________________________________
> MacRuby-devel mailing list
> MacRuby-devel at lists.macosforge.org
> http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
> 
> 
> 
> _______________________________________________
> MacRuby-devel mailing list
> MacRuby-devel at lists.macosforge.org
> http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.macosforge.org/pipermail/macruby-devel/attachments/20110929/3be182e7/attachment.html>


More information about the MacRuby-devel mailing list