[Xquartz-dev] XQ278rc2 is causing large 'hidden border' for me.

[Ken Thomases]

On Oct 12, 2015, at 10:09 PM, Jeremy Huddleston Sequoia <jeremyhu at apple.com> wrote:
> 
> Can you try installing 2.7.8_rc1, backing up /Applications/Utilities/XQuartz.app/Contents/MacOS/X11.bin, updating to 2.7.8_rc2, and then replacing that file with the old one?
> 
> I suspect this is a regression from http://cgit.freedesktop.org/~jeremyhu/xserver/commit/?h=server-1.16-apple&id=cac50177f99fb819bfaeea1f2ac33e38fc574eb3

> diff --git a/hw/xquartz/X11Application.m b/hw/xquartz/X11Application.m
> index 2efbd65..a9ee693 100644
> --- a/hw/xquartz/X11Application.m
> +++ b/hw/xquartz/X11Application.m
> @@ -1240,7 +1240,7 @@ X11ApplicationMain(int argc, char **argv, char **envp)
>  
>      /* Calculate the height of the menubar so we can avoid it. */
>      aquaMenuBarHeight = NSHeight([[NSScreen mainScreen] frame]) -
> -                        NSMaxY([[NSScreen mainScreen] visibleFrame]);
> +                        NSHeight([[NSScreen mainScreen] visibleFrame]);
>  
>  #ifdef HAVE_LIBDISPATCH
>      eventTranslationQueue = dispatch_queue_create(

+[NSScreen mainScreen] does not mean the primary display.  It used to mean the one with the key window.  When "Displays have separate spaces" is enabled, it means the active screen, the one whose menu bar is mostly opaque.  As such, it may not be the screen whose lower-left corner is located at (0, 0).  That's why its max-Y is not necessarily comparable to its height.  That only works for the primary display.

This code could use [[NSScreen screens] firstObject].  This is always the primary display, the one whose lower-left corner is at (0, 0).

Once that's done, the above change should be reverted.  The height of the visible frame would be the full height of the screen minus the menu bar _and the Dock_ if the Dock is along the bottom of the screen.

Actually, there's a theoretically-simpler approach: use -[NSMenu menuBarHeight].  That replaces a long-deprecated method +[NSMenuView menuBarHeight].  However, there was a bug in Tiger that led to the former not working while the latter still worked.  I haven't actually checked recently.

CrossOver's still-kicking X server code uses this code, which tries all of the above:

        NSScreen* primaryScreen = [[NSScreen screens] objectAtIndex:0];
        aquaMenuBarHeight = [[NSApp mainMenu] menuBarHeight];
        if (!aquaMenuBarHeight) aquaMenuBarHeight = [NSMenuView menuBarHeight];
        if (!aquaMenuBarHeight) aquaMenuBarHeight =
            NSHeight([primaryScreen frame]) - NSMaxY([primaryScreen visibleFrame]);


Regards,
Ken