XComposite transition, was Re: [Xquartz-dev] fb / rootless crash bugs

[Ben Byer]

On Dec 4, 2007, at 4:36 PM, Greg Parker wrote:

> On Dec 4, 2007, at 4:02 PM, Ben Byer wrote:
>> On Dec 3, 2007, at 5:28 PM, Jeremy Huddleston wrote:
>>>
>>> Do they involve gutting rootless and using kdrive perchance?  I'm  
>>> only half joking...
>>
>> No, but I *swore* I mentioned the analogous 100% serious suggestion  
>> from the X.org folks that we gut rootless and use xcomposite.  (I  
>> don't know that kdrive solves the problem that we're trying to  
>> solve here.)
>
> I took a quick look at Composite. It looks like it was designed to  
> work inside a window manager. I'm not sure that makes sense in our  
> environment. We could make quartz-wm also in charge of the Mac  
> windows via Composite, but that might leave other window managers  
> out in the cold. One alternative might be to use the Composite code  
> paths in the server, without using the Composite extension directly.

Yeah, there's definitely some subtlety here.

Keith Packard and I discussed this, here's some notes from a  
conversation that stretched over several months, while I tried and  
failed to make this happen:
- - - - -
If we call compRedirectSubwindows(serverClient, root,  
CompositeRedirectManual), then that will create pixmaps for all of the  
windows and keep those pixmaps up-to-date.   We could then create  
damage structures for each backing pixmap, and then our block handler  
sends the updated bits to the target windows.   If possible, we should  
point the composite pixmaps at the OS X backing buffer; that would  
make the code simpler and faster.

(elaborating further)
What composite gives you is per-window pixmaps.  If you set the root  
window to automatically redirect all children, you'll have pixmaps for  
every mapped child.  Then, hook CreateWindow to listen for damage on  
all children. Then, hack compWindowUpdateAutomatic to paint the right  
bits in a Quartz (Xplugin?)-specific manner, and once that's working  
we can abstract that out and get some hooks put into the DIX code.

compWindowUpdateAutomatic currently uses Render to copy the child  
window onto the parent; change that to copy the child onto the OSX  
surface.  Check if pParent->parent==NULL, and do your stuff.
We might be able to handle frame creation lazily from the update hook,  
but we might be better off hooking the following functions:
CreateWindow, DestroyWindow, RealizeWindow, UnrealizeWindow,  
ConfigureWindow, CloseScreen, PositionWindow, RestackWindow

This would be a simplification over the current Rootless code, which  
hooks all of the above, as well as  ValidateTree,  
ChangeWindowAttributes, MarkOverlappedWindows, ChangeBorderWidth,  
ReparentWindow, ResizeWindow, MoveWindow, SourceValidate, GetImage,  
CopyWindow, CreateGC, CreateScreenResources, and SetShape.
- - - - -

I tried for a week or two and utterly failed to gain any traction on  
this problem -- I got as far as that first compRedirectSubwindows  
call, but if I didn't have any of the rootless code enabled then I  
never got an initial Update message for anything, and if I left the  
rootless code enabled I got crashes and odd behavior.

There are certainly many more things we could do in the realm of  
compositing -- there may be things we can leverage more Quartz  
goodness for -- but for a first approximation, we'd just be using it  
to get pixmaps and updates.
--
Ben Byer
CoreOS / BSD Technology Group, XDarwin maintainer