OS X: Core Animation layer-backed QWindow

QCocoaWindow/QNSView should work well in layer-backed mode. Layer mode can be enabled several ways:

1. The view request a layer with [self setWantsLayer:true]
2. A parent view requests a layer (all child views become layer-backed as well)
3. Setting certain properties on the NSWindow will enable layer mode. (NSFullSizeContentViewWindowMask)

A view can create a custom layer by reimplementing [NSView makeBackingLayer]. The default layer is suitable for CoreGraphics based rendering, and may also be sufficient for OpenGL rendering.

A general benefit is that raster and OpenGL content is now freely composable, with proper z-order stacking. This includes mixing Qt content with native views.

NSView hierarchy control

Qt may have full control of the NSView hierarchy up to the NSWindow, or it may find itself in a 'foreign' NSView hierarchy that is or is not in layer mode. This means the the QCocoaWindow implementation should react to the "current" mode.

QWindows with raster content

Raster QWindows can draw themselves into a their layer using CoreGraphics. Alternatively, we can create a custom layer and set the content of the layer to be the backing store buffer, reducing the buffer count by one and omitting the "paint on layer" operation.

QWindows with OpenGL content

• Best option: use a custom layer, subclassed for NSOpenGLLayer.
• NSOpenGLView seems to work in both in both layer and non-layer mode,
• A straightforward NSView + NSOpenGLContext implementation has issues with flickering due to using the incorrect draw frame buffer.
• QOpenGLWindow (which also uses NSView + NSOpenGLContext) seem to fare better, but should theoretically have the same problem. Update: Moving updates/expose to drawRect (QTBUG-49859) seems to cause the breakage.

QWindows with OpenGL or raster content (QWidgetWindow)

• Option1: Use the default layer.
• Option2: Treat as an OpenGL window with a custom layer, use OpenGL compositing for Widgets. QTBUG-53160

Child QWindow / NSView

Child NSViews will normally get layers if the parent view is in layer mode. As an optimization, the parent NSView can set canDrawSubviewsIntoLayer to disable automatic child layers. Child views that call setWantsLayer::YES still gets layers.

We can use this for optimizing the QBackingStore case: The QWindow that owns the backing store gets a layer and sets
canDrawSubviewsIntoLayer. Child QWindows may then opt-in to having a layer, if needed.

The Qt for UIKit implementation

UIViews are a layer-backed by default so it makes sense to look at that implementation, provided that the available API is similar enough.

This seems to not be the case, in particular the QIOSContext uses CAEAGLLayer and EAGLContext API that does not have a Cocoa/AppKit equivalent:

[EAGLContext renderbufferStorage: fromDrawable:]
[EAGLContext presentDrawable: ]