Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-121781

Modernize iOS window management

    XMLWordPrintable

Details

    • iOS window management
    • iOS/tvOS/watchOS
    • 2da738f03 (dev), 5195c6683 (6.7), 0b70ca278 (6.6), 17f6c62d9 (dev), b7eb73b1f (dev), a43d76dcf (tqtc/lts-6.5), 447a82ebe (dev), f480b5991 (dev), e48a9aca5 (dev), eb9923c67 (dev), 1a0c113b2 (dev), c0f6d10bc (dev), bc014d5fc (dev), 76ebf51bc (dev)

    Description

      The Qt for iOS port was written when iOS didn't have support for advanced window management. Nowadays, with multi windows on iPadOS and iOS apps running on macOS the situation is different, and we should modernize the way we do window management on iOS in Qt.

      The current state

      QScreen

      For each UIScreen we have one QIOSScreen. We use UIScreen.bounds for the geometry, and UIScreen.applicationFrame for the availableGeometry.

      The orientation of the UIScreen is reflected in the QScreen geometry and orientation.

      Connecting external displays to an iOS device is reflected as an additional QScreen. Each screen is independent, and not a virtual sibling of any other screen. Moving/creating a window on an external QScreen (via setScreen) results in us disabling screen mirroring and setting the UIScreen up for independent content.

      QWindow

      Each QWindow is represented as a UIView (QUIView). The view handles callbacks for touch, keyboard input, etc, and has a Core Animation layer that we blit any rendering to (Metal or GL, depending on the graphics stack used in Qt).

      Child QWindows are subviews of their parent QWindow.

      Window management

      Each QIOSScreen manages a single UIWindow (QUIWindow).

      Each QUIWindow has a Qt specific root view controller (QIOSViewController). We use this to get callbacks about orientation changes.

      Each QIOSViewController has a a QIOSDesktopManagerView as its (root) view. We use this view for window management. Each top level QWindow (UIView) is added as a subview of this view. Depending on the window state of the QWindow (normal, maximized, fullscreen), we manage the frame of the QWindow's UIView within this QIOSDesktopManagerView.

      Issues

      Building Qt for iOS shows quite a few deprecation warnings for iOS APIs, eg:

       

      [1082/1117] Building OBJCXX object qtbase/src/plugins/platforms/ios/CMakeFiles/QIOSIntegrationPlugin.dir/qiosviewcontroller.mm.o
      /Users/torarne/dev/qt/qtbase/src/plugins/platforms/ios/qiosviewcontroller.mm:103:18: warning: 'setScreen:' is deprecated: first deprecated in iOS 13.0 [-Wdeprecated-declarations]
              uiWindow.screen = screen->uiScreen();
                                            ^
      /Users/torarne/dev/qt/qtbase/src/plugins/platforms/ios/qiosviewcontroller.mm:474:52: warning: 'statusBarOrientation' is deprecated: first deprecated in iOS 13.0 - Use the interfaceOrientation property of the window scene instead. [-Wdeprecated-declarations]
                  self.lockedOrientation = uiApplication.statusBarOrientation;
      
      [1063/1117] Building OBJCXX object qtbase/src/plugins/platforms/ios/CMakeFiles/QIOSIntegrationPlugin.dir/qiosscreen.mm.o
      /Users/torarne/dev/qt/qtbase/src/plugins/platforms/ios/qiosscreen.mm:160:17: warning: 'applicationFrame' is deprecated: first deprecated in iOS 9.0 [-Wdeprecated-declarations]
          return self.applicationFrame;
                      ^~~~~~~~~~~~~~~~
                      bounds
      [1087/1117] Building OBJCXX object qtbase/src/plugins/platforms/ios/CMakeFiles/QIOSIntegrationPlugin.dir/qioscolordialog.mm.o
      /Users/torarne/dev/qt/qtbase/src/plugins/platforms/ios/qioscolordialog.mm:121:40: warning: 'keyWindow' is deprecated: first deprecated in iOS 13.0 - Should not be used for applications that support multiple scenes as it returns a key window across all connected scenes [-Wdeprecated-declarations]
              : qt_apple_sharedApplication().keyWindow;

      Indicating it's high time to modernize and clean up a few things

      On iPadOS, moving application windows to external screens fails to use the entire screen (QTBUG-106709)

      When running an iOS app on macOS, UIScreen does not report the properties of the macOS screen. It reports a size that reflects a "fake" iPad display. 

      When running iOS apps on macOS, we don't reflect top level QWindows as top level windows in the macOS desktop environment. Instead they still live within the QIOSDesktopManagerView, which looks out of place.

      The QIOSDesktopManagerView on macOS still has a status bar offset, even though there's no status bar.

       

      Goal

      On iOS and iPadOS, Qt apps should work well, and take advantage of the features of the platform, including multi window and multiple screens.

      Running iOS apps on macOS is a less critical, as our cross platform story for macOS is Qt for macOS – not running iOS apps on macOS via Catalyst. But these iOS apps should still look and run reasonably well. It's important to note that we very much depend on the iOS on macOS compatibility layer for this, so as long as our iOS app is a well written/modern iOS app, we expect the underlying platform to do the work of making it run well in an macOS environment. If there are corner cases or quirks where we have to distinguish between running on iOS on a device and running on iOS inside a macOS environment, we can do selective fixes, but we don't want to build another macOS platform abstraction inside the Qt for iOS port.

      Possible improvements

      On the screen side, we should look into whether we can (and should) reflect the macOS screen size/properties when running an iOS app on macOS.

      For window management, we should likely start associating a UIWindowScene (and its resulting UIWindow and view controller hierarchy) with each top level window (depending on type, see below), similarly to how top level windows on macOS have both an NSView (for events and drawing), and an NSWindow. 

      We should potentially also use different view controllers for different top level window types. A dialog QWindow should perhaps show as a modal view controller, a popup should perhaps show using a popup view controller, etc. These types of top level windows would then not get their own UIWindowScene, but be presented from the view controller of the (transient parent) UIWindowScene.

      Tips and tricks

      Use the gui/rasterwindow example from qtbase for testing. tests/manual/windowflags is also useful for testing the various ways you can show a window and the various window types we have.

      Relevant links

      • Introducing Multiple Windows on iPad

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity