Find cross platform solution to QMimeData handling non-mime data

Our drag and drop system is based on marshalling data through QMimeData. Some platforms do not operate with mime as the identifier for data, so we've had to add bolt-on solutions that try to map from the native identifiers to MIME (QMacPasteboardMime and QWinMime). See https://doc.qt.io/qt-5/qmimedata.html#platform-specific-mime-types for more details.

Windows

On Windows, we have the QNativeInterface::Private::QWindowsMime abstract interface, with virtual functions to convert to and from native formats. Qt's built-in support for native formats is realised through implementations of that type, e.g. class QWindowsMimeImage : public QNativeInterface::Private::QWindowsMime.

The native format is described through a FORMATETC structure. When converting from native data, then we access it through an IDataObject interface implementation. When converting to native data, then we populate a STGMEDIUM struct.

QWindowsMime instances are managed through the QWindowsMimeConverter class, which provides static APIs to register QWindowsMime implementations, and to get access to the QWindowsMime instance that supports the requested conversion. A QWindowsMimeConverter instance is a member of QWindowsContext, and can be considered a singleton.

In addition, we have QWindowsInternalMimeData based on QInternalMimeData that wraps, and takes care of the the life-time management of, the IDataObject in Qt. We can ignore those types for this discussion.

macOS

On macOS, we have QMacInternalPasteboardMime. This class has a virtual interface to convert to and from native formats. Qt's built-in support for native formats is realised through implementations of that type, e.g. class QMacPasteboardMimeTraditionalMacPlainText : public QMacInternalPasteboardMime or class QMacPasteboardMimeTiff : public QMacInternalPasteboardMime. The interface provides flags to allow a converter implementation to indicate whether it supports drag'n'drop or the clipboard. All Qt built-in converters set MIME_ALL to indicate that they support both. The Qt specific flag MIME_QT_CONVERTOR (or even MIME_QT3_CONVERTOR) is set by two built-in converter implementations; there is however no code in Qt 6 that would test those flags.

The QMacPasteboardMime class in Qt 5.15's QtMacExtras is identical to this virtual interface of QMacInternalPasteboardMime. The comment in QMacInternalPasteboardMime "// Duplicate of QMacPasteboardMime in QtMacExtras. Keep in sync!" indicates that Qt 5.15 relied on those two virtual interfaces to be identical on a binary level, ie. implementations of QMacPasteboardMime could be treated as implementations of QMacInternalPasteboardMime.

However, at the same time as an abstract interface, QMacInternalPasteboardMime is a manager API with static functions to get access to the implementation suitable for a certain format. Qt defines global functions, qt_mac_addToGlobalMimeList, qt_mac_removeFromGlobalMimeList, qt_mac_registerDraggedTypes, and qt_mac_enabledDraggedTypes to register new converters.

Cross-platform opportunities?

Mime converter interface

Both macOS and Windows define virtual interfaces with platform specific APIs. Those are implemented by Qt internally to provide built-in support for types, so we can be rather confident that those interfaces work and are integrated correctly into the framework. Defining a single cross-platform virtual interface seems impossible; a Windows implementation MUST operate on Win32 data types (FORMATETC, IDataObject, STGMEDIUM), and a macOS implementation MUST map from macOS flavour strings to mime type. In theory, Qt could preprocess the native data in a way that allows us to operator on standard formats. Ie FORMATETC could be serialized into a string; Qt could read the bytes from an IDataObject or STGMEDIUM and put it into a QByteArray. However, the exact string and byte format would still be platform specific, so the code would still have to use #ifdef's to implement different paths, while losing a lot of flexibility (IDataObject might be a file on disk; Qt would have to read all that data into memory) and type safety ("CF_HTML" instead of CF_HTML).

A cross platform abstract type "QPlatformMime" or "QNativeMime" that QWindowsMime and QMacPasteboardMime subclass would add little value. Making a single class with a virtual interface for all platforms also provides no value. In fact, we might end up tying our hands when we want to add support for ie. and Android or WASM mime type, and instead of adding a dedicated QAndroidMime type we have to extend the virtual interface of an exsting class - which we can't.

Converter registration

Both macOS and Windows have manager APIs that are used by Qt internally and do not need to be exposed to applications. The constructor and destructor of QWindowsMime and QMacPasteboardMime transparently register and remove the instance, respectively.

Mime type registration

On Windows, applications that want to handle a native clipboard format need to know the corresponding clipboard format identifier. That value is provided to those QWindowsMime APIs that take a FORMATETC data structure. We need to register the mime type so that the corresponding clipboard format identifier FORMATETC.cfFormat is known to the converter implementation. This is at the very least required when Qt is the source of the clipboard data or drag object; to receive natively encoded data, a mime converter operates primarily on mime type strings (which it can compare; Qt maps the native format to a mime type application/x-qt-windows-mime;value="CF_TypeName"). However, QWindowsMime::mimeForFormat also needs to compare the FORMATETC.cfFormat value correctly. QWindowsApplication provides a registerMimeType method that takes a mime type string, and returns the clipboard format identifier. That identifier needs to be identical to what other Windows applications use for the same clipboard format name. Since QWindowsMime doesn't provide a virtual function to return the supported mime type, since even such a virtual function couldn't be called from the QWindowsMime constructor implicitly (as it would bypass virtual dispatch), and since the implementation would anyway need to know the resulting clipboard format ID for each of them, we need a public API for mime type registration. Currently, this is in QNativeInterface::Private::QWindowsApplication.

There is no equivalent mechanism on macOS. UTIs are in principle equivalent to mime types in that they are well-known strings. However, the 'widgets/draganddrop/dropsite' example does not show any mime types on macOS when dragging unknown content into the label (such as a sidebar item from the Finder; with the qt.qpa.clipboard logging category enabled, this shows that one of the item's UTIs would be e.g. "com.apple.finder.sidebarItem").

Proposal

Bring back types, perhaps QWindowsMimeConverter and QMacMimeConverter, as public types in Qt Gui (on macOS, the "Pasteboard" in the class name seems rather unnecessary and borderline misleading, given that drag'n'drop is supported as well). Those types can be pure virtual, with the exception of default constructor and destructor implementation that call the respective manager APIs to register and remove. QWindowsMime and QMacMime could be used instead, but those implementations are the actual converters. Refactor the existing code to split esp the macOS architecture up between the virtual interface and the manager API.

Applications can implement and instantiate those converter types during application startup. Based on existing implementations, instances don't allocate any member data, so there seems to be no benefit in allowing applications to add and remove instances dynamically.