Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-126233 Task related to supporting macOS 15 Sequoia
  3. QTBUG-126239

Investigate how we can handle NSAdaptiveImageGlyph (Genmojis, stickers)

    XMLWordPrintable

Details

    • Sub-task
    • Resolution: Unresolved
    • Not Evaluated
    • None
    • None
    • None

    Description

      https://developer.apple.com/videos/play/wwdc2024/10220

      Possible solutions

      • Object embedding via QTextObjectInterface
        • Custom AdaptiveImageGlyph object type
          • Implementing intrinsicSize via CTFontGetTypographicBoundsForAdaptiveImageProvider
            • And drawObject via CTFontDrawImageFromAdaptiveImageProviderAtPoint
          • Might miss out on some logic in consumers that's hard-coded for QTextFormat::ImageObject
            • We're technically also a QTextFormat::ImageObject, so would be good to get the same level of support for e.g. conversion to html
          • Registered as default handler for custom object type similar to QTextFormat::ImageObject
        • Register as QTextFormat::ImageObject handler
          • Since what we're really handling is images, just that the input property is not a file name but a  NSAdaptiveImageGlyph or NSAdaptiveImageGlyph-identifier
          • QAbstractTextDocumentLayout only maintains one handler per object type
            • Would need to extend QAbstractTextDocumentLayout to support multiple handlers per type, and iterate them
              • However that would conflict with handlerForObject()
            • Could possibly return a handler that internally iterates a set of handlers
            • Or implement a custom Apple image handler that subclasses QTextImageHandler and make sure that's used as the one and only QTextFormat::ImageObject
              • Forward to QTextFormat::ImageObject if it's not a NSAdaptiveImageGlyph
        • Register file engine backend for custom NSAdaptiveImageGlyph url scheme
          • And build on existing QTextImageHandler logic
          • Loses the ability to provide metrics and drawing based on font size
            • Only have access to DPR
      • Custom font
        • A custom font could map private-use code points to NSAdaptiveImageGlyphs, and render those using CTFontDrawImageFromAdaptiveImageProviderAtPoint
        • Could be that Apple Color Emoji already works this way, need to check
        • One down-side would be that text selection in an editor would then show the custom font for the glyph, instead of the surrounding font.

      All of these solutions have in common that they require text insertion via IM or otherwise to specify both text (QChar::ObjectReplacementCharacter or private-use code point), and QTextFormat for the inserted text (either to provide object replacement data, or custom font/apple emoji font).

      Attachments

        Issue Links

          For Gerrit Dashboard: QTBUG-126239
          # Subject Branch Project Status CR V

          Activity

            People

              vestbo Tor Arne Vestbø
              vestbo Tor Arne Vestbø
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There is 1 open Gerrit change