Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-118834

QStringConverter/Win might throw away some valid characters when restoring state during decode

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • P3: Somewhat important
    • None
    • 6.5, 6.6, 6.7
    • None
    • Windows
    • b099988f6 (dev), d5eb5d2f8 (dev)

    Description

      Note: This is only for the "System" Encoding for QStringConverter!

      After recently having enabled storing more than 1 character to state, the new status quo is that we try to restore what we have, and if it is still not valid we throw away what was stored and try to decode the rest of the string without it.
      What would be optimal would be to only throw away invalid characters until we reach the boundary of a new character, but this is not always trivially achievable. For instance, the second and third octet in a sequence might, by themselves, decode to a valid character, so there is no way to guess if this was the intent or not.

      Very concretely, given

          result = QLocal8Bit::convertToUnicode_sys("\xe4\xe4\xbd", UTF8, &state);
          result += QLocal8Bit::convertToUnicode_sys("\xa0", UTF8, &state);
      

      The logical correct output might be a replacement character (for the incomplete first \xe4) + 你. But currently the whole sequence would be discarded and you get 4 replacement characters.

      What's more important is that in general the QStringConverter machinery does not enable us to say that the internal buffers needs to be drained because it will not be called anymore.

      Attachments

        For Gerrit Dashboard: QTBUG-118834
        # Subject Branch Project Status CR V

        Activity

          People

            manordheim Mårten Nordheim
            manordheim Mårten Nordheim
            Vladimir Minenko Vladimir Minenko
            Alex Blasche Alex Blasche
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Gerrit Reviews

                There are no open Gerrit changes