Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-129710

Adding QJsonStreamWriter and QJsonStreamReader + Simplifying QCborStreamReader and QCborStreamWriter for QVariant Support

    XMLWordPrintable

Details

    • Suggestion
    • Resolution: Unresolved
    • P3: Somewhat important
    • None
    • 6.5, 6.8
    • Core: Serialization
    • None

    Description

      Hello,

      I would like to propose an enhancement for JSON and CBOR support in Qt by adding two new classes: QJsonStreamWriter and QJsonStreamReader, as well as simplifying the QCborStreamReader and QCborStreamWriter classes to allow for more intuitive handling of QVariant. (yes I know QCborValue::fromVariant exists but it has to go through the data twice, once to build the value and once to build the cbor, same goes for QJsonDocument::FromVariant) 

      Context and Problem:
      The current support for JSON in Qt through classes like `QJsonDocument`, `QJsonObject`, and `QJsonArray` is effective but has limitations when handling large volumes of JSON data, especially in streaming scenarios (incremental reading/writing). Furthermore, although the CBOR format is well-supported in Qt through QCborStreamReader and QCborStreamWriter, these classes remain somewhat complex when dealing with QVariant data. Currently, extra conversion steps are needed when working with `QVariant`, adding unnecessary complexity and reducing usability.

      Proposal for JSON Classes:

      1. Adding QJsonStreamWriter and QJsonStreamReader:
      I propose introducing two new classes to provide a more efficient way to handle JSON through a `QIODevice` (files, sockets, etc.) in a streaming (incremental) mode. These classes would allow developers to read and write JSON data progressively without loading the entire document into memory.

      Objectives:

      • Optimize performance when serializing and deserializing large JSON data.
      • Reduce memory consumption by enabling incremental JSON read/write, especially useful in the context of REST support in the QNetwork module.
      • Better REST compatibility: These classes would be particularly beneficial for handling large JSON responses and requests in real-time, using a `QIODevice`.

      Prototypes:

      class QJsonStreamWriter
      {
      public:
          explicit QJsonStreamWriter(QIODevice *device);
          ~QJsonStreamWriter();
          void writeStartObject();
          void writeEndObject();
          void writeStartArray();
          void writeEndArray();
          void writeKey(const QString &key);
          void writeValue(const QVariant &value);
          void setCompact(bool compact);
      private:
          QIODevice *m_device;
          bool m_compact;
      };
      
      class QJsonStreamReader
      {
      public:
          explicit QJsonStreamReader(QIODevice *device);
          ~QJsonStreamReader();
          bool readStartObject();
          bool readEndObject();
          bool readStartArray();
          bool readEndArray();
          QString readKey();
          QVariant readValue();
          bool atEnd() const;
      private:
          QIODevice *m_device;
      };
      

      Proposal for Improving CBOR Classes:

      2. Simplifying QCborStreamReader and QCborStreamWriter APIs for Better QVariant Support:
      While the QCborStreamReader and QCborStreamWriter classes already exist, they are currently difficult to use when it comes to serializing and deserializing QVariant data. Many operations require additional conversion layers, increasing the complexity of the code.

      I propose simplifying the API for these classes to allow direct handling of QVariant, making CBOR serialization more intuitive and easier to use for Qt developers.

      Suggested Improvements:

      • QCborStreamWriter::writeValue(const QVariant &value): This method would allow writing a QVariant directly to a CBOR stream, without manual conversion to CBOR.
      • QCborStreamReader::readValue(): This method would return a QVariant directly from a CBOR stream, automatically analyzing the underlying CBOR type (objects, arrays, numbers, strings, etc.).
      • These changes would streamline the workflow for handling QVariant and make CBOR streams more consistent with other Qt APIs.

      Expected Benefits:

      • Performance optimization: Handling JSON and CBOR through streams significantly reduces memory usage, particularly for large REST responses and requests.
      • Simplified API: Adding direct QVariant handling methods in QCborStreamReader and QCborStreamWriter will make these classes easier and more intuitive to use.
      • Improved REST support: Integrating these classes with Qt's REST framework would allow developers to handle streaming JSON or CBOR data efficiently, especially for large datasets.

      Conclusion:
      These proposals aim to improve Qt's support for handling JSON and CBOR data, providing streaming support and simplifying the QVariant handling in the CBOR API. This would be particularly useful in scenarios where large volumes of data need to be processed incrementally, such as in REST services (benefits would be even greater on low end devices).

      Some discussions and benchmarks have already been posted here https://bugreports.qt.io/browse/QTBUG-129633

      Attachments

        1. qjsonutils.cpp
          10 kB
          Romain Donzé

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              cnn Qt Core & Network
              romain.donze Romain Donzé
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes