Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-55090

OutOfMemory + crash because of missing error checking in QWindowsPrintDevice

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • P1: Critical
    • 5.10
    • 5.5.1, 5.6.0
    • GUI: Printing
    • None
    • * OS: Windows 10 64-Bit
      * Any Compiler like MSVS 2013
      * 32-Bit build
      * UAC enabled
      * user without admin-rights
    • 84ab88ce416fe5bb616ff64d26e5282e86f54cd8

    Description

      We found this callstack on a customer computer:

      custom.dll!new_handler(unsigned int s) Zeile 773	C++ ** OUT OF MEMORY **
      Qt5PrintSupport.dll!QPrintDevice::defaultPageSize() Zeile 137	C++
      Qt5PrintSupport.dll!QWin32PrintEngine::QWin32PrintEngine(QPrinter::PrinterMode mode) Zeile 88	C++
      Qt5PrintSupport.dll!QPrinterPrivate::initEngines(QPrinter::OutputFormat format, const QPrinterInfo & printer) Zeile 147	C++
      Qt5PrintSupport.dll!QPrinter::QPrinter(QPrinter::PrinterMode mode) Zeile 684	C++
      

      Unfortunately we were not able to reproduce it, because it seems to be system specitic.
      But some investigation in defaultPageSize() leads to a missusage of DocumentProperties that will definitely end in an OutOfMemory exception.

      The reason is simple:
      DocumentProperties is used to fetch the current DEVMODE-structure in several functions of QWindowsPrintDevice.
      But there is no error checking when DocumentProperties fails!
      When DocumentProperties fails it returns a negative (signed!) value of type LONG, which ist stored in an unsigned variable of type DWORD, which now have a huge positive value.
      Whithout error-checking ist is used to allocate memory of it's value, which will imho always fails.

      Suggested fix would be like this

          // Allocate the required DEVMODE buffer
          LONG dmSize = DocumentProperties(NULL, m_hPrinter, (LPWSTR)m_id.utf16(), NULL, NULL, 0);
          if (dmSize > 0)
          {
             LPDEVMODE pDevMode = (LPDEVMODE)malloc(dmSize);
      
             // Get the default DevMode
             DWORD result = DocumentProperties(NULL, m_hPrinter, (LPWSTR)m_id.utf16(), pDevMode, NULL, DM_OUT_BUFFER);
      
             // Get the desired value
      
             // Clean-up
             free(pDevMode);
          }
          else if (dmSize < 0) {
             // some error handling needed
          }
          // no error when dmSize == 0
      

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            johnlayt John Layt
            Alex_S Alexander Shaya
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes