Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-16967

sqlite driver eats/leaks memory (QTBUG-11210 clone)

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: P1: Critical
    • Resolution: Done
    • Affects Version/s: 4.6.2
    • Fix Version/s: 5.0.0
    • Component/s: SQL Support
    • Labels:
      None
    • Environment:
      Windows 7 professional 64 bit
      QT SDK 4.6.2 2010.2.1 with mingw and gcc

      also:
      Debian 64bit with Qt 4.7
    • Commits:
      cc75a26d

      Description

      The amount of memory used by an application continuously increases according to the number of INSERT statements executed
      A short program with these lines will eventually use all available memory (4Gig)

      EXAMPLE CODE:

          QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE");
          db.setDatabaseName(DBNAME);
      
          for (int i = 0; i < 40000; i++)
          {
              bOk = db.open();
              if ( !bOk )
              {
                  qDebug() << "db.Open() FAILED";
                  break;
              }
              QSqlQuery query(db);
              QString qStr = QString ("INSERT INTO socketlog ([ip],[outlet],[watts],[volts],[amps],[temp],[freq],[powerfactor]) "
                                           "VALUES(%1,0,10,240,2,15,50,1);").arg(19216811 + (i % 100));
              bOk = query.exec(qStr);
              db.close();
              if ( !bOk )
                  qDebug() << "DB insert_sockdata FAILED :: " << query.lastError();
              else
                  qDebug() << "DB insert_sockdata OK " << i;
          }
      

      ***

      this is valgrind's massif output ran on lokalize, debian 64-bit, with libsqlite3-0-dbg 3.7.4-2 and libqt4-sql-sqlite 4:4.7.1-2.

      ->28.19% (10,539,384B) 0x17BE9D9D: sqlite3MemMalloc (sqlite3.c:14221)
      | ->28.19% (10,539,384B) 0x17BB0385: mallocWithAlarm (sqlite3.c:17730)
      |   ->28.19% (10,539,384B) 0x17BB0466: sqlite3Malloc (sqlite3.c:17763)
      |     ->26.76% (10,005,056B) 0x17BB2301: pcache1Alloc (sqlite3.c:33493)
      |     | ->26.73% (9,996,800B) 0x17BC8A80: pcache1Fetch (sqlite3.c:33552)
      |     | | ->26.73% (9,996,800B) 0x17BBC534: sqlite3PcacheFetch (sqlite3.c:32956)
      |     | |   ->26.73% (9,996,800B) 0x17BD9A42: sqlite3PagerAcquire (sqlite3.c:39626)
      |     | |     ->26.73% (9,996,800B) 0x17BD9BC6: btreeGetPage (sqlite3.c:46753)
      |     | |       ->26.70% (9,984,000B) 0x17BD9C32: getAndInitPage (sqlite3.c:46806)
      |     | |       | ->26.60% (9,948,160B) 0x17BD9C92: moveToChild (sqlite3.c:49309)
      |     | |       | | ->10.78% (4,032,000B) 0x17BD9E25: moveToLeftmost (sqlite3.c:49474)
      |     | |       | | | ->10.72% (4,010,240B) 0x17C21A9A: sqlite3VdbeExec (sqlite3.c:64587)
      |     | |       | | | | ->10.72% (4,010,240B) 0x17C0481E: sqlite3_step (sqlite3.c:58706)
      |     | |       | | | |   ->10.72% (4,010,240B) 0x179A1CBF: ??? (in /usr/lib/qt4/plugins/sqldrivers/libqsqlite.so)
      |     | |       | | | |     ->10.72% (4,010,240B) 0x179A2ACD: ??? (in /usr/lib/qt4/plugins/sqldrivers/libqsqlite.so)
      |     | |       | | | |       ->10.72% (4,010,240B) 0x64977E4: QSqlQuery::exec(QString const&) (in /usr/lib/libQtSql.so.4.7.1)
      |     | |       | | | |         ->10.72% (4,010,240B) 0x4BD5A9: TM::OpenDBJob::run() (jobs.cpp:889)
      |     | |       | | | |           ->10.72% (4,010,240B) 0x6278933: ??? (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | | |             ->10.72% (4,010,240B) 0x6278C2C: ThreadWeaver::Job::execute(ThreadWeaver::Thread*) (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | | |               ->10.72% (4,010,240B) 0x6277C4D: ??? (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | | |                 ->10.72% (4,010,240B) 0x62781D6: ThreadWeaver::Thread::run() (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | | |                   ->10.72% (4,010,240B) 0x8A6330C: ??? (in /usr/lib/libQtCore.so.4.7.1)
      |     | |       | | | |                     ->10.72% (4,010,240B) 0xB6518B8: start_thread (pthread_create.c:300)
      |     | |       | | | |                       ->10.72% (4,010,240B) 0x970501B: clone (clone.S:112)
      |     | |       | | | |                         
      |     | |       | | | ->00.06% (21,760B) in 1+ places, all below ms_print's threshold (01.00%)
      |     | |       | | | 
      |     | |       | | ->10.76% (4,024,320B) 0x17C1C8DC: sqlite3VdbeExec (sqlite3.c:52583)
      |     | |       | | | ->10.76% (4,024,320B) 0x17C0481E: sqlite3_step (sqlite3.c:58706)
      |     | |       | | |   ->10.76% (4,024,320B) 0x179A1CBF: ??? (in /usr/lib/qt4/plugins/sqldrivers/libqsqlite.so)
      |     | |       | | |     ->10.76% (4,024,320B) 0x179A2ACD: ??? (in /usr/lib/qt4/plugins/sqldrivers/libqsqlite.so)
      |     | |       | | |       ->10.76% (4,024,320B) 0x64977E4: QSqlQuery::exec(QString const&) (in /usr/lib/libQtSql.so.4.7.1)
      |     | |       | | |         ->05.96% (2,229,760B) 0x4BDEE4: TM::OpenDBJob::run() (jobs.cpp:901)
      |     | |       | | |         | ->05.96% (2,229,760B) 0x6278933: ??? (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | |         |   ->05.96% (2,229,760B) 0x6278C2C: ThreadWeaver::Job::execute(ThreadWeaver::Thread*) (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | |         |     ->05.96% (2,229,760B) 0x6277C4D: ??? (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | |         |       ->05.96% (2,229,760B) 0x62781D6: ThreadWeaver::Thread::run() (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | |         |         ->05.96% (2,229,760B) 0x8A6330C: ??? (in /usr/lib/libQtCore.so.4.7.1)
      |     | |       | | |         |           ->05.96% (2,229,760B) 0xB6518B8: start_thread (pthread_create.c:300)
      |     | |       | | |         |             ->05.96% (2,229,760B) 0x970501B: clone (clone.S:112)
      |     | |       | | |         |               
      |     | |       | | |         ->04.80% (1,794,560B) 0x4BD635: TM::OpenDBJob::run() (jobs.cpp:895)
      |     | |       | | |           ->04.80% (1,794,560B) 0x6278933: ??? (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | |             ->04.80% (1,794,560B) 0x6278C2C: ThreadWeaver::Job::execute(ThreadWeaver::Thread*) (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | |               ->04.80% (1,794,560B) 0x6277C4D: ??? (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | |                 ->04.80% (1,794,560B) 0x62781D6: ThreadWeaver::Thread::run() (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | |                   ->04.80% (1,794,560B) 0x8A6330C: ??? (in /usr/lib/libQtCore.so.4.7.1)
      |     | |       | | |                     ->04.80% (1,794,560B) 0xB6518B8: start_thread (pthread_create.c:300)
      |     | |       | | |                       ->04.80% (1,794,560B) 0x970501B: clone (clone.S:112)
      |     | |       | | |                         
      |     | |       | | ->04.93% (1,841,920B) 0x17BE6682: sqlite3BtreeMovetoUnpacked (sqlite3.c:49769)
      |     | |       | | | ->04.93% (1,841,920B) 0x17C1BA1A: sqlite3VdbeExec (sqlite3.c:63914)
      |     | |       | | |   ->04.93% (1,841,920B) 0x17C0481E: sqlite3_step (sqlite3.c:58706)
      |     | |       | | |     ->04.93% (1,841,920B) 0x179A1CBF: ??? (in /usr/lib/qt4/plugins/sqldrivers/libqsqlite.so)
      |     | |       | | |       ->04.93% (1,841,920B) 0x179A2ACD: ??? (in /usr/lib/qt4/plugins/sqldrivers/libqsqlite.so)
      |     | |       | | |         ->04.93% (1,841,920B) 0x6496A44: QSqlQuery::exec() (in /usr/lib/libQtSql.so.4.7.1)
      |     | |       | | |           ->04.93% (1,841,920B) 0x48E6A8: cachedMetaInfo(KFileItem const&) (projectmodel.cpp:1285)
      |     | |       | | |             ->04.93% (1,841,920B) 0x48F1AB: UpdateStatsJob::run() (projectmodel.cpp:1336)
      |     | |       | | |               ->04.93% (1,841,920B) 0x6278933: ??? (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | |                 ->04.93% (1,841,920B) 0x6278C2C: ThreadWeaver::Job::execute(ThreadWeaver::Thread*) (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | |                   ->04.93% (1,841,920B) 0x6277C4D: ??? (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | |                     ->04.93% (1,841,920B) 0x62781D6: ThreadWeaver::Thread::run() (in /usr/lib/libthreadweaver.so.4.4.0)
      |     | |       | | |                       ->04.93% (1,841,920B) 0x8A6330C: ??? (in /usr/lib/libQtCore.so.4.7.1)
      |     | |       | | |                         ->04.93% (1,841,920B) 0xB6518B8: start_thread (pthread_create.c:300)
      |     | |       | | |                           ->04.93% (1,841,920B) 0x970501B: clone (clone.S:112)
      |     | |       | | |                             
      |     | |       | | ->00.13% (49,920B) in 1+ places, all below ms_print's threshold (01.00%)
      |     | |       | | 
      |     | |       | ->00.10% (35,840B) in 1+ places, all below ms_print's threshold (01.00%)
      |     | |       | 
      |     | |       ->00.03% (12,800B) in 1+ places, all below ms_print's threshold (01.00%)
      |     | |       
      |     | ->00.02% (8,256B) in 1+ places, all below ms_print's threshold (01.00%)
      |     | 
      |     ->01.43% (534,328B) in 15 places, all below massif's threshold (01.00%)
      |       
      

      you can find the source code at http://websvn.kde.org/trunk/KDE/kdesdk/lokalize/src/tm/jobs.cpp?revision=1215120&view=markup
      all TM::OpenDBJob::run() does is calling 'SELECT count(id) FROM main" on a database as large as 48mb. after that memory usage grows from 54 to 61 mb (viewed via top).
      if i add only two lines of code:
      db.close();
      db.open();
      after the call, then memory usage stays at 54 meg.

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

              People

              • Assignee:
                zhongle honglei zhang
                Reporter:
                shaforostoff Nick Shaforostoff
              • Votes:
                4 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Gerrit Reviews

                  There are no open Gerrit changes