When QTestLog intercepts a qFatal(), qtestlog.cpp's QTest::handleFatal() has the active loggers first leaveTestFunction() and then stopLogging(). The point of that is to wrap up logging cleanly, for example ensuring that XML logs are well-formed, before we return to calling code that's about to qAbort(). Loggers (should) end their stopLogging() by calling that of their base class (which is not as abstract as its name might suggest), but this does nothing, leaving the closing of the file handle (where it's an open file rather than stdout) to its destructor (which no derived class can accidentally forget to call). Since the subsequent qAbort() bypasses the destructor (as the stack doesn't unwind back through something in QtTest that would cause it), most platforms are left with incomplete output (including malformed XML). This could be fixed by flushing output and, when not stdout, closing the output stream in stopLogging(), and perhaps flagging that we have done so, for the destructor to check.

The above is P2; the following P3.

On Android, we apparently actually get log output that ends gracefully even after a QFATAL log line, as intended. The subsequent qAbort(), except on MS, calls std::abort(), whose spec says

Whether open resources such as files are closed is implementation defined.

This would seem to suggest Android's has opted to (flush and) close open file handles for us, where other platforms do not. We thus get a foretaste of what output shall look like if we fix the issue above.

That log output does not mention that some tests defined by the test-class whose tests were being run may have simply been skipped as a result of one of them causing a QFATAL. Indeed, the only hint of this is the presence of a QFATAL line before the FAIL! line for the last test run. It would be good to have the final summary report generated by QtTest contain some clue that some test-calls were not run.

It should be possible for qtestcase.cpp to tell the logging system how many test functions to expect to see, how many global data rows to see exercised for each and, for each test function (as it embarks on it), how many test data rows the logging system should expect to see for that function. That would enable the logging code to notice when there's a short-fall and duly include in its final logging output's summary of the test results seen a summary of the short-fall when there is one. For the counts of rows in global and test data tables, QTestTable::dataCount() has the answers; indeed, the logging code could consult those directly instead of needing qtestcase.cpp to tell it. For the number of test functions, TestMethods could add a size() function returning its m_methods.size() that its invokeTests() could communicate to the logging system. QTest::qRun() would also need to let the logging system know about the number of repetitions of the whole test class it intends to run.

Then the logging system could,

• when told to leaveTestFunction(), detect any shortfall in how many test data and global data iterations of it have been run, which it can report as appropriate,
• when told to stopLogging(), detect any short-fall in the tests run and duly report on that alongside its Totals line.

(The first of those may be complicated by the pattern of calls to start/leave functions around test functions, as there are various complications to that story. The logging system may also need to keep track of which function it was most recently logging results for, so that it can notice a change if asked to log for a new function before it expected to. It may in fact only be able to report an actual shortfall for a function at the point where a new function is sighted or the end of logging is reached.)

What to do:

• Flush the output stream in QAbstractTestLogger::stopLogging() and, if it's not stdout close it.
  • Optional: flag that this has been done and assert it has in the destructor. This should catch any test loggers (present or future) whose stopLogging() neglects to call that of the base.
• Have TestMethods::invokeTests tell QTestLog, in some way, its m_methods.size() to enable the following:
• Have QTestLog::leaveTestFunction() (or s/leave/enter/) track how many times it's called, so that stopLogging() can detect whether this falls below the number of test functions QTestLog was told to expect.
• Have QTestLog::stopLogging(), before it iterates loggers, telling each to stop, check for any shortfall in functions and report it in some form, as an omission of some of the tests that were meant to be run but apparently weren't.
  • Optionally have QTestResult::setCurrentGlobalTestData() let QTestLog know it's been called, so that QTestLog::leaveTestFunction() can check this matches QTestTable::globalTestTable()->dataCount() and raise some complaint if it doesn't.
  • Optionally have QTestLog() – either between enterTestFunction() and leaveTestFunction() or, when they happen, between successive global test data changes – track how many test results (pass, fail, xpass, skip but not xfail) it gets to record and check this matches QTestTable::currentTestTable()->dataCount(), likewise complaining if it doesn't.

Except that there's a whole heap of other complications to take into account, such as:

• The counts won't match if the test has been invoked with a selection of function and function:tag combinations so that not all of the apparently available candidates are in fact run, so we need to let QTestLog to forget all of the above when the command-line does that.
• If a *_data() function creates a partial table and then QSKIP() s, its function won't be run so the number of runs won't match the table size. This can probably be handled by the fact that a skipping *_data() should be the only case of zero calls to a function between its enterTestFunction() and leaveTestFunction().

and I've surely not thought of them all – you'll discover them when you try to do this.