Details
-
Epic
-
Resolution: Unresolved
-
P2: Important
-
None
-
None
-
None
-
Modernize qtestlib
Description
The Qt Test module is a central part of the Qt project, responsible for testing virtually every piece of code we have.
It is further used by other projects, for example KDE. An exact list of customers and users is, afaik, not available. But it is safe to assume that this library is not limited to internal usage and that it will have an effect of developers on choosing Qt.
The current status is considered stable and mature.
Nonetheless there are various pain points, reporting bugs, shortcomings, and gaps; see this Jira filter:
https://bugreports.qt.io/issues/?jql=status%20in%20(Reported%2C%20%22Need%20More%20Info%22%2C%20Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20component%20in%20(Tests%2C%20%22Testing%3A%20qtestlib%22%2C%20%22Testing%3A%20QuickTest%22%2C%20%22Testing%3A%20QtMetrics%22%2C%20%22Test%20Framework%22)
Various Qt developers try to fix with more or less coordinated patches, many of which do not get merged:
https://codereview.qt-project.org/q/project:qt/qtbase+status:open+path:%22%5Esrc/testlib/.*%22
TA bought this issues to attention at the 2025 Contributor Summit:
https://wiki.qt.io/QtCS25_-_QtTest:_state_of_the_module_and_the_way_forward
https://qt-wiki-uploads.s3.amazonaws.com/images/8/8c/Qt_Test_-_QtCS_2025.pdf
This task is research focused. Its goal is to show us a way forward with qtestlib, its modernization, replacement or sufficiency/adequacy. Points to start:
- Investigate our current problems, summarize and merge bugs; Are our CI problems related to this?
- Investigate the current patches on gerrit. Can they be merged into fewer, more systematic changes?
- Investigate where we fall short in required functionality and usability;
- Where our engineers have the biggest problems and how to make their job easier; Which parts of Qt are hard to test and/or flaky
- Investigate other testing frameworks. Can we use them as backend, or as inspiration?
- Investigate patterns and architectures that will improve qtestlib