Camera view matrix computation unstable (regression)

We have started to use Qt3D in QGIS with great success (thanks for that!), but recently we have found out that a bugfix for QTBUG-65896 introduced in recent Qt versions caused a regression. This is the change: https://codereview.qt-project.org/#/c/217105/

Since then, the camera in QGIS 3D view becomes very "shaky" during transitions when the camera's position coordinates get to the order of tens of thousands or more.

I am attaching a simple test case and two recorded videos of the test app running: one with Qt 5.9.0 (good) and the other one with Qt 5.9.5 (bad).

My assumption is that some part of the calculation introduces numerical errors causing the shakiness. I guess the values in order of 10^4 or 10^5 are still quite reasonable and so they should not cause such issues...