QFileDialog hidden behind main window

In my application, the main window is usually displayed as fullscreen with the WindowStaysOnTopHint window flag set.
If I then display the QFileDialog to open a new file, about half of the time the dialog is displayed behind the main window (I can see it, as the main window is partially transparent). Due to the window flag WindowStaysOnTopHint, I'm unable to bring it to the foreground (though keyboard navgation obiously still works). The other half of the time it works just fine.
Also, the dialog keeps the same behaviour per session of my application (i.e., if it's displayed once behind the main window it will do so all the time until I quit the app, and vice versa)

This happens whether it use the static QFileDialog::getOpenFileName() method or construct a QFileDialog object, setting its modality to true and display it via show(). This only happened since Qt 5.4.2, it worked before...