There are a few things to mention.

Sometimes users might want to toggle a feature either ON or OFF after an initial configuration of Qt.
We have a mechanism that tries to support that, but it might not always be reliable.

For example in a -developer-build, qt_find_package won't look for packages on reconfiguration, so even though a feature is toggled on, it's associated 3rd party package might not be found on reconfiguration.

And toggling -developer-build itself is risky, because of its overloaded meaning.
Are we switching from a -prefix build to a no-prefix one, or vice versa?
Is it just about private tests?

The safest thing we can offer (in my opinion) is to document that any toggling of Qt features should be done after CMakeCache.txt is removed.

We could document that at https://doc.qt.io/qt-6/configure-options.html#including-or-excluding-features

When using configure, that can also be done using the -redo option.

Also for -developer-builds, we can try to add some checks to ensure it's never accidentally toggled, or investigate if we can make it work just for the private_tests part, and not for the -no-prefix part.
The reason to do that check is that sometimes Qt Creator seems to toggle it. See

https://bugreports.qt.io/browse/QTBUG-112957?focusedId=739922&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-739922