Not possible to build Qt without rpath via -no-rpath

I passed -no-rpath but still see {{ -Wl,-rpath,/Users/torarne/build/qt/6.x-no-rpath/qtbase/lib}} in the linker lines and uses of `-install_name @rpath/libQt6OpenGLWidgets_debug.6.dylib`

 
❯ cat CMakeCache.txt | grep RPATH
//RPATH for installed binaries
CMAKE_INSTALL_RPATH:STRING=
CMAKE_SKIP_INSTALL_RPATH:BOOL=NO
CMAKE_SKIP_RPATH:BOOL=NO
//Build with RPATH
//ADVANCED property for variable: CMAKE_SKIP_INSTALL_RPATH
CMAKE_SKIP_INSTALL_RPATH-ADVANCED:INTERNAL=1
//ADVANCED property for variable: CMAKE_SKIP_RPATH
CMAKE_SKIP_RPATH-ADVANCED:INTERNAL=1
QT_FEATURE_LABEL_rpath:INTERNAL=Build with RPATH
 
Looks like our build system is not propagating our internal rpath config cmake variable to the cmake ones. 
 
Tried  explicitly setting -DCMAKE_SKIP_RPATH=ON but that only removed -Wl,-rpath, so something else is needed to not put @rpath into the install_name