We embed Qt controls into native floating palettes of Adobe Illustrator, i.e. of a 3rd-party non-Qt app. After the widget is created we do this:

window->setProperty("_q_embedded_native_parent_handle", (WId)hParent);

HWND h = static_cast<HWND>(QGuiApplication::platformNativeInterface()->nativeResourceForWindow("handle", window));

SetParent(h, hParent);

window->setFlags(Qt::FramelessWindowHint);

However, since Qt 5.9, our widgets are shifted up-left by (8, 31) pixels.

Before Qt 5.9, thanks to the fix for QTBUG-40578, QWindowsWindow::frameMarginsDp() always returned empty margins for Qt::FramelessWindowHint.

But there is no such explicit check on Qt::FramelessWindowHint since Qt 5.9, QWindowsGeometryHint::frame() now returns {8, 31, 8, 8} as the margins, based on the style. I believe WindowCreationData::style should not include WS_CHILD for embedded frameless windows. Then the margins will again be zeros. A patch (for Qt 5.12) is attached.