Part of the keyboard is not functional when working with vncserver without XKEYBOARD support

We are using the default realvnc (no XKEYBOARD support) on Debian 7. With Qt5.3 and Qt5.4 , the normal keys are mostly working, but we found that the special keys, lock keys, direction keys and the numeric keypad are acting weird, for example, the down arrow key acts like enter...

Here is the debug info dumped when adding this patch: https://codereview.qt-project.org/#/c/103933/

% ./fontsampler
Qt: XKEYBOARD extension not present on the X server.
qt.qpa.xkeyboard: updateKeymap
qt.qpa.xkeyboard: xkb context inlude path: /usr/share/X11/xkb
qt.qpa.xkeyboard: checking for xkeyboard...
qt.qpa.xkeyboard: xkeyboard extension not present on x server
qt.qpa.xkeyboard: read xkb config property
qt.qpa.xkeyboard: rules:
qt.qpa.xkeyboard: model:
qt.qpa.xkeyboard: layout:
qt.qpa.xkeyboard: variant:
qt.qpa.xkeyboard: options:

The example I used is the fontsampler.

Here is what we have in /usr/share/X11/xkb :
% ls
compat/ geometry.dir keymap/ semantics/ symbols.dir
compat.dir keycodes/ keymap.dir semantics.dir types/
geometry/ keycodes.dir rules/ symbols/ types.dir