Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-86533

Yocto and QNX builds are missing -rpath-link linker flag which causes non-prefix builds to fail

    XMLWordPrintable

Details

    Description

      Trying to build a non-prefix qtdeclarative build targeting yocto qemu armv7 fails when linking the qtdeclarative cross-compiled apps.

      Sample linking failure

      $ ninja -v qmleasing                                                                                                                                                                                                                                                  
      [1/1] : && /opt/b2qt/2.7.2/sysroots/x86_64-pokysdk-linux/usr/bin/arm-poky-linux-gnueabi/arm-poky-linux-gnueabi-g++   -march=armv7-a -mthumb -mfpu=neon -mfloat-abi=hard --sysroot=/opt/b2qt/2.7.2/sysroots/armv7at2hf-neon-poky-linux-gnueabi --sysroot=/opt/b2qt/2.7.2/sysroots/armv7at2hf-neon-poky-linux-gnueabi -O2 -pipe -g -feliminate-unused-debug-types  -g -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed tools/qmleasing/CMakeFiles/qmleasing.dir/qmleasing_autogen/mocs_compilation.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/main.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/mainwindow.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/segmentproperties.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/splineeditor.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/.rcc/qmlcache/resources/Button_qml.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/.rcc/qmlcache/resources/preview_qml.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/.rcc/qmlcache/resources/qmlcache_loader.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/.rcc/qrc_resources_qmlcache.cpp.o -o /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/bin/qmleasing  -Wl,-rpath,/home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib:  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Quick.so.6.0.0  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6QmlModels.so.6.0.0  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Qml.so.6.0.0  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Network.so.6.0.0  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6OpenGL.so.6.0.0  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Widgets.so.6.0.0  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Gui.so.6.0.0  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Core.so.6.0.0  -pthread  /opt/b2qt/2.7.2/sysroots/armv7at2hf-neon-poky-linux-gnueabi/usr/lib/libGLESv2.so  /opt/b2qt/2.7.2/sysroots/armv7at2hf-neon-poky-linux-gnueabi/usr/lib/libxkbcommon.so && :
      FAILED: /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/bin/qmleasing
      : && /opt/b2qt/2.7.2/sysroots/x86_64-pokysdk-linux/usr/bin/arm-poky-linux-gnueabi/arm-poky-linux-gnueabi-g++   -march=armv7-a -mthumb -mfpu=neon -mfloat-abi=hard --sysroot=/opt/b2qt/2.7.2/sysroots/armv7at2hf-neon-poky-linux-gnueabi --sysroot=/opt/b2qt/2.7.2/sysroots/armv7at2hf-neon-poky-linux-gnueabi -O2 -pipe -g -feliminate-unused-debug-types  -g -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed tools/qmleasing/CMakeFiles/qmleasing.dir/qmleasing_autogen/mocs_compilation.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/main.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/mainwindow.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/segmentproperties.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/splineeditor.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/.rcc/qmlcache/resources/Button_qml.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/.rcc/qmlcache/resources/preview_qml.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/.rcc/qmlcache/resources/qmlcache_loader.cpp.o tools/qmleasing/CMakeFiles/qmleasing.dir/.rcc/qrc_resources_qmlcache.cpp.o -o /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/bin/qmleasing  -Wl,-rpath,/home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib:  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Quick.so.6.0.0  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6QmlModels.so.6.0.0  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Qml.so.6.0.0  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Network.so.6.0.0  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6OpenGL.so.6.0.0  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Widgets.so.6.0.0  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Gui.so.6.0.0  /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Core.so.6.0.0  -pthread  /opt/b2qt/2.7.2/sysroots/armv7at2hf-neon-poky-linux-gnueabi/usr/lib/libGLESv2.so  /opt/b2qt/2.7.2/sysroots/armv7at2hf-neon-poky-linux-gnueabi/usr/lib/libxkbcommon.so && :
      /opt/b2qt/2.7.2/sysroots/x86_64-pokysdk-linux/usr/libexec/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/8.3.0/ld: warning: libQt6DBus.so.6, needed by /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Gui.so.6.0.0, not found (try using -rpath or -rpath-link)
      /opt/b2qt/2.7.2/sysroots/x86_64-pokysdk-linux/usr/libexec/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/8.3.0/ld: /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Gui.so.6.0.0: undefined reference to `QDBusMessage::type() const@Qt_6'
      /opt/b2qt/2.7.2/sysroots/x86_64-pokysdk-linux/usr/libexec/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/8.3.0/ld: /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Gui.so.6.0.0: undefined reference to `QDBusUnixFileDescriptor::QDBusUnixFileDescriptor(int)@Qt_6'
      /opt/b2qt/2.7.2/sysroots/x86_64-pokysdk-linux/usr/libexec/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/8.3.0/ld: /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Gui.so.6.0.0: undefined reference to `QDBusPendingCall::error() const@Qt_6'
      /opt/b2qt/2.7.2/sysroots/x86_64-pokysdk-linux/usr/libexec/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/8.3.0/ld: /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Gui.so.6.0.0: undefined reference to `QDBusAbstractInterface::~QDBusAbstractInterface()@Qt_6'
      /opt/b2qt/2.7.2/sysroots/x86_64-pokysdk-linux/usr/libexec/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/8.3.0/ld: /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Gui.so.6.0.0: undefined reference to `QDBusError::type() const@Qt_6'
      /opt/b2qt/2.7.2/sysroots/x86_64-pokysdk-linux/usr/libexec/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/8.3.0/ld: /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Gui.so.6.0.0: undefined reference to `QDBusArgument::operator>>(int&) const@Qt_6'
      /opt/b2qt/2.7.2/sysroots/x86_64-pokysdk-linux/usr/libexec/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/8.3.0/ld: /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Gui.so.6.0.0: undefined reference to `QDBusArgument::QDBusArgument()@Qt_6'
      /opt/b2qt/2.7.2/sysroots/x86_64-pokysdk-linux/usr/libexec/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/8.3.0/ld: /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Gui.so.6.0.0: undefined reference to `typeinfo for QDBusAbstractAdaptor@Qt_6'
      /opt/b2qt/2.7.2/sysroots/x86_64-pokysdk-linux/usr/libexec/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/8.3.0/ld: /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Gui.so.6.0.0: undefined reference to `QDBusArgument::endStructure()@Qt_6'
      /opt/b2qt/2.7.2/sysroots/x86_64-pokysdk-linux/usr/libexec/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/8.3.0/ld: /home/alex/Dev/qt/qt5_cmake/qtbase_built_developer_armv7/lib/libQt6Gui.so.6.0.0: undefined reference to `QDBusMessage::arguments() const@Qt_6'
      ......
      

      The problem is that the link line is missing an -rpath-link flag pointing to the qtbase ./lib dir, and because libQt6DBus.so is not explicitly mentioned on the link line, dbus symbols referenced from libQt6Gui.so are not found.

      As an aside, the equivalent qmake configurations do pass -rpath-link.

      As per
      https://stackoverflow.com/questions/49138195/whats-the-difference-between-rpath-link-and-l
      and the -rpath and -rpath-link documentation at https://sourceware.org/binutils/docs/ld/Options.html#Options
      one would think that passing just -rpath should be enough (it kind of implies -rpath-link), and usually it is enough, but not when combined with the --sysroot flag.

      As per https://cmake.org/pipermail/cmake-developers/2018-July/030746.html
      and the actual source code
      https://github.com/bminor/binutils-gdb/blob/5987401fcbc9933808fa0d84d1b01c93356c39a1/ld/ldelf.c#L1236

      If a -sysroot is provided in conjunction with -rpath, the sysroot is prepended to all absolute path values specified via -rpath. This will cause the linker not to find the needed libraries. The sysroot is not prepended for values provided via -rpath-link.

      Now for some reason CMake doesn't add -rpath-link flags as described in the mailing list post above, and yet the CI CMake qemu builds don't exhibit the problem.

      The reason is that CI builds are doing prefix builds. In a prefix build, $ORIGIN is embedded as an rpath directly into the shared libraries via qt_apply_rpaths().

      The linker has a separate code path where it tries to use the embedded rpaths here
      https://github.com/bminor/binutils-gdb/blob/5987401fcbc9933808fa0d84d1b01c93356c39a1/ld/ldelf.c#L1264

      Because $ORIGIN is not an absolute path, the sysroot is not prepended, and the linker successfully finds the libraries it needs.

      So prefix builds work by mere coincidence.

      Ideally, CMake should be fixed to add -rpath-link like qmake does, but i don't think that's fixed yet up to CMake 3.18 (present stable version).

      A possible workaround for non-prefix builds is to change qt_apply_rpaths() to embed an $ORIGIN rpath even in non-prefix builds (I've confirmed this works).

      By the way, this is probably also the reason why the qtbase cmake build tests were disabled for qemu, as mentioned in the commit message at
      https://codereview.qt-project.org/c/qt/qtbase/+/307774

      Here's the list of open -rpath-link issues on CMake's gitlab, but none of them seem relevant to our case

      https://gitlab.kitware.com/cmake/cmake/-/issues?scope=all&utf8=%E2%9C%93&state=all&search=-rpath-link

      Here's a simplified project that reproduces the problem
      https://github.com/neverpanic/cmake-rpath-link-example

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              alexandru.croitor Alexandru Croitor
              alexandru.croitor Alexandru Croitor
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes