Uploaded image for project: 'Qt for Python'
  1. Qt for Python
  2. PYSIDE-714

Implement proper way of copying msvc, opengl, d3d dlls into installed PySide package

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • P2: Important
    • None
    • 5.11.0
    • PySide
    • None
    • Windows

    Description

      Current State

      TheĀ opengl32sw.dll and d3d*.dll files are only present in a Qt/bin installation when installed via official package, but not during CI builds.
      These are needed for software GL rendering and ANGLE rendering.

      MSVC dlls are not present in the Qt installation at all.

      All of the above need to be bundled in the PySide2 package, to make the package completely standalone, and ready for further user application deployment.

      Current Workaround

      As requested in PYSIDE-696, there is currently a workaround fix in https://codereview.qt-project.org/#/c/232455/ which downloads an archive containing the required dlls and then copies them over, which is only done in Coin CI builds.

      This does not happen for non-Coin builds that users might do locally.

      Proper solution (that I can think of)

      opengl32sw.dll and d3dcompiler_47.dll

      The 'opengl' and 'd3d' dlls issue would be solved by passing additional command line arguments (similar to --openssl) to setup.py, and then copying the dlls from the provided locations.

      This means users will have to download opengl and d3d themselves (probably from download.qt.io). For convenience we could add a command line option that does that for them.

      We should also update the wiki to specify those dlls as requirements on Windows (if you want software rendering or ANGLE).

      MSVC dlls

      For MSVC dlls it's a bit more unclear, but will probably require automatic location detection and then copying the files.

      Due to not really supporting Python 2 on Windows, we are only concerned with the Python 3.5+ case, which is built using Visual Studio 2015. We will also have to take into account the cpu arch of the libraries (x86, x64).

      This means we need to detect and copy a couple of things:

      • C++ runtime library (msvcp140.dll) and its dependency (concrt140.dll / Windows Concurrency Runtime)
      • C runtime library divided into
        • vcruntime140.dll
        • ucrtbase.dll
          • all the 40+ forwarding dlls that are needed for Windows < 10 ( api-ms-
            win-*-l1-1-0.dll)

      The C++ runtime library and vcruntime140.dll can be found inside a Visual Studio installation.

      ucrtbase.dll and the forwarding dlls can only be found when the Windows SDK is installed (not entirely true, they seem to be present in some funky Visual Studio folders as well, but I don't know if those are usable).

      Fortunately for us, it seems that when a user installs Python 3.5+ from python.org, it also automatically installs the C runtime library (ucrtbase.dll + forwarders), so we don't actually need to copy those.

      Which leaves us only with detecting the location and copying of msvcp140.dll, concrt140.dll and vcruntime140.dll.

      And perhaps we could even skip "vcruntime140.dll" because that seems to be present next to the Python executable as well (needs testing). I'm not sure why Python does that instead of installing the vcredist alongside the visual c runtime library installer.

      Note that ucrtbase.dll and forwarders are not actually required on Windows 10, because they are shipped with the system from the get-go (not the forwarders, but they are not needed on Win 10).
      It's only Windows Vista, 7, 8, 8.1 that need them in case if Windows Recommended Updates are disabled, or the machines have no internet connection / updates.

      Additional notes

      This will probably need some revising if / when Python 3 switches to building with MSVC2017 or higher. MSVC2017 claims backwards compatibility with the C runtime library, but not with the C++ runtime library afaik. Which means it's possible other dlls will have to be copied (a fictional msvcp150.dll for example).

      What other tools do with MSVC dlls

      PyInstaller currently has limited support for MSVC deployment, as described in https://github.com/pyinstaller/pyinstaller/blob/develop/doc/usage.rst#id118 under Windows Platform-specific notes section.
      Lengthy discussion here https://github.com/pyinstaller/pyinstaller/issues/1566

      They recommend either:

      • shipping a MSVC redist package as part of an application installer (which installs both c++ libraries and c libraries ucrtbase.dll and forwarders)
      • installing Windows SDK and creating/modifying a .spec file manually to reference the necessary dlls (ucrtbase.dll, forwarders).

      In practice they find out the dll dependencies using their bindepend utility, and for the MSVC case, as long as the MSVC dlls are in PATH, they would be copied.

      Another tool worth mentioning is windeployqt.

      Currently windeployqt:

      • simply copies an MSVC redist installer package for Qt release builds (which doesn't help us because we can't run an .exe as part of wheel installation)
      • copies the actual MSVC dlls for Qt debug builds.

      We'd have to modify windeployqt to allow copying the MSVC dlls (instead of the redist) for release case if we wanted to use it for PySide2 (it wasn't done like this in the first place due to legal reasons afaik, which seems to be allowed now as per https://blogs.msdn.microsoft.com/vcblog/2016/04/18/c-runtime-deployment-why-choosing-applocal/).

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              crmaurei Cristian Maureira-Fredes
              alexandru.croitor Alexandru Croitor
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes