Uploaded image for project: 'Qt for Python'
  1. Qt for Python
  2. PYSIDE-996

Current COIN Wheel Tester Problems

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: P1: Critical
    • Resolution: Fixed
    • Affects Version/s: 5.12
    • Fix Version/s: 5.12.4, 5.13.2
    • Component/s: PySide, Shiboken
    • Labels:
      None
    • Commits:
      09f28e9e1d989a70c876138a4cf24e35c67e0fbb (qt/qt5/5.12) 4840fd8440eb3d6cab005f3a5a6a27767c57ec91 (pyside/pyside-setup/dev)

      Description

      When PySide and shiboken are tested, they are built as wheels. After the normal unit tests, the script wheel_tester.py performs additional integration tests. These tests show some problems on certain platforms.
      Instead of removing the small problems of the script, completely unrelated problems had to be circumvented in absence of the COIN staff.

      These errors may be hiding since a while, already. They became a problem, when I tried to build an extension for wheel_tester.py, that should use PyInstaller to ensure that a built demo script works on all platforms. Automatically Test Small Example With PyInstaller

      Linux openSUSE_42_3 Problem

      Error on Linux openSUSE_42_3: At the end of the log, you can see certain lines which report a problem with the sphinx package. The intended installer test could therefore not run.
      This was circumvented by COIN: Skip doc build for Python 2, but erroneously classified as a sphinx bug.
      If you take a closer look, you see:

      build.go:192: Traceback (most recent call last):
      build.go:192:   File "/home/qt/work/pyside/pyside-setup/env/bin/sphinx-build", line 7, in <module>
      build.go:192:     from sphinx.cmd.build import main
      build.go:192:   File "/home/qt/work/pyside/pyside-setup/env/lib/python2.7/site-packages/sphinx/cmd/build.py", line 39
      build.go:192:     file=stderr)
      build.go:192:         ^
      build.go:192: SyntaxError: invalid syntax
      

      When inspecting the source code of build.py, you realize that this code cannot belong to Python 2.7, which has some comment instead in line 39. The file=stderr belongs to the Python 3 version of sphinx/build.py line 39.

      That means: The Python 2 library contains Python 3 code, which of course is not suitable for Python 2. It is hard to imagine how such an error is possible in an automated environment.

      Linux RHEL_7_4 Problem

      Error on Linux RHEL_7_4: The end of the log shows PyInstaller trying to build a test script. It fails on Python 3 with the following traceback:

      build.go:192: Traceback (most recent call last):
      build.go:192:   File "/opt/rh/rh-python36/root/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
      build.go:192:     "__main__", mod_spec)
      build.go:192:   File "/opt/rh/rh-python36/root/usr/lib64/python3.6/runpy.py", line 85, in _run_code
      build.go:192:     exec(code, run_globals)
      build.go:192:   File "/home/qt/work/pyside/pyside-setup/env3/lib/python3.6/site-packages/PyInstaller/__main__.py", line 118, in <module>
      build.go:192:     run()
      build.go:192:   File "/home/qt/work/pyside/pyside-setup/env3/lib/python3.6/site-packages/PyInstaller/__main__.py", line 111, in run
      build.go:192:     run_build(pyi_config, spec_file, **vars(args))
      build.go:192:   File "/home/qt/work/pyside/pyside-setup/env3/lib/python3.6/site-packages/PyInstaller/__main__.py", line 63, in run_build
      build.go:192:     PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
      build.go:192:   File "/home/qt/work/pyside/pyside-setup/env3/lib/python3.6/site-packages/PyInstaller/building/build_main.py", line 838, in main
      build.go:192:     build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
      build.go:192:   File "/home/qt/work/pyside/pyside-setup/env3/lib/python3.6/site-packages/PyInstaller/building/build_main.py", line 784, in build
      build.go:192:     exec(text, spec_namespace)
      build.go:192:   File "<string>", line 17, in <module>
      build.go:192:   File "/home/qt/work/pyside/pyside-setup/env3/lib/python3.6/site-packages/PyInstaller/building/build_main.py", line 241, in __init__
      build.go:192:     self.__postinit__()
      build.go:192:   File "/home/qt/work/pyside/pyside-setup/env3/lib/python3.6/site-packages/PyInstaller/building/datastruct.py", line 158, in __postinit__
      build.go:192:     self.assemble()
      build.go:192:   File "/home/qt/work/pyside/pyside-setup/env3/lib/python3.6/site-packages/PyInstaller/building/build_main.py", line 573, in assemble
      build.go:192:     self._check_python_library(self.binaries)
      build.go:192:   File "/home/qt/work/pyside/pyside-setup/env3/lib/python3.6/site-packages/PyInstaller/building/build_main.py", line 676, in _check_python_library
      build.go:192:     raise IOError(msg)
      build.go:192: OSError: Python library not found: libpython3.6mu.so.1.0, libpython3.6m.so.1.0, libpython3.6.so.1.0
      build.go:192: This would mean your Python installation doesn't come with proper library files.
      build.go:192: This usually happens by missing development package, or unsuitable build parameters of Python installation.
      build.go:192: 
      build.go:192: * On Debian/Ubuntu, you would need to install Python development packages
      build.go:192:   * apt-get install python3-dev
      build.go:192:   * apt-get install python-dev
      build.go:192: * If you're building Python by yourself, please rebuild your Python with `--enable-shared` (or, `--enable-framework` on Darwin)
      

      The error message seems to make not much sense. For some reason, the correct library cannot be found.
      As an intermediate circumvention, the build process was simply skipped on any failure, because we wanted to test running the compiled script, and not the compilation itself.

      How To Resolve The Problem

      These errors became visible while developing the above mentioned PyInstaller test. Although PyInstaller has some issues as well, these problems are purely COIN misconfigurations and out of the scope of the original scripting task.

      The following steps are necessary to fix them completely:

      Amendment 2019-04-28: Follow-Up PyInstaller Problem

      While these two COIN problems were fixed, another issue came up:
      macOS with Python 2.7.16 crashes the generated PyInstaller test program.

      Reason:
      Python 2.7.16 includes the Tcl/Tk library in the Python distribution in a way that does not work with the usual framework layout that PyInstaller expects on macOS.
      More info: https://www.python.org/download/mac/tcltk/
      The error message is:

      Traceback (most recent call last):
        File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PyInstaller/loader/rthooks/pyi_rth__tkinter.py", line 28, in <module>
          raise FileNotFoundError('Tcl data directory "%s" not found.' % (tcldir))
      

      The hard-to-find reason for this error lies in the numpy module that was installed for other tests.
      It turns out that Python 3.7 has the same problem when Tcl/Tk is used, and this has been reported since a while in https://github.com/pyinstaller/pyinstaller/pull/3830 .
      That problem is not very visible because PyInstaller is not used often with Tcl/Tk. But on Python 2.7.16 the problem is more tricky, because the bug is triggered by the installation of numpy, which is a very frequently used module.
      The proposed patch is the same for both Python versions and works fine. Until this is fixed, we excluded Tcl/Tk in the spec file.

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

            • Assignee:
              falt Simo Fält
              Reporter:
              ctismer Christian Tismer
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Gerrit Reviews

                There are no open Gerrit changes