Uploaded image for project: 'Qt for Python'
  1. Qt for Python
  2. PYSIDE-749

Modularize / decouple the shiboken / PySide2 build process

    Details

    • Type: User Story
    • Status: Closed
    • Priority: P2: Important
    • Resolution: Done
    • Affects Version/s: 5.11.0
    • Fix Version/s: 5.12.0
    • Component/s: PySide, Shiboken
    • Labels:
      None

      Description

      Goal

      We want to improve the Qt For Python build process to provide 3 separate wheel packages for shiboken2, shiboken2-generator and PySide2.

      Nice to have (future)

      We would also like some additional nice to have things or at least pave the way towards implementing them in the future:
      1) Separate builds for each of the of the PySide2 modules. Which will
      2) Ease building external modules based on Qt (think non-essential QtCharts, external qwt library) and
      3) Allow separate wheels for each module (smaller wheels, use what you need, helps upstream distros)

      Proposal

      My proposal is to add a --build-type option (among others) to setup.py. This build type will represent:
      1) A monolithic build (all which is the default) - the same one we had so far, which builds shiboken2, pyside2, tools.
      2) shiboken2 - builds only the shiboken2 module
      3) shiboken2-generator - builds only the shiboken2 generator
      4) pyside2 - builds only pyside2, needs info where to find shiboken2 module and generator
      5) possibly tools - builds pyiside2-tools

      In the future extend this to allow:
      5) module-subset - allow building just one module or a list of pyside2 modules, needs info where to find the shiboken generator and already built pyside2 modules.
      e.g. first build core, gui. Then build only network. Then build only declarative. This will also allow separate wheel generation. And separate wheels for modules like qtcharts, or qtwebengine.
      6) possibly external-module - to ease building of custom modules based on qt (think qwt).

      Implementation

      My idea is to use a multi-step setup.py script which calls itself with different options.
      An example of a top-level setup.py invocation which calls additional setup.py invocations would look like:

      python setup.py bdist_wheel --build-type=all --qmake=/home/alex/qt/bin --jobs=8 --ignore-git --debug --limited-api=yes
      
      • python setup.py bdist_wheel --internal-build-type=shiboken --qmake=/home/alex/qt/bin --jobs=8 --ignore-git --debug --limited-api=yes
        
      • python setup.py bdist_wheel --internal-build-type=pyside2 --shiboken-path=path_to_previous_built_shiboken --qmake=/home/alex/qt/bin --jobs=8 --ignore-git --debug --limited-api=yes
        

      The -build-type of the top-level build can be "all", "shiboken2", "shiboken2-generator", "pyside2". The top level setup.py then calls itself with the same usual arguments, except build-type  is replaced with internal-build-type which actually triggers the whole build. We might also need -shiboken-path to specify where the previous invocation of shiboken was installed.

      If no --build-type is provided, "all" is implied to keep backwards compatibility with previous build commands.

      In the future we might need also a --built-modules=/path for supporting separate module building.

      General steps

      To implement this we will need to:

      1. Refactor and add certain functionality and options to setup.py to accommodate the new build strategy
      2. Separate the copying / installing of shiboken files from the PySide2 ones (currently it's a single step)
      3. Take care of rpaths / PATH in regards to new location of shiboken libraries
      4. Provide pyside2_config.py and shiboken2_config.py as part of the wheels (based on the scriptable application example ones) to allow end-users easier use of the components.
      5. Other things I haven't thought about

      For future per-module builds we would also need to:

      1. Change some of the cmake file logic
      2. Figure out how to pass dependency information for already pre-built modules
      3. Other things

      As the end result of the initial work, we will have 3 separate wheels for shiboken2, shiboken2-generator and pyside2.

       

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

              People

              • Assignee:
                alexandru.croitor Alexandru Croitor
                Reporter:
                alexandru.croitor Alexandru Croitor
                RnD Owner:
                Alex Blasche
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 2 weeks
                  2w
                  Remaining:
                  Remaining Estimate - 2 weeks
                  2w
                  Logged:
                  Time Spent - Not Specified
                  Not Specified

                    Gerrit Reviews

                    There are no open Gerrit changes