Change the structure of the build process and sources

Overview

When the work of porting the old PySide (Qt4) to Qt5 started in 2015, the repositories
were still split into multiple parts, also when the new team took over in 2016.
After about a year, the team decided to move all the different repositories
into a mono-repository structure, which helped to forget about
version problems, and a quicker development, branching, etc.

Before the initial official release for 5.12, the repository was
configured to build one big wheel containing everything: PySide, Shiboken,
and Shiboken Generator.

Even if this was convenient, again, for having everything on the same
place, there were many user stories, where people were relying
on the Shiboken generator, or only on PySide as user, so generating
different wheels was the next step.

Building three wheels from the same setup.py file was a challenging
task, that required a large set of changes related the build scripts
structure we currently have, and successfully ended into the 3 main
wheels that we currently have:

• PySide (on PyPi)
• Shiboken, the python helper module (on PyPi)
• Shiboken, the generator (not on PyPi due to the libclang dependency)

The Problem

Having a process that generates 3 wheels from the same setup.py file
is problematic if people are relying on editable build/install https://pip.pypa.io/en/latest/cli/pip_install/#editable-installs
because that approach is not compatible with more than one .egg-info
directory:

See errors here: https://gist.github.com/cmaureir/82fcb52bcbea5ce70362cb4270fc3f8c

Additionally,
when a user on a non-supported platform runs a pip install pyside6 command
if there is no pre-built wheel, pip will start building the source package
with a simple pip install, which generates the same errors.

Even if this approach would mean that users would need to build the whole
project in case they want to use it on an unsupported platform,
a process that will be really slow depending on the host, it's a solution
to at least be available on those platforms.

Complementary to that, the current source package we include
contains everything, and we cannot securely split the source only
for pyside (or shiboken), so people can build it directory with
a setup.py call.

Last but not least, going back to a many-repo approach for the project
will introduce a lot of extra work regarding the synchronization between
repositories, branches, etc.

Current scenario

At the moment the structure of the repository looks like this:

./build_scripts/
./setup.py
./sources/pyside6/
./sources/shiboken6/shibokenmodule
./sources/shiboken6/generator
./...

when calling globally python setup.py build ... that will trigger
three calls, with an option --internal-build-type for each project:

python setup.py build ... --internal-build-type=shiboken6
python setup.py build ... --internal-build-type=shiboken_module
python setup.py build ... --internal-build-type=pyside6

and thus, having independent processes, allows us to generate 3 different
wheels.

Solution proposal

Since having many repositories is not a solution,
the only solution will mean to have source directories for each project
so we could be able to build them independently:

./setup.py
./sources/pyside6/build_scripts/
./sources/pyside6/setup.py
./sources/shiboken6/shibokenmodule/build_scripts/
./sources/shiboken6/shibokenmodule/setup.py
./sources/shiboken6/generator/build_scripts/
./sources/shiboken6/generator/setup.py
./...

Now when the global setup.py is called, that will explicitly
go into the 3 different projects, and call internally
the same command:

python setup.py build ...
-> go on sources/shiboken6/shibokenmodule/
-> python setup.py build ...
-> go on sources/shiboken6/generator
-> python setup.py build ...
-> go on sources/pyside6
-> python setup.py build ...

This will ensure, that creating a zip file on the sources/pyside6, for example,
will contain the necessary scripts to build the project, as long
as the dependencies are fulfilled,
enabling us to provide the source packages, so people can build it on any
infrastructure.

Additionally, this new structure will help the efforts of making the whole project more cross-compatible friendly, in the sense that due to the independence of each project-source-structure, the options to cross compile tools for host and target will be simple than taking care of the general build process, and tackle all the possible conditions.

Lastly, this new structure might help the generation of new 'addons' wheels that people can install on top of a PySide6 one, like for example M2M Protocols, creating a new special directory:

...
./sources/pyside6m2m/build_scripts
./sources/pyside6m2m/setup.py
...

Complications

We rely on CMake for our build process, and distributions usually rely
on calling cmake directly, instead of using the setup.py file, so we need
to ensure, that also that process is properly supported.

PEP517 will probably become the standard to build python projects
https://www.python.org/dev/peps/pep-0517
and with the help of a pyproject.toml file https://www.python.org/dev/peps/pep-0518/
projects will change a bit the way of using setup.py,
however, that will not be the solution for C-extensions all the time,
and projects still would need to rely on them.
See https://stackoverflow.com/questions/66157987/how-to-build-a-c-extension-in-keeping-with-pep-517-i-e-with-pyproject-toml-ins