pkg-config module provider should decompose dependencies

Problem Statement

pkg-config --libs fully resolves transitive dependencies. this results in a naive implementation of the pkg-config provider creating modules which:

• name libraries which they don't actually "own"
• don't declare transitive module dependencies

with this, we sooner or later run into an anomaly: suppose an app that links against two libraries, each of which depends on a third one. suddenly there are two modules within one product which refer to the same physical files.

in principle, this goes well for linking, as the linker does not mind duplicates. however, command lines may get very long this way, which creates challenges. therefore, qbs will de-duplicate the linker arguments. but this is problematic in turn, as packages with static libraries containing circular dependencies must actually list these libraries multiple times - which is why de-duplication can be disabled via the cpp.removeDuplicateLibraries property, bringing us back to square one. therefore, it would be preferable to eliminate late de-duplication entirely - it should all happen only at the dependency graph level, which means that the modules need to be actually atomic.

other operations (e.g., related to deployment in the widest sense) may also rely on accurate dependencies. while these can in principle use ldd/otool to obtain low-level data, this is ugly, as it requires exclusion rules to be defined outside the existing module concept. it would be preferable to use such tools only to validate the declarations for the manual package provider.

generally speaking, it's just a bad idea to have the same physical files referred to from multiple "descriptors" (modules), especially ones that can end up in one context (product). it's always better to condition the input to fit the data model rather than post-processing the output to paper over problems. just like database normalization, this leads to a cleaner, more maintainable, and more efficient system.

another effect of ignoring the problem is that the pkg-config provider potentially produces a different module structure than other providers, which is at least confusing, if not actually problematic.

Possible Solutions

there are fundamentally two approaches:

• reverse pkg-config's resolution by subtracting the fully resolved dependencies (obtained via --print-requires[-private] and subsequent --libs)
• bypass the resolution in the first place by parsing the .pc files ourselves, which would also be a lot faster. see QBS-1615.

there is an additional complication: some packages (e.g., ncurses and capnp-rpc) come with .pc files that contain already resolved dependencies (tinfo resp. kj for the above ones) even though these libraries have own .pc files as well. this is presumably done to save the transitive lookup, which pkg-config would have to do anyway, but does us a disservice. i can think of the following approaches:

• let the user explicitly declare dependencies (see QBS-1452). this obviously imposes a continuous maintenance burden, but otoh that is required for the manual module provider anyway.
• make the provider single-shot/eager, i.e., have it preemptively process all .pc files and correlate them. due to the performance demands, this is realistic only when we parse the .pc files ourselves (possibly using a service written in c++).
• make the provider correlate only packages that actually appear in a Product (or better the entire Project). an infrastructure which makes the providers have a holistic view of the project would be generally advantageous, but would obviously require completely rewriting the provider interface. also, this approach would produce inconsistent results between projects which directly use some package which is also used transitively and those which don't - this might be a problem for example when a deployment database is kept on a target device which is used for multiple projects.

Backwards Compatibility Concerns

consumers of generated modules (i.e., Products) will see some transitive dependencies suddenly pop up. that must be expected any time anyway, so it won't be a problem.

the provider mechanism itself might require backwards incompatible changes.