parameterization of locating external dependencies

Products should never use cpp.staticLibraries and cpp.dynamicLibraries directly. instead, they should pull in all dependencies via Depends items. however, not all libraries external to a particular project come with qbs Module files. to overcome this issue, module providers have been implemented in QBS-1107. a qmake based provider for qt modules (as a replacement for qbs-setup-qt) and a pkg-config based generic one have been initially implemented.

one remaining question is how to tweak the detection and instantiation of particular modules by specific providers, e.g., providing name mappings for package manager based providers, or library+include file lists for a PathProbe based (manual) provider.

one way that comes to mind is explicitly instantiating specific providers for specific modules, thus preempting the fallback provider. however, this would cause configured and defaulted modules being instantiated differently even when ultimately the same provider class would be responsible. also, how global provider options should apply to specific providers is unclear.

a cleaner approach seems to be to have a purely declarative ExternalPackage item. module providers would try to locate matching instances of these (the mechanism could be presumably the same as for module providers themselves). if no package is found for a particular provider/module combo, the behavior would depend on the provider - package manager based providers could default to a package name algorithmically derived from the module name, while the manual provider would be skipped.

the ExternalPackage name is up to debate, Package being a shorter contender - however, such a generic name may collide with another concept, e.g., output artifacts for linux distros.

one question is whether the item would (1) directly contain the properties for all providers, or instead (2) contain sub-items for specific providers (source types).

(2) is pretty much a direct mapping of qt5's "new" configure system, with the exception that qt requires explicit instantiation of all sources.

for (1), the syntax could be like this:

ExternalPackage {
     // default providers to run for this package - QBS-1604
     qbsModuleProviders: ["pkgconfig", "conan"] 

     pkgconfigPackage: "mylibrary"
     conanPackage: "myCoolLibrary"

     // for manual provider
     libraryName: "mylibrary"
     includeNames: ["mylibrary.h"]
}

the available properties would be declared inside the providers themselves:

// module-providers/pkgconfig.qbs
Provider {
     PackageProperties {
          // has default, can be instantiated without config
          property string pkgconfigPackage: module.replace([...])
     }
}

// module-providers/manual.qbs
Provider {
     PackageProperties {
         // no defaults, is skipped without config
         property string libraryName
         property string includeNames
    }
}

the ExternalPackage item would basically do "magical multiple inheritance" from the found PackageProperties items, somewhat similarly to what is done for dependency parameters (QBS-585).

extending ExternalPackages "from the outside" can be easily done by collecting properties individually, instead of at the file level. the override priority would be dictated by the package search path order, as expected:

// <customSearchPath>/packages/<packageName>.qbs:
ExternalPackage {
    conanPackage: "package_name"
}
// <qbsSearchPath>/packages/<packageName>.qbs
ExternalPackage {
    pkgconfigPackage: "Package-Name"
    conanPackage: "Package-Name"  // this is shadowed by the override above
}

this should even allow somewhat clear qbsModuleProviders override semantics, as the evaluation order is clear.

for (2), the syntax could be like this:

ExternalPackage {
     // for qbs pkgconfig provider
     PkgConfigSource {
          packageName: "mylibrary"
     }

     // for conan provider
     ConanSource {
          packageName: "myCoolLibrary"
          // other props?
     }

     // for manual provider
     ManualSource {
          libraryName: "mylibrary"
          includeNames: ["mylibrary.h"]
          // other props
     }
}

the various FooSource items would be declared the usual way by deriving from a PackageSource base item. this is probably easier to implement than the magic inheritance of approach (1).

the code is more verbose than in (1), which is theoretically good for legibility, but also a lot of noise, considering that ExternalPackages are supposed to be "mass-produced".

the isolation between different providers' configs is better than in (1), but that might be actually counter-productive for some properties - a file list could be used also to feed the pkg-config based provider to make it produce modules compliant with QBS-942. to overcome this, common properties would have to be declared upstream in the ExternalPackage item itself, which somewhat limits extensibility, but is also good for consistency.

no explicit qbsModuleProviders property needs to be present, as the presence and order of the instantiated sub-items is sufficient. it is even possible to configure multiple sources of the same type, though it's not clear whether that is useful (qt does that for configuring platform-specific alternatives, but that can be done differently here by using complex expressions for the property values).

extending ExternalPackages can be done at the property level as in (1), but it would be also possible to externalize individual sources to separate files:

<qbsSearchPath>/packages/<packageName>/package.qbs
<qbsSearchPath>/packages/<packageName>/conan.qbs // contains ConanSource item
<qbsSearchPath>/packages/<packageName>/manual.qbs // contains ManualSource item 

note however that either extension scheme invalidates the statements made about qbsModuleProviders above. it would be necessary to explicitly declare a priority within the overriding item, and in case of declaring (or even overriding) multiple instances of one type things would be even less clear, esp. if the files actually live in the same folder (and thus the search path imposes no order).