WebAssembly: shared libraries and dynamic linking

Overview / Intro

Note: the Qt documentation contains the canonical dynamic linking overview and "getting started" instructions. This description duplicates some of that, and then covers development topics in the end.

By default Qt for WebAssembly binaries are statically linked, where all object files are combined into a single .wasm binary. This has several advantages such as allowing for relatively simple distribution, where only small number of files needs to be deployment. In addition static linking enables size and performance optimizations such as dead code elimination and link-time optimization.

Enabling shared libraries splits Qt up into several independent WebAssembly binaries:

qtcore.so
qtgui.so
qtwidgets.so
qtdeclarative.so

Each .so file can be downloaded, cached, and updated individually. This enables several use cases:
• Qt build sharing - several apps can use a single set of Qt binaries, perhaps hosted in a central location
• On-demand component loading via plugins.
• Incremental updates of single .so files.

“Simple” distribution use cases are may not benefit from shared libraries. The compiler can do less whole-program optimizations in this case, and dynamic linking also adds run-time overhead. For these static linking may be the best option. Deployment with static linking is also easier to set up.

Qt Shared Libraries Build

Enable by building Qt from source with the “-shared” configure argument. This will build each Qt library as an Emscripten side module, using the SIDE_MODLE=1 build option.

The main application binary is built with MAIN_MODULE=1, and links to the shared libraries it uses. Emscripten adds loading code which will download each library at startup. (See "deployment" below for details.)

Plugins
Plugins are shared libraries opened by calling dlopen() at runtime. Unlike normal shared libraries these are not added to the linker line, and Emscripten will not download them at startup. 

Emscripten provides two versions of dlopen:

• dlopen() “normal” dlopen: synchronously opens a shared library from the in-memory file system.
• emscripten_dlopen(): asynchronously opens a shared library from the in-memory file system, or by download.

Application code which is calling dlopen() directly today can possibly be ported to emscripten_dlopen(). It then needs to be changed from sync style to async via callback.

Using emscripten_dlopen() in Qt is difficult for a couple of reasons:
• We have several layers of sync API on top of dlopen(), which are not going to be changed to callback-style API. (e.g. The application calls QImage::save() which loads the jpeg plugin)
• QFactoryLoader expects to iterate over .so plugin files on the file system

Normal dlopen() is only partially supported, and can in practice not be used on Chrome since that browser limits the .so file size to 4KB (at least when called from the main thread).

Async Plugin loading

Plugins can be loaded with the following two-stage approach, where the plugin is "preloaded" before calling the sync plugin loading code provided by Qt:

1. Call the preparePlugin() helper function. This function downloads the plugin and places it on the in-memory files system (where e.g. QFactoryLoader can find it), and then calls emscripten_dlopen(). The function is async and completes with a callback.
2. Call the normal plugin loading code, which can now use blocking file access and also dlopen(), since the previous emscripten_dlopen() call completed the plugin loading.

This can also be combined with preloading the shared library for the plugin, in order to minimize the plugin loading delay. (See Deployment)

This functionality is currently implemented in an example:  https://git.qt.io/mosorvig/qt-web-utils/-/tree/main/experimental/dynamic_linking

Deployment

Qt downloads from "qt/" (relative to app.html) to "/qt/" on the in-memory file system, as far as Qt can control the downloads. The standard Qt install directory structure is used:

  qt/lib/libQt6Gui.so 
  qt/plugins/imageformats/libqjpeg.so
  qt/qml/QtQml/Base/libqmlplugin.so

The "qt" server location is configurable, and could potentially be set to a shared Qt build [TODO to be tested]. qtloader uses the Emscripten locateFile to set the location of the Qt shared libraries, but does not have complete control over how Emscripten loads them.

Qt (qtloader) provides functionality for downloading files at application startup, concurrently with the main wasm file download. This enables synchronous plugin loading at startup, without modifications to application code.

Current Limitations

• Asyncify is not supported.
• There are scalability issues with mulithreading, which cause browsers to run out of memory on startup.
• Only very minimal examples have been tested in order to verify the workflow.

Documentation

https://doc.qt.io/qt-6/wasm.html#shared-libraries-and-dynamic-linking-developer-preview

Work In progress (spring 2023)

Basic shared libraries usage and plugin loading for simple test applications work.

TODO

• Make qtloader.js use FS.createPreloadedFile: QTBUG-121817
• Find plugin dependencies by reading the dynlink.0 section: QTBUG-121833
• Make multithreaded shared library builds work: QTBUG-114256
• Investigate Qt Quick Controls missing symbol error: QTBUG-121914
• Test more modules (Qt Quick3D, Slate app, ++)
• Make the platform plugin be a plugin instead of an archive: QTBUG-121924
• Support cmake install and qt_generate_deploy_app_script() QTBUG-113849
• Should libQt6QmlBuiltins.a be a plugin? Or is it special?
• Investigate bug where emscripten_dlopen() appears to bypass the in-memory file system
• Investigate use of MAIN_MODULE=2

[Previous version of the bug description follows below \- may not be up to date]

What is the benefit? Why is this valuable?

Issue: Dynamic linking could provide smaller downloads by only downloading the modules that are needed. Also needed for binary QML and shared libraries that have been requested by several customers. There is one key customer project with millions of lines of code where DLLs are heavily used for instance. Shared libaries may also enable new distribution models for Qt.

What are common use cases?

Solution: Qt WebAssembly provides dynamic linking support to allow optimising download sizes and allows shared libraries. Wasm is being used in small and large projects in general, but larger projects are more likely to benefit from it. In general applications with more complexity and performance requirements are the target for Qt but the total performance consists of a number parts where shared libraries are one part, it may help with download size but consume more processing power.

Technical information

External Documentation

WebAssembly supports modules and dynamic linking. These are low-level APIs

Emscripten supports dynamic linking with compiler options and a run-time API.

Dynamic Linking for Qt

Emscripten supports dividing programs into one main module and several side modules, where the main module contains all system libraries. This is controlled by setting -s MAIN_MODULE=1/-s SIDE_MODULE=1 on the linker line.

There are two choices for the main module:

• Qt Core
• The application binary

The main module then needs to link/include all needed system libraries. It's possible to set EMCC_FORCE_STDLIBS=1 (on the environment) to enable "kitchen sink" mode, but this option is currently broken.

Setting MAIN_MODULE=1 makes emscripten not generate wasm fetching code: qtloader.js should either implement Module.readBinary or fetch the main wasm module and set Module.wasmBinary. (Going one step further we should look into compiling the wasm file from qtloader.js as well in order to enable streaming compiles. This is done by setting Module.instantiateWasm)

Side modules are loaded by setting Module.dynamicLibraries to an array of wasm file paths (not binaries!), which emscripten will fetch and compile. The module list can either be created manually or be provided by the build system (we'll aim to support both).

dlopen for Qt

Emscripten supports dlopen()ning side modules. The dlopen implementation expects to find the side module wasm binary on the filesystem: dlopen("foo.wasm") reads "/foo.wasm" and so on.

The easiest way to handle this is to preload the wasm files onto the (default-enabled) MEMFS file system in qtloader.js. All modules must then be specified at load time. This runs somewhat counter to the dynamic nature of dlopen. There are however other possible issues that prevents true dlopen-any-file:

• dlopen is synchronous API, which is hard/impossible to implement on the main thread
• The callers of dlopen may expect it to return in a reasonable time (and not wait for a download)

Prototypes and implementation

Minimal emscripten prototype: emscripten_dynamic

Build Qt libs as wasm modules WIP: https://codereview.qt-project.org/230044

Load libQtCore.wasm prototype: https://github.com/msorvig/qt-webassembly-examples/tree/master/core_dynamic

Qt Implementation Tasks

• EM_ASM is not supported for side modules; figure out what to do.
   ** Check if the EM_JS works: https://github.com/kripken/emscripten/commit/5f697f9349ea3662ee8f38ce439218ee4d22b5a0
• Set MAIN_MODULE/SIDE_MODULE as needed
• Add qtloader.js API and implementation for loading the main module / dynamic side modules / dlopen side modules
   ** Prototype: https://github.com/msorvig/qt-webassembly-examples/tree/master/emscripten_dynamic
• See if we can get the Qt library and plugin list from qmake
• Port/Test QLibrary and QPlugin(Loader)