Qt for Android integration cannot be used for complex QML apps

Over several month, again and again, we spent considerable time to tune and adapt the packaging, build and even the code of the reference UI in the Qt Automotive Suite to build and run on Android. Even though, we hacked it in by the end of the day, the result cannot be rated as an acceptable way to go for serious production-level development. This current solution in Qt (5.11 and later) for Android and related documentation is sufficient for simple apps. More sophisticated apps mixed C++ plugins, own QML modules with a complex import structure, cannot be built and used on Android without reverse engineering of tools and advanced Qt, especially qmake techniques.

The below is a copy of a project page from our internal Confluence. It outlines the key finding on the this journey. We wish that this report will be reviewed by the leads in the Qt Android and decomposed in a set of improvements along the 5.12 LTS scope.

*********************************

Currently, there are two "apps" which can be considered to be running on an Android device: Neptune 3 UI and NeptuneControlApp. Meanwhile, both can built for, and run on Android. It is still not an out-of-the-box experience and requires some work done. While the NeptuneControlApp is a relatively simple case. Getting the Neptune 3 UI running on Android requires a lot of tricks in packaging, work-arounds and insights for undocumented features. Even when Neptune 3 UI is once running on Android, it is still not fully featured, and merely meant for showing the general look and the basic functionality. The NeptuneControlApp on the other hand was specifically made to support Android and is fully functional.

Generally, building and running Qt-based applications on Android is however a different from desktop environments and there are android specific issues and things to consider that are discussed below. The fixes and outright hacks are neither necessarily the best practice nor they they are the only solution for the issues. Hopefully, they are still helpful for anyone trying to write new or port existing Qt-based applications. It is very pity that those topics are not addressed in the Qt documentation and there is no visible interest in The Qt Company to improve the Android support in Qt to simplify all that hassle...

The below sections contain observations and list changes made in the Neptune3 UI component on the code base for the the 5.12 release. The 5.11 code base does not contain most of the changes.

Setting up the development environment

Setting up Android and Qt tools:

• use Qt Maintenance Tool to install  the pre-build Qt for Android packages and tools
• install Android SDK
• install Android NDK, whereas you need to install the version that does not have unified headers (iirc, version 14 or older). We don't know how to get unified headers to work with Qt
• Set paths to SDK and NDK in Qt Creator in the "Tools" under "Options" or in "Preferences", the "Devices→Android" section

You will need Qt Creator since many steps are not sufficiently documented and "hidden" in the magic of Qt Creator integration. Due to this, developing and deploying Qt for Android is easiest when done with the Qt Creator. For some reasons, the building on command line is not mentioned in the docs and so probably not supported. This is a serious drawback for an integration of the steps on a CI which is mandatory for any serious development.

When deploying, pay attention to which executable you have picked from the list (the Kit selection button above the "Deploy/Start" button). When building the Neptune 3 UI project, the right executable is the neptune3-ui.

Since the Qt Automotive Suite packages in the Qt Maintenance Tool do not provide builds for Android at the moment, you have build and install the QtIVI and QtApplicationManager for Android on your own. Build the QtApplicationManager without the installer by passing -config disable-installer to qmake. For both projects, QtIVI and QtApplicationManager, disable the "Make Install" and the "Build APK" step in the Qt Creator Project Setup. In the "Build Steps" settings of the both projects, you have to add "install" as new argument under "Make arguments".

This is needed because  QtIVI and QtApplicationManager are Qt Modules and so their build artefacts must be installed into the Qt directory. The default "make Install" step is a different thing is not meant for this.

Neptune 3 UI specific points

• The QML files used in Neptune have to be stored under the special "assets://" path. Make sure that all files are listed this way. In Android, "assets://" is a read-only collection of free form data needed by the app. One of the special aspects here is that assets may not contain any executable code. This is a problem for QML plugins. In the {{build }}folder, an assets folder will be created for you
  All that content must be mase accessible using the assets:// URL.
• Qt provides the undocumented androiddeployqt tool which is thought to hide the specifics of the Android platform and do some magic in the background. The problem with the tool is that this magic is unknown in fine details which will stop your application from running. This especially applies to habit of the tool to copy all QML imports into one folder. If you have a simple importing structure in the QML code, this works well, but if you use, for example, relative path in imports, or the name for a plugin located in different paths, the tool will fail to make it right.
• The recent revisions of the pro files in the Neptune 3 UI has been changed to support Android. The undocumented QML_ROOT_PATH variable in the <NEPTUNE3_SRC>/src/neptune3-ui/neptune3-ui.pro points to the folder which contains the main.qml file. All the QML files in the folder specified in QML_ROOT_PATH are parsed using the qmlimportscanner (done by androiddeployqt) and scanned for dependencies. All the detected dependencies are packaged into the final APK. In order to handle the QML plugins provided in Neptune 3, the QML_IMPORT_PATH variable needs to be set correctly so that these plugins are found, e.g. to <BUILD_PATH>/plugins/shared
• The <NEPTUNE3_SRC>/qmake_features/qmlplugin.prf has been extended to work correctly for Android and now handles the manual steps explained here. All C++ QML plugins have to be installed into imports_shared_cpp instead of the imports_shared path. This is needed because the C++ plugins are deployed using the scanning mechanism in androiddeployqt, where they are copied into a special folder. On the other side, because the imports_shared folder also contains QML-only modules, those have to be deployed manually. When deploying that folder from the <BUILD_PATH>, it contains the C+  plugins as well. This causes a conflict, since the QML Engine sometimes tries to load these plugins from that place, and fails on that. The trick is to deploy the C+ plugins as part of androiddeployqt where they automatically are copied into the global QML import path, and then only separately deploy the QML-only modules into imports_shared.
• Another way would be to let all modules be imported by androiddeployqt as is, but that would not work either, since the Neptune3 UI loads some assets relative to the plugin location and this doesn't work anymore, since androiddeployqt adds the modules to a special folder and so changes the folder structure.
• On Android, the main() function the Neptune 3 UI sets a special UrlInterceptor object for the QML Engine used in the Qt Application Manager. The UrlInterceptor class is needed to fixing the issue on Android that adds a invalid file:assets: scheme to some QML file URLs, when the scheme should only be just assets:. The interceptor recognises these cases and removes the invalid "file:" part. It’s unclear why this happens, but this problem prevents local importing from working on Android, i.e. a QML code file is importing types from files located in the same directory or a subdirectory
• The Qt Application Manager configuration YAML-file needs to be different for Android than on Linux since APKs cannot write to a global tmp location, instead we need to point to the local directory.
• In the main()-function in the Neptune3 UI, we pass a YAML-config file with Android-specific changes to Qt Application Manager
• Additionally, we make sure the YAML-config file is also located in the assets directory, so that the special CONFIG_PWD value in the YAML-config file will point to the assets.
• As there is no command-line parsing on Android, we always pass the --recreate-database command to Qt Application Manager to make sure all deployed apps are scanned correctly, i.e from scratch.
• The QtVirtualKeyboard module is not supported on Android. This has been fixed by putting the VirtualKeyboard.qml into the Loader element, since loading succeeds only if the module is actually available available.
• Since the Neptune3 UI provides multiple executables, it is important to select the android3-ui executable in Qt Creator when deploying for Android

Other problems and general observations

• Qt versions are depended on NDK versions e.g. Qt5.11 ndk <= 12b Qt5.12 ndk >= 12b (clang)

• Qt Creator only allows to set the used NDK version globally which means you cannot have both 5.11 and 5.12 working when compiling for android

• The new Qt 5.12 Android kit is still called GCC although it uses clang. Switching to the g++ mkspec and using an older NDK is not possible as the new compiler flags are not supported in the older NDK

• The Qt 5.11 and 5.12 documentation doesn't reflect the new minimum Android NDK version

• The Qt Creator setup for Android is tailored for app-development, building a Qt module (e.g. qtapplicationmanager or qtivi) for Android is possible, but the install step is broken as it doesn't install it to the Android Qt even thought it's a valid Qt module

• As mentioned, developers have to understand details how apps are expected to be deployed on Android, e.g. which steps are done or why it's important for the dependency scanning.

• For some strange reason Neptune3 UI needs an URL interceptor (see above) because otherwise locally imported QML files are resolved with the path file:assets: .... which results in not finding the file and everything is broken, remove the file schema in the UrlInterceptor fixes the problem locally.

• Command line arguments and env environment variables cannot be set or don't have an effect on android, this makes it very hard to debug by using convenient debugging techniques e.g. QT_DEBUG_PLUGINS=1. The only way which helped so far is to set the env in main.cpp before the QApplication gets instantiated, this is just a workaround but the documentation also doesn't help with this problem