Qt Configuration System

Qt 5.10 introduced a new configuration system:

The goal of the new system is twofold. First of all, we are seeing a large need, especially with our embedded users, to customise how Qt is being built. Many of them have size restrictions or rather hard testing requirements, where being able to limit the functionality available in Qt is extremely helpful. We had such a system (see src/corelib/global/qfeatures.txt) before, but it was disconnected from the configure script, wasn’t really maintained since Qt 4 times, and had quite many limitations that I hope to solve with the new system.

Secondly, we have had to deal for a long time with two different configuration systems. We had a monster of a shell script for unix, and an Qt based executable on Windows. Changes to Qt often required changing both, and it was very easy to sneak subtle errors in. Maintainability of the old system was in general very poor.

- Lars Knoll (on the Qt development mailing list)

Overview

The new system basically moves most of the configuration handling from the shell script over to a declarative json file (currently configure.json) that is being processed by qmake. The change aims to be a 1 to 1 (or at least as close as possible) mapping of the old shell script to the new system.

Current State

This currently only affects Unix, Windows platforms are still being configured through configure.exe.

There are a couple of changes that will still come in over the next couple of weeks:

Give every module a qtmoduleglobal.h and qtmoduleglobal_p.h file

We already have per module global files for many modules (they are often required for the export/import handling of classes/symbols). With the new system we will add a public and private global file per module. I’ll start requiring that all public headers of the module include the public global file first, all private headers the private global file.

This is needed some steps below to modularise the configuration system and to allow us to track whether a certain feature we rely upon in code is actually available or not.

See https://codereview.qt-project.org/#/c/161143 and subsequent changes (the change adding global_p.h is still missing here, but will come as well)

Saner handling of 3rd party libs

Currently all our dependencies to external libs are handled in a rather ad-hoc way in the pro files. The goal is to unify this, see https://codereview.qt-project.org/#/c/161660/

Modularisation of the new configuration system

There are patches pending to modularise the new system. We will basically have one json configuration file per module/shared library. This will also allow us to use the system fully on repositories outside of qtbase, who currently have rather limited support for being configured.

With this change, we will start creating a qtmodule-config.h and qtmodule-config_p.h file as well as a corresponding public and private .pri files for each module. qtmodule-config.h will get included from the public global header for the module, qtmodule-config_p.h from the private one.

The public pro file will contain definitions for the features that are being exported by the module, the private one for features that are only relevant in the context of the module itself. As an example, ‘mimetypes’ would be a public feature of QtCore (as it changes the set of available API), whereas ‘glib’ would be most likely a private one (as it only determines which event loop to use and doesn’t change API).

See change https://codereview.qt-project.org/#/c/159604/40 and the following commit

Integration of the old feature system

There is the old feature system (see qfeatures.txt in corelib/global). With the work above done, integrating it into the system will be trivial (change https://codereview.qt-project.org/#/c/159763/)

With this done, we will also want to introduce a new mechanism to handle features in our cpp and header files. The current double negated #ifndef QT_NO_FOO is hard to read and unsafe. Unsafe means that the compiler won’t error out or warn us if the feature ‘foo’ isn’t available (because of a typo or because the feature is actually defined in widgets and you tried using it in gui). There is some pending work that would change this to use a macro function "#if QT_HAS_FEATURE(foo)” where we would actually get a compile error if the feature foo isn’t known to the system. As a nice side effect I’m planning on having the same names for features between .pro and c++ files.

Further reduce the shell script

The longer term goal will be to reduce the shell script further, until it’s basically a bootstrapping step for qmake, and hand all the other configuration work over to qmake. Some more patches for this are pending, but further help here would be appreciated.

Get rid of configure.exe

Finally, I’d also like to get rid of configure.exe, and do the same thing on Windows (ie. Bootstrap qmake from a small script, like we bootstrap configure.exe today) and then let qmake deal with the rest. With that we should be able to get to one cross-platform configuration system. For this task I’d be looking for volunteers :)

Detailed Description

Here’s a look at the details of the new system, in case you need to implement your own configure tests or want to add configurability to Qt. We now have one global configuration file qtbase/configure.json, that contains the details about how Qt can be configured. There is also a configure.pri file that contains some qmake methods to handle special cases (specialised tests our output handling).

You can find the infrastructure for the whole system in qt_configure.prf. The system basically works through two main systems.

1. The first one is a callback infrastructure, where all “type” fields provided in the json file actually refer to callback functions in qmake that do the actual handling. This allows you to e.g. implement your own custom test function in a .pri file and refer to that from the json. qt_configure has callbacks for the most common tasks, so you won’t need this feature in most cases. If you want to know how such custom callbacks can look like, have a look at configure.pri.
2. The other central piece is an expression evaluator, that is being used to evaluate conditions. It allows you to test for rather complex conditions (like e.g. “config.win32 && tests.mysql && features.opengl”) and is being used in many places. The evaluator understands boolean operators (!, && and ||), comparison operators (== and !=), braces and different types of arguments:

true/false

boolean values

‘foo’

The string literal foo

config.foo

Variables in the qmake CONFIG var (usually used for platform dependent stuff like config.unix)

tests.foo

Boolean whether the test foo succeeded

tests.foo.bar

Variable bar, set by tests.foo

features.foo

boolean whether feature foo is available

features.foo.bar

variable bar set by features.foo

arch.foo

architecture (like x86, arm, etc)

input.foo

input variable (set by command line handling)

far.foo

Any variable set in qmake

call.foo

return value of the replace function foo in qmake

This gives you quite some options to define conditions and dependencies between features. The system automatically runs the configure test, if you ask for tests.foo, so dependency handling is fully transparent.

configure.json

Now let’s have a look at configure.json:

The file contains a couple of sections. The first one is called “files”, and contains definitions for the output files. You should not need to touch this one.

The next section contains the command line parameters that you can pass to the configuration system. It basically defines the valid command line arguments that configure will recognise. They are being mapped to a config.input.option value in qmake, that is then being used in the next step of defining the features we will use for Qt.

Typical entries looks like

"harfbuzz": { "type": "enum", "values": [ "no", "qt", "system" ] },
"headersclean": "boolean",

This means harfbuzz is an option that can take a selection of args (-no/-qt/-system), whereas headersclean is a boolean argument (-headersclean and -no-headersclean accepted). The second form is a shorthand for

"headersclean”: { “type”: “boolean” }

Note that the type variable refers to a callback. In this case a test function qtConfCommandline_boolean.

Then comes a section with tests. Those define all the configure tests that so far have been executed by the shell script. The definition of a typical test looks like:

 "fontconfig": {
        "description": "Fontconfig”,
        "type": "compile”,
        "test": "unix/fontconfig”,
        "pkg-config-args": "fontconfig freetype2”,
        "libs": "-lfontconfig -lfreetype"
} 

This basically defines a configure test for fontconfig. It’s a compile test, the test being in config.tests/unix/fontconfig. It’ll try to use pig-config to determine the correct LIBS and CFLAGS to compile and link against the library, and there is a fallback for the libs in case fontconfig can’t be found.

Again, the type variable refers to a callback (qtConfTest_compile in this case).

After that we have the central section that defines all the features. Let’s take one example:

"icu": {
       "description": "ICU”,
       "autoDetect": "!config.win32”,
       "condition": "tests.icu”,
       "output": [ "publicQtConfig” ]
},

This defines the icu feature. It’s not auto-detected on windows, requires the ice configure test to pass, and will then generate one output called publicQtConfig. Here are some details of the fields:

description

A short description of the feature. Used by the summary section below

autoDetect

Should evaluate to a boolean value whether to automatically detect the feature. Defaults to true

emitIf

Skip the feature completely if this evaluated to false (don’t evaluate conditions or outputs). Defaults to true.

enable

Evaluates to a condition that will enable the feature (defaults to “input.feature == yes”)

disable

the opposite, (defaults to “input.feature == no”)

condition

A condition that determines whether the feature is enabled/disabled. Will generate an error if it conflicts with the enable field above

output

Different types of output to generate. Also here you can define arbitrary callbacks, but the standard types should cover most needs:

“publicQtConfig"

Add the feature name to the QT_CONFIG variable in the public pri file (here qconfig.pri)

“publicConfig"

Add the feature name to the CONFIG variable in the public pri file (here qconfig.pri)

"privateConfig"

Same for the private pri (qmodule.pri)

“feature”

Defines a feature. Adds it to QT_CONFIG if available, sets QT_NO_FEATURE otherwise

{ “type”: “define”, “name”: “FOO”, “value”: “expression” }
       #define FOO expression in the public header file, expression is evaluated
{ “type”: “libs”, “test”: “configtest” }

       Output QMAKE_LIBS/CFLAGS_FEATURE defining parameters required to use an external library

In addition, there are varAssign, varAppend and varRemove to assign, append and remove values from qmake variables

All outputs can have a ‘negative’: true/false field (default false), that would reverse when output is being generated (usually only if the feature is available, and a ‘condition’ field.

Finally there are two sections called ‘earlyReport’ and ‘report’. Use ‘report’ unless you know what you’re doing. These sections allow you to define conditions under which notes, warnings or errors are being reported back to the user.

Finally, there’s a summary section, that defines the configure summary we’re reporting back to the user. Lars leaves figuring out the details here as an exercise to the reader ;-)