QtCS2021 - Testing upstream changes with downstream modules

Session Summary

A discussion to find the best solution to the issue of changes in one module (upstream) not being tested with modules (downstream) that depend on that module.

Session Owners

• Mitch Curtis (mitch.curtis@qt.io)

Notes

Run tests for specific dependent modules

Have some way of telling CI that this module requires tests from other modules (doesn't have to be all dependent modules; can be a subset) to pass in addition to the module's own tests.

Pros

• Saves developer time (excluding CI integrations, which don’t count since they shouldn’t be wasting developer time).
  • qtdeclarative devs don't need to build qtquickcontrols2.
  • Git operations for both modules don't get slower.
  • Working on the modules in Creator doesn't get slower.

Cons

• Requires extra infrastructure to support it in CI.
• In the case of qtquickcontrols2, we miss out on the ability to use controls in the examples of qtdeclarative and its dependent modules.

Un-modularise the CI system

This is probably partially related to the first solution. Taken from https://bugreports.qt.io/browse/COIN-133:

Pros

• There is only one "module" in the integration setup: qt5.
• This means that we get full revdep coverage.
• All changes staged for the same branch in different submodules are integrated at the same time.
  • This conveniently introduces atomicity, and thus solves the problem introduced by revdeps in a modular CI.
• When an integration succeeds, the supermodule sha1 are instantly updated as well.
• The CI should also test the packaging stages, so we have end-to-end coverage.

Cons

• Significantly increases the performance demands of the system. https://bugreports.qt.io/browse/COIN-134 could help with that.

Move specific modules into the repository of the module they depend on

Where it makes sense, dependent modules can simply be moved into the repository of the module they depend on.

Pros

• Fairly straightforward solution that requires no extra infrastructure.
• In the case of qtquickcontrols2, we gain the ability to use controls in the examples of qtdeclarative and its dependent modules.

Cons

• Slower git operations.
• Slower build times for qtdeclarative devs. You can use e.g. ninja targets, but you would have to do this every time you build. Ideally we’d add a configure option for this, like -no-gui in qtbase, so that they don’t need to build qtquickcontrols2.
• Creator would probably be quite a bit slower to work with having both of these in one project.
• May need to rename qtdeclarative.git?