QtCS2021 - Testing upstream changes with downstream modules

From Qt Wiki
Revision as of 11:46, 22 June 2021 by Mitch (talk | contribs)
Jump to navigation Jump to search


Session Summary

A discussion to find the best solution to the issue of changes in one module (upstream) not being tested with modules (downstream) that depend on that module.

The goal of this work is to save the time of both upstream and downstream developers by providing faster feedback for upstream changes that may affect downstream modules.

Session Owners

  • Mitch Curtis (mitch.curtis@qt.io)

Notes

Run tests for specific dependent modules

Have some way of telling CI that this module requires tests from other modules (doesn't have to be all dependent modules; can be a subset) to pass in addition to the module's own tests.

Pros

  • Saves developer time (excluding CI integrations, which don’t count since they shouldn’t be wasting developer time).
    • qtdeclarative devs don't need to build qtquickcontrols2.
    • Git operations for both modules don't get slower.
    • Working on the modules in Creator doesn't get slower.

Cons

  • Requires extra infrastructure to support it in CI.
  • In the case of qtquickcontrols2, we miss out on the ability to use controls in the examples of qtdeclarative and its dependent modules.
  • Some changes need to break downstream modules (e.g. API changes, etc.), so blocking CI because of downstream modules would be an issue.

Un-modularise the CI system

This is probably partially related to the first solution. Taken from https://bugreports.qt.io/browse/COIN-133:

Pros

  • There is only one "module" in the integration setup: qt5.
  • This means that we get full revdep coverage.
  • All changes staged for the same branch in different submodules are integrated at the same time.
    • This conveniently introduces atomicity, and thus solves the problem introduced by revdeps in a modular CI.
  • When an integration succeeds, the supermodule sha1 are instantly updated as well.
  • The CI should also test the packaging stages, so we have end-to-end coverage.

Cons

Move specific modules into the repository of the module they depend on

Where it makes sense, dependent modules can simply be moved into the repository of the module they depend on.

Pros

  • Fairly straightforward solution that requires no extra infrastructure.
  • In the case of qtquickcontrols2, we gain the ability to use controls in the examples of qtdeclarative and its dependent modules.

Cons

  • Slower git operations.
  • Slower build times for qtdeclarative devs. You can use e.g. ninja targets, but you would have to do this every time you build. Ideally we’d add a configure option for this, like -no-gui in qtbase, so that they don’t need to build qtquickcontrols2.
  • Creator would probably be quite a bit slower to work with having both of these in one project.
  • May need to rename qtdeclarative.git?
Retrieved from "https://wiki.qt.io/index.php?title=QtCS2021_-_Testing_upstream_changes_with_downstream_modules&oldid=38419"