Qt Quality Gate Criteria

These criteria are the minimum standard for submitting code to any Qt branch.

Definitions

• The Submission is any change (addition, modification, or removal) that is submitted to a Qt branch. It may consist of code, comments, documentation, resources, or other content.
• Building The Package refers to compiling and linking the DLLs and executables, including test asset, that resides in a Qt package.

Basic Principles

• Development teams are responsible to ensure all criteria are met before any code is submitted to.
• These criteria are a minimum set. Teams are encouraged to set their standards higher.
• These quality criteria will be applied to all contributors.
• Code needs to compile on all platforms in the continuous integration system (no regressions, enforced by blocking continuous integration)
• Has been (API) reviewed (i.e. follows coding conventions)
• The code can be demonstrated (by means of demo/example/tests)
• Meets a defined minimum threshold of coverage by means of Unit/System tests
• All tests for the new feature pass plus no regressions in existing tests
• Well documented (functional specification, class documentation, architectural overview, etc)
• Maintenance agreed
• Test Plan available

Bug-fixes (i.e. work done in resolving a defect) are only accepted into the codebase if it meets the following criteria:

• Code needs to compile on all platforms in the continuous integration system (no regressions, enforced by blocking continuous integration)
• Has been reviewed (i.e. follows coding conventions)
• Fix fixes the reported error, and may also be verified by a new test (preferably automated Unit/System test)
• Corresponding documentation is updated as needed (functional specification, class documentation, architectural overview, etc)

Compiling

Rule	Responsibility	Explanation
Compiles against recent version of the target branch (usually dev)	Contributor	The Submission has been integrated with up-to-date Qt dev.
Compiles on all Tier 1 platforms	Contributor with QA Engineer	The Package builds on all Tier 1 platforms that are defined for the product. Note that The Package includes not only the DLLs, but also other executables and code, such as the test asset, performance tests and examples. This package is able to be installed on target platforms completely and without user intervention.

Static/code-quality criteria

Rule	Responsibility	Explanation
Compliant coding convention	Contributor	For QML see: qml coding conventions
qdoc - zero warnings	Community	Documentation for the submission should be complete and relative to the standard for the submission. For instance a major API contribution requires significant documentation whereas a feature will require only documentation related directly to its use.
Compile - zero warnings on Linux and minimal on remaining Tier 1 platforms	Contributor	Some warnings are more serious than the others, so we obviously care most about the important ones.

Performance criteria

Rule	Responsibility	Explanation
Autotest performance	Contributor with Integration Engineer	The measured unit test performance no worse unless approved if module has been submitted previously
Performance test coverage	Contributor with QA Engineer	Use-cases are covered by performance tests.
Streamlined platform interaction	Contributor	Native Qt classes no more than half speed of equivalent native implementation

Error management

Rule	Responsibility	Explanation
P0/1 tasks - zero open	QA Engineer	Monitoring of all teams active tasks against Qt and ensuring consistent error management and progressing of issues.
Untriaged tasked - zero	Contributor with QA Engineer	All bugs should be triaged and a priority assigned.
Relative reduction in bug count	Contributor with QA Engineer	Weighted bug total no worse than previous release.

Future Wishlist (i.e. pointless to implement this until we get the items above under control)

Rule	Responsibility	Explanation
Autotests - 80% function coverage, 70% executable	Contributor and QA Engineer	Autotests must, in bare minimum, call most functions in exported public API. Furthermore, line coverage of autotests should exceed 70%, unless there's a genuine technical reason why that cannot be reached. Note that these numbers are not necessarily the definition of good autotest coverage, but our attempt to define what would be the bare acceptable minimum.
Autotests pass-rate > 90%	Contributor and Integration Engineer	In principle: autotests (in other words, automatically executed unit and module tests) must pass. Regressions are not allowed.