Qt-Quality-Gate-Criteria: Difference between revisions

From Qt Wiki
Jump to navigation Jump to search
No edit summary
(Redirected page to Qt Quality Gate Criteria)
 
(One intermediate revision by one other user not shown)
Line 1: Line 1:
[[Category:Developing_Qt::QA]]
#REDIRECT [[Qt_Quality_Gate_Criteria]]
 
= Qt Quality Gate Criteria =
 
These criteria are the '''minimum standard''' for submitting code to any Qt master branch.
 
== Definitions ==
 
* The Submission is any change (addition, modification, or removal) that is submitted to a Qt branch. It may consist of code, comments, documentation, resources, or other content.
* Building The Package refers to compiling and linking the DLLs and executables, including test asset, that resides in a Qt package.
 
=== Basic Principles ===
 
* Development teams are responsible to ensure all criteria are met before any code is submitted to master.
* These criteria are a minimum set. Teams are encouraged to set their standards higher.
* These quality criteria will be applied to all contributors.
* Code needs to compile on all Tier 1 platforms (no regressions)
* Has been (API) reviewed (i.e. follows coding conventions)
* The code can be demonstrated (by means of demo/example/tests)
* Meets a defined minimum threshold of coverage by means of Unit/System tests
* All tests for the new feature pass plus no regressions in existing tests
* Well documented (functional specification, class documentation, architectural overview, etc)
* Maintenance agreed
* Test Plan available
 
Bug-fixes (i.e. work done in resolving a defect) are only accepted into the Master Code Line if it meets the following criteria:
 
* Code needs to compile on all Tier 1 platforms and work (no regressions)
* Has been reviewed (i.e. follows coding conventions)
* Fix fixes the reported error, and may also be verified by a new test (preferably automated Unit/System test)
* Corresponding documentation is updated as needed (functional specification, class documentation, architectural overview, etc)
 
=== Compiling ===
 
{|
!Rule
!Responsibility
!Explanation
|-
|Compiles against recent version of master
|Contributor
|The Submission has been integrated with up-to-date Qt master.
|-
|Compiles on all Tier 1 platforms
|Contributor with QA Engineer
|The Package builds on all Tier 1 platforms that are defined for the product. Note that The Package includes not only the DLLs, but also other executables and code, such as the test asset, performance tests and examples. This package is able to be installed on target platforms completely and without user intervention.
|}
 
=== Static/code-quality criteria ===
 
{|
|_. Rule
|_. Responsibility
|_. Explanation
|
|Compliant coding convention
|Contributor
|For QML see: [http://doc.qt.nokia.com/4.8-snapshot/qml-coding-conventions.html]
|-
|qdoc - zero warnings
|Community
|Documentation for the submission should be complete and relative to the standard for the submission. For instance a major API contribution requires significant documentation whereas a feature will require only documentation related directly to its use.
|-
|Compile - zero warnings on Linux and minimal on remaining Tier 1 platforms
|Contributor
|Some warnings are more serious than the others, so we obviously care most about the important ones.
|}
 
=== Performance criteria ===
 
{|
!Rule
!Responsibility
!Explanation
|-
|Autotest performance
|Contributor with Integration Engineer
|The measured unit test performance no worse unless approved if module has been submitted previously
|-
|Performance test coverage
|Contributor with QA Engineer
|Use-cases are covered by performance tests.
|-
|Streamlined platform interaction
|Contributor
|Native Qt classes no more than half speed of equivalent native implementation
|}
 
=== Error management ===
 
{|
!Rule
!Responsibility
!Explanation
|-
|P0/1 tasks - zero open
|QA Engineer
|Monitoring of all teams active tasks against Qt and ensuring consistent error management and progressing of issues.
|-
|Untriaged tasked - zero
|Contributor with QA Engineer
|All bugs should be triaged and a priority assigned.
|-
|Relative reduction in bug count
|Contributor with QA Engineer
|Weighted bug total no worse than previous release.
|}
 
== Future Wishlist (i.e. pointless to implement this until we get the items above under control) ==
 
{|
!Rule
!Responsibility
!Explanation
|-
|Autotests - 80% function coverage, 70% executable
|Contributor and QA Engineer
|Autotests must, in bare minimum, call most functions in exported public API. Furthermore, line coverage of autotests should exceed 70%, unless there's a genuine technical reason why that cannot be reached. Note that these numbers are not necessarily the definition of good autotest coverage, but our attempt to define what would be the bare acceptable minimum.
|-
|Autotests pass-rate > 90%
|Contributor and Integration Engineer
|In principle: autotests (in other words, automatically executed unit and module tests) must pass. Regressions are not allowed.
|}

Latest revision as of 20:37, 4 March 2015

Redirect to:

Retrieved from "https://wiki.qt.io/index.php?title=Qt-Quality-Gate-Criteria&oldid=12173"