TestCocoon Presentation Qt Summit 2011

Project name and goals

Presentation of TestCocoon at the "Qt Contributor’s Summit" in June 2011.

Description of TestCocoon

TestCocoon is a complete code coverage tool chain for C/C++ and C# programs available under AppleMac OS X, Linux™ or Microsoft® Windows. It analyzes the performance of a software validation and permits to measure the performance and optimizes the testing process of a C/C++ or C# applications:
* Finding untested code sections.
* Reducing the amount of tests by finding redundant tests.

With TestCocoon it is possible to find which portion of the source code is covered only by one execution of a test, and to detect if a new test does not cover more source code line than existing tests.
* Finding dead code trough displaying the code parts which are never executed.
* Specially useful for manual testing: TestCocoon is able to calculate the optimal execution order of tests which maximize the overall coverage after each run.
* Also, TestCocoon is able to perform its analysis on a difference of two applications.

This permits to find which tests are impacted by a source code modification and permits to measure the test quality of a patch or a hot fix.

TestCocoon can be uses for every testing steps and methodologies (unit tests, automatic tests, manual
white box tests, black box tests, etc. . . ), and permits to collect and merge the execution reports together.

It is composed of 3 tools:
# CoverageScanner, which analyzes, instruments and generates the C/C++ or C# application.
# CoverageBrowser, which displays and manages the results of the coverage analysis. black box interactive tests.
# An optional Microsoft® Visual Studio® 2005 & 2008 Add-In which permits to generate code coverage configurations from every C/C++ projects created by Microsoft® Visual Studio®.

Relationship with Qt

• All tools except CoverageScanner are generated using the Qt framework.
• TestCocoon provides a specific code coverage analysis for Qt applications which permits:

1. not instrument the contain of Q_OBJECT, Q_EXPORT_PLUGIN, Q_DECLARE_METATYPE, . . . macros.
2. instrument the emission of a signal, but not the engine code generated my the moc compiler.

Contact person by Nokia: Bill King.
Sandbox repository: "Qt-Creator instrumented using TestCocoon on Gitorious":https://gitorious.org/~billking/qt-creator/qt-creator-testcocoon-wip

Planed deliverables

Presentation of TestCocoon:
* General presentation of code coverage technique.
* Features and design of TestCocoon.
* How to instrument Qt applications.
* Live demonstration of Qt-Creator instrumented with TestCocoon.

Open discussion with Qt-stakeholders:
* Integration in build systems (qmake, cmake, . . . ).
* Integration in Qt-Creator.
* Integration in continuous build server (Hudson, Pulse, . . . ).
* New idea, or features.
* …

Amount of support required

820 Euro for hotel and fly ticket.

Timeline

Answer required before the 1st June 2011.

Contact person

Sébastien Fricker
sebastien.frickergmail.com
http://www.testcocoon.org