TestCocoon Presentation Qt Summit 2011

Project name and goals

Presentation of TestCocoon at the "Qt Contributor’s Summit" in June 2011.

Description of TestCocoon

TestCocoon is a complete code coverage tool chain for C/C++ and C# programs available under AppleMac OS X, Linux™ or Microsoft® Windows. It analyzes the performance of a software validation and permits to measure the performance and optimizes the testing process of a C/C++ or C# applications:

• Finding untested code sections.
• Reducing the amount of tests by finding redundant tests.

With TestCocoon it is possible to find which portion of the source code is covered only by one execution of a test, and to detect if a new test does not cover more source code line than existing tests.

• Finding dead code trough displaying the code parts which are never executed.
• Specially useful for manual testing: TestCocoon is able to calculate the optimal execution order of tests which maximize the overall coverage after each run.
• Also, TestCocoon is able to perform its analysis on a difference of two applications.

This permits to find which tests are impacted by a source code modification and permits to measure the test quality of a patch or a hot fix.

TestCocoon can be uses for every testing steps and methodologies (unit tests, automatic tests, manual white box tests, black box tests, etc. . . ), and permits to collect and merge the execution reports together.

It is composed of 3 tools:

1. CoverageScanner, which analyzes, instruments and generates the C/C++ or C# application.
2. CoverageBrowser, which displays and manages the results of the coverage analysis. black box interactive tests.
3. An optional Microsoft® Visual Studio® 2005 & 2008 Add-In which permits to generate code coverage configurations from every C/C++ projects created by Microsoft® Visual Studio®.

Relationship with Qt

• All tools except CoverageScanner are generated using the Qt framework.
• TestCocoon provides a specific code coverage analysis for Qt applications which permits:

1. not instrument the contain of Q_OBJECT, Q_EXPORT_PLUGIN, Q_DECLARE_METATYPE, . . . macros.
2. instrument the emission of a signal, but not the engine code generated my the moc compiler.

Contact person by Nokia: Bill King. Sandbox repository: "Qt-Creator instrumented using TestCocoon on Gitorious":https://gitorious.org/~billking/qt-creator/qt-creator-testcocoon-wip

Planed deliverables

Presentation of TestCocoon:

• General presentation of code coverage technique.
• Features and design of TestCocoon.
• How to instrument Qt applications.
• Live demonstration of Qt-Creator instrumented with TestCocoon.

Open discussion with Qt-stakeholders:

• Integration in build systems (qmake, cmake, . . . ).
• Integration in Qt-Creator.
• Integration in continuous build server (Hudson, Pulse, . . . ).
• New idea, or features.
• …

Amount of support required

820 Euro for hotel and fly ticket.

Timeline

Answer required before the 1st June 2011.

Contact person

Sébastien Fricker sebastien.fricker@gmail.com http://www.testcocoon.org