Qt Creator ManualTests DebuggerGdb: Difference between revisions

From Qt Wiki
Jump to navigation Jump to search
No edit summary
 
(Mention requirements for stepping into official Qt build)
 
(41 intermediate revisions by 6 users not shown)
Line 1: Line 1:
=Qt Creator Manual Tests: Debugger g++/gdb=
[[Category:Tools::QtCreator::QualityAssurance]]


tests/manual/debugger/simple/simple.pro and tests/manual/debugger/cli-io/cli-io.pro provide the needed code
For the following tests, you will need to build projects on Qt using gcc/MinGW and debug them using gdb. You can get such a build of Qt from the official [https://download.qt.io/official_releases/online_installers/ Online Installer].


{| class="infotable line"
{| class="wikitable"
|- style="background: #009900"
|+ Tests using ''tests/manual/debugger/simple/simple.pro'' from Qt Creator's source repository
! Test
! Test
! Result
! Result
! Annotation
! Annotation
|-
|+
| Create new project. Can you build, run and debug it?
| Set breakpoint, press F5 to build and run debugger, verify that program stops at a breakpoint that you set:
| automated, except on Mac
* in main function before program is started
|
|-
|
Set breakpoint, press F5 to build and run debugger, verify that program stops at breakpoint:
 
* on main before program is started
* while the program is running
* while the program is running
* in dynamically loaded plugins, especially in constructors
* in dynamically loaded plugins, especially in constructors
Line 22: Line 16:
|
|
|
|
|-
|+
| “Step into” a couple of times. Can you step into Qt source code ('''*.cpp''' file under <span class="caps">QTDIR</span>)?<br /> (Mac: switch on ‘Use Debug Versions of Frameworks’ in run configuration. You need Qt sources.)
| "Step into" a couple of times. Can you step into Qt source code (*.cpp file under QTDIR. You might need to configure source paths mapping under "Edit" -> "Preferences..." -> "Debugger" -> "General" -> "Add Qt sources...")?
 
(In case you are using a Qt version from the official installer, you may need to install "Sources" and "Qt Debug Information Files" for this version.)
 
('''Mac''': switch on 'Use Debug Versions of Frameworks' in run configuration. You need Qt sources.)
|
|
|
|
|-
|+
| Test debugging helpers/python gdb: Do classes like QImage or std::string show beautiful information instead of the raw structure?
| Test debugging helpers/python gdb: Do classes like QImage or std::string show beautiful information instead of the raw structure?
| automated
|
|+
| Step through some (not all) test* functions and check whether the displayed data looks correct
|
| The code contains comments with the expected displayed data. These are meant for semi-automatic runs and might differ from what you see. Use your own judgement what's correct and what's not.
|+
| Comment out the return statement inside the following functions one by one. Check whether you'll end up with a proper stack trace & locals display.
* testNullPointerDeref()
* testNullReference()
* testEndlessLoop(); (break manually)
* testEndlessRecursion();
* testUncaughtException(); (not on Windows)
|
|
|
|
|-
|+
| Step through some test* functions and check whether the displayed data looks sane
| Test a breakpoint in a QThread:
# Uncomment the call to qthread::testQThread()
# Place a breakpoint in qthread::Thread::run()
# Run this in the debugger
# Make sure that:
#* the breakpoint receives the right number of hits
#* "Locals and Expressions" and "Stack" views show reasonable data, including the object names of the threads
|
|
|
|
|-
|+
| Switch on temporarily 'Operate by Instruction' (small icon above the stack trace) and check whether you see disassembler output and can step by instruction
|
|
Comment out the return statement before the following code lines or inside the respective functions one by one. Check whether you’ll end up with a proper stack trace &amp; locals display.
|}


* <nowiki>*(int *)0 = a + b;</nowiki>
{| class="wikitable"
* testNullReferenceHelper(pp, qq);
|+ Tests using ''tests/manual/debugger/cli-io/cli-io.pro'' from Qt Creator's source repository
* testEndlessLoop(); (break manually)
! Test
* testEndlessRecursion();
! Result
* testUncaughtException();
! Annotation
|+
| Check I/O (qDebug, std::cout, std::cerr), on Win for both Debug and Release.
|
|stderr/stdout handling on Windows:
* An application needs to be built with 'console' for stderr/stdout to appear (use Creator's "Run in terminal" setting)
* Creator itself is built with 'console' in debug mode only.
* qDebug() prints to stderr for ‘console’ apps, else to the Windows debugger log, which can be shown with the DbgView utility
|+
| Check "Run in Terminal". Use Terminal for input. (Debbuging might not work on Ubuntu)
|
|
|
|+
| Create new project. Can you build, run and debug it?
| automated
|
|
|-
|+
| Switch on temporarily ‘Operate by Instruction’ and check whether you see disassembler output and can step by instruction
| Check nothing bad happens on a simple int main() {} program with no breakpoints set
| automated
|
|
|}
{| class="wikitable"
|+ Tests using ''tests/manual/debugger/gui/gui.pro'' from Qt Creator's source repository
! Test
! Result
! Annotation
|+
|
|
When testing on Linux or Mac:
# Run the project outside Creator.
# Click the button "Crash".<br/>It should write a core file to the working directory. Depending on the operating system, you might have to set "ulimit -c unlimited" and/or disable OS-specific crash-reporting mechanisms, e.g. on [https://stackoverflow.com/questions/2065912/core-dumped-but-core-file-is-not-in-the-current-directory/ Ubuntu].
# Attach Creator to the core file.<br/>Verify that you see a stack trace, variable values and code markers just as if you had run the application in the debugger from the beginning.
|
|
|-
|-
| Check I/O (qDebug, std::cout, std::cerr), on Win for both Debug and Release. Please read the below note!
|
|
|
# Start the project from outside Creator, e.g. from the command line.
# Attach Creator to the running process.<br/>If it doesn't work at first, Creator should show an error message why not. Make sure that this message is helpful and following its instruction will make attaching work.<br/>Verify that you see a stack trace, variable values and code markers just as if you had run the application in the debugger from the beginning. You should also be able to pause/continue the execution.
|
|
|-
|-
| Check “Run in Terminal”. Use Terminal for input. (Debbuging might not work on Ubuntu)
| Test unusual situations: Kill the running debugged program 'externally' with a signal that is neither SIGSTOP nor SIGKILL. Qt Creator should end up showing a backtrace.
|
|
|
|
|-
|+
| Check nothing bad happens on a simple int main() {} program with no breakpoints set
| Kill the stopped debugged program 'externally'. Nothing should happen immediately, on next continue or step Qt Creator should end up showing a backtrace.
| automated, except on Mac
|
|
|-
| Linux/Mac: Attach to a core file (ulimit -c unlimited)
|
|
|+
| Try remote debugging: start a gdbserver using e.g.
<code>gdbserver localhost:1234 ./manual_test_debugger_gui</code>
on a Linux machine and connect to it
|
|
|}
{| class="wikitable" style="width: 80%"
|+ Test run as root from inside QC - any simple application (Linux, macOS)
! Test
! Result
! Annotation
|+
|
Run the project without checking "Run as root user" on the run settings page.
# Verify application is starting and can be used as expected.
|
|
|-
|-
| Attach to a running process (might not work on Ubuntu)
|
|
Debug the project without checking "Run as root user" on the run settings page.
# Verify application and debugger are starting correctly.
# Verify you can interrupt the application and hit a breakpoint. Check some locals whether they contain sane data.
|
|
|-
|
|
Run the project with checking "Run as root user" on the run settings page.
# Verify application is starting correctly.
# Use the OS' means to verify that the application is actually running for user "root".
# Close the application and start it again immediately. Creator should not ask for entering credentials again.
# Run "sudo -K" to ensure credentials are reset.
# Re-run and check whether credentials are requested again.
|
|
|-
|-
| Try remote debugging: start a gdbserver using e.g.<br /> in creator/tests/manual/fakevim on a Linux machine and connect to it
|
|
Debug the project with checking "Run as root user" on the run settings page.
# Verify application and debugger are starting correctly.<br/>gdb running as "root" user is acceptable.
# Use the OS' means to verify that the application is actually running for user "root".
# Verify you can interrupt the application and hit a breakpoint. Check some locals whether they contain sane data.
# Close the debugging session and start it again immediately. Creator should not ask for entering credentials again.
# Run "sudo -K" to ensure credentials are reset.
# Re-run and check whether credentials are requested again.
# Re-do verifications above.
|
|
|
|-
|-
|
|
Test unusual situations: Kill X ‘externally’ while debugging (both in a ‘running’ and ‘stopped’ state), where X is
Is entering invalid credentials handled correctly?
 
* the debugged program
* gdb
|
|
|
|
|}
|}
==stderr/stdout handling on Windows==
* An application needs to be built with ‘console’ for stderr/stdout to appear (use Creators “Run in terminal” setting)
* Creator itself is built with ‘console’ in debug mode only.
* qDebug() prints to stderr for ‘console’ apps, else to the Windows debugger log, which can be shown with the DbgView utility
===Categories:===
* [[:Category:Tools|Tools]]
** [[:Category:Tools::QtCreator|QtCreator]]
* [[:Category:Tools::QtCreator::QualityAssurance|QualityAssurance]]

Latest revision as of 11:44, 7 November 2024


For the following tests, you will need to build projects on Qt using gcc/MinGW and debug them using gdb. You can get such a build of Qt from the official Online Installer.

Tests using tests/manual/debugger/simple/simple.pro from Qt Creator's source repository
Test Result Annotation
Set breakpoint, press F5 to build and run debugger, verify that program stops at a breakpoint that you set:
  • in main function before program is started
  • while the program is running
  • in dynamically loaded plugins, especially in constructors
  • on a bit of code that was commented out. Make sure it is moved on debugger startup to the first line producing real code below that position and that it is hit there.
"Step into" a couple of times. Can you step into Qt source code (*.cpp file under QTDIR. You might need to configure source paths mapping under "Edit" -> "Preferences..." -> "Debugger" -> "General" -> "Add Qt sources...")?

(In case you are using a Qt version from the official installer, you may need to install "Sources" and "Qt Debug Information Files" for this version.)

(Mac: switch on 'Use Debug Versions of Frameworks' in run configuration. You need Qt sources.)

Test debugging helpers/python gdb: Do classes like QImage or std::string show beautiful information instead of the raw structure? automated
Step through some (not all) test* functions and check whether the displayed data looks correct The code contains comments with the expected displayed data. These are meant for semi-automatic runs and might differ from what you see. Use your own judgement what's correct and what's not.
Comment out the return statement inside the following functions one by one. Check whether you'll end up with a proper stack trace & locals display.
  • testNullPointerDeref()
  • testNullReference()
  • testEndlessLoop(); (break manually)
  • testEndlessRecursion();
  • testUncaughtException(); (not on Windows)
Test a breakpoint in a QThread:
  1. Uncomment the call to qthread::testQThread()
  2. Place a breakpoint in qthread::Thread::run()
  3. Run this in the debugger
  4. Make sure that:
    • the breakpoint receives the right number of hits
    • "Locals and Expressions" and "Stack" views show reasonable data, including the object names of the threads
Switch on temporarily 'Operate by Instruction' (small icon above the stack trace) and check whether you see disassembler output and can step by instruction
Tests using tests/manual/debugger/cli-io/cli-io.pro from Qt Creator's source repository
Test Result Annotation
Check I/O (qDebug, std::cout, std::cerr), on Win for both Debug and Release. stderr/stdout handling on Windows:
  • An application needs to be built with 'console' for stderr/stdout to appear (use Creator's "Run in terminal" setting)
  • Creator itself is built with 'console' in debug mode only.
  • qDebug() prints to stderr for ‘console’ apps, else to the Windows debugger log, which can be shown with the DbgView utility
Check "Run in Terminal". Use Terminal for input. (Debbuging might not work on Ubuntu)
Create new project. Can you build, run and debug it? automated
Check nothing bad happens on a simple int main() {} program with no breakpoints set automated
Tests using tests/manual/debugger/gui/gui.pro from Qt Creator's source repository
Test Result Annotation

When testing on Linux or Mac:

  1. Run the project outside Creator.
  2. Click the button "Crash".
    It should write a core file to the working directory. Depending on the operating system, you might have to set "ulimit -c unlimited" and/or disable OS-specific crash-reporting mechanisms, e.g. on Ubuntu.
  3. Attach Creator to the core file.
    Verify that you see a stack trace, variable values and code markers just as if you had run the application in the debugger from the beginning.
  1. Start the project from outside Creator, e.g. from the command line.
  2. Attach Creator to the running process.
    If it doesn't work at first, Creator should show an error message why not. Make sure that this message is helpful and following its instruction will make attaching work.
    Verify that you see a stack trace, variable values and code markers just as if you had run the application in the debugger from the beginning. You should also be able to pause/continue the execution.
Test unusual situations: Kill the running debugged program 'externally' with a signal that is neither SIGSTOP nor SIGKILL. Qt Creator should end up showing a backtrace.
Kill the stopped debugged program 'externally'. Nothing should happen immediately, on next continue or step Qt Creator should end up showing a backtrace.
Try remote debugging: start a gdbserver using e.g.
gdbserver localhost:1234 ./manual_test_debugger_gui

on a Linux machine and connect to it

Test run as root from inside QC - any simple application (Linux, macOS)
Test Result Annotation

Run the project without checking "Run as root user" on the run settings page.

  1. Verify application is starting and can be used as expected.

Debug the project without checking "Run as root user" on the run settings page.

  1. Verify application and debugger are starting correctly.
  2. Verify you can interrupt the application and hit a breakpoint. Check some locals whether they contain sane data.

Run the project with checking "Run as root user" on the run settings page.

  1. Verify application is starting correctly.
  2. Use the OS' means to verify that the application is actually running for user "root".
  3. Close the application and start it again immediately. Creator should not ask for entering credentials again.
  4. Run "sudo -K" to ensure credentials are reset.
  5. Re-run and check whether credentials are requested again.

Debug the project with checking "Run as root user" on the run settings page.

  1. Verify application and debugger are starting correctly.
    gdb running as "root" user is acceptable.
  2. Use the OS' means to verify that the application is actually running for user "root".
  3. Verify you can interrupt the application and hit a breakpoint. Check some locals whether they contain sane data.
  4. Close the debugging session and start it again immediately. Creator should not ask for entering credentials again.
  5. Run "sudo -K" to ensure credentials are reset.
  6. Re-run and check whether credentials are requested again.
  7. Re-do verifications above.

Is entering invalid credentials handled correctly?