Reporting Bugs: Difference between revisions
AlexBlasche (talk | contribs) No edit summary |
Ahmad Samir (talk | contribs) |
||
(4 intermediate revisions by 2 users not shown) | |||
Line 43: | Line 43: | ||
===Components=== | ===Components=== | ||
*Select the component which best fits the location of the bug. | *Select the component which best fits the location of the bug. Note Components don't necessarily correlate to modules; for example, Qt Core has many sub-modules which you can select from. | ||
*You can select more than one | |||
=== Platforms === | |||
*You can select more than one platform by selecting one after the other from the drop-down menu. For example, if a widget problem is specific to Windows, you should select "Platform: Windows" and "Widgets". | |||
===Description=== | ===Description=== | ||
Line 90: | Line 93: | ||
*A screenshot, or longer stacktrace is also incredibly useful. | *A screenshot, or longer stacktrace is also incredibly useful. | ||
=== Assignee === | ===Assignee=== | ||
* By default this field is set to "Automatic" which means the assignee is chosen based on the first mentioned component. Only change this field if you really know what you are doing. A wrong assignment may lead to longer processing times as some aspects of the triaging process will be skipped. | *By default this field is set to "Automatic" which means the assignee is chosen based on the first mentioned component. Only change this field if you really know what you are doing. A wrong assignment may lead to longer processing times as some aspects of the triaging process will be skipped. | ||
===Platform/s=== | ===Platform/s=== | ||
Line 102: | Line 105: | ||
===Then click "Create"=== | ===Then click "Create"=== | ||
You can now either start | You can now either start coding a patch on Qt to find a solution and file a merge request, or wait until a Qtee [[Triaging Bugs|triages]], [[JIRA-Priorities|prioritizes]], schedules, and fixes the bug for a release. | ||
==Bugs with security-sensitive information== | ==Bugs with security-sensitive information== | ||
If you found a security problem, do ''not'' create a bug report on JIRA, but send email to security@qt-project(dot)org. | If you found a security problem, do ''not'' create a bug report on JIRA, but send email to security@qt-project(dot)org. See the [https://contribute.qt-project.org/quips/15 Qt Project Security Policy] for more details about how security problems should be reported, and how they get handled. | ||
==Reporting bugs as a commercial customer== | ==Reporting bugs as a commercial customer== | ||
Line 162: | Line 165: | ||
For more information on bisecting, please see the [http://www.kernel.org/pub/software/scm/git/docs/git-bisect.html git-bisect man page.] | For more information on bisecting, please see the [http://www.kernel.org/pub/software/scm/git/docs/git-bisect.html git-bisect man page.] | ||
== | == Reporting a bug that should block the next release == | ||
If you found a bug in a pre-release version of Qt that should, in your opinion, stop the release (for instance, a regression that breaks your application completely), then you can set the [[JIRA-Priorities|priority]] of the bug to "P0: Blocker", and the '''Affects Version''' field to the pre-release version that you are testing. | |||
==Reporting bugs for CI== | ==Reporting bugs for CI== | ||
Line 202: | Line 193: | ||
See [[Qt_Build_System_Glossary#Reporting build system issues]] | See [[Qt_Build_System_Glossary#Reporting build system issues]] | ||
== Marking a bug as a duplicate of another == | |||
* In the bug report, open the ''More'' menu at the top and select ''Link'' | |||
* From the ''This issue'' combobox select ''duplicates'' | |||
* From the ''Issue'' combobox set the parent bug report that this one duplicates | |||
* Finally click ''Link'' | |||
=Feature requests (named Suggestions in JIRA)= | |||
Feature requests are a bit different from bugs. They often (but not always) have a wider scope than bugs. For example, "Port Qt to platform X" is a feature request that sounds simple, but has a huge scope. | |||
The important thing to remember when submitting feature requests, particularly those with a big scope, is to be specific in the description: | |||
*Bad example: Please add QWidget::nudgeWidget(). | |||
*Good example: Please add a function to QWidget that will make the window shake for a short period of time, like when the user enters a bad password. The function can be called something like nudgeWidget(). | |||
In the second example, it is much clearer to the developer what the feature request actually means, since the meaning of "nudge" is ambiguous (it could also mean blink the colors of the window, like some advertisement popups do when browsing the Internet). | |||
Another very important piece of information in feature requests is to attach use cases. Use cases are concrete examples on situations where the feature will be used. For instance, the widget shaking in response to a bad password in the example above, is one such use case. | |||
Even though the use case may be crystal clear to the reporter, it may not be to the developer, so it is better to be explicit about it. If you have several distinct use cases, feel free to list more than one. That makes it more obvious that the feature can be useful for many people. |
Latest revision as of 13:06, 20 April 2024
En Ar Bg De El Es Fa Fi Fr Hi Hu It Ja Kn Ko Ms Nl Pl Pt Ru Sq Th Tr Uk Zh
Reporting bugs in Qt
Bugs in Qt can be reported at the bug tracker. The quality of the bug report can dramatically impact how likely it is that the bug will be fixed. A prerequisite for this is that a number of people (for example developers, code reviewers, QA and release managers) are able to quickly understand and verify the issue by reading the report and potentially running a code example. This page describes how to report bugs, and tips for making high quality reports.
In short
IMPORTANT: Created bug reports should always be self-contained and target a specific problem. Do not create collections of different failures with some generic title, eg. "Compilation failures", because commenting will become cluttered and the bug is difficult to close. If your issue concerns specific platform or environment, you should fill this information in the optional fields provided by the report template.
Always provide as much information as possible together with your bug report. The more the better; relevant information can be a stack trace if you are reporting a crash, or a CPU usage profile if you are reporting a performance issue. Also, please specify if a bug is a regression or not. If it is, then specify what version it worked in last. Finally, supplying a testcase main.cpp is a sure-fire way to land in the "fasttrack" of bug resolution and into the hearts of the Trolls. Also, providing a testcase is a great means of preventing the same bug from cropping up in the future as a regression.
By creating a high quality bug report, you will gain priority over those less descriptive, and your chances of a speedy resolution increased. If after all your hard work we still reject your bug, then please don't take it personally. Instead, add a comment in the report and explain your situation so that we can re-evaluate it. After all, we also make mistakes.
Step by step
Here is a nutshell description of how to report a bug:
- Visit bugreports.qt.io
- If it's your first time, create a Qt Account. Make sure to supply a valid email address; Qt developers sometimes need to ask for more details regarding a bug, and this is the only way they can reach you. Your email address will not be exposed; it is only possible to contact you by commenting on your reports (or commenting on bugs you're watching).
- Use the "Quick Search" field in the top right to try to find any similar bugs. If you find one, then…
- Leave a comment with any additional info you have.
- Click "Vote" — You will usually have a higher chance of having your bug fixed by voting on an already existing bug than creating a new one.
- Add yourself as a watcher, if you want to track the progress of the bug via email.
- If you don't find a bug report which matches your issue, then click "Create New Issues", and fill out the fields.
Summary
- Enter a short but descriptive text explaining the bug in a sentence.
- This should include an overall overview over how the bug happens without going into too much detail. They key is to think about what other people might search for.
- Bad example: "QLineEdit always goes wrong"
- Too long: "QLineEdit crashes in paintEvent at the for loop which draws characters. This happens if I use an input method based on XIM and then set it to Arabic, which also switches to right-to-left mode. Then I try to type something, and it crashes." A long text like this one should be put in the description instead; the summary should be kept short.
- Good example: "QLineEdit crashes in paintEvent when using an input method in right-to-left mode."
- If you are sure the bug affects only selected platforms, please pick the correct fields of the Platform/s field. Optionally you may prefix it by the platform and a colon: "Windows: QLineEdit crashes in paintEvent when using an input method in right-to-left mode."
- If the bug is a regression, that is, it worked in a previous version of Qt, indicate this as follows: "[REG 4.8.6->5.3.1] Windows: QLineEdit crashes in paintEvent when using an input method in right-to-left mode."
Affects Version
- Select the Qt version where you're able to reproduce the bug. If this is not the current minor version, please also verify against the current minor version to check whether the bug has been fixed in the meantime.
Components
- Select the component which best fits the location of the bug. Note Components don't necessarily correlate to modules; for example, Qt Core has many sub-modules which you can select from.
Platforms
- You can select more than one platform by selecting one after the other from the drop-down menu. For example, if a widget problem is specific to Windows, you should select "Platform: Windows" and "Widgets".
Description
- A longer text describing, step by step, the minimum amount of actions required to reproduce the bug.
- A good form to follow is this:
- What did you do? This ideally takes the form of short bullet points. However, try to be as precise as possible when describing something like a touch gesture.
- What did you expect to happen?
- What happened instead?
- If the bug is a crash, a vital piece of information in crash reports is the stacktrace. Retrieving a stracktrace is platform specific, however, all of them require that Qt be built with debug symbols. If you compiled Qt yourself, then make sure you supplied "-debug" during configure time. If you're using binaries, then make sure you've installed the corresponding debug packages. Paste the stacktrace in the description if it's not terribly long (surround it by to make it stand out from the rest), else attach it as a text file (use the suffix .txt so that it can be conveniently viewed by clicking). In Qt Creator, a verbose stack trace can be created by right-clicking on the stack window of the debugger and choosing Create full backtrace .
{noformat}My stack trace{noformat}
- If the bug is a performance problem, then a CPU usage profile can be useful for determining where the performance issue happens, and for verifying that the Qt developers are reproducing the same issue as the Qt user. Use a native profiler (like Instruments on macOS) to create the profile.
- Finally, please supply any more details that you may have in this field. For instance "This bug only occurs if I have compositing enabled", or "This is a regression from 4.6.1".
- Example: it will make the bug
- Launch attached example qtbug32777.zip
- Click on File Menu
- Choose item "Open"
- Note: Mouse cursor turns green
- EXPECTED: File dialog opens
- ACTUAL: Application crashes
Environment
- Input as detailed description of your operating system, window manager, compiler, and any other pertinent information needed to reproduce the bug.
- Also put the exact name of the package you used as well as the contents of the .tag file in the package if there is one. If using Git, please give the branch name and the commit ID.
- If you tested several platforms, it helps to add this information as well ("Windows: reproduced, Linux/KDE: not reproduced, OS X: reproduced"). As mentioned above, if you are sure the bug affects only one platform, it should be mentioned in the summary.
- Starting from version 5.3.0, there is a command line tool qtdiag, which outputs detailed information about operating system, compiler and graphics configuration. Particularly for bugs related to graphics, it is recommended to redirect its output into a file and attach that to the bug report, or add the output as a comment enclosed in {noformat} tags.
- For bugs related to OpenGL, High DPI scaling and multiple screens, please provide the output of the command line tool qtdiag (or at least the parts listing graphics driver and screens).
Attachment
- This often neglected field is perhaps one of the most important. Here is where you can make a simple application that reproduces the bug and attach it.
- If a single QML file is sufficient to reproduce the bug using the qml runtime (e.g. qml myfile.qml), just attach that. Keep the QML as simple as possible: e.g. don't use QtQuick.Controls if it's not related to the bug in any way, or doesn't make it easier to construct the example. Don't use ApplicationWindow if a plain Window or Item will work as the root item.
- If you need to use C++, then ideally you should attach an archive (tar.gz or zip file) consisting of a main.cpp and a .pro file that expands a directory whose name indicates the bug number and optionally some identifier (for example, qtbug32765_printbug.zip expanding a directory qtbug32765_printbug). The stand-alone main.cpp file should have the bare minimum amount of code needed to reproduce the bug. An application that reproduces the bug is the single most useful piece of information for a developer to try to identify and fix a bug. Qt Creator's Other Project / Code Snippet wizard is a useful tool to create such examples. Remember that the application (referred to as the test case) should follow the SSCCE rule:
- Short – Make it as short as possible. It is much easier to spot a problem in short code.
- Self Contained – Make sure you do not rely on any external libraries or files other than Qt. If you need extra nonstandard headers, copy paste what you need (but not more than you need) into the main.cpp file. If you need moc, you can put #include "main.moc" at the end of the file to force moc to include the metacode directly into your file.
- Correct – Make sure your application compiles and is correct code. For example, dereferencing a zero-pointer in the example code, is clearly not a bug in Qt.
- Example – Make sure that the application actually demonstrates the problem you are trying to solve.
- Avoid Qt Designer forms
- The example should output the version of Qt it uses (set QT_VERSION_STR as window title or output it via qDebug()).
- Avoid the use of #ifdef to show workarounds. Use simple command line options instead:
if (QCoreApplications::arguments().contains("-w")) …
- A unit test (of the kind you can find in the $QTDIR/tests/auto folder in the Git repository) is even better than a simple example, since we can add it to our automated tests to make sure this bug doesn't regress later.
- A screenshot, or longer stacktrace is also incredibly useful.
Assignee
- By default this field is set to "Automatic" which means the assignee is chosen based on the first mentioned component. Only change this field if you really know what you are doing. A wrong assignment may lead to longer processing times as some aspects of the triaging process will be skipped.
Platform/s
- Please pick the correct platform if the bug is limited to certain platforms only. If the bug is universal across all platforms you set the field blank for "All"..
- Leave the field blank if you are unsure.
- Note - This field is only different from the above mentioned Environment field as it permits a standardized quick filtering of bugs based on platforms.
Then click "Create"
You can now either start coding a patch on Qt to find a solution and file a merge request, or wait until a Qtee triages, prioritizes, schedules, and fixes the bug for a release.
Bugs with security-sensitive information
If you found a security problem, do not create a bug report on JIRA, but send email to security@qt-project(dot)org. See the Qt Project Security Policy for more details about how security problems should be reported, and how they get handled.
Reporting bugs as a commercial customer
The bug tracker is a community resource of the Qt Project, and may not always provide the experience you expect as a paying customer. If the problem you want to report has a high priority for you, you should use the Support Center accessible from your Qt Account to report it. The support team can help you with many of the steps detailed on this page, suggest workarounds, and will create a JIRA task on your behalf and tag it appropriately.
Tips for dealing with difficult bugs
Some bugs may be hard to reproduce, for example, it may happen only occasionally, it cannot be reproduced in a small example, or it only happens in a specific environment. However, for the Qt developers to be able to help you, it is still important that they are analyzed in enough detail so that the developer is able to see the problem for himself. This section describes some tips for dealing with such bugs.
Bugs that occur in a specific environment
This type of bug often follows this pattern:
- You identify a bug on your computer.
- You manage to create a test case for it and submit that to http://bugreports.qt.io/.
- A developer from Qt tries the example, but claims that he can not reproduce it.
- He/she may also close the bug with "Cannot reproduce" or change it to the "Need more info" state.
At this point, it important not to lose hope. These bugs are tricky, and solving them typically requires a lot of back and forth communication, since the bug can only be reproduced on the reporter's system.
You should start by asking for pointers in the comment field. A typical question would be: "Where would be a good place in the code to start debugging or place some debug print statements?" Most developers should respond favorably to this, and suggest places to look at in detail, or specific variables that you can check for consistency while the code runs. This, in turn, may lead to additional questions being asked, which hopefully leads to either the bug being identified, or the Qt developer being able to reproduce the problem on his machine using the new information.
The "Need more info" state is chosen if the developer requires more input as the given information is not complete. It is the reporters responsibility to provide further information (e.g. via further comments) which may ultimately lead to a verification of the bug. Once the info was provided the bug should be transitioned back to the "Reported" state by choosing the "Provide Missing Info" transition. Note that bugs in the "Need more info" may be closed automatically as described in the Jira Tips and Tricks. This may happen when no further information is forthcoming.
Bugs that cannot be reproduced in a small example
Typical for this type of bug is that in your big application, the bug can be easily triggered by enabling a certain attribute, for example
widget->setAttribute(Qt::WA_NativeWindow)
However, when you try to reproduce the same thing in a small example, the bug cannot be seen anymore.
One effective way of dealing with such bugs is the divide-and-conquer method. This method consists of a series of steps, where at each step, you remove a part of your program. Each time you remove something, you need to verify that the bug can still be reproduced, and if it can't you have to go back to the previous step and choose a different part to remove.
Start by removing code that is likely to be irrelevant to be bug, for example if the bug is a rendering bug, you should quite safely be able to remove networking code. Similarly, if your program has 20 QPushButtons, but only one is required to demonstrate the problem, remove the other 19.
Eventually, your program may reach a point where you are not able to remove any more code without the bug disappearing. Such a program is an excellent candidate for a test case that you can submit to the bug report. This whole process can take quite some time, but is sometimes the only way of isolating the real cause of a bug.
Bugs that only happen occasionally
These are usually the hardest types of bugs. The key here is to be prepared when the bug does happen. This can mean several things, for example, if the bug is a crash, make sure that you are always running in a debugger so that you can look at the stack when the crash happens. If there is a behavioral bug, for example a visual problem or corrupt data, try to put debug prints in various key locations, printing the value of variables that may be relevant to the problem, and make sure you always log those values. Then, when the problem happens, you can see at which point the variables started to go wrong.
Like the bugs that only happen in a certain environment, this is something that you can ask about in the bug comment field and most developers should give you some hints about which places and variables may be important to your type of problem. The Qt mailing lists are also a good place to ask, as well as the #qt-labs channel on the Freenode IRC network. (For a full list of IRC channels of interest to Qt developers, see OnlineCommunities.)
Things that used to work (regressions)
If you have a bug in Qt that is difficult to produce a test case for, but you know that the feature worked in the past, bisecting can be of a great help to pinpoint the problem. Bisection is a process in the Git version control tool that allows you to iteratively search for the exact commit that broke a given feature. A full description of the git-bisect process will not be given here, but this will give you an idea of the process involved:
git bisect start
- (Mark the previous version you know about where the feature worked)
git bisect good v4.6.2
- (Mark the current commit as broken)
git bisect bad
- Git should produce something like: Bisecting: 675 revisions left to test after this (roughly 7 steps)
- Compile Qt as well as your application, and test if the bug is present.
- (Good or bad depending on the outcome of the bug test)
git bisect <good|bad>
- Repeat the previous two steps.
- Eventually, git-bisect will tell you which commit introduced the problem. Include this commit in your bug report.
For more information on bisecting, please see the git-bisect man page.
Reporting a bug that should block the next release
If you found a bug in a pre-release version of Qt that should, in your opinion, stop the release (for instance, a regression that breaks your application completely), then you can set the priority of the bug to "P0: Blocker", and the Affects Version field to the pre-release version that you are testing.
Reporting bugs for CI
If you are experiencing problems with staging or merging a change in a qt-project repository, it might be because of an issue in our Continuous Integration system (CI). If that is the case, it is important that the created ticket contains complete information that speeds up the process to find the root cause and solution for the problem.
Needed Information
This list contains the minimum requirements for the bug report. If the reporter fails to fill in fields mentioned below, you might find your ticket status being set to Need More Information, which means that the problem cannot be resolved with the provided information. If that happens, you may update the required information and the work on the ticket may continue.
Project: Choose Qt Quality Assurance Infrastructure (QTQAINFRA) or Coin (COIN).
Issue type: Choose Bug, Suggestion or User Story.
Summary: Summary should be short and descriptive about what failed. Avoid generic titles such as "Module X does not compile", but be specific instead, e.g. "Project qt/qtdeclarative fails due to missing dbus library".
Affects Version/s: For QTQAINFRA issues, choose Qt software version or quarterly target. For COIN, you can should choose the product branch version 1.1 or master if the issue is related to CI self-tests or development branch.
Components: For QTQAINFRA issues, choose the affiliated CI infrastructure component (the Coin component in QTQAINFRA is now deprecated and new issues should be reported/moved under the new COIN project).
Description: You must include either related Gerrit Codereview URL (e.g. https://codereview.qt-project.org/#/c/206447/) or Coin integration task link (e.g. https://testresults.qt.io/coin/integration/qt/qt5/tasks/1511256273) in this section. This will help the assignee to start working on the solution without the need to dig into logs. This field is the placeholder for any information that might be relevant to solve the problem. If you know how to reproduce the issue, it might be helpful to include the instructions here.
Environment (optional): If you have a specific environment setup, you can describe it here.
Other: If there are any patch sets that are related to the problem, they should be added in the bug report. This will make sure that all the related fixes will be deployed in production.
Reporting Build System issues
See Qt_Build_System_Glossary#Reporting build system issues
Marking a bug as a duplicate of another
- In the bug report, open the More menu at the top and select Link
- From the This issue combobox select duplicates
- From the Issue combobox set the parent bug report that this one duplicates
- Finally click Link
Feature requests (named Suggestions in JIRA)
Feature requests are a bit different from bugs. They often (but not always) have a wider scope than bugs. For example, "Port Qt to platform X" is a feature request that sounds simple, but has a huge scope.
The important thing to remember when submitting feature requests, particularly those with a big scope, is to be specific in the description:
- Bad example: Please add QWidget::nudgeWidget().
- Good example: Please add a function to QWidget that will make the window shake for a short period of time, like when the user enters a bad password. The function can be called something like nudgeWidget().
In the second example, it is much clearer to the developer what the feature request actually means, since the meaning of "nudge" is ambiguous (it could also mean blink the colors of the window, like some advertisement popups do when browsing the Internet).
Another very important piece of information in feature requests is to attach use cases. Use cases are concrete examples on situations where the feature will be used. For instance, the widget shaking in response to a bad password in the example above, is one such use case.
Even though the use case may be crystal clear to the reporter, it may not be to the developer, so it is better to be explicit about it. If you have several distinct use cases, feel free to list more than one. That makes it more obvious that the feature can be useful for many people.