QtCS2021 - Wanna help handling oss-fuzz issues
Session Summary
oss-fuzz is fuzzing Qt since the beginning of 2020. While a number of people contributed to this in various ways (Thank you!), I still seem to be the bottle-neck when it comes to forwarding oss-fuzz' reports to the Qt project as well as documenting affected and fixed versions of Qt. After a brief introduction to oss-fuzz in general, in this talk I'd like to demo what are the required steps. My hope is, that some people will be interested in joining me, so we could share the work, extend the coverage and thus make Qt more secure and reliable.
Session Owners
- Robert Löhning (robert.loehning@qt.io)
Notes
Fuzzing in General
- Tool(s) generating lots of data, ideally in a clever way
- Feeding the data into the code under test
- Checks for things like crashes, timeouts, sanitizer issues, ...
- Good fuzzers
- Track code coverage and generate inputs accordingly
- Try to identify significant portions of input and only change these
One tool: libfuzzer
- Part of clang toolchain
- Has worked on Linux for years
- Newer versions claim to work on Windows, too. Presenter didn't try.
- Can be used with Qt
Testing Qt Code using lifuzzer
The following should also be documented in qtbase/tests/libfuzzer/README. If that's unclear, please inform Robert.
Doesn't strictly need a dedicated Qt build, but using one configured with the following switches is highly recommended:
-platform linux-clang -sanitize fuzzer-no-link -sanitize undefined
Have a source file which only contains a fuzzer test function like:
extern "C" int LLVMFuzzerTestOneInput(const char *Data, size_t Size) {
yourTestedFuncton(QByteArray::fromRawData(Data, Size));
return 0;
}
Build that source file with linker flag:
-fsanitize=fuzzer
See the test for QTextDocument::setHtml as an example.
Run the resulting binary. Keep an eye on printed "exec/s". Values above 1000 are nice. If it's below 100, you won't get very far. Speed up the tested code if possible.
oss-fuzz
Platform and service run by Google which runs fuzzing, also on Qt. It already found a low 3 digit number of issues in one and a half year. It generates daily public coverage reports about code the fuzzer ran.
Issues are written to Google's own bugtracker. First in confidential state, but get published after 90 days or after being fixed), whichever comes first.
That causes a bottleneck (because Qt's developers don't notice).
Currently, issues need to be forwarded to Qt's bugtracker/security mailing list. Also needs help reporting back to oss-fuzz where fix landed. Thus, there is no point in generating more reports (by adding more tests), unless we can handle the current findings.
Proposals for Improvement
Help with adding information
- Download "Reproducer Testcase" from public issue.
- Try reproducing it with different versions of Qt
- Build the test locally (see above). "Fuzz target" indicates the used test, "Job Type" indicates used sanitizers.
- Pass reproducer testcase to resulting binary
- If build is affected by issue, binary should report that.
- Note down in report which Qt version are affected/fixed.
- Needs Google account
- If fix is unclear: research, maybe bisect?
What's in it for you?
Might be interesting to to explore Qt's code and see where issues are/were.
Developers/Maintainers: pick up open issues from oss-fuzz directly
- Those issues are confidential => prefer to limit access to maintainers, people on security mailing list, or contributors with some track record in the project.
- For those, detailed report is available
- Needs Google account for granting access. Not necessary @gmail.com, but without a Google account you could only get update messages which won't let you reproduce. Would be added publicly in project.yaml.
- If you like to get access, please contact Robert
- If known, please reference oss-fuzz issue number in git commit message. Helps with manually checking/updating entry.
What's in it for you?
Quicker turn-around from issue to maintainer awareness.
Q&A
Q: Is there some fine-grained access control
A: No. Either all confidential issues in Qt or only public ones.
Q: Can we use a bot to automate this
A: Sometimes fix might not work in other branch, so better to verify manually
Any further proposals
Q: Address issues in company's bug fixing week?
A: Good idea, will bring that up.