The -fsanitize=integer analysis from UBSan can be helpful to detect signed and unsigned integer overflows in the codebase. Unfortunately, those occur very frequently, making it impossible to test anything with it without the use of a huge blacklist. This patch includes a blacklist that is broad enough to silence everything that would drain performance too much. But even with this blacklist, neither tests nor fuzzing is "clean". We can however in the future combine this with static analysis to limit ourselves to interesting places to look at, or improve the dynamic analysis to omit typical benign overflows.
It also adds another attribute that can be used on functions. It is not used right now because it was initially easier to add things to the compile-time blacklist to get started.
Finally, it includes a runtime suppression list and patches various parts in the test harnesses to support that. It is currently empty and it should not be used on frequent overflows because it is expensive. However, it has the advantage that it can be used to differentiate between signed and unsigned overflows while the compile-time blacklist cannot do that. So it can be used to e.g. silence unsigned integer overflows on a file or function while still reporting signed issues. We can also use this suppression list for any other UBSan related suppressions, should we ever want to use other features from that sanitizer.
MozReview-Commit-ID: C5ofhfJdpCS
--HG--
extra : rebase_source : 64aa804965d24bb90b103c00c692a2ac6859e408
Carsten "Tomcat" Book
Christian Holler
Ting-Yu Chou
David Major
Steve Fink