Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve dynamic analysis tooling #26

Open
Shnatsel opened this issue Jan 14, 2019 · 1 comment
Open

Improve dynamic analysis tooling #26

Shnatsel opened this issue Jan 14, 2019 · 1 comment

Comments

@Shnatsel
Copy link
Member

We already have some excellent tooling for generating test cases for programs to make them blow up:

We also want to add Angora to this collection which has raised the bar on fuzzing, see #17.

However, all of this glory is pretty much for naught if we don't have tooling to detect the runtime faults that these tests trigger. The current state is:

  • Address Sanitizer is great, but requires workarounds to work (see Improve ergonomics for Address Sanitizer #20)
  • Memory Sanitizer is lacking documentation and is very hard to use with Rust (see Make Memory Sanitizer actually usable #21)
  • MIRI doesn't handle arbitrary Rust code, is not easy to use, does not have a clear value proposition compared to sanitizers and does not compose with fuzzers
  • Nobody knows if Valgrind actually works now that Rust has ditched jemalloc, but it also doesn't compose with fuzzers, which limits its usefulness.
  • Obscure tools such as DUMA are obscure for a reason.
  • Bespoke tools such as libdiffuzz are not applicable in the general case, and are not easy/straightforward/foolproof enough for general use.
  • Rust-specific tooling that works on actual compiled binaries pretty much does not exist.

Getting Address Sanitizer to work without workarounds (see #20) would already put us in an excellent position. Getting Memory Sanitizer to work would be harder, it is not universally applicable (see #21). Perhaps a more usable Rust-specific alternative could be concocted.

@Shnatsel
Copy link
Member Author

Shnatsel commented Jan 14, 2019

FWIW I've taken a stab at "universally applicable alternative to Memory Sanitizer" by writing libdiffuzz, but my approach was very simplistic, so the tool is not any better than MSAN, it just has completely different limitations.

A more sophisticated approach that's closer to what MSAN is doing might yield better results. Or perhaps someone could just hammer MSAN into something generally applicable.

@Shnatsel Shnatsel changed the title Great dynamic analysis tooling Improve dynamic analysis tooling Jan 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant