Continuous verification of standard library

[Shnatsel]

Two serious vulnerabilities have been discovered in libstd to date. Another one was introduced but reverted before release because it was so bad that it caused crashes even on valid data. All of these were introduced during optimization or refactoring, and have passed manual code review.

The fact that humans are no good at analyzing unsafe code is the very reason for Rust’s existence. We need computers to assist in verification of Rust’s standard library.

There are several ways to go about that:

1. Static analysis would be a relatively cheap and scalable way to gain more confidence in the code. Rust is much more amenable to static analysis than C/C++ or dynamically typed languages, but there is no go-to security-oriented static analyzer yet. This is contingent on advances in static analyzers, see Safety-oriented static analysis tooling #22
2. Fuzzing or parametric testing could also scale well, assuming fuzzing harnesses could be automatically generated based on type definitions of stdlib functions. It would not find all the bugs, but it is easy to run continuously and feasible to scale to the entirety of the standard library with little maintenance burden. This is actionable right now since all the pieces are already there: AFL/libfuzzer/honggfuzz or even basic QuickCheck/Proptest RNGs for generating random bytes, Arbitrary from QuickCheck or Proptest to turn bytes into structured data, and syn for parsing the code to audit and automatically generating the code that does the "generate random bytes, turn them into data structures, feed them to the function being tested" loop.
3. Formal verification methods provide greater assurance in correctness, but require more effort and introduce a non-trivial maintenance burden. Even though verifying the entirety of standard library this way is probably not practical at this time, it would be great to apply them to verify the most essential parts of it. RustBelt tooling or SMACK would be the tools of the trade.

One of the already discovered vulnerabilities was trivial and would have been flagged by a static analyzer or easily discovered via fuzzing — if any of those were actually employed.

[Shnatsel]

https://github.com/Eh2406/auto-fuzz-test is an experiment in automatically generating fuzzing harnesses based on function signatures. No docs yet, but it seems to be a functional proof-of-concept.