Testmgr fuzz testing

[Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>]

Eric,

While fixing some issues in the inside-secure driver reported by the fuzz, I noticed that the
results are actually not repeatable: odds are  high that on the next run, the error case is 
actually not hit anymore since they're typically very specific  corner cases.

There's 2 problems with that:
a) Without repeatability, I cannot verify whether my fix actually worked. In fact, I cannot
even verify with any certainty that any modification I do won't fail somewhere else :-(
b) Odds are very significant that important corner cases are not hit by the fuzzing

Issue a) is usually solved by making the random generation deterministic, i.e. ensure
you seed it with a known constant and pull the random numbers strictly sequentially.
(you may or may not add the *option* to  pull the seed from some true random source)

Issue b) would be best solved by splitting the fuzz testing into two parts, a (properly
constrained!) random part and a part with fixed known corner cases where you use
constant parameters (like lengths and such) but depend on the generic implementation
for the actual vector generation (as specifications usually don't provide vectors for
all interesting corner cases but we consider the generic implementation to be correct) 

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com