Re: [gsoc] syscall/libc fuzzer proposal

[Steven Bellovin <smb%cs.columbia.edu@localhost>]

On Mar 20, 2010, at 3:35 PM, David Holland wrote:

> On Sat, Mar 20, 2010 at 12:40:12PM -0400, Thor Lancelot Simon wrote:
>>> As a part of my work I would like to write a translator for C
>>> language and a small library. Their goal would be to detect
>>> integer overflows, stack overflows, problems with static array
>>> indexing, etc (when such occur during the program execution). It
>>> will enable me to uncover more bugs in the software.
>> 
>> What is the benefit of this when compared to existing static-analysis
>> tools such as Coverity Scan, splint, or the Clang static analyzer?  Will
>> this cover any cases they don't?  If so, which ones?
> 
> AIUI from chat, the idea is to increase the probability that if the
> testing causes something bogus to happen, the bogus behavior will
> result in an easily identifiable abort.
> 
> This seems like a valid line of reasoning; all the same, implementing
> such a tool is a fairly big (and annoying) pile of grunt work. Plus
> various variations on it have been done before. (Some of which might
> be worth looking into, actually.)

Yes.  Beyond that, though, I think that fuzzing is a very useful, valid 
security *testing* tool.

Thor asked how this compared to Coverity and the like.  Unless I badly 
misunderstand what Coverity is about, it's a static analysis tool.  Static 
analysis is great -- when it does the job.  It can find bugs that would require 
really unlikely test cases and timing.  But there are things that static 
analysis can't do, even in principle.  Fuzzing is one tool, among many, for 
dynamic testing.

Now -- as has been noted, just saying "fuzzing" isn't enough.  There are tools 
around that do it; there's also the NetBSD testing framework.  Integrating all 
that, plus new code and perhaps new analysis, would probably be a better way to 
proceed.  Note carefully that I said "probably" -- I haven't fleshed out my 
ideas any more than the original poster.  But I do encourage the original 
poster to proceed and develop the proposal some more.


                --Steve Bellovin, http://www.cs.columbia.edu/~smb