Subject: Kernel modification that verifies execs against a md5 fingerprint
To: None <tech-security@netbsd.org>
From: Brett Lymn <blymn@baea.com.au>
List: tech-security
Date: 03/14/2000 13:36:46
  by redmail.netbsd.org with SMTP; 14 Mar 2000 03:07:31 -0000
	by myall.awadi.com.au (8.9.3/8.9.3) with SMTP id NAA00216
	for <tech-security@netbsd.org>; Tue, 14 Mar 2000 13:36:49 +1030 (CST)
	id NAA22900; Tue, 14 Mar 2000 13:36:47 +1030
	id NAA04792; Tue, 14 Mar 2000 13:36:47 +1030
From: blymn@baea.com.au (Brett Lymn)
Message-Id: <200003140306.NAA04792@mallee.awadi>
Subject: Kernel modification that verifies execs against a md5 fingerprint
To: tech-security@netbsd.org
Date: Tue, 14 Mar 2000 13:36:46 +1030 (CST)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit


Folks,
        I have a kernel modification that I have spouted off about on
and off for a while - I did offer it on tech-kern a while back which
sparked a bit of a discussion but noone seemed to think it worth
looking at.  I am hoping that some people here may take some time to
see what I have done and comment on it.

What I have done is hack the kernel exec function so that it performs
a md5 fingerprint evaluation on a file prior to execution.  It checks
this fingerprint against a list of fingerprints kept in kernel memory
(these are loaded in at boot time via a pseudo-device) and if the md5
fingerprint matches the execution proceeds otherwise the exec fails.
This means that if a person attempts to trojan a binary it will not
run, if someone attempts to run a file that is not in the kernel
fingerprint list it will not run.  A subtlety that I have added since
the last round is that you can prevent the _shell_interpreter_ from
being run directly but can allow shell scripts using this interpreter
to run.  For example, if you use perl then with my hacks you can
prevent /usr/pkg/bin/perl from being run from the a user shell prompt
but foo.pl which has #!/usr/pkg/bin/perl as it's first line will run -
of course foo.pl will have to pass the md5 fingerprinting first.

Naturally, evaluating a md5 fingerprint on a executable every time
does involve overhead - it slows the machine down to almost half speed
(things take about 70% longer).  To overcome this slow down the status
of the md5 fingerprint comparison is cached in the kernel which
markedly reduces the impact of the md5 fingerprinting on the running
of the system.

If people are interested I have patches available that apply cleanly
to a -current system as of a couple of weeks ago and I have a rough
draft of a paper on the system as well.  Please note that this hack is
at the proof of concept stage and some parts are still suboptimal.  I
have some ideas for improvements which are in the paper.  Apart from
one improvement - I believe that the md5 fingerprint need not only
apply to execs, I think it should be possible to do this on file open
as well which would be interesting.

Please CC me on any replies as I am not on tech-security.

-- 
===============================================================================
Brett Lymn, Computer Systems Administrator, BAE SYSTEMS
===============================================================================