Hopefully I can batch some replies to save some traffic - if I don't
cover something then prod me offline :-)

According to Jonathan Stone:
>
>To put it another way, are you asking about hashing keys into an
>associative hash-table, 

This is what I am after, I was hoping for an online reference
somewhere :-).  What I am doing at the moment is feeding md5
signature/filename pairs into the kernel via a pseudo-device (the
pseudo-device only accepts the pairs with securelevel <= 0).  The
inode & dev numbers are stashed into lists.  On exec the lists are
searched for the dev & inode pair and the associated signature
compared against the one that has just been calculated (or fetched
from cache).  From what I understand an associative hash table would
mean a number of smaller lists that can be searched more quickly which
means less negative impact on exec.  Another thing that occurred to me
today was to shuffle the order of the contents of the lists so that
the most often referenced entries migrate to the top of the signature
list which would mean that frequently executed binaries would be found
in the signature list more quickly.

The signature of a file is calculated on execution of the binary and
the result stored in the DNLC.  The calculation is done by reading the
entire file and evaluating the md5 signature.  This buggers the joys
of demand paged binaries but only if there is a cache miss on the
evaluated signature.  If someone overwrites the binary of a long
running process then one of a few things is going to happen:

1) the program dies because "they" have not been careful enough.
2) the old binary is deleted and a new file written - the pager would
still have hold of the old binary to page in from.
3) the program continues to run and "they" can trigger their exploit
by whatever means "they" thought of.  I would think that this is quite
an awesome hack if it worked :-)

What I have done is not the be all and end all in security, it is just
another string in the security bow.  It does prevent a lot of the
common cracking techniques such as replacing login with a trojan that
logs passwords and such like from working since the system will refuse
to execute the modified login binary.  It also stops the script
kiddies from starting up a network sniffer to snatch passwords
(assuming you did not already have one there ;-).  If "they" are
particularly stupid, and you should not assume so, it will stop them
running their shell scripts - there is an easy work around to that
though.  You are still vulnerable to trojans in shared libraries and
hacks to configuration files.

As for someone booting from floppy et al. - sure they can but this is
so for any security you put in place.  I would hope that if the
machine is important enough to trouble with securing in a software
sense then it will also be physically secured.  Try thinking of what I
have done as a sort of on-demand tripwire for executables :-)

>or designing a cryptographically-strong hash
>function?  For the former, see. e.,g, Knuth, or CLR; if you mean the
>latter, the short answer is: _dont_.  It's a hard problem, and if
>you're asking here, you probalby dont have the background in maths to
>get it right.
>

There are no worries about me trying that, my math just will not cut
it ;-)

-- 
===============================================================================
Brett Lymn, Computer Systems Administrator, British Aerospace Australia
===============================================================================