On Mon, Nov 14, 2005 at 11:17:28PM +1030, Brett Lymn wrote:
> 
> Folks,
> 
> Ever since verified exec has been around there has been an issue with
> files stored on media that is not under the direct control of the
> kernel running on the machine (for example, nfs mounts or even SAN
> attached storage).  The problem being that one could exploit the
> caching behaviour of veriexec by running an excutable to get it marked
> as "ok" and then overwrite the executable file from another machine
> with access to the storage and then go back and run the modified
> executable on the first machine, since the executable is cached as
> being ok the modified code would run.  Note that this is not a problem
> with storage such as direct attached disks because the kernel controls
> the storage and measures are in place to prevent the overwriting of
> veriexec protected files.
> 
> A simplistic but flawed solution would be to not cache the fingerprint
> comparison result on storage not under direct control of the kernel,
> thus force a fingerprint evaluation on the file every time.  For a
> short lived binary this is fine, a modification would be detected and
> execution blocked.  The problem with this approach is that if a long
> running binary, such as a daemon, is run then the fingerprint is only
> checked prior to running the binary _but_ the pager will read pages of
> the executable from the file as long as the binary is running.  This
> provides a window of opportunity - if an attacker can overwrite the
> executable file with a carefully crafted exploit, then by either
> resource exhaustion or running a simple program on the machine running
> the daemon force the pages for the daemon executable out of memory,
> once the pages are out of memory the exploit can be launched by
> causing pages for the daemon to be brought back in from storage by the
> pager.  In effect, the pager along with uncontrolled storage allows
> verifiedexec to be bypassed, previously the to this would have been
> "don't use untrusted storage" which is a bit limiting.
> 
> To address the problem of untrusted storage I modified veriexec to
> have an "untrusted" flag, this indicates that the file resides on
> storage that is not under direct control of the kernel.  When a file
> is marked untrusted, during the fingerprint evaluation phase not only
> is the overall fingerprint evaluated but a fingerprint of each page of
> the executable is made and stored (for those following along - both
> the overall and per-page fingerprints are done in parallel, so there
> can be no race condition).  If and only if the overall fingerprint
> checks ok, the page fingerprints are kept.  The pager was modified to
> compare the fingerprint of the page just read in with the per page
> fingerprints (if they existed) and reject pages that fail that
> fingerprint comparison.  At the time I shelved the fixes because the
> changes added even more storage to the vnode structure which was
> undesirable.
> 
> Elad Efrat took the original code, fixed it up and made it work with
> the new style verified exec, this code was committed to the tree but
> was partially backed out due to me putting the page check in the wrong
> place, Elad fixed this problem and attached is the resultant patch
> that will allow per page fingerprints.
> 
> If there are no objections, I would like to commit this update in the
> near future.

	I would have thought that something like this would be a mountpoint
option instead of a per-file thing.  
e.g. in veriexec_verify():
	if (vp->v_mount->mnt_flag & MNT_UNTRUSTED ||
	    do_something_extra_for_unionfs_or_nullfs_or_...)
		vhe->type |= VERIEXEC_UNTRUSTED;

In what case would you actually need a per-file setting?

> Index: sys/kern/kern_verifiedexec.c
> ===================================================================
> RCS file: /cvsroot/src/sys/kern/kern_verifiedexec.c,v
> retrieving revision 1.45
...
don't you also need to remove the "#if 0" around line 223, so do_perpage
gets turned on?

eric