--tKW2IUtsqtDRztdT
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline


Folks,

Ever since verified exec has been around there has been an issue with
files stored on media that is not under the direct control of the
kernel running on the machine (for example, nfs mounts or even SAN
attached storage).  The problem being that one could exploit the
caching behaviour of veriexec by running an excutable to get it marked
as "ok" and then overwrite the executable file from another machine
with access to the storage and then go back and run the modified
executable on the first machine, since the executable is cached as
being ok the modified code would run.  Note that this is not a problem
with storage such as direct attached disks because the kernel controls
the storage and measures are in place to prevent the overwriting of
veriexec protected files.

A simplistic but flawed solution would be to not cache the fingerprint
comparison result on storage not under direct control of the kernel,
thus force a fingerprint evaluation on the file every time.  For a
short lived binary this is fine, a modification would be detected and
execution blocked.  The problem with this approach is that if a long
running binary, such as a daemon, is run then the fingerprint is only
checked prior to running the binary _but_ the pager will read pages of
the executable from the file as long as the binary is running.  This
provides a window of opportunity - if an attacker can overwrite the
executable file with a carefully crafted exploit, then by either
resource exhaustion or running a simple program on the machine running
the daemon force the pages for the daemon executable out of memory,
once the pages are out of memory the exploit can be launched by
causing pages for the daemon to be brought back in from storage by the
pager.  In effect, the pager along with uncontrolled storage allows
verifiedexec to be bypassed, previously the to this would have been
"don't use untrusted storage" which is a bit limiting.

To address the problem of untrusted storage I modified veriexec to
have an "untrusted" flag, this indicates that the file resides on
storage that is not under direct control of the kernel.  When a file
is marked untrusted, during the fingerprint evaluation phase not only
is the overall fingerprint evaluated but a fingerprint of each page of
the executable is made and stored (for those following along - both
the overall and per-page fingerprints are done in parallel, so there
can be no race condition).  If and only if the overall fingerprint
checks ok, the page fingerprints are kept.  The pager was modified to
compare the fingerprint of the page just read in with the per page
fingerprints (if they existed) and reject pages that fail that
fingerprint comparison.  At the time I shelved the fixes because the
changes added even more storage to the vnode structure which was
undesirable.

Elad Efrat took the original code, fixed it up and made it work with
the new style verified exec, this code was committed to the tree but
was partially backed out due to me putting the page check in the wrong
place, Elad fixed this problem and attached is the resultant patch
that will allow per page fingerprints.

If there are no objections, I would like to commit this update in the
near future.

-- 
Brett Lymn

--tKW2IUtsqtDRztdT
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="veriexec_per_page.diff"

Index: sbin/veriexecctl/veriexecctl.8
===================================================================
RCS file: /cvsroot/src/sbin/veriexecctl/veriexecctl.8,v
retrieving revision 1.19
diff -u -r1.19 veriexecctl.8
--- sbin/veriexecctl/veriexecctl.8	5 Oct 2005 13:58:49 -0000	1.19
+++ sbin/veriexecctl/veriexecctl.8	14 Nov 2005 11:58:22 -0000
@@ -69,8 +69,11 @@
 .Em path
 is the full path to the file and
 .Em type
-is the type of fingerprint used, see above for the default list.
-Other fingerprints may be available depending on kernel configuration.
+is the type of fingerprint used.
+The fingerprint types available depends on the kernel configuration.
+A list of supported fingerprint types can by found by querying the
+variable kern.veriexec.algorithms using
+.Xr sysctl 8 .
 The
 .Em fingerprint
 field is a hexadecimal representation of the fingerprint for
@@ -78,7 +81,7 @@
 The field
 .Em options
 contains the associated options for the file.
-Currently there are seven valid options:
+Valid options are:
 .Pp
 .Bl -tag -width INTERPRETER -compact
 .It Dv DIRECT
@@ -117,10 +120,14 @@
 .Dv FILE
 option.
 .It Dv UNTRUSTED
-This option is used to indicate that the file is located on
-untrusted storage, and its fingerprint should not be cached,
-but calculated each time it is accessed and when pages with
-this file as backing store are paged in.
+Indicates that the storage the file is on is not under direct control
+of the kernel and, therefore, the file cannot be guaranteed not to be
+modified.
+Enabling this flag will cause the veriexec kernel subsystem
+to perform fingerprinting at the memory page level and verify these
+fingerprints when paging parts of the file into memory.
+Enabling this flag will cause an increase in executable start up time,
+an increase in kernel memory usage and decrease in paging performance.
 .El
 .Pp
 There must be only one executable/fingerprint pair per line.
Index: sbin/veriexecctl/veriexecctl.c
===================================================================
RCS file: /cvsroot/src/sbin/veriexecctl/veriexecctl.c,v
retrieving revision 1.17
diff -u -r1.17 veriexecctl.c
--- sbin/veriexecctl/veriexecctl.c	5 Oct 2005 13:48:48 -0000	1.17
+++ sbin/veriexecctl/veriexecctl.c	14 Nov 2005 11:58:22 -0000
@@ -141,7 +141,7 @@
 	 * If there's no access type specified, use the default.
 	 */
 	if (!(params.type & (VERIEXEC_DIRECT|VERIEXEC_INDIRECT|VERIEXEC_FILE)))
-		params.type |= VERIEXEC_DIRECT;
+		params.type |= (VERIEXEC_DIRECT | VERIEXEC_INDIRECT);
 	if (ioctl(gfd, VERIEXEC_LOAD, &params) == -1)
 		warn("Cannot load params from `%s'", params.file);
 	free(params.fingerprint);
Index: sys/kern/kern_verifiedexec.c
===================================================================
RCS file: /cvsroot/src/sys/kern/kern_verifiedexec.c,v
retrieving revision 1.45
diff -u -r1.45 kern_verifiedexec.c
--- sys/kern/kern_verifiedexec.c	12 Oct 2005 14:26:47 -0000	1.45
+++ sys/kern/kern_verifiedexec.c	14 Nov 2005 11:58:32 -0000
@@ -514,7 +514,7 @@
  */
 int
 veriexec_page_verify(struct veriexec_hash_entry *vhe, struct vattr *va,
-		     struct vm_page *pg, size_t idx)
+		     const struct vm_page *pg, size_t idx)
 {
 	void *ctx;
 	u_char *fp;
@@ -591,6 +591,35 @@
 	return (error);
 }
 
+int
+veriexec_block_verify(struct vnode *vp, const struct vm_page **pps,
+    voff_t offset, int npages)
+{
+	struct veriexec_hash_entry *vhe;
+	struct vattr va;
+	voff_t offidx, i;
+	int error;
+
+	error = VOP_GETATTR(vp, &va, curlwp->l_proc->p_ucred,
+	    curlwp->l_proc);
+	if (error)
+		return (error);
+
+	vhe = veriexec_lookup(va.va_fsid, va.va_fileid);
+	if ((vhe == NULL) || (vhe->page_fp == NULL))
+		return (0);
+
+	offidx = (offset >> PAGE_SHIFT);
+
+	for (i = 0; i < npages; i++) {
+		error = veriexec_page_verify(vhe, &va, pps[i], i + offidx);
+		if (error)
+			return (error);
+	}
+
+	return (0);
+}
+
 /*
  * Veriexec remove policy code.
  */
Index: sys/sys/verified_exec.h
===================================================================
RCS file: /cvsroot/src/sys/sys/verified_exec.h,v
retrieving revision 1.21
diff -u -r1.21 verified_exec.h
--- sys/sys/verified_exec.h	7 Oct 2005 18:07:46 -0000	1.21
+++ sys/sys/verified_exec.h	14 Nov 2005 11:58:33 -0000
@@ -199,7 +199,8 @@
 int veriexec_verify(struct proc *, struct vnode *, struct vattr *,
 		    const u_char *, int, struct veriexec_hash_entry **);
 int veriexec_page_verify(struct veriexec_hash_entry *, struct vattr *,
-			 struct vm_page *, size_t);
+			 const struct vm_page *, size_t);
+int veriexec_block_verify(struct vnode *, const struct vm_page **, voff_t, int);
 int veriexec_removechk(struct proc *, struct vnode *, const char *);
 int veriexec_renamechk(struct vnode *, const char *, const char *);
 void veriexec_init_fp_ops(void);
Index: sys/uvm/uvm_vnode.c
===================================================================
RCS file: /cvsroot/src/sys/uvm/uvm_vnode.c,v
retrieving revision 1.66
diff -u -r1.66 uvm_vnode.c
--- sys/uvm/uvm_vnode.c	27 Jun 2005 02:29:32 -0000	1.66
+++ sys/uvm/uvm_vnode.c	14 Nov 2005 11:58:33 -0000
@@ -55,6 +55,7 @@
 #include "fs_nfs.h"
 #include "opt_uvmhist.h"
 #include "opt_ddb.h"
+#include "opt_verified_exec.h"
 
 #include <sys/param.h>
 #include <sys/systm.h>
@@ -68,6 +69,9 @@
 #include <sys/conf.h>
 #include <sys/pool.h>
 #include <sys/mount.h>
+#ifdef VERIFIED_EXEC
+#include <sys/verified_exec.h>
+#endif /* VERIFIED_EXEC */
 
 #include <miscfs/specfs/specdev.h>
 
@@ -299,6 +303,14 @@
 	UVMHIST_LOG(ubchist, "vp %p off 0x%x", vp, (int)offset, 0,0);
 	error = VOP_GETPAGES(vp, offset, pps, npagesp, centeridx,
 			     access_type, advice, flags);
+
+#ifdef VERIFIED_EXEC
+	if (!error && !(flags & PGO_SYNCIO))
+		error = veriexec_block_verify(vp,
+					      (const struct vm_page **) pps,
+					      offset, *npagesp);
+#endif /* VERIFIED_EXEC */
+
 	return error;
 }
 

--tKW2IUtsqtDRztdT--