[src/trunk]: src/sys/uvm PR kern/55268: tmpfs is slow

[ad <ad%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/a28b1eda13fb
branches:  trunk
changeset: 932786:a28b1eda13fb
user:      ad <ad%NetBSD.org@localhost>
date:      Fri May 15 22:27:04 2020 +0000

description:
PR kern/55268: tmpfs is slow

uao_get(): in the PGO_LOCKED case, we're okay to allocate a new page as long
as the caller holds a write lock.  PGO_NOBUSY doesn't put a stop to that.

diffstat:

 sys/uvm/uvm_aobj.c |  13 ++++++++-----
 1 files changed, 8 insertions(+), 5 deletions(-)

diffs (50 lines):

diff -r a03347cf750f -r a28b1eda13fb sys/uvm/uvm_aobj.c
--- a/sys/uvm/uvm_aobj.c        Fri May 15 22:25:18 2020 +0000
+++ b/sys/uvm/uvm_aobj.c        Fri May 15 22:27:04 2020 +0000
@@ -1,4 +1,4 @@
-/*     $NetBSD: uvm_aobj.c,v 1.139 2020/03/22 18:32:42 ad Exp $        */
+/*     $NetBSD: uvm_aobj.c,v 1.140 2020/05/15 22:27:04 ad Exp $        */
 
 /*
  * Copyright (c) 1998 Chuck Silvers, Charles D. Cranor and
@@ -38,7 +38,7 @@
  */
 
 #include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: uvm_aobj.c,v 1.139 2020/03/22 18:32:42 ad Exp $");
+__KERNEL_RCSID(0, "$NetBSD: uvm_aobj.c,v 1.140 2020/05/15 22:27:04 ad Exp $");
 
 #ifdef _KERNEL_OPT
 #include "opt_uvmhist.h"
@@ -827,6 +827,7 @@
         */
 
        if (flags & PGO_LOCKED) {
+               krw_t lktype = rw_lock_op(uobj->vmobjlock);
 
                /*
                 * step 1a: get pages that are already resident.   only do
@@ -845,11 +846,11 @@
 
                        /*
                         * if page is new, attempt to allocate the page,
-                        * zero-fill'd.  we can only do this if busying
-                        * pages, as otherwise the object is read locked.
+                        * zero-fill'd.  we can only do this if the caller
+                        * holds a write lock.
                         */
 
-                       if ((flags & PGO_NOBUSY) == 0 && ptmp == NULL &&
+                       if (ptmp == NULL && lktype == RW_WRITER &&
                            uao_find_swslot(uobj,
                            current_offset >> PAGE_SHIFT) == 0) {
                                ptmp = uao_pagealloc(uobj, current_offset,
@@ -859,6 +860,8 @@
                                        ptmp->flags &= ~(PG_FAKE);
                                        uvm_pagemarkdirty(ptmp,
                                            UVM_PAGE_STATUS_UNKNOWN);
+                                       if ((flags & PGO_NOBUSY) != 0)
+                                               ptmp->flags &= ~PG_BUSY;
                                        goto gotpage;
                                }
                        }