kern/53474: mmap get anon execuable memory return MAP_FAILED

[zh_jq%outlook.com@localhost]

>Number:         53474
>Category:       kern
>Synopsis:       mmap get anon execuable memory return MAP_FAILED
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jul 26 14:15:00 +0000 2018
>Originator:     Zhang Jingqiang
>Release:        8.0
>Organization:
>Environment:
NetBSD localhost 8.0 NetBSD 8.0 (GENERIC) #0: Tue Jul 17 14:59:51 UTC 2018  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/amd64/compile/GENERIC amd64
>Description:
When use mmap to allocate MAP_ANON executable memory, it always returns MAP_FAILED.
This makes pcre2_jit_compile returns NOMEMORY error, as they use this in "src/sljit/sljitExecAllocator.c".
>How-To-Repeat:
Use the following test program:
--
#include <stdio.h>
#include <sys/mman.h>

int main(int argc, char *argv[])
{
        void *retval = mmap(NULL, 4096, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANON, -1, 0);
        printf("retval: %p\n", retval);
        return 0;
}
--

Compile using the following command:
--
gcc -o test test.c
--

Then run ./test, you will get
  retval: 0xffffffffffffffff

But if we run the program with gdb, it works as expected.
And if remove PROT_EXEC, the program works also.
>Fix: