kern/48962: NetBSD doesn't conform to POSIX with mlock(2) and per-process limits; returns EAGAIN instead of ENOMEM

[yanegomi%gmail.com@localhost]

>Number:         48962
>Category:       kern
>Synopsis:       NetBSD doesn't conform to POSIX with mlock(2) and per-process 
>limits; returns EAGAIN instead of ENOMEM
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jul 03 19:55:00 +0000 2014
>Originator:     Garrett Cooper
>Release:        6.0-RELEASE
>Organization:
EMC / Isilon Storage Division
>Environment:
n/a
>Description:
mlock(2) in NetBSD states that per-process limits should return EAGAIN whereas 
mlock(2) in POSIX states that per-process limits may (note, not a hard 
requirement) return ENOMEM.

This was caught after closely examining the tests/lib/libc/sys/t_mlock.c port 
to FreeBSD ( 
https://github.com/yaneurabeya/freebsd/commit/04792995a548a9f5dc3d70791a88625d1e831082#diff-c2edb8c374a36673ab235f51ebf706feR203
 ).

From NetBSD's mlock(2):

     [EAGAIN]           Locking the indicated range would exceed either the
                        system or per-process limit for locked memory.

From POSIX:

The mlock() function may fail if:

[ENOMEM]
    Locking the pages mapped by the specified range would exceed an 
implementation-defined limit on the amount of memory that the process may lock.
>How-To-Repeat:
- Change the EAGAIN check in tests/lib/libc/sys/t_mlock.c to ENOMEM.
- Compile and re-run the tests with ATF/Kyua.
>Fix: