Re: port-arm/55704: multi-threaded applications for earmv[45]{,hf} freeze on COMPAT_NETBSD32 of aarch64

[Rin Okuyama <rokuyama.rk%gmail.com@localhost>]

The following reply was made to PR port-arm/55704; it has been noted by GNATS.

From: Rin Okuyama <rokuyama.rk%gmail.com@localhost>
To: Tobias Nygren <tnn%NetBSD.org@localhost>
Cc: gnats-bugs%netbsd.org@localhost
Subject: Re: port-arm/55704: multi-threaded applications for earmv[45]{,hf}
 freeze on COMPAT_NETBSD32 of aarch64
Date: Sat, 10 Oct 2020 08:36:58 +0900

 On 2020/10/08 20:05, Tobias Nygren wrote:
 >   > I'm not sure whether we can fix this problem without modifying userland
 >   > binaries for earmv[45]{,hf}. While arm variants prior to v6 realize
 >   > atomic_ops(3) by swp instruction (we emulate it for COMPAT_NETBSD32),
 >   > they does not have membar_ops(3), since they are not intended for
 >   > multi-processor machines. Actually, you can see our membar_ops(3) are
 >   > no-op for arm processors prior to v6:
 >   
 >   There were, in another context, discussions about need of pinning
 >   processes to the CPU they were launched on in some situations. There
 >   was some (don't recall the model) Samsung big.LITTLE SoC that only
 >   implements crypto instruction set extensions on the big cores.
 >   
 >   Can the compat layer detect at exec time if the binary is v[45] or v6+?
 >   for a purely userland solution you might be able to pin the process
 >   in ld.elf_so or with an LD_PRELOAD library.
 >   The problem with that is only root has permision to use pset(3).
 
 Yes, kernel recognizes for which architecture a binary is built,
 from ELF note embedded in it.
 
 	https://nxr.netbsd.org/xref/src/sys/kern/exec_elf.c#1047
 
 Kernel saves it in an MD field of struct proc,
 
 	https://nxr.netbsd.org/xref/src/sys/arch/aarch64/aarch64/exec_machdep.c#89
 
 in order to export it as hw.machine sysctl node.
 
 	https://nxr.netbsd.org/xref/src/sys/arch/aarch64/include/netbsd32_machdep.h#122
 	https://nxr.netbsd.org/xref/src/sys/compat/netbsd32/netbsd32_sysctl.c#107
 
 Therefore, kernel, in principle, can detect a v[45] process to pin
 it onto a CPU core, without needing to bother userland with
 permission of pset(3), or sched_setaffinity(2).
 
 However, unfortunately, there still remain problems. (1) Scheduling
 for v[45] processes should becomes poor. Also, (2) there is still
 no memory coherence guaranteed between threads belonging to
 *different* processes; consider if they share some memory region
 with MAP_SHARED mapping.
 
 (2) may be pretty rare cases although...
 
 Thanks,
 rin