Re: prctl(2) of compat_linux for Firefox for Linux/x86_64

[Tobias Nygren <tnn%NetBSD.org@localhost>]

On Tue, 22 Aug 2023 04:56:18 +0900
Ryo ONODERA <ryo%tetera.org@localhost> wrote:

> Hi,
> 
> The recent improvements in compat_linux reduces unimplemented
> Linux system calls that is used by the official Firefox binary
> for Linux/x86_64.
> Great thanks for Theodore Preduta and their great works.
> 
> Sadly, the official Firefox 117.0b9 binary for Linux/x86_64 does not
> start its GUI yet.
> From the result of ktrace command, the following calls remain unimplemented.
> 
>  22518  12874 firefox-bin CALL  #157 (unimplemented prctl)
>  22518  12874 firefox-bin RET   #157 (unimplemented prctl) -1 errno -38 Too many processes
> 
> I have no idea whether prctl(2) is the last unimplemented system call.
> So I may find another unimplemented during starting process of GUI.
> 
> Is there any plan to implement prctl(2) for compat_linux?

It looks like a kitchen sink syscall that provides all sorts
of functionality. Implementing full emulation would be very difficult.
Partial support for some of the options looks doable.

grep through the Firefox source code found these uses:

PR_SET_NAME - this is straightforward to emulate
PR_SET_VMA - this can be made a no-op

These are difficult because they depend on the kernel security model,
but it should be OK to just return EINVAL for both.

PR_SET_SECCOMP
PR_SET_NO_NEW_PRIVS

-Tobias