Re: SOSEND_LOAN problems in MIPS

[coypu%SDF.ORG@localhost]

On Sun, Jun 19, 2016 at 04:25:20PM +0100, Robert Swindells wrote:
> 
> You can find what the option does using grep(1). Use the search button
> on the mailing list archives to see if anyone else has had a similar
> problem.
> 
> It looks to me as if the only place that it is used is in
> sys/kern/uipc_socket.c.
> 
> The behaviour that is enabled by the option is also enabled if
> MULTIPROCESSOR is true so I would guess that this code path is
> followed on most NetBSD systems in use today.
> 
> The mailing list archives suggest that some ports have had pmap bugs
> in the past that were triggered if the SOSEND_NO_LOAN option wasn't
> enabled.
> 
> It could also be that as the kernel size has grown over time the
> amount of free memory on a PMAX isn't large enough. How have you
> configured gxemul ?
> 
> It would probably also help to know where the file that you read
> using cat was stored, was it read over NFS ?

Local storage, tar x.. file works, cat file | tar x... does not.
Boundary where it stops working is with file over 4K. number may be
dependent on machine, so 'huge file' is suggested.
It was evident when unpacking tarballs manually worked, but sysinst did
not - it pipes to show progress.

This is a known issue with many MIPS CPUs, including some in production.
the CI20 config also has this option, likely because it was broken too.

I'm pretty glad to have this failure in emulation because I can look at
the emulator code.

I was hoping for some help figuring out what the actual problem is, so
it can be fixed, not just hacked away with a kernel option, which does
work, with this option enabled I am able to install normally, run tests,
etc.

my recently completed atf run shows:

Summary for 644 test programs:
    3975 passed test cases.
    201 failed test cases.
    35 expected failed test cases.
    102 skipped test cases.

(could be a lot worse :-))

Some stuff I've been told off-list:
SGI machines do not have this problem at all (tested on SGI O2, and none
of the SGI machines have it enabled in their configs), so not everything
is as broken (phew).

Some stuff I don't yet fully understand but sharing anyway:
SOSEND_LOAN prevents copying of memory contents and 'loans' it instead.
We end up with:
Two virtual addresses for the same physical address in TLB?
Kernel virtual address mapping of same physical address does not match
the rest of the virtual addresses it has?

CPU notices this, throws an exception (maybe T_VCED) which we're hoping
would go away if we just tell it the pages are uncached, but
apparently some MIPS CPUs do not like uncached mappings.

So, we must have a behaviour that the CPU is happy with, maybe take
advantage of the exception received.

Thanks for the responses.