Re: Please RUMPify tests in sbin/ifconfig

[Rin Okuyama <rokuyama.rk%gmail.com@localhost>]

Hi,

On 2020/07/18 18:09, Jukka Ruohonen wrote:
> I respectfully disagree. Arguments:
>
> 	A. As far as I know, (1) is not an issue.
>
> 	B. The explanation behind (3) is rump itself! Without these tests,
> 	   the whole issue would not have been caught.

As Greg and Roy pointed out, restoring old settings is not enough.
Tests should not modify any host configurations even if temporary.
NFS root pointed out by Martin should be one of the best examples.

> 	B. It is important to have tests for "real hardware" / device
> 	   drivers. It is likely that shmif(4) cannot reproduce many of the
> 	   PRs behind these tests.

OK, but I don't think it good idea to do such tests by default.
It should be an option for developers interested in NIC drivers.

Also, we should avoid crash of test runs as much as possible.
Our periodic tests, at least for evbarm-earmv7hf, crash due to
an ifconfig(8) test almost one month:

https://releng.netbsd.org/b5reports/evbarm-earmv7hf/2020/2020.06.26.10.14.32/test.log

This is very dangerous; this can make bugs invisible that are
much more serious than ones in a randomly chosen NIC driver.

> 	C. With respect to (2): different "name space" issues are common
> 	   across the whole test suite. There are literally hundreds of
> 	   comparable cases.

OK, I don't have strong opinions for this.

Thanks,
rin