Re: PATCH: Relax fdatasync checks to IEEE Std 1003.1-2008

[Paul Ripke <stix%stix.id.au@localhost>]

On Fri, Mar 13, 2020 at 08:26:30AM -0400, Greg Troxel wrote:
> Paul Ripke <stix%stix.id.au@localhost> writes:
> 
> >> Running atf on a GCP VM. Never run atf before, we'll see how it goes.
> >
> > Failed test cases:
> >     dev/fss/t_fss:basic, include/t_paths:paths, lib/libarchive/t_libarchive:libarchive,
> >     lib/libc/sys/t_ptrace_sigchld:traceme_raise1, lib/libc/sys/t_ptrace_wait:resume,
> >     lib/libc/sys/t_ptrace_wait3:resume, lib/libc/sys/t_ptrace_wait4:resume, lib/libc/sys/t_ptrace_wait6:resume,
> >     lib/libc/sys/t_ptrace_waitid:syscall_signal_on_sce, lib/libnvmm/t_io_assist:io_assist,
> >     lib/libnvmm/t_mem_assist:mem_assist, lib/librumphijack/t_tcpip:http, lib/librumphijack/t_tcpip:nfs,
> >     net/arp/t_arp:arp_rtm, net/if_pppoe/t_pppoe:pppoe6_chap, net/if_pppoe/t_pppoe:pppoe6_pap,
> >     net/if_pppoe/t_pppoe:pppoe_chap, net/if_pppoe/t_pppoe:pppoe_pap, net/mpls/t_mpls_fw6:mplsfw6,
> >     net/mpls/t_mpls_fw64:mplsfw64_expl, net/ndp/t_ndp:ndp_rtm, rump/rumpkern/t_vm:uvmwait,
> >     crypto/opencrypto/t_opencrypto:ioctl
> >
> > Summary for 854 test programs:
> >     8159 passed test cases.
> >     23 failed test cases.
> >     48 expected failed test cases.
> >     1134 skipped test cases.
> >
> > I don't have a baseline - so I don't know if anything has changed.
> > Glancing thru the above, nothing looks related, but I could run again
> > against a pristine kernel?
> 
> That would help, and there is
> 
> http://releng.netbsd.org/b5reports/amd64/
> 
> but the problem is that there is a constant stream of commits that cause
> new failures.  (We don't require proving that each commit doesn't cause
> atf failures, but I personally like to be careful.)

From a pristine kernel from the same sync:

Failed test cases:
    dev/fss/t_fss:basic, include/t_paths:paths, lib/libarchive/t_libarchive:libarchive,
    lib/libc/sys/t_ptrace_sigchld:traceme_raise1, lib/libc/sys/t_ptrace_waitid:resume,
    lib/libc/sys/t_ptrace_waitid:syscall_signal_on_sce, lib/libnvmm/t_io_assist:io_assist,
    lib/libnvmm/t_mem_assist:mem_assist, lib/librumphijack/t_tcpip:http, lib/librumphijack/t_tcpip:nfs,
    net/arp/t_arp:arp_rtm, net/if_pppoe/t_pppoe:pppoe6_chap, net/if_pppoe/t_pppoe:pppoe6_pap,
    net/if_pppoe/t_pppoe:pppoe_chap, net/ndp/t_ndp:ndp_rtm, rump/rumpkern/t_vm:uvmwait,
    crypto/opencrypto/t_opencrypto:ioctl

Summary for 854 test programs:
    8165 passed test cases.
    17 failed test cases.
    48 expected failed test cases.
    1134 skipped test cases.

Apart from duplicates, I think mplsfw64_expl is the only addition, which
doesn't sound relevant.

> There is also the issue that actually testing fsync* sort of requires
> controlled crashes, which is doable but expensive.
> 
> We do have fsync tests but no fdatasync tests.  I wonder how much a
> naive copy and s/fsync/fdatasync/ test file would make sense.
> See src/sys/lib/libc/sys/t_fsync.c

(that's src/tests/lib/libc/sys/t_fsync.c). Quick browse says s/fsync/fdatasync/g
would probably work fine, but it's not testing that much, just expected
error returns, no controlled crash tests.

> (I am not requesting that you do add atf tests as a precondition for
> committing, although I would like to hear that unifi/mongodb3 (without
> the patch) works.)

Building... I see -current has breaking changes, from kqueue(2):
The udata type was changed from intptr_t to void * in NetBSD 10.0.
Causes compile errors in databases/mongodb3:
src/third_party/asio-asio-1-11-0/asio/include/asio/detail/impl/kqueue_reactor.ipp.

Ok, build complete. I can confirm that unpatched mongodb3 still aborts
without the kernel patch, and that it at least starts and keeps
running with the kernel patch. I didn't have an easy way to create load,
but merely starting was enough to abort previously.

-- 
Paul Ripke
"Great minds discuss ideas, average minds discuss events, small minds
 discuss people."
-- Disputed: Often attributed to Eleanor Roosevelt. 1948.