Re: aio co-op with socket kqueue on Netbsd

[Justin Cormack <justin%specialbusservice.com@localhost>]

On Tue, Sep 2, 2014 at 2:19 PM, LeiMing <lmforreg%gmail.com@localhost> wrote:
> Hello, I'm just a Windows programmer and now learning *nix APIs. I want to
> know how people use aio and socket kqueue together.

Generally on Unix, people do not use file aio, unlike on Windows. The
interfaces are new, poorly standardised, only partially async anyway.

> Recently I'm learning I/O APIs and noticed that aio_* functions seeem to be
> the only way to perform asynchronism I/O on disk files. The interface is
> just like overlapped I/O on Windows. But the only usable way I found, to
> wait for the operation's completion or to notice me that it finished, which
> is usable on NetBSD, is the API aio_suspend.

Yes that is one of the older, less useful APIs. Traditionally these
APIs were usually implemented in userspace using threads, with no
kernel support at all, hence the lack of kernel completion
notification semantics in the specification.

> I searched the Internet and found 4 ways to implement this: realtime signal,
> kqueue, callback function and aio_suspend/aio_waitcomplete. Unluckily,
> according to this (
> http://ftp.netbsd.org/pub/NetBSD/NetBSD-current/src/doc/TODO.kqueue )
> document I know that currently NetBSD's kqueue doesn't support aio events.
> Sigevent manual ( http://netbsd.gw.com/cgi-bin/man-cgi?sigevent ) shows that
> neither realtime signal nor SIGEV_THREAD flag (by which callback
> notification implement) is supported on NetBSD.

The only one that can provide a sane performant interface is kqueue.
Your best bet would be to implement this on NetBSD if you really want
it.

> The existing practise to handle sockets in a network server prefers I/O
> completion port on Windows, kqueue on *BSD and epoll on Linux. I/O
> completion port handles file read/write operation as well, while kqueue on
> FreeBSD seems (I have not yet tried) to work with aio operations. Through
> fdevent/fdsignal API Linux handles signals via epoll. By this way there
> won't be situation that a process or thread is "waiting for a single
> operation" (synchronism I/O). So how could I use kqueue (to handle sockets)
> and aio (to handle regular files on disk) on NetBSD to achieve the same goal?

Well most of the event libraries use threads on Unix (eg libuv AFAIK).

> By the way, since the realtime signal feature is not usable on NetBSD and
> timer_create sends notice by realtime signal, does it mean that timer_create
> will not work on NetBSD? So generally how a time-out feature is implement in
> a network server on NetBSD?

Using signals is not going to perform very well, I would not do that...

You have to understand that Unix aio has no real history, largely
because IO goes through the page cache and there is no async semantics
for a page fault. This is why when aio has been grafted on it tends to
go via non cached IO, which is not terribly performant. It also tends
to block in metadata operations and so on (eg no Unix supports an
async fsync as far as I know). Windows always had a different
semantics, hence the mismatch you are seeing.

If you want to work on the kernel and make aio kqueue work, by all
means do. If you want to run async applciations, use an existing
library, or ignore file operations and just run a few more threads is
a good strategy for many applications - the best strategy depends on
the application.

Justin