Re: Real Asynchronous I/O

[David Holland <dholland-tech%netbsd.org@localhost>]

On Tue, Feb 05, 2019 at 11:09:03AM -0600, prajwal cvn wrote:
 > I am Prajwal, a final year masters student currently working towards my
 > master's thesis on I/O optimizations. I am very interested in learning and
 > contributing to Real Asynchronous I/O project. This project is in line with
 > my research work and can provide me valuable insights.

Good to hear; but keep in mind that it's overall a large project
involving a good deal of the kernel's I/O plumbing. It might be a good
idea to carve off a piece of it as a starting point, both to avoid
getting overwhelmed and to give you something that won't take forever
to finish.

What aspects are you interested in for your research? Feel free to
email me off-list if you like, but keep in mind that there's only one
of me and quite a few people who read this list and may have
suggestions.

 > I would be grateful if I am pointed towards the cases where the existing
 > code doesn't work properly as mentioned in project description.

Some years ago I was writing trace replay code for doing caching
simulations; I initially wrote it to use aio and it didn't work.
Unfortunately since I had a hard deadline there wasn't time to
investigate in detail so I ripped that logic out and replaced it with
a thread pool. More recently an officemate of mine ran into the same
symptoms while using an off-the-shelf I/O benchmarking tool that used
aio (I can find out which one) but it took a while for the role of aio
in the problem to become clear.

The symptoms are that some requests don't finish promptly -- don't
know if it's because they finish but the completion signaling drops
them or because they don't get scheduled -- at least in some cases
they do finish but several seconds later, in a pattern that looks
suspiciously like they've gotten stuck and then got bumped loose again
by a timer interrupt or timeout somewhere.

Other people may have more to report but I think that's the overall
pattern and I don't think any more is known.

At least one person went over the current thread pool code with a
fine-tooth comb looking for the problem without finding anything, so
whatever's going on is probably subtle or happens in some unexpected
way.

-- 
David A. Holland
dholland%netbsd.org@localhost