Re: bpf jit

[Mindaugas Rasiukevicius <rmind%netbsd.org@localhost>]

Alexander Nasonov <alnsn%yandex.ru@localhost> wrote:
> Mindaugas recently committed bpf_jit (just-in-time) code generator to
> the NetBSD i386 and amd64 kernels from FreeBSD code.
> The new code is still broken (see below for details) and I don't know
> when Mindaugas plans to fix it.

No, it is not broken if used in a certain way (which is actually the
common case) - see below.

> At the moment, sljit supports i386, amd64, arm, ppc32, ppc64 and mips32.
> There is no sparc support because the author of sljit neither have time
> nor hardware. I don't know a status of mips64.

Well, that covers most of the Tier I ports, which I think is good enough.

> Performace of generated code is on par with hand written C code on both
> arm and amd64.

Do you have any numbers?  With the current BPF JIT, it is more than 2x
performance increase:

http://www.netbsd.org/~rmind/bpf_jit_times.txt

> In my opinion, sljit generator is better than hand-coded generators
> for several reasons:
> 
> 1) bpfjit code is machine independent while bpf_jit has to be written
>    for every architecture. This argument has an opposite side of course.
>    If sljit doesn't support some particular architecture, it's a lot
>    more work to implement all sljit API for the new architecture than to
>    code up bpf generator manually.
> 
> 2) FreeBSD only supports i386 and amd64 at the moment. Some people
>    believe that there is a suport for some other arches but I don't
>    have any references to implementations. The best I could find is
>    a post by Robert Watson [3].
> 
> 3) bpf_jit expects an umlimate knowdedge of instructions encoding and
>    this badly affects both readability and extensibility. For example,
>    there are jumps by 12 bytes in bpf_jit generator. In order to tell
>    what is a destication of a jump (and whether it's valid at all), you
>    need to know a binary encoding format. On the other hand, sljit
>    supports jumps integrity using jump and label objects.

I guess (3) is a valid point, but as for (1) and (2) - I am not really
convinced that it is any easier to add support for sljit.  But perhaps
it is.. see below.

> 4) FreeBSD doesn't support (quite likely for the above reason) reading
>    data from mbuf chain. They instead fallback to interpreter but it
>    leads to suboptimal performance for bigger packets and this makes
>    their implementation look a bit like a toy. Why do they need jit
>    compiler at all if they only implement it for small packets?
> 
>    Current NetBSD code doesn't implement code generation for mbuf chain
>    either but it calls jit code nevertheless, it's a bug I mentioned at
>    the beginning of my post.

It is true that mbuf chains are not supported, but the size of the packet
does not really matter here.  Most of the filtering operations are done
only on layer 3 and/or 4 headers, i.e. it is a common case.  IP and other
layers (AFAIK in a vast majority of the cases) it is ensured that these
headers will be in a single (and the first) mbuf.  So, if one performs
m_copyback() (which would nearly always be a NOP), to guarantee one
contiguous mbuf, it is perfectly safe to use.  It also meets NPF/n-code
needs for now.

I fully agree, though, that adding support for mbuf chains would be great
or rather necessary, if we want to support e.g. L7 filtering (once we will
get there).  In any case, it is the reason why I kept BPF JIT disabled by
default for now (and, in fact, without a sysctl option to enable it yet).

> Sljit has downsides, of course. I mentioned one already in bullet 1) but
> there are some others:
> 
>  - No manual page. Most functionality is documented in sljitLir.h, it
>    should be moved or copied to a manual page.
>  - API is not stable yet.
>  - No proper build script and .so versioning. It's less a problem for
>    the kernel space but I'd like to build sljit in userspace as well 
>    to run unit tests.
> 
> I'd like to conclude my lengthy post with a question. Which of these two
> implementations should be used by NetBSD?

BPF JIT from FreeBSD is something what is here, works and is more tested.
Seems (or seemed?) smaller and simpler to work with as well, since SLJIT
was not ready to be used as an in-kernel JIT at all.  As I understand,
you have addressed this by now.

I guess the problem with SLJIT is that nobody else here is familiar with it
besides yourself, therefore it is difficult to decide on its suitability.
Lack of documentation, as you already pointed out, contributes to this as
well.  There are various other questions unanswered, e.g. would we diverge
SLJIT code in favour of better integration into NetBSD?  Would the changes
be committed to upstream or we would maintain SLJIT ourselves or perhaps
would even take over the maintenance?

Personally, I do not have a strong preference to use the current JIT code,
but not yet enough confidence to support SLJIT either.

-- 
Mindaugas