rumphijack, or how to use an unmodified firefox with rump tcp/ip

[Antti Kantee <pooka%cs.hut.fi@localhost>]

Hi,

I've been working on a system call "hijacking" library on and off
for the past 1.5 weeks.  Support is at a stage where TCP/IP works
and I do my normal web surfing through a rump tcp/ip server (plus
I run a third tcp/ip stack for testing).

In contrast to heavyweight virtualization (usermode OS etc.), the
only setup required is configuring the TCP/IP stack, no rootfs &
full installation & long waits are necessary.  Server "reboot"
takes about 0.01s, so there's hardly a loss of service for
some applications with good restart capability such as web
browsing (especially since the browser itself does not die).

rumphijack uses the very simple and old idea of LD_PRELOADing a
library between a dynamically linked application application and
libc, so that libc stubs are resolved to the hijack library.
The library then decides if a call should go to the host kernel
or rump kernel (or both!).  The granularity is rough and there
are no guarantees, but most things work (at least for TCP/IP
applications).

So, it's work in progress, but it works.  A howto mirroring the
state of -current src as of a minute ago follows.  I'll assume you
want to bridge the tcp/ip server with the outside world.  Routing is
possible too, but slightly different in details, and left as an
exercise to the reader.

    First, start a TCP/IP server listening for syscalls
    on e.g. /tmp/foo:

golem> rump_server -lrumpnet -lrumpnet_net -lrumpnet_netinet \
       -lrumpnet_virtif unix:///tmp/foo
golem> setenv RUMP_SERVER unix:///tmp/foo

    Note that anybody with host access to /tmp/foo will have "root"
    access to the virtual tcp/ip stack and conversely anybody
    without access to /tmp/foo will have no access to that tcp/ip
    stack.

    Then, assuming you want to be able to reach places other than
    127.0.0.1 (*), setup a host tap interface.
    (* note! that's 127.0.0.1 of the rump tcp/ip server, not
    127.0.0.1 of your host)

golem> ifconfig tap0 create
golem> ifconfig tap0 up
golem> ifconfig bridge0 create
golem> brconfig bridge0 add tap0 add yourif0
golem> brconfig bridge0 up

    Now, configure an interface within the rump kernel and set the
    default gateway:

golem> rump.ifconfig virt0 create
golem> rump.ifconfig virt0 inet foo netmask bar
golem> rump.route add default baz

    See virt(4) for details on how to attach to the right tap<n>.
    You can use rump.ping and rump.traceroute to test you've
    correctly configured everything.

    At this point you're pretty much good to go.  Just set LD_PRELOAD
    to the highjack library:

golem> setenv LD_PRELOAD /usr/lib/librumphijack.so

    and start your applications, e.g.:

golem> firefox -P anotherprofile -no-remote

    when you're done with the tcp/ip server, halt it with:

golem> rump.halt

Some other applications I've roughly tested:
socket, netcat (but you need nc110nb3, prior are statically linked),
bozohttpd, links, links-gui, w3m, ftp and wget.

caveats:
  + adobe-flash-plugin does not (yet?) work with rumphijack.
  + the shell with the LD_PRELOAD set should not be used for
    general purposes.  notably, it's not possible to execute
    anything [dynamically linked] if the server is dead.  i'll
    fix it some day.

If you plan on doing the above often, I recommend scripts, changing the
shell prompt to track the tcp/ip server you're talking to, and/or shell
aliases (which are left as an exercise).

Things like attaching gdb to rump_server and putting a breakpoint
on e.g. tcp_input is of course possible, provided you've compiled
sys/rump with DBG=-g.

Note: I'm running 5.1'ish, so I've not tested at all on a -current
host.  If you notice it doesn't work perfectly, please let me know.

I'll hopefully do a more thorough post at a later date.  It will
include details on e.g. how the whole thing works.

  - antti

-- 
älä karot toivorikkauttas, kyl rätei ja lumpui piisaa