Subject: Re: direct copy() between two proc address spaces
To: None <tech-kern@netbsd.org>
From: Ian Zagorskih <ianzag@megasignal.com>
List: tech-kern
Date: 06/20/2004 15:53:09
=F7 =D3=CF=CF=C2=DD=C5=CE=C9=C9 =CF=D4 20 =E9=C0=CE=D8 2004 06:30 Bill Stud=
enmund =CE=C1=D0=C9=D3=C1=CC(a):

> On Sat, Jun 19, 2004 at 06:46:43PM +0700, Ian Zagorskih wrote:
>
> Why do you want to do this? Usually this is handled by using either SysV
> shared memory, or having both processes mmap the same file. Then you have
> userland directly copy fron one place to the other.
>

In short: i'm trying to implement Send/Receive/Reply (SRR) IPC mechanism us=
ed=20
in QNX4 under NetBSD.

SRR is a very simple [and IMHO powerfull] IPC well described at=20
http://qdn.qnx.com/developers/docs/qnx_4.25_docs/qnx4/sysarch/microkernel.h=
tml#IPCMESSAGES

Reasons: ATM i have large project originally designed for QNX4 RTOS which=20
extensively uses SRR. Project itself includes about 40 separate application=
s=20
which work as a solid system through cooperative communication with each=20
other using SRR. Speaking in tech-kern@ it's needless to say that i need to=
=20
port this system on NetBSD :)

In general, QNX4 SRR IPC can be implemented completely inside userland usin=
g=20
existing IPC mechanisms i.e. on top of shm/sem, sockets, maybe message queu=
e=20
etc. On the other hand, some tests i made ATM leads to the conclusion that=
=20
implementing this IPC scheme inside the kernel is much more simplier. Note=
=20
that client and server processes aren't required to be in any child/parent=
=20
relations with each other. It means that i cannot relay on passing unnamed=
=20
shared resources between them like file descriptors, shared memory or=20
semaphores.

Technically, this isn't too hard to implement SRR under NetBSD and i'v wrot=
e=20
test LKM that does this. ATM it is designed as a character device with thre=
e=20
ioctl() calls:

=2D--sys/srr.h---
/* Send message */
struct srr_send {
	pid_t pid;			/* IN: dest PID */
	size_t snbytes;	/* IN: send msg length */
	const void *sdata;	/* IN: send msg data */
	size_t rnbytes;		/* IN: reply msg length */
	void *rdata;		/* IN: reply msg data */
};

/* Receive message */
struct srr_receive {
	pid_t pid;			/* IN/OUT: dest PID */
	size_t snbytes;	/* IN: recv msg length */
	void *sdata;		/* IN: recv msg data */
};

/* Reply message */
struct srr_reply {
	pid_t pid;			/* IN: dest PID */
	size_t rnbytes;		/* IN: reply msg length */
	const void *rdata;	/* IN: reply msg data */
};

/* ioctl() requests */
#define SRRSEND		_IOW('S', 1, struct srr_send)
#define SRRRECEIVE	_IOWR('S', 2, struct srr_receive)
#define SRRREPLY		_IOW('S', 3, struct srr_reply)

int Send(pid_t, const void *, void *, size_t, size_t);
pid_t Receive(pid_t, void *, size_t);
int Reply(pid_t, const void *, size_t);
=2D--sys/srr.h---

So generic client/server proto looks like below:

=2D--cut---
struct smsg smsg;
struct rmsg rmsg;

/* server */
while (1) {
    pid =3D Receive(0, &smsg, sizeof(smsg));
    /* do some work */
    Reply(pid, &rmsg, sizeof(rmsg));
}

/* client */
pid =3D find_server_pid_some_way();
while (1) {
    /* fill send msg */
    Send(pid, &smsg, &rmsg, sizeof(smsg), sizeof(rmsg));
    /* annalize reply msg and do some work */
}
=2D--cut---

At some point inside the kernel i have the following tasks:

1. Server issued Receive() and becomes ready to receive message. Client iss=
ued=20
Send() and becomes ready to send message. Here i have two proc structures,=
=20
proc *cnt + proc *svc, and i have two ioctl() argument structures - struct=
=20
srr_send *snd for client + struct srr_receive *rcv for server. Now i need t=
o=20
pass data between two user spaces snd->sdata and rcv->sdata with length=20
MIN(snd->snbytes, rcv->snbytes). Note that i do not care about the content =
of=20
this data inside kernel and it's is value has some sense only for client an=
d=20
server i.e. there's no need to access it anyway inside the kernel.

2. Server issued Reply() and client is pending in Send() call, waiting for=
=20
reply message. Again, i have client  + server proc structures and ioctl()=20
arguments struct srr_send *snd + struct srr_reply *rep. Now i need to pass=
=20
data between rep->rdata and snd->rdata between two user address spaces=20
without reasons to access it.

Well, probably my explanation isn't too clear but i tried hard :)

// wbr