Subject: ideas for loadable drivers and hot-plug - PCI and everything
To: None <tech-kern@netbsd.org>
From: Matthias Drochner <M.Drochner@fz-juelich.de>
List: tech-kern
Date: 08/03/2004 23:29:35
This is a multipart MIME message.
--==_Exmh_6417860905620
Content-Type: text/plain; charset=us-ascii
This continues the ideas brought in in the "lkm for pci device" thread
some weeks ago.
(Actually, the basic ideas are much older. I've beem playing with this
since NetBSD-1.3.)
While my implementation examples might look mature - there might be
wrong assumtions from the beginning, so consider them proof-of concept
only for now.
So what needs to be done, or can be done: We want to attach/detach devices
dynamically at runtime. There are some cases coming to mind:
a)The driver is already present and the hardware gets added/removed. This
works well already. Subcases are whether the hardware provides
some notification on hot-plug events (as USB and PCMCIA) or needs
to be manually notified (SCSI). Polling would be in-between.
b)Some hardware is present but not supported by the boot kernel. There
are some ad-hoc solutions to add drivers (actually there is only the
"pcilkm" I'm aware of), but this works around the autoconf framework
and should be replaced by something well-designed.
c)Hardware gets added, the driver is not yet present but will be loaded
"on demand". This needs some serious effort to get it "right", in
particular because the driver matching now done in the kernel (in the
individual drivers) needs to be duplicated in userland.
d)A newer driver for some already supported hardware gets available,
I want to replace the existing one. Needs proper detaching, otherwise
it is like the second case.
What's dealt here with is case (b) primarily. Case (d) too as far as
the bus drivers keep track of their childs. There are some ideas for
(c), but this needs much more work.
You know, to have a driver support some hardware it needs 3 pieces:
-a hardware specific driver backend (cfdriver)
-a bus specific driver frontend (cfattach)
-a directive in the config file connecting backend, frontend and possible
parents, using "locators" which are an abstraction of bus addresses
(somehow limited, but anyway...) called "cfdata"
A loadable kernel module might just contain a new cfdata for an existing
driver and frontend, a cfdata+cfattach for an existing driver, or all
of them.
How modules can be built which fit exactly into an existing kernel, ie
don't duplicate existing symbols, is a complex issue where I don't
know a solution for. It might turn out that modules are to be built for each
specific kernel incarnation as Linux does.
More control over exported symbols, in-kernel loader and so might help...
The autoconf framework has already explicite support for adding/removing
cfdriver and cfattach instances. The appended patch adds support for cfdata
entries (or rather tables of these).
What is also missing is some generic way to have the devices on a bus
probed again after "something has changed". What exists are bus specfic
scans after a hot-plug event (USB, PCMCIA) or by a specific user request
(SCSI).
The appended patch only implements a "scan the whole bus" method. It probably
makes sense to specify the "addresses" to scan further (as scsictl does),
but this is not easily done in a bus independant way.
I'd propose to rework the "locator names" thing as done by config(8), but
this is too much for today...
I'll append some patches which are close to something useful for me:
-subr_autoconf.c additions to support attach/detach of cfdata and bus rescans
-pci adaptions - keep track of child devices and support rescan
-a sample PCI driver LKM which just attaches to unowned PCI devices for testing
the framework
-a minimal driver (LKM) to trigger device detachs and rescans
-a minimal userland utility to invoke these driver functions
I'd like to explain more of the details, but I've run out of time a bit:-(
Hopefully, if you read until here and if you looked into the autoconf
stuff you'll understand what I meant anyway...
Since this stuff does imply serious framework decisions I'd like to get
as many opinions as possible.
best regards
Matthias
--==_Exmh_6417860905620
Content-Type: text/plain ; name="autoconf.txt"; charset=us-ascii
Content-Description: autoconf.txt
Content-Disposition: attachment; filename="autoconf.txt"
Index: sys/device.h
===================================================================
RCS file: /cvsroot/src/sys/sys/device.h,v
retrieving revision 1.68
diff -u -r1.68 device.h
--- sys/device.h 30 Apr 2004 23:00:03 -0000 1.68
+++ sys/device.h 3 Aug 2004 18:48:11 -0000
@@ -237,12 +237,22 @@
void (*ca_attach)(struct device *, struct device *, void *);
int (*ca_detach)(struct device *, int);
int (*ca_activate)(struct device *, enum devact);
+ /* technically, the next 2 belong into "struct cfdriver" */
+ int (*ca_rescan)(struct device *); /* scan for new children */
+ void (*ca_childdetached)(struct device *, struct device *);
};
LIST_HEAD(cfattachlist, cfattach);
#define CFATTACH_DECL(name, ddsize, matfn, attfn, detfn, actfn) \
struct cfattach __CONCAT(name,_ca) = { \
- ___STRING(name), { 0 }, ddsize, matfn, attfn, detfn, actfn \
+ ___STRING(name), { 0 }, ddsize, matfn, attfn, detfn, actfn, 0, 0 \
+}
+
+#define CFATTACH_DECL2(name, ddsize, matfn, attfn, detfn, actfn, \
+ rescanfn, chdetfn) \
+struct cfattach __CONCAT(name,_ca) = { \
+ ___STRING(name), { 0 }, ddsize, matfn, attfn, detfn, actfn, \
+ rescanfn, chdetfn \
}
/* Flags given to config_detach(), and the ca_detach function. */
@@ -317,6 +327,9 @@
int config_cfattach_attach(const char *, struct cfattach *);
int config_cfattach_detach(const char *, struct cfattach *);
+int config_cfdata_attach(struct cfdata *, int);
+int config_cfdata_detach(struct cfdata *);
+
struct cfdriver *config_cfdriver_lookup(const char *);
struct cfattach *config_cfattach_lookup(const char *, const char *);
Index: kern/subr_autoconf.c
===================================================================
RCS file: /cvsroot/src/sys/kern/subr_autoconf.c,v
retrieving revision 1.89
diff -u -r1.89 subr_autoconf.c
--- kern/subr_autoconf.c 17 Feb 2004 05:03:16 -0000 1.89
+++ kern/subr_autoconf.c 3 Aug 2004 18:48:12 -0000
@@ -465,7 +465,7 @@
* on `cfp'.
*/
static int
-cfparent_match(struct device *parent, const struct cfparent *cfp)
+cfparent_match(const struct device *parent, const struct cfparent *cfp)
{
struct cfdriver *pcd;
const char * const *cpp;
@@ -518,6 +518,102 @@
}
/*
+ * Helper for config_cfdata_attach(): check whether a device is
+ * a potential parent for any attachment in the config data table,
+ * and supports the "rescan" method.
+ */
+static int
+parent_candidate(const struct device *d, const struct cfdata *cf)
+{
+ const struct cfdata *cf1;
+
+ for (cf1 = cf; cf1->cf_name; cf1++)
+ if (cfparent_match(d, cf1->cf_pspec) &&
+ d->dv_cfattach->ca_rescan)
+ return (1);
+
+ return (0);
+}
+
+/*
+ * Attach a supplemental config data table and rescan potential
+ * parent devices if required.
+ */
+int
+config_cfdata_attach(struct cfdata *cf, int scannow)
+{
+ struct cftable *ct;
+ struct device *d;
+ int found = 0;
+
+ ct = malloc(sizeof(struct cftable), M_DEVBUF, M_WAITOK);
+ ct->ct_cfdata = cf;
+ TAILQ_INSERT_TAIL(&allcftables, ct, ct_list);
+
+ if (!scannow)
+ return (0);
+
+ TAILQ_FOREACH(d, &alldevs, dv_list) {
+ if (parent_candidate(d, cf))
+ found |= (*d->dv_cfattach->ca_rescan)(d);
+ }
+
+ return (found);
+}
+
+/*
+ * Helper for config_cfdata_detach: check whether a device is
+ * found through any attachment in the config data table.
+ */
+static int
+dev_in_cfdata(const struct device *d, const struct cfdata *cf)
+{
+ const struct cfdata *cf1;
+
+ for (cf1 = cf; cf1->cf_name; cf1++)
+ if (d->dv_cfdata == cf1)
+ return (1);
+
+ return (0);
+}
+
+/*
+ * Detach a supplemental config data table. Detach all devices found
+ * through that table (and thus keeping references to it) before.
+ */
+int
+config_cfdata_detach(struct cfdata *cf)
+{
+ struct device *d;
+ int error;
+ struct cftable *ct;
+
+again:
+ TAILQ_FOREACH(d, &alldevs, dv_list) {
+ if (dev_in_cfdata(d, cf)) {
+ error = config_detach(d, 0);
+ if (error) {
+ aprint_error("%s: unable to detach instance\n",
+ d->dv_xname);
+ return (error);
+ }
+ goto again;
+ }
+ }
+
+ TAILQ_FOREACH(ct, &allcftables, ct_list) {
+ if (ct->ct_cfdata == cf) {
+ TAILQ_REMOVE(&allcftables, ct, ct_list);
+ free(ct, M_DEVBUF);
+ return (0);
+ }
+ }
+
+ /* not found -- shouldn't happen */
+ return (EINVAL);
+}
+
+/*
* Invoke the "match" routine for a cfdata entry on behalf of
* an external caller, usually a "submatch" routine.
*/
@@ -994,6 +1090,13 @@
}
#endif
+ /* notify the parent that the child is gone */
+ if (dev->dv_parent) {
+ struct device *p = dev->dv_parent;
+ if (p->dv_cfattach->ca_childdetached)
+ (*p->dv_cfattach->ca_childdetached)(p, dev);
+ }
+
/*
* Mark cfdata to show that the unit can be reused, if possible.
*/
--==_Exmh_6417860905620
Content-Type: text/plain ; name="pci.txt"; charset=us-ascii
Content-Description: pci.txt
Content-Disposition: attachment; filename="pci.txt"
Index: dev/pci/pcivar.h
===================================================================
RCS file: /cvsroot/src/sys/dev/pci/pcivar.h,v
retrieving revision 1.63
diff -u -r1.63 pcivar.h
--- dev/pci/pcivar.h 2 Aug 2004 18:43:38 -0000 1.63
+++ dev/pci/pcivar.h 3 Aug 2004 18:48:50 -0000
@@ -166,6 +166,9 @@
u_int sc_intrswiz;
pcitag_t sc_intrtag;
int sc_flags;
+ /* accounting of child devices */
+ struct device *sc_devices[32*8];
+#define PCI_SC_DEVICESC(d, f) sc_devices[(d) * 8 + (f)]
};
extern struct cfdriver pci_cd;
Index: dev/pci/pci.c
===================================================================
RCS file: /cvsroot/src/sys/dev/pci/pci.c,v
retrieving revision 1.86
diff -u -r1.86 pci.c
--- dev/pci/pci.c 29 Jul 2004 16:51:01 -0000 1.86
+++ dev/pci/pci.c 3 Aug 2004 18:48:50 -0000
@@ -60,9 +60,11 @@
int pcimatch __P((struct device *, struct cfdata *, void *));
void pciattach __P((struct device *, struct device *, void *));
+int pcirescan(struct device *);
+void pcidevdetached(struct device *, struct device *);
-CFATTACH_DECL(pci, sizeof(struct pci_softc),
- pcimatch, pciattach, NULL, NULL);
+CFATTACH_DECL2(pci, sizeof(struct pci_softc),
+ pcimatch, pciattach, NULL, NULL, pcirescan, pcidevdetached);
int pciprint __P((void *, const char *));
int pcisubmatch __P((struct device *, struct cfdata *, void *));
@@ -196,6 +198,14 @@
}
int
+pcirescan(struct device *sc)
+{
+
+ pci_enumerate_bus((struct pci_softc *)sc, NULL, NULL);
+ return (0);
+}
+
+int
pciprint(aux, pnp)
void *aux;
const char *pnp;
@@ -270,6 +280,7 @@
struct pci_attach_args pa;
pcireg_t id, csr, class, intr, bhlcr;
int ret, pin, bus, device, function;
+ struct device *subdev;
pci_decompose_tag(pc, tag, &bus, &device, &function);
@@ -348,13 +359,47 @@
if (ret != 0 && pap != NULL)
*pap = pa;
} else {
- ret = config_found_sm(&sc->sc_dev, &pa, pciprint,
- pcisubmatch) != NULL;
+ /* a driver already attached? */
+ if (sc->PCI_SC_DEVICESC(device, function))
+ return (0);
+
+ subdev = config_found_sm(&sc->sc_dev, &pa, pciprint,
+ pcisubmatch);
+ sc->PCI_SC_DEVICESC(device, function) = subdev;
+ ret = (subdev != NULL);
}
return (ret);
}
+void
+pcidevdetached(struct device *sc, struct device *dev)
+{
+ struct pci_softc *psc = (struct pci_softc *)sc;
+ int i;
+#ifdef DEBUG
+ int found = 0;
+#endif
+
+ for (i = 0; i < sizeof(psc->sc_devices)/sizeof(psc->sc_devices[0]);
+ i++) {
+ if (psc->sc_devices[i] == dev) {
+ psc->sc_devices[i] = 0;
+#ifdef DEBUG
+ found++;
+#else
+ return;
+#endif
+
+ }
+ }
+#ifdef DEBUG
+ if (found != 1)
+ printf("pcidevdetach %s at %s: found %d times\n",
+ dev->dv_xname, sc->dv_xname, found);
+#endif
+}
+
int
pci_get_capability(pc, tag, capid, offset, value)
pci_chipset_tag_t pc;
--==_Exmh_6417860905620
Content-Type: application/x-gzip ; name="drvlkm.tar.gz"
Content-Description: drvlkm.tar.gz
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="drvlkm.tar.gz"
H4sIAAAAAAAAA+0ca3PaSDJfpV8xIZtEsIAlxKPKxNligSTUOsYHONmrXIqSJWHrLCROEk68
Kf/37Z7RYwTIJFmMkzp1FUaaV/f0zPRzsOFd64H96F5Blutyq9GAb7mm1Fv4LSvNOv0O4ZEi
K7LaaqitWhPKFUVVm48a90sWg6UfaB4hjwzP1S8d08tqp2vKPsjZNxh0/Q/gY5jX94TjG9a/
rqjYTqk1lVq+/nuA1PofsDfLrV7uEgcsrdys1zPXX20q0frXlJqK619vNuVH8i6JyIL/8/V/
YpgzyzFJb/SuOznu9d+RwkGyEwqi6AfeUg8IlBlmoOmXmnfhky+igE9Y6mhz84PS/NgWb9t8
a8/0dc1JtT5f+unWHPJef9LpvkH808HwvfS897xMYC+UySb8Rb7nqD/udk5+Pxunetb5ngkt
RfGhGf6DQfr8z6/uQwvg+b9L/qPMD8+/0mpR+a+21Pz87wOePD44t5wD/1IUTf3SJQXdM7XA
ci4IJwaINNf+63rkF7VYEL05qcz4WnF+5bhGqr0OTYmcn7UfH9Ln/612BXLVNneLY5v+r9WT
81+XFar/1Vp+/vcB4h9vh70jwrbB3DXE8ag7jt6revgwta/mVV08Gb7tnBwJT25MXxS7p6ev
jjuvx78eCZVeb9B5fTIcTwZdUukdD7t/9Pq/n70W33dGJ+MjoSZSNNPjYacX91mQX75Uu2ej
3mB0yxSPKFYtR7eXhklenPtG9Qroqc6vXuZy5P5gk/1f1XeLY8v5r6m1Ojv/9SZ4ADV2/tVa
fv73AE/iE+ff+AcLzdPm1cuXYroYPsGG4ivTc0x7vVx3ndl6KewvSzfXy81r0wnWiy0XdyIU
J+WFxD0FxwTGmwY3C3NKW0pRHfwpgmsBNPhB5ALo0Nb/FMmy8O0I/RJnadvuwnTKBJ902/VN
9ghWkMGePnlWAIXc+GXo5/qBu4AGbhDc4NfCtW38ns81Wnz1P9CjgemVQ7cIbCqdWE4gMi8G
SDi/QWdIYoRS/6gUulNFEShLvBfgGlTBKMKkMzj+1/TVcNQHX0kC+p5pNp5bH8i7ntqWHxRx
UoI1I9JjGECfL6RwTGhReQmNPlMERWgkeGaw9ByCFMysiykjDEeVi8BB4RYQRk36J38OhlB4
m5oKc6vAq0tNIvTydjqJcMyVSdBmtB0r1mdagHOovNS1KSOOTjSeRim7nWTgnAXBtH0z1ad/
MgSvlFbeZjJFpCub7BCJbk6cd5ksp7YL5rQ+h6nqmmF4WKEFWhl5SGa2dhG7qguQQaS0oKzD
SqANOeZ/sgL9EhZqbrA565qfdpoPY4fYIJK00WUvFRFpuPA+7P7VjYi8CfdfW3yydGBAYmDz
czgLV20ObexxZ6HlfP81tMmmAYTRXslACCXa0g4O+d3aP+2Mx5M3o+HZ6zdsm8ZVMDRdjoeW
qT8TbNL/zNrbHY5t9n+T0/9NtPtlpaFgHGB3JGRDrv/3pP9xU6E6Nz+DWnTuUs0g89BbALkm
FWLHpFCO9D88nZwdH5dJRQHlkeqJZaj9UXjGPXE7g4nh3UQSCt6ngXZug0aiQpj+CbuJX9UN
HoMFE+BUruPDtemh4BaF3mB82pmAbgtb0RZQW6YjOO5s6ehZz3uXX8n51xaLe8LxLfm/Zl0J
8z+NPP+zB0it/wPFfxS1Gcr/liy35DD+o+Tyfw8gno6Gr4+EMJK7OcJzJFQGXKymWg3NBS68
kw7cgB17kQdufgpIn/+Hif8ochz/VRvN8Pyr+fnfB3A2WmBgZCVltgWGbZ2ny5YOeOhGusz0
vHTBTHeCVcMw8Czn4jviPJGLOTydjLGqEMcgrl3LIEtfuzAlfCy2UzUiq2E22WwB2IMZmHIG
EAum5FOffKkY5Yp3iw7rfxwwKi/MAAUXdYZpBMT8bAWSwjx8NA3nmuVIaOqBZwtWGwt3lODl
OvbYoRhMR5N2pmYuFrqLwHKMNmsxA2OQ+vXCRjfd0LRUFedKe1glCp8uQUGDu62DMw0kw+AS
owcJKVM2FYvk8RGawjRaEIcPuODBc+P5Yfzs0WcBCYcx9fa60//8N9qE98dD7nJxEaSCVI6S
6SJB5NekQKSxGtrsBVEwJhAPAitkEGxpOlJ8FQEmMx313o8wItVmnaHVC3iDrrCMUg0WEruQ
pz6sX9yvGKFiMzrC2YasCDxbX9xIwOVqHBhDMj/IH8vEt/4y3RlfWaRDhZELFtnB9eNDL+iC
aBrlA6LEGASNIwF5KpDHNy3QeAXBEBNJkeetkAcrXY1DXqvkcZXZ5MUhGiDPu5u8uClHHuPv
Z6kOTZbOleN+coiOsU3HKFCcUchFzgMuPzeE+j8wQTJ693QB8JvufzZb9P6HnN//3Auk1/9g
Dsp9utCtKbzszAzcYv8paj3y/5pyq0Xv/zZaaiu3//YA9xH/SzJ9XAWGl2Fj4edao+ZidFmQ
7jnfnQU6WUsZ+foUntorSTTWZa6BWSOtpJjiXIo+w8QDFlAzsQSKijcb6QgsDZQ5RFKwOkRM
Q5g0S3UoxfHE7qvOBHX/tNfvHkvYIdbia3MvlkWBUEgmV+bJLPMYWQg0illmswMW1MQA5wpT
wtHZtLTlZ2YkR0odjmsq10et6TsYtoIkKvZNe7aGI7TDC4fEWM7nN0RHSiqabZPT7iDsCtb4
Wq4xm9uE4YmSaX56LrmBcjdkyf9dpoC2+f+y0qLyv9ZQVFYP8l/J/f99wN7kP58CSjx935ov
bBN2Hq0qoMjsjQbv+iNeZPbeTXs04RPJPE4Kw161Xf3DR7zOQRNCFYXctslBKRQPZYJplcBy
HVI6iDrG0pBJLhwkfKKXQgrwjukm9tV7d3YymEw7J/9GLbQ6ApWn9NSwZ0aJKHwpYCGOEn7L
ZfJqPOlM+lP4Cy4tI5yWP4vxw+st6IEvRCa3VOdhHuztYNyV2DA4+ZX8WXiXIaRBS+7g86yN
qYyeKY3PWB+8ghH2LvNToQQAjwWBFn5tFi1ZnLif7WpGVrfNPZbOlj6J3tt9mo4nO3ljJK0l
69aUVPZ80+Sk9RRbpynt+YxbqWImkkwWfQWasO86oocWRznsGVb0/70kALfe/67Fv/+SG+z3
f/WGqub6fw8Q3v9GQRBe/U6pZJ2kIwIkbSD+kxvh+V3vHwFWzn968XeEY5v932zE8T+5qSjM
/m/k538fsC/7f93SDy9bxVZPltlaMuLsmul5rofpB6PyUjcwA4L3WPEZP1MswFraDDNZ7GIz
WLOeBYZdFL2gjaOEEm2bXDgOux6Rfv/PwXgSXjGOghZsHPLUJxq9IH5DFp7pg/VJ84cxUewm
MX0FKmSaImNZn9RoT/1DsnSozRa4xDMvQLTC6AzJphEvXGjnLgMu57Y2VTbH1FTjm9O69k9m
zYZEW/suBmj0UnWaC9p3cCGaSebA4bVvWEl6XXt1raPb7NFafx33cMPxvJuViZLBM8ZPkUcd
8p5HvcL7Faq3EQ00AsGHifnOaIgT0mlr/lsO0OaZc/hn6WmLwhoNIscGypWsvbiNHxxjM3aF
YW7cF3ThN+2NFVKFZMUT/m8mOWv3fAedyTlmgd1IJMSiaiOt6z8zuN8I5l36f1f/BGLr/39Q
6lH8T2621FD/13P9vwfIEhpJJibayaSkG22uNIw7lXSNL2USR59BGf+jIHbyaR4nvJu9Xe8X
U223CTlo/dC8zCGHHHLIIYefBf4GWDehzQBQAAA=
--==_Exmh_6417860905620--