Re: fs-independent quotas

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Thu, Oct 20, 2011 at 04:39:21PM +0000, David Holland wrote:
>  > We're talking a few MB of ram here, isn't it ? the kernel can certainly
>  > allocate this without troubles (other subsystems do).
> 
> The proplib'd and XMLified complete dump for 50,000 users will
> probably make a blob of between 10 and 20 MB. (Note: this is an
> estimate; I haven't checked the size by trying it. It might be larger.
> I'd be surprised if it were much smaller.)

I tested with a few 10s or users; my estimate is about 35MB for 50k users.

> 
> I don't see why it's desirable to manifest such large objects when
> it's easily avoidable.

We don't agree on "easily". 

> 
>  > > There are two design truisms for database stuff that apply here:
>  > > first, you always end up wanting cursors, and second, you always end
>  > > up wanting bulk get (and not just single get) from those cursors. So
>  > > it's usually a good idea to anticipate this and design it all in up
>  > > front.
>  > 
>  > Maybe ... I know that in the end I want the whole set of data and not
>  > just a part of it.
> 
> Yes, probably. The cursor API I've floated so far is not general
> enough to support much else. Although it could be made more general.
> 
>  > But if you believe it's needed this can easily be added to the
>  > existing quotactl(2) (it would just be a new command).
> 
> Yes, perhaps it could... but why? What's to be gained by using a
> baroque proplib encoding of what can otherwise be handled as an array
> of simple structs?

it's an easily machine-parsable text. That's probably the reason why it's
used in other parts of the kernel too.

> 
> I remember asking this question when you first proposed the proplib
> interface last spring, and never really got a clear answer.

I see it as being the common format used for non-performance-critical
kernel/userland communication. It has been adopted by other kernel
subsystems, there's prior art there.

> 
>  > >  > > The reason to wrap the position in a cursor abstraction is to allow
>  > >  > > flexibility about how the position is represented.
>  > >  > 
>  > >  > But then the cursor would still be stored in userland ?
>  > > 
>  > > That's the idea, like reading a file with pread().
>  > > 
>  > > I think the kernel should know, or at least be able to know, how many
>  > > cursors are currently open; but I don't think there's any need to keep
>  > > the cursor state itself in the kernel.
>  > 
>  > So you want a quotaopen/quotaclose, with a file descriptor (or something
>  > similar) ?
> 
> The proposed API already has explicit open and close for cursors; what
> I'm saying is that this should be exposed to the kernel. (Open already
> has to be, to initialize the cursor position; close should be, so the
> filesystem can if necessary know if there are cursors open at any
> given time. Otherwise you can get into trouble; see for example nfsd
> and readdir.)

So you're close to have something like a file descriptor.

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--