Re: the bouyer-quota2 branch

[David Holland <dholland-tech%netbsd.org@localhost>]

On Wed, Mar 09, 2011 at 08:20:00PM +0100, Manuel Bouyer wrote:
 > On Wed, Mar 09, 2011 at 06:28:11PM +0000, David Holland wrote:
 > >  > struct quota2_entry (and so struct quota2_val) is used for both
 > >  > on-disk storage, and in-memory representation in tools and kernel.
 > >  > I agree this should be split; with an extra level of conversion
 > >  > (between in-memory and on-disk representation). The issue is struct
 > >  > quota2_val: I don't see any reason to have a different structure
 > >  > for on-disk and in-memory represenation at this level.
 > > 
 > > Well... for one thing N_QL isn't necessarily 2;
 > 
 > The tools rely on N_QL being defined and constant for all FS types at
 > this time. The string for each QL is also defined here

That should be fixed in favor of something more flexible before the
API gets cast in stone; as I was saying there's at least one piece of
prior art out there with three types of quotas. It isn't clear to me
that we'll ever care, but on the other hand the cost of not compiling
in the quota types isn't very high.

Especially since if the interface is really going to be proplib-based
they can just be arbitrary names.

 > > also the in-memory
 > > representation shouldn't have disk addresses in it (e.g. q2e_next),
 > > and in the syscall interface the types should be logical types, like
 > > uid_t, not sized types.
 > 
 > Sure (although in the syscall interface, all of this are strings now :)

that doesn't exactly make any difference...

 > struct quota2_entry isn't the real problem (it's not used much outside of
 > filesystem), struct quota2_val is.

Sure.

 > > Also the structures in quota2.h are too
 > > hierarchical; a single entry should be an id type (user or group,
 > > maybe others), an id number, a quota type (block, inode, or other
 > > things; SGI xfs has three types), the hard and soft limits and
 > > configured grace period, and the current usage and current expiration
 > > time.
 > 
 > The hierarchy in quota2.h reflects the proplib structure. A proplib
 > quota entry has a type, and an associated array of entries. Each
 > entry has an id (uid or gid depending on the type above) and array
 > of values for this id.  Each value has a type, current usage,
 > limits and grace times.

Well, yes, one of the problems with proplib is that it encourages
hierarchical structuring of things that should be relations.

ISTM that a bundle of quota records should be a single array of
tuples of the form I described above... at least in the canonical
format for communicating among system components.

 > > (Maybe the configured limits and current state should be
 > > separate structures though
 > 
 > I'm not sure.

I tend to think so because it allows separating the policy (which
might be independent of a given ID) from the operational statistics
(which can't be).

 > > And maybe the id information should be
 > > structured to allow ranges.)
 > 
 > I'm not sure we should allow too much at this level. The tools can
 > allow range if we want, but they should convert it to a list of discrete
 > entries. don't do too much in the kernel.

If I have 80,000 accounts of which 40,000 are undergrads with the same
quota policy, it certainly makes sense to pass this to the kernel
(and, where possible, store it) as one record rather than replicating
it 40,000 times.

I agree we shouldn't go off half-cocked and if we're going to set up a
policy language for quotas it should be designed with some care.
However, with the default stuff we're already moving in that direction
so it seems like a logical step.

 > > But at a minimum it's like struct dirent; it's not particularly
 > > different from the FFS on-disk structure, but it needs to be its own
 > > thing because it plays a different role.
 > 
 > I agree. What I don't get is how to split it to avoid too much extra
 > code which would just be a non-optimised memcpy.

We're talking about like a dozen lines of code, far less than the
proplist decoding that has to be replicated in every FS.

 > > Currently it looks like any fs that wants to implement quotas has to
 > > cut and paste quota2_prop.c. Surely the proplib gunk can be decoded
 > > fs-independently?
 > 
 > Parts of quota2_prop.c can, I guess. For the part in ufs_quota.c,
 > I'm not sure.

If quota2_val and friends can really be an fs-independent interface,
then I don't see that there's any value to passing the proplib bundle
to the FS. If they can't... then the data structures should be
strengthened.

Since we are not going to replicate the userland quota tools for every
different FS type that has quotas, the interface *they* talk has to be
FS-independent.

Even if there's some reason it needs to be proplib-based it's still a
proplib encoding of some physical data structure, and (especially in
the absence of any kind of proplib schemas) it would be helpful to
have that structure clearly defined somewhere.

-- 
David A. Holland
dholland%netbsd.org@localhost