In addition to Hubert's points (thanks!):

On Wed, Mar 10, 1999 at 10:45:12PM +1100, Julian Assange wrote:
> 
> I don't like to discourage, but I have some problems with this.
> 
> (1) are we trying to produce somethign that is human readable or
>     computer readable? There seems to be a conflict of goals here.

  Both.  I thought my proposed framework was flexible enough to
allow automated grubbing, as well as simple human parsing.  It
may not be _optimal_ for either, but I think it would be _pretty
good_ for both.

> (2) The file can't be versioned (at least not atomically, I suppose
>     you could version open()s for it) meaning you have to add 
>     another variable name each time you change a variable.

  You've lost me here.  What kind of versioning do you need, if
the first line says what format version the remaining data will
be?  If you really wanted to be evil, you could even have the
kernel change the format and version on-the-fly, and
under my proposal applications would pick up on it, if they
were written safely.  Is there something I'm not seeing?  :) 
They can ignore variable names that didn't exist when they were
compiled, and use the ones that they know about, even if the
order changes.

> (3) scanf and open etc are slow. look how slow top / ps is under linux,
>     further there is no random access to the information. you have
>     to read it all even if you only want a part.

  Once you have the file open, it takes a single syscall to grab
the data from then until when the program exits --
pread(fd, buff, nbytes, 0); sleep (1); pread(fd, buff2, nbytes, 0);
No more opens.  No seeks.

  It may be that the sscanf stuff will be slow, but I have some
ideas for handling that (I presume atoi and atof are
sufficiently fast, right?).  I'd like to get this nice, verbose
format out there before I spend too much time worrying about
tweaking performance to the utmost.  The way I figure it, xosview
currently needs to do on the order of 20 + #net-ifaces + #disks
kvm ops for each update (and it sort of defaults to 10 updates
per second).  At 10 ifaces (maybe only two of which are
active), and 2 disks, that's 32 syscalls per update.  

  I would posit that going from 32 syscalls to 1 syscall plus a
bunch of sscanfs (NOT scanf or fscanf -- that'd be SLOW, and
would not provide the atomicity guarantees that are needed!!!)
would be a big performance win.  Later on, I'll worry about even
removing the sscanfs.

> (4) it's nice to have a file you can apply shell tools quickly to,
>     but you can use sysctl -a|grep works just as well.

  But sysctl doesn't tell me everything.  How much net traffic is
there?  How many bytes have been transferred via disks?  How many
interrupts have occurred, and which ones were they?  sysctl is
great for some of this, but it doesn't provide the full
functionality.  Maybe it should provide such data, but I thought
my proposal was less radical than ``cluttering'' sysctl with
performance stats.

> (5) you have to create new ascii standards for representation of
>     non-trivial data types. e.g arrays, linked lists, trees,
>     flags etc, code to generate AND parse it.

  My immediate goal is, provide something that provides
system-wide statistics for tools like xosview and ps and top and
systat.  These tools, for the most part, don't need to walk the
net ifaces or disk structures -- they only care about _totals_,
but they are forced to walk these structures to get the totals.
Not _all_ statistics would be provided via the new file --
kvm_getprocs, for example, would not be provided in /kern/summary.  
(I believe it _could_ be provided, either here or in /proc, but
that's a different issue for a different day.) 

> (6) access to variables is split between the sysctl and and the
>     /kern framework. doubling the amount of work involved if you
>     want a variable in both.

  Yes, there are situations where this is true.  What I'm trying
to avoid, though, is having the code _more_ than duplicated --
some programs have embedded in them how to walk kernel
structures, just to grab a total.  Whenever that changes, sysctl
and _all_ of those programs will need to be changed.  At least
now, all the code that needs changing would be in the kernel, and
in those non-/kern-based kvm utilities.

> I think you would be far better off transmuting kernfs into an
> accurate representation of the sysctl namespace. you could even
> have a ".all" file at each level in the hierarchy which would
> represent all information at nodes under it. i.e ".all" in the
> root would be equivalent to sysctl -a. this would take advantage
> of the large existing sysctl namespace, and build upon it,
> making both sysctl and kernfs stronger.

  This is definitely a good idea, and I may try to do
this in addition to anything else I do with /kern.  It's
orthogonal to my main goal, though, which is to provide a concise
summary of the system statistics, achievable with a minimum of
cost (one syscall), and thus minimum impact on the system --
the whole observability thing and all that.

  I hope that answers some of your concerns!

  Brian