Subject: Is 'MAXUSERS' necessary?
To: None <tech-kern@netbsd.org>
From: David Laight <david@l8s.co.uk>
List: tech-kern
Date: 06/22/2002 19:27:16
ISTM that kernel configurables (like MAXUSERS) are an excuse for
not writing code that auto-sizes itself...
Now the current netbsd kernel will dynamically allocate almost
all of the structures that were originally fixed size data
structures.
The configured limits, however, still exist - and if you hit one
you (almost) always just want to increase the limit - because the
workload of that system requires a larger value. Only very
rarely do they stop a rampant process eating all the resources,
and if they do you can't rally get in to stop it!
I've just done a trawl through (a slightly old) current, looking
in particular at what MAXUSERS is used for.
item Used in definition of
----------------------------------------
MAXUSERS -> MAXFILES, NPROC
NPROC -> maxproc, ncallout, NTEXT, NVNODE, MAX_KMAPENT
NTEXT -> NVNODE
NVNODE -> desiredvnodes
MAX_KMAPENT -> static kernel_map_entry[ MAX_KMAPENT ]
ncallout -> hash table size for callout table
desiredvnodes -> number of kernel vnodes
maxproc -> hash table sizes for pid, pgid and ui tables
pmap_init for some ports?
Most of the above definition sare loose - like 3 files per
process + some for good measure etc.
The default number of vnodes doesn't seem large enough for modern
systems (because of the way the UBC uses them).
Also did I recall someone looking at dynamically increasing the
amount of kernel virtual address space?
What I've been looking at is the hash table sizes for pid (etc)
lookup. In particular deleting the hash tables completely.
The basic scheme is to have an array of pointers to proc
structures which would be doubled in size when full. This
would make pid lookup efficient regardless of the system size
and without requiring a tunable.
To avoid recycling pid numbers too quickly, the pid would be
masked with the table size before lookup then checked with the
number in the referenced entry.
An equivalent list for pgid (process group) entries is needed
(because you can't allocate a pid that matches an existing pgid.
I'd probably keep the pgid for a ssid allocated until the session
died.
It ought to be possible to overlay a 'free list' over the 'null'
proc_t pointers - thus removing the linear search for an unused
pid in fork().
Since the proc structure will no longer require p_list (nor
pg_hash in struct pgrp) anyone scanning through all the processes
yould have to skip the null pointers. OTOH the table won't be
much larger than the maximum number of processes the system has
had - so this wouldn't be a big problem.
Is this all worth coding? None of it looks too hard!
Should I wait for the nathan_smp branch to be merged first
(or download another CVS tree over my 56k modem?)
David
--
David Laight: david@l8s.co.uk