Subject: Configuring memory parameters for large network-based services
To: None <tech-kern@netbsd.org>
From: Michael Graff <explorer@flame.org>
List: tech-kern
Date: 03/22/1999 01:39:31
I have a machine running an IRC server.  Currently, it seems that a
small program that generates 700 "clones" can kill a machine.  FreeBSD
crashes hard, Linux just falls apart randomly after about 100 are
loaded, and NetBSD manages to load about 300 before the machine locks
with many mb_map full messages on 1.3.* systems, and processes stuck
waiting for memory from the pool (I think) on -current.

Here's the data:

BEFORE:

Memory resource pool statistics
 Name   Size Requests Releases Npage Idle InUse PageSpace FreePageSpace
 mbpl    128  1716842  1716652    16    9   23K       65K           36K
mclpl   2048   530752   530646   168  115  212K      672K          460K

WHILE 200 CLONES ARE CHATTING:

 Name   Size Requests Releases Npage Idle InUse PageSpace FreePageSpace
 mbpl    128  1755892  1752570   129    4  415K      516K           16K
mclpl   2048   544466   543178   726   58 2576K     2904K          232K

AFTER:

 Name   Size Requests Releases Npage Idle InUse PageSpace FreePageSpace
 mbpl    128  1765106  1764962   131  106  144K      524K          424K
mclpl   2048   546962   546856   727  661  212K     2908K         2644K

What these test clients do is open a connection and send a string.
That string is replicated to up to 200 others.  Each client sends one
string, so there are up to 200*200 strings in the TCP output queues.

My questions are:

(1)  What controls how much memory the pool system can use?

(2)  How do I make that space larger?

(3)  Will the pages marked as idle ever be reclaimed by the pool
     system, or released back into the system via kernel free()?

(4)  What knobs should I fiddle with to make this machine into a large
     network-based server, cable of handling a few thousand clients?

To be fair, these clones are much, much harder on a system than your
average IRC user would ever be, but it bothers me that a few hundred
clones (which are easy to create, of course...) can freeze my machine.

--Michael