On Wed, Jun 02, 2004 at 07:37:33PM -0600, Greg Oster wrote:
> Frank Kardel writes:
> > Hi *,
> > 
> > which config knobs would I need to tweak to avoid a
> > raidXX:  failed to allocate emergency buffer!
> > message. This message comes up when i configure about
> > the 34th raid device (i have pseudodevice raid 48 in the kernelconfig).
> 
> Um... "wow".  
Well, *grin*, what i have is 6 IDE disks with 200/250 GB. As amanda 2.4.2
currently cannot split a backup file between media i limit the file system
size to about 16-20GB. The backup is done on DAT4 tapes. Given the
'small' file system size and the big disks we are looking at 
30-40 raid 1 devices to accommodate all the file systems.
If amanda would be able to split backup file between media i could make
bigger file systems. But currently the small file system size is no limitation
for me.
> 
> > As this seems to be a malloc() failure I would suspect knobs in that
> > area.
> > BTW. That failure is no handles to gracefully. Processes accessing
> > that failed device hang fortunatly interuptable. But that does not seem
> > to be a sane state (will PR that).
> 
> "Emergency buffers" are tied to each individual array.  For the 
> typical case of 3-4 RAID arrays per system, that works fine.  For 34 
> RAID sets, it breaks down horribly with a whole whack of 
> pre-allocated space that probably should be shared.. 
> (10*34*64K --> lots of kernel VM used up!)
Yepp.
> 
> In particular, you've probably eaten up *all* available kernel 
> memory, and so anything else that needs kernel memory is going to be 
> starved too...
Very likely. 8-(
> 
> If you need 34+ RAID devices, then you'll need to bump up the 
> total memory allocated to kernel stuffs...  I'll need to think a bit 
> as to how to "fix" this problem -- it'd be easy if these buffers 
> really could be shared among all RAID sets, but it's quite nasty to 
> keep a shared pool between all RAID sets when the buffer sizes can be 
> quite different.. :( 
Maybe one could share a buffer set per size class if conditions are
suitable for that strategy.
> 
> Later...
> 
Thanks for pondering about that.
> Greg Oster
> 
Frank Kardel