Re: netbsd-6.1: squid from pkgsrc-2013-Q2 uses too much CPU time

[Brett Lymn <blymn%internode.on.net@localhost>]

(sorry if this messes up the threading I just joined the list to respone
to this thread)

Firstly, I have been using squid on NetBSD for many years and it works
fine for me.  I am running squid 3.3.8 on a recent-ish NetBSD-current
without issue.  I wouldn't totally rule out a problem with squid and
NetBSD but there are many many things that can impact the performance of
squid which need to be looked at first.

As a general note, if you are going to post your squid.conf to a mailing
list, please do everyone a favour and do something like:

grep -v '^\(#\|[         ]*$\)' /usr/pkg/etc/squid/squid.conf

and post that output - this will strip the comments and blank lines
leaving a more readable summary of what you actually have in the
configuration file (BTW there is a tab and a space between the square
brackets above).

One thing that can help a lot in working out what is going on with squid
is to look in the cache.log file, sometimes slowness and cpu utilisation
can be caused by squid restarting which can happen if its child
processes (redirector, authenticators, external acl and so on) are
exiting, if these processes exit too often/quickly then squid will
restart.  Similarly, if too many requests waiting to be serviced by
the helpers then squid will restart.  You should see indications of
these restarts and the reasons why in the cache.log.

Another thing it may be is squid running out of file descriptors, again,
this will be mentioned in the cache.log, try adding:

ulimit -n 3000

near the top of /etc/rc.d/squid and restarting squid.  This probably
should be added to the sample start up script as squid has recently
stopped managing the limit itself and now expects a suitable file handle
limit.

Also, it may be unrealistic expectations - one common problem is that
people configure a large disk cache on a memory limited machine.  Squid
requires around 10 - 14MB or memory per GB of cache you specify simply
for managing the cache contents.. it will use more on top of that.  So,
for example, trying to configure squid with a 100GB cache will result in
squid needing more than 1 - 1.4GB of RAM.  If you try that on a machine
with only 2GB of memory then performance will be poor because the
machine will be paging to try and keep up with squid and everything else
as well.

HTH - if not, please look in the cache.log and post any errors you find,
they may provide some clues... also a stripped squid.conf can help.
 
-- 
Brett Lymn
Staple Guns: because duct tape doesn't make that KerCHUNK sound - xkcd.com