Re: apache parked processes using CPU

[Jan Schaumann <jschauma%netmeister.org@localhost>]

Jan Schaumann <jschauma%netmeister.org@localhost> wrote:
 
>   PID USERNAME PRI NICE   SIZE   RES STATE      TIME WCPU    CPU COMMAND
> 21048 nobody    42    0   316M   14M parked     5:28 7.32%  7.32% httpd
> 17677 nobody    42    0   318M   14M parked     5:25 6.69%  6.69% httpd
> 16398 nobody    41    0   319M   17M parked    18:53 5.03%  5.03% httpd
>   829 nobody    41    0   320M   18M parked    18:59 4.83%  4.83% httpd
> 21512 nobody    42    0   322M   19M parked    18:49 4.74%  4.74% httpd
> 23308 nobody    42    0   322M   19M parked    18:51 1.61%  1.61% httpd
>     0 root     125    0     0K   20M vdrain    11:44 0.00%  0.00% [system]
> 
> This is new behavior that began without my having made
> any changes to the system on which httpd has been
> running for years without such problems.

I still can't make neither heads nor tails out of
this.  I've chased a few dead ends, such as
considering a filesystem problem on the disk that the
content is served from: I had recently done some
very heavy I/O on that disk with millions of files in
very large directories, but there were no signs of
filesystem corruption and serving content from a
different disk didn't solve this.

(I followed that trail only because it was literally
the only thing that I could think of that had changed
from when the problem started to occur.)

Right now I'm looking at vmstat output, and noting an
increase in page faults that seems to correlate with
the httpd server becoming increasingly unresponsive.
Restarting httpd brings that down, but I have no
explanation as to why they started to happen so
recently.

Given that this is a virtual server, could it be a
problem in the actual hardware or the virtualization
layer, i.e., outside of what I can see on the system?
I kinda feel like blaming some hardware... :-)

-Jan