Re: high cpu load with tcpdump

[Patrick Welche <prlw1%cam.ac.uk@localhost>]

On Mon, Feb 29, 2016 at 09:11:44PM +0100, 6bone%6bone.informatik.uni-leipzig.de@localhost wrote:
> On Mon, 29 Feb 2016, Christos Zoulas wrote:
> 
> >| Hello,
> >|
> >| the problem occurs only on one of my servers. I tried to find the
> >| difference. It is the bind9 (bind-9.10.3pl3). If I stop the bind9, tcpdump
> >| works without problems. When I restart the bind9, the CPU load goes back
> >| to 100%.
> >|
> >| Is it a problem of the kernel, tcpdump or bind9?
> >
> >Can you ktrace the bind? Perhaps it is waking up tcpdump spuriously.
> >That would indicate a kernel problem.

There seems to be a trend

>   2362      3 named    1456773618.152029244 RET   recvmsg -1 errno 35 Resource temporarily unavailable
>   2362     11 named    1456773618.152035390 GIO   fd 5 read 8 bytes ",\^B\0\0\M-{\M^?\M^?\M^?"
>   2362     11 named    1456773618.152041396 RET   read 8
>   2362      6 named    1456773618.152040698 CALL  recvmsg(0x260,0x7f7ff15fb370,0)
>   2362      9 named    1456773618.152033854 CALL  recvmsg(0x262,0x7f7ff09f8370,0)
>   2362     11 named    1456773618.152050266 CALL  __kevent50(8,0x7f7ff01f6f20,1,0,0,0)
>   2362      9 named    1456773618.152061650 MISC  msghdr: [name=0x7f7fef50f328, namelen=128, iov=0x7f7ff09f83a0, iovlen=1, control=0x7f7ff3d98fa0, controllen=96, flags=4000000]
>   2362      6 named    1456773618.152051663 MISC  msghdr: [name=0x7f7fef517088, namelen=128, iov=0x7f7ff15fb3a0, iovlen=1, control=0x7f7fec6ca1e0, controllen=96, flags=4000000]
>   2362      3 named    1456773618.152041047 CALL  write(7,0x7f7ff21fe3e0,8)
>   2362     11 named    1456773618.152075688 RET   __kevent50 -1 errno 2 No such file or directory
>   2362      9 named    1456773618.152081694 RET   recvmsg -1 errno 35 Resource temporarily unavailable
>   2362      6 named    1456773618.152085116 RET   recvmsg -1 errno 35 Resource temporarily unavailable

recvmsg() returns loads of errno 35 Resource temporarily unavailabel aka EAGAIN.

In the Citrix client thread

  http://mail-index.netbsd.org/netbsd-users/2016/02/03/msg017788.html

recvmsg() returns loads of EAGAIN, then a timer in linux_select1 terminates
=> EINTR/SEGV


Today we saw an instance of dansguardian core dump with a SEGV and a
backtrace says it was in recv(). This is happening repeatedly
(NetBSD7/amd64 stable), but unpredicably - all is well for days, then
a flurry of coredumps (restarted by watchdog). So not reproducible at
will, or at least no obvious conditions to set it off.

Really a coincidence that 3 different breakage scenarios all seem to involve
recv() returning EAGAIN?

Puzzled...

Cheers,

Patrick