Bad threading performance

[Sad Clouds <cryintothebluesky%googlemail.com@localhost>]

Hi, I've been working on a concurrent threading framework. It is a
userlevel library, which consists of a thread pool and it heavily uses
mutex locking and condition variable signaling.

I'm testing this on hyper-threaded Intel Atom D510 CPU, which shows up
as 4 CPUs. The test runs a tight loop, in which it assigns 4 parallel
tasks to threads, and waits for those tasks to complete.

For some reason I'm getting really bad performance results with NetBSD
(5.1-STABLE amd64). I suspect something is not quite right in pthreads
library, or at the kernel level.

Below are test results. Any ideas why concurrency on NetBSD is so bad
compared to Solaris? It seems as if on NetBSD threads are stuck on a
wait list for much longer.

Solaris running times and vmstat output for CPU user, system and idle:

real 28.7
user 1:38.4
sys 4.0

us sy id
--------
1  0 99
42 3 55
86 6 8
86 6 8
86 6 8
86 6 8
86 6 8
87 6 7
86 6 8
86 6 8
87 6 7
86 6 8
87 6 7
87 6 8
86 6 8
87 6 8
86 6 7
86 6 8
86 6 8
86 6 8
86 6 8
86 6 8
86 6 8
86 6 8
87 6 7
87 6 7
86 6 8
86 6 8
86 6 8


NetBSD running times and vmstat output for CPU user, system and idle:

57.61 real
142.19 user
1.86 sys

us sy id
--------
1  0 99
2  0 98
0  0 100
30 0 69
95 1 4
61 0 38
49 0 50
66 1 33
92 1 7
95 1 4
67 0 33
85 0 14
54 1 45
70 1 29
50 1 49
50 1 49
53 0 46
50 0 50
47 1 52
45 1 54
37 2 61
34 1 65
55 1 44
74 1 26
70 2 28
80 1 19
95 1 5
79 1 21
52 0 48
50 0 50
47 1 52
50 0 50
47 0 52
51 1 48
53 0 47
49 1 50
48 0 51
47 0 53
52 0 48
50 0 50
49 1 50
53 1 46
52 0 47
53 1 46
52 1 48
49 1 50
43 0 56
33 1 66
34 0 66
32 1 67
54 1 45
95 0 4
92 1 7
95 1 4
87 1 12
82 1 17
56 1 43
93 2 5
94 2 4
87 1 12
60 1 39