Re: NetBSD 5.1 TCP performance issue (lots of ACK)

[David Laight <david%l8s.co.uk@localhost>]

On Sat, Oct 29, 2011 at 01:37:40PM -0700, Dennis Ferguson wrote:
> 
> CPUs in general are allowed to reorder reads, but Intel and AMD
> x86 CPUs in particular won't do that.

The rules can also be different for cached v uncached values.

> The linux rmb() expands to
> an empty asm() statement, essentially (not quite) a NOP.

An empty asm statement isn't enough. You need:
    asm volatile ( ::: "memory" )
to tell gcc that the statement might modify any memory location.
However, if the data reference is volatile that isn't necessary.
> There might be another problem, though.  If your code looks like
> 
>     volatile int a, b;
>     <. . .>
>     my_a = a;
>     my_b = b;
> 
> I don't think it is guaranteed that the compiler will generate code
> which reads a before it reads b.  The volatile declarations ensure
> that a and b will be (re-)read, but don't indicate to the compiler that
> reads from different locations need to be done in the order you wrote
> them.

The volatiles do guarantee the order of the instructions.
(They don't guarantee the size of the bus cycles)

...

> I've been writing a lot of SMP data structure code recently, and
> you sometimes find bugs like this when you change compilers.

You also find a lot of code that just doesn't work...
Typical hints are locks that don't tie two actions together, and
places where locks are released over function calls (they can be ok).

        David

-- 
David Laight: david%l8s.co.uk@localhost