Re: xen and networking problems

[brook%biology.nmsu.edu@localhost (Brook Milligan)]

Manuel Bouyer writes:
 > How much RAM does you machine have ? If this systeme supports it it would 
 > also
 > be usefull to test amd64.

I have tested the machine with 4GB, 8GB and 16GB RAM.  The problems
all occur whenever there is more than 4GB of RAM.  Networking works in
all the various xen/kernel combinations with only 4GB.  

The CPU is an Intel Xeon that apparently lacks EM64T support so I
assume that amd64 kernels are out.

 > It looks like this driver doesn't deal well with 64bit physical addresses.
 > Would the attached patch help ?

No, this seems to make no difference.  See below for notes on the
patch, which would not compile as originally given.

Clearly, the combination of PAE support and >4GB memory triggers the
problems.  Does this mean that xen is not really supporting PAE or
that the hardware is not really correct or that there is some specific
interaction with the network interface or ... ?  How can any of this
be tested?  In all cases everything about the machine seems to work,
except for the network interface when PAE support is involved and
there is >4GB RAM.

Concrete guidance for troubleshooting is really needed as this is way
beyond my understanding of the internals of the kernel.  I am,
however, willing to test out suggestions and try new code on the
machine.

Thanks alot for your help.

Cheers,
Brook

===========================================================================

For future reference, the following portion of the patch will not
compile because of the context in which the macro is used.

      > Index: i82557var.h
      > ===================================================================
      > RCS file: /cvsroot/src/sys/dev/ic/i82557var.h,v
      > retrieving revision 1.47
      > diff -u -p -u -r1.47 i82557var.h
      > --- i82557var.h 12 May 2009 14:25:17 -0000      1.47
      > +++ i82557var.h 5 Nov 2009 15:56:42 -0000
      > @@ -179,7 +179,7 @@ struct fxp_softc {
      >          * fxp_rxdesc DMA map on a per-mbuf basis.
      >          */
      >         bus_dmamap_t sc_dmamap;
      > -#define        sc_cddma        sc_dmamap->dm_segs[0].ds_addr
      > +#define        sc_cddma        (uint32_t)sc_dmamap->dm_segs[0].ds_addr
      >  
      >         /*
      >          * Software state for transmit descriptors.

Instead, the cast must be done when the macro is used as follows:

     --- i82557var.h.orig       2009-11-06 12:12:48.000000000 -0700
     +++ i82557var.h    2009-11-06 13:10:22.000000000 -0700
     @@ -255,9 +255,9 @@
      #define   FXP_RXMAP_GET(sc)       ((sc)->sc_rxmaps[(sc)->sc_rxfree++])
      #define   FXP_RXMAP_PUT(sc, map)  (sc)->sc_rxmaps[--(sc)->sc_rxfree] = 
(map)

     -#define   FXP_CDTXADDR(sc, x)     ((sc)->sc_cddma + FXP_CDTXOFF((x)))
     -#define   FXP_CDTBDADDR(sc, x)    ((sc)->sc_cddma + FXP_CDTBDOFF((x)))
     -#define   FXP_CDTXPADADDR(sc)     ((sc)->sc_cddma + FXP_CDTXPADOFF)
     +#define   FXP_CDTXADDR(sc, x)     ((uint32_t)(sc)->sc_cddma + 
FXP_CDTXOFF((x)))
     +#define   FXP_CDTBDADDR(sc, x)    ((uint32_t)(sc)->sc_cddma + 
FXP_CDTBDOFF((x)))
     +#define   FXP_CDTXPADADDR(sc)     ((uint32_t)(sc)->sc_cddma + 
FXP_CDTXPADOFF)

      #define   FXP_CDTX(sc, x)         
(&(sc)->sc_control_data->fcd_txdescs[(x)])