Re: Xen modules

[John Nemeth <jnemeth%cue.bc.ca@localhost>]

On Nov 8,  2:40am, Jean-Yves Migeon wrote:
} Le 07/11/2013 03:32, John Nemeth a écrit :
} > On Nov 6,  2:30pm, Jean-Yves Migeon wrote:
} > } Le 31/10/2013 23:43, John Nemeth a écrit :
} > } >      Thus, this is NOT a module bug at all, but rather loading
} > } > modules tickles a latent bug in our Xen code.  For now, I ignore
} > } > the error instead of panic()ing.  It would be better not to make
} > } > the call at all, but I don't (yet) know how to test for privileges.
} > } >
} > } >      Having made this change, I can now successfully load simple
} > } > modules.  I need to go on and test more complex modules.
} > }
} > } First and foremost, thank you for working on this. Module usage inside
} >
} >       Your welcome.
} >
} > } Xen is a big plus :) Sorry for seeing the mail that late though :/
} >
} >       That's okay.
} >
} > } Regarding the hypercall privilege, I think that wbinvd gets caught and
} > } handled by hypervisor; what happens if you replace the hypercall with
} > }
} > } asm("wbinvd":::"memory")
} >
} >       Okay, I tried this.  The result is no crashes or other obvious
} > signs of problems.  Also, no complaints in "xl dmesg", whereas the
} > hypercall caused a warning to be logged.  Should I commit the
} > change?
} 
} Manuel being AFK, can I see the diff beforehand, please?

     There's basically two choices.  wbinvd() calls xpq_flush_cache().
It's the only caller.  So, the first choice is to change wbinvd()
to do the WBINVD and delete xpq_flush_cache().  The second option
is to modify xpq_flush_cache() to do the WBINVD instead of making
the hypercall, which is the method I tested since that's the smallest
change.  Right now, xpq_flush_cache() looks like:

void
xpq_flush_cache(void)
{
        struct mmuext_op op;
        int s = splvm(), err;

        xpq_flush_queue();

        XENPRINTK2(("xpq_queue_flush_cache\n"));
        op.cmd = MMUEXT_FLUSH_CACHE;
        if ((err = HYPERVISOR_mmuext_op(&op, 1, NULL, DOMID_SELF)) < 0) {
                panic("xpq_flush_cache, err %d", err);
        }
        splx(s); /* XXX: removeme */
}

The obvious change is to make it look like:

void
xpq_flush_cache(void)
{
        int s = splvm();

        xpq_flush_queue();

        XENPRINTK2(("xpq_queue_flush_cache\n"));
        asm("wbinvd":::"memory");
        splx(s); /* XXX: removeme */
}

} >       On a slightly different, but related note, do you (or anybody
} > else for that matter) know much about how symbols are handled in
} > Xen kernels?  I can successfully load and run the example module,
} > but the only functions it calls are printf and proplib stuff.
} > Trying to load the cd9660 module produces several linking errors:
} 
} Xen kernel are not built and linked the same way as native ones (unless 
} you modified it to test kernel modules loading).

     The only modifications are to fix the issue above, set the
module loading path, and adding "options MODULAR" to the kernel
config file.  In other words, no significant changes.

} It's been a while since I looked at the Makefiles (under 
} arch/xen/conf/), however Xen kernels are not built with OPT_MODULAR 
} turned on IIRC.
} 
} ldscripts should not differ much though (between i386/amd64 and xen).
} 
} Sorry for not being more specific :/
} 
} > # modload cd9660
} > kobj_checksyms, 881: [cd9660]: linker error: symbol `pool_destroy' not found
} > kobj_checksyms, 881: [cd9660]: linker error: symbol `brelse' not found
} > kobj_checksyms, 881: [cd9660]: linker error: symbol `uio_setup_sysspace' 
not found
} > kobj_checksyms, 881: [cd9660]: linker error: symbol `mutex_destroy' not 
found
} > WARNING: module error: unable to affix module `cd9660', error 8
} 
} Difference in nm output between the two kernels for those symbols?

     Other then the address, they look the same:

P4-3679GHz: {194} nm netbsd-XEN3PAE_DOMU | grep pool_destroy
c038d960 T pool_destroy
P4-3679GHz: {195} nm netbsd-GENERIC | grep pool_destroy
c0814d07 T pool_destroy

Same for the other symbols.

}-- End of excerpt from Jean-Yves Migeon