I'm far from a wizard at this area, but I've had pretty good luck 
with the following recipe for trimming the hardware section:

Boot up GENERIC or something similar that finds all your hardware.  
Put the output from dmesg in a window where you can easily search 
it. 

Edit your CONFIG file.  Whenever you come to something you don't 
recognize and think you can pitch, scan the dmesg output to see if/where 
it is used.  If it wasn't used you can get rid of it.