Steve,

Check out the ogrady power page.  http://ogrady.com/  It has a hack on
puting 8.5 on an upgraded 540c.  As I understand it, it only works
because the 68040 is gone completely(on a daughter card).  Since the
68040 is still in your box the OS 8.5 detects it and pukes.

GTK

By the by,

Do you have a 1.3.2 kernel that works on a quadra (built-in)in 832x624
w/o wrapping the video? I've got a later current kernel that works but
of course I get a proc size mismatch & can't us ps or top among others. 
You used to put up quadra friendly space case kernels and I used them
successfully for quite a while before the vidio got fixed in 1.3.?.

Tia

Jerry