Maybe group kmem read, but that might require more elevated
privileges in the programs that uses ksyms.
What program uses ksyms now that doesn't require at least group kmem?
You cannot give up kmem read privileges when calling ksyms read
routines.
I don't see why not - or, at least, I don't see the ksyms change as
being relevant. Just read /dev/ksyms at startup (at the same time as
you open /dev/kmem, probably), before dropping group kmem. Isn't that
all this change (making /dev/ksyms 440 root:kmem) requires?