Unix filenames have no "extension" - that's a concept from other
operating systems, and we should not pay that any regard at all.
A dot in a filename is simply a dot (though a leading one is
given some special status by some utilities).
Some utilities give specific meanings to filenames that end with
specific characters (like RCS likes ",v" endings) - but such
conventions belong to the utilities that implement them (having a
section name at the end of a man page is one like that - though it
is truly redundant, we don't really need the "1" twice in man1/cat.1
man1/cat would have worked just fine).
If the objective is to be portable to other systems, then some of those
impose a 6+3 naming rule, with a very limited char set (upper case letters,
digits, and a couple of other chars for some) - the only way to encode
anything reasonable for those would be to hash the original filename into
a 24 bit value, then use the name that that 24 bit value expands into).
And hope for no collisions (but 24 bits is 16 million, so you'd need 4 thousand
names in the same directory before the probability of a collision gets
above 50%).
If you're just going to aim for some other systems, then you'd have
to justify why those, and not others.
If the objective is to make reasonable, easy to manipulate (if perhaps
ugly to look at while encoded) names for unix systems, then there's a
totally different mindset when looking for an encoding method, and you
can easily find something where 99% of all real life man pages encode
into themselves (encoding changes nothing) which is, for this purpose,
ideal.
What you shouldn't be attempting to do is solve all of the issues,
generate something that is a panacea. That way just results in madness.