Re: libelf

[David Holland <dholland-tech%netbsd.org@localhost>]

On Fri, Jan 15, 2010 at 12:42:36PM -0500, der Mouse wrote:
 > >> E.g. consider libcurses, which re-exports libc's FILE * type.
 > > In the particular example you cite, all of the problems can be
 > > eliminated by simply making FILE * a truly opaque type and hiding all
 > > of the internals behind real accessors / mutators, rather than the
 > > pseudo-opacity we have now (where the guts are actually exposed in
 > > header files and manipulated by macros).
 > 
 > No, that wouldn't fix the problem; it would just expose it differently
 > (in an arguably less often problematic way, but still).  It would
 > permit redesigns of FILE's internals only when they still preserve the
 > under-the-hood accessors used by stdio.
 > 
 > To really fix it, FILE would have to become truly opaque, without
 > accessors/mutators except for stdio-exported interfaces.  This would
 > impair performance, of course....

That wouldn't fix it either, unless you never tried to make a change
to FILE and provide backwards ABI compatibility. To see why in gory
detail, let's construct a simpler example with no distractions like
getc().

Let's say we have two libraries, libfoo and libbar, and an application
baz.

libfoo defines a type "struct thing", which is fully opaque, i.e.,
only pointers to it appear in the ABI. It provides three functions:

   struct thing *thing_create(void);
   void thing_frob(struct thing *, int code);
   void thing_destroy(struct thing *);

Within libfoo, struct thing is defined as follows:

   struct thing {
        int state;
   };

libbar is a thing manipulation library, which provides one function:

   void thing_superfrob(struct thing *, const char *how);

This parses the provided string and calls thing_frob() accordingly.

The application baz contains this code:

   struct thing *t;
   t = thing_create();
   thing_superfrob(t, "fthagn");
   thing_destroy(t);

We compile, link, and run the lot and it works.

Now suppose the maintainers of libfoo extend the functionality so each
thing needs to hold more state. In order to do this in a compatible
way, they create the following internal types:

   struct compat10_thing {
        int state;
   };

   struct thing {
        long state;
   };

and build libfoo so it provides these entry points:

   struct thing *__thing_create_11(void);
   void __thing_frob_11(struct thing *, int code);
   void __thing_destroy_11(struct thing *);

   struct compat10_thing *thing_create(void);
   void thing_frob(struct __compat10_thing *, int code);
   void thing_destroy(struct __compat10_thing *);

arranging things the way we do so that new compilations use the new
_11 versions and old binaries will continue to bind to the original
names.

We compile and install the new libfoo, and our application still
works, because everything (both in libbar and in the baz image itself)
goes to the compat entry points.

Now we recompile baz, without recompiling libbar. This causes the
calls in baz to go to the new entry points and use the new thing type.
What happens when we run the resulting executable?

   1. It calls __thing_create_11 and gets a new-style thing back.
   2. It passes this to thing_superfrob in libbaz, which has not been
      recompiled and expected an old-style thing pointer. No problem
      though (yet) because it's an opaque type and just a pointer.
   3. libbaz calls thing_frob(), but because libbaz hasn't been
      recompiled this goes to the compat entry point in libfoo that
      expects an old-style thing pointer.
   4. libfoo uses the new-style thing as if it were an old-style
      thing, and if the user is lucky it crashes.

Recompiling just libbar and not the application baz fails in the
converse way.

 > > Again, this is something that could, in theory, be addressed within
 > > the framework of ELF.
 > 
 > Sure.  But, heck, ELF itself could in theory be addressed within the
 > a.out framework.  It's not about what's possible in theory, but what's
 > convenient in practice.

Right.

 > >> Apparently so. There's no "format: flowed" in the headers here, or
 > >> whatever the precise notation is, and there should be.
 > > Weird.  I wonder if the @netbsd.org mailing lists are stripping it
 > > out... or I wonder if my Mail app isn't putting it there in the first
 > > place.  I'm using a different version of Mail to send this one, so
 > > let's see if it's any different.
 > 
 > I got "Content-Type: text/plain; charset=us-ascii" and paragraph-length
 > lines, for what that's worth.  (Reflowed manually in the quotes above.)

Same here.

-- 
David A. Holland
dholland%netbsd.org@localhost