Re: usefulness of lint (was: Re: Proposal: remove usr.bin/mkstr)

[David Holland <dholland-tech%netbsd.org@localhost>]

On Sun, Apr 10, 2022 at 01:13:26AM +0200, Roland Illig wrote:
 > > We should not remove lint until we have another program checker to
 > > replace it with, but it is not itself very useful. And yes, I know
 > > you've been working on it and I haven't been following the details,
 > > but it will take far more than the time you've put in (and a ground-up
 > > restructuring) to become even as useful as lclint or sparse, let alone
 > > comparable to a state-of-the-art program checker.
 > 
 > Could you go into the details of what you mean by "ground-up"? The basic
 > concepts and data types (expression, type, symbol, operator) look
 > sensible to me. Sure, the memory management is hard to understand at
 > first, I'm preparing a README file for that.

When I looked it over in detail (some years back now) the internal
representation seemed entirely inadequate for doing more sophisticated
analyses and handling their results. It was far enough back that I
don't remember any details, just that I concluded that it didn't seem
worth spending time on.

But anyway, to do complicated analyses and then track the results you
need an intermediate representation you can iterate over easily and
also one that you can extend to hold additional discovered
information. Looking at lint1.h it still does not seem suitable at
all. For one thing it's too close to the concrete syntax; for example,
it has type specifiers in it instead of just types. For another thing,
declarations don't seem to have statements in them (!?) so it isn't
complete in some fashion and/or is limited to handling one a function
at a time. (While the latter isn't the end of the world, it means you
can't propagate information between functions in the same translation
unit unless they appear in the right order, which is a significant
limitation.)

And it isn't... how shall I say... sturdy. You want a robust framework
you can store a lot of arbitrary information in. If you start stuffing
large amounts of dataflow results, or extra type annotations in more
expressive type systems that you're using to carry extra information
(e.g. allowing functions to be polymorphic in const as I mentioned
before), or tracking ranges or impossible values or other stuff like
that, it will become a mess.

None of this is unreasonable; it was never intended to handle the
kinds of analysis I'm thinking of and wasn't written with the
possibility in mind. But, all the same, it wasn't. Back then (whenever
it was) my conclusion was that hacking it up was probably considerably
more expensive than starting over; now that you've fixed up some of
the more glaring problems that might be less true (that is, the
infrastructure that one might salvage at least works now), but it's
still a huge job.

 > > It used to be that lint did not do this in a useful way (via
 > > quantifier polymorphism) -- have you improved it?
 > 
 > Yes, but not in a general way. I just listed the few affected functions
 > explicitly. See
 > https://nxr.netbsd.org/search?q=%22effectively+discards%22 for the
 > implementation, the test msg_346.c contains a few examples.

Right. I was talking about doing it in a general way :-)

(To do that what you basically want is abstract interpretation over
the domain "is const", "is not const", "can be either", and "don't
know"; and then in support of that you want your types to be able to
express functions where the return constness is covariant in one of
the args's constness. And then to do the analogous thing with
volatile, though since volatile's a lot less common it's less
interesting.)

 > > I tried to check but running lint on a test file spews fatal errors
 > > trying to read the system headers. If one adds -Ac11 and
 > > -D'__attribute__(x)=' to avoid these, it does not complain about your
 > > example, let alone any other more complex ones.
 > 
 > Oh, I can see how lint's default settings result in a bad first-time
 > experience. Ideally, lint should just work out of the box, without even
 > having to read the manual. The minimal lint flags to make it usable are
 > currently -g for GCC mode and -S for C99 mode. That's what I use in the
 > unit tests as well. I'll try to improve the default settings to be more
 > user-friendly.

Yeah, the alphabet soup of options is hard to wade through too.

With -g -S it runs but doesn't complain on your example (or one where
the first arg is explicitly const char *, either) -- I assume that
there's some other flag that needs to be set to enable the warning,
but the only thing I see in the man page is an option for _disabling_
warnings by number. :-|

Might be nice to add an EXAMPLES section in the man page :-|

-- 
David A. Holland
dholland%netbsd.org@localhost