Re: CVS commit: src/sys/arch/amd64/conf

[David Holland <dholland-sourcechanges%netbsd.org@localhost>]

On Mon, Feb 21, 2011 at 12:20:14AM +0100, Jean-Yves Migeon wrote:
 > On 20.02.2011 22:58, David Holland wrote:
 > > 1. Traditionally, whether a driver/fs/option/whatever is listed in
 > > GENERIC is an indicator of how stable it's believed to be: entities
 > > that are missing are assumed not to work at all, entities that are
 > > commented out are assumed not to be stable, and only entities that are
 > > included and enabled by default are assumed to be production-ready.
 > 
 > How would you specify a module option considered production-ready (hint:
 > zfs)?

Traditionally, you don't; but, see the next section.

 > > This is a longstanding convention and it's wired into the built-in
 > > operating assumptions of many experienced sysadmins. There is now no
 > > way to tell which of the commented out entities are stable but meant
 > > to be used as modules and which of them are unstable/experimental or
 > > otherwise not ready yet. Having the stable drivers and other entities
 > > be present and uncommented in MONOLITHIC is not a good solution but
 > > it's the best currently available. (To solve this problem properly the
 > > config file syntax needs to be extended.)
 > 
 > Hmm -- adding a comment telling that the feature is experimental?

Right now some of the things commented out in amd64 GENERIC are
labeled experimental. Some of them are labeled "built as a module".
Most of them aren't labeled at all. I think relying on that is not
really a good idea, especially given that there often isn't a whole
lot of extra space to use.

 > What would it bring to extend config(5) to express such a feature?

Adding the syntax

   module options FOO
   module file-system PROCFS

etc. would cost almost nothing and would make it clear exactly what's
going on. It would also make it possible to build modules along with
the kernel instead of as part of userland, which I think is desirable
for a number of reasons.

 > > 2. While not removing FFS and ELF support from GENERIC is a start, few
 > > of the other robustness concerns that have been brought up over the
 > > last two years have been addressed properly.
 > > 
 > >    - End users who are trying to diagnose a driver problem still
 > >      will not be able to test -current by downloading and booting a
 > >      kernel. They will need to also download and install a module
 > >      set. This vastly increases the overhead and complexity of trying
 > >      out -current and puts it past the reach of many newbies. This is
 > >      a serious problem.
 > 
 > This is more a matter of giving appropriate sets, or
 > installation/release media, rather than being modular/monolithic.

To some extent. It should be possible for someone using a stable
release to download and test-boot a nightly -current, without knowing
much about the system and without having to install anything that's
difficult or complicated to remove again.

A monolithic kernel satisfies this requirement; so do other things,
such as tarball kernels that include both a boot image and a module
collection. However, the existing scheme we have doesn't serve.

 > Newbies do fine with Bubuntu/Linuxeries, and modules do live "happily"
 > there.

Modules are also bound much more tightly to kernels in Linux. If you
go install a new kernel package in any major Linux distribution you
get an associated batch of modules with it; this happens transparently
to the user.

 > >    - There is still no way to avoid building modules you don't want,
 > >      and while we have a start on a method to prevent autoloading
 > >      them, it's not ready for prime time yet. Therefore, the only safe
 > >      way to disable any functionality that's available as a module is
 > >      to turn off "options MODULAR". This requires either expert
 > >      attention or a maintained MONOLITHIC config.
 > 
 > Well, same goes for MONOLITHIC: turning off an option requires editing
 > config(5) file + kernel recompile. This requires "expert"
 > intervention... fetching src, building toolchain then kernel. Hum.

Building a kernel might qualify as "advanced" but it's not "expert" in
the sense I meant: formulating your own MONOLITHIC that works requires
knowing quite a bit more about the system than just commenting out
some lines in the kernel config does. This is one of the reasons the
i386 MONOLITHIC was added.

 > How is that less/more expert than playing rm(1) in /stand? That will
 > seriously block autoloading.

That does indeed block autoloading, but it's not an adequate solution
because (1) it causes the nightly scripts to start whining (or at
least I'd expect it to, I haven't tried specifically), and (2) it gets
undone silently if you rebuild the world.

 > >    - The versioning of installed modules is still restricted to one
 > >      set per sys/param.h kernel version, and there's no way to tie a
 > >      particular set of modules and a particular kernel together. This
 > >      means that anyone working on broad kernel changes that affect
 > >      module ABIs (e.g. me, but I'm hardly the only one) must in
 > >      practice stick to monolithic configurations -- if any
 > >      adjustment I make affects a module ABI I need to do a private
 > >      sys/param.h bump and do a nearly full rebuild, instead of
 > >      recompiling half a dozen files actually affected by the change.
 > >      This is a non-starter; furthermore, in such an environment if I
 > >      screw up with the versions I can easily render my test machine
 > >      unbootable.
 > 
 > 100% agreed. Regular problem with Xen kernels.

Glad at least some things can be agreed on :-)

 > ----------------------------------------------------------------
 > [...]
 > Right; which one? i386. Curiously, I never heard of complains regarding
 > the amd64 one, which is MODULAR. Now why is that?
 >
 > MONOLITHIC was added in a rush to shut up complaints, because people
 > couldn't make a difference between these types:
 > 1 - MONOLITHIC kernel, where options are compiled in (_provided_ you
 > enabled them), with module support disabled
 > 2 - MODULAR kernel, with most common options(7) compiled as "builtin"
 > modules (essentially making it like 1, except "options MODULAR")
 > 3 - MODULAR kernel, with most options(7) compiled as third party modules.

Well right, except that case 2 isn't really a modular kernel. The
existence of "options MODULAR" is not the issue that matters (mostly,
modulo the autoload issues) -- it's case 3 that causes serious
complications. 

Even without a fix for the autoloading issues, I don't think anyone
really objects much to having "options MODULAR" turned on. Anyone
who's setting up their own kernel config to disable things and doesn't
want the autoloader providing them anyway can turn off "options
MODULAR" at the same time they turn off whatever else they're
concerned about.

(Knowing that they need to do this isn't desirable and the autoloading
issues should be resolved, but it's not critical at the same level.)

 > Please stop opposing MONOLITHIC and MODULAR. Given the current
 > architecture, modules can be turned into builtins, which is, in essence,
 > the same thing as a monolithic kernel, when you turn on most options.

I do not understand what you mean by this.

The premise of your commit was that GENERIC should be modular, that
is, you took out a large number of standard features so they'd be only
available via moduldes. Any MONOLITHIC config should have those
features compiled in.

There shouldn't be any difference between a compiled-in feature and
the same feature present as a "builtin module".

 > I also stated (also privately with Matthew) that "MONOLITHIC" was a bad
 > name. "NOMODULAR" would be more appropriate. If we enable most options
 > in GENERIC, MONOLITHIC would just be a "include <...>/GENERIC\n no
 > options MODULAR", with almost zero difference regarding code (well,
 > except that you won't have module support anymore).

No, that's not what MONOLITHIC is meant to be. It should turn on the
options that, in GENERIC, are turned off so as to be modules.

 > So, one more time: the current issue is to define what people consider
 > as options that should be enabled by default, and what could stay out
 > (or as a third party module if you urgently need it back). One example:
 > accf_* is a questionable choice, whether its inside GENERIC (as a
 > builtin), or enabled by default in MONOLITHIC.

I don't think that's the issue. Running GENERIC is supposed to make
all stable features and functionality available. The question at hand
is whether those features are compiled in (which would be
"monolithic") or loaded as modules (which would be "modular").

-- 
David A. Holland
dholland%netbsd.org@localhost