Re: In-kernel units for block numbers, etc ...

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Fri, 27 Nov 2015 01:54:04 +0100
    From:        Jan Danielsson <jan.m.danielsson%gmail.com@localhost>
    Message-ID:  <5657A9AC.8030602%gmail.com@localhost>

  | I took a quick stab at making cgd work on a disk with 4K sectors a
  | while back, but $OTHER_STUFF got in the way.

That's largely what has happened with me, so far....  If it isn't fixed
before early next year, I'll start again then.

  | I got beyond any hard crashes; I believe that newfs was failing.

If the underlying cgd is setup correctly, I don't think newfs ought
to be a problem - it works OK on a 4K disk that isn't encrypted, so
living on cgd ought to make no difference.

  |    Heads-up; the panic you're seeing may be a division by zero.
  |    dg->dg_nsectors = 1024 * (1024 / dg->dg_secsize);

No, saw that one, and fixed it (anything related to sector sizes got
fixed just from reading the code).

  |    If you check those old posts you'll see that there were requests to
  | make cgd binary compatible between 512 and 4096 byte sectors.

That's an interesting idea - I certainly did not attempt to do anything
like that, but it could certainly be done.  I was kind of hoping I would
not need to go near the encryption code or logic, and so far at least,
that seems to have been successful...

  | I assume the idea is that
  | you should be able to dd a 512 byte sector drive to a 4KB sector drive
  | and configuring a cgd device on it and mounting the partition  should
  | Just Work(tm).

Unfortunately, that, and any other dd'ing of images between 512 byte and
4K sector size drives may be doomed by design before we even start.  That
is, regardless of whatever the encryption method does to help (or not) in
this process (or even if it is there).

The problem is that there is a bunch of stuff which has been (probably just
from lack of thought) designed to be at fixed sector number addresses, and
fixed sizes in units of sectors.   So, if you have a 4K based CGD for example,
and it contains GPT labelling (which it might, one day) the GPT primary
label is at sector 1, that's byte offset 4096 on that drive, but if copied
to a 512 byte sector drive, it would need to move to byte offset 512 (still
sector 1) and what's more, the contents of the following GPT table would
need to have all of its sector numbers (offsets and sizes) translated,
as they're all in units of the drive's basic sector size.

The same (except for the sector number, as it is always 0, any units) for
an MBR (though CGDs containing MBRs are probably rare.)  I think the same
is true for a BSD disklabel - it is less clear what units those contain, but
I believe they are intended to be sector numbers as well.

To fix this the CGD would need to fully emulate a 512 byte drive, regardless
of sector size, actually doing RMW cycles to allow updating 512 bytes in the
middle of a 4K sector (or whatever size it happens to be.)

For CGD (more than any other layered device) that might actually be the
right solution, regardless of the code complexity or speed implications.
That's because it probably is a good idea to be able to move the contents
of a CGD around without decrypting it.

But I think all of this is a discussion for another day.   I'd just like
to make the thing work rationally first, then its algorithms can be
improved if desired.

kre