Subject: Re: TS-72xxx flash and NetBSD
To: Francis Dupont <Francis.Dupont@enst-bretagne.fr>
From: Jesse Off <joff@embeddedARM.com>
List: tech-embed
Date: 02/04/2005 16:41:00
I'm Cc'ing this to the tech-kern@netbsd.org mailing list in order to 
hopefully solicit a few other expert opinions on the subject.

I haven't implemented anything for the onboard TS72xx flash support because 
the best course of action isn't yet perfectly clear (to me anyway).  I know 
what I'd like to see avoided in an implementation, but don't yet have a good 
plan of action to offer that would avoid all the chaos thats currently in 
Linux in the form of "MTD" drivers and special filesystems unique to each 
type of flash chip (JFFS2, YAFFS, YAFFS2, FTL/EXT2, NFTL/EXT2, etc).  There 
are certain things unique about using direct mapped NAND/NOR flash (as 
opposed to something like CF, or USB flash which has an onboard controller), 
but a completely new filesystem (and one also for each type and variant of 
chip) seems a bit overkill.  Some ideas:

* Come up with some sort of  "unreliable/quirky block device" layer that can 
be used to implement the same sort of internal logic already present in 
devices like CompactFlash and hard drives such that a regular FFS filesystem 
could be placed on it.  Devices (512B NAND, NOR, 2KB NAND) simply register 
some generic block device access functions and their quirks with this layer, 
and then a regular FFS filesystem could be placed on the "de-quirked" block 
device.  The de-quirking driver has all the intelligence of filesystem 
agnostic wear-leveling, ECC generation/checking, and bad block management 
(by reserving a driver-defined percentage of blocks)

* Perhaps 1) extend ffs's block allocation policy to be random within free 
space, (with preference to a sector already erased or in an area already 
marked for erase/rewrite) 2) force block rewrites to go through a block 
reallocation instead of using the same block, 3) rate-limit/aggressively 
buffer FS metadata (inode, block bitmap, etc) writes.  This would still have 
issue w/NAND for bad block management, and having to erase/rewrite an entire 
erase block at a time would expose ourselves to potential major data-lossage 
on crashes.  The more I think about this one, the more it seems it won't 
work.

* LFS would be a good place to start, since it naturally wear levels, though 
there are certain "near-full" runaway conditions that are quite hard to 
accomodate the possibility for unless you (for instance) reserve 50% of the 
disk.  They are statistically unlikely to appear in most real workloads, 
though still quite possible, especially if you run the disk close to full. 
I understand the Linux logging flash FS's also have issue with RAM usage 
with larger chips.

* Maybe do something by using something like fss with mfs and periodically 
writing out/commiting changes to the real FFS.  (Still have bad block 
managment problems)  By rate-limiting sector erase/rewrites you can acheive 
results similar to wear leveling for a given target flash minimum lifetime.

* Maybe relegate the whole thing to userland and use some of the Linux 
GPL'ed filesystems and an NFS loopback mount.  (prohibits use as root 
filesystem)

* Just use a ramdisk image and mfs union mount and have an easy way to say 
"save me" and thereafter write out a new ramdisk to the original flash 
location the kernel+ramdisk reside.

//Jesse Off

> Why is there on support in NetBSD for TS-72xx on-board flash and
> how can I help? I.e., is it a problem of file-system (NetBSD has
> no flash oriented file-system) or just a lack of a flash driver?
>
> Thanks
>
> Francis.Dupont@enst-bretagne.fr