Terry Moore wrote:
> CF cards are done with NAND flash.  They don't "know where things
> are", they just have an intermediate layer that provides an
> abstraction.  Due to patents, this is done in a variety of ways.

In order to provide that layer, they have to have somewhere to store
their persistent data.  So they are able to make assumptions about where
they can stick stuff on flash.

If I follow that approach, the end result will be a flash system that is
only useful to itself.  I.e. you won't have a way to directly access
blocks of data at a specific physical sector.

This can have bad effects, e.g. for some flash systems which need to
store data like bootloaders at a specific offset in flash.

The translation layers that work well for turning a *specific* device in
to a general mass storage device don't work well for the kinds of things
that I think I want to be able to achieve.

    -- Garrett
>
> Don't obsess on block numbers "above the line", as it were, having
> anything necessary to do with block numbers below the line; they (in
> general) will not.  Block-oriented file systems don't actually care;
> they depend on very specific abstract semantics (I wrote X to location
> Z and when I read X, I get Z; and for all Y not equal to X, nothing
> got changed by writing Z, and I can write X to Z as often as I like.) 
> Generally these days there's lots of translation going on downstream
> from them (think about RAID, striping, etc).  Performance benefits
> from a design that assumes mostly one-to-one correspondence are pretty
> modest, when you consider that flash needs substantial error management.
>
> Breaking the one-to-one correspondence paradigm lets you separate
> flash management from the rest of the world, and there are techniques
> to let things be consecutive most of the time, which gets your data
> transfer performance up where you want it.
>
> --Terry
>
> At 12:28 PM 3/23/2006 -0800, Garrett D'Amore wrote:
>> Hmm.. with something like CF cards, you *know* where you can store the
>> relevant data, and its easy to export a simple disk-like device.  The
>> translation layer makes sense here (I think).  (You still have an
>> interesting problem about how to handle writes of < 1 sector though.  Do
>> NAND devices have the big sector sizes that you find in NOR devices?)
>>
>> The problem is that if you want to make something that is reusable, its
>> much harder, because e.g. you don't know where you can stash the
>> book-keeping you need to provide that translation layer.   On a specific
>> device its easy, because you know the layout of data for that device,
>> and can hide this kind of info.
>>
>> I'm interesting in operating on a large variety of devices where those
>> kind of assumptions (the kind where say, its safe to write accounting
>> data at this place in flash, or at this part of OOB) may not hold true.
>>
>> I guess, in other words, I think creating something that is a generic
>> solution for an operating system like NetBSD has a different set of
>> tradeoffs than if I'm inventing something that is only going to be used
>> in one kind of device or in a set of devices where the layout of data on
>> media can be strictly controlled.
>>
>>     -- Garrett
>>
>> Terry Moore wrote:
>> > Hi Garret,
>> >
>> > My point was that to have a common layer, you have to design for NAND,
>> > not for NOR, and that it's not something anyone can do casually.
>> >
>> > Based on my experience, trying to couple this kind of detail into a
>> > file system is a really bad idea.  FTLs (the generic term in the
>> > industry for "Flash Transfer Layers", the transfer layers that hide
>> > the media from the file system) are much better.  The commercial
>> > market has come to this conclusion too; this is why CF cards don't
>> > expose the flash to the host machine.  In the early 90s, I helped
>> > write the PCMCIA spec that introduced standards for all this, and
>> > worked for a company that used both Microsoft's original FFS and then
>> > M-Systems TFFS (which is really an FTL).  Microsoft's original FFS
>> > never worked well. The problem was that the FFS systems that knew
>> > about the media details were much too complex, and were unmaintainable
>> > as the media evolved.  It is possible to get everything (including
>> > performance and reliability) via an FTL; then you can use existing
>> > file systems (FAT, UDF, etc).  There really aren't enough hours in the
>> > day to deal with all these details.  I've written file system and FTL
>> > code, and the thought of trying to test all the possible interactions
>> > between the file system, FTL and memory technology is daunting.  Since
>> > the performance of the file system is critical for system integrity
>> > and security, keeping things simple is a really good idea.  In
>> > addition to all their other properties, FTLs have a really clean
>> > unit-test interface.
>> >
>> > Somewhat off topic, "MTD" is the term of art that was invented by
>> > PCMCIA to describe the low level drivers for memory access (SRAM
>> > cards, EPROM cards, Flash cards).  Five years from now something may
>> > well replace "flash", but wear levelling and data integrity
>> > considerations that are peculiar to non-rotating inexpensive media are
>> > likely still to be an issue.   I'd strongly suggest using "mtd" rather
>> > than "flash" as the name of the bus, for this reason.  If I understood
>> > the earlier comments, Linux got this one right.
>> >
>> > --Terry
>> >
>> > At 11:45 AM 3/23/2006 -0800, Garrett D'Amore wrote:
>> >> I *think* the existence of
>> >> the MTD framework in Linux is evidence that the notion that the
>> >> abstractions have *something* in common is not totally insane.
>>
>>
>> -- 
>> Garrett D'Amore, Principal Software Engineer
>> Tadpole Computer / Computing Technologies Division,
>> General Dynamics C4 Systems
>> http://www.tadpolecomputer.com/
>> Phone: 951 325-2134  Fax: 951 325-2191


-- 
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134  Fax: 951 325-2191