Re: write alignment matters?

[Johnny Billquist <bqt%softjar.se@localhost>]

On 2011-06-25 16:29, Robert Elz wrote:
>      Date:        Fri, 24 Jun 2011 20:26:19 -0400 (EDT)
>      From:        der Mouse<mouse%Rodents-Montreal.ORG@localhost>
>      Message-ID:<201106250026.UAA12885%Sparkle.Rodents-Montreal.ORG@localhost>
>
> mouse%Rodents-Montreal.ORG@localhost said:
>    |>  Yes, so it keeps being said.  It would truly be a pity to see that
>    |>  happen.
>
>    | Why?
>
> David Holland answered most of that already - but for me, this one is because
> that's exactly what you want, access to a device in a way that makes
> the device act like any other file.   That's the interface that block
> devices present, access at random offsets, random lengths, ... all the
> kinds of things that you could do to a file, but applied to the device
> directly with no filesystem, remapping.   I really would not like to
> lose that ability, and you seem to not want to lose it either (but you seem
> to want to achieve that by forcing the raw access mechanism to make it
> work.)

I might be confused here. I thought that if you accessed the block 
device, you were restricted to blocks. So you can in fact not seek to an 
arbitrary byte, nor read an arbitrary length, like for a normal file, 
but instead offsets and sizes must be multiples of the block size.

But reading through the man-pages now, I cannot find any references to 
this, so maybe I just dreamt that whole thing?

> Thor Lancelot Simon<tls%panix.com@localhost>  said:
>    | At least for NetBSD, that's never been true.  The most glaring problem is
>    | that there's no protection against causing the same underlying disk blocks
>    | to be multiply cached by accessing the buffer cache with a different 
> stride.
>    | And no way to keep those multiple cachings coherent...
>
> While there's no enforcement, the basic rule has always been "one at a time",
> the same problem applies to raw devices if you have multiple processes
> accessing the things without coordination.   While this one is worse,
> because of the caching delays and unpredictable write order, it all just
> goes away if we prohibit multiple access, doesn't it?   That is, open,
> i/o from the process that opens, and close (with accompanying sync and
> buffer invalidation).   Next open it all starts again, and the new process
> isn't bothered by the previous one.
>
> However, I don't expect that this is the only problem that exists.

I think Thor was thinking of more than just several users (user 
programs) accessing the device simultaneously, but I might have read it 
wrong. Think user program, and then also something like the file system 
code.

> David Holland<dholland-tech%NetBSD.org@localhost>  said:
>    | There's also a problem that the buffer cache code wasn't ever designed to
>    | cope with removable devices, so bad things happen if you try to use the
>    | block device for something that isn't there (e.g. a floppy drive with no
>    | media in it) or that you eject before writeback has completed.
>
> Oh you young'uns ...  believe it or not, removable media was just about
> all that existed when the buffer cache code was designed.   Non-removable
> discs didn't appear until we started seeing winchester class drives, which
> was long after unix was invented (sometime during the 80's...)   Before
> then, the only non-removable mass storage that existed (that I recall anyway)
> were drums, and while there's no reason unix wouldn't have been able to
> use such things (and Steve Bellovin will probably tell us that at the Labs
> they actually did), they were incredibly rare (and expensive, and low
> capacity - like 128KB would have been typical).
>
> Normal unix used rp06's, rk07's, rm03's, rm05's, etc - all removable.
> What didn't exist at the time (rather than removable media) was the
> expectation that it should be possible to remove the media without bothering
> to inform the OS that was about to happen, and give the OS type to update the
> media properly before releasing it.   It took the invention of that paragon of
> OS's - DOS - to teach the populace that simply pulling the device/media was
> an acceptable operating procedure.
>
> The one difference we have today, is that the controllers have moved
> out to accompany the media, rather than remaining in the system, and we
> haven't done a very good job (yet) of adapting to that change of
> methodology - but that's just a SMOP (or not-so-S perhaps) not anything
> that's a fundamental paradigm shift.
>
> A device with no media isn't a problem, never was, accessing that is
> simply an error, today as it always was.  You only really notice it with
> floppies because that horrible interface made the only way to detect the
> absence of media was by actually trying I/O, failing, repeatedly, and
> then eventually concluding the disc must be absent.  On anything sane,
> the controller just says "not there" and the error is immediate.

True. However, Unix have never really gracefully handled file systems or 
devices that comes and goes. :-)

> David Laight<david%l8s.co.uk@localhost>  said:
>    | Apparantly it was useful to put a bad block at the end of every tape
>    | track so that otherwise sequential data wouldn't cross track boundaries.
>
> I don't recall ever doing that - DECtape capacity was so small that
> artificially reducing it doesn't seem like a good idea to me.

And it is incorrect. DECtapes don't have some bad block at the end. And 
unless you are careful when you play around, you will spin off the end 
of the reel on a DECtape.
The tape is furthermore formatted into blocks, and "sequential" data is 
handled no different than on any other block addressable disk.

It's all up to the OS to put some higher form of interpretation on the 
blocks. (Like regarding the data contents as a stream of bytes, if you 
want that.)

>    | IIRC I have seen DECtape in action - on a pdp8.
>
> My fingers used to remember the key change sequence to boot a pdp8 from
> DECtape (the optimal way to move the data entry keys to enter the
> boot sequence with as little pain as possible) - fortunately, that's now
> so long ago that I now remember no more than that I used to remember!

That's what the OS/8 handbook is for. :-)

        Johnny

--
Johnny Billquist                  || "I'm on a bus
                                  ||  on a psychedelic trip
email: bqt%softjar.se@localhost             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol