I've assembled some notes from old manuals and other sources
on the formats used for on-disk file systems through the
Additional notes, comments on style, and whatnot are welcome.
(It may be sensible to send anything in the last two categories
directly to me, rather than to the whole list.)
Hi all, I received an e-mail looking for the ksh-88 source code. A quick
search for it on-line doesn't reveal it. Does anybody have a copy?
I recently built a PiDP11 and have been enjoying going back in time
to 2.11BSD.. I was at UC Davis in the the early 1980's and we had
a few PDP-11/70's running 2.8/2.9 BSD. Back then we reached out to
David Korn and he sent us the source for KSH -- this would have been
in 1985ish if I remember, and we compiled it for 2.9 & 4.1BSD, Xenix,
and some other variants that used K&R C. It may have been what was
later called ksh88. I wish I still had the files from then..
I was wondering if you might know if there's an older version like this
or one that's been ported for 2.11BSD?
I've recently started to implement a set of helper functions and
procedures for parsing Unix-like command-line interfaces (i.e., POSIX +
GNU-style long options, in this case) in Ada.
While doing that, I learned that there is a better way to approach
this problem – beyond using getopt(s) (which never really made sense to
me) and having to write case statements in loops every time: Define a
grammar, let a pre-built parser do the work, and have the parser
provide the results to the program.
Now, defining such a grammar requires a thoroughly systematic approach
to the design of command-line interfaces. One problem with that is
whether that grammar should allow for sub-commands. And that leads to
the question of how task-specific tool sets should be designed. These
seem to be a relatively new phenomenon in Unix-like systems that POSIX
doesn't say anything about, as far as I can see.
So, I've prepared a bit of a write-up, pondering on the pros and cons
of two different ways of having task-specific tool sets
(non-hierarchical command sets vs. sub-commands) that is available at
I tend to think the sub-command approach is better. But I'm neither a UI
nor a Unix expert and have no formal training in computer things. So, I
thought this would be a good place to ask for comment (and get some
This is all just my pro-hobbyist attempt to make some people's lives
easier, especially mine. I mean, currently, the "Unix" command line is
quite a zoo, and not in a positive sense. Also, the number of
well-thought-out command-line interfaces doesn't seem to be a growing
one. But I guess that could be changed by providing truly easy ways to
make good interfaces.
I recently upgraded my machines to fc34. I just did a stock
uncomplicated installation using the defaults and it failed miserably.
Fc34 uses btrfs as the default filesystem so I thought that I'd give it
a try. I was especially interested in the automatic checksumming because
the majority of my storage is large media files and I worry about bit
rot in seldom used files. I have been keeping a separate database of
file hashes and in theory btrfs would make that automatic and transparent.
I have 32T of disk on my system, so it took a long time to convert
everything over. A few weeks after I did this I went to unload my
camera and couldn't because the filesystem that holds my photos was
mounted read-only. WTF? I didn't do that.
After a bit of poking around I discovered that btrfs SILENTLY remounted the
filesystem because it had errors. Sure, it put something in a log file,
but I don't spend all day surfing logs for things that shouldn't be going
wrong. Maybe my expectation that filesystems just work is antiquated.
This was on a brand new 16T drive, so I didn't think that it was worth
the month that it would take to run the badblocks program which doesn't
really scale to modern disk sizes. Besides, SMART said that it was fine.
Although it's been discredited by some, I'm still a believer in "stop and
fsck" policing of disk drives. Unmounted the filesystem and ran fsck to
discover that btrfs had to do its own thing. No idea why; I guess some
think that incompatibility is a good thing.
Ran "btrfs check" which reported errors in the filesystem but was otherwise
useless BECAUSE IT DIDN'T FIX ANYTHING. What good is knowing that the
filesystem has errors if you can't fix them?
Near the top of the manual page it says:
Do not use --repair unless you are advised to do so by a developer
or an experienced user, and then only after having accepted that
no fsck successfully repair all types of filesystem corruption. Eg.
some other software or hardware bugs can fatally damage a volume.
Whoa! I'm sure that operators are standing by, call 1-800-FIX-BTRFS.
Really? Is a ploy by the developers to form a support business?
Later on, the manual page says:
enable the repair mode and attempt to fix problems where possible
Note there’s a warning and 10 second delay when this option
is run without --force to give users a chance to think twice
before running repair, the warnings in documentation have
shown to be insufficient
Since when is it dangerous to repair a filesystem? That's a new one to me.
Having no option other than not being able to use the disk, I ran btrfs
check with the --repair option. It crashed. Lesson so far is that
trusting my data to an unreliable unrepairable filesystem is not a good
idea. Since this was one of my media disks I just rebuilt it using ext4.
Last week I was working away and tried to write out a file to discover
that /home and /root had become read-only. Charming. Tried rebooting,
but couldn't since btrfs filesystems aren't checked and repaired. Plugged
in a flash drive with a live version, managed to successfully run --repair,
and rebooted. Lasted about 15 minutes before flipping back to read only
with the same error.
Time to suck it up and revert. Started a clean reinstall. Got stuck
because it crashed during disk setup with anaconda giving me a completely
useless big python stack trace. Eventually figured out that it was
unable to delete the btrfs filesystem that had errors so it just crashed
instead. Wiped it using dd; nice that some reliable tools still survive.
Finished the installation and am back up and running.
Any of the rest of you have any experiences with btrfs? I'm sure that it
works fine at large companies that can afford a team of disk babysitters.
What benefits does btrfs provide that other filesystem formats such as
ext4 and ZFS don't? Is it just a continuation of the "we have to do
everything ourselves and under no circumstances use anything that came
from the BSD world" mentality?
So what's the future for filesystem repair? Does it look like the past?
Is Ken's original need for dsw going to rise from the dead?
In my limited experience btrfs is a BiTteR FileSystem to swallow.
Or, as Saturday Night Live might put it: And now, linux, starring the
not ready for prime time filesystem. Seems like something that's been
under development for around 15 years should be in better shape.
Not to get into what is soemthing of a religious war,
but this was the paper that convinced me that silent
data corruption in storage is worth thinking about:
A key point is that the character of the errors they
found suggests it's not just the disks one ought to worry
about, but all the hardware and software (much of the latter
inside disks and storage controllers and the like) in the
I had heard anecdotes long before (e.g. from Andrew Hume)
suggesting silent data corruption had become prominent
enough to matter, but this paper was the first real study
I came across.
I have used ZFS for my home file server for more than a
decade; presently on an antique version of Solaris, but
I hope to migrate to OpenZFS on a newer OS and hardware.
So far as I can tell ZFS in old Solaris is quite stable
and reliable. As Ted has said, there are philosophical
reasons why some prefer to avoid it, but if you don't
subscribe to those it's a fine answer.
I've been hearing anecdotes since forever about sharp
edges lurking here and there in BtrFS. It does seem
to be eternally unready for production use if you really
care about your data. It's all anecdotes so I don't know
how seriously to take it, but since I'm comfortable with
ZFS I don't worry about it.
PS: Disclosure: I work in the same (large) CS department
as Bianca Schroeder, and admire her work in general,
though the paper cited above was my first taste of it.
This may be due to logic similar to that of a classic feature that I
always deemed a bug: troff begins a new page when the current page is
exactly filled, rather than waiting until forced by content that
doesn't fit. If this condition happens at the end of a document, a
spurious blank page results. Worse, if the page header happens to
change just after the exactly filled page, the old heading will be
produced before the new heading is read.
> fork() is a great model for a single-threaded text processing pipeline to do
> automated typesetting. (More generally, anything that is a straightforward
> composition of filter/transform stages.) Which is, y'know, what Unix is *for*.
> It's not so great for a responsive GUI in front of a multi-function interactive program.
"Single-threaded" is not a term I would apply to multiple processes in
a pipeline. If you mean a single track of data flow, fine, but the
fact that that's a prevalent configuration of cooperating processes in
Unix is an artifact of shell syntax, not an inherent property of
pipe-style IPC. The cooperating processes in Rob Pike's 20th century
window systems and screen editors, for example, worked smoothly
without interrupts or events - only stream connections. I see no
abstract distinction between these programs and "stuff people play
with on their phones."
It bears repeating, too, that stream connections are much easier to
reason about than asynchronous communication. Thus code built on
streams is far less vulnerable to timing bugs.
At last a prince has come to awaken the sleeping beauty of stream
connections. In Go (Pike again) we have a widely accepted programming
language that can fully exploit them, "[w]hich is, y'know, what Unix
(If you wish, you may read "process" above to include threads, but
I'll stay out of that.)
once again i am taken aback at the good taste of the residents of the unix room.
As a whilom denizen of that esteemed playroom, I question
both the accuracy and the relevance of that metric.
Besides, what happened to the sheep shelf? Was it scrubbed
away after I left? And, Ken, whatever happened to Dolly the
Sheep (after she was hidden to avoid upsetting visitors)?
No longer a subscriber to sheep! magazine
> I don't think anyone knows. Nobody relevant, I believe.
I understand that Dave Presotto bought that photo at a garage sale for $1. The photo hung in
the Unix Room for years, at one point labeled “Peter Weinberger.”
One day I removed it from its careful mounting and scanned in the photo. It bore the label
“what, no steak?”
The photo was stolen from a wall sometime after I left. The scanned image is at