On 3/23/17, Larry McVoy <lm(a)mcvoy.com> wrote:
>
> I only know a tiny amount about Plan 9, never dove into deeply. QNX,
> on the other hand, I knew quite well. I was pretty good friends with
> one of the 3 or 4 people who were allowed to work on the microkernel, Dan
> Hildebrandt. He died in 1998, but before then he and I used to call each
> other pretty often to talk about kernel stuff, how stuff ought to be done.
> The calls weren't jerk off sessions, they were pretty deep conversations,
> we challenged each other. I was very skeptical about microkernels,
> I'd seen Mach and found it lacking, seen Minix and found it lacking
> (though that's a little unfair to Minix compared to Mach). Dan brought
> me around to believing in the microkernel but only if the kernel was
> actually a kernel. Their kernel didn't use up a 4K instruction cache,
> it left room for the apps and the processes that did the rest. That's
> why only a few people were allowed to work on the kernel, they counted
> every single instruction cache footprint.
>
> So tell me more about your OS, I'm interested.
Where do I start? I've got much of the design planned out. I've
thought about the design for many years now and changed my plans on
some things quite a few times. Currently the only code I have is a
bootloader that I've been working on somewhat intermittently for a
while now, as well as a completely untested patch for seL4 to add
support for QNX-ish single image boot. I am planning to borrow Linux
and BSD code where it makes sense, so I have less work to do.
It will be called UX/RT (for Universally eXtensible Real Time
operating system, although it's also a kind of a nod to QNX originally
standing for Quick uNiX), and it will be primarily intended as a
workstation and embedded OS (although it would be also be good for
servers where security is more important than I/O throughput, and also
for HPC). It will be a microkernel-based multi-server OS.
Like I said before, it will superficially resemble QNX in its general
architecture (for example, much like QNX, the lowest-level user-mode
component will be a single root server called "proc", which will
handle process management and filesystem namespace management),
although the specifics of the IPC model will be somewhat different. I
will be using seL4 and/or Rux as the microkernel (so unlike that of
QNX, UX/RT's process server will be a completely separate program).
proc and most other first-party components will be mostly written in
Rust rather than C, although there will still be a lot of third-party
C code. The network stack, disk filesystems, and graphics drivers will
be based on the NetBSD rump kernel and/or LKL (which is a
"librarified" version of the Linux kernel; I'll probably provide both
Linux and NetBSD implementations and allow switching between them and
possibly combining them) with Rust glue layers on top. For
performance, the disk drivers and disk filesystems will run within the
same server (although there will be one disk server per host adapter),
and the network stack will also be a single server, much like in QNX
(like QNX, UX/RT will mostly avoid intermediary servers and will
usually follow a process-per-subsystem architecture rather than a
process-per-component one like a lot of other multi-sever OSes, since
intermediary servers can hurt performance).
As I said before, UX/RT will take file-oriented architecture even
further than Plan 9 does. fork() and some thread-related APIs will be
pretty much the only APIs implemented as non-file-based primitives.
Pretty much the entire POSIX/Linux API will be implemented although
most non-file-based system calls will have file-based implementations
underneath (for example, getpid() will do a readlink() of /proc/self
to get the PID). Even process memory like the process heap and stack
will be implemented as files in a per-process memory filesystem (a bit
like in Multics) rather than being anonymous like on most other OSes.
ioctl() will be a pure library function for compatibility purposes,
and will be implemented in terms of read() and write() on a secondary
file.
Unlike other microkernel-based OSes, UX/RT won't provide any way to
use message passing outside of the file-based API. read() and write()
will use kernel calls to communicate directly with the process on the
other end (unlike some microkernel Unices in which they go through an
intermediary server). There will be APIs that expose the underlying
transport (message registers for short messages and a shared per-FD
buffer for long ones), although they will still operate on file
descriptors, and read()/write() and the low-level messaging APIs will
all use the same header format so that processes don't have to care
which API the process on the other
end uses (unlike on QNX where there are a few different incompatible
messaging APIs). There will be a new "message special" file type that
will preserve message boundaries, similar to SEQPACKET Unix-domain
sockets or SCTP (these will be ideal for RPC-type APIs).
File descriptors will be implemented as sets of kernel capabilities,
meaning that servers won't have to check permissions like they do on
QNX. The API for servers will be somewhat socket-like. Each server
will listen on a "port" file in a special filesystem internal to the
process server, sort of like /mnt on Plan 9 (although it will be
possible for servers to export ports within their own filesystems as
well). Reads from the port will produce control messages which may
transfer file descriptors. Each client file descriptor will have a
corresponding file descriptor on the server side, and servers will use
a superset of the regular file API to transfer data. Device numbers as
such will not exist (the device number field in the stat structure
will be a port ID that isn't split into major and minor numbers), and
device files will normally be exported directly by their server,
rather than residing on a filesystem exported by one driver but being
serviced by another as in conventional Unix. However, there will be a
sort of similar mechanism, allowing a server to export "firm links"
that are like cross-filesystem hard links (similar to in QNX).
Per-process namespaces like in Plan 9 will be supported. Unlike in
Plan 9, it will be possible for processes with sufficient privileges
to mount filesystems in the namespaces of other processes (to allow
more flexible scoping of mount points). Multiple mounts on one
directory will produce a union like in both QNX and Plan 9. Binding
directories as in Plan 9 will also be supported. In addition to the
per-process root on / there will also be a global root directory on //
into which filesystems are mounted. The per-process name spaces will
be constructed by binding directories from // into the / of the
process (neither direct mounts under / nor bindings in // will be
supported, but bindings between parts of / will of course be
supported).
The security model will be based on a per-process default-deny ACL
(which will be purely in memory; persisting process ACLs will be
implemented with an external daemon). It will be possible for ACL
entries to explicitly specify permissions, or to use the permissions
from the filesystem with (basically) normal Unix semantics. It will
also be possible for an entry to be a wildcard allowing access to all
files in a directory. Unlike in conventional Unix, there will be no
root-only system calls (anything security-sensitive will have a
separate device file to which access can be controlled through process
ACLs), and running as root will not automatically grant a process full
privileges. The suid and sgid bits will have no effect on executables
(the ACL management daemon will handle privilege escalation instead).
The native API will be mostly compatible with that of Linux, and a
purely library-based Linux compatibility layer will be available. The
only major thing the Linux compatibility layer specifically won't
support will be stuff dealing with logging users in (since it won't be
possible to revert to traditional Unix security, and utmp/wtmp won't
exist). The package manager will be based on dpkg and apt with hooks
to make them work in a functional way somewhat like Nix but using
per-process bindings, allowing for multiple environments or universes
consisting of different sets of packages (environments will be able to
be either native UX/RT or Linux compatibility environments, and it
will be possible to create Linux environments that aren't managed by
the UX/RT package manager to allow compatibility environments for
non-dpkg Linux distributions).
The init system will have some features in common with SMF and
systemd, but unlike those two, it will be modular, flexible, and
lightweight. System initialization such as checking/mounting
filesystems and bringing up network interfaces will be script-driven
like in traditional init systems, whereas starting daemons will be
done with declarative unit files that will be able to call (mostly
package-independent) hook scripts to set up the environment for the
daemon.
Initially I will use X11 as the window system, but I will replace it
with a lightweight compositing window server that will export
directories providing a low-level DRI-like interface per window.
Unlike a lot of other compositing window systems, UX/RT's window
system will use X11-style central client-side window management. I was
thinking of using a default desktop environment based on GNUstep
originally but I'm not completely sure if I'll still do that (a long
time ago I had wanted to put together a Mac OS X-like or NeXTStep-like
Linux distribution using a GNUstep-based desktop, a DPS-based window
server, and something like a fork of mkLinux with a stable module ABI
for a kernel, but soon decided I wanted to write a QNX-like OS
instead).
>
> Sockets are awesome but I have to agree with you, they don't "fit".
> Seems like they could have been plumbed into the file system somehow.
>
Yeah, the almost-but-not-quite-file-based nature of the socket API is
my biggest complaint about it. UX/RT will support the socket API but
it will be implemented on top of the normal file system.
> Can't speak to your plan 9 comments other than to say that the fact
> that you are looking for provided value gives me hope that you'll get
> somewhere. No disrespect to plan 9 intended, but I never saw why it
> was important that I moved to plan 9. If you do something and there
> is a reason to move there, you could be worse but still go farther.
I'd say a major problem with Plan 9 is the way it changes things in
incompatible ways that provide little advantage over the traditional
Unix way of doing things (for example, instead of errno, libc
functions set a pointer to an error string instead, which I don't
think provides enough of a benefit to break compatibility). Another
problem is that some facilities Plan 9 provides aren't general enough
(e.g. the heavy focus on SSI clustering, which never really was widely
adopted, or the rather 80s every-window-is-a-terminal window system).
UX/RT will try to be compatible with conventional Unices and
especially Linux wherever it is reasonable, since lack of applications
would significantly hold it back. It will also try to be as general as
possible without overcomplicating things.
All, I'm setting up a uucp site 'tektronix'. When I send e-mail, I'm seeing
this error:
ASSERT ERROR (uux) pid: 235 (3/24-00:09) CAN'T OPEN D.tektronX00D0 (0)
Something seems to be trimming the hostname to seven chars. If I do:
# hostname
tektronix
Thanks, Warren
On Fri, Mar 24, 2017 at 4:08 PM, Andy Kosela <andy.kosela(a)gmail.com> wrote:
>
> [snip]
> Dan, that was an excellent post.
>
Thanks! I thought it was rather too long/wordy, but I lacked the time to
make it shorter.
I always admired the elegance and simplicity of Plan 9 model which indeed
> seem to be more UNIX like than todays BSDs and Linux world.
>
> The question though remains -- why it has not been more successfull? The
> adoption of Plan 9 in the real world is practically zero nowadays and even
> its creators like Rob Pike moved on to concentrate on other things, like
> the golang.
>
I think two big reasons and one little one.
1. It wasn't backwards compatible with the rest of the world and forced you
to jump headlong into embracing a new toolset. That is, there was no
particularly elegant way to move gradually to Plan 9: you had to adopt it
all from day one or not at all. That was a bridge too far for most. (Yeah,
there were some shims, but it was all rather hacky.)
2. Relatedly, it wasn't enough of an improvement over its predecessor to
pull people into its orbit. Are acme or sam really all that much better
than vi or emacs? Wait, don't answer that...but the reality is that people
didn't care enough whether they were or not. The "everything is a
file(system)" idea is pretty cool, but we've already had tools that worked
then and work now. Ultimately, few people care how elegant the abstractions
are or how well the kernel is implemented.
And the minor issue: The implementation. Plan 9 was amazing for when it was
written, but now? Not so much.
I work on two related kernels: one that is directly descended from Plan 9
(Harvey, for those interested) and one that's borrowed a lot of the code
(Akaros) and in both we've found major, sometimes decades old bugs. There
are no tests, and there are subtle race conditions or rarely tickled bugs
lurking in odd places. Since the system is used so little, these don't
really get shaken out the way they do in Linux (or to a lesser extent the
BSDs or commercial Unixes). In short, some code is better than other code
and while I'd argue that the median quality of the implementation is
probably higher than that of Linux or *BSD in terms of elegance and
understandability, it's not much higher and it's certainly buggier.
And the big implementation issue is lack of hardware support. I stood up
two plan 9 networks at work for Akaros development and we ran into major
driver issues with the ethernets that took a week or two to sort out. On
the other hand, Linux just worked.
Eventually, one of those networks got replaced with Linux and the other is
probably headed that way. In fairness, this has to do with the fact that no
one besides Ron and I was interested in using them or learning how they
work: people *want* Linux and the idea that there's this neat system out
there for them to explore and learn about the cool concepts it introduced
just isn't a draw. I gave a presentation on Plan 9 concepts to the Akaros
team a year and a half or so ago and a well-known figure in the Linux
community who working with us at the time had only to say that, "the user
interface looks like it's from 1991." None of the rest didn't interest him
at all: the CPU command usually kind of blows people's minds, but after I
went through the innards of it the response was, "why not just use SSH?"
I've had engineers ask me why Plan 9 used venti and didn't "just use git"
(git hadn't been invented yet). It's somewhat lamentable, but it's also
reality.
- Dan C.
> From: Random832
> "a stream consisting of a serialized sequence of all of whatever
> information would have been supplied to/by the calls to the special
> function" seems like a universal solution at the high level.
Yes, and when the only tool you have is a hammer, everything look like
a nail.
Noel
> From: Nick Downing
> Programming is actually an addiction.
_Can be_ an addition. A lot of people are immune... :-)
> What makes it addictive to a certain type of personality is that little
> rush of satisfaction when you try your code and it *works*... ... It was
> not just the convenience and productivity improvements but that the
> 'hit' was coming harder and faster.
Joe Weizenbaum wrote about the addiction of programming in his famous book
"Computer Power and Human Reason" (Chapter 4, "Science and the Compulsive
Programmer"). He attributes it to the sense of power one gets, working in a
'world' where things do exactly what you tell them. There might be something
to that, but I suspect your supposition is more likely.
> This theory is well known to those who design slot machines and other
> forms of gambling
Oddly enough, he also analogizes to gamblers!
Noel
> From: "Ron Natalie"
> I was thinking about Star Wars this morning and various parodies of it
> (like Ernie Foss's Hardware Wars)
The best one ever, I thought, was Mark Crispin's "Software Wars". (I have an
actual original HAKMEM!)
> I rememberd the old DEC WARS.
I seem to vaguely recall a multi-page samizdat comic book of this name? Or am
I mis-remembering its name? Does this ring any bells for anyone?
Noel
I realized after writing that I was being slightly unfair since one valid
use case that DOES work correctly is something like:
ssh -X <some host> <command that uses X>
This is occasionally handy, although the best use case I can think of is
running a browser on some internet-facing machine so as to temporarily
change your IP address, and this use case isn't exactly bulletproof since
at least google chrome will look for a running instance and hand over to it
(despite that instance having a different DISPLAY= setting). Nevertheless
my point stands which is that IMO a programmatic API (either through .so or
.dll linkage, or through ioctls or dedicated syscalls) should be the first
resort and anything else fancy such as remoting, domain specific languages,
/proc or fuse type interfaces, whatever, should be done through extra
layers as appropriate. You shouldn't HAVE to use them.
cheers, Nick
On Mar 15, 2017 9:15 PM, "Tim Bradshaw" <tfb(a)tfeb.org> wrote:
On 15 Mar 2017, at 01:13, Nick Downing <downing.nick(a)gmail.com> wrote:
>
> But the difficulty with X Windows is that the remoting layer is always
there, even though it is almost completely redundant today.
It's redundant if you don't ever use machines which you aren't physically
sitting next to and want to run any kind of graphical tool run on them. I
do that all the time.
--tim
> From: Tim Bradshaw
> I don't know about other people, but I think the whole dope thing is why
> computer people tend *not* to be hippies in the 'dope smoking' sense. I
> need to be *really awake* to write reasonably good code ... our drugs
> of choice are stimulants not depressants.
Speak for yourself! :-)
(Then again, I have wierd neuro-chemistry - I have modes where I have a large
over-sppply of natural stimulant... :-)
My group (which included Prof. Jerry Salzter, who's about as straight an arrow
as they make) was remarkably tolerant of my, ah, quirks... I recall at one
point having a giant tank of nitrous under the desk in my office - which they
boggled at, but didn't say anything about! ;-)
Noel
"Two Bacco, here, my Bookie.”
Awesome.
David
> Date: Wed, 22 Mar 2017 21:26:16 -0400
> From: "Ron Natalie" <ron(a)ronnatalie.com>
> To: <tuhs(a)minnie.tuhs.org>
> Subject: [TUHS] DEC Wars
> Message-ID: <001d01d2a374$77e02dc0$67a08940$(a)ronnatalie.com>
> Content-Type: text/plain; charset="utf-8"
>
> I was thinking about Star Wars this morning and various parodies of it (like
> Ernie Foss's Hardware Wars) and I rememberd the old DEC WARS. Alas when I
> tried to post it, it was too big for the listserv. So here's a link for
> your nostalgic purposes. I had to find one that was still in its
> fixed-pitch glory complete with the ASCII-art title.
>
>
>
> http://www.inwap.com/pdp10/decwars.txt
>
> From: Steffen Nurpmeso
> This "We owe it all to the Hippies"
Well, yes and no. Read "Hackers". There wasn't a tremendous overlap between
the set of 'nerds' (specifically, computer nerds) and 'hippies', especially in
the early days. Not that the two groups were ideologically opposed, or
incompatible, or anything like that. Just totally different.
Later on, of course, there were quite a few hackers who were also 'hippies',
to some greater or lesser degree - more from hackers taking on the hippie
vibe, than the other way around, I reckon. (I think that to be a true computer
nerd, you have to start down that road pretty early on, and with a pretty
severe commitment - so I don't think a _lot_ of hippied turned into hackers.
Although I guess the same thing, about starting early, is true of really
serious musicians.)
> "The real legacy of the 60s generation is the Computer Revolution"
Well, there is something to that (and I think others have made this
observation). The hippie mentality had a lot of influence on everyone in that
generation - including the computer nerds/hackers. Now, the hackers may have
had a larger, impact, long-term, than the hippies did - but in some sense a
lot of hippie ideals are reflected in the stuff a lot of hackers built:
today's computer revolution can be seen as hippie idealism filtered through
computer nerds...
But remember things like this, from the dust-jacket of the biography of
Prof. Licklider:
"More than a decade will pass before personal computers emerge from the
garages of Silicon Valley, and a full thirty years before the Internet
explosion of the 1990s. The word computer still has an ominous tone,
conjuring up the image of a huge, intimidating device hidden away in an
over-lit, air-conditioned basement, relentlessly processing punch cards for
some large institution: _them_. Yet, sitting in a nondescript office in
McNamara's Pentagon, a quiet ... civilian is already planning the revolution
that will change forever the way computers are perceived. Somehow, the
occupant of that office ... has seen a future in which computers will empower
individuals, instead of forcing them into rigid conformity. He is almost
alone in his conviction that computers can become not just super-fast
calculating machines, but joyful machines: tools that will serve as new media
of expression, inspirations to creativity, and gateways to a vast world of
online information.
Now, technically Lick wasn't a hippie (he was, after all, 40 years old in
1965), and he sure didn't have a lot of hippie-like attributes - but he was,
in some ways, an ideological close relative of some hippies.
Noel
