10 Dec
2022
10 Dec
'22
7:38 p.m.
Good morning all. I've been doing some historical research on the UUCP cu utility
this morning and have come across a little discrepancy between the various UNIX streams I
was wondering if someone could illuminate.
So cu as of V7 supported the ~$ escape, a means of calling a local procedure and emitting
stdout over the TTY line to the remote machine, all fine and good for packaging a
character stream to emit. However, what I'm not finding in that age of documentation
is any means of requesting std*in* from the TTY line as input to a local procedure (in
essence running a text filter or handshake-driven protocols over cu). The context in
which I'm researching this is integrating cu into my bare-metal SBC programming using
XMODEM so I can rest a little easier my process is based on tools I'll probably find
in most places.
So old fashioned Mike Lesk-era cu only seems to do stdout redirect, but no stdin. I did
some further digging and it looks like different UUCP implementations cracked this nut
with different escapes, with BSD eventually going with ~C and Taylor UUCP opting for ~+.
Checking the current illumos manual pages (for a SVR4-ish example) doesn't turn up
any command for this. This is indicative of there never being an agreed-upon mechanism
for doing this, although I could see this being a very useful mechanism.
What I'm curious about is if the lack of a bi-directional redirect in early cu is
reflective of a lack of need for that sort of functionality at the time or that matters
such as that were handled through a different mechanism. One thought I did have is that
there wasn't file locking at the time (right?) and so theoretically nothing would
prevent one from using a cu session on one terminal to send interactive commands and then
a second using fd redirects in the shell to run filters/protocols separately of the
interactive stream.
- Matt G.
11 Dec
11 Dec
12:22 a.m.
On Sat, Dec 10, 2022 at 2:39 PM segaloco via TUHS <tuhs(a)tuhs.org> wrote:
Good morning all. I've been doing some
historical research on the UUCP cu
utility this morning and have come across a little discrepancy between the
various UNIX streams I was wondering if someone could illuminate.
Maybe. see below....
So cu as of V7 supported the ~$ escape, a means of calling a local
procedure and emitting stdout over the TTY line to the remote machine, all
fine and good for packaging a character stream to emit. However, what I'm
not finding in that age of documentation is any means of requesting std*in*
from the TTY line as input to a local procedure (in essence running a text
filter or handshake-driven protocols over cu). The context in which I'm
researching this is integrating cu into my bare-metal SBC programming using
XMODEM so I can rest a little easier my process is based on tools I'll
probably find in most places.
So old fashioned Mike Lesk-era cu only seems to do stdout redirect, but no
stdin. I did some further digging and it looks like different UUCP
implementations cracked this nut with different escapes, with BSD
eventually going with ~C and Taylor UUCP opting for ~+. Checking the
current illumos manual pages (for a SVR4-ish example) doesn't turn up any
command for this. This is indicative of there never being an agreed-upon
mechanism for doing this, although I could see this being a very useful
mechanism.
maybe -- need to see more of what your session was like. I never remember
missing anything I needed.
What I'm curious about is if the lack of a bi-directional redirect in
early cu is reflective of a lack of need for that sort of functionality at
the time or that matters such as that were handled through a different
mechanism.
I'm not sure I get the question. We did all sorts of redirection and
used/abused cu and its friends all the time. I suspect I'm not
understanding what you are trying to do.
From a history standpoint, cu(1) is just one of many programs in that
family. In the mid/late 1970s, we used a program called 'connect' for
Sixth Edition at CMU, IIRC the Purdue folks had a similar one which was
called attach(1) and there was tip(1) which was from Case/UCB [Sam
Leffler]. If you look in the USENIX archives, I bet you will find a 1/2
doz or so of programs in the ilk before V7. With V7 uucp. was delivered,
so cu(1) began to make inroads as it had the advantage that it was set up
to work on concert to uucico(8). Simply, V7 came out, and UUCP started to
used and eventually the 'USENET' born, cu(1) sort of 'won' because
most of
the other programs tended to conflict with uucico(8) - plus since it was
already there, people did not need something else. But if you had written
one before V7, you often find sites sticking with what they had.
There were a number of UNIX implementations of XMODEM and friends. The C
version of Kermit (ckermit) was quite popular plus has
connect(1)/tip(1)/cu(1) style functionality built into it, but .... IIRC
does not obey the locks that uucico(8) wants so if you used it on TTYs that
had a modem that uucico was trying grab, bad things happened. That said,
in a microprocessor lab where you often dedicated. serial port to 'target'
micro/pc, kermit worked well. My memory is there were also a bunch of two
letter programs, rx/sx and rz/sz and the like. Frankly its been so long
since I had any use for them, I've forgotten. Look in the both USENIX and
the USENET source archives.
Frankly, the last time I think I was trying to do this sort of thing, I was
using Kermit.
YMMV
Clem
ᐧ
2:37 a.m.
Luckily I can toss you my exact use-case with code.
https://gitlab.com/segaloco/riscv-bits/-/blob/master/util/sxj.c
Above is a little utility I spit out to very specifically perform XMODEM-CRC for the
VisionFive RISC-V single-board computer. Word on the street is their recovery bootloader
has a bug that prevents sx and other utilities from working properly. As far as I could
tell, they don't send the standard ACK after each packet receipt, but instead some
other character. I just opted to check for NAK and proceed if not received.
In any case, the above should achieve XMODEM-CRC transmission to the JH7100 RISC-V chip
over UART.
The phenomenon I'm experiencing (and sorry, I'm not trying to turn this into
Taylor UUCP troubleshooting, I promise) is that when using the ~$ escape, my code never
receives the 'C' from the device to initiate XMODEM-CRC. I know it's
reading local stdin still because if I type and submit a 'C', it picks up and
starts transferring, likewise requiring me to submit a non-NAK character every packet.
Otherwise it does work, so the stdout is going to the remote machine, but at least with
~$, the stdin from the serial line does stop showing on my screen, which made me think the
redirect was happening, but it just seems to disappear off into the void (/dev/null?)
while the local program reads local stdin and puts data out on the remote stdout. This
does hold consistent with the manual verbiage for ~$, it only mentions stdout to remote,
doesn't mention any ferrying of remote stdin into the local application being run.
I know my utility works because I've used it in the place of sx in GNU screen and it
works like a charm. Plus it does send to the device if I provide the expected
acknowledgement characters from my local machine in cu.
So this doesn't just turn into troubleshooting, all I hope to highlight here is
getting stdin from the remote machine into a program launched via a ~ escape in cu, be it
like the ~C option in BSD cu or ~+ option in Taylor. My curiosity is whether such a thing
was in historic cu, and, if not, how it might have been accommodated otherwise. As always,
thankful for the insight and feedback folks here provide!
- Matt G.
------- Original Message -------
On Saturday, December 10th, 2022 at 4:22 PM, Clem Cole <clemc(a)ccc.com> wrote:
On Sat, Dec 10, 2022 at 2:39 PM segaloco via TUHS
<tuhs(a)tuhs.org> wrote:
Good morning all. I've been doing some
historical research on the UUCP cu utility this morning and have come across a little
discrepancy between the various UNIX streams I was wondering if someone could illuminate.
Maybe. see below....
So cu as of V7 supported the ~$ escape, a means
of calling a local procedure and emitting stdout over the TTY line to the remote machine,
all fine and good for packaging a character stream to emit. However, what I'm not
finding in that age of documentation is any means of requesting std*in* from the TTY line
as input to a local procedure (in essence running a text filter or handshake-driven
protocols over cu). The context in which I'm researching this is integrating cu into
my bare-metal SBC programming using XMODEM so I can rest a little easier my process is
based on tools I'll probably find in most places.
So old fashioned Mike Lesk-era cu only seems to do stdout redirect, but no stdin. I did
some further digging and it looks like different UUCP implementations cracked this nut
with different escapes, with BSD eventually going with ~C and Taylor UUCP opting for ~+.
Checking the current illumos manual pages (for a SVR4-ish example) doesn't turn up
any command for this. This is indicative of there never being an agreed-upon mechanism for
doing this, although I could see this being a very useful mechanism.
maybe -- need to see more of what your session was like. I never remember missing
anything I needed.
What I'm curious about is if the lack of a
bi-directional redirect in early cu is reflective of a lack of need for that sort of
functionality at the time or that matters such as that were handled through a different
mechanism.
I'm not sure I get the question. We did all sorts of redirection and used/abused cu
and its friends all the time. I suspect I'm not understanding what you are trying to
do.
From a history standpoint, cu(1) is just one of many programs in that family. In the
mid/late 1970s, we used a program called 'connect' for Sixth Edition at CMU,
IIRC the Purdue folks had a similar one which was called attach(1) and there was tip(1)
which was from Case/UCB [Sam Leffler]. If you look in the USENIX archives, I bet you will
find a 1/2 doz or so of programs in the ilk before V7. With V7 uucp. was delivered, so
cu(1) began to make inroads as it had the advantage that it was set up to work on concert
to uucico(8). Simply, V7 came out, and UUCP started to used and eventually the
'USENET' born, cu(1) sort of 'won' because most of the other programs
tended to conflict with uucico(8) - plus since it was already there, people did not need
something else. But if you had written one before V7, you often find sites sticking with
what they had.
There were a number of UNIX implementations of XMODEM and friends. The C version of
Kermit (ckermit) was quite popular plus has connect(1)/tip(1)/cu(1) style functionality
built into it, but .... IIRC does not obey the locks that uucico(8) wants so if you used
it on TTYs that had a modem that uucico was trying grab, bad things happened. That said,
in a microprocessor lab where you often dedicated. serial port to 'target'
micro/pc, kermit worked well. My memory is there were also a bunch of two letter programs,
rx/sx and rz/sz and the like. Frankly its been so long since I had any use for them,
I've forgotten. Look in the both USENIX and the USENET source archives.
Frankly, the last time I think I was trying to do this sort of thing, I was using
Kermit.
YMMV
Clem
ᐧ 
1:59 p.m.
On 10 Dec 2022 19:22 -0500, from clemc(a)ccc.com (Clem Cole):
My memory is there were also a bunch of two
letter programs, rx/sx and rz/sz and the like. Frankly its been so long
since I had any use for them, I've forgotten.
I remember at least sz/rz from my early (for me) forays into UNIX,
back when ZModem was pretty much state of the art at least on micros
and I had files locally that I wanted remotely or vice versa. The
mnenomic being s(end)/r(eceive) z(modem); "send" and "receive", of
course, being local to the remote host, so the opposite sense of what
one would do with the terminal emulator program that one interacted
with locally. So "sz <some file>" at the prompt, then activate the
"receive ZModem transfer" function locally; or "rz <some file>",
then
activate the "send using ZModem" function locally.
I think the host I was on at the time (which appears to have been some
Solaris) also offered XModem and YModem variants as [rs][xy], but I
never used those because someone had at some point told me that ZModem
was better. :-)
--
✍ Michael Kjörling 🏡 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”
2:28 p.m.
On Sun, 11 Dec 2022, Michael Kjörling wrote:
On 10 Dec 2022 19:22 -0500, from clemc(a)ccc.com (Clem
Cole):
Kind-of a pain to type as I'm SSHing in from my phone, but they still
exist and I have "lrzsz" installed on my Debian box.
-uso.
My memory is there were also a bunch of two
letter programs, rx/sx and rz/sz and the like. Frankly its been so long
since I had any use for them, I've forgotten.
I remember at least sz/rz from my early (for me) forays into UNIX,
back when ZModem was pretty much state of the art at least on micros
and I had files locally that I wanted remotely or vice versa. The
mnenomic being s(end)/r(eceive) z(modem); "send" and "receive", of
course, being local to the remote host, so the opposite sense of what
one would do with the terminal emulator program that one interacted
with locally. So "sz <some file>" at the prompt, then activate the
"receive ZModem transfer" function locally; or "rz <some
file>", then
activate the "send using ZModem" function locally.
I think the host I was on at the time (which appears to have been some
Solaris) also offered XModem and YModem variants as [rs][xy], but I
never used those because someone had at some point told me that ZModem
was better. :-)
3:04 p.m.
On Sun, Dec 11, 2022 at 9:28 AM Steve Nickolas <usotsuki(a)buric.co> wrote:
On Sun, 11 Dec 2022, Michael Kjörling wrote:
And as I mentioned earlier, they're still very much used.
I recently wrote the initial bootstrap program that runs when the x86
cores on our machine come out of reset
(https://github.com/oxidecomputer/phbl/). However, that's for
production use; for internal engineering use we have _another_ tool,
also written in Rust, that is an interactive standalone program. This
lets us do things like poke at physical memory, read/write MSRs, and
all of that fun stuff, before loading an operating system. So how does
one load the OS? Our machine has a part that includes a 3
MBAUD-capable UART, and we use xmodem to transfer the kernel (and a
ramdisk image!) over that, where the engineering tool can load and
jump into it. The standalone program implements xmodem internally,
whereas we use `sx` on the dev host side. Basically, there is a
built-in commands to perform the transfer, another that effectively
says "there's an ELF image in physical memory $here: load it and
return the entry point" and finally another command that can jump to
an arbitrary virtual address (yes, we load the host OS in 64-bit mode
and treat the kernel like any old "normal" ELF binary).
As the ramdisk image has grown (more and more debugging tools are
included as we move ever forward in the bringup and development
process), latency here is annoying. I added support for compressed
kernel and ramdisk images, which gave a 2.5x speedup (directly
proportional to the usual compression ratio we see on these images)
which ameliorated but did not eliminate the pain. We briefly tossed
around the idea of implementing zmodem, but decided it wasn't worth
it. We _could_, in theory, make use of the enhanced kermit with
sliding windows and so on, but again decided it wasn't worth the
effort for an engineering tool.
Incidentally, my colleague Matt Keeter ran into performance and
reliability issues with xmodem over the USB serial driver in macOS,
and wrote a small shim that side-stepped the host OS device. He did a
nice write-up of it here:
https://www.mattkeeter.com/blog/2022-05-31-xmodem/
I think this is in line with Larry's question about non-legacy
use-cases for these ancient serial transfer protocols in late 2022.
- Dan C.
On 10 Dec 2022 19:22 -0500, from clemc(a)ccc.com
(Clem Cole):
Kind-of a pain to type as I'm SSHing in from my phone, but they still
exist and I have "lrzsz" installed on my Debian box. My memory is there were also a bunch of two
letter programs, rx/sx and rz/sz and the like. Frankly its been so long
since I had any use for them, I've forgotten.
I remember at least sz/rz from my early (for me) forays into UNIX,
back when ZModem was pretty much state of the art at least on micros
and I had files locally that I wanted remotely or vice versa. The
mnenomic being s(end)/r(eceive) z(modem); "send" and "receive", of
course, being local to the remote host, so the opposite sense of what
one would do with the terminal emulator program that one interacted
with locally. So "sz <some file>" at the prompt, then activate the
"receive ZModem transfer" function locally; or "rz <some
file>", then
activate the "send using ZModem" function locally.
I think the host I was on at the time (which appears to have been some
Solaris) also offered XModem and YModem variants as [rs][xy], but I
never used those because someone had at some point told me that ZModem
was better. :-) 
13 Dec
13 Dec
1:54 a.m.
On Sun, Dec 11, 2022 at 10:04:54AM -0500, Dan Cross wrote:
I think this is in line with Larry's question
about non-legacy
use-cases for these ancient serial transfer protocols in late 2022.
Indeed, this was just what I was looking for. I had no idea that
serial transfer protocols were still useful.
It makes me wish I had kept the source for quicknet, I wrote my
own file transfer and remote terminal for CP/M in the middle
1980s. I did it just because I wanted things to work a certain
way and none of the other options did what I wanted. quicknet was
really pleasant on a 128K Z80 (not really 128K because the screen
was mapped in there, but more than 64K). I spent many a pleasant
hour logged in to uwvax in quicknet, got a lot of work done.
Thanks to you and others who had shown that serial transfer still
matters. Who knew? You did.
--
---
Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat
11 Dec
11 Dec
5:18 p.m.
In the late 1990s I did a brief consulting project to move files between sites. Something
to do with customer billing and record transmission, from some large system that was not
Internet-connected. It ended up being a terrible teetering tower of kermit, sz or sx, and
expect. But it got the job done!
The funny thing about that...the project spanned the July 4th weekend. I didn't get
it working before the 4th. Then when I went in very hungover after the weekend, I
couldn't figure out quite what I had been trying to do, because it was something that
was attempting to be very clever. So in my dulled state, I rewrote the bottom layers to
be much more straightforward. Worked the first time. That taught me the virtue of not
overthinking it.
Adam
6:54 p.m.
On 11 Dec 2022 10:18 -0700, from athornton(a)gmail.com (Adam Thornton):
The funny thing about that...the project spanned the
July 4th
weekend. I didn't get it working before the 4th. Then when I went in
very hungover after the weekend, I couldn't figure out quite what I
had been trying to do, because it was something that was attempting
to be very clever. So in my dulled state, I rewrote the bottom
layers to be much more straightforward. Worked the first time. That
taught me the virtue of not overthinking it.
By definition, if you write code as cleverly as you can, then you
aren't clever enough to debug it...
There's something to be said for simplicity unless for some reason you
really need to be clever. Let's save the clever for the, say, 1% where
it's actually needed; premature application of cleverness is the root
of much evil.
--
✍ Michael Kjörling 🏡 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”
7:55 p.m.
On Sun, 11 Dec 2022, Michael Kjörling wrote:
By definition, if you write code as cleverly as you
can, then you aren't
clever enough to debug it...
Indeed...
I've always used the maxim "Write code as though the next person to
maintain it is an axe-wielding psychopath who knows where you live".
-- Dave
8:03 p.m.
On Mon, Dec 12, 2022 at 06:55:31AM +1100, Dave Horsfall wrote:
On Sun, 11 Dec 2022, Michael Kj??rling wrote:
My main job, for the 20 years until I retired, was to keep telling
people that code that you wrote 6 months ago might as well have been
written by someone else. Optimize for reading the code, not writing
the code. It's read many.
99.9% of the time, I detest clever code. .1% of the time, I need it.
The problem is that smart engineers adore writing clever code. They
usually, eventually, wise up.
By definition, if you write code as cleverly as
you can, then you aren't
clever enough to debug it...
Indeed...
I've always used the maxim "Write code as though the next person to
maintain it is an axe-wielding psychopath who knows where you live".
11:22 p.m.
Readable code also contributes to easier documentation. It's easier to maintain a
manual when you can just translate the code into the local language. My current team
started small and I learned pretty soon that time invested in a habit of readable code
meant less time spent providing the person assigned documentation the juicy details.
- Matt G.
------- Original Message -------
On Sunday, December 11th, 2022 at 12:03 PM, Larry McVoy <lm(a)mcvoy.com> wrote:
On Mon, Dec 12, 2022 at 06:55:31AM +1100, Dave
Horsfall wrote:
On Sun, 11 Dec 2022, Michael Kj??rling wrote:
My main job, for the 20 years until I retired, was to keep telling
people that code that you wrote 6 months ago might as well have been
written by someone else. Optimize for reading the code, not writing
the code. It's read many.
99.9% of the time, I detest clever code. .1% of the time, I need it.
The problem is that smart engineers adore writing clever code. They
usually, eventually, wise up. By definition, if you write code as cleverly as
you can, then you aren't
clever enough to debug it...
Indeed...
I've always used the maxim "Write code as though the next person to
maintain it is an axe-wielding psychopath who knows where you live". 
12 Dec
12 Dec
2:15 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Dec 11, 2022, at 12:04 PM, Larry McVoy <lm(a)mcvoy.com> wrote:
On Mon, Dec 12, 2022 at 06:55:31AM +1100, Dave Horsfall wrote:
Agree that clear code is preferable to complicated code. But in practice
people sacrifice clarity for performance improvement all the time. Look
at the kernel code of any modern os. Everybody pays lip service to this
but most anything other than toy programs ends up getting needlessly
complicated over time. As an example, building "Unix as a service" as
user processes on top of a small microkernel could provide the same
functionality using much clearer and much less code but it would be
slower so we don't do it. Plan9 sort of went in that direction and it
is much simpler (but that could also be because it is not hacked on so
much).
I do prefer clever/smart design to locally clever/smart code. For example,
using Schönhage-Strassen algorithm for multiplying very large numbers.
Or transforming a problem to use a much more efficient data structure
or making equivalent transforms which may be more efficient to compute.
Such code may not be immediately clear but with proper documentation
it is not difficult + you can solve much larger problems. But agreed
these come up much less often.
On Sun, 11 Dec 2022, Michael Kj??rling wrote:
My main job, for the 20 years until I retired, was to keep telling
people that code that you wrote 6 months ago might as well have been
written by someone else. Optimize for reading the code, not writing
the code. It's read many.
99.9% of the time, I detest clever code. .1% of the time, I need it.
The problem is that smart engineers adore writing clever code. They
usually, eventually, wise up. By definition, if you write code as cleverly as
you can, then you aren't
clever enough to debug it...
Indeed...
I've always used the maxim "Write code as though the next person to
maintain it is an axe-wielding psychopath who knows where you live".
2:44 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Sun, 11 Dec 2022, Bakul Shah wrote:
Agree that clear code is preferable to complicated
code. But in practice
people sacrifice clarity for performance improvement all the time. Look
at the kernel code of any modern os. Everybody pays lip service to this
but most anything other than toy programs ends up getting needlessly
complicated over time. As an example, building "Unix as a service" as
user processes on top of a small microkernel could provide the same
functionality using much clearer and much less code but it would be
slower so we don't do it. Plan9 sort of went in that direction and it
is much simpler (but that could also be because it is not hacked on so
much).
I do prefer clever/smart design to locally clever/smart code. For example,
using Schönhage-Strassen algorithm for multiplying very large numbers.
Or transforming a problem to use a much more efficient data structure
or making equivalent transforms which may be more efficient to compute.
Such code may not be immediately clear but with proper documentation
it is not difficult + you can solve much larger problems. But agreed
these come up much less often.
My attitude is: if I'm doing an ugly on nonobvious hack, I'll drop a
comment saying what it does.
Most recently I wrote code that set a "magic flag" - and the comment
explained why the flag was set (so that it would be immediately altered,
forcing a redraw).
-uso.
3:09 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On 12/11/22, Bakul Shah <bakul(a)iitbombay.org> wrote:
Agree that clear code is preferable to complicated code. But in practice
people sacrifice clarity for performance improvement all the time. Look
at the kernel code of any modern os. Everybody pays lip service to this
but most anything other than toy programs ends up getting needlessly
complicated over time. As an example, building "Unix as a service" as
user processes on top of a small microkernel could provide the same
functionality using much clearer and much less code but it would be
slower so we don't do it. Plan9 sort of went in that direction and it
is much simpler (but that could also be because it is not hacked on so
much).
It's not necessarily true that microkernels are significantly slower.
They mostly got that reputation because of Mach and kernels like it
with their heavyweight IPC. Lightweight microkernels like QNX and the
L4 family generally have significantly better performance (in fact,
QNX 4 outperformed SysV/386 back in the 90s on certain benchmarks, and
a proof-of-concept network driver on a current version of seL4 is
significantly faster than a Linux network driver).
3:34 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Sun, Dec 11, 2022 at 08:09:32PM -0700, Andrew Warkentin wrote:
On 12/11/22, Bakul Shah <bakul(a)iitbombay.org>
wrote:
I was friends with one of the 3 people who were allowed to commit to QNX
kernel, Dan Hildebrandt (RIP 1998). Those 3 people actually understood
the "micro" in "microkernel". Early versions of QNX kernel fit in
the 4K instruction cache and left space for user programs. They had
benchmarks that measured cache misses and refused commits that added to
the number of cache misses.
I met Dan because I had 10 terminals on a 80286 with I think 256KB
of memory and we were all editing and compiling and it worked way the
heck better than a 11/780 with 4MB with 30-40 students trying to do the
same thing. I just wanted to talk to the people who made that possible,
it was after I had made a connection to Dennis. The early QNX people
were special. Dan and I had many conversations about the mono kernel
vs micro kernel, it was super fun because we weren't trying to convince
each other, we were comparing approaches.
I've never seen a team like Dan's team that cared as much as they did
about "micro" actually being micro. That's the only way that that
approach works. I can not state that enough, if you want to have a
micro kernel with message passing, if you don't fit in L1 cache,
you are a joke compared to a mono kernel. You have to embrace
micro.
I had hoped that Mach would be like that but it was a giant mess. I know
my opinion is not liked but I so wanted to like Mach, read the papers,
loved it, then read the code, hated it, a few years ago I was working
with the BSD team at Netflix, looked through the VM system from Mach
and it was a mess, it was the exact opposite of my experience at Sun
where I looked and looked and things came in focus and it made sense.
The Mach stuff has never made sense to me, I know Clem says it was a
research platform, OK, but the platform never got to a nice clean place.
If you think of microkernels and think Mach is it, nope, look at QNX.
--
---
Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat
Agree that clear code is preferable to complicated code. But in practice
people sacrifice clarity for performance improvement all the time. Look
at the kernel code of any modern os. Everybody pays lip service to this
but most anything other than toy programs ends up getting needlessly
complicated over time. As an example, building "Unix as a service" as
user processes on top of a small microkernel could provide the same
functionality using much clearer and much less code but it would be
slower so we don't do it. Plan9 sort of went in that direction and it
is much simpler (but that could also be because it is not hacked on so
much).
It's not necessarily true that microkernels are significantly slower.
They mostly got that reputation because of Mach and kernels like it
with their heavyweight IPC. Lightweight microkernels like QNX and the
L4 family generally have significantly better performance (in fact,
QNX 4 outperformed SysV/386 back in the 90s on certain benchmarks, and
a proof-of-concept network driver on a current version of seL4 is
significantly faster than a Linux network driver). 
5 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Sun, Dec 11, 2022 at 8:35 PM Larry McVoy <lm(a)mcvoy.com> wrote:
On Sun, Dec 11, 2022 at 08:09:32PM -0700, Andrew Warkentin wrote:
Mach seems not incompatible to me to Windows NT ideas of layering,
especially in modern macOS guise, more so than a microkernel I'd call
it something like a "macrokernel" with a variety of ways to do
things.. dealer's choice for better and worse. It is interesting to
note the success, often overlooked by more hardline *nix people, that
macOS (and iOS and tvOS and watchOS etc) is indeed mach (more so than
it is BSD as some people claim) and wildly successful.
On 12/11/22, Bakul Shah
<bakul(a)iitbombay.org> wrote:
I was friends with one of the 3 people who were allowed to commit to QNX
kernel, Dan Hildebrandt (RIP 1998). Those 3 people actually understood
the "micro" in "microkernel". Early versions of QNX kernel fit in
the 4K instruction cache and left space for user programs. They had
benchmarks that measured cache misses and refused commits that added to
the number of cache misses.
I met Dan because I had 10 terminals on a 80286 with I think 256KB
of memory and we were all editing and compiling and it worked way the
heck better than a 11/780 with 4MB with 30-40 students trying to do the
same thing. I just wanted to talk to the people who made that possible,
it was after I had made a connection to Dennis. The early QNX people
were special. Dan and I had many conversations about the mono kernel
vs micro kernel, it was super fun because we weren't trying to convince
each other, we were comparing approaches.
I've never seen a team like Dan's team that cared as much as they did
about "micro" actually being micro. That's the only way that that
approach works. I can not state that enough, if you want to have a
micro kernel with message passing, if you don't fit in L1 cache,
you are a joke compared to a mono kernel. You have to embrace
micro.
I had hoped that Mach would be like that but it was a giant mess. I know
my opinion is not liked but I so wanted to like Mach, read the papers,
loved it, then read the code, hated it, a few years ago I was working
with the BSD team at Netflix, looked through the VM system from Mach
and it was a mess, it was the exact opposite of my experience at Sun
where I looked and looked and things came in focus and it made sense.
The Mach stuff has never made sense to me, I know Clem says it was a
research platform, OK, but the platform never got to a nice clean place.
Agree that clear code is preferable to complicated code. But in practice
people sacrifice clarity for performance improvement all the time. Look
at the kernel code of any modern os. Everybody pays lip service to this
but most anything other than toy programs ends up getting needlessly
complicated over time. As an example, building "Unix as a service" as
user processes on top of a small microkernel could provide the same
functionality using much clearer and much less code but it would be
slower so we don't do it. Plan9 sort of went in that direction and it
is much simpler (but that could also be because it is not hacked on so
much).
It's not necessarily true that microkernels are significantly slower.
They mostly got that reputation because of Mach and kernels like it
with their heavyweight IPC. Lightweight microkernels like QNX and the
L4 family generally have significantly better performance (in fact,
QNX 4 outperformed SysV/386 back in the 90s on certain benchmarks, and
a proof-of-concept network driver on a current version of seL4 is
significantly faster than a Linux network driver). If you think of microkernels and think Mach is it,
nope, look at QNX.
--
---
Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat
5:26 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On 12/11/22, Larry McVoy <lm(a)mcvoy.com> wrote:
If you think of microkernels and think Mach is it, nope, look at QNX.
And yet, for some reason, QNX has had almost no influence on anything
else besides a few hobby OSes (only one of which ever reached a
reasonably mature state) as far as I can tell. I really don't get why
that is. As far as I know, I'm the only one who's working on a
QNX-like OS that's not QNX itself.
3:02 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Sun, Dec 11, 2022 at 10:26:37PM -0700, Andrew Warkentin wrote:
On 12/11/22, Larry McVoy <lm(a)mcvoy.com> wrote:
It influenced me.
Lots of things that were the right answer failed to achieve success.
Beta vs VHS. Plan 9.
Does that mean they were worthless? I don't think so.
If you think of microkernels and think Mach is it, nope, look at QNX.
And yet, for some reason, QNX has had almost no influence on anything
else besides a few hobby OSes (only one of which ever reached a
reasonably mature state) as far as I can tell. I really don't get why
that is. As far as I know, I'm the only one who's working on a
QNX-like OS that's not QNX itself.
3:29 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin <andreww591(a)gmail.com>
wrote:
And yet, for some reason, QNX has had almost no
influence on anything
Be careful with a statement like that. It's likely running in something
in your car. and very likely to be running in something in the last Boeing
or Airbus-based flight you took, and it was used when Amazon made the last
delivery to you. It has long been popular in process control/materials
handling/robotics/fly-by-wire systems.
When a small, very lightweight UNIX-style programming API needed to be
used, QNX was often a favorite.
I sometimes think QNX must have had a really good salesperson in the
'Rust-Belt.' I know I talked to several fans in companies doing that
work. I do know of a least one firm that still uses it. An inexpensive
x86 can be designed into a custom controller, and the only 'development' is
the customer interface to private HW. The development system is a PC or
Vmware on an engineer's desk.
After Blackberry bought the company, it's interesting that they seem to be
all that is left of BB. But they are still going strong: QNX Neutrino RTOS
<https://streaklinks.com/BUDwcGeMTFsTSXcFxQzjvNX5/https%3A%2F%2Fwww.qnx.com%2Fproducts%2Fintl%2Fneutrino_rtos%2F>
Funny, I loved Mach (still do), but it tried to be all things to all
people. The QNX guys did not. I also think it help in their success --
by the fact that the QNX folks concentrated on RT, while Mach tried to be
the replacement for all of BSD. They both have their place ... I'm
typing this on my macOS 13.0.1 (Ventura) M1 system, which is just the
current flavor of Mach. As Tru64 hacker, as well as one of the folks that
work on Intel Paragon, which was OSF/1, all three are Mach based.
I also did some work with QNX back in the day and, like Larry, was always
very impressed. At one time, I did some consulting in the Rust-Belt, and
the executive (*i.e.* -- Havard Business. School types) asked me if *"this QNX
thing their engineers were using -- after all it was not from Microsoft,
IBM or DEC, of course.''* [they had converted/were in the process of
converting from DEC PMAX-based controllers running Ultrix to PCs running
QNX]. My analysis at the time, for a bunch of ex-Fortran Mech E's, had
done extremely well. I told the execs then that this is good stuff; it's
going to save them buckets of money as it 'just works' (that was the core
SW in the automatic 'sorter' that at the time was being done under contract
for Amazon -- I know the CEO of that firm and they sold the same basic
system to UPS/FedEx/USPS -- they used to have a very cool movie taken
during the testing at FedEx with glasses full of champagne moving at 45
miles through the sorter without spilling -- the PMAX would never have been
able to do that).
Frankly, for anyone learning either about microkernels or RT, I would
certainly tell them to look at QNX. Neither topic are what we call
'research' projects as much these days, but both have extremely practical
applications.
ᐧ
3:39 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Mon, Dec 12, 2022 at 10:30 AM Clem Cole <clemc(a)ccc.com> wrote:
On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin <andreww591(a)gmail.com> wrote:
For a while, Neutrino was open source, but that seemed to change
quietly at some point. I can't seem to find the source code online
anywhere, which surprises me a bit: I'd have assumed anyone who got a
copy of the code under the OSS license would still be able to treat it
as open source, but maybe I'm just wrong.
- Dan C.
And yet, for some reason, QNX has had almost no influence on anything
Be careful with a statement like that. It's likely running in something in your
car. and very likely to be running in something in the last Boeing or Airbus-based flight
you took, and it was used when Amazon made the last delivery to you. It has long been
popular in process control/materials handling/robotics/fly-by-wire systems.
When a small, very lightweight UNIX-style programming API needed to be used, QNX was
often a favorite.
I sometimes think QNX must have had a really good salesperson in the
'Rust-Belt.' I know I talked to several fans in companies doing that work. I
do know of a least one firm that still uses it. An inexpensive x86 can be designed into a
custom controller, and the only 'development' is the customer interface to
private HW. The development system is a PC or Vmware on an engineer's desk.
After Blackberry bought the company, it's interesting that they seem to be all that
is left of BB. But they are still going strong: QNX Neutrino RTOS
4:04 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Mon, Dec 12, 2022 at 10:29:04AM -0500, Clem Cole wrote:
On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin
<andreww591(a)gmail.com>
wrote:
Thanks for that Clem. One question though, all those companies want
support. Is there anyone still providing support? I see QNX.com is
a thing but is there an actual team of good people working on it?
And yet, for some reason, QNX has had almost no
influence on anything
Be careful with a statement like that. It's likely running in something
in your car. and very likely to be running in something in the last Boeing
or Airbus-based flight you took, and it was used when Amazon made the last
delivery to you. It has long been popular in process control/materials
handling/robotics/fly-by-wire systems.
When a small, very lightweight UNIX-style programming API needed to be
used, QNX was often a favorite.
4:26 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
I don't know. I'll try to poke and ask. The engineer I used to work with
at the Mill and later Intelligrated has retired (a lot of that with our
generation). But I still know some of the execs at the former
Intelligrated (now part of Honeywell), so I'll see if I can find out who
owns that product these days. If I learn anything interesting, I'll pass
it back.
FWIW: BB is offering support: i.e.: https://blackberry.qnx.com/en/support
claims such, but what does that mean?
Clem
ᐧ
On Mon, Dec 12, 2022 at 11:04 AM Larry McVoy <lm(a)mcvoy.com> wrote:
On Mon, Dec 12, 2022 at 10:29:04AM -0500, Clem Cole
wrote:
On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin
<andreww591(a)gmail.com>
wrote:
Boeing
And yet, for some reason, QNX has had almost no
influence on anything
Be careful with a statement like that. It's likely running in something
in your car. and very likely to be running in something in the last or Airbus-based flight you took, and it was used
when Amazon made the
last
delivery to you. It has long been popular in
process control/materials
handling/robotics/fly-by-wire systems.
When a small, very lightweight UNIX-style programming API needed to be
used, QNX was often a favorite.
Thanks for that Clem. One question though, all those companies want
support. Is there anyone still providing support? I see QNX.com is
a thing but is there an actual team of good people working on it?
10:20 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Mon, 12 Dec 2022 at 17:04, Larry McVoy <lm(a)mcvoy.com> wrote:
Thanks for that Clem. One question though, all those companies want
support. Is there anyone still providing support? I see QNX.com is
a thing but is there an actual team of good people working on it?
QNX is alive and well and owned by Blackberry Ltd, the former Research
in Motion. One big Canadian tech co bought another.
They aren't doing _great_ but they're not dead by any means.
https://en.wikipedia.org/wiki/BlackBerry_Limited
--
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven(a)cix.co.uk ~ gMail/gTalk/FB: lproven(a)gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053
11:10 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
Well this has certainly piqued my curiosity on QNX. One of my more long-term projects is
to write a kernel (micro or otherwise), initially for RISC-V that can, if nothing else,
serve as a springboard for my own projects. I've thus far just been using AT&T
and BSD UNIX kernels for research and inspiration, with a peek at Darwin every now and
then to see how Apple/Mach does things, but could certainly stand to broaden my horizons.
That said, if I was to straight up port something as an exercise, I'd probably go for
V7 given its ubiquity and the fact that work done there would likely serve as a good
template for doing the same with other AT&T and BSD variants. Plus, I think
someone's already gone and done a V6 port to RISC-V, so I'd have stuff to refer
to.
- Matt G.
------- Original Message -------
On Monday, December 12th, 2022 at 2:20 PM, Liam Proven <lproven(a)gmail.com> wrote:
On Mon, 12 Dec 2022 at 17:04, Larry McVoy lm(a)mcvoy.com
wrote:
Thanks for that Clem. One question though, all
those companies want
support. Is there anyone still providing support? I see QNX.com is
a thing but is there an actual team of good people working on it?
QNX is alive and well and owned by Blackberry Ltd, the former Research
in Motion. One big Canadian tech co bought another.
They aren't doing great but they're not dead by any means.
https://en.wikipedia.org/wiki/BlackBerry_Limited
--
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven(a)cix.co.uk ~ gMail/gTalk/FB: lproven(a)gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053
11:24 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
It's been decades since I've used it, my comments are based on the QNX that
predated their POSIX conformance. It was extremely light weight, fast,
and used as little memory as possible. I literally had 10 active logins
(tty) with people editing and compiling on a 256KB 80286. Nothing else
came close. But it wasn't POSIX so it might be more bloated today.
On Mon, Dec 12, 2022 at 11:10:08PM +0000, segaloco via TUHS wrote:
Well this has certainly piqued my curiosity on QNX.
One of my more long-term projects is to write a kernel (micro or otherwise), initially for
RISC-V that can, if nothing else, serve as a springboard for my own projects. I've
thus far just been using AT&T and BSD UNIX kernels for research and inspiration, with
a peek at Darwin every now and then to see how Apple/Mach does things, but could certainly
stand to broaden my horizons. That said, if I was to straight up port something as an
exercise, I'd probably go for V7 given its ubiquity and the fact that work done there
would likely serve as a good template for doing the same with other AT&T and BSD
variants. Plus, I think someone's already gone and done a V6 port to RISC-V, so
I'd have stuff to refer to.
- Matt G.
------- Original Message -------
On Monday, December 12th, 2022 at 2:20 PM, Liam Proven <lproven(a)gmail.com> wrote:
> On Mon, 12 Dec 2022 at 17:04, Larry McVoy lm(a)mcvoy.com wrote:
>
> > Thanks for that Clem. One question though, all those companies want
> > support. Is there anyone still providing support? I see QNX.com is
> > a thing but is there an actual team of good people working on it?
>
>
> QNX is alive and well and owned by Blackberry Ltd, the former Research
> in Motion. One big Canadian tech co bought another.
>
> They aren't doing great but they're not dead by any means.
>
> https://en.wikipedia.org/wiki/BlackBerry_Limited
>
> --
> Liam Proven ~ Profile: https://about.me/liamproven
> Email: lproven(a)cix.co.uk ~ gMail/gTalk/FB: lproven(a)gmail.com
> Twitter/LinkedIn: lproven ~ Skype: liamproven
> UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053
--
---
Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat
13 Dec
13 Dec
2 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On 12/12/22, Clem Cole <clemc(a)ccc.com> wrote:
On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin
<andreww591(a)gmail.com>
wrote:
I'm well aware that QNX has been extremely successful commercially and
can be found in a wide range of embedded systems. I'm specifically
talking about architectural influence on other OSes.
The only OSes with significant QNX influence to reach anything
resembling a mature state of which I am aware are VSTa
<https://vsta.org/> and WiNGs <http://wingsos.org/>, both of which
have been abandoned for quite a while. Besides those two, there was
also OpenBLT, which was able to run a simple shell and a few basic
utilities from its boot image but not much more, and RadiOS, which was
abandoned at a point where it couldn't run much more than a hello
world. I'm also working on my own QNX-like OS
<https://gitlab.com/uxrt/uxrt-toplevel> like I said earlier, although
it doesn't run user programs yet (I'm working on the VFS and IPC
transport layer at the moment).
The extensive commercial success of QNX makes it even more surprising
to me that it has had so little architectural influence. Similarly, I
don't get why people who bash microkernels always seem to think that
all of them are like Mach despite QNX being quite successful.
On 12/12/22, Larry McVoy <lm(a)mcvoy.com> wrote:
And yet, for some reason, QNX has had almost no
influence on anything
Be careful with a statement like that. It's likely running in something
in your car. and very likely to be running in something in the last Boeing
or Airbus-based flight you took, and it was used when Amazon made the last
delivery to you. It has long been popular in process control/materials
handling/robotics/fly-by-wire systems.
It's been decades since I've used it, my
comments are based on the QNX that
predated their POSIX conformance. It was extremely light weight, fast,
and used as little memory as possible. I literally had 10 active logins
(tty) with people editing and compiling on a 256KB 80286. Nothing else
came close. But it wasn't POSIX so it might be more bloated today.
QNX 4 and Neutrino are heavier than QNX "Classic", but they're still
fairly lightweight. QNX 4 had the fairly well-known demo disk with a
Photon desktop and browser on a single 1.4M floppy. Neutrino never had
anything quite like that (the smallest official live CD images of the
early versions are something like 60M in size) but it's still possible
to build images with it that are pretty small.
1:37 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Mon, Dec 12, 2022 at 07:00:34PM -0700, Andrew Warkentin wrote:
On 12/12/22, Clem Cole <clemc(a)ccc.com> wrote:
Minix? QNX predated that.
On Mon, Dec 12, 2022 at 12:27 AM Andrew Warkentin
<andreww591(a)gmail.com>
wrote:
I'm well aware that QNX has been extremely successful commercially and
can be found in a wide range of embedded systems. I'm specifically
talking about architectural influence on other OSes. And yet, for some reason, QNX has had almost no
influence on anything
Be careful with a statement like that. It's likely running in something
in your car. and very likely to be running in something in the last Boeing
or Airbus-based flight you took, and it was used when Amazon made the last
delivery to you. It has long been popular in process control/materials
handling/robotics/fly-by-wire systems.
11 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On 12/13/22, Larry McVoy <lm(a)mcvoy.com> wrote:
Yes, QNX predated Minix by several years, but Minix was completely
independent and there was no QNX influence on it at all AFAIK.
I'm well aware that QNX has been extremely successful commercially and
can be found in a wide range of embedded systems. I'm specifically
talking about architectural influence on other OSes.
Minix? QNX predated that.
14 Dec
14 Dec
1:05 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Tue, Dec 13, 2022 at 04:00:46PM -0700, Andrew Warkentin wrote:
On 12/13/22, Larry McVoy <lm(a)mcvoy.com> wrote:
Yes, QNX predated Minix by several years, but Minix was completely
independent and there was no QNX influence on it at all AFAIK.
Have you talked to Andy and confirmed that? I'd be quite surprised if
he hadn't played with QNX but who knows. I wouldn't assume he hadn't.
And forgive me for asking, do you have some axe to grind against QNX
or something?
To me, it's not that surprising that the rest of the world didn't copy
QNX because the rest of the world was either a mono-kernel or it was
Mach. Don't get me started on Mach, it has defenders but I absolutely
hate it. Mach is more of a distributed research OS that advertised
itself as a microkernel. There is _nothing_ micro about Mach. It's
a big bloated mess.
--
---
Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat
I'm well aware that QNX has been extremely successful commercially and
can be found in a wide range of embedded systems. I'm specifically
talking about architectural influence on other OSes.
Minix? QNX predated that.
1:40 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
Has anything material come of later Mach work other than knowledge and influence on other
systems? I don't have a complete picture but it's my understanding that there
were plenty of developments after NeXT and OSF had their way with it that aren't
reflected in, say, macOS. Something about said OSs being forked before Mach 3, which is
listed in some references as being more true to the micro kernel architecture.
As an aside, I wish Apple would actually engage with their "open source" nature
more. I'd love to plink around in the Darwin kernel but even with Google-fu have
never managed to actually get a kernel build all the way through. Their lack of
documentation is a painful matter. I'd love to bootstrap it on RISC-V if I could get
there...
- Matt G.
------- Original Message -------
On Tuesday, December 13th, 2022 at 5:05 PM, Larry McVoy <lm(a)mcvoy.com> wrote:
On Tue, Dec 13, 2022 at 04:00:46PM -0700, Andrew
Warkentin wrote:
On 12/13/22, Larry McVoy lm(a)mcvoy.com wrote:
Yes, QNX predated Minix by several years, but Minix was completely
independent and there was no QNX influence on it at all AFAIK.
Have you talked to Andy and confirmed that? I'd be quite surprised if
he hadn't played with QNX but who knows. I wouldn't assume he hadn't.
And forgive me for asking, do you have some axe to grind against QNX
or something?
To me, it's not that surprising that the rest of the world didn't copy
QNX because the rest of the world was either a mono-kernel or it was
Mach. Don't get me started on Mach, it has defenders but I absolutely
hate it. Mach is more of a distributed research OS that advertised
itself as a microkernel. There is nothing micro about Mach. It's
a big bloated mess.
--
---
Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat I'm
well aware that QNX has been extremely successful commercially and
can be found in a wide range of embedded systems. I'm specifically
talking about architectural influence on other OSes.
Minix? QNX predated that. 
6:32 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Dec 13, 2022, at 17:40, segaloco via TUHS
<tuhs(a)tuhs.org> wrote:
... I wish Apple would actually engage with their "open source" nature more.
I'd love to plink around in the Darwin kernel but even with Google-fu have never
managed to actually get a kernel build all the way through. Their lack of documentation
is a painful matter. I'd love to bootstrap it on RISC-V if I could get there...
You might want to take a look at http://www.puredarwin.org/ ...
-r
2:01 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On 12/13/22, Larry McVoy <lm(a)mcvoy.com> wrote:
Have you talked to Andy and confirmed that? I'd be quite surprised if
he hadn't played with QNX but who knows. I wouldn't assume he hadn't.
I haven't actually talked to him about it. He definitely is aware of
QNX since he's mentioned it on a few occasions, but I'm not sure if he
was aware of it when he wrote the first version of Minix.
Personally I don't see a lot of resemblance between the two, besides
both being single-personality Unix-like microkernel OSes with
lightweight IPC. Minix is more akin to a "serverized" conventional
Unix, whereas QNX seems to embrace its microkernel-ness more fully
with its focus on extensibility and its fairly tight integration of
IPC transport layer and filesystem. There may have been a little bit
of influence, but it's not all that obvious to me.
The pre-3.x versions seem especially un-QNX-like with their more or
less closed set of servers. Even in 3.x, the kernel still seems to
have quite a bit of knowledge about what servers are present and what
messages they accept. QNX does colocate the process server in the
kernel, but it makes very few assumptions about user-mode servers.
And forgive me for asking, do you have some axe to grind against QNX
or something?
Quite the opposite, hence why I'm writing my own OS with a similar architecture.
To me, it's not that surprising that the rest of the world didn't copy
QNX because the rest of the world was either a mono-kernel or it was
Mach. Don't get me started on Mach, it has defenders but I absolutely
hate it. Mach is more of a distributed research OS that advertised
itself as a microkernel. There is _nothing_ micro about Mach. It's
a big bloated mess.
Yes, I agree 100% that Mach is a complete and utter failure as a
microkernel, and seems to have almost single-handedly destroyed the
reputation of microkernels. I don't get why everyone was so focused on
Mach-like kernels when there was a better alternative that had been
around in some form for almost a decade before Mach (QNX wasn't the
first of its kind; it seems to have had pretty significant influence
from Thoth).
7:49 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
Andrew Warkentin <andreww591(a)gmail.com> wrote:
Yes, I agree 100% that Mach is a complete and utter
failure as a
microkernel, and seems to have almost single-handedly destroyed the
reputation of microkernels. I don't get why everyone was so focused on
Mach-like kernels when there was a better alternative that had been
around in some form for almost a decade before Mach (QNX wasn't the
first of its kind; it seems to have had pretty significant influence
from Thoth).
I suspect because Mach was available if you had the right Unix licenses
and because it was hot in the research world in the mid 80s. Researchy
types tend to look at what other researchers are doing / using, it seems
to me often without knowledge of or caring about what people are using
in industry. (My two cents, from having worked at universities.)
Arnold
11:54 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
arnold(a)skeeve.com writes:
Andrew Warkentin <andreww591(a)gmail.com> wrote:
In that time frame there was a number of microkernel designs. One that
has not been mentioned was OS-9 for the 6809/68000 processor. I used it
pretty extensively. OS-9 was very unix like from the userland POV, when
you consider something like V5 unix, however it didn't share any of the
same command names, just many of the same concepts. It was close enough
that if you had the C compiler, a very basic K&R compiler, you could get
some of the unix command to compile without too much trouble. I ported
sed from the DEC user's group source and the termcap library from BSD
and created a varargs library for it. OS-9 was very microkernel and
nothing like Mach or even Minix. It was also very much positioned to
real time OS needs of the time and was not really marketed generally and
unless you happened to have a Color Computer from Radio Shack or was a
part of the nitch they served you would probably have never come across
it and it would have seemed to be expensive to acquire. It was very
clean, but you needed to know 6809 or 68000 assembly to create anything
new for the OS itself, although at least one person had figured out how
to use the C compiler, sort of, to produce assembly that could be
assembled into device managers and device drivers.
--
Brad Spencer - brad(a)anduin.eldar.org - KC8VKS - http://anduin.eldar.org
Yes, I agree 100% that Mach is a complete and
utter failure as a
microkernel, and seems to have almost single-handedly destroyed the
reputation of microkernels. I don't get why everyone was so focused on
Mach-like kernels when there was a better alternative that had been
around in some form for almost a decade before Mach (QNX wasn't the
first of its kind; it seems to have had pretty significant influence
from Thoth).
I suspect because Mach was available if you had the right Unix licenses
and because it was hot in the research world in the mid 80s. Researchy
types tend to look at what other researchers are doing / using, it seems
to me often without knowledge of or caring about what people are using
in industry. (My two cents, from having worked at universities.)
Arnold
12:08 p.m.
New subject: (TUHS -> COFF?) Re: Clever code
On 14 Dec 2022 06:54 -0500, from brad(a)anduin.eldar.org (Brad Spencer):
[...] but you needed to know 6809 or 68000 assembly to
create anything
new for the OS itself,
Wasn't that the norm at the time, though? As I recall one of the
things that really set UNIX apart from other operating systems up
until about the early 1990s was precisely how machine-independent it
was by virtue of (with the exception of the early versions) having
been written in something other than assembler.
--
✍ Michael Kjörling 🏡 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”
3:14 p.m.
New subject: Microware's OS-9 (was: Clever code)
At 2022-12-14T06:54:04-0500, Brad Spencer wrote:
arnold(a)skeeve.com writes:
> I suspect because Mach was available if you had the right Unix
> licenses and because it was hot in the research world in the mid
> 80s. Researchy types tend to look at what other researchers are
> doing / using, it seems to me often without knowledge of or caring
> about what people are using in industry. (My two cents, from having
> worked at universities.)
The UNSW CSE department seemed to be a bit more outward facing than
that, at least in my brief exposure to it, long after the 1980s.
In that time frame there was a number of microkernel
designs. One
that has not been mentioned was OS-9 for the 6809/68000 processor. I
used it pretty extensively. OS-9 was very unix like from the userland
POV, when you consider something like V5 unix, however it didn't share
any of the same command names, just many of the same concepts.
This is emphatically true. I used this system as a kid on a 64KiB
machine, and I don't remember even a mention of Unix in the doorstop of
a manual by Dale Puckett and Peter Dibble (who gave you something like 6
chapters of architectural background before introducing the shell
prompt). Maybe they did mention Unix , but since it had no meaning to
me at the time, it didn't sink in. I think it is also possible they
avoided any names that they thought might draw legal ire from AT&T.
It was close enough that if you had the C compiler, a
very basic K&R
compiler, you could get some of the unix command to compile without
too much trouble.
Years later I went to college, landed on Sun IPC workstations, and
quickly recognized OS-9's "T/S Edit" as a vi clone, and its "T/S
Word"
as a version of nroff. There was also a "T/S Spell" product but I don't
recall it clearly enough to venture whether it was a clone of ispell.
OS-9 was very microkernel
In that deployment environment, it had to be.
and nothing like Mach or even Minix.
With the source of all three available, a technical paper analyzing and
contrasting them would be a worthwhile thing to have. (It's unclear to
me if even a historical version of QNX is available for study.)
It was also very much positioned to real time OS needs
of the time and
was not really marketed generally and unless you happened to have a
Color Computer from Radio Shack
Lucky me! How I yearned for a 128KiB Color Computer 3 so I could
upgrade to OS-9 Level 2 and the windowing system. (512KiB was
preferred, but there had been a spike in RAM prices right about the time
the machine was released. Not that greater market success would have
kept Tandy from under-promoting and eventually killing the machine.[1])
It was very clean, but you needed to know 6809 or
68000 assembly to
create anything new for the OS itself,
The 6809 was my first exposure to a (relatively) clean ISA design,
having come from the Z80. It probably helped that I was born with a
big-endian head and thus had an instinctive revulsion to Intel byte
order at an extremely young age.
In the late 1990s, Apple decided they wanted to rebrand their operating
system (still "MacOS [Classic]" at the time), looked at Microware's
name for its system, and said, "right, we'll be having that".
Microware, having apparently so carefully followed the letter of
trademark law with respect to AT&T Unix, sued Apple for peddling "OS/9"
in the operating system market, and promptly got their asses handed to
them by the federal district court, which dutifully honored the foremost
principle of law: big people get to stomp smaller people as often, and
as hard, as they would like.[2]
(Later, apparently, Apple pointed out this precedent to Cisco with a
shark-toothed grin when Apple decided they wanted the name "iOS" for yet
another revitalizing rebrand of familiar technology. Cisco rolled over
and took some undisclosed amount of money, which they promptly spent on
acquisitions--they then were suddenly startled by the proportion of op
ex going to salaries, and initiated layoffs.)
Apple's never changed its stripes, but OS-9 lives on, as Free Software,
under the name NitrOS-9.[4]
Regards,
Branden
[1] Here's a story you may have to sit down for from Frank Durda IV (now
deceased) about how the same company knifed their m68k-based
line--which ran XENIX--in the gut repeatedly. It's hard to find
this story via Web search so I've made a Facebook post
temporarily(?) public. I'd simply include it, but it's pretty long.
https://www.facebook.com/g.branden.robinson/posts/pfbid0F8MrvauQ6KPQ1tytme9…
[2] https://www.cnn.com/2000/TECH/computing/03/21/os9.suit.idg/index.html
[3]
https://appleinsider.com/articles/10/06/08/cisco_licenses_ios_name_to_apple…
[4] https://sourceforge.net/projects/nitros9/
10:41 p.m.
New subject: Microware's OS-9 (was: Clever code)
On Wed, Dec 14, 2022 at 10:15 AM G. Branden Robinson <
g.branden.robinson(a)gmail.com> wrote:
Microware, having apparently so carefully followed the letter of
trademark law with respect to AT&T Unix, sued
Apple for peddling "OS/9"
in the operating system market, and promptly got their asses handed to
them by the federal district court, which dutifully honored the foremost
principle of law: big people get to stomp smaller people as often, and
as hard, as they would like.[2]
To be fair, the judge decided that there wasn't a whole lot of risk of
consumer confusion between a consumer OS that ran only on Apple Macintoshes
and an OEM OS that did not. The only actual victims were Mac people who
stumbled into comp.os.os9 and got seriously confused.
In any case, Microsoft v. Lindows pretty much established that sometimes
the little guy wins, even in trademark cases. Microsoft claimed that
Lindows was infringing their trademark for Windows (it was a Linux distro
that came with Wine and some glue) and lost the case on prior-use grounds
(both Xerox and Apple). So rather than risking all on a retrial and maybe
losing the Windows trademark altogether, they settled for US$20M and
Lindows changed its corporate and distro names to Linspire.
That said, "Windows" is a descriptive trademark, and those are always shaky
legally.
Cisco rolled over
and took some undisclosed amount of money,
Money undoubtedly flowed from Apple Computers to Apple Records when the
first company started selling music, too. This sort of thing is routine.
While I was working at Chase Bank, they paid a small fortune to Chase
Research for the rights to "chase.com".
9:46 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
Larry McVoy [14/12/2022 02.05]:
To me, it's not that surprising that the rest of
the world didn't copy
QNX because the rest of the world was either a mono-kernel or it was
Mach. Don't get me started on Mach, it has defenders but I absolutely
hate it. Mach is more of a distributed research OS that advertised
itself as a microkernel. There is_nothing_ micro about Mach. It's
a big bloated mess.
There was a rumour back in the early 90s that a new version of AmigaOS
would be based on QNX. This was just before Commodore went bust, so
nothing came out of it. A shame, I rather liked the Amiga and its
so-called Workbench.
--
Hilsen Harald
15 Dec
15 Dec
6:33 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Wed, 14 Dec 2022 at 10:47, Harald Arnesen <skogtun(a)gmail.com> wrote:
There was a rumour back in the early 90s that a new version of AmigaOS
would be based on QNX.
Oh, it wasn't a rumour. The deal got quite advanced.
My now-employers covered it at the time:
https://www.theregister.com/1998/11/14/amiga_2_to_use_qnx/
This was just before Commodore went bust, so
nothing came out of it.
It was after. By this point it was Gateway Inc paying for it. There's
an account of what happened here:
https://www.trollaxor.com/2005/06/how-qnx-failed-amiga.html
The next-gen Amiga project was pretty much why QNX gained its GUI, Neutrino:
https://guidebookgallery.org/screenshots/qnx621
This is also the basis of the famous QNX Demo Disk, build by the late
great Dan Hildebrandt:
http://qnx.puslapiai.lt/qnxdemo/qnx_demo_disk.htm
A shame, I rather liked the Amiga and its
so-called Workbench.
It was a good OS in its time and its admirers often call it a
microkernel, but IMVHO if all the code is in the same memory space,
that's not really a true microkernel.
Amiga Inc then went on to nearly do a deal with Tao Group for the much
more advanced Taos, in its later v2 incarnation as Intent/Elate.
https://wiki.c2.com/?TaoIntentOs
https://www.osnews.com/story/157/tao-group-on-elateos-amigade-and-more/
Ars has some info on this nearly-forgotten OS:
https://arstechnica.com/gadgets/2018/03/a-history-of-the-amiga-part-12-red-…
--
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven(a)cix.co.uk ~ gMail/gTalk/FB: lproven(a)gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053
16 Dec
16 Dec
10:42 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
Liam Proven [15/12/2022 19.33]:
On Wed, 14 Dec 2022 at 10:47, Harald
Arnesen<skogtun(a)gmail.com> wrote:
Thanks! Very interesting, I didn't know much of this.
--
Hilsen Harald
There was a rumour back in the early 90s that a
new version of AmigaOS
would be based on QNX.
Oh, it wasn't a rumour. The deal got quite advanced.
My now-employers covered it at the time:
https://www.theregister.com/1998/11/14/amiga_2_to_use_qnx/
18 Dec
18 Dec
2:05 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Fri, 16 Dec 2022 at 11:43, Harald Arnesen <skogtun(a)gmail.com> wrote:
Thanks! Very interesting, I didn't know much of this.
My pleasure.
I am a daily Unix user but my main interests tend to lie elsewhere.
I don't have any citations for this, but it looked to me, watching in
the magazines at the time, that _before_ the Amiga deal, QNX did run
on PCs for testing purposes, but I am not totally sure if it had a GUI
at all, and little to no multimedia support.
My impression is that QNX implemented that for Amiga Inc and then were
left with it when Amiga turned its gaze on Tao and Elate.
(Some of the best remaining info about Intent and Elate is in HN comments, e.g.
https://news.ycombinator.com/item?id=16053726
https://news.ycombinator.com/item?id=9807269
Which leads to:
http://www.uruk.org/emu/Taos.html )
I am curious to know if the TAOS virtual-processor,
all-binaries-are-CPU-independent, model influenced or inspired
Inferno. I think that Inferno postdates TAOS.
But returning to QNX: my impression is, they implemented a GUI and
multimedia frameworks for Amiga, Amiga changed its mind, and QNX
offered it as a PC dev kit for a while.
E.g.
https://archive.org/details/qnx-neutrino-rtos-x86-runtime-kit-6.3.0-sp3
But the only mass-market end-user-facing graphical multimedia-capable
QNX devices I know of were the Blackberry X smartphones. (And
cancelled tablet and netbook.)
I owned a Blackberry Passport. A lovely device with a lovely OS... but
too late and it flopped.
So, oddly, and accidentally and unintentionally, Amiga Inc was
directly responsible for the Blackberry smartphone OS.
--
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven(a)cix.co.uk ~ gMail/gTalk/FB: lproven(a)gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053
3:08 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On 2022-12-18 09:05, Liam Proven wrote (in part):
[...]
But returning to QNX: my impression is, they
implemented a GUI and
multimedia frameworks for Amiga, Amiga changed its mind, and QNX
offered it as a PC dev kit for a while.
E.g.
https://archive.org/details/qnx-neutrino-rtos-x86-runtime-kit-6.3.0-sp3
QNX initially ran only on x86 kit. I do not recall a GUI at that time.
But the only mass-market end-user-facing graphical
multimedia-capable
QNX devices I know of were the Blackberry X smartphones. (And
cancelled tablet and netbook.)
The BB10 i/f was designed by TAT
(https://www.engadget.com/2010-12-02-rim-buys-tat-blackberry-ui-in-danger-of…).
I owned a Blackberry Passport. A lovely device with a
lovely OS... but
too late and it flopped.
Quite so.
N.
19 Dec
19 Dec
11:47 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Sun, 18 Dec 2022 at 16:08, Stuff Received <stuff(a)riddermarkfarm.ca> wrote:
One of the slightly stranger things in the smartphone OS space
recently, for me, has been watching Huawei's response to the licensing
fight with Google.
Huawei now promotes its devices as running "Harmony OS" with a lot of
hot air about microkernels, convergence, cross-platform, blah blah.
It's nothing of the kind. It's AOSP, the open-source core of Android,
with their own skin on top.
I don't think Huawei has the tech skills to build a new OS as it's
claimed it has...
But it could make some of those claims come true if it licensed BB10
and spent a bit on updating it.
--
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven(a)cix.co.uk ~ gMail/gTalk/FB: lproven(a)gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053
The BB10 i/f was designed by TAT
(https://www.engadget.com/2010-12-02-rim-buys-tat-blackberry-ui-in-danger-of…).
Oh, fascinating. I did not know that -- thank you!
I owned a
Blackberry Passport. A lovely device with a lovely OS... but
too late and it flopped.
Quite so.
20 Dec
20 Dec
8:30 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On 12/18/22, Liam Proven <lproven(a)gmail.com> wrote:
I don't have any citations for this, but it looked to me, watching in
the magazines at the time, that _before_ the Amiga deal, QNX did run
on PCs for testing purposes, but I am not totally sure if it had a GUI
at all, and little to no multimedia support.
My impression is that QNX implemented that for Amiga Inc and then were
left with it when Amiga turned its gaze on Tao and Elate.
QNX Classic and 4.x only ran on x86 machines, most of which were
either standard PCs or at least sort of PC-like (although AFAIK the
ability to run without a BIOS was present very early on). It was quite
common to run QNX on desktops as a development host for embedded
systems AFAIK.
GUIs for QNX predate the Amiga deal, and have existed since the late
80s. The original was QNX Windows, which was either a reimplementation
or port (not quite sure which) of Open Look on a custom non-X11 window
server, running on later versions of 2.x and all versions of 4.x.
Later versions of 4.x added Photon 1, which looks like a cross between
Motif and Windows 9x, again based on a custom window server (this time
with a rather unconventional multi-process architecture). The 90s-era
demo disk was based on 4.25 and Photon 1 (there was also a 2.x demo
disk back in the 80s but this didn't have a GUI).
6.0 came with Photon 2, which is still Win9x-ish in terms of
organization and is a fairly straightforward evolution of Photon 1,
although the widgets look very vaguely Amiga-like in 6.0-6.2. 6.0 came
out slightly after the Amiga deal, so that might be the reason for the
Amiga-like theming.
Neutrino was not the name of a GUI, but rather of the entire OS that
succeeded QNX 4 (the first versions of Neutrino used Photon 1 but were
incapable of self-hosting and were developed alongside 4.x; 6.0 was
the first mainline QNX version to be Neutrino-based).
But the only mass-market end-user-facing graphical multimedia-capable
QNX devices I know of were the Blackberry X smartphones. (And
cancelled tablet and netbook.)
There were also the i-Opener (running 4.25) and Audrey (running 6.0)
internet appliances of the late 90s.
11:57 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Tue, 20 Dec 2022 at 09:31, Andrew Warkentin <andreww591(a)gmail.com> wrote:
QNX Classic and 4.x only ran on x86 machines, most of
which were
either standard PCs or at least sort of PC-like (although AFAIK the
ability to run without a BIOS was present very early on). It was quite
common to run QNX on desktops as a development host for embedded
systems AFAIK.
GUIs for QNX predate the Amiga deal, and have existed since the late
80s. The original was QNX Windows, which was either a reimplementation
or port (not quite sure which) of Open Look on a custom non-X11 window
server, running on later versions of 2.x and all versions of 4.x.
Later versions of 4.x added Photon 1, which looks like a cross between
Motif and Windows 9x, again based on a custom window server (this time
with a rather unconventional multi-process architecture). The 90s-era
demo disk was based on 4.25 and Photon 1 (there was also a 2.x demo
disk back in the 80s but this didn't have a GUI).
Fascinating. Thanks for setting the record straight, and I apologise
for my inaccurate speculation.
6.0 came with Photon 2, which is still Win9x-ish in terms of
organization and is a fairly straightforward evolution of Photon 1,
although the widgets look very vaguely Amiga-like in 6.0-6.2. 6.0 came
out slightly after the Amiga deal, so that might be the reason for the
Amiga-like theming.
Neutrino was not the name of a GUI, but rather of the entire OS that
succeeded QNX 4 (the first versions of Neutrino used Photon 1 but were
incapable of self-hosting and were developed alongside 4.x; 6.0 was
the first mainline QNX version to be Neutrino-based).
Aha!
There were also the i-Opener (running 4.25) and Audrey
(running 6.0)
internet appliances of the late 90s.
I did not realise they were QNX devices. I vaguely thought the 3Com
Audrey was based on BeIA, in fact, but I must be confusing it with
something else -- perhaps the Sony eVilla.
--
Liam Proven ~ Profile: https://about.me/liamproven
Email: lproven(a)cix.co.uk ~ gMail/gTalk/FB: lproven(a)gmail.com
Twitter/LinkedIn: lproven ~ Skype: liamproven
UK: (+44) 7939-087884 ~ Czech [+ WhatsApp/Telegram/Signal]: (+420) 702-829-053
15 Dec
15 Dec
12:29 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Dec 11, 2022, at 7:09 PM, Andrew Warkentin <andreww591(a)gmail.com> wrote:
It's not necessarily true that microkernels are significantly slower.
uKernels are usually quite fast as they do so little. What can be slow
is emulating a Unix like OS on top due to context switches. For instance,
a user process doing read() will have the following context switches:
userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
or worse (I didn't account for a few things). Because of this even some
uKernels run a few critical services + drivers in the supervisor mode.
But overall slowdown of such a unix emulation will very much depend on the
workload and also what kind of performance improvements you are willing to
try in a complex kernel vs same services running in user mode.
At present the linux kernel has about 31+ Million lines (accounting for
all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
This LoC growth is entirely understandable but I wonder how things may
have turned out in an alternate universe of uKernel based designs....
2:54 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
Wasn't there some statement that QNX dropped some of these? Copy plus
context switch?
On Wed, Dec 14, 2022 at 04:29:45PM -0800, Bakul Shah wrote:
On Dec 11, 2022, at 7:09 PM, Andrew Warkentin
<andreww591(a)gmail.com> wrote:
--
---
Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat
It's not necessarily true that microkernels are significantly slower.
uKernels are usually quite fast as they do so little. What can be slow
is emulating a Unix like OS on top due to context switches. For instance,
a user process doing read() will have the following context switches:
userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
or worse (I didn't account for a few things). Because of this even some
uKernels run a few critical services + drivers in the supervisor mode.
But overall slowdown of such a unix emulation will very much depend on the
workload and also what kind of performance improvements you are willing to
try in a complex kernel vs same services running in user mode.
At present the linux kernel has about 31+ Million lines (accounting for
all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
This LoC growth is entirely understandable but I wonder how things may
have turned out in an alternate universe of uKernel based designs....
5:36 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
Don't see how unless they put multiple related services in the same
address space, which reduces context switching but tends toward a
monokernel (& increased coupling). Unless I am misunderstanding you.
On Dec 14, 2022, at 6:54 PM, Larry McVoy
<lm(a)mcvoy.com> wrote:
Wasn't there some statement that QNX dropped some of these? Copy plus
context switch?
On Wed, Dec 14, 2022 at 04:29:45PM -0800, Bakul Shah wrote:
On Dec 11, 2022, at 7:09 PM, Andrew Warkentin
<andreww591(a)gmail.com> wrote:
--
---
Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat
It's not necessarily true that microkernels are significantly slower.
uKernels are usually quite fast as they do so little. What can be slow
is emulating a Unix like OS on top due to context switches. For instance,
a user process doing read() will have the following context switches:
userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
or worse (I didn't account for a few things). Because of this even some
uKernels run a few critical services + drivers in the supervisor mode.
But overall slowdown of such a unix emulation will very much depend on the
workload and also what kind of performance improvements you are willing to
try in a complex kernel vs same services running in user mode.
At present the linux kernel has about 31+ Million lines (accounting for
all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
This LoC growth is entirely understandable but I wonder how things may
have turned out in an alternate universe of uKernel based designs....
2:02 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Thu, Dec 15, 2022 at 12:38 AM Bakul Shah <bakul(a)iitbombay.org> wrote:
Don't see how unless they put multiple related
services in the same
address space, which reduces context switching but tends toward a
monokernel (& increased coupling). Unless I am misunderstanding you.
I don't see why two services in a microkernel couldn't arrange to
share a region of memory and implement bidirectional queues
between themselves. With an appropriate signalling mechanism,
you'd still be context switching but avoiding a lot of copying.
- Dan C.
On Dec 14,
2022, at 6:54 PM, Larry McVoy <lm(a)mcvoy.com> wrote:
Wasn't there some statement that QNX dropped some of these? Copy plus
context switch?
On Wed, Dec 14, 2022 at 04:29:45PM -0800, Bakul Shah wrote:
On Dec 11, 2022, at 7:09 PM, Andrew Warkentin
<andreww591(a)gmail.com> wrote:
--
---
Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat
It's not necessarily true that microkernels are significantly slower.
uKernels are usually quite fast as they do so little. What can be slow
is emulating a Unix like OS on top due to context switches. For instance,
a user process doing read() will have the following context switches:
userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
or worse (I didn't account for a few things). Because of this even some
uKernels run a few critical services + drivers in the supervisor mode.
But overall slowdown of such a unix emulation will very much depend on the
workload and also what kind of performance improvements you are willing to
try in a complex kernel vs same services running in user mode.
At present the linux kernel has about 31+ Million lines (accounting for
all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
This LoC growth is entirely understandable but I wonder how things may
have turned out in an alternate universe of uKernel based designs....
2:06 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Thu, Dec 15, 2022 at 09:02:08AM -0500, Dan Cross wrote:
On Thu, Dec 15, 2022 at 12:38 AM Bakul Shah
<bakul(a)iitbombay.org> wrote:
My mind went to similar thoughts. How did QNX manage the page cache?
Did they have mmap?
Don't see how unless they put multiple
related services in the same
address space, which reduces context switching but tends toward a
monokernel (& increased coupling). Unless I am misunderstanding you.
I don't see why two services in a microkernel couldn't arrange to
share a region of memory and implement bidirectional queues
between themselves. With an appropriate signalling mechanism,
you'd still be context switching but avoiding a lot of copying.
2:18 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On Thu, Dec 15, 2022 at 9:07 AM Larry McVoy <lm(a)mcvoy.com> wrote:
On Thu, Dec 15, 2022 at 09:02:08AM -0500, Dan Cross
wrote:
I don't know if they did historically, but I kind of doubt it; QNX predates
4.2BSD, and I imagine there wasn't much influence from TENEX/PMAP).
But they seem to now:
https://www.qnx.com/developers/docs/7.1/#com.qnx.doc.neutrino.getting_start…
No idea how they handle page caching. That's an interesting question,
given how they adopted POSIX and try to at least give the outward
appearance of being Unix-y.
Given the emphasis on the message-based architecture, where messages
can be sent between nodes, I doubt there's that much support for shared
memory as a basis for IPC; I think they favored explicit messaging copying
facilitated by the ukernel. Whether you could build something bespoke using
the provided primitives is another matter.
- Dan C.
On Thu, Dec 15, 2022 at 12:38 AM Bakul Shah
<bakul(a)iitbombay.org> wrote:
My mind went to similar thoughts. How did QNX manage the page cache?
Did they have mmap? Don't see how unless they put multiple
related services in the same
address space, which reduces context switching but tends toward a
monokernel (& increased coupling). Unless I am misunderstanding you.
I don't see why two services in a microkernel couldn't arrange to
share a region of memory and implement bidirectional queues
between themselves. With an appropriate signalling mechanism,
you'd still be context switching but avoiding a lot of copying.
2:02 p.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
I wasn't clear. There was someone who said that the copy to another
address space included a context switch to the receiving process
(I think that was it) that resulted in better performance. Perhaps
the person who said that can elaborate.
On Wed, Dec 14, 2022 at 09:36:12PM -0800, Bakul Shah wrote:
Don't see how unless they put multiple related
services in the same
address space, which reduces context switching but tends toward a
monokernel (& increased coupling). Unless I am misunderstanding you.
> On Dec 14, 2022, at 6:54 PM, Larry McVoy <lm(a)mcvoy.com> wrote:
>
> Wasn't there some statement that QNX dropped some of these? Copy plus
> context switch?
>
> On Wed, Dec 14, 2022 at 04:29:45PM -0800, Bakul Shah wrote:
>> On Dec 11, 2022, at 7:09 PM, Andrew Warkentin <andreww591(a)gmail.com>
wrote:
>>>
>>> It's not necessarily true that microkernels are significantly slower.
>>
>> uKernels are usually quite fast as they do so little. What can be slow
>> is emulating a Unix like OS on top due to context switches. For instance,
>> a user process doing read() will have the following context switches:
>>
>>
userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
>>
>> or worse (I didn't account for a few things). Because of this even some
>> uKernels run a few critical services + drivers in the supervisor mode.
>> But overall slowdown of such a unix emulation will very much depend on the
>> workload and also what kind of performance improvements you are willing to
>> try in a complex kernel vs same services running in user mode.
>>
>> At present the linux kernel has about 31+ Million lines (accounting for
>> all architectures, filesystems, device drivers etc.). The FreeBSD 13.x
>> kernel is about 8.7M LoC (of which 44-45% are in device drivers). I only
>> counted .c and .h files. In contract FreeBSD 2.2.2 kernel has ~554K LoC.
>> This LoC growth is entirely understandable but I wonder how things may
>> have turned out in an alternate universe of uKernel based designs....
>
> --
> ---
> Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat
--
---
Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat
8:01 a.m.
New subject: Clever code (was Re: Re: Stdin Redirect in Cu History/Alternatives?
On 12/14/22, Larry McVoy <lm(a)mcvoy.com> wrote:
Wasn't there some statement that QNX dropped some
of these? Copy plus
context switch?
Yeah, QNX/L4-type IPC is usually just copying followed by a context
switch. Some such kernels (like seL4) don't even have full message
queues per endpoint (the scheduler queue sort of functions as an IPC
queue). Mach IPC is slow because of stuff like permission checking,
which I would assume involves iteration over a list of permitted
threads. QNX/L4-type kernels usually either use constant-time
permission checks (like the old "clans and chiefs" model used by L3
and early L4, or the more modern capability-oriented model used by
seL4 and some others), or lack kernel permission checking entirely
leaving it up to servers.
Another issue is that Mach-type kernels don't have what is known as
"direct process switching" AFAIK. When a synchronous message is sent
on a QNX/L4-type kernel, the kernel immediately switches to the
receiving process, bypassing the scheduler queue entirely, with the
remainder of the sender's timeslice being given to the receiver
(depending on the kernel, priorities may factor into this so it isn't
always quite that simple though). Mach-like kernels often require the
sender to wait for the kernel to decide to schedule the receiver based
on the queue, and then once the reply is sent there's another wait for
the kernel to again decide to schedule the sender again, which makes
for rather poor performance.
On 12/14/22, Bakul Shah <bakul(a)iitbombay.org> wrote:
On Dec 11, 2022, at 7:09 PM, Andrew Warkentin
<andreww591(a)gmail.com> wrote:
Yeah, excessive vertical layering can be bad for performance. QNX
normally follows a process-per-subsystem-instance architecture, so the
chain is just:
client -> (kernel) -> diskServer -> (kernel) -> client
where the disk server includes the disk driver, partition table driver
(if applicable), and disk filesystem. The VFS layer isn't even
involved at all on reads, writes, and the like (it's pretty much only
there to deal with operations that involve paths and not those that
only involve FDs), whereas some other Unix-like microkernel OSes have
it act as an intermediary on all FS operations. A lot of the time,
protection domains correspond more to subsystem instances rather than
layer instances, so there really isn't much harm in merging all layers
of a subsystem into a single process for the sake of performance. When
there is a benefit to separation of layers into different processes,
it is possible to use tap-type drivers to allow running subsystem
instances that only contain some of the layers (QNX doesn't do this
AFAIK, but my OS will).
It's not necessarily true that microkernels are significantly slower.
uKernels are usually quite fast as they do so little. What can be slow
is emulating a Unix like OS on top due to context switches. For instance,
a user process doing read() will have the following context switches:
userProc->uK->FileSystem->uK->diskDriver->uk->FileSysem->uK->userProc
or worse (I didn't account for a few things). Because of this even some
uKernels run a few critical services + drivers in the supervisor mode.
But overall slowdown of such a unix emulation will very much depend on the
workload and also what kind of performance improvements you are willing to
try in a complex kernel vs same services running in user mode.
12 Dec
12 Dec
9:48 a.m.
New subject: Clever code
On 11 Dec 2022 18:15 -0800, from bakul(a)iitbombay.org (Bakul Shah):
Agree that clear code is preferable to complicated
code. But in practice
people sacrifice clarity for performance improvement all the time. Look
at the kernel code of any modern os. Everybody pays lip service to this
but most anything other than toy programs ends up getting needlessly
complicated over time.
Performant code does not need to stand in opposition to clear code.
And if you are writing performance-critical or size-critical code
(such as the QNX microkernel that someone brought up), then of course
you might end up needing to do things in non-obvious ways; that's not
the point here. Clever is fine IMO _where the cleverness provides an
actual benefit in a real-world scenario_, and an operating system
kernel (and a language standard library) is a rather special type of
program where sometimes you have to do things in somewhat non-obvious
ways because of the environment the code is meant to execute in. But a
significant portion of the time, where I see "clever" code there is
_no_ significant benefit to the cleverness; it's often more about
"showing off", or saving a few source code characters at the expense
of at-a-glance readability, than it is about actual usefulness and
necessity.
Necessary complexity in order to solve the problem, combined with
things like performance requirements, may sometimes require clever
code. At that point, there is a benefit to the cleverness. But one can
still aim to write the clever code _clearly_, with everything from
comments to good variable and function names to formatting that
enhances readability by for example grouping related operations
together in the source code to keeping related parts grouped together
and separate from other code. Just such a simple thing as that I often
add extra parenthesis over and beyond what's actually required based
on operator precedence, because doing so makes it clearer what goes
together and unless the compiler is severely braindead by modern
standards, doing so costs _nothing_ past the compilation stage.
--
✍ Michael Kjörling 🏡 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”
9:34 p.m.
On Sun, 11 Dec 2022, Larry McVoy wrote:
I should clarify that on the odd occasion that I write obscure code it's
for efficiency, and I comment it well because *I* might be the one to
maintain it six months hence :-)
-- Dave
I've
always used the maxim "Write code as though the next person to
maintain it is an axe-wielding psychopath who knows where you live".
My main job, for the 20 years until I retired, was to keep telling
people that code that you wrote 6 months ago might as well have been
written by someone else. Optimize for reading the code, not writing the
code. It's read many. 
9:46 p.m.
On 12/12/22 4:34 PM, Dave Horsfall wrote:
Six months? Try ten years. :-)
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU chet(a)case.edu http://tiswww.cwru.edu/~chet/
My main job,
for the 20 years until I retired, was to keep telling
people that code that you wrote 6 months ago might as well have been
written by someone else. Optimize for reading the code, not writing the
code. It's read many.
I should clarify that on the odd occasion that I write obscure code it's
for efficiency, and I comment it well because *I* might be the one to
maintain it six months hence :-)
500
days inactive
510
days old
56 comments
20 participants
participants (20)
-
Adam Thornton -
Andrew Warkentin -
arnold＠skeeve.com -
Bakul Shah -
Brad Spencer -
Chet Ramey -
Clem Cole -
Dan Cross -
Dave Horsfall -
G. Branden Robinson -
Harald Arnesen -
John Cowan -
Kevin Bowling -
Larry McVoy -
Liam Proven -
Michael Kjörling -
Rich Morin -
segaloco -
Steve Nickolas -
Stuff Received