14 Nov
2022
14 Nov
'22
10:11 p.m.
Date: Sat, 12 Nov 2022 17:56:24 -0800
From: Larry McVoy <lm(a)mcvoy.com>
Subject: [TUHS] Re: DG UNIX History
It sounds like they could have supported mmap() easily. I'd love to see
this kernel, it sounds to me like it was SunOS with nicely done SMP
support. The guy that said he'd never seen anything like it before or
since, just makes me want to see it more.
I know someone who was friends with one of the kernel
guys, haven't talked
to her in years but I'll see if I can find anything.
Following on from the exchange on TUHS about DG-UX, it would seem to me that the (Unix)
unified cache was invented at least three times for Unix:
- John Reiser at AT&T
- At Sun
- At DG
As to the latter I could find two leads that might help you finding out more. It would
seem that this unique Unix is specifically DG-UX version 4:
https://web.archive.org/web/20070930212358/http://www.accessmylibrary.com/c…
and
Michael H. Kelly and Andrew R. Huber, "Engineering a (Multiprocessor) Unix
Kernel", Proceedings of the Autumn 1989 EUUG Conference, European Unix Systems User
Group, Vienna, Austria, 1989, pp. 7- 19.
The unified cache isn’t mentioned, but it would seem that the multiprocessor redesign
might have included it. Maybe the author names are helpful. I could not find the paper
online, but there was a web page suggesting that a paper copy still exists in a
(university?) library in Sweden.
=====
Publication: DG Review
Publication Date: 01-NOV-88
Author: Huber, Andrew R.
DG-UX 4.00: DG's redesigned kernel lays the foundation for future UNIX systems.
(includes related article on DG-UX 4.00's file system and an excerpt from Judith S.
Hurwitz's 'Data General's UNIX strategy: an evaluation' report)
COPYRIGHT 1988 New Media Publications
DG/UX 4.00
Revision 4.00 of Data General's native UNIX operating system siginificantly enhances
the product and adds unique capabilities not found in other UNIX implementations. This
article reviews the goals of DG/UX 4.00 and discusses some of its features.
When DG released DG/UX 1.00 in March, 1985, it was based on AT&T's System V
Release 2 and incorporated the Berkeley UNIX file system and networking.
As DG/UX grew, it continued to incorporate functions of the major standard UNIX systems,
as illustrated in the following timeline:
* DG/UX 1.00 March, 1985 Based on System V Release 2 and Berkely 4.1.
Included Berkely 4.2 file system and TCP/IP (LAN).
* DG/UX 2.00, September, 1985 Added Berkeley 4.2 system calls.
* DG/UX 3.00, April 1986 Added support for new DG hardware.
* DG/UX 3.10 March, 1987 Added Sun Microsystem's Network File System.sup.(R) Added X
Windows.
* DG/UX 4.00, August, 1988 Re-designed and re-implemented kernel and file system.
14 Nov
14 Nov
10:31 p.m.
New subject: DG UNIX History
On Mon, Nov 14, 2022 at 5:12 PM Paul Ruizendaal <pnr(a)planet.nl> wrote:
Following on from the exchange on TUHS about DG-UX, it
would seem to me
that the (Unix) unified cache was invented at least three times for Unix:
Not to quibble too much, but s/cache/memory/ I think is a fairer way of
saying that.
- John Reiser at AT&T
- At Sun
- At DG
- At CMU (Mach)
The interesting thing again, is that while they while all of these
implementations seem to have been technologically 'better' - only Mach
lived on from the original developers. And in the case of Mach, by the
time it was mainstream (macOS) the original implementation had been
replaced a few times - so while the concepts are there, I don't think much
of the Original CMU code is left in XNU/Darwin [or for that matter in the
OSF flavors -- Tru64 rewrote it but it died and the OSF/RI kernel never
went anywhere either].
As I said, the lesson to TUHS -- as much as I'm a techie and I am
interested in the 'proper' way of doing things ... "good enough" is
often
what rules.
It's too bad none of the good memory implementations made it into
>systems<< that lasted.
Clem
ᐧ
11:54 p.m.
New subject: DG UNIX History
On Mon, Nov 14, 2022 at 3:33 PM Clem Cole <clemc(a)ccc.com> wrote:
On Mon, Nov 14, 2022 at 5:12 PM Paul Ruizendaal <pnr(a)planet.nl> wrote:
Both FreeBSD and NetBSD re-wrote the vm layer. FreeBSD incrementally, and
NetBSD with a new uvm. At least for FreeBSD, this is when the buffer cache
was fully (or more fully?) unified because it wasn't quite complete in
4.4BSD as shipped IIRC (or maybe it was that it was really buggy, it's been
so long ago now that I've forgotten). Neither are, imho, as good as Sun's
in this respect, but both have been better tuned to scale better than
SunOS. IIRC from OS/MP days, the buffer cache lock contention started to
get bad around 8 or so CPUs (to be fair: SunOS was never MP, these were
Solbourne's locking modifications that weren't super-fine grained).
Following on from the exchange on TUHS about
DG-UX, it would seem to me
that the (Unix) unified cache was invented at least three times for Unix:
Not to quibble too much, but s/cache/memory/ I think is a fairer way of
saying that.
- John Reiser at AT&T
- At Sun
- At DG
- At CMU (Mach)
The interesting thing again, is that while they while all of these
implementations seem to have been technologically 'better' - only Mach
lived on from the original developers. And in the case of Mach, by the
time it was mainstream (macOS) the original implementation had been
replaced a few times - so while the concepts are there, I don't think much
of the Original CMU code is left in XNU/Darwin [or for that matter in the
OSF flavors -- Tru64 rewrote it but it died and the OSF/RI kernel never
went anywhere either].
As I said, the lesson to TUHS -- as much as I'm a
techie and I am
interested in the 'proper' way of doing things ... "good enough" is
often
what rules.
Good enough, and a little more polish to make it even better :)
It's too bad none of the good memory
implementations made it into
Yea...
Warner
>systems<< that lasted.ᐧ
15 Nov
15 Nov
6:03 a.m.
New subject: DG UNIX History
On Mon, Nov 14, 2022 at 04:54:54PM -0700, Warner Losh wrote:
FWIW, Linux also has a (mostly) unified page cache. Which is to say,
for all of the major file systems, there is only a single unified page
cache for file data.
There are some legacy file systems in Linux (e.g., befs, qnx4, qnx6,
minix, vfat, etc.) that still use the buffer cache, but the buffer
cache is implemented in terms of the page cache infrastructure, and
that's mainly because no one has really bothered to update those file
systems to use the unified page cache. After all, you're using vfat
to read and write for super-slow USB thumb drive, who cares if data is
getting copied from the USB thumb drive, to the buffer cache, and then
to the page cache? :-)
Good enough, and backwards compatibility, sure. But for the file
systems where performance is an issue, having a unified page is the
only way to go. :-)
The
interesting thing again, is that while they while all of these
implementations seem to have been technologically 'better' - only Mach
lived on from the original developers. And in the case of Mach, by the
time it was mainstream (macOS) the original implementation had been
replaced a few times - so while the concepts are there, I don't think much
of the Original CMU code is left in XNU/Darwin [or for that matter in the
OSF flavors -- Tru64 rewrote it but it died and the OSF/RI kernel never
went anywhere either].
Both FreeBSD and NetBSD re-wrote the vm layer. FreeBSD incrementally, and
NetBSD with a new uvm. At least for FreeBSD, this is when the buffer cache
was fully (or more fully?) unified because it wasn't quite complete in
4.4BSD as shipped IIRC (or maybe it was that it was really buggy, it's been
so long ago now that I've forgotten).
As I said, the
lesson to TUHS -- as much as I'm a techie and I am
interested in the 'proper' way of doing things ... "good enough" is
often
what rules.
Good enough, and a little more polish to make it even better :)
> It's too bad none of the good memory
implementations made it into
> >>systems<< that lasted.ᐧ
At least in my book, it's not the implementations, but the ideas that
matter. And sometimes implementations are constrained by hardware
limitations (example: clists), and for less contrained hardware,
sometimes you're better re-evaluating whether the ideas in use, and if
that means that you need to reimplement the ideas that still make
sense, there's nothing wrong with that.
- Ted
P.S. I remember back in the day when I was engaging in a friendly
competition with a FreeBSD hacker on improving the serial driver for
the 8250 chip (with no FIFO's!), and I shared with him my idea of
using a pair of ring buffers that would get flipped back and forth
between the interrupt handler and the tty "bottom-half" (read:
software interrupt) handler, and I was told that clists were handed
down from Olympus by the AT&T/Unix Gods and he could never get that
kind of change into the FreeBSD tty layer. Of course, I was free to
make all of the radical changes to Linux's tty layer --- and I did,
all in the name of the number of 115kbaud connections that could be
handled on a single 40 MHz 386 processor...
3:11 p.m.
New subject: DG UNIX History
On Tue, Nov 15, 2022 at 01:03:05AM -0500, Theodore Ts'o wrote:
P.S. I remember back in the day when I was engaging
in a friendly
competition with a FreeBSD hacker on improving the serial driver for
the 8250 chip (with no FIFO's!), and I shared with him my idea of
using a pair of ring buffers that would get flipped back and forth
between the interrupt handler and the tty "bottom-half" (read:
software interrupt) handler, and I was told that clists were handed
down from Olympus by the AT&T/Unix Gods and he could never get that
kind of change into the FreeBSD tty layer. Of course, I was free to
make all of the radical changes to Linux's tty layer --- and I did,
all in the name of the number of 115kbaud connections that could be
handled on a single 40 MHz 386 processor...
I remember being pleasantly surprised that Linus/Linux was open to
that sort of change. I get why the traditional Unix shops resisted
be wacks like that but they went too far and it prevented good work.
Linux seemed far more willing to realize that they had it wrong and
there was a better way. That was refreshing.
Of course they got beat up for it with "Linux is stable/compatible".
--
---
Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat
3:48 p.m.
New subject: DG UNIX History
On Nov 15, 2022, at 7:11 AM, Larry McVoy <lm(a)mcvoy.com> wrote:
On Tue, Nov 15, 2022 at 01:03:05AM -0500, Theodore Ts'o wrote:
At Fortune Systems Dave Yost was able to achieve full-duplex 9600
baud speed on up to 5 ports in V7 Unix without changing the clist
design. This on a 5.6Mhz machine (with 4 cycle memory). The trick
was to specialize interrupt handlers for each port.
P.S. I remember back in the day when I was
engaging in a friendly
competition with a FreeBSD hacker on improving the serial driver for
the 8250 chip (with no FIFO's!), and I shared with him my idea of
using a pair of ring buffers that would get flipped back and forth
between the interrupt handler and the tty "bottom-half" (read:
software interrupt) handler, and I was told that clists were handed
down from Olympus by the AT&T/Unix Gods and he could never get that
kind of change into the FreeBSD tty layer. Of course, I was free to
make all of the radical changes to Linux's tty layer --- and I did,
all in the name of the number of 115kbaud connections that could be
handled on a single 40 MHz 386 processor...
I remember being pleasantly surprised that Linus/Linux was open to
that sort of change. I get why the traditional Unix shops resisted
be wacks like that but they went too far and it prevented good work.
Linux seemed far more willing to realize that they had it wrong and
there was a better way. That was refreshing.
Of course they got beat up for it with "Linux is stable/compatible".
5:47 p.m.
New subject: DG UNIX History
On Mon, Nov 14, 2022 at 11:03 PM Theodore Ts'o <tytso(a)mit.edu> wrote:
On Mon, Nov 14, 2022 at 04:54:54PM -0700, Warner Losh
wrote:
cache
Good enough, and backwards compatibility, sure. But for the file
systems where performance is an issue, having a unified page is the
only way to go. :-)
Thankfully, clists have been gone forever from the FreeBSD tty layer (since
FreeBSD 2 or 3 if memory serves). The soft interrupt stuff had to wait to
be replaced until the SMPng efforts to make FreeBSD scale on multicore.
Some cruft remains in the tty layer, to be sure, but it's somewhat better
than before.
Bruce was quite active in FreeBSD and did drive much change in our
thinking, though mostly through getting others to rewrite it in the right
way after the main proponents of that dogma left the project and we could
grow beyond that past. It was a mistake in the earliest days to think
anything from AT&T was sacrosanct and the project is much healthier for
it... We'd never been able to scale with the old AT&T inherited design on
many many things.
Warner
> The interesting thing again, is that while
they while all of these
> implementations seem to have been technologically 'better' - only Mach
> lived on from the original developers. And in the case of Mach, by the
> time it was mainstream (macOS) the original implementation had been
> replaced a few times - so while the concepts are there, I don't think
much
> of the Original CMU code is left in
XNU/Darwin [or for that matter in
the
OSF
flavors -- Tru64 rewrote it but it died and the OSF/RI kernel never
went anywhere either].
Both FreeBSD and NetBSD re-wrote the vm layer. FreeBSD incrementally, and
NetBSD with a new uvm. At least for FreeBSD, this is when the buffer was fully (or more fully?) unified because it
wasn't quite complete in
4.4BSD as shipped IIRC (or maybe it was that it was really buggy, it's
been
so long ago now that I've forgotten).
FWIW, Linux also has a (mostly) unified page cache. Which is to say,
for all of the major file systems, there is only a single unified page
cache for file data.
There are some legacy file systems in Linux (e.g., befs, qnx4, qnx6,
minix, vfat, etc.) that still use the buffer cache, but the buffer
cache is implemented in terms of the page cache infrastructure, and
that's mainly because no one has really bothered to update those file
systems to use the unified page cache. After all, you're using vfat
to read and write for super-slow USB thumb drive, who cares if data is
getting copied from the USB thumb drive, to the buffer cache, and then
to the page cache? :-)
> As I said, the lesson to TUHS -- as much as
I'm a techie and I am
> interested in the 'proper' way of doing things ... "good enough"
is
often
what
rules.
Good enough, and a little more polish to make it even better :) > It's too bad none of the good memory
implementations made it into
> >>systems<< that lasted.ᐧ
At least in my book, it's not the implementations, but the ideas that
matter. And sometimes implementations are constrained by hardware
limitations (example: clists), and for less contrained hardware,
sometimes you're better re-evaluating whether the ideas in use, and if
that means that you need to reimplement the ideas that still make
sense, there's nothing wrong with that.
- Ted
P.S. I remember back in the day when I was engaging in a friendly
competition with a FreeBSD hacker on improving the serial driver for
the 8250 chip (with no FIFO's!), and I shared with him my idea of
using a pair of ring buffers that would get flipped back and forth
between the interrupt handler and the tty "bottom-half" (read:
software interrupt) handler, and I was told that clists were handed
down from Olympus by the AT&T/Unix Gods and he could never get that
kind of change into the FreeBSD tty layer. Of course, I was free to
make all of the radical changes to Linux's tty layer --- and I did,
all in the name of the number of 115kbaud connections that could be
handled on a single 40 MHz 386 processor...
1:21 a.m.
New subject: DG UNIX History
On 2022-11-14 17:31, Clem Cole wrote:
On Mon, Nov 14, 2022 at 5:12 PM Paul Ruizendaal
<pnr(a)planet.nl> wrote:
Following on from the exchange on TUHS about DG-UX, it would seem
to me that the (Unix) unified cache was invented at least three
times for Unix:
Not to quibble too much, but s/cache/memory/ I think is a fairer way
of saying that.
- John Reiser at AT&T
- At Sun
- At DG
- At CMU (Mach)
The interesting thing again, is that while they while all of these
implementations seem to have been technologically 'better' - only Mach
lived on from the original developers. And in the case of Mach, by
the time it was mainstream (macOS) the original implementation had
been replaced a few times - so while the concepts are there, I don't
think much of the Original CMU code is left in XNU/Darwin [or for that
matter in the OSF flavors -- Tru64 rewrote it but it died and the
OSF/RI kernel never went anywhere either].
The CMU copyrights are still there
(https://github.com/apple-opensource/xnu/tree/master/osfmk/mach).
Perhaps someone far more knowledgeable than me could spelunk.
As I said, the lesson to TUHS -- as much as I'm a techie and I am
interested in the 'proper' way of doing things ... "good enough" is
often what rules.
Indeed.
It's too bad none of the good memory implementations made it into
>systems<< that lasted.
Clem
ᐧ
529
days inactive
530
days old
7 comments
7 participants
participants (7)
-
Bakul Shah -
Clem Cole -
Larry McVoy -
Paul Ruizendaal -
Stuff Received -
Theodore Ts'o -
Warner Losh