I'm trying to figure out how tcp/ip networking worked in 8th edition Unix.
I'm starting from dmr's paper about streams (http://cm.bell-labs.co/who/dmr/st.html), the V8 man pages (http://man.cat-v.org/unix_8th/3/) and browsing the source code (tarball here http://www.tuhs.org/Archive/Distributions/Research/Dan_Cross_v8/).
In the below I use 'socket' to mean a file descriptor bound to a network connection. My current understanding is like this:
- The hardware interface is exposed as a character device; for tcp/ip only ethernet is supported. Directly accessing this device reads/writes ethernet frames.
- One can push an 'ip' module (only) onto an ethernet device; this module also handles ARP. Once this is done IP messages are queued to the virtual ip devices, /dev/ipXX. The device minor number XX equals the protocol number, i.e. the ip packets are demultiplexed into separate virtual devices. IP packets from multiple ethernet cards all end up on the same virtual ip devices. I'm not sure if one can still read/write to the ethernet device after pushing the ip module, but I think you can, creating a raw IP interface so to say.
- On /dev/ip6 one can push a TCP module. The TCP module handles the TCP protocol and demultiplexes incoming traffic to the virtual /dev/tcpXX devices. On /dev/ip17 one can push a UDP module. The UDP module handles the UDP protocol and demultiplexes incoming traffic to the virtual /dev/udpXX devices. Not sure wether the ip6 and ip17 devices can still be read/written after pushing these disciplines.
- There are 100 udp devices, /dev/updXX. To open a UPD socket, one opens an unused udp device (through linear search). This socket accepts binary commands ('struct upduser') through the read()/write() system calls. There is a command to set the local port (effectively 'bind') and a comment to also set the foreign address and port (effectively 'bind+connect'). As long as the socket is not connected actual datagrams are preceded by a command header with the address/port information (effectively 'sendto'/'recvfrom'). Once the socket is connected, it is no longer possible to send further commands, but each write/read is a datagram. For udp sockets it is not possible to specify the local address: it is chosen by the system to match with the given foreign address.
- There are 100 tcp devices /dev/tcpXX. Initial connection is always over an odd numbered device. To open a TCP socket, one opens an unused tcp device (through linear search). This socket accepts binary commands ('struct tcpuser') through the read()/write() system calls. There is a command to actively connect (effectively 'connect' with optional 'bind'), and a command to passively listen (effectively 'bind'+'listen'). If the connect command is sent, one can read one more response block and then the socket becomes a regular tcp socket. If the listen command is sent, one can read multiple response blocks, one for each new client (effectively 'accept'). Those response blocks contain a device number for the new client connection, i.e. one has to subsequently open device /dev/tcpXY to talk to the client. This number is always even, i.e. locally initiated tcp connections are over odd numbered tcp devices, and remotely initiated connections are over even numbered tcp devices - not sure what the significance of this is.
- The above seems to be modeled on the Datakit setup, where the network is exposed as 520 virtual devices, one for each channel, as /dev/dk/dkXXX. These channels than also seem to accept binary command blocks through the read()/write() interface, with a 'connect' type command changing the connection into a data only channel.
Anybody on the list with 8th edition experience who can confirm that the above understanding is about correct?
> From: Will Senn <will.senn(a)gmail.com>
> I am curious about how the Harvard Architecture relates to Unix,
> historically. If the Harvard Architecture is predicated on the
> separation of code from data in order to prevent self-modifying code (my
That's not the 'dictionary' definition, which is 'separate paths for
instructions and data'. But let's go with the 'no self-modifying code' one for
The thing is that self-modifying code is pretty much an artifact of the dawn
of computers, before the economics of gates moved from that of tubes, to
transistors, and also before people understood how important good support for
subroutines was. (This latter is a reference to how Whirlwind did subroutines,
with self-modifying code.) Once people had index registers, and lots of
registers in general, self-modifying code (except for a few small, special
hacks like bootstraps which had to fit in tiny spaces) became as dead as the
It's just a Bad Idea.
> then it would seem to me to be somewhat at odds with a Unix philosophy
> of extreme abstraction (code, data, it's all 0's and 1's, after all).
The people who built Unix were fundamentally very practical. Self-modifing
code is not 'practical'. (And note that Unix from V4:
onward has support for pure text - for practical reasons).
> the PDP-11 itself, with the Unibus and apparently agnostic ISA seem to
> summarily reject the Harvard Architecure...
You could say that of a zillion computers. The only recent computer I can
think of offhand with separate instruction and data paths was the AMD 42K
(nice chip, I used it in a product we built at Proteon). They had separate
ports for instructions and data purely for performance reasons. (Our card had
a pathway which allowed the CPU to write the instruction memory, needed during
booting, obviously; the details as to how we did it escape me now.)
> From: Jon Steinhart
> For all intents and purposes instructions were separate from data from
> the PDP 11/70 on.
And the other -11 memory management (as on the /40, /23, etc) does allow for
execute-only 'segments' (they call them 'pages' in the later versions of the
manual, but they're not) - again, separating code from data. Unix used this
for shared pure texts.
And note that those machines with separate I+D space don't meet the dictionary
definition either, because they only have one bus from the CPU to memory,
shared between data and instruction fetches.
> From: Doug McIlroy
> Optimal code for bitblt (raster block transfers) in the Blit
Interesting case. I'm not familiar with BitBLT codes, do they actually modify
the existing program, or rather do they build small custom ones? Only the
former is what I was thinking of.
>From the discussion of self-modifying code:
>> Optimal code for bitblt (raster block transfers) in the Blit
> Interesting case. I'm not familiar with BitBLT codes, do they actually modify
> the existing program, or rather do they build small custom ones? Only the
> > former is what I was thinking of.
It built small custom fragments of code. But if that had been in D
space, it couldn't have been executed.
>> Surely JIT compiling must count as self-modifying code.
> If it does, then my computer just runs one program from when I turn it
> on. It switches memory formats and then is forever extending itself and
> throwing chunks away.
Exactly. That is the essence of stored-program computers. The exec
system call is self-modification with a vengeance.
Fill memory-and-execute is the grandest coercion I know. What is
data one instant is code the next.
It's all a matter of viewpoint and scale. Where is the boundary
between changing one instruction and changing them all? Or is
this boundary a figment of imagination?
> From: "Ron Natalie"
> Every PDP-11 UNIX I ever used had the console KL-11 as /dev/tty8.
> The question is why.
Blast! I have this memory of reading an explanation for that somewhere - but
I cannot remember what it was, or where! I've done a grep through my hoard of
Unix documents, looking for "tty8", but no hits.
> The thing is that self-modifying code is pretty much an artifact of the dawn
> of computers, [...]
> It's just a Bad Idea.
Surely JIT compiling must count as self-modifying code.
Optimal code for bitblt (raster block transfers) in the Blit
Repeat, slightly modified, of a previous post that got
shunted to the attachment heap.
> I am curious if anyone on the list remembers much
> about the development of the first spell checkers in Unix?
Yes, intimately. They had no relationship to the PDP 10.
The first one was a fantastic tour de force by Bob Morris,
called "typo". Aside from the file "eign" of the very most common
English words, it had no vocabulary. Instead it evaluated the
likelihood that any particular word came from a source with the
same letter-trigram frequencies as the document as a whole. The
words were then printed in increasing order of likelihood. Typos
tended to come early in the list.
Typo, introduced in v3, was very popular until Steve Johnson wrote
"spell", a remarkably short shell script that (efficiently) looks
up a document's words in the wordlist of Webster's Collegiate
Dictionary, which we had on line. The only "real" coding he did
was to write a simple affix-stripping program to make it possible
to look up plurals, past tenses, etc. If memory serves, Steve's
program is described in Kernighan and Pike. It appeared in v5.
Steve's program was good, but the dictionary isn't an ideal source
for real text, which abounds in proper names and terms of art.
It also has a lot of rare words that don't pull their weight in
a spell checker, and some attractive nuisances, especially obscure
short words from Scots, botany, etc, which are more likely to
arise in everyday text as typos than by intent. Given the basic
success of Steve's program, I undertook to make a more useful
spelling list, along with more vigorous affix stripping (and a
stop list to avert associated traps, e.g. "presenation" =
pre+senate+ion"). That has been described in Bentley's "Programming
Pearls" and in http://www.cs.dartmouth.edu/~doug/spell.pdf.
Morris's program and mine labored under space constraints, so
have some pretty ingenious coding tricks. In fact Morris has
a patent on the way he counted frequencies of the 26^3 trigrams
in 26^3 bytes, even though the counts could exceed 255. I did
some heroic (and probabilistic) encoding to squeeze a 30,000
word dictionary into a 64K data space, without severely
affecting lookup time.
> From: "Nelson H. F. Beebe"
> The PDF URLs for bstj.bell-labs.com no longer work, and the ones for
> www.alcatel-lucent.com ... now redirect to an HTML page.
With any luck, someone scraped them before they went.
I've gotten in the habit of scraping all the Web content I look at, since it
has (as above) a distressing tendency to vapourize.
On 24 November 2017 at 10:11, Nemo <cym224(a)gmail.com> wrote:
> On 22 November 2017 at 03:48, <arnold(a)skeeve.com> wrote (in part):
>>> As a former developer and manager, I would be really pissed off if my
>>> programmers wasted their time on writing useless frippery instead of
>>> quality code, and I would certainly have a little chat with them...
>> I think that this is totally appropriate for code being developed
>> for a paid product.
> I would say this is context-sensitive (industry, customers, ...). One
> version of MS Word had an animation of a cartoon monster crushing "WP'
> (somewhere in the credits, I recall).
I really must be more careful with replies. The above was meant for
TUHS, not just Arnold.