> On Sun, Jul 3, 2022 at 1:33 PM Marc Donner wrote:
>
> I've been ruminating on the question of whether networks are different from
> disks (and other devices). Here are a couple of observations:
[...]
From my perspective most of these things are not unique to networks, they happen with disks and/or terminals. Only out-of-order delivery seems new. However, in many early networking contexts (Spider/Arpanet/Datakit/UUCP) this aspect was not visible to the host (and the same holds for a single segment ethernet).
To me, in some ways networks are like tty’s (e.g. completing i/o can take arbitrarily long, doing a seek() does not make sense), in other ways they are like disks (raw devices are organised into byte streams, they have a name space). Uniquely, they have two end-points, only one of which is local (but pipes come close).
Conceptually, a file system does two things: (i) it organises raw blocks into multiple files; these are the i-nodes and (ii) it provides a name space; these are directories and the namei routine. A network stack certainly does the first: a raw network device is organised into multiple pipe-like connections; depending on the network, it optionally offers a naming service.
With the first aspect one could refer to any file by “major device number, minor device number, i-node number”. This is not very different from referring to a network stream by “network number, host number, port number” in tcp/ip (and in fact this is what bind() and connect() in the sockets API do), or “switch / host / channel” in Datakit. For disks, Unix offers a clean way to organise the name spaces of multiple devices into a unified whole. How to do this with networks is not so easy, prior to the invention of the file system switch.
Early on (Arpanet Unix), it was tried to incorporate host names into a net directory by name (RFC 681) but this is not scalable. Another way would be to have a virtual directory and include only names for active connections. The simple way would be to use a text version of the numeric name as described above - but that is not much of an improvement. Better to have a network variant of namei that looks up symbolic names in a hosts file or in a network naming service. The latter does not look very performant on the hardware of 40 years ago, but it appears to have worked well on the Alto / PuPs network at Xerox PARC.
With the above one could do
open(“/net/inet/org.tuhs.www:80”, O_RDWR | O_STREAM)
to connect to the TUHS web server, and do
open(“/net/inet/any:80”, O_RDWR | O_STREAM | O_CREAT, 0600)
to create a ‘listening’ (rendez-vous) socket.
Paul
On Sun, Jul 03, 2022 at 05:55:15PM +1000, steve jenkin wrote:
>
> > On 3 Jul 2022, at 12:27, Larry McVoy <lm(a)mcvoy.com> wrote:
> >
> > I love the early Unix releases because they were so simple, processors
> > were simple then as well.
>
>
> Bell???s Observation on Computer Classes has brought surprises
> - we???ve had some very popular new devices appear at the bottom end of the market and sell in the billions.
Yes, and they all run Linux or some tiny OS. Has anyone ported v7 to
any of these devices and seen it take off? Of course not, it doesn't
have TCP/IP.
On June 28 Rob Pike wrote:
"One of the reasons I'm not a networking expert may be relevant here. With
networks, I never found an abstraction to hang my hat on. Unlike with file
systems and files, or even Unix character devices, which provide a level of
remove from the underlying blocks and sectors and so on, the Unix
networking interface always seemed too low-level and fiddly, analogous to
making users write files by managing the blocks and sectors themselves."
I've been ruminating on the question of whether networks are different from
disks (and other devices). Here are a couple of observations:
1 - Two different packets may take two different paths from the sender to
the receiver.
1a - The transit time for one packet may vary widely from that of the other.
1b - The two packets may arrive in an order different from the order in
which they were transmitted.
(Note - recently I have been reading Bob Gezelter's monograph [and PhD
dissertation] and I've learned that modern high-performance disk systems
behave more like networks in 1a and 1b.)
2 - A packet may never arrive.
3 - Behavior 2 not a sign of hard failure for networks, whereas it is
generally considered so for other I/O devices.
There is probably more to why networks are weird, but these are some of the
big dissonances that seem to me to make Rob's comment resonate so loudly to
me.
Best,
Marc
=====
nygeek.netmindthegapdialogs.com/home <https://www.mindthegapdialogs.com/home>
As part of some of simh work, I've been immersed in some licensing
discussions. Thanks for the V8-10, Plan-9 and Inferno notes - they are
relevant.
Anyway, WRT to TUHS, I'm thinking that at least in the case of the Unix
style bits, I propose a small change to Waren's top-level directory. Add
a new dir called something like 'Legal Docs' or 'Copyrights+Licenses'.
Then move the Caldera document and Warren's current note into that area.
Then add copies of anything we can collect like the Dan Cross's V8-10,
anything WRT to Plan9/Inferno or anything we from the UNIX world - such as
something Sun, DEC or HP or like might have added. Maybe add a
subdirectory with the AT&T/USL case details. And maybe add a
sub-directory with known FOSS licenses used by the UNIX community and add a
copy of the 3-clause BSD and maybe even the two GPLs.
Then update the README in the current top-level dir. Adding to the
contents something like "*the IP contained on this website is covered by
different licenses depending on the specific IP. Copies of these can be
found with the source code itself, but have also been all collected
together in the top-level directory: ...*."
I think these all have both historical values, as well as practical
values. As I said, I was not sure myself and I think other would be less
ignorant if they could find it all easily. In the case of the practical,
a for instance, in an email with some lawyers last week, I had pointed them
at the Caldera document. I'ld have loved to have been able to say look in
this directory. The Caldera and later Nokia Licenses are what we are
considering as examples.
Thoughts?
I've enjoyed reading this thread as networking has always been a passion of
mine. Lawrence Livermore had, at one time, their own networking system
they called Spider. Is this the same Spider technology that Sandy Fraiser
references in his Datakit notes?
Geoff
>> I don't know the answer to Ctrl-D.
The Unix command "man ascii" has the answer:
Oct Dec Hex Char Oct Dec Hex Char
------------------------------------------------------------------------
000 0 00 NUL '\0' 100 64 40 @
001 1 01 SOH (start of heading) 101 65 41 A
002 2 02 STX (start of text) 102 66 42 B
003 3 03 ETX (end of text) 103 67 43 C
004 4 04 EOT (end of transmission) 104 68 44 D
....
Ctrl-D signifies end of transmission. Some other O/Ses have used
Ctrl-Z for that purpose, presumably because Z is the final letter
of numerous alphabets.
There is a good book about the history of character sets (pre-Unicode)
in the book described at this URL:
http://www.math.utah.edu/pub/tex/bib/master.html#Mackenzie:1980:CCS
Bob Bemer (1920--2004), known as Dr. ASCII to some of us, was a key
person in the standardization of character sets:
https://en.wikipedia.org/wiki/Bob_Bemerhttps://en.wikipedia.org/wiki/ASCII
-------------------------------------------------------------------------------
- Nelson H. F. Beebe Tel: +1 801 581 5254 -
- University of Utah -
- Department of Mathematics, 110 LCB Internet e-mail: beebe(a)math.utah.edu -
- 155 S 1400 E RM 233 beebe(a)acm.org beebe(a)computer.org -
- Salt Lake City, UT 84112-0090, USA URL: http://www.math.utah.edu/~beebe/ -
-------------------------------------------------------------------------------
What are the 1970’s & 1980’s Computing / IT skills “our grandkids won’t have”?
Whistling into a telephone while the modem is attached, because your keyboard has a stuck key
- something I absolutely don’t miss.
Having a computer in a grimy wharehouse with 400 days of uptime & wondering how a reboot might go?
steve j
=========
9 Skills Our Grandkids Will Never Have
<https://blog.myheritage.com/2022/06/9-skills-our-grandkids-will-never-have/>
1: Using record players, audio cassettes, and VCRs
2: Using analog phones [ or an Analog Clock ]
3. Writing letters by hand and mailing them
4. Reading and writing in cursive
5. Using manual research methods [ this is a Genealogy site ]
6. Preparing food the old-fashioned way
7. Creating and mending clothing
8. Building furniture from scratch
9. Speaking the languages of their ancestors
--
Steve Jenkin, IT Systems and Design
0412 786 915 (+61 412 786 915)
PO Box 38, Kippax ACT 2615, AUSTRALIA
mailto:sjenkin@canb.auug.org.au http://members.tip.net.au/~sjenkin
Warner Losh:
Alcatel-Lucent gave an official grant to V8, V9 and V10. See
https://www.tuhs.org/Archive/Distributions/Research/Dan_Cross_v8/statement_…
====
Quite so. I believe this was announced on a mailing list called TUHS.
Those here who are interested in such things might want to subscribe;
I have and find it quite useful and interesting, with occasional
disappointment.
Norman Wilson
Toronto ON
(typing this on a train in Texas)
> I understand UNIX v7 is under this BSD-style license by Caldera Inc.
> https://www.tuhs.org/Archive/Caldera-license.pdf
The eqn document by Kernighan and Cherry also appears in the v10
manual, copyright by AT&T and published as a trade book. Wouldn't the
recent release of v10 also pertain to the manual?
Doug
Following an insightful post by Norman Wilson (https://minnie.tuhs.org/pipermail/tuhs/2022-June/025929.html) and re-reading a few old papers (https://minnie.tuhs.org/pipermail/tuhs/2022-June/026028.html) I was thinking about similarities and differences between the various Unix networking approaches in the 1975-1985 era and came up with the following observations:
- First something obvious: early Unix was organised around two classes of device: character based and block based. Arguably, it is maybe better to think of these classes conceptually as “transient” and “memoizing”. A difference between the two would be wether or not it makes conceptual sense to do a seek operation on them and pipes and networks are in the transient class.
- On the implementation side, this relates two early kernel data structures: clists and disk buffers. Clists were designed for slow, low volume traffic and most early Unix network code creates a third kind: the mbufs of Arpanet Unix, BBN-TCP Unix and BSD, the packets of Chesson's V7 packet driver, Ritchie's streams etc. These are all the same when seen from afar: higher capacity replacements for clists.
- Typically devices are accessed via a filter. At an abstract level, there is not much difference between selecting a line discipline, pushing a stream filter or selecting a socket type. At the extreme end one could argue that pushing a TCP stack on a network device is conceptually the same as mounting a file system on a disk device. Arguably, both these operations could be performed through a generalised mount() call.
- Another implementation point is the organisation of the code. Is the network code in the kernel, or in user land? Conceptually connection management is different from stream management when connected (e.g. CMC and URP with Datakit, or RTP and BSP in Xerox Pups). In the BSD lineage all is in the kernel, and in the Research lineage connection management is done in a user space daemon.
Arpanet Unix (originally based on V5) had a curious solution: the network code was organised in a single process, but with code both in kernel mode and in user mode. The user code would make a special system call, and the kernel code would interact with the IMP driver, manage buffers and deliver packets. Only when a state-changing event happened, it would return to user mode and the user code would handle connection management (followed by a new call into kernel mode). Interestingly, this approach mostly hid the IMP connection, and this carried through to the BSD’s where the network devices were also buried in the stack. Arpanet Unix made this choice to conserve kernel address space and to minimize the amount of original kernel code that had to be touched.
- Early Unix has three ways to obtain a file descriptor: open, creat and pipe. Later also fifo. In this context adding more (like socket) does not seem like a mortal sin. Arguably, all could be rolled into one, with open() handling all cases. Some of this was done in 4.2BSD. It is possible to combine socket() & friends into open() with additional flags, much as was done in Arpanet Unix and BBN-TCP Unix.
- Network connections have different meta data than disk files, and in sockets this handled via specialised calls. This seems a missed opportunity for unified mechanisms. The API used in BBN-TCP handles most of this via ioctl. However, one could (cheekily!) argue that V7 unix has a somewhat byzantine meta data API, with the functionality split over seek, ioctl, fcntl, stat and fstat. These could all be handled in a generalised ioctl. Conceptually, this could also be replaced by using read/write on a meta data file descriptor, which could for example be the regular descriptor with the high bit set. But this, of course, did never exist.
- A pain point in Arpanet Unix was that a listening connection (i.e. a server endpoint) would block until a client arrived and then turn into the connection with the client. This would fork out into a service process and the main server process would open a new listening socket for the next client. In sockets this was improved into a rendez-vous type server connection that would spawn individual client connections via ‘accept’. The V8/V9 IPC library took a similar approach, but also developed the mechanism into a generalized way to (i) create rendez-vous points and (ii) ship descriptors across local connections.
- The strict blocking nature of IO in early Unix was another pain point for writing early network code. The first solution to that were BBN’s await and capac primitives, which worked around the blocking nature. With SysIII, non-blocking file access appeared and 4.1a BSD saw the arrival of 'select’. Together these offer a much more convenient way to deal with multiple tty or network streams in a single threaded process (although it did modify some of the early Unix philosophy). Non-blocking IO and select() also appeared in the Research lineage with 8th edition.
- The file system switch (FSS) arrived around 1983, during the gestation of 8th edition. This was just 1 or 2 years after the network interfaces for BSD and Datakit got their basic shape. Had the FSS been part of V7 (as it well could have been), probably the networking designs would have been a bit different, using virtual directories for networking connections. The ‘namei hack’ in MIT’s CHAOS network code already points in this direction. A similar approach could have been extended to named pipes (arriving in SysIII), where the fifo endpoint could have been set up through creating a file in a virtual directory, and making connections through a regular open of such a virtual file (and 9th edition appears to implement this.)
oOo
To me it seems that the V1-V7 abstractions, the system call API, etc. were created with the experience of CTSS, Multics and others fresh in mind. The issues were understood and it combined the best of the ideas that came before. When it came to networking, Unix did not have this advantage and was necessarily trying to ride a bike whilst inventing it. Maybe in a different time line it would have been possible to pick the best ideas in this area as well and combine these into a coherent framework.
I concur with the observation that this list should be about discussion of what once was and only tangentially about what might have been, so it is only after considerable hesitation that I write the below.
Looking at the compare and contrast above (and having been tainted by what became dominant in later decades), I would say that the most “Unixy” way to add networking to V7/SysIII era Unix would have been something like:
- Network access via open/read/write/close, in the style of BBN-TCP
- Network namespace exposed via a virtual file system, a bit like V9
- Meta data via a generalised ioctl, or via read/write on a meta data descriptor
- Connection rendez-vous via a generalised descriptor shipping mechanism, in the style of V8/V9
- Availability of non-blocking access, together with a waiting primitive (select/poll/etc.), in the style of BSD
- Primary network device visible as any other device, network protocol mounted similar to a file system.
- Both connection management and stream management located in kernel code, in the style of BSD
