[TUHS] 8th Edition timeline

Tue Mar 31 12:44:06 AEST 2020

> [...] the whole mbuf's stuff is something [...]

I would prefer not to go into the BSD code history at this point in time, as I’d like to stay focused on what was going on at Bell Labs between 7th and 8th edition, as far as networking is concerned. Of course it was all interconnecting, and the topic of mbuf's is a nice arc to take us ultimately back too streams.

Looking at network buffering in Unix, I think the story begins in 1973 with Spider. In Spider, the network driver was a character mode device, but used disk block buffering for storage. This worked well, because the Spider interface sent/received data through DMA, in cells of 256 bytes and at speeds comparable to contemporary disks:
https://minnie.tuhs.org/cgi-bin/utree.pl?file=V4/nsys/dmr/tdir/tiu.c.

In 1975 the UoI guys doing Arpa Unix had to interface to an IMP, which was much slower than a disk but much faster than a terminal. They concluded that neither clists not disk buffers were a good match and Steve Bunch came up with ’netbuf’s, which were 64 byte blocks (carved out of a pool of disk blocks). Later (1978?) this code was modified to optionally use separate buffer heads and buffer data, as part of the UoC buffer mod’s (to alleviate the pressure on a limited address space):
https://minnie.tuhs.org/cgi-bin/utree.pl?file=SRI-NOSC/h/net/netbuf.h

In 1979 the first Datakit software was done (Chesson), where I don’t have much information. My hypothesis is that this used clists, but I’m still trying to find a copy of his paper about this implementation. Maybe the driver used a precursor to a streams like data structure. Maybe this is also where MPX files fit in. Unfortunately, currently I just don’t know.

In 1980 new TCP software for the VAX is being written, and again it needs to find a data buffer approach. This is ‘mbuf’s, which are similar to the earlier ’netbuf’s. The main changes are that the size is increased to 128 bytes and that a start offset is added, which makes it possible to remove headers without copying the buffer. The ‘memberof’ field is replaced by a second linkage pointer. The UoC idea to separate the header and data is dropped again. They (mbufs) may have other origins as well, but as far as Unix networking is concerned they are an evolution of the Arpa Unix ‘netbuf’s:
https://minnie.tuhs.org/cgi-bin/utree.pl?file=BBN-Vax-TCP/bbnnet/mbuf.h

Then we get to (late) 1981. On the Labs side streams are created, in my current understanding mainly to support networking and in particular Datakit. The difference with Spider is that the Datakit interface was not DMA based and that the data arrives in 9 bit envelopes instead of 8 bit bytes. The 9th bit distinguishes between data bytes and control bytes. Again neither clists nor disk buffers are a good match for buffering and a new solution is created. Looking at buffers, these consist of a header and a separate datablock. The datablock can be of various sizes: 4, 16, 64 and 1024 bytes - a match with the varying nature of network traffic. Each block has a type (a.o. DATA and CONTROL) which has various uses, but certainly gives a home to the 9th Datakit bit where needed:
https://github.com/Alhadis/Research-Unix-v8/blob/master/v8/usr/include/sys/stream.h
https://github.com/Alhadis/Research-Unix-v8/blob/master/v8/usr/sys/dev/stream.c

Over on the west coast the mbuf structure is extended with a new field, “type” at the same time, and it transitions from being a pure network data buffer to a more generic buffer. It remains at a fixed size of 128 bytes with inline data; the option to store data in a separate block of varying size will only come back late in 4.3BSD. For the 4.2 version:
https://minnie.tuhs.org/cgi-bin/utree.pl?file=4.2BSD/usr/src/sys/h/mbuf.h

When viewed at this code level it would seem to me that by various routes everybody had arrived at more or less the same the point in network buffering approach by 1981. That has some interesting implications. At this level, is there really a difference between a pipe implemented as cross connected streams and a pipe implemented as cross connected sockets? Is an unbound socket not about the same as an unconnected stream head?

But before getting into such discussions, I’d really like to get the development story from 7th to 8th edition as clear and precise as possible after 40 years.

> On 30 Mar 2020, at 21:31, Clem Cole <clemc at ccc.com> wrote:
> 
> 
> 
> On Mon, Mar 30, 2020 at 12:02 PM Paul Ruizendaal <pnr at planet.nl> wrote:
> Comments in line:
> 
> > On 30 Mar 2020, at 15:26, Clem Cole <clemc at ccc.com> wrote:
> Your date of June '81 for the 4.1BSD release seems late, but I'll accept it.  3BSD was 1979, and I thought 4BSD was a year later, with 4.1BSD a few months after 4BSD (few people actually got 4BSD)
> 
> I am aware of that. “20 years of Berkely Unix” says November 1980 for 4BSD and June 1981 for 4.1BSD. From the SCCS log I’d say November 9th and July 1st respectively.
> Ok, that sounds right 6-7 months between them.   That was the time of the 'FASTVAX' work and the fight with Stanford over if BSD or VMS was to be the official Arpa OS.  I remember Joy was incredibly prolific during that time.  From a user standpoint, 4BSD and 4.1BSD are really similar, but he was hacking the kernel.  It seemed like a new change came out all the time.  The CAD group (where I was) always wanted the fastest system, so there was pressure on me to follow him, but I was trying to make progress on the AP work (my thesis) so I was a little loath to take the churn.
> 
>  
> 
> 
> As far as I can tell CSRG integrated the BBN stack with (almost) 4.1 in April 1981 - most of that is in a separate tree, with a few #ifdef’s in the main tree.
> I'm pretty sure that was mostly Eric Cooper (ecc) IIRC.  We had a tape directly from Gurwitz running in the CAD lab in Cory Hall, running on Xerox 3M Ethernet and because we needed it between our three 780s systems. We had ordered the 3C501's from 3COM and we were going to be the Beta for the Interlan board - which I wrote the original driver and gave it to Sam.  I'm not sure when IngVAX arrived, Eric Schmidt had Ing70 downstairs (and that was the ArpaNet connect via a VDH up the hill to the IMP at LBL.  We had been running BerkNET over serial lines.  Ernie and Kim were the two 780s over in Evan's (the big pile of 750s and the C30 IMP were still in the future).
> 
> 
>  
> From SCCS it seems that implementation work on sockets started in Oct/Nov 1981 and 4.1a was released in April or March 1982.
> That sounds about right.  It was a bunch of wnj marathon hacking sessions.
> 
>  
> From the partial coverage in SCCS I’d say there is no BBN code in 4.1a anymore.
> Mumble ...   I sort of have a hard time with that.  I knew and worked with both of them in my day and have heard both sides of the story. Joy had Gurwitz's code and was hacking it - that is just a fact.  Claiming pure authorship is a tad extreme.   For instance, Andy VanDam told me that the whole mbuf's stuff is something Rob originally wrote for a project for him at Brown when he was an undergrad.  Rob brought it BBN and he used it for the OS-independent TCP (i.e. the HP3000 version) originally and it landed in the UNIX version.  Joy hacked on it, but that's a data structure directly from Rob.    On the other hand, Joy types open curly brace, close curly brace, and hacks in code the fastest of anyone I ever knew (someone, maybe Mary Ann Horton, once said he 'wrote code at 9600').  But if you look at it, its sometimes not pretty.  It works and its usually fast, which is hard to argue with.
> 
> 
>  
> The socket API in that release was different from what ended being in 4.1c and 4.2,
> Definitely.   I thought other hands besides Joy started to mess with it (like Sam)
> 
>  
> and seems to have had some wider visibility in 2.9BSD and UniPlus System V - both of which used the 4.1a API (and probably code base).
> Yeah, Keith was taking things was Sam at pretty hectic pace trying to keep the 11's running.    I would guess that took more of a 'good enough' attitude, they got the TCP stack in the limited address space and did not want to try much more
> 
> 
> Yes, Summer 81 - Summer 82 must have been a vibrant year, with much interaction between various Unix groups.
> It was a fun time to be apart of it all.
> 
>  
> By the way, dmr was on the 4.2BSD steering group.
> Yes, I know.   That's how I first got to know him. 