25 Jun
2022
25 Jun
'22
11:01 p.m.
Wanted to post my notes as plain text, but the bullets / sub-bullets get lost.
Here is a 2 page PDF with my notes on Research Datakit:
https://www.jslite.net/notes/rdk.pdf
The main takeaway is that connection build-up and tear-down is considerably more expensive
than with TCP. The first cost is in the network, which builds up a dedicated path for each
connection. Bandwidth is not allocated/reserved, but a path is and routing information is
set up at each hop. The other cost is in the relatively verbose switch-host communication
in this phase. This compares to the 3 packets exchanged at the hosts’ driver level to set
up a TCP connection, with no permanent resources consumed in the network.
In compensation, the cost to use a connection is considerably lower: the routing is known
and the host-host link protocol (“URP") can be light-weight, as the network
guarantees in-order delivery without duplicates but packets may be corrupted or lost (i.e.
as if the connection is a phone line with a modem). No need to deal with packet
fragmentation, stream reassembly and congestion storms as in the TCP of the early 80’s.
Doing UDP traffic to a fixed remote host is easily mapped to using URP with no error
correction and no flow control. Doing UDP where the remote host is different all the time
is not practical on a Datakit network (i.e. a virtual circuit would be set up anyway).
A secondary takeaway is that Research Datakit eventually settled on a three-level ascii
namespace: “area/trunk/switch”. On each switch, the hosts would be known by name, and each
connection request had a service name as parameter. In an alternate reality we would maybe
have used “ca/stclara/mtnview!google!www” to do a search.
25 Jun
25 Jun
11:09 p.m.
Nice. Any chance you want to do a TCP/XTP comparison?
On Sun, Jun 26, 2022 at 01:01:07AM +0200, Paul Ruizendaal wrote:
Wanted to post my notes as plain text, but the bullets
/ sub-bullets get lost.
Here is a 2 page PDF with my notes on Research Datakit:
https://www.jslite.net/notes/rdk.pdf
The main takeaway is that connection build-up and tear-down is considerably more
expensive than with TCP. The first cost is in the network, which builds up a dedicated
path for each connection. Bandwidth is not allocated/reserved, but a path is and routing
information is set up at each hop. The other cost is in the relatively verbose switch-host
communication in this phase. This compares to the 3 packets exchanged at the hosts???
driver level to set up a TCP connection, with no permanent resources consumed in the
network.
In compensation, the cost to use a connection is considerably lower: the routing is known
and the host-host link protocol (???URP") can be light-weight, as the network
guarantees in-order delivery without duplicates but packets may be corrupted or lost (i.e.
as if the connection is a phone line with a modem). No need to deal with packet
fragmentation, stream reassembly and congestion storms as in the TCP of the early 80???s.
Doing UDP traffic to a fixed remote host is easily mapped to using URP with no error
correction and no flow control. Doing UDP where the remote host is different all the time
is not practical on a Datakit network (i.e. a virtual circuit would be set up anyway).
A secondary takeaway is that Research Datakit eventually settled on a three-level ascii
namespace: ???area/trunk/switch???. On each switch, the hosts would be known by name, and
each connection request had a service name as parameter. In an alternate reality we would
maybe have used ???ca/stclara/mtnview!google!www??? to do a search.
--
---
Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat
11:57 p.m.
One of the things we liked about Datakit was that the computer didn't have
to establish the connection before it could reject the call, unlike TCP/IP
where all validation happens after the connection is made. This is also why
sockets and Datakit never worked together; sockets pretty much assume
Ethernet-like connection rules.
I am not a networking expert, but to me in this regard at least Datakit
seemed like a prettier picture. I suppose you can DOS-attack the network,
but not the machines. Datakit had other issues, for sure, like the
expensive racks of hardware, but then that's because, for better and worse,
it was designed by phone engineers rather than.... however you'd
characterize Ethernet and its original "I scream while listening to your
whisper", 5V into 50Ω Schmidt-triggered craziness. Ethernet's come a long
way, but the engineering of the original Radio Shack parts was not favored
by the Bell Labs crowd.
-rob
On Sun, Jun 26, 2022 at 9:09 AM Larry McVoy <lm(a)mcvoy.com> wrote:
Nice. Any chance you want to do a TCP/XTP
comparison?
On Sun, Jun 26, 2022 at 01:01:07AM +0200, Paul Ruizendaal wrote:
Wanted to post my notes as plain text, but the
bullets / sub-bullets get
lost.
Here is a 2 page PDF with my notes on Research Datakit:
https://www.jslite.net/notes/rdk.pdf
The main takeaway is that connection build-up and tear-down is
considerably more
expensive than with TCP. The first cost is in the
network, which builds up a dedicated path for each connection. Bandwidth is
not allocated/reserved, but a path is and routing information is set up at
each hop. The other cost is in the relatively verbose switch-host
communication in this phase. This compares to the 3 packets exchanged at
the hosts??? driver level to set up a TCP connection, with no permanent
resources consumed in the network.
In compensation, the cost to use a connection is considerably lower: the
routing
is known and the host-host link protocol (???URP") can be
light-weight, as the network guarantees in-order delivery without
duplicates but packets may be corrupted or lost (i.e. as if the connection
is a phone line with a modem). No need to deal with packet fragmentation,
stream reassembly and congestion storms as in the TCP of the early 80???s.
Doing UDP traffic to a fixed remote host is easily mapped to using URP
with no
error correction and no flow control. Doing UDP where the remote
host is different all the time is not practical on a Datakit network (i.e.
a virtual circuit would be set up anyway).
A secondary takeaway is that Research Datakit eventually settled on a
three-level
ascii namespace: ???area/trunk/switch???. On each switch, the
hosts would be known by name, and each connection request had a service
name as parameter. In an alternate reality we would maybe have used
???ca/stclara/mtnview!google!www??? to do a search.
--
---
Larry McVoy Retired to fishing
http://www.mcvoy.com/lm/boat
26 Jun
26 Jun
1:17 a.m.
On 26 Jun 2022, at 01:57, Rob Pike
<robpike(a)gmail.com> wrote:
One of the things we liked about Datakit was that the computer didn't have to
establish the connection before it could reject the call, unlike TCP/IP where all
validation happens after the connection is made. This is also why sockets and Datakit
never worked together; sockets pretty much assume Ethernet-like connection rules.
I am not a networking expert, but to me in this regard at least Datakit seemed like a
prettier picture. I suppose you can DOS-attack the network, but not the machines. Datakit
had other issues, for sure, like the expensive racks of hardware, but then that's
because, for better and worse, it was designed by phone engineers rather than.... however
you'd characterize Ethernet and its original "I scream while listening to your
whisper", 5V into 50Ω Schmidt-triggered craziness. Ethernet's come a long way,
but the engineering of the original Radio Shack parts was not favored by the Bell Labs
crowd.
I was not putting Datakit down, just trying to explain why the V8 approach to networking
may seem a little odd from a 1980’s TCP/IP perspective, but makes perfect sense from a
Datakit perspective.
In the end technology often becomes a hybrid of various solutions, and maybe in this case
as well. By coincidence there was a post in the Internet History mailing list earlier
today that appears to make this point.
In his video (https://www.youtube.com/watch?v=ojRtJ1U6Qzw), Sandy explains why he became
dissatisfied with Spider and the main reason was that doing switching/routing on a mini
computer was just plain inefficient as compared to a telephone switch (at 37:06). This was
1972. The result was a new design, Datakit, that could route/switch packets at high speed
and in parallel.
On the internet history list, someone quipped: "Yeah, back then the joke was that
McQuillan was the only one making money from ATM. :-) That did change in a big way (for a
while) in the late 90s and early 2000s, before router silicon caught up."
To this Craig Partridge responded: "Wasn't just router silicon -- it was router
design. What made ATM appealing is that it made the inside of the router or switch
parallel, which was necessary to push into multigigabit rates. Folks had to figure out how
to rework an Internet router to be parallel and it took at least two major innovations:
fully-standalone forwarding tables with associating forwarding engines and breaking
packets apart (essentially into cells), squirting those parts through the parallel
backplane, and then reassembling the packet at the outbound interface for
transmission." This was around 2000.
It is not my field of expertise, but it would seem to me that Sandy had figured out a core
problem some 30 years before the TCP/IP world would come up with a similar solution. I
would not even be surprised if I learned that modern telco routers transparantly set up
virtual circuits for tcp traffic.
2 Jul
2 Jul
2:51 a.m.
On 6/25/22 7:17 PM, Paul Ruizendaal wrote:
In his video
(https://www.youtube.com/watch?v=ojRtJ1U6Qzw), Sandy
explains why he became dissatisfied with Spider and the main reason
was that doing switching/routing on a mini computer was just plain
inefficient as compared to a telephone switch (at 37:06). This was
1972. The result was a new design, Datakit, that could route/switch
packets at high speed and in parallel.
I didn't realize, or even fathom that something preceded ATM.
Now it seems like Spider preceded Datakit which preceded ATM. Very
interesting.
--
Grant. . . .
unix || die
2:57 a.m.
On Fri, Jul 01, 2022 at 08:51:53PM -0600, Grant Taylor via TUHS wrote:
On 6/25/22 7:17 PM, Paul Ruizendaal wrote:
They were Bell Labs, funded by the phone company. It makes sense that
they would look at networking through the phone company lens.
In his video
(https://www.youtube.com/watch?v=ojRtJ1U6Qzw), Sandy explains
why he became dissatisfied with Spider and the main reason was that doing
switching/routing on a mini computer was just plain inefficient as
compared to a telephone switch (at 37:06). This was 1972. The result was a
new design, Datakit, that could route/switch packets at high speed and in
parallel.
I didn't realize, or even fathom that something preceded ATM.
Now it seems like Spider preceded Datakit which preceded ATM. Very
interesting. 
28 Jun
28 Jun
10:38 a.m.
On Sun, Jun 26, 2022 at 09:57:17AM +1000, Rob Pike wrote:
One of the things we liked about Datakit was that the
computer didn't have
to establish the connection before it could reject the call, unlike TCP/IP
where all validation happens after the connection is made.
Nor does TCP, one can send a RST to a SYN, and reject the call before it is
established. That would then look to the caller just like a non listening
endpoint, unless one added data with the RST.
So this is really just a consequence of the sockets API, and the current implementations.
I've a vague recall of folks suggesting ways to expose that facility via the sockets
layer, possibly using setsockopt(), but don't know if anyone ever did it.
As I recall that TCP capability was actually exposed via the TLI/XTI API,
and (for some STREAMS based TCP stacks) it did function. Although I may be
thinking of embedded STREAMS TCP stacks, not unix based stacks.
Or by 'connection' are you referring to an end-to-end packet delivery,
and that Datakit allowed a closer switch to reject a call before the packet
got to the far end?
DF
12:36 p.m.
I am not a networking expert. I said that already. The issue could well be
a property more of sockets than TCP/IP itself, but having the switch do
some of the call validation and even maybe authentication (I'm not sure...)
sounds like it takes load off the host.
-rob
On Tue, Jun 28, 2022 at 8:39 PM Derek Fawcus <
dfawcus+lists-tuhs(a)employees.org> wrote:
On Sun, Jun 26, 2022 at 09:57:17AM +1000, Rob Pike
wrote:
One of the things we liked about Datakit was that
the computer didn't
have
to establish the connection before it could
reject the call, unlike
TCP/IP
where all validation happens after the connection
is made.
Nor does TCP, one can send a RST to a SYN, and reject the call before it is
established. That would then look to the caller just like a non listening
endpoint, unless one added data with the RST.
So this is really just a consequence of the sockets API, and the current
implementations.
I've a vague recall of folks suggesting ways to expose that facility via
the sockets
layer, possibly using setsockopt(), but don't know if anyone ever did it.
As I recall that TCP capability was actually exposed via the TLI/XTI API,
and (for some STREAMS based TCP stacks) it did function. Although I may be
thinking of embedded STREAMS TCP stacks, not unix based stacks.
Or by 'connection' are you referring to an end-to-end packet delivery,
and that Datakit allowed a closer switch to reject a call before the packet
got to the far end?
DF
12:45 p.m.
One of the reasons I'm not a networking expert may be relevant here. With
networks, I never found an abstraction to hang my hat on. Unlike with file
systems and files, or even Unix character devices, which provide a level of
remove from the underlying blocks and sectors and so on, the Unix
networking interface always seemed too low-level and fiddly, analogous to
making users write files by managing the blocks and sectors themselves. It
could be all sockets' fault, but when I hear networking people talk about
the protocols and stacks and routing and load shedding and ....my ears
droop. I know it's amazing engineering and all that, but why aren't we
allowed to program the I/O without all that fuss? What makes networks so
_different_? A telling detail is that the original sockets interface had
send and recv, not read and write. From day 1 in Unix land at least,
networking was special, and it remains so, but I fail to see why it needs
to be.
It just seems there has to be a better way. Sockets are just so unpleasant,
and the endless nonsense around network configuration doubly so.
Rhetorical questions. I'm not asking or wanting an answer. I'm happy to
remain a greenhorn, oblivious to the wonder.
To adapt a reference some may recognize, I just want to read 5 terabytes.
-rob
On Tue, Jun 28, 2022 at 10:36 PM Rob Pike <robpike(a)gmail.com> wrote:
I am not a networking expert. I said that already. The
issue could well be
a property more of sockets than TCP/IP itself, but having the switch do
some of the call validation and even maybe authentication (I'm not sure...)
sounds like it takes load off the host.
-rob
On Tue, Jun 28, 2022 at 8:39 PM Derek Fawcus <
dfawcus+lists-tuhs(a)employees.org> wrote:
On Sun, Jun 26, 2022 at 09:57:17AM +1000, Rob
Pike wrote:
One of the things we liked about Datakit was that
the computer didn't
have
to establish the connection before it could
reject the call, unlike
TCP/IP
where all validation happens after the connection
is made.
Nor does TCP, one can send a RST to a SYN, and reject the call before it
is
established. That would then look to the caller just like a non listening
endpoint, unless one added data with the RST.
So this is really just a consequence of the sockets API, and the current
implementations.
I've a vague recall of folks suggesting ways to expose that facility via
the sockets
layer, possibly using setsockopt(), but don't know if anyone ever did it.
As I recall that TCP capability was actually exposed via the TLI/XTI API,
and (for some STREAMS based TCP stacks) it did function. Although I may be
thinking of embedded STREAMS TCP stacks, not unix based stacks.
Or by 'connection' are you referring to an end-to-end packet delivery,
and that Datakit allowed a closer switch to reject a call before the
packet
got to the far end?
DF
1:33 p.m.
On Tue, Jun 28, 2022 at 8:46 AM Rob Pike <robpike(a)gmail.com> wrote:
One of the reasons I'm not a networking expert
may be relevant here.
With networks, I never found an abstraction to hang my hat on.
Hmm, this raises some design questions.
Unlike with file systems and files, or even Unix
character
devices, which provide a level of remove from the underlying
blocks and sectors and so on, the Unix networking interface
always seemed too low-level and fiddly, analogous to making
users write files by managing the blocks and sectors themselves.
I can see this. Sockets in particular require filling in abstruse
and mostly opaque data structures and then passing pointers
to them into the kernel. It's not so terribly onerous once one
gets it going, particularly for a streaming protocol like TCP,
but eronomically it also doesn't sit particularly well in the larger
paradigm. Something like plan9's dial() seems more in line with
the Unix model than what Unix actually gives us in terms of
TLI/Sockets/whatever. I'll note that 10th Edition had its IPC
library that handled some of this.
It could be all sockets' fault, but when I hear
networking people talk
about the protocols and stacks and routing and load shedding and
....my ears droop. I know it's amazing engineering and all that, but
why aren't we allowed to program the I/O without all that fuss?
What makes networks so _different_? A telling detail is that the
original sockets interface had send and recv, not read and write.
From day 1 in Unix land at least, networking was special, and it
remains so, but I fail to see why it needs to be.
Of course, the semantics of networking are a little different than
the (mostly) stream-oriented file model of Unix, in that datagram
protocols must be accommodated somehow and they have metadata
in the form of sender/receiver information that accompanies each
IO request. How does one model that neatly in the read/write case,
except by prepending a header or having another argument?
But the same is true of streaming to/from files and file-like things
as well. I can't `seek` on a pipe or a serial device, for obvious
reasons, but that implies that that model is not completely regular.
Similarly, writes to, say, a raw disk device that are not a multiple
of the sector size have weird semantics. The best we have done
is document this and add it to the oral lore.
One may argue that the disk device thing is a special case that
is so uncomment and only relevant to extremely low-level systems
programs that it doesn't count, but the semantics of seeking are
universal: programs that want to work as Unix filters have to
accommodate this somehow. In practice this doesn't matter much;
most filters just don't seek on their input.
Once again I am in awe of how Unix got it right for 90% of use
cases, and makes the last 10% possible, even if painful.
It just seems there has to be a better way. Sockets
are just so
unpleasant, and the endless nonsense around network
configuration doubly so.
No argument there.
Rhetorical questions. I'm not asking or wanting
an answer.
I'm happy to remain a greenhorn, oblivious to the wonder.
As we continue forward, I wonder how much this matters. We
talk about sockets, but how many programmers _actually_ reach
for that interface when they want to talk over a network? I'd
wager that _most_ of the time now days it's hidden behind a
library interface that shields the consumer from the gritty details.
Sure, that library probably uses sockets internally, but most
people probably never look under that rock.
To adapt a reference some may recognize, I just want
to read 5 terabytes.
Believe it or not, I actually had Borgmon readability.
- Dan C.
On Tue, Jun 28, 2022 at 10:36 PM Rob Pike
<robpike(a)gmail.com> wrote:
>
> I am not a networking expert. I said that already. The issue could well be a property
more of sockets than TCP/IP itself, but having the switch do some of the call validation
and even maybe authentication (I'm not sure...) sounds like it takes load off the
host.
>
> -rob
>
>
> On Tue, Jun 28, 2022 at 8:39 PM Derek Fawcus <dfawcus+lists-tuhs(a)employees.org>
wrote:
>>
>> On Sun, Jun 26, 2022 at 09:57:17AM +1000, Rob Pike wrote:
>> > One of the things we liked about Datakit was that the computer didn't
have
>> > to establish the connection before it could reject the call, unlike TCP/IP
>> > where all validation happens after the connection is made.
>>
>> Nor does TCP, one can send a RST to a SYN, and reject the call before it is
>> established. That would then look to the caller just like a non listening
>> endpoint, unless one added data with the RST.
>>
>> So this is really just a consequence of the sockets API, and the current
implementations.
>> I've a vague recall of folks suggesting ways to expose that facility via the
sockets
>> layer, possibly using setsockopt(), but don't know if anyone ever did it.
>>
>> As I recall that TCP capability was actually exposed via the TLI/XTI API,
>> and (for some STREAMS based TCP stacks) it did function. Although I may be
>> thinking of embedded STREAMS TCP stacks, not unix based stacks.
>>
>> Or by 'connection' are you referring to an end-to-end packet delivery,
>> and that Datakit allowed a closer switch to reject a call before the packet
>> got to the far end?
>>
>> DF
9:19 p.m.
On 2022, Jun 28, at 9:33 AM, Dan Cross <crossd(a)gmail.com> wrote:
On Tue, Jun 28, 2022 at 8:46 AM Rob Pike <robpike(a)gmail.com> wrote:
My theory is that networking is different because it breaks all the time.
One can get away with writing local file system programs without checking the return codes
on every single call and figuring out what you should do in the case of every possible
error.
It is much harder to get away with that sort of thing when writing network applications.
And of course there is a long history of things that look like file systems but which have
network failure modes, and treating them like they are reliable often does not end well.
Full up reliable applications with replicated storage and multiple network availability
zones and raft/paxos/byzantine generals level coding are pretty arcane.
I know I am not qualified to write one.
File systems are “good enough” that you can depend on them, mostly. Networks are not.
-L
One of the reasons I'm not a networking
expert may be relevant here.
With networks, I never found an abstraction to hang my hat on.
9:34 p.m.
My theory is that networking is different because it
breaks all the time.
This Sun research paper from 1994
https://scholar.harvard.edu/waldo/publications/note-distributed-computing
is a classic.
29 Jun
29 Jun
6:07 a.m.
Yes, I'd thought of Sun's Jini here, part of their Java push -- which IIRC
exposed network (un)reliability and tried to address it through service
lookup and limited-lifetime lease/renew for remote resource access.
Though I'm not sure it went anywhere?
(or how TUHS-relevant this is/was... apologies if not)
-----
Hmm, apparently Jini became Apache River; last release in 2016, then
retired Feb 2022:
https://en.wikipedia.org/wiki/Jini
On Wed, 29 Jun 2022, 05:36 Richard Salz, <rich.salz(a)gmail.com> wrote:
My theory is
that networking is different because it breaks all the time.
This Sun research paper from 1994
https://scholar.harvard.edu/waldo/publications/note-distributed-computing
is a classic.
28 Jun
28 Jun
4:11 p.m.
On 6/28/22 8:45 AM, Rob Pike wrote:
One of the reasons I'm not a networking expert
may be relevant here.
With networks, I never found an abstraction to hang my hat on. Unlike
with file systems and files, or even Unix character devices, which
provide a level of remove from the underlying blocks and sectors and
so on, the Unix networking interface always seemed too low-level and
fiddly, analogous to making users write files by managing the blocks
and sectors themselves. It could be all sockets' fault, but when I
hear networking people talk about the protocols and stacks and routing
and load shedding and ....my ears droop. I know it's amazing
engineering and all that, but why aren't we allowed to program the I/O
without all that fuss? What makes networks so _different_? A telling
detail is that the original sockets interface had send and recv, not
read and write. From day 1 in Unix land at least, networking was
special, and it remains so, but I fail to see why it needs to be.
Two observations:
At the time, I think everyone was searching for the right abstraction. I
don't remember the whole talk, but just an anecdote by, I think, David
Clark. I don't remember if this was just a seminar at MIT LCS or perhaps
at SIGOPS. In any case, David talked about trying to get some database
people to use the virtual memory and file systems abstractions that had
been built by Multics. They agreed that these were nice abstractions,
but in the mean time, "get out of our way and let us at the disk."
Since networking developers and users were all searching for the right
abstraction, and new hardware, protocols, and software interfaces were
being proposed on what seemed like a weekly basis, many of the proposed
interfaces tried to expose low level mechanisms as well as high level
stream abstractions, preserving the hope that something like TCP could
be implemented at user code level rather than the kernel.
Secondly, I had to dig up a reference for the Chaosnet software (MIT AI
memo 628 available at
http://bitsavers.trailing-edge.com/pdf/mit/ai/AIM-628_chaosnet.pdf and
probably other places). The Unix implementation used the rest of the
path name to specify connection setup parameters in the typical case
which seemed more unix-like than sockets. But the Chaosnet software was
definitely swept away in the Ethernet/sockets storm surge.
6:28 p.m.
On 6/28/22 05:45, Rob Pike wrote:
[...] I know it's amazing engineering and
all that, but why aren't we allowed to program the I/O without all that
fuss? What makes networks so _different_? A telling detail is that the
original sockets interface had send and recv, not read and write. From
day 1 in Unix land at least, networking was special, and it remains so,
but I fail to see why it needs to be.
It just seems there has to be a better way. Sockets are just so
unpleasant, and the endless nonsense around network configuration doubly so.
I was pretty sad when netchans were discontinued. A colleague was
sufficiently attached to them that he kept his own branch of the library
going for a while.
john
12:47 p.m.
On Jun 28, 2022, at 05:36, Rob Pike
<robpike(a)gmail.com> wrote:
I am not a networking expert. I said that already. The issue could well be a property
more of sockets than TCP/IP itself, but having the switch do some of the call validation
and even maybe authentication (I'm not sure...) sounds like it takes load off the
host.
Some years ago, we set up a front end email server to reject incoming message attempts
that didn't match our list of valid users. This resulted in a better then 90%
reduction.
-r
1:13 p.m.
In the mid-1980s I returned to IBM Research after finishing up at CMU. I
smuggled a bunch of Sun machines in and strung Ethernet between my office
and my lab so that the desktop could talk to the server.
Then I went around IBM giving talks about TCP/IP and why IBM should commit
to it. At the time IBM Research was the center of development of IBM's SNA
stuff, so there was some (!) tension. (Particularly because Paul Greene,
one of the key leaders of the SNA work, was very close to my undergraduate
mentor, so I had socialized with him.) They proposed running TCP/IP
encapsulated in SNA, but I told them that the best they could expect was to
encapsulate SNA in TCP/IP. That turned out to be what happened.
My perception of the debate at the time was that it pitted proprietary
networking (SNA, DECNet, ...) against open networking (TCP/IP). The
hardware vendors wanted proprietary networking to lock customers into their
equipment, but that dog would not hunt.
Meanwhile, our community had recently discovered how horrible proprietary
tech was for our careers ... the mid-1980s recession led to serious layoffs
in the system programmer community and the newly unemployed geeks
discovered that the skills so assiduously honed were not portable. Enter
FSK and the open source movement.
It was pretty clear that except for the clever encapsulation stuff that
Vint had done with IP, the TCP/IP world was quick and dirty and quite
slapdash. But it was non-proprietary and that is what won the race.
What I don't understand is whether Rob's observation about networking is
*fundamental* to the space or *incidental* to the implementation. I would
love to be educated on that.
Marc
=====
nygeek.net
mindthegapdialogs.com/home <https://www.mindthegapdialogs.com/home>
On Tue, Jun 28, 2022 at 8:48 AM Rich Morin <rdm(a)cfcl.com> wrote:
On Jun 28,
2022, at 05:36, Rob Pike <robpike(a)gmail.com> wrote:
I am not a networking expert. I said that already. The issue could well
be a
property more of sockets than TCP/IP itself, but having the switch do
some of the call validation and even maybe authentication (I'm not sure...)
sounds like it takes load off the host.
Some years ago, we set up a front end email server to reject incoming
message attempts that didn't match our list of valid users. This resulted
in a better then 90% reduction.
-r
2:41 p.m.
On Tue, Jun 28, 2022 at 9:15 AM Marc Donner <marc.donner(a)gmail.com> wrote:
My perception of the debate at the time was that it
pitted proprietary
networking (SNA, DECNet, ...) against open networking (TCP/IP). The
hardware vendors wanted proprietary networking to lock customers into their
equipment, but that dog would not hunt.
Metcalfe's law: "*value of a network is proportional to the square of
the
number of connected users of the system*." The problem with a walled garden
is that it can only grow as large as the walls allow.
It was pretty clear that except for the clever encapsulation stuff that
Vint had done with IP, the TCP/IP world was quick and dirty and quite
slapdash. But it was non-proprietary and that is what won the race.
Point taken, but I actually think it is more of a Christensen-style disruption
where the 'lessor technology' outstrips the more sophisticated one because
it finds/creates a new market that values that new technology for what it
is and cares less about the ways it may be 'lessor.'
I described this in a talk I did at Asilomar a few years back. This is the
most important slide:
[image: ColesLaw20190222.png]
ᐧ
3:54 p.m.
Christensen-style disruption sounds rather like Gresham's Law ("bad
money drives out good"), but I don't think the mechanism is the same:
one can hoard old silver coins and sell those at a profit for the silver
content, but there's no premium I know of for better technology --
probably because "better technology" seems to imply aesthetics, but
newer, "lessor technology" is likely to be much faster while using less
energy.
On 6/28/22 10:41 AM, Clem Cole wrote:
On Tue, Jun 28, 2022 at 9:15 AM Marc Donner <marc.donner(a)gmail.com> wrote:
My perception of the debate at the time was that it pitted
proprietary networking (SNA, DECNet, ...) against open networking
(TCP/IP). The hardware vendors wanted proprietary networking to
lock customers into their equipment, but that dog would not hunt.
Metcalfe's law: "/value of a network is proportional to the square of
the number of connected users of the system/."The problem with a
walled garden is that it can only grow as large as the walls allow.
It was pretty clear that except for the clever encapsulation stuff
that Vint had done with IP, the TCP/IP world was quick and dirty
and quite slapdash. But it was non-proprietary and that is what
won the race.
Point taken, but I actually think it is more of a
Christensen-style disruption where the 'lessor technology' outstrips
the more sophisticated one because it finds/creates a new market that
values that new technology for what it is and cares less about the
ways it may be 'lessor.'
I described this in a talk I did at Asilomar a few years back. This
is the most important slide:
ColesLaw20190222.png
ᐧ
5:05 p.m.
On Jun 28, 2022, at 6:13 AM, Marc Donner
<marc.donner(a)gmail.com> wrote:
What I don't understand is whether Rob's observation about networking is
*fundamental* to the space or *incidental* to the implementation. I would love to be
educated on that.
And there it is! THAT was the sentence--well, ahort paragraph--that jogged my memory as
to why this seemed familiar.
If you go back to _The Unix-Hater's Handbook_ (I know, I know, bear with me), one of
the things I noticed and pointed out in my review
(https://athornton.dreamwidth.org/14272.html) is how many of the targets of hatred, twenty
years down the line, turned out to be unix-adjacent, and not fundamental.
In the book, these were things like Usenet and sendmail.cf (indeed, those were the two big
ones).
But the current discussion: is the thing we don't like Berkeley Sockets? Is it
TCP/IP itself? Is the the lack of a Unixy abstraction layer over some lower-level
technology? To what degree is it inherent?
I mean, obviously, to some degree it's all three, and I think a large but fairly
unexamined part of it is that TCP/IP these days almost always at least pretends to be
sitting on top of Ethernet at the bottom...but of course Classic Ethernet largely died in
the...early 2000s, I guess?...when even extremely cheap home multiple-access-devices
became switches rather than hubs.
Some sort of inter-machine networking is clearly inherent in a modern concept of Unix. I
think we're stuck with the sockets interface and IP, whether we like them or not.
They don't bother me a great deal, but, yes, they do not feel as unixy as, say,
/dev/tcp does. But the interesting thing is that I think that is Unix-adjacent or, like
the UHH distate for Unix filesystems, it's at least incidental and could be replaced
if the desire arose. And I think we already have the answer about what the abstraction
is, albeit at an application rather than the kernel level.
To answer Rob's question: I think the abstraction is now much farther up the stack.
To a pretty good first approximation, almost all applications simply definte their own
semantics on top of HTTP(S) (OK, OK, Websockets muddy the waters again) and three-to-five
verbs. There's an incantation to establish a circuit (or a "session" if
you're under the age of 50, I guess), and then you GET, DELETE, and at least one of
PUT/POST/PATCH, for "read", "unlink", and "write". This
does seem to be a more record-oriented (kids these days get snippy if you call them
"records" rather than "objects" but w/e) format than a stream of bytes
(or at least you put an abstraction layer in between your records and the stream-of-octets
that's happening).
This is certainly not efficient at a wire protocol level, but it's a fairly small
cognitive burden for people who just want to write applications that communicate with each
other.
Adam
5:43 p.m.
I’m generally a lurker here, but this has been an interesting conversation to observe. I
often hesitate to post, as to not offend you folks who literally invented this stuff, but
I thought it might be helpful to share some experiences with socket-based I/O.
For the first time in ~20 years, I’ve recently been writing low-level (eg, not through a
library layer/framework) socket code in straight C, for an art project based on an ESP32
embedded/SoC. I first played around with the socket API in the mid-80s, and then wrote a
lot of socket-using code in the 1990s for the Watchguard Firebox, the first Linux-based
appliance firewall.
I have to say that I really enjoying programming with sockets. I feel that it *does* make
a lot of sense if I'm thinking directly about the TCP/IP stack, *and* if my code has
a good 'impedance match' to the protocols. If I’m writing a server, I’m dealing
with connections and queues and various-sized packets/messages/blocks, which have to fit
into some decision of memory usage (often true in embedded systems). Usually I’m not
simply writing, say, a file server that simply reads from disc and sends bytes out through
a stream and then calls close().
I also believe that the sockets API really comes into its own with high-capacity,
non-threaded, non-blocking servers or clients — that is, ones that use select() or poll()
and then recv() and send() or their variants. I’m sure that if I didn’t have a sockets API
and only had open(), read(), write(), etc., I could make it work, but there’s something
almost beautiful that happens at a large scale with many non-blocking sockets (see: the
reactor pattern) that I don’t think would translate as well with a typical
everything-is-a-file model.
My opinion solely, of course. But I’m simply happy that both socket- and file-based APIs
exist. Each has their purpose.
—John
On Jun 28, 2022, at 19:05, Adam Thornton
<athornton(a)gmail.com> wrote:
On Jun 28, 2022, at 6:13 AM, Marc Donner
<marc.donner(a)gmail.com> wrote:
What I don't understand is whether Rob's observation about networking is
*fundamental* to the space or *incidental* to the implementation. I would love to be
educated on that.
And there it is! THAT was the sentence--well, ahort paragraph--that jogged my memory as
to why this seemed familiar.
If you go back to _The Unix-Hater's Handbook_ (I know, I know, bear with me), one of
the things I noticed and pointed out in my review
(https://athornton.dreamwidth.org/14272.html) is how many of the targets of hatred, twenty
years down the line, turned out to be unix-adjacent, and not fundamental.
In the book, these were things like Usenet and sendmail.cf (indeed, those were the two
big ones).
But the current discussion: is the thing we don't like Berkeley Sockets? Is it
TCP/IP itself? Is the the lack of a Unixy abstraction layer over some lower-level
technology? To what degree is it inherent?
I mean, obviously, to some degree it's all three, and I think a large but fairly
unexamined part of it is that TCP/IP these days almost always at least pretends to be
sitting on top of Ethernet at the bottom...but of course Classic Ethernet largely died in
the...early 2000s, I guess?...when even extremely cheap home multiple-access-devices
became switches rather than hubs.
Some sort of inter-machine networking is clearly inherent in a modern concept of Unix. I
think we're stuck with the sockets interface and IP, whether we like them or not.
They don't bother me a great deal, but, yes, they do not feel as unixy as, say,
/dev/tcp does. But the interesting thing is that I think that is Unix-adjacent or, like
the UHH distate for Unix filesystems, it's at least incidental and could be replaced
if the desire arose. And I think we already have the answer about what the abstraction
is, albeit at an application rather than the kernel level.
To answer Rob's question: I think the abstraction is now much farther up the stack.
To a pretty good first approximation, almost all applications simply definte their own
semantics on top of HTTP(S) (OK, OK, Websockets muddy the waters again) and three-to-five
verbs. There's an incantation to establish a circuit (or a "session" if
you're under the age of 50, I guess), and then you GET, DELETE, and at least one of
PUT/POST/PATCH, for "read", "unlink", and "write". This
does seem to be a more record-oriented (kids these days get snippy if you call them
"records" rather than "objects" but w/e) format than a stream of bytes
(or at least you put an abstraction layer in between your records and the stream-of-octets
that's happening).
This is certainly not efficient at a wire protocol level, but it's a fairly small
cognitive burden for people who just want to write applications that communicate with each
other.
Adam
10:45 p.m.
New subject: HTTP (was Re: Re: Research Datakit notes)
On Tue, Jun 28, 2022 at 10:05:26AM -0700, Adam Thornton wrote:
And I think we already have the answer about what the abstraction is, albeit at an
application rather than the kernel level.
To answer Rob's question: I think the abstraction is now much farther up the stack.
To a pretty good first approximation, almost all applications simply definte their own
semantics on top of HTTP(S) (OK, OK, Websockets muddy the waters again) and three-to-five
verbs. There's an incantation to establish a circuit (or a "session" if
you're under the age of 50, I guess), and then you GET, DELETE, and at least one of
PUT/POST/PATCH, for "read", "unlink", and "write". This
does seem to be a more record-oriented (kids these days get snippy if you call them
"records" rather than "objects" but w/e) format than a stream of bytes
(or at least you put an abstraction layer in between your records and the stream-of-octets
that's happening).
Yes it is effectively records, or RPCs w/o state between requests. So akin to DNS queries
over UDP, rather than DNS queries over TCP.
As the whole HTTP model allows for proxies, at which point the request (GET, HEAD, POST,
etc) includes
an endpoint address (DNS name and port) and path at that endpoint. Having
'session' state is built at
a level above HTTP, by cookies or redirecting to magic URIs encoding the session ID after
the query (the '?').
The fact that this all happens to run over a TCP connection, simply means that we have
stacked sets of
VCs. An application VC on top of a connectionless HTTP on top of a TCP 'VC'.
One could argue that
Websockets simplifies this by ripping out the HTTP layer, and having the top VC then just
being message
framing within the TCP session.
(Ignoring for the moment TLS on top of TCP, or HTTP/3 being on top of QUIC, hence UDP)
This is certainly not efficient at a wire protocol
level, but it's a fairly small cognitive burden for people who just want to write
applications that communicate with each other.
Sort of, in that I recently investigated the Go HTTP client/server APIs as someone was
asking us about
implementing a MiTM HTTP(S) "firewall". Depending upon how one deals with it,
it hides the VCs and exposes
the connectionless RPCs.
With such a MiTM in place, the 'nice' RPC in a session is effectivly forced back
to the 'nasty' RPC in datagrams, even though the endpoints may be unaware, but
they already include the whole datagram remote address in the request. Responses come
back by 'magic' without explicit address.
So maybe from that perspective one could model HTTP request/response as RPCs over SVCs,
those being raised and torn down for each exchange.
DF
26 Jun
26 Jun
1:41 a.m.
Paul Ruizendaal <pnr(a)planet.nl> once said:
Wanted to post my notes as plain text, but the bullets
/ sub-bullets get lost.
Here is a 2 page PDF with my notes on Research Datakit:
https://www.jslite.net/notes/rdk.pdf
Really nice outline. I felt some déjà vu when opening the document: I
remember reading your notes in April of 2020.
Would you happen to know where I can find copies of these three
papers?
A. G. Fraser, "Datakit - A Modular Network for Synchronous and
Asynchronous Traffic", Proc. ICC 79, June 1979, Boston, Ma.,
pp.20.1.1-20.1.3
G. L. Chesson, "Datakit Software Architecture", Proc. ICC 79, June
1979, Boston Ma., pp.20.2.1-20.2.5
G. L. Chesson and A. G. Fraser, "Datakit Network Architecture," Proc.
Compcon 80, February 1980, San Fransisco CA., pp.59-61
A secondary takeaway is that Research Datakit
eventually settled on a
three-level ascii namespace: “area/trunk/switch”. On each switch, the
hosts would be known by name, and each connection request had a
service name as parameter. In an alternate reality we would maybe have
used “ca/stclara/mtnview!google!www” to do a search.
To connect to one of the Plan 9 cpu servers at Bell Labs, you would
dial "nj/astro/helix!9fs". I do wonder how the relative hierarchical
naming would have evolved to encompass the entire Internet. Would
it have been more like "com/google/search!http"? Who knows? ☺
Thanks,
Anthony
9:52 a.m.
Hi Anthony,
Would you happen to know where I can find copies of
these three
papers?
A. G. Fraser, "Datakit - A Modular Network for Synchronous and
Asynchronous Traffic", Proc. ICC 79, June 1979, Boston, Ma.,
pp.20.1.1-20.1.3
G. L. Chesson, "Datakit Software Architecture", Proc. ICC 79, June
1979, Boston Ma., pp.20.2.1-20.2.5
G. L. Chesson and A. G. Fraser, "Datakit Network Architecture," Proc.
Compcon 80, February 1980, San Fransisco CA., pp.59-61
I had no luck looking for those. I did find other Datakit ones
which may interest the list; all Bell Labs.
- A Virtual Circuit Switch as the Basis for Distributed Systems.
Luderer, Che, Marshall, Bell Labs, Murray Hill. ~1981.
‘...we have implemented... a new distributed operating system derived
from Unix’.
https://dl.acm.org/doi/pdf/10.1145/1013879.802670
- Methods for routing, performance management, and service problem
relief in Datakit networks.
Fendick, Harshavardhana, Jidarian, Bell Labs, Holmdel. 1991.
https://libgen.rocks/get.php?md5=27c34fe1fce3ed20b79aaf65362cb5f9&key=V…
- FIBERKIT: a Datakit compatible 500 megabit virtual circuit switch.
Follett, Levy, Sobin, Tourgee, AT&T Bell Labs. 1988.
https://libgen.rocks/get.php?md5=577344ef77634022868f52969dbda62b&key=C…
--
Cheers, Ralph.
11:04 a.m.
On 26 Jun 2022, at 03:41, Anthony Martin
<ality(a)pbrane.org> wrote:
Really nice outline. I felt some déjà vu when opening the document: I
remember reading your notes in April of 2020.
Thank you. Yes, I have not updated these notes since then.
Would you happen to know where I can find copies of
these three
papers?
A. G. Fraser, "Datakit - A Modular Network for Synchronous and
Asynchronous Traffic", Proc. ICC 79, June 1979, Boston, Ma.,
pp.20.1.1-20.1.3
G. L. Chesson, "Datakit Software Architecture", Proc. ICC 79, June
1979, Boston Ma., pp.20.2.1-20.2.5
G. L. Chesson and A. G. Fraser, "Datakit Network Architecture," Proc.
Compcon 80, February 1980, San Fransisco CA., pp.59-61
Two of these three were in Sandy Fraser’s archives:
https://www.jslite.net/notes/dk1.pdf
https://www.jslite.net/notes/dk2.pdf
The middle one is I fear lost - unless a paper copy of the proceedings is still lurking in
some university library. I think it will be mostly about the “CMC” switch control
software, which is somewhat described in the Luderer paper as well. As far as I can tell,
there is no surviving Unix code that was designed for the CMC version of Datakit (unless
someone is sitting on a V7 tape with the Datakit drivers still included).
To connect to one of the Plan 9 cpu servers at Bell
Labs, you would
dial "nj/astro/helix!9fs”.
Out of interest, would you know if the switch-host interface for Datakit in the days of
Plan9 worked more or less the same as in 8th Edition? Or had it much evolved by then?
29 Jun
29 Jun
8:21 p.m.
Would you happen to know where I can find copies of
these three
papers?
A. G. Fraser, "Datakit - A Modular Network for Synchronous and
Asynchronous Traffic", Proc. ICC 79, June 1979, Boston, Ma.,
pp.20.1.1-20.1.3
G. L. Chesson, "Datakit Software Architecture", Proc. ICC 79, June
1979, Boston Ma., pp.20.2.1-20.2.5
G. L. Chesson and A. G. Fraser, "Datakit Network Architecture," Proc.
Compcon 80, February 1980, San Fransisco CA., pp.59-61
I just remembered that I had received a copy of a file note (50+ pages) that Greg Chesson
wrote in 1982 about the "CMC” control software for Datakit. I think it covers the
same ground as the 1979 paper, but in far greater detail and with two more years of
development. In short, the connection protocol in CMC is based on the exchange of binary
messages. That was replaced (for the most part) by text-based messages in the later TDK
control software.
It is here (it is a 16MB pdf):
https://www.jslite.net/notes/dk3.pdf
To compare, here are the first two design documents on sockets. I looked for these for
many years (even had the Berkeley library manually search the boxes with CSRG documents
that Kirk McKusick had sent there - to no avail), and then in 2021 Rich Morin found them
in the papers of Jim Joyce. I’m still very thankful for this.
These two papers were written in the summer of 1981 and circulated to the newly formed
steering committee for what was to become 4.2BSD (note: ~5MB pdf each).
The first is specifically on networking:
https://www.jslite.net/notes/joy1.pdf
The second outlines the overall ambitions for the new version (including a summary of the
above document). It has an interesting view of John Reiser’s VM code in its section 3.17
as well:
https://www.jslite.net/notes/joy2.pdf
What was proposed is not quite the sockets we know, but the general direction is set and
the reasoning is explained. Reading the Chesson and Joy paper side by side makes for an
interesting comparison of thinking on these topics in the early 80’s.
Maybe they are worth storing in the TUHS archive.
Wbr,
Paul
662
days inactive
669
days old
25 comments
18 participants
participants (18)
-
Adam Thornton -
Anthony Martin -
Clem Cole -
Dan Cross -
Derek Fawcus -
Grant Taylor -
John Floren -
John Labovitz -
Larry McVoy -
Lawrence Stewart -
Marc Donner -
Paul Ruizendaal -
Ralph Corderoy -
Rich Morin -
Richard Salz -
Rob Pike -
Stuart Remphrey -
Tom Teixeira