Below some additional thoughts on the various observations posted
about this. Note that I was not a contemporary of these developments,
and I may stand corrected on some views.
> I'm pretty sure the two main System V based TCP/IP stacks were STREAMS
> based: the Lachman one (which I ported to the ETA-10 and to SCO Unix)
> and the Mentat one that was done for Sun. The socket API was sort of
> bolted on top of the STREAMS stuff, you could get to the STREAMS stuff
> directly (I think, it's been a long time).
Yes, that is my understanding too. I think it goes back to the two
roots of networking on Unix: the 1974 Spider network at Murray Hill and
the 1975 Arpanet implementation of the UoI.
It would seem that Spider chose to expose the network as a device, whereas
UoI chose to expose it as a kind of pipe. This seems to have continued in
derivative work (Datakit/streams/STREAMS and BBN/BSD sockets respectively).
When these systems were developed networking was mostly over serial lines,
and to use serial drivers was not illogical (i.e. streams->STREAMS). By 1980
fast local area networks were spreading, and the idea to see the network as
a serial device started to suck.
Much of the initial modification work that Joy did on the BBN code was to
make it perform on early ethernet -- it had been designed for 50 kbps arpanet
links. Some of his speed hacks (such as trailing headers) were later
Interestingly, Spider was conceived as a fast network (1.5 Mbps); the local
network at Murray Hill operated at that speed, and things were designed to
work over long distance T1 connections as well. This integrated fast LAN/WAN
idea seems to have been abandoned in Datakit. I have a question out to Sandy
Fraser to ask about the origins of this, but have not yet received a reply.
> The sockets stuff was something Joy created to compete with the CMU Accent
> networking system. [...] CMU was developing Accent on the Triple Drip
> PascAlto (aka the Perq) and had a formal networking model that was very clean
> and sexy. There were a lot of people interested in workstations, the Andrew
> project (MIT is about to start Athena etc). So Bill creates the sockets
> interface, and to show that UNIX could be just as modern as Accent.
I've always thought that the Joy/Leffler API was a gradual development of
the UoI/BBN API. The main conceptual change seem to have been support for
multiple network systems (selectable network stack, expansion
of the address space to 14 bytes).
I don't quite see the link to Accent and Wikipedia offers little help here
Could you elaborate on how Accent networking influenced Joy's sockets?
> * There's no reason for
> a separate listen() call (it takes a "backlog" argument but
> in practice everyone defaults it and the kernel does strange
> manipulations on it.)
Perhaps there is. The UoI/BBN API did not have a listen() call;
instead the open() call - if it was for a listening connection - blocked until
a connection occurred. The server process would then fork of a worker process
and re-issue the listening open() call for the next connection. This left a
time gap where the server would not be 'listening'.
The listen() call would create up to 'backlog' connection blocks in the
network code, so that this many clients could connect simultaneously
without user space intervention. Each accept() would hand over a (now
connected) connection block and add a fresh unconnected one to the backlog
list. I think this idea came from Sam Leffler, but perhaps he was inspired
by something else (Accent?, Chaos?)
Of course, this can be done with fewer system calls. The UoI/BBN system
used the open() call, with a pointer to a parameter data block as the 2nd
argument. Perhaps Joy/Leffler were of the opinion that parameter data
blocks were not very Unix-y, and hence spread it out over
The UoI choice to overload the open() call and not create a new call
(analogous to the pipe() call) was entirely pragmatic: they felt this
was easier for keeping up with the updates coming out of Murray Hill
all the time.
> In particular, I have often thought that it would have been a better
> and more consistent with the philosophy to have it implemented as
> open("/dev/tcp") and so on.
I think this is perhaps an orthogonal topic: how does one map network names
to network addresses. The most ambitious was perhaps the "portal()" system
call contemplated by Joy, but soon abandoned. It may have been implemented
in the early 90's in BSD, but I'm not sure this was fully the same idea.
That said, making the the name mapping a user concern rather than a kernel
concern is indeed a missed opportunity.
Last and least, when feeling argumentative I would claim that connection
strings like "/dev/tcp/host:port" are simply parameter data blocks encoded
in a string :^)
> This also knocks out the need for
> SO_REUSEADDR, because the kernel can tell at the time of
> the call that you are asking to be a server. Either someone
> else already is (error) or you win (success).
Under TCP/IP I'm not sure you can. The protocol specifies that you must
wait for a certain period of time (120 sec, if memory serves my right)
before reusing an address/port combo, so that all in-flight packets have
disappeared from the network. Only if one is sure that this is not an
issue one can use SO_REUSEADDR.
> Also, the profusion of system calls (send, recv, sendmsg, recvmsg,
> recvfrom) is quite unnecessary: at most, one needs the equivalent
> of sendmsg/recvmsg.
Today that would indeed seem to make sense. Back in 1980 there seems
to have been a lot of confusion over message boundaries, even in
stream connections. My understanding is that originally send() and
recv() were intended to communicate a borderless stream, whereas
sendmsg() and recvmsg() were intended to communicate distinct
messages, even if transmitted over a stream protocol.
> On Sep 23, 2017, at 3:06 PM, Nelson H. F. Beebe <beebe(a)math.utah.edu> wrote:
> Not that version, but I have the 4.4BSD-Lite source tree online with
> these files in the path 4.4BSD-Lite/usr/src/usr.bin/uucp:
Thanks, but I have the 44BSD CDs.
> If they look close enough to what you need, I can put
> a bundle online for you.
I'm looking for the seismo/uunet version that Rick hacked on for so many years. It started off as the 4.3BSD version, but grew to embrace the volume of traffic uunet handled in its heyday. It wasn't your daddy's uucico ;-)
Dario Niedermann <dario(a)darioniedermann.it> wrote on Sat, 23 Sep 2017
>> I just can't forgive FreeBSD for abandoning the proc filesystem ...
It can be there, if you wish.
Here are two snippets from a recent log of a recent "pkg update -f ;
pkg upgrade" run on a one of my many *BSD family systems, this one
running FreeBSD 11.1-RELEASE-p1:
Message from openjdk8-8.131.11:
This OpenJDK implementation requires fdescfs(5) mounted on
/dev/fd and procfs(5) mounted on /proc.
If you have not done it yet, please do the following:
mount -t fdescfs fdesc /dev/fd mount -t procfs proc
To make it permanent, you need the following lines in
fdesc /dev/fd fdescfs rw 0 0 proc /proc procfs rw 0 0
Message from rust-1.18.0:
Printing Rust backtraces requires procfs(5) mounted on /proc .
If you have not already done so, please do the following:
mount -t procfs proc /proc
To make it permanent, you need the following lines in /etc/fstab:
proc /proc procfs rw 0 0
I've seen such messages in many package installations in the *BSD
family, and I generally add the suggested lines to /etc/fstab.
Perhaps others more familiar with BSD internals might comment on
whether it is many non-BSD software, like the Java Developer's Kit,
and Mozilla's rust language, that mostly would like /proc support, or
whether there are plenty of native-BSD packages that expect it too.
The second edition of
Marshall Kirk McKusick, George V. Neville-Neil, and Robert N. M. Watson
The Design and Implementation of the FreeBSD Operating System
ISBN 0-201-70245-2 (hardcover), 0-321-96897-2 (hardcover)
has 5 pages with mention of the /proc filesystem, and nothing that
suggests that it is in any way deprecated.
- Nelson H. F. Beebe Tel: +1 801 581 5254 -
- University of Utah FAX: +1 801 581 4148 -
- Department of Mathematics, 110 LCB Internet e-mail: beebe(a)math.utah.edu -
- 155 S 1400 E RM 233 beebe(a)acm.org beebe(a)computer.org -
- Salt Lake City, UT 84112-0090, USA URL: http://www.math.utah.edu/~beebe/ -
Sadly no longer with us (he exited in 2011), he was forked in 1941. Just
think, if it wasn't for him and Ken, we'd all be running Windoze, and
thinking it's wonderful.
A Unix bigot through and through, I remain,
Dave Horsfall DTM (VK2KFU) "Those who don't understand security will suffer."
Tom Ivar Helbekkmo:
Why should anyone need to? Of all the mailing lists I'm on, this one is
the only one that has this problem.
Beware tunnel vision. Another mailing list I'm on has exactly
the same problem, made worse because it's being run by a central
Big Company Mailing List Provider so the rules keep changing under
foot and it's up to the poor-sod list maintainer (who is not a
programmer) to cope.
To bring the focus back to this mailing list, not every program
runs on a little-endian computer with arbitrary word alignment
and pointers that fit in an int.
Does anyone have a copy of Rick's uunet version of the 4.3BSD UUCP source? The disk I had it on seized up, and I can't figure out a fine-grained-enough set of search keywords to find it through a web search :-(
I really like mk. 8ed was where it first rolled out? I remember
reading about it in the 10ed books. It's a joy to use in Plan 9.
Later than that. I was around when Andrew wrote mk, so it
definitely post-dated the 8/e manual.
mk does a number of things better, but to my mind not quite
enough to justify carrying it around. Just as I decided long
ago (once I'd come out of the ivory hothouse of Murray Hill)
that I was best off writing programs that hewed to the ISO C
and POSIX interfaces (and later put some work into bringing
my private copy of post-V10 nearer to the standards), because
that way I didn't have to think much about porting; so I
decided eventually that it is better just to use make.
As with any other language, of course, it's best to use it
in as simple a way as possible. So I don't care much whether
it's gmake or pmake or qmake as long as it implements more
or less the 7/e core subset without breaking anything.
I do wish that some simple make had stuffed a scripting language in there.
Anything, tcl, lua, even (horrors, can't believe I'm saying this) a little
lisp. Or ideally a built in shell compat language. All those backslashes
to make shell scripts work get old.
This is something mk got right, and it's actually very simple to do:
every recipe is a shell script. Not a collection of lines handed
one by one to the shell, but a block of text. No backslashes (or
extra semicolons) required for tests. Each script is run with sh -e,
so by default one failed command will abort the rest, which is
usually what one wants; but if you don't want that you just insert
(So it's not that I dislike mk. Were it available as an easy
add-on package on all modern systems, rather than something I'd
have to carry around and compile, I'd be happy to use it.)
This question is motivated by the posters for whom FreeBSD is not Unix
Probably the best known contribution of the Berkeley branch of Unix is
the sockets API for IP networking. But today, if for no other reason
than the X/Open group of standards, sockets are the preferred networking
API everywhere, even on true AT&T derived UNIX variants. So they must
have been merged back at some point, or reimplemented. My question is,
when and how did that happen?
And if there isn't a simple answer because it happened at different
times and in different ways for each variant, all the better :-)
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
Do obvious transformation on domain to reply privately _only_ on Usenet.
I run my own mail server, on systems in my basement.
It is a setup that no one in their right mind would
replicate, but the details may actually be proper for
A firewall/gateway system runs a custom SMTP server,
which can do simple filtering based on the SMTP envelope,
SMTP commands, calling IP address and hostname. It is
also able to call external commands to pass judgement on
a caller or a particular message.
If mail is accepted, it is passed through a simple
MTA and a stupidly-simple queueing setup (the latter
made of shell scripts) to be sent via SMTP to a
different internal system, which uses the same SMTP
server and MTA to deliver to local mailboxes.
Outbound mail is more or less the obvious inverse.
I have put off naming names for dramatic effect. The
two systems in question are MicroVAX IIIs running
my somewhat-hacked-up version of post-10/e Research
UNIX. The MTA is early-1990s-vintage upas. The SMTP
server, SMTP sender, and queuing stuff are my own.
I wrote the SMTP server originally not long after I left
Bell Labs; I was now in a world where sendmail was the
least-troublesome MTA, but in those days every month
brought news of a new sendmail vulnerability, so I wrote
my own simple server to act as a condom. Over time it
grew a bit, as I became interested in problems like
what sorts of breakin attempts are there in real life
(back then one received occasional DEBUG or WIZ commands,
but I haven't seen any since the turn of the century);
what sorts of simple filtering at the SMTP level will
get rid of most junk mail. The code is more complicated
than it used to be, but is still small enough that I am
reasonably confident that it is safe to expose to the
The SMTP sender and the queueing scripts came later,
when I decided to host my own mail. Both were designed
in too much of a hurry.
There is no official spam filtering (no bogofilter or
the like). A few simple rules that really just enforce
aspects of the SMTP standard seem to catch most junk
callers: HELO argument must contain at least one . (standard
says it must be your FQDN) and must not be *.* (I see dozens
of those every day!); sender must not speak until my server
has issued a complete greeting (I follow Wietse Venema in
this: send a line with a continuation marker first, then
sleep five seconds or so, then send a finish). I also
have a very simple, naive greylisting implementation that
wouldn't work well for a site with lots of users, but is
fine for my personal traffic. The greylisting is implemented
with a pair of external shell scripts.
I have had it in mind for a long time to consult the Spamhaus
XBL too. It would be easy enough to do with another plug-in
shell script. There are stupid reasons having to do with my
current DNS setup that make that impractical for now.
The mail setup works, but is showing its age, as is the
use of Research UNIX and such old, slow hardware as a network
gateway. One of these years, when I have the time, I'd like
first to redo the mail setup so that mailboxes are stored
on my central file server (a Sun X2200 running Solaris 10,
or perhaps something illumos-based by the time I actually
do all this); then set up a new gateway, probably based on
OpenBSD. Perhaps I should calculate how much hardware I
could buy from the power savings of turning off just one of
the two MicroVAXes for a year.
I have yet to see an MTA that is spare enough for my taste,
but the old upas code just doesn't quite do what I want any
more, and is too messy to port around. (Pursuant to the
conversation earlier here about autoconf: these days I try
to need no configuration magic at all, which works as long
as I stick to ISO C and POSIX and am careful about networking.
upas was written in messier days.) At the moment I'm leaning
toward qmail, just because for other reasons I'm familiar with
it, though for my personal use I will want to make a few changes
here and there. But I'll want to keep my SMTP server because
I am still interested in what goes on there.