From clemc at ccc.com Mon Jun 1 00:53:57 2020 From: clemc at ccc.com (Clem Cole) Date: Sun, 31 May 2020 10:53:57 -0400 Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) In-Reply-To: <1jfNb2-7JV-00@marmaro.de> References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> Message-ID: On Sun, May 31, 2020 at 9:02 AM markus schnalke wrote: > Would you be so kind to explain a bit about the hm version of MH. > I can try ... At the time V6 (and later V7) had a program called /bin/mail. The important thing about this program is that it was both a Mail Transport (and delivery) Agent as well as a Mail User Interface. The original version only delivered mail locally. When networking, like UUCP, was added to UNIX, it was easy to build a scheme that could send a transfer a message using UUCP and then fork /bin/mail on the proper system to deliver it. IIRC it was in Seventh Edition that /bin/mail was extended to recognize email addresses in the prefix form of: host!user, strip the first host! part and pass it to uucp. I never used it on anything like Spider, so I have not idea what it did internally and in fact, the prefix address form could have predated UUCP. We would need someone like Doug, Steve or Ken to tell us how email worked host to host, pre-UUCP. I believe it was Bruce Borden and team developed the original Rand Message Handler or MH. I do not know who was the primary author, we need to ask Bruce or one of the old Rand folks. MH ran on 6th edition when it was first released to rest of the USENIX community (at least I never saw it on V5, but it's possible it went back further). I just remember talking to Bruce about it at any early USENIX. MH used a new Mailbox (on-disk) scheme and formatted all messages in RFC733 form with addresses being flat an in the form: user at host. The new format, used a line a control-As followed by a nl, before and after the message. The headers of course where RFC733 (type: value with a trailing nl) and were separated from the body of the message by a single nl. MH assumed a traditional UNIX command line for the user interface and had a number of programs that ran behind the scene for delivery. This is important because the original ARPAnet NCP used FTP to do mail transfer. At least one version could call the early ARPAnet subsystem to perform host to host communications. The key point is that MH separated the MTA and MUI. A number of us ran MH at different places. I don't remember if it was on a USENIX tape or I had sent Bruce a tape @ Rand when I got a copy in the late 1970's (77-79 timeframe, I've forgotten). I'm pretty sure Goble, as did the Purdue crew ran it, as did Holmgrem, Greg Chesson at al, @ UofI and I would not surprised if it was the mailer at Harvard, given the Harvard/Rand connection in those days. By the time of Seventh Edition of UNIX, the Mail Transport Agent (MTA) that was part of the MH subsystem, could recognize ArpaNET address postfix and had a hack in it, that allowed the 'user' part of the address to include UUCP addresses and /bin/mail replacement knew to work like the AT&T mailer and pass it off to UUCP. So, at that point, life was good for those of us in strickly ARPA and/or UUCP land. BTW: At some point, the BBN TCP code was releases and a separate SMTPD was included in the release. I don't remember if it was the Rand folks or someone else, but at some point, the MH delivery agent was updated to call it or be called by it. Similarly, by the late 1970s, when Bruce, Greg Shaw, and Bob Metcalfe formed 3Com; Bruce and Greg brought MH with them and added an SMTPD that they wrote for their commercial product, UNET. This was the smtpd, I ran on the Teklabs machine before I went off to UCB. Meanwhile, as Mary Ann and I noted in earlier messages, Kurt Shoen's wrote a different MTA called Mail(1ucb). Unlike MH, Kurt continues to support the original UNIX mailbox format (later named 'mbox' format). The header lines were in a specific UNIX style prose starting with the ACSII 'From' and there are no special characters to demark the messages in the format, so recovery can be fraught with error, there are the famous From-line munging issues etc. (lots of details in other places start with https://en.wikipedia.org/wiki/Mbox). At some point in time, UCB built the 'Berk-net' (whose original code was written by ABC/Google's Eric Schmidt). BTW: Eric would have seen the Spider Network in his summers at BTL. The key thing with Berknet was cheap. It ran over 3 wire RS-232C between hosts at 9600 baud. Like UUCP was used to transfer files and email. Like UUCP it used a pre-fix addressing form: host:user ; but like RFC733 and unlike UUCP was flat. Where Mary Ann and I differ in our memories is who wrote the original version of UCB's delivermail (8ucb) program. We both agree that it is possible it was Eric Schmidt, as the switch to using delivermail(8ucb) was were Berknet was spliced into the email namespace. I had thought Kurt wrote it, Mary Ann thought it was Eric Allman. We agree Eric Allman was hacking on it for the Ing70. For this response, it doesn't really matter other than to try to get the history right, because it does not matter for the Rand Mail subsystem. Around, now I arrive at UCB. I was not the only person that had used MH, but I had the advantage of having 3 Vaxen 780 in the CAD lab. The key change we made at that point was to stop using the Rand MTA and make it use delivermail. There was hackery to allow the shared mbox to say, but in each user directory it stored the messages separately, MH style so the MH 'ui' suite of tools 'just worked.' [There were a number of arguments at the time. I remember having one with Sam. He hated MH because 'of all the small files and it chewed up inodes]. Anyway around this time, the curses library was created by Ken Arnold (originally to support Rogue) by pulling the screen code out of vi and using Mary Ann's termcap stuff. A couple of us in the USENIX community started playing with screen-based front ends to MH, including the folks at Rand. Numerous messages were exchanged, and a collaboration started (Rand team should get 99% percent of the work, I knew how to make curses go). The comment in the main code was, 'Well it sure isn't MH' -- so it was called HM. I later brought it Masscomp, and it was the start of their mailer. The key with HM, is that it uses the MH backend for all the real work. It would fork scan, but save the output in a text file that could be manipulated with the arrow keys. The user interface is not unlike Gmail. I only stopped running HM when I finally switched to Gmail a few years ago. In fact, when I was a DEC, the precursor to Gmail, was called Pachyderm which used the Altavista search. I glued MH to Pachyderm for a short period. I even ported them both to FreeBSD to run at home in the mid-1990s. Then I left DEC, the authors of Pachyderm headed for Google, *etc*... I got a chance to be an early Gmail/Google Apps tester for ccc.com and I have not switched back since. FWIW: It's been long enough, that the sources are no longer easy to find. It's all very incestuous. We all have seen other ideas and good ideas seem to have a way of reappearing in different places. Clem Note the part of the story I left out was when Rand MH spun out to one of the other Universities, UCI I think. I was never much involved with that team, so I can not tell you much about it and their code base. -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Mon Jun 1 02:05:47 2020 From: clemc at ccc.com (Clem Cole) Date: Sun, 31 May 2020 12:05:47 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: On Sun, May 31, 2020 at 7:10 AM Paul Ruizendaal wrote: > This behaviour seems to have continued into SysVR1, I’m not sure when > EAGAIN came into use as a return value for this use case in the SysV > lineage. Maybe with SysVR3 networking? Actually, I'm pretty sure that was a product of the POSIX discussions. BSD already had networking an EWOULDBLOCK. We had argued about EWOULDBLOCK a great deal, we also were arguing about signal semantics. I've forgotten many of the details, Heinz may remember more than I do. EAGAIN was created as a compromise -- IIRC neither system had it yet. SVR3 networking was where it went into System V, although some of the AT&T representatives were none too happy about it. > > In the Research lineage, the above SysIII approach does not seem to exist, > although the V8 manual page for open() says under BUGS "It should be > possible [...] to optionally call open without the possibility of hanging > waiting for carrier on communication lines.” In the same location for V10 > it reads "It should be possible to call open without waiting for carrier on > communication lines.” > > The July 1981 design proposals for 4.2BSD note that SysIII non-blocking > files are a useful feature and should be included in the new system. In > Jan/Feb 1982 this appears to be coded up, although not all affected files > are under SCCS tracking at that point in time. Non-blocking behaviour is > changed from the SysIII semantics, in that EWOULDBLOCK is returned instead > of 0 when progress is not possible. The non-blocking behaviour is extended > beyond TTY’s and pipes to sockets, with additional errors (such as > EINPROGRESS). At this time EWOULDBLOCK is not the same error number as > EGAIN. > My memory is that Keith was the BSD (CSRG) person at the POSIX meeting (he, Jim McGinness of DEC, and I created PAX at one point as a compromise). I wish I could remember all of the details, but this was all argued at the POSIX meetings. As I said before the folks from AT&T just wanted to take the SVID and rubber stamp it at the specification. Part of it the problem was they wanted to be free to do what things/make choices that the rest of us might or might not like (for instance, they did not want the sockets interface). > > It would seem that the differences between the BSD and SysV lineages in > this area persisted until around 2000 or so. > Yep - cause around then POSIX started to settle out and both systems began to follow it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich.salz at gmail.com Mon Jun 1 02:25:47 2020 From: rich.salz at gmail.com (Richard Salz) Date: Sun, 31 May 2020 12:25:47 -0400 Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) In-Reply-To: References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> Message-ID: "I'll try" Wow, that was quite some "try" Kudo's! > This is important because the original ARPAnet NCP used FTP to do mail transfer. If you look at RFC 765 (dated June 1980) , there are nearly a dozen mail-related FTP commands. By RFC 959, it's successor five years later, all of them were gone. Another eary MTA was the MMDF, the Multichannel Memorandum Distribution Facility, from U Delware. BBN ran it. One key point is that somehow when used with MH you could get real-time address verification before sending, "user rs at bbn.com doesn't exist." BBN ran MMDF for a long time because a key exec liked/needed that feature, long after 4.2 and sendmail. (My group was one of the first to run sendmail, which I liked because of the "R$" lines in its CF files. (Not really :) My Usenet/email gateway code had to support MMDF but it was only ever used on bbn.com) > could recognize ArpaNET address postfix and had a hack in it, that allowed the 'user' part of the address to include UUCP addresses Originally ihnp4!mirror!rs at seismo.arpa, then ihnp4!mirror!rs at seismo.css.gov and then rs%mirror.uucp at seismo.css.gov and then rs at mirror.tmc.com Thanks to Mary Ann and the "UUCP Mapping Project" for making much of that possible and Peter Honeyman for Pathlias, and MX records for the last part. The pre-MX styles received much scorn from Research, and if you search for "that hideous name" you can find a paper on it. There's also a pathlias paper, which Honey later said "was too good for Usenix" but he had nowhere else to submit it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Mon Jun 1 02:46:37 2020 From: imp at bsdimp.com (Warner Losh) Date: Sun, 31 May 2020 10:46:37 -0600 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: Sorry to top post, but LSX or Miniunix had non blocking I/O as well. It was in one of the documents that Clem scanned in the last year. It specifically was an experiment into how to do it. Warner On Sun, May 31, 2020, 10:07 AM Clem Cole wrote: > > > On Sun, May 31, 2020 at 7:10 AM Paul Ruizendaal wrote: > >> This behaviour seems to have continued into SysVR1, I’m not sure when >> EAGAIN came into use as a return value for this use case in the SysV >> lineage. Maybe with SysVR3 networking? > > Actually, I'm pretty sure that was a product of the POSIX discussions. > BSD already had networking an EWOULDBLOCK. We had argued about > EWOULDBLOCK a great deal, we also were arguing about signal semantics. > I've forgotten many of the details, Heinz may remember more than I do. > EAGAIN was created as a compromise -- IIRC neither system had it yet. > SVR3 networking was where it went into System V, although some of the AT&T > representatives were none too happy about it. > > > >> >> In the Research lineage, the above SysIII approach does not seem to >> exist, although the V8 manual page for open() says under BUGS "It should be >> possible [...] to optionally call open without the possibility of hanging >> waiting for carrier on communication lines.” In the same location for V10 >> it reads "It should be possible to call open without waiting for carrier on >> communication lines.” >> >> The July 1981 design proposals for 4.2BSD note that SysIII non-blocking >> files are a useful feature and should be included in the new system. In >> Jan/Feb 1982 this appears to be coded up, although not all affected files >> are under SCCS tracking at that point in time. Non-blocking behaviour is >> changed from the SysIII semantics, in that EWOULDBLOCK is returned instead >> of 0 when progress is not possible. The non-blocking behaviour is extended >> beyond TTY’s and pipes to sockets, with additional errors (such as >> EINPROGRESS). At this time EWOULDBLOCK is not the same error number as >> EGAIN. >> > My memory is that Keith was the BSD (CSRG) person at the POSIX meeting > (he, Jim McGinness of DEC, and I created PAX at one point as a > compromise). I wish I could remember all of the details, but this was all > argued at the POSIX meetings. > > As I said before the folks from AT&T just wanted to take the SVID and > rubber stamp it at the specification. Part of it the problem was they > wanted to be free to do what things/make choices that the rest of us might > or might not like (for instance, they did not want the sockets interface). > > > >> >> It would seem that the differences between the BSD and SysV lineages in >> this area persisted until around 2000 or so. >> > Yep - cause around then POSIX started to settle out and both systems began > to follow it. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Mon Jun 1 02:50:50 2020 From: lm at mcvoy.com (Larry McVoy) Date: Sun, 31 May 2020 09:50:50 -0700 Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) In-Reply-To: References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> Message-ID: <20200531165050.GS22016@mcvoy.com> On Sun, May 31, 2020 at 12:25:47PM -0400, Richard Salz wrote: > Originally ihnp4!mirror!rs at seismo.arpa, then ihnp4!mirror!rs at seismo.css.gov > and then rs%mirror.uucp at seismo.css.gov and then rs at mirror.tmc.com Thanks > to Mary Ann and the "UUCP Mapping Project" for making much of that possible > and Peter Honeyman for Pathlias My first real research paper was with Udi Manber and it was all about compressing the pathalias maps. I had them on a 40MB disk on a Masscomp that had 20 users. The format was seismo.css.gov geowhiz!geophiz!uwisc!ihnp4!mirror in other words destination path!to!get!there I went to Udi and explained that all of our stuff started with geowhiz!geophiz!uwisc so that string was replicated over and over and it took too much space. Once Udi understood the problem he went "You have time best case, space worst case, you can reverse them like so" seismo -> mirror mirror -> ihnp4 ihnp4 -> uwisc uswisc -> geophiz geophiz -> geowhiz and you'll have time worst case and space best case. I was like "oh, yeah, you are right, cool, thanks" and got up to leave. Not so fast, says Udi, that's not "interesting". What would be interesting is if you could approximate time best case and space best case. With that, I started down the only dynamic programming problem I have ever solved. Udi's idea was to break the graph at what he called pivot points, where the pivot points as much as possible got rid of the replicated strings. He also wanted to limit lookups to just 2. So I had to read in the maps and build a graph from them on my creaky microvax with 4MB of ram. Doing so lead me to the first place I taught Udi about systems. I was fgets() each line and mallocing a copy of it. Then I sorted with qsort() and it took overnight to get the sorted array. Udi asked what sort? Qsort. Oh, that's garbage, use a radix sort, that will be better. WTF is a radix sort, look that up, implemented it, it was worse. So I start poking around and somehow realized we were thrashing VM, we didn't fit, and I have no idea how I figured out that malloc was fragmenting memory but I did. Probably figured out that most of my program was backed by swap, not files, and that's usually malloc or brk. So I wrote my own malloc that grabbed memory in 300KB chunks (why that size, I dunno, seemed right). Then I allocated memory out of those chunks. Got rid of the fragmentation and we fit in 4MB. qsort worked fine, took about 20 minutes (we didn't really fit but we were way closer). Udi was watching as I was doing all this, I think I was working in his office, it was his microvax. The fragmentation thing taught him that when Unix says it has virtual memory and you can just use it, it only works well when you fit. So the programming problem turned out that you calculated the space it work take for every possible pivot point and then at the end, you took the set that gave you the minimum space. We wrote, heh, who am I kidding, Udi wrote, a paper about it. I was still green on writing papers, they were hard for me. I told Udi that and he stared at me blankly and said "Writing papers is easy if you know the content. You just make a good outline and fill it in." He's right, that's how you do it, and it is easy if you know and can make the outline. I went on to do a lot of work with/for him. I wrote a user level thread library that made me write swtch() in assembler for the VAX (I believe I did a 68K version too but the source I have is VAX only). I "cheated" and did 99% of the work in C and popped down the rabbit hole to assembler, did the nasty, and popped back out as a different thread. 112 lines: http://www.mcvoy.com/lm/T/src/Tasm.S Funny Udi story, I was young and wild and drove too fast and got tickets. So I got a radar detector that beeped when it saw the radar gun. Udi upgraded the microvax to some newer release of X10 or X11 and the new xclock beeped on the hour. Every time it beeped, I jumped because it sounded just like the radar detector. Udi asked about it, I explained, he laughed and said "Larry, you're hacking too fast" each time I jumped :) Good times. I almost did a PhD under him but I didn't care for the CS department at Tucson where he went and I liked industry too much. From ralph at inputplus.co.uk Mon Jun 1 03:09:24 2020 From: ralph at inputplus.co.uk (Ralph Corderoy) Date: Sun, 31 May 2020 18:09:24 +0100 Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) In-Reply-To: References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> Message-ID: <20200531170924.0609821F85@orac.inputplus.co.uk> Hi Clem, > I believe it was Bruce Borden and team developed the original Rand > Message Handler or MH. I do not know who was the primary author, we > need to ask Bruce or one of the old Rand folks. https://en.wikipedia.org/wiki/MH_Message_Handling_System#History says MH was proposed by R. Stockton Gaines and Norman Shapiro and developed by Bruce S. Borden and others at RAND Corporation. and cites https://www.rand.org/content/dam/rand/pubs/notes/2009/N3017.pdf Norman Shapiro, or Norm, is still using nmh well into his eighties, nmh is a descendant of MH, and pops up on the nmh-workers mailing list to either fill in bits of history or ask for installation advice as the Linux distro shifts under his feet. He did visit Bell Labs, though in '54. :-) Details at https://en.wikipedia.org/wiki/Norman_Shapiro > The new format, used a line a control-As followed by a nl, before and > after the message. MH's mts.conf(5) had the ability to set mmdelim{1,2} to something other than \ca\ca\ca\ca\n; no idea why. And that magic marker would be escaped if it occurred in the middle of a message, to \cb\ca\ca\ca\n. > Note the part of the story I left out was when Rand MH spun out to one > of the other Universities, UCI I think. Wikipedia again: Subsequently development was taken over by Marshall T. Rose and John L. Romine working at University of California, Irvine. nmh... was forked [from that] by Richard Coleman while working at the Georgia Institute of Technology. It incorporates the "LBL changes" made to MH in the late 80s by Van Jacobson, Mike Karels and Craig Leres. MH also had support for UCI's bulletin boards called ‘BBoards’, including on ZOTnet. They were optionally available over POP3 and NNTP, but only one or the other. :-) There was an MH shell of some kind called msh, and a visual mh, vmh. ZOTnet was a LAN for mail by Marshall Rose at UCI with gateways for CSnet and ARPAnet. https://escholarship.org/content/qt4bm4k2vh/qt4bm4k2vh.pdf The ZOTnet: a local area mailing network, 1983 (I still use nmh and I'm using it for this reply.) -- Cheers, Ralph. From jon at fourwinds.com Mon Jun 1 03:25:07 2020 From: jon at fourwinds.com (Jon Steinhart) Date: Sun, 31 May 2020 10:25:07 -0700 Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) In-Reply-To: <20200531170924.0609821F85@orac.inputplus.co.uk> References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> <20200531170924.0609821F85@orac.inputplus.co.uk> Message-ID: <202005311725.04VHP7bX3814695@darkstar.fourwinds.com> Ralph Corderoy writes: > ... > > (I still use nmh and I'm using it for this reply.) +1 on that. I also still use nmh. To me, the set of shell commands exemplifies the UNIX philosophy as I can easily "show | wc -l" or whatever that is cumbersome with monolithic packages. My minor contributions were adding the attachment support and the hooks. The hooks allow other programs to be triggered behind the scenes when nmh commands are run; I use it to keep an elasticsearch database of my messages. I have two additional command script to support thise: gpick and gscan which are the equivalent of pick and scan except that they search the elasticsearch database and pretty much return results instantaneously. Jon From mah at mhorton.net Mon Jun 1 03:28:56 2020 From: mah at mhorton.net (Mary Ann Horton) Date: Sun, 31 May 2020 10:28:56 -0700 Subject: [TUHS] mh/hm, mmh In-Reply-To: References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> Message-ID: What a wonderful historical tour, Clem! Thank you! On 5/31/20 7:53 AM, Clem Cole wrote: > MH used a new Mailbox (on-disk) scheme and formatted all messages in > RFC733 form with addresses being flat an in the form: user at host. The > new format, used a line a control-As followed by a nl, before and > after the message.   The headers of course where RFC733 (type: value > with a trailing nl) and were separated from the body of the message by > a single nl. MH's message/file format was also the inspiration for B news, which kept each Usenet message in a separate RFC733 format file. (The command line interface, however, was more like a feed, not separate shell commands.) No control-A separators, though, just separate files. > At some point in time, UCB built the 'Berk-net' (whose original code > was written by ABC/Google's Eric Schmidt).  BTW: Eric would have seen > the Spider Network in his summers at BTL.  The key thing with Berknet > was cheap.  It ran over 3 wire RS-232C between hosts at 9600 baud.  >  Like UUCP was used to transfer files and email.    Like UUCP it used > a pre-fix addressing form: host:user ; but like RFC733 and unlike UUCP > was flat. > > Where Mary Ann and I differ in our memories is who wrote the original > version of UCB's delivermail (8ucb) program.   We both agree that it > is possible it was Eric Schmidt, as the switch to using > delivermail(8ucb) was were Berknet was spliced into the email > namespace.  I had thought Kurt wrote it, Mary Ann thought it was Eric > Allman.   We agree Eric Allman was hacking on it for the Ing70.  For > this response, it doesn't really matter other than to try to get the > history right, because it does not matter for the Rand Mail subsystem. To clarify, I was joking when I mentioned Eric Schmidt as a possible author of delivermail. (I should have inserted a smiley.) I wasn't directly involved, as Eric Allman was in the Ingres project on a different system, but I thought he (Eric Allman) wrote delivermail, and so does Wikipedia. I think Kurt Shoens wrote only the MTA, but i could be mistaken. > Anyway around this time, the curses library was created by Ken Arnold > (originally to support Rogue) by pulling the screen code out of vi and > using Mary Ann's termcap stuff. I didn't write termcap, Bill Joy did. I wrote terminfo, but that was later when I was at Bell Labs in 1982. Ken's curses used termcap, my 1982 rewrite used terminfo.     Mary Ann -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Mon Jun 1 03:43:46 2020 From: imp at bsdimp.com (Warner Losh) Date: Sun, 31 May 2020 11:43:46 -0600 Subject: [TUHS] LSX on the PDP-11/03 (LSI-11) In-Reply-To: <193c770d408ee131dbaa9a07dba6f068.squirrel@squirrelmail.tuffmail.net> References: <193c770d408ee131dbaa9a07dba6f068.squirrel@squirrelmail.tuffmail.net> Message-ID: On Sun, May 31, 2020 at 6:34 AM wrote: > > > Anyway, I've tried compiling Hello World on LSX, and I get "1: External > > definition syntax" error. Some help would be nice, but more generally, is > > anyone on this list more than vaguely familiar with LSX, or 6th Edition > > itself? > > > > void main () { > > printf("Hello World!"); > > } > > > > It seems that the 7th Edition was the beginning of the standard library > in > > C, and that this is missing in LSX. I'm not sure if printf is an > intrinsic > > function in (6th Edition) C, or if it's from a library. > > > > First off, VOID MAIN is not legal in any standard version of C. Even when > the language allows implementation defined extensions to the main > signature, it must still return int. > > If you have a later version of language supported, you have to define > printf rather than allowing it default define as an int returning > function. > > Add #include > Or, alternatively, LSX is really old. The compiler there likely doesn't support 'void'. >From the lsx part of tuhs: # cp /dev/tty8 addr.c main() { int a; printf("main address: %u (0%o)\n", main, main); printf("stack address: %u (0%o)\n", &a, &a); printf("heap address: %u (0%o)\n", sbrk(0), sbrk(0)); } ^D # cc addr.c # a.out main address: 16410 (040032) stack address: 40934 (0117746) heap address: 17920 (043000) None of the disks appear to have a /usr/include. The only .h files look like they were from the kernel. Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at eric.allman.name Mon Jun 1 04:56:07 2020 From: tuhs at eric.allman.name (Eric Allman) Date: Sun, 31 May 2020 11:56:07 -0700 Subject: [TUHS] mh/hm, mmh In-Reply-To: References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> Message-ID: <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com> Ah, time for some "Email at Berkeley" history.... First the back story.  In the beginning, /bin/mail was the mail system.  All mail was local to one machine so the distinction between MUA, MTA, and MSA (Mail Storage Agent) didn't exist.  At this point there were no options, at least at Berkeley, although I suspect that Bruce was working on MH by then. If my memory serves, networking at Berkeley started with BerkNET.  Clem got it right: Eric Schmidt wrote it, and it was designed to be cheap.  The main purpose was that Berkeley was getting more Unix machines and carrying tapes was problematic.  It was purely batch (no interactive sessions; file copy and email only) and ran over 9600 baud TTY lines.  Eric hacked /bin/mail to do the appropriate calls if it saw a colon in the address, so the flow of a typical message would be keyboard → /bin/mail → BerkNET → /bin/mail → /var/mail. In about 1976 the INGRES project at Berkeley got an ARPAnet connection.  6th Edition was current then, and we got the software from (iirc) UC San Diego.  It was kind of a mess, but it did work.  It was a VDH (Very Distant Host) interface running at 9600 baud.  But the mail system it included, MSG I think, was a world unto itself, and definitely did not use /bin/mail.  So now we had two universes: to send something to both someone on BerkNET and someone on ARPAnet you had to type the message into two different MUAs.  But at first this was good enough.  But not for long. The problem was that to use the ARPAnet you had to be logged into our PDP-11/70.  Back then the cool kids had "glass TTYs" (i.e., adm3a).  But BerkNET didn't have interactive connections, so you had to actually have a TTY line into our machine.  Two problems arose: first, INGRES was in a different building (Cory Hall) than the rest of the department (Evans Hall).  But that's OK, there were already wires running between the two buildings that were probably inspired by BerkNET.  Second, our PDP-11 had a single DH-11 multiport interface with 16 ports.  Only two of those ports were available, and new DH-11s cost on the order of $15,000, money which we did not have. But no problem.  At this point there was already a "plug board" in the mail room in Evans that had ports to the half dozen or so machines that were by then on campus and lines to each office.  To connect you would plug in a wire between the machine you wanted to use and your office.  Protocol was that you remove the cross connect after use, but that was widely ignored.  Ing70 ports were quite frequently being pulled out while in use and no one was happy.  Unfortunately since I was kind of managing that machine, I was the one who got yelled at. Also in that period Berkeley got connected into UUCP.  UUCP had also done a hack to /bin/mail, so it was becoming quite the big mess.  I made the observation that I could hack /bin/mail to forward messages with an "@" sign in the address to our ARPAnet-connected machine, but at that point the code was becoming unwieldy.  Enter delivermail, which was essentially what we now call an MTA.  It was still a hack, but it was easier to deal with because it didn't have MUA and MSA functionality.  I changed /bin/mail to call delivermail unless it got a "-d" (deliver) flag, so now the sequence for local mail was /bin/mail → delivermail → /bin/mail -d → /var/mail.  And most importantly, I hacked the ARPAnet code to use delivermail as well, so now Ing70 could forward mail from BerkNET to ARPAnet and back again.  The fights over our two plug board ports mostly dissipated and my life got better. I'm pretty sure that at this point Kurt Schoens was working on Mail.  He read /var/mail (at that point it had been migrated to /var/spool/mail) directly, so there was no hiding of the mailbox format, so he had MUA + a half of an MSA.  He converted Mail to use delivermail for outgoing mail.  I don't recall the details of how MH got hooked in, but it did have a separate program for sending (called "send" I think) that would have been fairly easy to hack.  It still had "inc" (incorporate) to bring mail from the /var/mail one-file-per-mailbox format into the MH one-file-per-message format, so the three functions were not completely broken out, but it was really the closest to that ideal model available at the time, so far as I know. That's the gist of that story.  Sendmail is a completely different story, driven by the onset of the Internet, and I won't make this already long message even longer by going into it here. By the way, I have a 9-track tape sent to me by Marshall Rose containing the November 1983 version of MH that UCI did (based on the Rand system), complete with original versions of the documentation.  If anyone has any way of reading it I'm happy to pass it on. eric From clemc at ccc.com Mon Jun 1 05:49:07 2020 From: clemc at ccc.com (Clem Cole) Date: Sun, 31 May 2020 15:49:07 -0400 Subject: [TUHS] mh/hm, mmh In-Reply-To: <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com> References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com> Message-ID: On Sun, May 31, 2020 at 3:38 PM Eric Allman wrote: > but it did have a separate program for sending (called "send" I think) > that would > have been fairly easy to hack. It still had "inc" (incorporate) to > bring mail from the /var/mail one-file-per-mailbox format into the MH > one-file-per-message format > Right, the hooks were in send(1mh) and inc(1mh). Clem -------------- next part -------------- An HTML attachment was scrubbed... URL: From mah at mhorton.net Mon Jun 1 07:31:30 2020 From: mah at mhorton.net (Mary Ann Horton) Date: Sun, 31 May 2020 14:31:30 -0700 Subject: [TUHS] mh/hm, mmh In-Reply-To: <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com> References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com> Message-ID: Wonderful! Thank you for the history, Eric!     Mary Ann On 5/31/20 11:56 AM, Eric Allman wrote: > Ah, time for some "Email at Berkeley" history.... > From robpike at gmail.com Mon Jun 1 08:01:58 2020 From: robpike at gmail.com (Rob Pike) Date: Mon, 1 Jun 2020 08:01:58 +1000 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: > > I’m not quite sure why the Research lineage did not include non-blocking > behaviour, especially in view of the man page comments. Maybe it was seen > as against the Unix philosophy, with select() offering sufficient mechanism > to avoid blocking (with open() the hard corner case)? That's it. Select was good enough for our purposes. -rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave at horsfall.org Mon Jun 1 13:32:56 2020 From: dave at horsfall.org (Dave Horsfall) Date: Mon, 1 Jun 2020 13:32:56 +1000 (EST) Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: On Mon, 1 Jun 2020, Rob Pike wrote: > I’m not quite sure why the Research lineage did not include > non-blocking behaviour, especially in view of the man page comments. > Maybe it was seen as against the Unix philosophy, with select() > offering sufficient mechanism to avoid blocking (with open() the hard > corner case)? > > That's it. Select was good enough for our purposes. After being dragged through both Berserkley and SysVile, I never did get the hang of poll()/select() etc,,, -- Dave From Caipenghui_c at 163.com Mon Jun 1 16:32:01 2020 From: Caipenghui_c at 163.com (Caipenghui) Date: Mon, 01 Jun 2020 14:32:01 +0800 Subject: [TUHS] mh/hm, mmh In-Reply-To: References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com> Message-ID: <36190BD8-0A00-4729-9B81-2FF441552CF0@163.com> it's beautiful. 于 2020年6月1日 GMT+08:00 上午5:31:30, Mary Ann Horton 写到: >Wonderful! Thank you for the history, Eric! > >     Mary Ann > >On 5/31/20 11:56 AM, Eric Allman wrote: >> Ah, time for some "Email at Berkeley" history.... >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From meillo at marmaro.de Mon Jun 1 17:14:06 2020 From: meillo at marmaro.de (markus schnalke) Date: Mon, 01 Jun 2020 09:14:06 +0200 Subject: [TUHS] mh/hm, mmh In-Reply-To: References: <1jeHk5-5LM-00@marmaro.de> <1jfNb2-7JV-00@marmaro.de> <7180f41c-39ea-4271-578e-184057d5a723@neophilic.com> Message-ID: <1jfee7-82Z-00@marmaro.de> Thanks to everyone who joined in. That was more than I had expected. :-) meillo [2020-05-31 14:31] Mary Ann Horton > > Wonderful! Thank you for the history, Eric! > >     Mary Ann > > On 5/31/20 11:56 AM, Eric Allman wrote: > > Ah, time for some "Email at Berkeley" history.... > > > From paul at rileyriot.com Mon Jun 1 20:18:41 2020 From: paul at rileyriot.com (Paul Riley) Date: Mon, 1 Jun 2020 18:18:41 +0800 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator Message-ID: Team Unix, Is there a Windows or Linux utility to create a disk image in any of the above formats, from a local folder tree? Paul *Paul Riley* Mo: +86 186 8227 8332 Email: paul at rileyriot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron at ronnatalie.com Mon Jun 1 23:01:13 2020 From: ron at ronnatalie.com (Ronald Natalie) Date: Mon, 1 Jun 2020 09:01:13 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: Eh? An RL is just a cartridge disk drive, not a format. It would depend on what the operating system you are using wanted to put on it. Similarly the RX is just a floppy drive. In my memory, there wasn’t much UNIX use of these drives. The main reason the driver existed on the VAX unices was to write the Console Floppy Disk in the 780 (which was in RT-11 format). What are you trying to do? > On Jun 1, 2020, at 6:18 AM, Paul Riley wrote: > > Team Unix, > > Is there a Windows or Linux utility to create a disk image in any of the above formats, from a local folder tree? > > Paul > > Paul Riley > > Mo: +86 186 8227 8332 > Email: paul at rileyriot.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Mon Jun 1 23:04:28 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 1 Jun 2020 09:04:28 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: On Mon, Jun 1, 2020 at 6:19 AM Paul Riley wrote: > Is there a Windows or Linux utility to create a disk image in any of the > above formats, from a local folder tree? > What I think you are asking, is there a utility for a modern OS that will walk a local folder tree on my OS and create a new file whose structure is that of the file system for OS . The issue is not the device as much as the OS and disk file layout. As far as UNIX (or simh at the OS level) is concerned, the disk is just a linear array of bytes, addressed by blocks. The physical format is not seen by UNIX. There are numerious utilities, as well as 'foreign file systems' that are available. For instance, many Unix's can write RT-11 and MS-DOS format with standard utilities. It really depends the OS. That said, if the target OS is modern enough to support NFS or Samba, the easiest way might be export the file system from local system, and then running a simulated OS, 'mount' the file system. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Tue Jun 2 00:58:02 2020 From: lm at mcvoy.com (Larry McVoy) Date: Mon, 1 Jun 2020 07:58:02 -0700 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: <20200601145801.GE22016@mcvoy.com> On Mon, Jun 01, 2020 at 01:32:56PM +1000, Dave Horsfall wrote: > On Mon, 1 Jun 2020, Rob Pike wrote: > > > I???m not quite sure why the Research lineage did not include > > non-blocking behaviour, especially in view of the man page comments. > > Maybe it was seen as against the Unix philosophy, with select() > > offering sufficient mechanism to avoid blocking (with open() the hard > > corner case)? > > > >That's it. Select was good enough for our purposes. > > After being dragged through both Berserkley and SysVile, I never did get the > hang of poll()/select() etc,,, I'm sure you could, select is super handy, think a network server like apache. From pnr at planet.nl Tue Jun 2 01:04:22 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Mon, 1 Jun 2020 17:04:22 +0200 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator Message-ID: <94C31414-D9DF-41C5-B0D9-9FB92994B513@planet.nl> > Team Unix, Is there a Windows or Linux utility to create a disk image in any of the above formats, from a local folder tree? Paul *Paul Riley* It seems you are asking for two tools in the BK-UNIX toolbox, fsutil and dskutil https://github.com/sergev/bkunix/tree/master/fsutil https://github.com/sergev/bkunix/tree/master/diskutil The first generates a Unix 6th edition file system from a local directory tree. The result is a binary file with the 512 byte disk blocks stored in sequence. Maybe this is what SIMH needs, I’m not into the details of SIMH. The second appears to be a tool to take the a file generated by the fsutil tool and split that into sectors and tracks. I’m not familiar with this tool, but it looks like you might need something similar (I assume that you have some way to hook up a 8” drive to your PC?). Sector interleaving may be an issue to look out for. When preparing a LSX system disk, you will need to think carefully about the layout: Presumably the disk works with 128 byte sectors and 4 sectors are grouped together to create a 512 byte unix block. Check out the disk driver code for details: https://www.tuhs.org/cgi-bin/utree.pl?file=LSX/sys/decfd.c Block 0 contains bootstrap code which is loaded/called from the monitor rom (or from a short code sequence keyed in on a “blinkenpanel” console). The filesystem itself starts at block 1 and runs up to a block N (you need to tell fsutil how big the filesystem needs to be). After block N up to the end of the disk is space to contain 2 swapped out programs plus 1 block for the return code of the third (default LSX has a maximum of 3 processes). You have to figure out how many 512 blocks are on your floppy and subtract out the swap space to arrive at a figure for N. In the LSX source code ’N’ is known as the define SWPLO, see param.h for details: https://www.tuhs.org/cgi-bin/utree.pl?file=LSX/sys/param.h In this file, 99 blocks are reserved for swap, corresponding to processes with 24KB memory; total disk size is defined as 500 blocks, 250KB - note that this slightly exceeds the 241KB offered by the standard IBM 77 track 26 sector formatting for 8” disks. Probably you will need to tweak the values in param.h Be careful with size units in the source code. Often sizes are expressed in words (2 bytes). Where memory is concerned it is often expressed in ‘clicks’, 64 bytes. From pnr at planet.nl Tue Jun 2 01:18:44 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Mon, 1 Jun 2020 17:18:44 +0200 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator Message-ID: > > In this file, 99 blocks are reserved for swap, corresponding to processes with 24KB memory; total disk size is defined as 500 blocks, 250KB - note that this slightly exceeds the 241KB offered by the standard IBM 77 track 26 sector formatting for 8” disks. Probably you will need to tweak the values in param.h Oops: 77 tracks by 26 sectors of 128 bytes, makes 250.25KB, so it should work as configured. From heinz at osta.com Tue Jun 2 02:41:03 2020 From: heinz at osta.com (Heinz Lycklama) Date: Mon, 1 Jun 2020 09:41:03 -0700 Subject: [TUHS] LSX on the PDP-11/03 (LSI-11) In-Reply-To: References: Message-ID: Paul, I'm surprised that copies of the LSX system are still around. I'm assuming that you acquired only the binary code, and not the source code. The LSX system source code could be compiled on an LSX system running on floppy disks. I see you are getting your C compiler issues addressed by others already. Heinz Lycklama Author of LSX at Bell Labs On 5/30/2020 11:26 PM, Paul Riley wrote: > I've managed to acquire a PDP-11/03 with twin floppy drives (Sykes > Datatronics RX01 or RX02 equivalents, not sure yet which). > > I've stumbled across LSX, and I have it running on SimH. I'm quite > inexperienced with Unix, but it's something I want to learn well, > having brushed against it at university in the '80s, and having played > with Linux somewhat. > > I have some interest in Forth, but I don't like the block system of > early forths such as FigForth, and I plan to create a new Forth based > on FigForth, but supporting external source files. > > Anyway, I've tried compiling Hello World on LSX, and I get "1: > External definition syntax" error. Some help would be nice, but more > generally, is anyone on this list more than vaguely familiar with LSX, > or 6th Edition itself? > > void main () { >         printf("Hello World!"); > } > > It seems that the 7th Edition was the beginning of the standard > library in C, and that this is missing in LSX. I'm not sure if printf > is an intrinsic function in (6th Edition) C, or if it's from a library. > > My questions are a bit random, but looking to converse with others > with LSX experience. > > Paul > > *Paul Riley* > > Email: paul at rileyriot.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From heinz at osta.com Tue Jun 2 02:58:47 2020 From: heinz at osta.com (Heinz Lycklama) Date: Mon, 1 Jun 2020 09:58:47 -0700 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: <6aaca54f-83da-515f-e5b0-e3baa4560c44@osta.com> I did  add a few new features to LSX to deal with contiguous files and to handle asynchronous read/write's for real time applications. They are described in the LSX paper in the 1978 BSTJ on the UNIX Time-Sharing System. Heinz On 5/31/2020 9:46 AM, Warner Losh wrote: > Sorry to top post, but LSX or Miniunix had non blocking I/O as well. > It was in one of the documents that Clem scanned in the last year. It > specifically was an experiment into how to do it. > > Warner > > On Sun, May 31, 2020, 10:07 AM Clem Cole > wrote: > > > > On Sun, May 31, 2020 at 7:10 AM Paul Ruizendaal > wrote: > >  This behaviour seems to have continued into SysVR1, I’m not > sure when EAGAIN came into use as a return value for this use > case in the SysV lineage. Maybe with SysVR3 networking? > > Actually, I'm pretty sure that was a product of the POSIX > discussions.  BSD already had networking an EWOULDBLOCK.   We had > argued about EWOULDBLOCK a great deal, we also were arguing about > signal semantics. I've forgotten many of the details, Heinz may > remember more than I do.  EAGAIN was created as a compromise -- > IIRC neither system had it yet.   SVR3 networking was where it > went into System V, although some of the AT&T representatives were > none too happy about it. > > > In the Research lineage, the above SysIII approach does not > seem to exist, although the V8 manual page for open() says > under BUGS "It should be possible [...] to optionally call > open without the possibility of hanging waiting for carrier on > communication lines.” In the same location for V10 it reads > "It should be possible to call open without waiting for > carrier on communication lines.” > > The July 1981 design proposals for 4.2BSD note that SysIII > non-blocking files are a useful feature and should be included > in the new system. In Jan/Feb 1982 this appears to be coded > up, although not all affected files are under SCCS tracking at > that point in time. Non-blocking behaviour is changed from the > SysIII semantics, in that EWOULDBLOCK is returned instead of 0 > when progress is not possible. The non-blocking behaviour is > extended beyond TTY’s and pipes to sockets, with additional > errors (such as EINPROGRESS). At this time EWOULDBLOCK is not > the same error number as EGAIN. > > My memory is that Keith was the BSD (CSRG) person at the POSIX > meeting (he, Jim McGinness of DEC, and I created PAX at one point > as a compromise).   I wish I could remember all of the details, > but this was all argued at the POSIX meetings. > > As I said before the folks from AT&T just wanted to take the SVID > and rubber stamp it at the specification.  Part of it the problem > was they wanted to be free to do what things/make choices that the > rest of us might or might not like (for instance, they did not > want the sockets interface). > > > It would seem that the differences between the BSD and SysV > lineages in this area persisted until around 2000 or so. > > Yep - cause around then POSIX started to settle out and both > systems began to follow it. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul at rileyriot.com Tue Jun 2 08:35:21 2020 From: paul at rileyriot.com (Paul Riley) Date: Tue, 2 Jun 2020 06:35:21 +0800 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: Ron, Trying to move a large number of source files from my Windows machine to the LSX simulation (or V6 simulation). This is a Unix forum, so I didn’t explicitly say in a Unix format, but that was implied, sorry. Paul On Mon, 1 Jun 2020 at 9:01 pm, Ronald Natalie wrote: > Eh? An RL is just a cartridge disk drive, not a format. It would > depend on what the operating system you are using wanted to put on it. > Similarly the RX is just a floppy drive. In my memory, there wasn’t > much UNIX use of these drives. The main reason the driver existed on the > VAX unices was to write the Console Floppy Disk in the 780 (which was in > RT-11 format). > > What are you trying to do? > > On Jun 1, 2020, at 6:18 AM, Paul Riley wrote: > > Team Unix, > > Is there a Windows or Linux utility to create a disk image in any of the > above formats, from a local folder tree? > > Paul > > *Paul Riley* > > Mo: +86 186 8227 8332 > Email: paul at rileyriot.com > > > -- *Paul Riley* Mo: +86 186 8227 8332 Email: paul at rileyriot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul at rileyriot.com Tue Jun 2 08:45:07 2020 From: paul at rileyriot.com (Paul Riley) Date: Tue, 2 Jun 2020 06:45:07 +0800 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: Clem, Thanks for your help. You’ve correctly interpreted my question. Is the disk image independent from the disk hardware? I’d assume that different disks may have different block sizes etc, so the disk type may be important. The target system is LSX, a cut-down version of V6 designed to run on the LSI-11. There are very few system utilities in the standard build (no mount for example). The second floppy is permanently mounted at boot time. I’m interested in making source file floppies on my modern system to use on the LSX, so I want to be able to create an image file from a source folder tree. Paul On Mon, 1 Jun 2020 at 9:05 pm, Clem Cole wrote: > > > On Mon, Jun 1, 2020 at 6:19 AM Paul Riley wrote: > >> Is there a Windows or Linux utility to create a disk image in any of the >> above formats, from a local folder tree? >> > What I think you are asking, is there a utility for a modern OS that will > walk a local folder tree on my OS and create a new file whose structure is > that of the file system for OS . > > The issue is not the device as much as the OS and disk file layout. As > far as UNIX (or simh at the OS level) is concerned, the disk is just a > linear array of bytes, addressed by blocks. The physical format is not > seen by UNIX. > > There are numerious utilities, as well as 'foreign file systems' that are > available. For instance, many Unix's can write RT-11 and MS-DOS format > with standard utilities. It really depends the OS. That said, > if the target OS is modern enough to support NFS or Samba, the easiest way > might be export the file system from local system, and then running a > simulated OS, 'mount' the file system. > -- *Paul Riley* Mo: +86 186 8227 8332 Email: paul at rileyriot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul at rileyriot.com Tue Jun 2 08:54:28 2020 From: paul at rileyriot.com (Paul Riley) Date: Tue, 2 Jun 2020 06:54:28 +0800 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: <94C31414-D9DF-41C5-B0D9-9FB92994B513@planet.nl> References: <94C31414-D9DF-41C5-B0D9-9FB92994B513@planet.nl> Message-ID: Paul, Many thanks for your detailed reply, it’s much appreciated. I’ll start with the Bkunix utility and see if SimH can use the output, then move on to your other suggestions if not. Paul On Mon, 1 Jun 2020 at 11:04 pm, Paul Ruizendaal wrote: > > Team Unix, Is there a Windows or Linux utility to create a disk image in > any of the above formats, from a local folder tree? Paul *Paul Riley* > > It seems you are asking for two tools in the BK-UNIX toolbox, fsutil and > dskutil > https://github.com/sergev/bkunix/tree/master/fsutil > https://github.com/sergev/bkunix/tree/master/diskutil > > The first generates a Unix 6th edition file system from a local directory > tree. The result is a binary file with the 512 byte disk blocks stored in > sequence. Maybe this is what SIMH needs, I’m not into the details of SIMH. > > The second appears to be a tool to take the a file generated by the fsutil > tool and split that into sectors and tracks. I’m not familiar with this > tool, but it looks like you might need something similar (I assume that you > have some way to hook up a 8” drive to your PC?). Sector interleaving may > be an issue to look out for. > > When preparing a LSX system disk, you will need to think carefully about > the layout: > > Presumably the disk works with 128 byte sectors and 4 sectors are grouped > together to create a 512 byte unix block. Check out the disk driver code > for details: > https://www.tuhs.org/cgi-bin/utree.pl?file=LSX/sys/decfd.c > > Block 0 contains bootstrap code which is loaded/called from the monitor > rom (or from a short code sequence keyed in on a “blinkenpanel” console). > > The filesystem itself starts at block 1 and runs up to a block N (you need > to tell fsutil how big the filesystem needs to be). > > After block N up to the end of the disk is space to contain 2 swapped out > programs plus 1 block for the return code of the third (default LSX has a > maximum of 3 processes). You have to figure out how many 512 blocks are on > your floppy and subtract out the swap space to arrive at a figure for N. > > In the LSX source code ’N’ is known as the define SWPLO, see param.h for > details: > https://www.tuhs.org/cgi-bin/utree.pl?file=LSX/sys/param.h > > In this file, 99 blocks are reserved for swap, corresponding to processes > with 24KB memory; total disk size is defined as 500 blocks, 250KB - note > that this slightly exceeds the 241KB offered by the standard IBM 77 track > 26 sector formatting for 8” disks. Probably you will need to tweak the > values in param.h > > Be careful with size units in the source code. Often sizes are expressed > in words (2 bytes). Where memory is concerned it is often expressed in > ‘clicks’, 64 bytes. > > > > > -- *Paul Riley* Mo: +86 186 8227 8332 Email: paul at rileyriot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jnc at mercury.lcs.mit.edu Tue Jun 2 09:17:42 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Mon, 1 Jun 2020 19:17:42 -0400 (EDT) Subject: [TUHS] non-blocking IO Message-ID: <20200601231742.CBAB118C073@mercury.lcs.mit.edu> > From: Paul Ruizendaal > This time looking into non-blocking file access. (... right now my > scope is 'communication files' (tty's, pipes, network connections). > ... > First appearance of non-blocking behaviour seems to have been with > Chesson's multiplexed files ... in 1979. At around that point in time (I don't have the very _earliest_ code, to get an exact date, but the oldest traces I see [in mapalloc(), below] are from September '78), the CSR group at MIT-LCS (which were the people in LCS doing networking) was doing a lot with asynchronous I/O (when you're working below the reliable stream level, you can't just do a blocking 'read' for a packet; it pretty much has to be asynchronous). I was working in Unix V6 - we were building an experimental 1Mbit/second ring - and there was work in Multics as well. I don't think the wider Unix community heard about the Unix work, but our group regularly filed updates on our work for the 'Internet Monthly Reports', which was distributed to the whole TCP/IP experimental community. If you can find an archive of early issues (I'm too lazy to go look for one), we should be in there (although our report will alsocover the Multics TCP/IP work, and maybe some other stuff too). There were two main generations of code; I don't recall the second one well, and I'm too lazy to go look, but I can tell you off the top of my head a bit about how the first one worked. Open/read/write all looked standard to the user in the process (the latter two were oriented to packets, a bit like raw disks being blocks); multiple operations could be queued in each direction. (There was only one user allowed at a time for the network device; no input demultiplexing.) Whenever an I/O operation completed, the process was sent a signal. Since the read/write call had long since returned, it had to do a getty() to get info about that operation - the size of the packet, error indications, etc. One complication was that for a variety of reasons (we wanted to avoid having to copy data, and the interface did not have packet buffers) we did DMA directly to/from the user's memory; this meant the process has to be locked in place while I/O was pending. (I didn't realize it at the time, but we dodged a bullet there; a comment in xalloc(), which I only absorbed recently, explains the problem. More here: https://gunkies.org/wiki/UNIX_V6_internals#exec()_and_pure-text_images if anyone wants the gory details.) That all (the queing, signals for I/O completion, locking the process to a fixed location in memory while it continued to run) etc all worked well, as I recall (although I guess it couldn't do an sbrk() while locked), but one complication was the UNIBUS map on the -11/70. The DSSR/RTS group at LCS wanted to have a ring interface, but their machine was a /70 (ours, the one the driver was initially done on/for, was a /40), so with DMA we had to use the UNIBUS map. The stock V6 code had mapalloc(), (and mapfree(), both called on all DMA operations), but... it allocated the whole map to whatever I/O operation asked for the map. Clearly, if you're about to start a network input operation, and wait a packet to show up, you don't want the disk controller to have to sit and wait for for a packet to show up so _it_ can have the map. Luckily, mapalloc() was called with a pointer to the buffer header (which had all the info about the xfer), so I added a 'ubmap' array, and called the existing malloc() on it, to allocate only a big enough chunk of the UNIBUS map for the I/O operation defined by the header. Since there was 248KB of map space, and the largest single DMA transfer possible in V6 was about 64KB (maybe a little more, for a max-sized process with its 'user' block), there was never a problem with contention for the map, and we didn't have to touch any of the other drivers at all. That was dandy, and only a couple of lines of extra code, but I somehow made a math error in my changes, and as I recall I had to debug it with a printf() in mapalloc(). I was not popular that day! Luckily, the error was quickly obvious, a fix was applied, and we were on our way. Noel From ron at ronnatalie.com Tue Jun 2 09:59:20 2020 From: ron at ronnatalie.com (Ronald Natalie) Date: Mon, 1 Jun 2020 19:59:20 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: As far as the early UNIXs go, any disk is collection of 512-byte blocks. The filesystems either the early (what I’ll call V6) and the later (V7) don’t differ much. The primary difference is that the later V7 had 16-bit uids and a provision for larger file systems/sizes. The V6 file system was limited to 2^24 blocks while V7 did 2^32. The 512 block size corresponded to the native sector size of all the DEC hardware except the RX which I think only had 128-byte sectors. But again, we didn’t do much with that other than write the standalone console disks for the 780 (in RT format) and I also used it to make “unix” file system disks for the BRL “LOS” (little operating system…no time for sharing, uniprocessor system) that ran our internet routers and the IO hardware on the HEP supercomputer. From jnc at mercury.lcs.mit.edu Tue Jun 2 10:08:45 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Mon, 1 Jun 2020 20:08:45 -0400 (EDT) Subject: [TUHS] non-blocking IO Message-ID: <20200602000845.CF26E18C073@mercury.lcs.mit.edu> > when you're working below the reliable stream level, you can't just do a > blocking 'read' for a packet; it pretty much has to be asynchronous Oh, you should look at the early BBN TCP for V6 Unix - they would have faced the same issue, with their TCP process. They did have the capac() call (which kind of alleviates the need for non-blocking I/O), but that may have only been available for ports/pipes; I'm not sure if the ARPANET device supported it. (With the NCP as well, that did some amount of demultiplexing in the kernel, and probably had buffering there, so, if so, in theory capac() could have been done there. Of course, with the ARPANET link being only 100Kbit/sec maximum - although only to a host on the same IMP - the overhead of copying buffered data made kernel buffering more 'affordable'.) Noel From jnc at mercury.lcs.mit.edu Tue Jun 2 10:14:46 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Mon, 1 Jun 2020 20:14:46 -0400 (EDT) Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator Message-ID: <20200602001446.359F618C073@mercury.lcs.mit.edu> > From: Ronald Natalie > The V6 file system was limited to 2^24 blocks No, 2^16; from filsys.h: int s_fsize; /* size in blocks of entire volume */ and of course on an -11 an int is 16 bits. Maybe you're thinking of the file size, which was 2^24 bytes (max). Noel From ron at ronnatalie.com Tue Jun 2 11:26:56 2020 From: ron at ronnatalie.com (Ronald Natalie) Date: Mon, 1 Jun 2020 21:26:56 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: <20200602001446.359F618C073@mercury.lcs.mit.edu> References: <20200602001446.359F618C073@mercury.lcs.mit.edu> Message-ID: <8C214FC8-A7F3-4DC1-9600-DA3CE357F041@ronnatalie.com> You are right. 16 bit block numbers 24 bit file size for V6. > On Jun 1, 2020, at 8:14 PM, Noel Chiappa wrote: > >> From: Ronald Natalie > >> The V6 file system was limited to 2^24 blocks > > No, 2^16; from filsys.h: > > int s_fsize; /* size in blocks of entire volume */ > > and of course on an -11 an int is 16 bits. > > Maybe you're thinking of the file size, which was 2^24 bytes (max). > > Noel From paul at rileyriot.com Tue Jun 2 14:57:54 2020 From: paul at rileyriot.com (Paul Riley) Date: Tue, 2 Jun 2020 12:57:54 +0800 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: Thanks Ron, and all others. Theres plenty of grist for the mill, time for me to grind it. Paul *Paul Riley* Mo: +86 186 8227 8332 Email: paul at rileyriot.com On Tue, 2 Jun 2020 at 07:59, Ronald Natalie wrote: > As far as the early UNIXs go, any disk is collection of 512-byte blocks. > The filesystems either the early (what I’ll call V6) and the later (V7) > don’t differ much. > The primary difference is that the later V7 had 16-bit uids and a > provision for larger file systems/sizes. The V6 file system was limited > to 2^24 blocks while V7 > did 2^32. > > The 512 block size corresponded to the native sector size of all the DEC > hardware except the RX which I think only had 128-byte sectors. But > again, we didn’t > do much with that other than write the standalone console disks for the > 780 (in RT format) and I also used it to make “unix” file system disks for > the BRL “LOS” > (little operating system…no time for sharing, uniprocessor system) that > ran our internet routers and the IO hardware on the HEP supercomputer. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pnr at planet.nl Tue Jun 2 18:22:12 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Tue, 2 Jun 2020 10:22:12 +0200 Subject: [TUHS] non-blocking IO Message-ID: <46A39BDD-A854-40EF-B82A-176F1AB7B92E@planet.nl> > > when you're working below the reliable stream level, you can't just do a > blocking 'read' for a packet; it pretty much has to be asynchronous > Oh, you should look at the early BBN TCP for V6 Unix - they would have faced the same issue, with their TCP process. They did have the capac() call (which kind of alleviates the need for non-blocking I/O), but that may have only been available for ports/pipes; I'm not sure if the ARPANET device supported it. I did. There is capac() support also for the IMP interface: https://www.tuhs.org/cgi-bin/utree.pl?file=BBN-V6/dmr/imp11a.c (see bottom two functions) BBN took the same approach as Research: with capac() or select() one can prevent blocking on read() and write(). From pnr at planet.nl Tue Jun 2 19:34:54 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Tue, 2 Jun 2020 11:34:54 +0200 Subject: [TUHS] non-blocking IO (morphing to "The serial I/O loop") Message-ID: > On 5/31/2020 9:46 AM, Warner Losh wrote: > > Sorry to top post, but LSX or Miniunix had non blocking I/O as well. > > It was in one of the documents that Clem scanned in the last year. It > > specifically was an experiment into how to do it. > > > > Warner > I did add a few new features to LSX to deal with contiguous files > and to handle asynchronous read/write's for real time applications. > They are described in the LSX paper in the 1978 BSTJ on the > UNIX Time-Sharing System. > > Heinz Thanks for highlighting this! The realm here is async I/O to disk, my original scope was limited to “communication” files (tty’s, pipes, network connections). Still, I find it an interesting topic. For others, the paper that you refer to can also be found here: https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/UNIX_on_a_Microprocessor_19780322.pdf If I read correctly, the async functionality was available only in the stand alone program version of LSX. Is that correct? In any case, the source code appears lost. From another paper in that set, I get the impression that the async functionality for LSX builds on earlier work for a very early version of Unix: https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-74-1352-1_Implementation_of_Large_Contiguous_Files_and_Asynchronous_IO_in_UNIX_19740104.pdf - - - When reading through the papers in that TUHS directory, something else caught my attention: early networking at the labs. For a while I have been puzzled by the “serial I/O loop” in use at the labs in the early seventies. Last Fall I found some 1970/1971 BSTJ papers about it, but there it stopped. I see in the memo on Glance (https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-75-1352-3_GLANCE_Terminals_on_UNIX_Time-Sharing_19750303.pdf) that D.R. Weller continued to work on it up to 1973 at least and that it was integrated with Unix in some way. Is that correct? Did the two memo’s referred to (MM 70-1384-1 and TM 73-1356-8) survive? Then the memo on satellite processors is very interesting (https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-78-3114-2_A_Minicomputer_Satellite_Processor_System_19780322.pdf). This appears to show the serial I/O loop in use as late as 1978, with a very intriguing use case involving system call forwarding over the network. Can you tell me more about the serial I/O loop and its use cases? From clemc at ccc.com Tue Jun 2 23:31:28 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 2 Jun 2020 09:31:28 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: Frankly, using a common tape format, and then a raw disk as the 'tape' tends to be easy. For instance, if it's an older system like v6, there is a binary for v6tar kicking around that was created as part of the v7 conversion utilities (or you can build it yourself with a little work - the issue a few changes in some headers). Put that binary on your v6 system. Then create a tar image on your modern system. Tar uses a threaded ASCII header and actually had a bug in it, that is exploited as an extension mechanism and became a feature (I'll explain off line if need be). So modern tar's will produce a checksum that the original's will correctly accept. Note the modern tar can create >>file types<< that the old tar will not understand, but it will just skip them. Going the back works too, but is limited by the original's handling of things like directories. It is generally not a problem. So, the result is that you can attach that 'tarball' as a raw disk on simh and then read it with v6tar. Another possible way to go it to try to get stp(1) to compile on the more modern system [it's on the Harvard tape IIRC -- it was the first version of tp in C -- earlier versions were in PDP-11 assembler). But ... since that was written with the a pre-Typesetter C compiler, and has a PDP-11 binary format knowledge in it and I think used Lesk's portable I/O package, so it might be a little more difficult to get running on a more modern C. On Mon, Jun 1, 2020 at 6:45 PM Paul Riley wrote: > Clem, > > Thanks for your help. You’ve correctly interpreted my question. > > Is the disk image independent from the disk hardware? I’d assume that > different disks may have different block sizes etc, so the disk type may be > important. > > The target system is LSX, a cut-down version of V6 designed to run on the > LSI-11. There are very few system utilities in the standard build (no mount > for example). The second floppy is permanently mounted at boot time. I’m > interested in making source file floppies on my modern system to use on the > LSX, so I want to be able to create an image file from a source folder tree. > > Paul > > On Mon, 1 Jun 2020 at 9:05 pm, Clem Cole wrote: > >> >> >> On Mon, Jun 1, 2020 at 6:19 AM Paul Riley wrote: >> >>> Is there a Windows or Linux utility to create a disk image in any of the >>> above formats, from a local folder tree? >>> >> What I think you are asking, is there a utility for a modern OS that will >> walk a local folder tree on my OS and create a new file whose structure is >> that of the file system for OS . >> >> The issue is not the device as much as the OS and disk file layout. As >> far as UNIX (or simh at the OS level) is concerned, the disk is just a >> linear array of bytes, addressed by blocks. The physical format is not >> seen by UNIX. >> >> There are numerious utilities, as well as 'foreign file systems' that are >> available. For instance, many Unix's can write RT-11 and MS-DOS format >> with standard utilities. It really depends the OS. That said, >> if the target OS is modern enough to support NFS or Samba, the easiest >> way might be export the file system from local system, and then running a >> simulated OS, 'mount' the file system. >> > -- > *Paul Riley* > > Mo: +86 186 8227 8332 > Email: paul at rileyriot.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Tue Jun 2 23:52:44 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 2 Jun 2020 09:52:44 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: On Mon, Jun 1, 2020 at 6:45 PM Paul Riley wrote: > Is the disk image independent from the disk hardware? I’d assume that > different disks may have different block sizes etc, so the disk type may be > important. > A reasonable thought, but no, other than efficiency. It's just an array of bytes exposed to the OS. The good news is that all the DEC systems used 512 byte fixed blocks (Nothing like the funky Prime, Apollo or Alto tricks of putting some of the file system metadata into the physical disk format -- if you ever wondered why some of the original SCSI controllers supported a 1072 byte block size - it was for Domain/Aegis - 1024 block + 48 byte meta data]. > > The target system is LSX, a cut-down version of V6 designed to run on the > LSI-11. > Everything should be able to be simulated using is 512-byte blocks. > There are very few system utilities in the standard build (no mount for > example). The second floppy is permanently mounted at boot time. I’m > interested in making source file floppies on my modern system to use on the > LSX, so I want to be able to create an image file from a source folder tree. > You'll have to check the docs, but I thought Heinz supported the RK05 [RKV11 controller]. That will give you 2.5M of 'raw' disk blocks, instead of floppy. Open it raw, and use it like mag tape. If you don't have a coming format, you can create something and then use dd(1) on the LSI-11 from the RKV11 -- a little clumsy but as a bootstrap, it should work fine. The RL11 and RK711 controllers (RL01/02, RK07 disk) came later to the PDP-11 (post 11/34 and 11/60). I think there was an RLV1[12] at some point, but I don't think DEC even made an RKV71[12] -- however, I'm not up on late-generation PDP-11 lore and they would have been much later than LSX so driver support is doubtful. Good luck, Clem -------------- next part -------------- An HTML attachment was scrubbed... URL: From pnr at planet.nl Wed Jun 3 00:19:26 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Tue, 2 Jun 2020 16:19:26 +0200 Subject: [TUHS] non-blocking IO Message-ID: > At around that point in time (I don't have the very _earliest_ code, to get an exact date, but the oldest traces I see [in mapalloc(), below] are from September '78), the CSR group at MIT-LCS (which were the people in LCS doing networking) was doing a lot with asynchronous I/O (when you're working below the reliable stream level, you can't just do a blocking 'read' for a packet; it pretty much has to be asynchronous). I was working in Unix V6 - we were building an experimental 1Mbit/second ring - and there was work in Multics as well. > I don't think the wider Unix community heard about the Unix work, but our group regularly filed updates on our work for the 'Internet Monthly Reports', which was distributed to the whole TCP/IP experimental community. If you can find an archive of early issues (I'm too lazy to go look for one), we should be in there (although our report will alsocover the Multics TCP/IP work, and maybe some other stuff too). Sounds very interesting! Looked around a bit, but I did not find a source for the “Internet Monthly Reports” for the late 70’s (rfc-editor.org/museum/ has them for the 1990’s). In the 1970’s era, it seems that NCP Unix went in another direction, using newly built message and event facilities to prevent blocking. This is described in "CAC Technical Memorandum No. 84, Illinois Inter-Process Communication Facility for Unix.” - but that document appears lost as well. Ah, well, topics for another day. From ron at ronnatalie.com Wed Jun 3 01:44:00 2020 From: ron at ronnatalie.com (Ronald Natalie) Date: Tue, 2 Jun 2020 11:44:00 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: > On Jun 2, 2020, at 9:52 AM, Clem Cole wrote: > > > The good news is that all the DEC systems used 512 byte fixed blocks much later than LSX so driver support is doubtful. > Except on the RX’s. The RX01 was 128 byte sectors and the later ones 256 and a odd interleaving strategy. However, the boot block and the rest of the file systems (both UNIX and RT at least) just aggregated the smaller (logical) sectors together to make a 512 byte one. -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Wed Jun 3 02:26:54 2020 From: imp at bsdimp.com (Warner Losh) Date: Tue, 2 Jun 2020 10:26:54 -0600 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: On Tue, Jun 2, 2020, 9:45 AM Ronald Natalie wrote: > > > On Jun 2, 2020, at 9:52 AM, Clem Cole wrote: > > > The good news is that all the DEC systems used 512 byte fixed blocks much > later than LSX so driver support is doubtful. > > > Except on the RX’s. The RX01 was 128 byte sectors and the later ones 256 > and a odd interleaving strategy. However, the boot block and the rest of > the file systems (both UNIX and RT at least) > just aggregated the smaller (logical) sectors together to make a 512 byte > one. > RX50 also had weird interleaving... but it was quite the odd duck. Warner > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.winalski at gmail.com Wed Jun 3 03:45:55 2020 From: paul.winalski at gmail.com (Paul Winalski) Date: Tue, 2 Jun 2020 13:45:55 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: The operating systems that I cut my teeth on (OS/360, DOS/360, VAX/VMS) all had basic I/O system calls that were non-blocking. Blocking I/O calls were all built on top of that framework. I thus found it curious that Unix took the opposite tack, and non-blocking I/O was an afterthought. So I'm curious as to what the rationale was for Unix to have been designed with basic I/O being blocking rather than asynchronous. Especially that non-blocking I/O primitives were the norm for OSes in those days. -Paul W. From arnold at skeeve.com Wed Jun 3 03:59:05 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Tue, 02 Jun 2020 11:59:05 -0600 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: <202006021759.052Hx5Et022619@freefriends.org> Paul Winalski wrote: > So I'm curious as to what the rationale was for Unix to have been > designed with basic I/O being blocking rather than asynchronous. I don't doubt that it was "simplify, simplify, simplify". Async I/O is much messier than Unix's read/write model. The Unix model was simpler to design, simpler to code, simpler to get right, and undoubtedly took much less OS code than an async model would have; on the PDP-11 that would have mattered. Also, the early Unixs were on smaller -11s, not the /45 or /70 with split I&D space and the ability to address lost more RAM. My guess, anyway. :-) Arnold From clemc at ccc.com Wed Jun 3 04:08:36 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 2 Jun 2020 14:08:36 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: On Tue, Jun 2, 2020 at 11:44 AM Ronald Natalie wrote: > Except on the RX’s. The RX01 was 128 byte sectors and the later ones 256 > and a odd interleaving strategy. However, the boot block and the rest of > the file systems (both UNIX and RT at least) just aggregated the smaller > (logical) sectors together to make a 512 byte one. > Indeed, I was less than precise -- the key is that the SW treats them everything as being in 512 byte 'hunks' no matter the underlying physical formatting/interleaving actually is. The point is that if you Paul is trying to move data from the two systems, just treat the device as a serial byte stream of 512 byte blocks on both sides and you you be fine. That means if using dd, add conv=sync to fill any write buffer to a complete multiple of 512. -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Wed Jun 3 04:23:59 2020 From: crossd at gmail.com (Dan Cross) Date: Tue, 2 Jun 2020 14:23:59 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: On Tue, Jun 2, 2020 at 1:47 PM Paul Winalski wrote: > The operating systems that I cut my teeth on (OS/360, DOS/360, > VAX/VMS) all had basic I/O system calls that were non-blocking. > Blocking I/O calls were all built on top of that framework. I thus > found it curious that Unix took the opposite tack, and non-blocking > I/O was an afterthought. > > So I'm curious as to what the rationale was for Unix to have been > designed with basic I/O being blocking rather than asynchronous. > Especially that non-blocking I/O primitives were the norm for OSes in > those days. Doug addressed this, albeit in an oblique manner, on this list back in 2015: https://minnie.tuhs.org/pipermail/tuhs/2015-September/007509.html Quoting him: """ Unix was what the authors wanted for a productive computing environment, not a bag of everything they thought somebody somewhere might want. One objective, perhaps subliminal originally, was to make program behavior easy to reason about. Thus pipes were accepted into research Unix, but more general (and unruly) IPC mechanisms such as messages and events never were. The infrastructure had to be asynchronous. The whole point was to surmount that difficult model and keep everyday programming simple. User visibility of asynchrony was held to a minimum: fork(), signal(), wait(). Signal() was there first and foremost to support SIGKILL; it did not purport to provide a sound basis for asynchronous IPC. The complexity of sigaction() is evidence that asynchrony remains untamed 40 years on. """ My response at the time was to question whether asynchrony itself remains untamed, as Doug put it, or if rather it has proved difficult to retrofit asynchrony onto a system designed around fundamentally synchronous primitives? - Dan C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.winalski at gmail.com Wed Jun 3 04:53:59 2020 From: paul.winalski at gmail.com (Paul Winalski) Date: Tue, 2 Jun 2020 14:53:59 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: <202006021759.052Hx5Et022619@freefriends.org> References: <202006021759.052Hx5Et022619@freefriends.org> Message-ID: On 6/2/20, arnold at skeeve.com wrote: > Paul Winalski wrote: > >> So I'm curious as to what the rationale was for Unix to have been >> designed with basic I/O being blocking rather than asynchronous. > > Also, the early Unixs were on smaller -11s, not the /45 or /70 with > split I&D space and the ability to address lost more RAM. I first encountered DOS/360 on a System/360 model 25 with 48K of memory. This was a one-job-at-a-time batch system, but the I/O primitive (EXCP--execute channel program) was asynchronous. So I don't think the small memory rationale really applies. -Paul W. From paul.winalski at gmail.com Wed Jun 3 04:56:59 2020 From: paul.winalski at gmail.com (Paul Winalski) Date: Tue, 2 Jun 2020 14:56:59 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: On 6/2/20, Dan Cross wrote: > > My response at the time was to question whether asynchrony itself remains > untamed, as Doug put it, or if rather it has proved difficult to retrofit > asynchrony onto a system designed around fundamentally synchronous > primitives? I think that's a very good question. It's analogous to record-oriented I/O vs. byte stream I/O. It's easy to build record-oriented I/O on top of a byte stream, but it's a real bear to do it the other way around. Similarly, it's easy to build synchronous I/O on top of asynchronous I/O but the reverse ends up looking contrived. -Paul W. From paul.winalski at gmail.com Wed Jun 3 04:59:37 2020 From: paul.winalski at gmail.com (Paul Winalski) Date: Tue, 2 Jun 2020 14:59:37 -0400 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: On 6/2/20, Ronald Natalie wrote: > > Except on the RX’s. The RX01 was 128 byte sectors and the later ones 256 > and a odd interleaving strategy. However, the boot block and the rest of > the file systems (both UNIX and RT at least) > just aggregated the smaller (logical) sectors together to make a 512 byte > one. At least they all supported fixed-length sectors. Was there ever a Unix implementation that supported IBM-style CKD disk devices? -Paul W. From athornton at gmail.com Wed Jun 3 05:08:23 2020 From: athornton at gmail.com (Adam Thornton) Date: Tue, 2 Jun 2020 12:08:23 -0700 Subject: [TUHS] DEC RL01/RL02 RX01/RX02 Disk Image Creator In-Reply-To: References: Message-ID: <3A17BE73-777A-4CF2-8D55-A5C802FC767D@gmail.com> > On Jun 2, 2020, at 11:59 AM, Paul Winalski wrote: > > On 6/2/20, Ronald Natalie wrote: >> >> Except on the RX’s. The RX01 was 128 byte sectors and the later ones 256 >> and a odd interleaving strategy. However, the boot block and the rest of >> the file systems (both UNIX and RT at least) >> just aggregated the smaller (logical) sectors together to make a 512 byte >> one. > > At least they all supported fixed-length sectors. Was there ever a > Unix implementation that supported IBM-style CKD disk devices? If you count Linux, sure, Linux for the S/390 and zSeries. If you don’t…well, for one brief shining moment before IBM squeezed too hard on the price and Oracle bought Sun instead, there was an OpenSolaris port to the zSeries, and it did support 3390 type devices. I may have had something to do with that latter device driver. Adam From clemc at ccc.com Wed Jun 3 05:18:16 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 2 Jun 2020 15:18:16 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: <202006021759.052Hx5Et022619@freefriends.org> Message-ID: On Tue, Jun 2, 2020 at 2:54 PM Paul Winalski wrote: > I first encountered DOS/360 on a System/360 model 25 with 48K of > memory. This was a one-job-at-a-time batch system, but the I/O > primitive (EXCP--execute channel program) was asynchronous. So I > don't think the small memory rationale really applies. > Hrrmpt... it was single task. Being asynchronous in the I/O and allowed process asynchrony takes a lot more housekeeping. Paul you know I agree with you, it was always an issue with UNIX IMO. The problem is how to do it differently. At Masscomp RRU, our solution was not to try to 'fix it' as much as add a new scheme beside the synchronous one. We added a general AST's mechanism that anyone could use (very much like RSX and VMS in semantics), but left signals alone. We added new async calls, which were just implemented via ioctl's (the universal hack) for the specific HW that supported it. In retrospect, that was an error, it should have been aread(2)/awrite(2) and then added the completion routine/call back as the 4th & 5th parameters. Since Stellix was not Real-Time, we left them out. [ tjt and I have argued about that for years ]. So back to Doug/Dan's answer -- I think for a small system, like the original PDP-7 and the PDP-11/20 putting the effort to make it multiprocess and leaving the I/O synchronous definitely made it easier. Particularly, since what was created with the signal semantics. To me, UNIX got way more right than wrong. I suspect if signals had been more like AST's, the idea of being async might have gone farther. But to use Dennis's like about C being quirky, signals are very quirky. wnj tried to 'fix' them and frankly it just made matters worse. And IMO, signaction(3) as Doug says, is a nightmare. I've generally been a fan of introducing a new idea separately as a 'stronger strain' and then seeing if people likely it. FWIW: A couple of us did try to get AST's into the POSIX.4 (they were there in a draft), but ultimately *.4 was rejected as 'not UNIX.' -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Wed Jun 3 05:23:20 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 2 Jun 2020 15:23:20 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: Message-ID: On Tue, Jun 2, 2020 at 2:58 PM Paul Winalski wrote: > I think that's a very good question. It's analogous to > record-oriented I/O vs. byte stream I/O. It's easy to build > record-oriented I/O on top of a byte stream, but it's a real bear to > do it the other way around. Similarly, it's easy to build synchronous > I/O on top of asynchronous I/O but the reverse ends up looking contrived. > Which was exactly the point I tried to make in the POSIX.4 discussions, but it does take more work in the basic housekeeping and you need a way to handle events and completions that are priority based, queued, and a few other details. As Doug said, they stayed away from some features (like messaging). async I/O was one of them. But as I said, Ken, Dennis and the rest of the crew did an amazing job with very little. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jnc at mercury.lcs.mit.edu Wed Jun 3 06:13:34 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Tue, 2 Jun 2020 16:13:34 -0400 (EDT) Subject: [TUHS] non-blocking IO Message-ID: <20200602201334.95D9718C079@mercury.lcs.mit.edu> > From: Paul Winalski > I'm curious as to what the rationale was for Unix to have been designed > with basic I/O being blocking rather than asynchronous. It's a combination of two factors, I reckon. One, which is better depends a lot on the type of thing you're trying to do. For many typical thing (e.g. 'ls'), blocking is a good fit. And, as As Arnold says, asyhchronous I/O is more complicated, and Unix was (well, back then at least) all about getting the most bang for the least bucks. More complicated things do sometimes benefit from asynchronous I/O, but complicated things weren't Unix's 'target market'. E.g. even though pipes post-date the I/O decision, they too are a better match to blocking I/O. > From: Arnold Skeeve > the early Unixs were on smaller -11s, not the /45 or /70 with split I&D > space and the ability to address lost more RAM. Ahem. Lots more _core_. People keeep forgetting that we're looking at decicions made at a time when each bit in main memory was stored in a physically separate storage device, and having tons of memory was a dream of the future. E.g. the -11/40 I first ran Unix on had _48 KB_ of core memory - total! And that had to hold the resident OS, plus the application! It's no surprise that Unix was so focused on small size - and as a corollary, on high bang/buck ratio. But even in his age of lighting one's cigars with gigabytes of main memory (literally), small is still beautiful, because it's easier to understand, and complexity is bad. So it's too bad Unix has lost that extreme parsimony. > From: Dan Cross > question whether asynchrony itself remains untamed, as Doug put it, or > if rather it has proved difficult to retrofit asynchrony onto a system > designed around fundamentally synchronous primitives? I'm not sure it's 'either or'; I reckon they are both true. Noel From clemc at ccc.com Wed Jun 3 06:43:25 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 2 Jun 2020 16:43:25 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: <20200602201334.95D9718C079@mercury.lcs.mit.edu> References: <20200602201334.95D9718C079@mercury.lcs.mit.edu> Message-ID: On Tue, Jun 2, 2020 at 4:14 PM Noel Chiappa wrote: > Ahem. Lots more _core_. People keeep forgetting that we're looking at > decicions made at a time when each bit in main memory was stored in a > physically separate storage device, and having tons of memory was a dream > of > the future. > Yeah -- that is something that forgotten. There's a kit/hackday project to make 32-byte core for an Arduino I did with some of my boy scouts doing electronic MB a while back just to try to give them a feel what a 'bit' was. Similarly, there was a update of in late 1960's children's book originally called 'A Million' it's now called: A Million Dots Each page has 10K dots. The idea is to help young readers get a real feel for what 'a million' means visually. > > E.g. the -11/40 I first ran Unix on had _48 KB_ of core memory - total! > And that had to hold the resident OS, plus the application! It's no > surprise that Unix was so focused on small size - and as a corollary, on > high bang/buck ratio.' Amen -- I ran an 11/34 with 64K under V6 for about 3-6 months while we were awaiting the 256K memory upgrade. > > > But even in his age of lighting one's cigars with gigabytes of main memory > (literally), small is still beautiful, because it's easier to understand, > and > complexity is bad. So it's too bad Unix has lost that extreme parsimony. > Yep -- I think we were discussing this last week WRT to cat -v/fmt et al. I fear some people confuse 'progress' with 'feature creep.' Just because we can do something, does not mean we should. As I said, I'm a real fan of async I/O and like Paul, feel that it is a 'better' primitive. But I fully understand and accept, that given the tradeoffs of the time, UNIX did really well and I much prefer what we got than the alternative. I'm happy we ended up with simply and just works. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stewart at serissa.com Wed Jun 3 07:15:21 2020 From: stewart at serissa.com (Lawrence Stewart) Date: Tue, 2 Jun 2020 17:15:21 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: <202006021759.052Hx5Et022619@freefriends.org> Message-ID: <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> I remember working on getting Arpanet access on an 11/34 running V7 around 1978 or 1979. (SU-ISL). We used an 11/23 as a front end to run NCP, using a variation of Rand’s code. I wrote some sort of bisync driver for packet communications between the /23 and the /34, and I think added an IOCTL or some hack to ask if there was a message ready. So a polling variety of non-blocking I/O :) Meanwhile, on the Alto, surely an underpowered machine, the style for UI programming and asynch I/O was mostly event driven, much like libevent in more recent years. I found that very easy to understand. The main trajectory of PARC was lightweight threads calling synchronous I/O which really has to be counted as a Bad Idea not yet fully stamped out. I’ve always thought select(2) was a pretty good thing. In 1995 at Open Market we had a single-process web server that had no difficulties running 1000 to 1200 connections. I think that was BSD, and later OSF-1. -L From rdm at cfcl.com Wed Jun 3 08:14:10 2020 From: rdm at cfcl.com (Rich Morin) Date: Tue, 2 Jun 2020 15:14:10 -0700 Subject: [TUHS] non-blocking IO In-Reply-To: References: <20200602201334.95D9718C079@mercury.lcs.mit.edu> Message-ID: <0BF3AFA4-0B0E-45AA-A3AA-609704AF493F@cfcl.com> Case study (read, war story :-) of non-blocking I/O... Back in the mid-70's, I was doing scientific support programming for an X-Ray sky survey, using Fortran-63 under DRUM SCOPE on a CDC 3800 system which NRL's Space Sciences Division had (sigh) acquired and cobbled together from the US Government "excess property" list. Most of the time, our satellite was used in "scan mode", spinning on an axis which pointed at the Sun. This let us collect data on a two degree wide strip of sky. So, every six months we scanned the entire X-ray sky. The track was basically a helix, bent to match the shape of the Earth's orbit. Every so often, in order to look at a particularly interesting area, the spinning would be stopped, so the instrument could be put into "point mode". The challenge was to "bin" this data for further analysis. We did this by loading sets of 7-track tapes onto a CDC 813 drive, then dumping it out into 36 2x10 degree bins. This put a quarter of the data onto a set of tapes. Rinse, repeat for the other 3/4 of the data (after skipping past the already-written data). The result was 36 tapes, each holding an 8x10 degree bin (plus one extra tape for any point mode data). IIRC, we had five tape drives; my challenge was to keep them all as busy as possible, so as to dump the data set expeditiously. Because I had asynchronous I/O (mostly in the form of BUFFER IN and BUFFER OUT commands), I was able to implement a simple but quite effective polling loop. The machine room was a bit of a madhouse, but the tapes were written about as quickly as the hardware allowed. Asynchronous I/O FTW... https://en.wikipedia.org/wiki/United_States_Naval_Research_Laboratory#Space_sciences https://en.wikipedia.org/wiki/High_Energy_Astronomy_Observatory_1#A1:_Large-Area_Sky_Survey_instrument https://ub.fnwi.uva.nl/computermuseum/cdcdisk.html -r From jon at fourwinds.com Wed Jun 3 08:03:58 2020 From: jon at fourwinds.com (Jon Steinhart) Date: Tue, 02 Jun 2020 15:03:58 -0700 Subject: [TUHS] non-blocking IO - threads In-Reply-To: <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> Message-ID: <202006022203.052M3wpa167150@darkstar.fourwinds.com> Lawrence Stewart writes: > I’ve always thought select(2) was a pretty good thing. In 1995 at Open > Market we had a single-process web server that had no difficulties > running 1000 to 1200 connections. I think that was BSD, and later OSF-1. So select was a great thing until interactive graphics came along. Just got too messy. I recall writing assembly language code to support threads in the mid to late 1980s, and I remember reading some papers from Sun about the same. Everybody was doing it. Anybody have enough of the history to paint a picture of how we got from there to standardized threads? Jon From robpike at gmail.com Wed Jun 3 09:05:39 2020 From: robpike at gmail.com (Rob Pike) Date: Wed, 3 Jun 2020 09:05:39 +1000 Subject: [TUHS] non-blocking IO - threads In-Reply-To: <202006022203.052M3wpa167150@darkstar.fourwinds.com> References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <202006022203.052M3wpa167150@darkstar.fourwinds.com> Message-ID: It's only messy if you make a messy job of it. -rob On Wed, Jun 3, 2020 at 8:36 AM Jon Steinhart wrote: > Lawrence Stewart writes: > > I’ve always thought select(2) was a pretty good thing. In 1995 at Open > > Market we had a single-process web server that had no difficulties > > running 1000 to 1200 connections. I think that was BSD, and later OSF-1. > > So select was a great thing until interactive graphics came along. Just > got too messy. I recall writing assembly language code to support threads > in the mid to late 1980s, and I remember reading some papers from Sun about > the same. Everybody was doing it. Anybody have enough of the history to > paint a picture of how we got from there to standardized threads? > > Jon > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robpike at gmail.com Wed Jun 3 09:09:08 2020 From: robpike at gmail.com (Rob Pike) Date: Wed, 3 Jun 2020 09:09:08 +1000 Subject: [TUHS] non-blocking IO - threads In-Reply-To: References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <202006022203.052M3wpa167150@darkstar.fourwinds.com> Message-ID: To be more forthcoming, there was a long period when the received wisdom was that threads were unmanageable. But that was a category error: The programming model in POSIX and others was what made it hard. The fundamental concepts are a tiny step harder than trivial, and it was not hard to build a programming model that matched them. The first mistake people made was to split "thread" from "process". What those words mean today is not what they meant a generation ago. For me, it's like the mess sockets made of networking. Our programming models could be so much simpler than what we are required to deal with. -rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon at fourwinds.com Wed Jun 3 09:12:22 2020 From: jon at fourwinds.com (Jon Steinhart) Date: Tue, 02 Jun 2020 16:12:22 -0700 Subject: [TUHS] non-blocking IO - threads In-Reply-To: References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <202006022203.052M3wpa167150@darkstar.fourwinds.com> Message-ID: <202006022312.052NCMCR178629@darkstar.fourwinds.com> Rob Pike writes: > > It's only messy if you make a messy job of it. Maybe I should explain what I meant by messy. To me, the issue with select is that in many cases one has to either keep a lot of state somewhere, or live with long response times while handlers complete. Threads allowed handlers to be written in the "normal" way with state kept on the stack. From lm at mcvoy.com Wed Jun 3 09:21:14 2020 From: lm at mcvoy.com (Larry McVoy) Date: Tue, 2 Jun 2020 16:21:14 -0700 Subject: [TUHS] non-blocking IO - threads In-Reply-To: References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <202006022203.052M3wpa167150@darkstar.fourwinds.com> Message-ID: <20200602232114.GA22016@mcvoy.com> On Wed, Jun 03, 2020 at 09:09:08AM +1000, Rob Pike wrote: > The first mistake people made was to split "thread" from "process". What > those words mean today is not what they meant a generation ago. So I'm a fan of your quote, Rob, "If you think you need threads, your processes are too fat". Years ago, I asked Linus if he could fix the one flaw with that point of view. Which is, processes can't share page tables. So N processes will have N page tables, need an N way bigger TBL, to get the same performance. What I asked Linus to try and do is allow a process to share all or part of its page table with another process. If that could be done, we could pretty do what you said, just use processes and if they want to act like threads, just share the page table. And I just looked, he seems to have done it. man 2 clone on a Linux box. It's the swiss army knife of features, feels messy. I don't know if CLONE_VM will share the page tables, I'll ask Linus. --lm From robpike at gmail.com Wed Jun 3 10:39:45 2020 From: robpike at gmail.com (Rob Pike) Date: Wed, 3 Jun 2020 10:39:45 +1000 Subject: [TUHS] non-blocking IO - threads In-Reply-To: <20200602232114.GA22016@mcvoy.com> References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <202006022203.052M3wpa167150@darkstar.fourwinds.com> <20200602232114.GA22016@mcvoy.com> Message-ID: See the description of rfork at http://man.cat-v.org/plan_9/2/fork. -rob On Wed, Jun 3, 2020 at 9:21 AM Larry McVoy wrote: > On Wed, Jun 03, 2020 at 09:09:08AM +1000, Rob Pike wrote: > > The first mistake people made was to split "thread" from "process". What > > those words mean today is not what they meant a generation ago. > > So I'm a fan of your quote, Rob, "If you think you need threads, your > processes are too fat". > > Years ago, I asked Linus if he could fix the one flaw with that point > of view. Which is, processes can't share page tables. So N processes > will have N page tables, need an N way bigger TBL, to get the same > performance. > > What I asked Linus to try and do is allow a process to share all or > part of its page table with another process. If that could be done, > we could pretty do what you said, just use processes and if they want > to act like threads, just share the page table. > > And I just looked, he seems to have done it. man 2 clone on a > Linux box. It's the swiss army knife of features, feels messy. > > I don't know if CLONE_VM will share the page tables, I'll ask > Linus. > > --lm > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bakul at iitbombay.org Wed Jun 3 10:43:18 2020 From: bakul at iitbombay.org (Bakul Shah) Date: Tue, 2 Jun 2020 17:43:18 -0700 Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) Message-ID: <655420D4-EB34-4B75-A6BA-A874113B3680@iitbombay.org> > Would you be so kind to explain a bit about the hm version of MH. Not sure what Clem meant but I used "hm" by Jim Guyton (@Rand) on my Fortune box until I retired it. It provided a 2D interface. Later I tried xmh but didn't like it. The early history of MH is covered in some detail in Willis Ware's "RAND and the Information Evolution" book from page 128 onward. hm & Guyton get a paragraph on page 136. https://www.rand.org/content/dam/rand/pubs/corporate_pubs/2008/RAND_CP537.pdf From clemc at ccc.com Wed Jun 3 10:49:46 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 2 Jun 2020 20:49:46 -0400 Subject: [TUHS] mh/hm, mmh (was: fmt(1): history, POSIX, -t, -c) In-Reply-To: <655420D4-EB34-4B75-A6BA-A874113B3680@iitbombay.org> References: <655420D4-EB34-4B75-A6BA-A874113B3680@iitbombay.org> Message-ID: On Tue, Jun 2, 2020 at 8:43 PM Bakul Shah wrote: > > Would you be so kind to explain a bit about the hm version of MH. > > Not sure what Clem meant but I used "hm" by Jim Guyton (@Rand) on my > Fortune box until I retired it. It provided a 2D interface. > Yep, that would be the program .. Jim did a great job with it. It was built using a bunch of MH programs behind it, and was a curses front end. -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnold at skeeve.com Wed Jun 3 15:12:27 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Tue, 02 Jun 2020 23:12:27 -0600 Subject: [TUHS] Looking for vi files from http://alf.uib.no/pub/vi Message-ID: <202006030512.0535CRFC020727@freefriends.org> Hi. Does anyone have a mirror of the files that once upon a time were at http://alf.uib.no/pub/vi? They were mirrored at ftp://ftp.uu.net/pub/text-processing/vi also. If so, can you please send me a tarball off list (or tell me where I can download a copy from)? Thanks! Arnold From heinz at osta.com Wed Jun 3 15:41:16 2020 From: heinz at osta.com (Heinz Lycklama) Date: Tue, 2 Jun 2020 22:41:16 -0700 Subject: [TUHS] non-blocking IO (morphing to "The serial I/O loop") In-Reply-To: References: Message-ID: <3e482f51-2c2e-19a8-c9fd-f23c67f1df8c@osta.com> See comments below. Heinz On 6/2/2020 2:34 AM, Paul Ruizendaal wrote: >> On 5/31/2020 9:46 AM, Warner Losh wrote: >>> Sorry to top post, but LSX or Miniunix had non blocking I/O as well. >>> It was in one of the documents that Clem scanned in the last year. It >>> specifically was an experiment into how to do it. >>> >>> Warner >> I did add a few new features to LSX to deal with contiguous files >> and to handle asynchronous read/write's for real time applications. >> They are described in the LSX paper in the 1978 BSTJ on the >> UNIX Time-Sharing System. >> >> Heinz > Thanks for highlighting this! > > The realm here is async I/O to disk, my original scope was limited to “communication” files (tty’s, pipes, network connections). Still, I find it an interesting topic. > > For others, the paper that you refer to can also be found here: > https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/UNIX_on_a_Microprocessor_19780322.pdf > If I read correctly, the async functionality was available only in the stand alone program version of LSX. Is that correct? In any case, the source code appears lost. Any source code for LSX would be archived somewhere within Nokia Bell Labs? > From another paper in that set, I get the impression that the async functionality for LSX builds on earlier work for a very early version of Unix: > https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-74-1352-1_Implementation_of_Large_Contiguous_Files_and_Asynchronous_IO_in_UNIX_19740104.pdf Yes, as this memo describes, I implemented a very simple version of large contiguous files and asynchronous file I/O for a special application. The MERT system developed a little later on, had a completely integrated of implementation of contiguous files and asynchronous file I/O. > - - - > > When reading through the papers in that TUHS directory, something else caught my attention: early networking at the labs. For a while I have been puzzled by the “serial I/O loop” in use at the labs in the early seventies. Last Fall I found some 1970/1971 BSTJ papers about it, but there it stopped. Jon Steinhart would have more information about the serial I/O loo. > I see in the memo on Glance (https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-75-1352-3_GLANCE_Terminals_on_UNIX_Time-Sharing_19750303.pdf) that D.R. Weller continued to work on it up to 1973 at least and that it was integrated with Unix in some way. Is that correct? Did the two memo’s referred to (MM 70-1384-1 and TM 73-1356-8) survive? I do not believe that the software for the Glance terminals was ever integrated into the standard UNIX software distribution > Then the memo on satellite processors is very interesting (https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-78-3114-2_A_Minicomputer_Satellite_Processor_System_19780322.pdf). This appears to show the serial I/O loop in use as late as 1978, with a very intriguing use case involving system call forwarding over the network. Carl Christensen and I developed this software to handle small computer systems with disk storage, as described in the memo, and in the BSTJ article. The satellite diskless computers could be attached to a powerful central computer by means the serial I/O loop or various standard DEC controllers. Each of the satellite computers could run different tailored applications > Can you tell me more about the serial I/O loop and its use cases? I think Jon Steinhart has described this elsewhere? > > > > From lars at nocrew.org Wed Jun 3 15:38:29 2020 From: lars at nocrew.org (Lars Brinkhoff) Date: Wed, 03 Jun 2020 05:38:29 +0000 Subject: [TUHS] Unix on the Arpanet In-Reply-To: <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> (Lawrence Stewart's message of "Tue, 2 Jun 2020 17:15:21 -0400") References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> Message-ID: <7wtuzs3e0a.fsf_-_@junk.nocrew.org> Lawrence Stewart wrote: > I remember working on getting Arpanet access on an 11/34 running V7 > around 1978 or 1979. (SU-ISL). We used an 11/23 as a front end to run > NCP, using a variation of Rand’s code. Has this, or Rand's code, been preserved? I'm only aware of one Arpanet NCP implementation for Unix that is online, the one from University of Illinois. From stewart at serissa.com Wed Jun 3 22:23:33 2020 From: stewart at serissa.com (Lawrence Stewart) Date: Wed, 3 Jun 2020 08:23:33 -0400 Subject: [TUHS] Unix on the Arpanet In-Reply-To: <7wtuzs3e0a.fsf_-_@junk.nocrew.org> References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <7wtuzs3e0a.fsf_-_@junk.nocrew.org> Message-ID: <68231FF8-6E05-49AE-A33A-A22D5A05966A@serissa.com> > On 2020, Jun 3, at 1:38 AM, Lars Brinkhoff wrote: > > Lawrence Stewart wrote: >> I remember working on getting Arpanet access on an 11/34 running V7 >> around 1978 or 1979. (SU-ISL). We used an 11/23 as a front end to run >> NCP, using a variation of Rand’s code. > > Has this, or Rand's code, been preserved? > > I'm only aware of one Arpanet NCP implementation for Unix that is > online, the one from University of Illinois. Alas I do not know. I may have some old emails from that era. I will check. From clemc at ccc.com Wed Jun 3 22:33:10 2020 From: clemc at ccc.com (Clem Cole) Date: Wed, 3 Jun 2020 08:33:10 -0400 Subject: [TUHS] Looking for vi files from http://alf.uib.no/pub/vi In-Reply-To: <202006030512.0535CRFC020727@freefriends.org> References: <202006030512.0535CRFC020727@freefriends.org> Message-ID: I think so... I'll send you a URL off line. Clem On Wed, Jun 3, 2020 at 1:14 AM wrote: > Hi. > > Does anyone have a mirror of the files that once upon a time were > at http://alf.uib.no/pub/vi? They were mirrored at > ftp://ftp.uu.net/pub/text-processing/vi also. > > If so, can you please send me a tarball off list (or tell me where > I can download a copy from)? > > Thanks! > > Arnold > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Wed Jun 3 22:49:10 2020 From: clemc at ccc.com (Clem Cole) Date: Wed, 3 Jun 2020 08:49:10 -0400 Subject: [TUHS] Looking for vi files from http://alf.uib.no/pub/vi In-Reply-To: <7EA7EAE7-991B-4B04-A89A-43C2B62995E1@icloud.com> References: <202006030512.0535CRFC020727@freefriends.org> <7EA7EAE7-991B-4B04-A89A-43C2B62995E1@icloud.com> Message-ID: It is too big to attach to email. On Wed, Jun 3, 2020 at 8:39 AM Michael Stiller wrote: > Hi Clem, > > why offline? Other people are also interested. :-) > > Cheers, > > Michael > > > > On 3. Jun 2020, at 14:33, Clem Cole wrote: > > > > I think so... I'll send you a URL off line. > > > > Clem > > > > On Wed, Jun 3, 2020 at 1:14 AM wrote: > > Hi. > > > > Does anyone have a mirror of the files that once upon a time were > > at http://alf.uib.no/pub/vi? They were mirrored at > > ftp://ftp.uu.net/pub/text-processing/vi also. > > > > If so, can you please send me a tarball off list (or tell me where > > I can download a copy from)? > > > > Thanks! > > > > Arnold > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.winalski at gmail.com Thu Jun 4 02:31:57 2020 From: paul.winalski at gmail.com (Paul Winalski) Date: Wed, 3 Jun 2020 12:31:57 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: <0BF3AFA4-0B0E-45AA-A3AA-609704AF493F@cfcl.com> References: <20200602201334.95D9718C079@mercury.lcs.mit.edu> <0BF3AFA4-0B0E-45AA-A3AA-609704AF493F@cfcl.com> Message-ID: On 6/2/20, Rich Morin wrote: > > IIRC, we had five tape drives; my challenge was to keep them all as busy as > possible, so as > to dump the data set expeditiously. Because I had asynchronous I/O (mostly > in the form of > BUFFER IN and BUFFER OUT commands), I was able to implement a simple but > quite effective > polling loop. The machine room was a bit of a madhouse, but the tapes were > written about > as quickly as the hardware allowed. Asynchronous I/O FTW... With 9-track magnetic tape devices, reading and writing can't start until the tape is up to speed. Once up to speed the drive can read and write records while keeping the tape moving at speed. This is called streaming. If there's a pause in the read/write requests from the CPU, time is lost as the drive stops and starts moving the tape. It was essential that applications doing large amounts of tape I/O keep up the I/O requests at a rate that allows streaming. Asynchronous I/O with multi-buffering is a straightforward way to accomplish this. The IBM S/360 channel commands for tape devices provided a mechanism for the tape control unit to send an interrupt to the CPU when a read or write channel command completed. This notified the sequential access method (the user program I/O interface) when I/O to each buffer had completed and the buffer was available for reuse. OS/360's Sequential Access Method could read or write an entire tape using a single SIO (start I/O) instruction, as long as no read or write errors were encountered. -Paul W. From paul.winalski at gmail.com Thu Jun 4 02:42:28 2020 From: paul.winalski at gmail.com (Paul Winalski) Date: Wed, 3 Jun 2020 12:42:28 -0400 Subject: [TUHS] non-blocking IO - threads In-Reply-To: <202006022312.052NCMCR178629@darkstar.fourwinds.com> References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <202006022203.052M3wpa167150@darkstar.fourwinds.com> <202006022312.052NCMCR178629@darkstar.fourwinds.com> Message-ID: On 6/2/20, Jon Steinhart wrote: > > Maybe I should explain what I meant by messy. To me, the issue with select is > that in many cases one has to either keep a lot of state somewhere, or live with > long response times while handlers complete. Threads allowed handlers to be > written in the "normal" way with state kept on the stack. > TOPS-10 had a nifty program called ptycon that would allow one to control multiple login sessions, each on its own pseudo-terminal device, from a single physical terminal. In 1985 our software tools group implemented a pseudo-terminal device driver for VMS, and I wrote a program called ptycon-32. It was multithreaded, with one thread of control for each pseudo-terminal. The problem is that VMS at the time didn't have a threads package. I had to roll my own threads by using the asynchronous system traps (ASTs) delivered by completion of I/O to the pseudo-terminals. Talk about messy.... -Paul W. From nobozo at gmail.com Thu Jun 4 03:57:33 2020 From: nobozo at gmail.com (Jon Forrest) Date: Wed, 3 Jun 2020 10:57:33 -0700 Subject: [TUHS] non-blocking IO - threads In-Reply-To: References: <202006021759.052Hx5Et022619@freefriends.org> <515ABF82-79D4-4DB8-90F1-4DDD512C7706@serissa.com> <202006022203.052M3wpa167150@darkstar.fourwinds.com> <202006022312.052NCMCR178629@darkstar.fourwinds.com> Message-ID: <332ca1eb-f862-b028-a2a2-e168399576a1@gmail.com> On 6/3/2020 9:42 AM, Paul Winalski wrote: > The problem is that VMS at the time > didn't have a threads package. I had to roll my own threads by using > the asynchronous system traps (ASTs) delivered by completion of I/O to > the pseudo-terminals. Talk about messy.... That's also how the Sybase SQL Server worked on VMS at the same time. Jon From jpl.jpl at gmail.com Thu Jun 4 05:19:40 2020 From: jpl.jpl at gmail.com (John P. Linderman) Date: Wed, 3 Jun 2020 15:19:40 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: References: <20200602201334.95D9718C079@mercury.lcs.mit.edu> <0BF3AFA4-0B0E-45AA-A3AA-609704AF493F@cfcl.com> Message-ID: On a distantly related note, when I worked part time for the MIT administration in the 60's, we'd do processing on the 7094 in the main comp center, then bring a tape of results back to our office to do the printing and card punching. For reasons that were never explained to me, the blocks of print and punch data were combined on a single tape, and were distinguished by being written with different parities. If you attempted to read a block with the correct parity setting, the tape could stream. If you guessed wrong, you had to stop, back up a block, change the parity, and try again. So I wrote a simple 360 assembly program to keep track how often blocks of each parity followed the observed parities of the previous 8 blocks. Essentially, 256 pairs of parity observations, indexed by the previous 8 parity observations. Blocks of print and punch data tended to fall into patterns that depended on what job was being run on the 7094, so detecting those patterns and correctly anticipating the upcoming parity made the tapes move much more smoothly. It was fun to watch the tape at the start of a run. It was mostly just a coin-toss, so the tape was jerking around fitfully. As the patterns started to emerge, the predictions got better, the jerking got less and less common, and the tapes were streaming most of the time. My introduction to learning algorithms. On Wed, Jun 3, 2020 at 12:33 PM Paul Winalski wrote: > On 6/2/20, Rich Morin wrote: > > > > IIRC, we had five tape drives; my challenge was to keep them all as busy > as > > possible, so as > > to dump the data set expeditiously. Because I had asynchronous I/O > (mostly > > in the form of > > BUFFER IN and BUFFER OUT commands), I was able to implement a simple but > > quite effective > > polling loop. The machine room was a bit of a madhouse, but the tapes > were > > written about > > as quickly as the hardware allowed. Asynchronous I/O FTW... > > With 9-track magnetic tape devices, reading and writing can't start > until the tape is up to speed. Once up to speed the drive can read > and write records while keeping the tape moving at speed. This is > called streaming. If there's a pause in the read/write requests from > the CPU, time is lost as the drive stops and starts moving the tape. > It was essential that applications doing large amounts of tape I/O > keep up the I/O requests at a rate that allows streaming. > Asynchronous I/O with multi-buffering is a straightforward way to > accomplish this. The IBM S/360 channel commands for tape devices > provided a mechanism for the tape control unit to send an interrupt to > the CPU when a read or write channel command completed. This notified > the sequential access method (the user program I/O interface) when I/O > to each buffer had completed and the buffer was available for reuse. > OS/360's Sequential Access Method could read or write an entire tape > using a single SIO (start I/O) instruction, as long as no read or > write errors were encountered. > > -Paul W. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gnu at toad.com Thu Jun 4 11:24:52 2020 From: gnu at toad.com (John Gilmore) Date: Wed, 03 Jun 2020 18:24:52 -0700 Subject: [TUHS] non-blocking IO: Simplicity has been subtituted for efficiency In-Reply-To: <20200602201334.95D9718C079@mercury.lcs.mit.edu> References: <20200602201334.95D9718C079@mercury.lcs.mit.edu> Message-ID: <8066.1591233892@hop.toad.com> Noel Chiappa wrote: > > I'm curious as to what the rationale was for Unix to have been designed > > with basic I/O being blocking rather than asynchronous. > > It's a combination of two factors, I reckon. One, which is better depends a > lot on the type of thing you're trying to do. For many typical thing (e.g. > 'ls'), blocking is a good fit. And, as As Arnold says, asyhchronous I/O is > more complicated, and Unix was (well, back then at least) all about getting > the most bang for the least bucks. I just happened to open a binder today of old papers about UNIX(tm), including Ken Thompson's "Unix Implementation" paper, which says at the bottom of the introduction: "What is or is not implemented in the kernel represents both a great responsibiity and a great power. It is a soap-box platform on "the way things should be done." Even so, if "the way" is too radical, no one will follow it. Every important decision was weighted carefully. Throughout, simplicity has been substituted for efficiency. Complex algorithms are used only if their complexity can be localized." John From imp at bsdimp.com Thu Jun 4 13:38:43 2020 From: imp at bsdimp.com (Warner Losh) Date: Wed, 3 Jun 2020 21:38:43 -0600 Subject: [TUHS] 2.11BSD pl 0 recovered (mostly) PRELIMINARY Message-ID: OK. I've written a script to take the 2.11BSD pl 195 tape and reverse apply all the patches that we have to get back to 2.11BSD original. There's some problems. The biggest one is that ld.c was rewritten during this series and what it replaced is lost. And the 2.10.1 ld.c isn't what's in 2.11BSD and the patches to get from 2.10.1 to 2.11 don't seem to be out there. And the other patches in the series make it clear that patches are missing. 2.11BSD likely will work with 2.10.1's ld, so this isn't fatal. There's 122 other files that I recovered from 2.10.1. Almost all of those are a good guess, if not what's actually in 2.11. Many of these can be verified via other means. Some of these can be snagged from comp.bugs.2bsd (that's how as was recovered). There's a number of small cosmetic changes made via copying, some to get rid of redundant files, etc. I think these don't matter for the function of the system, but are small differences from the actual tape that shipped. I still need to try to still create a tape and try to compile. And forward apply all the patches and create a git repo from it. All things considered 99% of the files have been recovered at this point. It's early days, but I've pushed this to github for comments. http://github.com/bsdimp/mk211bsd It just assumes you have the Tuhs archive (including the new Usenet section) and that apout works on your computer. It works on FreeBSD for sure. No clue about anything else... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnold at skeeve.com Thu Jun 4 16:27:10 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Thu, 04 Jun 2020 00:27:10 -0600 Subject: [TUHS] non-blocking IO: Simplicity has been subtituted for efficiency In-Reply-To: <8066.1591233892@hop.toad.com> References: <20200602201334.95D9718C079@mercury.lcs.mit.edu> <8066.1591233892@hop.toad.com> Message-ID: <202006040627.0546RAMW020424@freefriends.org> John Gilmore wrote: > I just happened to open a binder today of old papers about UNIX(tm), > including Ken Thompson's "Unix Implementation" paper, which says at the > bottom of the introduction: > > "What is or is not implemented in the kernel represents both a great > responsibiity and a great power. It is a soap-box platform on "the > way things should be done." Even so, if "the way" is too radical, no > one will follow it. Every important decision was weighted carefully. > Throughout, simplicity has been substituted for efficiency. Complex > algorithms are used only if their complexity can be localized." Words to live by. I remember reading that paper early on in my career and that this statement certainly influenced my subsequent thinking about programming in general. Arnold From peter at rulingia.com Thu Jun 4 19:04:36 2020 From: peter at rulingia.com (Peter Jeremy) Date: Thu, 4 Jun 2020 19:04:36 +1000 Subject: [TUHS] non-blocking IO In-Reply-To: <20200601145801.GE22016@mcvoy.com> References: <20200601145801.GE22016@mcvoy.com> Message-ID: <20200604090436.GJ279@server.rulingia.com> On 2020-Jun-01 07:58:02 -0700, Larry McVoy wrote: >On Mon, Jun 01, 2020 at 01:32:56PM +1000, Dave Horsfall wrote: >> On Mon, 1 Jun 2020, Rob Pike wrote: >> >> > I???m not quite sure why the Research lineage did not include >> > non-blocking behaviour, especially in view of the man page comments. >> > Maybe it was seen as against the Unix philosophy, with select() >> > offering sufficient mechanism to avoid blocking (with open() the hard >> > corner case)? >> > >> >That's it. Select was good enough for our purposes. >> >> After being dragged through both Berserkley and SysVile, I never did get the >> hang of poll()/select() etc,,, > >I'm sure you could, select is super handy, think a network server like >apache. My view may be unpopular but I've always been disappointed that Unix implemented blocking I/O only and then had to add various hacks to cover up for the lack of asynchonous I/O. It's trivial to build blocking I/O operations on top of asynchonous I/O operations. It's impossible to do the opposite without additional functionality. I also found it disappointing that poll()/select() only worked on TTY and network operations. HDDs are really slow compared to CPUs and it would be really nice if a process could go and do something else whilst waiting for a file to open. -- Peter Jeremy -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From arnold at skeeve.com Thu Jun 4 23:39:35 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Thu, 04 Jun 2020 07:39:35 -0600 Subject: [TUHS] Looking for vi files from http://alf.uib.no/pub/vi In-Reply-To: <7EA7EAE7-991B-4B04-A89A-43C2B62995E1@icloud.com> References: <202006030512.0535CRFC020727@freefriends.org> <7EA7EAE7-991B-4B04-A89A-43C2B62995E1@icloud.com> Message-ID: <202006041339.054DdZk7008288@freefriends.org> Hi All. I've put Clem's file up at http://www.skeeve.com/text-processing.tar.bz2. It's a little over 16 meg and includes more than just the vi stuff. Enjoy, Arnold Michael Stiller wrote: > Hi Clem, > > why offline? Other people are also interested. :-) > > Cheers, > > Michael > > > > On 3. Jun 2020, at 14:33, Clem Cole wrote: > > > > I think so... I'll send you a URL off line. > > > > Clem > > > > On Wed, Jun 3, 2020 at 1:14 AM wrote: > > Hi. > > > > Does anyone have a mirror of the files that once upon a time were > > at http://alf.uib.no/pub/vi? They were mirrored at > > ftp://ftp.uu.net/pub/text-processing/vi also. > > > > If so, can you please send me a tarball off list (or tell me where > > I can download a copy from)? > > > > Thanks! > > > > Arnold From imp at bsdimp.com Fri Jun 5 00:19:58 2020 From: imp at bsdimp.com (Warner Losh) Date: Thu, 4 Jun 2020 08:19:58 -0600 Subject: [TUHS] non-blocking IO In-Reply-To: <20200604090436.GJ279@server.rulingia.com> References: <20200601145801.GE22016@mcvoy.com> <20200604090436.GJ279@server.rulingia.com> Message-ID: On Thu, Jun 4, 2020 at 3:06 AM Peter Jeremy wrote: > On 2020-Jun-01 07:58:02 -0700, Larry McVoy wrote: > >On Mon, Jun 01, 2020 at 01:32:56PM +1000, Dave Horsfall wrote: > >> On Mon, 1 Jun 2020, Rob Pike wrote: > >> > >> > I???m not quite sure why the Research lineage did not include > >> > non-blocking behaviour, especially in view of the man page comments. > >> > Maybe it was seen as against the Unix philosophy, with select() > >> > offering sufficient mechanism to avoid blocking (with open() the hard > >> > corner case)? > >> > > >> >That's it. Select was good enough for our purposes. > >> > >> After being dragged through both Berserkley and SysVile, I never did > get the > >> hang of poll()/select() etc,,, > > > >I'm sure you could, select is super handy, think a network server like > >apache. > > My view may be unpopular but I've always been disappointed that Unix > implemented blocking I/O only and then had to add various hacks to cover > up for the lack of asynchonous I/O. It's trivial to build blocking I/O > operations on top of asynchonous I/O operations. It's impossible to do > the opposite without additional functionality. > > I also found it disappointing that poll()/select() only worked on TTY and > network operations. HDDs are really slow compared to CPUs and it would be > really nice if a process could go and do something else whilst waiting for > a file to open. > Lest anybody think this is a theoretical concern, Netflix has spent quite a bit of effort to reduce the sources of latency in our system. The latency for open doesn't happen often, due to caching, but when it does this causes a hickup for nginx worker thread (since open is blocking). If you get enough hickups, you wind up consuming all your worker threads and latency for everybody suffers while waiting for these to complete (think flaky disk that suddenly takes a really long time for each of its I/Os for one example). We've hacked FreeBSD in various ways to reduce or eliminate this delay.... In FreeBSD we have to translate from the pathname to a vnode, and to do that we have to look up directories, indirect block tables, etc. All these are surprising places that one could bottleneck at... And if you try to access the vnode before all this is done, you'll wait for it (though in the case of sendfile it doesn't matter since that's async and only affect the one I/O)... So I could see how having a async open could introduce a lot of hair into the mix depending on how you do it. Without a robust callback/AST mechanism, my brain is recoiling from the EALREADY errors in sockets for things that are already in progress... reads and write are easy by comparison :) The kicker is that all of the kernel is callback driven. The upper half queues the request and then sleeps until the lower half signals it to wakeup. And that signal is often just a wakeup done from the completion routine in the original request. All of that would be useful in userland for high volume activity, none of it is exposed... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From dot at dotat.at Fri Jun 5 02:34:54 2020 From: dot at dotat.at (Tony Finch) Date: Thu, 4 Jun 2020 17:34:54 +0100 Subject: [TUHS] non-blocking IO In-Reply-To: References: <20200601145801.GE22016@mcvoy.com> <20200604090436.GJ279@server.rulingia.com> Message-ID: Warner Losh wrote: > > So I could see how having a async open could introduce a lot of hair into > the mix depending on how you do it. Linux io_uring, for example. I would be interested if anyone has a historical perspective on its design :-) Tony. -- f.anthony.n.finch http://dotat.at/ Wight, Portland, Plymouth, Biscay, Fitzroy, Sole, Lundy, Fastnet: North or northwest 4 to 6, occasionally 7 in Biscay. Slight or moderate at first except in Biscay and Fitzroy, otherwise moderate or rough. Showers. Good, occasionally moderate. From lm at mcvoy.com Fri Jun 5 02:50:11 2020 From: lm at mcvoy.com (Larry McVoy) Date: Thu, 4 Jun 2020 09:50:11 -0700 Subject: [TUHS] non-blocking IO In-Reply-To: References: <20200601145801.GE22016@mcvoy.com> <20200604090436.GJ279@server.rulingia.com> Message-ID: <20200604165011.GC18437@mcvoy.com> On Thu, Jun 04, 2020 at 08:19:58AM -0600, Warner Losh wrote: > The kicker is that all of the kernel is callback driven. The > upper half queues the request and then sleeps until the lower half signals > it to wakeup. And that signal is often just a wakeup done from the > completion routine in the original request. All of that would be useful in > userland for high volume activity, none of it is exposed... Yeah, I've often wondered why this stuff wasn't exposed. We already have signal handlers, seems like that maps. I tried to get the NFS guys at Sun to rethink the biod junk and do it like UFS does, where it queues something and gets a callback. I strongly suspect that two processes, one to queue, one to handle callbacks, would be more efficient and actually faster than the biod nonsense. That's one of the arguments I lost unfortunately. Warner, exposing that stuff in FreeBSD is not really that hard, I suspect. Might be a fun project for a young kernel hacker with some old dude like you or me or someone, watching over it and thinking about the API. --lm From crossd at gmail.com Sat Jun 6 02:00:52 2020 From: crossd at gmail.com (Dan Cross) Date: Fri, 5 Jun 2020 12:00:52 -0400 Subject: [TUHS] non-blocking IO In-Reply-To: <20200604165011.GC18437@mcvoy.com> References: <20200601145801.GE22016@mcvoy.com> <20200604090436.GJ279@server.rulingia.com> <20200604165011.GC18437@mcvoy.com> Message-ID: On Thu, Jun 4, 2020 at 12:51 PM Larry McVoy wrote: > On Thu, Jun 04, 2020 at 08:19:58AM -0600, Warner Losh wrote: > > The kicker is that all of the kernel is callback driven. The > > upper half queues the request and then sleeps until the lower half > signals > > it to wakeup. And that signal is often just a wakeup done from the > > completion routine in the original request. All of that would be useful > in > > userland for high volume activity, none of it is exposed... > > Yeah, I've often wondered why this stuff wasn't exposed. We already have > signal handlers, seems like that maps. > Was it Rob who said that signals were really just for SIGKILL? Here, signals would be gang-pressed into service as a general IPC mechanism. In fairness, they've mutated that way, but they didn't start out that way. While I obviously wasn't there, the strong impression I get is that by the time people were seriously thinking about async IO in Unix, the die had already been cast for better or worse. > I tried to get the NFS guys at Sun to rethink the biod junk and do it like > UFS does, where it queues something and gets a callback. I strongly > suspect > that two processes, one to queue, one to handle callbacks, would be more > efficient and actually faster than the biod nonsense. > > That's one of the arguments I lost unfortunately. > > Warner, exposing that stuff in FreeBSD is not really that hard, I suspect. > Might be a fun project for a young kernel hacker with some old dude like > you or me or someone, watching over it and thinking about the API. > I'm going to actually disagree with you here, Larry. While I think a basic mechanism wouldn't be THAT hard to implement, it wouldn't compose nicely with the existing primitives. I suspect the edge cases would be really thorny, particularly without a real AST abstraction. For instance, what happens if you initiate an async IO operation, then block on a `read`? Where does the callback happen? If on the same thread, The real challenge isn't providing the operation, it's integrating it into the existing model. As a counter-point to the idea that it's completely unruly, in Akaros this was solved in the C library: all IO operations were fundamentally asynchronous, but the C library provided blocking read(), write(), etc by building those from the async primitives. It worked well, but Akaros had something akin to an AST environment and fine-grain scheduling decisions were made in userspace: in Akaros the unit of processor allocation is a CPU core, not a thread, and support exists for determining the status of all cores allocated to a process. There are edge cases (you can't roll-your-own mutex, for example, and the basic threading library does a lot of heavy lifting for you making it challenging to integrate into the runtime of a language that doesn't use the same ABI), but by and large it worked. It was also provided by a kernel that was a pretty radical departure from a Unix-like kernel. - Dan C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave at horsfall.org Sat Jun 6 06:57:06 2020 From: dave at horsfall.org (Dave Horsfall) Date: Sat, 6 Jun 2020 06:57:06 +1000 (EST) Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <95e6e8de901c837a28b84e62556ba326@firemail.de> Message-ID: On Wed, 27 May 2020, Greg A. Woods wrote: > Sadly most compilers, including GCC and Clang/LLVM will, at best, warn > (and warnings are only treated as errors by the most macho|wise); and > compilers only do that now because they've been getting flack from > developers whenever the optimizer does something unexpected. Don't talk to me about optimisers... That's not the code that I wrote! I've seen code simply disappear, because the "optimiser" though that it was cleverer than I was. > The Linux kernel example I've referred to involved dereferencing a > pointer to do an assignment in a local variable definition, then a few > lines later testing if the pointer was NULL before using the local > variable. Unoptimised the code will dereference a NULL pointer and load > junk from location zero into the variable (because it's kernel code), > then the NULL test will trigger and all will be good. The optimizer > rips out the NULL check because "obviously" the programmer has assumed > the pointer is always a valid non-NULL pointer since they've explicitly > dereferenced it before checking it and they wouldn't want to waste even > a single jump-on-zero instruction checking it again. (It's also quite > possible the code was written "correctly" at first, then someone mushed > all the variable initialisations up onto their definitions.) Typical Penguin/OS behaviour... > In any case there's now a GCC option: -fno-delete-null-pointer-checks > (to go along with -fno-strict-aliasing and -fno-strict-overflow, and > -fno-strict-enums, all of which MUST be used, and sometimes > -fno-strict-volatile-bitfields too, on all legacy code that you don't > want to break) I'm sure that there's a competition somewhere, to see who can come with GCC's -fmost-longest-and-most-obscure-option flags... > It's even worse when you have to write bare-metal code that must > explictly dereference a NULL pointer (a not-so-real example: you want > to use location zero in the CPU zero-page (e.g. on a 6502 or 6800, or > PDP-8, etc.) as a pointer) -- it is now impossible to do that in strict > Standard C even though trivially it "should just work" despite the silly > rules. As far as I can tell it always did just work in "plain old" C. I've programmed a PDP-8! 'Twas way back in high school, and I found a bug in my mentor's program; it controlled traffic lights... > The crazy thing about modern optimizers is that they're way more > persistent and often somewhat more clever than your average programmer. > They follow all the paths. They apply all the rules at every turn. Optimisers... Grrr... -- Dave From cym224 at gmail.com Sat Jun 6 07:40:50 2020 From: cym224 at gmail.com (Nemo Nusquam) Date: Fri, 5 Jun 2020 17:40:50 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <95e6e8de901c837a28b84e62556ba326@firemail.de> Message-ID: <5f5d5c7f-ce33-d067-2a62-aa369175d9cb@gmail.com> On 06/05/20 16:57, Dave Horsfall wrote (in part): > [...] > Optimisers... Grrr... Steve Johnson's position paper on optimising compilers may amuse you: https://dl.acm.org/doi/abs/10.1145/567532.567542 N. > > -- Dave From rich.salz at gmail.com Sat Jun 6 07:47:23 2020 From: rich.salz at gmail.com (Richard Salz) Date: Fri, 5 Jun 2020 17:47:23 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <95e6e8de901c837a28b84e62556ba326@firemail.de> Message-ID: > I'm sure that there's a competition somewhere, to see who can come with > GCC's -fmost-longest-and-most-obscure-option flags... > At least one of the GCC maintainers is German, so possibly. Can clang keep up? :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From bakul at iitbombay.org Sat Jun 6 08:01:29 2020 From: bakul at iitbombay.org (Bakul Shah) Date: Fri, 5 Jun 2020 15:01:29 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <95e6e8de901c837a28b84e62556ba326@firemail.de> Message-ID: <63245F94-E434-42D9-A294-8AE6971FE66D@iitbombay.org> On Jun 5, 2020, at 2:47 PM, Richard Salz wrote: > > > | I'm sure that there's a competition somewhere, to see who can come with > | GCC's -fmost-longest-and-most-obscure-option flags... > > At least one of the GCC maintainers is German, so possibly. Can clang keep up? :) Clang has more than kept up! clang: -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang gcc-9: -print-sysroot-headers-suffix Not counting gcc's --help={common|optimizers|params|target|warnings|[^]{joined|separate|undocumented}}[,...]. From imp at bsdimp.com Sat Jun 6 08:51:27 2020 From: imp at bsdimp.com (Warner Losh) Date: Fri, 5 Jun 2020 16:51:27 -0600 Subject: [TUHS] My BSDcan talk Message-ID: OK. Must be off my game... I forgot to tell people about my BSDcan talk earlier today. It was streamed live, and will be online in a week or three... It's another similar to the last two. I've uploaded a version to youtube until the conference has theirs ready. It's a private link, but should work for anybody that has it. Now that I've given my talk it's cool to share more widely... https://www.youtube.com/watch?v=NRq8xEvFS_g The link at the end is wrong. https://github.com/bsdimp/bsdcan2020-demos is the proper link. Please let me know what you think. Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From pnr at planet.nl Sat Jun 6 21:30:58 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Sat, 6 Jun 2020 13:30:58 +0200 Subject: [TUHS] Unix on Aprpanet Message-ID: >> On 2020, Jun 3, at 1:38 AM, Lars Brinkhoff wrote: >> >> Lawrence Stewart wrote: >>> I remember working on getting Arpanet access on an 11/34 running V7 >>> around 1978 or 1979. (SU-ISL). We used an 11/23 as a front end to run >>> NCP, using a variation of Rand’s code. I wrote some sort of bisync >>> driver for packet communications between the /23 and the /34, and I >>> think added an IOCTL or some hack to ask if there was a message ready. >>> So a polling variety of non-blocking I/O :) >> >> Has this, or Rand's code, been preserved? >> >> I'm only aware of one Arpanet NCP implementation for Unix that is >> online, the one from University of Illinois. > Alas I do not know. I may have some old emails from that era. I will check. I have not so far come across an NCP Unix that was not based on the UoI code base. At a time there was a “Rand Unix”, that combined the UoI code with a few other extensions. A 1978 document describes the kernel as: "UNIX version 6 with modifications some of which are: - Accounting - A system call to write end of files on pipes - A system call which checks whether a process would sleep upon doing a read on either a pipe or tty - Pseudo teletypes - The network control program NCP written at the University of Illinois - A new method of interprocess communication called ports. Ports are similar to pipes but have names and can be accessed by any process.” The system call to write EOF on pipes is eofp() and the blocking read test is empty(). Both are available in the NCP Unix source code as included on the TUHS Unix Tree page. It could well be that the hack Lawrence refers to above was a modification to empty() to also cover his bisync driver. I guess empty() is the first step on the path that includes capac()/await() and later select()/poll() as the preferred mechanism to prevent blocking on IPC files (incl. tty’s). From jnc at mercury.lcs.mit.edu Sat Jun 6 23:29:06 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Sat, 6 Jun 2020 09:29:06 -0400 (EDT) Subject: [TUHS] non-blocking IO Message-ID: <20200606132906.B7FAB18C079@mercury.lcs.mit.edu> > From: Peter Jeremy > My view may be unpopular but I've always been disappointed that Unix > implemented blocking I/O only and then had to add various hacks to cover > up for the lack of asynchonous I/O. It's trivial to build blocking I/O > operations on top of asynchonous I/O operations. It's impossible to do > the opposite without additional functionality. Back when I started working on networks, I looked at other kinds of systems to see what general lessons I could learn about the evolution of systems, which might apply to the networks we were building. (I should have written all that up, never did, sigh.) One major one was that a system, when small, often collapses multiple needs onto one machanism. Only as the system grows in size do scaling effects, etc necessitate breaking them up into separate mechanisms. (There are some good examples in file systems, for example.) I/O is a perfect example of this; a small system can get away with only one kind; it's only when the system grows that one benefits from having both synchronous and asynchronous. Since the latter is more complicated, _both_ in the system and in the applications which use it, it's no surprise that synchronous was the pick. The reasons why synchronous is simpler in applications have a nice illustration in operating systems, which inevitably support both blocking (i.e. implied process switching) and non-blocking 'operation initiation' and 'operation completed notification' mechanisms. (The 'timeout/callout' mechanism is Unix is an example of the latter, albeit specialized to timers.) Prior to the Master Control Program in the Burroughs B000 (there may be older examples, but I don't know of them - I would be more than pleased to be informed of any such, if there are), the technique of having a per-process _kernel_ stack, and on a process block (and implied switch), switching stacks, was not used. This idea was picked up for Jerry Saltzer's PhD thesis, used in Multics, and then copied by almost every other OS since (including Unix). The advantage is fairly obvious: if one is deep in some call stack, one can just wait there until the thing one needs is done, and then resume without having to work one's way back to that spot - which will inevitably be complicated (perhaps more in the need to _return_ through all the places that called down - although the code to handle a 'not yet' return through all those places, after the initial call down, will not be inconsiderable either). Exactly the same reasoning applies to blocking I/O; one can sit where one is, waiting for the I/O to be done, without having to work one's way back there later. (Examples are legion, e.g. in recursive descent parsers - and can make the code _much_ simpler.) It's only when one _can't_ wait for the I/O to complete (e.g. for a packet to arrive - although others have mentioned other examples in this thread, such as 'having other stuff to do in the meanwhile') than having only blocking I/O becomes a problem... In cases where blocking would be better, one can always build a 'blocking' I/O subsystem on top of asynchronous I/O primitives. However, in a _tiny_ system (remember my -11/40 which ran Unix on a system with _48KB_ of main memory _total_- i.e. OS and application together had to be less than 48KB - no virtual memory on that machine :-), building blocking I/O on top of asynchonous I/O, for those very few cases which need it, may not be the best use of very limited space - although I agree that it's the way to go, overall. Noel From pnr at planet.nl Sun Jun 7 00:55:50 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Sat, 6 Jun 2020 16:55:50 +0200 Subject: [TUHS] My BSDcan talk Message-ID: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl> > It's another similar to the last two. I've uploaded a version to youtube until the conference has theirs ready. It's a private link, but should work for anybody that has it. Now that I've given my talk it's cool to share more widely. > The link at the end is wrong. https://github.com/bsdimp/bsdcan2020-demos is the proper link. > Please let me know what you think. Watched it & liked it a lot! I have one nit-pick in the section on early networking: BBN's VAX TCP did not allow the ‘/dev/net/host’ syntax. That particular semantic comes from UoI’s NCP Unix, where the 8-bit host number was encoded in the minor number of character special file ‘host’ - but it did not carry through to the BBN code. Other systems used something similar. The Chaos network code made namei() break when it recognised the Chaos driver and left the remainder of the path for the driver to fetch & parse. I’m also being told that Greg Chesson experimented with using the given name of a Datakit channel device as the connection string for the switch, but that this approach was abandoned early on. In my view, exposing the host names through integration in the Unix file name space makes a lot of conceptual sense, but it unfortunately falls down on the practicalities, with the host name set being hard to enumerate (it is large, distributed and not stable - even back then). A question mark is hard pin-pointing the start of Unix networking to V4 / 1974. Yes, that is the earliest evidence we currently have. However, Sandy Fraser says that Spider came into operation in 1972 and it must have connected to something. Maybe that something was a lab-bench test setup, but it could have been a computer - maybe even one running Unix. There is another candidate for earliest Unix networking as well. The tech memo’s from Heinz Lycklama include one on the Glance terminal. That memo includes a section on the network used, referencing a 1973 report by D.R. Weller, "A High-Speed I/O Loop Communication System for the DEC PDP-11 Computer”. That computer appears to be an 11/45 running Unix and the loop is not Spider (nor the Pierce loop discussed in 1970/71 BSTJ). I have an off-list question outstanding to better understand this. From clemc at ccc.com Sun Jun 7 02:01:08 2020 From: clemc at ccc.com (Clem Cole) Date: Sat, 6 Jun 2020 12:01:08 -0400 Subject: [TUHS] My BSDcan talk In-Reply-To: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl> References: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl> Message-ID: On Sat, Jun 6, 2020 at 10:56 AM Paul Ruizendaal wrote: > The Chaos network code made namei() break when it recognised the Chaos > driver and left the remainder of the path for the driver to fetch & parse. > Yep - cute hack -- Domain (Aegis) did the same trick (same code author most likely). The biggest advantage is that open(2) can be used unchanged and 'parameters' are passed in ASCII to the lower code. You mentioned that BBN Vax code did not use it. I've forgotten than how parameters like IP addresses were passed down? I had thought that Rob had is also in at least one version of the BBN code, as I remember talking to him about it at one point; but I have frankly forgotten. I'm pretty sure that wnj thought it was an ugly solution; which is why UCB did not try it. We played with this solution also in the early Masscomp networking code (I had seen it in the Chaos driver from MIT); but by then the UCB code was coming out and user code was using sockets, so we just kept with that interface for code reuse reasons. Clem -------------- next part -------------- An HTML attachment was scrubbed... URL: From erc at pobox.com Sun Jun 7 06:49:36 2020 From: erc at pobox.com (Ed Carp) Date: Sat, 6 Jun 2020 15:49:36 -0500 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> Message-ID: On 5/27/20, Ronald Natalie wrote: > The large areas of undefined and unspecified behavior has always been an > issue in C. It was somewhat acceptable when you were using it as a direct > replacement for assembler, but Java and many of other follow-ons endevaored to be more > portable/rigourous. Of course, you can write crap code in any language. "It's not a bug, it's a feature" C was written when the programmer had to be more rigorous instead of just letting things slide and having the language do their thinking for them. I remember being laughed at for using static arrays instead of malloc() and friends, until people found out that safety-critical systems were written the same way. I have C code that was written 35 years ago that's still in production. Back then, you had to be careful, and you actually had to think about what you were writing. We've gotten soft and lazy, and now we're paying for it. From thomas.paulsen at firemail.de Sun Jun 7 07:08:43 2020 From: thomas.paulsen at firemail.de (Thomas Paulsen) Date: Sat, 06 Jun 2020 23:08:43 +0200 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> Message-ID: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> 'C was written when the programmer had to be more rigorous instead of just letting things slide and having the language do their thinking for them. ' I fully subscribe to that. Today the company owners have to pay a lot for programmers having the language do their thinking for them. The memory hunger of the soa java business services of the company I worked prior to retirement, is sheer endless. Arnold once told that there is more demand for C developers in Israel. I envy you From lm at mcvoy.com Sun Jun 7 07:13:28 2020 From: lm at mcvoy.com (Larry McVoy) Date: Sat, 6 Jun 2020 14:13:28 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> References: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> Message-ID: <20200606211328.GD1635@mcvoy.com> I did one stint at a Java shop, Charles Schwab's web group. No talent, no architecture, no vision. Lots of politics and back stabbing and claiming credit for other people's work. Totally toxic, hands down the worst job I've ever had. I lasted less than 6 months and am surprised I made it that far. On Sat, Jun 06, 2020 at 11:08:43PM +0200, Thomas Paulsen wrote: > 'C was written when the programmer had to be more rigorous instead of > just letting things slide and having the language do their thinking > for them. ' > I fully subscribe to that. > Today the company owners have to pay a lot for programmers having the language do their thinking > for them. The memory hunger of the soa java business services of the company I worked prior to > retirement, is sheer endless. Arnold once told that there is more demand for C developers > in Israel. I envy you > -- --- Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm From doug at cs.dartmouth.edu Sun Jun 7 07:49:23 2020 From: doug at cs.dartmouth.edu (Doug McIlroy) Date: Sat, 06 Jun 2020 17:49:23 -0400 Subject: [TUHS] History of popularity of C Message-ID: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> > Steve Johnson's position paper on optimising compilers may amuse you: > https://dl.acm.org/doi/abs/10.1145/567532.567542 Indeed. This passage struck a particular chord: "I contend that the class of applications that depend on, for example, loop optimization and dead code elimination for their efficient solution is of modest size, growing smaller, and often very susceptible to expression in applicative languages where the optimization is built into the individual applicative operators." I don't know whether I saw that note at the time, but since then I've come to believe, particularly in regard to C, that one case of dead-code elmination should be guaranteed. That case is if(0), where 0 is the value of a constant expression. This guarantee would take the place of many--possibly even most--ifdefs. Every ifdef is an ugly intrusion and a pain to read. Syntactically it occurs at top level completely out of sync with the indentation and flow of text. Conversion to if would be a big win. Doug From imp at bsdimp.com Sun Jun 7 07:55:07 2020 From: imp at bsdimp.com (Warner Losh) Date: Sat, 6 Jun 2020 15:55:07 -0600 Subject: [TUHS] History of popularity of C In-Reply-To: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> Message-ID: On Sat, Jun 6, 2020 at 3:50 PM Doug McIlroy wrote: > > Steve Johnson's position paper on optimising compilers may amuse you: > > https://dl.acm.org/doi/abs/10.1145/567532.567542 > > Indeed. This passage struck a particular chord: > > "I contend that the class of applications that depend on, for example, loop > optimization and dead code elimination for their efficient solution is of > modest size, growing smaller, and often very susceptible to expression in > applicative languages where the optimization is built into the individual > applicative operators." > > I don't know whether I saw that note at the time, but since then I've > come to believe, particularly in regard to C, that one case of dead-code > elmination should be guaranteed. That case is if(0), where 0 is the > value of a constant expression. > > This guarantee would take the place of many--possibly even > most--ifdefs. Every ifdef is an ugly intrusion and a pain to read. > Syntactically it occurs at top level completely out of sync with the > indentation and flow of text. Conversion to if would be a big win. > I'd love something like this to work, but the semantic interpretation would need to also somehow be omitted, otherwise how do you replace #ifdef AIX ioct(fd, AIX_SPECIFIC_IOCTL, ...) #endif on a HUPX system that doesn't define AIX_SPECIFIC_IOCTL... I remember hearing that BLISS could cope because it deferred the semantic interpretation of the identifiers until after a round of dead code elimination so it didn't need a pre-processor... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From erc at pobox.com Sun Jun 7 08:27:19 2020 From: erc at pobox.com (Ed Carp) Date: Sat, 6 Jun 2020 17:27:19 -0500 Subject: [TUHS] History of popularity of C In-Reply-To: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> Message-ID: "Arnold once told that there is more demand for C developers in Israel. I envy you" Maybe I ought to move to Israel. Sounds like they have more common sense there.

Virus-free. www.avast.com
On 6/6/20, Thomas Paulsen wrote: > 'C was written when the programmer had to be more rigorous instead of > just letting things slide and having the language do their thinking > for them. ' > I fully subscribe to that. > Today the company owners have to pay a lot for programmers having the > language do their thinking > for them. The memory hunger of the soa java business services of the company > I worked prior to > retirement, is sheer endless. Arnold once told that there is more demand for > C developers > in Israel. I envy you > > > From coppero1237 at gmail.com Sun Jun 7 09:14:55 2020 From: coppero1237 at gmail.com (Tyler Adams) Date: Sun, 7 Jun 2020 02:14:55 +0300 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> Message-ID: > "C was written when the programmer had to be more rigorous instead of > just letting things slide and having the language do their thinking > for them" True, but a wise man once said "let the computer do the dirty work". Tyler On Sun, Jun 7, 2020 at 1:28 AM Ed Carp wrote: > "Arnold once told that there is more demand for C developers > in Israel. I envy you" > > Maybe I ought to move to Israel. Sounds like they have more common > sense there. > > On 6/6/20, Thomas Paulsen wrote: > > 'C was written when the programmer had to be more rigorous instead of > > just letting things slide and having the language do their thinking > > for them. ' > > I fully subscribe to that. > > Today the company owners have to pay a lot for programmers having the > > language do their thinking > > for them. The memory hunger of the soa java business services of the > company > > I worked prior to > > retirement, is sheer endless. Arnold once told that there is more demand > for > > C developers > > in Israel. I envy you > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bakul at iitbombay.org Sun Jun 7 09:31:42 2020 From: bakul at iitbombay.org (Bakul Shah) Date: Sat, 6 Jun 2020 16:31:42 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> Message-ID: <64F96FBA-2BA7-4282-8516-5BD72E94B652@iitbombay.org> On Jun 6, 2020, at 1:49 PM, Ed Carp wrote: > > On 5/27/20, Ronald Natalie wrote: > >> The large areas of undefined and unspecified behavior has always been an >> issue in C. It was somewhat acceptable when you were using it as a direct >> replacement for assembler, but Java and many of other follow-ons endevaored to be more >> portable/rigourous. Of course, you can write crap code in any language. > > "It's not a bug, it's a feature" A snippet of a recent comp.arch post by someone (the subject was C and safety): What you call "misfeatures", some other people call "features". If you expect people to take you and your opinions seriously, you'll get on better if you stop mocking other opinions. I've written several times why undefined behaviour lets me write better and safer code, as well as more efficient code. If you remain determinedly unconvinced, at least agree to disagree without sounding childish about it. From woods at robohack.ca Sun Jun 7 10:12:35 2020 From: woods at robohack.ca (Greg A. Woods) Date: Sat, 06 Jun 2020 17:12:35 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: <64F96FBA-2BA7-4282-8516-5BD72E94B652@iitbombay.org> References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <64F96FBA-2BA7-4282-8516-5BD72E94B652@iitbombay.org> Message-ID: At Sat, 6 Jun 2020 16:31:42 -0700, Bakul Shah wrote: Subject: Re: [TUHS] History of popularity of C > > On Jun 6, 2020, at 1:49 PM, Ed Carp wrote: > > > > On 5/27/20, Ronald Natalie wrote: > > > >> The large areas of undefined and unspecified behavior has always > >> been an issue in C. It was somewhat acceptable when you were using > >> it as a direct replacement for assembler, but Java and many of > >> other follow-ons endevaored to be more portable/rigourous. Of > >> course, you can write crap code in any language. > > > > "It's not a bug, it's a feature" > > A snippet of a recent comp.arch post by someone (the subject was C and > safety): > > What you call "misfeatures", some other people call "features". > If you expect people to take you and your opinions seriously, > you'll get on better if you stop mocking other opinions. I've > written several times why undefined behaviour lets me write > better and safer code, as well as more efficient code. If you > remain determinedly unconvinced, at least agree to disagree > without sounding childish about it. Heh. W.r.t. efficiency, well undefined behaviour does allow the compiler to turn their code, or anyone's else's code, into more "efficient" code if they happen to (accidentally or otherwise) trip over undefined behaviour. However I don't think it can be argued in any valid way that "undefined behaviour" can ever lead to "better and safer" code, in any way, or from any viewing angle, whatsoever. "Undefined behaviour" just means that the language definition is somehow adversely compromised in such a way that it is impossible to prevent the programmer from writing compilable and executable code that will always produce some well defined behaviour in all standards-compliant implementations. I.e. the language allows that there are ways to write syntactically correct code that cannot be guaranteed to do anything particular whatsoever in _all_ standards-compatible implementations. We can argue until the cows come home whether "undefined behaviour" is a "necessary" part of the language definition (e.g. to keep the language implementable, or backward-compatible, or whatever), but I don't see how any valid argument can ever be made for it being a "good" and "useful" thing from the perspective of a programmer using the language. Undefined behaviours are black holes for which the language standard offers no real guidance nor maps for safe passage other than the stern warning to avoid them as best as possible. Perhaps it is such scare-mongering that the author above justifies as their influence to write "better and safer" code, but that's no good argument for having such pits of despair in the language definition in the first place. If we were arguing theology then I would say the bible we call the "C Standard" is actually actively trying to trap its followers into committing sins. Luckily the real world of C is made of actual implementations, and they are free to either offer definitions for how various (ab)uses of the language will work, or to maintain the black holes of mystery that we must try to avoid, or even sometimes to give us the choice in how they will treat our code. As programmers we should try to choose which implementation(s) we use, and how we control _their_ behaviour, while at the same time still doing our best to avoid giving them the rope to hang us with. -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP Digital Signature URL: From andreww591 at gmail.com Sun Jun 7 15:15:42 2020 From: andreww591 at gmail.com (Andrew Warkentin) Date: Sat, 6 Jun 2020 23:15:42 -0600 Subject: [TUHS] My BSDcan talk In-Reply-To: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl> References: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl> Message-ID: On 6/6/20, Paul Ruizendaal wrote: > > In my view, exposing the host names through integration in the Unix file > name space makes a lot of conceptual sense, but it unfortunately falls down > on the practicalities, with the host name set being hard to enumerate (it is > large, distributed and not stable - even back then). > With a proper dynamic VFS architecture, there is no reason why a resolver with a filesystem API has to bother supporting enumeration at all. All it needs to be able to do is respond to open() and stat() calls, returning ENOENT when resolution fails. From arnold at skeeve.com Sun Jun 7 15:57:46 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Sat, 06 Jun 2020 23:57:46 -0600 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> Message-ID: <202006070557.0575vkwU011918@freefriends.org> Ed Carp wrote: > "Arnold once told that there is more demand for C developers > in Israel. I envy you" The market in Israel for software developers is VERY hot. Based entirely on the emails I get from Linked-In about jobs that may interest me, there's some C, but a lot more C++, both Windows and Linux. Also a lot of Python. > Maybe I ought to move to Israel. Moving here isn't a trivial decision, especially if you don't speak any Hebrew. Off-topic. Sorry. > Sounds like they have more common sense there. We do, but the strong influence of western (US) culture is eroding it, which is saddening and frustrating. This definitely off-topic. Arnold From akosela at andykosela.com Sun Jun 7 19:22:21 2020 From: akosela at andykosela.com (Andy Kosela) Date: Sun, 7 Jun 2020 11:22:21 +0200 Subject: [TUHS] History of popularity of C In-Reply-To: <202006070557.0575vkwU011918@freefriends.org> References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: On 6/7/20, arnold at skeeve.com wrote: > Ed Carp wrote: > >> "Arnold once told that there is more demand for C developers >> in Israel. I envy you" > > The market in Israel for software developers is VERY hot. > > Based entirely on the emails I get from Linked-In about jobs that may > interest me, there's some C, but a lot more C++, both Windows and Linux. > Also a lot of Python. Seriously, is anyone still doing any real development in C besides kernel programming and embedded world?? Maybe I was living under a rock, but I always had an impression that the industry moved to C++ in the late 90s and stayed with it ever since. The last bastion of C was open source Linux/*BSD programming but I remember the time when C was a truly universal programming language used for _everything_ including games (e.g. Doom). Maybe I just miss the 90s. --Andy From erc at pobox.com Sun Jun 7 19:39:28 2020 From: erc at pobox.com (Ed Carp) Date: Sun, 7 Jun 2020 04:39:28 -0500 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: On 6/7/20, Andy Kosela wrote: > Seriously, is anyone still doing any real development in C besides > kernel programming and embedded world?? Maybe I was living under a > rock, but I always had an impression that the industry moved to C++ in > the late 90s and stayed with it ever since. Absolutely. C++ isn't the panacea that it's made out to be, it's not "superior" to C++, it's just C with other useful stuff bolted on, but it didn't make C obsolete, not by a long shot. Some find the operator overloading and classes to be a lot more confusing than just function calls and such, without adding a lot to the language itself. Others, of course, feel differently. It's more of a religious discussion than anything, like debating the merits of vi vs. emacs - a pointless discussion, since you're not going to change anyone else's mind anyway. Lots of C still being written out there. My most recent project was writing a crypto library in C. Writing it was the hard part - validating it was something else entirely. One of the nicest things about C++ is that you can write your code entirely in C and the C++ compiler will compile it, no problem. From brantley at coraid.com Sun Jun 7 20:02:23 2020 From: brantley at coraid.com (Brantley Coile) Date: Sun, 7 Jun 2020 10:02:23 +0000 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: <7563F472-96B8-4CF8-9176-404FB9761508@coraid.com> This might register low on the useful information index, but I decided a few years back as VC Coraid was coming apart and New Coraid was being resurrected under the auspices of SouthSuite Inc., that I would have a C mono-culture and use the language for everything. Specifically no JavaScript and no Python. Our website is written in C. Webpages generated on the server side using C. Our evolving ERP is written in C. Our subscription system is written in C. Our test systems are written in it. All work quite well. The focus on C was for three main reasons. First, our products are infrastructure products and are meant to be simple, fast, and affordable. Forty-two years of experience plus C gives us the ability to squeeze all the performance possible from any hardware platform. We "see" the instructions our code will generate. C is a great choice for that. (As would have been Oberon, but that's another discussion.) Second, if instead of having a set of complex languages, each with its own adherents, using a single language removes all the distracting and divisive language wars having multiple complex languages create. Little language like AWK and the shell script are fine. It's the more complex ones that divide people. Lastly, a single, powerful, simple (on the other side of complexity) language that a single person can maintain is essential to our Software Atelier model of doing business. Like the workshops of the renaissance, we have to understand and work on all our own tools. I use Ken's C compiler under Plan 9. It weights in at a light, 20K lines of code. As I said, I'm not sure how useful this data point is for you. Over the last thirty years I carefully chose my foot falls through the software swamp to avoid getting sucked under by one of the quagmires of complexity. Brantley > On Jun 7, 2020, at 5:22 AM, Andy Kosela wrote: > > On 6/7/20, arnold at skeeve.com wrote: >> Ed Carp wrote: >> >>> "Arnold once told that there is more demand for C developers >>> in Israel. I envy you" >> >> The market in Israel for software developers is VERY hot. >> >> Based entirely on the emails I get from Linked-In about jobs that may >> interest me, there's some C, but a lot more C++, both Windows and Linux. >> Also a lot of Python. > > Seriously, is anyone still doing any real development in C besides > kernel programming and embedded world?? Maybe I was living under a > rock, but I always had an impression that the industry moved to C++ in > the late 90s and stayed with it ever since. > > The last bastion of C was open source Linux/*BSD programming but I > remember the time when C was a truly universal programming language > used for _everything_ including games (e.g. Doom). Maybe I just miss > the 90s. > > --Andy From emu at e-bbes.com Sun Jun 7 21:04:23 2020 From: emu at e-bbes.com (emanuel stiebler) Date: Sun, 7 Jun 2020 07:04:23 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> Message-ID: <0fec52c3-0f5d-7e20-6247-8ffe1de40ebc@e-bbes.com> On 2020-06-06 16:49, Ed Carp wrote: > C was written when the programmer had to be more rigorous instead of > just letting things slide and having the language do their thinking > for them. I remember being laughed at for using static arrays instead > of malloc() and friends, until people found out that safety-critical > systems were written the same way. Still avoiding malloc and friends in safety critical systems, for a good reason ... From thomas.paulsen at firemail.de Sun Jun 7 21:30:03 2020 From: thomas.paulsen at firemail.de (Thomas Paulsen) Date: Sun, 07 Jun 2020 13:30:03 +0200 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: <5d6f1074119107ee81ff119c7a9463f6@firemail.de> >Seriously, is anyone still doing any real development in C besides >kernel programming and embedded world?? Maybe I was living under a >rock, but I always had an impression that the industry moved to C++ in >the late 90s and stayed with it ever since. I repeat myself: C never was intended for writing business applications. It was made for OS development in the sense of a portable assembler replacement. It also has found itself as the main devel language for fundamental services like db engines etc. From thomas.paulsen at firemail.de Sun Jun 7 21:33:21 2020 From: thomas.paulsen at firemail.de (Thomas Paulsen) Date: Sun, 07 Jun 2020 13:33:21 +0200 Subject: [TUHS] History of popularity of C In-Reply-To: <0fec52c3-0f5d-7e20-6247-8ffe1de40ebc@e-bbes.com> References: <0fec52c3-0f5d-7e20-6247-8ffe1de40ebc@e-bbes.com> Message-ID: <065f8b450e1884b5f4ab97d64d7ab05c@firemail.de> >Still avoiding malloc and friends in safety critical systems, for a good reason ... all modern languages allowing dynamical memory allocation one way or another. From pnr at planet.nl Sun Jun 7 21:41:41 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Sun, 7 Jun 2020 13:41:41 +0200 Subject: [TUHS] My BSDcan talk In-Reply-To: References: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl> Message-ID: <465A2942-5740-4B74-AF7E-8E58B37DC499@planet.nl> > On 6 Jun 2020, at 18:01, Clem Cole wrote: > > You mentioned that BBN Vax code did not use it. I've forgotten than how parameters like IP addresses were passed down? I had thought that Rob had it also in at least one version of the BBN code, as I remember talking to him about it at one point; but I have frankly forgotten. In the surviving source code, BBN VAX TCP re-purposed the ‘mode’ parameter of open() to be a pointer to a block with connection data (“struct con”). Details are here: https://www.tuhs.org/cgi-bin/utree.pl?file=BBN-Vax-TCP/doc/net.5.P From pnr at planet.nl Sun Jun 7 22:16:20 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Sun, 7 Jun 2020 14:16:20 +0200 Subject: [TUHS] network name space (was: My BSDcon talk) Message-ID: <42C43756-9800-41AC-88FF-4186510F83D4@planet.nl> > On 6/6/20, Paul Ruizendaal > wrote: > > > > In my view, exposing the host names through integration in the Unix file > > name space makes a lot of conceptual sense, but it unfortunately falls down > > on the practicalities, with the host name set being hard to enumerate (it is > > large, distributed and not stable - even back then). > > > With a proper dynamic VFS architecture, there is no reason why a > resolver with a filesystem API has to bother supporting enumeration at > all. All it needs to be able to do is respond to open() and stat() > calls, returning ENOENT when resolution fails. That is an intriguing thought. In Research Unix terms it would be a virtual directory that was not readable or writable, but still explorable (i.e. only the x bit set). Maybe enumeration is only impractical for the networks that were designed to be ‘large’, such as Arpanet and Datakit. It would have been feasible for contemporary networks that were designed to be local only, such as Chaosnet or ARCnet. A half-way house would be to only enumerate the local network and leaving everything else merely explorable. That is conceptually very messy, though. From clemc at ccc.com Mon Jun 8 01:26:45 2020 From: clemc at ccc.com (Clem Cole) Date: Sun, 7 Jun 2020 11:26:45 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: On Sun, Jun 7, 2020 at 5:23 AM Andy Kosela wrote: > Seriously, is anyone still doing any real development in C besides > kernel programming and embedded world?? Maybe I was living under a > rock, but I always had an impression that the industry moved to C++ in > the late 90s and stayed with it ever since. > Hardly, in my 45+ years, I have seen way more C projects than C++. The C projects tended to last longer, have a more profound impact and many are still being developed. I've been at Intel for the last qtr of my career and without a doubt, C is the #1 language in use internally, with C++ probably #2. Most of our work is actually in user space, although obviously we do a great deal of low-level work. The Intel compilers are a mix of both languages. That said, as we move to more LLVM work (we actually have the largest number of paid LLVM developers ). For your amusement check out: https://www.archer.ac.uk/status/codes/ Archer is a large HPC site in the UK. Many supercomputer centers off similar stats, but I often point to Archer because it easy to get an understanding of what programming languages are used for codes that are actually used in day-to-day production. That said, I personally am the most excited about Go theses day, but I'm also thinking Rust looks pretty interesting, but my experience with both compared to C is extremely nominal. Neither language is used for anything in production in our world at this point. Clem -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Mon Jun 8 01:52:39 2020 From: lm at mcvoy.com (Larry McVoy) Date: Sun, 7 Jun 2020 08:52:39 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: <20200607155239.GL1635@mcvoy.com> On Sun, Jun 07, 2020 at 11:26:45AM -0400, Clem Cole wrote: > That said, I personally am the most excited about Go theses day, but I'm > also thinking Rust looks pretty interesting, but my experience with both > compared to C is extremely nominal. Neither language is used for anything > in production in our world at this point. If I had to move to a modern language it would be Go. I looked at Rust and barfed. But if I had the juice, I'd just evolve C. Make a C+ that isn't object oriented but does have some of the stuff that modern languages have. I'd pretty much take http://little-lang.org and evolve C to that. From bakul at iitbombay.org Mon Jun 8 03:26:48 2020 From: bakul at iitbombay.org (Bakul Shah) Date: Sun, 7 Jun 2020 10:26:48 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: <9FB6B5CB-A8A2-4EA7-8613-474C2B376FEC@iitbombay.org> On Jun 7, 2020, at 8:26 AM, Clem Cole wrote: > > That said, I personally am the most excited about Go theses day, but I'm also thinking Rust looks pretty interesting, but my experience with both compared to C is extremely nominal. Neither language is used for anything in production in our world at this point. People who use Rust seem to really like it but so far I have not done anything in it. I will take another look if I do any bare metal coding. In contrast I use Go for almost all my own coding now. I am not entirely happy with it but the ecosystem around it is great. And the really nice thing about it is that not only I can cross-compile programs but also, I believe it is the only compiled language (much more so than C) where a lot of stuff just works on on plan9. From bakul at iitbombay.org Mon Jun 8 03:35:29 2020 From: bakul at iitbombay.org (Bakul Shah) Date: Sun, 7 Jun 2020 10:35:29 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: On Jun 7, 2020, at 8:26 AM, Clem Cole wrote: > > That said, I personally am the most excited about Go theses day, but I'm also thinking Rust looks pretty interesting, but my experience with both compared to C is extremely nominal. Neither language is used for anything in production in our world at this point. People who use Rust seem to really like it but so far I have not done anything in it. I will take another look if I do any bare metal coding. In contrast I use Go for almost all my own coding now. I am not entirely happy with it but the ecosystem around it is great. And the really nice thing about it is that not only I can cross-compile programs but also, I believe it is the only compiled language (much more so than C) where a lot of stuff just works on on plan9. From cym224 at gmail.com Mon Jun 8 04:50:34 2020 From: cym224 at gmail.com (Nemo Nusquam) Date: Sun, 7 Jun 2020 14:50:34 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8a2e9b1b-8890-a783-5b53-c8480c070f2e@telegraphics.com.au> <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> Message-ID: <17d519b0-384c-fddd-9fe1-700bbe9821be@gmail.com> On 06/07/20 11:26, Clem Cole wrote (in part): > That said, I personally am the most excited about Go theses day, but > I'm also thinking Rust looks pretty interesting, but my experience > with both compared to C is extremely nominal. Neither language is > used for anything in production in our world at this point. They seem to be used in some worlds: https://blog.golang.org/10years and https://www.rust-lang.org/production N. From imp at bsdimp.com Mon Jun 8 06:03:46 2020 From: imp at bsdimp.com (Warner Losh) Date: Sun, 7 Jun 2020 14:03:46 -0600 Subject: [TUHS] My BSDcan talk In-Reply-To: <465A2942-5740-4B74-AF7E-8E58B37DC499@planet.nl> References: <9E4F4C8A-A2F6-4993-9144-30A4D3721CF1@planet.nl> <465A2942-5740-4B74-AF7E-8E58B37DC499@planet.nl> Message-ID: On Sun, Jun 7, 2020 at 5:42 AM Paul Ruizendaal wrote: > > > On 6 Jun 2020, at 18:01, Clem Cole wrote: > > > > You mentioned that BBN Vax code did not use it. I've forgotten than how > parameters like IP addresses were passed down? I had thought that Rob had > it also in at least one version of the BBN code, as I remember talking to > him about it at one point; but I have frankly forgotten. > > In the surviving source code, BBN VAX TCP re-purposed the ‘mode’ parameter > of open() to be a pointer to a block with connection data (“struct con”). > Details are here: > https://www.tuhs.org/cgi-bin/utree.pl?file=BBN-Vax-TCP/doc/net.5.P That's... clever? Gross? I don't even know how I'd want to react to this... which likely means "too clever for its own good" :) In any event, I've created a short, 2:00 correction to my talk. I'll have the conference organizers tack onto the end before they upload it to youtube. https://www.youtube.com/watch?v=9uTjyfO6MM8 If people care :) Thanks everybody for corrections... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From torek at torek.net Mon Jun 8 07:15:26 2020 From: torek at torek.net (Chris Torek) Date: Sun, 07 Jun 2020 14:15:26 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: Your message of "Sun, 07 Jun 2020 14:50:34 -0400." <17d519b0-384c-fddd-9fe1-700bbe9821be@gmail.com> Message-ID: <202006072115.057LFV6v089953@elf.torek.net> Yes, both Rust and Go are being used. Rust has some advantages: like C++, it can compile to very fast code that does not need a garbage collector but that provides type-safety. It also gives you thread-safety through clever compiler analysis of who "owns" any given variable or data. The ownership / lifetime-analysis / borrow-checker is quite complicated and takes a lot of getting-used-to. I have not written anything "real" in Rust and had not had time to really learn it (I was hoping to learn it for real and write code in it the last few years, but that never actually happened). Meanwhile, Go is actually a really nice language to use. It has a few quirks, but it gives you reasonably-fast-running code that (because there is a garbage collector) does not require nearly as much skull-sweat when figuring out who owns memory and how to make sure it gets released appropriately. Its built in channels and goroutine support makes multi-threaded cod, and using all the CPUs effectively, also much easier. You pay (sometimes noticeably) for the GC, but the price is not too bad in less time-critical situations. The GC has a few short stop-the-world points but since Go 1.6 or so, it's pretty smooth, unlike what I remember from 1980s Lisp systems. :-) (Note: I started with Go 1.11, so I don't have a lot of history, nor that much experience in it. But I do like it.) Both Go and Rust have build systems. In Rust, this is a separate front-end from the compiler proper (rustc): you run "cargo build", for instance. In Go, you run "go build", "go test", etc., to build and invoke unit tests and so on. Rust has generics (think C++ templates, except sane) and Go lacks them although there's a plan for them in Go 2.0. Rust has, or at least had, rather bad array support when I last looked at it: you could make an array out of any type, but with only up to 32 elements. Chris From crossd at gmail.com Mon Jun 8 08:16:36 2020 From: crossd at gmail.com (Dan Cross) Date: Sun, 7 Jun 2020 18:16:36 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: <202006072115.057LFV6v089953@elf.torek.net> References: <17d519b0-384c-fddd-9fe1-700bbe9821be@gmail.com> <202006072115.057LFV6v089953@elf.torek.net> Message-ID: On Sun, Jun 7, 2020 at 5:37 PM Chris Torek wrote: > Yes, both Rust and Go are being used. > Indeed. The languages complement each other quite nicely. Disclaimer: I've been programming in Rust for my day job for almost two years now and I sit in the office next to the Go tech lead and in close proximity a number of people working on Go. Rust has some advantages: like C++, it can compile to very fast > code that does not need a garbage collector but that provides > type-safety. It also gives you thread-safety through clever > compiler analysis of who "owns" any given variable or data. > Safe Rust code is data race free, but not free of race conditions, let alone thread safe in all ways. The ownership / lifetime-analysis / borrow-checker is quite > complicated and takes a lot of getting-used-to. Once you get used to it, it doesn't feel that complex but yes, it takes a while to get used to. When we switched our project to it (from C++, at my behest) our tech lead quipped that Rust has a "near vertical" learning curve. Suffice it to say we're all quite glad we switched but there was an adjustment period. Interestingly, we find it a selling point for attracting new engineers regardless of the challenges in coming up to speed. [snip] Rust has, or at least had, rather bad > array support when I last looked at it: you could make an array > out of any type, but with only up to 32 elements. I'm afraid this is incorrect. Rust arrays are indexed by a `usize`, which is basically whatever `size_t` would be in C. Rust arrays in general can be essentially arbitrarily large (up to limitations imposed by the target machine, of course). However, Rust does not support dependent types, most certainly not for arrays. In other words, an array's size is considered part of its type and so when specializing traits on arrays, one must do so explicitly for each supported array size. For practical reasons such implementations are often limited to a relatively small number of distinct sizes; 32 is a believable number. Perhaps that's what you're thinking of? For example, the `Vec` type was recently modified to implement the `From` trait on arrays, but only on arrays up to size 32. This means that one can essentially create a dynamic, growable vector seeded from an array, but such source arrays are limited in how large they can be. My current biggest frustrations are in assumptions made by the memory allocator, the magical nature of `box` (the allocation primitive), and the under-defined memory model. - Dan C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From torek at torek.net Mon Jun 8 08:56:45 2020 From: torek at torek.net (Chris Torek) Date: Sun, 07 Jun 2020 15:56:45 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: Your message of "Sun, 07 Jun 2020 18:16:36 -0400." Message-ID: <202006072256.057Muo1h090326@elf.torek.net> >Safe Rust code is data race free, but not free of race >conditions, let alone thread safe in all ways. Er, yes. I mainly wanted to contrast to Go, where you look for race conditions by building with a flag that enables runtime checking. This can only detect races that actually occur, and if there are paths that would still have races that didn't occur on your test run, well... >>[rust array limitations] >I'm afraid this is incorrect. Rust arrays are indexed by a >`usize`, which is basically whatever `size_t` would be in C. Rust >arrays in general can be essentially arbitrarily large (up to >limitations imposed by the target machine, of course). >However, Rust does not support dependent types, most certainly >not for arrays. In other words, an array's size is considered >part of its type and so when specializing traits on arrays, one >must do so explicitly for each supported array size. For >practical reasons such implementations are often limited to a >relatively small number of distinct sizes; 32 is a believable >number. Perhaps that's what you're thinking of? Ah, yes, that was it. (As I said, I never really had time to do anything "real" in Rust.) Go's slices are nice to use, but slices (and maps, for that matter) trip people up because they are headers that point to shared spaces. Chris From wkt at tuhs.org Mon Jun 8 09:14:34 2020 From: wkt at tuhs.org (Warren Toomey) Date: Mon, 8 Jun 2020 09:14:34 +1000 Subject: [TUHS] Comparative languages In-Reply-To: <202006072256.057Muo1h090326@elf.torek.net> References: <202006072256.057Muo1h090326@elf.torek.net> Message-ID: <20200607231434.GA13571@minnie.tuhs.org> All, as the discussion on C has moved on to other languages, we might move the followups to COFF. Thanks, Warren From bramwyllie at gmail.com Mon Jun 8 10:24:38 2020 From: bramwyllie at gmail.com (Bram Wyllie) Date: Sun, 7 Jun 2020 21:24:38 -0300 Subject: [TUHS] History of popularity of C In-Reply-To: <202006072256.057Muo1h090326@elf.torek.net> References: <202006072256.057Muo1h090326@elf.torek.net> Message-ID: Dependent types aren't needed for sum types though, which is what you'd normally use for an array that carries its size, correct? On Sun, Jun 7, 2020 at 7:57 PM Chris Torek wrote: > >Safe Rust code is data race free, but not free of race > >conditions, let alone thread safe in all ways. > > Er, yes. I mainly wanted to contrast to Go, where you look for > race conditions by building with a flag that enables runtime > checking. This can only detect races that actually occur, > and if there are paths that would still have races that didn't > occur on your test run, well... > > >>[rust array limitations] > > >I'm afraid this is incorrect. Rust arrays are indexed by a > >`usize`, which is basically whatever `size_t` would be in C. Rust > >arrays in general can be essentially arbitrarily large (up to > >limitations imposed by the target machine, of course). > > >However, Rust does not support dependent types, most certainly > >not for arrays. In other words, an array's size is considered > >part of its type and so when specializing traits on arrays, one > >must do so explicitly for each supported array size. For > >practical reasons such implementations are often limited to a > >relatively small number of distinct sizes; 32 is a believable > >number. Perhaps that's what you're thinking of? > > Ah, yes, that was it. (As I said, I never really had time to > do anything "real" in Rust.) > > Go's slices are nice to use, but slices (and maps, for that matter) > trip people up because they are headers that point to shared > spaces. > > Chris > -------------- next part -------------- An HTML attachment was scrubbed... URL: From athornton at gmail.com Mon Jun 8 11:02:18 2020 From: athornton at gmail.com (Adam Thornton) Date: Sun, 7 Jun 2020 18:02:18 -0700 Subject: [TUHS] History of popularity of C In-Reply-To: <20200607155239.GL1635@mcvoy.com> References: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> <20200607155239.GL1635@mcvoy.com> Message-ID: > On Jun 7, 2020, at 8:52 AM, Larry McVoy wrote: > > On Sun, Jun 07, 2020 at 11:26:45AM -0400, Clem Cole wrote: >> That said, I personally am the most excited about Go theses day, but I'm >> also thinking Rust looks pretty interesting, but my experience with both >> compared to C is extremely nominal. Neither language is used for anything >> in production in our world at this point. > > If I had to move to a modern language it would be Go. I looked at Rust > and barfed. Several years ago, this was a job talk I gave, based on my experience at the time developing a pretty nifty system that never found traction. The new job (I got it!) doesn’t use Go, so I’ve grudgingly gone back to Python. But I stand by most of what I wrote (although I am sure parts of it are outdated and wrong now). The tl;dr is the title of the talk: https://athornton.github.io/go-it-mostly-doesnt-suck I make the claim that Go *is* pretty much C with 35 years of lessons learned about what did and didn’t work in C, and 35 years of machine time getting cheaper and programmer time getting more expensive. Adam From lars at nocrew.org Mon Jun 8 15:48:03 2020 From: lars at nocrew.org (Lars Brinkhoff) Date: Mon, 08 Jun 2020 05:48:03 +0000 Subject: [TUHS] History of popularity of C In-Reply-To: <202006072115.057LFV6v089953@elf.torek.net> (Chris Torek's message of "Sun, 07 Jun 2020 14:15:26 -0700") References: <202006072115.057LFV6v089953@elf.torek.net> Message-ID: <7w4krmw1kc.fsf@junk.nocrew.org> Chris Torek wrote: > You pay (sometimes noticeably) for the GC, but the price is not > too bad in less time-critical situations. The GC has a few short > stop-the-world points but since Go 1.6 or so, it's pretty smooth, > unlike what I remember from 1980s Lisp systems. :-) I'm guessing those 1980s Lisp systems would also be pretty smooth on 2020s hardware. From thomas.paulsen at firemail.de Mon Jun 8 18:04:41 2020 From: thomas.paulsen at firemail.de (Thomas Paulsen) Date: Mon, 08 Jun 2020 10:04:41 +0200 Subject: [TUHS] History of popularity of C In-Reply-To: References: <8b8d2ff3ed704c5631997387bf78e9b7@firemail.de> <202006070557.0575vkwU011918@freefriends.org> <20200607155239.GL1635@mcvoy.com> Message-ID: <7a76a39c7a0ceaa53a58a85fea3d6be2@firemail.de> >I make the claim that Go *is* pretty much C with 35 years of lessons learned >about what did and didn’t work in C, and 35 years of machine time getting >cheaper and programmer time getting more expensive. see Rob Pike 'Notes on Programming in C' freely available in the internet at: https://www.lysator.liu.se/c/pikestyle.html From clemc at ccc.com Mon Jun 8 23:10:37 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 8 Jun 2020 09:10:37 -0400 Subject: [TUHS] UNESCO's Software Heritage Foundation Message-ID: I'm seeding this URL to TUHS as one would expect them to be interested in the work from Warren and friends. FWIW: I tried to browse their archives and was not impressed (I couldn't find anything). https://www.softwareheritage.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dfawcus+lists-tuhs at employees.org Mon Jun 8 23:56:19 2020 From: dfawcus+lists-tuhs at employees.org (Derek Fawcus) Date: Mon, 8 Jun 2020 14:56:19 +0100 Subject: [TUHS] History of popularity of C In-Reply-To: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> Message-ID: <20200608135619.GA22471@clarinet.employees.org> On Sat, Jun 06, 2020 at 05:49:23PM -0400, Doug McIlroy wrote: > Syntactically it occurs at top level completely out of sync with the > indentation and flow of text. Conversion to if would be a big win. Well, doesn't that rather depend upon how one indents it? As I recall, that freedom came with ANSI C. But yeah, I'd prefer the if(0) type of elimination. DF $ cc yes-no.c; ./a.out; cc -DYES yes-no.c; ./a.out; cat yes-no.c No Yes #include int main(void) { #ifdef YES puts("Yes"); #else puts("No"); #endif return 0; } From aek at bitsavers.org Tue Jun 9 00:44:55 2020 From: aek at bitsavers.org (Al Kossow) Date: Mon, 8 Jun 2020 07:44:55 -0700 Subject: [TUHS] UNESCO's Software Heritage Foundation In-Reply-To: References: Message-ID: On 6/8/20 6:10 AM, Clem Cole wrote: > I'm seeding this URL to TUHS as one would expect them to be interested in the work from Warren and friends. FWIW: I tried to browse their > archives and was not impressed (I couldn't find anything). > > https://www.softwareheritage.org/ Don't diss them too badly. They are good folks, especially with how they are dealing with archiving and deduping almost-identical source trees. They stop by CHM every few years. From clemc at ccc.com Tue Jun 9 01:14:13 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 8 Jun 2020 11:14:13 -0400 Subject: [TUHS] UNESCO's Software Heritage Foundation In-Reply-To: References: Message-ID: Al, I'm not diss them or their efforts, just I wanted to know who they were. This is the first I had heard of them, I am, however, dissing the web site which I could not figure out how to use. On Mon, Jun 8, 2020 at 10:51 AM Al Kossow wrote: > On 6/8/20 6:10 AM, Clem Cole wrote: > > I'm seeding this URL to TUHS as one would expect them to be interested > in the work from Warren and friends. FWIW: I tried to browse their > > archives and was not impressed (I couldn't find anything). > > > > https://www.softwareheritage.org/ > > Don't diss them too badly. They are good folks, > especially with how they are dealing with archiving and deduping > almost-identical > source trees. > > They stop by CHM every few years. > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at kdbarto.org Tue Jun 9 00:58:30 2020 From: david at kdbarto.org (David Barto) Date: Mon, 8 Jun 2020 07:58:30 -0700 Subject: [TUHS] UNESCO's Software Heritage Foundation In-Reply-To: References: Message-ID: <2A179FD5-E1DA-459A-8B06-8025F5238ACF@kdbarto.org> As a quick look, I entered ‘unix’ to the search bar. 8^) In the list was a UNIX_V6PP. A quick browse it looked like a (not very good IMO) c++ rewrite of the v6 kernel for the x86 processor. David https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://github.com/lonway/UNIX_V6PP&path=UNIX_V6%2B%2B > On Jun 8, 2020, at 7:44 AM, Al Kossow wrote: > > On 6/8/20 6:10 AM, Clem Cole wrote: >> I'm seeding this URL to TUHS as one would expect them to be interested in the work from Warren and friends. FWIW: I tried to browse their archives and was not impressed (I couldn't find anything). >> https://www.softwareheritage.org/ > > Don't diss them too badly. They are good folks, > especially with how they are dealing with archiving and deduping almost-identical > source trees. > > They stop by CHM every few years. > > > > From rich.salz at gmail.com Tue Jun 9 01:20:58 2020 From: rich.salz at gmail.com (Richard Salz) Date: Mon, 8 Jun 2020 11:20:58 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: <20200608135619.GA22471@clarinet.employees.org> References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> <20200608135619.GA22471@clarinet.employees.org> Message-ID: > But yeah, I'd prefer the if(0) type of elimination. > Some parts of OpenSSL do this, and it seems to work well across a pretty wide variety of platforms/toolchains. A really important benefit is that you get code syntax-checked as opposed to unused #ifdef branches rotting. A minor benefit is you don't have ifdef spaghetti around local variables, to avoid "xxx is not used" warnings. -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Tue Jun 9 01:30:50 2020 From: crossd at gmail.com (Dan Cross) Date: Mon, 8 Jun 2020 11:30:50 -0400 Subject: [TUHS] History of popularity of C In-Reply-To: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> Message-ID: On Sat, Jun 6, 2020 at 5:50 PM Doug McIlroy wrote: > > Steve Johnson's position paper on optimising compilers may amuse you: > > https://dl.acm.org/doi/abs/10.1145/567532.567542 > > Indeed. This passage struck a particular chord: > > "I contend that the class of applications that depend on, for example, loop > optimization and dead code elimination for their efficient solution is of > modest size, growing smaller, and often very susceptible to expression in > applicative languages where the optimization is built into the individual > applicative operators." > > I don't know whether I saw that note at the time, but since then I've > come to believe, particularly in regard to C, that one case of dead-code > elmination should be guaranteed. That case is if(0), where 0 is the > value of a constant expression. > > This guarantee would take the place of many--possibly even > most--ifdefs. Every ifdef is an ugly intrusion and a pain to read. > Syntactically it occurs at top level completely out of sync with the > indentation and flow of text. Conversion to if would be a big win. > This came up on this list back in 2017 and I posted something about this at the time: https://minnie.tuhs.org/pipermail/tuhs/2017-January/009325.html If the language had been extended to support a "compile-type conditional" syntax ("if #(LINUX)", for example) it would have obviated the need for most preprocessor conditionals. As long as, syntactically, the compiler could tokenize a block in absence of e.g. type definitions, dead-code elimination could do the job. - Dan C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lars at nocrew.org Tue Jun 9 02:06:00 2020 From: lars at nocrew.org (Lars Brinkhoff) Date: Mon, 08 Jun 2020 16:06:00 +0000 Subject: [TUHS] UNESCO's Software Heritage Foundation In-Reply-To: (Clem Cole's message of "Mon, 8 Jun 2020 09:10:37 -0400") References: Message-ID: <7wpna9v8yf.fsf@junk.nocrew.org> Clem Cole wrote: > I'm seeding this URL to TUHS as one would expect them to be interested > in the work from Warren and friends. FWIW: I tried to browse their > archives and was not impressed (I couldn't find anything). They seem to have scraped GitHub, so they backed up everything I put there. Thanks guys. From aek at bitsavers.org Tue Jun 9 02:20:21 2020 From: aek at bitsavers.org (Al Kossow) Date: Mon, 8 Jun 2020 09:20:21 -0700 Subject: [TUHS] UNESCO's Software Heritage Foundation In-Reply-To: <7wpna9v8yf.fsf@junk.nocrew.org> References: <7wpna9v8yf.fsf@junk.nocrew.org> Message-ID: On 6/8/20 9:06 AM, Lars Brinkhoff wrote: > Clem Cole wrote: >> I'm seeding this URL to TUHS as one would expect them to be interested >> in the work from Warren and friends. FWIW: I tried to browse their >> archives and was not impressed (I couldn't find anything). > > They seem to have scraped GitHub, so they backed up everything I put > there. Thanks guys. > They are trying to archive and de-dup EVERYTHING that is open-source. Unix is more difficult, because of the encumbrances. https://www.softwareheritage.org/2019/06/06/announcing-the-software-heritage-graph-dataset/ From dot at dotat.at Tue Jun 9 02:32:04 2020 From: dot at dotat.at (Tony Finch) Date: Mon, 8 Jun 2020 17:32:04 +0100 Subject: [TUHS] History of popularity of C In-Reply-To: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> References: <202006062149.056LnNsb074899@tahoe.cs.Dartmouth.EDU> Message-ID: Doug McIlroy wrote: > Every ifdef is an ugly intrusion and a pain to read. I maintain `unifdef`, which is not a complicated program to port but it does require some platform-specific stuff on Windows (partly because of the poor standards conformance of the Windows C library). There's a fun rule that `unifdef` must contain no #ifdef (or #if etc.) so the portability stuff is handled by the build scripts adjusting the include path and list of object files. Not a very scalable solution :-) Tony. -- f.anthony.n.finch http://dotat.at/ Biscay: North or northwest 3 or 4, occasionally 5 at first. Slight or moderate, becoming slight later. Showers, thundery in east. Good, occasionally moderate. From cym224 at gmail.com Tue Jun 9 04:21:18 2020 From: cym224 at gmail.com (Nemo Nusquam) Date: Mon, 8 Jun 2020 14:21:18 -0400 Subject: [TUHS] UNESCO's Software Heritage Foundation In-Reply-To: References: Message-ID: On 06/08/20 11:14, Clem Cole wrote: > Al, I'm not diss them or their efforts, just I wanted to know who they > were. > This is the first I had heard of them, I am, however, dissing the web > site which I could not figure out how to use. I have surfed there over the years but I have always found their UI nothing short of terrible. They have archived over 126 million projects but my search queries return little and their browsing i/f seems non-existant. N. From paul at rileyriot.com Thu Jun 11 21:38:04 2020 From: paul at rileyriot.com (Paul Riley) Date: Thu, 11 Jun 2020 19:38:04 +0800 Subject: [TUHS] Unix V6: Assembler Listings Message-ID: Team, I am now writing code in assembly for the PDP-11. I remember reading somewhere that the output from "AS" (my caps) is a bit meagre. I can't find an option to produce a text listing. Is it possible from AS, using command options (I can't see one) or perhaps from "LD"? Paul *Paul Riley* -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Thu Jun 11 22:55:06 2020 From: clemc at ccc.com (Clem Cole) Date: Thu, 11 Jun 2020 08:55:06 -0400 Subject: [TUHS] Unix V6: Assembler Listings In-Reply-To: References: Message-ID: If you want something like this, there is a DEC Macro-11 compatible assembler in C originally for V6 kicking around. IIRC it was first on the Harvard tape in about 1977-78. Last winter there was a large discussion about it and I believe a fairly current version that can be compiled with modern compilers can be found here: https://github.com/Rhialto/macro11 On Thu, Jun 11, 2020 at 7:39 AM Paul Riley wrote: > Team, > > I am now writing code in assembly for the PDP-11. I remember reading > somewhere that the output from "AS" (my caps) is a bit meagre. I can't find > an option to produce a text listing. Is it possible from AS, using command > options (I can't see one) or perhaps from "LD"? > > Paul > > *Paul Riley* > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Thu Jun 11 23:40:30 2020 From: clemc at ccc.com (Clem Cole) Date: Thu, 11 Jun 2020 09:40:30 -0400 Subject: [TUHS] Unix V6: Assembler Listings In-Reply-To: <9CCCBCA1-DD64-49EA-AD3E-8E2B9C548B99@icloud.com> References: <9CCCBCA1-DD64-49EA-AD3E-8E2B9C548B99@icloud.com> Message-ID: below... On Thu, Jun 11, 2020 at 9:04 AM Paul Riley wrote: > Clem, > > Thanks for that. So this would compile on modern machines to a cross > compiler for V6 also running on a modern machine? I note you say macro11, > so not a Unix “as” style syntax, is that right? > Yes - the AT&T syntax was much simpler/less sugar than the DEC assembler. But the differences are pretty easy to see. IIRC that assembler generates DEC style linker objects and there is a companion linker that can create DEC binary objects (*i.e.* 'obj' files) as well as traditional UNIX a.out format. The entire tool suite was created originally to move code from RT-11 to UNIX at Harvard and passed around the nascent USENIX community. IIRC that version was forked from a BSD 2.x/NetBSD source repository and folks were adding some fields/features in the DEC obj format that RSX supported that RT-11 did not. Go hunting and see what you find. My memory was that with the BSD 2.x project, somebody added a DEC obj to UNIX binary (a.out) converter tool, so that you could use ld(1) instead of using the DEC style linker that had been included in the original. It has been >>years<< since I was really familiar with any of this stuff. A question about it came up last fall/winter on the simh mailing listing, which is where I found the the URL. FWIW: I offered the modern port, assuming you might want to run some of it as a cross-systems on a newer OS with a modern compiler. But if you are content running this on V6, then you might just want to go back to the original. As I said, my memory is that's in the original USENIX Harvard tape. All that should be Warner's archives if not other places on the Internet. Just remember that a big problem with the original code is that it will be written in pre-'White Book' C (that many of us learned years ago - not even ANSI of Second edition - this used Lesk's portable C library etc.). It sometimes looks a little strange to modern eyes. Also if you go looking, IIRC, someone at Harvard ported the DEC Macro RT-11 library to UNIX v6. In the late 1970s, I remember tjk, Danny Klein, Tron McConnell and I, plus some of the folks over in the bio-med group (whose names I have forgotten) had to a number assembler codes that had been written for the earlier RT-11 systems to Unix for one of the projects we had. Some of it got re-written in C, but I do remember we managed to use the Harvard assembler somehow for parts of it. If my memory is correct, early VMS and messing with BLISS compatibility could have been mixed up in the project somehow, but I've long forgotten the details of what we were doing at the time. Have fun. -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Fri Jun 12 00:56:22 2020 From: imp at bsdimp.com (Warner Losh) Date: Thu, 11 Jun 2020 08:56:22 -0600 Subject: [TUHS] Unix V6: Assembler Listings In-Reply-To: References: <9CCCBCA1-DD64-49EA-AD3E-8E2B9C548B99@icloud.com> Message-ID: On Thu, Jun 11, 2020 at 7:41 AM Clem Cole wrote: > below... > > On Thu, Jun 11, 2020 at 9:04 AM Paul Riley wrote: > >> Clem, >> >> Thanks for that. So this would compile on modern machines to a cross >> compiler for V6 also running on a modern machine? I note you say macro11, >> so not a Unix “as” style syntax, is that right? >> > Yes - the AT&T syntax was much simpler/less sugar than the DEC assembler. > But the differences are pretty easy to see. IIRC that assembler generates > DEC style linker objects and there is a companion linker that can create > DEC binary objects (*i.e.* 'obj' files) as well as traditional UNIX a.out > format. The entire tool suite was created originally to move code from > RT-11 to UNIX at Harvard and passed around the nascent USENIX community. > IIRC that version was forked from a BSD 2.x/NetBSD source repository and > folks were adding some fields/features in the DEC obj format that RSX > supported that RT-11 did not. > > Go hunting and see what you find. My memory was that with the BSD 2.x > project, somebody added a DEC obj to UNIX binary (a.out) converter tool, so > that you could use ld(1) instead of using the DEC style linker that had > been included in the original. > It's hard to trace. There's one on the UNSW distributions called macro-linkr with file dates from 76 in tuhs/Distributions/UNSW/7/source/macro-linkr (the last few elements in the path are from an extracted tarball). The 2.9 and newer has a m11 command that looks like a macro assembler. 2.79 (from 1980) has a number of lisp source files that are in m11 with Harvard copyrights on them. There's a file called 'macro' that is a binary and another called 'linkr'. The UBC tapes posted here a while ago had rt11 emulators of various flavors (from libraries to trap interception) and those were dated in the 76-80 time frame. These dates seem to line up with the lisp that's on the 77 usenix tape we have, but I've delved no further there since there's no sources. There's also a macro/linker from stanford on the 81 tape... But it says it's from Harvard: >> This directory contains the sources for a version of MACRO assembly language >> for the PDP-11. It produces Version 6 Unix executables that run under .> the compatibility package on the vax. They are essentially unmodified from >> those distributed on the 2nd Unix Users group distribution tape from Harvard. >> The only change was to make dates on listings work beyond 1979. NetBSD, as cool as it is, is about a decade and a half too recent :) The 2.9BSD and newer have the m11 sources, but they are written in m11. I've not seen if apout can be used to compile/assemble them on modern hardware. It has been >>years<< since I was really familiar with any of this stuff. > A question about it came up last fall/winter on the simh mailing listing, > which is where I found the the URL. > > FWIW: I offered the modern port, assuming you might want to run some of it > as a cross-systems on a newer OS with a modern compiler. But if you are > content running this on V6, then you might just want to go back to the > original. As I said, my memory is that's in the original USENIX Harvard > tape. All that should be Warner's archives if not other places on the > Internet. > Yea, that's the stuff I talked about in my talks... The files I think are somewhere in tuhs/Applications/Usenix_77, though I've not diving in too deeply there this morning. > Just remember that a big problem with the original code is that it will be > written in pre-'White Book' C (that many of us learned years ago - not > even ANSI of Second edition - this used Lesk's portable C library etc.). > It sometimes looks a little strange to modern eyes. Also if you go > looking, IIRC, someone at Harvard ported the DEC Macro RT-11 library to > UNIX v6. In the late 1970s, I remember tjk, Danny Klein, Tron McConnell > and I, plus some of the folks over in the bio-med group (whose names I have > forgotten) had to a number assembler codes that had been written for the > earlier RT-11 systems to Unix for one of the projects we had. Some of it > got re-written in C, but I do remember we managed to use the Harvard > assembler somehow for parts of it. If my memory is correct, early VMS and > messing with BLISS compatibility could have been mixed up in the project > somehow, but I've long forgotten the details of what we were doing at the > time. > The files are in the TUHS archive for the curious, but the github stuff looks like the best place to start... > Have fun. > Definitely. You could spend weeks exploring this area... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From pnr at planet.nl Fri Jun 12 01:26:36 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Thu, 11 Jun 2020 17:26:36 +0200 Subject: [TUHS] Unix V6: Assembler Listings Message-ID: <84BE6471-7004-466E-BE0C-2622DD2D41A3@planet.nl> > I am now writing code in assembly for the PDP-11. I remember reading > somewhere that the output from "AS" (my caps) is a bit meagre. I can't find > an option to produce a text listing. Is it possible from AS, using command > options (I can't see one) or perhaps from "LD"? > > Paul > > *Paul Riley* I had the same problem. As I was porting to a different mini I had to write a new assembler. As you have undoubtedly seen, early ‘as’ was written in assembler and not so easy to use as a base. Hence I used Richard’s Miller’s AS for the Interdata as a base (available on Tuhs): https://www.tuhs.org/cgi-bin/utree.pl?file=Interdata732/usr/source/as Later I discovered that the TUHS archive has source code for the original ‘as’ rewritten in C, a work by Roger Jaeger: https://minnie.tuhs.org/Archive/Distributions/USDL/Mini-Unix/ Maybe adding a listing module to this version of ‘as’ is another possible route. From imp at bsdimp.com Fri Jun 12 01:57:47 2020 From: imp at bsdimp.com (Warner Losh) Date: Thu, 11 Jun 2020 09:57:47 -0600 Subject: [TUHS] Unix V6: Assembler Listings In-Reply-To: <84BE6471-7004-466E-BE0C-2622DD2D41A3@planet.nl> References: <84BE6471-7004-466E-BE0C-2622DD2D41A3@planet.nl> Message-ID: On Thu, Jun 11, 2020 at 9:27 AM Paul Ruizendaal wrote: > > > I am now writing code in assembly for the PDP-11. I remember reading > > somewhere that the output from "AS" (my caps) is a bit meagre. I can't > find > > an option to produce a text listing. Is it possible from AS, using > command > > options (I can't see one) or perhaps from "LD"? > > > > Paul > > > > *Paul Riley* > > I had the same problem. As I was porting to a different mini I had to > write a new assembler. As you have undoubtedly seen, early ‘as’ was written > in assembler and not so easy to use as a base. Hence I used Richard’s > Miller’s AS for the Interdata as a base (available on Tuhs): > https://www.tuhs.org/cgi-bin/utree.pl?file=Interdata732/usr/source/as > > Later I discovered that the TUHS archive has source code for the original > ‘as’ rewritten in C, a work by Roger Jaeger: > https://minnie.tuhs.org/Archive/Distributions/USDL/Mini-Unix/ > > Maybe adding a listing module to this version of ‘as’ is another possible > route. There's also https://minnie.tuhs.org/cgi-bin/utree.pl?file=2.11BSD/src/new/m11/macxrf.c which looks to be decent or better K&R code from a quick look, but I don't know if it works, or if it would be easy to adopt to AT&T/Bell syntax which decorates things less, making it a little harder to infer semantic meanings. Also more digging shows the UNSW tapes / sources also are from Harvard. All roads lead to Harvard for this it seems :) Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave at horsfall.org Fri Jun 12 18:18:30 2020 From: dave at horsfall.org (Dave Horsfall) Date: Fri, 12 Jun 2020 18:18:30 +1000 (EST) Subject: [TUHS] non-blocking IO In-Reply-To: References: <20200601145801.GE22016@mcvoy.com> <20200604090436.GJ279@server.rulingia.com> <20200604165011.GC18437@mcvoy.com> Message-ID: On Fri, 5 Jun 2020, Dan Cross wrote: > Was it Rob who said that signals were really just for SIGKILL? Here, > signals would be gang-pressed into service as a general IPC mechanism. > In fairness, they've mutated that way, but they didn't start out that > way. While I obviously wasn't there, the strong impression I get is that > by the time people were seriously thinking about async IO in Unix, the > die had already been cast for better or worse. I will quite happily strangle anyone who uses signals for IPC. Why? I got bitten quite badly by that, if anyone here remembers BSD/OS... It seemed that "fdump" forked off several kiddies, and they chatted amongst themselves using signals. Anyway, let's just say that after a disk crash this was a poor time to discover that *some* of my backups were screwed in a weird way; for example, there was a 1/4" QIC[*] tape with files, but no inodes to put them into their corresponding home directories... I wrote something to extract whatever I could, excavate what I could, and then either rebuild or rely upon memory from there. Not fun. Did I mention that I will quite happily strangle anyone who uses signals for IPC? Signals mean "stop what you're doing now, do this instead, then hopefully go back to whatever you thought you were doing". [*] Don't ask me about those !@#$% QIC tapes, that chose to use whatever density they wished depending upon the phase of the moon etc. -- Dave, who lost a lot of valuable files that day From scj at yaccman.com Sun Jun 14 07:24:47 2020 From: scj at yaccman.com (scj at yaccman.com) Date: Sat, 13 Jun 2020 14:24:47 -0700 Subject: [TUHS] v7 K&R C [really lexers] In-Reply-To: References: <20200511005745.GL17035@mcvoy.com> <357EFE54-BD94-4C10-8C43-C6735BF7D317@via.net> <20200511202555.GU17035@mcvoy.com> <20200514173206.GJ20771@mcvoy.com> <20200516235308.icuQH%steffen@sdaoden.eu> <202005162359.04GNxalN3783011@darkstar.fourwinds.com> Message-ID: <50a4ee872b7be9770f96a1c1db10fb86@yaccman.com> I also gave up on lex for parsing fairly early. The problem was reserved words. These looked like identifiers, but the state machine to pick out a couple of dozen reserved words out of all identifiers was too big for the PDP-11. When I wrote spell, I ran into the same problem. I had some rules that wanted to convert plurals to singular forms that would be found in the dictionary. Writing a rule to recognize .*ies and convert the "ies" to "y" blew out the memory after only a handful of patterns. My solution was to pick up words and reverse them before passing them through lex, so I looked for the pattern "sei.*", converted it to "y" and then reversed the word again. As it turned out, I only owned spell for a few weeks because Doug and others grabbed it and ran with it... Steve --- On 2020-05-16 18:23, Warner Losh wrote: > On Sat, May 16, 2020, 6:05 PM Brantley Coile wrote: > >> "The asteroid to kill this dinosaur is still in orbit." >> >> --- Plan 9 lex man page >> >> I always hand craft my lexers and use yacc to parse. Most code on plan 9 does that as well. > > Wow! That is the most awesome thing I've seen in a while.... > > Warner > > Brantley > > On May 16, 2020, at 8:00 PM, Jon Steinhart wrote: > > Steffen Nurpmeso writes: > Tony Finch wrote in : |Larry McVoy wrote: |> |> It's got some perl goodness, regexps are part of the syntax, .... | |I got into Unix after perl and I've used it a lot. Back in the 1990s I saw |Henry Spencer's joke that perl was the Swiss Army Chainsaw of Unix, as a |riff on lex being its Swiss Army Knife. I came to appreciate lex |regrettably late: lex makes it remarkably easy to chew through a huge pile |of text and feed the pieces to some library code written in C. I've been |using re2c recently (http://re2c.org/), which is differently weird than |lex, though it still uses YY in all its variable names. It's remarkable |how much newer lexer/parser generators can't escape from the user |interface of lex/yacc. Another YY example: http://www.hwaci.com/sw/lemon/ P.S.: i really hate automated lexers. I never ever got used to use them. For learning i once tried to use flex/bison, but i failed really hard. I like that blood, sweat and tears thing, and using a lexer seems so shattered, all the pieces. And i find them really hard to read. If you can deal with them they are surely a relief, especially in rapidly moving syntax situations. But if i look at settled source code which uses it, for example usr.sbin/ospfd/parse.y, or usr.sbin/smtpd/parse.y, both of OpenBSD, then i feel lost and am happy that i do not need to maintain that code. --steffen > Wow, I've had the opposite experience. I find lex/yacc/flex/bison really > easy to use. The issue, which I believe was covered in the early docs, > is that some languages are not designed with regularity in mind which makes > for ugly code. But to be fair, that code is at least as ugly with hand-crafted > code. > > I believe that the original wisecrack was directed towards FORTRAN. My ancient > experience was that it was using lex/yacc for HSPICE was not going to work so I > had to hand-craft code for that. > > Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: From dds at aueb.gr Sun Jun 14 18:33:07 2020 From: dds at aueb.gr (Diomidis Spinellis) Date: Sun, 14 Jun 2020 11:33:07 +0300 Subject: [TUHS] Accessing the PDP-11/70 MMU registers and the kernel's u area Message-ID: From the 2.11 BSD sources I understand that the PDP-11/70 MMU address and data registers, KDSA and KDSD, start at 0172360 and 0172320 respectively [1]. Yet, when I read the register contents I don't get what I would expect to see: increasing by 0200 memory values for KDSA and the same constant value for KDSD [2]. I checked this by looking at /dev/mem. # od -o /dev/mem 0172360 | head -1 0172360 000002 000016 001403 012700 000400 000402 012700 000200 # od /dev/mem 0172320 | head -1 0172320 101016 005064 000026 005067 175456 016467 000006 175430 I get the same results when I examine the memory through SIMH: sim> examine 172360 172360: 000002 sim> examine 172362 172362: 000016 sim> examine 172364 172364: 001403 sim> examine 172320 172320: 101016 sim> examine 172322 172322: 005064 The MMU kernel instruction registers, KISA and KISD, contain similarly nonsensical values as do the registers located at a different memory location (077320, 0772360) indicated in another source [3]. What am I missing? My goal is to access from the console the kernel's u area. According to mem(4) and the symbols in /unix, this should be at address 0140000. Indeed, accessing it through /dev/kmem I get the expected results for e.g. u_comm and u_uid. However, I have been unable to find it in the machine's physical memory, hence my question regarding the MMU's operation. [1] https://github.com/RetroBSD/2.11BSD/blob/master/usr/sys/pdpstand/M.s#L346 [2] https://github.com/RetroBSD/2.11BSD/blob/master/usr/sys/pdpstand/M.s#L247 [3] https://gunkies.org/wiki/PDP-11_Memory_Management Diomidis From arnold at skeeve.com Sun Jun 14 18:47:32 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Sun, 14 Jun 2020 02:47:32 -0600 Subject: [TUHS] v7 K&R C [really lexers] In-Reply-To: <50a4ee872b7be9770f96a1c1db10fb86@yaccman.com> References: <20200511005745.GL17035@mcvoy.com> <357EFE54-BD94-4C10-8C43-C6735BF7D317@via.net> <20200511202555.GU17035@mcvoy.com> <20200514173206.GJ20771@mcvoy.com> <20200516235308.icuQH%steffen@sdaoden.eu> <202005162359.04GNxalN3783011@darkstar.fourwinds.com> <50a4ee872b7be9770f96a1c1db10fb86@yaccman.com> Message-ID: <202006140847.05E8lW4c004167@freefriends.org> scj at yaccman.com wrote: > As it turned out, I only owned spell for a few weeks because Doug and > others grabbed it and ran with it... > > Steve Who else besides Doug worked on spell? Do I understand correctly that you invented the original pipeline based spell: tr ... | # split words to one line tr ... | # lower case sort -u | # sort and uniq comm -12 - dict # find words not in dictionary I didn't know that you'd worked on a rewrite in C. Thanks, Arnold From rich.salz at gmail.com Sun Jun 14 22:52:18 2020 From: rich.salz at gmail.com (Richard Salz) Date: Sun, 14 Jun 2020 08:52:18 -0400 Subject: [TUHS] v7 K&R C [really lexers] In-Reply-To: <50a4ee872b7be9770f96a1c1db10fb86@yaccman.com> References: <20200511005745.GL17035@mcvoy.com> <357EFE54-BD94-4C10-8C43-C6735BF7D317@via.net> <20200511202555.GU17035@mcvoy.com> <20200514173206.GJ20771@mcvoy.com> <20200516235308.icuQH%steffen@sdaoden.eu> <202005162359.04GNxalN3783011@darkstar.fourwinds.com> <50a4ee872b7be9770f96a1c1db10fb86@yaccman.com> Message-ID: Does anyone have the Usenix paper (80's timeframe I think) about making lex go fast? It was by Vern Paxson or Van Jacobson IIRC. Maybe only an abstract was published. The talk ended with a chart showing some CPU times, and the modified lex was only slightly slower than cat. Maybe Vern since he contributed to what became flex. -------------- next part -------------- An HTML attachment was scrubbed... URL: From doug at cs.dartmouth.edu Sun Jun 14 23:55:05 2020 From: doug at cs.dartmouth.edu (Doug McIlroy) Date: Sun, 14 Jun 2020 09:55:05 -0400 Subject: [TUHS] v7 K&R C [really lexers] Message-ID: <202006141355.05EDt5YM067818@tahoe.cs.Dartmouth.EDU> Interesting. My "speak" program had a trivial lexer that recognized literal tokens, many of which were prefixes of others, by maximum-munch binary search in a list of 1600 entries. Entries gave token+translation+rewrite. The whole thing fit in 15K. Many years later I wrote a regex recognizer that special-cased alternations of lots of literals. I believe Gnu's regex.c does that, too. (My regex also supported conjunction and negation-- legitimate regular-language operations--implemented by continuation-passing to avoid huge finite-state machines.) We have here a case of imperfect communication in 1127. Had I been conscious of the lex-explosion problem, I might have thought of speak and put support for speak-like tables into lex. As it happened, I only used yacc/lex once, quite successfully, for a small domain-specific language. Doug Steve Johnson wrote: I also gave up on lex for parsing fairly early. The problem was reserved words. These looked like identifiers, but the state machine to pick out a couple of dozen reserved words out of all identifiers was too big for the PDP-11. When I wrote spell, I ran into the same problem. I had some rules that wanted to convert plurals to singular forms that would be found in the dictionary. Writing a rule to recognize .*ies and convert the "ies" to "y" blew out the memory after only a handful of patterns. My solution was to pick up words and reverse them before passing them through lex, so I looked for the pattern "sei.*", converted it to "y" and then reversed the word again. As it turned out, I only owned spell for a few weeks because Doug and others grabbed it and ran with it. From ralph at inputplus.co.uk Mon Jun 15 00:03:03 2020 From: ralph at inputplus.co.uk (Ralph Corderoy) Date: Sun, 14 Jun 2020 15:03:03 +0100 Subject: [TUHS] v7 K&R C [really lexers] In-Reply-To: References: <20200511005745.GL17035@mcvoy.com> <357EFE54-BD94-4C10-8C43-C6735BF7D317@via.net> <20200511202555.GU17035@mcvoy.com> <20200514173206.GJ20771@mcvoy.com> <20200516235308.icuQH%steffen@sdaoden.eu> <202005162359.04GNxalN3783011@darkstar.fourwinds.com> <50a4ee872b7be9770f96a1c1db10fb86@yaccman.com> Message-ID: <20200614140303.E5C2A221A1@orac.inputplus.co.uk> Hi Rich, > Does anyone have the Usenix paper (80's timeframe I think) about making lex > go fast? It was by Vern Paxson or Van Jacobson IIRC. Maybe only an > abstract was published. The talk ended with a chart showing some CPU > times, and the modified lex was only slightly slower than cat. Maybe Vern > since he contributed to what became flex. Google Scholar's ‘Vancouver’ style citation: Jacobson V. Tuning UNIX Lex or it's NOT true what they say about Lex. InUSENIX Conference Proceedings (Washington, DC, Winter 1987) 1987 (pp. 163-164). Lack of space after ‘In’ is Google, not me. I didn't find the paper itself, just citations of it. -- Cheers, Ralph. From arnold at skeeve.com Mon Jun 15 00:26:17 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Sun, 14 Jun 2020 08:26:17 -0600 Subject: [TUHS] v7 K&R C [really lexers] In-Reply-To: <20200614140303.E5C2A221A1@orac.inputplus.co.uk> References: <20200511005745.GL17035@mcvoy.com> <357EFE54-BD94-4C10-8C43-C6735BF7D317@via.net> <20200511202555.GU17035@mcvoy.com> <20200514173206.GJ20771@mcvoy.com> <20200516235308.icuQH%steffen@sdaoden.eu> <202005162359.04GNxalN3783011@darkstar.fourwinds.com> <50a4ee872b7be9770f96a1c1db10fb86@yaccman.com> <20200614140303.E5C2A221A1@orac.inputplus.co.uk> Message-ID: <202006141426.05EEQHEm013973@freefriends.org> Ralph Corderoy wrote: > Hi Rich, > > > Does anyone have the Usenix paper (80's timeframe I think) about making lex > > go fast? It was by Vern Paxson or Van Jacobson IIRC. Maybe only an > > abstract was published. The talk ended with a chart showing some CPU > > times, and the modified lex was only slightly slower than cat. Maybe Vern > > since he contributed to what became flex. > > Google Scholar's ‘Vancouver’ style citation: > > Jacobson V. Tuning UNIX Lex or it's NOT true what they say > about Lex. InUSENIX Conference Proceedings (Washington, > DC, Winter 1987) 1987 (pp. 163-164). > > Lack of space after ‘In’ is Google, not me. > > I didn't find the paper itself, just citations of it. I'm pretty sure I have those proceedings. Given that it's 2 pages, it's probably just an abstract. If there's interest, I can try to scan the pages. Arnold From jnc at mercury.lcs.mit.edu Mon Jun 15 00:46:43 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Sun, 14 Jun 2020 10:46:43 -0400 (EDT) Subject: [TUHS] Accessing the PDP-11/70 MMU registers and the kernel's u area Message-ID: <20200614144643.E4BBF18C09D@mercury.lcs.mit.edu> > From: Diomidis Spinellis > From the 2.11 BSD sources I understand that the PDP-11/70 MMU address > and data registers, KDSA and KDSD, start at 0172360 and 0172320 > respectively ... Expressed as 16-bit addreses, on a PDP-11 with mapping disabled, yes. > I checked this by looking at /dev/mem. I don't know about 2.11, but in other PDP-11 Unixes, /dev/mem gives access to the actual CPU memory bus (which on a /34, etc, is the 18-bit address UNIBUS; on a /70 it's a separate 22-bit address bus). In the /70 memory address space, the 'I/O page' (which is where the PxR's live) is at the top end of it, i.e. the registers are at 017772360 (KDSAR0), etc. > What am I missing? PDP-11's have a plethora of address spaces, of different sizes. You need to always be aware of which one you're working in. > My goal is to access from the console the kernel's u area. According to > mem(4) and the symbols in /unix, this should be at address 0140000. In the kernel virtual address space, yes. > Indeed, accessing it through /dev/kmem I get the expected results for > e.g. u_comm and u_uid. Because /dev/kmem gives access to kernel address space for the _current_ process. > I have been unable to find it in the machine's physical memory By far and away the easiest thing, for the _current_ process, is to use /dev/kmem, which automagally applies the correct mapping. For other processes, if the process is swapped in, there's some field in the proc structure which says where in physical memory it us. IIRC, the user struct and the kernel stack are stored in the very bottom of that. (This article: http://gunkies.org/wiki/Unix_V6_dump_analysis#Memory_layouts goes into some detail for V6. Not sure how different 2.11 is; I know it uses one block of kernel address space to map in code overlays, but I don't know all the details of how it works.) Anyway, using that, one could read the user area in /dev/mem, at the appropriate location. For swapped-out processes, a similar algorithm applies, but you'll have to look in the swap device (obviously). Noel From ralph at inputplus.co.uk Mon Jun 15 00:48:19 2020 From: ralph at inputplus.co.uk (Ralph Corderoy) Date: Sun, 14 Jun 2020 15:48:19 +0100 Subject: [TUHS] v7 K&R C [really lexers] In-Reply-To: <202006141426.05EEQHEm013973@freefriends.org> References: <20200511005745.GL17035@mcvoy.com> <357EFE54-BD94-4C10-8C43-C6735BF7D317@via.net> <20200511202555.GU17035@mcvoy.com> <20200514173206.GJ20771@mcvoy.com> <20200516235308.icuQH%steffen@sdaoden.eu> <202005162359.04GNxalN3783011@darkstar.fourwinds.com> <50a4ee872b7be9770f96a1c1db10fb86@yaccman.com> <20200614 140303.E5C2A221A1@orac.inputplus.co.uk> <202006141426.05EEQHEm013973@freefriends.org> Message-ID: <20200614144819.91834221A2@orac.inputplus.co.uk> Hi Arnold, > I'm pretty sure I have those proceedings. Given that it's 2 pages, > it's probably just an abstract. Yes, it's just the abstract, says https://compilers.iecc.com/comparch/article/87-05-012 -- Cheers, Ralph. From imp at bsdimp.com Mon Jun 15 01:52:26 2020 From: imp at bsdimp.com (Warner Losh) Date: Sun, 14 Jun 2020 09:52:26 -0600 Subject: [TUHS] Accessing the PDP-11/70 MMU registers and the kernel's u area In-Reply-To: <20200614144643.E4BBF18C09D@mercury.lcs.mit.edu> References: <20200614144643.E4BBF18C09D@mercury.lcs.mit.edu> Message-ID: On Sun, Jun 14, 2020 at 8:47 AM Noel Chiappa wrote: > Not sure how different 2.11 is; I know it uses > one block of kernel address space to map in code overlays, but I don't know > all the details of how it works. > Yes. There's 1 8kB instruction segment that implements the overlays. The calls to these routines are bounced through thunks that flips the segment register if needed on the call. Each overlay is limited to 8k. The 2.11BSD kernel has about 70k of overlays these days... and they are all almost full, as is the text segment (which is limited to 56k). They get around this by running the TCP/IP stack in supervisor mode, kinda as a process.. This means the total size of the kernel and data for the kernel is 320k on a machine with a 128k address space :). I've been looking a lot at that code lately... I'm back to patch 0, but am missing ld.c and ranlib.c in my push to get back to the start since they were augmented between 2.10.1 and 2.11 for the new portable archive format... Once I solve those two, it's off to build testing in simh. Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From dds at aueb.gr Mon Jun 15 02:13:08 2020 From: dds at aueb.gr (Diomidis Spinellis) Date: Sun, 14 Jun 2020 19:13:08 +0300 Subject: [TUHS] Accessing the PDP-11/70 MMU registers and the kernel's u area In-Reply-To: <20200614144643.E4BBF18C09D@mercury.lcs.mit.edu> References: <20200614144643.E4BBF18C09D@mercury.lcs.mit.edu> Message-ID: <6d0060c0-f577-9402-5e9a-e3fdc5cb25ea@aueb.gr> On 14-Jun-20 17:46, Noel Chiappa wrote: > I don't know about 2.11, but in other PDP-11 Unixes, /dev/mem gives access to > the actual CPU memory bus (which on a /34, etc, is the 18-bit address UNIBUS; > on a /70 it's a separate 22-bit address bus). In the /70 memory address > space, the 'I/O page' (which is where the PxR's live) is at the top end of it, > i.e. the registers are at 017772360 (KDSAR0), etc. Indeed, fetching data from the I/O page region gives plausible values and everything works beautifully. Thank you! The value of the kernel's u is 0140000 so it begins exactly at the start of the memory mapped by kernel D-space PAR #6 sim> examine 17772374 17772374: 016226 Offset of u_uid in struct user is 0242, so its physical memory address is: 016226 * 0100 + 0242 = 01623042 sim> examine 1623042 1623042: 000145 sim> examine 1623044 1623044: 000145 sim> examine 1623046 1623046: 000145 This indeed matches my uid (0145) repeated for svuid and ruid. Even better (and this was my original proof of concept goal), setting those addresses to 0 provides root access. $ id uid=101(dds) gid=101 groups=101, 0(wheel) $ while : ; do : ; done Simulation stopped, PC: 040214 (BNE 40232) sim> deposit 1623042 0 sim> deposit 1623044 0 sim> deposit 1623046 0 sim> cont $ id uid=0(root) gid=101 groups=101, 0(wheel) One remaining puzzle is why doesn't this work when examining the kernel's virtual address. I would expect to see again my user id below. sim> examine -v -k -d -o 140242 140242: 000026 Diomidis From wkt at tuhs.org Mon Jun 15 08:52:08 2020 From: wkt at tuhs.org (Warren Toomey) Date: Mon, 15 Jun 2020 08:52:08 +1000 Subject: [TUHS] Fwd: Origins and life of the pg pager Message-ID: <20200614225208.GB6122@minnie.tuhs.org> All, I just received this e-mail from a non-TUHS list member. If you have an answer for Michael, could you reply to him and pop a cc here as well? Thanks, Warren ----- Forwarded message from Michael Siegel ----- Date: Sun, 14 Jun 2020 16:37:59 +0200 From: Michael Siegel To: wkt at tuhs.org Subject: Origins and life of the pg pager Hi there, I'm trying to find out where the pg pager originated. The research I've done so far vaguely suggests it came with one of the System V versions, though Internet claims it to be “the name of the historical utility on BSD UNIX systems” occasionally.[1] I think System V because the source code of pg.c in the util-linux package says that this utility is “a clone of the System V CRT paging utility.”[2] I'd also like to find out when pg was discarded and if it ever made it into POSIX before that. Linux still has pg to the very day, but none of the current major BSDs (Free/Net/Open) offer it. POSIX 2001, 2004 Edition lists it as an excluded utility.[3] I've not been able to get the text of any prior POSIX documents. It seems they aren't freely available. Any ideas on how to proceed? Best Michael [1] This one's from Wikipedia (https://en.wikipedia.org/wiki/Pg_(Unix)), but I've also found other sites stating the same. [2] https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/tree/text-utils/pg.c [3] https://pubs.opengroup.org/onlinepubs/009696899/xrat/xcu_chap04.html ----- End forwarded message ----- From mah at mhorton.net Mon Jun 15 09:37:35 2020 From: mah at mhorton.net (Mary Ann Horton) Date: Sun, 14 Jun 2020 16:37:35 -0700 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: <20200614225208.GB6122@minnie.tuhs.org> References: <20200614225208.GB6122@minnie.tuhs.org> Message-ID: Eric Shienbrood at Berkeley wrote "more" around 1979, and it was the standard BSD pager. It was inspired by the --More-- option in the ITS terminal driver at MIT. "pg" did not come from Berkeley. My recollection is that it came from AT&T in response to requests to include "more", which was in exptools at Bell Labs but not in any standard AT&T system. "pg" is not in my UNIX 5.0 manual, but it's in my SVID with the comment "New in System V Release 2". UNIX 5.0 was the AT&T internal pre-release of System V.  SVID was the System V Interface Definition from AT&T.     Mary Ann On 6/14/20 3:52 PM, Warren Toomey wrote: > All, I just received this e-mail from a non-TUHS list member. If you have > an answer for Michael, could you reply to him and pop a cc here as well? > > Thanks, Warren > > ----- Forwarded message from Michael Siegel ----- > > Date: Sun, 14 Jun 2020 16:37:59 +0200 > From: Michael Siegel > To: wkt at tuhs.org > Subject: Origins and life of the pg pager > > Hi there, > > I'm trying to find out where the pg pager originated. > > The research I've done so far vaguely suggests it came with one of the > System V versions, though Internet claims it to be “the name of the > historical utility on BSD UNIX systems” occasionally.[1] > > I think System V because the source code of pg.c in the util-linux > package says that this utility is “a clone of the System V CRT paging > utility.”[2] > > I'd also like to find out when pg was discarded and if it ever made it > into POSIX before that. Linux still has pg to the very day, but none of > the current major BSDs (Free/Net/Open) offer it. POSIX 2001, 2004 > Edition lists it as an excluded utility.[3] I've not been able to get > the text of any prior POSIX documents. It seems they aren't freely > available. > > Any ideas on how to proceed? > > > Best > Michael > > > [1] This one's from Wikipedia (https://en.wikipedia.org/wiki/Pg_(Unix)), > but I've also found other sites stating the same. > > [2] > https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/tree/text-utils/pg.c > > [3] https://pubs.opengroup.org/onlinepubs/009696899/xrat/xcu_chap04.html > > ----- End forwarded message ----- From david at kdbarto.org Mon Jun 15 09:52:45 2020 From: david at kdbarto.org (David Barto) Date: Sun, 14 Jun 2020 16:52:45 -0700 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> Message-ID: My January 1983 printing of the System V Unix* System User’s Manual doesn’t have pg in it. And my 1986 AT&T The Unix(tm) System Users Manual for System V Release 2 also doesn’t have it. David > On Jun 14, 2020, at 4:37 PM, Mary Ann Horton wrote: > > Eric Shienbrood at Berkeley wrote "more" around 1979, and it was the standard BSD pager. It was inspired by the --More-- option in the ITS terminal driver at MIT. > > "pg" did not come from Berkeley. My recollection is that it came from AT&T in response to requests to include "more", which was in exptools at Bell Labs but not in any standard AT&T system. "pg" is not in my UNIX 5.0 manual, but it's in my SVID with the comment "New in System V Release 2". > > UNIX 5.0 was the AT&T internal pre-release of System V. SVID was the System V Interface Definition from AT&T. > > Mary Ann > > On 6/14/20 3:52 PM, Warren Toomey wrote: >> All, I just received this e-mail from a non-TUHS list member. If you have >> an answer for Michael, could you reply to him and pop a cc here as well? >> >> Thanks, Warren >> >> ----- Forwarded message from Michael Siegel ----- >> >> Date: Sun, 14 Jun 2020 16:37:59 +0200 >> From: Michael Siegel >> To: wkt at tuhs.org >> Subject: Origins and life of the pg pager >> >> Hi there, >> >> I'm trying to find out where the pg pager originated. >> >> The research I've done so far vaguely suggests it came with one of the >> System V versions, though Internet claims it to be “the name of the >> historical utility on BSD UNIX systems” occasionally.[1] >> >> I think System V because the source code of pg.c in the util-linux >> package says that this utility is “a clone of the System V CRT paging >> utility.”[2] >> >> I'd also like to find out when pg was discarded and if it ever made it >> into POSIX before that. Linux still has pg to the very day, but none of >> the current major BSDs (Free/Net/Open) offer it. POSIX 2001, 2004 >> Edition lists it as an excluded utility.[3] I've not been able to get >> the text of any prior POSIX documents. It seems they aren't freely >> available. >> >> Any ideas on how to proceed? >> >> >> Best >> Michael >> >> >> [1] This one's from Wikipedia (https://en.wikipedia.org/wiki/Pg_(Unix)), >> but I've also found other sites stating the same. >> >> [2] >> https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/tree/text-utils/pg.c >> >> [3] https://pubs.opengroup.org/onlinepubs/009696899/xrat/xcu_chap04.html >> >> ----- End forwarded message ----- From ggm at algebras.org Mon Jun 15 10:06:52 2020 From: ggm at algebras.org (George Michaelson) Date: Mon, 15 Jun 2020 10:06:52 +1000 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> Message-ID: I recently found myself wanting a pager which respected ^L formfeed page throw. looks like when people wrote pagers, they were seeking to reject the tyranny of the paper formalism. None of the current set recognize it as a marker worthy of pause. The thing is, that now we are not bound by formalisms of page size, it turns out the semantic intent of a page throw might be .. useful. -G From salewski at att.net Mon Jun 15 10:31:13 2020 From: salewski at att.net (Alan D. Salewski) Date: Sun, 14 Jun 2020 20:31:13 -0400 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> Message-ID: <20200615003113.GF18267@att.net> I first encountered 'pg' on an IBM RT[0] running AIX 2.x, whose wikipedia page[1] describes it this way: "AIX is based on UNIX System V with 4.3BSD-compatible extensions" -Al [0] https://en.wikipedia.org/wiki/IBM_RT_PC On 2020-06-14 16:52:45, David Barto spake thus: > My January 1983 printing of the System V Unix* System User’s Manual doesn’t have pg in it. > > And my 1986 AT&T The Unix(tm) System Users Manual for System V Release 2 also doesn’t have it. > > David > > > On Jun 14, 2020, at 4:37 PM, Mary Ann Horton wrote: > > > > Eric Shienbrood at Berkeley wrote "more" around 1979, and it was the standard BSD pager. It was inspired by the --More-- option in the ITS terminal driver at MIT. > > > > "pg" did not come from Berkeley. My recollection is that it came from AT&T in response to requests to include "more", which was in exptools at Bell Labs but not in any standard AT&T system. "pg" is not in my UNIX 5.0 manual, but it's in my SVID with the comment "New in System V Release 2". > > > > UNIX 5.0 was the AT&T internal pre-release of System V. SVID was the System V Interface Definition from AT&T. > > > > Mary Ann > > > > On 6/14/20 3:52 PM, Warren Toomey wrote: > >> All, I just received this e-mail from a non-TUHS list member. If you have > >> an answer for Michael, could you reply to him and pop a cc here as well? > >> > >> Thanks, Warren > >> > >> ----- Forwarded message from Michael Siegel ----- > >> > >> Date: Sun, 14 Jun 2020 16:37:59 +0200 > >> From: Michael Siegel > >> To: wkt at tuhs.org > >> Subject: Origins and life of the pg pager > >> > >> Hi there, > >> > >> I'm trying to find out where the pg pager originated. > >> > >> The research I've done so far vaguely suggests it came with one of the > >> System V versions, though Internet claims it to be “the name of the > >> historical utility on BSD UNIX systems” occasionally.[1] > >> > >> I think System V because the source code of pg.c in the util-linux > >> package says that this utility is “a clone of the System V CRT paging > >> utility.”[2] > >> > >> I'd also like to find out when pg was discarded and if it ever made it > >> into POSIX before that. Linux still has pg to the very day, but none of > >> the current major BSDs (Free/Net/Open) offer it. POSIX 2001, 2004 > >> Edition lists it as an excluded utility.[3] I've not been able to get > >> the text of any prior POSIX documents. It seems they aren't freely > >> available. > >> > >> Any ideas on how to proceed? > >> > >> > >> Best > >> Michael > >> > >> > >> [1] This one's from Wikipedia (https://en.wikipedia.org/wiki/Pg_(Unix)), > >> but I've also found other sites stating the same. > >> > >> [2] > >> https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/tree/text-utils/pg.c > >> > >> [3] https://pubs.opengroup.org/onlinepubs/009696899/xrat/xcu_chap04.html > >> > >> ----- End forwarded message ----- > -- ----------------------------------------------------------------- a l a n d. s a l e w s k i salewski at att.net 1024D/FA2C3588 EDFA 195F EDF1 0933 1002 6396 7C92 5CB3 FA2C 3588 ----------------------------------------------------------------- From cym224 at gmail.com Mon Jun 15 10:32:33 2020 From: cym224 at gmail.com (Nemo Nusquam) Date: Sun, 14 Jun 2020 20:32:33 -0400 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> Message-ID: <33259344-9c9c-aca0-db93-224e7e0a5dc9@gmail.com> For what it's worth, the OpenSolaris pg.c (https://minnie.tuhs.org/cgi-bin/utree.pl?file=OpenSolaris_b135/cmd/pg/pg.c) contains the following: /* * Copyright 2009 Sun Microsystems, Inc. All rights reserved. * Use is subject to license terms. */ /* Copyright (c) 1984, 1986, 1987, 1988, 1989 AT&T */ /* All Rights Reserved */ N. On 06/14/20 19:37, Mary Ann Horton wrote: > Eric Shienbrood at Berkeley wrote "more" around 1979, and it was the > standard BSD pager. It was inspired by the --More-- option in the ITS > terminal driver at MIT. > > "pg" did not come from Berkeley. My recollection is that it came from > AT&T in response to requests to include "more", which was in exptools > at Bell Labs but not in any standard AT&T system. "pg" is not in my > UNIX 5.0 manual, but it's in my SVID with the comment "New in System V > Release 2". > > UNIX 5.0 was the AT&T internal pre-release of System V. SVID was the > System V Interface Definition from AT&T. > > Mary Ann > > On 6/14/20 3:52 PM, Warren Toomey wrote: >> All, I just received this e-mail from a non-TUHS list member. If you >> have >> an answer for Michael, could you reply to him and pop a cc here as well? >> >> Thanks, Warren >> >> ----- Forwarded message from Michael Siegel ----- >> >> Date: Sun, 14 Jun 2020 16:37:59 +0200 >> From: Michael Siegel >> To: wkt at tuhs.org >> Subject: Origins and life of the pg pager >> >> Hi there, >> >> I'm trying to find out where the pg pager originated. >> >> The research I've done so far vaguely suggests it came with one of the >> System V versions, though Internet claims it to be “the name of the >> historical utility on BSD UNIX systems” occasionally.[1] >> >> I think System V because the source code of pg.c in the util-linux >> package says that this utility is “a clone of the System V CRT paging >> utility.”[2] >> >> I'd also like to find out when pg was discarded and if it ever made it >> into POSIX before that. Linux still has pg to the very day, but none of >> the current major BSDs (Free/Net/Open) offer it. POSIX 2001, 2004 >> Edition lists it as an excluded utility.[3] I've not been able to get >> the text of any prior POSIX documents. It seems they aren't freely >> available. >> >> Any ideas on how to proceed? >> >> >> Best >> Michael >> >> >> [1] This one's from Wikipedia (https://en.wikipedia.org/wiki/Pg_(Unix)), >> but I've also found other sites stating the same. >> >> [2] >> https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/tree/text-utils/pg.c >> >> >> [3] https://pubs.opengroup.org/onlinepubs/009696899/xrat/xcu_chap04.html >> >> ----- End forwarded message ----- From salewski at att.net Mon Jun 15 10:35:30 2020 From: salewski at att.net (Alan D. Salewski) Date: Sun, 14 Jun 2020 20:35:30 -0400 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: <20200615003113.GF18267@att.net> References: <20200614225208.GB6122@minnie.tuhs.org> <20200615003113.GF18267@att.net> Message-ID: <20200615003530.GG18267@att.net> [missed a footnote; fixed] On 2020-06-14 20:31:13, Alan D. Salewski spake thus: > I first encountered 'pg' on an IBM RT[0] running AIX 2.x, whose wikipedia > page[1] describes it this way: > > "AIX is based on UNIX System V with 4.3BSD-compatible extensions" > > -Al > > [0] https://en.wikipedia.org/wiki/IBM_RT_PC [1] https://en.wikipedia.org/wiki/IBM_AIX -- ----------------------------------------------------------------- a l a n d. s a l e w s k i salewski at att.net 1024D/FA2C3588 EDFA 195F EDF1 0933 1002 6396 7C92 5CB3 FA2C 3588 ----------------------------------------------------------------- From wkt at tuhs.org Mon Jun 15 11:12:30 2020 From: wkt at tuhs.org (Warren Toomey) Date: Mon, 15 Jun 2020 11:12:30 +1000 Subject: [TUHS] v7 K&R C [really lexers] In-Reply-To: <20200614144819.91834221A2@orac.inputplus.co.uk> References: <20200516235308.icuQH%steffen@sdaoden.eu> <202005162359.04GNxalN3783011@darkstar.fourwinds.com> <50a4ee872b7be9770f96a1c1db10fb86@yaccman.com> <20200614140303.E5C2A221A1@orac.inputplus.co.uk> <202006141426.05EEQHEm013973@freefriends.org> <20200614144819.91834221A2@orac.inputplus.co.uk> Message-ID: <20200615011230.GA4621@minnie.tuhs.org> On Sun, Jun 14, 2020 at 03:48:19PM +0100, Ralph Corderoy wrote: > Hi Arnold, > > > I'm pretty sure I have those proceedings. Given that it's 2 pages, > > it's probably just an abstract. > > Yes, it's just the abstract, says > https://compilers.iecc.com/comparch/article/87-05-012 Clem was kind enough to send the scan to me, and it's now here: https://www.tuhs.org/Archive/Documentation/Papers/TuningUnixLex_Jacobson_USENIX_Winter_1987_pp163_164.pdf Cheers, Warren From wkt at tuhs.org Mon Jun 15 11:29:20 2020 From: wkt at tuhs.org (Warren Toomey) Date: Mon, 15 Jun 2020 11:29:20 +1000 Subject: [TUHS] v7 K&R C [really lexers] In-Reply-To: <20200615011230.GA4621@minnie.tuhs.org> References: <20200516235308.icuQH%steffen@sdaoden.eu> <202005162359.04GNxalN3783011@darkstar.fourwinds.com> <50a4ee872b7be9770f96a1c1db10fb86@yaccman.com> <20200614140303.E5C2A221A1@orac.inputplus.co.uk> <202006141426.05EEQHEm013973@freefriends.org> <20200614144819.91834221A2@orac.inputplus.co.uk> <20200615011230.GA4621@minnie.tuhs.org> Message-ID: <20200615012920.GA7807@minnie.tuhs.org> On Mon, Jun 15, 2020 at 11:12:30AM +1000, Warren Toomey wrote: > Clem was kind enough to send the scan to me, and it's now here: > https://www.tuhs.org/Archive/Documentation/Papers/TuningUnixLex_Jacobson_USENIX_Winter_1987_pp163_164.pdf John R. Levine might be a person to ask to see if he has a copy of the paper. Warren From imp at bsdimp.com Mon Jun 15 11:38:38 2020 From: imp at bsdimp.com (Warner Losh) Date: Sun, 14 Jun 2020 19:38:38 -0600 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: <20200615003530.GG18267@att.net> References: <20200614225208.GB6122@minnie.tuhs.org> <20200615003113.GF18267@att.net> <20200615003530.GG18267@att.net> Message-ID: Here's what I've found: There's a pg.c, starting in System Vr2. R1 didn't have it. It changed in a number of fussy ways, but nothing substantial. It grew that copyright notice in SVr3, but the sun one quoted is missing /* Copyright (c) 1990, 1991 UNIX System Laboratories, Inc. */ that should have been there to, suggesting it was snagged between r2 and r3. There's also two other copies of pg as well. There's one in 32V. Though a quick diff suggests little in common with the System Vr2 version. There's also one in the BRL/JHU pdp-11 version of unix that is nearly identical to the System Vr2 one. Though it's dated 1985 while the SVr2 one is 1983. Both with a 1984 copyright by AT&T. SunOS 4.1.3 shipped with the System Vr2 version. in 5bin. Irix 6.5.5 had what looks like the System Vr3 copy in it, though I didn't delve into it. So this supports the recollections here that it was from System Vr2... However, there's also a Doug Gwen version from the Delaware 1980 Usenix tape submitted by geotronics and on other tapes. Haven't delved on what makes these different from each other though. There's one in the NOSC unix we have in the archive, unrelated, it seems to either of these. It's dated in 1979. There's still another one on the UNSW tapes from 1978 by Sape Mullender Informatics staff Vrije Universiteit Amsterdam. Also independent. It's on the 4th tape we have in the archive. 10th edition also has pg.c, but it's an I/O pager for the kernel. Which explains another comment in the System Vr2 one: * Note: The reason that there are so many commands to do * the same types of things is to try to accommodate * users of other paginators. Warner On Sun, Jun 14, 2020 at 6:36 PM Alan D. Salewski wrote: > [missed a footnote; fixed] > > On 2020-06-14 20:31:13, Alan D. Salewski spake thus: > > I first encountered 'pg' on an IBM RT[0] running AIX 2.x, whose wikipedia > > page[1] describes it this way: > > > > "AIX is based on UNIX System V with 4.3BSD-compatible extensions" > > > > -Al > > > > [0] https://en.wikipedia.org/wiki/IBM_RT_PC > > [1] https://en.wikipedia.org/wiki/IBM_AIX > > -- > ----------------------------------------------------------------- > a l a n d. s a l e w s k i salewski at att.net > 1024D/FA2C3588 EDFA 195F EDF1 0933 1002 6396 7C92 5CB3 FA2C 3588 > ----------------------------------------------------------------- > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Mon Jun 15 11:47:32 2020 From: lm at mcvoy.com (Larry McVoy) Date: Sun, 14 Jun 2020 18:47:32 -0700 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> <20200615003113.GF18267@att.net> <20200615003530.GG18267@att.net> Message-ID: <20200615014732.GJ29495@mcvoy.com> On Sun, Jun 14, 2020 at 07:38:38PM -0600, Warner Losh wrote: > SunOS 4.1.3 shipped with the System Vr2 version. in 5bin. Yeah, but nobody used it. pg was not invented here in a nutshell. more(1) was fine, less(1) was actually an improvement. pg(1) was a shitty also ran. From doug at cs.dartmouth.edu Mon Jun 15 12:26:21 2020 From: doug at cs.dartmouth.edu (Doug McIlroy) Date: Sun, 14 Jun 2020 22:26:21 -0400 Subject: [TUHS] Origins and life of the pg pager Message-ID: <202006150226.05F2QLL4066364@tahoe.cs.Dartmouth.EDU> > less(1) was actually an improvement Hmm. Less is my favorite whipping boy for featuritis. % less --help |wc 242 1331 12202 I am very happy with p. Doug From alec.muffett at gmail.com Mon Jun 15 12:38:25 2020 From: alec.muffett at gmail.com (Alec Muffett) Date: Mon, 15 Jun 2020 03:38:25 +0100 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: <20200615014732.GJ29495@mcvoy.com> References: <20200614225208.GB6122@minnie.tuhs.org> <20200615003113.GF18267@att.net> <20200615003530.GG18267@att.net> <20200615014732.GJ29495@mcvoy.com> Message-ID: On Mon, 15 Jun 2020, 02:48 Larry McVoy, wrote: > > Yeah, but nobody used it. pg was not invented here in a nutshell. more(1) > was fine, less(1) was actually an improvement. pg(1) was a shitty also > ran. I thought that the whole point of "pg" was that it was a print system filter; that you integrated it into pipelines to throw proper formfeeds every 66 (etc) lines rather than print over the margins of DECWriters (etc), and that it wasn't really a tool for humans. At least, that's how it was explained to me. -a -------------- next part -------------- An HTML attachment was scrubbed... URL: From bakul at iitbombay.org Mon Jun 15 12:41:27 2020 From: bakul at iitbombay.org (Bakul Shah) Date: Sun, 14 Jun 2020 19:41:27 -0700 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: <202006150226.05F2QLL4066364@tahoe.cs.Dartmouth.EDU> References: <202006150226.05F2QLL4066364@tahoe.cs.Dartmouth.EDU> Message-ID: <8DD10EA6-2AF8-4557-BF75-8F9C65780E78@iitbombay.org> On Jun 14, 2020, at 7:26 PM, Doug McIlroy wrote: > >> less(1) was actually an improvement > > Hmm. Less is my favorite whipping boy for featuritis. > > % less --help |wc > 242 1331 12202 % wc p.c 90 219 1504 p.c From alec.muffett at gmail.com Mon Jun 15 12:46:01 2020 From: alec.muffett at gmail.com (Alec Muffett) Date: Mon, 15 Jun 2020 03:46:01 +0100 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> <20200615003113.GF18267@att.net> <20200615003530.GG18267@att.net> <20200615014732.GJ29495@mcvoy.com> Message-ID: On Mon, 15 Jun 2020, 03:38 Alec Muffett, wrote: > On Mon, 15 Jun 2020, 02:48 Larry McVoy, wrote. > > I thought that the whole point of "pg" was that it was a print system > filter; that you integrated it into pipelines to throw proper formfeeds > every 66 (etc) lines rather than print over the margins of DECWriters > (etc), and that it wasn't really a tool for humans. > Okay, I just checked, that was "pr". My bad. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sauer at technologists.com Mon Jun 15 12:26:08 2020 From: sauer at technologists.com (Charles H. Sauer) Date: Sun, 14 Jun 2020 21:26:08 -0500 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> <20200615003113.GF18267@att.net> <20200615003530.GG18267@att.net> Message-ID: <6D89AB08-4BE1-4809-BE20-5CE3C284DA97@technologists.com> My memory is that pg was “always” in AIX for the RT. The group of us that worked on BSD convergence deprecated pg in favor of more, if I recall correctly. So far I don’t know how to verify any of this. It is also my memory that PC/IX (Sys III-based?) had pg. If so, ISC may have also included similar pg in both early AIX and PC/IX. Charlie > On Jun 14, 2020, at 8:38 PM, Warner Losh wrote: > > Here's what I've found: > > There's a pg.c, starting in System Vr2. R1 didn't have it. > > It changed in a number of fussy ways, but nothing substantial. > > It grew that copyright notice in SVr3, but the sun one quoted is missing > /* Copyright (c) 1990, 1991 UNIX System Laboratories, Inc. */ > that should have been there to, suggesting it was snagged between r2 and r3. > > There's also two other copies of pg as well. There's one in 32V. Though a quick diff suggests little in common with the System Vr2 version. > > There's also one in the BRL/JHU pdp-11 version of unix that is nearly identical to the System Vr2 one. Though it's dated 1985 while the SVr2 one is 1983. Both with a 1984 copyright by AT&T. > > SunOS 4.1.3 shipped with the System Vr2 version. in 5bin. > > Irix 6.5.5 had what looks like the System Vr3 copy in it, though I didn't delve into it. > > So this supports the recollections here that it was from System Vr2... > > However, there's also a Doug Gwen version from the Delaware 1980 Usenix tape submitted by geotronics and on other tapes. Haven't delved on what makes these different from each other though. > > There's one in the NOSC unix we have in the archive, unrelated, it seems to either of these. It's dated in 1979. > > There's still another one on the UNSW tapes from 1978 by Sape Mullender Informatics staff Vrije Universiteit Amsterdam. Also independent. It's on the 4th tape we have in the archive. > > 10th edition also has pg.c, but it's an I/O pager for the kernel. > > Which explains another comment in the System Vr2 one: > * Note: The reason that there are so many commands to do > * the same types of things is to try to accommodate > * users of other paginators. > > Warner > > On Sun, Jun 14, 2020 at 6:36 PM Alan D. Salewski > wrote: > [missed a footnote; fixed] > > On 2020-06-14 20:31:13, Alan D. Salewski spake thus: > > I first encountered 'pg' on an IBM RT[0] running AIX 2.x, whose wikipedia > > page[1] describes it this way: > > > > "AIX is based on UNIX System V with 4.3BSD-compatible extensions" > > > > -Al > > > > [0] https://en.wikipedia.org/wiki/IBM_RT_PC > > [1] https://en.wikipedia.org/wiki/IBM_AIX > > -- > ----------------------------------------------------------------- > a l a n d. s a l e w s k i salewski at att.net > 1024D/FA2C3588 EDFA 195F EDF1 0933 1002 6396 7C92 5CB3 FA2C 3588 > ----------------------------------------------------------------- -- voice: +1.512.784.7526 e-mail: sauer at technologists.com fax: +1.512.346.5240 web: https://technologists.com/sauer/ Facebook/Google/Skype/Twitter: CharlesHSauer -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Mon Jun 15 12:55:59 2020 From: lm at mcvoy.com (Larry McVoy) Date: Sun, 14 Jun 2020 19:55:59 -0700 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: <8DD10EA6-2AF8-4557-BF75-8F9C65780E78@iitbombay.org> References: <202006150226.05F2QLL4066364@tahoe.cs.Dartmouth.EDU> <8DD10EA6-2AF8-4557-BF75-8F9C65780E78@iitbombay.org> Message-ID: <20200615025559.GK29495@mcvoy.com> I believe that was my statement, not Dougs, and I stand behind it. Less was a huge improvement. If you want me to dig out why I will but I don't want to because it will make you look bad, that is not my goal. On Sun, Jun 14, 2020 at 07:41:27PM -0700, Bakul Shah wrote: > On Jun 14, 2020, at 7:26 PM, Doug McIlroy wrote: > > > >> less(1) was actually an improvement > > > > Hmm. Less is my favorite whipping boy for featuritis. > > > > % less --help |wc > > 242 1331 12202 > > % wc p.c > 90 219 1504 p.c -- --- Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm From robpike at gmail.com Mon Jun 15 14:26:21 2020 From: robpike at gmail.com (Rob Pike) Date: Mon, 15 Jun 2020 14:26:21 +1000 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: <20200615025559.GK29495@mcvoy.com> References: <202006150226.05F2QLL4066364@tahoe.cs.Dartmouth.EDU> <8DD10EA6-2AF8-4557-BF75-8F9C65780E78@iitbombay.org> <20200615025559.GK29495@mcvoy.com> Message-ID: Yeah, p is all we need. I think it originated with td at UofT. I might have brought it with me to Bell Labs, or recreated it. Probably the former. The Plan 9 version is, except for a change of I/O library, likely the same code. 90 lines of C. More and less are both more and less than p. -rob On Mon, Jun 15, 2020 at 12:56 PM Larry McVoy wrote: > I believe that was my statement, not Dougs, and I stand behind it. > Less was a huge improvement. If you want me to dig out why I will > but I don't want to because it will make you look bad, that is not > my goal. > > On Sun, Jun 14, 2020 at 07:41:27PM -0700, Bakul Shah wrote: > > On Jun 14, 2020, at 7:26 PM, Doug McIlroy wrote: > > > > > >> less(1) was actually an improvement > > > > > > Hmm. Less is my favorite whipping boy for featuritis. > > > > > > % less --help |wc > > > 242 1331 12202 > > > > % wc p.c > > 90 219 1504 p.c > > -- > --- > Larry McVoy lm at mcvoy.com > http://www.mcvoy.com/lm > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bakul at iitbombay.org Mon Jun 15 14:27:55 2020 From: bakul at iitbombay.org (Bakul Shah) Date: Sun, 14 Jun 2020 21:27:55 -0700 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: <20200615025559.GK29495@mcvoy.com> References: <202006150226.05F2QLL4066364@tahoe.cs.Dartmouth.EDU> <8DD10EA6-2AF8-4557-BF75-8F9C65780E78@iitbombay.org> <20200615025559.GK29495@mcvoy.com> Message-ID: <88839FF5-A964-40AD-AACB-B5424262D436@iitbombay.org> No one is beating you up for your opinion. One should be able to express alternate points of view without being made to look bad. But let us face it, less is not only more, it is far too much :-) Less gone wild. @ Fortune Yost had added page mode as a line discipline. That would've avoided the temptation to turn less into a mini editor. > On Jun 14, 2020, at 7:55 PM, Larry McVoy wrote: > > I believe that was my statement, not Dougs, and I stand behind it. > Less was a huge improvement. If you want me to dig out why I will > but I don't want to because it will make you look bad, that is not > my goal. > > On Sun, Jun 14, 2020 at 07:41:27PM -0700, Bakul Shah wrote: >> On Jun 14, 2020, at 7:26 PM, Doug McIlroy wrote: >>> >>>> less(1) was actually an improvement >>> >>> Hmm. Less is my favorite whipping boy for featuritis. >>> >>> % less --help |wc >>> 242 1331 12202 >> >> % wc p.c >> 90 219 1504 p.c > > -- > --- > Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm From ggm at algebras.org Mon Jun 15 14:38:55 2020 From: ggm at algebras.org (George Michaelson) Date: Mon, 15 Jun 2020 14:38:55 +1000 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: <88839FF5-A964-40AD-AACB-B5424262D436@iitbombay.org> References: <202006150226.05F2QLL4066364@tahoe.cs.Dartmouth.EDU> <8DD10EA6-2AF8-4557-BF75-8F9C65780E78@iitbombay.org> <20200615025559.GK29495@mcvoy.com> <88839FF5-A964-40AD-AACB-B5424262D436@iitbombay.org> Message-ID: I get two hits in this, aside from the bikeshed of 'what is the status line telling me about things I am looking at' which is a disease Emacs has. 1) I use more, and the screen DOESNT clear when I finish. DOH! 2) I use less, and the screen DOES clear when I finish. DOH! I kind of tend to 1. The 'clear' command works at a tty to flush screen so for me, noting backwards scroll, less is 'more; clear' But backwards scroll has been useful to me. Oh I know, I have scrollbars on the terminal. But still. I don't like having to engage the mouse for this stuff. I want commands, modal or otherwise to do things which go to 'where am I, in the buffer' Both are wildly complex. Who knew tail on a log was something integrated into a pager? (I think it is for less. I don't think more has that) -G From lars at nocrew.org Mon Jun 15 15:41:17 2020 From: lars at nocrew.org (Lars Brinkhoff) Date: Mon, 15 Jun 2020 05:41:17 +0000 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: (Mary Ann Horton's message of "Sun, 14 Jun 2020 16:37:35 -0700") References: <20200614225208.GB6122@minnie.tuhs.org> Message-ID: <7wsgewophe.fsf@junk.nocrew.org> Mary Ann Horton wrote: > Eric Shienbrood at Berkeley wrote "more" around 1979, and it was the > standard BSD pager. It was inspired by the --More-- option in the ITS > terminal driver at MIT. Daniel Halbert wrote to comp.society.folklore in 1994: "I was a first-year graduate student at UC Berkeley in 1978. I had been an undergraduate at MIT, and had used the ITS timesharing systems there, which ran on PDP-10's. ITS put a "--MORE--" at the bottom of the screen when one typed out files [..] So I wrote a simple "cr3"-like program, but had it print "--More--" instead of ringing the bell. I had it accept space instead of carriage return to continue, because that was what I was used to from ITS." Wikipedia says "more" was written by Daniel Halbert and expanded by Eric Shienbrood and Geoff Peck. From arnold at skeeve.com Mon Jun 15 16:06:54 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Mon, 15 Jun 2020 00:06:54 -0600 Subject: [TUHS] v7 K&R C [really lexers] In-Reply-To: <20200615012920.GA7807@minnie.tuhs.org> References: <20200516235308.icuQH%steffen@sdaoden.eu> <202005162359.04GNxalN3783011@darkstar.fourwinds.com> <50a4ee872b7be9770f96a1c1db10fb86@yaccman.com> <20200614140303.E5C2A221A1@orac.inputplus.co.uk> <202006141426.05EEQHEm013973@freefriends.org> <20200614144819.91834221A2@orac.inputplus.co.uk> <20200615011230.GA4621@minnie.tuhs.org> <20200615012920.GA7807@minnie.tuhs.org> Message-ID: <202006150606.05F66sd2007108@freefriends.org> Warren Toomey wrote: > On Mon, Jun 15, 2020 at 11:12:30AM +1000, Warren Toomey wrote: > > Clem was kind enough to send the scan to me, and it's now here: > > https://www.tuhs.org/Archive/Documentation/Papers/TuningUnixLex_Jacobson_USENIX_Winter_1987_pp163_164.pdf > > John R. Levine might be a person to ask to see if he has a copy of > the paper. > Warren I've just done so, by posting to comp.compilers. Arnold From thomas.paulsen at firemail.de Mon Jun 15 17:12:10 2020 From: thomas.paulsen at firemail.de (Thomas Paulsen) Date: Mon, 15 Jun 2020 09:12:10 +0200 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: <8DD10EA6-2AF8-4557-BF75-8F9C65780E78@iitbombay.org> References: <202006150226.05F2QLL4066364@tahoe.cs.Dartmouth.EDU> <8DD10EA6-2AF8-4557-BF75-8F9C65780E78@iitbombay.org> Message-ID: <476bbcc6330b701361e685796d576077@firemail.de> > Hmm. Less is my favorite whipping boy for featuritis. > > % less --help |wc > 242 1331 12202 Fedora release 32: less --help |wc 252 1364 12625 From erc at pobox.com Mon Jun 15 17:40:06 2020 From: erc at pobox.com (Ed Carp) Date: Mon, 15 Jun 2020 02:40:06 -0500 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: <20200615025559.GK29495@mcvoy.com> References: <202006150226.05F2QLL4066364@tahoe.cs.Dartmouth.EDU> <8DD10EA6-2AF8-4557-BF75-8F9C65780E78@iitbombay.org> <20200615025559.GK29495@mcvoy.com> Message-ID: Agreed. I use "less -c" daily. :)

Virus-free. www.avast.com
On 6/14/20, Larry McVoy wrote: > I believe that was my statement, not Dougs, and I stand behind it. > Less was a huge improvement. If you want me to dig out why I will > but I don't want to because it will make you look bad, that is not > my goal. > > On Sun, Jun 14, 2020 at 07:41:27PM -0700, Bakul Shah wrote: >> On Jun 14, 2020, at 7:26 PM, Doug McIlroy wrote: >> > >> >> less(1) was actually an improvement >> > >> > Hmm. Less is my favorite whipping boy for featuritis. >> > >> > % less --help |wc >> > 242 1331 12202 >> >> % wc p.c >> 90 219 1504 p.c > > -- > --- > Larry McVoy lm at mcvoy.com > http://www.mcvoy.com/lm > From robpike at gmail.com Mon Jun 15 18:17:26 2020 From: robpike at gmail.com (Rob Pike) Date: Mon, 15 Jun 2020 18:17:26 +1000 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: References: <202006150226.05F2QLL4066364@tahoe.cs.Dartmouth.EDU> <8DD10EA6-2AF8-4557-BF75-8F9C65780E78@iitbombay.org> <20200615025559.GK29495@mcvoy.com> Message-ID: The Blit was nice. -rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Mon Jun 15 23:56:46 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 15 Jun 2020 09:56:46 -0400 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: <7wsgewophe.fsf@junk.nocrew.org> References: <20200614225208.GB6122@minnie.tuhs.org> <7wsgewophe.fsf@junk.nocrew.org> Message-ID: First, as for pg(1) itself, I thought it came from one of the non-Research labs, like Holmdel or Indian Hill. I certainly remember using it on AT&T based system before System V arrived on the scene. It's possible, tjk had it from USG, but I think it's more likely I saw it from someone like Phil Karn. We also had pg(1) at Tek until I was introduced to more(1) probably originally from Jim Kleckner and the CAD folks [which worked better and pg(1) -- which means I probably brought pg(1) to Tek, unless it came from one the Purdue folks like Ward, I don't remember]. I also know pg(1) was available for PWB 3.0 at some point, as we had it at Masscomp before we got System V and we had it the AT&T 'universe' but I don't remember the provenance of that code (again I could have brought it with me, but I think it came to Masscomp via MIT). That said, Warren's PWB 3.0/SYS III source tree does not show binary or source, which makes the thought that it was AT&T based, but originated at a lab other than Summit or MH. On Mon, Jun 15, 2020 at 1:41 AM Lars Brinkhoff wrote: > Daniel Halbert wrote to comp.society.folklore in 1994: > "I was a first-year graduate student at UC Berkeley in 1978. I had > been an undergraduate at MIT, and had used the ITS timesharing systems > there, which ran on PDP-10's. ITS put a "--MORE--" at the bottom of > the screen when one typed out files [..] > I believe that to be true. Wikipedia says "more" was written by Daniel Halbert and expanded by Eric Shienbrood > and Geoff Peck. > I also believe that is partially true and Mary Ann is actually correct in the provenance. I think Eric arrived later than Dan (maybe a year later), but was also ex-MIT, and he too had used/seen --MORE-- on ITS as Dan had. But I was under the impression Eric started over. Then a few years after Eric, Geoff worked with Eric's sources to add a few features. As for starting over/hacking on the program from Dan's original code base, it's hard to call that one, as I recall that Dan's version was not much more than a hack on to cat(1). i.e. the original version was pretty simple, and I don't remember that it 'knew' about the type of terminal being used, get info from TERMCAP *et al*. [My intro to Dan's version was at Tek, BTW but was quickly replaced with Eric's TERMCAP version which I think I got from Mary Ann, but might have also been via the CAD connection - we in TekLabs had written all of the TERMCAP support for the Tek 4025 and family and I was working to get it all back into the UCB database, so we got a bunch of early PDP-11 based TERMCAP/curses code to help debug things since the 4025 supported some interesting modes]. To be fair, ITS and Dan certainly should get credit for introducing the idea of the terminal pager to UCB, but it really was Eric that created the more(1) program framework that took off and eventually did begat less(1), pg(1), p(1) and others. As for less(1) itself, the Gnu folks seem to have started with Geoff's version (which I think is the version in 4.2 BSD if you look at SCCS delta in Kurt's disks), with the biggest addition was for the user to be able to go backward and look at some of the text that had already scrolled off the page. But as pointed out elsewhere, after the Gnu folks had peed all over it; as Doug says, (and I suspect most of us agree) it really became a sort of lesson in featuritis. But as discussed on many things in the historical computer world, more/less/pg/p often comes back to taste -> more(1) and p(1) were much more directed a doing one job well. As Larry says, there are features of less(1) which can be handy depending on your environment -- if you don't have a window manager/BLIT or today's user interfaces with unlimited scrolling (*i.e. *still on an ADM3A or VT-100), then many of the features of less(1) probably are considered a nice to have, you got used to them and it became your standard [burned in the ROMs in your fingers like csh(1) and vi(1) into my own]. But as Rob and Doug have pointed out, p(1) is more than sufficient and the 'extras' that programs like less(1) in a modern and clean environment start having doubtful long term value. Certainly, the 'cost' in complexity and 'code bloat' seems like a poor trade-off. But the assumption to me is that the desire for those type of features are fulfilled with different tools in the current world, but if you grew up with using them (say like Larry) I can see the value. Clem -------------- next part -------------- An HTML attachment was scrubbed... URL: From mah at mhorton.net Tue Jun 16 00:03:53 2020 From: mah at mhorton.net (Mary Ann Horton) Date: Mon, 15 Jun 2020 07:03:53 -0700 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> <20200615003113.GF18267@att.net> <20200615003530.GG18267@att.net> Message-ID: My 32V manual does not include pg. We got UNIX/32V at Berkeley shortly after we got our Vax and decided we didn't like VMS. Early 1979, I think. If it had had pg, Eric probably wouldn't have written more. So if there was a pg in 32V, it must have been added later.     Mary Ann On 6/14/20 6:38 PM, Warner Losh wrote: > There's also two other copies of pg as well. There's one in 32V. > Though a quick diff suggests little in common with the System Vr2 version. From mah at mhorton.net Tue Jun 16 00:15:03 2020 From: mah at mhorton.net (Mary Ann Horton) Date: Mon, 15 Jun 2020 07:15:03 -0700 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> <7wsgewophe.fsf@junk.nocrew.org> Message-ID: <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> Hmm. Clem has a far richer recollection than I do. I don't recall Dan's version, but I defer to Clem. I do have a vague recollection of a program called cr3 intended to mimic the tty driver thing. Perhaps that was Dan's version. I note that my 4.1BSD manual more(1) says the author is "Eric Shienbrood, minor revisions by John Foderaro and Geoffrey Peck." On 6/15/20 6:56 AM, Clem Cole wrote: > > Wikipedia says "more" was written by Daniel Halbert and expanded > by EricShienbrood and Geoff Peck. > > I also believe that is partially true and Mary Ann is actually correct > in the provenance. > > I think Eric arrived later than Dan (maybe a year later), but was also > ex-MIT, and he too had used/seen --MORE-- on ITS as Dan had.  But I > was under the impression Eric started over. Then a few years after > Eric, Geoff worked with Eric's sources to add a few features. As for > starting over/hacking on the program from Dan's original code base, > it's hard to call that one, as  I recall that Dan's version was not > much more than a hack on to cat(1). i.e. the original version was > pretty simple, and I don't remember that it 'knew' about the type of > terminal being used, get info from TERMCAP /et al/. -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Tue Jun 16 00:56:02 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 15 Jun 2020 10:56:02 -0400 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> References: <20200614225208.GB6122@minnie.tuhs.org> <7wsgewophe.fsf@junk.nocrew.org> <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> Message-ID: On Mon, Jun 15, 2020 at 10:15 AM Mary Ann Horton wrote: > Hmm. Clem has a far richer recollection than I do. I don't recall Dan's > version, but I defer to Clem. > > I do have a vague recollection of a program called cr3 intended to mimic > the tty driver thing. Perhaps that was Dan's version. > You know, indeed it might have been called cr3 originally. We used it for such a short time. I just remember we had something that was an alternative to pg(1), there was something different about it, maybe it better used the try driver's canonical modes, I really don't remember. I do remember when the TERMCAP based more(1) showed up, we all switched to it overnight. Funny, when Steve Zimmerman was at Masscomp, one of the things he did was put the ITS style 'more' into the TTY driver (along with Tenex style ^T), both of which I loved having until I went to Stellar -- it was always on my list of things that would be cool. But we had window managers by then, and we never thought adding more into the TTY driver it was worth it. The truth is, either put it in the driver as a base feature so everything that runs gets support for it without having to remember to add it to the pipeline, or solves the problem more globally like a window manager does. I can see good arguments to both schemes -- that later is a tad more elegant and see seems simpler / less complex (Doug's rules). In fact, we have lived with that style of solution for years, but ... I still need to use more(1) even with a window manager/terminal emulator that allows me to scroll 'forever' [I don't need many of the features of less(1)], but something like p/pg/more does seem to be desirable. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich.salz at gmail.com Tue Jun 16 01:04:41 2020 From: rich.salz at gmail.com (Richard Salz) Date: Mon, 15 Jun 2020 11:04:41 -0400 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> <7wsgewophe.fsf@junk.nocrew.org> <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> Message-ID: > > Funny, when Steve Zimmerman was at Masscomp, one of the things he did was > put the ITS style 'more' into the TTY driver (along with Tenex style ^T) > Did Steve go to UofMD? They were big on those things. (I seem to recall a patch adding it to Pyramid) -------------- next part -------------- An HTML attachment was scrubbed... URL: From lars at nocrew.org Tue Jun 16 01:45:55 2020 From: lars at nocrew.org (Lars Brinkhoff) Date: Mon, 15 Jun 2020 15:45:55 +0000 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: (Clem Cole's message of "Mon, 15 Jun 2020 10:56:02 -0400") References: <20200614225208.GB6122@minnie.tuhs.org> <7wsgewophe.fsf@junk.nocrew.org> <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> Message-ID: <7wr1ugmix8.fsf@junk.nocrew.org> Clem Cole wrote: > I think Eric arrived later than Dan (maybe a year later), but was also > ex-MIT, and he too had used/seen --MORE-- on ITS as Dan had. But I was > under the impression Eric started over. That's not what Dan writes (see below), but I'll ask him to confirm his version. Clem Cole wrote: > Mary Ann Horton wrote: >> I do have a vague recollection of a program called cr3 intended to >> mimic the tty driver thing. Perhaps that was Dan's version. > > You know, indeed it might have been called cr3 originally. Dan's full account is on his web site here: https://danhalbert.org/more.html He writes he was not happy with cr3 and wrote more to replace it. From clemc at ccc.com Tue Jun 16 02:14:42 2020 From: clemc at ccc.com (Clem Cole) Date: Mon, 15 Jun 2020 12:14:42 -0400 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> <7wsgewophe.fsf@junk.nocrew.org> <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> Message-ID: No, that the same Steve Zimmerman was (is) and MIT type (he is also the 'Zimmerman' of EMACS fame. On Mon, Jun 15, 2020 at 11:04 AM Richard Salz wrote: > Funny, when Steve Zimmerman was at Masscomp, one of the things he did was >> put the ITS style 'more' into the TTY driver (along with Tenex style ^T) >> > > Did Steve go to UofMD? They were big on those things. (I seem to recall > a patch adding it to Pyramid) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lars at nocrew.org Tue Jun 16 05:08:31 2020 From: lars at nocrew.org (Lars Brinkhoff) Date: Mon, 15 Jun 2020 19:08:31 +0000 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: <7wr1ugmix8.fsf@junk.nocrew.org> (Lars Brinkhoff's message of "Mon, 15 Jun 2020 15:45:55 +0000") References: <20200614225208.GB6122@minnie.tuhs.org> <7wsgewophe.fsf@junk.nocrew.org> <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> <7wr1ugmix8.fsf@junk.nocrew.org> Message-ID: <7wh7vcm9jk.fsf@junk.nocrew.org> > Clem Cole wrote: >> I think Eric arrived later than Dan (maybe a year later), but was also >> ex-MIT, and he too had used/seen --MORE-- on ITS as Dan had. But I was >> under the impression Eric started over. > That's not what Dan writes (see below), but I'll ask him to confirm his > version. He responded and asked me to post his reply to the mailing list: As described in https://danhalbert.org/more.html, I wrote the original version of more. It was only a page or two or code, and was meant to replace the dysfunctional cr3. It went into raw mode so it could read a space as a "next page" character, but otherwise didn't care about what kind of terminal it was on. (The ADM-3's (not 3A's) were barely a terminal.) Clem Cole's timeline is a bit off. Eric Shienbrood, Geoff Peck, and I all arrived at Berkeley at the same time in fall 1978 as CS grad students, and became friends (along with some other folks). After I wrote the first version, I did not have time to continue to working on it, so Eric picked it up and added many more features. My contribution is that I had the original idea (to replace cr3) and coined the name, and made the very first version. I am grateful to Eric, Geoff, and others taking it much further with many more features. I am still in touch with Eric, and will ask him to see what he has to say. We are both still alive so I appreciate Lars getting in touch with me to verify. I also wrote a hack less as a joke, which was more except that it scrolled up the screen rather than down . I think it was Bill Joy who then wrote another less that paged from the end of the file (and also scrolled up the screen). From woods at robohack.ca Tue Jun 16 06:19:50 2020 From: woods at robohack.ca (Greg A. Woods) Date: Mon, 15 Jun 2020 13:19:50 -0700 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> <20200615003113.GF18267@att.net> <20200615003530.GG18267@att.net> Message-ID: At Mon, 15 Jun 2020 07:03:53 -0700, Mary Ann Horton wrote: Subject: Re: [TUHS] Origins and life of the pg pager > > My 32V manual does not include pg. > > We got UNIX/32V at Berkeley shortly after we got our Vax and decided > we didn't like VMS. Early 1979, I think. If it had had pg, Eric > probably wouldn't have written more. > > So if there was a pg in 32V, it must have been added later. There is a /usr/src/cmd/pg.c and /usr/bin/pg binary in Unix-32V, but no manual page. The pg.c I've seen has the date Nov. 5, 1978. It uses newline to proceed one page, '/' or control-Y to restart from the beginning, '-' or '_' to move back one page (via a 16kb buffer), and '!' to start a sub-shell. It prints 20 lines at a time with a formfeed between each page. The error message about failing to open a file is prefixed by "dk:" suggesting it may have had a different name at one point. There was also of course a pg.c on the 1980 and 1981 Usenix tapes, and on the 1983 tape a pg.1l manual page accompanies it. It was written by D. A. Gwyn starting sometime before June 1980. It is unique amongst programs called "pg". The pg.c in System V Release 2 has an sccsid of 1.5 (but no date), so may or may not have evolved from the one in 32V -- however it operates quite differently and the source doesn't appear to bear any noticeable resemblance. The source for this one remains quite recognizable in all System V derivatives, right up to OpenSolaris. -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP Digital Signature URL: From norman at oclsc.org Tue Jun 16 06:25:32 2020 From: norman at oclsc.org (Norman Wilson) Date: Mon, 15 Jun 2020 16:25:32 -0400 Subject: [TUHS] Origins and life of the pg pager Message-ID: <1592252735.3176.for-standards-violators@oclsc.org> Rob Pike: Yeah, p is all we need. I think it originated with td at UofT. I might have brought it with me to Bell Labs, or recreated it. Probably the former. ==== The former, I think. The source code in V10 is very similar to that you left behind at Caltech (where I first encountered p). Most differences have to do with using opendir and readdir rather than reading raw directories in the SPname code. A further clue is that, even in V10, p.c begins /*%cc p.c pad.o spname.o */ The tool that looked for such lines to tell it how to compile things (I forget its name; was it comp?) doesn't seem to have survived in the archival backup I have from Caltech HEP, but I'm quite sure it came from U of T as well. Norman Wilson Toronto ON From mah at mhorton.net Tue Jun 16 06:50:17 2020 From: mah at mhorton.net (Mary Ann Horton) Date: Mon, 15 Jun 2020 13:50:17 -0700 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> <20200615003113.GF18267@att.net> <20200615003530.GG18267@att.net> Message-ID: Fascinating. My UNIX/32V manual, from Berkeley, bears the date "December, 1978" on the cover. I've hand-written "Unix/32V" on it. The 32V archive on minnie.tuhs.org bears the date "February, 1979" in /usr/man/man0/title. The text of the cover sheets is somewhat different.  My copy is "UNIX PROGRAMMER'S MANUAL / Seventh Edition / VAX-11 Version" compared to TUHS "UNIX/32V PROGRAMMER'S MANUAL / Version 1.0". I also note there is no pg.c in the 3BSD distribution. It seems likely that what Berkeley based 3BSD on was a prerelease of UNIX/32V, prior to the "official" 1.0 version, and what while pg.c was written in November, it didn't make it onto the tape we got at Berkeley. Imagine if pg.c had been on that tape! more might never have been written. (Of course, we all read TFM, so we might have missed the source code all by its lonesome.)     Mary Ann On 6/15/20 1:19 PM, Greg A. Woods wrote: >> My 32V manual does not include pg. >> >> We got UNIX/32V at Berkeley shortly after we got our Vax and decided >> we didn't like VMS. Early 1979, I think. If it had had pg, Eric >> probably wouldn't have written more. >> >> So if there was a pg in 32V, it must have been added later. > There is a /usr/src/cmd/pg.c and /usr/bin/pg binary in Unix-32V, but no > manual page. The pg.c I've seen has the date Nov. 5, 1978. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robpike at gmail.com Tue Jun 16 07:36:42 2020 From: robpike at gmail.com (Rob Pike) Date: Tue, 16 Jun 2020 07:36:42 +1000 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: <1592252735.3176.for-standards-violators@oclsc.org> References: <1592252735.3176.for-standards-violators@oclsc.org> Message-ID: I had forgotten about that tool, along with many others. "Comp" sounds plausible but as I said, I forget. (Not a sign of age; my memory for details is no match for Clem's; to me stage actors are superheroes). I do remember the tool existing, though, and now see it as related to a long list of similar things, including "go generate". I wonder if my subconscious held on to it. This history stuff is fun because of the reminder of a time when tools were simple and you could create a whole new one in an afternoon. -rob On Tue, Jun 16, 2020 at 6:26 AM Norman Wilson wrote: > Rob Pike: > > Yeah, p is all we need. I think it originated with td at UofT. I might > have > brought it with me to Bell Labs, or recreated it. Probably the former. > > ==== > > The former, I think. The source code in V10 is very similar > to that you left behind at Caltech (where I first encountered > p). Most differences have to do with using opendir and readdir > rather than reading raw directories in the SPname code. > > A further clue is that, even in V10, p.c begins > > /*%cc p.c pad.o spname.o > */ > > The tool that looked for such lines to tell it how to compile > things (I forget its name; was it comp?) doesn't seem to have > survived in the archival backup I have from Caltech HEP, but > I'm quite sure it came from U of T as well. > > Norman Wilson > Toronto ON > -------------- next part -------------- An HTML attachment was scrubbed... URL: From henry.r.bent at gmail.com Tue Jun 16 08:55:40 2020 From: henry.r.bent at gmail.com (Henry Bent) Date: Mon, 15 Jun 2020 18:55:40 -0400 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: References: <1592252735.3176.for-standards-violators@oclsc.org> Message-ID: On Mon, 15 Jun 2020 at 17:37, Rob Pike wrote: > I had forgotten about that tool, along with many others. "Comp" sounds > plausible but as I said, I forget. (Not a sign of age; my memory for > details is no match for Clem's; to me stage actors are superheroes). > Remembering the blocking and the actor's names for a play you saw thirty years ago would be an impressive feat, but I would argue that it is much more beneficial in the long run to remember the basic plot and the message that the production was attempting to convey. > This history stuff is fun because of the reminder of a time when tools were simple and you could create a whole new one in an afternoon. Indeed, that's a nearly impossible task now. We have transitioned to a world where you can say, "here's my sketch for a program that I think would be useful," and that sketch is either a basic chapter outline or you've only finished writing chapter one. You put it up on Github, try to give it a little publicity, and hope that others can share your vision or at least provide constructive criticism. -Henry -------------- next part -------------- An HTML attachment was scrubbed... URL: From pnr at planet.nl Wed Jun 17 08:41:09 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Wed, 17 Jun 2020 00:41:09 +0200 Subject: [TUHS] updated: loop networks at Bell Labs Message-ID: <1F6AD3BF-E7E8-49BD-8993-2539A10F1A89@planet.nl> With some further reading and research (and the kind help of Heinz Lycklama and Jon Steinart) I’ve found that my understanding of early loop networks at Bell Labs confused several different systems. As far as I can currently tell there were at least 4 different loop networks developed around 1970 at Murray Hill. 1. The first one is the “Newhall Loop” (paper published in 1969). This loop used twisted pair cabling, ran at about 3Mhz and used variable sized messages. It seems to have used some sort of token to coordinate between hosts. This might have been the network that Ken Thompson recalled as having been in operation when he arrived at the labs in 1966. 2. The second one appears to have been the “Pierce Loop”, as described in 3 BSTJ papers submitted in 1970/71. This one was coax based, used T1 compatible frames and was used to connect H516 computers with various bits of equipment. It seems to have had a very short life span. Part of my confusion was that the term Pierce Loop also appears to have been used in a generic sense to denote loop networks with fixed-sized frames. 3. The third one is the “Weller Loop” (paper published in 1971). This loop used coax cabling, ran at 3.3Mhz and used fixed 35 bit frames/cells. Each cell carried one address byte and two data bytes. One participant on the loop was the controller and effectively polled the other stations. In its 1971 form it appears to have been for the H516’s only and was referred to as a “Serial I/O bus”. This is what Jon Steinhart was talking about. The Weller loop was later redesigned (memo written in 1973) to interface with PDP-11’s as well. Heinz Lycklama used this loop in 1974 to connect several systems running (rump) Unix - see his paper about peripheral Unix here: https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-75-1352-2_Emulation_of_UNIX_on_Peripheral_Processors_19750109.pdf This Serial I/O bus remained in use for several years at least. 4. The fourth and best known one is the “Spider Loop” (memo published in 1974, but operational from 1972). Twisted pair cabling, using T1 compatible frames. In use until about 1978. Main uses appear to have been the ‘nfs’ file store and the ‘npr’ remote printing service. My conclusion from all that is that in 1974 Unix had access to two networks, Spider and the Serial I/O bus. For both, first experiments would have been in 1973. It is hard to be sure which one came first. If I had to venture a guess today, I’d say that Spider connected to Unix several months before the Weller loop (even though the 1st generation Weller loop preceded Spider). Maybe the conclusion is that both happened more or less in parallel: Weller was also one of the designers of the Spider hardware. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gregg.drwho8 at gmail.com Wed Jun 17 12:01:33 2020 From: gregg.drwho8 at gmail.com (Gregg Levine) Date: Tue, 16 Jun 2020 22:01:33 -0400 Subject: [TUHS] Interesting problem Message-ID: Hello! I normally download items from the Society FTP site rather than from the archival area on the website. And my clients kept getting bounced, claiming that I should stop trying to connect frequently. Oddly enough that was not the case a while ago. Was the archive recently updated regarding hosting software? On Windows it was FileZilla, and spun down to one rather than eight or ten. And on the WSL SLES12 SP3 arrangement it was ncftp. Now downloading the reports to go along with the one Paul made available from the location is not important for today, I can wait until tomorrow or the next day. ----- Gregg C Levine gregg.drwho8 at gmail.com "This signature fought the Time Wars, time and again." From msi at malbolge.net Thu Jun 18 22:49:00 2020 From: msi at malbolge.net (Michael Siegel) Date: Thu, 18 Jun 2020 14:49:00 +0200 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: <20200614225208.GB6122@minnie.tuhs.org> References: <20200614225208.GB6122@minnie.tuhs.org> Message-ID: <76a036bf-c897-19c2-3461-ad85212060bb@malbolge.net> Hello everyone, now that I'm subscribed to this list as well, first let me say thank you for all the info on pg(1) you've provided. I really wasn't expecting that much. The only thing I'm able to add here is that, shortly after I had sent my question to Warren, I discovered that the Rationale section in the POSIX description of more(1) explicitly says: The more utility, available in BSD and BSD-derived systems, was chosen as the prototype for the POSIX file display program since it is more widely available than either the public-domain program less or than pg, a pager provided in System V.[1] This also strongly suggests that pg(1) never made it into POSIX. Well, if I'm not mistaken, that quote actually says that pg(1) didn't make it into POSIX. Michael [1] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/more.html#tag_20_81_18 From heinz at osta.com Fri Jun 19 11:01:16 2020 From: heinz at osta.com (Heinz Lycklama) Date: Thu, 18 Jun 2020 18:01:16 -0700 Subject: [TUHS] updated: loop networks at Bell Labs In-Reply-To: <1F6AD3BF-E7E8-49BD-8993-2539A10F1A89@planet.nl> References: <1F6AD3BF-E7E8-49BD-8993-2539A10F1A89@planet.nl> Message-ID: <229637de-55f7-56a7-b5ee-4b0a1578b707@osta.com> A later memo in 1978 provides a Figure that shows how the I/O Loop was used to support multiple satellite processors. See here: https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-78-3114-2_A_Minicomputer_Satellite_Processor_System_19780322.pdf Heinz On 6/16/2020 3:41 PM, Paul Ruizendaal wrote: > With some further reading and research (and the kind help of Heinz > Lycklama and Jon Steinart) I’ve found that my understanding of early > loop networks at Bell Labs confused several different systems. As far > as I can currently tell there were at least 4 different loop networks > developed around 1970 at Murray Hill. > > 1. The first one is the “Newhall Loop” (paper published in 1969). This > loop used twisted pair cabling, ran at about 3Mhz and used variable > sized messages. It seems to have used some sort of token to coordinate > between hosts. This might have been the network that Ken Thompson > recalled as having been in operation when he arrived at the labs in 1966. > > 2. The second one appears to have been the “Pierce Loop”, as described > in 3 BSTJ papers submitted in 1970/71. This one was coax based, used > T1 compatible frames and was used to connect H516 computers with > various bits of equipment. It seems to have had a very short life > span. Part of my confusion was that the term Pierce Loop also appears > to have been used in a generic sense to denote loop networks with > fixed-sized frames. > > 3. The third one is the “Weller Loop” (paper published in 1971). This > loop used coax cabling, ran at 3.3Mhz and used fixed 35 bit > frames/cells. Each cell carried one address byte and two data bytes. > One participant on the loop was the controller and effectively polled > the other stations. In its 1971 form it appears to have been for the > H516’s only and was referred to as a “Serial I/O bus”. This is what > Jon Steinhart was talking about. > > The Weller loop was later redesigned (memo written in 1973) to > interface with PDP-11’s as well. Heinz Lycklama used this loop in 1974 > to connect several systems running (rump) Unix - see his paper about > peripheral Unix here: > https://www.tuhs.org/Archive/Documentation/TechReports/Heinz_Tech_Memos/TM-75-1352-2_Emulation_of_UNIX_on_Peripheral_Processors_19750109.pdf > > This Serial I/O bus remained in use for several years at least. > > 4. The fourth and best known one is the “Spider Loop” (memo published > in 1974, but operational from 1972). Twisted pair cabling, using T1 > compatible frames. In use until about 1978. Main uses appear to have > been the ‘nfs’ file store and the ‘npr’ remote printing service. > > My conclusion from all that is that in 1974 Unix had access to two > networks, Spider and the Serial I/O bus. For both, first experiments > would have been in 1973. It is hard to be sure which one came first. > If I had to venture a guess today, I’d say that Spider connected to > Unix several months before the Weller loop (even though the 1st > generation Weller loop preceded Spider). Maybe the conclusion is that > both happened more or less in parallel: Weller was also one of the > designers of the Spider hardware. > From imp at bsdimp.com Fri Jun 19 11:50:20 2020 From: imp at bsdimp.com (Warner Losh) Date: Thu, 18 Jun 2020 19:50:20 -0600 Subject: [TUHS] 2.9BSD-Seismo Message-ID: So in BSD family tree, there's a 2.9BSD-Seismo and a 2.9.1BSD listed. Yet I can't seem to find them in the TUHS archive. There is a 2.9-Patch directory dated 1985 the same as the date that's found in the family tree. Is that the Seismo version? And Kirk's archive has a 2.9pucc directory, which appears appears to be from Purdue dated1987. How does this relate? Thanks for any help you can provide Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Fri Jun 19 11:59:35 2020 From: lm at mcvoy.com (Larry McVoy) Date: Thu, 18 Jun 2020 18:59:35 -0700 Subject: [TUHS] 2.9BSD-Seismo In-Reply-To: References: Message-ID: <20200619015935.GG23788@mcvoy.com> I can't provide any help other than I remember BSD-seismo. I have no memory of what is in it, I just know that was a thing. On Thu, Jun 18, 2020 at 07:50:20PM -0600, Warner Losh wrote: > So in BSD family tree, there's a 2.9BSD-Seismo and a 2.9.1BSD listed. > > Yet I can't seem to find them in the TUHS archive. > > There is a 2.9-Patch directory dated 1985 the same as the date that's found > in the family tree. Is that the Seismo version? > > And Kirk's archive has a 2.9pucc directory, which appears appears to be > from Purdue dated1987. How does this relate? > > Thanks for any help you can provide > > Warner -- --- Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm From reed at reedmedia.net Sat Jun 20 05:50:37 2020 From: reed at reedmedia.net (Jeremy C. Reed) Date: Fri, 19 Jun 2020 14:50:37 -0500 (CDT) Subject: [TUHS] 2.9BSD-Seismo In-Reply-To: References: Message-ID: On Thu, 18 Jun 2020, Warner Losh wrote: > So in BSD family tree, there's a 2.9BSD-Seismo and a 2.9.1BSD listed. > Yet I can't seem to find them in the TUHS archive. > > There is a 2.9-Patch directory dated 1985 the same as the date that's found > in the family tree. Is that the Seismo version? 2.9-Patch appears to the Seismo version -- Harvard/CSS (Center for Seismic Studies) release from Bostic. (It is not a diff file but is a tarball.) See the README and GOTCHA files. And see https://www.krsaborio.net/bsd/research/1985/0831.htm I don't know if the URL will change as it was elsewhere before.) From rich.salz at gmail.com Sat Jun 20 09:06:32 2020 From: rich.salz at gmail.com (Richard Salz) Date: Fri, 19 Jun 2020 19:06:32 -0400 Subject: [TUHS] Dennis's list thesis is found Message-ID: https://computerhistory.org/blog/discovering-dennis-ritchies-lost-dissertation/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at kdbarto.org Sat Jun 20 10:44:39 2020 From: david at kdbarto.org (David Barto) Date: Fri, 19 Jun 2020 17:44:39 -0700 Subject: [TUHS] Dennis's list thesis is found In-Reply-To: References: Message-ID: <65F7EFE9-960B-40BB-A6B0-DEDEE4546A95@kdbarto.org> > On Jun 19, 2020, at 4:06 PM, Richard Salz wrote: > > https://computerhistory.org/blog/discovering-dennis-ritchies-lost-dissertation/ Thanks for finding this. An excellent read. David -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnold at skeeve.com Sun Jun 21 15:35:38 2020 From: arnold at skeeve.com (Arnold Robbins) Date: Sun, 21 Jun 2020 08:35:38 +0300 Subject: [TUHS] Jon Bentley's DFORMAT preprocessor now available Message-ID: Hello All. I have reconstituted Jon Bentley's DFORMAT preprocessor (troff and awk source) and made it available on GitHub. See https://github.com/arnoldrobbins/dformat. This is an awk program that reads a description of data layouts, such as registers or network packets, and generates PIC descriptions. A Makefile generates PDF from the dformat.ms file using GNU troff. Jon Bentley was helpful in the final stages, including contributing his original files. Enjoy! Arnold Robbins From jaapna at xs4all.nl Sun Jun 21 19:34:10 2020 From: jaapna at xs4all.nl (Jaap Akkerhuis) Date: Sun, 21 Jun 2020 11:34:10 +0200 Subject: [TUHS] Jon Bentley's DFORMAT preprocessor now available In-Reply-To: References: Message-ID: > On Jun 21, 2020, at 7:35, Arnold Robbins wrote: > > Hello All. > > I have reconstituted Jon Bentley's DFORMAT preprocessor (troff and awk > source) and made it available on GitHub. See > https://github.com/arnoldrobbins/dformat. Hom much does it differ from the one in "Plan 9 from User Space", https://github.com/9fans/plan9port . BTW, there it is called "bwk's dformat"[1]. jaap [1] https://github.com/9fans/plan9port/tree/master/src/cmd/dformat -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 267 bytes Desc: Message signed with OpenPGP URL: From arnold at skeeve.com Sun Jun 21 23:08:08 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Sun, 21 Jun 2020 07:08:08 -0600 Subject: [TUHS] Jon Bentley's DFORMAT preprocessor now available In-Reply-To: References: Message-ID: <202006211308.05LD88jU003738@freefriends.org> > > On Jun 21, 2020, at 7:35, Arnold Robbins wrote: > > > > Hello All. > > > > I have reconstituted Jon Bentley's DFORMAT preprocessor (troff and awk > > source) and made it available on GitHub. See > > https://github.com/arnoldrobbins/dformat. Jaap Akkerhuis wrote: > Hom much does it differ from the one in "Plan 9 from User Space", https://github.com/9fans/plan9port . > BTW, there it is called "bwk's dformat"[1]. > > jaap > > [1] https://github.com/9fans/plan9port/tree/master/src/cmd/dformat It's misattributed. It's Bentley's code. But nice of them to include it. I opened a github issue. Arnold From msi at malbolge.net Mon Jun 22 04:49:31 2020 From: msi at malbolge.net (Michael Siegel) Date: Sun, 21 Jun 2020 20:49:31 +0200 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> <7wsgewophe.fsf@junk.nocrew.org> <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> Message-ID: Am 15.06.20 um 16:56 schrieb Clem Cole: > Funny, when Steve Zimmerman was at Masscomp, one of the things he did > was put the ITS style 'more' into the TTY driver (along with Tenex > style ^T), both of which I loved having until I went to Stellar -- > it was always on my list of things that would be cool. But we had > window managers by then, and we never thought adding more into the > TTY driver it was worth it. The truth is, either put it in the > driver as a base feature so everything that runs gets support for it > without having to remember to add it to the pipeline, or solves the > problem more globally like a window manager does. I can see good > arguments to both schemes -- that later is a tad more elegant and see > seems simpler / less complex (Doug's rules). In fact, we have lived > with that style of solution for years, but ... I still need to use > more(1) even with a window manager/terminal emulator that allows me > to scroll 'forever' [I don't need many of the features of less(1)], > but something like p/pg/more does seem to be desirable. >From my perspective as a user, I would love to see paging built in. Having to use pagers to view terminal output conveniently is, in my opinion, a major nuisance. In fact, my question about the origin and life of pg(1) arose while writing a yet unfinished addendum to a blog article that details my failed attempt to hack together a solution for auto-paged ls(1).[1] The conclusion there was: Those attempts (you can find them in quite a few places throughout the Web) are futile, just pipe to a pager when you need it. However, there's no need to write out "less" every time. You can just alias that to "pg" without causing any harm and save two letters, which is an improvement for a task that is performed manually rather often. Michael [1] https://www.msiism.org/blog/2019/07/17/no_fun_with_auto-paged_ls.html PS: I hope it's okay that I chose to reply just to the list address and take all the other addresses out. If not, please let me know. From jnc at mercury.lcs.mit.edu Mon Jun 22 08:40:18 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Sun, 21 Jun 2020 18:40:18 -0400 (EDT) Subject: [TUHS] Fwd: Origins and life of the pg pager Message-ID: <20200621224018.DBFFA18C09A@mercury.lcs.mit.edu> > From: Michael Siegel > there's no need to write out "less" every time. You can just alias that > to "pg" without causing any harm and save two letters, which is an > improvement for a task that is performed manually rather often. It always surpised me that there wasn't more of this - abbreviating the names of the most-used commands, to minimize typing - or more specifically, run-length encoding them based on how frequently they were used, with the most-used ones given the shortest names. The MIT-DSSR PWB1 system had a pager called just 'p' (source here: http://ana-3.lcs.mit.edu/~jnc/tech/unix/s2/p.c if anyone wants to see it; the man page is dated Apr/77); and 'ls' was tweaked to recognize the command name 'l' as an alias for 'ls -ls'. Of course, aliases didn't exist in the shell back then, which was why the command had to be coded to recognize the short form, and act differently. (And /bin/l was linked to /bin/ls.) 'l' and 'p' are _still_ aliased in my shell,to this day! > I hope it's okay that I chose to reply just to the list address and take > all the other addresses out. 'That's not a bug, that's a feature!' I always delete other addresses when replying to a list, unless I think someone might not be subscribed to that list. Noel From woods at robohack.ca Mon Jun 22 10:35:28 2020 From: woods at robohack.ca (Greg A. Woods) Date: Sun, 21 Jun 2020 17:35:28 -0700 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> <7wsgewophe.fsf@junk.nocrew.org> <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> Message-ID: At Sun, 21 Jun 2020 20:49:31 +0200, Michael Siegel wrote: Subject: Re: [TUHS] Fwd: Origins and life of the pg pager > > In fact, my question about the origin and life of pg(1) arose while > writing a yet unfinished addendum to a blog article that details my > failed attempt to hack together a solution for auto-paged ls(1).[1] > > The conclusion there was: Those attempts (you can find them in quite a > few places throughout the Web) are futile, just pipe to a pager when you > need it. Indeed! Just pipe to a pager when you need to! There's also the benefit use of a modern "terminal" gives, e.g. Xterm: (a) on a modern screen it can display a whole lot more than an old-fashioned 24x80 terminal; (b) it offers a scroll-back buffer so if you get somewhat more output than expected you can just scroll back a bit to scan through it all. Such features further reduce the need for any pager. In a related vein what annoys me are so-called modern programs like "git", "hg", and others which default to always piping their output through $PAGER, along with such things as colour decorations enabled, but when you tack on "|$PAGER" to their command-line then they turn off the decorations! They cause me to have to undo decades of finger memory. > However, there's no need to write out "less" every time. You can just > alias that to "pg" without causing any harm and save two letters, which > is an improvement for a task that is performed manually rather often. I've used an alias and/or shell function called "ds" (for display) since approximately forever, typically to invoke whatever I've set $PAGER to, at least in more recent times. Others I've know have called it just "p", and indeed there's been a pager called just "p" in Research Unix since the Eighth Edition (and in 8th Edition the manual page suggests is also known as "more" and by 10th Edition the manual adds "pg" too, though it's not installed directly with those aliases by the makefile). I've stuck with "ds", partly for finger memory, and also partly because it is for the most part unique (at least in all environments I've typically used). That gives me the option of using a pager called "pg" directly should I want to, while getting my "default" favourite pager (from those available on a given system) when I use "ds". -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP Digital Signature URL: From dfawcus+lists-tuhs at employees.org Tue Jun 23 02:24:06 2020 From: dfawcus+lists-tuhs at employees.org (Derek Fawcus) Date: Mon, 22 Jun 2020 17:24:06 +0100 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> <7wsgewophe.fsf@junk.nocrew.org> <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> Message-ID: <20200622162406.GA48733@clarinet.employees.org> On Sun, Jun 21, 2020 at 05:35:28PM -0700, Greg A. Woods wrote: > > In a related vein what annoys me are so-called modern programs like > "git", "hg", and others which default to always piping their output > through $PAGER, along with such things as colour decorations enabled, > but when you tack on "|$PAGER" to their command-line then they turn off > the decorations! They cause me to have to undo decades of finger > memory. Well, for further fun, one could always export GIT_PAGER=cat, then git will not use a pager for those commands, recognising that 'cat' is a no-op in such cases. :-) DF From robpike at gmail.com Tue Jun 23 07:33:31 2020 From: robpike at gmail.com (Rob Pike) Date: Tue, 23 Jun 2020 07:33:31 +1000 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: <20200622162406.GA48733@clarinet.employees.org> References: <20200614225208.GB6122@minnie.tuhs.org> <7wsgewophe.fsf@junk.nocrew.org> <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> <20200622162406.GA48733@clarinet.employees.org> Message-ID: There is only one correct way. % grep PAGER .bashrc export PAGER='col -b' % -rob On Tue, Jun 23, 2020 at 2:26 AM Derek Fawcus < dfawcus+lists-tuhs at employees.org> wrote: > On Sun, Jun 21, 2020 at 05:35:28PM -0700, Greg A. Woods wrote: > > > > In a related vein what annoys me are so-called modern programs like > > "git", "hg", and others which default to always piping their output > > through $PAGER, along with such things as colour decorations enabled, > > but when you tack on "|$PAGER" to their command-line then they turn off > > the decorations! They cause me to have to undo decades of finger > > memory. > > Well, for further fun, one could always export GIT_PAGER=cat, > then git will not use a pager for those commands, recognising that > 'cat' is a no-op in such cases. :-) > > DF > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bakul at iitbombay.org Tue Jun 23 07:59:48 2020 From: bakul at iitbombay.org (Bakul Shah) Date: Mon, 22 Jun 2020 14:59:48 -0700 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> <7wsgewophe.fsf@junk.nocrew.org> <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> <20200622162406.GA48733@clarinet.employees.org> Message-ID: <0F78CB9D-870B-4008-B975-23756EFC6F83@iitbombay.org> $ PAGER="col -b" git log 33mcommit fe147581ff1c5b0571a40369cd257fe60bfee58cm33m (m1;36mHEAD -> m1;32mmasterm33m, m1;31morigin/masterm33m)m ... $ git log | cat commit fe147581ff1c5b0571a40369cd257fe60bfee58c There is probably a way to turn off syntax coloring but I haven't bothered to learn. $ man git-log | wc 2512 14056 127024 > On Jun 22, 2020, at 2:33 PM, Rob Pike wrote: > > There is only one correct way. > > % grep PAGER .bashrc > export PAGER='col -b' > % > > -rob > > > On Tue, Jun 23, 2020 at 2:26 AM Derek Fawcus > wrote: > On Sun, Jun 21, 2020 at 05:35:28PM -0700, Greg A. Woods wrote: > > > > In a related vein what annoys me are so-called modern programs like > > "git", "hg", and others which default to always piping their output > > through $PAGER, along with such things as colour decorations enabled, > > but when you tack on "|$PAGER" to their command-line then they turn off > > the decorations! They cause me to have to undo decades of finger > > memory. > > Well, for further fun, one could always export GIT_PAGER=cat, > then git will not use a pager for those commands, recognising that > 'cat' is a no-op in such cases. :-) > > DF -------------- next part -------------- An HTML attachment was scrubbed... URL: From khm at sciops.net Tue Jun 23 07:58:57 2020 From: khm at sciops.net (Kurt H Maier) Date: Mon, 22 Jun 2020 14:58:57 -0700 Subject: [TUHS] Fwd: Origins and life of the pg pager In-Reply-To: References: <20200614225208.GB6122@minnie.tuhs.org> <7wsgewophe.fsf@junk.nocrew.org> <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> <20200622162406.GA48733@clarinet.employees.org> Message-ID: <20200622215857.GB36730@wopr> On Tue, Jun 23, 2020 at 07:33:31AM +1000, Rob Pike wrote: > There is only one correct way. > > > ... .bashrc > I have a doubt. khm From steffen at sdaoden.eu Tue Jun 23 08:43:01 2020 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Tue, 23 Jun 2020 00:43:01 +0200 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: <0F78CB9D-870B-4008-B975-23756EFC6F83@iitbombay.org> References: <20200614225208.GB6122@minnie.tuhs.org> <7wsgewophe.fsf@junk.nocrew.org> <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> <20200622162406.GA48733@clarinet.employees.org> <0F78CB9D-870B-4008-B975-23756EFC6F83@iitbombay.org> Message-ID: <20200622224301.kdouj%steffen@sdaoden.eu> Bakul Shah wrote in <0F78CB9D-870B-4008-B975-23756EFC6F83 at iitbombay.org>: |$ PAGER="col -b" git log | |33mcommit fe147581ff1c5b0571a40369cd257fe60bfee58cm33m (m1;36mHEAD \ |-> m1;32mmasterm33m, m1;31morigin/masterm33m)m | |... | |$ git log | cat | |commit fe147581ff1c5b0571a40369cd257fe60bfee58c | |There is probably a way to turn off syntax coloring but I haven't bothered \ |to learn. | |$ man git-log | wc | |2512 14056 127024 Hihihi. I wanted to reply Rob Pike wrote in : |There is only one correct way. | |% grep PAGER .bashrc | |export PAGER='col -b' As long as groff(1) is used for manual display i assume you also have "export GROFF_NO_SGR=1". The source of #?0|kent:steffen$ pkginfo -o `command -v col` Package File util-linux usr/bin/col util-linux usr/bin/colcrt util-linux usr/bin/colrm util-linux usr/bin/column #?0|kent:steffen$ prt-get info util-linux Name: util-linux Path: /usr/ports/core Version: 2.35.2 Release: 1 Description: Miscellaneous system utilities URL: https://www.kernel.org/pub/linux/utils/util-linux/ Maintainer: CRUX System Team, core-ports at crux dot nu Dependencies: eudev,ncurses,zlib states /* * This command is deprecated. The utility is in maintenance mode, * meaning we keep them in source tree for backward compatibility * only. Do not waste time making this command better, unless the * fix is about security or other very critical issue. * * See Documentation/deprecated.txt for more information. */ Plan9port has a version, though. And i searched in golang/cmd, but did not find a copy. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From pnr at planet.nl Tue Jun 23 19:09:57 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Tue, 23 Jun 2020 11:09:57 +0200 Subject: [TUHS] VFS prior to 1984 Message-ID: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> When googling for File System Switch or Virtual File System most sources mention Sun NFS and SysVr3 as the earliest implementations. Some sources mention 8th Edition. I did a (short) search on FSS/VFS in earlier, non-Unix OS’s (Tenex, Multics, CTSS, etc.), but none of those seem to have had a comparable concept. Does anybody recall prior art (prior to 1984) in this area? Paul From lm at mcvoy.com Wed Jun 24 00:01:24 2020 From: lm at mcvoy.com (Larry McVoy) Date: Tue, 23 Jun 2020 07:01:24 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> Message-ID: <20200623140124.GR22291@mcvoy.com> I like to claim credit for creating the ChangeSet concept, the grouping of a set of deltas across a number of files. But I wasn't the first. Back when dejanews was a thing and you could search usenet posts in a date range, if you searched before I started talking about ChangeSets, you could find 6, count 'em, 6 hits. There was a really obscure system called Aide De Camp, that had a similar concept. But if you searched after I started talking about them you would see millions of hits. So I didn't invent the concept but I sure as heck made the world understand the concept. I suspect Sun could be in a similar position. The VFS concept is pretty sweet so there might have been someone before Sun. I'll long odds that if there was, it didn't gain traction until Sun did it. If there is nothing that predates it, then the inspiration was almost certainly the device driver interface. One interface, many devices. VFS is the same. On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote: > When googling for File System Switch or Virtual File System most sources mention Sun NFS and SysVr3 as the earliest implementations. Some sources mention 8th Edition. > > I did a (short) search on FSS/VFS in earlier, non-Unix OS???s (Tenex, Multics, CTSS, etc.), but none of those seem to have had a comparable concept. > > Does anybody recall prior art (prior to 1984) in this area? > > Paul -- --- Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm From clemc at ccc.com Wed Jun 24 01:12:40 2020 From: clemc at ccc.com (Clem Cole) Date: Tue, 23 Jun 2020 11:12:40 -0400 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <20200623140124.GR22291@mcvoy.com> References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200623140124.GR22291@mcvoy.com> Message-ID: I agree with Larry with his observations. The only thing I would add is that Popek and Walker had a file system in the UCLA Locus distributed system for the 11/70 that was published in SIGOPS in late 1970s. In fact, Gerry came up with the idea after a sabbatical @ PARC. But it was not formalized like NFS as a separate (layered) FS, it was just part of the basic Locus FS. Peter Weinberger created the file system switch (FSS) for V8 and in the UNIX world, it was first. Dave Arnovitz used it for RFS in System V, while Perry Flynn and I used it for EFS, both of us after talking to Peter. Rusty and team did the VFS layer Sun @ (Larry will have to tell you who actually built it, I never knew). Rusty and I both published papers in the '85 Summer USENIX (NFS and EFS were contemporaries -- the difference is that Sun gave away NFS and Masscomp was not smart enough to do that). I talked to Steve Kleinman extensively about VFS at one point, and I'm pretty sure he and the rest of the Sun guys had talked to the PARC folks who after Gerry went back to UCLA started working on a DFS. The idea of the state-less (idempotent) file system RPC that NFS used based on stuff PARC did. But I'm not sure PARC had anything like the FSS or the VFS layer. Peter, Dave, and I used a stateful scheme because we chose to have full UNIX FS semantics, which NFS did not. In the end, early NFS was notorious for putting 'holes' in the files because of the automatic seek in every operation and errors not coming until close(2) time. EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it was not nearly as flexible as what Sun built [which had a crude interface generator], although until years later when Mike Leibensger built PIG (the Paceline Interface Generator) NFS RPC was always a PITA and not much better in practice than what we had at Masscomp. In fact, the point of the EFS paper was it's all about the recovery when there is a failure/error. If you read his paper, Rusty's point was who cares if there is an error (I've always felt vindicated that while I lost the war, over time everyone came to our way of thinking and now NFS V4 looks a whole lot like EFS did). Having a DFS as a feature was an incredible advance and proved we needed something (and it needed to be standard in all systems). NFS really lead the market with that advance, although it sort of took a few years to make it really good. The fact is that others had the same idea before or at least contemporary with it. That said, and to give the NFS team* a huge amount of credit *(and great applause), the VFS layer was better thought out than the FSS and in fact made it possible to add a lot of different file systems into UNIX later. FSS was much more ad hoc. At LCC, when we built VPROC for TNC a few years later, we used some of the same ideas from VFS and of course used VFS for the file system layer since TNC had to have full POSIX semantics. (It's a shame VPROC never caught on the way VFS did). Clem On Tue, Jun 23, 2020 at 10:02 AM Larry McVoy wrote: > I like to claim credit for creating the ChangeSet concept, the grouping > of a set of deltas across a number of files. But I wasn't the first. > Back when dejanews was a thing and you could search usenet posts in a > date range, if you searched before I started talking about ChangeSets, > you could find 6, count 'em, 6 hits. There was a really obscure system > called Aide De Camp, that had a similar concept. > > But if you searched after I started talking about them you would see > millions of hits. > > So I didn't invent the concept but I sure as heck made the world > understand the concept. > > I suspect Sun could be in a similar position. The VFS concept is > pretty sweet so there might have been someone before Sun. I'll > long odds that if there was, it didn't gain traction until Sun > did it. > > If there is nothing that predates it, then the inspiration was > almost certainly the device driver interface. One interface, > many devices. VFS is the same. > > On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote: > > When googling for File System Switch or Virtual File System most sources > mention Sun NFS and SysVr3 as the earliest implementations. Some sources > mention 8th Edition. > > > > I did a (short) search on FSS/VFS in earlier, non-Unix OS???s (Tenex, > Multics, CTSS, etc.), but none of those seem to have had a comparable > concept. > > > > Does anybody recall prior art (prior to 1984) in this area? > > > > Paul > > -- > --- > Larry McVoy lm at mcvoy.com > http://www.mcvoy.com/lm > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdm at cfcl.com Wed Jun 24 01:55:42 2020 From: rdm at cfcl.com (Rich Morin) Date: Tue, 23 Jun 2020 08:55:42 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200623140124.GR22291@mcvoy.com> Message-ID: <79609B26-AA5C-4F73-98DE-9353CC8B2078@cfcl.com> I recall listening to an interesting survey talk on DFS (pronounced Doofus) variants at a USENIX ATC conference in the 80's. Someone here may be able to recall the specifics... -r From robpike at gmail.com Wed Jun 24 06:38:53 2020 From: robpike at gmail.com (Rob Pike) Date: Wed, 24 Jun 2020 06:38:53 +1000 Subject: [TUHS] VFS prior to 1984 In-Reply-To: References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200623140124.GR22291@mcvoy.com> Message-ID: For my taste, the various Unix file system switches that I've seen are too firmly tied to the idea of blocks and disks and all that, making them less flexible than they should have been. That's why the Plan 9 version is about names and byte streams, to make it as general as possible. That's one of the reasons the possibilities of the file system approach to data has not reached its potential in Unix. -rob On Wed, Jun 24, 2020 at 1:14 AM Clem Cole wrote: > I agree with Larry with his observations. The only thing I would add is > that Popek and Walker had a file system in the UCLA Locus distributed > system for the 11/70 that was published in SIGOPS in late 1970s. In fact, > Gerry came up with the idea after a sabbatical @ PARC. But it was not > formalized like NFS as a separate (layered) FS, it was just part of the > basic Locus FS. > > Peter Weinberger created the file system switch (FSS) for V8 and in the > UNIX world, it was first. Dave Arnovitz used it for RFS in System V, > while Perry Flynn and I used it for EFS, both of us after talking > to Peter. Rusty and team did the VFS layer Sun @ (Larry will have to tell > you who actually built it, I never knew). Rusty and I both published > papers in the '85 Summer USENIX (NFS and EFS were contemporaries -- the > difference is that Sun gave away NFS and Masscomp was not smart enough to > do that). > > I talked to Steve Kleinman extensively about VFS at one point, and I'm > pretty sure he and the rest of the Sun guys had talked to the PARC folks > who after Gerry went back to UCLA started working on a DFS. The idea of > the state-less (idempotent) file system RPC that NFS used based on stuff > PARC did. But I'm not sure PARC had anything like the FSS or the VFS > layer. Peter, Dave, and I used a stateful scheme because we chose to have > full UNIX FS semantics, which NFS did not. In the end, early NFS was > notorious for putting 'holes' in the files because of the automatic seek in > every operation and errors not coming until close(2) time. > > EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it > was not nearly as flexible as what Sun built [which had a crude interface > generator], although until years later when Mike Leibensger built PIG (the > Paceline Interface Generator) NFS RPC was always a PITA and not much better > in practice than what we had at Masscomp. > > In fact, the point of the EFS paper was it's all about the recovery when > there is a failure/error. If you read his paper, Rusty's point was who > cares if there is an error (I've always felt vindicated that while I lost > the war, over time everyone came to our way of thinking and now NFS > V4 looks a whole lot like EFS did). > > Having a DFS as a feature was an incredible advance and proved we needed > something (and it needed to be standard in all systems). NFS really lead > the market with that advance, although it sort of took a few years to make > it really good. The fact is that others had the same idea before or at > least contemporary with it. > > That said, and to give the NFS team* a huge amount of credit *(and great > applause), the VFS layer was better thought out than the FSS and in fact > made it possible to add a lot of different file systems into UNIX later. > FSS was much more ad hoc. > > At LCC, when we built VPROC for TNC a few years later, we used some of the > same ideas from VFS and of course used VFS for the file system layer since > TNC had to have full POSIX semantics. (It's a shame VPROC never caught on > the way VFS did). > > Clem > > > On Tue, Jun 23, 2020 at 10:02 AM Larry McVoy wrote: > >> I like to claim credit for creating the ChangeSet concept, the grouping >> of a set of deltas across a number of files. But I wasn't the first. >> Back when dejanews was a thing and you could search usenet posts in a >> date range, if you searched before I started talking about ChangeSets, >> you could find 6, count 'em, 6 hits. There was a really obscure system >> called Aide De Camp, that had a similar concept. >> >> But if you searched after I started talking about them you would see >> millions of hits. >> >> So I didn't invent the concept but I sure as heck made the world >> understand the concept. >> >> I suspect Sun could be in a similar position. The VFS concept is >> pretty sweet so there might have been someone before Sun. I'll >> long odds that if there was, it didn't gain traction until Sun >> did it. >> >> If there is nothing that predates it, then the inspiration was >> almost certainly the device driver interface. One interface, >> many devices. VFS is the same. >> >> On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote: >> > When googling for File System Switch or Virtual File System most >> sources mention Sun NFS and SysVr3 as the earliest implementations. Some >> sources mention 8th Edition. >> > >> > I did a (short) search on FSS/VFS in earlier, non-Unix OS???s (Tenex, >> Multics, CTSS, etc.), but none of those seem to have had a comparable >> concept. >> > >> > Does anybody recall prior art (prior to 1984) in this area? >> > >> > Paul >> >> -- >> --- >> Larry McVoy lm at mcvoy.com >> http://www.mcvoy.com/lm >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Wed Jun 24 06:57:32 2020 From: lm at mcvoy.com (Larry McVoy) Date: Tue, 23 Jun 2020 13:57:32 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200623140124.GR22291@mcvoy.com> Message-ID: <20200623205732.GM14302@mcvoy.com> The SunOS one hid the block stuff pretty well. It had mmap(). Where it got hairy when I was working on it was block size 4k vs page size 8k. And frags, I hate frags. On Wed, Jun 24, 2020 at 06:38:53AM +1000, Rob Pike wrote: > For my taste, the various Unix file system switches that I've seen are too > firmly tied to the idea of blocks and disks and all that, making them less > flexible than they should have been. That's why the Plan 9 version is about > names and byte streams, to make it as general as possible. > > That's one of the reasons the possibilities of the file system approach to > data has not reached its potential in Unix. > > -rob > > > On Wed, Jun 24, 2020 at 1:14 AM Clem Cole wrote: > > > I agree with Larry with his observations. The only thing I would add is > > that Popek and Walker had a file system in the UCLA Locus distributed > > system for the 11/70 that was published in SIGOPS in late 1970s. In fact, > > Gerry came up with the idea after a sabbatical @ PARC. But it was not > > formalized like NFS as a separate (layered) FS, it was just part of the > > basic Locus FS. > > > > Peter Weinberger created the file system switch (FSS) for V8 and in the > > UNIX world, it was first. Dave Arnovitz used it for RFS in System V, > > while Perry Flynn and I used it for EFS, both of us after talking > > to Peter. Rusty and team did the VFS layer Sun @ (Larry will have to tell > > you who actually built it, I never knew). Rusty and I both published > > papers in the '85 Summer USENIX (NFS and EFS were contemporaries -- the > > difference is that Sun gave away NFS and Masscomp was not smart enough to > > do that). > > > > I talked to Steve Kleinman extensively about VFS at one point, and I'm > > pretty sure he and the rest of the Sun guys had talked to the PARC folks > > who after Gerry went back to UCLA started working on a DFS. The idea of > > the state-less (idempotent) file system RPC that NFS used based on stuff > > PARC did. But I'm not sure PARC had anything like the FSS or the VFS > > layer. Peter, Dave, and I used a stateful scheme because we chose to have > > full UNIX FS semantics, which NFS did not. In the end, early NFS was > > notorious for putting 'holes' in the files because of the automatic seek in > > every operation and errors not coming until close(2) time. > > > > EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it > > was not nearly as flexible as what Sun built [which had a crude interface > > generator], although until years later when Mike Leibensger built PIG (the > > Paceline Interface Generator) NFS RPC was always a PITA and not much better > > in practice than what we had at Masscomp. > > > > In fact, the point of the EFS paper was it's all about the recovery when > > there is a failure/error. If you read his paper, Rusty's point was who > > cares if there is an error (I've always felt vindicated that while I lost > > the war, over time everyone came to our way of thinking and now NFS > > V4 looks a whole lot like EFS did). > > > > Having a DFS as a feature was an incredible advance and proved we needed > > something (and it needed to be standard in all systems). NFS really lead > > the market with that advance, although it sort of took a few years to make > > it really good. The fact is that others had the same idea before or at > > least contemporary with it. > > > > That said, and to give the NFS team* a huge amount of credit *(and great > > applause), the VFS layer was better thought out than the FSS and in fact > > made it possible to add a lot of different file systems into UNIX later. > > FSS was much more ad hoc. > > > > At LCC, when we built VPROC for TNC a few years later, we used some of the > > same ideas from VFS and of course used VFS for the file system layer since > > TNC had to have full POSIX semantics. (It's a shame VPROC never caught on > > the way VFS did). > > > > Clem > > > > > > On Tue, Jun 23, 2020 at 10:02 AM Larry McVoy wrote: > > > >> I like to claim credit for creating the ChangeSet concept, the grouping > >> of a set of deltas across a number of files. But I wasn't the first. > >> Back when dejanews was a thing and you could search usenet posts in a > >> date range, if you searched before I started talking about ChangeSets, > >> you could find 6, count 'em, 6 hits. There was a really obscure system > >> called Aide De Camp, that had a similar concept. > >> > >> But if you searched after I started talking about them you would see > >> millions of hits. > >> > >> So I didn't invent the concept but I sure as heck made the world > >> understand the concept. > >> > >> I suspect Sun could be in a similar position. The VFS concept is > >> pretty sweet so there might have been someone before Sun. I'll > >> long odds that if there was, it didn't gain traction until Sun > >> did it. > >> > >> If there is nothing that predates it, then the inspiration was > >> almost certainly the device driver interface. One interface, > >> many devices. VFS is the same. > >> > >> On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote: > >> > When googling for File System Switch or Virtual File System most > >> sources mention Sun NFS and SysVr3 as the earliest implementations. Some > >> sources mention 8th Edition. > >> > > >> > I did a (short) search on FSS/VFS in earlier, non-Unix OS???s (Tenex, > >> Multics, CTSS, etc.), but none of those seem to have had a comparable > >> concept. > >> > > >> > Does anybody recall prior art (prior to 1984) in this area? > >> > > >> > Paul > >> > >> -- > >> --- > >> Larry McVoy lm at mcvoy.com > >> http://www.mcvoy.com/lm > >> > > -- --- Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm From norman at oclsc.org Wed Jun 24 08:17:14 2020 From: norman at oclsc.org (Norman Wilson) Date: Tue, 23 Jun 2020 18:17:14 -0400 Subject: [TUHS] VFS prior to 1984 Message-ID: <1592950638.2831.for-standards-violators@oclsc.org> Rob Pike: For my taste, the various Unix file system switches that I've seen are too firmly tied to the idea of blocks and disks and all that, making them less flexible than they should have been. That's why the Plan 9 version is about names and byte streams, to make it as general as possible. ===== The only file-system switch I know well is that in the later Research systems. It has nothing to do with disks or blocks. It is about names and inodes: turn name to struct inode, read arbitrary chunk of data, write arbitrary chunk of data, create file, unlink file, abominable ill-defined system call that just wouldn't die erm I mean ioctl, and so on. It was certainly a cheap hack (as its author cheerfully admitted) but it really was about I/O operations, not about simulating disks. That's why it so easily supported /proc (the one that was just about processes, not the misnamed Linux one that is really about all things in the kernel so you don't need /dev/kmem). I don't know a lot about later VFSes like those in SunOS or Linux or the BSDs. Blockiness might well have crept in in support of memory-mapped I/O or in collaboration with the buffer-cache implementation. The Research version was never used directly to support different on-disk file system formats (we did a different pjw cheap hack for the one case that mattered in the kernel, and used the idea later developed more by FUSE for other cases because speed didn't matter). Norman Wilson Toronto ON From dave at horsfall.org Wed Jun 24 08:24:30 2020 From: dave at horsfall.org (Dave Horsfall) Date: Wed, 24 Jun 2020 08:24:30 +1000 (EST) Subject: [TUHS] VFS prior to 1984 In-Reply-To: <1592950638.2831.for-standards-violators@oclsc.org> References: <1592950638.2831.for-standards-violators@oclsc.org> Message-ID: On Tue, 23 Jun 2020, Norman Wilson wrote: > The only file-system switch I know well is that in the later Research > systems. It has nothing to do with disks or blocks. It is about names > and inodes: turn name to struct inode, read arbitrary chunk of data, > write arbitrary chunk of data, create file, unlink file, abominable > ill-defined system call that just wouldn't die erm I mean ioctl, and so > on. Ioctl: the Swiss Army knife of system calls. I thought it was a neat idea when it arrived (much better then those primitive stty/gtty calls) but now... -- Dave From imp at bsdimp.com Wed Jun 24 14:09:23 2020 From: imp at bsdimp.com (Warner Losh) Date: Tue, 23 Jun 2020 22:09:23 -0600 Subject: [TUHS] VFS prior to 1984 In-Reply-To: References: <1592950638.2831.for-standards-violators@oclsc.org> Message-ID: On Tue, Jun 23, 2020, 4:25 PM Dave Horsfall wrote: > On Tue, 23 Jun 2020, Norman Wilson wrote: > > > The only file-system switch I know well is that in the later Research > > systems. It has nothing to do with disks or blocks. It is about names > > and inodes: turn name to struct inode, read arbitrary chunk of data, > > write arbitrary chunk of data, create file, unlink file, abominable > > ill-defined system call that just wouldn't die erm I mean ioctl, and so > > on. > > Ioctl: the Swiss Army knife of system calls. I thought it was a neat idea > when it arrived (much better then those primitive stty/gtty calls) but > now... > More like the swiss army chainsaw with engine trouble and a dull blade.... Warner -- Dave > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave at horsfall.org Wed Jun 24 15:10:26 2020 From: dave at horsfall.org (Dave Horsfall) Date: Wed, 24 Jun 2020 15:10:26 +1000 (EST) Subject: [TUHS] VFS prior to 1984 In-Reply-To: References: <1592950638.2831.for-standards-violators@oclsc.org> Message-ID: On Tue, 23 Jun 2020, Warner Losh wrote: > > Ioctl: the Swiss Army knife of system calls.  I thought it was a neat > > idea when it arrived (much better then those primitive stty/gtty > > calls) but now... > > More like the swiss army chainsaw with engine trouble and a dull > blade.... (Slowly drifting off-topic (as all mailing lists do) so perhaps a new thread would be better.) Love it! On the other hand though, if an extra service is required of the OS and you don't want to extend ioctl(), then that means Yet Another System Call (tm). I really like the idea of /proc, so if there's an equivalent for ioctl() then I'd love to see it; /sys would be an obvious choice, but it's already taken (Harris' Lament). No doubt, someone will now chime in and say that there's an existing mechanism involving shared memory or something... My Unix-fu is getting rather long in the tooth now (just like me). -- Dave From arnold at skeeve.com Wed Jun 24 15:14:01 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Tue, 23 Jun 2020 23:14:01 -0600 Subject: [TUHS] VFS prior to 1984 In-Reply-To: References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200623140124.GR22291@mcvoy.com> Message-ID: <202006240514.05O5E1u9008730@freefriends.org> Rob Pike wrote: > For my taste, the various Unix file system switches that I've seen are too > firmly tied to the idea of blocks and disks and all that, making them less > flexible than they should have been. That's why the Plan 9 version is about > names and byte streams, to make it as general as possible. > > That's one of the reasons the possibilities of the file system approach to > data has not reached its potential in Unix. > > -rob Well, it's a progression, isn't it? As ideas are tried out, they are used and then refined, and used and refined further. Multics did that for its predecessors, Unix did that for Multics, and Plan 9 did that for Unix. The key difference in Unix and Plan 9 from others being the continual goal towards refinement through simplification. Arnold From bakul at iitbombay.org Wed Jun 24 15:33:56 2020 From: bakul at iitbombay.org (Bakul Shah) Date: Tue, 23 Jun 2020 22:33:56 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: References: <1592950638.2831.for-standards-violators@oclsc.org> Message-ID: On Jun 23, 2020, at 10:10 PM, Dave Horsfall wrote: > > On Tue, 23 Jun 2020, Warner Losh wrote: > >> > Ioctl: the Swiss Army knife of system calls. I thought it was a neat > idea when it arrived (much better then those primitive stty/gtty > calls) but now... >> More like the swiss army chainsaw with engine trouble and a dull blade.... > > (Slowly drifting off-topic (as all mailing lists do) so perhaps a new thread would be better.) > > Love it! On the other hand though, if an extra service is required of the OS and you don't want to extend ioctl(), then that means Yet Another System Call (tm). I really like the idea of /proc, so if there's an equivalent for ioctl() then I'd love to see it; /sys would be an obvious choice, but it's already taken (Harris' Lament). > > No doubt, someone will now chime in and say that there's an existing mechanism involving shared memory or something... My Unix-fu is getting rather long in the tooth now (just like me). plan9 has ctl devices that accept commands as ascii strings and mostly return results as ascii strings too. Very handy. For example if you have a optical disk drive you can send commands via its ctl device. There is no technical reason why this model can't be adapted in Unix. The fact is that the Unix byte/block read/write model goes only so far. Every device has its own peculiarities that need to be queried or controlled. And then there are controllers or USB device treees where you can hang useful devices at varius places. Bottom line: while dealing with real devices you can push this complexity around. You can't make it disappear. At one point a friend and I were looking at building a product using plan9 and one of the things I wanted to do was to make sure that the "ctl" file for every device responded to a "help" message that yielded one line per command and "help " for more details. That way you can discover their capabilities at runtime. From andreww591 at gmail.com Wed Jun 24 19:10:12 2020 From: andreww591 at gmail.com (Andrew Warkentin) Date: Wed, 24 Jun 2020 03:10:12 -0600 Subject: [TUHS] VFS prior to 1984 In-Reply-To: References: <1592950638.2831.for-standards-violators@oclsc.org> Message-ID: On 6/23/20, Bakul Shah wrote: > > The fact is that the Unix byte/block read/write model goes only so far. > Every device has its own peculiarities that need to be queried or > controlled. And then there are controllers or USB device treees where > you can hang useful devices at varius places. Bottom line: while dealing > with real devices you can push this complexity around. You can't make it > disappear. At one point a friend and I were looking at building a > product using plan9 and one of the things I wanted to do was to make > sure that the "ctl" file for every device responded to a "help" message > that yielded one line per command and "help " for more details. > That way you can discover their capabilities at runtime. > Yes, complexity is unavoidable. IMO, the reason for going with a purely file-oriented architecture isn't to make all user-visible APIs use normal file calls, even though they are all going to be implemented in terms of files (there's definitely a place for libraries that provide a friendlier API on top of the file-based transport). "Everything is a file" should not mean "all APIs are visibly file-based". The real advantages of a pure file-oriented architecture I can think of off the top of my head are basically: All security for all services in the system is enforced through a single mechanism (rather than having many security primitives like modern conventional OSes have) On a microkernel, a purely stream-oriented IPC transport layer will have better performance when dealing with bulk unstructured data than a structured RPC transport layer with dynamic marshalling invoked on every message (for servers that need RPC with dynamic marshalling, the transport layer can easily support a type of file that preserves message boundaries but otherwise has standard Unix semantics) A purely stream-oriented IPC transport layer is simpler than one with dynamic marshalling and has less attack surface All services in the system can be extended, overridden, and exported through the same mechanism (this includes making all parts of a purely file-oriented OS network transparent for basically free) In some cases a higher-level service that extends another one doesn't have to fully intercept all parts of the underlying interface (e.g. if the underlying interface has multiple files and only some need to be overridden, or possibly if some messages are treated as opaque) From jnc at mercury.lcs.mit.edu Thu Jun 25 00:31:29 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Wed, 24 Jun 2020 10:31:29 -0400 (EDT) Subject: [TUHS] VFS prior to 1984 Message-ID: <20200624143129.1913318C0BD@mercury.lcs.mit.edu> > From: Dave Horsfall > Ioctl: the Swiss Army knife of system calls. I thought it was a neat > idea when it arrived (much better then those primitive stty/gtty calls) > but now... Like they say, when the only tool you have is a hammer... Better syntax than stty/gtty, maybe but I'm not sure the semantics are that much better. The problem is that, especially with devices, what the I/O commands do is so widely varied that it's hard to fit them all under a unified umbrella. Maybe some (e.g. asynchronous I/O), but not all. Noel From jruohonen at iki.fi Thu Jun 25 00:39:20 2020 From: jruohonen at iki.fi (Jukka Ruohonen) Date: Wed, 24 Jun 2020 17:39:20 +0300 Subject: [TUHS] About the history of cfree() In-Reply-To: <98DC6210-A6ED-4E51-8EF8-085CB553D966@planet.nl> References: <98DC6210-A6ED-4E51-8EF8-085CB553D966@planet.nl> Message-ID: <20200624143920.GA8127@sartre> Hello, does anyone remember details on how the cfree() function ended to of various systems? >From the only available documentation I could easily find [1], I'd reckon it came from SCO's malloc implementation that tried to conform to Intel's i386 iBCS. But almost all other systems wrapped it to free(). Interestingly, it seems that also SunOS 4.0 (1988) used the free()-wrapper. So was the 3-parameter cfree() only used in SCO XENIX 386 (1987) and the like [2]? Anyways, a small but interesting detail, and I suppose a good example on the developments that led to ANSI C. - Jukka [1] https://man7.org/linux/man-pages/man3/cfree.3.html [2] https://www.landley.net/history/mirror/unix/scohistory.html From clemc at ccc.com Thu Jun 25 01:21:41 2020 From: clemc at ccc.com (Clem Cole) Date: Wed, 24 Jun 2020 11:21:41 -0400 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <20200624143129.1913318C0BD@mercury.lcs.mit.edu> References: <20200624143129.1913318C0BD@mercury.lcs.mit.edu> Message-ID: On Wed, Jun 24, 2020 at 10:32 AM Noel Chiappa wrote: > > From: Dave Horsfall > > > Ioctl: the Swiss Army knife of system calls. I thought it was a neat > > idea when it arrived (much better then those primitive stty/gtty > calls) > > but now... > > Like they say, when the only tool you have is a hammer... > > Better syntax than stty/gtty, maybe but I'm not sure the semantics are that > much better. The problem is that, especially with devices, what the I/O > commands do is so widely varied that it's hard to fit them all under a > unified umbrella. Maybe some (e.g. asynchronous I/O), but not all. > I remember the change from stty/gtty which was peculiar to tty's in v6 to ioctl in v7 and thinking it was a nice clean generalization. Then I realized it had at least two big issues -- no concept of direction nor the size of the container being moved. So, then I see it start to get abused and I said -- oh my -- that was clearly not well thought out. Plan 9's solution of the CTL file with an ASCII interface, I thought was cleaner. IIRC TRIX had a controller interface too, but I've forgotten the details now. Here is the problem.... if you make everything a byte stream (which is a great fundamental way of building up a small cooperating program with a common interface), how do you handle the mechanisms to modify the stream in a manner than make sense (control if you will) in an out-of-band manner, but not end up with a fisherman's stew of different and special things (access methods, frameworks, *et al*) that are peculiar to that specific stream? The problem with stty/getty was is was specific to TTY's. It was a fine OOB scheme, but did not make sense for say changing parameters on other devices (tape as an example). The tape solution was different names, which worked there because you were not supposed to change things like density mid tape write. Other peripherals worked differently. ioctl became the only way out, as Noel points out. So ... this begs the question? Does the ctl file idea work better than ioctl? You still have to know what the ctl interface is for that device? It is ASCII which is nice and huge improvement over the binary command encoding crap. Clem -------------- next part -------------- An HTML attachment was scrubbed... URL: From ality at pbrane.org Thu Jun 25 02:51:07 2020 From: ality at pbrane.org (Anthony Martin) Date: Wed, 24 Jun 2020 09:51:07 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> Message-ID: <20200624165107.GA5737@alice> Paul Ruizendaal once said: > When googling for File System Switch or Virtual File System most sources > mention Sun NFS and SysVr3 as the earliest implementations. Some sources > mention 8th Edition. > > I did a (short) search on FSS/VFS in earlier, non-Unix OS’s (Tenex, > Multics, CTSS, etc.), but none of those seem to have had a comparable > concept. > > Does anybody recall prior art (prior to 1984) in this area? The concept of the "remote inode" was introduced in: Luderer, G. W., Che, H., Haggerty, J. P., Kirslis, P. A., & Marshall, W. T. (1981). A distributed UNIX system based on a virtual circuit switch. ACM SIGOPS Operating Systems Review, 15(5), 160-168. Cheers, Anthony From ality at pbrane.org Thu Jun 25 03:31:49 2020 From: ality at pbrane.org (Anthony Martin) Date: Wed, 24 Jun 2020 10:31:49 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <20200624165107.GA5737@alice> References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200624165107.GA5737@alice> Message-ID: <20200624173149.GA58805@alice> The only other prior art I know of is Toda, I. W. A. O. (1980). DCNA higher level protocols. IEEE Transactions on Communications, 28(4), 575-584. wherein they state the following: A virtual file system, from the viewpoint of application programs on a computer, models the file system functions of other computers. This is done in the same way as with virtual terminals, a virtual file system consists of a UPP having virtual files (VF), and a UCP which executes virtual file system protocols. I'd be interested if you find anything earlier. Anthony From jnc at mercury.lcs.mit.edu Thu Jun 25 03:51:49 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Wed, 24 Jun 2020 13:51:49 -0400 (EDT) Subject: [TUHS] VFS prior to 1984 Message-ID: <20200624175149.7947118C0BC@mercury.lcs.mit.edu> > From: Anthony Martin > wherein they state the following: > A virtual file system, from the viewpoint of application programs on a > computer, models the file system functions of other computers. This is > done in the same way as with virtual terminals, a virtual file system > consists of a UPP having virtual files (VF), and a UCP which executes > virtual file system protocols. > I'd be interested if you find anything earlier. MLDEV on ITS would, I think, fit under that description. I don't know if there's a paper on it; it's mid-70's. Noel From rich.salz at gmail.com Thu Jun 25 04:13:20 2020 From: rich.salz at gmail.com (Richard Salz) Date: Wed, 24 Jun 2020 14:13:20 -0400 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <20200624175149.7947118C0BC@mercury.lcs.mit.edu> References: <20200624175149.7947118C0BC@mercury.lcs.mit.edu> Message-ID: > MLDEV on ITS would, I think, fit under that description. > > I don't know if there's a paper on it; it's mid-70's. > > A web search for "its mldev" finds several things (mostly by Lars Brinkhoff it seems) , including https://github.com/larsbrinkhoff/mldev/blob/master/doc/mldev.protoc -------------- next part -------------- An HTML attachment was scrubbed... URL: From pnr at planet.nl Thu Jun 25 04:31:00 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Wed, 24 Jun 2020 20:31:00 +0200 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <20200624165107.GA5737@alice> References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200624165107.GA5737@alice> Message-ID: <17CD58F0-2474-4308-86BA-C8847D7ABA21@planet.nl> On 24 Jun 2020, at 18:51, Anthony Martin wrote: > > Paul Ruizendaal once said: >> When googling for File System Switch or Virtual File System most sources >> mention Sun NFS and SysVr3 as the earliest implementations. Some sources >> mention 8th Edition. >> >> I did a (short) search on FSS/VFS in earlier, non-Unix OS’s (Tenex, >> Multics, CTSS, etc.), but none of those seem to have had a comparable >> concept. >> >> Does anybody recall prior art (prior to 1984) in this area? > > The concept of the "remote inode" was introduced in: > > Luderer, G. W., Che, H., Haggerty, J. P., Kirslis, P. A., & Marshall, > W. T. (1981). A distributed UNIX system based on a virtual circuit > switch. ACM SIGOPS Operating Systems Review, 15(5), 160-168. > The only other prior art I know of is > > Toda, I. W. A. O. (1980). DCNA higher level protocols. IEEE > Transactions on Communications, 28(4), 575-584. > > wherein they state the following: > > A virtual file system, from the viewpoint of application programs on a > computer, models the file system functions of other computers. This is > done in the same way as with virtual terminals, a virtual file system > consists of a UPP having virtual files (VF), and a UCP which executes > virtual file system protocols. Thanks for those pointers: I had not thought to include distributed systems in my initial search. Clem also pointed out LOCUS in this area. It would seem that S/F-Unix (the Luderer paper) and LOCUS happened at about the same time and both have the concept of a remote inode. The Luderer paper has only a few details about implementation and besides describing the remote inode concept they mention: “Of 27 system call types related to files, 18 result in message traffic if remote files are involved. Of these, 10 contain a path name as an argument, and the remaining 8 refer to already opened files. Path names or the data read or written can be up to 64K bytes long. The structure of each message is a type code followed by type-dependent data.” In a way, the approach resembles the system call forwarding that Heinz Lycklama did for his 'satellite systems' work. I asked Bill Marshall (one of the S/F co-authors) about it a few months ago and he recalls that they ran into problems with corner case semantics in their implementation, and that Peter Weinberger later approached the issue afresh 'thinking about the hard cases first.' Bill does not have S/F-Unix source code, and I have not been able to contact Gottfried Luderer yet. I have not yet been able to look at the DCNA paper yet. > I'd be interested if you find anything earlier. I came across this 1979 paper by P.M. Lu from Bell Labs Naperville https://ieeexplore.ieee.org/document/762533 The abstract says: "RIDE (Resource-Sharing in a Distributed Environment) has been designed and implemented for the UNIX operating system to provide sharing of remote files, remote process invocation and interprocess communication in a distributed computing environment. The system is designed to support a uniform interface for both local and remote access in a network. The user programs can be executed in a single or multiple machine environment without any program modification.” I do not know what RIDE was and if its approach qualifies as a FSS precursor. Maybe it is like 'Unix United / Newcastle Connection' in its approach. It does seem an interesting paper in any case, as its front page includes: 'This system was designed to replace the network interface programs that previously operated on the SPIDER system.' From norman at oclsc.org Thu Jun 25 04:31:34 2020 From: norman at oclsc.org (Norman Wilson) Date: Wed, 24 Jun 2020 14:31:34 -0400 (EDT) Subject: [TUHS] VFS prior to 1984 Message-ID: <20200624183134.7B27E4422E@lignose.oclsc.org> Reaching outside of UNIX, RSX/11 used external supervisor-mode processes called ACPs (ancillary control processes) to implement file systems. I don't know exactly how they were plugged in, but I do know they were pluggable, so their interface must have constituted a file-system switch of some sort. RSX dates back into the 1970s. At some point in the latter part of the 1980s, Ralph Stamerjohn (a name instantly recognizable in the 16-bit DEC software world) gave a DECUS talk about implementing a remote file system through ACPs: a stub ACP on the client exporting RPCs over the network, a real one at the server end. I remember chatting with him about how that did and didn't resemble the way pjw had done it; interesting architectural comparison. Norman Wilson Toronto ON From lars at nocrew.org Thu Jun 25 04:46:01 2020 From: lars at nocrew.org (Lars Brinkhoff) Date: Wed, 24 Jun 2020 18:46:01 +0000 Subject: [TUHS] VFS prior to 1984 In-Reply-To: (Richard Salz's message of "Wed, 24 Jun 2020 14:13:20 -0400") References: <20200624175149.7947118C0BC@mercury.lcs.mit.edu> Message-ID: <7wsgek5ml2.fsf@junk.nocrew.org> Redirected to COFF, since this isn't very TUHS related. Richard Salz wrote: > Noel Chiappa wote: >> MLDEV on ITS would, I think, fit under that description. >> I don't know if there's a paper on it; it's mid-70's. I don't think there's anything like an academic paper. The earliest evidence I found for the MLDEV facility, or a predecessor, is from 1972: https://retrocomputing.stackexchange.com/questions/13709/what-are-some-early-network-file-systems > A web search for "its mldev" finds several things (mostly by Lars Brinkhoff > it seems) , including > https://github.com/larsbrinkhoff/mldev/blob/master/doc/mldev.protoc That's just a copy I made. From pnr at planet.nl Thu Jun 25 05:05:07 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Wed, 24 Jun 2020 21:05:07 +0200 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <20200624165107.GA5737@alice> References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200624165107.GA5737@alice> Message-ID: > I came across this 1979 paper by P.M. Lu from Bell Labs Naperville > Full paper is available here: https://www.computer.org/csdl/pds/api/csdl/proceedings/download-article/12OmNzd7c1v/pdf > I do not know what RIDE was and if its approach qualifies as a FSS precursor. Maybe it is like 'Unix United / Newcastle Connection' in its approach. Indeed it appears to have been like the (later) Newcastle Connection in its approach. From clemc at ccc.com Thu Jun 25 05:30:18 2020 From: clemc at ccc.com (Clem Cole) Date: Wed, 24 Jun 2020 15:30:18 -0400 Subject: [TUHS] VFS prior to 1984 In-Reply-To: References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200623140124.GR22291@mcvoy.com> Message-ID: Paul -- slow cache refill ... but I finally remembered some more of this enough to reply... I'm not sure if Larry remembers any of this and if he does, he might have more details from the Sun side. In the early/mid 90's, Dave Rosenthal was working on what become LOCKSS. I think he was at still at Sun at that point, but he would eventually move to Stanford fulltime, but he could have left Sun already - I've forgotten. The UCLA folks (some of Popek's guys) had built a stackable file system layer for BSD (which I think is in the contributed stuff in 4.4 - the original BSD union FS and the UCLA replication FS for FICUS used it IIRC). Some of the UMICH folks were messing with enhancements to NFS and it was an active time. Under the aegis of Unix International (UI), Gerry sponsored a pow-wow at LCC (in the LA office by the airport) to talk about how stacking and VFS layers. IIRC Kleinman from then NetApp was there. Besides' Dave, I've forgotten now who else from Sun was there. There must have been someone from USL and maybe Mical Gien of Chorus. But the whole discussion came back to having FSS/VFS and N different DFS's, PFS's and layers like replication or union, *etc* in everyone's kernel. Gerry saw there was going to be a 'market' for support of different filing schemes, but at that point, everything needed to be 'ported' because they were all different. But also this idea of stacking was appealing. If we could come up with something that clean, then maybe we could be more independent of the different physical file systems and we could 'push' behavior into the stack more invisible. But to do we needed to get some sort of agreement and all use the same thing. USL (Sun) was insisting that a Sun VFS++ -- one interface to rule them all - NFS was the standard already of course. This was not going over with the Academic (UCLA/BSD/UMICH folks IIRC), particularly because of the lack of a stacking property. Dave proposed an idea of a general interposition layer on the inode that would allow pushing and popping routines similar to the way STREAMS worked. Which we landed on -- everybody had to change a little. I had thought Dave implemented in an SVR4 kernel at some point and Chorus may have had it. There was talk of trying to get the spec out of UI and LCC was acting either as an arms merchant or the swiss, was trying to get IBM/DEC/HP/OSF to come along. I went looking in my files and so far have not been able to dig it up. But you might find something hunting around the LOCKSS work as Dave was the mastermind behind it. Eventually, is all blew up and I'm not sure what happened to either the specs or the code. Clem On Tue, Jun 23, 2020 at 4:39 PM Rob Pike wrote: > For my taste, the various Unix file system switches that I've seen are too > firmly tied to the idea of blocks and disks and all that, making them less > flexible than they should have been. That's why the Plan 9 version is about > names and byte streams, to make it as general as possible. > > That's one of the reasons the possibilities of the file system approach to > data has not reached its potential in Unix. > > -rob > > > On Wed, Jun 24, 2020 at 1:14 AM Clem Cole wrote: > >> I agree with Larry with his observations. The only thing I would add is >> that Popek and Walker had a file system in the UCLA Locus distributed >> system for the 11/70 that was published in SIGOPS in late 1970s. In fact, >> Gerry came up with the idea after a sabbatical @ PARC. But it was not >> formalized like NFS as a separate (layered) FS, it was just part of the >> basic Locus FS. >> >> Peter Weinberger created the file system switch (FSS) for V8 and in the >> UNIX world, it was first. Dave Arnovitz used it for RFS in System V, >> while Perry Flynn and I used it for EFS, both of us after talking >> to Peter. Rusty and team did the VFS layer Sun @ (Larry will have to tell >> you who actually built it, I never knew). Rusty and I both published >> papers in the '85 Summer USENIX (NFS and EFS were contemporaries -- the >> difference is that Sun gave away NFS and Masscomp was not smart enough to >> do that). >> >> I talked to Steve Kleinman extensively about VFS at one point, and I'm >> pretty sure he and the rest of the Sun guys had talked to the PARC folks >> who after Gerry went back to UCLA started working on a DFS. The idea of >> the state-less (idempotent) file system RPC that NFS used based on stuff >> PARC did. But I'm not sure PARC had anything like the FSS or the VFS >> layer. Peter, Dave, and I used a stateful scheme because we chose to have >> full UNIX FS semantics, which NFS did not. In the end, early NFS was >> notorious for putting 'holes' in the files because of the automatic seek in >> every operation and errors not coming until close(2) time. >> >> EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it >> was not nearly as flexible as what Sun built [which had a crude interface >> generator], although until years later when Mike Leibensger built PIG (the >> Paceline Interface Generator) NFS RPC was always a PITA and not much better >> in practice than what we had at Masscomp. >> >> In fact, the point of the EFS paper was it's all about the recovery when >> there is a failure/error. If you read his paper, Rusty's point was who >> cares if there is an error (I've always felt vindicated that while I lost >> the war, over time everyone came to our way of thinking and now NFS >> V4 looks a whole lot like EFS did). >> >> Having a DFS as a feature was an incredible advance and proved we needed >> something (and it needed to be standard in all systems). NFS really lead >> the market with that advance, although it sort of took a few years to make >> it really good. The fact is that others had the same idea before or at >> least contemporary with it. >> >> That said, and to give the NFS team* a huge amount of credit *(and great >> applause), the VFS layer was better thought out than the FSS and in fact >> made it possible to add a lot of different file systems into UNIX later. >> FSS was much more ad hoc. >> >> At LCC, when we built VPROC for TNC a few years later, we used some of >> the same ideas from VFS and of course used VFS for the file system layer >> since TNC had to have full POSIX semantics. (It's a shame VPROC never >> caught on the way VFS did). >> >> Clem >> >> >> On Tue, Jun 23, 2020 at 10:02 AM Larry McVoy wrote: >> >>> I like to claim credit for creating the ChangeSet concept, the grouping >>> of a set of deltas across a number of files. But I wasn't the first. >>> Back when dejanews was a thing and you could search usenet posts in a >>> date range, if you searched before I started talking about ChangeSets, >>> you could find 6, count 'em, 6 hits. There was a really obscure system >>> called Aide De Camp, that had a similar concept. >>> >>> But if you searched after I started talking about them you would see >>> millions of hits. >>> >>> So I didn't invent the concept but I sure as heck made the world >>> understand the concept. >>> >>> I suspect Sun could be in a similar position. The VFS concept is >>> pretty sweet so there might have been someone before Sun. I'll >>> long odds that if there was, it didn't gain traction until Sun >>> did it. >>> >>> If there is nothing that predates it, then the inspiration was >>> almost certainly the device driver interface. One interface, >>> many devices. VFS is the same. >>> >>> On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote: >>> > When googling for File System Switch or Virtual File System most >>> sources mention Sun NFS and SysVr3 as the earliest implementations. Some >>> sources mention 8th Edition. >>> > >>> > I did a (short) search on FSS/VFS in earlier, non-Unix OS???s (Tenex, >>> Multics, CTSS, etc.), but none of those seem to have had a comparable >>> concept. >>> > >>> > Does anybody recall prior art (prior to 1984) in this area? >>> > >>> > Paul >>> >>> -- >>> --- >>> Larry McVoy lm at mcvoy.com >>> http://www.mcvoy.com/lm >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Thu Jun 25 05:36:47 2020 From: lm at mcvoy.com (Larry McVoy) Date: Wed, 24 Jun 2020 12:36:47 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200623140124.GR22291@mcvoy.com> Message-ID: <20200624193647.GB14302@mcvoy.com> On Tue, Jun 23, 2020 at 11:12:40AM -0400, Clem Cole wrote: > Rusty and team did the VFS layer Sun @ (Larry will have to tell > you who actually built it, I never knew). Steve Kleiman is the only author on the VFS paper so I'm guessing that was him. Rusty was NFS. Joe Moran (sun!mojo if I remember correctly) did the VM implementation, it was co-designed by Joe, Shannon, and Gingell. Side story: part of the reason I wanted to go to Sun was because when I was a CS student at UW-Madison, Joe was a little older than me, and I sat and watched him do a BSD port to a 68K (I think) in a little more than 24 hours (sketchy on the time, I think that's right, almost positive it was more than 20 hours, it was a chunk of time). As a guy who wanted to be next to the metal, it was impressive. > In the end, early NFS was > notorious for putting 'holes' in the files because of the automatic seek in > every operation and errors not coming until close(2) time. You have no idea how many of those holes that 16 bit SCCS checksum has found (BitKeeper kept it). > EFS used an RPC and a RUDP layer that Perry and Alan Atlas built, but it > was not nearly as flexible as what Sun built [which had a crude interface > generator], although until years later when Mike Leibensger built PIG (the > Paceline Interface Generator) NFS RPC was always a PITA and not much better > in practice than what we had at Masscomp. Oh, I don't know if I agree. I took the RPC mechanism and vectorized it. I probably have the code somewhere. It was pretty neat, 10Mbit was the common networking and I did RPC Vector calls to the rstatd on literally every machine in the Mountain View campus. It was thousands of machines and it worked. The code split the RPC back into send and receive and the code had to start gathering up answers before it was finished sending the questions or the packets would be dropped. It worked surprisingly well. Ron Minnich and Gene Kim made an NFS client that worked as demo-ware. There is a paper about below, note that I'm co-author only because I wrote the RPC vector code, I believe Gene did the in kernel code, it was Ron and Gene's project. http://mcvoy.com/lm/papers/bigfoot.pdf > In fact, the point of the EFS paper was it's all about the recovery when > there is a failure/error. If you read his paper, Rusty's point was who > cares if there is an error (I've always felt vindicated that while I lost > the war, over time everyone came to our way of thinking and now NFS > V4 looks a whole lot like EFS did). Yeah, they were sort of cavalier about errors in public. The weird thing is Sun ran their entire network on Sun machines. Everything was NFSv2 when I got there, the routers were Sun workstations, the servers were Sun servers, the clients were Sun workstations. They ate their own dogfood and NFS just worked, it was pretty unheard of to hear about NFS issues other than performance tuning (stupid biods etc). > Having a DFS as a feature was an incredible advance and proved we needed > something (and it needed to be standard in all systems). NFS really lead > the market with that advance, although it sort of took a few years to make > it really good. The fact is that others had the same idea before or at > least contemporary with it. NFS was solid in v2 at Sun. I went to SGI and NFS was complete garbage. SGI's networking principle engineer told me, when I complained about NFS not being reliable, "That's what rcp is for". I was flabbergasted, the networking group generated $18M/year in NFS sales. It wasn't just SGI, every vender other Sun had a crappy NFS implementation. I'm surprised NFS didn't have a worse rep because while NFS worked on Suns, it really didn't work anywhere near as well elsewhere. > That said, and to give the NFS team* a huge amount of credit *(and great > applause), the VFS layer was better thought out than the FSS and in fact > made it possible to add a lot of different file systems into UNIX later. > FSS was much more ad hoc. The VFS layer was Steve, NFS was Rusty. > At LCC, when we built VPROC for TNC a few years later, we used some of the > same ideas from VFS and of course used VFS for the file system layer since > TNC had to have full POSIX semantics. (It's a shame VPROC never caught on > the way VFS did). VPROC was very cool and a requirement for any sort of distributed single system image OS. I agree it is a shame that it didn't catch on. --lm From dfawcus+lists-tuhs at employees.org Thu Jun 25 06:27:33 2020 From: dfawcus+lists-tuhs at employees.org (Derek Fawcus) Date: Wed, 24 Jun 2020 21:27:33 +0100 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> Message-ID: <20200624202733.GA96818@clarinet.employees.org> On Tue, Jun 23, 2020 at 11:09:57AM +0200, Paul Ruizendaal wrote: > I did a (short) search on FSS/VFS in earlier, non-Unix OS’s (Tenex, Multics, CTSS, etc.), but none of those seem to have had a comparable concept. > > Does anybody recall prior art (prior to 1984) in this area? Surely any OS, even non unix, which offered a remote FS in addition to a local FS implicitly implements something like a FSS/VFS? So we could look to CP/M (and CP/Net, CP/NOS), where this[1] seems to suggest a date between 1980 and 1982. An appendix to that paper has the CPM file system operations (such as they were) remotely available and listed with their codes for the messages. That said, another OS from DR (FlexOS) has a recognisable VFS, is written in C, but dates to early 1985. I'm not sure when the design dates from[3]. If one looks at this[2] article in Byte, each of the Resource Managers operates through a VFS layer, in terms of: #define I_GET 3 /* NET DISK CON MSC PIPE KERN */ #define I_SET 4 /* NET DISK CON MSC PIPE KERN */ #define I_LOOKUP 5 /* NET DISK CON MSC PIPE KERN */ #define I_OPEN 6 /* NET DISK CON MSC PIPE */ #define I_CLOSE 7 /* NET DISK CON MSC PIPE */ #define I_READ 8 /* NET DISK CON MSC PIPE */ #define I_WRITE 9 /* NET DISK CON MSC PIPE */ #define I_SEEK 10 /* NET DISK CON MSC PIPE */ #define I_CREATE 11 /* NET DISK CON PIPE */ #define I_DELETE 12 /* NET DISK CON PIPE */ Which can be seen in the public code for CDOS-68k. Not all RMs implement all calls, but the NET, DISK and PIPE sort of did. DF [1] https://www.autometer.de/unix4fun/z80pack/cpnet/cpnet.htm [2] http://tech-insider.org/personal-computers/research/1985/05.html https://archive.org/stream/byte-magazine-1985-05/1985_05_BYTE_10-05_Multiprocessing#page/n380/mode/1up/search/concurrent [3] The 80286 was released in '82, but I don't know when Intel started giving early info to customers. From dave at horsfall.org Thu Jun 25 07:08:24 2020 From: dave at horsfall.org (Dave Horsfall) Date: Thu, 25 Jun 2020 07:08:24 +1000 (EST) Subject: [TUHS] VFS prior to 1984 In-Reply-To: <202006240514.05O5E1u9008730@freefriends.org> References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200623140124.GR22291@mcvoy.com> <202006240514.05O5E1u9008730@freefriends.org> Message-ID: On Tue, 23 Jun 2020, arnold at skeeve.com wrote: > Well, it's a progression, isn't it? As ideas are tried out, they are > used and then refined, and used and refined further. Multics did that > for its predecessors, Unix did that for Multics, and Plan 9 did that for > Unix. The key difference in Unix and Plan 9 from others being the > continual goal towards refinement through simplification. "Make things as simple as possible, but no simpler." -- Dave From woods at robohack.ca Thu Jun 25 07:33:16 2020 From: woods at robohack.ca (Greg A. Woods) Date: Wed, 24 Jun 2020 14:33:16 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> Message-ID: At Tue, 23 Jun 2020 11:09:57 +0200, Paul Ruizendaal wrote: Subject: [TUHS] VFS prior to 1984 > > When googling for File System Switch or Virtual File System most > sources mention Sun NFS and SysVr3 as the earliest > implementations. Some sources mention 8th Edition. > > I did a (short) search on FSS/VFS in earlier, non-Unix OS’s (Tenex, > Multics, CASS, etc.), but none of those seem to have had a comparable > concept. > > Does anybody recall prior art (prior to 1984) in this area? Well, with regard to Multics, it had what were called "Device Interface Modules" (DIMs) for building device drivers to either hardware or pseudo-devices (an example of a pseudo-DIM is the File System Interface Module (FSIM) which associates a segment (file) in the filesystem (by path name) with a stream, i.e. it implements I/O redirection). DIMs had a well defined API, and as Organick said in his 1972 book: "The DIM converts a device independent request into a device dependent one." ... "The file-system interface DIM is used to make a segment look like an I/O device." A DIM also had a name, an identifier, and that's how you attached a "stream" to it: call attach (stream_name, DIM_name, device_name); All of those parameters are strings. As an aside I haven't found any evidence that DIM identifiers were ever listed in the filesystem, e.g. as Unix would have in "/dev", but that would be an obvious thing to do these days. Then they were just listed in the programmer's manual, and presumably user-written ones were added to the online documentation. Today such an idea would be "obvious". As far as I can remember Multics didn't really have the concept of a "mount point". All storage was single-level, i.e. segments (equivalent in some respects to inodes, but they are also actually the value of the segment register in the virtual memory hardware), and so files were either physically in memory or paged out on physical disk devices or similar, or even out on tape. Where they actually resided was entirely and permanently hidden from the user. What was called the "filesystem" was a form of database representing a hierarchical namespace which pointed at all the known segments (files) regardless of where they were actually stored. In terms of virtual filesystems for Multics, I think it would be (have been) relatively easy enough (i.e. with the addition of some APIs to the filesystem itself) to implement a way of "mounting" a pseudo-DIM into the filesystem namespace and thus being able to implement anything from the likes of /kern and /proc to remote filesystems, network directories, etc. I.e. create an inverse twin of the filesystem DIM that takes filesystem requests and turns them into back-end pseudo-DIM requests. Effectively this would be implementing a VFS for Multics. Indeed today, and perhaps any time after Apollo Domain was designed, it would be plainly obvious, and probably desirable, to do such a thing, at least for some kinds of concepts, such as /proc. Note though that I suspect the original Multics designers would have preferred it if remote filesystems, in particular, were also effectively invisible to the user, though I do not know how they might have approached the issue of remote segments not always being available, including indeed parts of the filesystem itself, i.e. directories and metadata. Perhaps in a similar way to how files migrated to backup storage were handled -- if you referenced a file that had been migrated offline then your process trapped to a retrieval process that involved prompting a human tape operator to load the appropriate backup tape(s). In fact Organick says the following about this idea of an inverse relationship between the filesystem and I/O subsystem in his 1972 book (p351,352): "We have just seen how the I/O system may behave as a customer of the file system, which supplies needed services. One might wonder if the reverse of those roles is ever true. That is, does the file system through its Page Control Module, when seeking to transfer a page of information to/from core [[memory]] and disk or tape storage, ever find itself to be a customer of the I/O system? Emphasis on objectives of modularity might cause one to guess the affirmative. In actual fact however, considerations of efficiency have dictated that paging I/O in Multics be treated with special-purpose I/O software that greatly streamlines the processor's task in initiating and controlling such I/O." So, at least early on, he appears to have denied the idea of the filesystem routing requests through a DIM or pseudo-DIM, though he did so on the basis that this would be done at the pager level and that doing so could be too inefficient to consider. However he doesn't seem to have considered, or at least does not mention in the 1972 book, the idea of a pseudo-DIM extending the filesystem namespace by implementing a mount point, which as I say would today be considered a quite obvious, desirable, and feasible task. In any case the idea of pseudo device drivers interacting with the filesystem, albeit the other way around, was firmly in place by 1965 (the file-system interface pseudo-DIM was described in the 1965 Multics I/O system paper). -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP Digital Signature URL: From athornton at gmail.com Thu Jun 25 10:45:38 2020 From: athornton at gmail.com (Adam Thornton) Date: Wed, 24 Jun 2020 17:45:38 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> Message-ID: On Wed, Jun 24, 2020 at 2:34 PM Greg A. Woods wrote: > > As far as I can remember Multics didn't really have the concept of a > "mount point". All storage was single-level, i.e. segments (equivalent > in some respects to inodes, but they are also actually the value of the > segment register in the virtual memory hardware), and so files were > either physically in memory or paged out on physical disk devices or > similar, or even out on tape. Where they actually resided was entirely > and permanently hidden from the user. What was called the "filesystem" > was a form of database representing a hierarchical namespace which > pointed at all the known segments (files) regardless of where they were > actually stored. > > Coming to it from a Unix perspective, it's like all storage (core, disk, tape) is mmap()ed. The segment-name database then is just an index relating symbolic names to particular memory locations. It all feels very upside-down to me, but that's probably because I grew up in Unix and never actually used a Multics system until I emulated one with dps8m. Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From gingell at computer.org Thu Jun 25 10:56:58 2020 From: gingell at computer.org (Rob Gingell) Date: Wed, 24 Jun 2020 17:56:58 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <17CD58F0-2474-4308-86BA-C8847D7ABA21@planet.nl> References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200624165107.GA5737@alice> <17CD58F0-2474-4308-86BA-C8847D7ABA21@planet.nl> Message-ID: <57bb4779-66e3-6501-c19c-0bc2afb8fbd7@computer.org> On 6/24/2020 11:31 AM, Paul Ruizendaal wrote: > I came across this 1979 paper by P.M. Lu from Bell Labs Naperville > https://ieeexplore.ieee.org/document/762533 > > The abstract says: > > "RIDE (Resource-Sharing in a Distributed Environment) ... RIDE seems very reminiscent of the RSEXEC (Resource Sharing EXEC) done on TENEX earlier in the 1970s. The implementation of RSEXEC involved encapsulation of a process fork by its parent rather than an object-oriented abstraction inserted behind an existing interface like VFS is. Both are functionally transparent to their users but are kind of going in opposite directions in implementation. As with other mechanisms that have been referenced in this thread, these illustrate the claim that "all problems in computer science can be solved by [adding] another layer of indirection". I don't know that that's absolutely true but I've certainly applied it a lot, and like a lot of things, excessive application can create more problems. (I never knew who was the first to utter the claim as I've heard it said by lots of people. However a quick check shows an attribution to David Wheeler in the form of the "Fundamental theorem of software engineering".) https://en.wikipedia.org/wiki/Fundamental_theorem_of_software_engineering RSEXEC was a pretty remarkable facility in action. It (approximately) was like having a command interpreter for the aggregated TENEX resources of the ARPAnet. It was at least partly intended to serve users of TIPs (the terminal interface processors used to access hosts) by letting them treat all TENEXes as a single resource and not worry about which one they got hosted onto "this time" with respect to their files, applications, etc., and importantly also not worry about the where their collaborators were hosted at the moment. A description of RSEXEC is at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.830.1403&rep=rep1&type=pdf The encapsulation mechanism was JSYS Traps, JSYS being the system call instruction on PDP-10s modified to run TENEX. A description of that is at: https://archive.org/details/bitsavers_bbntenexTh_1137399 In it's way, RSEXEC illustrated one characteristic of many excellent innovations in that when used as intended it was "deceptively unimpressive". As in: harder to demo. NFS and other things share this trait, and indeed early trade show demonstrations of NFS were frustrating (in those days, the engineers did booth duty). At first, demos reduced to typing "ls". Which, unsurprisingly, produced a list of files. It's not until you know how it was done that it's impressive. There can be a real utilitarian pleasure in things that, when shown to others, produce a "yeah, wasn't it supposed to do that?" reaction. Part of that aesthetic appreciation for me originated with RSEXEC. Which brings me to VFS and such. Earlier in this thread Clem Cole observed: On 6/23/2020 8:12 AM, Clem Cole wrote: > That said, and to give the NFS team_a huge amount of credit _(and great > applause), the VFS layer was better thought out than the FSS and in fact > made it possible to add a lot of different file systems into UNIX > later.  FSS was much more ad hoc. I think VFS and several related things are another instance of that aesthetic appreciation. [I'm not claiming any credit, the initial NFS work was just finishing up as I arrived at Sun. That work like that happened there was part of why I wanted to be there and so I share Clem's appreciation for the team.] NFS could have been implemented by sprinkling "if (network)" statements around a lot of code. A user or application program would not know any differently had it been done that way. (Though they would over time as the code became less maintainable with that implementation approach.) That VFS has the qualities leading to Clem's applause is I think due to a confluence of several things that likely hadn't occurred previously and thus entitle it to claim "first" in response to the thread-starting question. One of the motivators was the desire to have a network filing abstraction, and to replace "nd" for diskless workstations among other things. (That took a while, "nd" really hung on for various reasons.) That's occurring in an environment of heterogeneity and "open systems" thinking (which is a different thing from "open source"), and the desire to be more "of the network" in the product offerings. There was kind of a group-innate understanding that to be successful in introducing these capabilities that they had to occur largely "behind" the extant interfaces used by the nascent UNIX application market. To some extent Sun's market success was based on being deliberately "more vanilla than thou" with respect to APIs. The variations already extant were a market-fragmenting influence and thus didn't need an additional player doing more of it. To be "of the network" made NFS a "file system for the Network (that happens to look like UNIX)". The similarity and the known semantic differences lead some to look at it as "it failed to do UNIX" but it's not. The intent was "do the network, the network is NOT 'a' computer (irony alert), the fallacies of networking apply and we resolve the irreconcilable in favor of the network". (An intangible factor relates to the group that Sun formed, talented engineers, strikingly different people who shared good taste in design and the desire to do the work to find the "simple" in a problem. A different group of people with the same problem set wouldn't have automatically gotten the same or even similar answer, sometimes it's just in the air. Of course, a different group might have made an even better answer, "innovation happens elsewhere" is more true than the earlier-referenced layer of indirection principle.) Sometimes (mostly even), good taste is expressed in framing the problem appropriately. The "N" in NFS is deliberately not a "D". That's obvious from the resulting definition but earlier in the effort it was a choice. The intended distinction was that we weren't doing operating system work in order to transcend the network by submerging it in the abstractions of an OS. Rather, we wanted to build a network environment populated by systems which were of a piece with it. While not codified until much later, Deutsch's "Fallacies of Networking" were the controlling notions in a lot of thinking and influenced things that at first wouldn't seem to have anything to do with networking (e.g., msync()). However, it would be wrong to say we realized enough at the time to *know* that our goal was to build a system defined by a metric that it avoid all of the networking fallacies. We knew some of it, were incrementally hunting it, instincts were oriented that way. Had we been a research organization that might have given rise to an initiative that was more clearly defined. (But, I think we did a reasonable job of pursuing the ideas while making products that people who weren't thinking of such things bought. As it happens, a lot of FORTRAN users as has been noted here before. Regarding the earlier irony alert: to the extent that the evocative marketing slogan of "The Network is the Computer" had any technical underpinnings, it was an expression of this aim to design in favor of the network. It isn't really irony so much as consequence. The real irony is that when the slogan was first trotted out almost all of the company but particularly engineering barfed at it. I recall the marketing person(s) (individual contributors, not the executives later claiming credit) trying to push it receiving huge rations of abuse. But it stuck. And we came around once we looked beyond its technical deficiency as a literal statement and saw it as evocative of what we had been doing. They knew us better than we did. And, independent of any technical matter, it worked really well. 'Find the simple" is a useful admonition for marketing as well as engineering. From tytso at mit.edu Thu Jun 25 11:50:09 2020 From: tytso at mit.edu (Theodore Y. Ts'o) Date: Wed, 24 Jun 2020 21:50:09 -0400 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: <20200622224301.kdouj%steffen@sdaoden.eu> References: <7wsgewophe.fsf@junk.nocrew.org> <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> <20200622162406.GA48733@clarinet.employees.org> <0F78CB9D-870B-4008-B975-23756EFC6F83@iitbombay.org> <20200622224301.kdouj%steffen@sdaoden.eu> Message-ID: <20200625015009.GB4655@mit.edu> On Tue, Jun 23, 2020 at 12:43:01AM +0200, Steffen Nurpmeso wrote: > #?0|kent:steffen$ pkginfo -o `command -v col` > Package File > util-linux usr/bin/col > > states > > /* > * This command is deprecated. The utility is in maintenance mode, > * meaning we keep them in source tree for backward compatibility > * only. Do not waste time making this command better, unless the > * fix is about security or other very critical issue. > * > * See Documentation/deprecated.txt for more information. > */ I'll note that the Austin Group / The Open Group marked those commands as LEGACY[1] in the Single Unix Specification V2 in 1997: The utilities in the table below are marked LEGACY. Various factors may have contributed to the decision to class a utility LEGACY. Application writers should not use functionality marked LEGACY. If a migration path exists, advice is given to application developers regarding alternative means of obtaining similar functionality. This information may be found in the APPLICATION USAGE sections on the relevant pages. No requirement beyond that which was in effect at the time that these utilities were marked LEGACY shall be applied to these utilities. calendar cancel cc col cpio cu dircmp dis egrep fgrep line lint lpstat mail pack pcat pg spell sum tar unpack uulog uuname uupick uuto [1] https://pubs.opengroup.org/onlinepubs/7908799/xcu/intro.html#tag_001_003_003 The quote from the Single Unix Specification: "No requirement beyond that which was in effect at the time that these utilities were marked LEGACY shall be applied to these utilities." is not that different from "Do not waste time making this command better, unless the fix is about security or other very critical issue." And while I'm sure the Linux haters will be happy to try to blame Linux for the decision to declare pg, col, et.al as "legacy", in 1997 The Open Group was hardly filled with Linux developers; '97 predates IBM and Oracle declaring their support for Linux (1998), the publication of "The Cathedral and the Bazaar (also 1998), the start of the GNOME project (1999), Red Hat Enterprise Linux (first released in 2002), and Ubuntu Linux (2004). - Ted P.S. Of course, the fact that the The Open Group tried to convince the world to stop using tar and cpio in favor to pax seems to be a strong indication that they forgot the lesson of King Canute. :-) From rdm at cfcl.com Thu Jun 25 14:15:32 2020 From: rdm at cfcl.com (Rich Morin) Date: Wed, 24 Jun 2020 21:15:32 -0700 Subject: [TUHS] Oh, things were very different. In-Reply-To: <57bb4779-66e3-6501-c19c-0bc2afb8fbd7@computer.org> References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200624165107.GA5737@alice> <17CD58F0-2474-4308-86BA-C8847D7ABA21@planet.nl> <57bb4779-66e3-6501-c19c-0bc2afb8fbd7@computer.org> Message-ID: https://imgs.xkcd.com/comics/old_days_2.png -r From arnold at skeeve.com Thu Jun 25 16:22:05 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Thu, 25 Jun 2020 00:22:05 -0600 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <20200624183134.7B27E4422E@lignose.oclsc.org> References: <20200624183134.7B27E4422E@lignose.oclsc.org> Message-ID: <202006250622.05P6M5ws003909@freefriends.org> Pr1me had remote filesystems circa 1981 at least, maybe even earlier. It was quite transparent, something equivalent to //node/path/to/file. I only ever used Primos through the Georgia Tech Software Tools Subsytem which made the filesystem look Unix-y. I don't know any more details. HTH, Arnold From gingell at computer.org Thu Jun 25 16:52:33 2020 From: gingell at computer.org (Rob Gingell) Date: Wed, 24 Jun 2020 23:52:33 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <20200624193647.GB14302@mcvoy.com> References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200623140124.GR22291@mcvoy.com> <20200624193647.GB14302@mcvoy.com> Message-ID: <99278109-df44-9ec3-df7e-fb1f24fdb0b5@computer.org> On 6/24/2020 12:36 PM, Larry McVoy wrote: > On Tue, Jun 23, 2020 at 11:12:40AM -0400, Clem Cole wrote: Larry's enumeration of NFS/VFS players was good. I'd add Bob Lyon who led the NFS team (and who later founded Legato along with Rusty, Joe Moran and Jon Kepecs -- all except Bob being UW alumni.) Bob came to Sun via PARC and brought with him a number of the ideas that Clem had referenced earlier in this thread about RPC, etc. Although he's on few papers his influence on defining and building NFS etc. was essential. Another important contributor was David Goldberg, for building the YP services that, while functionally separate, enabled the administrative and deployment environment that helped make NFS useful. David also wrote the floating-point paper in the appendix to the Hennessy-Patterson architecture textbook. From stewart at serissa.com Fri Jun 26 01:45:24 2020 From: stewart at serissa.com (Lawrence Stewart) Date: Thu, 25 Jun 2020 11:45:24 -0400 Subject: [TUHS] Oh, things were very different. In-Reply-To: References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200624165107.GA5737@alice> <17CD58F0-2474-4308-86BA-C8847D7ABA21@planet.nl> <57bb4779-66e3-6501-c19c-0bc2afb8fbd7@computer.org> Message-ID: <7EC6B8F7-133A-455A-93E8-464FF368019A@serissa.com> > On 2020, Jun 25, at 12:15 AM, Rich Morin wrote: > > https://imgs.xkcd.com/comics/old_days_2.png > > -r > The trick to these is to include one nugget of truth. If anyone asks, I’m saying the sweater part is absolutely true. After all, BITNET actually did work by connecting a virtual card punch to a remote virtual card reader. -L From imp at bsdimp.com Fri Jun 26 02:24:42 2020 From: imp at bsdimp.com (Warner Losh) Date: Thu, 25 Jun 2020 10:24:42 -0600 Subject: [TUHS] Oh, things were very different. In-Reply-To: <7EC6B8F7-133A-455A-93E8-464FF368019A@serissa.com> References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200624165107.GA5737@alice> <17CD58F0-2474-4308-86BA-C8847D7ABA21@planet.nl> <57bb4779-66e3-6501-c19c-0bc2afb8fbd7@computer.org> <7EC6B8F7-133A-455A-93E8-464FF368019A@serissa.com> Message-ID: On Thu, Jun 25, 2020 at 9:52 AM Lawrence Stewart wrote: > > > > On 2020, Jun 25, at 12:15 AM, Rich Morin wrote: > > > > https://imgs.xkcd.com/comics/old_days_2.png > > > > -r > > > > The trick to these is to include one nugget of truth. If anyone asks, I’m > saying the sweater part is absolutely true. > > After all, BITNET actually did work by connecting a virtual card punch to > a remote virtual card reader. > I have thousands of virtual sweaters in a drawer upstairs. They were what inspired virtual machines and the cloud after all... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From krewat at kilonet.net Fri Jun 26 03:57:45 2020 From: krewat at kilonet.net (Arthur Krewat) Date: Thu, 25 Jun 2020 13:57:45 -0400 Subject: [TUHS] Oh, things were very different. In-Reply-To: <7EC6B8F7-133A-455A-93E8-464FF368019A@serissa.com> References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200624165107.GA5737@alice> <17CD58F0-2474-4308-86BA-C8847D7ABA21@planet.nl> <57bb4779-66e3-6501-c19c-0bc2afb8fbd7@computer.org> <7EC6B8F7-133A-455A-93E8-464FF368019A@serissa.com> Message-ID: <3d18a828-3684-5162-af3d-6a62488615e8@kilonet.net> On 6/25/2020 11:45 AM, Lawrence Stewart wrote: > The trick to these is to include one nugget of truth. If anyone asks, I’m saying the sweater part is absolutely true. > > After all, BITNET actually did work by connecting a virtual card punch to a remote virtual card reader. > > -L I always knew Mr. Rogers was a deep state operative. All those sweaters... probably generated by years of anti-encryption efforts... From jnc at mercury.lcs.mit.edu Fri Jun 26 05:31:30 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Thu, 25 Jun 2020 15:31:30 -0400 (EDT) Subject: [TUHS] VFS prior to 1984 Message-ID: <20200625193130.9690E18C0AB@mercury.lcs.mit.edu> > From: "Greg A. Woods" > as Organick said in his 1972 book A word of warning: i) Organick describes Multics as it was planned, not as it was actually implemented; and ii) then it changed considerably during its service life. Examples of each: i) there was one linkage segment per processs, not per segment (linkage info was copied across into the shared linkage segment when a segment was dynamically added to a process' address space); ii) the New Storage System. The basic concepts (single level store, dynamic linking etc) are correct in Organick, but be wary of anything past that. Not his fault, of course; things just changed. > As far as I can remember Multics didn't really have the concept of a > "mount point". In the original design, no. In 1973-75, with the New Storage System: https://multicians.org/nss.html mountable volumes were added (see MTB 229, "Use of Demountable Logical Volumes", linked from the page above). It was released to customers in June, 1976. Noel From woods at robohack.ca Fri Jun 26 05:40:50 2020 From: woods at robohack.ca (Greg A. Woods) Date: Thu, 25 Jun 2020 12:40:50 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> Message-ID: At Wed, 24 Jun 2020 17:45:38 -0700, Adam Thornton wrote: Subject: Re: [TUHS] VFS prior to 1984 > > On Wed, Jun 24, 2020 at 2:34 PM Greg A. Woods wrote: > > > > As far as I can remember Multics didn't really have the concept of a > > "mount point". All storage was single-level, i.e. segments (equivalent > > in some respects to inodes, but they are also actually the value of the > > segment register in the virtual memory hardware), and so files were > > either physically in memory or paged out on physical disk devices or > > similar, or even out on tape. Where they actually resided was entirely > > and permanently hidden from the user. What was called the "filesystem" > > was a form of database representing a hierarchical namespace which > > pointed at all the known segments (files) regardless of where they were > > actually stored. > > > > > Coming to it from a Unix perspective, it's like all storage (core, disk, > tape) is mmap()ed. > > The segment-name database then is just an index relating symbolic names to > particular memory locations. Yes, exactly. A process is just a collection of segments of (virtual) memory, some containing code, some containing data, and more can be attached, some can be detached and/or swapped, as execution proceeds. E.g. for shared code, a module can be swapped out for another equivalent one, e.g. with a bug fix, without stopping or restarting the process (and perhaps, if it is a system library, without the user or application even knowing that has happened). > It all feels very upside-down to me, but that's probably because I grew up > in Unix and never actually used a Multics system until I emulated one with > dps8m. I kind of thought of it that way in my early years too, even though Multics was one of the first systems I learned, at the same time as I learned Unix. In Multics I/O takes a far lesser role than it does in Unix (or most any other OS then and since). Modeling everything as if it were I/O has some advantages in terms of implementation of the model (I guess it makes for far simpler operating systems), but personally I now think it's completely upside down and inside out from the user's perspective. Having to read all data before you can actually access it, and/or even just having to manage your allotted memory space and pointer offsets, is quite an annoying addition to any algorithm -- i.e. adding I/O in front of everything is like adding a significant problem on top of every other problem, every time, all the time. (Well using full-on only mmap() and a good language and compiler could probably hide much of the nastiness, but.... we don't really seem to do that even to this day, even in languages that could make it easy.) So, personally, I now see modeling everything as I/O as if it is forcing you to access all your data through the eye of a needle, every time, all the time. No matter how big the hole you still have to coil all your data up and push/pull it through if it is, or could be, bigger than the hole. I'm also quite sad that the extremely wide and near pervasive and long-term availability of a CPU family with half-decent support for segmented virtual memory has effectively been wasted for a generation or two or more of possibilities in OS design. -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP Digital Signature URL: From dave at horsfall.org Fri Jun 26 06:22:35 2020 From: dave at horsfall.org (Dave Horsfall) Date: Fri, 26 Jun 2020 06:22:35 +1000 (EST) Subject: [TUHS] Oh, things were very different. In-Reply-To: References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200624165107.GA5737@alice> <17CD58F0-2474-4308-86BA-C8847D7ABA21@planet.nl> <57bb4779-66e3-6501-c19c-0bc2afb8fbd7@computer.org> Message-ID: On Wed, 24 Jun 2020, Rich Morin wrote: > https://imgs.xkcd.com/comics/old_days_2.png Got the original cartoon number for that? I like to see the comment when I hover over it (and the explanation site can be funny, too). -- Dave From jnc at mercury.lcs.mit.edu Fri Jun 26 06:23:51 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Thu, 25 Jun 2020 16:23:51 -0400 (EDT) Subject: [TUHS] VFS prior to 1984 Message-ID: <20200625202351.BCC8418C0AB@mercury.lcs.mit.edu> > From: Richard Salz > A web search for "its mldev" finds several things (mostly by Lars > Brinkhoff it seems) Yeah, other than the source: https://github.com/PDP-10/its/blob/master/src/sysen2/mldev.106 there's not a lot on it. That's typical of a lot of the innovative work done at the AI Lab, LCS, etc; they built some extraordinary tools, but to them they were just tools they used to work on their research, their _real_ work. So why bother to write that stuff up? So people who actually used them remember them, but other than that... Welcome to tomorrow, where everything gets re-invented, because everyone is too busy to waste time learning about the past. Noel From jnc at mercury.lcs.mit.edu Fri Jun 26 06:25:14 2020 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Thu, 25 Jun 2020 16:25:14 -0400 (EDT) Subject: [TUHS] VFS prior to 1984 Message-ID: <20200625202514.4209818C0AB@mercury.lcs.mit.edu> > From: Rob Gingell > RSEXEC (Resource Sharing EXEC) done on TENEX earlier in the 1970s. That was associated with the National Software Works project: https://apps.dtic.mil/dtic/tr/fulltext/u2/a132320.pdf I think? (It's been four decades since I looked at NSW, so I don't recall much about it.) Neither was TENEX-specific; e.g. there were Multics implementations of RSEXEC and NSW. Noel From clemc at ccc.com Fri Jun 26 06:49:08 2020 From: clemc at ccc.com (Clem Cole) Date: Thu, 25 Jun 2020 16:49:08 -0400 Subject: [TUHS] Oh, things were very different. In-Reply-To: References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200624165107.GA5737@alice> <17CD58F0-2474-4308-86BA-C8847D7ABA21@planet.nl> <57bb4779-66e3-6501-c19c-0bc2afb8fbd7@computer.org> Message-ID: 2324 On Thu, Jun 25, 2020 at 4:23 PM Dave Horsfall wrote: > On Wed, 24 Jun 2020, Rich Morin wrote: > > > https://imgs.xkcd.com/comics/old_days_2.png > > Got the original cartoon number for that? I like to see the comment when > I hover over it (and the explanation site can be funny, too). > > -- Dave > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael at kjorling.se Fri Jun 26 06:42:48 2020 From: michael at kjorling.se (Michael =?utf-8?B?S2rDtnJsaW5n?=) Date: Thu, 25 Jun 2020 20:42:48 +0000 Subject: [TUHS] Oh, things were very different. In-Reply-To: References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200624165107.GA5737@alice> <17CD58F0-2474-4308-86BA-C8847D7ABA21@planet.nl> <57bb4779-66e3-6501-c19c-0bc2afb8fbd7@computer.org> Message-ID: <5bb50095-9d09-4a7f-aa5f-0b1f06e3f414@localhost> On 26 Jun 2020 06:22 +1000, from dave at horsfall.org (Dave Horsfall): >> https://imgs.xkcd.com/comics/old_days_2.png > > Got the original cartoon number for that? It's brand new. https://xkcd.com/2324/ The first one is https://xkcd.com/1755/ -- Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se “Remember when, on the Internet, nobody cared that you were a dog?” From steffen at sdaoden.eu Fri Jun 26 07:31:37 2020 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Thu, 25 Jun 2020 23:31:37 +0200 Subject: [TUHS] Origins and life of the pg pager In-Reply-To: <20200625015009.GB4655@mit.edu> References: <7wsgewophe.fsf@junk.nocrew.org> <10160f8c-62a3-014b-43a1-65025f27cde5@mhorton.net> <20200622162406.GA48733@clarinet.employees.org> <0F78CB9D-870B-4008-B975-23756EFC6F83@iitbombay.org> <20200622224301.kdouj%steffen@sdaoden.eu> <20200625015009.GB4655@mit.edu> Message-ID: <20200625213137.lkVlu%steffen@sdaoden.eu> Hello. Theodore Y. Ts'o wrote in <20200625015009.GB4655 at mit.edu>: |On Tue, Jun 23, 2020 at 12:43:01AM +0200, Steffen Nurpmeso wrote: |> #?0|kent:steffen$ pkginfo -o `command -v col` |> Package File |> util-linux usr/bin/col |> |> states |> |> /* |> * This command is deprecated. The utility is in maintenance mode, |> * meaning we keep them in source tree for backward compatibility |> * only. Do not waste time making this command better, unless the |> * fix is about security or other very critical issue. |> * |> * See Documentation/deprecated.txt for more information. |> */ | |I'll note that the Austin Group / The Open Group marked those commands |as LEGACY[1] in the Single Unix Specification V2 in 1997: | | The utilities in the table below are marked LEGACY. Various factors | may have contributed to the decision to class a utility | LEGACY. Application writers should not use functionality marked | LEGACY. | | If a migration path exists, advice is given to application | developers regarding alternative means of obtaining similar | functionality. This information may be found in the APPLICATION | USAGE sections on the relevant pages. | | No requirement beyond that which was in effect at the time that | these utilities were marked LEGACY shall be applied to these | utilities. | | calendar cancel cc col cpio cu dircmp dis egrep fgrep line lint lpstat | mail pack pcat pg spell sum tar unpack uulog uuname uupick uuto | |[1] https://pubs.opengroup.org/onlinepubs/7908799/xcu/intro.html#tag_001\ |_003_003 | |The quote from the Single Unix Specification: "No requirement beyond |that which was in effect at the time that these utilities were marked |LEGACY shall be applied to these utilities." is not that different |from "Do not waste time making this command better, unless the fix is |about security or other very critical issue." | |And while I'm sure the Linux haters will be happy to try to blame |Linux for the decision to declare pg, col, et.al as "legacy", in 1997 |The Open Group was hardly filled with Linux developers; '97 predates |IBM and Oracle declaring their support for Linux (1998), the |publication of "The Cathedral and the Bazaar (also 1998), the start of |the GNOME project (1999), Red Hat Enterprise Linux (first released |in 2002), and Ubuntu Linux (2004). | | - Ted | |P.S. Of course, the fact that the The Open Group tried to convince |the world to stop using tar and cpio in favor to pax seems to be a |strong indication that they forgot the lesson of King Canute. :-) --End of <20200625015009.GB4655 at mit.edu> Ok. Yes, POSIX is a condensed pool of tradition, history and knowledge. I do not play it well, let alone in its entirety. STREAMS not at all. After the time when the "Unix community" came together, more or less, as i understand it, in POSIX, to finally find a common ground (again), the current "POSIX group" more or less started collecting "behind the pack" what can be standardized as a common foundation, to fill gaps, and to clarify ambiguities. I have the impression that POSIX is referred to quite often --- sometimes i have to search the net to find more or less nothing good, stackoverflow, superuser, you-know (or maybe even not, having grown in this environment and even at such an outstanding position, and for such a long time), then POSIX shines through and stands out from this mess. One of the main contributors stated to me years ago something like "POSIX is an excellent foundation.." The current Issue 7, 2018 edition of POSIX states on page 3481, line 117589 ff. legacy The term ``legacy’’ was included in earlier versions of this standard but is no longer used in the current version. For col(1) that means, unfortunately, to abandon all hope. It does not exist. (It never did in any POSIX document i have in full text, not a moment i had an intent to look there.) Actually i was astonished. You said IBM (i do not know Oracle) declared support for Linux in 1998! I would have sworn that there was a big billion dollar announcement in 2003 or around that time, with "evangelists" appearing on eg Linux Magazine (of Germany, what i know..). I always, foolishly, state this must have been around 2005, the time when the wonderful Linux HOWTOs entered their bit rot age. Just yesterday, i use Linux for real since April 2019 (1999-2002 does not count, with all the out-of-the box SuSE, RedHat and Halloween Linux, and Debian Woody) and take a week of learning Linux better, ZFS on Monday, switch from proxy_arp=1 network for my VMs and such to separate netns with net<->host:veth-netns-veth:bridge<->vms++ on Tuesday (with almost 400 percent performance penalty, but: cool!), and since yesterday i try for the first time to do my own kernel with initrd (only big one for now, efi stub, builtin firmware, no modules except one from staging which cannot be linked in etc.), in order to get an allyesmod thing which i can use to plug into a box, lsmod>MODS, and then do localyesconfig, you know. I hope i find some time. However, just yesterday i wondered what would happen if suddenly virt-manager etc. would stop working, how many people would know how to continue? You really find nothing good on the web, and all the billion dollar players and their forums etc, they are no help at all. Luckily for ZFS there is the FreeBSD handbook (people who care), which is a kind starter (or would be if you would really go that route, i not for the moment, at least), and that i can apply a bridge onto a veth on a wifi network device that allows no bridge to be attached crossed my mind, finally, yay. For that proxy_arp thing i could read iptables-tut and Linux Advanced Routing & Traffic Control HOWTO, which says "Can be very useful when building 'ip pseudo bridges'. Do take care that your netmasks are very correct before enabling this! Also be aware that the rp_filter, mentioned elsewhere, also operates on ARP queries!" How could anyone hate people who write such HOWTOs? But other than that, POSIX is also always worth looking at, right. Quite off topic that all, though.. Ciao, --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From dave at horsfall.org Fri Jun 26 10:48:21 2020 From: dave at horsfall.org (Dave Horsfall) Date: Fri, 26 Jun 2020 10:48:21 +1000 (EST) Subject: [TUHS] Oh, things were very different. In-Reply-To: References: <4FC7FA55-5035-41A2-B52F-AE26DC8BED2C@planet.nl> <20200624165107.GA5737@alice> <17CD58F0-2474-4308-86BA-C8847D7ABA21@planet.nl> <57bb4779-66e3-6501-c19c-0bc2afb8fbd7@computer.org> Message-ID: On Thu, 25 Jun 2020, Clem Cole wrote: > > Got the original cartoon number for that?  I like to see the comment > > when I hover over it (and the explanation site can be funny, too). > > 2324 Thanks! The hover-over makes a bit more sense now, and explainxkcd.com is fascinating. -- Dave From lars at nocrew.org Fri Jun 26 15:49:46 2020 From: lars at nocrew.org (Lars Brinkhoff) Date: Fri, 26 Jun 2020 05:49:46 +0000 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <20200625202514.4209818C0AB@mercury.lcs.mit.edu> (Noel Chiappa's message of "Thu, 25 Jun 2020 16:25:14 -0400 (EDT)") References: <20200625202514.4209818C0AB@mercury.lcs.mit.edu> Message-ID: <7wbll64br9.fsf@junk.nocrew.org> Noel Chiappa wrote: > Neither was TENEX-specific; e.g. there were Multics implementations of > RSEXEC and NSW. RFC 441 specifies a protocol for TALK, and apparently the RSEXEC protocol is involved somehow. ITS has a server for this protocol. https://tools.ietf.org/html/rfc441 From dave at horsfall.org Fri Jun 26 18:34:29 2020 From: dave at horsfall.org (Dave Horsfall) Date: Fri, 26 Jun 2020 18:34:29 +1000 (EST) Subject: [TUHS] VFS prior to 1984 In-Reply-To: <7wbll64br9.fsf@junk.nocrew.org> References: <20200625202514.4209818C0AB@mercury.lcs.mit.edu> <7wbll64br9.fsf@junk.nocrew.org> Message-ID: On Fri, 26 Jun 2020, Lars Brinkhoff wrote: > RFC 441 specifies a protocol for TALK, and apparently the RSEXEC > protocol is involved somehow. ITS has a server for this protocol. "talk"... Well, at least it was a *bit* better than "write" :-) "mesg n" was your friend. -- Dave From arnold at skeeve.com Fri Jun 26 21:13:32 2020 From: arnold at skeeve.com (arnold at skeeve.com) Date: Fri, 26 Jun 2020 05:13:32 -0600 Subject: [TUHS] VFS prior to 1984 In-Reply-To: References: <20200625202514.4209818C0AB@mercury.lcs.mit.edu> <7wbll64br9.fsf@junk.nocrew.org> Message-ID: <202006261113.05QBDW2W017518@freefriends.org> Dave Horsfall wrote: > "talk"... Well, at least it was a *bit* better than "write" :-) > > "mesg n" was your friend. Unless you actually wanted to talk to the person on the other end. Having it over the Internet was cool. In 1989 when my wife and I were dating, we lived in different cities, and would occasionally use talk to chat with each other in real time, w/o the expense of a phone call. It was fun. :-) Arnold From dscherrer at solar.stanford.edu Sat Jun 27 11:24:25 2020 From: dscherrer at solar.stanford.edu (Deborah Scherrer) Date: Fri, 26 Jun 2020 18:24:25 -0700 Subject: [TUHS] Mike Hawley has died Message-ID: Some very sad news: http://news.mit.edu/2020/michael-hawley-former-professor-media-arts-sciences-dies-0625 From agrier at poofygoof.com Mon Jun 29 08:39:55 2020 From: agrier at poofygoof.com (Aaron J. Grier) Date: Sun, 28 Jun 2020 15:39:55 -0700 Subject: [TUHS] Okistations and OKIX? Message-ID: <20200628223955.GH170@ra.poofy.goof.com> Anybody on the list know much about the provenance of SVr4 and OKIX for the Okistation 7300 model 30? http://museum.ipsj.or.jp/en/computer/work/0029.html indicates that OKIX came later on the model 75, but many years ago as a staff programmer at Oregon Graduate Institute, a surplus 7300/30 followed me home, but the sysadmins referred to it as OKIX. if anybody's interested in the hardware to do some spelunking, email me off-list. -- Aaron J. Grier | "Not your ordinary poofy goof." | agrier at poofygoof.com "The price of reliability is the pursuit of the utmost simplicity. It is a price which the very rich find most hard to pay." -- Tony Hoare From lm at mcvoy.com Mon Jun 29 09:41:01 2020 From: lm at mcvoy.com (Larry McVoy) Date: Sun, 28 Jun 2020 16:41:01 -0700 Subject: [TUHS] Okistations and OKIX? In-Reply-To: <20200628223955.GH170@ra.poofy.goof.com> References: <20200628223955.GH170@ra.poofy.goof.com> Message-ID: <20200628234101.GU1324@mcvoy.com> Huh, that's a trip down memory lane. I didn't have that one but the first PC I ever owned was a CP/M machine made by the same company. Marketed in this country under the BMC name but it was made by Oki. 2nd one down here: http://www.classiccmp.org/dunfield/miscpm/index.htm I took out a $2000 loan and bought it in 1984 or 1985 because the alternative was a VAX 11/780 that had about 30-40 people trying to compile their projects at the same time. I believe it had 4M of ram, definitely not enough for the load. You'd think a 64K z80 with floppies would be a down grade, it was not, the performance was predictable, I didn't get swapped out for minutes, it was always working just for me. I wrote a _ton_ of code on that machine. On Sun, Jun 28, 2020 at 03:39:55PM -0700, Aaron J. Grier wrote: > Anybody on the list know much about the provenance of SVr4 and OKIX for the > Okistation 7300 model 30? > > http://museum.ipsj.or.jp/en/computer/work/0029.html indicates that OKIX > came later on the model 75, but many years ago as a staff programmer at > Oregon Graduate Institute, a surplus 7300/30 followed me home, but the > sysadmins referred to it as OKIX. > > if anybody's interested in the hardware to do some spelunking, email me > off-list. > > -- > Aaron J. Grier | "Not your ordinary poofy goof." | agrier at poofygoof.com > "The price of reliability is the pursuit of the utmost simplicity. It > is a price which the very rich find most hard to pay." -- Tony Hoare -- --- Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm From cym224 at gmail.com Mon Jun 29 10:35:10 2020 From: cym224 at gmail.com (Nemo) Date: Sun, 28 Jun 2020 20:35:10 -0400 Subject: [TUHS] Okistations and OKIX? In-Reply-To: <20200628234101.GU1324@mcvoy.com> References: <20200628223955.GH170@ra.poofy.goof.com> <20200628234101.GU1324@mcvoy.com> Message-ID: On 28/06/2020, Larry McVoy wrote: > Huh, that's a trip down memory lane. I didn't have that one but the first > PC I ever owned was a CP/M machine made by the same company. Marketed in > this country under the BMC name but it was made by Oki. > > 2nd one down here: > > http://www.classiccmp.org/dunfield/miscpm/index.htm > > I took out a $2000 loan and bought it in 1984 or 1985 because the > alternative was a VAX 11/780 that had about 30-40 people trying to > compile their projects at the same time. I believe it had 4M of > ram, definitely not enough for the load. > > You'd think a 64K z80 with floppies would be a down grade, it was not, > the performance was predictable, I didn't get swapped out for minutes, > it was always working just for me. > > I wrote a _ton_ of code on that machine. What did you write, I mean, what were your development tools? N. From pnr at planet.nl Mon Jun 29 19:11:17 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Mon, 29 Jun 2020 11:11:17 +0200 Subject: [TUHS] VFS prior to 1984 Message-ID: <888A9360-A6E0-4EE0-84A5-8ADD478807E1@planet.nl> Date: Wed, 24 Jun 2020 14:31:34 -0400 (EDT) From: norman at oclsc.org (Norman Wilson) > Reaching outside of UNIX, RSX/11 used external supervisor-mode processes called ACPs (ancillary control processes) to implement file systems. I don't know exactly how they were plugged in, but I do know they were pluggable, so their interface must have constituted a file-system switch of some sort. RSX dates back into the 1970s. At some point in the latter part of the 1980s, Ralph Stamerjohn (a name instantly recognizable in the 16-bit DEC software world) gave a DECUS talk about implementing a remote file system through ACPs: a stub ACP on the client exporting RPCs over the network, a real one at the server end. I remember chatting with him about how that did and didn't resemble the way pjw had done it; interesting architectural comparison. > Norman Wilson Toronto ON I am still digesting all the inputs (thanks, all!) The above post made me realise that the delineation of what is a FSS/VFS or not, is not so easy. I did a little bit of reading, and the concept of an ACP arrived with RSX11D in May 1973, but only matured in RSX11M in November 1974. As I understand it, originally in RSX11 file system code was closely tied to the low-level device driver for each device. ACP’s separated the file system code from the device driver itself, and became separate processes. In essence there were two switches: one switch into abstract devices, implemented in ACP code and one kernel switch to deal with hardware interfacing. The first is indeed like a file system switch (although still tied to specific devices). Looking at this stuff made me realise that my retro machine of choice (the TI990) went through a similar evolution. In the early seventies it had a sort of abstract device switch that linked to individual ‘device service routines’ (drivers). Initially, these modelled batch oriented ‘logical units’ that tied to files at the job control level. Later (late 70’s), the ‘open’ command would carry a file name and the file system was delegated to the device service routine. Still later (say 1983) this was used for networked disks. As several people have observed in this topic, indeed there appears to be a close relationship between a device switch and a file system switch. From lm at mcvoy.com Tue Jun 30 00:45:31 2020 From: lm at mcvoy.com (Larry McVoy) Date: Mon, 29 Jun 2020 07:45:31 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <888A9360-A6E0-4EE0-84A5-8ADD478807E1@planet.nl> References: <888A9360-A6E0-4EE0-84A5-8ADD478807E1@planet.nl> Message-ID: <20200629144531.GD16252@mcvoy.com> On Mon, Jun 29, 2020 at 11:11:17AM +0200, Paul Ruizendaal wrote: > As several people have observed in this topic, indeed there appears to be a close relationship between a device switch and a file system switch. Yep. If you think in objects, they are both objects with a set of methods. Pretty much two instances of the same idea. From pnr at planet.nl Tue Jun 30 00:53:50 2020 From: pnr at planet.nl (Paul Ruizendaal) Date: Mon, 29 Jun 2020 16:53:50 +0200 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <20200629144531.GD16252@mcvoy.com> References: <888A9360-A6E0-4EE0-84A5-8ADD478807E1@planet.nl> <20200629144531.GD16252@mcvoy.com> Message-ID: <294D0B70-CDBB-41FE-8F3D-035562863D44@planet.nl> > On 29 Jun 2020, at 16:45, Larry McVoy wrote: > > On Mon, Jun 29, 2020 at 11:11:17AM +0200, Paul Ruizendaal wrote: >> As several people have observed in this topic, indeed there appears to be a close relationship between a device switch and a file system switch. > > Yep. If you think in objects, they are both objects with a set of methods. > Pretty much two instances of the same idea. Phrased that way all OO programs have a close relationship between them. Maybe: "they are both rather similar objects with a rather similar set of methods"? From lm at mcvoy.com Tue Jun 30 01:14:02 2020 From: lm at mcvoy.com (Larry McVoy) Date: Mon, 29 Jun 2020 08:14:02 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <294D0B70-CDBB-41FE-8F3D-035562863D44@planet.nl> References: <888A9360-A6E0-4EE0-84A5-8ADD478807E1@planet.nl> <20200629144531.GD16252@mcvoy.com> <294D0B70-CDBB-41FE-8F3D-035562863D44@planet.nl> Message-ID: <20200629151402.GE16252@mcvoy.com> On Mon, Jun 29, 2020 at 04:53:50PM +0200, Paul Ruizendaal wrote: > > > On 29 Jun 2020, at 16:45, Larry McVoy wrote: > > > > On Mon, Jun 29, 2020 at 11:11:17AM +0200, Paul Ruizendaal wrote: > >> As several people have observed in this topic, indeed there appears to be a close relationship between a device switch and a file system switch. > > > > Yep. If you think in objects, they are both objects with a set of methods. > > Pretty much two instances of the same idea. > > Phrased that way all OO programs have a close relationship between them. Maybe: "they are both rather similar objects with a rather similar set of methods"? They are two different classes, similar, but not the same. It's really 4 classes, vfs, vnode for FS and block/char for devices. This is from SunOS 4.1.1 that I happened to have an install image sitting around. Took me forever to find the device definitions, I resorted to find . -type f -name '*.h' -print | xargs grep strategy It's in include/sys/conf.h because, well, that's obvious, right? struct vfsops { int (*vfs_mount)(); /* mount file system */ int (*vfs_unmount)(); /* unmount file system */ int (*vfs_root)(); /* get root vnode */ int (*vfs_statfs)(); /* get fs statistics */ int (*vfs_sync)(); /* flush fs buffers */ int (*vfs_vget)(); /* get vnode from fid */ int (*vfs_mountroot)(); /* mount the root filesystem */ int (*vfs_swapvp)(); /* return vnode for swap */ }; struct vnodeops { int (*vn_open)(); int (*vn_close)(); int (*vn_rdwr)(); int (*vn_ioctl)(); int (*vn_select)(); int (*vn_getattr)(); int (*vn_setattr)(); int (*vn_access)(); int (*vn_lookup)(); int (*vn_create)(); int (*vn_remove)(); int (*vn_link)(); int (*vn_rename)(); int (*vn_mkdir)(); int (*vn_rmdir)(); int (*vn_readdir)(); int (*vn_symlink)(); int (*vn_readlink)(); int (*vn_fsync)(); int (*vn_inactive)(); int (*vn_lockctl)(); int (*vn_fid)(); int (*vn_getpage)(); int (*vn_putpage)(); int (*vn_map)(); int (*vn_dump)(); int (*vn_cmp)(); int (*vn_realvp)(); int (*vn_cntl)(); }; /* two different device classes, block and char */ struct bdevsw { int (*d_open)(); int (*d_close)(); int (*d_strategy)(); int (*d_dump)(); int (*d_psize)(); int d_flags; }; struct cdevsw { int (*d_open)(); int (*d_close)(); int (*d_read)(); int (*d_write)(); int (*d_ioctl)(); int (*d_reset)(); int (*d_select)(); int (*d_mmap)(); struct streamtab *d_str; int (*d_segmap)(); }; -- --- Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm From heinz at osta.com Tue Jun 30 02:34:49 2020 From: heinz at osta.com (Heinz Lycklama) Date: Mon, 29 Jun 2020 09:34:49 -0700 Subject: [TUHS] VFS prior to 1984 In-Reply-To: <888A9360-A6E0-4EE0-84A5-8ADD478807E1@planet.nl> References: <888A9360-A6E0-4EE0-84A5-8ADD478807E1@planet.nl> Message-ID: <59a01bce-3076-f4db-acb1-99089faac01e@osta.com> We implemented filesystems as a supervisor process so that it could support file access from both a UNIX supervisor process and an RSX/11 supervisor process in the MERT operating system. See Figure 1 in the MERT paper in the July/August 1978 BSTJ. Heinz On 6/29/2020 2:11 AM, Paul Ruizendaal wrote: > Date: Wed, 24 Jun 2020 14:31:34 -0400 (EDT) > From: norman at oclsc.org (Norman Wilson) >> Reaching outside of UNIX, RSX/11 used external supervisor-mode processes called ACPs (ancillary control processes) to implement file systems. I don't know exactly how they were plugged in, but I do know they were pluggable, so their interface must have constituted a file-system switch of some sort. RSX dates back into the 1970s. At some point in the latter part of the 1980s, Ralph Stamerjohn (a name instantly recognizable in the 16-bit DEC software world) gave a DECUS talk about implementing a remote file system through ACPs: a stub ACP on the client exporting RPCs over the network, a real one at the server end. I remember chatting with him about how that did and didn't resemble the way pjw had done it; interesting architectural comparison. >> Norman Wilson Toronto ON > I am still digesting all the inputs (thanks, all!) > > The above post made me realise that the delineation of what is a FSS/VFS or not, is not so easy. > > I did a little bit of reading, and the concept of an ACP arrived with RSX11D in May 1973, but only matured in RSX11M in November 1974. As I understand it, originally in RSX11 file system code was closely tied to the low-level device driver for each device. ACP’s separated the file system code from the device driver itself, and became separate processes. > > In essence there were two switches: one switch into abstract devices, implemented in ACP code and one kernel switch to deal with hardware interfacing. The first is indeed like a file system switch (although still tied to specific devices). > > Looking at this stuff made me realise that my retro machine of choice (the TI990) went through a similar evolution. In the early seventies it had a sort of abstract device switch that linked to individual ‘device service routines’ (drivers). Initially, these modelled batch oriented ‘logical units’ that tied to files at the job control level. Later (late 70’s), the ‘open’ command would carry a file name and the file system was delegated to the device service routine. Still later (say 1983) this was used for networked disks. > > As several people have observed in this topic, indeed there appears to be a close relationship between a device switch and a file system switch. > > > >