I got to wondering, based on the sendmail discussions, how many shell
escapes have appeared over the years?
uucp
sendmail
xdvi : "The "allowShell" option enables the shell escape in PostScript specials"
There must be a lot of them, however.
When looking at old xmodem code I noticed that it calculated its CRC bit-by-bit, switching to byte-wise using a table in the late 80’s. It never seems to have used the byte-wise, “on-the-fly” algorithm. This seems to match a pattern: I often come across bit-wise and table implementations, but rarely on-the-fly implementations if any. The on-the-fly algorithm was known at least since 1983, following a paper by Perez: http://www.bitsavers.org/components/fairchild/_appNotes/Byte-wise_CRC_Jun83…
The paper was noted, for example it is on the citation list of RFC1134, describing the PPP protocol. Today, a wikipedia page gives implementations for various polynomials: https://en.wikipedia.org/wiki/Computation_of_cyclic_redundancy_checks#Paral…
Now, it would seem to me that on memory constrained personal computers and PDP-11’s the “on-the-fly” algorithm would have been a good choice, being just a few lines of code and maybe 30-50% slower than table lookup. The tables aren’t big, but a kilobyte is a lot when you only have 64.
Any suggestions as to why the on-the-fly algorithm did not catch on more in the 1980’s? Maybe it was simply less well known than I think?
> From: Paul Ruizendaal
> Any suggestions as to why the on-the-fly algorithm did not catch on
> more in the 1980's? Maybe it was simply less well known than I think?
I can't answer that directly, but I will point you at IEN-56, "CRC Checksum
Calculation", by Dave Reed (11-Sep-78):
https://www.rfc-editor.org/ien/ien56.pdf
Dave wanted the INWG to use a more powerful (in terms of detecting errors)
CRC, instead of the simple summation eventually adopted, in TCP and IP. So he
produced code to implement a particular CRC, to show people that it would not
be particularly expensive (whether in time, or space, I don't alas recall
definitively; speed would have been an important consideration, when
competing with the summation, though).
This would have been close to the leading edge of our knowledge at the time;
Dave liked playing around with math, and at about that time he did a very
fast DES implementation.
Noel
Howdy all, just passing along this auction I spotted on eBay: https://www.ebay.com/itm/256216372208
This is a subset of papers from the Documents for UNIX 4.0 set compiled as a handy volume (and likely a forerunner to the packaged volumes released with System V.) Missing is the Typing Documents with MM pamphlet which is listed in the TOC of these issues. I've already got one of these or I'd scoop it up myself. Enjoy!
- Matt G.
P.S. I have seen this and its companion document (Programming Starter Package) with the AT&T deathstar in the upper right rather than Bell Labs and the Bell logo. What I don't know is if it's an exact repackaging just with a change in logo or if those versions have revisions to any of the papers. Most notably as far as version association, the Programming Starter Package includes the Documentation Roadmap, which in my Bell Labs-branded copy, indicates it is for use with UNIX 4.0. I'd be curious if the AT&T variant retains the UNIX 4.0 reference. I'll continue to keep my eye out for an AT&T-branded set, it might reveal some differences.
>> Any suggestions as to why the on-the-fly algorithm did not catch on more in the 1980’s? Maybe it was simply less well known than I think?
>
> Could it have been the per-cpu-second billing that was (fairly?) common at the time. I was only getting in to Unix in the early 90s but saw the tail end of that.
Good point, but wasn’t per-cpu-second billing mostly used for big iron? For machines without memory constraint the table method makes the most sense, also if billing was not a factor.
>> Any suggestions as to why the on-the-fly algorithm did not catch on more in the 1980’s? Maybe it was simply less well known than I think?
>
> The CRC algorithm I'm familiar with shows up in Dragon Quest for the Famicom in 1986[1], written in 6502 assembly. Admittedly though I only recognized it due to the EOR with 0x1021 on lines 318-323. That I then only know from a quick and dirty CRC I threw in an XMODEM-CRC client [2] I did to accommodate for a bug in the JH7100 RISC-V board's recovery ROM implementation. Not sure if this is along the lines of the approach you're talking about ...
>
> [1] - https://gitlab.com/segaloco/dq1disasm/-/blob/master/src/chr3/start_pw.s
> [2] - https://gitlab.com/segaloco/riscv-bits/-/blob/master/util/sxj.c
Both of these are what I call the bit-wise method: a loop iterating over bytes, with an inner loop iterating over bits. An example of the table method is here:
https://github.com/u-boot/u-boot/blob/master/lib/crc16-ccitt.c
and an example of the on-the-fly method is here:
https://github.com/tio/tio/blob/master/src/xymodem.c#L44-L54
Note how the latter also only has one loop iterating over the bytes, but effectively calculates the table entry ‘on the fly’ for each byte. That is only a handful of instructions more than doing the table lookup. Maybe it is a “stuck in the middle” solution that was quickly forgotten.
Last december Matt brought up xmodem and recently I needed it for an almost identical use case. Studied it a bit and in the context of Unix, it is an interesting piece of software history.
Although xmodem originated on CP/M in 1977, it seems to have arrived on Unix soon thereafter. The first Unix implementation seems to have gone by the name of “rbsb” and must have originated when V7 was prevalent: it uses alarm() to simulate non-blocking I/O. Over the course of the 80’s ymodem and zmodem were added and the package became lrzsz; the source continued to have a very V7-ish feel to it at least to the early 90’s. In the mid-90’s it was converted to ansi-C and had some other modernization, but it looks like it would still have run on SysIII (apart from the ansi).The last update to the upstream source package appears to be from 1998, i.e. 25 years ago.
It is still packaged today for FreeBSD and major Linux distros. On Unix it must be one the oldest code bases still in regular use, with the 1980 source still recognizable in its current incarnation. Any other contenders come to mind?
I had always associated x/y/zmodem with CP/M and MSDOS, not so much with Unix. Last December Clem already pointed out that it was popular for file exchange in the Unix scene as well, along with several other similar tools. Also, the ymodem approach to file metadata is very unix oriented, suggesting it originated on Unix or at least that Unix users were an important user demographic. Yet, I could find little trace of x/y/zmodem in the TUHS Unix Tree. The search tool finds it in 2.11BSD, in Minix 1.5 and 2.0 and in V10. Kermit is in those as well, and in 4.3BSD and 4.4BSD on top.
Maybe these programs were commonly pulled from a bulletin board and hence not on distribution media. Not sure how that would be bootstrapped, although the 2.11BSD files for x/y/zmodem include “minirb.c”, a 175 line implementation to receive files using the ymodem protocol. It is possible that this was transferred as plain text as a first step, or even retyped.
Any recollections?
Joe Ossanna worked diligently to see that WECo's ASCII teletype really
would come to market and would meet the needs of Unix. He famously
estimated that Bell Labs alone was a sizable market: 777 machines. He
also leaned on WECo to make the return key issue a single newline. And
he specified what non-ascii characters would fill out the 128
positions on the Bell Labs type boxes. This requires encoding 7-bit
ASCII in 8 bits. I don't know whether WECo had been planning to do so
anyway.
Doug
Good morning folks, something I've been pondering on this morning is the potential close connection between UNIX development and Teletype interfacing designs as the 70s and 80s marched on. Seeing as Teletype was a part of the Bell System (albeit a little less obviously in marketing than it's kin), was there any sort of official rapport between folks working on UNIX and those designing subsequent Teletypes, Dataspeed terminals, etc?
For instance, would there have been any influence an up-and-coming Teletype design would've had on developments in the UNIX tty drivers, or would particulars of UNIX tty drivers "rub off" on specifics of terminals being developed? Or were those units so distant from one another that there would've been little cross-talk between teams? Granted, an argument could be made for specifically avoiding any significant knowledge of internals, that way UNIX tty driver folks don't potentially paint into a Teletype corner and vice versa. Still, with the tightly integrated nature of Bell System R&D, manufacturing, supply, etc. it would be very "Bell System" for there to be some sort of interplay between Teletype and UNIX.
Anyone got the scoop on whether Teletype hardware enjoyed a special place in UNIX tty-interfacing considerations or vice versa, or if they were just two relatively insular developments from one another in the same general field from the same big umbrella organization?
- Matt G.
All,
My apologies to you for sending a note to the entire list that was meant
to be sent only to Warren. I definitely had not intended to send it to
the entire list (reply list... ugh.). If I were to get a mulligan, I
would make it clear that I do not take issue with anyone's signature,
including Mary Ann's. Signatures, in my view, are a matter of free
speech. I would simply state my preference that we keep the discussion
to a more technical level and consider moving cultural topics to another
list altogether.
Thanks,
Will
Hello All.
For whoever's interested, the csv code has been merged into the master
branch of the Git repo. Have fun!
Arnold
> From: arnold at skeeve.com (arnold at skeeve.com)
> Date: Sun, 10 Sep 2023 13:41:34 -0600
> Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new?
>
> Hi.
>
> markus schnalke <meillo at marmaro.de> wrote:
>
> > Hoi,
> >
> > I just discovered that one of my favorite computer books about my
> > best liked programming language (besides C) releases in a second
> > edition. Does anyone know what the differences of 1st and 2nd
> > edition are?
> >
> > As the original book is almost perfect, the only rework and
> > extension direction I can think of is towards different
> > implementations like gawk, mawk, portability and such things.
> >
> > Does anyone know more about it? Maybe some inside information? ;-)
> >
> > meillo
>
> Inside information? As it happens, yes, I do have some. :-)
> (I was a reviewer.)
>
> [In the below, "awk" means Brian Kernighan's awk.]
>
> In the 36 (!) years since the first edition was published, awk
> has undergone, shall we say, a large number of small changes. These
> are listed in the FIXES file currently in the master branch of
> https://github.com/onetrueawk/awk.
>
> In addition, Brian Kernighan decided to add support for UTF-8 input,
> which is what awk now expects, and support for CSV input files when
> invoked with the --csv option. Furthermore, there is a new \u escape
> sequence which must be followed by 1-8 hexadecimal digits for specifying
> Unicode code points.
>
> The book itself has been carefully revised. The large second chapter
> which was a reference to the full language was moved to an appendix.
> Many of the example programs from the first edition were retained
> and updated, but there is also quite of lot of pleasing new material.
>
> There is mention of, and occasional comparison with, gawk, mawk and
> Ben Hoyt's GoAwk, but by and large the focus is on the authors' version.
>
> The new code is currently in the "csv" branch of the above Github
> repo. The maintainer is in the process of tidying up the repo (dealing
> with issues and pull requests) and will merge the csv branch into
> master sometime in the very near future.
>
> I'm told that the printed books with get to the publisher's warehouse
> towards the end of September. The book is available now on O'Reilly's
> Safari learning site (safari.oreilly.com) for anyone who has a
> subscription.
>
> Matching code (--csv and \u) are in gawk's master branch now. I will
> make a release this fall, after the new code has moved into master
> in BWK's awk.
>
> I heartily recommend the book; it is totally up to Brian Kernighan's
> usual very high standard.
>
> Enjoy,
>
> Arnold
>
