Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]
3 Sep
2017
3 Sep
'17
2:08 a.m.
On 02/09/2017, Dave Horsfall <dave(a)horsfall.org> wrote:
On Thu, 31 Aug 2017, Nemo wrote:
Hhhmmm... This begs the historical question: When did LF replace CR/LF in UNIX?
N.
(By the way, why all the ^M is the files?)
Err, because CR/LF was the line terminator?
--
Dave Horsfall DTM (VK2KFU) "Those who don't understand security will
suffer."
3:08 p.m.
On Sat, Sep 2, 2017 at 8:54 PM, Dave Horsfall <dave(a)horsfall.org> wrote:
On Sat, 2 Sep 2017, Nemo wrote:
Hhhmmm... This begs the historical question: When did LF replace CR/LF in
<CR><LF> was the line terminator in DEC operating systems that grew up
around the same time as Unix. CP/M and MS-DOS inherited that from them
since those systems were developed, in part, using cross compilers running
on DEC gear with DEC OSes. Unix came from the Multics world where LF was
used as the line terminator... Thankfully, neither CP/M nor MS-DOS picked
up DEC's RMS...
Warner
UNIX?
Unix has always used NL as the terminator :-) 
10:28 p.m.
New subject: Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]
I think the reason for this is pretty obvious. At the time -- for
many of the paper terminals, line feed simply rotated the platen, and
the print head stayed at the same column position. Carriage return
returned the print head to the first column without advancing the
paper (remember overstriking?). But there were also some terminals
that both advanced the paper and returned the print head (I'm hazy
about this, but I think the IBM Selectric was one...).
The Honeywell system in Murray Hill had a teletype driver that
attempted to paper over these differences. If it saw a linefeed on
some tty's they would issue the line feed and then space out to the
previous column location. If it saw a carriage return it would
backspace to the first column. It did recognize CR/LF and did
something more reasonable.
I do remember one guy who wrote his program to output LF/CR instead of
CR/LF. The teletype driver would do a LF and then space out to the
previous column and then do a carriage return. Pretty painful at 300
baud!
Doug might remember better than I but I suspect some terminal did the
combined CR/LF only, so that's why Unix adopted it. Certainly by the
time glass teletypes came along, making overstriking (and often
underlining) impossible, I don't think anybody objected.
Steve
----- Original Message -----
From: "Dave Horsfall" <dave(a)horsfall.org>
To:"The Eunuchs Hysterical Society" <tuhs(a)tuhs.org>
Cc:
Sent:Sun, 3 Sep 2017 12:54:02 +1000 (EST)
Subject:Re: [TUHS] Line Terminators in Text Files [Was: Re: Why Pascal
is Not My Favorite Programming Language - Unearthed!]
On Sat, 2 Sep 2017, Nemo wrote:
Hhhmmm... This begs the historical question: When did
LF replace
CR/LF
in UNIX?
Unix has always used NL as the terminator :-)
--
Dave Horsfall DTM (VK2KFU) "Those who don't understand security will
suffer."
11:14 p.m.
New subject: Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]
I think the reason for this is pretty obvious. At the
time -- for
many of the paper terminals, line feed simply rotated the platen, and
the print head stayed at the same column position. Carriage return
returned the print head to the first column without advancing the
paper (remember overstriking?).
In what was either a remarkable coincidence, or actual planning,
the old ASR 33 at I think 110 baud (maybe 300) took two "character
times" to do the carriage return, so that if you sent CR and LF
in that order, the LF occurred mid-CR and the print head was
ready right at the time the next character arrived.
(One of the Stupid Pet Tricks of the day was to print a line with
an unfilled character halfway through, then send CR, then print
the middle character and have it come out in the right position.)
But there were also some terminals that both advanced
the paper
and returned the print head (I'm hazy about this, but I think
the IBM Selectric was one...).
Possibly, since as I recall there was a keyboard key that did the
equivalent of pushing the return bar on a manual typewriter. If
you look at photos of Selectrics, there is no external bar (though
of course there are knobs to rotate the platen for inserting and
removing pages).
(I used a modified Selectric that had been converted into a
printer, hooking it to a custom driver that I wrote, and custom
hardware that came with the modified Selectric, on my home TRS-80,
back in high school -- 1980-81 or so. I remember seven solenoids,
which must have been divided into 1 for shift -- 180 degree rotate
-- and 2 for tilt plus 4 for rotate. I had access to an actual
Selectric as well, and when you pushed the shift key, the ball
would do the 180 degree rotate at that point, before doing the
tilt/rotate for each character.)
Doug might remember better than I but I suspect some
terminal did the
combined CR/LF only, so that's why Unix adopted it.
As I recall, the original definition of ASCII suggested that the
LF character was either "line feed" or "new line", and that if it
*was* new-line, it would be stand-alone.
Wikipedia (https://en.wikipedia.org/wiki/ASCII) implies that Unix
got this from Multics.
Of course, DEL was code 0177 because it punched all the holes in
the paper tape, which would always work to "erase" any partially
punched holes....
Chris
11:37 p.m.
On 09/03/2017 05:14 PM, Chris Torek wrote:
Of course, DEL was code 0177 because it punched all
the holes in
the paper tape, which would always work to "erase" any partially
punched holes....
I remember having a face palm moment when I realized / understood this
for the first time.
I've had the fortune of sharing that epiphany with others a handful of
times.
I think the order of the carriage return followed by the line feed will
become another such epiphany inducing pieces of trivia.
--
Grant. . . .
unix || die
4 Sep
4 Sep
12:31 a.m.
On 04/09/2017 00:14, Chris Torek wrote:
In what was either a remarkable coincidence, or actual
planning,
the old ASR 33 at I think 110 baud (maybe 300) took two "character
times" to do the carriage return, so that if you sent CR and LF
in that order, the LF occurred mid-CR and the print head was
ready right at the time the next character arrived.
That's not right. Yes, the ASR33 runs at 110 baud (10cps), but all -
including mine and those of my friends - take more like half a dozen
character times to perform a CR from the right hand side. Moreover, the
time it takes depends in a non-linear fashionon how far along the line
the printhead has travelled; it's driven by a spring and decelerated and
stopped by an air dashpot. Even an LF takes more than two character
times. That's why ASR33 drivers send a string of nulls after a CR.
Many are smart enough to send the LF instead of the first null, if they
see that LF is the first character after CR.
> But there were also some terminals that both
advanced the paper
> and returned the print head (I'm hazy about this, but I think
> the IBM Selectric was one...).
The ASR33 friction feed models, but not the sprocket fed ones, can be
configured to do that but AFAIR only at the 72nd character position at
the end of a line.
Possibly, since as I recall there was a keyboard key
that did the
equivalent of pushing the return bar on a manual typewriter. If
you look at photos of Selectrics, there is no external bar (though
of course there are knobs to rotate the platen for inserting and
removing pages).
That's correct. Selectrics have a RETURN key, which performs carriage
return and linefeed, and on mine there was a switch for single/double
line spacing too. I no longer have one, but I used to have two.
--
Pete
Pete Turnbull
1:47 a.m.
New subject: Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]
That's not right. Yes, the ASR33 runs at 110 baud
(10cps), but all -
including mine and those of my friends - take more like half a dozen
character times to perform a CR from the right hand side.
Hm, OK. I recall actually seeing the trick performed on
*something* though.
(I hooked up an old ASR 33 to someone's system with a current loop
circuit at one point, plus a bit-bang driver, and I thought I
remembered it being this particular ASR 33 on which I saw the trick,
but it might have been something else.)
Chris
2:58 a.m.
On Sun, Sep 3, 2017 at 7:47 PM, Chris Torek <torek(a)torek.net> wrote:
Time delay for physical movement of the print head is the main reason that
termcap has entries for the number of NUL characters to insert after
carriage return. Not sure it is for the ASR 33, but when it takes a while
for the carriage to return to column 1, you have to insert NULs to wait for
it to be ready to print characters. If it takes 100ms to do the return,
that's still 10 characters of NUL needed...
Warner
That's not
right. Yes, the ASR33 runs at 110 baud (10cps), but all -
including mine and those of my friends - take more like half a dozen
character times to perform a CR from the right hand side.
Hm, OK. I recall actually seeing the trick performed on
*something* though.
(I hooked up an old ASR 33 to someone's system with a current loop
circuit at one point, plus a bit-bang driver, and I thought I
remembered it being this particular ASR 33 on which I saw the trick,
but it might have been something else.)
9:05 a.m.
On 04/09/2017 03:58, Warner Losh wrote:
Time delay for physical movement of the print head is
the main reason
that termcap has entries for the number of NUL characters to insert
after carriage return. Not sure it is for the ASR 33, but when it takes
a while for the carriage to return to column 1, you have to insert NULs
to wait for it to be ready to print characters. If it takes 100ms to do
the return, that's still 10 characters of NUL needed...
I'm going to assume that's a typo :-) A single NUL at 110 baud,or
10cps, is 100ms.
--
Pete
Pete Turnbull
3:33 p.m.
New subject: Line Terminators in Text Files [Was: Re: Why Pascal is Not My Favorite Programming Language - Unearthed!]
Time delay for physical movement of the print head is
the main
reason that termcap has entries for the number of NUL characters
to insert after carriage return.
This is not quite right. For instance, there are delay values for
line and character insert and delete operations, which were
required back when terminals were slow and bit rates were inching
up ("stty 19200" finally going in as a name for EXTA, one of the
two externally-controlled bit rates in the old drivers...).
(The 4BSD tty driver had CR/LF delays built in, too, no doubt
inherited from 32V. It had to be done in the tty driver, not in
termcap, since command line utilities mostly did not use termcap
in the first place.)
As for termcap delays: some of the software on those terminals was
particularly bad. The Heathkit H19, for instance, implemented the
repeat-counted line and character insert and delete ANSI escape
sequences ESC [ %d <code> with a loop that inserted or deleted one
line or character, N times. (I had an H19, and kept it in this
mode instead of the other one that did not have the N-character
ops -- even with the required delays, if you got the editor to use
them, they were much faster over slow modems.)
(I hacked on the Gosling Emacs display driver quite a bit to tune
it up for our displays, including the Ann Arbor Ambassador displays
that did up to 60 lines in Squint Mode.)
Chris
10:07 a.m.
On 04/09/2017 02:47, Chris Torek wrote:
I don't doubt it's possible, just not on an ASR33. Apart from the
timing issue, the carriage return mechanism on Teletype 33s inhibits
printing while the carriage is moving back, until it stops at the left
margin.
--
Pete
Pete Turnbull
That's
not right. Yes, the ASR33 runs at 110 baud (10cps), but all -
including mine and those of my friends - take more like half a dozen
character times to perform a CR from the right hand side.
Hm, OK. I recall actually seeing the trick performed on
*something* though.
(I hooked up an old ASR 33 to someone's system with a current loop
circuit at one point, plus a bit-bang driver, and I thought I
remembered it being this particular ASR 33 on which I saw the trick,
but it might have been something else.) 
5 Sep
5 Sep
3:23 p.m.
On 9/4/17, Pete Turnbull <pete(a)dunnington.plus.com> wrote:
I don't doubt it's possible, just not on an ASR33. Apart from the
timing issue, the carriage return mechanism on Teletype 33s inhibits
printing while the carriage is moving back, until it stops at the left
margin.
I could swear I saw our old ASR33 at college do this (print characters
in the middle of the line while the carriage is moving back). But
maybe my memory is faulty and it was another printing terminal of that
era. Or maybe our ASR33 was broken.
-Paul W.
3:37 p.m.
On Tue, Sep 5, 2017 at 11:23 AM, Paul Winalski <paul.winalski(a)gmail.com>
wrote:
On 9/4/17, Pete Turnbull
<pete(a)dunnington.plus.com> wrote:
I agree with Paul, maybe later models worked this way, but the first ASR33
I used definitely did print at strange times. I have the same memories
from 1967 on a GE Mark III with Dartmouth Basic using an acoustic coupler,
where line noise was definitely a problem. It would print random
characters, particularly during carriage motion. Since the only way to
get a copy of your program was with the LIST command, it was rare to get a
clean copy. If the program was of any length, there was usually a random
character or two overprinted in the middle of listing.
Clem
I don't doubt it's possible, just not on an ASR33. Apart from the
timing issue, the carriage return mechanism on Teletype 33s inhibits
printing while the carriage is moving back, until it stops at the left
margin.
I could swear I saw our old ASR33 at college do this (print characters
in the middle of the line while the carriage is moving back). But
maybe my memory is faulty and it was another printing terminal of that
era. Or maybe our ASR33 was broken.
6:55 p.m.
On 9/3/17, Pete Turnbull <pete(a)dunnington.plus.com> wrote:
That's not right. Yes, the ASR33 runs at 110 baud (10cps), but all -
including mine and those of my friends - take more like half a dozen
character times to perform a CR from the right hand side. Moreover, the
time it takes depends in a non-linear fashionon how far along the line
the printhead has travelled; it's driven by a spring and decelerated and
stopped by an air dashpot. Even an LF takes more than two character
times. That's why ASR33 drivers send a string of nulls after a CR.
Many are smart enough to send the LF instead of the first null, if they
see that LF is the first character after CR.
I distinctly recall the ASR33 printing exactly two characters while
the print head is in motion, if a CR is issued while the print head is
at the right margin. The ASR33 had a very fast and forceful print
head return mechanism--you definitely didn't want to get your fingers
in the way! In my programs that sent data to an ASR33, I always
followed CR with two non-printing characters (typically LF/NUL) and I
never had a problem with characters printing during the print head
return.
I don't recall LF taking more than one character time to execute on
the ASR33. If it did, I never noticed it.
The DEC LA36 dot matrix terminal had a rather leisurely carriage
return mechanism, but it also could buffer up characters received
while the CR was in progress. It would then speed up over the normal
printing rate to catch up and empty the buffer. There was a third
party mod to the LA36 that you could buy that would cause it to always
think it was in catch-up mode and to print at the higher speed all the
time. This shortened the life of the print mechanism, but at our shop
we found the extra speed in printing worth the expense of repairing
early breakdown of the mechanism.
-Paul W.
7:13 p.m.
On 05/09/2017 19:55, Paul Winalski wrote:
I distinctly recall the ASR33 printing exactly two
characters while
the print head is in motion, if a CR is issued while the print head is
at the right margin. The ASR33 had a very fast and forceful print
head return mechanism--you definitely didn't want to get your fingers
in the way! In my programs that sent data to an ASR33, I always
followed CR with two non-printing characters (typically LF/NUL) and I
never had a problem with characters printing during the print head
return.
Well, I may have exaggerated the time a CR takes slightly, in fact
thinking of 5 or 6 perhaps I was remembering my other teleprinter: a 50
baud Creed 7B which I no longer have. I'm still sure it's normally more
than two character times. Yes, I know about keeping fingers away - and
how to adjust various parts of an ASR33, and I have the tech manuals.
I'm sitting next to a working ASR33 right now.
--
Pete
Pete Turnbull
4 Sep
4 Sep
5:44 p.m.
On 9/3/17, Steve Johnson <scj(a)yaccman.com> wrote:
I think the reason for this is pretty obvious. At the
time -- for
many of the paper terminals, line feed simply rotated the platen, and
the print head stayed at the same column position. Carriage return
returned the print head to the first column without advancing the
paper (remember overstriking?). But there were also some terminals
that both advanced the paper and returned the print head (I'm hazy
about this, but I think the IBM Selectric was one...).
IIRC, you're right about this. I'm pretty sure I remember the IBM
2741 and friends (the Selectric terminals) operating this way.
I do remember one guy who wrote his program to output LF/CR instead of
CR/LF. The teletype driver would do a LF and then space out to the
previous column and then do a carriage return. Pretty painful at 300
baud!
At 110 baud, the model 33 Teletype took up to two character times to
return the typeball to the left margin when executing a CR. You
needed to follow CR with two non-printing characters to avoid
erroneously striking a character in the middle of the line. With
CR/LF, the LF served as one of these non-printables. CR/LF/NUL was
the common sequence used to return the carriage and advance the
platen. If you did LF/CR you'd need LF/CR/NUL/NUL, thus wasting one
character time. So just about everyone implemented newline as CR/LF
rather than LF/CR.
Doug might remember better than I but I suspect some terminal did the
combined CR/LF only, so that's why Unix adopted it.
I suspect UNIX used LF as EOL for two reasons: [1] as just mentioned,
nearly everyone did CR/LF vs LF/CR, and [2] breaking lines on the LF
allowed for overstriking.
-Paul W.
8:30 p.m.
using cr/lf as end of line also introduced all kinds of problems on the DEC
OSes I used, to wit, what do
CR alone
LF alone
and LF/CR
mean? It was just a pain in the neck. I was grateful to get to an OS that
was sensible and just used LF.
8:41 p.m.
CR by itself could be used for overstrike (e.g. if you wanted to underline
words). Faster than char, backspace, overstrike-char if you wanted lots
of this. LF by itself, not so much. Could've been if there was an inverse
linefeed.
On Sep 4, 2017, at 1:30 PM, ron minnich
<rminnich(a)gmail.com> wrote:
using cr/lf as end of line also introduced all kinds of problems on the DEC OSes I used,
to wit, what do
CR alone
LF alone
and LF/CR
mean? It was just a pain in the neck. I was grateful to get to an OS that was sensible
and just used LF.
2419
days inactive
2421
days old
18 comments
11 participants
participants (11)
-
Bakul Shah -
Chris Torek -
Clem Cole -
Dave Horsfall -
Grant Taylor -
Nemo -
Paul Winalski -
Pete Turnbull -
ron minnich -
Steve Johnson -
Warner Losh