All, recently I saw on Bruce Schneier "Cryptogram" blog that he has had
to change the moderation policy due to toxic comments:
https://www.schneier.com/blog/archives/2024/06/new-blog-moderation-policy.h…
So I want to take this opportunity to thank you all for your civility
and respect for others on the TUHS and COFF lists. The recent systemd
and make discussions have highlighted significant differences between
people's experiences and opinions. Nonetheless, apart from a few pointed
comments, the discussions have been polite and informative.
These lists have been in use for decades now and, thankfully, I've
only had to unsubscribe a handful of people for offensive behaviour.
That's a testament to the calibre of people who are on the lists.
Cheers and thank you again,
Warren
P.S. I'm a happy Devuan (non-systemd) user for many years now.
[Moved to COFF. Mercifully this really has nothing to do with Unix]
On Wednesday, 19 June 2024 at 22:09:11 -0700, Luther Johnson wrote:
> On 06/19/2024 10:01 PM, Scot Jenkins via TUHS wrote:
>> "Greg A. Woods" <woods(a)robohack.ca> wrote:
>>
>>> I will not ever allow cmake to run, or even exist, on the machines I
>>> control...
>>
>> How do you deal with software that only builds with cmake (or meson,
>> scons, ... whatever the developer decided to use as the build tool)?
>> What alternatives exist short of reimplementing the build process in
>> a standard makefile by hand, which is obviously very time consuming,
>> error prone, and will probably break the next time you want to update
>> a given package?
>>
>> If there is some great alternative, I would like to know about it.
>
> I just avoid tools that build with CMake altogether, I look for
> alternative tools. The tool has already told me, what I can expect from
> a continued relationship, by its use of CMake ...
That's fine if you have the choice. I use Hugin
(https://hugin.sourceforge.io/) a panorama stitcher, and the authors
have made the decision to use cmake. I don't see any useful
alternative to to Hugin, so I'm stuck with cmake.
Greg
--
Sent from my desktop computer.
Finger grog(a)lemis.com for PGP public key.
See complete headers for address and phone numbers.
This message is digitally signed. If your Microsoft mail program
reports problems, please read http://lemis.com/broken-MUA.php
Sort of between kernel and user mode, Unix [zillion trademarks etc] never
used it. but did RSX-11?
I used the latter long enough to hate it until Edition 5 arrived at UNSW,
and I still remember being blown away by the fact that there was nothing
privileged about the Shell :-)
-- Dave
This report [link at end ] about a security issue with VMware Vsphere, stemming from the design/ architecture, resonated with me and the recent TUHS “Unix Philosophy” thread.
Many of the criticisms of Unix relate to not understanding it’s purpose and design criteria:
A platform on which to develop (other) Software. Which implies ‘running, profiling, testing & debugging’ that code.
Complaining that Unix tools/utilities are terse and arcane for non-developers & testers, needing a steep Learning Curve,
is the same as complaining a large truck doesn’t accelerate or corner like a sports car.
Plan 9, by the same core team twenty years later, addresses the same problems with modern hardware & graphics, including with Networking.
The system they developed in 1990 would’ve been proof against both vSphere attacks because of its security-by-design:
No ‘root’ user, hence no ’sudo’
and no complex, heavyweight RPC protocol with security flaws, instead the simple, lightweight & secure 9P protocol.
It seems Eric Raymond’s exposition on the “Unix Philosophy” is the basis of much of the current understanding / view.
In the ESR & other works cited on Wikipedia, I see a lot about “Userland” approaches,
nothing about the Kernel, Security by Design and innovations like ’shells’, ‘pipes’ and the many novel standard tools, which is
being able to Reuse standard tools and ’stand on the shoulders of giants’ [ versus constantly Reinventing the Wheel, poorly ]
ESR was always outside CSRC and from his resume, not involved with Unix until 1983 at best.
He’s certainly been a mover & shaker in the Linux and associated (GNU led) Open Source community.
<http://catb.org/~esr/resume.html>
ESR baldly states "The Unix philosophy is not a formal design method”,
which isn’t strictly untrue, but highly misleading IMHO.
Nor is the self-description by members of CSRC as having “good taste” a full and enlightening description of their process.
There’s not a general appreciation, even in Research & Academic circles, that “Software is Performance Discipline”,
in the same way as Surgery, Rocketry, Aviation, Music, Art and physical disciplines (dance, gymnastics, even rock climbing) are “Performance” based.
It requires both Theory and Practice.
If an educator hasn’t worked on at least one 1M LOC system, how can they teach “Programming in the Large”, the central problem of Software Engineering?
[ an aside: the problem “golang” addressed was improving Software Engineering, not simply a language & coding. ]
There’s a second factor common to all high-performance disciplines,
why flying has become cheaper, safer and faster since the first jet & crashes in 1950’s:
- good professionals deliberately improve, by learning from mistakes & failures and (perhaps) adopting better practices,
- great professionals don’t just ‘improve’, they actively examine how & why they created Errors, Faults & Failures and detect / remove root causes.
The CSRC folk used to hate Corporate attempts at Soft Skills courses, calling them “Charm School”.
CSRC's deliberate and systematic learning, adaption and improvement wasn’t accidental or incidental,
it was the same conscious approach used by Fairchild in its early days, the reason it quickly became the leader in Silicon devices, highly profitable, highly valued.
Noyce & Moore, and I posit CSRC too, applied the Scientific Method to themselves and their practices, not just what their research field.
IMO, this is what made CSRC unique - they were active Practitioners, developing high-quality, highly-performant code, as well as being astute Researchers,
providing quantifiably better solutions with measurable improvements, not prototypes or partial demonstrators.
Gerard Holtzman’s 1127 Alumni page shows the breadth & depth of talent that worked at CSRC.
The group was unusually productive and influential. [ though I’ve not seen a ‘collected works’ ]
<http://spinroot.com/gerard/1127_alumni.html>
CSRC/1127 had a very strong culture and a very deliberate, structured ‘process’
that naturally led to a world-changing product in 1974 from only ~30 man-years of effort, a minor effort in Software Projects.
perfective “iterative design”, rigorous testing, code quality via a variation of pair-programming,
collaborative design with group consultation / discussion
and above all “performant” code - based first on ‘correct’ and ’secure’,
backed by Doug McIlroy’s insistence on good documentation for everything.
[ It’s worth noting that in the original paper on the “Waterfall” development process, it isn’t "Once & Done”, its specifically “do it twice”, ]
[ the Shewhart Cycle, promoted by Deming, Plan - Do - Check - Act, was well known in Engineering circles, known to be very Effective ]
Unix - the kernel & device drivers, the filesystem, the shell, libraries, userland and standard tools - weren’t done in hurry between 1969 & 1974’s CACM article.
It was written and rewritten many times - far more than the ‘versions’, derived from the numbering of the manuals, might suggest.
Ken’s comment on one of his most productive days, “throwing away 1,000 lines of code”,
demonstrates this dynamic environment dominated by trials, redesign and rewriting - backed by embedded ‘instrumentation’ (profiling).
Ken has also commented he had to deliberately forget all his code at one point (maybe after 1974 or 77).
He was able to remember every line of code he’d written, in every file & program.
I doubt that was an innate skill, even if so, it would’ve improved by deliberate practice, just as in learning to play a musical instrument.
There’s a lot of research in Memory & Recall, all of which documents ‘astonishing’ performance by ‘ordinary’ people, with a only little tuition and deliberate practice.
CSRC had a scientific approach to software design and coding, unlike any I’ve seen in commercial practice, academic research or promoted “Methodologies”.
There’s a casual comment by Dennis in “Evolution of Unix”, 1979, about rewriting the kernel, improving its organisation and adding multiprogramming.
By one person in months.. A documented, incontestable level of productivity, 100x-1000x programmers practising mainstream “methodologies”.
Surely that performance alone would’ve been worthy of intensive study as the workforce & marketplace implications are profound.
<https://www.bell-labs.com/usr/dmr/www/hist.pdf>
Perhaps the most important watershed occurred during 1973, when the operating system kernel was rewritten in C.
… The success of this effort convinced us that C was useful as a nearly universal tool for systems programming, instead of just a toy for simple applications.
The CSRC software evolution methodology is summed by perfectly in Baba Brinkman’s Evolution Rap:
"Performance, Feedback, Revision”
<https://www.youtube.com/watch?v=gTXVo0euMe4>
Website: <https://bababrinkman.com/>
ABC Science Show, 2009, 54 min audio, no transcript
This is the performance Baba gave at the Darwin Festival in Cambridge England, July 2009.
<https://www.abc.net.au/listen/programs/scienceshow/the-rap-guide-to-evoluti…>
Ken also commented that they divided up the work coding, seemingly informally but in a disciplined way,
so that there was only ever one time they created the same file. [ "mis-coordination of work”, Turing Award speech ]
To prove they had well defined coding / naming standards and followed them, the two 20-line files were identical…
———————
There’s a few things with the “Unix Philosophy” that are critical and not included in the commentaries I’ve read or seen quoted:
- The Unix kernel was ‘conservative’, not inventive or novel.
It deliberately used only known, proven solutions, with a focus on small, correct, performant. “Just Worked”, not “Worked, Just”.
Swapping was used, while Virtual Memory not implemented because they didn’t know of a definitive solution.
They avoided the “Second System Effect” - showing how clever they were - working as professional engineers producing a robust, reliable, secure system.
- Along with Unix (kernel, fsys, userland), CSRC developed a high-performance high-quality Software Development culture and methodology,
The two are inseparable, IMO.
- Professionals do not, can not, write non-trivial code in a “One and Done” manner. Professional quality code takes time and always evolves.
It takes significant iterative improvement, including redesign, to develop large systems,
with sufficient security, reliability, maintainability and performance.
[ Despite 60 years of failed “Big Bang” projects using “One & Done”, Enterprises persist with this idioticy, wasting billions every year ]
- Unix was developed to provide CSRC with a great environment for their own work. It never attempted to be more, but has been applied ‘everywhere’.
Using this platform, members of the team developed a whole slew of important and useful tools,
now taken as a given in Software Development: editors, type settings, ‘diff’ and Version Control, profile, debug, …
This includes the computer Language Tools, now core to every language & system.
- Collaboration and Sharing, both ways, was central to the Unix Philosophy developed at CSRC.
Both within the team, within Bell Labs and other Unix installations, notably USENIX & UCB and it’s ARPA-IPTO funded CSRG.
The world of Software and Code Development is clearly in two Eras, “Before Unix” and “After”.
Part of this is “Open Source”, not just shared source targeted for a single platform & environment, but source code mechanically ported to new platforms.
This was predicated on the original CSRC / Bell Labs attitude of Sharing the Source…
Source was shared in & out,
directly against the stance of the Legal Dept, intent on tightly controlling all Intellectual Property with a view of extracting “revenue streams” from clients.
Later events proved CSRC’s “Source Code Sharing” was far more powerful and profitable than a Walled Garden approach, endlessly reinvesting the wheel & competing, not cooperating with others.
Senior Management and the old school lawyers arguably overestimated their marketing & product capability
and wildly underestimated the evolution of computing and failed to understand completely the PC era, with Bill Gates admonisment,
“You guys don’t get it, it’s all about Volume”.
In 1974, Unix was described publicly in CACM.
In 1977, USG then later Unix System Labs was formed to work on and sell Unix commercially, locking day the I.P., with no free source code.
In 1984, AT&T ‘de-merged’, keeping Bell Labs, USL and Western Digital - all the hardware and software to “Rule the World” and beat IBM.
In 1994, AT&T gave up being the new IBM and sold its hardware and software divisions.
In 2004, AT&T was bought by one of its spinoff’s, SBC (Southern Bell),
who’d understood Mobile Telephony (passing on to customers savings from new technology), merged and rebranded themselves as “A&T”.
The “Unix Wars” of the 1990’s, where vendors bought AT&T licenses, confusing “Point of Difference” with “Different & Incompatible”.
They attempted Vendor lock-in, a monopoly tactic to create captive markets that could be gouged.
This failed for two reasons, IMO:
- the software (even binaries) and tools were all portable, the barriers to exit were low.
- Unix wasn’t the only competitor
Microsoft used C to write Windows NT and Intel-based hardware to undercut Unix Servers & Workstations by 10x.
Bill Gates understood ‘Volume’ and the combined AT&T and Unix vendors didn’t.
================
VMware by Broadcom warns of two critical vCenter flaws, plus a nasty sudo bug
<https://www.theregister.com/2024/06/18/vmware_criticial_vcenter_flaws/>
VMware's security bulletin describes both of the flaws as "heap-overflow vulnerabilities in the implementation of the DCE/RPC protocol” …
DCE/RPC (Distributed Computing Environment/Remote Procedure Calls)
is a means of calling a procedure on a remote machine as if it were a local machine – just the ticket when managing virtual machines.
================
CHM, 2019
<https://computerhistory.org/blog/the-earliest-unix-code-an-anniversary-sour…>
As Ritchie would later explain:
“What we wanted to preserve was not just a good environment to do programming, but a system around which a fellowship could form.
We knew from experience that the essence of communal computing, as supplied from remote-access, time-shared machines,
is not just to type programs into a terminal instead of a keypunch, but to encourage close communication.”
================
Ken Thompson, 1984 Turing Award paper
Reflections on Trusting Trust To what extent should one trust a statement that
a program is free of Trojan horses?
Perhaps it is more important to trust the people who wrote the software.
That brings me to Dennis Ritchie.
Our collaboration has been a thing of beauty.
In the ten years that we have worked together, I can recall only one case of mis-coordination of work.
On that occasion, I discovered that we both had written the same 20-line assembly language program.
I compared the sources and was astounded to find that they matched character-for-character.
The result of our work together has been far greater than the work that we each contributed.
================
The Art of Unix Programming
by ESR
<http://www.catb.org/~esr/writings/taoup/html/index.html>
Basics of the Unix Philosophy
<http://www.catb.org/~esr/writings/taoup/html/ch01s06.html>
================
Wiki
ESR
<https://en.wikipedia.org/wiki/Eric_S._Raymond>
Unix Philosophy
<https://en.wikipedia.org/wiki/Unix_philosophy>
================
--
Steve Jenkin, IT Systems and Design
0412 786 915 (+61 412 786 915)
PO Box 38, Kippax ACT 2615, AUSTRALIA
mailto:sjenkin@canb.auug.org.au http://members.tip.net.au/~sjenkin
There seems to be some confusion, but I've heard enough sources now
that I believe it to be confirmed. Notably, faculty at UMich EECS have
shared that it was passed to them internally.
RIP Lynn Conway, architect of the VLSI revolution and long-time
transgender activist. She apparently died from heart failure; she was
86. http://www.myhusbandbetty.com/wordPressNEW/2024/06/11/lynn-conway-january-2…
- Dan C.
Could interest a few OFs here... I've used the -8 and of course the -11,
but not the -10 so I may as well start now.
-- Dave
---------- Forwarded message ----------
From: A fellow geek
To: Dave Horsfall <dave(a)horsfall.org>
Subject: PiDP-10 — The MagPi magazine
RasPi is now masquerading as a PDP-10…
https://magpi.raspberrypi.com/articles/pidp-10
Explore this gift article from The New York Times. You can read it for free
without a subscription.
C. Gordon Bell, Creator of a Personal Computer Prototype, Dies at 89
It cost $18,000 when it was introduced in 1965, but it bridged the world
between room-size mainframes and the modern desktop.
https://www.nytimes.com/2024/05/21/technology/c-gordon-bell-dead.html?unloc…
ᐧ
Good evening, I recently came into possession of a "Draft Proposed American National Standard for BASIC" circa September 15, 1980, 6 years prior to the publication of the Full BASIC standard.
I did some brief searching around, checked bitsavers, archive.org, but I don't happen to see this archived anywhere. Is anyone aware of such a scan? If not, I'll be sure to add it to my scan docket...which is a bit slow these days due to a beautiful spring and much gardening...but not forgotten!
Flipping through this makes me wonder how things might have been different if this dropped in 1980 rather than 1986.
- Matt G.
The Newcastle Connection, aka Unix United, was an early experiment in
transparent networking: see <
https://web.archive.org/web/20160816184205/http://www.cs.ncl.ac.uk/research…>
for a high-level description. A name of the form "/../host/path"
represented a file or device on a remote host in a fully transparent way.
This was layered on V7 at the libc level, so that the kernel did not need
to be modified (though the shell did, since it was not libc-based at the
time). MUNIX was an implementation of the same idea using System V as the
underlying system.
This appears to be a VHS vs. Betamax battle: NFS was not transparent, but
Sun had far more marketing clout. However, the Manchester Connection
required a single uid space (as far as I can tell), which may also have
been a (perceived) institutional barrier.
On Thu, May 16, 2024 at 3:34 AM Ralph Corderoy <ralph(a)inputplus.co.uk>
wrote:
> Hi,
>
> I've set ‘mail-followup-to: coff(a)tuhs.xn--org-to0a.
>
> > > Every so often I want to compare files on remote machines, but all
> > > I can do is to fetch them first (usually into /tmp); I'd like to do
> > > something like:
> > >
> > > rdiff host1:file1 host2:file2
> > >
> > > Breathes there such a beast?
>
> No, nor should there. It would be slain less it beget rcmp, rcomm,
> rpaste, ...
>
> > > Think of it as an extension to the Unix philosophy of "Everything
> > > looks like a file"...
>
> Then make remote files look local as far as their access is concerned.
> Ideally at the system-call level. Less ideal, at libc.a.
>
> > Maybe
> >
> > diff -u <(ssh host1 cat file1) <(ssh host2 cat file2)
>
> This is annoyingly noisy if the remote SSH server has sshd_config(5)'s
> ‘Banner’ set which spews the contents of a file before authentication,
> e.g. the pointless
>
> This computer system is the property of ...
>
> Disconnect NOW if you have not been expressly authorised to use this
> system. Unauthorised use is a criminal offence under the Computer
> Misuse Act 1990.
>
> Communications on or through ...uk's computer systems may be
> monitored or recorded to secure effective system operation and for
> other lawful purposes.
>
> It appears on stderr so doesn't upset the diff but does clutter.
> And discarding stderr is too sloppy.
>
> --
> Cheers, Ralph.
>
While the idea of small tools that do one job well is the core tenant of
what I think of as the UNIX philosophy, this goes a bit beyond UNIX, so I
have moved this discussion to COFF and BCCing TUHS for now.
The key is that not all "bloat" is the same (really)—or maybe one person's
bloat is another person's preference. That said, NIH leads to pure bloat
with little to recommend it, while multiple offerings are a choice. Maybe
the difference between the two may be one person's view over another.
On Fri, May 10, 2024 at 6:08 AM Rob Pike <robpike(a)gmail.com> wrote:
> Didn't recognize the command, looked it up. Sigh.
>
Like Rob -- this was a new one for me, too.
I looked, and it is on the SYS3 tape; see:
https://www.tuhs.org/cgi-bin/utree.pl?file=SysIII/usr/src/man/man1/nl.1
> pr -tn <file>
>
> seems sufficient for me, but then that raises the question of your
> question.
>
Agreed, that has been burned into the ROMs in my fingers since the
mid-1970s 😀
BTW: SYS3 has pr(1) with both switches too (more in a minute)
> I've been developing a theory about how the existence of something leads
> to things being added to it that you didn't need at all and only thought of
> when the original thing was created.
>
That is a good point, and I generally agree with you.
> Bloat by example, if you will. I suspect it will not be a popular theory,
> however accurately it may describe the technological world.
>
Of course, sometimes the new features >>are<< easier (more natural *for
some people*). And herein lies the core problem. The bloat is often
repetitive, and I suggest that it is often implemented in the wrong place -
and usually for the wrong reasons.
Bloat comes about because somebody thinks they need some feature and
probably doesn't understand that it is already there or how they can use
it. But they do know about it, their tool must be set up to exploit it - so
they do not need to reinvent it. GUI-based tools are notorious for this
failure. Everyone seems to have a built-in (unique) editor, or a private
way to set up configuration options et al. But ... that walled garden is
comfortable for many users and >>can be<< useful sometimes.
Long ago, UNIX programmers learned that looking for $EDITOR in the
environment was way better than creating one. Configuration was as ASCII
text, stored in /etc for system-wide and dot files in the home for users.
But it also means the >>output<< of each tool needs to be usable by each
other [*i.e.*, docx or xlx files are a no-no).
For example, for many things on my Mac, I do use the GUI-based tools --
there is no doubt they are better integrated with the core Mac system >>for
some tasks.<< But only if I obey a set of rules Apple decrees. For
instance, this email read is easier much of the time than MH (or the HM
front end, for that matter), which I used for probably 25-30 years. But on
my Mac, I always have 4 or 5 iterm2(1) open running zsh(1) these days. And,
much of my typing (and everything I do as a programmer) is done in the shell
(including a simple text editor, not an 'IDE'). People who love IDEs swear
by them -- I'm just not impressed - there is nothing they do for me that
makes it easier, and I have learned yet another scheme.
That said, sadly, Apple is forcing me to learn yet another debugger since
none of the traditional UNIX-based ones still work on the M1-based systems.
But at least LLDB is in the same key as sdb/dbx/gdb *et al*., so it is a
PITA but not a huge thing as, in the end, LLDB is still based on the UNIX
idea of a single well-designed and specific to the task tool, to do each
job and can work with each other.
FWIW: I was recently a tad gob-smacked by the core idea of UNIX and its
tools, which I have taken for a fact since the 1970s.
It turns out that I've been helping with the PiDP-10 users (all of the
PiDPs are cool, BTW). Before I saw UNIX, I was paid to program a PDP-10. In
fact, my first UNIX job was helping move programs from the 10 to the UNIX.
Thus ... I had been thinking that doing a little PDP-10 hacking shouldn't
be too hard to dust off some of that old knowledge. While some of it has,
of course, come back. But daily, I am discovering small things that are so
natural with a few simple tools can be hard on those systems.
I am realizing (rediscovering) that the "build it into my tool" was the
norm in those days. So instead of a pr(1) command, there was a tool that
created output to the lineprinter. You give it a file, and it is its job to
figure out what to do with it, so it has its set of features (switches) -
so "bloat" is that each tool (like many current GUI tools) has private ways
of doing things. If the maker of tool X decided to support some idea, they
would do it like tool Y. The problem, of course, was that tools X and Y
had to 'know about' each type of file (in IBM terms, use its "access
method"). Yes, the engineers at DEC, in their wisdom, tried to
"standardize" those access methods/switches/features >>if you implemented
them<< -- but they are not all there.
This leads me back to the question Rob raises. Years ago, I got into an
argument with Dave Cutler RE: UNIX *vs.* VMS. Dave's #1 complaint about
UNIX in those days was that it was not "standardized." Every program was
different, and more to Dave's point, there was no attempt to make switches
or errors the same [getopt(3) had been introduced but was not being used by
most applications). He hated that tar/tp used "keys" and tools like cpio
used switches. Dave hated that I/O was so simple - in his world all user
programs should use his RMS access method of course [1]. VMS, TOPS, *etc.*,
tried to maintain a system-wide error scheme, and users could look things
like errors up in a system DB by error number, *etc*. Simply put, VMS is
very "top-down."
My point with Dave was that by being "bottom-up," the best ideas in UNIX
were able to rise. And yes, it did mean some rough edges and repeated
implementations of the same idea. But UNIX offered a choice, and while Rob
and I like and find: pr -tn perfectly acceptable thank you, clearly someone
else desired the features that nl provides. The folks that put together
System 3 offer both solutions and let the user choose.
This, of course, comes as bloat, but maybe that is a type of bloat so bad?
My own thinking is this - get things down to the basics and simplest
privatives and then build back up. It's okay to offer choices, as long as
the foundation is simple and clean. To me, bloat becomes an issue when you
do the same thing over and over again, particularly because you can not
utilize what is there already, the worst example is NIH - which happens way
more than it should.
I think the kind of bloat that GUI tools and TOPS et al. created forces
recreation, not reuse. But offering choice and the expense of multiple
tools that do the same things strikes me as reasonable/probably a good
thing.
1.] BTW: One of my favorite DEC stories WRT to VMS engineering has to do
with the RMS I/O system. Supporting C using VMS was a bit of PITA.
Eventually, the VMS engineers added Stream I/O - which simplified the C
runtime, but it was also made available for all technical languages.
Fairly soon after it was released, the DEC Marketing folks discovered
almost all new programs, regardless of language, had started to use Stream
I/O and many older programs were being rewritten by customers to use it. In
fact, inside of DEC itself, the languages group eventually rewrote things
like the FTN runtime to use streams, making it much smaller/easier to
maintain. My line in the old days: "It's not so bad that ever I/O has
offer 1000 options, it's that Dave to check each one for every I/O. It's a
classic example of how you can easily build RMS I/O out of stream-based
I/O, but the other way around is much harder. My point here is to *use
the right primitives*. RMS may have made it easier to build RDB, but it
impeded everything else.