On Sat, Dec 12, 2020 at 8:07 PM Theodore Y. Ts'o <tytso@mit.edu> wrote:
On Wed, Dec 09, 2020 at 02:58:27PM -0500, Dan Cross wrote:
> To circle back to plan 9 for a moment, this was something that the open
> source folks who found their way to 9fans just couldn't grok. The answer to
> the question, "why don't you do all this work to support (emacs|a web
> browser|a C++ compiler|whatever du jour)?" was, "because there's little
> inherent research value in doing that, and this is a research system." That
> it was also a workaday system for a handful of folks was incidental; the
> goal wasn't world domination, it was advancing research and providing a
> comfortable environment for that work. Linus's response exemplifies this
> lack of understanding. (Disclaimer: I was very much an outsider looking in
> there, but it seems clear enough in retrospect.)

There was a similar dynamic with Minix, where Prof. Tanenbaum rejected
contributions to Minix because Minix wa a teaching system, and he
wanted to keep it simple.

Yes, but his aim there was pedagogy, not general use. In that I would argue that he was successful.

The contrast is that with Linux, that contributions are accepted from
a large number of people, working at a large number of companies, that
all have different goals, and the challenge of maintainers is to
balance off the goals of many different contributors.  Contributions
don't get rejected just because "this is a {research,testing} OS".
The goal is to make the open source project as generally useful as
possible.

Sure, but that's Linux. Linux is not (Minix|Plan9|L4). QED, no?

Judging (Minix|Plan9|L4) by the same metrics by which one judges Linux is not useful if those systems never aspired to those metrics.

>     And notice how they aren't all that popular or well known? "Design" is
> >     like a religion - too much of it makes you inflexibly and unpopular.
>
> That's a terrible metric.
>
> I submit that neither of those systems were created with the explicit goal
> to become "popular", and the claim of inflexibility is unwarranted. Within
> their domain, that is as research systems, both are quite well known and
> remain highly influential.

>From the open source perspective, it's an extremely important metric,

But the systems Linus mentioned weren't trying to be "successful" open source systems by that definition. That's the point: the central thesis here is that _that may be an important metric for Linux_, and by that metric, Linux is very successful.

However Linus Torvalds is suggesting that other systems that explicitly didn't judge themselves along those metrics are _unsuccessful_ in an absolute sense because they didn't "succeed" by the metrics along which Linux succeeded. This is analogous to suggesting that Yo-Yo Ma isn't "successful" because he can't dunk a basketball like Michael Jordan, and consequently isn't as famous. I'm willing to bet that more people know Jordan's name than Ma's, and Ma himself would likely be among the first to admit he's not NBA material, but that doesn't mean he isn't wildly successful as a classically trained cellist. For that matter, Mick Jagger is probably better known than Yo-Yo Ma, but I don't think the latter ever aspired to be the frontman for the Rolling Stones, but I would never suggest that he isn't an accomplished musician as a result.

Recall that the original context was a defense of Linux's purely organic evolution vs some notion of "design up front" that Linus suggests both L4 and Plan9 sprang from. Nevermind that that is almost certainly not true of either L4 or plan 9, but rather it's nonsensical to suggest that a more design-oriented focus was the reason they weren't as successful as Linux (as measured by popularity and deployment) as it completely ignores that neither L4 nor plan9 were trying to do what Linux did in the first place.

since if a system is generally useful, such that many different
entities find the system to be useful, that means that the project
will have more and more contributors.  Yes, those contributors may
have differing objectives, but this also gives you a larger
development community to make the project more useful.

At one time, MS-DOS was wildly successful. There was a ton of software written for it. That software was useful to a lot of people. But it would be hard to argue that DOS was "better" on some technical plane than Unix.

I realize that some of this is moving the goalposts because no one defined what "good" means in context. My definition includes things like complexity, maintainability, and elegance. Some parts of Linux are nice here; some ... not so much.

The challenge is how to structure the project so that you can usefully
use a larger and larger number of contributors, and how to mediate
conflicts when objectives are in tension with each other.  (For
example, sometimes adding lots of fine-grained locking to improve CPU
scalability often comes at the cost of trashing UP and small SMP
performance.)

However, it's surprising how often that with the right amount of
encouragement, things like SMP vs UP performance is not an either/or,
but a both/and.  Granted, at the extremes, this isn't always going to
be true.  If you have to squeeze an OS into super-tiny
micro-controller, or if you want to optimize scalability for a
massiely large Sunfire E10k/E12k/E15k server, the only way to do this
is with a huge number of fine-grained locks in Solaris.  (And given
the profit margins on million dollar E10k versus a cheap Ultrasparc 5
workstation, it's not surprising that Solaris would optimize
performance for an E10k.)

> This is a common but annoying line of thought in the computer world:
> because something is useful and popular, it is good. My first car was a
> 1985 AMC Eagle; it was undeniably useful. It may have even been moderately
> popular at one point. But damn it was an awful car.
>
> Linux is undeniably useful and it's arguably the most popular operating
> system in the world. And parts of it are really, really good. But simply
> put, that doesn't mean that its evolutionary path has landed in an
> inherently good place.

The question is what your objective function such that you consider
the endpoint evolutionary path is "a good place"?

Yes, I cheated here by not offering a definition.

My pre-existing
values are that a system is "good" if it can add value for many
different applications.

Well, my AMC Eagle got me around town, I drove to Bell Labs and back (from central PA) in it when I was 16, and it had a hitch attachment that could haul a camper. The engine was awesome and so was the heater, but every time I drove over a speedbump a part fell off. If it was "good" by the metric of adding value for different applications, then a car that did the same things and where the rearview _didn't_ fall off every time I ran over a pothole would have been better.

So I have a bit of an engineer's perspective of a system is good
because it is useful --- and part of being useful is that it is
secure, and reliable, and cost effective.  Having a clean architecture
is useful in so far as it makes reduces maintenance overhead and
improves reliability.  But forcing everything to use a file interface
merely for aethestics' sake is not terribly important for _my_
objective function.

One might argue that that misses the point of the interface: using the file interface now allows pretty much all resources to be shared over the network transparently, for example. Was it a purely aesthetic decision, or was it a research angle that opened up new areas for exploration? Indeed, some of the ideas that fell out of that as a consequence of "forcing" everything to look like a file are now in Linux. Per-process namespaces and /proc are obvious examples.

But taking it back to Linus's point...Whether one system uses that file interface and another does not is immaterial, and I don't know that anyone seriously argued that the popularity of the system wasn't predicated on that. What you're describing here is a consequence of Linux's popularity; Linux wasn't more popular than plan9 necessarily because it had mmap and sockets and TTYs, but rather because it cloned an existing design that was already very popular and packaged that up free of the legal and financial entanglements of Unix, and also because plan9 just _wasn't trying to be popular in the same way_.

And if popularity means that I can have engineers from Tencent, and
Huawei, and IBM, and SuSE, and Oracle, and Google all helping me make
a better file system for Linux, as opposed to having one company
shoulder all of the development costs --- then heck yes, I'll take
popularity any day.

That's fair, but that wasn't the original context.

        - Dan C.