More probably speed; those old disks were not fast, and on a PDP-11, disk caches were so small that converting the path to the current directory to its in memory inode could take a bunch of disk reads.

Depends; if a directory _has_ to be empty before it can be deleted, maybe; but if not, no. (Consider if /a/b/c/d exists, and /a/b is removed; the tree underneath it has to be walked and the components deleted. That could take a while...) In the general case (e.g. without the restriction to a tree), it's basically the same problem as garbage collection in LISP.

Noel

On Wed, Dec 29, 2021 at 10:45:12PM -0500, Noel Chiappa wrote: The other problem with storing the path as a string is that if higher-level directories get renamed, the path would become invalidated. If you store the cwd as "/foo/bar/baz/quux", and someone renames "/foo/bar" to "/foo/sadness" the cwd-stored-as-a-string would become invalidated.

Sure, you could scan the pwd of all the processes running on the system and do edit all of the strings, but once you go SMP, the potential locking pitfalls would be numerous.

On Dec 30, 2021, at 2:31 PM, Dan Cross <crossd(a)gmail.com> wrote: Without the ".." entry you can't map a dir inode back to a path. Note that something similar can happen even today: $ mkdir ~/a; cd ~/a; rm -rf ~/a; cd .. cd: no such file or directory: .. $ mkdir -p ~/a/b; ln -s ~/a/b b; cd b; mv ~/a/b ~/a/c; cd ../b ls: ../b: No such file or directory You can't protect the user from every such case. Storing a path instead of the cwd inode simply changes the symptoms.

? I was just explaining Ts'o's point, not agreeing with it. The first example I gave works just fine on plan9 (unlike on unix). And since it doesn't allow renames, the scenario T'so outlines can't happen there! But we were discussing Unix here. As for symlinks, if we have to have them, storing a path actually makes their use less surprising. We're in the 6th decade of Unix and we still suffer from unintended, fixable consequences of decisions made long long ago. No argument here. Perhaps you can suggest a path for fixing? On Dec 30, 2021, at 5:00 PM, Rob Pike <robpike(a)gmail.com> wrote: Grumpy hat on. Sometimes the Unix community suffers from the twin attitudes of a) believing if it can't be done perfectly, any improvement shouldn't be attempted at all and b) it's already done as well as is possible anyway. I disagree with both of these positions, obviously, but have given up pushing against them. We're in the 6th decade of Unix and we still suffer from unintended, fixable consequences of decisions made long long ago. Grumpy hat off. -rob On Fri, Dec 31, 2021 at 11:44 AM Bakul Shah <bakul(a)iitbombay.org> wrote:

On Thu, Dec 30, 2021 at 05:31:09PM -0500, Dan Cross wrote: was done disk directory to its I was responding to Bakul's suggestion that the original Unix could/should have stored the cwd as a string, instead of a pointer to a directory inode. If you stored the cwd as a string, then you could interpret .. by chopping off the last file name component of the string, and so now you could have hard links to directories and the file system tree could then be a file system DAG. I don't think Bakul's proposal would have worked --- although I suppose if you disallowed directory renames, I guess it could. Noel's objection above was a performance one, in that if you had to parse the path every single time, you'd be much more dependent on the namei cache --- and that PDP-11 didn't have the memory to support a sufficiently large cache. - Ted

In broad strokes, that's exactly what we did in Plan 9, with great results, as was said here a while back. The paper ended with a plea to do the same to Unix, which I think is quite doable. https://9p.io/sys/doc/lexnames.html

On Fri, Dec 31, 2021 at 02:23:15PM +1100, Rob Pike wrote: results, I'm sure it *could* be done. In fact, Linux has a number of the pieces of the implementations already, which could be reused for a Plan-9-style lexical namespace. Linux has bind mounts, and a dentry is essentially a cached directory lookup (it contains a pointer to its parent dentry, the filename used to do the lookup and a pointer to the resulting inode). And a struct file contains a struct dentry, which means in Linux it is possible to determine the path used to open a particular file. There are some differences of course; so it's not exact, but I agree with you that it is quite "doable". (In particular, how Linux dentries and symlinks interact appears to be different from how Plan 9 and Channels did things, if I understand the paper correctly.) The question though, is whether Linux *should* do it. Or rather, *could* we do it without breaking backwards compatibility with users and application which expect that dot-dot works the way it does in classical Unix implementations for the past 4 decades or so. Granted that the combination of symlinks, and hard links, and in Linux, bind mounts (not to mention bind namespaces, chroots, and pivot_root operations), can be confusing. But there is a whole ecosystem of shell scripts, programs, user's finger macros, etc., that would break if we "made things better" or "made things simpler". One of the benefits of research OS's is that you can experiment with things without having to worry about the howls of angry users when you break backwards compatibility, or eliminate some particular awkward API or feature. - Ted P.S. For example, how I wish I could eliminate telldir/seekdir, or at least change things so that telldir/seekdir used a substantially larger "cookie" to memoize a position in a directory's readdir stream which has to be valid effectively forever. Today, you can pass a 32-bit telldir cookie to seekdir, and readdir MUST not return a directory entry that it had already returned, and MUST not skip a directory entry that was present at the time of the telldir(). Thus spake POSIX, and there are applications which depend on telldir/seekdir/readdir having this precise set of semantics If your directory is stored internally as a linear stream, that's not too bad; but if you're using a more complex data structure, life gets interesting. I know of one file system (JFS) which added an extra, separate, on-disk b-tree structure to each directory just to be able to provide stable telldir/seekdir cookies. (And of course, there's a performance impact to needing to update an extra b-tree each time a directory entry is added or removed.) With a research OS, it's at least possible to say, "Well, *that* was a mistake; let's pretend it never happened and let's see how much better life would be." :-)

On 31 Dec 2021 16:55 +1100, from robpike(a)gmail.com (Rob Pike): Unix to and Seems to me that we have a decent, relatively current case study of this with Python 2 vs Python 3. Which is probably a good bit easier actually, because maintaining the ability to run Python 2 applications while introducing the ability to run Python 3 applications is likely easier than introducing changes to how something as relatively fundamental as how ".." behaves in paths while at the same time maintaining backwards compatibility with the old behavior, where different users and different applications on the same system might very well expect different behavior... -- Michael Kjörling • https://michael.kjorling.se • michael(a)kjorling.se “Remember when, on the Internet, nobody cared that you were a dog?”

Slightly older, but also slightly more fundamental to the system, you need look no farther than Solaris's `/bin/sh` for an illustrated example of the pros and cons of maintaining backwards compatibility. [snip]

I suppose one could always install `bash` on Solaris as a shell lingua franca, as well

Without the ".." entry you can't map a dir inode back to a path. Note that something similar can happen even today: $ mkdir ~/a; cd ~/a; rm -rf ~/a; cd .. cd: no such file or directory: .. $ mkdir -p ~/a/b; ln -s ~/a/b b; cd b; mv ~/a/b ~/a/c; cd ../b ls: ../b: No such file or directory

On Fri, Dec 31, 2021 at 01:17:18PM -0500, Dan Cross wrote: I'm sure that there were people at Sun who *did* care.

The story I had heard was that it was a decision made at the C-suite level, and was a quid-pro-quo where to get that sweet, sweet, cash from AT&T so Sun could stay afloat, they had to switch over to System V.

(No matter that Solaris 2 was a major step *backwards* in terms of performance and stability compared to Sun OS....)

Are there distros who are no longer supplying ifconfig/netstat/route, at least as an optional package? That's surprising if so.

All of the kernel interfaces to allow the old-style net-tools packages to work, as well as the BSD-style ioctls/setsockopt, etc., are still around, and fully supported. At least on my systems, I still install net-tools because my finger macros are still used to using ifconfig, netstat, and friends.

The reason why ip and ss were added was because there was a significant amount of new functionality that was added to the Linux networking stack (especially relating to routing and address aliasing) that couldn't be expressed using the older C programming interfaces as well as the ifconfig/route shell commands.

There were two north star principles about the new networking interfaces: 1) The old interfaces were always supposed to continue to work, and if you didn't need the new functionality, there was no reason to use the newer interfaces. 2) The new interfaces were *supposed* to be a strict superset of the old interfaces. If in fact ip and ss don't support AX.25, or other "exotic address families" --- that's a bug, and should be reported as such.

That being said, if you don't need the fancy new features, there's no reason to switch away from ifconfig. The whole *point* of the first principle was that we didn't want to force users to do a forced Python 2.7 -> Python 3 style "long march" migration.

On Mon, Jan 03, 2022 at 08:35:24AM -0500, Dan Cross wrote: Yeah, to be fair, by the time Solaris 2.3 or 2.4 came around, it was mostly up to par. (Or maybe it was because Moore's law meant that we didn't care any more. :-)

Sure, but there's no *point* to sunset the old tools. The git tree for net-tools is still being actively developed (the last commit was dated December 12, 2021). And the kernel interfaces are not going to be disappear, because of the prime directive: Thou Shalt Not Break Userspace. Well, take a look at the ip-route man page. The BSD route command assumes fundamentally there is a single routing table that you can update. In Linux, there are multiple routing tables --- to support NAT, VRF (virtual routing and forwarding), etc.

I suspect the other consideration was that all of this extra functionality and complexity were done by folks who wanted the Linux networking stack to essentially have pretty much all of the functionality of a Cisco Router. So it made sense to create a new user interface interface that was inspired by the Cisco IOS configuration language. Now, if you weren't trying to do something ala a router in the default-free zone, and were just simply doing what most leaf nodes on the internet (99.99999% of the hosts), there really is no reason to need to use the ip/ss interface. For that matter, you probably don't need to use ifconfig/route --- just let the DHCP client server of your choice take care of setting up the network, and you're done.

That's a distro-level choice. And for most users, their networking is automatically brought up using NetworkManager, or some such, so they probably don't care. And it's not like installing a package is that painful. I don't see users of say, mysql complaining that they have to install that package should they want to use it. I'm old school, and since I generally tend to install BIND, that will drag in net-tools as dependency, so all my systems have ifconfig installed. But I'm not going to have a lot of sympathy for someone who thinks that "sudo apt-get install net-tools" is massive inconvenience.

That'll work on a laptop or on my home network (where I set up the DHCP server). In a large-scale datacenter environment, maybe not so much.

I would suggest that the number of users who want to run MySQL is much smaller than the number who want to have a functioning network. But you're right; it's not that hard to adapt. That was kind of the point; there have been cases where Linux users have adapted to one degree or another.

The `apt install net-tools` thing is a red-herring, though: that's explicitly why I mentioned Google prod. What works on a single system doesn't necessarily scale to O(10^6) machines supporting O(10^7) separately running instances of O(10^4) services in O(10) globally distributed datacenters, that's just a single organization.

On Mon, Jan 03, 2022 at 04:15:08PM -0500, Dan Cross wrote: I've seen a number of large-scale data center environments that use DHCP; and those that don't in Linux-land generally just edit a config file in /etc/network/interfaces or /etc/network/interfaces.d --- and then the init scripts would just use "ifup -a" which would bring up the network either using DHCP or the hard-coded values in /etc/network/interaces. No need to have sysadmins manually typing "route add ..." commands except in extreme debugging situations.

I actually suspect the number of users who want to run MySQL is actually larger than the number of users who need to manually configure a network using /sbin/ifconfig and /sbin/route.... :-)

Google prod's easy --- Google maintains its own minimal prod distribution where a small set of packages (less than a few dozen) that were originally forked from Debian a number of years ago. So it's trivial for us, because net-tools is installed by default and there is **no** engineering upside for us to switch to ip/ss --- so we haven't. :-)

Similarly, we're still using a *very* old version of bash (having stayed back across multiple major releases) because it's easier to stick with an old version of bash and keep up with the security fixes, than it would be to audit a gazillion shell scripts and shell script fragments for the various potential and subtle backwards-incomaptible changes that bash has made over the past decade.

Larry has told a similar story about the advantages of using a "dead" language like Tcl for Bitkeeper, since he didn't have to deal with gratuitous version skew problems caused by backwards incomaptible changes in Perl or Python. But if you manage your own userspace, including affording to do your own FEDRAMP-compliant security updates, you can just choose *not* to upgrade to newer versions of bash or network utilities. Who *says* you have to use bash 5.1? Or switch to ip/ss?

On the other hand, we did spend untold engineering hours migrating from Python 2.7 to 3.x (and *wow* was that painful) because in that case, that was considered less work and more secure than independently supporting Python 2.7 ad infinitum (also, we have fewer Python programs than we have shell script fragments in config files like Makefiles as well actual shell scripts).

So the decision about whether to follow breaking interfaces/language changes is an engineering decision that's made on a case by case basis, as it should be for all organizations. In the case of bash and net-tools, one decision was made.

In the case of Python, a difference decision was made --- although a lot of people really weren't happy that the Python developers didn't appreciate the cost of breaking backwards compatibility (or rather, they decided that they didn't care and prioritized their own convenience over that of their users). This is why I'm a big supporter of Linus's, "Thou Shalt Not Break Userspace" rule; backwards compatibility at the lowest levels is important! I'd draw that line higher than some people, but hey, if you don't like the instability of Python, you can do what Bitkeeper did and base your extension language/scripts on something like Tcl instead.

P.S. If you want to see the horror of trying to support a Python program that has to be able to run on a wide variety of Linux distros, running different versions of Python (from RHEL 6 and Python 2.3 up to the newest bleeding edge Python 3.x), take a look at the Google's gcloud program. Alas, unlike the Google prod environment, we can't dictate what distro our Cloud customers might choose to use. The gcloud command (part of the Google Cloud SDK) has to play all sorts of portability games, and people need to continually test across a wide range of Python versions and Linux distros to provide assurance that it continues to correctly as new features are added and bugs are fixed. I *completely* understand why Larry chose to implement Bitkeeper extensions in Tcl, since it avoids this problem.

Bt yeah, it's why I wouldn't want to change dot-dot, since having seen the pain the Python developers inflected on me, I wouldn't want to inflict it on others...

On Mon, Jan 03, 2022 at 04:15:08PM -0500, Dan Cross wrote: I'd agree, 2.4 was pretty slow and chunky, 2.5 was alright, but 2.5.1 was quite usable and stable. Also by this time, the hardware was going in directions that SunOS wouldn't keep up with.

On Mon, Jan 03, 2022 at 05:21:51PM -0600, Doug McIntyre wrote: Yeah, Doug is right, SunOS was pretty simple, it didn't really take advantage of SMP, Greg Limes tried to thread it but it was too big a job for one guy. That's not to say that SunOS couldn't have evolved into SMP, I'm 100% sure it could have. It just didn't. It's a shame.

On Mon, Jan 03, 2022 at 05:21:51PM -0600, Doug McIntyre wrote: Yeah, Doug is right, SunOS was pretty simple, it didn't really take advantage of SMP, Greg Limes tried to thread it but it was too big a job for one guy. That's not to say that SunOS couldn't have evolved into SMP, I'm 100% sure it could have. It just didn't. It's a shame.

Of course, that's assuming that Sun could have stayed afloat without that injection of cash from AT&T....

On 1/3/2022 6:28 PM, Theodore Ts'o wrote: What causes you to assert that Sun was at risk of not staying afloat? Sun went from $0/yr to over $1B/yr in revenue between 1982 and 1988, the prototype for what people now call "unicorns". Between 1985 and 1989 Sun grew at a compounded rate of 145% a year, and according to Forbes was the fastest growing company in the US in those years. Doesn't sound like a company foundering to me, certainly not in 1987 and 1988. Didn't sound like it to AT&T either, who wanted in on the action and so bought a bunch of Sun stock on terms very favorable to Sun (and financially speaking of benefit to both companies when AT&T divested some years later). The injection of capital was certainly useful to Sun, not because of distress or failure, but because it was bursting at the seams from all the growth. Sun was going to get that capital without AT&T by going to the market anyway, that it was able to do so on more favorable terms with an already established partner was literally an example of the rich getting richer. The partnership to inject SunOS technologies, do SVR4, harmonize the various UNIX flavors had already been committed and launched some months before the investment occurred and wasn't contingent upon it. They're not unrelated of course, the investment occurred in the context of the already committed partnership. And if you examine the announced expectations of that partnership it included some strong dependencies on Sun products and technologies by AT&T in both the near and long term. The gestalt of the investment was that it was a consequence of "well, if we're already doing all this, then..." Certainly the later transitions in Sun's products had lots of issues. But lessons aren't gained from "well, they meant well, but, poor sods, they were barely staying alive" especially when the premise isn't even remotely accurate. It's a much more interesting examination to consider: "they had agency, they made choices, the context was <pretty complex>, why that and not this, what was the alternative, etc." Since we can have the facts, why not premise the discussions on those?

OK, serious question. Why did Sun "sell out" to AT&T if they could have raised capital in the market? Did the executive team just not care about what moving to System V would mean? Or did they not understand it? How desperate were they? And why were they so desparate? Were there other alternatives they could have pursued? I too lived through those times, from the outside, and it was indeed a shocker when they moved from SunOS to Solaris. Thanks, Arnold Larry McVoy &lt;lm(a)mcvoy.com&gt; wrote: > Rob, you've painted this picture before and it just doesn't match up > at all with what I saw. I was there. Ken Okin paid me for 6 months to > try and convince the execs to not go to SVR4. If it was all sweetness > and light, why did he do that? > > You make it sound like it was a nice friendly deal. It was $200M of > Sun stock at 35% over market rate. Who does that just to be friends? > You say you could have gone to market and gotten that, well, sure, > but not at 35% over market. > > SunOS was winning all the deals, as you point out, Sun was growing > like crazy. I did lots and lots of customer presentations and not once > did I hear "oh, when are you guys going to standardize on System V?" > > Quite the opposite in fact. I was doing a talk at the Moscone center > on the first Sun cluster that I had built, Scott refused to let me ship > it with SunOS, so I was being a good soldier and talking up Solaris. > Over and over and over I was asked why we were forcing people to use > an obviously worse OS and I finally lost it and said "I know, I have > everything working in SunOS but they won't let me ship it." > > I was on tape. Okin listened to the tape and said "get every copy of > those tapes and destroy them". Not exactly the friendly everyone > agreeing picture you paint. Not remotely like that. > > Rob, I lived through that time. One of my close friends, John Pope, > did the bring up. He felt alienated from all the other kernel people > because we all loved SunOS and we hated Solaris, it made zero sense to > go that far backward. And look what it got you. Sun is gone. > > If you could have gone to market and gotten the money, you would > have. Sun was killing it, everyone wanted a Sun over every other > workstation, in the SunOS days, every open source thing just built > on Suns. Everyone else had to twiddle the makefiles and the source. > Your story that people wanted a standard just doesn't hold up when > you were the standard. It also doesn't hold up when every single > customer I talked to (and I talked to them at least twice a month > in formal presentations and frequently more like twice a week, > the sales people loved me) never asked for SVR4. Not once did that > come up. And when SVR4 was being pushed on them, 100% of them > pushed back and wanted SunOS. > > On Tue, Jan 04, 2022 at 01:28:20AM -0800, Rob Gingell wrote: > > On 1/3/2022 6:28 PM, Theodore Ts'o wrote: > > >Of course, that's assuming that Sun could have stayed afloat without > > >that injection of cash from AT&T.... > > > > What causes you to assert that Sun was at risk of not staying afloat? > > > > Sun went from $0/yr to over $1B/yr in revenue between 1982 and 1988, the > > prototype for what people now call "unicorns". > > > > Between 1985 and 1989 Sun grew at a compounded rate of 145% a year, and > > according to Forbes was the fastest growing company in the US in those > > years. > > > > Doesn't sound like a company foundering to me, certainly not in 1987 and > > 1988. Didn't sound like it to AT&T either, who wanted in on the action and > > so bought a bunch of Sun stock on terms very favorable to Sun (and > > financially speaking of benefit to both companies when AT&T divested some > > years later). > > > > The injection of capital was certainly useful to Sun, not because of > > distress or failure, but because it was bursting at the seams from all the > > growth. Sun was going to get that capital without AT&T by going to the > > market anyway, that it was able to do so on more favorable terms with an > > already established partner was literally an example of the rich getting > > richer. The partnership to inject SunOS technologies, do SVR4, harmonize the > > various UNIX flavors had already been committed and launched some months > > before the investment occurred and wasn't contingent upon it. > > > > They're not unrelated of course, the investment occurred in the context of > > the already committed partnership. And if you examine the announced > > expectations of that partnership it included some strong dependencies on Sun > > products and technologies by AT&T in both the near and long term. The > > gestalt of the investment was that it was a consequence of "well, if we're > > already doing all this, then..." > > > > Certainly the later transitions in Sun's products had lots of issues. But > > lessons aren't gained from "well, they meant well, but, poor sods, they were > > barely staying alive" especially when the premise isn't even remotely > > accurate. > > > > It's a much more interesting examination to consider: "they had agency, they > > made choices, the context was <pretty complex>, why that and not this, what > > was the alternative, etc." Since we can have the facts, why not premise the > > discussions on those? > > -- > --- > Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm

you can just choose *not* to upgrade to newer versions of bash or network utilities. Who *says* you have to use bash 5.1? Or switch to ip/ss?

On Monday, 3 January 2022 at 15:44:11 -0800, Larry McVoy wrote: was quite usable and stable. wouldn't keep up with. advantage guy. An interesting question. I had always thought that SMP was (one of?) the technical reasons why Sun moved from a BSD to a System V base. Since then, of course, we've done lots of work on SMP support for at least FreeBSD. Does anybody have an overview of how good the support is compared to modern Solaris? Is there any intrinsic reason why one should be better than the other?

Larry has told a similar story about the advantages of using a "dead" language like Tcl for Bitkeeper, since he didn't have to deal with gratuitous version skew problems caused by backwards incomaptible changes in Perl or Python.