[TUHS] "What UNIX Cost Us"

[Adam Thornton]

> On Jan 19, 2020, at 9:33 AM, Warner Losh <imp at bsdimp.com> wrote:
> 
> Benno's talks (systemd and this one) weren't wrong. Systemd *is* a dumpster fire. It has a lot of cool ideas, but is coded by someone that has poor listening skills and is more stubborn than he's technically competent. It's had a crapton of severe security bugs in it. It's given us abominations like eth4156 as a NIC name. It doesn't like it when you & a job and log out for Pete's sake. It's a total mess that breaks everything to try to push the state of the art.


Hoo boy.

I just had this argument on one of the Slacks I’m on.


> Beno's talk on it may have been a little over the top, but he's not wrong about much of his criticism. Systemd has swung too far from the do one thing and do it well philosophy, admittedly in ways that are ham-fisted and don't necessarily mean that it's philosophically wrong, that it shows at least some of thee wisdom of simplicity.


No, it’s philosophically wrong.

I will grant that there is a problem there that needs solving.  Let’s look at BSD rc and SysVinit; they’re both extremely difficult to automate installing and uninstalling services.  SysVInit works better than rc in that regard, but they’re both kind of crappy.  Let’s look at the things that suck about SysVInit:

1: no actual dependency graphs.  Sure, there was a whole framework of magic comments to sorta-kinda-glue one in.  Didn’t work very well.
2: tons of shell ceremony around start/stop/restart

What you want in an init system is:
First and foremost, a process manager, that acts as a signal handler of last resort.
  * It would be nice if it _can_ and _usually does_ run as PID 1, but doesn’t insist on it.
  * Has a firm notion of “this service depends on _that_ service” and can sequence startup and shutdown appropriately.
  * Has a declarative syntax that lets you configure most services with just a little declarative text, probably with an escape hatch for more complicated services

Here’s what you don’t want _in your init system_:

A new approach to logging that puts your logs in a human-unreadable binary format.

A system-wide event bus.  This is actually quite a good idea, but it doesn’t need to be part of the init system.

A sound manager.

You get the idea.

Yeah, SysVInit needed to be superseded.  Runit, Daemontools, Upstart all had reasonable approaches.  But the devouring hippo of SystemD won, mostly because Lennart worked at Red Hat.  And it’s a terrible idea because it tangles all these things together.  It’s also a terrible idea because your init system should have goals other than time from pressing the power button to getting a login prompt on Lennart’s laptop, but I digress.

> 
> Figure a  dozen file paths out, cat the right thing to them so other files show up and  then you can  do the same thing again? That's  not a sane interface. Everything isn't a file.  

[…]

To be fair: BSD sockets are not pretty, and they’re not elegant.  They won, and SysV Streams were worse in a lot of ways, but I still suspect there was some way of doing a TCP/IP stack that seemed more Unixy, without (maybe) needing DNS (or a service locator) in a sidecar userland process…but as soon as you start thinking about it, yeah, it gets gross.

> At Netflix we use sendfile for our stuff. It's one of the least unixy things in the kernel. It reads from a file, then TLS encrypts the file and sends it out the socket. This means state has to be carefully managed with some setup in userland before the handoff. The other non-unixy thing is that it's all non blocking. sendfile asks for a set of pages from a file. When they are ready, it gets a callback to schedule encryption, and when that fires it's scheduled to the NIC for transmission and either retransmission or freeing up depending on the ACKs that come back. At ~190Gbps, this isn't something one can do with the normal Unix interfaces, which was the point of his talk. He's not wrong, but his examples could use some work.

And here we see Unix as a victim of its own success.  It’s pretty damn weird, when you think about it, that we’re using a half-century-old typesetting system to power both the front and back ends of sending cat pictures and porn videos in real-time all over the planet all the time.

This also speaks to https://www.microsoft.com/en-us/research/uploads/prod/2019/04/fork-hotos19.pdf (mentioned here several months ago).     The fork()/exec() model is great in a single-threaded pipeline-processing model.  But, yeah, it’s totally true that it (and read()/write()) is not the best thing for writing user-facing nonblocking GUIs.

> The real world is messy, and often requires complexity. Going too simple for simplicity's sake is just as bad as going too complicated for complexity's sake. A proper balance is needed. And he's not wrong to make that point.

That said: Linux now has _way_ too many syscalls.  I would be flabbergasted if the rarely-used ones are not crawling with exploitable bugs.  I am surprised that Google hasn’t done more with fuchsia + gvisor (well, maybe they have but haven’t showed us yet).  Gvisor is a fascinating experiment in worse-is-better, in that, no, it makes no attempt to replace _all_ the Linux system calls, just the ones that _the programs you really want to run_ use.

Things do grow cruft over time.  And things change over time.  Containers are absolutely essential to what I’m doing these days.  They were not ten years ago.  Alpine is going there (in terms of a minimal container-focused support system (== OS layer), but clearly the more-right thing would be something like gvisor-plus-something-like-fuschia: my container only contains _code_ for the syscalls my application actually uses.  Can’t exploit what isn’t there.

This also hits the “People shouldn’t be using C in 2020” argument.  But _that_ in turn is less about C not providing modern strong typing features and letting you play fast and loose with pointers and making you do your own memory management, and more about: your computer is really, really not a PDP-11 anymore.  A language that provides the abstraction of a simple in-order execution stream is _lying_ to you.

The problem with that is that thinking about something as bizarrely complex as modern CPUs and writing software to exploit the way they actually work is _really really hard_.

Wow, that was a ramble.  Anyway: Benno’s not wrong.  But everything’s historically contingent, right?  We’ve got a half-century of C and a half-century of a typesetting system that got too big for its britches, that’s now running, basically, the entire underpinnings of the 21st century…I was going to say “economy” but really, it’s “world.”

Adam