28 Feb
2022
28 Feb
'22
1:04 a.m.
The X11 tree was a heavily ifdef-ed. And it needed to
be, I don't have
an answer as to how you would reuse all that code on different hardware
in a better way.
Plan 9 did it with #include. The name of the included file was the same for
every architecture. Only the search path for include files changed. Done with
care, this eliminates the typical upfront #ifdefs.that define constants and set
flags.
Other preprocessor conditionals can usually be replaced by a regular if, letting
the compiler optimize away the unwanted alternative. This makes conditionals
obey the scope rules of C.
Doug
28 Feb
28 Feb
7:22 a.m.
New subject: run commands at login in v6 and stty
Plan 9 had the distinct advantage of a constant system interface at the
source level. X11 did not, but it also made essentially no attempt to
abstract it away, so the lines starting #ifdef often outnumbered the actual
code. I couldn't make hide nor hair of it, and had no way to reliably test
any change.
C with #ifdefs is not portable, it is a collection of 2^n overlaid
programs, where n is the number of distinct #if[n]def tags. It's too bad
the problems of that approach were not appreciated by the C standard
committee, who mandated the #ifndef guard approach that I'm sure could
count as a provable billion dollar mistake, probably much more. The cost of
building #ifdef'ed code, especially with C++, which decided to be more
fine-grained about it, is unfathomable.
Google alone might well count for many millions of dollars in wasted
compilation equipment. I remember giving a Plan 9 demo to someone soon
after I got to Google. None of the features of the system were of interest.
The thing that astounded my audience was the ability to build the kernel on
a P90 in 20 seconds or so, and the window system in under 3. At that time,
a build of a Google server would require hours on a large distcc cluster.
I still shudder to think of it. It's worse now, of course, far worse, but
Google has far larger clusters to handle it and some improvement in
tooling. However, the #ifdefs persist.
Tom Cargill warned Bjarne about this around 1984, but the plea fell on deaf
ears.
-rob
On Mon, Feb 28, 2022 at 12:07 PM Douglas McIlroy <
douglas.mcilroy(a)dartmouth.edu> wrote:
The X11 tree
was a heavily ifdef-ed. And it needed to be, I don't have
an answer as to how you would reuse all that code on different hardware
in a better way.
Plan 9 did it with #include. The name of the included file was the same for
every architecture. Only the search path for include files changed. Done
with
care, this eliminates the typical upfront #ifdefs.that define constants
and set
flags.
Other preprocessor conditionals can usually be replaced by a regular if,
letting
the compiler optimize away the unwanted alternative. This makes
conditionals
obey the scope rules of C.
Doug
2:07 p.m.
New subject: run commands at login in v6 and stty
So I'm curious, did Plan 9 run on a similar number of architectures and
support a similar number of graphics cards?
If it did that, nicely, then you have a great point, the X11 people
(and most of us) clearly missed a better way to do things. If this is
the case, I'd like to understand how you did it because just including
different definitions doesn't begin to scratch the surface of what the
#ifdefs did in X11, there were tons in the actual code.
Maybe the constant system interface made all the difference, that
could be.
If it supported a much smaller number of targets, well, sure, it's
easier to be clean if you have a clean set of targets.
I remain grateful for the #ifdefs, I could make enough sense of it
all to bring up X11 on every platform I worked on. It wasn't pretty,
there was a ton of "I don't recognize this $WHATEVER, what happens if
I just #if 0 around the whole thing? Wow, it compiles, lets see if
I get a window system. Yep, I do. Shrug." That limited what I had
to understand to the much smaller subset of code I was actually
going to run, yeah, that set of #ifdefs was a mess but not such
a mess that I didn't get a working X11.
I'm not arguing that #ifdef is good, I'm just acknowledging it had
a lot of benefit to me, and _for me_, the cost was worth it in that
instance. I could have X11 working in less than a day.
I do get your point about abstracting the interface differences away,
I actually hate #ifdefs in the code with a passion so in BitKeeper we
had all that stuff buried under the abstractions. We made everything
look like Unix, even on Windows, except for fork(). I haven't called
fork() directly in close to 20 years, we picked up spawn() and made that
work everywhere. The abstraction layer cuts down on the #ifdefs in the
code a LOT.
On Mon, Feb 28, 2022 at 06:22:28PM +1100, Rob Pike wrote:
Plan 9 had the distinct advantage of a constant system
interface at the
source level. X11 did not, but it also made essentially no attempt to
abstract it away, so the lines starting #ifdef often outnumbered the actual
code. I couldn't make hide nor hair of it, and had no way to reliably test
any change.
C with #ifdefs is not portable, it is a collection of 2^n overlaid
programs, where n is the number of distinct #if[n]def tags. It's too bad
the problems of that approach were not appreciated by the C standard
committee, who mandated the #ifndef guard approach that I'm sure could
count as a provable billion dollar mistake, probably much more. The cost of
building #ifdef'ed code, especially with C++, which decided to be more
fine-grained about it, is unfathomable.
Google alone might well count for many millions of dollars in wasted
compilation equipment. I remember giving a Plan 9 demo to someone soon
after I got to Google. None of the features of the system were of interest.
The thing that astounded my audience was the ability to build the kernel on
a P90 in 20 seconds or so, and the window system in under 3. At that time,
a build of a Google server would require hours on a large distcc cluster.
I still shudder to think of it. It's worse now, of course, far worse, but
Google has far larger clusters to handle it and some improvement in
tooling. However, the #ifdefs persist.
Tom Cargill warned Bjarne about this around 1984, but the plea fell on deaf
ears.
-rob
On Mon, Feb 28, 2022 at 12:07 PM Douglas McIlroy <
douglas.mcilroy(a)dartmouth.edu> wrote:
> > The X11 tree was a heavily ifdef-ed. And it needed to be, I don't have
> > an answer as to how you would reuse all that code on different hardware
> > in a better way.
>
> Plan 9 did it with #include. The name of the included file was the same for
> every architecture. Only the search path for include files changed. Done
> with
> care, this eliminates the typical upfront #ifdefs.that define constants
> and set
> flags.
>
> Other preprocessor conditionals can usually be replaced by a regular if,
> letting
> the compiler optimize away the unwanted alternative. This makes
> conditionals
> obey the scope rules of C.
>
> Doug
>
--
---
Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm
6:18 p.m.
New subject: run commands at login in v6 and stty
On Sun, Feb 27, 2022 at 6:07 PM Douglas McIlroy <
douglas.mcilroy(a)dartmouth.edu> wrote:
The qemu project does this, for the most part. It makes things a lot
easier, but
does take some getting used to to find the proper file to be included. It
helps
a lot to keep the twisty maze of #ifdefs down to a dull roar.
FreeBSD (and all the BSDs) has a similar philosophy. You define the page
size,
for example, of the target, and the rest of the system is written to that
abstraction
rather than having #ifdefs in a lot of places. NetBSD is much better about
keeping
the total number of #ifdefs out of MI code than FreeBSD. In both cases, most
of the #ifdefs are basically #ifdef __HAS_FUNKY_THING <do funky thing here>
#endif that are needed on a couple of architectures, though usually that
idiom
is expressed by a macro (that's empty for most platforms) or an inline
function.
The X11 tree
was a heavily ifdef-ed. And it needed to be, I don't have
an answer as to how you would reuse all that code on different hardware
in a better way.
Plan 9 did it with #include. The name of the included file was the same for
every architecture. Only the search path for include files changed. Done
with
care, this eliminates the typical upfront #ifdefs.that define constants
and set
flags.
Other preprocessor conditionals can usually be
replaced by a regular if,
letting
the compiler optimize away the unwanted alternative. This makes
conditionals
obey the scope rules of C.
As long as they are always defined... :) Though older gcc/clang compilers
get cranky
when they view the if expression as a tautology (warning about the idiom
rather than
encouraging it: stupid clang tricks).
Warner
6:47 p.m.
New subject: run commands at login in v6 and stty
On Mon, Feb 28, 2022 at 9:10 AM Larry McVoy <lm(a)mcvoy.com> wrote:
So I'm curious, did Plan 9 run on a similar
number of architectures and
support a similar number of graphics cards?
Yes, it did.
If it did that, nicely, then you have a great point, the X11 people
(and most of us) clearly missed a better way to do
things. If this is
the case, I'd like to understand how you did it because just including
different definitions doesn't begin to scratch the surface of what the
#ifdefs did in X11, there were tons in the actual code.
#ifdef has more or less always struck me as the solution to the wrong
problem. "We have all this code and we want to shoehorn it into a new
environment," instead of, "we have many environments, so we carefully
structure the code to accommodate the differences." Of course, the latter
is harder than the former, but it also pays larger dividends over time as
compared to the former.
Maybe the constant system interface made all the difference, that
could be.
If it supported a much smaller number of targets, well, sure, it's
easier to be clean if you have a clean set of targets.
In fairness, a meaningful comparison is hard: plan9 ran on ~a dozen
different architectures over its lifetime, but never supported near the
variety of software or workloads of Unix, and didn't exactly "compete" in
the market the way Unix did. The PC port in particular supported oodles of
graphics cards at one point, but not as many as X did, mostly because there
weren't all that many folks working with the unsupported cards, so there
wasn't a lot of motivation to write tons of drivers.
Could it have retained its elegantly clean structure over time had things
evolved differently? That's impossible to answer, but I'm sad that we never
got the chance to find out.
I remain grateful for the #ifdefs, I could make enough sense of it
all to bring up X11 on every platform I worked on. It
wasn't pretty,
there was a ton of "I don't recognize this $WHATEVER, what happens if
I just #if 0 around the whole thing? Wow, it compiles, lets see if
I get a window system. Yep, I do. Shrug." That limited what I had
to understand to the much smaller subset of code I was actually
going to run, yeah, that set of #ifdefs was a mess but not such
a mess that I didn't get a working X11.
I'm not arguing that #ifdef is good, I'm
just acknowledging it had
a lot of benefit to me, and _for me_, the cost was worth it in that
instance. I could have X11 working in less than a day.
I do think that #ifdef, used extremely judiciously, can have some utility:
for example, compiling out debug code by setting a constant. Consider:
#ifdef NDEBUG
const bool DEBUGGING = false;
#else
const bool DEBUGGING = true;
#endif
static inline
void
DBG(const char *fmt, ...)
{
if (DEBUGGING) {
va_list ap;
va_start(ap, fmt);
vfprintf(stderr, fmt ap);
va_end(ap);
}
}
DBG(foo);
if `NDEBUG` is defined, then the body of the static inline becomes, `if (0)
{ ... }` and one hopes our compilers are sufficiently smart to elide the
entire thing.
The biggest problem with #ifdef wasn't so much that it existed, but rather
that it was used for too many things that it wasn't well-suited for. The
second biggest problem was that it was semantically unaware of the
language; it was purely textual. Bummer.
I do get your point about abstracting the interface differences away,
I actually hate #ifdefs in the code with a passion so
in BitKeeper we
had all that stuff buried under the abstractions. We made everything
look like Unix, even on Windows, except for fork(). I haven't called
fork() directly in close to 20 years, we picked up spawn() and made that
work everywhere. The abstraction layer cuts down on the #ifdefs in the
code a LOT.
Agreed. But getting those abstractions right requires experience and taste.
#ifdef is a blunt took for a nuanced problem.
- Dan C.
On Mon, Feb 28, 2022 at 06:22:28PM +1100, Rob Pike wrote:
Plan 9 had the
distinct advantage of a constant system interface at the
source level. X11 did not, but it also made essentially no attempt to
abstract it away, so the lines starting #ifdef often outnumbered the
actual
code. I couldn't make hide nor hair of it,
and had no way to reliably
test
any change.
C with #ifdefs is not portable, it is a collection of 2^n overlaid
programs, where n is the number of distinct #if[n]def tags. It's too bad
the problems of that approach were not appreciated by the C standard
committee, who mandated the #ifndef guard approach that I'm sure could
count as a provable billion dollar mistake, probably much more. The cost
of
building #ifdef'ed code, especially with
C++, which decided to be more
fine-grained about it, is unfathomable.
Google alone might well count for many millions of dollars in wasted
compilation equipment. I remember giving a Plan 9 demo to someone soon
after I got to Google. None of the features of the system were of
interest.
The thing that astounded my audience was the
ability to build the kernel
on
a P90 in 20 seconds or so, and the window system
in under 3. At that
time,
a build of a Google server would require hours on
a large distcc cluster.
I still shudder to think of it. It's worse now, of course, far worse, but
Google has far larger clusters to handle it and some improvement in
tooling. However, the #ifdefs persist.
Tom Cargill warned Bjarne about this around 1984, but the plea fell on
deaf
ears.
-rob
On Mon, Feb 28, 2022 at 12:07 PM Douglas McIlroy <
douglas.mcilroy(a)dartmouth.edu> wrote:
> > The X11 tree was a heavily ifdef-ed. And it needed to be, I don't
have
> > an answer as to how you would reuse all
that code on different
hardware
> > in a better way.
>
> Plan 9 did it with #include. The name of the included file was the
same for
> every architecture. Only the search path for
include files changed.
Done
> with
> care, this eliminates the typical upfront #ifdefs.that define constants
> and set
> flags.
>
> Other preprocessor conditionals can usually be replaced by a regular
if,
> letting
> the compiler optimize away the unwanted alternative. This makes
> conditionals
> obey the scope rules of C.
>
> Doug
>
--
---
Larry McVoy lm at mcvoy.com
http://www.mcvoy.com/lm
7:25 p.m.
New subject: run commands at login in v6 and stty
On Mon, Feb 28, 2022 at 1:47 PM Dan Cross <crossd(a)gmail.com> wrote:
[snip]
Oops, I forgot to mention something. I imagine a lot of readers of this
list are familiar with this paper, but for those that aren't, this
resonates:
https://www.usenix.org/legacy/publications/library/proceedings/sa92/spencer…
On a personal note, when I was learning to write C for Unix, I read a lot
of open source code to see how to do it. So much of it was riddled with
complex #ifdefs that I kind of got the impression that that was how things
were _supposed_ to be written. Then I read the Spencer/Collyer paper, and
modified some program that I'd written to use abstract interfaces; it was
so much easier to reason about that I was kind of shocked that others
didn't do the same.
I suppose the point is that many programmers learn by copying what has come
before. However, so much of what's out there is a poor example of how to do
things; we're stuck with a feedback loop of poor design.
- Dan C.
7:26 p.m.
New subject: run commands at login in v6 and stty
On Mon, Feb 28, 2022 at 1:49 PM Dan Cross <crossd(a)gmail.com> wrote:
#ifdef has more or less always struck me as the solution to the wrong
problem. "We have all this code and we want to shoehorn it into a new
environment," instead of, "we have many environments, so we carefully
structure the code to accommodate the differences." Of course, the latter
is harder than the former, but it also pays larger dividends over time as
compared to the former.
Absolutely - but the problem is the development process (more in a minute).
The biggest problem with #ifdef wasn't so much that it existed, but rather
that it was used for too many things that it wasn't well-suited for.
Agreed.
The second biggest problem was that it was
semantically unaware of the
language; it was purely textual. Bummer.
Dan - I'll argue with you on that one actually. One of the reasons why C
was successful in a professional language is that it could be easily
preprocessed, which other languages like Pascal and even Ada sucked
when you tried. There are times when a preprocessor is just really handy
as Larry said, particularly in a product programming house.
And here is the issue ...
Doug and Rob are 100% right about Plan 9 but .. it's not a fair
comparison. Plan9 was a research system. X started out as that, but by
the time of X11 it was a production system and very different types of
programmers. When you are porting code, particularly when under type
constraints, Larry described exactly how it's done ... #ifdef NOTDEF ...
ok this works.... and off you go the next issue with the product. In my
experience, the development process often (tends to) reinforce this
behavior.
Note I am not telling you that's a good idea ... but *it is a typical
behavior* (*a.k.a. *whiskey to teenagers). You intend to go back and redo
it, but it never gets done. The next person adds their hack and pretty
soon you have the BSD kernel if #ifdef FASTVAXl or #ifdef BIG_ENDIAN scattered
all over the code base.
As Larry and others have pointed out, an abstraction layer, when you can
enforce it, clearly is the best. But to be fair, one of the reasons why C
was so popular, is that so much code could be moved. One of the things I
hated about Pascal, was that so much code had been written that assumed 36
or 40 integers, or that string limits varied depending on byte or work size
and how the len was encoded -- yeech.
The problem, as Larry said, is that it is a powerful tool that was easy to
and often was, abused by both neophytes and people that should have known
better.
8:03 p.m.
New subject: run commands at login in v6 and stty
From the Spencer paper:
"In such cases,it is important to go back and fix the kludges. The time is
not wasted; it is an investment in the future"
Yeah, well, that's the problem, isn't it? Investment in the future is not
incentivized. The manager needs a win *now* so he can collect his bonus
and then leave for someplace that will pay him more. Even as a lowly
engineer I've never seen a place where it made financial sense to stick
around. You'd never get compensation, authority, and responsibility as
fast by being internally promoted as by hopping to the competition across
town.
If the goal were building a sustainable and maintainable ecosystem, then,
sure. But it isn't. Good software engineering doesn't pay. Barfing out a
checklist of feature bullet points that marketing wants pays.
That does suggest that there's some scope for doing this properly in big
Open Source projects--except those, in reality, are almost always driven by
a very few large funders, who have particular features they want and are
willing to pay for. Shipping those features then becomes the priority, not
keeping the codebase manageable.
I guess it comes down to "hate the game, not the player" on my part and
I'm
having a grumpy day.
Adam
9:25 p.m.
New subject: run commands at login in v6 and stty
Hoi.
[2022-02-28 14:25] Dan Cross <crossd(a)gmail.com>
On a personal note, when I was learning to write C for Unix, I read a lot of
open source code to see how to do it. So much of it was riddled with complex #
ifdefs that I kind of got the impression that that was how things were
_supposed_ to be written. Then I read the Spencer/Collyer paper, and modified
some program that I'd written to use abstract interfaces; it was so much easier
to reason about that I was kind of shocked that others didn't do the same.
I suppose the point is that many programmers learn by copying what has come
before. However, so much of what's out there is a poor example of how to do
things; we're stuck with a feedback loop of poor design.
... and I am very happy that I found my way into a different feedback
loop, called Unix philosophy ... with sources meant to be read, great
documentation, and books that actually feature real life code (already
a thing by itself) but also that the production code is on the same
``acts as a good example'' level.
If programmers have never experienced such wonderful worlds -- how
poor their programming lifes must be!
meillo
796
days inactive
796
days old
8 comments
8 participants
participants (8)
-
Adam Thornton -
Clem Cole -
Dan Cross -
Douglas McIlroy -
Larry McVoy -
markus schnalke -
Rob Pike -
Warner Losh