On Tue, Jun 26, 2018 at 01:03:24PM -0500, Cornelius Keck wrote: ACM Communications is behind a paywall. If you have a subscription it's available here: https://cacm.acm.org/magazines/2018/7/229036-c-is-not-a-low-level-language/… khm

On Tue, Jun 26, 2018 at 05:16:56PM -0400, Arthur Krewat wrote: So I agree, had the same initial reaction. But I read the paper a second time and the point about Fortran, all these years later, still being a thing resonated. The hardware guys stand on their heads to give us coherent caches. That's a great quote. But I do sort of grudgingly see the author's point of view, at least somewhat. --lm

On Tue, Jun 26, 2018 at 05:54:32PM -0400, Ronald Natalie wrote: I think you are more or less agreeing with the author. (I also think, as Unix die hards, we all bridle a little when anyone dares to say anything negative about C. We should resist that if it gets in the way of making things better.) The author at least has me thinking about how you could make a C like language that didn't ask as much from the hardware. -- --- Larry McVoy lm at mcvoy.com http://www.mcvoy.com/lm

On 6/26/2018 6:20 PM, Bakul Shah wrote: Then let's discuss why caching is the problem. If thread X reads memory location A, why is thread Y able to access that cached value? Shouldn't that cached value be associated with memory location A which I would assume would be in a protected space that thread Y shouldn't be able to access? I know the nuts and bolts of how this cache exploit works, that's not what I'm asking. What I'm asking is, why is cache accessible in the first place? Any cache offset should have the same memory protection as the value it represents. Isn't this the CPU manufacturer's fault? art k.

On 26 Jun 2018, at 23:20, Bakul Shah <bakul(a)bitblocks.com> wrote: That's the strangest thing to see: *why do people think the point he's making is in any way controversial*, because it's so obvious. (But then I'm also annoyed by the paper because I've been talking about 'giant PDP-11s' for a long time and now he's stolen (obviously not stolen: independently come up with) my term, pretty much.)

On Jun 26, 2018, at 3:33 PM, Andy Kosela <akosela(a)andykosela.com> wrote: I primarily write code in Go these days and like it a lot (as a "better" C) but I am not sure it will have C's longevity. It still uses a flat shared memory model. This is harder and harder for hardware to emulate efficiently (and comes with more complexity) at smaller and smaller minimum feature sizes and higher & higher CPU clock rates & on-chip comm speeds. We need something other than a better C to squeeze maximum performance out of a CPU built out of 100s to 1000s of cores.

One of the things which is probably causing a lot of hang ups is the phrase "low-level language". What is exactly meant by that? The article actually uses multiple definitions of "low-level language", and switches between them when it is convenient, or as a rhetorical trick. There is at least one of his points that I very much agree with, which is the abstract model which is exported by the C programming paradigm is very different from what totality of what the CPU can do. Consider the oft-made admonition that most humans can't program in assembler more efficiently than the compiler. Think about that for a moment. That saying is essentially telling us that the compiler is capable of doing a lot of things behinds our back (which goes against the common understanding of what low-level language does), and second, that modern CPU's has gotten too complicated for most humans to be able to program directly, either comfortably or practically. Also, all of these compiler optimizations are not _mandatory_. They are there because people really want performance. You are free to disable them if you care about predictability and avoiding the "any sufficiently advanced compiler is indistinguishable from a malicious adversary" dynamic. In fact, if are writing a program where you are, more often than not, I/O bound and/or fundamentally bound by memory bandwidth, turning off a lot of these compiler optimizations might be the right thing to do, as they may not buy you much, and given you much in the way of headaches. There is a reason why the Linux kernel compiles with the GCC flags: -fno-asynchronous-unwind-tables -fno-delete-null-pointer-checks \ -fstack-protector-strong -fno-var-tracking-assignments \ -femit-struct-debug-baseonly -fno-var-tracking \ -fno-inline-functions-called-once -fno-strict-overflow \ -fno-merge-all-constants -fmerge-constants -fno-stack-check By the way, I disagree with Chisnall's assertion that low-level languages are fast. If you believe that compilers can often do a better job than humansh writing assembly, does that mean that since assembly code isn't fast that it's not a low-level language?) And the performance wars is an important part of dynamic --- at the end of the day, the vast majority of people choose performance as the most important driver of their purchasing decision. They may pay lip service to other things, including security, robustness, etc., but if you take a look at what they actually _do_, far more often than not, they choose fast over slow. I see this an awful lot in file system design. People *talk* a great game about how they care about atomicity, and durability, etc., but when you look at their actual behavior as opposed to expressed desires, more often than not what they choose is again, fast over slow. And so I have an extension to the fictional dialog to Pillai's presentation for his 2014 OSDI paper: Experienced Academic: Real file systems don't do that. But <...> does just that. Academic: My students would flunk class if they built a file system that way. Cynical Industry Developer's rejoinder: A file system designed the way you propose would likely be a commercial failure, and if it was the only/default FS for an OS, would probably drag the OS down to failure with it. And this goes to Spectre, Meltdown, TLBleed, and so on. Intel built processors this way because the market _demanded_ it. It's for the same reason that Ford Motor Company built gas-guzzling pickup trucks. Whether or not Maimi Beach will end up drowning under rising sea levels never entered into the the consumer or the manufacturer's consideration, at least not in any real way. It's very easy to criticize Intel for engineering their CPU's the way they did, but if you want to know who to blame for massively high ILP's and caching, and compilers that do all sorts of re-ordering and loop rewriting behind the developer's back, just look in the mirror. It's the same as why we have climate change and sea level rise. "We have met the enemy and he is us". -- Pogo (Walt Kelly) - Ted

Theodore Y. Ts'o wrote in &lt;20180627021843.GA31920(a)thunk.org&gt;rg>: ... |There is at least one of his points that I very much agree with, which |is the abstract model which is exported by the C programming paradigm |is very different from what totality of what the CPU can do. Consider ... |That saying is essentially telling us that the compiler is capable of |doing a lot of things behinds our back (which goes against the common |understanding of what low-level language does), and second, that |modern CPU's has gotten too complicated for most humans to be able to |program directly, either comfortably or practically. ... |It's very easy to criticize Intel for engineering their CPU's the way |they did, but if you want to know who to blame for massively high |ILP's and caching, and compilers that do all sorts of re-ordering and |loop rewriting behind the developer's back, just look in the mirror. |It's the same as why we have climate change and sea level rise. | |"We have met the enemy and he is us". -- Pogo (Walt Kelly) And my thinking is that the C language has been misused. If you can cast in stone a language policy that then allows compilers to apply optimization X, and it gains your product ten percent performance without the necessity to spend a single minute payed manpower. But even otherwise. For example loop unrolling -- how could a compiler know whether i want a loop to become unrolled or not? It may have very clever heuristics, but despite that these have to become coded themselves, they will not know whether i have the desire to make this piece of code fast or not, shall it ever execute. In my opinion this "giving away decisions to the compiler" made the C language worse, not better. I think it would be better if one could place attributes, like @unroll, @inline (since inline is only a hint and placing not-to-be-inlined after their use cases does not matter to intelligent CCs: @no-inline would be needed), @prefetch[[X]], and @parallel. And @align[[X]] and all these. "@parallel @unroll for(;;)" could for example automatically split a loop into X subranges. Or it could hint secure prediction. Or at least indicate the programmer's intention or clearance of that, if the hardware is capable. And a good compiler could indicate whether the desire could ever be fulfilled or not. This looks like the Julia language, but there these are user-defined "macros" not compiler hints. I mean, at least until quantum computers happen C seems to map to what i know about hardware pretty good. It is up to the programmer to localize data and avoid that threads trumple on each other. It is not so easy as if the language offers a completely asynchronous shot with a single operator, but in an ideal world you can get the same with a much smaller footprint. Unfortunately what you say seems to be right, and recalling what the largest German computer magazine had to say about compilers, it was mostly a series of graphs indicating the speed of the generated code. Whereas i had to deal with working around compiler bugs for the same (free) CCs at the same time. (Of course, or maybe not of course, people did care to get them fixed, but the story was not about "the walk", but what the eagles saw.) There were two or three large interesting articles with quite some context (except Mr. Stiller's "Prozessorgeflüster"; i recall he once won a bottle of wine from some Intel manager), about at the same time, one about the ARM architecture, one accompanying Donald Knuth's 64th birthday regarding MMIX, and if you want one about Smalltalk. And indeed most think it is all about education, despite the fact that each epoch seems to end regardless what you. It has always been the fight to make people reflect, realize, and be good with it. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)

I guess my take on it is mixed. I see some of his points but over all I disagree with most of them. I firmly believe if you look at anything long enough you will find flaws. There is no perfect. I think Fortran, C, even Algol are credits for more what people were able to think about at the time and how well they lasted. As I have said in other places, Fortran is not going away. Clem Cole's answer is the Future of Fortran Programming Dead <https://www.quora.com/Is-the-future-of-Fortran-programming-dead/answer/Clem-Cole> also applies to C. It's just not broken and he's wrong. Go, Rust *et al* is not going to magically overtake C, just as Fortran as not been displaced in my lifetime (BTW, I >>like<< both Go and Rust and think they are interesting new languages). He thinks C is not long a low level language because when Ken abstracted the PDP-7 into B and then Dennis abstracted the PDP-11 into C, the systems were simple. The HW designers are in a giant fake out at this point, so things that used to work like 'register' no longer make sense. Now its the compiler that binds to the primitives available to the functions under the covers and there is more to use than the PDP-11 and PDP-7 offered. But wait, that is not always true. So I think he's wrong. I think you leave the language alone and if the HW moves on great. But if we have a simple system like you have on the Amtel chips that most Arduino's and lots of other embedded C programs use, C is very low level and most his arguments go away. Cken ᐧ

What Clem said. Chisnall is right about C having been designed for a sequential-programming world. That's why Fortran (with array and other parallel/vector operations built in) rules in the HPTC parallel programming space. But I don't buy most of his arguments. Making parallel programming easy and natural has been an unsolved problem during my entire 30+ year career in designing software development tools. It's still an unsolved problem. Modern compiler technology helps to find the hidden parallelism in algorithms expressed sequentially, but I think the fundamental problem is that most human beings have great difficulty conceptualizing parallel algorithms. It's also always been true that to get maximum performance you have to somehow get close to the specific hardware you're using--either by explicitly programming for it, or by having a compiler do that for you. Note also that there have been extensions to C/C++ to support parallelism. Cilk, for example. -Paul W.

Arthur Krewat &lt;krewat(a)kilonet.net&gt; wrote: After many years working in a variety of places, I have come to the conclusion that Sturgeon's Law ("90% of everything is crud") applies to working programmers as well. :-( W.R.T. the statement as given, the point is good; when a language is huge and complex (yes C++, I'm looking at you) it becomes really hard to be effective in it unless you are a super-star genius. Arnold

On Jun 27, 2018, at 9:00 AM, Steve Johnson &lt;scj(a)yaccman.com&gt; wrote: Very well put. The whole concept of address-spaces is rather low level. There is in fact a close parallel to this model that is in current use. Cloud computing is essentially a collection of "micro-services", orchestrated to provide some higher level service. External to some micro-service X, all other services care about is how to reach X and what comm. protocol to use to talk to it but not about any details of how it is implemented. Here concerns are more about reliability, uptime, restarts, updates, monitoring, load balancing, error handling, DoS, security, access-control, latency, network address space & traffic management, dynamic scaling, etc. A subset of these concerns would apply to parallel computers as well. Current cloud computing solutions to these problems are quite messy, complex and heavyweight. There is a lot of scope here for simplification....

On Thu, Jun 28, 2018 at 10:15:38AM -0400, Theodore Y. Ts'o wrote: Yep. Lots of cpus are nice when doing a parallel make but there is always some task that just uses one cpu. And then you want the fastest one you can get. Lots of wimpy cpus is just, um, wimpy.

On Thu, Jun 28, 2018 at 10:55:38AM -0400, Perry E. Metzger wrote: You completely missed my point, I never said I was in favor of single cpu systems, I said I the speed of a single cpu to be fast no matter how many of them I get. The opposite of wimpy. Which was, I think, Ted's point as well when he said the market rejected the idea of lots of wimpy cpus.

On Thu, Jun 28, 2018 at 04:39:58PM +0100, Tim Bradshaw wrote: That's not what I, or Ted, or the Market was saying. We were not comparing yesterday's cpu against todays. We were saying that at any given moment, a faster processor is better than more processors that are slower. That's not an absolute, obviously. If I am running AWS and I get 10x the total CPU processing speed at the same power budget, yeah, that's interesting. But for people who care about performance, and there are a lot of them, more but slower is less desirable than less but faster. There too much stuff that hasn't been (or can't be) parallelized (and I'll note here that I'm the guy that built the first clustered server at Sun, I can argue the parallel case just fine). People still care about performance and always will. Yeah, for your laptop or whatever you could probably use what you have for the next 10 years and be fine. But when you are doing real work, sorting the genome, machine learning, whatever, performance is a thing and lots of wimpy cpus are not.

n re modern architectures, admiral Grace Hopper gave a talk at DEC's Nashua software development plant ca. 1982 on the future of computer architecture. She predicted the modern multi-core situation. As she put it, if you were a carter and had to haul double the load in your cart, you wouldn't breed bigger a horse twice as big--you'd hook up a team of horses. Another problem with solving problems fast is I/O bandwidth. Much of digital audio file, for example, is an embarrassingly parallel problem. But no matter how many cores you throw at the problem, and no matter how fast they are, the time it takes to process the audio file is limited by how fast you can get the original off the disk and the modified file back onto the disk. Or main memory, for that matter. Relative to the processor speed, a cache miss takes an eternity to resolve. Get your cache management wrong and your program ends up running an order of magnitude slower. It's a throwback to the situation in the 1960s, where compute speeds were comparable to main memory speeds, but vastly higher than the I/O transmission rates to disk and tape. Only now first-level cache is the new "main memory", and for practical purposes main memory is a slow storage medium. -Paul W.

On 28 Jun 2018, at 18:09, Larry McVoy &lt;lm(a)mcvoy.com&gt; wrote: [I said I wouldn't reply more: I'm weak.] I think we have been talking at cross-purposes, which is probably my fault. I think you've been using 'wimpy' to mean 'intentionally slower than they could be' while I have been using it to mean 'of very tiny computational power compared to the power of the whole system'. Your usage is probably more correct in terms of the way the term has been used historically. But I think my usage tells you something important: that the performance of individual cores will, inevitably, become increasingly tiny compared to the performance of the system they are in, and will almost certainly become asymptotically constant (ie however much money you spend on an individual core it will not be very much faster than the one you can buy off-the-shelf). So, if you want to keep seeing performance improvements (especially if you want to keep seeing exponential improvements for any significant time), then you have no choice but to start thinking about parallelism. The place I work now is an example of this. Our machines have the fastest cores we could get. But we need nearly half a million of them to do the work we want to do (this is across three systems). I certainly don't want to argue that choosing intentionally slower cores than you can get is a good idea in general (although there are cases where it may be, including, perhaps, some HPC workloads). --- However let me add something about the Sun T-series machines, which were 'wimpy cores' in the 'intentionally slower' sense. When these started appearing I worked in a canonical Sun customer at the time: a big retail bank. And the reason we did not buy lots of them was nothing to do with how fast they were (which was more than fast enough), it was because Sun's software was inadequate. To see why, consider what retail banks IT looked like in the late 2000s. We had a great mass of applications, the majority of which ran on individual Solaris instances (at least two, live and DR, per application). A very high proportion (not all) of these applications had utterly negligible computational requirements. But they had very strong requirements on availability, or at least the parts of the business which owned them said they did and we could not argue with that, especially given that this was 2008 and we knew that if we had a visible outage there was a fair chance that it would be misread as the bank failing, resulting in a cascade failure of the banking system and the inevitable zombie apocalypse. No one wanted that. Some consolidation had already been done: we had a bunch of 25ks, many of which were split into lot of domains. The smallest domain on a 25k was a single CPU board which was 4 sockets and therefore 8 or 16 (I forget how many cores there were per socket) cores. I think you could not partition a 25k like that completely because you ran out of IO assemblies, so some domains had to be bigger. This smallest domain was huge overkill for many of these applications, and 25ks were terribly expensive as well. So, along came the first T-series boxes and they were just obviously ideal: we could consolidate lots and lots of these things onto a single T-series box, with a DR partner, and it would cost some tiny fraction of what a 25k cost, and use almost no power (DC power was and is a real problem). But we didn't do that: we did some experiments, and some things moved I think, but on the whole we didn't move. The reason we didn't move was nothing, at all, to do with performance, it was, as I said, software, and in particular virtualisation. Sun had two approaches to this, neither of which solved the problems that everyone had. At the firmware level there were LDOMs (which I think did not work very well early on or may not have existed) which let you cut up a machine into lots of smaller ones with a hypervisor in the usual way. But all of these smaller machines shared the same hardware of course. So if you had a serious problem on the machine, then all of your LDOMs went away, and all of the services on that machine had an outage, at once. This was not the case on a 25k: if a CPU or an IO board died it would affect the domain it was part of, but everything else would carry on. At the OS level there were zones (containers). Zones had the advantage that they could look like Solaris 8 (the machine itself, and therefore the LDOMs it got split into, could only run Solaris 10), which all the old applications were running, and they could be very fine-grained. But they weren't really very isolated from each other (especially in hindsight), they didn't look *enough* like Solaris 8 for people to be willing to certify the applications on them, and they still had the all-your-eggs-in-one-basket problem if the hardware died. The thing that really killed it was the eggs-in-one-basket problem. We had previous experience with consolidating a lot of applications onto one OS & hardware instance, and no-one wanted to go anywhere near that. If you needed to get an outage (say to install a critical security patch, or because of failing hardware) you had to negotiate this with *all* the application teams, all of whom had different requirements and all of whom regarded their application as the most important thing the bank ran (some of them might be right). It could very easily take more than a year to get an outage on the big shared-services machines, and when the outage happened you would have at least 50 people involved to stop and restat everything. It was just a scarring nightmare. So, to move to the T-series machines what we would have needed was a way of partitioning the machine in such a way that the partitions ran Solaris 8 natively, and in such a way that the partitions could be moved, live, to other systems to deal with the eggs-in-one-basket problem. Sun didn't have that, anywhere near (they knew this I think, and they got closer later on, but it was too late). So, the machines failed for us. But this failure was nothing, at all, to do with performance, let alone performance per core, which was generally more than adequate. Lots of wimpy, low power, CPUs was what we needed, in fact: we just needed the right software on top of them, which was not there. (As an addentum: what eventually happened / is happening I think is that applications are getting recertified on Linux/x86 sitting on top of ESX, *which can move VMs live between hosts*, thus solving the problem.)

On Fri, Jun 29, 2018 at 04:32:59PM +0100, tfb(a)tfeb.org wrote: Not "intentionally" as "let me slow this down" but as in "it's faster and cheaper to make a slower cpu so I'll just give you more of them". The market has shown, repeatedly, that more slow cpus are not as fun as less faster cpus. It's not a hard concept and I struggle to understand why it's a point to discuss. Yeah, so what? That wasn't the point being discussed though you and Perry keep pushing it.

On Fri, 29 Jun 2018 10:51:26 -0700 Larry McVoy &lt;lm(a)mcvoy.com&gt; wrote: Perhaps we're not agreed on what was originally motivating the discussion. I believe we started with whether modern parallel computing environments might need something more than C. I think the original hardware question this spawned was: is dealing with lots of cores simultaneously here to stay, and do you get something out of having language support to help with it? We were trying to address that question, and given that, I think we've been on point. Single programs that have to handle large numbers of threads and cores are now common. Every interesting use of the GPU on your machine is like this, including the classic ones like rendering images to your screen, but also newer ones like being exploited for all sorts of purposes by your browser that aren't related to video as such. Your desktop browser is a fine example of other sorts of parallelism, too: it uses loads of parallelism (not merely concurrency) on a modern desktop to deal with rapid processing of the modern hideous and bloated browsing stack. As for whether there's an advantage to modern languages here, the answer there is also "yes", Mozilla only really managed to get a bunch of parallelism into Firefox because they used a language (Rust) with a pretty advanced linear type system to assure that they didn't have concurrency problems. Perry -- Perry E. Metzger perry(a)piermont.com

On Thu, Jun 28, 2018 at 10:40 AM, Larry McVoy &lt;lm(a)mcvoy.com&gt; wrote: ​Larry Stewart would be better to reply as SiCortec's CTO - but that was the basic logic behind their system -- lots of cheap MIPS chips. Truth is they made a pretty neat system and it scaled pretty well. My observation is that they, like most of the attempts I have been a part, *in the end architecture does not matter nearly as much as economics*. In my career I have build 4 or 5 specially architecture systems. You can basically live through one or two generations using some technology argument and 'win'. But in the end, people buy computers to do a job and the really don't give a s*t about how the job gets done, as long as it get done cheaply.​ Whoever wins the economic war has the 'winning' architecture. Look x66/Intel*64 would never win awards as a 'Computer Science Architecture' or in SW side; Fortran *vs*. Algol *etc*...; Windows beat UNIX Workstations for the same reasons... as well know. Hey, I used to race sailboats ... there is a term called a 'sea lawyer' - where you are screaming you have been fouled but you drowning as your boating is sinking. I keep thinking about it here. You can scream all you want about goodness or badness of architecture or language, but in the end, users really don't care. They buy computers to do a job. You really can not forget that is the purpose. As Larry says: Lots of wimpy cpus is just wimpy. Hey, Intel, nVidia and AMD's job is sell expensive hot rocks. They are going to do what they can to make those rocks useful for people. They want to help people get there jobs done -- period. That is what they do. Amtel and RPi folks take the 'jelly bean' approach - which is one of selling enough it make it worth it for the chip manufacture and if the simple machine can do the customer job, very cool. In those cases simple is good (hey the PDP-11 is pretty complex compared to say the 6502). So, I think the author of the paper trashing as too high level C misses the point, and arguing about architecture is silly. In the end it is about what it costs to get the job done. People will use what it is the most economically for them. Clem

On Thu, 28 Jun 2018 10:15:38 -0400 "Theodore Y. Ts'o" &lt;tytso(a)mit.edu&gt; wrote: I'll be contrary for a moment. Huge numbers of wimpy cores is the model already dominating the world. Clock rates aren't rising any longer, but (in spite of claims to the contrary) Moore's law continues, very slightly with shrinkage of feature size (which is about to end) and more dominantly with increasing the number of transistors per square mil by going into 3D. Dynamic power usage also scales as the square of clock rate, so larger numbers of lower clocked cores save a boatload of heat, and at some point you have too many transistors in too small an area to take heat out if you're generating too much. Some data points: 1. All the largest compute platforms out there (Google, Amazon, etc.) are based on vast numbers of processors integrated into a giant distributed system. You might not see this as evidence for the trend, but it is. No one can make a single processor that's much faster than what you get for a few hundred bucks from Intel or AMD, so the only way to get more compute is to scale out, and this is now so common that no one even thinks of it as odd. 2. The most powerful compute engines out there within in a single box aren't Intel microprocessors, they're GPUs, and anyone doing really serious computing now uses GPUs to do it. Machine learning, scientific computing, etc. has become dependent on the things, and they're basically giant bunches of tiny processors. Ways to program these things have become very important. Oh, and your iPhone or Android device is now pretty lopsided. By far most of the compute power in it comes from its GPUs, though there are a ridiculous number of general purpose CPUs in these things too. 3. Even "normal" hacking on "normal" CPUs on a singe box now runs on lots of fairly wimpy processors. I do lots of compiler hacking these days, and my normal lab machine has 64 cores, 128 hyperthreads, and a half T of RAM. It rebuilds one system I need to recompile a lot, which takes like 45 minutes to build on my laptop, in two minutes. Note that this box is both on the older side, the better ones in the lab have a lot more RAM, newer and better processors and more of them, etc. This box also costs a ridiculously small fraction of what I cost, a serious inversion of the old days when I started out and a machine cost a whole lot more compared to a human. Sadly my laptop is stalled out and hasn't gotten any better in forever, but the machines in the lab still keep getting better. However, the only way to take advantage of that is parallelism. Luckily parallel builds work pretty well. Perry -- Perry E. Metzger perry(a)piermont.com

On Thu, Jun 28, 2018 at 8:56 AM, Larry McVoy &lt;lm(a)mcvoy.com&gt; wrote: Well, we want to be able to manage 100G or more of encrypted traffic sanely. We currently get this by lots (well 20) of not-so-wimpy cores to do all the work since none of the offload solutions can scale. The problem is that there's no systems with lots (100's) of wimpy cores that we can do the offload with that also have enough bandwidth to keep up. And even if there were, things like NUMA and slow interprocessor connects make the usefulness of the boatloads of cores a lot trickier to utilize than it should.... Then again, a lot of what we do is rather special case, even if we do use off the shelf technology to get there... Warner

On 6/28/18, Perry E. Metzger &lt;perry(a)piermont.com&gt; wrote: That, and power consumption/heat dissipation. -Paul W.

On Thu, 28 Jun 2018 14:42:47 -0600 Warner Losh &lt;imp(a)bsdimp.com&gt; wrote: Tens of thousands of machines is a lot more than one. I think the point stands. This is the age of distributed and parallel systems. I believe all of this, but I think it is consistent with the point. You're not trying to buy $100,000 CPUs that are faster than the several-hundred-per-core things you can get, because no one sells them. You're building systems that scale out by adding more CPUs and more boxes. You might want very high end CPUs even, but the high end isn't vastly better than the low, and there's a limit to what you can spend per CPU because there just aren't better ones on the market. So, all of this means that, architecturally, we're no longer in an age where things get designed to run on one processor. Systems have to be built to be parallel and distributed. Our kernels are no longer one fast core and need to handle multiprocessing and all it entails. Our software needs to run multicore if it's going to take advantage of the expensive processors and motherboards we've bought. Thread pools, locking, IPC, and all the rest are now a way of life. We've got ways to avoid some of those things by using share nothing and message passing, but even so, the fact that we've structured our software to deal with parallelism is unavoidable. Why am I belaboring this? Because the original point, that language support for building distributed and parallel systems does help, isn't wrong. There are a lot of projects out there using things like Erlang and managing nearly miraculous feats of uptime because of it. There are people replacing C++ with Rust because they can't reason about concurrency well enough without language support and Rust's linear types mean you can't write code that accidentally shares memory between two writers by accident. The stuff does matter. Perry -- Perry E. Metzger perry(a)piermont.com

On Thu, Jun 28, 2018 at 06:29:54PM -0400, Theodore Y. Ts'o wrote: As usual, Ted gets it. Precisely (and well put Ted!) Perry, please take this in the spirit in which is intended, but you're arguing with people who have been around the block (there are people on this list that have 5 decades of going around the block - looking at you Ken). I designed the first clustered product at Sun, I was the 4th guy at google (working on clusters there), Ted is a Linux old timer, Clem goes back in Unix farther than I do, Ken did much of Unix, etc. There are a ton of people on this list that make me look like a nobody, you want to be careful in that crowd. This is a really poor place for a younger person to come in and make loud points, that is frowned upon. It's a fantastic place for a younger person to come in and learn. All of us old farts want to pass on what we know and will gladly do so. But some of us old farts, like me, are really tired of arguing with people that don't see the whole picture. This is not the place to bring the whole picture into focus for you, sorry. If you want to argue about stuff I'll eventually go away and so will other old farts. I kinda think you don't want to chase me away or other old farts away, this is a place where we (mostly) talk about Unix history and if the old farts go away, so does the history. I'm not saying you can't voice your opinion and argue all you want, just saying this might not be the list for that. But that's just my view, Warren will step in if he needs to. Cheers and welcome, --lm

Heh, check this out: $ call Viktor Viktor Dukhovni 718-754-2126 (W/Morgan Stanley) Sorry Perry, I had completely forgotten about all that stuff. Now that you mention it, I do remember your install stuff, it was very cool. But I'm horrible with names, when I was teaching at Stanford I'd have students come up to me from a couple of semesters ago and I'd have no idea what their names were. Sorry about that. Well welcome to the old farts club, I'll cut you some slack :) I still think you are missing the point I was trying to make, it's amusing, a bit, that you are preaching what you are to me as a guy who moved Sun in that direction. I'm not at all against your arguments, I was just making a different point.

On 28 Jun 2018 18:29 -0400, from tytso(a)mit.edu (Theodore Y. Ts'o): What's more, I suspect that in order to get good performance out of those wimpy cores, you'd rather need _more_ cache per core, than less or the same, simply because there's less of an advantage in raw clock. One doesn't have to look hard to find examples of where adding or increasing cache in a CPU (these days on-die) have, at least for workloads that are able to use such cache effectively, led to huge improvements in overall performance, even at similar clock rates. Of course, I can't help but find it interesting that we're having this discussion at all about a language that is approaching 50 years old by now (Wikipedia puts the earliest design in 1969, which sounds about right, and even K&R C is 40 years old by now). Sure, C has evolved -- for example, C11 added language constructs for multithreaded programming, including the _Thread_local storage class specifier) -- but it's still in active use and it's still recognizably an evolved version of the language specified in K&R. I can pull out the manual for a pre-ANSI C compiler and look at the code samples, and sure there are things about that code that a modern compiler barfs at, but it's quite easy to just move a few things around a little and end up with pretty close to modern C (albeit code that doesn't take advantage of new features, obviously). I wonder how many of today's programming languages we'll be able to say the same thing about in 2040-2050-ish. -- Michael Kjörling • https://michael.kjorling.se • michael(a)kjorling.se “The most dangerous thought that you can have as a creative person is to think you know what you’re doing.” (Bret Victor)

On Thu, 28 Jun 2018 16:52:26 -0400 Lawrence Stewart &lt;stewart(a)serissa.com&gt; wrote: Of course, Tilera is a many core architecture, so again, I think this supports the original point, which is we're now in the parallel and distributed age, and that demands software design techniques to match. (Indeed, If I Recall Correctly, the guys at MIT that designed the original Tilera stuff are now looking at building 1000 core devices.) Perry -- Perry E. Metzger perry(a)piermont.com

On Thu, 28 Jun 2018 12:45:39 -0400 Paul Winalski &lt;paul.winalski(a)gmail.com&gt; wrote: Dunno. Rust does some amazing things because it has a linear type system, which means both that it can be a fully safe language even though it doesn't have a garbage collector, and that it can allow sharing of memory without any fear of multiple writers touching the same block of memory. I used to think that there hadn't been much progress in computer science in decades and then I fell down the rabbit hole of modern type theory. The evolution of type systems over the last few decades has changed the game in a lot of ways. Most people aren't aware of the progress that has been made, which is a shame. You're not mentioning things like linear types, effect systems, etc. Perry -- Perry E. Metzger perry(a)piermont.com

On Thu, 28 Jun 2018 10:15:38 -0400 "Theodore Y. Ts'o" &lt;tytso(a)mit.edu&gt; wrote: Let me try to explain my thinking on this topic. 1. Cobol/C/C++/Fortran/Go/Java/Perl/Python/Ruby/... will be around for a long time. Lots of computing platforms & existing codes using them so them languages will continue to be used. This is not particularly interesting or worth debating. 2. A lot of processor architecture evolution in the last few decades has been to make such programs run as fast as possible but this emulation of flat shared/virtual address spaces and squeezing out parallelism at run time from sequential code is falling further and further behind. The original Z80 had 8.5E3 transistors while an 8 core Ryzen has 4.8E9 transistors (about 560K Z80s worth of transistor). And yet, it is only 600K times faster in Dhrystone MIPS even though it has > thousand times faster clock rate, its data bus is 8 times wider and there is no data/address multiplexing. I make this crude comparison to point out how much less efficient these resources are used due to the needs of above languages. 3. As Perry said, we are using parallel and distributed computing more and more. Even the RaspberryPi Zero has a several times more powerful GPU than its puny ARM "cpu"! Most all cloud services use multiple cores & nodes. We may not set up our services but we certainly use quite a few of them via the Internet. Even on my laptop at present there are 555 processes and 2998 threads. Most of these are indeed "embarrassingly" parallel -- most of them don't talk to each other! Even local servers in any organization run a large number of processes. Things like OpenCL is being used more and more to benefit from whatever parallelism we can squeeze out of a GPU for specialized applications. 4. The reason most people prefer to use one very high perf. CPU rather than a bunch of "wimpy" processors is *because* most of our tooling uses only sequential languages with very little concurrency. And just as in the case of processors, most of our OSes also allow use of very little parallelism. And most performance metrics focus on single CPU performance. This is what is optimized so given these assumptions using faster and faster CPUs makes the most sense but we are running out that trick. 5. You may well be right that most people don't need faster machines. Or that machines optimized for parallel languages and codes may never succeed commercially. But as a techie I am more interested in what can be built (as opposed to what will sell). It is not a question of whether problems amenable to parallel solutions are the *only problems that matter*. I think about these issues because a) I find them interesting (one among many). b) Currently we are using resources rather inefficiently and I'm interested in what can be done about it. c) This is the only direction in future that may yield faster and faster solutions for large set of problems. And in *this context* our current languages do fall short. 6. The conventional wisdom is parallel languages are a failure and parallel programming is *hard*. Hoare's CSP and Dijkstra's "elephants made out of mosquitos" papers are over 40 years old. But I don't think we have a lot of experience with parallel languages to know one way or another. We are doing adhoc distributed systems but we don't have a theory as to how they behave under stress. But see also [1] 7. As to distributed vs parallel systems, my point was that even if they different, there are a number of subproblems common to them both. These should be investigated further (beyond using adhoc solutions like Kubernetes). It may even be possible to separate a reliability layer to simplify a more abstract layer that is more common to the both. Not unlike the way disks handle reliability so that at a higher level we can treat them like a sequence of blocks (or how IP & TCP handle reliability). Here is a somewhat tenuous justification for why this topic does make sense on this list: Unix provides *composable* tools. It isn'just that one unix program did one thing well but that it was easy to make them worked together to achieve a goal + a few abstractions were useful from most programs. You hid device peculiarities in device drivers and filesystem peculiarities in filesystem code and so on. I think these same design principles can help with distributed/parallel systems. Even if we get an easy to use parallel language, we may still have to worry about placement and load balancing and latencies and so forth. And for that we may need a glue language. [1] What I have discovered is that it takes some experience and experimenting to think in a particular way that is natural to the language at hand. As an example, when programming in k I often start out with a sequential, loopy program . But if I can think about in terms of "array" operations, I can iterate fast and come up with a better solution. Not have to think about locations and pointers make this iterativer process very fast. Similarly it takes a while to switch to writing forth (or postscript) code idiomatically. Writind idiomatic code in any language is not just a matter learning the language but being comfortable with it, knowing what works well and in what situation. I suspect the same is true with parallel programming as well. Also note that Unix hackers routinely write simple parallel programms (shell pipelines) but these may seem quite foreign to people who grew up using just GUI.

On Fri, 29 Jun 2018 08:58:48 -0400 "Theodore Y. Ts'o" &lt;tytso(a)mit.edu&gt; wrote: On the other hand, when people are doing machine learning stuff on GPUs or TPUs, or are playing a video game on a GPU, or are doing weather prediction or magnetohydrodynamics simulations, or are calculating the bonding energies of molecules, they very much _are_ going to use each and every computation unit they can get their hands on. Yah, a programmer doesn't use very much CPU unless they're compiling, but the world's big iron isn't there to make programmers happy these days. Even if "all" you're doing is handling instant message traffic for a couple hundred million people, you're going to be doing a lot of stuff that isn't easily classified as concurrent vs. parallel vs. distributed any more, and the edges all get funny. The parallelism winter already happened, in the late 1980s. Larry may think I'm a youngster, but I worked over 30 years ago on a machine called "DADO" built at Columbia with 1023 processors arranged in a tree with MIMD and SIMD features depending on how it was arranged. Later I worked at Bellcore on something called the Y machine built for massive simulations. All that parallel stuff died hard because general purpose CPUs kept getting faster too quickly for it to be worthwhile. Now, however, things are different. I'm going to dispute that. Sun bet the company on the notion that people wanted machines they could sell them at ridiculously high margins with pretty low performance per dollar when they could go out and buy Intel boxes running RHEL instead. As has been said elsewhere in the thread, what failed wasn't Niagra as much as Sun's entire attempt to sell people what were effectively mainframes at a point where they no longer wanted giant boxes with low throughput per dollar. Frankly, Sun jumped the shark long before. I remember some people (Hi Larry!) valiantly keeping adding minor variants on the SunOS release number (what did it get to? SunOS 4.1.3_u1b or some such?) at a point where the firm had committed to Solaris. They stopped shipping compilers so they could charge you for them, stopped maintaining the desktop, stopped updating the userland utilities so they got ridiculously behind (Solaris still, when I last checked, had a /bin/sh that wouldn't handle $( ) ), and put everything into selling bigger and bigger iron, which paid off for a little while, until it didn't any more. Your laptop's GPUs beat it's CPUs pretty badly on power, and you're using them every day. Ditto for the GPUs in your cellphone. You're invoking big parallel back-ends at Amazon, Google, and Apple all the time over the network, too, to do things like voice recognition and finding the best route on a large map. They might be "specialized" uses, but you're invoking them enough times an hour that maybe they're not that specialized? Er, Erlang? Rust? Go has CSP and is in very wide deployment? There's multicore stuff in a bunch of functional languages, too. If you're using Firefox right now, a large chunk of the code is running multicore using Rust to assure that parallelism is safe. Rust does that using type theory work (on linear types) from the PL community. It's excellent research that's paying off in the real world. (And on the original point, you're also using the GPU to render most of what you're looking at, invoked from your browser, whether Firefox, Chrome, or Safari. That's all parallel stuff and it's going on every time you open a web page.) Perry -- Perry E. Metzger perry(a)piermont.com