On Mar 29, 2018 5:12 PM, "Dave Horsfall" <dave(a)horsfall.org> wrote: On Thu, 29 Mar 2018, Paul Winalski wrote: [...] What is the history of shared objects on Unix? When did they first appear, I first saw 'em when they appeared in SunOS (can't remember which release) and thought they were wonderful, along with loadable drivers. Shared libraries were 4.0 due to the new mmap call. These were a.out based I'm pretty sure. Warner

Paul, The point is that there were a number of different shared library implementations for UNIX over the years. That was one of the 'knocks' when comparing VMS to 4.1BSD in the early 80s. VMS had shared libraries from the start. I'm pretty sure the first unix to support shared libraries was CMU's Mach using its modified about called macho, which lives today in Mac OSX. BSD4.2 had started to implement it, but it was incomplete so folks like Sun and Masscomp each did their own scheme, both based on modified a.out. A big problem was every vendor messed with a.out in a different way -- so Sun's, Masscomp and Apollo's versions were all a little different and you a linker guy, you know that a.out was not a great format for same. With System V.3, AT&T introduced was an a new file format, called the Common Object File Format - *a.k.a.* COFF. SVR3 supported shared libs. In fairness, COFF was a huge improvement over a.out, but it was done when AT&T was in its 'consider it standard' time and trying to force its will and wanted licensing fees. Let's say that failed for non-technical reasons. Unfortunately, it lead to more confusion and we ended up a number of different COFF-almost, sort-of, extensions. IIRC, IBM went with a modified COFF, but again we were in a cold war of who could do what. I remember that the time, the Gnu guys wrote tool called 'robitussin' - which 'cured COFFs.' With, AT&T's SVR4 release the world was introduced to the 'Extended Linker Format' - ELF, which fixed a number of issues with COFF, the primary one being that it could loaded images faster and you could page from directly, which neither a.out nor COFF could easily. Again IIRC, SVR4's linker could handled AT&T's COFF files. I have never known the legals on it, but some how the details of ELF did become public and somebody reimplemented the GNU compilers to use it and AT&T for whatever reason did not complain (maybe the had their hands full at time with BSDi case). Anyway, eventually both Linux and FreeBSD switched to that version of Gnu and its pretty much been stable since. ᐧ

You might be right i don’t exactly remember the dates of each release. that said I do remember Mach ran on a number of systems before Next, but they were not commercial. It was similar to BSD where CMU ported to that hardware - I’ve forgotten the name of the guys that spun out of DG that had an early MP based system that used reflected memory- but I remember CMU had very early version of Mach running there that was based on the 4.1mash up like the original Vax code from CMU that they started. The point is that early Mach was working before 4.2 way final and released to the DARPA community not much later. CRSG was doing Val support for DARPA and CMU was more researchy (which was a point of contention in a lot of fronts that I’ll not scratch open any further). I was not commenting on the implementation of the goodness of the CMU or CSRG code by the wsy. Sun could have had the first commercial version of shared libs that worked well, although IBM might have been around the same time as Charlie pointed. What had Paul asked when shared libs came to being for Unix. I was trying to say it did not come from any one point. I do believe Mach and it’s parent Accent (which was not Unix) was the first time any Unix had it but like BSD was a DARPA funded project and not commercial. But a lot of different places were working on the idea as it was commonly held to be an issue with Unix. Also, joy / BSD 4.2 was heavily influenced by Accent (and RIG )and the Mach memory system would eventually go back into BSD (4.3 IIRC) - which we have talked about before wrt to sockets and Accent/Mach’s port concept. FWIW If I'm remembering the sequence right I believe Mach 2.5 was quickly created as a update to the original release after joy et al released the final 4.2. 4.1c was 83 as I was leaving UCB and I brought it with me. But I think we had an early 4.1 based Mach tape at Masscomp not too much later ??a year maybe?? And one of the reasons we were interested in it was the shared library code because the exVMS at Masscomp all felt that was a deficiency of Unix and at the time the only ‘open source’ in a HLL if you will that had it was Mach. Sent from my PDP-7 Running UNIX V0 expect things to be almost but not quite.

On Fri, Mar 30, 2018 at 4:52 PM, Jon Forrest &lt;nobozo(a)gmail.com&gt; wrote: ​Two things... 1.) I think we are in complete agreement wrt to code quality - and I think LM is probably correct that Sun had the first real quality implementation. 2.) Paul's question was a little different and I was able to chat with him about it at lunch today. Until a year or so ago, Paul was a compiler/linker guy and was interested less in the OS extension to support shared libraries, but more when did different tool chains start to support it. * i.e.* How did we get from a.out to ELF. The key is that make shared libraries real for the tool chain, you need OS support for shared memories. It turns out that Joy's BSD Unix took its shared memory API (mmap) from Tenex's 'PMAP JSYS' (which BTW, the '72 Tenex CACM paper calls out shared libraries modelled after Multics - I was trying to google the API specific and failed - I think its darned near 1x1). I'm also pretty sure that PMAP influenced the VMS shared memory call that Cutler would build in the mid 70s. Not only that, In Rashid's 81 Accent paper, he too referenced Tenex's PMAP too (more in a minute). Also, please remember, that BSD's mmap was not the first shared memory call for any UNIX. Horton & Johnson can correct me, but I think Dale's team in Columbus had a shared memory in a PDP-11 ??PWB 1.0?? style system - that would eventually begat the System V shmem call (i've got a hard copy of it somewhere in my archives). What I do not remember, and this is where someone like Steve might know, was could COFF use the Columbus shmem to do shared libs? Certainly by that time, shared libraries for UNIX was being heavily discussed both in and out of AT&T. I remember that either V.2 or V.3 had a shared lib scheme that put the libraries at a static address as the System V kernel support was not very flexible. But to Paul's question, was that AT&T had started to put shared libs into their flavor around the same time? *i.e. *when did Summit start to support it. I was trying to get an account of which versions of which OS distributions/releases had some sorts of kernel extensions that allowed shared libraries and that also supported tool chains that could provide them -- *i.e.* what Paul was asking about. I was not commenting on who's kernel to support same was good or bad. I did comment on the file format. COFF was a dead end because of the way Summit's management introduced it to larger Unix community, even if it was 'better' and solved some of the earlier issues with a.out. Macho was already there for instance, and it was an 'extensible' version of a.out and had some level of support for architecture of the binary contained within. How v6/v7 a.out got extended to support shared libs, had varied widely by different manufacturers tool chains. Originally, most manufacturers still called it a.out and just hacked on their own version of the toolchain. CMU gave it a different name (macho) and IIRC made the kernel handle both *i.e. *still supported the original BSD a.out but you were limited in what could be done with it to only the 4.1BSD api. Remember, Mach 1.0, reacting to Accent' failure, was BSD4.1++ so they wanted to be able to run BSD binaries, at least on Vaxen. In fact the way you installed Mach1.0 was build a new kernel boot it on your 4.1BSD system. Binary compatibility was sort of new concept for the University community --- in fact, around that time at UCB I remember Joy once saying he thought it was good to force people to recompile as it ensured the code was not stale. So moving forward, AT&T's COFF had tried to repair the many different a.outs, but AT&T made other major non-technical missteps so only firms that were late in the Unix world bothered with it. And even after trying to be 'common', it too, got extended in N different incompatible (uncommon) ways. As I said, I am not sure how ELF really became the universal format, other than timing. AT&T for whatever reason did not have a fit when it was reimplemented by Gnu - but by the Gnu guys doing that, and the FOSS community switching to that version of the compiler, ELF did win in the long run. And fortunately, the Gnu version of ELF and the SVR4 version seems/appears to be were pretty close [as far as I know they matched, but if someone knows otherwise, I defer to them]. As for the kernel stuff, my other point is that >>idea<< of mmap/pmap was kicking around the Unix community by numerous folks hacking on the kernel for a longer time then when it was finally implemented. LM reminds us that you need something for memory sharing to implement shared libraries. But memory sharing added to Unix before shared libraries was added to the tool chains. I do find it interesting the UCB mmap API call pretty much was all of the Unix kernel implementation agreed to be the memory sharing API and that it happens to be pretty much the same call as Tenex's PMAP. As Paul mentioned at lunch today for him, VMS, Tenex/TOP-20 and eventually Unix it was pretty much the same call for him implementing the user space linker/loader calls, which is part of why he was asking when did the tools start to support it all. Clem ᐧ ᐧ

On Fri, Mar 30, 2018 at 06:42:09PM -0400, Clem Cole wrote: Well, there was a subtle difference between the Solaris SPARC version and the Linux i386 version which we ran in to when porting a program. I can't quite recall, something to do with symbol visibility for symbols defined in the main executable vs those in a shared object. DF

On Thu, Mar 29, 2018 at 08:35:50PM -0500, Charles H. Sauer wrote: What was the underlying technology that enabled them in AIX?

On Thu, Mar 29, 2018 at 8:34 PM, Charles H. Sauer &lt;sauer(a)technologists.com&gt; wrote: Wikipedia says AIX v3 did shared libraries in 1990. SunOS 4.0 had them in 1988, also from wikipedia (though I can confirm 4.0 had shared libraries now). I recall vaguely working with them in 1991 when porting OI from SunOS to AIX 3.mumble, iirc. xlC was the weirdest C++ compiler at the time (which effectively means it wasn't cfront based). But I can't say with absolute certainty.... I’ve sent Jack a note about this discussion. Unless he or ISC folks chime I'd love to see something more definitive than my memory at the time + vague backup from Wikipedia... http://ps-2.kev009.com/rs6000/aix_ps_pdf/programming/using_shared_libraries… is a primary source, but talks of AIX 4.1 Warner

I’m fairly sure Unix didn’t invent the dynamic linking symbol preemption. It was fundamental in the UNIVAC Exec8 operating system. EXEC8 also predates UNIX on the concept of fork() (or as they call it ER FORK$).