>> Does anyone know whether there are implementations of mmap that
>> do transparent file sharing? It seems to me that should be possible by
>> making the buffer cache share pages with mmapping processes.
> These days they all do. The POSIX rationale says:
> ... When multiple processes map the same memory object, they can
> share access to the underlying data.
Notice the weasel word "can". It is not guaranteed that they will do so
automatically without delay. Apparently each process may have a physically
distinct copy of the data, not shared access to a single location.
The Linux man page mmap(2), for example, makes it very clear that mmap
has a cache-coherence problem, at least in that system. The existence
of msync(2) is a visible symptom of the problem.
[Weasel words of my own: I have not read the POSIX definition of mmap.]
Doug
On Mon, 16 Dec 2024, Konstantin Belousov wrote:
> On Mon, Dec 16, 2024 at 02:08:43PM -0500, John Levine wrote:
>> PS: I can believe there are some versions of linux that screwed up disk cache
>> coherency, but that just means they don't properly implement the spec, not for
>> the first time. I mean, it's not *that* hard to make all the maps point to the
>> same physical page frame, even on a machine like POWER with reverse page maps.
>
> This is not enough. There are (were ?) architectures, typically with the
> virtually addressed caches, which require all mappings of the same page
> to be suitably aligned, at least. ...
>
> If addresses of different mappings are not aligned, caches were not coherent.
I think we're in "so don't do that" territory. mmap() normally lets the
system pick the memory address to map so it can pick something suitably
aligned. You can pass the MAP_FIXED flag to tell it to map at a
particular address, but it can return EINVAL if the address doesn't work.
The POSIX description says "The use of MAP_FIXED is discouraged, as it may
prevent an implementation from making the most effective use of
resources."
It's not always trivial to make this work. On systems with reverse maps,
a physical page can only be mapped to one virtual address at a time, so
for shared pages it has to mark all of the aliases nonresident and on a
fault remap the page into the map of the process that is running. But
it's not rocket science, either.
R's,
John
> "John Levine" <johnl(a)taugh.com> wrote:
>> M4 was written in the 1970s by Kernighan and Ritchie in C ...
> In private mail, BWK told me that it was DMR who wrote m4. He
t> hen reimplemented it in Ratfor for "Software Tools".
> Arnold
The book says colorfully, "... [and] we are grateful to him for
letting us steal it."
Doug
> well after Unix had fledged, its developers at CSRC found it necessary
> and/or desirable to borrow back a Multics concept: they named it mmap().
As far as I know no Research version of Unix ever had mmap.
Multics had a segmented universal memory. A process incorporated
segments into its address space The universal memory was normally
addressed via a hierachical segment-name directory. With enhancement
to provide for multisegment "files", the directory could serve as a file
system and file I/O became data transfer between segments.
Unix originally imitated the Multics file system, but not the universal
memory. mmap(2) weakly imitates universal memory by allowing a process
to nominally incorporate a portion of a file into the process address space
at page-level granularity. However, an update is guaranteed to be visible
to the file and other processes only upon specific request.
Does anyone know whether there are implementations of mmap that
do transparent file sharing? It seems to me that should be possible by
making the buffer cache share pages with mmapping processes.
Doug
I'm curious if anyone has any history they can share about the BSD
"talk" program.
I was fond of this back when it was still (relatively) common, but
given the way it's architected I definitely see why it fell out of use
as the Internet grew. Still, does anybody know what the history behind
it is? Initially, I thought it was written by Mike Karels, but that
was just my speculation from SCCS spelunking, and looking at the
sources from 4.2, I see RCS header strings that indicate it was
written by "moore" (Peter Moore?). talk.c says, "Written by Kipp
Hickman".
It seems to have arrived pretty early on with respect to the
introduction of TCP/IP in BSD: the README alludes to some things
coming up in 4.1c. Clem, you seem to have had a hand in it, and are
credited (along with Peter Moore) for making it work on 4.1a.
So I guess the question is, what was the motivation? Was it just to
have a more pleasing user-to-user communications experience, or was
discussion across the network an explicit goal? There's a note in
talk.c ("Modified to run between hosts by Peter Moore, 8/19/82") that
suggests this wasn't the original intent. Who thought up the
character-at-a-time display mode?
Thanks for any insights.
- Dan C.
IEEE Transactions on Software Engineering has asked me to write a
retrospective on the influence of SCCS over the last 50 years, as my SCCS
paper was published in 1975. They consider it one of the most influential
papers from TSE's first decade.
There's a funny quote from Ken Thompson that circulates from time-to-time:
"SCCS, the source motel! Programs check in and never check out!"
But nobody seems to know what it means exactly. As part of my research, I
asked Ken what the quote meant, sunce I wanted to include it. He explained
that it refers to SCCS storing binary data in its repository file,
preventing UNIX text tools from operating on the file.
Of course, this is only one of SCCS's many weaknesses. If you have anything
funny about any of the others, post it here. I already have all the boring
usual stuff (e.g., long-term locks, file-oriented, no merging).
Marc Rochkind
mrochkind.com
I was thinking about this some more.
IIRC: Peter and I sketched out the protocol for the sockets version on a
whiteboard in our office one night after a beer and pizza run. Rick
Spicklemeir, Tom Quarles, and Jim Kleckner also participated in those bull
sessions. I started writing the program soon after that and had it working
to a point in a couple of hours. I don't remember the issues, but a couple
of them were when I left for the USENIX conference later that week. When I
got back Peter had finished it and put it into RCS. The key is that the
coding was primarily Peter and myself, but Rick, TQ, and Jim all had
contributed in some manner, too,
Although the famous bug of using a vax integer, you can squarely blame me —
and as I said, having worked on networking for several years before my time
at UCB, I should have known better. But did not even think about it. I
failed Henry's ten programming commandments and concluded that the world
was a Vax. Mei culpa.
ᐧ
On Fri, Dec 13, 2024 at 10:03 AM Mark Seiden <mseiden(a)gmail.com> wrote:
> (I know there is a special place in hell for those who explain a joke,
> but, you asked…)
>
> it’s just an allusion to the Black Flag Roach Motel product (still being
> produced)
> which has a trademark on the phrase “Roaches Check in… But they Don’t
> Check Out”.
>
> Yeah, I knew that much. My question to Ken was about what this was saying
about SCCS.
Marc
This is verging on COFF material, and I won't mind if someone
moves the discussion thither:
Clem Cole:
As a satisfied user of SCCS (and later Bitkeeper), it's still my preferred
choice.
====
I have to admit SCCS is one of the many pieces of software
I tried for a week or two > 40 years ago and abandoned because
I couldn't stand it. I don't think I ever tried RCS, because
I could see it didn't what I saw as the underlying problems.
CVS likewise. Subversion was the earliest version-control
system that felt usable to me.
What bugged me so much? The earlier systems were focussed
entirely (or for CVS, almost entirely) on individual files.
There was no way to link changes that affected more than one
file:
-- SCCS and RCS don't (so far as I can tell) understand any
relation between files at all. When I'm working on a real
project of any size, there is almost always more than one
file, and there are often changes that affect multiple files
at once: if an interface changes, the changes to definitions
and uses really all should be a single commit, logged in one
go and reversible as a single coordinated operation; if I
refactor code, moving it between files and perhaps adding
files or removing them, likewise. It is possible to check
in a bunch of files at once and reuse the log message, but
I couldn't find a way to make it a true coordinated single
commit; and neither SCCS nor RCS has a way I could find to
record, as a structured commit, that files are added or
removed from a directory or directory tree.
-- CVS can track adds and deletes and can bundle changes
into a single commit on the command line, but the changes
are still stored individually per file.
-- None of the systems even tries to track file renames,
something I also do occasionally as programs and especially
documentation evolves.
It wasn't until I discovered Subversion that I started using
version control regularly in my own work.
Ironically, a few years after I began using svn for myself,
I ended up working in places that had great compost heaps
of RCS and CVS. This would have made sense in the 1990s,
but it was in the 2010s, and continues in the 2020s.
I now use hg for my personal work, and have been attempting
to drag my workplace into using git if only so new hires
don't have to learn clumsy outdated stuff just to do their
jobs. I expect to retire (not for this reason) before I
really succeed, though.
Norman Wilson
Toronto ON
----- Forwarded message from Rick Smith <rick(a)rbsmith.com> -----
Date: Fri, 13 Dec 2024 18:22:29 -0500
From: Rick Smith <rick(a)rbsmith.com>
To: Larry McVoy <lm(a)mcvoy.com>
Subject: Re: [TUHS] SCCS roach motel
X-Mailer: Apple Mail (2.3826.300.87.4.3)
Hi Larry,
I likely can???t post to TUHS without joining, though I can read it fine.
Anyway, feel free to edit and post:
Marc,
I remember in 1992 driving to the UCSD library to get that issue of TSE and
make make copy of the article. I still have it and wrote about it on HN [1].
The link I posted at the end there is dead (wayback [2]). It is a post
by J??rg Schilling about your post here on TUHS about SCCS.
Yes, we could get lost in the weeds about choice of control-A, lack of
merge bookkeeping and the corner cases with -i and -x when applying strict
order to a partial order when computing the graph-to-set but in the big
picture, the SCCS system is a huge contribution. Congratulations on
writing one of the most influential papers in TSE's first decade!
Certainly is that for me.
Aside, Larry wrote:
> ... He [Rick] did point out that my weave implementation was the only
> one written such that I could have N serial sets in my hand, and do one
> pass through the weave and get N different checked out files. I don't
> think we ever used that but if we did it would be in smerge.c.
The original makepatch() uses it in sccs_getdiffs() which walks the weave
calling changestate() once to track weave structure and printstate() twice,
generating a diff in one pass.
While this is a hats off to Marc, there is also a hats off to Larry
for extending the SCCS work with TeamWare and Bitkeeper. I studied SCCS
(and other engines).
Larry built systems.
[1] https://news.ycombinator.com/item?id=26225001
[2] https://web.archive.org/web/20190403213137/http://sccs.sourceforge.net/sccs…
----- End forwarded message -----
--
---
Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat
