[TUHS] [simh] 2bsd tarball

Johnny Billquist bqt at softjar.se
Thu Jul 30 00:30:11 AEST 2020


Yes. That is a much better way to look at it.

   Johnny

On 2020-07-29 15:52, John Cowan wrote:
> When I talk about DECtape in my capacity as the local old fart, i 
> describe it as "a disk with one track and about 1500 small sectors that 
> spins ve-ry ve-ry slow-ly..
> 
> On Wed, Jul 29, 2020 at 5:58 AM Johnny Billquist <bqt at softjar.se 
> <mailto:bqt at softjar.se>> wrote:
> 
>     Just a small comment. Whoever it was that thought DECtape was a tape
>     was
>     making a serious mistake. DECtapes are very different from magtapes.
> 
>         Johnny
> 
>     On 2020-07-29 02:21, Clement T Cole wrote:
>      >
>      > Cross posting to simh - since much of this has been discussed in the
>      > last few days there also....
>      >
>      > in for penny, in for pound ... here is the history ...  man ... I
>     lived
>      > this and I'll need a strong drink later tonight after I write it
>     all up.
>      >
>      >
>      > On Tue, Jul 28, 2020 at 7:04 PM Will Senn <will.senn at gmail.com
>     <mailto:will.senn at gmail.com>
>      > <mailto:will.senn at gmail.com <mailto:will.senn at gmail.com>>> wrote:
>      >
>      >     I recall having to do something with cont.a files, which are not
>      >     present on these images. So, my questions is, does anyone
>     know of or
>      >     have an actual 2bsd tape/tape image?
>      >
>      > cont.a is a tp-v6 and earlier ism.
>      >
>      > DECtape had a directory at the front of the tape (think
>      > superblock/ilist), but could do cool things and be treated more
>     like a disk.
>      > When tp was created for a very early version of Unix (I'm not sure
>      > which, could be V2), Ken/Dennis et al had DECtape units and so the
>      > original scheme followed the media.   This also meant that the
>     program
>      > could write files and go back and update the directory, which is
>     a no-no
>      > with many tape systems.  Then research got a 9-track unit.   So
>     tp was
>      > changed to calculate how much space was going to be needed, write
>     the
>      > directory, then the datablocks.  All good ... except...
>      >
>      > 9-track could write more files than the directory could take. 
>       So for
>      > many years, people would use the ar(1) program to take a number
>     of files
>      > in a directory, create a file called cont.a and then delete the
>     files.
>      > Then the tree would be written with tp, when you read it, you
>     reversed
>      > the ar(1) process.  If you look at the USENIX/Harvard tape on the
>     TUHS
>      > you'll see this scheme in use.
>      >
>      > BTW: tp was written in assembler and all the data structures were
>     using
>      > PDP-11 binary formats.  Eventually, Harvard wrote stp (super-tp)
>     in C
>      > (which is on the USENIX tape Warren has in the archives) that worked
>      > like the original assembler tp but also put a redundant directory
>     at the
>      > end of the tape.  Another issue with tp was if the you had a bad
>     block
>      > in the first few blocks you could not decode the rest of the tape.
>      > [There were some other issues with the UNIX tree structure as
>     disks got
>      > bigger but I'm going to ignore all that - other than to say, tp had
>      > lived it life].
>      >
>      > Enter Mashey and the PWB 1.0 folks (which is based on V6). 
>     Someone in
>      > USG created cpio (and volcpy) as part of the PWB 1.0.   Like tp
>     it was a
>      > PDP-11 binary format, but unlike tp, the tape directory is threaded.
>      > /i.e./ block one describes the first file only and includes the
>     size of
>      > the following file, then file itself, then a new directory block
>     for the
>      > next file and again that file (rinse and repeat).  So it improved
>     on tp
>      > in the directory threading, but was still binary and for a
>     reasons I'll
>      > leave out had a different user interface.
>      >
>      > As part of V7, Ken wrote a new program, tar [you can ask him
>     why].  But
>      > like cpio he used a threaded tape directory, but unlike cpio it was
>      > always ASCII and not PDP-11 specific.  Furthermore, the user
>     interface
>      > was made to parrot tp.  So, certainly, it had the advantage that
>      > changing tp scripts to use tar was pretty easy i.e. s/tp/tar/   
>       not so
>      > for coil.  And it was muscle memory compliant.
>      >
>      > For PWB 2.0, cpio was updated to allow a -c option to write the
>     header
>      > in ascii and -s to byte swap the binary.   But the damage had
>     been done ...
>      >
>      > Thus began 'tar wars' which was a battle that raged officially
>     over tape
>      > archive formats, but really was an argument about user interfaces.
>      > Since tar was part of Research and the Universities and commercial
>      > people used it, only USG and the folks inside the Bell System
>     were using
>      > cpio, as officially none of us had it since PWB was not released
>     to us
>      > (although thanks to many AT&T employees doing an OYOC year, many
>     schools
>      > like UCB, MIT and CMU all had the sources to cpio anyway -- for
>     instance
>      > you'll see it hidden away on Kirk's CD).
>      >
>      > I personally had used both, preferred tar for easy of use and ASCII
>      > directories.  But, note I had written car at Masscomp, but never
>     tpio.
>      > This was our trick to use the file scripting list that cpio could
>     do,
>      > but create tar format tapes - which was handy.  I never wrote
>     tpio which
>      > would have been cpio format using tp/tar user interface as I did not
>      > need it.
>      >
>      > Roll forward to the /usr/group UNIX standard that Heinz chaired.  We
>      > ended up not being able to agree on a distribution format, but
>     the ISVs
>      > were PO because now they could create UNIX programs that might
>     actually
>      > work across systems, but they had not standard way to distribution.
>      > Roll forward again to IEEE.  Heinz's committee was officially
>     disbanded
>      > (story discussed elsewhere) and we were created as IEEE P1003
>     with Jim
>      > Issack as Chair. This time the ISV's said we had to have a
>     distribution
>      > format.  Since *.1 was only an API we were allowed to avoid the user
>      > interface issue but only examine the on tape format.
>      >
>      > It turns out while it seems to have been unintended, Ken's
>     original V7
>      > implementation has an interesting coding feature/bug which turns
>     out to
>      > be what clinched the deal.   When Ken creates the directory block
>     for
>      > each file, he did bcopy of 0's to the buffer before he wrote that
>     data
>      > that fills it in.  Then when he calculated the checksum for the
>      > directory header block, he summed the entire block (which because
>     of the
>      > bcopy was zeros).  This means if you write beyond the end of Ken's
>      > original header and include that extra data in the chksum, the
>     original
>      > program will ignore the new information but accept the directory
>     block
>      > as valid.  i.e. he had built an extension mechanism into the tar
>     on-tape
>      > format.
>      >
>      > cpio's ASCII on tape format was not able to do that as the
>     checksum used
>      > a sizeof(header struct) in the checksum routine.
>      >
>      > USTAR was born ... Ken had written things like the UID/GID as ASCII
>      > representations of the integer value in the original header.  USTAR
>      > added the ASCII representation of the username and the group name
>     since
>      > that was more often portable between systems than the numbers. 
>       There
>      > were other additions like more room for the pathname new file types
>      > /etc/.  But the key is that a USTAR tape can be read by the
>     original V7
>      > (and follow on) tape formats, although may not recognize all the
>      > filetype or use all of the new information.
>      >
>      > A few years later during *.2 discussions, we finally got into the
>     user
>      > interface stuff and pax(1) was born.  Knowing my hack with car,
>     Keith
>      > Bostic, Jim McGuiness and I wrote up a description of a program that
>      > could with both users interfaces scheme.  USENIX provided funding
>     for a
>      > student to implement it and put the sources out on
>     comp.unix.sources at
>      > some point.  That proposal was originally accepted at the first tape
>      > user interface program in *.2 [a few years later after I stopped
>     being
>      > part of the committee, the USG folks did get an alternate CPIO
>     format
>      > accepted and cpio as an allowed program.   USENIX paid to have the
>      > program updated to operate like cpio if it was called that, pure
>     V7 tar
>      > if called that and if pax, user USTAR].
>      >
>      > 'nuf said ... I hope.
>      >
>      > Clem
>      >
>      > _._,_._,_
>      >
>     ------------------------------------------------------------------------
>      > Groups.io Links:
>      >
>      > You receive all messages sent to this group.
>      >
>      > View/Reply Online (#62) <https://groups.io/g/simh/message/62> |
>     Reply To
>      > Group
>      > <mailto:simh at groups.io
>     <mailto:simh at groups.io>?subject=Re:%20Re%3A%20%5Bsimh%5D%20%5BTUHS%5D%202bsd%20tarball>
> 
>      > | Reply To Sender
>      > <mailto:clemc at ccc.com
>     <mailto:clemc at ccc.com>?subject=Private:%20Re:%20Re%3A%20%5Bsimh%5D%20%5BTUHS%5D%202bsd%20tarball>
> 
>      > | Mute This Topic <https://groups.io/mt/75856261/4814011> | New
>     Topic
>      > <https://groups.io/g/simh/post>
>      >
>      > Your Subscription <https://groups.io/g/simh/editsub/4814011> |
>     Contact
>      > Group Owner <mailto:simh+owner at groups.io
>     <mailto:simh%2Bowner at groups.io>> | Unsubscribe
>      > <https://groups.io/g/simh/leave/8625569/104597204/xyzzy>
>     [bqt at softjar.se <mailto:bqt at softjar.se>]
>      >
>      > _._,_._,_
> 
>     -- 
>     Johnny Billquist                  || "I'm on a bus
>                                         ||  on a psychedelic trip
>     email: bqt at softjar.se <mailto:bqt at softjar.se>             || 
>     Reading murder books
>     pdp is alive!                     ||  tryin' to stay hip" - B. Idol
> 

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt at softjar.se             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol


More information about the TUHS mailing list