[TUHS] Question about BSD disklabel history

[Warner Losh]

On Sun, Dec 31, 2023 at 2:31 PM Clem Cole <clemc at ccc.com> wrote:

> Below... apologies if I forgot something -- this is pretty much as I knew
> it/play it out.
>
> On Sun, Dec 31, 2023 at 12:31 PM Grant Taylor via TUHS <tuhs at tuhs.org>
> wrote:
>
>> Hi,
>>
>> I've found myself wondering about partitions inside of BSD disk labels.
>>
>> Specifically, when and where was the convention that "a" is root, "b" is
>> swap, etc?
>>
>> I also understand the "c" partition to be the entire disk, unless it
>> isn't, at which point it's the entire slice (BIOS / MBR partition)
>> containing the BSD disklabel and "d" is the entire disk.
>>
>> I also found something last night that indicated that OpenBSD uses disk
>> labels somewhat differently than FreeBSD.
>>
>> Aside:  This is one of the dangers of wondering how something curious
>> came to be and why it came to be when working on 10-15 year old FreeBSD
>> systems.
>>
>> A long, strange trip as were.
>
> Step one was the introduction of DEC RP04 disks -- 19 Heads, 21
> Sectors/Track, 411 cylinders or 163989 blocks which will overflow a 16-bit
> PDP-11 register. You find it in the V5 code, but I have Dennis's rp.c
> driver showing split the RP04 into from v6:
>
> struct {
>
> char *nblocks;
>
> int cyloff;
>
> } rp_sizes[] {
>
> 40600, 0, /* cyl 0 thru 202 */
> 40600, 203, /* cyl 203 thru 405 */
> 9200, 0, /* cyl 0 thru 45 */
> 9200, 360, /* cyl 360 thru 405 */
> -1, 0, /* cyl 0 thru 327 */
> -1, 78, /* cyl 78 thru 405 */
> 15600, 0, /* cyl 0 thru 77 */
> 15600, 328, /* cyl 328 thru 405 */
>
> };
>
> (V5/V6 C does not have an unsigned or longs; char * is how the compiler
> created a type that was treated as an unsigned 16-bit integer).
>
> If you look at things like the rk05 in V6, we reserve a few blocks at the
> end of the disk for swap.  Remember quotas and file single file management
> are not there yet, so the scheme with the RP04 starts to be the way of the
> world.  Also, remember Ted has not introduced what would later be called
> fsck(1), so recovery from a crash was painful.   The corruption was often
> on the root disk, a quick dd(1) of the root FS you were quickly back - or
> at least bookable and then you could use  check, ncheck, *et al *on the
> other partitions.
>
> Also, with PWB - a new set of tools was created for disk management that
> Research did not pick up: volcopy, cpio, et al.  The V6 version of PWB is
> where *disklabels are first added to the superblock* BTW.     In fact,
> when we did the original fsck(8) at CMU, we had v5 then v6.  No, PWB
> pollution into our IP stream.  So the original fsck I recently recovered
> for V6 does not have the disklabel code in it.   When Ted goes to USG
> later, he adds the label support (which is why if you compile the version
> in the V7 addenda, it will get errors -- that one is the version on UNIX/TS
> - which the kernel supported disklabels). I had to look at the code, I
> think labels are simple, just names not information about that specific
> FS.  The labels are not per disk pack.
>
> Meanwhile back in Murray Hill ....
>
> With V7, we have long int, but the partition scheme is already in use and
> has some nice side  effects, so it is now a feature, and with the RP06 --
> so Dennis's new  scheme:
>
> struct size
> {
>
> daddr_t nblocks;
> int cyloff;
>
>
> } hp_sizes[8] =
>
> {
> 9614, 0, /* cyl 0 thru 22 */
> 8778, 23, /* cyl 23 thru 43 */
> 0, 0,
> 0, 0,
> 161348, 44, /* cyl 44 thru 429 */
> 160930, 430, /* cyl 430 thru 814 */
> 153406, 44, /* cyl 44 thru 410 (rp04, rp05) */
> 322278, 44, /* cyl 44 thru 814 (rp06) */
>
> };
>
> At this point, as Arnold said, people would change the size/shape of the
> tables locally by recompiling and setting up the disk they way you wanted.
> It could be awkward, you typically needed a second disk pack, *etc*.
>
> With DEC's release of V7m, 2.x (2.1, I think), Fred Cantor of DEC put the
> first versions of labels into what would later become Ultrix-11.   That
> code was released to CRSG (it is also where 2.9 et al. get Fred's wonderful
> swapping support).   Shannon (still at DEC) helped Sam add some of that
> into a 4.1a/b/c stream...  This was where Sam added things like newfs(8)
> and /etc/disktab and started to try to make BSD more resilient to
> different types of disks.   But my memory is that it was nascent in the 4.2
> release, so you had /etc/disktab and some of the tools but not everything
> until later, 4.3, when what you know today came into being.
>

Except I can't find any of this in our V7m or ultrix-11 that we have. This
is 2.1. And there all the device drivers still have the fixed tables:
grep sizes * | grep '\[[0-9]'
hk.c:} hk_sizes[8] =
hk.c_i:} hk_sizes[8] =
hk.c_id:} hk_sizes[8] =
hm.c:} hm_sizes[24] =
hp.c:} hp_sizes[24] =

and the ultrix-11 3.1 sources we have are similar, but move all of the
above into /usr/sys/conf/dksizes.c.

The early disktab stuff encoded the default tables into a text file to make
it easier to cope with all the different types of disks, but didn't affect
what we know as bsdlabels until after 4.3BSD. The naming convention was
there, but the 'write bits to the disk to describe partitioning' I couldn't
find at all in ultrix. I think you may be confusing that stuff (which was a
very clever way to cope with all these disks that have different
partitioning in a increasingly generic way) with disk labeling which I
think. Sun was first to market with.

Maybe you can help me find it?

I did find the SunOS 0.x images (UniSoft V7 port) didn't appear have a disk
partition, but 1.0 (post 4BSD pre 4.1BSD) and later did. That puts it at
1982. Although based off BSD, the disk label stuff wasn't in BSD for
another 5 years when the BSD folks started the Tahoe port.

It's entirely possible one of the early UniSoft ports had this feature, and
that was adapted into SunOS and I've not seen the paper trail (studying the
early UniSoft ports is rather problematic for a variety of logistical
reasons)... Not least of which is the fragmentary nature of the surviving
source, the difficulty in obtaining boot images, etc. This is a very
under-documented time. It wouldn't surprise me if someone produced evidence
of Sun's partitioning scheme coming from this mess...  Or a very similar
thing. And Pixar may have been involved (though that's a memory I have that
I don't have in my notes documented). Maybe that's the Sam Leffler
connection (though I thought his time at Pixar post-dated this period
somewhat).

Meanwhile, in Pittsburgh ...
>
> CMU  is working on the Mach2.5 project.   Intel had donated some of the
> original 386 PC support to Rasid's group (they were partial funding) that
> they had developed with ISC/AT&T *et al* to handle things particular to
> the PC architecture -- such as the PC's fdisk(1) since PC's have a
> completely different way of partitioning things. Bob Baron did the Mach
> version - as to what was new and what was from Intel I'm not 100% sure.
> But CMU did have the tools that Intel felt they owned and were not part of
> System III/V.
>
> The key is that a PC's fdisk(1) - creates a UNIX partition within the DOS
> partition table so DOS knows to leave it alone, but then Unix
> sub-partitions using its own scheme.   My memory here is hazy, I think the
> sub-partition code Bob used is different from the ATT's 386 [which most
> people saw as ISC's 386/ix who did the original work under contract for
> AT&T]
>
> Meanwhile, across the pond ...
> Andy's team starts to want a V7 for the PC/XT and starts the Minux
> project.   I know they had CMU's Mach 2.5 stuff and there fdisk, while it
> seems to be independent (i.e. clean room) is very similar to Bob Baron's
> scheme.
>

Yea, the fdisk stuff came from early versions of PC-DOS to partition hard
drives in 2.0 / 1983. I do know the DEC Rainbow had yet another set of
partitioning (which the PRO and DecMATE didn't share). It never adopted
what today is the MBR partitioning scheme because it's hard disk support
pre-dates this. It quickly became standard, and we see evidence of support
for it in the Venix/86 sources that we have (both the XT version and the AT
version, though I think that was disk controller version, and not MBR data).


> Meanwhile, in Walnut Creek ...
>
> Jolitiz *et al.* started 386BSD, which would begat
> FreeBSD/NetBSD/OpenBSD.    At some point, Baron's fdisk (1) and support
> tools make their way to that project.    This was why all the original
> UNIX for a PC used the same PC fdisk "type" ID number (the ISC 386/ix type
> -- from the original 386 port). [Linus built his own tools, which I think
> started from the Minux tools, and thus he did his own thing --- so Linux
> disks were not marked with the same types). If you look at the sources to
> both Mach and early 386BSD, you'll see Intel markings as well as Bob's name
> in a couple of places [Truth in advertising here (different story) -- I
> wrote most of the original AT disk driver Bill used -- I was consulting for
> NCR and had access to the WD specs as well as the CMU code but I was not
> the conduit of the CMU code base to  Bill].
>
At some point, 386BSD started to go one way, and what would become FreeBSD
> started to ramp up.  Jordan Hubbard wants an "easy install" like Microsoft
> has for Win98, so very early FreeBSD has the first PC-based install that
> can run without much intervention.    At some point, Jordon or someone else
> in the  FreeBSD team introduced what they called slices, furthering the BSD
> scheme but designed to overlay the PC better.   The NetBSD folks had
> already split off, so they did not pick it up.  When OpenBSD split from
> NetBSD, they did something similar but different to FreeBSD's slices...
>

Yea, this was 1992 or 1993. In FreeBSD 1.x you just had BSD Labels which
would default in interesting ways to what the MBR of the disk was (I've not
double checked the code, this is my memory). The 'c' partition was the
entire BSD partition while the 'd' partition was the entire disk. And that
duality lived on for a while. The slice code came in during the 3.x series
(or after 2.0, which used the same scheme as 1.x). And it was mostly an
attempt to make the BSD labels independent of each other. There was some
weird code for bsd-label only disks but most of the visible changes from
slices came in device naming (if you had only one disk label, and it was on
the first partition it was compatible with old bsd labels by and large).
FreeBSD wouldn't have true nested MBR and bsd labels until 5.0 and geom
entered the tree... A lot of the 'differences' between the BSDs were in
presentation of the data to the user, rather than physical differences on
the disk (though such differences did exist, especially in endian). It was
a real mess of NIH + incompatible UI + almost-but-not-quite compatible
on-disk formats coupled with a bunch of bugs that differed among the BSDs.
I do not miss these days.


> Meanwhile, in Cupertino ....
>
> Next (ney Apple) started with the Mach code base from CMU.  There was a
> push in the Valley in those days for something called OpenFirmware [Warner
> help here -- I think that it was forth based IIRC, and Sun may have had
> their hand in it also].  But the key is that it ran on 68K's.   So Next's
> original partition scheme was influenced if not based on some of Bob's
> work, although they did not need the silly BIOS stuff, so they did make it
> all talk to OpenFW (BTW: I think that was still all Next)
>

I'm not sure about Next's partitioning scheme. I do know that Apple adopted
the OpenFirmware partitioning scheme for PPC Mac leading to the APM scheme
which  FreeBSD supports to this day for PowerPC. This was different from
Sun's sunlabels (which differed from BSD labels since they pre-dated them
by half a decade.

By the time the Intel Mac's BIOS had begun to be replaced in the WINTEL
> world by UEFI, Apple committed to using a flavor of it. My old lab partner,
> Guy Sotomayer (*a.k.a.* gss on many mailing lists) was mixed up during
> his time at Apple. He might be able to fill in some details if I ask him.
> I also know that Apple was not happy with how disks were handled with the
> original partition scheme, and by later release started over with their
> partitioning scheme(s) at least once.
>

Yea, GPT partitioning came along a lot later, kinda sort with EFI which
Intel and Apple did (intel for ia64, Apple for 32-bit x86 macs). It's one
of the few partitioning schemes I rather like.


> I've undoubtedly left stuff out, such as much of Linux - but this is all
> part of the UNIX war I was a player.
>

Yea, not to mention the HP, AIX, DG and other partitioning schemes that
grew up in the late 80s and early 90s to have parity with MBR disks at the
feature level, but rarely at the disk format level. A lot of it is
documented, but in hard to find places and I've not done a systematic
survey.

Warner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.tuhs.org/pipermail/tuhs/attachments/20231231/84734d9f/attachment-0001.htm>