23 Apr
2018
23 Apr
'18
11:44 p.m.
On 2018-04-24 01:30, Grant Taylor <gtaylor(a)tnetconsulting.net> wrote:
On 04/23/2018 04:15 PM, Warner Losh wrote:
That must have been somewhere in the middle of history in that case. Old
(proper) drives had/have track 0 at the outer edge. The disk loaded the
heads after spin up, and that was at the outer edge, and then you just
locked on to track 0, which should be near.
Heads had to be retracted for the disk pack to be replaced.
But this whole optimization for swap based on transfer speeds makes no
sense to me. The dominating factor in spinning rust is seek times, and
not transfer speed. If you place the swap at one end of the disk, it
won't matter much that transfers will be faster, as seek times will on
average be much longer, and that will eat up any transfer gain ten times
over before even thinking. (Unless all your disk ever does is swapping,
at which time the heads can stay around the swapping area all the time.)
Which is also why the file system for RSX (ODS-1) placed the index file
(equivalent of the inode table) at the middle of the disk by default.
Not sure if Unix did that optimization, but I would hope so. (Never dug
into that part of the code.)
Johnny
--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: bqt(a)softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol
It's weird. These days lower LBAs perform
better on spinning drives.
We're seeing about 1.5x better performance on the first 30% of a drive
than on the last 30%, at least for read speeds for video streaming....
I think
manufacturers have switched things around on us. I'm used to
higher LBA numbers being on the outside of the disk. But I've seen
anecdotal indicators that the opposite is now true. 
23 Apr
23 Apr
11:57 p.m.
New subject: /dev/drum
On Tue, 24 Apr 2018, Johnny Billquist wrote:
Which is also why the file system for RSX (ODS-1)
placed the index file
(equivalent of the inode table) at the middle of the disk by default.
Not sure if Unix did that optimization, but I would hope so. (Never dug into
that part of the code.)
That reminds me of the Apple ][, whose original DOS put the directory and
bitmap table on track 17 (0-indexed) on a 35-track floppy disk.
-uso.
24 Apr
24 Apr
12:24 a.m.
New subject: /dev/drum
It also makes no sense because disks of the day were constant angular density (unlike CDs
for example, which are constant linear). There’s no different in transfer rate anywhere
on the disk. Each track has the same number of sectors.
We spent a lot of time in those days with “elevator” algorithms and clustering inodes to
try to minimize seek time. The other thing that was done on the fancier devices (not
often found on PDP-11s) was optimizing where you were in the rotational angle.
The original UNIX filesystems were dumb. It was <boot
block><superblock><inodes><datablocks>. Wasn’t until later things
like the Berkeley file systems that things started to get more clever in layout.
12:25 a.m.
New subject: /dev/drum
On Tue, Apr 24, 2018 at 01:44:06AM +0200, Johnny Billquist wrote:
Which is also why the file system for RSX (ODS-1)
placed the index
file (equivalent of the inode table) at the middle of the disk by
default.
Not sure if Unix did that optimization, but I would hope so. (Never
dug into that part of the code.)
Boston Children's Museum RK05 driver for 6th Ed springs to mind!
Cheers, Warren
12:27 a.m.
New subject: i-nodes in middle of disk
On Tue, Apr 24, 2018 at 10:25:17AM +1000, Warren Toomey wrote:
On Tue, Apr 24, 2018 at 01:44:06AM +0200, Johnny
Billquist wrote:
See the blurb for the UNSW 01 image here:
http://www.tuhs.org/Archive/Distributions/UNSW
UNSW 01
-------
Tape label: System Source Disk
DD format URK? BS=24B count=203 800bpi 9track
UNIX System Source 1 of 1
25/1/78
A distribution of UNIX source from UNSW, with several changes. record0.gz is
an RK05 image laid out according to the `Boston Children's Museum' format
(i-nodes in the middle). Latest file timestamp is Jan 24 1978. There is only
kernel source, plus a `unswbatch' directory. The latter seems to hold the
source to a UNIX batch system developed by Ian Johnstone and other at the
School of Electrical Engineering at UNSW.
record0.tar.gz is a tar archive of the RK05 image.
Cheers, Warren
Which is also why the file system for RSX (ODS-1)
placed the index
file (equivalent of the inode table) at the middle of the disk by
default.
Not sure if Unix did that optimization, but I would hope so. (Never
dug into that part of the code.)
Boston Children's Museum RK05 driver for 6th Ed springs to mind! 
10:19 a.m.
New subject: i-nodes in middle of disk
On Tue, 24 Apr 2018, Warren Toomey wrote:
UNSW 01
-------
Tape label: System Source Disk
DD format URK? BS=24B count=203 800bpi 9track
UNIX System Source 1 of 1
25/1/78
A distribution of UNIX source from UNSW, with several changes. record0.gz is
an RK05 image laid out according to the `Boston Children's Museum' format
(i-nodes in the middle). Latest file timestamp is Jan 24 1978. There is only
kernel source, plus a `unswbatch' directory. The latter seems to hold the
source to a UNIX batch system developed by Ian Johnstone and other at the
School of Electrical Engineering at UNSW.
Odd; I could've sworn that "URK" was un-rotated i.e. traditional format;
are you sure about that?
And "unswbatch"... Ahhh... I must take a look at that distribution some
time, to see whether it has my fingerprints on it; Kevin Hill and I
totally rewrote the system, throwing out IanJ's rubbish, with him doing
the application stuff ("submit" etc[*]) and me doing the driver. After
that, it actually worked (once I'd figured out a nasty bug in KRONOS'
UT-200 driver, that led to a POLL/REJECT loop).
[*]
I did a memorable hack to "submit" once; you see, we had an old VT-05
in the fishbowl, facing outwards for the sheeple, and it displayed the
batch queue (by title) in real time. Well, me being me, I hacked up the
aforesaid "submit" command to take multiple arbitrary files as input with
a specified title, so for a while it displayed "xxx... LLAMAS ARE
BIGGER THAN FROGS" (job names were 8 characters, and I have no idea
how that will display in people's MUAs)..
(I *think* I was actually working for them at that time.)
--
Dave Horsfall DTM (VK2KFU) "Those who don't understand security will
suffer."
12:31 a.m.
New subject: /dev/drum
On Tue, 24 Apr 2018, Warren Toomey wrote:
Yep, with the inodes in the middle of the disk! It also helped that we
(UNSW) put the superblock in the middle of said slice...
--
Dave Horsfall BSc DTM (VK2KFU) -- FuglySoft -- Gosford IT -- Unix/C/Perl (AbW)
Not sure if
Unix did that optimization, but I would hope so. (Never dug
into that part of the code.)
Boston Children's Museum RK05 driver for 6th Ed springs to mind! 
4:32 a.m.
New subject: /dev/drum
On 04/23/2018 05:44 PM, Johnny Billquist wrote:
But this whole optimization for swap based on transfer
speeds makes no
sense to me. The dominating factor in spinning rust is seek times, and
not transfer speed. If you place the swap at one end of the disk, it
won't matter much that transfers will be faster, as seek times will on
average be much longer, and that will eat up any transfer gain ten times
over before even thinking. (Unless all your disk ever does is swapping,
at which time the heads can stay around the swapping area all the time.)
I wonder if part of the (perceived?) performance gain was from the
likelihood that swap at one end of the drive meant that things could be
contiguous. Seek, lay down / pick up a large (or at least not small)
number of sectors, and seek back.
I had always assumed that the outer edge (what I thought was the end of
the disk) was faster than the inner edge (what I thought was the
beginning of the disk) because of geometry. However, as Ronald stated,
hard drives were constant angular density. Thus negating what I
originally thought about speed.
--
Grant. . . .
unix || die
4:49 a.m.
New subject: /dev/drum
On Mon, 23 Apr 2018 22:32:26 -0600 Grant Taylor via TUHS <tuhs(a)minnie.tuhs.org>
wrote:
I had always assumed that the outer edge (what I thought was the end of
the disk) was faster than the inner edge (what I thought was the
beginning of the disk) because of geometry. However, as Ronald stated,
hard drives were constant angular density. Thus negating what I
originally thought about speed.
Constant angular velocity means faster "linear" velocity for
tracks further away from the center. Since 1990 or so disk
tracks are divided up in 16 or so "zones", where outer zones
have more blocks per track. This translates to higher
throughput.
A modern Seagate Exos SAS disk may have a range of 279MB/s
(outermost) to 136MB/s (innermost) or 300MB/s to 210MB/s for
faster disks (15Krpm). Disk vendors don't seem to break this
range out for consumer drives. But you can measure it using
tools like diskinfo on FreeBSD. For example:
# diskinfo -t /dev/ada4 # this is an 5 year old 1TB WD "Black" disk.
/dev/ada4
...
Not_Zoned # Zone Mode <<== this seems wrong.
...
Transfer rates:
outside: 102400 kbytes in 0.972176 sec = 105331 kbytes/sec
middle: 102400 kbytes in 1.088977 sec = 94033 kbytes/sec
inside: 102400 kbytes in 1.804460 sec = 56748 kbytes/sec
4:59 a.m.
New subject: /dev/drum
On Mon, Apr 23, 2018 at 10:49 PM, Bakul Shah <bakul(a)bitblocks.com> wrote:
On Mon, 23 Apr 2018 22:32:26 -0600 Grant Taylor via
TUHS <
tuhs(a)minnie.tuhs.org> wrote:
That's right. This is for BIO_ZONE stuff, which has to do with host managed
and host aware SMR drive zones. That's different than the zones you are
talking about.
I had always assumed that the outer edge (what I thought was the end of
the disk) was faster than the inner edge (what I thought was the
beginning of the disk) because of geometry. However, as Ronald stated,
hard drives were constant angular density. Thus negating what I
originally thought about speed.
Constant angular velocity means faster "linear" velocity for
tracks further away from the center. Since 1990 or so disk
tracks are divided up in 16 or so "zones", where outer zones
have more blocks per track. This translates to higher
throughput.
A modern Seagate Exos SAS disk may have a range of 279MB/s
(outermost) to 136MB/s (innermost) or 300MB/s to 210MB/s for
faster disks (15Krpm). Disk vendors don't seem to break this
range out for consumer drives. But you can measure it using
tools like diskinfo on FreeBSD. For example:
# diskinfo -t /dev/ada4 # this is an 5 year old 1TB WD "Black" disk.
/dev/ada4
...
Not_Zoned # Zone Mode <<== this seems wrong.
...
Transfer rates:
outside: 102400 kbytes in 0.972176 sec = 105331
kbytes/sec
middle: 102400 kbytes in 1.088977 sec = 94033
kbytes/sec
inside: 102400 kbytes in 1.804460 sec = 56748
kbytes/sec
Yes. This matches our experience where we get 1.5x better on the low LBAs
than the high LBAs. We're looking to 'short stroke' the drive to the first
part of it to get better performance... Toss a filesystem on top of it, and
have a more random workload and it's down to about 30% better than using
the whole drive....
Warner
6:22 a.m.
New subject: /dev/drum
On Mon, 23 Apr 2018 22:59:19 -0600 Warner Losh <imp(a)bsdimp.com> wrote:
Ah. Thanks! Does host management of SMR zones provide better
throughput for sequential writes? Enough to make it worht it?
[I guess this may be something you guys may care about?]
Haven't had a chance to work on storage stuff for ages. [Last
I played with Ceph was 5 years ago and at a higher level than
disks.]
...
Not_Zoned # Zone Mode <<== this seems wrong.
That's right. This is for BIO_ZONE stuff, which has to do with host managed
and host aware SMR drive zones. That's different than the zones you are
talking about. Yes. This matches our experience where we get 1.5x
better on the low LBAs
than the high LBAs. We're looking to 'short stroke' the drive to the
first
part of it to get better performance... Toss a filesystem on top of it, and
have a more random workload and it's down to about 30% better than using
the whole drive....
Is the tradeoff worth it? Now you have choices like Sata vs
SAS vs SDD vs PCIe....
We've come a long way from /dev/drum :-)
2:57 p.m.
New subject: /dev/drum
On Tue, Apr 24, 2018 at 12:22 AM, Bakul Shah <bakul(a)bitblocks.com> wrote:
On Mon, 23 Apr 2018 22:59:19 -0600 Warner Losh
<imp(a)bsdimp.com> wrote:
managed
Right now, I don't think that we do anything in stock FreeBSD with the
zones. I've looked at trying to create some kind of FS that copes with the
large granularity writes blocks that host managed SMR drives would need to
do and changes we'd need to a write-in-place FS to take advantage of it.
It's possible, but it would turn UFS from a write-in-place system to a
write-in-place, but only for meta-data, and different free block allocation
methods. So far, it hasn't been enough of a win to be worth bothering with
for our application (eg, we can get 10-20% more storage, but that delta is
likely to remain constant, and the effort to make it happen is high enough
that the savings isn't there to pay for the development).
...
Not_Zoned # Zone Mode <<== this seems wrong.
That's right. This is for BIO_ZONE stuff, which has to do with host and host aware SMR drive zones. That's
different than the zones you are
talking about.
Ah. Thanks! Does host management of SMR zones provide better
throughput for sequential writes? Enough to make it worht it?
[I guess this may be something you guys may care about?]
Haven't had a chance to work on storage stuff for ages. [Last
I played with Ceph was 5 years ago and at a higher level than
disks.] Yes. This matches our experience where we get 1.5x
better on the low LBAs
We have a multi-tiered storage architecture. When you want to play a video
from our service, we see if any of the close, fast boxes has a copy we can
use. If they are too busy, we go back to slower, but more complete tiers.
The last tier is made up of machines with lots of spinning disks. Some
catalogs are small enough that only using 1/2 the drive, but getting 30%
better throughput is the right engineering decision since it would improve
network utilization w/o needing to deploy more servers. We use all those
technologies in the different tiers: our fastest 100G boxes are NVMe, the
40G boxes we have are JBOD of SSDs, the 10G storage boxes are spinning
rust. We use SATA for SSDs since the SAS SSDs are super pricy, but we use
SAS HDDs since we need the deeper queues and other features of SAS that are
absent from SATA. We also sometimes oversubscribe PCIe lanes to get better
storage density at a cheaper price point.
There's lots of tradeoffs that can be made....
Warner
than the high LBAs. We're looking to
'short stroke' the drive to the
first
part of it to get better performance... Toss a
filesystem on top of it,
and
have a more random workload and it's down to
about 30% better than using
the whole drive....
Is the tradeoff worth it? Now you have choices like Sata vs
SAS vs SDD vs PCIe....
6:46 a.m.
New subject: /dev/drum
Johnny Billquist wrote:
Which is also why the file system for RSX (ODS-1)
placed the index
file (equivalent of the inode table) at the middle of the disk by
default. Not sure if Unix did that optimization, but I would hope so.
I know of an operating system predating Unix which has that
optimization.
2193
days inactive
2194
days old
13 comments
10 participants
participants (10)
-
Bakul Shah -
Dave Horsfall -
Grant Taylor -
Johnny Billquist -
Lars Brinkhoff -
Lyndon Nerenberg -
Ronald Natalie -
Steve Nickolas -
Warner Losh -
Warren Toomey