[TUHS] scaling on TCP socket connections

[Larry McVoy]

On Fri, Mar 10, 2023 at 10:14:01AM +0000, Ralph Corderoy wrote:
> Hi Larry,
> 
> > SGI made TCP go very fast on 200Mhz MIPS processors.  The tricks were
> > to mark the page copy on write on output so the driver could use the
> > page without copying
> 
> I understand that bit.
> 
> > and to page flip on input.
> 
> Could you expand that to a sentence or two.

Sure.  If you are using something like Hippi where the data size is
at least as big as a page, the networking stack can arrange to send 
data that is a multiple of a page size.  When that data works its
way up the stack to where a process is waiting in a read(), if the
process has asked for a read of a size that is a multiple of a
page size, it can just diddle the page tables and take out the
pages that the process had and give them to the kernel and put
in the pages that the kernel had and give them to the process.
That seems like a lot but it's basically what happens in TLB
miss and that code is very small and fast.


As I was thinking about this there was another thing SGI did that
was new to me at the time.  Interrupt coalescing.  When a packet
came in and the driver pulled it off the wire, the driver could
look to see if there was going to be another interrupt because
another packet had arrived.  Then the driver grabbed that packet
as well, saving all the overhead of an interrupt.

MIPS cpus were not fast but SGI wall papered over that problem
by being very clever when they needed to be.