[TUHS] the sin of buffering [offshoot of excise process from a pipeline]

[Larry McVoy]

What is being provided is a generic layering system on top of stdio.
Any sort of conversion you want.  Encryption, CRC, XOR block, compression
(we support gzip and lz4).  Those are just the layers we use right now,
it's easy to imagine others being added.

The point is that there is no one API that is going to pleasantly encode
all of the options to all of those layers and any that may come later.
Are you seriously suggesting that you want to read the freopen(3)
man page and see all of these options explained?  That's the classic
open source way, dump everything in one poorly thought out man page.
It's not the Unix way, people think about it harder.

For the record, I pushed for the single string encoding as well but got
pushed off it as I realized the API wasn't as simple as I imagined.
While you could do it that way you shouldn't do it that way, it's just
not a good API.

I'm very pleased with how it turned out in our code, other than a handful
of fpush() calls, it just looks like stock stdio.  

On Wed, Jul 16, 2014 at 10:56:57AM -0400, Dan Cross wrote:
> Why can't those be embedded in the relevant string?  freopen(fp, "rx{128}")
> or something?
> 
> 
> On Wed, Jul 16, 2014 at 10:30 AM, Larry McVoy <lm at mcvoy.com> wrote:
> 
> > On Wed, Jul 16, 2014 at 02:03:58AM -0400, John Cowan wrote:
> > > Larry McVoy scripsit:
> > >
> > > > We tried that but the problem is that you can't encode all the options
> > you
> > > > want in just a character.  Compression doesn't take options, the
> > CRC/XOR
> > > > layer wants to know how big you might think the file is (because we
> > > > support blocksizes from about 256B to 256K and we want to know the
> > > > file size to guess the block size).
> > >
> > > It's a string: you can have as many characters as you want.
> >
> > I understand your desire to have one API.  We tried and it just wasn't
> > practical.  Imagine pushing an encryption layer that wants a key,
> > XOR layer that wants block size, etc.
> > _______________________________________________
> > TUHS mailing list
> > TUHS at minnie.tuhs.org
> > https://minnie.tuhs.org/mailman/listinfo/tuhs
> >

-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm