[COFF] [TUHS] Re: Command Line Editing in the Terminal Driver (Was: A few comments on porting the Bourne shell)

[Adam Thornton]

(moving to COFF)

On Tue, Jan 3, 2023 at 9:55 AM Marshall Conover <marzhall.o at gmail.com>
wrote:

> Along these lines but not quite, Jupyter Notebooks have stood out to me as
> another approach to this concept, with behavior I'd like to see implemented
> in a shell.
>
>
Well, you know, if you're OK with bash:
https://github.com/takluyver/bash_kernel

Or zsh: https://pypi.org/project/zsh-jupyter-kernel/

One of the big things I do is work on the Notebook Aspect of the Rubin
Science Platform.  Each JupyterLab notebook session is tied to a "kernel"
which is a specific language-and-extension environment.  At the Rubin
Observatory, we support a Python 3.10 environment with our processing
pipeline included.  Although JupyterLab itself is capable of doing other
languages (notably Julia and R, which are the other two from which the name
"Jupyter" came), many others have been adapted to the notebook environment
(including at least the two shells above).  And while researchers are
welcome to write their own code and work with raw images, we work under the
presumption that almost everyone is going to use the software tools the
Rubin Observatory provides to work with the data it collects, because
writing your own data processing pipeline from scratch is a pretty
monumental project.

Most astronomers are perfectly happy with what we provide, which is Python
plus our processing pipelines, which are all Python from the
scientist-facing perspective (much of the pipeline code is implemented in
C++ for speed, but it then gets Python bindings, so unless you're actually
working very deep down in the image processing code, it's Python as far as
you're concerned).  However, a certain class of astronomers still loves
their FORTRAN.  This class, unfortunately, tends to be the older ones,
which means the ones who have societal stature, tenure, can relatively
easily get big grants, and wield a lot of power within their institutions.

I know that it is *possible* to run gfortran as a Jupyter kernel.  I've
seen it done.  I was impressed.

Fortunately, no one with the power to make it stick has demanded we provide
a kernel like that.  The initial provision of the service wouldn't be the
problem.  It's that the support would be a nightmare.  No one on my team is
great at FORTRAN; I'm probably the closest to fluent, and I'm not very, and
I really don't enjoy working in FORTRAN, and because FORTRAN really doesn't
lend itself easily to the kind of Python REPL exploration that notebooks
are all about, and because someone who loves FORTRAN and hates Python
probably has a very different concept of what good software engineering
practices look like than I do, trying to support someone working in a
notebook environment in a FORTRAN kernel would very likely be exquisitely
painful.  And fortunately, since there are not FORTRAN bindings to the C++
classes providing the core algorithms, much less FORTRAN bindings to the
Python implementations (because all the things that don't particularly need
to be fast are just written in Python in the first place), a gfortran
kernel wouldn't be nearly as useful as our Python-plus-our-tools.

Now, sure, if we had paying customers who were willing to throw enough
money at us to make it worth the pain and both bring a FORTRAN
implementation to feature-parity with the reference environment and make a
gfortran kernel available, then we would do it.  But I get paid a salary
that is not directly tied to my support burden, and I get to spend a lot
more of my time working on fun things and providing features for
astronomers who are not mired in 1978 if I can avoid spending my time
supporting huge time sinks that aren't in widespread use.  We are scoped to
provide Python.  We are not scoped to provide FORTRAN.  We are not making
money off of sales: we're employed to provide stable infrastructure
services so the scientists using our platform and observatory can get their
research done.  And thus far we've been successful in saying "hey, we've
got finite resources, we're not swimming in spare cycles, no, we can't
support [ x for x in
things-someone-wants-but-are-not-in-the-documented-scope ]".  (To be fair,
this has more often been things like additional visualization libraries
than whole new languages, but the principle is the same.)  We have a
process for proposing items for inclusion, and it's not at all rare that we
add them, but it's generally a considered decision about how generally
useful the proposed item will be for the project as a whole and how much
time it's likely to consume to support.

So this gave me a little satori about why I think POSIX.2 is a perfectly
reasonable spec to support and why I'm not wild about making all my shell
scripts instead compatible with the subset of v7 sh that works (almost)
everywhere.  It's not all that much more work up front, but odds are that a
customer that wants to run new software, but who can't guarantee a POSIX
/bin/sh, will be a much more costly customer to support than one who can,
just as someone who wants a notebook environment but insists on FORTRAN in
it is very likely going to be much harder to support than someone who's
happy with the Python environment we already supply.

Adam
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/coff/attachments/20230103/0c379bde/attachment.htm>