On 19-Mar-23 7:00, segaloco via TUHS wrote: I think the most significant development that came out of Unix regarding internationalization was the proposal and adoption of Unicode and UTF-8. This was published in 1993 in the USENIX Technical Conference proceedings: Pike, Rob, and Ken Thompson. "Hello World or Καλημέρα κόσμε or こんにちは世界." Proceedings of the Winter 1993 USENIX Conference. 1993. At the time of the decision to adopt Unicode and UTF-8 in Unix (Plan 9 actually) there was no consensus on international character representations and encodings. Many systems extended ASCII with 8 bit characters to represent those required in a particular country These "code pages" were standardized in numerous mutually incompatible ISO-8859-X variants. My understanding is the for many Asian (Chinese, Japanese, and Korean) languages the situation was even worse, with ISO 2022 being used to shift mid-string from one character set encoding to another. In addition, Unicode was a draft standard for unified 16-bit character codes promoted by a group US companies. It was battling against the ISO 10646 draft, which had taken the approach of allocating character set blocks to national bodies, thus creating a sparse 32-bit representation with considerable redundancy between similar languages. Furthermore, the ISO 10646 standard proposed a (non-required) UTF multibyte encoding (now known as UTF-1), which was not self-synchronized, because bytes used for representing ASCII characters were also employed as parts of multibyte sequences. The Bell Labs team took the bold approach of adopting the draft Unicode standard and an X-Open proposal for encoding multibyte characters only using bytes with the top bit set. At the time the encoding was known as UTF-2; it is what we now call UTF-8. UTF-8 makes it easier to achieve backward compatibility in existing code; for example code scanning for the "/" file path separation character in a string, will never encounter it in the UTF-8 representation of non-ASCII characters. The Plan 9 choices proved wise and prescient. I do not know how much the Plan 9 implementation and the USENIX paper influenced further developments (its authors may enlighten us), but in the end Unicode converged with ISO 10646 becoming a single standard, and UTF-8 was widely adopted. The Plan 9 team's decision to adopt UTF-8 was by no means a given. Consider the case of Microsoft, which released Windows NT with Unicode support in the same year. Microsoft's Windows NT 1993 offering supported a wide character encoding, not UTF-8: initially UCS-2 and later UTF-16. To achieve backward compatibility the Windows API offers two functions for each call involving strings: a so-called "ANSI" version (actually using the currently active code page) and a "Wide" (Unicode) version. Furthermore, text files use a byte order mark to inform programs regarding their character representation, and in C/C++ code strings are often enclosed in a special macro to facilitate porting to wide characters. In the end, in 2019 Microsoft yielded, supporting UTF-8 in its Windows API through code page 65001 (CP_UTF8), and recommending its use. The double APIs and BOM files are still with us as a reminder that deficient technical decisions come at a cost. Diomidis - https://www.spinellis.gr

Hi, I just got off the phone with my father, who was working at the CII, then Bull, in the eighties through to the early nineties. Here is what I learned: Bull ported UNIX to their own line of Mitra 125: - https://fr.wikipedia.org/wiki/Mitra_15 and later to the SPS-9 which is from what I understand a license-built ridge 32, somewhat comparable to a VAX : - https://www.lemonde.fr/archives/article/1984/11/21/bull-va-fabriquer-en-fra… - http://www.histoireinform.com/Histoire/+infos2/chr5infa.htm - https://retrocomputingforum.com/t/ridge-32-a-bitsliced-early-risc-graphical… These efforts included first a translation of all the manuals from english to french. The translated manuals sometimes still wore the AT&T logo. The strings associated with error codes, and such were also tranlated in the source. This proved insufficient and awkward, and then a real internationalization effort was spearheaded by Bull. Anecdote: the abbrevation i18n for "Internationalization" was coined by Pascal Beyls, his boss at the time. My father was the representative for Bull at X/Open. Internationalization was part of the normalization process. I have on my desk volume 3 of the X/Open portability guide, whose section 3 is entirely dedicated to internationalization. The document is dated december 1988 and therefore predates UTF8 (92 if I'm correct) and even unicode (but I was blowing my first birthday candle at the time so my memory of the events may be a bit fuzzy). The document state that 8 bits are enough for western european languages, and 16 bits should do for asian languages. It promotes the use of ISO-8859-1. It notes that UNIX is limited to 7-bit ASCII and is therefore as is unsuitable for internationalization. It promotes the LANG and LC_* env variables as an annoucement mechanism, and gives examples where the locale is set to french. It also explains the "Message catalogues" with stores "messages [...] separate from the logic of a program, to be translated into several languages, and to be retrieved at run-time according to the language requirements of each user". If you want to dig into a fun part of foreign UNIX history, my father mentions that along with the corresponding Bull hardware, UNIX was sold to the USSR, along with the translated (in russian) manuals. So somewhere in a gas field in siberia may exist a russian UNIX manual with the BULL and AT&T logo. They only had the binaries, not the source of the system. There were a few "homegrown unix-like" efforts in europe, and bull was part of a few of them. BULL then moved to Linux, that's another phone call to make. If you want to dig more into that, the CNAM (a school/museum in Paris, if you are there, come see the museum it is absolutely awesome) is documenting early UNIX history and did a conference a few years back about it. Clem Cole was there and spoke. Here are some links about their efforts: - https://technique-societe.cnam.fr/colloque-international-unix-en-europe-ent… - https://www.youtube.com/watch?v=-mCSFSF-i1A <- begins with Clem's talk, but the rest is worth watching as well. Hope that helps, Cheers, Edouard. segaloco via TUHS &lt;tuhs(a)tuhs.org&gt; writes: