Making the Tibetan language a first-class citizen in the digital world

306 pointsposted a year ago
by thunderbong

88 Comments

heydenberk

a year ago

Jim Woolsey, a hippie and early-ish computer hacker from New Hope, Pennsylvania, was an important and early force in the digitization of the Tibetan language. This interview[0] with him from 1993 is a fascinating time capsule, and interesting in its own right. He was a family friend and I always admired his singular commitment to this important and underappreciated work.

[0] https://www.mcall.com/1993/10/08/new-hope-man-computer-guru-...

zombot

a year ago

Too bad, that link only gives me "This content is not available in your region".

heydenberk

a year ago

If you doubt the computer’s influence has made its way into every walk of life, you haven’t met New Hope’s Jim Woolsey.

For a decade, Woolsey has worked with the Tibetan government-in-exile in Dharmshala, India, to put the Tibetan language on computer.

The free-lance computer whiz has compiled a source book of Tibetan literature and also has worked to create a Tibetan computer keyboard for the exiles from that ancient Asian kingdom. In the course of his work, he’s met the Dalai Lama, the spiritual leader of Tibet, and Aung San Suu Kyi, the Burmese political leader since placed under house arrest in that country.

Both leaders are recent Nobel Peace Prize winners, the Dalai Lama in 1989, Aung San Suu Kyi in 1991.

“If the Tibetan language isn’t put on computers — because of the fact that there are fewer than 1,000 Tibetan typewriters in the world and they’re more expensive than computers –the Tibetan language might not be saved from being put on the shelf with all those other dusty, musty languages of the scholars,” said Woolsey. “This is its only hope.”

Woolsey’s work with the Tibetan Buddhist government-in-exile in India began with an interest in Tibetan literature.

Formerly a technician with various rock’n’roll groups in the 1970s, he would read anything he could lay his hands on concerning Tibet, then enter the titles of the books in a bibliography he kept. He was traveling both for work and pleasure, and decided it was time to journey to one of the farthest corners of the globe.

“I booked a 120-day round-trip ticket to India,” he said. “I threw on my backpack and went to India. I was coming in from the airport in New Delhi at 3 o’clock in the morning and passed a camel pulling a cart down the street. I said, ‘We’re not in Kansas anymore.'”

He saw a sign for Tibet House in Dharmshala and decided to go there, even though he had no idea what Dharmshala was like or what awaited him there. His first trip to the Tibetan exiles’ home was a short one, and he later traveled to Kashmir, Darjeeling and Nepal.

By the time he returned to Dharmshala in 1983, the Tibetans knew him.

“Before I left, the Office of Tibet in New York asked me if they could have a copy of the notes that I had been keeping on my computer about Tibetan studies, mainly my reading list,” he explained. “I looked at it and it was a mess. I thought I had better clean it up. I wrote a couple computer programs to make it an organized matter. I printed it up and gave them a copy.”

A friend who was learning word processing wanted a copy and Woolsey also gave him one.

“He sent a copy to the Dalai Lama,” said Woolsey. “The Dalai Lama must have figured it was going to be published, so he wrote a forward to it.

“By the time I got back to the library in Dharmshala, I didn’t know anything about this. I got to the Western reference section and they said to me, ‘Oh, we’ve been wanting to meet you.'”

They asked Woolsey at the library what his background was and he answered rock’n’roll. They asked who his teacher was and he told them he didn’t have one. They asked if he was a Tibetan Buddhist and he said no. They asked if he wanted to become one, and he again answered no.

“I was raised a Quaker, and that was close enough,” he said. “They meditate, but they don’t call it that.”

At the Tibetans’ request, Woolsey settled down to begin work organizing by computer the chaos that was the Dharmshala Library.

Realizing the power of the age of information had fallen into their laps, the Tibetans decided he was to be their computer guru, and designated him as such.

They told him he could consider them his affiliation in the academic world.

The chaos inflicted on the Tibetans by the Chinese invasion of the late 1950s had not yet been alleviated. Books and manuscripts lay in unsorted piles in the library, so Woolsey’s computer was the perfect tool to help put things in order.

“Later on, in 1984, they sent me a list of letters to all the high lamas in the United States, from the director of the library, telling them that I was their computer guy, and would they please aid and abet me in my endeavors,” said Woolsey. “Of course, they sent them the letters before they sent me one asking me if I wanted to do it, which makes it a little strange.”

Woolsey returned to India several times at the invitation of the Tibetans. He had discovered in the United States that no one was working on computerizing the Tibetan language with much interest.

By 1985, he was acting as the consultant to the library in developing the language on computer.

Once he was given the assignment, he was besieged with students, one of whom was the abbot of the Mahayana Buddhist Temple in St. Petersburg, Russia. Tenzing Samaev was visiting Dharmshala in 1990, and Woolsey had arrived just after Tibetan New Year.

“This Tibetan brought over a monk and said ‘He wants to know something about computers,'” said Woolsey. “I said, ‘OK.’ I answered his question.”

The monk returned after the New Year with two more questions, and then the next day with two more, and then two more the next morning, and two more by that noon.

This went on for four days.

“Tenzing, in the true Tibetan tradition, formally presented himself and requested me to become his teacher, to accept him as a student,” said Woolsey.

The abbot came to this country in 1991 and went home with a computer and laser printer. With Woolsey’s help he set up the computer to work in Cyrillic, the alphabet used in Russia, and is now publishing the temple’s newsletters and other proclamations on it.

Norbu Chompel, director of book sales for the Office of Tibet in New York City, said, “Jim has done quite a lot. He’s the main person responsible for introducing computers to the Tibetan administration … He came with a lap-top and talked computers to several staff members. That’s how computers came.

“Before, we used to use typewriters,” Chompel said. “He taught computers, and then everybody got into buying computers.” With all the work he was doing for the Tibetans becoming known, perhaps it was inevitable that the Dalai Lama take more notice of him.

The Tibetan spiritual leader wanted to know what was going on with the development of Tibetan on computer, Woolsey said.

“A couple years ago, I was given the opportunity to brief the Dalai Lama about what’s going on,” said Woolsey. “We had some interesting conversation, but I feel that he’s got better things to do.”

The Dalai Lama had the pursuits of freeing his country from Chinese domination and leading his people in their exile as more pressing problems.

Aung San Suu Kyi also had more important pursuits to consider.

Woolsey met her in Dharmshala, prior to her house arrest in Burma (now officially called the Union of Myanmar) as a political dissident.

She’s been detained by the Burmese government for the last four years because of her political activities and her great personal power. Her father, Aung San, founded modern Burma and was assassinated in 1947.

She has followed in his footsteps in an attempt to free her people from military rule.

Woolsey recounted an incident at a rally in which Aung San Suu Kyi prevented a slaughter by the army of an unarmed crowd of 20,000 people. The army approached to smash the rally, Aung San Suu Kyi positioned herself between the crowd and the soldiers and halted the military with her words.

“She told 20,000 people to sit down and be quiet and they sat down and were quiet,” said Woolsey. “She out-positioned the army and did it non-violently. That’s the key, non-violence.

“She’s a very, very learned person,” continued Woolsey. “She really has the rights of her people in her heart more than worries about herself.”

As is the case with Woolsey and the Tibetans.

Woolsey’s source book of Tibetan literature is under consideration for Internet, the international computer-user network.

With his help, Tibetan might go from being an endangered language to one available to everyone who can hook up a computer to a phone line.

And that might bring an ancient kingdom into today’s electronic age.

“I feel that you should be able to leapfrog over the industrial age into the information age as an agricultural society, and perhaps be farther ahead than where we in the West are trying to get to,” said Woolsey.

Push a few computer keys and it might happen.

Originally Published: October 8, 1993 at 4:00 a.m.

skybrian

a year ago

Apparently "documents have reasonably short paragraphs" should be added to "falsehoods programmers believe about text."

khaled

a year ago

In some countries, legal documents are required to not have any paragraph breaks, so you can have a document with one paragraph spanning 100s of pages. OpenOffice has a hard limit of 65534 per paragraph, and it took LibreOffice quite some work to left it: https://bugs.documentfoundation.org/show_bug.cgi?id=30668

briandear

a year ago

Why? Sounds ridiculous — intentionally making documents hard to understand in order to subsidize the administrative class.

pbronez

a year ago

I never thought about this element of cross language structure before. Text direction, diacritics, punctuation, sure - but I always assumed that chunking was universal. Turns out no:

“the typographical notion of the paragraph does not really exist in a Tibetan text the way it does in European languages. As a result, Tibetan texts often need to be processed as a long stream of uninterrupted text with no forced line breaks, sometimes over hundreds or thousands of pages. “

crazygringo

a year ago

Tens of pages, sure.

But hundreds? Thousands?

Do they not have the concepts of headers? Sections? Chapters?

Both in non-fiction and fiction, there are a lot more means of content separation than just paragraphs.

jdub

a year ago

Count yourself lucky (by mere hundreds of years) you have spaces.

tokai

a year ago

Paragraphs in the west as, indentation or separation of text, are not even a thousand years old. In the ancient world a paragraph was denoted with a typographical mark.

AlienRobot

a year ago

Somewhere, a programmer created a 4096 character buffer and sought the next '\n' only to be defeated by Tibetan.

wslh

a year ago

With all due respect, the innovation side of Tibetans is also appreciated in "The Nine Billion Names of God" [1].

[1] <https://en.wikipedia.org/wiki/The_Nine_Billion_Names_of_God>

asimovfan

a year ago

i don't know how it is phrased in the book itself but in Tibetan Buddhism there is no god. And their innovation is far beyond this book (at least the plot summary on wikipedia).

benterix

a year ago

Although there is no God, there are many gods (lha, sometimes translated as deity).

To add to the confusion, the same word is used for the so-called mundane gods like elementals and supramundane gods, i.e. beings who transcended subject-object dychotomy and can manifest also under a form of a god, whatever it may be. An inspiration for Clarke could be the famous Mañjuśrīnāmasamgīti or "Chanting the names of Manjushri".

To add even more confusion, if instead of gods we consider God and identify that being with characteristics such as omnipresence, all-pervasiveness, being beyond ordinary mind and so on - then one could attempt do identify it with the central topic of all Tibetan Buddhist tradition that exists under various names (primordial wisdom, the union of appearance and emptiness, self-existing wisdom and so on).

wslh

a year ago

If I were a Tibetan Buddhist, I might say we were just having some fun with Arthur C. Clarke's imagination.

Reubachi

a year ago

"asimovfan", would you say the same type of criticism of Asimov's injection of the Cristian God in "the Last question"?

In this story from Clarke, the sentiment you just portrayed is acknowledged, and the point of the whole story.

The two westerners cannot understand not only the grand project being undertaken by the monks, but also what "god" even is to them. Before they can really understand, their project is completed and reality changes.

stahorn

a year ago

"... relatively short paragraphs (possibly up to a few pages)"

I love things like this that just shows me how much I view the world from a certain perspective. I don't think I've ever had a paragraph even on one page! The closest I know is that some writer, that I forgot the name of, had several pages of stream of consciousness that I think was without paragraphs and punctuations.

cnity

a year ago

Modernist and post-modernist writers are known for this (James Joyce and David Foster Wallace, for example).

ron_k

a year ago

But also Saramago, Gadda, García Márquez, or Victor Hugo and his 800+ words sentence. Stream of consciousness usually has long paragraphs, but it’s quite common in other genres (I feel like genres is not the right word for it. Techniques?). Popular fiction is the only one that usually avoids long sentences/paragraphs/chapters.

hyperhello

a year ago

This has been in the works for a while. There is an old HyperCard stack to teach Tibetan pronunciation (with 16bit sound) you can try: https://hcsimulator.com/Learn-Tibetan

fsckboy

a year ago

the only vowel is AH ?

shanekandy

a year ago

In text, the singular vowels are built on the ah syllable with modifying marks.

cosignal

a year ago

The site seems incomplete. Tibetan does have 5 vowels, and it looks like the non intrinsic vowels are written at the bottom section of the view, but I can't get them to work. I assume the intention would be that you click one of the other vowels to toggle it, but it no worky.

zokier

a year ago

It's bit surprising that word processors would struggle with long paragraphs considering that various stream-of-consciousness and related styles of writing also eschew paragraphs and possibly other conventional structures. They might not be super common, but not exactly unheard of either. I'd assume writers and publishers manage them somehow.

Random recent example:

https://en.wikipedia.org/wiki/Ducks,_Newburyport

hoseja

a year ago

AFAIU there are no spaces either.

java-man

a year ago

I want to know the details how they achieved it (the support for super-long paragraphs, or rather, the absence thereof).

Does anyone know?

l1n

a year ago

https://gerrit.libreoffice.org/c/core/+/172801

Pretty short change for reducing O(n^2) impact with a cache.

This change includes the following scalability improvements for documents containing extremely large paragraphs:

- Reduces the size of layout contexts to account for LF control chars.

- Due to typical access patterns while laying out paragraphs, VCL was making O(n^2) calls to vcl::ScriptRun::next(). VCL now uses an existing global LRU cache for script runs, avoiding much of this overhead.

lappet

a year ago

I know Bengali and Assamese use a Tibetan script, anyone know how similar is it to the one Tibetans use for their language?

abe94

a year ago

While the scripts (Bengali/Assamese and Tibetan) both evolved from the Gupta script, the actual languages are very different. Bengali and Assamese are Indo-Aryan, while Tibetan is from a completely different language family (Sino-Tibetan).

When I (bengali speaker) visited Bhutan where they speak a language that is 50% mutually intelligble with Tibetan I didn't understand anything. I was surprised because I thought they might use a number of buddhist loan words, but even the words for dharma, karma, etc. sound completely different in tibetan

InDubioProRubio

a year ago

Languages, are very interesting beasts. As in, they are easy to learn and marshal communication across large swaths of the world- or they are hard to master and allow to construct very complex constructs and ideas- which are then transported from one speaker to another. In which part of the field does the Tibetan language fall?

somat

a year ago

I am not sure the two are mutually exclusive.

I mean, a highly technical subject will have highly technical jargon. but I am not convinced that the amount of subtle nuance in the language has a relationship to the complexity of ideas you can express in that language.

einpoklum

a year ago

Hey everyone, I'm Eyal, a LibreOffice project volunteer who does a lot of QA regarding Right-to-Left and Complex-Text-Layout scripts (= written languages). I want to thank thunderbong3 for posting a link to that post - and heartily thank Jonathan Clark, the new RTL-CTL-CJK-focused developer at The Document Foundation, who implemented the performance improvement for Tibetan.

Most bugs we encounter and report in LibreOffice are more general, and aren't script specific (e.g. code which forgets that the content may be right-to-left resulting in wrong behavior in those cases); and a lot of the script-specific bugs are about the most popular script, which is Arabic (that is also used for Farsi, Urdu, Javanese etc.)

But we do have some issues regarding less-commonly-used scripts, like Tibetan or Mongolian. Here:

https://bugs.documentfoundation.org/show_bug.cgi?id=115607

is the meta-bug which tracks issues with: Mongolian, Tibetan, Uyghur, Zhuang,Kazak, Xibo, Dai, Yi, Miao, Jingpo, Lisu, Lahu, Wa, etc.

We don't know if there are really very few issues specific to those languages (which is quite possible), or whether it's just that they're not used so much and the users aren't motivated enough to file bugs.

Still, as Jonathan's recent fix demonstrates, there is certainly the interest to address them when developer-time-resources become available.

I would like to encourage everyone who cares about these scripts, and "document editing fairness" across countries and cultures, to consider:

1. Try using LibreOffice with such languages which you know at least a little bit of - and if you find any bugs, file them at our BugZilla: https://bugs.documentfoundation.org/

2. Consider supproting The Document Foundation, which manages the LibreOffice project, financially:

https://www.libreoffice.org/donate/

We are one of the larger FOSS projects in the world, with tens of Millions of regular users (if not > 100 Million) and a board of trustees with members from dozens of countries; but - we don't have large corporations investing money nor time in the project. While a few commercial companies do contribute to LibreOffice (like Collabora and Allotropia) - many fundamental issues are not close enough to their customers' needs - which is why it was decided to hire Jonathan directly to give RTL-CTL-CJK support a boost. Individual user donations are what enables this work.

mmooss

a year ago

Hi Eyal - Your hard work as a volunteer does so much for so many - look at that blog post, for example. I really admire it.

buovjaga

a year ago

The bug report about long paragraphs and the blog post are by Élie Roux, the CTO of BDRC :)

whereistimbo

a year ago

I would appreciate you if you supported Dzongkha as well!

einpoklum

a year ago

Well, the "minimal" support is there, as buovjaga noted - but... we need you to tell us what aspects of that support is missing - by filing bugs, or at the very least talking to us about this (for example - there are "LibreOffice RTL" and a "LibreOffice CJK" groups on Telegram).

xmly

a year ago

There are over 50 tibetic languages, which one do you choose?

mihaic

a year ago

Honestly, given the particularities of Tibetan, I'm surprised it didn't adapt to the digital world. So while I'd congratulate the LibreOffice developers for improving their software, as least in displaying legacy text, I would expect Tibetan to evolve just like almost any writing system has done over time, and introduce some spaces and paragraphs.

emilamlom

a year ago

An actively used written script is not "legacy text". We're not talking about some ancient, dead language, but even if it was that, there's merit in being able to accurately display it digitally just from an academic standpoint of making historical research easier.

mihaic

a year ago

Sure, I understand the merit of it being displayed properly, that's something worthwhile.

But at the same time, if it's an actively used script, don't you think current users should consider changing its conventions?

My point was that all writing systems change, especially when the medium changes. Modern punctuation was invented at some point simply because the previous form of writing words one glued to the other was not ideal.

nottorp

a year ago

Frankly when I clicked I expected to read that the Tibetan language has been recently added to Unicode.

I mean, they're too busy adding emojis over there to work on support for human scripts any more...

TorKlingberg

a year ago

Tibetan script has been in Unicode since version 2 from 1996, with some characters added in later versions. Is there are particular human script you want added to Unicode?

nottorp

a year ago

I don't know, I want them to self disband if they're done instead of adding emojis.

mmooss

a year ago

> I mean, they're too busy adding emojis over there to work on support for human scripts any more...

What makes you say that there are human scripts being left out, or which nobody is working on (or that the work is displaced by emoji support)?

One challenge of large projects is that what's essential for some users is not even within the experience of other users. Sometimes humans make the fundemental error of thinking the range of their own experience defines the range of everyone's experiences, when the truth is that each of us sees only a tiny portion of an enormous canvas.

soheil

a year ago

[flagged]

vinay427

a year ago

This is a well-established phrase in computer science and programming languages, and it’s likely that its use here is meant to be evocative of those principles rather than of an anthropomorphic sense.

https://en.wikipedia.org/wiki/First-class_citizen

soheil

a year ago

Well let's not use it anymore if I agree on principle, computer science itself is a new field, so not sure how deeply entrenched it really is.

nottorp

a year ago

I suppose even if ... then ... is political to you?

How about { } ? I'm sure even that has a deep political meaning!

user

a year ago

[deleted]

blackeyeblitzar

a year ago

Tibet as a land and people should be made a first class citizen of the physical and political world as well. I am not sure why the Free Tibet movement died out in America and Europe, but less than 75 years ago, China forcibly and illegally took over Tibet and since then has been engaged in a forced reeducation and erasure of the people and their culture (Sinicization):

https://en.wikipedia.org/wiki/Annexation_of_Tibet_by_the_Peo...

https://en.wikipedia.org/wiki/Sinicization_of_Tibet

This is one of the great injustices of the world, and unfortunately the Tibetan leader (the real Dalai Lama) had to flee and live in exile in India.

EA-3167

a year ago

> I am not sure why the Free Tibet movement died out in America and Europe

Realism. Movements without a realistic goal tend to become quite niche, unless there's another hook to keep them going. With the rise of China as an economic and nuclear power, and one with no interest in even talking about Tibet as something other than a part of their empire, a populist movement to pressure politicians to "Free Tibet" is about as useful as a populist movement to get politicians to conquer the concept of entropy.

People, even very passionate and politically active people, moved on to causes with at least some hope, or at least the perception of hope.

g-b-r

a year ago

What? From my recollection, when it was decided to admit China in the WTO, every problem was set aside for the asserted belief that growth would result in democracy, human rights and hence the solution of those problems.

By the time it became clear that things wouldn't have gone that way, the west had bound itself to China too much to be able to do something about Tibet (without reshoring the manufacturing).

justin66

a year ago

> I am not sure why the Free Tibet movement died out in America and Europe

The US could have conditioned most favored nation trade status for China on improving its behavior in Tibet, could have kept them out of the WTO, etc. That would have cost the wrong people money.

Tainnor

a year ago

I'm inclined to agree with you politically (with what little I know about this conflict which admittedly is... almost nothing), but I don't think this kind of political discourse is very appropriate for HN. You can already see from some of the replies to your comment how the quality of the discussion deteriorates.

blackeyeblitzar

a year ago

I get what you’re saying but I shared it because the motives are the same. Why does Tibetan language matter at all? This BDRC article about digital representation of the language is tied back to that motivation. And the language is under threat due to Sinicization (the program of brainwashing Tibet so it looks like the rest of China). In other words, this discussions around a digital effort is tied to a greater effort to preserve Tibet (land, people, culture).

modernpink

a year ago

[flagged]

mtalantikite

a year ago

Just last weekend in Manhattan I saw a group of young people protesting for a free Tibet along with signs in support of Palestine. It was small, but as someone in their 40s who saw a lot of protests calling for a free Tibet in the 90s, it was really nice to see.

g-b-r

a year ago

How is it much greater?

Greater, maybe yes, since more decades passed since the Palestinians expulsion.

Or maybe not, because Tibetans at least can still live there?

Either way, I don't see how can it be much greater

blackeyeblitzar

a year ago

It’s also weird that Xinjiang and the Uyghur genocide get almost no commentary from the Arab Islamic world, even though the scale of violence against Muslims is far greater than the Israel-Arab conflict, by several orders of magnitude. To me it looks a lot like antisemitism - the smaller conflict matters only because it involves Jews.

debit-freak

a year ago

[flagged]

paulryanrogers

a year ago

Is anyone suggesting first people's in North America deserve less consideration than Tibet?

I've heard concerns about both Tibet and treatment of first peoples in the US (historical and at present).

hash872

a year ago

As OP notes, the 'Free Tibet' cause was very popular in the 90s when the US was not seen as being in a great-power conflict with China. So I don't think your 2nd sentence is correct. Your 1st sentence is just classic whataboutism

Retric

a year ago

There’s an expiration date on being able to do anything on this kind of conquest and forceful relocations.

I’m personally more concerned with China’s genocide of the Uyghur because you can’t undo such atrocities after a population is gone as happened to many Native American populations. There’s a few relatively tiny “Native American” populations that are still genetically distinct, but a wide range of cultures have been effectively erased with only fragments remaining. Hundreds of dead languages we don’t even know enough about to classify let alone stories.

Imagine if everything from Spain to Japan was referenced as Eurasian culture and thrown into basically the same bucket.

blackeyeblitzar

a year ago

Native Americans have far more consideration than Tibetans do. They are free to live per their culture. They can practice their religion. They can use their language. They are afforded every right American citizens are. There are several hundred Indian reservations, totaling something like 2.5% of US soil.

This is by no means a justification for the genocide of native Americans or a way to say they’ve been made whole. But there is simply no fair comparison between their situation today (despite huge injustices in the past) and the situation of Tibetans.

ninetyninenine

a year ago

[flagged]

hackernewds

a year ago

A very loaded and weird take comparing tibetan culture to the worst denominator of cannibalism, scientology and brainwashing. We have much to learn from Eastern wisdom as we already find with TCM, meditation, Ayurveda and yoga to mention a few.

morkalork

a year ago

Well, hopefully your preferred beliefs and way of life stay in favor with the majority, otherwise it could get awkward. In the case they do not, perhaps you can reflect upon your comment here and reassure yourself that your persecution is not unjustified.

blackeyeblitzar

a year ago

China’s fight against religion and culture is about control. This is due to their modern support of Communism, which tries to eliminate every threat to its ideology (it cannot survive otherwise). That’s why texts on Communism and Socialism often call for the elimination of religion, family structures, etc. It’s not about fighting a “cult” for some virtuous reason. If anything, authoritarianism and Communism are cults, and the Chinese version was visible clearly in the cultural revolution (https://en.wikipedia.org/wiki/Four_Olds). I’d also note that the absence of religion isn’t cult-free, and there are many cultish movements in the world even from non-religious parts of the world.

From the Chinese perspective pre-Communism, there would be no issue with Tibetan culture. But even if there were, it isn’t for them to care about or interfere with right?

As for your other comparisons. I don’t see cannibalism or Islamic fundamentalism (Jihad) as being comparable to Tibetan culture. Islam has explicit calls for violence against other belief systems. Tibetan culture does not. I can see treating Islam as a threat because of what it states openly, but it would be weird to group that with Tibetan culture, whether before or after Buddhism.