Happy Africa Day: Keyboards for All African Wikipedia Languages

Happy Africa Day!

To celebrate this, I am happy to make a little announcement: It is now possible to write in all the Wikipedias of all the languages of Africa, with all the special letters that are difficult to find on common keyboards. You can do it on any computer, without buying any new equipment, installing any software, or changing operating system preferences. Please see the full list of languages and instructions.

This release completes a pet project that I began a year ago: to make it easy to write in all the languages of Africa in which there is a Wikipedia or an active Wikipedia Incubator.

Most of these languages are written in the Latin alphabet, but with addition of many special letters such as Ŋ, Ɛ, Ɣ, and Ɔ, or letters with accents such as Ũ or Ẹ̀. These letters are hard to type on common keyboards, and in my meetings with African people who would write in Wikipedia in their language this is very often brought up as a barrier to writing confidently.

Some of these languages have keyboard layouts that are built into modern operating systems, but my experience showed me that to enable them one has to dig deep in the operating system preferences, which is difficult for many people, and even after enabling the right thing in the preferences, some keyboards are still wrong and hard to use. I hope that this will be built into future operating system releases in a more convenient way, just as it is for languages such as French or Russian, but in the mean time I provide this shortcut.

The new software released this week to all Wikimedia sites and to translatewiki.net makes it possible to type these special characters without installing any software or pressing any combining keys such as Ctrl or Alt. In most cases you simply need to press the tilde character (~) followed by the letter that is similar to the one you want to type. For example:

  • Ɓ is written using ~B
  • Ɛ is written using ~E
  • Ɔ is written using ~O
    … and so on.

Some of these languages are written in their own unique writing systems. N’Ko and Vai keyboards were made by myself, mostly based on ideas from freely licensed keyboard layouts by Keyman. (A keyboard for the Amharic language, also written with its own script, has had keyboards made by User:Elfalem for a while. I am mentioning it here for completeness.)

This release addresses only laptop and desktop computers. On mobile phones and tablets most of these languages can be typed using apps such as Gboard (also in iPhone), SwiftKey (also on iPhone), or African Keyboard. If you aren’t doing this already, try these apps on your phone, and start exchanging messages with your friends and family in your language, and writing in Wikipedia in your language on your phone! If you are having difficulties doing this, please contact me and I’ll do my best to help.

The technology used to make this is the MediaWiki ULS extension and the jquery.ime package.

I would like to thank all the people who helped:

  • Mahuton Possoupe (Benin), with whom I made the first of these keyboards, for the Fon language, at the Barcelona Hackathon.
  • Kartik Mistry, Santhosh Thottingal (India), Niklas Laxström (Finland), and Petar Petkovich (Serbia), who reviewed the numerous code patches that I made for this project.

This is quite a big release or code. While I made quite a lot of effort to test everything, code may always have bugs: missing languages, wrong or missing letters, mistakes in documentation, and so on. I’ll be happy to hear any feedback and to fix the bugs.

And now it’s all up to you! I hope that these keyboard layouts make it easier for all of you, African Wikimedians, to write in your languages, to write and translate articles, and share more knowledge!

Again, happy Africa day!

The full list of languages for which there is now a keyboard in ULS and jquery.ime:

  • Afrikaans
  • Akan
  • Amharic
  • Bambara
  • Berber
  • Dagbani
  • Dinka
  • Ewe
  • Fula
  • Fon
  • Ga
  • Hausa
  • Igbo
  • Kabiye
  • Kabyle
  • Kikuyu
  • Luganda
  • Lingala
  • Malagasy
  • N’Ko
  • Sango
  • Sotho
  • Northern Sotho
  • Koyraboro Senni Songhay
  • Tigrinya
  • Vai
  • Venda
  • Wolof
  • Yoruba
Advertisement

Disease of Familiarity, the Flaw of Wikipedia

Originally written as an answer to the question What are some major flaws in Wikipedia? on Quora. Republished here with some changes.

Wikipedia has a whole lot of flaws, and its basic meta-flaw is the disease of familiarity.

It does not mean what you think it means. The disease of familiarity is knowing so much about something that you don’t understand what it is like to not understand it.

I recognized this phenomenon in 2011 or so, and called it The Software Localization Paradox. I later realized that it has a lot of other aspects beyond software localization, so I thought a lot about it and struggled for years with giving it a name. I learned about the term “disease of familiarity” from Richard Saul Wurman, best known as the creator of the TED conference (see a note about it at the end of this post). Some other names for this phenomenon are “curse of knowledge” and “mind blindness”. See also Is there a name for “knowing so much about something that you don’t understand what is it like not to know it”?

Unfortunately, none of these terms is very famous, and their meaning is not obvious without some explanation. What’s even worse, the phenomenon is in general hard to explain because of its very nature. But I’ll try to give a few examples.


Wikipedia doesn’t make it easy for people to understand its jargon.

Wikipedia calls itself “The Free encyclopedia”; what does it mean that it’s “free”? I wrote Wikipedia:The Free Encyclopedia, one of the essays on this topic (there are others), but it’s not official or authoritative, and more importantly, the fact that this essay exists doesn’t mean that everybody who starts writing for Wikipedia reads it and understands the ideology behind it, and its implications. An important implication of this ideology is that according to the ideology of the Free Culture movement, of which Wikipedia is a part, is that some images and pieces of text can be copied from other sites into Wikipedia, and some cannot. The main reason for this is copyright law. People often copy text or images that are not compatible with the policies, and since this is heavily enforced by experienced Wikipedia editors, this causes misunderstandings. Wikipedia’s interface could communicate these policies better, but experienced Wikipedians, who already know them, rarely think about this problem. Disease of familiarity.

Wikipedia calls itself “a wiki”. A lot of people think that it’s just a meaningless catchy brand name, like “Kodak”. Some others think that it refers to the markup language in which the site is written. Yet others think that it’s an acronym that means “what I know is”. None of these interpretations is correct. The actual meaning of “wiki” is “a website that anyone can edit”. The people who are experienced with editing Wikipedia know this, and assume that everybody else does, but the truth is that a lot of new people don’t understand it and are afraid of editing pages that others had written, or freak out when somebody edits what they had written. Disease of familiarity.

The most common, built-in way for communication between the different Wikipedians is the talk page. Only Wikipedia and other sites that use the MediaWiki software use the term “talk page”. Other sites call such a thing “forum”, “comments”, or “discussion”. (To make things more confusing, Wikipedia itself occasionally calls it “discussion”.) Furthermore, talk pages, which started on Wikipedia in 2001, before commenting systems like Disqus, phpBB, Facebook, or Reddit were common, work in a very weird way: you need to manually indent each of your posts, you need to manually sign your name, and you need to use a lot of obscure markup and templates (“what are templates?!”, every new user must wonder). Experienced editors are so accustomed to doing this that they assume that everybody knows this. Disease of familiarity.

A lot of pages in Wikipedia in English and in many other languages have infoboxes. For example, in articles about cities and towns there’s an infobox that shows a photo, the name of the mayor, the population, etc. When you’re writing an article about your town, you’ll want to insert an infobox. Which button do you use to do this? There’s no “Infobox” button, and even if there were, you wouldn’t know that you need to look for it because “Infobox” is a word in Wikipedia’s internal jargon. What you actually have to do is Insert → Template → type “Infobox settlement”, and fill a form. Every step here is non-intuitive, especially the part where you have to type the template’s name. Where are you supposed to know it from? Also, these steps are how it works on the English Wikipedia, and in other languages it works differently. Disease of familiarity.

And this brings us to the next big topic: Language.

You see, when I talk about Wikipedia, I talk about Wikipedia in all languages at once. Otherwise, I talk about the English Wikipedia, the Japanese Wikipedia, the Arabic Wikipedia, and so on. Most people are not like me: when they talk about Wikipedia, they talk about the one in the language in which they read most often. Quite often it’s not their first language; for example, a whole lot of people read the Wikipedia in English even though English is their second language and they don’t even know that there is a Wikipedia in their own language. When these people say “Wikipedia” they actually mean “the English Wikipedia”.

There’s nothing bad in it by itself. It’s usually natural to read in a language that you know best and not to care very much about other languages.

But here’s where it gets complicated: Technically, there are editions of Wikipedia in about 300 languages. This number is pretty meaningless, however: There are about 7,000 languages in the world, so not the whole world is covered, and only in 100 languages or so there is a Wikipedia in which there is actually some continuous writing activity. In the other 200 the activity is only sporadic, or there is no activity at all—somebody just started writing something in that language, and a domain was created, but then the first people who started it lost interest and nobody else came to continue their work.

This is pretty sad because it’s frequently forgotten that a whole lot of people cannot read what they want in Wikipedia because they don’t know a language in which there is an article about what they want to learn. If you are reading this post, you have the privilege of knowing English, and it’s hard for you to imagine how does a person who doesn’t know English feel. Disease of familiarity: You think you can tell everybody “if you want to know something, read about it in Wikipedia”, but you cannot actually tell this to most people because most people don’t know English.

The missed opportunity becomes even more horrific when you realize that the people who would have the most appropriate skills for breaking out of this paradox are the people who are least likely to notice it, and the people who are hurt by it the most are the least capable of fixing it themselves. Think about it:

  • If you know, for example, Russian and English, and you need to read about a topic on which there is an article in the English Wikipedia, but not in Russian, you can read the English Wikipedia, and it’s possible that you won’t even notice that an article in Russian doesn’t exist. Unless you exercise mindfulness about the issue, you won’t empathize with people who don’t know English. To break out of this cycle, one can practice the following:
    • Always look for articles in Russian first.
    • Dedicate some time every week to translating articles. (See How does Wikipedia handle page translation?)
    • When you talk to people in your language, don’t assume that they know English.
  • A person who doesn’t know English is just stuck without an article, and there’s not much to do. It’s possible that you don’t even know that the article you need exists in another language. And maybe you cannot even read the user manual that teaches you how to edit. What can you do?
    • Try to be bold and ask your friends who do know English to translate it for you and publish the translation for the benefit of all the people who speak your language.
    • (Of course, there’s the solution of learning English, but we can’t assume that it works. Evidently, there are billions of people who don’t know English, and they won’t all learn English any time soon.)

(In case it isn’t clear, you can replace “English” and “Russian” in the example above with any other pair of languages.)

It’s particularly painful in countries where English, French, or Portuguese is the dominant language of government and education, even though a lot of the people, often the majority, don’t actually know it. This is true for many countries in Africa, as well as for Philippines, and to a certain extent also in India and Pakistan.

People who know English have a very useful aid for their school studies in the form of Wikipedia. People who don’t know English are left behind: the teachers don’t have Wikipedia to get help with planning the lessons and the students don’t have Wikipedia to get help with homework. The people who know English and study in English-medium schools have these things and don’t even notice how the other people—often their friends!—are left behind. Disease of familiarity.

Finally, most of the people who write in the 70 or so most successful Wikipedias don’t quite realize that the reason the Wikipedia in their language is successful is that before they had a Wikipedia, they had had another printed or digital encyclopedia, possibly more than one; and they had public libraries, and schools, and universities, and all those other things, which allowed them to imagine quite easily how would a free encyclopedia look like. A lot of languages have never had these things, and a Wikipedia would be the first major collection of educational materials in them. This would be pretty awesome, but this develops very slowly. People who write in the successful Wikipedia projects don’t realize that they just had to take the same concepts they already knew well and rebuild them in cyberspace, without having to jump through any conceptual epistemological hoops.

Disease of familiarity.


It’s hard to explain this.

I unfortunately suspect that very few, if any, people will understand this boring, long, and conceptually difficult post. If you disagree, please comment. If you think that you understand what I’m trying to say, but you have a simpler or shorter way to say it, please comment or suggest an edit (and tell your friends). If you have more examples of the disease of familiarity in Wikipedia and elsewhere, please speak up.

Thank you.


(As promised above, a note about Richard Saul Wurman. I heard him introduce the “disease of familiarity” concept in an interview with Debbie Millman on her podcast Design Matters, at about 23 minutes in. That interview was one of this podcast’s weirdest episodes: you can clearly hear that he’s making Millman uncomfortable, and she also mentioned it on Twitter. This, in turn, makes me uncomfortable to discuss something I learned from that interview, but I am just unable to find any better terminology for the phenomenon in question. If you have suggestions, please send them my way.)


Disclaimer: I’m a contractor working with the Wikimedia Foundation, but this post, as well as all my other posts on the topic of Wikimedia, Wikipedia, and related projects, are my own opinions and do not represent the Wikimedia Foundation.

Amir Aharoni’s Little Take on the Lodestar Affair

In case you haven’t heard, an op-ed called I Am Part of the Resistance Inside the Trump Administration was published in the New York Times on September 5. It was allegedly written by an anonymous senior person in the White House, and it made a whole lot of noise in the news.

People immediately started guessing who this is. One of the popular guesses is that it’s vice president Mike Pence, because the article uses the word “lodestar”, which is relatively rare, but unusually common in Pence’s past speeches.

And here’s my tiny, tiny conspiracy theory about it: “lodestar” was Merriam-Webster’s word of the day on August 28. Being a dictionary lover, I listen to Merriam-Webster’s Word of the Day podcast every day using Podcast Addict, a simple RSS-based podcast player. I didn’t hear this episode. If you try to download this episode using Podcast Addict, you’ll see that the title is “lodestar”, but in fact it’s the episode for “rubric“, the previous day’s episode.

It’s kind of weird, but maybe it’s a total coincidence. Maybe the person who wrote the op-ed just follows the word of the day not through the podcast, but elsewhere on the web. And maybe it has nothing to do with Merriam-Webster, and they are just an educated person who knows words like “lodestar”.

But hey, feel free to spread the rumor that Merriam-Webster is trying to subvert the government, or make up whatever other nonsense you want.

Wikimedia Strategy Phase 1: What Does It Mean for Me and (Maybe) for Language Diversity in Wikipedia

The Wikimedia Foundation is leading a process to write a strategy for the Wikimedia movement. This process takes over a year. A few months ago, the conclusion of Phase 1 of this process was published: The strategic direction.

Some central concepts in this document are “knowledge as a service” and “knowledge equity”. Some people said that it’s too vague and high-level, and that it can be interpreted in a lot of ways. This is true, especially in a movement that is as culturally and linguistically diverse as Wikimedia. Perhaps this is intentional, so that people will be able to interpret this in any way that feels right for them.

Recently I was filling a registration form for Wikimedia Conference 2018. This form was very long, and it asked what do the concepts that appear in the strategic direction document mean to me. My answers were longish, and since there’s nothing secret about them, and they may (or may not) interest some people, I copied them from the form to this blog post. I edited them slightly for publishing here so that the context will be clearer, but the essence is the same as what I submitted.

Knowledge as a service

The knowledge that Wikimedia projects already contain is available through all common channels of communication: in addition to being available on the website, it must be findable on all search engines in all languages and countries, browsable on devices of all operating systems whether open or not, browsable as much as possible through social networks and chat applications, embeddable in other apps, etc.

It must be easy for all people, whether they are knowledgeable about computers or not, to contribute their knowledge to Wikimedia sites, and humanity in general should know that Wikimedia sites is the place where they contribute their knowledge and not only learn it.

Knowledge equity

What it means to me is:

  • That all people, of all ages and all kinds of identities, of all countries, who speak all languages, must be able to read and write in their language.
  • That we will fight whenever it’s reasonable against censorship and against all kinds of chilling effects that deter potential contributors or threaten their well-being.
  • That we remain independent of commercial and political entities by strictly refusing to carry political and commercial advertising and to accept unreasonable limited grants.
  • That all the software that is useful for reading and writing on our sites must be easily usable in all languages, whether it’s core software, extensions, templates, or gadgets.
  • That we don’t depend on any non-Free or otherwise unethical software, even if it appears to make consuming and contributing knowledge easier.
  • That we set a goal of having good coverage for core content in all languages and actively pursue it and not leave it only to the community’s “invisible hand”.
  • That we set a goal that the most popular Wikimedia projects in each country are in that country’s most spoken languages and not in a foreign language.

What kind of conditions do you need to realize these activities?

Describe what you think would be good conditions for you to move forward in this direction. Think of conditions in the broadest sense; e.g., capacity, skills, partnerships, clarification, structures and processes, room for development or experimentation, financial resources, people, access to other means of support etc.

We need to partner with academic institutions that work on topics that are not currently covered by our projects because of systemic bias.

We need to partner more with organizations that have expertise in developing minorized and under-resourced languages, working on the ground in the countries where these languages are spoken.

We need easy access to data about the social and political situations in poorer countries, and if such data doesn’t exist at all, we need to lead research that creates such data ourselves.

We need a new attitude to developing software for our sites: we need to understand what do our communities actually do on the sites with gadgets and templates rather than just developing new extensions that may be shiny, but are hard to integrate into the sites, each of which is heavily customized.

What I wrote in that form is a good description of my current attitude to what the priorities of Wikimedia movement should be, at least in terms of ideology and values. You can clearly see my interests: remembering that language support is important and that most people don’t speak English; remembering that we are not supposed to be an American non-profit organization, but an international movement that happens to have an office in the U.S.; remembering that we are also a part of the Free Software movement; remembering that good software engineering are important, even if engineering alone can’t solve all the problems.

For people who have doubts: This post represents my own opinions, and doesn’t express the opinion of the Wikimedia Foundation or any of its employees or managers.

The Curious Problem of Belarusian and Igbo in Twitter and Bing Translation

Twitter sometimes offers machine translation for tweets that are not written in the language that I chose in my preferences. Usually I have Hebrew chosen, but for writing this post I temporarily switched to English.

Here’s an example where it works pretty well. I see a tweet written in French, and a little “Translate from French” link:

Emmanuel Macron on Twitter.png

The translation is not perfect English, but it’s good enough; I never expect machine translation to have perfect grammar, vocabulary, and word order.

Now, out of curiosity I happen to follow a lot of people and organizations who tweet in the Belarusian language. It’s the official language of the country of Belarus, and it’s very closely related to Russian and Ukrainian. All three languages have similar grammar and share a lot of basic vocabulary, and all are written in the Cyrillic alphabet. However, the actual spelling rules are very different in each of them, and they use slightly different variants of Cyrillic: only Russian uses the letter ⟨ъ⟩; only Belarusian uses ⟨ў⟩; only Ukrainian uses ⟨є⟩.

Despite this, Bing gets totally confused when it sees tweets in the Belarusian language. Here’s an example form the Euroradio account:

Еўрарадыё   euroradio    Twitter double.pngBoth tweets are written in Belarusian. Both of them have the letter ⟨ў⟩, which is used only in Belarusian, and never in Ukrainian and Russian. The letter ⟨ў⟩ is also used in Uzbek, but Uzbek never uses the letter ⟨і⟩. If a text uses both ⟨ў⟩ and ⟨і⟩, you can be certain that it’s written in Belarusian.

And yet, Twitter’s machine translation suggests to translate the top tweet from Ukrainian, and the bottom one from Russian!

An even stranger thing happens when you actually try to translate it:

Еўрарадыё   euroradio    Twitter single Russian.pngNotice two weird things here:

  1. After clicking, “Ukrainian” turned into “Russian”!
  2. Since the text is actually written in Belarusian, trying to translate it as if it was Russian is futile. The actual output is mostly a transliteration of the Belarusian text, and it’s completely useless. You can notice how the letter ⟨ў⟩ cannot be transliterated.

Something similar happens with the Igbo language, spoken by more than 20 million people in Nigeria and other places in Western Africa:

 4  Tweets with replies by Ntụ Agbasa   blossomozurumba    Twitter.png

This is written in Igbo by Blossom Ozurumba, a Nigerian Wikipedia editor, whom I have the pleasure of knowing in real life. Twitter identifies this as Vietnamese—a language of South-East Asia.

The reason for this might be that both Vietnamese and Igbo happen to be written in the Latin alphabet with addition of diacritical marks, one of the most common of which is the dot below, such as in the words ibụọla in this Igbo tweet, and the word chọn lọc in Vietnamese. However, other than this incidental and superficial similarity, the languages are completely unrelated. Identifying that a text is written in a certain language only by this feature is really not great.

If I paste the text of the tweet, “Nwoke ọma, ibụọla chi?”, into translate.bing.com, it is auto-identified as Italian, probably because it includes the word chi, and a word that is written identically happens to be very common in Italian. Of course, Bing fails to translate everything else in the Tweet, but this does show a curious thing: Even though the same translation engine is used on both sites, the language of the same text is identified differently.

How could this be resolved?

Neither Belarusian nor Igbo languages are supported by Bing. If Bing is the only machine translation engine that Twitter can use, it would be better to just skip it completely and not to offer any translation, than to offer this strange and meaningless thing. Of course, Bing could start supporting Belarusian; it has a smaller online presence than Russian and Ukrainian, but their grammar is so similar, that it shouldn’t be that hard. But what to do until that happens?

In Wikipedia’s Content Translation, we don’t give exclusivity to any machine translation backend, and we provide whatever we can, legally and technically. At the moment we have Apertium, Yandex, and YouDao, in languages that support them, and we may connect to more machine translation services in the future. In theory, Twitter could do the same and use another machine translation service that does support the Belarusian language, such as Yandex, Google, or Apertium, which started supporting Belarusian recently. This may be more a matter of legal and business decisions than a matter of engineering.

Another thing for Twitter to try is to let users specify in which languages do they write. Currently, Twitter’s preferences only allow selecting one language, and that is the language in which Twitter’s own user interface will appear. It could also let the user say explicitly in which languages do they write. This would make language identification easier for machine translation engines. It would also make some business sense, because it would be useful for researchers and marketers. Of course, it must not be mandatory, because people may want to avoid providing too much identifying information.

If Twitter or Bing Translation were free software projects with a public bug tracking system, I’d post this as a bug report. Given that they aren’t, I can only hope that somebody from Twitter or Microsoft will read it and fix these issues some day. Machine translation can be useful, and in fact Bing often surprises me with the quality of its translation, but it has silly bugs, too.

Five More Privileges of English Speakers, part 2: Language and Software

For the previous part in the series, see Five Privileges of English Speakers, part 1.

I’m continuing the series of posts in each of which I write about five privileges that English speakers have without giving it a lot of thought. The examples I give mostly come from my experience translating software, Wikipedia articles, blog posts, and some other texts between English, Hebrew, and Russian. Hebrew and Russian are the languages I know best. If you have interesting examples from other languages, I am very interested in hearing them and writing about them.

I’m writing them mostly as they come into my mind, without a particular order, but the five items in this part of the series will focus on usage of the English language in software, and try to show that the dominance of English is not only a consequence of economics and history, but that it’s further reinforced by features of the language itself.

1. Software usually begins its life in English

English is the main language of software development worldwide.

The world’s best-known place for software development is Silicon Valley, an English-speaking place. That’s the place of Facebook, Google, Apple, Oracle and many others. California is also the home of Adobe.

There are several other hubs of software development in United States: Seattle (Microsoft, Amazon), North Carolina (Red Hat), New York (IBM, CA), Massachusets (TripAdvisor, Lotus, RSA), and more. The U.S. is also the source for much of computer science research and education, coming from Berkeley, MIT, and plenty of other schools. The U.S. is also the birthplace of the Internet, originally supported by the U.S. Department of Defense and several American universities. The world wide web, which brought the Internet to the masses, was created in Switzerland by an English speaker.

Software is developed in other countries—India, Russia, Israel, France, Germany, Estonia, and many other countries. But the dominance of the U.S. and of the English language is clear. The reason for this is not only that the U.S. is the source for much of computer technologies, but also—and probably more importantly—that the U.S. is the biggest consumer market for software. So developers in all countries tend to optimize the product for the highest-paying consumers, and these only need English.

When engineers write the user interface of their software in English, they often do not give any thought to other languages at all, or make translation possible, but complicated by English-centric assumptions about number, gender, text direction, text size, personal names, and plenty of other things, which will be explored in further points.

2. Terminology

English is also the source for much of the computer world’s terminology. Other languages have to adapt terms like smartphone, network, token, download, authentication, and thousands of others.

Some language communities work hard to translate them all meticulously into native words; Icelandic, Lithuanian, French, Chinese, and Croatian are famous examples. This is nice, but requires effort on behalf of terminology committees, who need to keep up with the fast pace of technological development, and on behalf of the software translators, who have to keep with the committees.

Some just transliterate most of them: keep the term essentially in English, but rewritten in the native alphabet. Hindi and Japanese are examples of that. This seems easy, but it is based on a problematic assumption: that the target language speakers who will use the software know at least some English! This assumption is correct for the translators, who don’t just know the English terms, but are probably also quite accustomed to it, but it’s not necessarily correct for the end users. Thus, the privilege is perpetuated.

Some languages, such as Hebrew, German, and Russian, are mid-way, with language academics and purists pulling to purer native language, engineers pulling to more English-based words, and the general public settling somewhere in between—accepting the neologisms for some terms, and going for English-based words for others.

For the non-English languages it provides fertile ground for arguments between purists and realists, in which the needs of the actual users are frequently forgotten. All the while, English speakers are not even aware of all this.

3. Easy binary logic word formation

One particular area of computer terminology is binary logic. This sounds complicated, but it’s actually simple: in electronics and software opposite notions such as true / false, success / failure, OK / Cancel, and so forth, are very common.

This translates to a great need for words that express opposites: enable / disable, do / undo, log in / log out, delete / undelete, block / unblock, select / deselect, online / offline, connect / disconnect, read / unread, configured / misconfigured.

Notice something? All of the above words are formed with the same root, with the addition of a prefix (un-, dis-, de-, mis-, a-), or with the words “on” and “off”.

A distinct, but closely related need, is words for repetition. Computers are famously good at doing things again and again, and that’s where the prefix re- is handy: reconnect, retry, redo, retransmit.

These features happen to be conveniently built into the English language. While English has extremely simple morphology for declension and conjugation (see the section “Spell-checking” in part 1 of the series), it has a slightly more complex morphology for word formation, but it’s still fairly easy.

It is also productive. That is, a software developer can create new words using it. For example, the MediaWiki software has the concept of “oversight”—hiding a problematic page in such a way that only users with a particular permission can read it. What happens if a page was hidden by mistake? Correct: “unoversight”. This word doesn’t quite exist elsewhere, but it doesn’t sound incorrect, because familiar English word formation rules were used to coin it.

As it always happens, English-speaking software engineers either don’t think about it at all, or think that other languages also have similar word formation rules. If you haven’t guessed it already, it is not true. Sime other European languages have similar constructs, but not necessarily as consistent as in English. And for Semitic languages like Hebrew it’s a disaster, because in Semitic languages prefixes are used for entirely different things, and the grammar doesn’t have constructs for repetition and negation. So when translating software user interface strings into Hebrew, we have to use different words as opposites. For example the English pair connect / disconnect is translated as lehitḥabér / lehitnaték—completely different roots, which Hebrew is just lucky to have. Another option is to use negative words like lo and bilti, or bitul, but they are often unnatural or outright wrong. Having to deal with something like “Mark as unread” is every Hebrew software translator’s nightmare, even though it sounds pretty straightforward in English.

English itself also has pairs of negative words that are not formed using the above prefixes, for example next / previous and open / close, but in many other languages they are much more common.

4. Verbing

“Verbing weirds language”, as one of the famous Calvin and Hobbes panels says.

Despite being a funny joke in the comic, it’s a real feature of the English language: because of how English morphology and syntax work, nouns can easily jump into the roles of adjectives and verbs without changing the way they are written.

For English, this is a useful simplification, and it works in labeling, as well as in advertising. “Enjoy Coca-Cola” is something more than an imperative. The fact that it’s a short single word and that it’s the same in all genders and numbers, makes it more usable as a call to action than it would be in other languages. And, other than advertising, where are calls to action very common? Software, of course. When you’re trying to tell a user to do something, a word that happens to be both the abstract concept and the imperative is quite useful.

Perhaps the most famous example of this these days is Facebook’s “Like”. Grammatically, what is it in English? Imperative? A noun describing an abstract action? Maybe a plain old noun, as in “chasing likes” (this is a plural noun—English verb don’t have a plural form!)? Answer: it’s all of them and more.

When translated to Hebrew in Facebook’s interface, it’s Ahávti, which literally means “I loved it”. Actually, this translation is mostly good, because it’s understandable, idiomatic, and colloquial enough without compromising correctness. Still, it’s a verb, which is not imperative, and it’s definitely not a noun, so you cannot use it in a sentence as if it was a noun. Indeed, Hebrew speakers are comfortable using this button, but when they speak and write about this feature, they just use its English name: “like” (in plural láykim). It even became a slightly awkward, but commonly used verb: lelaykék. Something similar happens in Russian.

It would be impossible in Hebrew and Russian to use the exact same word for the noun and the verb, especially in different persons and genders. Sometimes the languages are lucky enough to be able to adapt an English verb in a way that is more or less natural, but sometimes it’s weird, and hurts the user experience.

5. Word length

This one is relatively simple and not unique to English, but should be mentioned anyway: English words are neither very long, nor very short. Examples of languages where words are, on average, longer than in English, are Finnish, Tamil, German, and occasionally Russian. Hebrew tends to be shorter, although sometimes a single English word has to be translated with several Hebrew words, so it can get also get longer. This is true for a pretty much any language, really.

In designing interfaces, especially for smaller screens, the length of the text is often important. If a button label is too long, it may overflow from the button, or be truncated, making the display ugly, or unusable, or both.

If you’re an English speaker, it probably won’t happen with you, because almost all software is usually designed with the word length of your language in mind. Other languages are almost always an afterthought.

The good practice for software engineers and designers is to make sure that translated strings can be longer. Their being shorter is rarely a problem, although sometimes a string is so short that the button may become to small to click or tap conveniently.


Generally, what can you do about these privileges?

Whoever you are, remember it. If you know English, you are privileged: Software is designed more for you than for people who speak other languages.

If you are a software engineer or a designer, at the very least, make your software translatable. Try to stick to good internationalization practices and to standards like Unicode and CLDR. Write explanations for every translatable string in as much detail as possible. Listen to users’ and translators’ complaints patiently—they are not whining, they are trying to improve your software! The more internationalizable it is, the more robust it is for you as a developer, and for your English-speaking users, too, because better design thinking will be going into each of its components, and less problematic assumptions will be made.

Five Privileges of English Speakers, part 1

It’s very common today on progressive blogs to urge people to check their privilege.

Being an English speaker, native or non-native, is a privilege.

It’s not as often as discussed as other forms of privilege, such as white, male, cis, hetero, or rich privilege. The reason for this is simple: The world’s media is dominated by the English language. English-language movies are more popular in many countries than movies in these countries’ own languages, English-language news networks are quoted by the rest of the world, the world’s most popular social networks are based in the U.S. and are optimized for U.S. audiences, etc.

So, when English speakers discuss privilege among each other, English is not much of an issue, and they dedicate more time to race, gender, wealth, religion, and other factors that differentiate between people in English-speaking countries.

Despite this, I am not the first one to describe English as a privilege. A simple Google search for english language privilege will yield many interesting results.

What I do want to try to do in this series of posts is to list the particular nuances that make English such a privilege in as much detail as possible. I wanted to write this for a long time, but there are many such nuances, so I’ll just do it in batches of five, in no particular order:

1. Keyboard

If you speak English, congratulations: A keyboard on which your language can be written is available on all electronic devices.

All of them.

All desktops, laptops, phones, tablets, watches. The only notable exception I can think of is typewriters, which only makes the point more tragic: technology moved forward and made writing easier in English, but harder in many other languages, where local-language typewriters were replaced with computers with English-only keyboard.

At the very worst case, writing English on a computer will be slightly inconvenient in countries like Germany, France, or Turkey, where the placement of the Latin letters on the keys is slightly different from the U.S. and U.K. QWERTY standard. Oh, poor American tourists.

On a more serious note, though, even though a lot of languages use the Latin alphabet, a lot of them also use a lot of extra diacritics and special characters, and English is one of the very few that doesn’t. Of the top 100 world’s languages by native speakers, only Malay, Kinyarwanda, Somali, and Uzbek have standardized orthographies that can be written in the basic 26-letter Latin alphabet without any extra characters. We can also add Swahili, which has a large number of non-native speakers, but that’s it. With other languages you can get stuck and not be able to write your language at all (Hindi, Chinese, Russian, etc.), or you may have to write in a substandard orthography because you can’t type letters like é or ł (French, Vietnamese, Polish, etc.).

The above is just the teeny-tiny tip of the iceberg; the keyboard problem will be explored in more points later.

2. Spell-checking

English word morphology is laughably simple.

There’s -s for plurals and for third person present tense verbs, there’s -‘s for possession, and there are -ed and -ing verb forms. There are also some contractions (‘d, ‘s, ‘ll, ‘ve), and a long, but finite list of irregular verb forms, and an even shorter list of irregular plural noun forms. And that’s it.

Most languages aren’t like that. In most languages words change with prefixes, suffixes, infixes, clitics, and so on, according to their role in the sentence.

Beyond the fact that English writing is (arguably) easier for children and foreigners to learn, this means that software tools for processing a language are easy to develop for English and hard to develop for other languages.

The first simple example is spell-checking.

English has had not just spelling, but also grammar and style checkers built into common word processors for decades, and many languages of today don’t even have spelling checkers, not to mention grammar, or style, or convenient searching. (See below.)

So in English, when you type “kinh”, most word processors will suggest correcting it to “king”, but then, some of them may also suggest replacing this word with “monarch” to be more inclusive for women, and this is just one of the hundreds of style improvement suggestions that these tools can make. For a lot of other languages, even simple spell-checking of single words hasn’t been developed yet, and grammar checking is a barely-imaginable dream.

3. Autocompletion

Simpler morphology has many other effects.

Even though Russian is my first native language and I speak it more fluently than I speak English, I am much slower when I’m typing in Russian on my phone. In English, the autocompleting keyboard makes it possible to write just two or three letters of a word and let the software complete the rest. In Russian, the ending of the word must be typed, and autocompletion rarely guesses it correctly. Typing an incorrect ending will make a sentence convey incorrect information, or just make it completely ungrammatical.

4. Searching

A yet-another issue of the previous point, English’s very simple morphology makes searching easier.

For example word processors have a search and replace function. For English, it will likely find all forms of the word, because there are so few of them anyway. But in Hebrew and Arabic, letters are often inserted or changed in the middle of the word according to its grammatical state, and you need to search for each form, which is quite agonizing. It’s comparable to “man” vs. “men” in English, except that in English such changes are very rare, while in many other languages it happens in almost every word.

With search engines that must find words across thousands of documents it gets even harder. Google can easily figure out that if you’re searching for “drive”, you may also be interested in “driving”, “drove”, and “driven”, but Russian has dozens of other forms for this word. A few languages are lucky: special support was developed for them in search engines, and tasks of this kind are automated, but most languages our just out in the cold. But English barely needs extra support like this in the first place.

5. Very little gender

A lot can be said about gendered language, but as far as basic grammar goes, English has very little in the area of gender. “He” and “She”, and that’s about it. There are also man/woman, actor/actress, boy/girl, etc., but these distinctions are rarely relevant in technology.

In many other languages gender is far more pervasive. In Semitic and Slavic languages, a lot of verb forms have gender. In English, the verb “retweeted” is the same in “Helen retweeted you” and “Michael retweeted you”, but in Hebrew the verb is different. Because Twitter doesn’t know that Helen needs a different verb, it uses the masculine verb there, which sounds silly to Hebrew speakers.

I asked Twitter developers about this many times, and they always replied that there’s no field for gender in the user profile. It becomes more and more amusing lately, now that it has become so common —and for good reasons!— to mention what one’s preferred pronouns are in the Twitter profile bio. So people see it, but computers don’t.

On a more practical note, in the relatively rare cases when third person pronouns must be used in software strings, English will often use the singular “they” instead of “he” or “she”. So English-speaking developers do notice it, but not as often as they should, and when they do, they just use the lazy singular-they solution, which is socially acceptable and doesn’t require any extra coding. If only they’d notice it more often, using their software in other languages would be much more convenient for people of all genders.

The only software packages that I know that have reasonably good support for grammatical gender are MediaWiki and Facebook’s software. I once read that Diaspora had a very progressive solution for that, but I don’t know anybody who actually uses it. There may be other software packages that do, but probably very few.


These are just the first five examples of English-language privilege I can think of. There will be many, many more. Stay tuned, and send me your ideas!

Link Wikipedia Articles in Different Languages

OK THIS IS AWESOME, and “awesome” is not a word that I use lightly.

As a gift for the second birthday of the Wikidata project, nice people at Google created a tool that helps people link articles in different languages that are not linked yet. They prepared a list with thousands of pairs of articles in different languages that are supposed to be about the same subject according to their automatic guesswork. The tool only shows such articles, and a human editor must check whether they actually match, and if they do—make the linking automatically.

There were thirty six such articles for the Hebrew–English pair. About four of them were unrelated, and I fixed the linking between the rest of them. Some of them required manual intervention, because there were interfering links to unrelated subjects. For some simple cases it took me just a few seconds, and for a few complicated ones—a few minutes.

I also tried doing the same for Russian–English, but there are over a thousand article pairs there, so I only did a few. I also did a few for Catalan and Greek, and I finished all ten pairs for Bengali, even though I don’t actually know Greek or Bengali. I just used a bit of healthy intuition and Google Translate, and I’m pretty sure that I did it well.

You can help!

Here are my suggested instructions for doing this.

Preparation:

  1. Log in to mediawiki.org. This account is used also for the tool.
  2. Now go to the tool’s site. Click Login, and allow the tool to use your mediawiki.org account.
  3. Go to settings, and choose your pair of languages.
  4. Go to “Check by list” and you’ll see a list of article pairs. If there are no suggested article pairs for the language pair you selected, go back to number 3 choose some other languages. As I wrote above, from my experience, you don’t need to know a language thoroughly to perform this useful work ;)

Now click a link to a pair of articles that looks reasonable. Articles in both languages will open side by side.

  1. If the articles are definitely not about the exact same subject, click “No” in the list and find another pair.
  2. If the articles are about the same subject and one of them doesn’t have any interlanguage links, click “Add links” in the interlanguage area. In the box that will open, write the language name of the other language in the first field and the title of the article in the other field, and then click the “Link with page” button. A list of articles in other languages will be shown. If it looks reasonable, click “Confirm”, and then “Close dialog and reload page”. That’s it, the pages are linked! Click “Yes” in the list in the linking tool and proceed to another article pair.
  3. If the articles are about the same subject, but both of them appear to have links to other language, it’s possible that explicit interlanguage links are written in the source code of the articles. To resolve this, do the following:
    1. Open both articles for editing in source mode.
    2. Scroll all the way down and find whether they have explicit interlanguage links.
    3. If these are correct links to articles about the same subjects in other languages, go to those articles, and link them using Wikidata. Note that it often happens in such cases that these are links to redirects, so the actual current title may be different.
    4. If these are links to articles about other subjects, even if they are related, remove those links. For example, if the article in Bengali is about an island, and the article in Dutch is about a city on that island, remove the link – these subject are distinct enough. Ditto if the article in English is about an American human rights organization and the article in French is about a French human rights organization.
    5. If you were able to remove all the explicit links from the source, go back to point 2 above and link the articles using Wikidata.
    6. If it’s too complicated to remove these links for any reason, feel free to go to another article, but it would be nice to leave a note about this on the articles’ talk pages so that other editors would clean this up some time.

That’s it. It may get a tad complicated for some cases, but if you ask me, it’s a lot of fun.

Serbian Spam

I always celebrate when I receive spam in a language in which I haven’t yet received spam. I just received spam in Serbian for the first time. It was in the Cyrillic alphabet; Serbian can also be written in Latin, and it is frequently done in Serbia, possibly even more frequently than in Cyrillic, even though the government prefers Cyrillic.

This makes me wonder: Is Serbian in Cyrillic popular and important enough for spamming in it, or did the silly spammer just use Google Translate to translate to Serbian and got the result in Cyrillic, because that’s what Google Translate does?

If you know Serbian, can you please tell me whether it looks real or machine-translated? Words like “5иеарс” and the spaces before the punctuation marks give me a strong suspicion that it’s machine translation, but I might be wrong.

Молим вас за попустљивост за нежељене природи овог писма , али је рођена из очаја и тренутног развоја . Молимо носе са мном . Моје име је сер Алекс Бењамин Хубертревизор Африке развојне банке открио постојећи налог за успавану 5иеарс .

Када сам открио да није било ни наставак ни исплате са овог рачуна на овог дугог периода и наши банкарских закона предвиђа да ће било неупотребљивим чине више од 5иеарс иду на банковни прихода као неостварен фонда .

Ја сам се распитивала за личне депонента и његове најближе , али нажалост ,депонент и његове најближе преминуо на путу до Сенегала за тајкун , а он је оставио иза себе нема тело за ову тврдњу само сам направио ову истрагу само да буде двоструко сигурни у ту чињеницу , а пошто сам био неуспешан у лоцирању родбину .

So, how does it look? And do you receive Serbian spam? Thanks.

A Relevant Tower of Babel

The Tower of Babel is frequently used as a symbol of foreign languages. For example, several language software packages are named after it, such as the Babylon electronic dictionary, MediaWiki’s Babel extension and the Babelfish translation service (itself named after the Babel fish from The Hitchhiker’s Guide).

In this post I shall use the Tower of Babel in a somewhat more relevant and specific way: It will speak about multilingualism and about Babel itself.

This is how most people saw the Wikipedia article about the Tower of Babel until today:

The Tower of Babel article. Notice the pointless squares in the Akkadian name. They are called "tofu" in the jargon on internationalization programmers.
The tower of Babel. Notice the pointless squares in the Akkadian name. They are called “tofu” in the jargon on internationalization programmers.

And this is how most people will see it from today:

And we have the name written in real Akkadian cuneiform!
And we have the name written in real Akkadian cuneiform!

Notice how the Akkadian name now appears as actual Akkadian cuneiform, and not as meaningless squares. Even if you, like most people, cannot actually read cuneiform, you probably understand that showing it this way is more correct, useful and educational.

This is possible thanks to the webfonts technology, which was enabled on the English Wikipedia today. It was already enabled in Wikipedias in some languages for many months, mostly in languages of India, which have severe problems with font support in the common operating systems, but now it’s available in the English Wikipedia, where it mostly serves to show parts of text that are written in exotic fonts.

The current iteration of the webfonts support in Wikipedia is part of a larger project: the Universal Language Selector (ULS). I am very proud to be one of its developers. My team in Wikimedia developed it over the last year or so, during which it underwent a rigorous process of design, testing with dozens of users from different countries, development, bug fixing and deployment. In addition to webfonts it provides an easy way to pick the user interface language, and to type in non-English languages (the latter feature is disabled by default in the English Wikipedia; to enable it, click the cog icon near “Languages” in the sidebar, then click “Input” and “Enable input tools”). In the future it will provide even more abilities, so stay tuned.

If you edit Wikipedia, or want to try editing it, one way in which you could help with the deployment of webfonts would be to make sure that all foreign strings in Wikipedia are marked with the appropriate HTML lang attribute; for example, that every Vietnamese string is marked as <span lang=”vi” dir=”ltr”>. This will help the software apply the webfonts correctly, and in the future it will also help spelling and hyphenation software, etc.

This wouldn’t be possible without the help of many, many people. The developers of Mozilla Firefox, Google Chrome, Safari, Microsoft Internet Explorer and Opera, who developed the support for webfonts in these browsers; The people in Wikimedia who designed and developed the ULS: Alolita Sharma, Arun Ganesh, Brandon Harris, Niklas Laxström, Pau Giner, Santhosh Thottingal and Siebrand Mazeland; The many volunteers who tested ULS and reported useful bugs; The people in Unicode, such as Michael Everson, who work hard to give a number to every letter in every imaginable alphabet and make massive online multilingualism possible; And last but not least, the talented and generous people who developed all those fonts for the different scripts and released them under Free licenses. I send you all my deep appreciation, as a developer and as a reader of Wikipedia.