Archive for the 'Wikipedia' Category

Yak Shaving, part 2: Wikidata, Busiati col Pesto Trapanese, and Other Slow Food

Some time ago I celebrated a birthday in an Italian restaurant in Haifa, and I saw a pack of pasta of a curious shape on a shelf there. I asked whether they serve it or sell it.

“No”, they told me, “it’s just a display”.

This answer didn’t satisfy me.

I added the pasta’s name, Busiate, to my shopping list.

I searched for it in a bunch of stores. No luck.

I googled for it and found an Israeli importer of this pasta. But that importer only sell in bulk, in crates of at least 12 items. That’s too much.

And of course, I searched Wikipedia, too. There’s an article about Busiate in the English Wikipedia. There also an article about this pasta in Arabic and in Japanese, but curiously, there’s no article about it in the Wikipedia in the Italian language, nor in the Sicilian language, given that this type of pasta is Sicilian.

So I… did a few things about it.

I Improved the article about Busiate in the English Wikipedia: cleaned up references, cleaned up formatting, and updated the links to references.

I also improved the references and the formatting to the article about Pesto alla trapanese, the sauce with which this pasta is traditionally served.

I also cleaned up the Wikidata items associated with the two articles above: Q48852218 (busiate) and Q3900766 (pesto alla trapanese).

I also translated all the names of the Wikidata properties that are used on these items to Hebrew. I usually do this when I do something with any Wikidata item: I only need to translate these property names once, and after that all the people who use Wikidata in Hebrew will see items in which these properties are used in Hebrew. There are more than 6000 properties, and the number is constantly growing, so it’s difficult to have everything translated, but every little translation makes the experience more completely translated for everyone.

I added references to the Wikidata item about the sauce. Wikidata must have references, too, and not only Wikipedia. I am not enthusiastic about adding random recipe sites that I googled up as references, but luckily, I have The Slow Food Dictionary of Italian Regional Cooking, which I bought in Italy, or more precisely in Esino Lario, where I went for the 2016 Wikimania conference.

Now, a book in Wikidata is not just a book. You need to create an item about the book, and another item about the edition of a book. And since I created those, I create Wikidata items for the dictionary’s original Italian author Paola Gho, for the English translator John Irving, and for the publishing house, Slow Food.

And here’s where it gets really nerdy: I added each of the sauce’s ingredients as values of the “has part” property, and added the dictionary as a reference for each entry. I initially thought that it’s overdone, but you know what?—When we’ll have robot cooks, as in the movie I, Robot, busiati col pesto trapanese will be one of the first things that they will know how to prepare. One of the main points of Wikidata is that it’s supposed to be easy to read for both people and machines.

And since I have a soft spot for regional languages, I also added the sauce’s Sicilian name under the “native label” property: pasta cull’àgghia. The aforementioned Slow Food Dictionary of Italian Regional Cooking actually does justice to the regional part in its title, and gives the names of the different food items in the various regional languages of Italy, so I could use it as a reliable source.

And I translated the Wikipedia article into Hebrew: בוזיאטה.

And I also created the “Sicilian cuisine” category in the Hebrew Wikipedia. A surprisingly large number of articles already existed, filed under “Italian cuisine”: Granita, Arancini, Cannoli, and a few others. Now they are organized under Sicilian cuisine. (I hope that some day Wikipedia categories will be managed more automatically with the help of Wikidata, so that I wouldn’t have to create them by hand.)

Finally, I found the particular issue of the Gazzetta Ufficiale of the Italian Republic, in which busiati col pesto trapanese was declared as a traditional agricultural food product, and I added that issue as a reference to the Wikidata item, as well.

And all of this yak shaving happened before I even tasted the damn thing!

So anyway, I couldn’t find this pasta anywhere, and I couldn’t by it from the importer’s website, but I wanted it really badly, so I called the importer on the phone.

They told me they don’t have any stores in Jerusalem that buy from them, but they suggested checking a butcher shop in Mevaseret Tsiyon, a suburb of Jerusalem. Pasta in a butcher shop… OK.

So I took a bus to Mevaseret, and voila: I found it there!

And I made Busiate, and I made the sauce! It’s delicious and totally worth the effort.

Of course, I could just eat it without editing Wikipedia and Wikidata on the way, but to me that would be boring.

My wife and my son loved it.

These are the busiate with pesto alla trapanese that I made at home. I uploaded this photo to Wikimedia Commons and added it to the English Wikipedia article as an illustration of how Busiate are prepared. I wonder what do Wikipedians from Sicily think of it.

There is a story behind every Wikipedia article, Wikidata item, and Commons image. Millions and millions of stories. I wrote mine—you should write yours!

Advertisements

Happy Africa Day: Keyboards for All African Wikipedia Languages

Happy Africa Day!

To celebrate this, I am happy to make a little announcement: It is now possible to write in all the Wikipedias of all the languages of Africa, with all the special letters that are difficult to find on common keyboards. You can do it on any computer, without buying any new equipment, installing any software, or changing operating system preferences. Please see the full list of languages and instructions.

This release completes a pet project that I began a year ago: to make it easy to write in all the languages of Africa in which there is a Wikipedia or an active Wikipedia Incubator.

Most of these languages are written in the Latin alphabet, but with addition of many special letters such as Ŋ, Ɛ, Ɣ, and Ɔ, or letters with accents such as Ũ or Ẹ̀. These letters are hard to type on common keyboards, and in my meetings with African people who would write in Wikipedia in their language this is very often brought up as a barrier to writing confidently.

Some of these languages have keyboard layouts that are built into modern operating systems, but my experience showed me that to enable them one has to dig deep in the operating system preferences, which is difficult for many people, and even after enabling the right thing in the preferences, some keyboards are still wrong and hard to use. I hope that this will be built into future operating system releases in a more convenient way, just as it is for languages such as French or Russian, but in the mean time I provide this shortcut.

The new software released this week to all Wikimedia sites and to translatewiki.net makes it possible to type these special characters without installing any software or pressing any combining keys such as Ctrl or Alt. In most cases you simply need to press the tilde character (~) followed by the letter that is similar to the one you want to type. For example:

  • Ɓ is written using ~B
  • Ɛ is written using ~E
  • Ɔ is written using ~O
    … and so on.

Some of these languages are written in their own unique writing systems. N’Ko and Vai keyboards were made by myself, mostly based on ideas from freely licensed keyboard layouts by Keyman. (A keyboard for the Amharic language, also written with its own script, has had keyboards made by User:Elfalem for a while. I am mentioning it here for completeness.)

This release addresses only laptop and desktop computers. On mobile phones and tablets most of these languages can be typed using apps such as Gboard (also in iPhone), SwiftKey (also on iPhone), or African Keyboard. If you aren’t doing this already, try these apps on your phone, and start exchanging messages with your friends and family in your language, and writing in Wikipedia in your language on your phone! If you are having difficulties doing this, please contact me and I’ll do my best to help.

The technology used to make this is the MediaWiki ULS extension and the jquery.ime package.

I would like to thank all the people who helped:

  • Mahuton Possoupe (Benin), with whom I made the first of these keyboards, for the Fon language, at the Barcelona Hackathon.
  • Kartik Mistry, Santhosh Thottingal (India), Niklas Laxström (Finland), and Petar Petkovich (Serbia), who reviewed the numerous code patches that I made for this project.

This is quite a big release or code. While I made quite a lot of effort to test everything, code may always have bugs: missing languages, wrong or missing letters, mistakes in documentation, and so on. I’ll be happy to hear any feedback and to fix the bugs.

And now it’s all up to you! I hope that these keyboard layouts make it easier for all of you, African Wikimedians, to write in your languages, to write and translate articles, and share more knowledge!

Again, happy Africa day!

The full list of languages for which there is now a keyboard in ULS and jquery.ime:

  • Afrikaans
  • Akan
  • Amharic
  • Bambara
  • Berber
  • Dagbani
  • Dinka
  • Ewe
  • Fula
  • Fon
  • Ga
  • Hausa
  • Igbo
  • Kabiye
  • Kabyle
  • Kikuyu
  • Luganda
  • Lingala
  • Malagasy
  • N’Ko
  • Sango
  • Sotho
  • Northern Sotho
  • Koyraboro Senni Songhay
  • Tigrinya
  • Vai
  • Venda
  • Wolof
  • Yoruba

Disease of Familiarity, the Flaw of Wikipedia

Originally written as an answer to the question What are some major flaws in Wikipedia? on Quora. Republished here with some changes.

Wikipedia has a whole lot of flaws, and its basic meta-flaw is the disease of familiarity.

It does not mean what you think it means. The disease of familiarity is knowing so much about something that you don’t understand what it is like to not understand it.

I recognized this phenomenon in 2011 or so, and called it The Software Localization Paradox. I later realized that it has a lot of other aspects beyond software localization, so I thought a lot about it and struggled for years with giving it a name. I learned about the term “disease of familiarity” from Richard Saul Wurman, best known as the creator of the TED conference (see a note about it at the end of this post). Some other names for this phenomenon are “curse of knowledge” and “mind blindness”. See also Is there a name for “knowing so much about something that you don’t understand what is it like not to know it”?

Unfortunately, none of these terms is very famous, and their meaning is not obvious without some explanation. What’s even worse, the phenomenon is in general hard to explain because of its very nature. But I’ll try to give a few examples.


Wikipedia doesn’t make it easy for people to understand its jargon.

Wikipedia calls itself “The Free encyclopedia”; what does it mean that it’s “free”? I wrote Wikipedia:The Free Encyclopedia, one of the essays on this topic (there are others), but it’s not official or authoritative, and more importantly, the fact that this essay exists doesn’t mean that everybody who starts writing for Wikipedia reads it and understands the ideology behind it, and its implications. An important implication of this ideology is that according to the ideology of the Free Culture movement, of which Wikipedia is a part, is that some images and pieces of text can be copied from other sites into Wikipedia, and some cannot. The main reason for this is copyright law. People often copy text or images that are not compatible with the policies, and since this is heavily enforced by experienced Wikipedia editors, this causes misunderstandings. Wikipedia’s interface could communicate these policies better, but experienced Wikipedians, who already know them, rarely think about this problem. Disease of familiarity.

Wikipedia calls itself “a wiki”. A lot of people think that it’s just a meaningless catchy brand name, like “Kodak”. Some others think that it refers to the markup language in which the site is written. Yet others think that it’s an acronym that means “what I know is”. None of these interpretations is correct. The actual meaning of “wiki” is “a website that anyone can edit”. The people who are experienced with editing Wikipedia know this, and assume that everybody else does, but the truth is that a lot of new people don’t understand it and are afraid of editing pages that others had written, or freak out when somebody edits what they had written. Disease of familiarity.

The most common, built-in way for communication between the different Wikipedians is the talk page. Only Wikipedia and other sites that use the MediaWiki software use the term “talk page”. Other sites call such a thing “forum”, “comments”, or “discussion”. (To make things more confusing, Wikipedia itself occasionally calls it “discussion”.) Furthermore, talk pages, which started on Wikipedia in 2001, before commenting systems like Disqus, phpBB, Facebook, or Reddit were common, work in a very weird way: you need to manually indent each of your posts, you need to manually sign your name, and you need to use a lot of obscure markup and templates (“what are templates?!”, every new user must wonder). Experienced editors are so accustomed to doing this that they assume that everybody knows this. Disease of familiarity.

A lot of pages in Wikipedia in English and in many other languages have infoboxes. For example, in articles about cities and towns there’s an infobox that shows a photo, the name of the mayor, the population, etc. When you’re writing an article about your town, you’ll want to insert an infobox. Which button do you use to do this? There’s no “Infobox” button, and even if there were, you wouldn’t know that you need to look for it because “Infobox” is a word in Wikipedia’s internal jargon. What you actually have to do is Insert → Template → type “Infobox settlement”, and fill a form. Every step here is non-intuitive, especially the part where you have to type the template’s name. Where are you supposed to know it from? Also, these steps are how it works on the English Wikipedia, and in other languages it works differently. Disease of familiarity.

And this brings us to the next big topic: Language.

You see, when I talk about Wikipedia, I talk about Wikipedia in all languages at once. Otherwise, I talk about the English Wikipedia, the Japanese Wikipedia, the Arabic Wikipedia, and so on. Most people are not like me: when they talk about Wikipedia, they talk about the one in the language in which they read most often. Quite often it’s not their first language; for example, a whole lot of people read the Wikipedia in English even though English is their second language and they don’t even know that there is a Wikipedia in their own language. When these people say “Wikipedia” they actually mean “the English Wikipedia”.

There’s nothing bad in it by itself. It’s usually natural to read in a language that you know best and not to care very much about other languages.

But here’s where it gets complicated: Technically, there are editions of Wikipedia in about 300 languages. This number is pretty meaningless, however: There are about 7,000 languages in the world, so not the whole world is covered, and only in 100 languages or so there is a Wikipedia in which there is actually some continuous writing activity. In the other 200 the activity is only sporadic, or there is no activity at all—somebody just started writing something in that language, and a domain was created, but then the first people who started it lost interest and nobody else came to continue their work.

This is pretty sad because it’s frequently forgotten that a whole lot of people cannot read what they want in Wikipedia because they don’t know a language in which there is an article about what they want to learn. If you are reading this post, you have the privilege of knowing English, and it’s hard for you to imagine how does a person who doesn’t know English feel. Disease of familiarity: You think you can tell everybody “if you want to know something, read about it in Wikipedia”, but you cannot actually tell this to most people because most people don’t know English.

The missed opportunity becomes even more horrific when you realize that the people who would have the most appropriate skills for breaking out of this paradox are the people who are least likely to notice it, and the people who are hurt by it the most are the least capable of fixing it themselves. Think about it:

  • If you know, for example, Russian and English, and you need to read about a topic on which there is an article in the English Wikipedia, but not in Russian, you can read the English Wikipedia, and it’s possible that you won’t even notice that an article in Russian doesn’t exist. Unless you exercise mindfulness about the issue, you won’t empathize with people who don’t know English. To break out of this cycle, one can practice the following:
    • Always look for articles in Russian first.
    • Dedicate some time every week to translating articles. (See How does Wikipedia handle page translation?)
    • When you talk to people in your language, don’t assume that they know English.
  • A person who doesn’t know English is just stuck without an article, and there’s not much to do. It’s possible that you don’t even know that the article you need exists in another language. And maybe you cannot even read the user manual that teaches you how to edit. What can you do?
    • Try to be bold and ask your friends who do know English to translate it for you and publish the translation for the benefit of all the people who speak your language.
    • (Of course, there’s the solution of learning English, but we can’t assume that it works. Evidently, there are billions of people who don’t know English, and they won’t all learn English any time soon.)

(In case it isn’t clear, you can replace “English” and “Russian” in the example above with any other pair of languages.)

It’s particularly painful in countries where English, French, or Portuguese is the dominant language of government and education, even though a lot of the people, often the majority, don’t actually know it. This is true for many countries in Africa, as well as for Philippines, and to a certain extent also in India and Pakistan.

People who know English have a very useful aid for their school studies in the form of Wikipedia. People who don’t know English are left behind: the teachers don’t have Wikipedia to get help with planning the lessons and the students don’t have Wikipedia to get help with homework. The people who know English and study in English-medium schools have these things and don’t even notice how the other people—often their friends!—are left behind. Disease of familiarity.

Finally, most of the people who write in the 70 or so most successful Wikipedias don’t quite realize that the reason the Wikipedia in their language is successful is that before they had a Wikipedia, they had had another printed or digital encyclopedia, possibly more than one; and they had public libraries, and schools, and universities, and all those other things, which allowed them to imagine quite easily how would a free encyclopedia look like. A lot of languages have never had these things, and a Wikipedia would be the first major collection of educational materials in them. This would be pretty awesome, but this develops very slowly. People who write in the successful Wikipedia projects don’t realize that they just had to take the same concepts they already knew well and rebuild them in cyberspace, without having to jump through any conceptual epistemological hoops.

Disease of familiarity.


It’s hard to explain this.

I unfortunately suspect that very few, if any, people will understand this boring, long, and conceptually difficult post. If you disagree, please comment. If you think that you understand what I’m trying to say, but you have a simpler or shorter way to say it, please comment or suggest an edit (and tell your friends). If you have more examples of the disease of familiarity in Wikipedia and elsewhere, please speak up.

Thank you.


(As promised above, a note about Richard Saul Wurman. I heard him introduce the “disease of familiarity” concept in an interview with Debbie Millman on her podcast Design Matters, at about 23 minutes in. That interview was one of this podcast’s weirdest episodes: you can clearly hear that he’s making Millman uncomfortable, and she also mentioned it on Twitter. This, in turn, makes me uncomfortable to discuss something I learned from that interview, but I am just unable to find any better terminology for the phenomenon in question. If you have suggestions, please send them my way.)


Disclaimer: I’m a contractor working with the Wikimedia Foundation, but this post, as well as all my other posts on the topic of Wikimedia, Wikipedia, and related projects, are my own opinions and do not represent the Wikimedia Foundation.

There’s Nothing Particularly Good About Long Wikipedia Articles. Let’s Make Them Shorter

Wikipedia used to have a warning about articles of a certain size. If I recall correctly, it was 64KB. As far as I understand, the reason for this was more engineering-oriented than user-experience-oriented: Loading a larger page was slower, because networks were slower, or at least so some people thought.

Wikipedia no longer has this warning. It’s not unusual to have a page of 250KB or more. I don’t participate in discussions about performance, but the discussions that I do see are about the time that it takes to parse the templates server-side, to load JavaScript modules, and to render the CSS; they are not so much about the kilobyte size of the pages themselves.

I suspect, however, that there is a problem with page length. Not one of performance engineering, but of user experience. Do people actually read whole encyclopedic articles in Wikipedia? In case you haven’t guessed it already, my hypothesis is that most people don’t.

This is my hypothesis because of the famous debunking of a designer myth: people usually don’t read texts.

It should be clarified right away that the notion that people don’t read whole Wikipedia article is not, by itself, a problem. It may be a bit sad for people who invest hours (or years!) in writing the brilliant prose of each excellent article, but the point of Wikipedia is not supposed to be getting millions of people to read very long articles. Rather, it’s making information that they need accessible, and making it as easy as possible for everybody to edit this information.

Do long articles make finding information easy? Probably not. Experienced Wikipedia editors are familiar with article structure, with tricks like Find in Page, and so on, but a lot of readers are not.

So here’s my call: Let’s bring back article length warning in some form. The importance of a topic doesn’t necessarily justify having a very long article about it. The purpose is not to have a long page, but to make information easy to find. If splitting an article to several pages makes the information easier to find, then the readers will of course be happy, and the editors who invest their effort in writing a lot about a topic should be happy, too, because their writing is more likely to be actually read.

Wikimedia Strategy Phase 1: What Does It Mean for Me and (Maybe) for Language Diversity in Wikipedia

The Wikimedia Foundation is leading a process to write a strategy for the Wikimedia movement. This process takes over a year. A few months ago, the conclusion of Phase 1 of this process was published: The strategic direction.

Some central concepts in this document are “knowledge as a service” and “knowledge equity”. Some people said that it’s too vague and high-level, and that it can be interpreted in a lot of ways. This is true, especially in a movement that is as culturally and linguistically diverse as Wikimedia. Perhaps this is intentional, so that people will be able to interpret this in any way that feels right for them.

Recently I was filling a registration form for Wikimedia Conference 2018. This form was very long, and it asked what do the concepts that appear in the strategic direction document mean to me. My answers were longish, and since there’s nothing secret about them, and they may (or may not) interest some people, I copied them from the form to this blog post. I edited them slightly for publishing here so that the context will be clearer, but the essence is the same as what I submitted.

Knowledge as a service

The knowledge that Wikimedia projects already contain is available through all common channels of communication: in addition to being available on the website, it must be findable on all search engines in all languages and countries, browsable on devices of all operating systems whether open or not, browsable as much as possible through social networks and chat applications, embeddable in other apps, etc.

It must be easy for all people, whether they are knowledgeable about computers or not, to contribute their knowledge to Wikimedia sites, and humanity in general should know that Wikimedia sites is the place where they contribute their knowledge and not only learn it.

Knowledge equity

What it means to me is:

  • That all people, of all ages and all kinds of identities, of all countries, who speak all languages, must be able to read and write in their language.
  • That we will fight whenever it’s reasonable against censorship and against all kinds of chilling effects that deter potential contributors or threaten their well-being.
  • That we remain independent of commercial and political entities by strictly refusing to carry political and commercial advertising and to accept unreasonable limited grants.
  • That all the software that is useful for reading and writing on our sites must be easily usable in all languages, whether it’s core software, extensions, templates, or gadgets.
  • That we don’t depend on any non-Free or otherwise unethical software, even if it appears to make consuming and contributing knowledge easier.
  • That we set a goal of having good coverage for core content in all languages and actively pursue it and not leave it only to the community’s “invisible hand”.
  • That we set a goal that the most popular Wikimedia projects in each country are in that country’s most spoken languages and not in a foreign language.

What kind of conditions do you need to realize these activities?

Describe what you think would be good conditions for you to move forward in this direction. Think of conditions in the broadest sense; e.g., capacity, skills, partnerships, clarification, structures and processes, room for development or experimentation, financial resources, people, access to other means of support etc.

We need to partner with academic institutions that work on topics that are not currently covered by our projects because of systemic bias.

We need to partner more with organizations that have expertise in developing minorized and under-resourced languages, working on the ground in the countries where these languages are spoken.

We need easy access to data about the social and political situations in poorer countries, and if such data doesn’t exist at all, we need to lead research that creates such data ourselves.

We need a new attitude to developing software for our sites: we need to understand what do our communities actually do on the sites with gadgets and templates rather than just developing new extensions that may be shiny, but are hard to integrate into the sites, each of which is heavily customized.

What I wrote in that form is a good description of my current attitude to what the priorities of Wikimedia movement should be, at least in terms of ideology and values. You can clearly see my interests: remembering that language support is important and that most people don’t speak English; remembering that we are not supposed to be an American non-profit organization, but an international movement that happens to have an office in the U.S.; remembering that we are also a part of the Free Software movement; remembering that good software engineering are important, even if engineering alone can’t solve all the problems.

For people who have doubts: This post represents my own opinions, and doesn’t express the opinion of the Wikimedia Foundation or any of its employees or managers.

The Curious Problem of Belarusian and Igbo in Twitter and Bing Translation

Twitter sometimes offers machine translation for tweets that are not written in the language that I chose in my preferences. Usually I have Hebrew chosen, but for writing this post I temporarily switched to English.

Here’s an example where it works pretty well. I see a tweet written in French, and a little “Translate from French” link:

Emmanuel Macron on Twitter.png

The translation is not perfect English, but it’s good enough; I never expect machine translation to have perfect grammar, vocabulary, and word order.

Now, out of curiosity I happen to follow a lot of people and organizations who tweet in the Belarusian language. It’s the official language of the country of Belarus, and it’s very closely related to Russian and Ukrainian. All three languages have similar grammar and share a lot of basic vocabulary, and all are written in the Cyrillic alphabet. However, the actual spelling rules are very different in each of them, and they use slightly different variants of Cyrillic: only Russian uses the letter ⟨ъ⟩; only Belarusian uses ⟨ў⟩; only Ukrainian uses ⟨є⟩.

Despite this, Bing gets totally confused when it sees tweets in the Belarusian language. Here’s an example form the Euroradio account:

Еўрарадыё   euroradio    Twitter double.pngBoth tweets are written in Belarusian. Both of them have the letter ⟨ў⟩, which is used only in Belarusian, and never in Ukrainian and Russian. The letter ⟨ў⟩ is also used in Uzbek, but Uzbek never uses the letter ⟨і⟩. If a text uses both ⟨ў⟩ and ⟨і⟩, you can be certain that it’s written in Belarusian.

And yet, Twitter’s machine translation suggests to translate the top tweet from Ukrainian, and the bottom one from Russian!

An even stranger thing happens when you actually try to translate it:

Еўрарадыё   euroradio    Twitter single Russian.pngNotice two weird things here:

  1. After clicking, “Ukrainian” turned into “Russian”!
  2. Since the text is actually written in Belarusian, trying to translate it as if it was Russian is futile. The actual output is mostly a transliteration of the Belarusian text, and it’s completely useless. You can notice how the letter ⟨ў⟩ cannot be transliterated.

Something similar happens with the Igbo language, spoken by more than 20 million people in Nigeria and other places in Western Africa:

 4  Tweets with replies by Ntụ Agbasa   blossomozurumba    Twitter.png

This is written in Igbo by Blossom Ozurumba, a Nigerian Wikipedia editor, whom I have the pleasure of knowing in real life. Twitter identifies this as Vietnamese—a language of South-East Asia.

The reason for this might be that both Vietnamese and Igbo happen to be written in the Latin alphabet with addition of diacritical marks, one of the most common of which is the dot below, such as in the words ibụọla in this Igbo tweet, and the word chọn lọc in Vietnamese. However, other than this incidental and superficial similarity, the languages are completely unrelated. Identifying that a text is written in a certain language only by this feature is really not great.

If I paste the text of the tweet, “Nwoke ọma, ibụọla chi?”, into translate.bing.com, it is auto-identified as Italian, probably because it includes the word chi, and a word that is written identically happens to be very common in Italian. Of course, Bing fails to translate everything else in the Tweet, but this does show a curious thing: Even though the same translation engine is used on both sites, the language of the same text is identified differently.

How could this be resolved?

Neither Belarusian nor Igbo languages are supported by Bing. If Bing is the only machine translation engine that Twitter can use, it would be better to just skip it completely and not to offer any translation, than to offer this strange and meaningless thing. Of course, Bing could start supporting Belarusian; it has a smaller online presence than Russian and Ukrainian, but their grammar is so similar, that it shouldn’t be that hard. But what to do until that happens?

In Wikipedia’s Content Translation, we don’t give exclusivity to any machine translation backend, and we provide whatever we can, legally and technically. At the moment we have Apertium, Yandex, and YouDao, in languages that support them, and we may connect to more machine translation services in the future. In theory, Twitter could do the same and use another machine translation service that does support the Belarusian language, such as Yandex, Google, or Apertium, which started supporting Belarusian recently. This may be more a matter of legal and business decisions than a matter of engineering.

Another thing for Twitter to try is to let users specify in which languages do they write. Currently, Twitter’s preferences only allow selecting one language, and that is the language in which Twitter’s own user interface will appear. It could also let the user say explicitly in which languages do they write. This would make language identification easier for machine translation engines. It would also make some business sense, because it would be useful for researchers and marketers. Of course, it must not be mandatory, because people may want to avoid providing too much identifying information.

If Twitter or Bing Translation were free software projects with a public bug tracking system, I’d post this as a bug report. Given that they aren’t, I can only hope that somebody from Twitter or Microsoft will read it and fix these issues some day. Machine translation can be useful, and in fact Bing often surprises me with the quality of its translation, but it has silly bugs, too.

Five More Privileges of English Speakers, part 2: Language and Software

For the previous part in the series, see Five Privileges of English Speakers, part 1.

I’m continuing the series of posts in each of which I write about five privileges that English speakers have without giving it a lot of thought. The examples I give mostly come from my experience translating software, Wikipedia articles, blog posts, and some other texts between English, Hebrew, and Russian. Hebrew and Russian are the languages I know best. If you have interesting examples from other languages, I am very interested in hearing them and writing about them.

I’m writing them mostly as they come into my mind, without a particular order, but the five items in this part of the series will focus on usage of the English language in software, and try to show that the dominance of English is not only a consequence of economics and history, but that it’s further reinforced by features of the language itself.

1. Software usually begins its life in English

English is the main language of software development worldwide.

The world’s best-known place for software development is Silicon Valley, an English-speaking place. That’s the place of Facebook, Google, Apple, Oracle and many others. California is also the home of Adobe.

There are several other hubs of software development in United States: Seattle (Microsoft, Amazon), North Carolina (Red Hat), New York (IBM, CA), Massachusets (TripAdvisor, Lotus, RSA), and more. The U.S. is also the source for much of computer science research and education, coming from Berkeley, MIT, and plenty of other schools. The U.S. is also the birthplace of the Internet, originally supported by the U.S. Department of Defense and several American universities. The world wide web, which brought the Internet to the masses, was created in Switzerland by an English speaker.

Software is developed in other countries—India, Russia, Israel, France, Germany, Estonia, and many other countries. But the dominance of the U.S. and of the English language is clear. The reason for this is not only that the U.S. is the source for much of computer technologies, but also—and probably more importantly—that the U.S. is the biggest consumer market for software. So developers in all countries tend to optimize the product for the highest-paying consumers, and these only need English.

When engineers write the user interface of their software in English, they often do not give any thought to other languages at all, or make translation possible, but complicated by English-centric assumptions about number, gender, text direction, text size, personal names, and plenty of other things, which will be explored in further points.

2. Terminology

English is also the source for much of the computer world’s terminology. Other languages have to adapt terms like smartphone, network, token, download, authentication, and thousands of others.

Some language communities work hard to translate them all meticulously into native words; Icelandic, Lithuanian, French, Chinese, and Croatian are famous examples. This is nice, but requires effort on behalf of terminology committees, who need to keep up with the fast pace of technological development, and on behalf of the software translators, who have to keep with the committees.

Some just transliterate most of them: keep the term essentially in English, but rewritten in the native alphabet. Hindi and Japanese are examples of that. This seems easy, but it is based on a problematic assumption: that the target language speakers who will use the software know at least some English! This assumption is correct for the translators, who don’t just know the English terms, but are probably also quite accustomed to it, but it’s not necessarily correct for the end users. Thus, the privilege is perpetuated.

Some languages, such as Hebrew, German, and Russian, are mid-way, with language academics and purists pulling to purer native language, engineers pulling to more English-based words, and the general public settling somewhere in between—accepting the neologisms for some terms, and going for English-based words for others.

For the non-English languages it provides fertile ground for arguments between purists and realists, in which the needs of the actual users are frequently forgotten. All the while, English speakers are not even aware of all this.

3. Easy binary logic word formation

One particular area of computer terminology is binary logic. This sounds complicated, but it’s actually simple: in electronics and software opposite notions such as true / false, success / failure, OK / Cancel, and so forth, are very common.

This translates to a great need for words that express opposites: enable / disable, do / undo, log in / log out, delete / undelete, block / unblock, select / deselect, online / offline, connect / disconnect, read / unread, configured / misconfigured.

Notice something? All of the above words are formed with the same root, with the addition of a prefix (un-, dis-, de-, mis-, a-), or with the words “on” and “off”.

A distinct, but closely related need, is words for repetition. Computers are famously good at doing things again and again, and that’s where the prefix re- is handy: reconnect, retry, redo, retransmit.

These features happen to be conveniently built into the English language. While English has extremely simple morphology for declension and conjugation (see the section “Spell-checking” in part 1 of the series), it has a slightly more complex morphology for word formation, but it’s still fairly easy.

It is also productive. That is, a software developer can create new words using it. For example, the MediaWiki software has the concept of “oversight”—hiding a problematic page in such a way that only users with a particular permission can read it. What happens if a page was hidden by mistake? Correct: “unoversight”. This word doesn’t quite exist elsewhere, but it doesn’t sound incorrect, because familiar English word formation rules were used to coin it.

As it always happens, English-speaking software engineers either don’t think about it at all, or think that other languages also have similar word formation rules. If you haven’t guessed it already, it is not true. Sime other European languages have similar constructs, but not necessarily as consistent as in English. And for Semitic languages like Hebrew it’s a disaster, because in Semitic languages prefixes are used for entirely different things, and the grammar doesn’t have constructs for repetition and negation. So when translating software user interface strings into Hebrew, we have to use different words as opposites. For example the English pair connect / disconnect is translated as lehitḥabér / lehitnaték—completely different roots, which Hebrew is just lucky to have. Another option is to use negative words like lo and bilti, or bitul, but they are often unnatural or outright wrong. Having to deal with something like “Mark as unread” is every Hebrew software translator’s nightmare, even though it sounds pretty straightforward in English.

English itself also has pairs of negative words that are not formed using the above prefixes, for example next / previous and open / close, but in many other languages they are much more common.

4. Verbing

“Verbing weirds language”, as one of the famous Calvin and Hobbes panels says.

Despite being a funny joke in the comic, it’s a real feature of the English language: because of how English morphology and syntax work, nouns can easily jump into the roles of adjectives and verbs without changing the way they are written.

For English, this is a useful simplification, and it works in labeling, as well as in advertising. “Enjoy Coca-Cola” is something more than an imperative. The fact that it’s a short single word and that it’s the same in all genders and numbers, makes it more usable as a call to action than it would be in other languages. And, other than advertising, where are calls to action very common? Software, of course. When you’re trying to tell a user to do something, a word that happens to be both the abstract concept and the imperative is quite useful.

Perhaps the most famous example of this these days is Facebook’s “Like”. Grammatically, what is it in English? Imperative? A noun describing an abstract action? Maybe a plain old noun, as in “chasing likes” (this is a plural noun—English verb don’t have a plural form!)? Answer: it’s all of them and more.

When translated to Hebrew in Facebook’s interface, it’s Ahávti, which literally means “I loved it”. Actually, this translation is mostly good, because it’s understandable, idiomatic, and colloquial enough without compromising correctness. Still, it’s a verb, which is not imperative, and it’s definitely not a noun, so you cannot use it in a sentence as if it was a noun. Indeed, Hebrew speakers are comfortable using this button, but when they speak and write about this feature, they just use its English name: “like” (in plural láykim). It even became a slightly awkward, but commonly used verb: lelaykék. Something similar happens in Russian.

It would be impossible in Hebrew and Russian to use the exact same word for the noun and the verb, especially in different persons and genders. Sometimes the languages are lucky enough to be able to adapt an English verb in a way that is more or less natural, but sometimes it’s weird, and hurts the user experience.

5. Word length

This one is relatively simple and not unique to English, but should be mentioned anyway: English words are neither very long, nor very short. Examples of languages where words are, on average, longer than in English, are Finnish, Tamil, German, and occasionally Russian. Hebrew tends to be shorter, although sometimes a single English word has to be translated with several Hebrew words, so it can get also get longer. This is true for a pretty much any language, really.

In designing interfaces, especially for smaller screens, the length of the text is often important. If a button label is too long, it may overflow from the button, or be truncated, making the display ugly, or unusable, or both.

If you’re an English speaker, it probably won’t happen with you, because almost all software is usually designed with the word length of your language in mind. Other languages are almost always an afterthought.

The good practice for software engineers and designers is to make sure that translated strings can be longer. Their being shorter is rarely a problem, although sometimes a string is so short that the button may become to small to click or tap conveniently.


Generally, what can you do about these privileges?

Whoever you are, remember it. If you know English, you are privileged: Software is designed more for you than for people who speak other languages.

If you are a software engineer or a designer, at the very least, make your software translatable. Try to stick to good internationalization practices and to standards like Unicode and CLDR. Write explanations for every translatable string in as much detail as possible. Listen to users’ and translators’ complaints patiently—they are not whining, they are trying to improve your software! The more internationalizable it is, the more robust it is for you as a developer, and for your English-speaking users, too, because better design thinking will be going into each of its components, and less problematic assumptions will be made.


Archives

Advertisements