Aharoni in Unicode, ya mama

Always define the language and the direction of your HTML documents, part 02: Backwards English

aharoni — Sat, 18 May 2013 12:14:26 +0000

In part 01 of these series, I showed why is it important to always define the language and the direction of all HTML content and not rely on the defaults: The content may get embedded in a document with different direction and be displayed incorrectly.

This issue is laughably easy to avoid: If you are writing the content, you are supposed to know in what language it is written, so if it’s English, just write even though these seem to be the defaults. Nineteen or so characters that ensure your content is readable and not displayed backwards. Please do it always and tell all your friends to do it.

The problem is that you don’t only have to explicitly set the language and the direction, but, as silly as it sounds, you have to set them correctly, too. A more subtle, but nevertheless quite frequent and disruptive bug is displaying presumably, but not actually, translated content in a different direction. This happens quite frequently when a website supports the browser language detection feature, known as Accept-Language:

The web server sees that the browser requests content in Hebrew.
The web server sends a response with , but because the website is not actually translated, the text is shown in the fallback language, which is usually English.
The user sees the content just like this numbered list, which I intentionally set to dir=”rtl”: with the numbers and the punctuation on the wrong side, and possibly invisible, because English is not a right-to-left language.

Of course, it can go even worse. Arrows can point the wrong way and buttons and images can overlap and hide each other, rendering the page not just hard to read, but totally unusable.

This bug is also an example of the Software Localization Paradox: It manifests itself when Accept-Language is not English, but most developers install English operating systems and don’t bother to change the preferred language settings in the browser, so they never see how this bug manifests itself. The site developers don’t bother to test for it either.

The solution, of course, is to set a different language and direction only if the site is actually translated, and not to pretend that it’s translated if it’s not.

Here are two examples of such brokenness. Both sites are important and useful, but hard to use for people whose Accept-Language is Hebrew, Persian or Arabic.

Here’s how the Mozilla Developer Network website looks in fake Hebrew:

Mozilla Developer Network website, in English, but right-to-left

Notice how the full stops are on the left end and how the text overlaps the images in the tiles on the right-hand side. This is how it is supposed to look, more or less:

Mozilla Developer Network home page in English, left-to-right

I manually changed dir=”rtl” to dir=”ltr” using the element inspector from Firefox’s developer tools and I also had to tweak a CSS class to move the “mozilla” tab at the top.

The above troubles are reported as bug 816443 – lang and dir attributes must be used only if the page is actually translated.

After showing an example of a web development bug from a site for, ahem, web developers, here is an even funnier example: The home page of Unicode’s CLDR. That’s right: Unicode’s own website shows text with incorrect direction:

The Unicode CLDR website, in English but right-to-left

The only words translated here are “Contents” (תוכן) and “Search this site” (חיפוש באתר זה), which is not so useful. The rest is shown in English, and the direction is broken: Notice the strange alignment of the content and the schedule table. A few months ago that table was so broken that its content wasn’t visible at all, but that was probably patched.

Here’s how it is supposed to look:

The CLDR home page in English, appropriately left-to-right

I tried reporting the CLDR home page direction bug, but it was closed as “out-of-scope”: The CLDR developers say that the Google Sites infrastructure is to blame. This is frustrating, because as far as I know Google Sites doesn’t have a proper bug reporting system and all I can do is write a question about that direction problem in the Google Sites forum and hope that somebody notices it or poke my Googler friends.

One thing that I will not do is switch my Accept-Language to English. Whenever I can, I don’t just want to see the website correctly, but to try to help my neighbor: see the possible problems that can affect other users who use different language. Somebody has to break the Software Localization Paradox.

Filed under: Firefox, Free Software, localization

Marriage in Dictionaries

aharoni — Thu, 18 Apr 2013 12:22:15 +0000

The definition of marriage is the hottest topic in US news lately.

My favorite place for looking up definitions of English words is, unsurprisingly, the Merriam-Webster dictionary.

And indeed, the editors of M-W’s website noticed the public interest in the definition of marriage, and here’s what they had to write about it:

The word became the subject of renewed scrutiny as the Supreme Court heard arguments in cases seeking to overturn California’s ban on gay marriage and the federal government’s Defense of Marriage Act.

Marriage has become a controversial definition, although its original sense – “the state of being united to a person of the opposite sex” – has not changed.

However, because the word is used in phrases such as “same-sex marriage” and “gay marriage” (by proponents and opponents alike), a second definition – “the state of being united to a person of the same sex in a relationship like that of a traditional marriage” – was added to the dictionary to provide an accurate picture of the word’s current use.

I recently read Herbert Morton’s excellent book The Story of Webster’s Third: Philip Gove’s Controversial Dictionary and Its Critics. It’s excellent because it’s very well written and because it could be a handbook in how to make dictionaries in general: how to balance scientific linguistic precision with usefulness to the general public.

Sadly, this remark about the definition of marriage is a departure from the principles of excellence that guided the editors of Webster’s Third. If the sentence says “same-sex marriage”, then “same-sex” means, literally, “same-sex”; there’s no need to say “the state of being united to a person of the same sex“.

Why not just say that “marriage” is “the state of being united to a person”? Maybe “legally united”, or “religiously united”. Or “united in a family”. It neatly avoids the political problems around sex and gender and all that, and is correct linguistically.

The official dictionary of the Catalan language already did it:

Comparison of two versions of a dictionary definition in the Catalan language.

The Institute of Catalan Studies, which publishes the dictionary, also publishes a list of updates in each edition. In this image you can see how the definition of marriage changed from “a legal union of a man and a woman” to “a legitimate union of two people who promise each other a common life, established through certain rituals or legal formalities”. The last usage example also says: “In some countries the legislation provides for marriage between two persons of the same sex”.

And well, yes, before you ask: of course there is a political background. Catalonia was one of the first jurisdictions that made same-sex marriage equal to different-sex marriage. But from the purely linguistic point of view the newer definition, which doesn’t mention a man and a woman, is perfectly correct. And saying that the definition of “marriage” is different in “marriage” and in “same-sex marriage” is not correct. Simple, really.

Filed under: Catalan, gay, gender, linguistics, sex, USA

The Fateful March of 1998 – my #webstory

aharoni — Sun, 07 Apr 2013 10:06:53 +0000

I first connected to the web in the summer of 1997. I bought a new computer with Windows 95 and Microsoft Internet Explorer 2. For about a week I thought that that’s how the web is supposed to look, but I kept seeing messages saying “Your browser doesn’t support frames” on a lot of sites. And then I found that there’s this thing called Microsoft Internet Explorer 3. I went to microsoft.com and downloaded it. It was the first piece of software that I downloaded. It was about 10 megabytes and took about an hour on my dial-up connection.

Most notably, Microsoft Internet Explorer 3 supported frames and animated GIFs. I loved animated GIFs! I guess that it makes me quite a hipster.

House cat. Sorry, it’s an anachronism— this animated GIF is from mid-2000s. 1997′s animated GIFs were quite different.

And then Microsoft Internet Explorer 4 came out. I thought—”well, if the move from IE2 to IE3 made such a big difference, then I guess that I should try number 4, and it will be even cooler”. And I tried. And it was a disaster. The installation screwed up everything on my computer. I had no idea how to disable the dreaded Active Desktop, which it introduced. It didn’t work so well with my Hebrew version of Windows 95. So I did what a lot of people did very often back then and formatted my hard drive and re-installed Windows.

And the question arose—which browser should I use? IE3 was stable, but I didn’t like that it was getting old. So I went to netscape.com, to try that Netscape Navigator browser that I kept hearing everybody talking about it.

And I loved it.

I loved its nifty toolbars and its bookmarks manager. I loved the crash reporting; it crashed quite often, actually, but I didn’t feel so bad about it, because Microsoft’s programs crashed often, too, and in case of Netscape I felt good about reporting these crashes. Netscape’s email program, Netscape Messenger, was truly outstanding. I especially loved the green dot, which marked messages as read and unread in one click. Most of all, it said very clearly something that I came to realize only years later: “I am a program that lets you browse the web as well as possible. I am not trying to do anything else.”

Fast forward to March 1998. Netscape made the big announcement that the development of its browser becomes an open source project code-named “Mozilla”. I started hearing about “open source”, “free software” and Linux shortly before that, but it was mostly in the context of crazy geek hobbyists. And then suddenly a big famous end-user product that I love becomes open source—that felt really cool.

I followed Mozilla news since then. I heard about Bugzilla before its first version was released. I liked Mozilla’s decision to redo the whole rendering based on standards, even though many people criticized it. The thing that annoyed me the most in Mozilla’s early years was the lack of support for proper right-to-left text support, which was present in Internet Explorer. That’s why I, sadly, used mostly IE, and even became a bit of an IE power user. But I waited eagerly for Mozilla to do it and tried every alpha release.

The famous New York Times ad.

I was thrilled about the announcement of Firefox, the first stable version of Mozilla’s browser. I gave 10$ to the famous 2004 New York Times Firefox advertisement, and I still have the poster of that advertisement at home.

And there’s my name. Third line in the middle.

It always seemed natural to me that I follow Mozilla news so eagerly. I thought that everybody does it. I mean, how is it even possible to use the web in any way without being at least a bit curious about the technology that runs it?

And then in 2008 I wrote a little unimportant post in my Hebrew blog about a funny spelling correction. Tomer Cohen commented on it and suggested me to try the Hebrew spelling dictionary and Hebrew Firefox in general. And that’s how my big love story with software localization began.

I started sending corrections to the translation of Firefox’s interface translation. I started sending corrections to the Hebrew spelling dictionary. I got so curious about the way the spelling dictionary was built that I ended up doing a whole university degree in Hebrew Language. Really.

And in 2011 I started working in the Language Engineering team in the Wikimedia Foundation. I love it, and it probably wouldn’t have happened without my involvement with Mozilla. In the same year I also became a Mozilla Rep—a volunteer representative of Mozilla at conferences, blogs and forums.

Probably the most important thing that I learned from my Mozilla story is that loving the web and being curious about it is not something obvious. Most people just want something that works for checking weather, news, Facebook friends updates, homework help and kitten videos. And for the most part, that is perfectly fine. But the people’s freedom to read reliable and complete news on any electronic device cannot actually be taken for granted. Neither the people’s freedom and privacy to share their thoughts in social networks. Mozilla is among the most important organizations that care for these things and it develops technologies that make them possible. Technologies that let you browse the web as well as possible and don’t try to do anything else.

We do it for one simple reason: We love the web.

Do you love it, too?

P.S. As I began writing this post, I realized that Microsoft’s Active Desktop was not so different from today’s devices, which are heavily based on web technologies: Firefox OS, Chrome OS and others. I can’t say that I love Microsoft, but as it often happens, it was quite pioneering with ideas, and not so good with their execution. Credit where credit’s due.

Filed under: Firefox, Free Software, Internet, privacy, software

Hugo Chávez Is Still Not Dead

aharoni — Wed, 06 Mar 2013 10:16:21 +0000

There are articles about Chávez in Wikipedias in ninety-six languages. He’s still not dead according to thirteen of them:

Cantonese (about the language) – FIXED
Central Bikol (about the language) – FIXED
Ido (about the language) – FIXED
Ladino (about the language) – FIXED
Min Nan (about the language)
Ossetic (about the language) – FIXED
Papiamento (about the language) – FIXED
Samogitian (about the language) – FIXED
Sicilian (about the language) – FIXED
Somali (about the language)
Upper Sorbian (about the language) – FIXED
Võro (about the language) – FIXED
Walloon (about the language) – FIXED

Looking at the different language Wikipedias often brings about other useful things. For example, Chávez’ death date was marked in the Manx Wikipedia, but the name of the month of March was spelled incorrectly, so I corrected it. In the Russian Wikipedia I noticed that the banner that invites people to Wikimania 2013 in Hong Kong is translated incorrectly, and I corrected it.

If you know one of the above languages, consider adding the death date of Hugo Chávez to the articles, and writing some other things there, too. Millions of people will appreciate your contribution.

Filed under: language, Wikipedia

Pay it Forward Soviet-style

aharoni — Wed, 12 Dec 2012 21:52:11 +0000

Remember Misha, the Soviet Olympic mascot?

Here’s another example that in the Soviet Union the good things were good: The animated short film “Thank you!” by Vladimir Tarasov.

It’s very Soviet, but in a good way. Three children fly in a plane, enjoy the flight and thank the pilot. The pilot is flattered, but he suggests them to thank the engineer who designed the plane, so they do. The engineer is hinted to be Jewish, and he’s smoking a cigarette while designing the plane—in the 1970s nobody complained that depicting smoking is dangerous to children. The engineer suggests the children to thank the factory workers who built the plane. The factory worker turns out to be Georgian and is depicted as an orchestra conductor; he suggests the children to thank the forgery worker who made the metal for the plane.

The forgery worker, who turns out to be Ukrainian, and even says a couple of Ukrainian words, suggests the children to thank the miner who brought the ore to the forgery. The miner suggests to thank the geologist, who found the ore. And the geologist suggest to thank the pilot, who brought him from Moscow to the far easy, where he found the ore.

As with many Soviet animated films, this one is both simple and arty.

Filed under: movies, Russia

Possible censorship of Putin and Medvedev’s names on Russian television

aharoni — Mon, 10 Dec 2012 18:12:13 +0000

Here’s a somewhat curious story: The Russian TV channel NTV showed a performance by the rock band “Leningrad”, which is famous for incorporating many Russian expletives in its lyrics. The expletives were censored by beeping, which is the usual and expected practice, comparable to beeping on words like “fuck” in American TV. The surprise in this performance, however, was that the names of president Putin and prime minister Medvedev, who were mentioned in the song, were censored the same way. The name of the the Church of Christ the Savior, which recently became famous as the stage of Pussy Riot’s notorious performance, was partly censored as well, although the name “Pussy Riot” itself was not censored.

NTV started out in the early 1990s as one of Russia’s first independent TV channels, but now it’s controlled by the Kremlin.

Here’s the original story at Lenta.ru in Russian. The only thing in English I could find about it, was this story, which is probably machine-translated. So I made a rough edited version so the English would be readable. It’s a reaction of the NTV host and of the former Culture Minister Mikhail Shvydkoi to the incident:

“Quacking’ over the names of Vladimir Putin and Dmitry Medvedev during the performance of the band “Leningrad” on NTV was irony, not an act of censorship. On this, as reported by RIA Novosti , December 10, told the host Vadim Takmenev. “We thought it was a funny trick in this song, so here it was sung openly with a giant duck will quack over the names of the president and the prime minister and the famous building with a dome,” – said Takmenev. He expressed regret that no one understood self-irony. The words “Putin” and “Medvedev” were masked duck quacking during the performance of Sergei Shnurov’s song “Moscow” in the program “CCTV” on December 9. Some cursing was masked, too. Many media saw this as an act of censorship. Meanwhile, according to Vadim Takmenev, if the purpose was censorship, the authors of the program would have chosen a different way: “The options were a few – well, for example, ask the “Leningrad” to sing the song in a more ethereal version. There was an option to score the words that are now all the talk , the names of two people, the music, so no one noticed… Or we could do the song and not put it on the air. If it would be about a censorship. What prevents us to throw this song?” Takmenev also noticed that topical songs on the air has been retained, and by “Leningrad” no questions to transfer arose. The presenter also said that accusations of censorship to the management of NTV in connection with the release of “Central Television” in any case unfounded, since the program is in the external production.

Earlier on Monday, President of the Foundation “Russian Television Academy,” the former Culture Minister Mikhail Shvydkoi said that the names of Putin and Medvedev have been removed from the air legally, as is the norm in the law on the protection of the dignity of the president. He did not see political implications in the actions of the authors of ” Central Television”.

Filed under: Russia

Web sight

aharoni — Sun, 25 Nov 2012 09:37:22 +0000

Because of some not-so-interesting technical reasons I ended up on the mailing list for reporting bugs in Wikipedia’s mobile app (please see disclaimer in the end).

Reading real Wikipedia readers’ reactions is fascinating.

A lot of the emails there are just empty. People just press the button to report a problem and don’t actually write anything at all.

Sometimes they are just slightly less than empty. For example, quite a lot of people write things like “When will you fix your stupid app already???!?!!”. This may seem pointless and unconstructive, but actually these people think that there is context to what they say, because they see complaints from other people at Google’s or Apple’s app store and they assume that the app’s maintainers are aware of them. Some people also threaten to give the app a low rating in the app store; it’s not really wrong, but it’s not very helpful either.

A lot of the emails are about connectivity problems in Android 2.2.2 and about screen rotation problems on iPad. The developers are aware of both issues and are working on them.

And a whole lot of reports suggest fixes in content, rather than technical problems. Some of them are pointless, for example “The facts on this web sight is wrong and i want it changed to the corrected statement”. It never occurred to that person that it would be helpful to say what information is wrong or what should be written there (it can also be a troll). And some people do make useful suggestions. For example, one person reported that Obama didn’t write “How the Grinch Stole Christmas“. The report was correct: somebody indeed vandalized the article about the children’s book and wrote that its author is Obama. It was an easy fix, so I just fixed it myself and replied, thanking the person for the report and saying that in the future she can fix it herself by pressing the “edit” button.

If I see that fixing the problem will take more than a minute, I just reply with “you can fix it yourself”. This does make me think that a more robust way of telling people that they can fix the problems themselves is needed.

All these issues aside, there is something truly wonderful about this app: People write these emails in their language without caring at all about who will read them. Reporting a bug in Bugzilla is hard for many reasons, one of which is certainly the language. But the app gives the user a completely localized experience, so the users don’t think twice before sending a bug report in their language.

And this is a good thing. Some People from Some Companies told me explicitly that they give up on processing reports from too many people in too many languages; not Wikimedia. Wikimedia may acknowledge that it’s hard, Wikimedia won’t commit to replying to each email, but Wikimedia wouldn’t just shut it down and ignore it completely, either. We would rather think about more efficient ways to get volunteers to reply to people efficiently or to help people fix the issues themselves – that’s what the whole “wiki” idea is about in the first place.

(Important disclaimer: I am involved with this mailing list as a volunteer. It has nothing to do with the paid work that I do for the Wikimedia Foundation. I do not officially represent the Foundation in any actions that I take with regard to that mailing list.)

Filed under: localization, Wikipedia Tagged: Android, iPad

The Case for Localizing Names

aharoni — Tue, 13 Nov 2012 19:56:25 +0000

I often help my friends and family members open email accounts. Sometimes they are starting to use the Internet and sometimes they move from old email services (Yahoo, Walla!, ISP) to something modern (like it or not, GMail).

At some point they have to fill their name, which will appear in the “from” field. And then I have to suggest them to write it in Latin characters, even though most of them speak languages that aren’t written in Latin characters – mostly Hebrew and Russian. Chances are that some day they will send an email to somebody who cannot read Russian or Hebrew, and Latin is relatively better known.

Only relatively, though. It may seem obvious to you that everybody knows the Latin script, but in fact, a lot of people are not comfortable with it at all. There are also other complications: lossy and inconsistent transliteration rules (is Amir אמיר or עמיר?), potential right-to-left rendering problems, and more. And of course, all people are happy to see their name in their language.

And people are also happy to see their friends’ names in their own language and not in a foreign or a neutral language. I have, for example, a lot of friends in India. Most of them write their names in English, but some write it in Marathi or in Malayalam. It’s certainly good for them, but in practice it’s much harder for me to find them this way, so English would be better – but Hebrew or Russian would be better yet.

Finally, there are a lot of people in the world who have more than one linguistic background. Mine are Russian, Hebrew and English, and I am really not such a special case. There are many millions of immigrants who have mixed backgrounds: Punjabi-Hindi-Urdu-English, Kurdish-Turkish-German, Kazakh-Russian-Norwegian, and others, and others and others. From each of these backgrounds they have friends, co-workers and family members, with whom they would love to communicate in the respective language. In each of these backgrounds they have friends who would want to find them using the name under which they know them there and using the appropriate language and writing system.

And sometimes people change their names, too. I did once, and so have many other people.

All this means that people’s names should be translatable, just like books, articles and software interfaces. Facebook and Google+ allow me to add a very limited number of names in foreign languages. Why wouldn’t they let me write my name in four, five, ten languages? This would make it easier for people who speak these languages to find me and to communicate with me. I would go even further and allow people who speak languages that I don’t know well to write my name as their hear it in their language and to add it to my details. Yet again, this would make me easier to find to even more people.

Some degree of automation can be possible. A lot of names are, after all, repetitive, so social networks would be able to suggest people with common names how their name would be written in other languages.

Wikipedia is actually quite good in this regard: Usually people have the same username across projects, and this username is not necessarily written in Latin letters, but people can customize the appearance of their signature in each project. I did it in a few languages, and people who speak those languages appreciate it.

I can only hope that social networks and email systems will allow as much flexibility as possible with this.

Filed under: Facebook, Google, localization, software, translation

English typing computer

aharoni — Sat, 03 Nov 2012 14:03:04 +0000

I’m in an Internet cafe in Mumbai. I tried to install Firefox with the Marathi interface, but on the computers here fonts for languages of India are not installed. That’s right – on computers in India fonts for languages of India are not installed. Hence, installing Firefox in Marathi failed at the very first stage, because the fonts are needed for the installation wizard.

Actually, I’m not surprised that these fonts are not installed, because it’s not my first time in India. I know that it happens a lot in this country. I would install them, but I don’t have a permission.

I find it incredibly weird – and tragic – that so many people in India don’t even try to use computers in any language except English. The one curious thing that I did find was an “English typing computer” shop. It’s just a place where you can use a computer to write Word documents in Hindi or Marathi, but using an English-based transliteration keyboard rather than the standard Indian Devanagari InScript keyboard, because they find transliteration keyboards easier. Of course, they could just install such a keyboard layout on their computers… but they prefer to go to an “English typing computer” shop.

We, software internationalization people, have so much more work to do.

Filed under: English, Firefox, India, localization, translation

Always define the language and the direction of your HTML documents, part 01

aharoni — Tue, 07 Aug 2012 08:42:49 +0000

I received this email from Safari Books Online:

Email in English from Safari Books, oriented like Hebrew. Click to enlarge.

The email is written in English, but notice how the text is aligned unusually to the right. Notice also that the punctuation marks appear at the wrong end of the sentence. I used Firefox developer tools to apply the correct direction, and saw it correctly:

The same email, with corrected left-to-right formatting using Firefox developer tools

This happens because I use GMail with the Hebrew interface. GMail has to guess the direction of the emails that I receive, because in plain text there’s no easy way to specify the direction (I hope to discuss it in a separate post soon). Usually GMail guesses correctly. Ironically, for HTML-formatted emails like this one, GMail often guesses incorrectly, even though in HTML, unlike in plain text, it’s quite easy to specify the direction by simply adding dir=”ltr” to the root element of the email.

Unfortunately a lot of HTML authors don’t bother to specify explicit direction. Many are not even aware of this exotic dir attribute. Others think that because “ltr” is the default, they don’t have to specify it. They are wrong: As this email shows, the left-to-right HTML content is embedded in a right-to-left environment, and the “rtl” definition propagates to the embedded content.

You could blame GMail, of course, but it’s much more practical to always define the direction of your HTML content, even if it’s the default. You can never know where will your content end up.

P.S.: I read this post before publishing and suddenly realized that its style is quite similar to “Best Practices” books, such as Damian Conway’s classic “Perl Best Practices” – it tells you to do something that is not obviously needed, and explains why it is needed nevertheless. I like to acknowledge sources of inspiration. Thank you, Damian.

Filed under: Firefox, Internet, localization, Perl, programming, translation Tagged: right to left