Texte gehören zu den Teilen einer Anwendung, die häufig geändert werden. Wenn dann noch externe Übersetzer im Spiel sind, kann es leicht zu fehlenden Übersetzungen kommen

Wir möchten deshalb automatisch testen, dass es zu allen Texten in Sprache A auch Übersetzungen in Sprache B vorhanden sind. Ein solcher Test allein hilft schon sehr, und sollte ja nicht so schwierig sein.

Leider gibt es dabei eine Überraschung: in der Internationalisierung-API findet sich zur Zeit (Rails 2.3.5) keine Möglichkeit, alle Schlüssel aufzuzählen (sprich iterieren). Man muss also offenbar die Yaml-Dateien selbst einlesen, um an alle Schlüssel zu kommen. Unserer Test ist dabei leider abhängig von der konkreten Art der Datenhaltung.

texts_hash = YAML::load(IO.read('config/locales/de.yml'))

Wie haben nun einen rekursiven Hash. Um alle Schlüssel zu überprüfen, extrahieren wie sie mit einer gesonderten Funktion. Diese Funktion ist einfach rekursiv aufgebaut.

def deep_hash_key(hash, prefix)
  hash.keys.sort.collect{|key2|
    val = hash[key2]
    newprefix = prefix.nil? ? key2 : "#{prefix}.#{key2}"
    if val.is_a? Hash
     deep_hash_key(val, newprefix)
    else
      newprefix
    end
  }
end

Wollen wir jetzt die Übersetzungen für alle Schlüssel prüfen lauert die Zweite Falle: Wenn die Übersetzung eine Variableninterpolation enthält, erhalten wir einen Fehler beim Aufrufen der translate() Methode ohne den passenden Hash. Offenbar liefert uns die API auch keinen Zugriff auf den String ohne Interpolierung. Da uns in diesen Test nur die Existenz einer Übersetzung interessiert, und nicht ihr konkreter Wert, fangen wir die Exception “MissingInterpolationArgument” einfach auf.

Für eine deutsche Anwendung mit Übersetzungen in Englisch und Französisch sieht dann der Test so aus:

def test_all_texts_should_exists
  texts_hash = YAML::load(IO.read('config/locales/de.yml'))
  de_keys = deep_hash_key(texts_hash['de'], nil).flatten
  for lang in ['de', 'fr', 'en']
    I18n::locale = lang
    for key in de_keys
      begin
        val = I18n.translate(key, :default => nil, :raise => true)
      rescue I18n::MissingTranslationData
        val = nil
      rescue I18n::MissingInterpolationArgument
        val = true  # there was a value, only an argument is missing
      end
      assert_not_nil(val, "key missing for locale #{lang}: #{key}")
    end
  end
end

Wenn jetzt eine Übersetzung fehlt, liefert der Test eine Fehlermeldung der Art:

test_all_texts_should_exists(LocalisationTest): key missing for locale en: users.edit.title

Die fehlende Übersetzung können wir dann ergänzen, bevor wir eine neue Version der Anwendung veröffentlichen.

Web pages often contains sentences in which a single word is changing. For example, “You are logged in as xxxx.” For the translation, we can obviously assemble the sentence from the components. There is often a problem, because the word order is different in different languages .

Better is the so-called interpolation, as Ruby itself also offers. The string contains a marked variable, which will be replaced at run time. The Yaml file specification does not use the Ruby syntax. Instead, you mark the variable with double curly brackets.

# Definition
  logmsg: "You are logged in as {{name}}."
 
# call
I18n.translate(:logmsg, :name => 'Administrator')
# or short form
t :logmsg, :name => 'Administrator'

We need to make sure that the parameters of the definition and the call are matching. The translate ()-method is relatively tolerant: Superfluous parameters are ignored.

Attention: the person who translates must know that the variable names in the brackets must not be translated.

Another small trap lurks in the variable names: They may not have the names one of the other parameters of the translate () method. Not allowed are therefore “default” and “scope”.

Texts of Validations

Even if our application is monolingual in another language than English, we get surprised in one area: The automatically generated error messages from the validation of the model classes. Very convenient, but unfortunately only in English.

Also, the messages contains the class name and attribute name from the ruby code directly. Classes and attribute names are chosen to serve the needs of the programmers. For users, other words are sometimes more appropriate.

You see the composition of such an error message in this picture:

To display the validation message with texts in our local language, we need only the translations with the right keys in our translation Yaml file. The template sentences are defined in the area (scope) activerecord.messages. Since these are are equal for all applications, there exists already standard translations for many languages.

Model Translation

We define the names of the fields and objects in the Yaml file with the key activerecord.models.classname or activerecord.attributs.classname.attributname. Rails recognizes these keys and uses the translations correctly in the automatic validation notifications.

activerecord:
  models:
    term: "Begriff"
  attributes:
    term:
      name: "Name"
      topic: "Sachgebiet"

The central translation of class names and field names is also useful for the views. So we don’ t need to translate the the labels of input fields a second or third time.

For the view this does not always make sense. The user interface should not stick too narrow to the model. The important thing is the user.

Validations with own Messages

A typical use for a specific message is to check of a phone number. The methods for the specification of the validations have a parameter “message”.

So the translation should be no problem: we should just call t ()-method, right? Unfortunately it is not that easy:

# funktioniert so nicht! Does not work!
validates_format_of( :fax,{
    :with => /\A[0-9 \+\-\/\(\)]+\Z/i,
    :message => t('activerecord.errors.messages.bad_phone'),
    :allow_blank => true})

Depending on the version and the deployment, Rails either did not even start or it does not delivers the required translation. What’s wrong?

The language is set for each request of the web user. But the declarative validations are defined as part of the model class, there is no request at this time! It therefore makes no sense to call a translate method in this stage.

But how do we solve this problem? Typical of Rails, the solution is already built in. But if you don’t know, you can go in round circles.

We can put a symbol rather than a string as the message parameter. Now the symbol will be evaluated at the time when the error message will be displayed.

The symbol gets evaluated in these specific contexts (scope) one after another:

• activerecord.errors.models.[model_name].attributes.[attribute_name]
• activerecord.errors.models.[model_name]
• activerecord.errors.messages

Therefore we can simply write our validation like this:

validates_format_of( :fax, {
     :with => /\A[0-9\+\-\/\(\)]+\Z/i,
     :message => :bad_phone,
     :allow_blank => true } )

Sources:

Description (Guide) of the Internationalization API: guides.rubyonrails.org/i18n.html

Language files for standard translations: github.com/svenfuchs/rails-i18n/rails/locale

Recently, I received a job request. An existing web application will be greatly expanded, and therfore converted to use the CMS “Joomla”. For the management of members  the extension “Community Builder” will be used.
Previously, users had a number as login name. This number should now be available in a field “member number” (german: mitgliedsnummer), and the numbers from existing users should be transferred. So much for the request from a customer perspective.

Analysis

Joomla manages the users in a table jos_user. The “community builder” allows to add additonal fields to the user record. The fields are placed in the separate table “jos_profiler”. These records are linked with the ID of the user. In this way, the Community Builder does not need to change the table of the Joomla core. However, fiddling with the two tables is technically more complex and error-prone.

The requirement is therefore: For all jos_user records with a number in the field username , put this number in the field member_number of the corresponding member record in the table jos_profiler.

Is it possible with SQL?

You can surely make a small PHP program to solve the problem . But if we can do it just with a SQL command, it is much easier. This was the reason for me to inspect the documentation for MySQL.

Fortunately, the MySQL database can use regular expressions. So you can retrieve all user names that consist only of digits with this statement:

SELECT username FROM jos_users WHERE username REGEXP '^[0-9]+$'

Linking two tables

The link between two database tables is also called a “join”, in our case a “inner equijoin”.
To show the username field and the membership number at the same time, we use:

SELECT jos_users.username, jos_comprofiler.cb_mitgliedsnummer
FROM jos_comprofiler, jos_users
WHERE jos_comprofiler.user_id = jos_users.id

This statement will also helps us to check whether everything is set correctly

Setting data

As a final step remains: set the field of one record with the contents of the connected record of the other table. Fortunately, the above two techniques are also working in the SQL UPDATE command.

UPDATE jos_comprofiler, jos_users
SET jos_comprofiler.cb_mitgliedsnummer = jos_users.username
WHERE jos_comprofiler.user_id = jos_users.id
  AND jos_users.username REGEXP '^[0-9]+$'

With a single SQL command, the customer problem is solved. The magic that makes this possible is the mathematical foundation of SQL, the relational algebra.

Writing software for several languages is an ever recurrent challenge. Ruby on Rails provides solutions to most of the standard problems in web development. But it needed time until the version 2.2. for the “Internationalization API ” to become an official part of Ruby on Rails. .

Structure of the framework

The framework consists of a “public” and a “private” part. The I18N module forms the public part with the methods that we call in our code. The private backend is interchangeable. The standard implementation reads the texts from files in Yaml format or from normal ruby files.

Calling

In our code, we replace all the text to be translated with the I18N.translate() method. Thanks to an alias definition, we can simply write t(). We give a key as parameter, and the backend uses this key to look up the correct translation.

The key is a string or a symbol. Normally, look up by symbol is slightly more efficient. But since the key gets decomposed into its components during the processing, I expect that this advantage disappears.

In a typical web application, many of the texts to be translated are found in the views, ie. Erb templates. As an abbreviation for pure HTML text, there is the following tag:

<%=t 'key' %>

Organization of keys

The keys have a hierarchical structure. The suggested organization is:

“Controllername.viewname.key”, for example ‘companies.show.header’

The language files

The language files can either be Ruby code files or text files in Yaml format. The person who translates is often a person without Ruby knowledge. That is why the Yaml file format is recommended.

The format looks like this:

de:
  users:
    show:
      header: "Company Data"
      intro: "For this company, we have stored the following data:"

The first key part the language code. This has the disadvantage that it separates the text entries of different languages. To help the translators, you can insert comments by marking a line with a beginning ‘#’ character . And of course, you should name the keys as clear as possible.

But even then, when developer and translator are not the same person, I think the Yaml file format is not optimal.

Part 2 of this article will soon be translated to English.