can you get more random?

Symfony i18n : dumping and loading translation strings

September 13th, 2012 Posted in geeky stuff

I’ve been meaning to write about this for literally years, but never got round to it. Hopefully this will be useful to someone. After an only partially sucessful attempt to manage symfony i18n strings in a MySQL DB, I finally gave up and reverted back to the default XLIFF file method. One of the reasons for initially wanting to store the translations in a DB was to build an easy to use interface to allow translators to easily find untranslated strings. Unfortunatley this didn’t work as well as I was hoping, and the alternatives (installing pootle, or getting translators to install and learn how to use an xliff tool) werent’ much better.

I finally decided that the easiest way to update translations would be by dumping untranslated strings to a file before sending them to a translator. Unfortunately I couldn’t find a way to easily dump strings to a file and reload translated strings, so I wrote a couple of new tasks to do this for me.

You need to create the following two files: lib/tasks/sfI18nDumpTask.class.php


<?php
/*
* Current known limitations:
* - Can only works with the default "messages" catalogue
* - For file backends (XLIFF and gettext), it only saves/deletes strings in the "most global" file
*/

/**
* Dumps i18n strings from messages.xml files.
*
*/
class sfI18nDumpTask extends sfBaseTask
{
/**
* @see sfTask
*/
protected function configure()
{
$this->addArguments(array(
new sfCommandArgument('application', sfCommandArgument::REQUIRED, 'The application name'),
new sfCommandArgument('culture', sfCommandArgument::REQUIRED, 'The target culture'),
));

$this->addOptions(array(
new sfCommandOption('untranslated', null, sfCommandOption::PARAMETER_NONE, 'Output all new found strings'),
));

$this->namespace = 'i18n';
$this->name = 'dump';
$this->briefDescription = 'dumps i18n strings from messages.xml files';

$this->detailedDescription = <<<EOF
The [i18n:dump|INFO] task extracts i18n strings from your project files
for the given application and target culture:

[./symfony i18n:dump frontend fr|INFO]

By default, the task dumps all translations it found in the current project.

[./symfony i18n:dump frontend fr|INFO]

If you want to dump the untranslated strings, use the [--untranslated|COMMENT] option:

[./symfony i18n:dump --untranslated frontend fr|INFO]

EOF;
}

public function execute($arguments = array(), $options = array())
{
$this->logSection('i18n', sprintf('dumping i18n strings for the "%s" application', $arguments['application']));

// get i18n configuration from factories.yml
$config = sfFactoryConfigHandler::getConfiguration($this->configuration->getConfigPaths('config/factories.yml'));

$class = $config['i18n']['class'];
$params = $config['i18n']['param'];
unset($params['cache']);

$i18n = new $class($this->configuration, new sfNoCache(), $params);
$i18n->getMessageSource()->setCulture($arguments['culture']);
$i18n->getMessageSource()->load();

$messages = $i18n->getMessageSource()->read();

$count=0;
foreach ($messages as $catalogue => $translations) {
$this->logSection('i18n', sprintf('found "%d" i18n strings', count($translations)));
foreach ($translations as $key => $values) {
if ($options['untranslated'] && ('' != $values[0])) {
continue;
}
$count++;
$source = str_replace('"', '""', $key);
$translation = str_replace('"', '""', $values[0]);
print "$values[1]|\"$arguments[culture]\"|\"$source\"|\"$translation\"\n";
}
}

$this->logSection('i18n', sprintf('dumped "%d" i18n strings', $count));

}
}

and lib/task/sfI18nLoadTask.class.php


<?php

/*
* Current known limitations:
* - Can only works with the default "messages" catalogue
* - For file backends (XLIFF and gettext), it only saves/deletes strings in the "most global" file
*/

/**
* loads i18n strings from php files.
*
*/
class sfI18nLoadTask extends sfBaseTask
{
/**
* @see sfTask
*/
protected function configure()
{
$this->addArguments(array(
new sfCommandArgument('application', sfCommandArgument::REQUIRED, 'The application name'),
new sfCommandArgument('culture', sfCommandArgument::REQUIRED, 'The target culture'),
new sfCommandArgument('filename', sfCommandArgument::REQUIRED, 'The translation csv file'),
));

$this->addOptions(array(
new sfCommandOption('no-overwrite', null, sfCommandOption::PARAMETER_NONE, 'Dont overwrite existing translations'),
));

$this->namespace = 'i18n';
$this->name = 'load';
$this->briefDescription = 'loads i18n strings from php files';

$this->detailedDescription = <<<EOF
The [i18n:load|INFO] task loads i18n strings from your project files
for the given application and target culture:

[./symfony i18n:load frontend fr filename|INFO]

By default, the task will load all strings from the file
into the current project.

If you only want to load previously untranslated strings, use the [--no-overwrite|COMMENT] option:

[./symfony i18n:load --no-overwrite frontend fr filename|INFO]

EOF;
}

/**
* @see sfTask
*/
public function execute($arguments = array(), $options = array())
{
$this->logSection('i18n', sprintf('loading i18n strings for the "%s" application', $arguments['application']));

// get i18n configuration from factories.yml
$config = sfFactoryConfigHandler::getConfiguration($this->configuration->getConfigPaths('config/factories.yml'));

$class = $config['i18n']['class'];
$params = $config['i18n']['param'];
unset($params['cache']);

$i18n = new $class($this->configuration, new sfNoCache(), $params);
$i18n->getMessageSource()->setCulture($arguments['culture']);
$i18n->getMessageSource()->load();

$messageSource = $i18n->getMessageSource();
$messages = $messageSource->read();

if ($loaded_list = file($arguments['filename'])) {
foreach ($loaded_list as $line) {
$line = chop($line);
if (preg_match('/(\d+)\|"(.*)"\|"(.*)"\|"(.*)"/', $line, $matches)) {
list($id, $culture, $source, $translation) = array($matches[1], $matches[2], $matches[3], $matches[4]);
if ($culture == $arguments['culture']) {
$messageSource->update($source, $translation, '');
}
}
}
} else {
$this->logSection('i18n', sprintf('Cant open file "%s"', $arguments['filename']));
}

if ($options['no-overwrite'])
{
$this->logSection('i18n', 'saving only new i18n strings');
} else {
}
$messageSource->save();

}
}

Before dumping the new strings to a file, you should first add any new untranslated strings using the builtin symfony i18n:extract command:

 ./symfony i18n:extract --auto-save --display-new frontend <culture> 

Then you’re ready to dump the new untranslated strings to a file :

 ./symfony i18n:dump frontend <culture> --untranslated > untranslated.csv 

You can then import this file into your favourite spreadsheet program. In NeoOffice this is done by opening a fresh spreadsheet and importing the data via “Insert” > “Sheet from external file” and making sure to add pipe ‘|’ as field seperator. YMMV in your spreadsheet tool of choice. Once the translations have been added in the 4th column of the spreadsheet, save the document as a pipe separated csv file before using the following command to reload the translated strings:

 ./symfony i18n:dump frontend <culture> translated.csv 
Share

Post a Comment