Preparing for i18n right now


My understanding of the discussions about i18n during the past two weeks is that the SecureDrop team is willing to add i18n as soon as possible. They are however focused on publishing the 0.4 release which is going to feature freeze after the last pending PRs are merged. It means the release/0.4 branch will be created next week and we can start sending i18n pull requests against the develop branch without disturbing the stable release.

Since the SecureDrop team attention won’t be focused on i18n and their bandwidth will be limited, we (as a community), need a strategy to avoid wasting good work. This is not a problem specific to SecureDrop, it is pretty much the same with every project when trying to introduce a significant change that is not the top priority of the core team driving the development.

I think it would be enough to make progress as follows:

  • create a topic branch for i18n with a patch series made of minimal commit, each passing the CI
  • work with this topic branch to provide translations in at least one language
  • send a PR for the first commit in the patch series and wait for the SecureDrop team to merge it on their own time. And submit again, one commit, when it is merged.
  • rebase at least once a day the patch series so it stays up to date with the develop branch

I realize this is more work on our part (us the community :wink: but we’re not the bottleneck and there are many of us. I don’t mind if things take a very long time to land upstream, as long as we have a strategy ensuring it does not go to waste.

What do you think ?

After discussing with heartsucker today I created the tiniest pull request I could think of at (adding a requirement ;-). Hopefully that will be merged for 0.4.1 ?

Next step will be to split heartsucker initial pull request ( ) into smaller ones and test it manually.

Round trip with weblate almost complete: import successful to . Now for export there needs to be a git repository to push edits.

Weblate has a cli:

$ pip3 install wcl
$ wlc --url list-projects
name: SecureDrop
slug: securedrop
source_language: en

Using the same method as phpmyadmin the SecureDrop documentation can also be translated via weblate so it is all in the same place (website strings and localized documentation). Not sure I understand how they do it just yet but it looks simple.

Sent a mail to asking for the scripts to create the POT from RST files. They should be in but I can’t find them. He replied they are created with sphinx-build -b gettext… which seems fairly straightforward.

And the first translation was pushed to git by weblate. Looks like this is actually going to work :slight_smile:

I’m optimistic this will work out fine, it has been a good week-end :slight_smile:

I’m not too worried about the translation workflow now, weblate is good enough.

@heartsucker I was reading the documentation to better understand best_match and it looks like the behaviour is somewhat imperfect in werkzeug. Combined with the undocumented assumption of flask-babel about the translation directory it got me worried about the maintenance of i18n in flask. Do you have an opinion about that ?

This is what I’ve done in the past. I can’t remember what my old PR did.

1 Like

We could store the locale in the session for SD or use a query string param. The initial would default to English with someone clicking the locale they want to switch it.

I see you manually parsed the Accept-Language. At least there is no complicated match going on and problems are easier to diagnose :slight_smile:

As a first minimal step to load the locale, I used best_match. I’ll most likely revert to your method if it gives me trouble. The user has no control over it, this is very limited. I did not want to merge the commit focusing on loading the locale with the commit that implements a user interface to select the language (via query arguments or via a select in the HTML page).

Also, instead of explicitly listing the locales in the config file, I chose to only define LOCALE which set the default. The list of available locales is defined by exploring the content of the translations directory. My motivation was primarily to reduce the change of to the strict minimum because it needs to be carefully upgraded / verified. If there was a list of available locales, including human readable labels, it would make it more complicated. Both can be defined elsewhere (the human readable strings are part of the GUI, the list of available translations can be read from the directory, restricting the list of translations displayed to the user can also be part of the GUI configuration).

This is not carved in stone, reason why I did not elaborate more in the commit messages. Please let me know if you have concerns :slight_smile:

Yes, but you’d want to have a mapping of en_US to English (American) or whatever for the buttons a user clicks. Don’t put that in, but somewhere else that’s hard coded. The app I pulled it from had a that is totally unlike what SD uses.

1 Like

The minimal version of i18n works.

The next step is to go over each source/template file and gettextize it. It may be good to l10n in parallel, just to see how manageable the fixed size font / layout is and what compromises it requires. It would be better to run into blockers before all files are processed. Since @heartsucker already did the work for German not too long ago I’m not very worried.

It would go like this:

and repeat until it’s done. In firefox Preference -> Content -> Languages can be used to select the language by moving the prefered language in front of the list. It changes the Accept-Language header accordingly and source/journalist pages will respond to the change on every request.

While making progress with the french translation of the journalist interface it is most useful to verify the output using screenshots automatically generated (see the implementation). Here is an example that show where the display is not right (Edition and Suppression are too close to each other):

It was generated with:

ACCEPT_LANGUAGES=fr-fr pytest -v tests/pages-layout/
1 Like

@heartsucker your original commit is a great time saver. It is so much easier to follow your footsteps than to figure out how to proceeed :slight_smile: It also turns out the majority of the templates gettext() are still valid too, which is a nice suprise. Moving on to javascript i18n now and I would not know where to begin if it was not for the work you did.

Hey thanks. Yeah, that was some fun little hack to work around the embedded strings in the JS. :slight_smile:

I suspect it will need to be reworked but it does the job right now :slight_smile: Here is how it looks now if you’re curious: . To ease testing I derived the functional tests that create a text file containing the translated alert (cannot conveniently be captured by a screenshot):

I also simplified the workflow so it creates all screenshots / txt files for supported locales with one call:

pytest -v tests/pages-layout/

All journalist templates localized in french. Moving on the source templates now. Here is how they look now. It took longer than I thought. It should be a little quicker for the source templates if all the i18n is complete (I think it is but … ;-). Here is how the automated screenshots look like. There are 26 different screenshots, these show how they vary depending on the application state / javascript enabled or not etc.

The French translation is complete. And the German translation was resurrected from @heartsucker past efforts: about 50% of the ~200 strings are still good.

It is probably time to call for new translators. The weblate workflow is good. Hopefully good enough to be understandable with a one page documentation targeting a new translator.

1 Like