Add i18n support to Docusaurus

[rudokemper]

We can look to how Awana did this for their Docusaurus instance: digidem/comapeo-docs#3

This should be a future epic handled once documentation writing has stabilized, including:

• Adding i18n support itself
• Translating all pages to ES, PT

[nicopace]

There is this blog that explains how they are doing it:
https://semaphore.io/blog/docusaurus
I think it is worth to take a look.

[rudokemper]

There is this blog that explains how they are doing it: https://semaphore.io/blog/docusaurus I think it is worth to take a look.

This blog provides a helpful CLI way to set up the translations pages e.g. /i18n/en, /i18n/pt, ...

To actually provide the translations, ideas discussed today:

• Adding something like a Google Translate widget as lowest hanging fruit
• Using an LLM to handle translations, either
  • Running them manually
  • Some kind of CI workflow where they are automatically generated and submitted as PR, to be reviewed and approved by a human

Most important is Portuguese. Then Spanish (opportunistic for new partners in Latam), then Dutch (for Suriname, but many of our partners can read English)

[nicopace]

We want for the documentation to be accesible through different languages.
Documentation is produced in english, and we want at least to be translated to Portuguese, and potentially to Spanish too.
The process should involve automated translations with human-in-the-loop to ensure the quality of the translation.
It should happen as an automated process.
One potential approach would be to have a github action that translates the merged changes and sends a PR for each of the languages supported.

Key documentation pages:

• https://docusaurus.io/docs/i18n/introduction
• https://docusaurus.io/docs/api/plugins/@docusaurus/plugin-content-docs#i18n
• https://docusaurus.io/docs/api/plugins/@docusaurus/plugin-content-blog#i18n

The Docusaurus recommended approach is to use Crowdin, that has automated free translations for open source projects with human-in-the-loop workflows, so it fits right in our workflow.

Here is the official documentation on how to use Crowdin: https://docusaurus.io/docs/i18n/crowdin

And here the process within Crowdin to use the open source free license: https://crowdin.com/product/for-open-source

[rudokemper]

Worth pinging Awana / Lu. They have used Crowdin extensively for years for the applications. I wonder if they are also using it for CoMapeo docs.

Originally posted by @rudokemper in #89

[rudokemper]

Awana:

We have LLM translate the markdown as to retain the rich text formatting
No crowdin
More LLM does first translation then human reviews it on notion, without having to worry about maintaining the rich text
But we're finding some challenges with our approach, as the rich MD becomes quite large (a few some pages become large that can be larger then a context window)
There's a plan to include the glossary, but hasn't been implemented

[rudokemper]

It does feel like human-in-the-loop translation via CrowdIn would be a very heavy lift.

[nicopace]

It does feel like human-in-the-loop translation via CrowdIn would be a very heavy lift.

In my mind:

1. human merges english PR
2. machine triggers translation PR
3. human reviews translation PR

It feels like a heavy lift to do the first chunk as we have a bunch written, but will have to do either way.

Crowdin contributes with the automation, integration and the institutional memory.

Could you describe more about what you mean by very heavy lift?

[rudokemper]

Oh, I didn't read your description above about CrowdIn offering automation, sorry. I was imagining that we would have to manually translate every single string.

[nicopace]

Some discussion around whether to use Crowdin vs LLMs here: facebook/docusaurus#11226

[nicopace]

Clickhouse uses their own llm-based python translation: https://github.com/ClickHouse/clickhouse-docs/tree/main/scripts/translate (mentioned in the docusaurus link before).

[nicopace]

As far as my research went, doing the automation doesn't necessarily solve big issues for us.
I've set up translation for pt for our documentation, and translated the main sections, and will keep exploring options, as none of the integrations or automations seem to suit our needs.

In particular, crowdin and lokalize are very focused in multi user translation, while this is much simpler than that.

Till I find a setup that allows:

1. get a changeset from new commit
2. do a Machine TRanslated translation of it
3. send a PR of it for a human to review

Will do it by hand one by one.