Merge pull request #666 from TownSquareXYZ/l10n_localization

feat: New mandarin localization workflow

Author: reveloper

.gitignore

@@ -24,3 +24,5 @@ npm-debug.log*
 yarn-debug.log*
 yarn-error.log*
 .gitpod.yml
+
+.env

crowdin.yml

@@ -0,0 +1,12 @@
+project_id: '663652'
+api_token_env: CROWDIN_PERSONAL_TOKEN
+preserve_hierarchy: 1
+files:
+  - source: /i18n/en/**/*
+    translation: /i18n/%two_letters_code%/**/%original_file_name%
+  - source: /docs/**/*
+    translation: /i18n/%two_letters_code%/docusaurus-plugin-content-docs/current/**/%original_file_name%
+    ignore:
+      - /docs/**/*.png
+  - source: /src/pages/learn/**/*
+    translation: /i18n/%two_letters_code%/docusaurus-plugin-content-pages/learn/**/%original_file_name%

docs/contribute/archive/hacktoberfest-2022/README.mdx

@@ -1,13 +1,13 @@
 import Button from '@site/src/components/button'
 
 # What is Hacktoberfest?
-
+````mdx-code-block 
 <div style={{ textAlign: 'center', margin: '50px 0' }}>
   <img alt="tlb structure"
        src="/docs/img/docs/hacktoberfest.webp"
        width="100%" />
 </div>
-
+````
 [Hacktoberfest](https://hacktoberfest.digitalocean.com/) is a month-long celebration of *open-source projects*, their *maintainers*, and the entire community of *contributors*. Each October, open source maintainers give new contributors extra attention as they guide developers through their first pull requests.
 
 For the TON Community it's time to help ecosystem growth together, so let's join the whole world with our **Hack-TON-berfest** party and become *#1 open-source ecosystem of the year*!
@@ -30,14 +30,16 @@ For everyone in TON it's an opportunity to drive the growth of the entire ecosys
 ## What are the rewards?
 
 To motivate the community to contribute to open source projects in the TON Ecosystem, you'll be able to receive a special reward from TON Foundation. Every participant will receive a **Limited Hack-TON-berfest NFT** achievement as a proof of participating:
-
+````mdx-code-block 
 <div style={{width: '100%', textAlign:'center', margin: '0 auto'}}>
   <video width="300" style={{width: '100%', borderRadius:'10pt', margin:'15pt auto'}} muted={true} autoPlay={true} loop={true}>
     <source src="/docs/files/nft-sm.mp4" type="video/mp4" />
-        Your browser does not support the video tag.
+````
+Your browser does not support the video tag.
+````mdx-code-block 
   </video>
 </div>
-
+````
 :::info IMPORTANT!
 TON Foundation will mint a collection in November to all wallet addresses submitted to the [@toncontests_bot](https://t.me/toncontests_bot). It will happen after the calculation and validation of all contribution results.
 :::
@@ -46,8 +48,18 @@ You have plenty of time to participate in the event. Let's build decentralized I
 
 
 
-  <Button href="/contribute/hacktoberfest/as-contributor"
+````mdx-code-block 
+<Button href="/contribute/hacktoberfest/as-contributor"
      colorType="primary" sizeType={'lg'}>
-    I want to become a Contributor
-  </Button>
-  <Button href="/contribute/hacktoberfest/as-maintainer" colorType={'secondary'} sizeType={'lg'}>I'm a Maintainer</Button>
+````
+I want to become a Contributor
+````mdx-code-block 
+</Button>
+````
+````mdx-code-block 
+<Button href="/contribute/hacktoberfest/as-maintainer" colorType={'secondary'} sizeType={'lg'}>
+````
+I'm a Maintainer
+````mdx-code-block 
+</Button>
+````

docs/contribute/docs/schemes-guidelines.mdx

@@ -29,7 +29,7 @@ If the order of transactions doesn't matter, you can omit their labels. This sim
 
 
 #### Message Processing Example
-
+````mdx-code-block
 <br></br>
 <ThemedImage
   alt=""
@@ -39,7 +39,7 @@ If the order of transactions doesn't matter, you can omit their labels. This sim
   }}
 />
 <br></br>
-
+````
 
 
 Learn references directly from Visio [message-processing.vsdx](/static/schemes-visio/message-processing.vsdx).
@@ -72,7 +72,7 @@ Learn references directly from Visio [message-processing.vsdx](/static/schemes-v
 In the case of complex and repetitive communication schemes between 2-3 actors, it is advisable to use a sequence diagram. For messages, use the notation of a common synchronous message arrow.
 
 #### Example
-
+````mdx-code-block 
 <br></br>
 <div class="text--center">
 <ThemedImage
@@ -84,7 +84,7 @@ In the case of complex and repetitive communication schemes between 2-3 actors,
 />
 </div>
 <br></br>
-
+````
 
 ### Scheme References
 

docs/contribute/localization-program/how-it-works.md

@@ -0,0 +1,141 @@
+# How It Works
+
+![how it works](/img/localizationProgramGuideline/localization-program.png)
+
+The **TownSquare Labs Localization Program** comprises several key components. This chapter will provide an overview of how the program operates, helping you understand its workings and how to use it effectively.
+
+Within this system, we integrate several applications to function seamlessly as a unified program:
+
+- **GitHub**: Hosts the documentation, synchronizes docs from the upstream repository, and syncs translations to specific branches.
+- **Crowdin**: Manages translation processes, including translating, proofreading, and setting language preferences.
+- **AI Systems**: Utilizes advanced AI to assist translators, ensuring smooth workflow.
+- **Customized Glossary**: Guides translators and ensures AI generates accurate translations based on the project’s context. Users can also upload their glossaries as needed.
+
+:::info
+This guide won't cover the entire process in detail, but it will highlight the key components that make the TownSquare Labs Localization Program unique. You can explore the program further on your own.
+:::
+
+## GitHub Synchronization for Documentation and Translations
+
+Our repository utilizes several branches for managing documentation and translations. Below is a detailed explanation of the purpose and function of each special branch:
+
+### Branches Overview
+
+#### 1. `dev`
+The `dev` branch runs GitHub Actions to handle synchronization tasks. You can find the workflow configurations in the [**`.github/workflows`**](https://github.com/TownSquareXYZ/ton-docs/tree/dev/.github/workflows) directory:
+
+- **`sync-fork.yml`**: This workflow synchronizes documentation from the upstream repository. It runs daily at 00:00.
+- **`sync-translations.yml`**: This workflow synchronizes updated translations to the respective language branches for preview purposes on the corresponding language websites.
+
+#### 2. `localization`
+This branch stays in sync with the upstream repository through GitHub Actions running on the `dev` branch. It is also used for updating certain codes that we intend to propose to the original repository.
+
+#### 3. `l10n_localization`
+This branch includes all changes from the `localization` branch and translations from Crowdin. All modifications in this branch are committed to the upstream repository.
+
+#### 4. `[lang]_localization`
+These branches are designated for specific language previews, such as `ko_localization` for Korean and `ja_localization` for Japanese. They allow us to preview the website in different languages.
+
+By maintaining these branches and using GitHub Actions, we efficiently manage the synchronization of our documentation and translation updates, ensuring that our multilingual content is always up to date.
+
+## How to Set Up a New Crowdin Project
+
+1. Log in to your [**Crowdin account**](https://accounts.crowdin.com/login).
+2. Click `Create new project` in the menu.
+![Create new project](/img/localizationProgramGuideline/howItWorked/create-new-project.png)
+3. Set your Project name and Target languages. You can change the languages in the settings later.
+![Create project setting](/img/localizationProgramGuideline/howItWorked/create-project-setting.png)
+4. Go to the project you just created, select the Integrations tab, click the `Add Integration` button, search for `GitHub`, and install it.
+![install-github-integration](/img/localizationProgramGuideline/howItWorked/install-github-integration.png)
+5. Before configuring GitHub integrations on Crowdin, specify which files to upload to Crowdin to avoid uploading unnecessary files:
+
+    1. Create a **crowdin.yml** file in the root of **your GitHub repo** with the basic configuration:
+
+      ```yml
+      project_id: <Your project id>
+      preserve_hierarchy: 1
+      files:
+        - source: <Path of your original files>
+          translation: <Path of your translated files>
+      ```
+
+    2. Get the correct configuration values:
+        - **project_id**: In your Crowdin project, go to the Tools tab, select API, and find the **project_id** there.
+        ![select-api-tool](/img/localizationProgramGuideline/howItWorked/select-api-tool.png)
+        ![projectId](/img/localizationProgramGuideline/howItWorked/projectId.png)
+        - **preserve_hierarchy**: Maintains the GitHub directory structure on the Crowdin server.
+        - **source** and **translation**: Specify the paths for the files to upload to Crowdin and where the translated files should be output.   
+
+          Refer to [**our official config file**](https://github.com/TownSquareXYZ/ton-docs/blob/localization/crowdin.yml) for an example.   
+          More details can be found in the [**Crowdin configuration documentation**](https://developer.crowdin.com/configuration-file/).
+
+6. Configure Crowdin to connect to your GitHub repo:
+    1. Click `Add Repository` and select `Source and translation files mode`.
+    ![select-integration-mode](/img/localizationProgramGuideline/howItWorked/select-integration-mode.png)
+    2. Connect your GitHub account and search for the repo you want to translate.
+    ![search-repo](/img/localizationProgramGuideline/howItWorked/search-repo.png)
+    3. Select the branch on the left, which will generate a new branch where Crowdin will post the translations.
+    ![setting-branch](/img/localizationProgramGuideline/howItWorked/setting-branch.png)
+    4. Choose the frequency for updating translations to your GitHub branch. Default settings can be kept for other configurations, then click save to enable the integration.
+    ![frequency-save](/img/localizationProgramGuideline/howItWorked/frequency-save.png)
+
+Refer to the [**GitHub integration documentation**](https://support.crowdin.com/github-integration/) for more details.
+
+7. Finally, you can click the `Sync Now` button to sync the repo and translations whenever needed.
+
+## Glossary
+
+### What is a Glossary?
+
+Sometimes, AI translators can't recognize specific terms that shouldn't be translated. For instance, we don't want "Rust" translated when referring to the programming language. To prevent such mistakes, we use a glossary to guide translations.
+
+A **glossary** allows you to create, store, and manage project-specific terminology in one place, ensuring terms are translated correctly and consistently.
+
+You can check our [**ton-i18n-glossary**](https://github.com/TownSquareXYZ/ton-i18n-glossary) for reference.
+![ton-i18n-glossary](/img/localizationProgramGuideline/howItWorked/ton-i18n-glossary.png)
+
+### How to Set Up a Glossary for a New Language?
+
+Most translation platforms support glossaries. In Crowdin, after setting up a glossary, each term appears as an underlined word in the Editor. Hover over the term to see its translation, part of speech, and definition (if provided).
+![github-glossary](/img/localizationProgramGuideline/howItWorked/github-glossary.png)
+![crowdin-glossary](/img/localizationProgramGuideline/howItWorked/crowdin-glossary.png)
+
+In DeepL, simply upload your glossary, and it will be used automatically during AI translation.
+
+We have created [**a program for glossary**](https://github.com/TownSquareXYZ/ton-i18n-glossary) that auto-uploads updates.
+
+To add a term to the glossary:
+1. If the English term already exists in the glossary, find the corresponding line and column for the language you want to translate, input the translation, and upload it.
+2. To upload a new glossary, clone the project and run:
+
+    - `npm i`
+    - `npm run generate -- <glossary name you want>`
+
+Repeat step 1 to add the new term.
+
+**Simple and efficient, isn’t it?**
+
+## How to Take Advantage of AI Translation Copilot?
+
+AI translation copilot helps break down language barriers with several advantages:
+
+- **Enhanced Consistency**: AI translations are based on up-to-date information, providing the most accurate and current translations.
+- **Speed and Efficiency**: AI translation is instantaneous, handling large volumes of content in real-time.
+- **Robust Scalability**: AI systems continuously learn and improve, enhancing translation quality over time. With the provided glossary, AI translations can be tailored to the specific needs of different repositories.
+
+To use AI translation in Crowdin (we use DeepL in our project):
+1. Select Machine Translation in the Crowdin menu and click edit on the DeepL line.
+![select-deepl](/img/localizationProgramGuideline/howItWorked/select-deepl.png)
+2. Enable DeepL support and input the DeepL Translator API key.
+      > [How to get DeepL Translator API key](https://www.deepl.com/pro-api?cta=header-pro-api)
+
+![config-crowdin-deepl](/img/localizationProgramGuideline/howItWorked/config-crowdin-deepl.png)
+
+3. Our DeepL setup uses a customized glossary. Check [**ton-i18n-glossary**](https://github.com/TownSquareXYZ/ton-i18n-glossary) for details on uploading the glossary.
+
+4. In the repo, click Pre-translation and select via Machine Translation.
+![pre-translation](/img/localizationProgramGuideline/howItWorked/pre-translation.png)
+5. Choose DeepL as the Translation Engine, select the target languages, and select the files to translate.
+![pre-translate-config](/img/localizationProgramGuideline/howItWorked/pre-translate-config.png)
+
+That's it! Now you can take a break and wait for the pre-translation to complete.

docs/contribute/localization-program/how-to-contribute.md

@@ -0,0 +1,119 @@
+# How to Contribute
+
+In our quest to make **TON the most successful blockchain**, ensuring that TON documentation is comprehensible to people worldwide is crucial. Localization is key, and we're **excited** to have you join us in this effort.
+
+## Prerequisites
+
+The **TownSquare Labs Localization Program** is open to everyone! Here are a few steps you need to take before you start contributing:
+
+1. Log in to your [**Crowdin**](https://crowdin.com) account or sign up.
+2. Select the language you want to contribute to.
+3. Familiarize yourself with the [**How to Use Crowdin**](/contribute/localization-program/how-to-contribute) guide and the [**Translation Style Guide**](/contribute/localization-program/translation-style-guide) for tips and best practices.
+4. Use machine translations to aid your work but do not rely solely on them.
+5. All translation results can be previewed on the website one hour after they have been proofread.
+
+## Roles
+
+Here are the **roles** you can assume in the system:
+
+- **Language Coordinator** – Manages project features within assigned languages.
+- **Developer** – Uploads files, edits translatable text, connects integrations, and uses the API.
+- **Proofreader** – Translates and approves strings.
+- **Translator** (in-house or community) – Translates strings and votes on translations added by others.
+
+Our localization project is hosted on [Crowdin](https://crowdin.com/project/ton-docs).
+
+:::info
+Before you start contributing, **read the guidelines below** to ensure standardization and quality, making the review process much faster.
+
+## Side-by-Side Mode
+
+All tasks are performed in **side-by-side** mode in the Crowdin Editor. To enable this, click a file you want to work on. At the top right of the page, click the **Editor view** button and select **side-by-side** mode for a clearer editor view.  
+![side-by-side mode](/img/localizationProgramGuideline/side-by-side.png)
+:::
+
+### Language Coordinator
+- **Translate and approve strings**
+- **Pre-translate project content**
+- **Manage project members and join requests**
+  ![manage-members](/img/localizationProgramGuideline/manage-members.png)
+- **Generate project reports**
+  ![generate-reports](/img/localizationProgramGuideline/generate-reports.png)
+- **Create tasks**
+  ![create-tasks](/img/localizationProgramGuideline/create-tasks.png)
+
+### Developer
+- **Upload files**
+- **Edit translatable text**
+- **Connect integrations** (e.g., add GitHub integration)
+  ![install-github-integration](/img/localizationProgramGuideline/howItWorked/install-github-integration.png)
+- **Use the [Crowdin API](https://developer.crowdin.com/api/v2/)**
+
+### Proofreader
+
+As a **Proofreader**, you'll work on files with a **blue progress bar**.
+![proofread step1](/img/localizationProgramGuideline/proofread-step1.png)
+Click on a file to enter the editing interface.
+
+#### Let's Start Contributing
+
+1. Make sure you're in [**side-by-side mode**](#side-by-side-mode). Filter by **Not Approved** translations to see strings needing proofreading.
+![proofread filter](/img/localizationProgramGuideline/proofread-filter.png)
+
+2. Follow these rules:
+   - Select strings with a **blue cube icon**. Check each translation:
+      - If **correct**, click the ☑️ button.
+      - If **incorrect**, move to the next line.
+
+![proofread approved](/img/localizationProgramGuideline/proofread-approved.png)
+
+:::info
+You can also review approved lines:
+  1. Filter by **Approved**.
+  2. If an approved line has issues, click the ☑️ button to revert it to needing proofreading.
+:::
+
+3. To move to the next file, click the file name at the top, select the new file from the pop-up window, and continue proofreading.
+![to next](/img/localizationProgramGuideline/redirect-to-next.png)
+
+#### Previewing Your Work
+Every approved content will be deployed to a preview website within one hour. Check [**our repo**](https://github.com/TownSquareXYZ/ton-docs/pulls) for the **preview** link in the newest PR.
+![preview link](/img/localizationProgramGuideline/preview-link.png)
+
+### Translator
+
+As a **Translator**, your goal is to ensure translations are faithful and expressive, making them as close to the original meaning and as understandable as possible. Your mission is to make the **blue progress bar** reach 100%.
+
+#### Let's Start Translating
+
+Follow these steps for a successful translation process:
+
+1. Select files that haven't reached 100% translation.
+![translator select](/img/localizationProgramGuideline/translator-select.png)
+
+2. Ensure you're in [**side-by-side mode**](#side-by-side-mode). Filter by **Untranslated** strings.
+![translator filter](/img/localizationProgramGuideline/translator-filter.png)
+
+3. Your workspace has four parts:
+   - **Top left:** Input your translation based on the source string.
+   - **Bottom left:** Preview the translated file. Maintain the original format.
+   - **Bottom right:** Suggested translations from Crowdin. Click to use, but verify for accuracy, especially with links.
+  
+4. Save your translation by clicking the **Save** button at the top.
+![translator save](/img/localizationProgramGuideline/translator-save.png)
+
+5. To move to the next file, click the file name at the top and select the new file from the pop-up window.
+![to next](/img/localizationProgramGuideline/redirect-to-next.png)
+
+## How to Add Support for a New Language
+
+Currently, we have all desired languages in Crowdin. If you are a community manager, follow these steps:
+
+1. Add a new branch named `[lang]_localization` (e.g., `ko_localization` for Korean) on [TownSquareXYZ/ton-docs](https://github.com/TownSquareXYZ/ton-docs).
+2. **Contact the Vercel owner of this repo** to add the new language to the menu.
+3. Create a PR request to the dev branch. **Do not merge to dev**; this is for preview purposes only.
+
+Once these steps are completed, you can see the preview of your language in the PR request.
+![ko preview](/img/localizationProgramGuideline/ko_preview.png)
+
+When your language is ready for the TON docs, create an issue, and we'll set your language into the production environment.