finish-mds for full project #3
@@ -1 +1,122 @@
|
||||
# Contributing to Core
|
||||
|
||||
`foreignthon-core` is the transpiler engine, CLI, and pack loader. Contributions that improve correctness, performance, or usability are welcome.
|
||||
|
||||
---
|
||||
|
||||
## Repository
|
||||
|
||||
[https://git.keshavanand.net/foreign-thon/foreignthon-core](https://git.keshavanand.net/foreign-thon/foreignthon-core)
|
||||
|
||||
---
|
||||
|
||||
## Dev setup
|
||||
|
||||
```bash
|
||||
git clone https://git.keshavanand.net/foreign-thon/foreignthon-core
|
||||
cd foreignthon-core
|
||||
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate # Windows: .venv\Scripts\activate
|
||||
|
||||
pip install -e ".[dev]" # installs foreignthon + pytest + ruff
|
||||
```
|
||||
|
||||
Verify the install:
|
||||
|
||||
```bash
|
||||
fpy --version
|
||||
pytest
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Running tests
|
||||
|
||||
```bash
|
||||
pytest # all tests
|
||||
pytest -v # verbose
|
||||
pytest -k postfix # filter by name
|
||||
```
|
||||
|
||||
Tests live in `tests/test_engine.py`. They use a local `test_pack.json` fixture (a copy of the Spanish pack) so no installed language pack is needed.
|
||||
|
||||
---
|
||||
|
||||
## Code style
|
||||
|
||||
ForeignThon uses [Ruff](https://docs.astral.sh/ruff/) for linting and formatting:
|
||||
|
||||
```bash
|
||||
ruff check src/ # lint
|
||||
ruff format src/ # format
|
||||
```
|
||||
|
||||
The CI gate runs both. A PR that fails either will not be merged.
|
||||
|
||||
Line length is 88. Import order follows the `I` rule set (isort-compatible).
|
||||
|
||||
---
|
||||
|
||||
## Project layout
|
||||
|
||||
```
|
||||
foreignthon-core/
|
||||
├── src/
|
||||
│ └── foreignthon/
|
||||
│ ├── __init__.py # version
|
||||
│ ├── cli.py # click commands (fpy)
|
||||
│ ├── errors.py # bilingual error hook
|
||||
│ ├── pack.py # pack discovery and validation
|
||||
│ ├── template.json # canonical list of all pack keys
|
||||
│ └── transpiler.py # tokenizer-based engine
|
||||
└── tests/
|
||||
├── test_engine.py
|
||||
└── test_pack.json # Spanish fixture (no install needed)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Adding new keywords or builtins
|
||||
|
||||
`template.json` is the **single source of truth** for the full set of keywords, builtins, exceptions, and stdlib names that all language packs must cover.
|
||||
|
||||
To add a new entry (e.g. a new builtin):
|
||||
|
||||
1. Add it to `template.json` with the English value as the default.
|
||||
2. Add a matching entry to `test_pack.json` (Spanish values).
|
||||
3. Update `_build_mapping()` in `transpiler.py` if a new section is needed.
|
||||
4. Bump `foreignthon-core` version.
|
||||
5. Announce to language pack maintainers so they can update their packs.
|
||||
|
||||
Do not add entries to `template.json` that are specific to one language pack.
|
||||
|
||||
---
|
||||
|
||||
## Adding a new CLI command
|
||||
|
||||
All commands are defined in `cli.py` using [Click](https://click.palletsprojects.com/).
|
||||
|
||||
- Add your command as a function decorated with `@main.command()`.
|
||||
- Include a docstring — Click uses it as the `--help` text.
|
||||
- Add `CONTEXT_SETTINGS` to every command for consistent `-h` support.
|
||||
- Write tests in `tests/test_engine.py` or a new test file.
|
||||
|
||||
---
|
||||
|
||||
## Submitting a PR
|
||||
|
||||
1. Fork the repository on Gitea.
|
||||
2. Create a branch: `git checkout -b feature/my-change`.
|
||||
3. Make your change, add or update tests, run `pytest` and `ruff check`.
|
||||
4. Open a PR with a clear description of the problem and the fix.
|
||||
|
||||
For large changes (new features, engine behaviour changes), open an issue first to discuss before writing code.
|
||||
|
||||
---
|
||||
|
||||
## What not to change
|
||||
|
||||
- **`template.json`** keys should only grow, never shrink — removing a key is a breaking change for all existing language packs.
|
||||
- The tokenizer-based approach in `transpiler.py` is intentional. Do not replace it with a regex or AST-based approach without a very strong reason — the tokenizer correctly handles strings, comments, and f-strings without any special casing.
|
||||
- The bilingual error format (`[XX] ForeignName: msg` / `[EN] EnglishName: msg`) is part of the public interface. Do not change the format without a major version bump.
|
||||
|
||||
@@ -1 +1,189 @@
|
||||
# Contributing Language Packs
|
||||
|
||||
Language packs are published independently to PyPI — no access to the `foreignthon-core` repo is needed. There are more packs than the ones listed here; the ones on the official docs site are the ones hosted under the [`foreign-thon` org on Gitea](https://git.keshavanand.net/foreign-thon/).
|
||||
|
||||
---
|
||||
|
||||
## The right starting point
|
||||
|
||||
**Do not start from scratch.** Fork the official language template:
|
||||
|
||||
```
|
||||
https://git.keshavanand.net/foreign-thon/language-template.git
|
||||
```
|
||||
|
||||
The template gives you:
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `src/foreignthon_xx/__init__.py` | Exposes `get_pack_path()` and reads version/author metadata |
|
||||
| `src/foreignthon_xx/xx.json` | The pack file — all keys are English stubs to replace |
|
||||
| `tests/test_pack.py` | Universal test suite shared by every official pack |
|
||||
| `.gitea/workflows/ci.yml` | Runs `pytest` on every push and PR |
|
||||
| `.gitea/workflows/publish.yml` | Builds and uploads to PyPI on a `v*` tag |
|
||||
| `.gitea/workflows/trigger-docs.yml` | Pings the docs site to rebuild when `README.md` changes |
|
||||
| `pyproject.toml` | Pre-wired entry points, hatchling build, GPL v3 |
|
||||
|
||||
---
|
||||
|
||||
## Step 1 — Fork and rename
|
||||
|
||||
Fork the template on Gitea (or GitHub mirror). Rename your fork to `foreignthon-xx` where `xx` is the [ISO 639-1](https://en.wikipedia.org/wiki/List_of_ISO_639_language_codes) two-letter code for your language (use ISO 639-2 three-letter code if no two-letter code exists).
|
||||
|
||||
Clone your fork locally:
|
||||
|
||||
```bash
|
||||
git clone https://git.keshavanand.net/yourname/foreignthon-xx
|
||||
cd foreignthon-xx
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 2 — Rename the template files
|
||||
|
||||
The template uses placeholder names. Rename:
|
||||
|
||||
```
|
||||
src/foreignthon_xx/xx.json → src/foreignthon_fr/fr.json (example: French)
|
||||
src/foreignthon_xx/ → src/foreignthon_fr/
|
||||
```
|
||||
|
||||
Then update every occurrence of `xx` in:
|
||||
|
||||
- `pyproject.toml` — `name`, `entry-points` key, `packages` path, `keywords`, `description`
|
||||
- `src/foreignthon_fr/__init__.py` — the `get_pack_path()` return value
|
||||
|
||||
---
|
||||
|
||||
## Step 3 — Fill in the JSON
|
||||
|
||||
Open `src/foreignthon_fr/fr.json`. Every value is an English Python identifier — **never change the values**. Replace the **keys** with your language's words.
|
||||
|
||||
```json
|
||||
{
|
||||
"meta": {
|
||||
"name": "French",
|
||||
"native_name": "Français",
|
||||
"code": "fr"
|
||||
},
|
||||
"keywords": {
|
||||
"si": "if",
|
||||
"sinon": "else",
|
||||
...
|
||||
},
|
||||
"builtins": { ... },
|
||||
"exceptions": { ... },
|
||||
"error_messages": { ... },
|
||||
"stdlib": { ... },
|
||||
"postfix_keywords": []
|
||||
}
|
||||
```
|
||||
|
||||
The filename must exactly match `meta.code` (e.g. `fr.json` for code `"fr"`).
|
||||
|
||||
Set `postfix_keywords` to a list of English keywords (e.g. `["if", "elif", "while"]`) if your language is SOV and benefits from `@@` postfix style on decompile. Set it to `[]` for SVO languages.
|
||||
|
||||
For a full reference on the JSON schema see [Custom Packs → Pack schema](../custom-packs.md#pack-schema).
|
||||
|
||||
---
|
||||
|
||||
## Step 4 — Validate and test
|
||||
|
||||
Install your pack in editable mode and run the test suite:
|
||||
|
||||
```bash
|
||||
pip install -e .
|
||||
pip install pytest
|
||||
|
||||
fpy pack src/foreignthon_fr/fr.json
|
||||
# ✓ Pack 'French' is valid.
|
||||
|
||||
pytest tests/ -v
|
||||
```
|
||||
|
||||
The shared test suite (`tests/test_pack.py`) checks:
|
||||
|
||||
- All required sections exist (`meta`, `keywords`, `builtins`, `exceptions`, `error_messages`, `stdlib`, `postfix_keywords`)
|
||||
- Every keyword value is a real Python keyword
|
||||
- Every builtin/exception value is a real Python builtin
|
||||
- The filename matches `meta.code`
|
||||
- All `postfix_keywords` entries have a matching translation in `keywords`
|
||||
|
||||
The CI (`ci.yml`) runs this same suite on every push and pull request automatically.
|
||||
|
||||
---
|
||||
|
||||
## Step 5 — Publish to PyPI
|
||||
|
||||
You can publish from your own fork at any time — the pack works as soon as it is on PyPI, regardless of whether it is in the official org.
|
||||
|
||||
Tag a release and push:
|
||||
|
||||
```bash
|
||||
git tag v0.1.0
|
||||
git push origin v0.1.0
|
||||
```
|
||||
|
||||
The `publish.yml` workflow builds the wheel and uploads it to PyPI using the `PYPI_TOKEN` secret. If you are working from your personal fork you will need to add your own `PYPI_TOKEN` secret in your fork's Gitea/GitHub settings.
|
||||
|
||||
Once published, anyone can install your pack:
|
||||
|
||||
```bash
|
||||
pip install foreignthon-fr
|
||||
fpy new myproject --lang fr
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Getting hosted under the official org
|
||||
|
||||
The `foreign-thon` org on Gitea is where official packs live and where the org-level CI secrets (`PYPI_TOKEN`, `GIT_RELEASE_TOKEN`, `DOCS_TRIGGER_TOKEN`) are inherited automatically.
|
||||
|
||||
**Nobody has direct repo-creation access in the org.** To get your pack hosted there:
|
||||
|
||||
1. Make sure your pack is working and published to PyPI.
|
||||
2. Open an issue or PR on [foreignthon-docs](https://git.keshavanand.net/foreign-thon/foreignthon-docs) requesting inclusion, and include:
|
||||
- Your language code, English name, and native name
|
||||
- A link to your published PyPI package
|
||||
- Your Gitea handle
|
||||
3. The maintainer will transfer or mirror your repo into the org and wire up the secrets.
|
||||
|
||||
---
|
||||
|
||||
## Getting listed on the docs site
|
||||
|
||||
The docs site table in [Language Packs → Overview](../language-packs/index.md) is maintained in `languages.yml` in the `foreignthon-docs` repo. Open a PR to add your entry:
|
||||
|
||||
```yaml
|
||||
- code: fr
|
||||
name: French
|
||||
repo: foreignthon-fr
|
||||
```
|
||||
|
||||
The `trigger-docs.yml` workflow in your pack repo fires a rebuild of the docs site whenever `README.md` is pushed to `main`.
|
||||
|
||||
---
|
||||
|
||||
## Quality bar
|
||||
|
||||
Before submitting for listing, your pack should satisfy:
|
||||
|
||||
- [ ] `fpy pack xx.json` passes
|
||||
- [ ] `pytest tests/` passes
|
||||
- [ ] Every foreign key is unique within the pack
|
||||
- [ ] No English words used as foreign keys unless the language genuinely keeps them (e.g. `lambda`)
|
||||
- [ ] `meta.code` matches the JSON filename and the entry-point key in `pyproject.toml`
|
||||
- [ ] Package name is `foreignthon-xx` (hyphen), module name is `foreignthon_xx` (underscore)
|
||||
- [ ] `pip install foreignthon-xx` works from PyPI
|
||||
|
||||
---
|
||||
|
||||
## Versioning and updates
|
||||
|
||||
When `foreignthon-core` releases a new version that adds entries to `template.json`, update your pack to cover them and release a new version. Update the `foreignthon` lower-bound dependency in `pyproject.toml` if your pack requires the new core:
|
||||
|
||||
```toml
|
||||
dependencies = ["foreignthon>=0.5.4"]
|
||||
```
|
||||
|
||||
Follow [Semantic Versioning](https://semver.org/). Use `0.x.y` while the keyword set is still being refined; move to `1.0.0` once stable.
|
||||
|
||||
@@ -92,4 +92,4 @@ fpy pack custom.json
|
||||
|
||||
Once your `custom.json` is complete and working, you can turn it into a proper `foreignthon-xx` package on PyPI so others can install it with `pip install foreignthon-xx`.
|
||||
|
||||
See [Contributing → Language Packs](contributing/language-packs.md) for the full guide. Custom Packs
|
||||
See [Contributing → Language Packs](contributing/language-packs.md) for the full guide.
|
||||
|
||||
@@ -1 +1,160 @@
|
||||
# Architecture
|
||||
|
||||
This page describes how `foreignthon-core` works internally.
|
||||
|
||||
---
|
||||
|
||||
## Pipeline
|
||||
|
||||
```
|
||||
source.xx.py
|
||||
│
|
||||
▼
|
||||
_check_shebang() ← reads "# foreignthon: xx" if present
|
||||
│
|
||||
▼
|
||||
load_pack(lang_code) ← discovers + validates the JSON pack
|
||||
│
|
||||
▼
|
||||
_apply_postfix_syntax() ← rewrites "expr @@keyword:" lines
|
||||
│
|
||||
▼
|
||||
_swap_tokens() ← tokenizer pass: replaces NAME tokens
|
||||
│
|
||||
▼
|
||||
standard Python string ← ready to compile or write to disk
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Module overview
|
||||
|
||||
| Module | Responsibility |
|
||||
|---|---|
|
||||
| `transpiler.py` | The engine — postfix rewriter and tokenizer pass |
|
||||
| `pack.py` | Pack discovery, loading, and validation |
|
||||
| `cli.py` | Click commands (`fpy run`, `fpy compile`, etc.) |
|
||||
| `errors.py` | Bilingual exception hook |
|
||||
| `template.json` | Canonical set of all keywords/builtins a pack must cover |
|
||||
|
||||
---
|
||||
|
||||
## Tokenizer-based translation
|
||||
|
||||
ForeignThon uses Python's standard `tokenize` module rather than regex or AST manipulation.
|
||||
|
||||
`tokenize.generate_tokens()` splits source code into typed tokens. ForeignThon only looks at `NAME` tokens — identifiers. It replaces any `NAME` token whose string appears as a key in the active pack mapping. All other token types (strings, comments, operators, numbers) pass through unchanged.
|
||||
|
||||
This gives three important guarantees:
|
||||
|
||||
1. **Strings are safe.** A keyword inside `"..."` or `f"..."` is a `STRING` token, never a `NAME` — it is never touched.
|
||||
2. **Comments are safe.** Comment tokens are passed through verbatim.
|
||||
3. **Variable names are safe.** A variable like `si_condition` contains `si` only as a substring; as a `NAME` token it is `si_condition`, which is not in the mapping.
|
||||
|
||||
The whitespace between tokens is preserved by tracking `(row, col)` positions and copying the gaps from the original source.
|
||||
|
||||
---
|
||||
|
||||
## Pack discovery
|
||||
|
||||
Language packs register themselves using Python [entry points](https://packaging.python.org/en/latest/specifications/entry-points/):
|
||||
|
||||
```toml
|
||||
# in foreignthon-es/pyproject.toml
|
||||
[project.entry-points."foreignthon.langs"]
|
||||
es = "foreignthon_es"
|
||||
```
|
||||
|
||||
`pack.py` calls `importlib.metadata.entry_points(group="foreignthon.langs")` at runtime to discover all installed packs. Installing a pack is sufficient — no configuration file needs to be edited.
|
||||
|
||||
Each pack module must expose:
|
||||
|
||||
```python
|
||||
def get_pack_path() -> Path:
|
||||
return files(__name__) / "xx.json"
|
||||
```
|
||||
|
||||
The core calls `get_pack_path()` to locate the JSON, loads it, and validates that all required sections are present.
|
||||
|
||||
Results are cached with `@lru_cache` so each pack is loaded at most once per process.
|
||||
|
||||
---
|
||||
|
||||
## Pack mapping
|
||||
|
||||
Four sections of the JSON are merged into a single flat dict for translation:
|
||||
|
||||
```python
|
||||
mapping = {}
|
||||
mapping.update(pack["keywords"])
|
||||
mapping.update(pack["builtins"])
|
||||
mapping.update(pack["exceptions"])
|
||||
mapping.update(pack["stdlib"])
|
||||
```
|
||||
|
||||
The merged mapping is `{ foreign_word: english_word }`. It is passed directly to `_swap_tokens()`.
|
||||
|
||||
If two sections define the same foreign key, later sections win (stdlib last). In practice this does not occur because pack authors ensure uniqueness.
|
||||
|
||||
---
|
||||
|
||||
## Postfix syntax (`@@`)
|
||||
|
||||
The `@@` operator is a source-level pre-processing step that runs **before** tokenization.
|
||||
|
||||
A line like:
|
||||
|
||||
```
|
||||
x > 0 @@si:
|
||||
escribir(x)
|
||||
```
|
||||
|
||||
is rewritten to:
|
||||
|
||||
```
|
||||
si x > 0:
|
||||
escribir(x)
|
||||
```
|
||||
|
||||
The rewriter uses a regex that matches `(.+?)@@(<keyword>)` and moves the keyword to the front. It only operates on lines that contain `@@`, preserving indentation and line endings.
|
||||
|
||||
`@@` is never valid Python and never appears in the tokenizer output.
|
||||
|
||||
**Decompile direction:** `fpy decompile --postfix` does the reverse — it looks for lines of the form `foreign_kw expr:` where `foreign_kw` is in the pack's `postfix_keywords` list, and rewrites them to `expr @@foreign_kw:`.
|
||||
|
||||
---
|
||||
|
||||
## Bilingual error hook
|
||||
|
||||
`errors.py` installs a custom `sys.excepthook` before running user code:
|
||||
|
||||
1. On exception, it looks up the exception type name in the pack's `exceptions` section (reverse map: English → foreign).
|
||||
2. It looks up a translated message in `error_messages`.
|
||||
3. It prints `[XX] ForeignName: translated_msg` then `[EN] EnglishName: original_msg`.
|
||||
4. It always calls `traceback.print_exception()` afterwards so the full traceback is shown.
|
||||
|
||||
Tracebacks point to the original `.xx.py` file. This is achieved by populating `linecache.cache` with the original source before `exec()`-ing the compiled code, so Python's traceback machinery reads the right lines.
|
||||
|
||||
---
|
||||
|
||||
## Custom pack override
|
||||
|
||||
When `.foreignthon.toml` declares `custom_pack = "path/to/custom.json"`:
|
||||
|
||||
- If the custom JSON has `meta.code` set, it is treated as a **standalone pack** and used directly.
|
||||
- If `meta.code` is absent, it is treated as an **override** — it is merged on top of the installed pack, replacing only the keys it defines.
|
||||
|
||||
The CLI (`cli.py`) handles this in `_load_effective_pack()` by walking up the directory tree to find `.foreignthon.toml`.
|
||||
|
||||
---
|
||||
|
||||
## File naming and language detection
|
||||
|
||||
Language detection order (highest priority first):
|
||||
|
||||
1. `--lang` CLI flag
|
||||
2. Shebang comment `# foreignthon: xx` on the first line
|
||||
3. Double extension `.xx.py` → `xx`
|
||||
4. Fallback to `"en"` (no-op — English is Python)
|
||||
|
||||
`_detect_lang()` and `_check_shebang()` in `transpiler.py` implement steps 3 and 2 respectively. Step 1 is handled by the `run` command in `cli.py`.
|
||||
|
||||
@@ -1 +1,136 @@
|
||||
# Releasing
|
||||
|
||||
This page covers how to release `foreignthon-core` and how language pack maintainers should respond to a new core release.
|
||||
|
||||
---
|
||||
|
||||
## Core release (`foreignthon`)
|
||||
|
||||
### 1. Update the version
|
||||
|
||||
Version is set in `pyproject.toml`:
|
||||
|
||||
```toml
|
||||
[project]
|
||||
name = "foreignthon"
|
||||
version = "0.5.4" # bump this
|
||||
```
|
||||
|
||||
ForeignThon follows [Semantic Versioning](https://semver.org/):
|
||||
|
||||
| Change | Version bump |
|
||||
|---|---|
|
||||
| Bug fix, no API change | Patch (`0.5.3` → `0.5.4`) |
|
||||
| New feature, backward compatible | Minor (`0.5.x` → `0.6.0`) |
|
||||
| Breaking change (removes a key, changes CLI contract) | Major (`0.x.y` → `1.0.0`) |
|
||||
|
||||
### 2. Update `template.json` if needed
|
||||
|
||||
If the release adds new keywords, builtins, or sections, update `template.json` to include them. This is the canonical list all language packs are expected to cover. Document what changed in your commit message and PR so pack maintainers know what to add.
|
||||
|
||||
### 3. Run CI locally
|
||||
|
||||
```bash
|
||||
pytest
|
||||
ruff check src/
|
||||
ruff format --check src/
|
||||
```
|
||||
|
||||
All must pass.
|
||||
|
||||
### 4. Tag and push
|
||||
|
||||
The Gitea CI publishes to PyPI on any tag matching `v*`:
|
||||
|
||||
```bash
|
||||
git tag v0.5.4
|
||||
git push origin v0.5.4
|
||||
```
|
||||
|
||||
The `publish.yml` workflow builds the wheel and uploads it using the `PYPI_TOKEN` secret configured in Gitea.
|
||||
|
||||
### 5. Verify
|
||||
|
||||
```bash
|
||||
pip install --upgrade foreignthon
|
||||
fpy --version
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Language pack release (`foreignthon-xx`)
|
||||
|
||||
Each language pack is independently versioned and released.
|
||||
|
||||
### Responding to a core release
|
||||
|
||||
When `foreignthon-core` adds new entries to `template.json`:
|
||||
|
||||
1. Add the new keys to your `xx.json` with translations.
|
||||
2. Run `fpy pack src/foreignthon_xx/xx.json` to validate.
|
||||
3. Run `pytest tests/` to confirm all checks pass.
|
||||
4. Bump your pack's version in `pyproject.toml`.
|
||||
5. Tag and push — the `publish.yml` workflow handles PyPI upload.
|
||||
|
||||
Update the `foreignthon` dependency lower bound in `pyproject.toml` if your pack requires the new core version:
|
||||
|
||||
```toml
|
||||
dependencies = ["foreignthon>=0.5.4"]
|
||||
```
|
||||
|
||||
### Releasing independently
|
||||
|
||||
You can release a pack at any time — to fix a translation, add missing entries, or extend coverage. The process is the same:
|
||||
|
||||
```bash
|
||||
# bump version in pyproject.toml
|
||||
git add pyproject.toml src/foreignthon_xx/xx.json
|
||||
git commit -m "v0.3.3: fix translation for 'return'"
|
||||
git tag v0.3.3
|
||||
git push origin v0.3.3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Docs release (`foreignthon-docs`)
|
||||
|
||||
The docs site rebuilds automatically when a language pack fires its `trigger-docs.yml` workflow after a successful publish. No manual step is needed for routine pack releases.
|
||||
|
||||
To manually trigger a docs rebuild (e.g. after editing `foreignthon-docs` directly):
|
||||
|
||||
```bash
|
||||
git push origin main
|
||||
```
|
||||
|
||||
The `deploy.yml` workflow builds the MkDocs site and deploys it to [foreignthon.keshavanand.net](https://foreignthon.keshavanand.net).
|
||||
|
||||
---
|
||||
|
||||
## Gitea secrets
|
||||
|
||||
| Secret | Used by |
|
||||
|---|---|
|
||||
| `PYPI_TOKEN` | `publish.yml` in core and all packs |
|
||||
| `DOCS_WEBHOOK` | `trigger-docs.yml` in language packs |
|
||||
|
||||
Secrets are set per-repository in **Gitea → Settings → Actions → Secrets**.
|
||||
|
||||
---
|
||||
|
||||
## Checklist
|
||||
|
||||
### Core
|
||||
- [ ] Version bumped in `pyproject.toml`
|
||||
- [ ] `template.json` updated if new keys added
|
||||
- [ ] `pytest` passes
|
||||
- [ ] `ruff check` passes
|
||||
- [ ] Tagged `v*` and pushed
|
||||
- [ ] PyPI upload confirmed
|
||||
|
||||
### Language pack
|
||||
- [ ] New `template.json` entries translated in `xx.json`
|
||||
- [ ] `fpy pack xx.json` passes
|
||||
- [ ] `pytest tests/` passes
|
||||
- [ ] Version bumped
|
||||
- [ ] `foreignthon` dependency lower bound updated if required
|
||||
- [ ] Tagged and pushed
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
# Coming soon
|
||||
@@ -1 +1,240 @@
|
||||
# Add Your Language
|
||||
|
||||
Anyone can publish a `foreignthon-xx` language pack to PyPI — no access to the core repo is needed, and the pack works for anyone who installs it the moment it is on PyPI.
|
||||
|
||||
The recommended path is to start from the official language template. See [Contributing → Language Packs](../contributing/language-packs.md) for the full contribution and review process. This page is a quick reference.
|
||||
|
||||
---
|
||||
|
||||
## Start from the template
|
||||
|
||||
Fork the official language template on Gitea:
|
||||
|
||||
```
|
||||
https://git.keshavanand.net/foreign-thon/language-template.git
|
||||
```
|
||||
|
||||
The template gives you the full correct structure — `__init__.py`, JSON pack, the universal `tests/test_pack.py`, and all three CI workflows — pre-wired and ready to rename.
|
||||
|
||||
!!! tip "Why fork instead of starting fresh?"
|
||||
The template includes the shared test suite (`tests/test_pack.py`) that every official pack uses. Starting from it means your CI is already correct and your pack will pass the same checks as `foreignthon-es` and `foreignthon-ta` from day one.
|
||||
|
||||
Alternatively, if you just want a local pack for your own project without publishing, use:
|
||||
|
||||
```bash
|
||||
fpy new myproject --lang custom
|
||||
```
|
||||
|
||||
This scaffolds a `custom.json` locally — no PyPI account needed. See [Custom Packs](../custom-packs.md) for details.
|
||||
|
||||
---
|
||||
|
||||
## Package structure
|
||||
|
||||
After renaming the template for your language (e.g. French, code `fr`):
|
||||
|
||||
```
|
||||
foreignthon-fr/
|
||||
├── .gitea/workflows/
|
||||
│ ├── ci.yml # pytest on push / PR
|
||||
│ ├── publish.yml # PyPI upload on v* tag
|
||||
│ └── trigger-docs.yml # docs rebuild on README.md change
|
||||
├── .gitignore
|
||||
├── LICENSE # GPL v3
|
||||
├── README.md
|
||||
├── pyproject.toml
|
||||
└── src/
|
||||
└── foreignthon_fr/
|
||||
├── __init__.py
|
||||
└── fr.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `pyproject.toml`
|
||||
|
||||
```toml
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "foreignthon-fr"
|
||||
version = "0.1.0"
|
||||
description = "French language pack for ForeignThon."
|
||||
license = { text = "GPL v3" }
|
||||
requires-python = ">=3.9"
|
||||
authors = [
|
||||
{ name = "Your Name", email = "you@example.com" }
|
||||
]
|
||||
keywords = ["foreignthon", "french", "français"]
|
||||
dependencies = ["foreignthon>=0.5.3"]
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://git.keshavanand.net/foreign-thon/foreignthon-fr"
|
||||
|
||||
[project.entry-points."foreignthon.langs"]
|
||||
fr = "foreignthon_fr"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["src/foreignthon_fr"]
|
||||
```
|
||||
|
||||
The `[project.entry-points."foreignthon.langs"]` block is what makes ForeignThon auto-discover your pack — the key is the language code, the value is the Python module name.
|
||||
|
||||
---
|
||||
|
||||
## `__init__.py`
|
||||
|
||||
```python
|
||||
from importlib.resources import files
|
||||
from importlib.metadata import version, metadata, PackageNotFoundError
|
||||
|
||||
try:
|
||||
package_name = (__package__ or "").replace("_", "-")
|
||||
__version__ = version(package_name)
|
||||
|
||||
pkg_metadata = metadata(package_name)
|
||||
raw_authors = pkg_metadata.get_all("Author") or []
|
||||
raw_emails = pkg_metadata.get_all("Author-email") or []
|
||||
|
||||
combined = []
|
||||
for item in (raw_authors + raw_emails):
|
||||
clean_name = item.split("<")[0].strip()
|
||||
if clean_name and clean_name not in combined:
|
||||
combined.append(clean_name)
|
||||
|
||||
__authors__ = combined
|
||||
|
||||
except PackageNotFoundError:
|
||||
__version__ = "0.0.0"
|
||||
__authors__ = []
|
||||
|
||||
|
||||
def get_pack_path():
|
||||
return files(__name__) / "fr.json"
|
||||
```
|
||||
|
||||
Change `"fr.json"` to match your actual filename.
|
||||
|
||||
---
|
||||
|
||||
## The JSON pack
|
||||
|
||||
Keys are your language's words. Values are English Python identifiers — **never change the values**.
|
||||
|
||||
```json
|
||||
{
|
||||
"meta": {
|
||||
"name": "French",
|
||||
"native_name": "Français",
|
||||
"code": "fr"
|
||||
},
|
||||
"keywords": {
|
||||
"si": "if",
|
||||
"sinon": "else",
|
||||
"sinonsi": "elif",
|
||||
"pour": "for",
|
||||
"tantque": "while",
|
||||
"déf": "def",
|
||||
"classe": "class",
|
||||
"importer": "import",
|
||||
"depuis": "from",
|
||||
"comme": "as",
|
||||
"retourner": "return",
|
||||
"arrêter": "break",
|
||||
"continuer": "continue",
|
||||
"passer": "pass",
|
||||
"essayer": "try",
|
||||
"sauf": "except",
|
||||
"finalement": "finally",
|
||||
"lever": "raise",
|
||||
"avec": "with",
|
||||
"dans": "in",
|
||||
"est": "is",
|
||||
"et": "and",
|
||||
"ou": "or",
|
||||
"non": "not",
|
||||
"supprimer": "del",
|
||||
"global": "global",
|
||||
"nonlocal": "nonlocal",
|
||||
"affirmer": "assert",
|
||||
"générer": "yield",
|
||||
"attendre": "await",
|
||||
"asynchrone": "async",
|
||||
"lambda": "lambda",
|
||||
"Vrai": "True",
|
||||
"Faux": "False",
|
||||
"Rien": "None"
|
||||
},
|
||||
"builtins": {
|
||||
"afficher": "print",
|
||||
"saisir": "input",
|
||||
"longueur": "len",
|
||||
"intervalle": "range",
|
||||
...
|
||||
},
|
||||
"exceptions": {
|
||||
"ErreurDeValeur": "ValueError",
|
||||
...
|
||||
},
|
||||
"error_messages": {
|
||||
"ValueError": "Erreur de valeur",
|
||||
...
|
||||
},
|
||||
"stdlib": {
|
||||
"mathématiques": "math",
|
||||
...
|
||||
},
|
||||
"postfix_keywords": []
|
||||
}
|
||||
```
|
||||
|
||||
### `postfix_keywords`
|
||||
|
||||
Set this to a list of English keywords if your language is SOV (subject-object-verb) and benefits from `@@` postfix style on decompile. Example (Tamil):
|
||||
|
||||
```json
|
||||
"postfix_keywords": ["if", "elif", "while", "class", "with"]
|
||||
```
|
||||
|
||||
Set it to `[]` for SVO languages like Spanish or French.
|
||||
|
||||
### Filename rule
|
||||
|
||||
The JSON filename **must** match `meta.code` exactly. `fr.json` for code `"fr"`.
|
||||
|
||||
---
|
||||
|
||||
## Validate and test
|
||||
|
||||
```bash
|
||||
pip install -e .
|
||||
pip install pytest
|
||||
|
||||
fpy pack src/foreignthon_fr/fr.json
|
||||
# ✓ Pack 'French' is valid.
|
||||
|
||||
pytest tests/ -v
|
||||
```
|
||||
|
||||
The shared `tests/test_pack.py` checks that all sections exist, all values are real Python identifiers, the filename matches `meta.code`, and `postfix_keywords` entries each have a translation.
|
||||
|
||||
---
|
||||
|
||||
## Publish
|
||||
|
||||
Tag and push — the CI handles the rest:
|
||||
|
||||
```bash
|
||||
git tag v0.1.0
|
||||
git push origin v0.1.0
|
||||
```
|
||||
|
||||
The `publish.yml` workflow builds the wheel and uploads it to PyPI using the `PYPI_TOKEN` secret. Add your own `PYPI_TOKEN` to your fork's secrets if you are not yet under the official org.
|
||||
|
||||
---
|
||||
|
||||
## Getting listed
|
||||
|
||||
Once your pack is working and on PyPI, open a PR or issue on [foreignthon-docs](https://git.keshavanand.net/foreign-thon/foreignthon-docs) to get added to the [Language Packs overview](index.md). See [Contributing → Language Packs](../contributing/language-packs.md) for the full process.
|
||||
|
||||
@@ -84,4 +84,4 @@ Which keywords get rewritten is controlled by the `postfix_keywords` list in the
|
||||
| Direction | Mechanism | Controlled by |
|
||||
|---|---|---|
|
||||
| Input (writing `.xx.py`) | `@@` in source | Always available for any keyword |
|
||||
| Output (`fpy decompile --postfix`) | Pack's `postfix_keywords` | Language pack author | Postfix Syntax
|
||||
| Output (`fpy decompile --postfix`) | Pack's `postfix_keywords` | Language pack author |
|
||||
|
||||
Reference in New Issue
Block a user