1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
|
# anthy as a WebAssembly module
This directory vendors anthy-9100 (the kana-kanji conversion engine) built as a WASI
WebAssembly module, so the paper plugin can convert romaji to Japanese in-process: no
JNI, no second process, no per-platform native library.
```
src/ the C sources of the embedder API and the WASI shims (anthy's own
tree is NOT vendored here - see "Regenerating")
build-wasm.sh anthy + src/ -> prebuilt/anthy.wasm
prebuilt/anthy.wasm the module (wasm32-wasip1, reactor, ~630 KB)
prebuilt/anthy.dic the dictionary (22,897,008 B, sha256 4a6ba918...417f8a)
prebuilt/aot/ Chicory build-time AOT output, committed (see below)
```
## How the plugin uses it
`AnthyWasmConverter` (platform-paper, `converter/anthy/`) loads the module with Chicory,
a pure-JVM WebAssembly runtime, and calls the `lcanthy_*` exports declared in
`src/anthy_wasm.c`:
| export | role |
|---|---|
| `_initialize` | libc startup (constructors). Called once, right after instantiation |
| `lcanthy_alloc` / `lcanthy_free` | linear-memory allocation, so the host can stage strings |
| `lcanthy_set_dic` | dictionary path (relative to a WASI preopen) |
| `lcanthy_set_home` | `HOME` / `ANTHYDIR` (WASI has no passwd database) |
| `lcanthy_set_log` | anthy's log threshold (default 99 = silent) |
| `lcanthy_init` | `anthy_init`, once |
| `lcanthy_convert` | hiragana in, kanji out (UTF-8 byte lengths) |
| `lcanthy_segments` / `lcanthy_version` | inspection helpers |
The dictionary is committed here and packaged into the plugin jar; the engine extracts it into
the plugin data folder the first time it starts (`features.japaneseConversion.anthy.dictionary`,
default `anthy.dic`). An existing file there is never overwritten, so a server that uses a
different dictionary is not clobbered by an upgrade. The build copies it into the jar from this
one location, so the repository holds a single 22.9 MB copy.
## AOT (why prebuilt/aot is committed)
Chicory can run the module straight through its interpreter, but that measured ~1,000 ms
per conversion - the plugin's conversion budget is 1000 ms for the whole message, so the
interpreter is not usable. The build-time compiler turns the module into JVM bytecode
ahead of time; the generated machine plus a code-less ".meta" copy of the module (which
is all the runtime still needs) measured 30 ms per conversion and 406 ms of startup.
`prebuilt/aot/` holds
```
dev/m1sk9/lunaticChat/paper/converter/anthy/AnthyAotMachine*.class the generated machine
dev/m1sk9/lunaticChat/paper/converter/anthy/AnthyAot.meta module structure only
```
`platform-paper/build.gradle.kts` puts the classes on the classpath and the `.meta` into
the jar next to them (that is where the generated code looks for it).
## Regenerating
The wasm module and the AOT output are committed so that `./gradlew build` works on a
machine with no wasm toolchain and no anthy source. Regenerate only when anthy, the
dictionary format, or the embedder API changes.
```sh
# 1. the module (needs clang with wasm32 + a WASI sysroot + an anthy source tree)
ANTHY_SRC=/ROM/anthy-build/anthy-9100 ./anthy/build-wasm.sh
# 2. the AOT classes and the .meta, from the plugin build
cd .. && ./gradlew :platform-paper:anthyAot -PanthyAot=true
```
Both steps are also what a future Gradle task would automate; they are kept explicit
because they need inputs (the toolchain, the anthy tree, the 22.9 MB dictionary) that do
not belong in this repository.
## Notes and traps (measured)
- **Do not use the FreeBSD port's `wasi-libc19`**: its `libc.a` is built with
`-fstack-protector-strong`, which breaks wasi-libc's own SSP constructor
(`__wasilibc_init_ssp` writes `__stack_chk_guard`, then its own canary check fails and
traps in `__wasm_call_ctors`). Measured under both node's WASI and wasmtime, so it is
the library, not the runtime. Use the reference wasi-sdk sysroot (25.0 here).
- **anthy's source is unmodified.** `src/shim/pwd.h`, `src/shim/fcntl.h` and
`src/shim_posix.c` supply what WASI lacks: a passwd database, record-lock constants
(`F_WRLCK`/`F_SETLKW`, hidden in the sysroot under `__wasilibc_unmodified_upstream`),
and `getuid`/`getpwuid`/`gethostname`.
- **`filemap.c` needs mmap**, which WASI only provides through the emulated mman library
(`-lwasi-emulated-mman`). The 22.9 MB dictionary is mapped read-only and read into the
module's linear memory; that is also why each module instance costs ~25 MB of heap and
instances cannot share the dictionary.
- The 4-byte `anthy.dic` in the root of an anthy source tree is a stub; pointing
`DIC_FILE` at it makes `anthy_init` fail. Use `mkanthydic/anthy.dic`.
## License
anthy is LGPL-2.1, the dictionaries (alt-cannadic) are GPL, LunaticChat is GPL-3.0, so
redistribution is compatible. The module is a derivative of anthy: ship its license and
point at the source (this directory plus the anthy-9100 tarball it was built from).
|
