# Blau Dictionary JSON v6 → TEI P5 mapping

## Scope

This document describes Blau Digital Dictionary release 1.1 (2026-08-25), TEI mapping 1.3. The export targets TEI P5 4.12.0. The application JSON remains the canonical working data model; TEI is the interoperable scholarly export layer. Persistent JSON identifiers are reused verbatim as `xml:id` values.

## Core hierarchy

| JSON v6 | TEI |
|---|---|
| Dictionary | `<TEI><text><body><div type="dictionary">` |
| Entry | `<entry xml:id="blau-e-…">` |
| `name_ja` | `<form type="lemma"><orth xml:lang="jrb-Hebr">` |
| `arabic_spelling` | `<orth xml:lang="ar-Arab">` |
| Subentry | nested `<entry type="subentry" xml:id="blau-se-…">` |
| Stem | `<sense type="stem" xml:id="blau-st-…">` |
| JSON `form` | nested `<sense type="form" xml:id="blau-f-…">` |
| Orthographic variant | `<form type="variant"><orth …>` |
| Hebrew definition | `<def xml:lang="he">` |
| English definition | `<def xml:lang="en">` |
| Binyan / government / complements | `<gramGrp><gram type="…">` |
| Classical flag | `<usg type="variety">Classical Arabic</usg>` |
| Reconstructed / erroneous flag | `<note type="editorial-status">` |
| Dictionary citation | `<cit type="dictionary" xml:id="blau-dc-…"><bibl …>` |
| Reference block | `<cit type="reference-block" xml:id="blau-r-…">` |
| Parsed citation | `<cit type="example|source" xml:id="blau-c-…">` |
| Cited JA text | `<quote xml:lang="jrb-Hebr">` |
| Translation | nested `<cit type="translation"><quote …>` |
| Additional note | `<note xml:id="blau-n-…">` |
| Bibliographic item | `<listBibl><bibl xml:id="blau-bib-…">` |

The project deliberately maps the JSON object named `form` to TEI `<sense type="form">`, because in Blau JSON it contains definitions, attestations, citations, references and editorial status. TEI `<form>` is reserved for lemma/variant orthographic forms.

## Cross-references

Cross-reference arrows in the source are resolved against a normalized lexical index built from entry headwords plus attested subentry/variant headwords. Matching is conservative:

1. Unicode and diacritics are normalized.
2. Exact unique lexical matches are accepted.
3. If several entries share the normalized form, a single non-cross-reference canonical entry may be preferred.
4. Internal “see Binyan X” arrows are linked to a persistent `blau-st-*` stem only when the destination stem is unique in context.
5. Top-level redirect entries are promoted to their unique canonical entry where possible.
6. No fuzzy lexical match is written automatically.

A resolved source segment receives `target_id` pointing to a stable entry or stem ID. TEI emits `<ref target="#blau-e-…">` or `<ref target="#blau-st-…">`. Unresolved or ambiguous cases remain without an invented target and are marked `ana="#unresolved-cross-reference"` in TEI. They are listed in `data/reports/unresolved_cross_references.json`.

## Citations and bibliography

Bibliographic records receive persistent `blau-bib-*` IDs. Parsed citations and dictionary citations receive `source_id` only when a controlled matcher can identify a bibliography record with high confidence. Matching uses, in descending priority:

1. exact and normalized bibliography abbreviations;
2. controlled spelling/order aliases for recurrent printed variants (for example reversed author order in dictionary names);
3. a unique bibliography abbreviation at the beginning of the printed citation, after a small controlled set of editorial cue words such as “השווה” or “עיין”;
4. a unique known authority embedded in parser prose;
5. high-confidence fuzzy matching only for sufficiently long `source_work` strings with a strict score and margin;
6. explicit `ibid` inheritance only when a preceding citation in the same parsed reference block is already resolved.

When one printed block explicitly names several known authorities, JSON uses `source_ids` rather than forcing a single `source_id`; TEI serializes all of them as whitespace-separated pointers in `corresp`. A single source is serialized as `<bibl corresp="#blau-bib-…">`. Unresolved values are retained as text and summarized in `data/reports/unresolved_citation_sources.json` and `data/reports/unresolved_dictionary_citations.json`; no uncertain authority link is invented.

## Languages

- `jrb-Hebr` — Judeo-Arabic in Hebrew script.
- `ar-Arab` — Arabic in Arabic script.
- `he` — Hebrew definitions/translations/editorial prose.
- `en` — English.
- `grc` — Greek when explicitly labelled in the source data.

## Formal validation

`schema/blau-tei.odd` is the project TEI customization. It explicitly constrains the TEI structures emitted by the exporter. `scripts/odd_to_rng.py` compiles the ODD content models to `schema/blau-tei.rng`. `scripts/validate_tei.py` then validates the complete generated dictionary against that Relax NG schema with `lxml.etree.RelaxNG` and checks `xml:id` uniqueness and structural counts.

Build and validate:

```bash
python scripts/build_v6_links.py
python scripts/validate_data.py
node scripts/buildTei.mjs
python scripts/odd_to_rng.py
python scripts/validate_tei.py
```

The website can generate the complete `.xml` file on demand in the browser from the canonical compressed JSON v6 data. For auditability, the full local research package also includes the validated TEI XML and gzip outputs together with the validation reports and schemas.


## Persistent entry URLs

Every top-level dictionary entry has a stable `blau-e-*` identifier. The web reader exposes canonical entry routes of the form `/entry/blau-e-000001`. These routes resolve through Netlify and through the bundled local HTTP launchers. Search results and in-dictionary links use the persistent ID route when an ID is available. The reader also exposes a “Copy permanent link” control.

## Release validation layers

Release 1.1 has two validation layers. The self-contained project ODD is compiled locally by `scripts/odd_to_rng.py`, and the entire generated dictionary is validated against the resulting Relax NG schema by `scripts/validate_tei.py`. This validation is part of `scripts/run_release_checks.py` and passes for the frozen release snapshot.

A second reproducible validation path is provided for the TEI Consortium toolchain. `scripts/official_tei_validate.sh` runs `teitorelaxng` from a supplied TEI Stylesheets checkout; `scripts/official_tei_validate_docker.sh` pins the Stylesheets checkout to `v7.61.0` inside the TEI development Docker environment. The same step is included in `.github/workflows/validate-release.yml`. Its machine-readable result is written to `data/reports/official_tei_validation.json`.

The full release package contains a frozen TEI snapshot, the ODD, generated project RNG, all validation reports, and the canonical compressed JSON v6 source data.