Data Standardization SOP

This page is the source of truth for how the Public Data API spells names and notations. Use it when you $filter institutions or companies, and when you map a new national dataset into the catalog.

The rules below match what the public API already documents and what new imports are required to emit. They do not change historical rows by themselves.

Source of Truth

Resolve values in this order:

1. Public API contract — the English strings integrators already filter on, as shown in Getting Started and the list-endpoint samples.
2. ISO 3166-1 alpha-2 — the code key for a country (flags, coverage tables, national identifier prefixes).
3. English CLDR short names — new Intl.DisplayNames(['en'], { type: 'region' }).of(code) is how we turn that code into the stored addresses.country value. The coverage table is generated from the same function.
4. Closed enums in @epiphany/types for institution type and address type. Do not invent new values.

Dashboard geography pickers (apps/dashboard/src/constants.js) are a separate UI catalog. Do not copy those labels into Public Data addresses when they disagree with CLDR (for example dashboard China vs coverage China mainland).

Country Names

addresses.country stores the English CLDR short name, not an ISO code.

Do not store USA, United States of America, UK, Great Britain, Holland, or Czech Republic as addresses.country.

How new imports must map countries

1. Trim the source value.
2. If it is already a two-letter code, treat it as ISO 3166-1 alpha-2 (uppercase for Intl.DisplayNames, which is unreliable with lowercase codes).
3. Apply the aliases we already use in coverage generation: UK → GB, FX → FR.
4. Otherwise look the name up in a source-specific name→ISO map (the EQAR importer in utils/entities/countries/eqar/eqar.ts is the existing example), then expand with Intl.DisplayNames.
5. Write the English display name into the CSV country column and into entities.institution_addresses.country / entities.company_addresses.country. Do not leave the alpha-2 code in that field.

Common expansions:

The coverage table is the published spelling list for every flag we ship. If you add a country, regenerate that page with utils/entities/_gen-coverage-tables.mjs so the public name stays in lockstep with CLDR.

Filtering by country

const filter = {
    'addresses.country': [{ operator: 'equals', value: 'United States' }]
};

Use the same English name you see on the coverage table.

States, Regions, and Provinces

addresses.state stores the full region name, not a postal abbreviation.

Rules:

• Prefer the official English name, or the source’s Latin-script name when that is the national standard (for example French Île-de-France, Italian Lazio, Swedish Stockholm County).
• If the source only has a code, expand it with a national mapping table in the importer. Australia already does this (ACT → Australian Capital Territory, and so on). Do not persist the abbreviation.
• Use the administrative unit the source treats as the region: US state, Canadian province, French région, UK government office region or county, Chilean región, Singapore zone, and similar. Do not invent a second geography field.
• Leave state empty when the source has no region, as EQAR currently does.

const filter = {
    'addresses.state': [{ operator: 'equals', value: 'California' }],
    'addresses.country': [{ operator: 'equals', value: 'United States' }]
};

Cities, Postal Codes, and Coordinates

• City: Latin characters when a reasonable transliteration exists (Tokyo, not 東京都). Use proper case, not AUSTIN.
• Postal code: Trim whitespace; keep the national format (do not strip spaces from UK postcodes or hyphens from US ZIP+4).
• Coordinates: WGS 84 decimal degrees in latitude / longitude.

Institution Types

type is the closed InstitutionType enum. Unknown source types become other. Do not add new enum members from an importer.

Companies do not have a type field.

Address Types

addresses.type is the closed AddressType enum: physical, mailing, post_office_box, billing, shipping, headquarters, branch, other.

New institution imports default to physical. Company seeds default to headquarters. Only use another value when the source clearly distinguishes it.

Names, Descriptors, and Websites

These rules apply to institution and company name, descriptor, and website.

• Websites start with https://. A school subpage is acceptable when that is the school’s public URL.
• Case: Correct ALL CAPS source names to normal title case.
• Abbreviations in names: Prefer letters without periods or spaces (ISD, not I.S.D.). Do not leave common district abbreviations in the name — expand ISD, CSD, CISD, USD, and similar. Keep periods when they are part of a personal name (H.C. Crittenden Middle School).
• Irish / Scottish (and similar) expansions: Scoil Náisiúnta instead of S N / Scoil Nais; Boys National School / Girls National School instead of B N S / G N S; Infant Boys / Infant Girls instead of I B / I G.
• Script: Prefer Latin characters for cities, states, and countries. Institution names may stay in the source script when there is no reasonable transliteration.
• Generic names: If the name is only a number or a generic label (Ecole élémentaire), add a distinguishing place or official qualifier.
• Descriptor: At most three words. Keep it general (High School, Private School, National governing body). It appears in search results to disambiguate records. Do not write long source dumps into descriptor.

Relationships

Parent/child rows use relationship_type governing_body. That is the only relationship type current Public Data importers write. Leave it empty when there is no parent.

Seed parents before children. When several institutions share a parent name, the CSV seeder prefers a government, district, or lea parent over a school.

Identifiers

Identifier type names are stored on entities.institution_identifier_types.name. They are not currently returned on the public institution or company objects; they are how we dedupe imports (identifier_type|identifier_value).

Convention for new national IDs: {ISO2}_{SOURCE_KEY} in SCREAMING_SNAKE (for example FR_UAI, IE_ROLL_NUMBER). Existing exceptions to keep:

Reuse an existing type name when the same national register is imported again. Do not create phone identifier types.

Import Checklist

Before a new source lands in utils/entities:

1. Map country → ISO 3166-1 alpha-2 → English CLDR name. Write the name, not the code.
2. Expand state/province abbreviations to full names.
3. Map each row to one InstitutionType value from the table above.
4. Normalize website, name, and descriptor as specified.
5. Assign a stable identifier_type / identifier_value pair and a source_dataset slug (schools, governing_body, eqar, …).
6. Emit governing_body relationships with parent rows first.
7. Update coverage when the country or sector is new.

Related Docs