Normalizing OSM Yes/No Tag Values

The tag looks boolean and is not. access can be yes, no, private, permissive, destination, customers or designated, and a pipeline that maps everything not equal to yes onto false has just told a routing engine that a permissive path is closed.

Prerequisites

Conceptual minimum

Three distinctions have to survive normalization, and collapsing any of them loses real information.

Absent is not false. A way with no lit tag has not been surveyed for lighting; it has not been recorded as unlit. A pipeline that treats the absence as a negative is inventing data, and it will report that a city’s streets are overwhelmingly unlit when in fact they are overwhelmingly unsurveyed.

Some values are boolean in disguise. true, 1 and occasionally T appear as synonyms for yes, usually from imports or from editors that did not enforce the vocabulary. They are safe to fold.

Some values are not boolean at all. private, permissive, destination, customers, designated, limited and unknown each say something a boolean cannot carry. Folding them into true or false is where the real damage happens, because the result is plausible and the nuance is gone.

The right output is a tri-state — yes, no, unknown — plus a preserved original value, so a consumer that understands the richer vocabulary can still reach it.

What each family of boolean-ish value actually means A grid of five value families against how each should normalise and why. The affirmative family, covering yes, true and one, normalises to yes and is safe to fold. The negative family, covering no, false and zero, normalises to no and is equally safe. The conditional family, covering permissive, destination and customers, means access is allowed under conditions and must not become a plain yes. The restrictive family, covering private and no entry variants, means access is denied to the general public and is closer to no but not identical. The absent case means unsurveyed and must stay distinct from no. Five families, and only two fold safely Normalises to Why yes, true, 1 yes synonyms, safe to fold no, false, 0 no synonyms, safe to fold permissive, destination yes, flagged allowed with conditions private, customers no, flagged denied to the public absent unknown unsurveyed, not negative The flagged rows keep the original value alongside the tri-state, so a consumer that understands the nuance can still use it.
Collapsing the middle two rows into plain yes and no is what tells a routing engine a permissive path is a public road.

Runnable solution

python 
from __future__ import annotations

import logging
from enum import Enum
from dataclasses import dataclass

logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
logger = logging.getLogger("osm.tags.boolean")


class Tri(str, Enum):
    YES = "yes"
    NO = "no"
    UNKNOWN = "unknown"


AFFIRMATIVE = {"yes", "true", "1", "t", "y"}
NEGATIVE = {"no", "false", "0", "f", "n"}
# Values that resolve to a state but carry a condition worth keeping.
CONDITIONAL_YES = {"permissive", "destination", "designated", "official",
                   "customers", "permit", "agricultural", "forestry", "delivery"}
CONDITIONAL_NO = {"private", "no_entry", "restricted", "military"}
# Values that resolve to nothing: they say the surveyor did not know.
EXPLICIT_UNKNOWN = {"unknown", "unspecified", "fixme", ""}


@dataclass(frozen=True)
class Normalized:
    state: Tri
    original: str | None
    conditional: bool
    recognised: bool

    @property
    def is_definite(self) -> bool:
        return self.state is not Tri.UNKNOWN and not self.conditional


def normalize(value: str | None) -> Normalized:
    """Map an OSM boolean-ish value to a tri-state, keeping what was lost."""
    if value is None:
        # Absent is NOT false: nobody recorded anything either way.
        return Normalized(Tri.UNKNOWN, None, conditional=False, recognised=True)

    raw = value.strip()
    folded = raw.casefold()

    if folded in EXPLICIT_UNKNOWN:
        return Normalized(Tri.UNKNOWN, raw, False, True)
    if folded in AFFIRMATIVE:
        return Normalized(Tri.YES, raw, False, True)
    if folded in NEGATIVE:
        return Normalized(Tri.NO, raw, False, True)
    if folded in CONDITIONAL_YES:
        return Normalized(Tri.YES, raw, conditional=True, recognised=True)
    if folded in CONDITIONAL_NO:
        return Normalized(Tri.NO, raw, conditional=True, recognised=True)

    # An unrecognised value is not a negative. Say so, and keep it.
    logger.debug("unrecognised boolean-ish value %r", raw)
    return Normalized(Tri.UNKNOWN, raw, conditional=False, recognised=False)


def normalize_key(tags: dict[str, str], key: str) -> Normalized:
    return normalize(tags.get(key))


def audit(values: list[str | None]) -> dict[str, int]:
    """Count how a corpus distributes, so the vocabularies can be extended."""
    counts = {"yes": 0, "no": 0, "unknown": 0,
              "conditional": 0, "unrecognised": 0}
    unrecognised: dict[str, int] = {}
    for value in values:
        result = normalize(value)
        counts[result.state.value] += 1
        if result.conditional:
            counts["conditional"] += 1
        if not result.recognised:
            counts["unrecognised"] += 1
            key = result.original or ""
            unrecognised[key] = unrecognised.get(key, 0) + 1

    top = sorted(unrecognised.items(), key=lambda kv: -kv[1])[:10]
    logger.info("distribution %s", counts)
    if top:
        logger.info("most common unrecognised: %s", top)
    return counts


if __name__ == "__main__":
    sample = ["yes", "no", "private", "permissive", None, "1", "maybe", ""]
    audit(sample)

Step-by-step walkthrough

  1. Separate absent from empty. A missing key and a key whose value is an empty string both mean unknown, but only the first is a legitimate state — the second is a data-quality signal worth keeping visible.
  2. Fold case, not meaning. Case folding handles the editor variants safely; folding a conditional value into a plain state does not.
  3. Keep conditional and definite apart. A path that is permissive is usable and a path that is yes is usable, and a routing engine that plans a route through private land on the strength of a permissive tag has made a real mistake.
  4. Treat an unrecognised value as unknown. It is neither affirmative nor negative; defaulting it either way is guessing, and defaulting it to false is the guess that silently closes things.
  5. Preserve the original. The tri-state is for consumers that want simplicity; the original is for the ones that want the nuance, and keeping both costs one column.
  6. Audit the corpus. Counting the most common unrecognised values is how the vocabularies get extended from evidence rather than from memory of what the documentation says.
  7. Record whether the value was recognised. That flag is what lets a quality report distinguish “the surveyor did not know” from “we did not understand the answer”.
Three collapses that each lose something specific Three panels. Collapsing absent into no reports unsurveyed features as negative, which turns a coverage gap into a factual claim and makes a city's streets look unlit when they are merely unsurveyed. Collapsing conditional into definite tells a consumer that a permissive path is a public right of way, which a routing engine will act on. Collapsing unrecognised into no closes anything whose value the pipeline did not understand, which is the worst default because the failure scales with how unusual the data is. Three collapses, three different damages Absent to no Unsurveyed reads as negative Coverage gap becomes a claim Streets look unlit Statistics quietly invented Conditional to definite Permissive reads as public Private reads as merely closed Routing acts on it Nuance gone, plausible result Unrecognised to no Anything unknown is closed Worst possible default Scales with unusual data Fails hardest where it matters All three produce numbers that look entirely reasonable, which is why none of them is caught by a sanity check on the output.
Each collapse saves one column and costs a distinction that cannot be recovered afterwards.
How one key's values actually distribute across a country extract Five outcome categories with their approximate share of features for a typical access-style key across a country extract. Features with no tag at all dominate by a wide margin, because most things are simply unsurveyed for that property. Plain affirmative values are the next largest group. Plain negative values are a small share. Conditional values such as permissive and private form a smaller but significant share. Unrecognised values are a fraction of a percent and are where the vocabulary needs extending. Where the values actually are, for one key No tag at all about 88% Plain yes about 7% Plain no about 2% Conditional about 2.5% Unrecognised under 1% Folding the first bar into the third would report ninety percent of features as negative, which is the shape of a statistic nobody questions.
The dominance of the first bar is why the absent-versus-negative distinction matters more than every other decision here.

Verification

  • Absent and negative differ in the output. Count features whose tri-state is unknown versus no; a corpus where unknown is zero means absence is being folded.
  • Conditional values are flagged. Find a permissive feature and confirm it carries both the affirmative state and the conditional flag.
  • Unrecognised values surface. The audit’s unrecognised list should be short and, where non-empty, should contain real values worth adding.
  • Case variants fold. Yes, YES and yes must produce identical output.
  • Originals survive. Every normalised record should still carry the value it came from.

Common errors and fixes

Symptom Root cause One-line fix
Most features reported negative Absent folded into no Return unknown for a missing key
Routing uses private land Conditional folded into definite Keep a conditional flag alongside the state
Unusual regions behave worst Unrecognised defaulted to no Return unknown for anything not in the vocabulary
Vocabulary never grows Unrecognised values discarded Audit and report the most common ones
Case variants treated as distinct No case folding Casefold before matching
Nuance irrecoverable downstream Original value dropped Keep the raw value alongside the tri-state
Empty string treated as a state Empty and absent conflated Map empty to unknown but flag it separately

Specification reference

Keys such as access, oneway, lit and bridge accept yes and no alongside a wider vocabulary, and several keys define conditional values such as permissive, destination and private with distinct meanings. An absent key indicates that the property has not been recorded rather than that it is false. See the access tag documentation for the conditional vocabulary and Tag Taxonomy & Key-Value Standards for the wider key conventions.

Frequently Asked Questions

Why should an absent tag be unknown rather than no?

Because absence records that nobody surveyed the property, not that the property is false. Treating the two the same converts a coverage gap into a factual claim, and the resulting statistics are confidently wrong: a city whose streets are largely unsurveyed for lighting will be reported as largely unlit. The distinction also tells you where survey effort would be worth spending, which the collapsed version cannot.

Is permissive the same as yes?

For the question “can I pass”, nearly; for anything a consumer will act on, no. Permissive means the owner currently tolerates access and may withdraw it, which is a materially different statement from a public right of way. A routing engine planning a walking route can reasonably use permissive paths; one planning a delivery vehicle’s route probably should not, and it can only make that distinction if the value survived normalization.

What should happen to a value nobody recognises?

It becomes unknown, and it gets counted. Defaulting it to false is the worst available choice because the failure scales with how unusual the data is — regions with local tagging conventions, or features with genuinely unusual access arrangements, are exactly where the default does most damage. Counting the unrecognised values is how the vocabulary grows from evidence.

Is a tri-state enough, or should the full vocabulary survive?

Both, which is why the original is kept alongside. The tri-state serves the many consumers that want a simple answer and would otherwise invent one; the preserved original serves the few that need the nuance. Storing only the tri-state discards information irreversibly, and storing only the original pushes the same normalization decision onto every consumer separately.

Up one level: Tag Taxonomy & Key-Value Standards.