Authoring Osmose Rule DSL Checks

Surface a data-quality issue on the Osmose QA map — for example every amenity=fuel node with no name — by writing a backend analyser that selects the offending features with SQL and emits a typed error class the frontend can render as map markers.

Prerequisites

Have each of these in place; an analyser that runs cleanly but produces zero markers is nearly always a schema-name or class-registration slip below.

Conceptual minimum

Osmose is a server-side quality-assurance system: it ingests an OSM extract into PostgreSQL, runs a battery of analysers over the database, and publishes each analyser’s findings as issue markers on a web map. An analyser is a Python class, and the two families that matter are the Analyser_Osmosis base, which runs raw SQL against the osmosis snapshot schema, and the Analyser_Merge family, which cross-references OSM against an external open dataset. This page uses the Analyser_Osmosis pattern because most house rules are expressible as a query: the offending features are exactly the rows a SELECT returns, and the analyser’s job is to turn each returned row into a typed error. Unlike the editor-time checks in a JOSM validation preset, an Osmose check runs over the whole database on a schedule, so it catches issues in data nobody is actively editing.

Every issue Osmose raises is identified by a small set of fields that the frontend keys on. class_id is a number unique within the analyser; it groups all instances of one problem so mappers can filter by it. item is a broad category number (for example the ranges used for tagging, geometry, or routing problems) that colours and groups issues across analysers. level is severity from 1 (most serious) to 3 (minor). The analyser declares each class once in its constructor with a translated title, then, for every offending row the SQL returns, calls self.error with that class_id and the feature’s type, id, and coordinates. The backend writes those errors to its issue tables; the frontend reads them and drops a marker at each feature’s location. The pipeline below shows that path end to end.

Osmose analyser pipeline from database to QA map A PostgreSQL osmosis schema feeds a SQL rule in an Analyser_Osmosis class, whose returned rows become a typed issue class via self.error, which the backend stores and the frontend renders as markers on the QA map. PostgreSQL osmosis schema from PBF SQL rule Analyser_Osmosis SELECT offending rows rows issue class self.error(...) class_id · item · level store Osmose QA map one marker per row Each returned row → one self.error() call → one map marker at the feature's coordinates.

Runnable solution

Save this as analyser_merge_fuel_name.py under the Osmose backend analysers/ directory. It declares one issue class and selects every fuel node lacking a name, emitting an error per row.

python 
import logging

from modules.OsmoseTranslation import T_
from Analyser_Osmosis import Analyser_Osmosis

logger = logging.getLogger(__name__)

# SQL that selects offending features from the osmosis snapshot schema.
# nodes.tags is an hstore column; the ? operator tests key presence.
SQL_FUEL_WITHOUT_NAME = """
SELECT
    nodes.id,
    ST_X(nodes.geom) AS lon,
    ST_Y(nodes.geom) AS lat
FROM
    nodes
WHERE
    nodes.tags ? 'amenity'
    AND nodes.tags -> 'amenity' = 'fuel'
    AND NOT (nodes.tags ? 'name')
"""


class Analyser_Merge_Fuel_Name(Analyser_Osmosis):
    """Flag fuel stations (amenity=fuel) that are missing a name tag."""

    def __init__(self, config, logger=None):
        super().__init__(config, logger)
        # class_id: unique within this analyser. item: broad tagging category.
        self.classs[1] = self.def_class(
            item=3220,
            level=3,
            tags=["fuel", "fix:survey"],
            title=T_("Fuel station without a name"),
            detail=T_("This amenity=fuel node carries no name tag."),
            fix=T_("Survey the station and add its name."),
        )

    def analyser_osmosis_common(self):
        self.run(SQL_FUEL_WITHOUT_NAME, self._emit)

    def _emit(self, res):
        node_id, lon, lat = res[0], res[1], res[2]
        # self.error groups by class_id and positions the marker at (lon, lat).
        self.error(
            self.classs[1],
            {"self": lambda r: {"en": T_("Fuel station without a name")}},
            subclass=1,
            osm_id={"N": node_id},
            geom={"position": [{"lat": lat, "lon": lon}]},
        )

Run it against the loaded database from the backend root:

bash 
python osmose_run.py --analyser=merge_fuel_name --country=my-extract

Step-by-step walkthrough

  1. Imports and base class. Analyser_Osmosis provides self.run (execute SQL and iterate rows) and self.error (emit an issue). T_ marks strings for translation, which the frontend uses to localise titles.
  2. The SQL contract. The query must return the columns the emit callback expects — here id, lon, lat. Selecting the geometry as ST_X/ST_Y gives the marker coordinates directly; without them the frontend has nowhere to place the issue.
  3. hstore presence tests. In the osmosis schema, tags live in an hstore column: tags ? 'name' tests key presence, and tags -> 'amenity' reads a value. NOT (tags ? 'name') is the SQL equivalent of the editor rule’s [!name] absence test.
  4. Declaring the class. self.classs[1] = self.def_class(...) registers one issue class. item=3220 places it in the tagging-issue band, level=3 marks it minor, and title is the label mappers read on the map. Declaring the class in the constructor means it exists even when a run finds zero issues.
  5. analyser_osmosis_common. This is the entry point Osmose calls for a database analyser. self.run(sql, callback) streams each result row into the callback rather than materialising the whole result set, which keeps memory bounded on continental extracts.
  6. Emitting an error. self.error writes one issue: subclass disambiguates variants within a class, osm_id={"N": node_id} links the marker to the node so JOSM can open it, and geom={"position": [...]} fixes the marker’s map location.
  7. The item numbering. item values are shared conventions across all Osmose analysers so the frontend can group and colour by category; reusing an established band keeps your check consistent with the rest of the catalogue rather than inventing an orphan category.
  8. Running and re-running. The command loads no data itself; it assumes the osmosis schema is already populated, so a re-run after fixing tags in the database re-evaluates the same SQL and clears resolved markers.

Verification

Confirm the check produces the markers you expect:

  • Class exists. After a run, the analyser’s class table must contain a row for class_id 1 with your title, even on a clean extract — proof the constructor registered it.
  • Row count matches errors. Run SQL_FUEL_WITHOUT_NAME by hand in psql; the number of returned rows must equal the number of issues Osmose emitted for that class.
  • Marker placement. Open the generated issue output and confirm each marker’s lat/lon sits on the corresponding fuel node, not at (0, 0) — a (0, 0) marker means the geometry columns were missing from the SELECT.
  • Severity renders. The issue should appear at level 3 (minor) styling on the map; a wrong level argument shows up as the wrong colour band.
  • Idempotent re-run. Add name to one node in the database, re-run, and confirm that node’s marker disappears while the others remain.

Common errors and fixes

Symptom Root cause One-line fix
Analyser runs, zero markers Query returns nothing due to wrong schema/table Confirm the osmosis schema is loaded and nodes is populated.
KeyError on self.classs self.error used a class_id never declared Register every class in __init__ via def_class.
All markers at (0, 0) Geometry not selected or geom position empty Select ST_X/ST_Y and pass them in geom["position"].
operator does not exist: hstore ? unknown hstore extension not enabled CREATE EXTENSION hstore; in the target database.
Title shows as raw msgid String not wrapped for translation Wrap user-facing text in T_(...).
Duplicate markers per feature Callback emits inside a loop over joined rows Ensure the SQL returns one row per offending feature.
Wrong colour grouping on map item category number outside its band Reuse an established item band for the issue type.

For issues that also need catching while a mapper edits, pair this server-side class with the editor guardrail in Writing Custom JOSM Validation Presets so both the live and the batch path enforce the same rule.

Specification reference

An Osmose backend analyser subclasses Analyser_Osmosis and runs SQL against the osmosis snapshot schema; each issue class is registered with def_class carrying an item category, a level from 1 to 3, and a translated title, and each offending feature is reported with self.error naming the class, the osm_id, and a geom position. See the Osmose overview on the OSM Wiki and the Osmose backend source and analyser docs for the analyser API, the item numbering conventions, and the translation helpers.

Frequently Asked Questions

When should I use Analyser_Osmosis versus Analyser_Merge?

Use Analyser_Osmosis when the offending features are fully describable by a SQL query over the OSM snapshot alone — missing tags, bad geometry relationships, inconsistent attributes. Reach for the Analyser_Merge family when the check compares OSM against an external open dataset, such as matching official addresses or public facility registries, because that family handles fetching, conflating, and diffing the reference source for you.

What do class_id, item, and level actually control?

class_id is unique within one analyser and groups every instance of a single problem so mappers can filter by it. item is a broad category number shared across all analysers that the frontend uses to colour and group issue types. level is severity from 1, the most serious, to 3, minor. Together they decide how an issue is labelled, coloured, grouped, and prioritised on the QA map.

Why does my analyser produce no markers even though the SQL looks right?

The usual cause is that the database was never loaded into the osmosis schema the analyser queries, so the tables are empty and the query returns nothing. Run the SELECT directly in psql first; if it returns zero rows there, the problem is the data load or the schema name, not the analyser. If psql returns rows but Osmose shows none, check that the emit callback passes valid geometry positions.

How do markers get their location on the map?

Each self.error call includes a geom argument with a position list of latitude and longitude pairs. Those coordinates come straight from the SQL, typically as ST_X(geom) and ST_Y(geom) for a node. The frontend drops one marker per emitted error at that position. If you omit the geometry columns from the SELECT, the markers collapse onto null-island at zero-zero.

Up one level: Authoring OSM Validation Rules.