How OmniJudge keeps game rules separate from AI commentary
Following one turn through OmniJudge: dice input, balance updates, a structured ruling, and optional generated speech.
OmniJudge is a board-game referee project that combines a camera, a Python rules engine, hardware feedback, and spoken commentary. The camera observes pieces and dice; the rules engine tracks the game; the narrator explains the result.
The narrator is the part people hear, but it should not decide who owes money. A good way to understand the design is to follow a rent payment through the code.
Follow one rent payment
Suppose a player lands on a property owned by someone else. The game needs the property, its owner, the applicable rent, and the two balances. These are structured values in Python. They do not have to be recovered from a sentence such as “Pay Alex one million dollars.”
The main action handler processes the dice roll before requesting narration:
1
2
3
4
5
6
rulings = self.rules_engine.process_turn_roll(d1, d2)
for ruling in rulings:
narration = self.llm_referee.narrate_ruling(ruling)
narrations.append(narration)
self.mcu.play_buzzer(ruling.audio_cue)
This is a shortened extract from process_roll. One action can produce several rulings: passing GO and then landing on a property, for example. The narrator receives the results of those decisions.
A Ruling includes the event type, player, board space, amount, and an optional target player. It also carries a message, a narration field, and hardware cues. The amount remains a number even if the spoken wording changes.
A ruling is produced before the narrator is called.
If the narrator produces a sentence with the wrong amount, the announcement is still a problem, but it does not become a new balance update. Nothing in this path parses the generated sentence to calculate what a player owes. That is the part of the separation worth preserving in other applications, too: store the transaction first, then generate an explanation from it.
Why a template speaks first
narrate_ruling immediately fills in a template using the ruling’s fields. For a rent event, the template already knows the payer, recipient, property, and amount. It does not need a model to invent the sentence structure on every turn.
If the local model adapter is available, a background thread asks it for another version. The game can respond with the template while that work happens. Without the adapter, the template is still usable.
There is a detail to handle before treating this as a complete update mechanism. The background thread changes ruling.narration, but the immediate response already contains the initial narration strings. Changing an object later does not automatically change a response that has been sent to a browser.
A later-commentary feature would need an update channel and a way to identify the turn. For example, attach a ruling ID to the initial response and any later narration event. The client can then update the matching item, or discard it if the turn is no longer relevant. Otherwise a slow response can announce an earlier result during the next player’s action.
The rules still depend on the input being right
The camera can mistake a die face or place a token in the wrong board space. The rules engine can then make a perfectly consistent calculation from the wrong observation. Keeping the narrator out of the arithmetic does not solve that earlier problem.
The current board mapping uses object centers and fixed edge regions for a controlled overhead view. It does not apply a perspective homography to correct an arbitrary camera angle. When setting up the camera, that assumption matters more than a neat architecture diagram.
The training labels are also worth inspecting. The repository has 93 training images, but the number of objects in each class varies a lot:
Object instances in the training labels, not recognition scores. Several objects can appear in one photograph.
There are far more office and house annotations than examples of any one dice face. That does not tell us the detector’s accuracy. It tells us to evaluate dice faces separately rather than report one number for all objects. The dataset configuration uses the same directory for validation and test, so an independent test would need a separate set of images.
Test the parts that can fail independently
The rules engine can be tested with explicit dice values and a known game state. Set up ownership and balances, process the action, and check the resulting balances and ruling fields. You do not need the camera or narrator for that test.
Then test perception with labeled images, and narration with a fixed ruling. For narration, check both an unavailable model and a delayed response. For the camera path, include ambiguous inputs and a way for a person to correct them. Repeating the same physical action should also be checked so one roll cannot accidentally become two turns.
One other practical limit: the microphone code calls Google’s speech-recognition service. Some commentary can run locally, but voice transcription in this version still has an online dependency. That matters if you intend to demonstrate it without an internet connection.
The rules engine, ruling definition, and narrator show how these parts fit together. If you are adding generated explanations to an application, the useful starting point is the structured result you already know how to verify.
Project work: August 2026. Written in October 2026; revised November 2026.
