A loop of Tsvetok · daily since April 2026

Every morning, the photos I already took on my shift become a report I didn't have to write.

The model is the easy 20%. The hard 80% is the engineering around it — a signed PhotoKit binary, a confidence-aware re-read, a local-inference circuit breaker, layered fallbacks — that turns the geofenced rear-camera photos from my overnight security walk into a shift report I can trust without watching it. AI does the seeing and the reading; I just walked the property and photographed what was off, which I was already doing.

02

A real job, and a chore at the end of it

the gap

Premise A real-world job problem, solved on the surfaces and devices I already use, with AI doing only the part only AI can do. The missing piece was the connecting tissue, not a new app.

I work overnight security at a hotel. Part of the job is walking the property and documenting anything off — unauthorized vehicles in the lot, side-door or room damage, the perimeter gate state, arrivals. I already photograph these on my walk; the photos are already in Photos.app, geotagged, taken on the rear camera.

The chore was never the photographing. It was the write-up: at the end of a shift, those photos had to become a legible report — vehicles identified, plates read, repeat offenders flagged, incidents described, the gate logged. That is the part the photos couldn't do for themselves, and the part I didn't want to spend the tail of an overnight shift on.

This isn't computer-vision research. It's a working engineer's automation of a nightly chore, built well enough to trust unattended. That last clause is the whole pitch — and it's where the engineering lives.

03

What it produces

receipts first

At 06:30 every morning a LaunchAgent (com.tsvetok.journal-writer.plist, Hour=6 Minute=30) fires agents/journal-writer.py. By the time I wake up, the report is written and on my phone. The outputs, in order of how much I rely on them:

From the pipeline's own logs, counted — not asserted
Pipeline runs
152
Mornings journaled
97
Photos read by the vision pass
197
Runs routed to local inference
67

On 33 mornings with nothing to report, it filed nothing — instead of inventing a journal.

The report itself renders in a fixed order: Gate → Parking (vehicles) → Incidents & Notes → Documents → Summary → Photos. Same shape every day, so I read it the same way every day.

04

Let the model do only what only it can

the pipeline

The daily run is a ten-step pipeline. Only two steps — vision and OCR — ask a model to interpret an image. The other eight are the reason I trust it without watching it: photo selection, a confidence-aware re-read, a local-inference circuit breaker, dedup, a critic gate, and layered fallbacks. The four below are where the judgment lives.

Step A

Find the right photos — a signed binary, not a guess

photos-query.swift · PhotoKit · launchd-only TCC

Which photos even belong in the report is itself an engineering decision, and I didn't hand it to the model. A code-signed PhotoKit binary, photos-query.swift, does the selection: it queries an overnight time window from config, then applies four independent post-filters — within the geofence (centered on the work site), from the rear camera (detected by reading the EXIF LensModel and excluding anything marked "front"), not a screenshot, and carrying real GPS and EXIF. It logs its own filter stats so a bad night is debuggable.

The systems detail that makes this real: Photos and iCloud access only works from a code-signed binary spawned by launchd — the TCC permission is identity-bound and survives recompile only when the same identity re-signs. The binary exits with a distinct code on a permission denial, and the Python orchestrator retries with backoff. Getting an unattended 06:30 job to reliably touch the photo library is a systems problem, not a scripting one.

Step B

Read each photo — with a confidence-aware second pass

analyze_photo · enhance_low_confidence_plates

Each photo is resized and converted to JPEG (≤1024 px, quality 70) — HEIC isn't a valid vision input, and the resize caps token cost. The vision call returns structured JSON: a category (gate, vehicle, damage, document, other), a description, and for vehicles the make, model, color, plate, plate state, a plate-reading confidence, and a fractional bounding box for the plate. The category rules are hard-won — a "gate" means only the perimeter gate, never an interior door; that distinction only got written after the model got it wrong in the field.

The signature move is the two-pass plate enhancement, and it's the most transferable idea in the system. A re-read fires on either of two triggers. The first is obvious: low confidence (< 0.80). The second is the interesting one: a plate that contains a known-confusable character (O 0 Q D I 1 B 8 S 5 Z 2 G 6 9) gets re-read even at high confidence (< 0.95). The insight, in the code's own words: a model can be confidently wrong on a character whose glyph is ambiguous. The field failure that motivated it — a plate read as OJG659 that should have been QJG659, the Q's tail lost to streetlamp glare while the model's prior leaned toward O.

On a re-read the pipeline crops the plate region (model bounding box, falling back to a heuristic bottom-center crop), grayscales it, boosts contrast 1.8×, sharpens 2×, and upscales small crops, then re-reads with a prompt that threads the first-pass state hint while explicitly telling the model not to anchor on the first read. Acceptance is guarded: a lower-confidence re-read is accepted only if it changed confusable-character positions and nothing else — a wild swing or a length change is rejected. The general principle, which outlives plates: when a model does OCR in a domain with known-confusable tokens, the presence of a confusable token is itself a signal the confidence may be mis-calibrated. Force a second look with domain context baked in.

Step C

Route plate-reading to local hardware — a circuit breaker

jetson_client.probe() · /ready · /v1/plates

At the start of every run, jetson_client.probe() calls the Jetson's /ready endpoint with a 5-second timeout. If it answers, vehicle photos go to /v1/plates for the local detector's bounding box and OCR; if it times out, the run keeps the Claude read, logs the fallback, and continues. The breaker is that timeout plus the route decision — and it fires every morning in production: the logs show plate reads routing to local inference daily, with Claude as the floor underneath.

The merge logic is deliberately cautious. The local detector's bounding box is always preferred — it's more accurate than the model's estimated box — but its OCR text is trusted only if it passes three gates: at least five characters, at least 0.70 confidence, and not meaningfully shorter than the existing read. Fail any gate and it's "bounding-box only": keep the model's text, but feed the better crop into the enhancement pass from Step B. That guard came from a real bug — a low-confidence two-character local read overwriting a valid seven-character one — and every rejection is logged so a regression is visible. Local when available, cloud as the floor, with the seams instrumented.

Step D

Catch the model's structural mistakes — a critic and a deduper

deduplicate_vehicles · critic_review · check_repeat_offenders

The model reads photos one at a time; the report is about the night as a whole, and that gap is where structural errors live. Four deterministic passes close it. Vehicle dedup collapses the same car shot twice from different angles into one, using multi-signal scoring — fuzzy plate distance, make/model/color agreement, time proximity — against a grouping threshold. A critic_review pass checks the assembled manifest for four defect classes — malformed plates, vehicle overcounts, category errors, implausible plate states — and its return is constrained to corrections the orchestrator applies programmatically, so a review note can't quietly become report prose. Cross-day repeat-offender matching runs the same fuzzy multi-signal comparison against a 90-day history, so a returning car is caught even when OCR got its plate slightly wrong on a different night. And a silent-loss detector reconciles the exported photo count against the count that made it into the report's sections, and warns on a mismatch — born from a real incident where surplus photos in a single slot dropped silently.

05

The pipeline, end to end

one diagram

One linear flow with one branch. Read it top to bottom; the model sits in the middle, and almost everything else is selection, verification, and fallback. Every fallback is logged — the system is built to fail loudly, never silently.

flow journal-writer · daily 06:30 pipeline

Two details the diagram leaves to prose: photo selection is deliberate engineering, not a model decision; and the verify stage reconciles silently-lost photos so nothing drops without a logged trace.

06

What the AI does, and what it doesn't

named precisely

The line between "I called an API" and "I engineered a system" is exactly this table. The model is good at a narrow, genuinely hard thing. Everything that decides whether to trust it sits outside the model, in code I can read and reason about.

The AI doesThe AI does NOT
Scene understanding — what is this photo of? Decide which photos to look at (the signed geofence binary does)
Structured extraction — make, model, color, plate Decide what's a duplicate (deterministic multi-signal scoring does)
OCR under hard conditions — night, glare, motion blur Decide the report layout (the renderer does)
A second-opinion critic pass over the assembled manifest Get trusted blindly — confidence thresholds, confusable-char re-reads, and the critic gate all sit between the model and the report

Cost discipline runs through all of it. Every model call routes through the Max subscription credit pool — there's no standalone API key (it was retired 2026-04-15). The pipeline caps the photo count, logs estimated cost per run, prefers free local inference on the Jetson when it's up, and the resize-to-1024 / quality-70 step is itself a token-cost lever. Reliability and cost were design inputs, not afterthoughts.

07

What was hard

four problems

Confident-but-wrong OCR

The hardest failures weren't low-confidence reads — those announce themselves. They were high-confidence wrong reads on ambiguous glyphs. The cure is the confusable-character-triggered re-read with a cleaner crop and a state prior, guarded by a rule that accepts only confusable-only diffs. It's the most transferable idea in the system, and it generalizes to any OCR domain with a known-confusable alphabet.

TCC and signed binaries

Photos and iCloud access only works from a code-signed binary spawned by launchd; ad-hoc signing breaks the identity-bound TCC permission. Getting an unattended 06:30 job to reliably touch the photo library — through permission denials, retries, and recompiles — is a real systems problem, not a scripting one.

Hybrid inference without regressions

Letting the local Jetson help without letting it hurt required the three-gate text-acceptance logic and per-rejection logging. The naive version — trust whatever's local and fast — actively degraded reads. This path runs in production daily, and the guard is the reason it's an improvement rather than a liability.

Never fail silently

An unattended daily job that quietly drops data is worse than one that loudly fails. The markdown-backup safety net, the photo-accounting reconciliation, and the layered fallbacks all exist because of that conviction. When something goes wrong, I want a report that says so — not a clean-looking report that's missing a car.

08

Status, and where this sits

journal-writer is one loop of Tsvetok, my personal AI assistant — the full system, its architecture, and its operating discipline are on the umbrella case study.

As of 2026-07-01

Running daily since April 2026. The model is the easy 20%; the engineering is the 80% that makes an unattended output trustworthy — which is the only kind of output worth waking up to.

Building something that needs Apple-platform systems work or agentic-AI pipeline engineering? david@davidjknudson.com.