At-most-once delivery, immutable approvals, and safety tests that can't be broken by the calendar.
An unattended social publisher for AstroAI that treats a public post as irreversible. Every post needs a sealed, append-only human approval of its exact bytes, and any send whose outcome can't be proven is held for a person instead of retried.
- tests across the three publish-safety suites
- 128
- tests across the three publish-safety suites
- automatic retries once a send's outcome is ambiguous
- 0
- automatic retries once a send's outcome is ambiguous
- tests a calendar date once broke; now clock-independent
- 23
- tests a calendar date once broke; now clock-independent
- clock shifts both safety suites were run under
- -10 to +100 yrs
- clock shifts both safety suites were run under
The problem
AstroAI's social pipeline generates captions and images and schedules them to public accounts through Buffer from an unattended drain job. The original publisher retried any transport exception up to four times. A connection reset or read timeout that arrives after the request was sent doesn't mean the post was rejected; it may already be live. Retrying those errors published the same post twice.
Approval had the same weakness. Caption and image review verdicts were the only gate, so an edit made after review could still reach a public account.
Constraints
- A public post is the least reversible action in the system. When a send's outcome can't be proven, the right number of extra sends is zero, not one.
- The publisher runs without a person watching, so a held or unproven post has to fail loudly instead of quietly.
- Buffer's failure signals are often ambiguous: resets, read timeouts, 5xx responses, and bodies that can't be parsed.
- Safety tests must never open a socket or read a credential.
- There is one operator, so the recovery step has to be simple and safe to run more than once.
What I built
- An outcome classifier with four results: not_sent, delivered, refused, and ambiguous. Success counts only when the provider explicitly returns success. Anything the classifier doesn't recognize is ambiguous, and an ambiguous send is never retried automatically.
- Automatic retries only for failures that provably happened before the request left the process (DNS failure, connection refused, connect timeout, proven TLS failure), capped at 4 attempts.
- A write-ahead attempt ledger that is flock-guarded and written atomically. Each attempt is recorded before the request goes out, so a process killed mid-request leaves an in_flight record that blocks any second attempt. The queue row is also stamped ambiguous, which gives two independent guards that both survive a restart.
- An operator reconcile command (list, then reconcile as delivered or not-delivered) with no default and no auto-resolve. Re-running it is idempotent, and it refuses to turn a recorded delivery back into a pending row.
- Immutable final approvals in an append-only SQLite ledger. Each approval seals the caption text and SHA-256, the exact image bytes (or an ordered multi-image package), the destination channel, and the publish slot with its timezone. Triggers block UPDATE and DELETE. Any later edit, remap, reorder, or slot change invalidates the approval for good, and nothing can revive it.
- Revalidation of the approval against current source state immediately before transport, plus staging of content-addressed snapshot copies to R2, so Buffer never receives a mutable, filename-based asset.
How it's verified
- 128 tests across three safety suites: test_publisher_at_most_once.py (38), test_final_publish_approval.py (42), and test_social_package_approval.py (48). They run against in-memory fake transports and temp files.
- The publisher suite proves each rule: retry before the request, no retry after a reset or timeout, delivered, refused, ledger persistence, behavior after a restart, and that no path sends more than one request once delivery became possible.
- The approval suites prove that review-only records can't schedule, that every mutation of a bound component invalidates the approval, and that an unbound legacy queue row can't reach HTTP transport.
- Validation found a real gap: queue revalidation didn't check the caption-to-image mapping. After the fix, remapping an image in the same batch invalidates the approval even when the replacement image has identical bytes.
- Clock independence: on 2026-08-05, hardcoded fixture dates passed their own publish slot and turned 6 approval tests and 17 publisher tests red with no code change. Instead of moving the dates further out, slots are now derived from the run's own clock. A ClockIndependenceTestCase re-runs the approval and delivery path under clocks shifted from one day to a century, and it fails statically if a hardcoded slot date is reintroduced. Both suites were also run end to end on clocks from -10 to +100 years, with no test weakened, skipped, or retried.
Results
- The duplicate-post failure mode is gone. An unprovable send stops the batch, prints the reconcile command, and makes the scheduled check exit non-zero until a person decides.
- Scheduling authority is now a sealed record of exactly what will be posted, where, and when. It is no longer a review verdict that can drift away from the content.
- The safety suites no longer have a hidden expiry date.
TODO (Matt): numbers to add
- TODO: Number of posts published through the guarded path since August 2026.
- TODO: Number of ambiguous sends held and reconciled (and how many turned out to be live), if tracked.
How coding agents fit in
- Shipped as three agent-authored pull requests (#152 approvals, #155 at-most-once delivery, #158 clock independence). Each went through an automated review, lint, and CI gate before merge, and that gate's fix-up commits are part of the history.
- The rules are written down for future agents in docs/publisher-delivery-outcomes.md and docs/final-publish-approval.md. For example: never widen the retry-safe class, and never resolve an ambiguous record from code.