Skip to main content

Module persistence

Module persistence 

Source
Expand description

Optional sqlite persistence for captured events.

The ring buffer in super::ring::TopicRing is the primary read path — fast, in-memory, bounded. Sqlite is the historical store: every event we push to a ring is also appended to a single events table so the GET /events/{topic}/history?since_ts=… endpoint can reach back beyond ring capacity (and beyond container lifetime).

Schema:

CREATE TABLE events (
  id           INTEGER PRIMARY KEY AUTOINCREMENT,
  cursor       INTEGER NOT NULL,
  topic        TEXT    NOT NULL,
  received_at  TEXT    NOT NULL,    -- ISO 8601
  reset_epoch  INTEGER NOT NULL,
  payload      TEXT    NOT NULL     -- JSON
);
CREATE INDEX events_topic_received_at ON events(topic, received_at);

Retention is configurable per topic:

  • orders — 90 days (default)
  • pnl — 90 days
  • marketdata:* — 14 days (high volume; bigger window doesn’t pay)
  • gap — 365 days (low volume, useful forever)

[EventLog::prune_older_than] is intended to be called from a once-an-hour task; nothing in the read path waits on it.

Structs§

EventLog
Sqlite-backed historical event store. Wraps a [Connection] in a Mutex so it can be shared across the connector + axum tasks without unsafe juggling. Reads are fast; writes serialise.
RetentionPolicy
Per-topic retention policy. Topics not in the map default to 30 days.