Design notes on the fastest published verifiable matching engine, and what it took to get there.
Centralized exchanges beat DEXs on latency and UX. That's not controversial. Every recent DEX has been some version of the same bet: can you close the latency gap without giving up the thing that makes a DEX worth building, which is that anyone can independently verify what the exchange did.
Pulse answered part of that question. They built a Rust matching engine that hit 125k orders per second on a laptop with 7.92 μs mean latency, which was enough to prove the shape of the design worked. We started Vela to push the same shape further and to actually settle the resulting state on Ethereum in a way a normal user could audit.
Vela is a central limit order book written in Rust. Each market runs as its own concurrent shard. Every batch produces a keccak256 state root that gets anchored on Ethereum every ten minutes. Deposits and withdrawals go through an operator-signed smart contract with a seven-day emergency exit escape hatch. The matcher runs at 0.38 μs p50 and sustains 2.5 million operations per second in isolation. The end-to-end path across HTTP and WebSocket, including signature verification and WAL fsync, closes at 16.9 ms p50 on Apple M3 over loopback.
The numbers are the headline but they aren't the point. The point is that every match is deterministic and replayable from the write-ahead log, hashed into a state root, and posted on a public chain. If we ever misreport what happened, anyone with the log can prove it. This piece walks through how the system fits together, the specific changes that made the numbers move, and the parts we haven't built yet.
The other thing that happened between the first writeup and this one is that the surface got wider. Session keys, iceberg orders, self-trade prevention, a points-and-referrals stack, historical data exports, an IEX-style speed bump, and an MM credit-vault LP product landed as the growth-unblock tier. A permissionless listing market, sub-accounts, volume-tiered maker rebates, a WebSocket drop-copy channel, and multi-chain deposit routing landed as the product-surface tier. On top of that we added an agentic surface (MCP server, capability-scoped session keys, ERC-8004 reputation attestations, verifiable-intent orders, prompt-injection firewall, copy-trading strategy contracts, signed backtest receipts) and a large-scale surface (spot borrow-lend money market, SPAN-style portfolio margin, a perpetuals crate scaffold, and a FIX 4.4 gateway). Sections 11 through 14 are the new material.
There are two sets of numbers to look at, and mixing them up is how every “fastest DEX” benchmark ends up rightly torn apart.
Tier 1 is the isolated matching engine. One thread calling engine.process(), no HTTP, no signature verification, no I/O. Ten markets loaded at book capacity, fifty makers, one taker, roughly 98% cancels and 2% fills. This is the same workload Pulse published, so the comparison at the engine layer is like-for-like.
| Metric | Vela (Apple M3) | Pulse (Apple M2 Pro) |
|---|---|---|
| Match latency p50 | 0.38 μs | 7.92 μs |
| Match latency p99.9 | 0.92 μs | not reported |
| Throughput | 2,500,000 ops/sec | 125,000 ops/sec |
| FOK rollback (CoW) | 841 ns | not reported |
| Fee calculation overhead | ~0.2 μs | not reported |
Tier 2 is the end-to-end path a real client actually walks: sign an order in the browser, ship it over WebSocket, verify the ECDSA signature in the API layer, run it through the batch dispatcher, hit the shard engine, write to the WAL, fan the response back out.
| Metric | Value |
|---|---|
| Round-trip p50 | 16.9 ms |
| Round-trip p99 | 19.9 ms |
| Per-client ceiling | ~59 ops/sec (single connection, single client) |
The gap between the two tiers is the interesting part. Tier 1 says the matcher can move at silicon speed. Tier 2 says the wire around it moves at network and disk speed. Both are true simultaneously and neither is misleading if you keep them straight.
Vela is a ten-crate Cargo workspace. The split follows lock granularity: everything mutated on the hot path lives in engine, everything that touches disk or the network lives in api, each cryptographic concern gets its own crate, and the two big product bets (perpetuals math and the FIX 4.4 codec) live in their own dependency-light crates so they can move without touching the matcher.
Everything from the browser to the Ethereum settlement layer transits through one Axum process. There is no separate matching service, no separate WebSocket gateway, no sidecar. This is a deliberate constraint. Inter-process messaging is where every DEX we've profiled bleeds tens of microseconds, and colocating the matcher, the WAL, and the response fan-out keeps the whole hot path inside one address space with one lock hierarchy.
All exchange state lives in memory. Balances, resting orders, user metadata (nonce windows, open order IDs, credit ratios, quoted notional). Everything mutable flows through a StateCache that mirrors writes into a depth-32 sparse Merkle tree, so producing a state root at commit time is O(dirty × 32) rather than O(state size).
Durability comes from two things running in parallel. Every sixty seconds the process snapshots the entire engine to a JSON file on the Fly volume. On every order the API layer appends WAL records (an ORDER_POST, then an ORDER_PROCESSED, then a FILL_CREATED for each fill) with a single amortized sync_all per dispatch batch. On restart the process loads the newest snapshot and replays the WAL from the last checkpoint to reconstruct the exact pre-crash state. Clean shutdowns write a clean_shutdown: true marker into the final snapshot so recovery can skip replay when the previous exit was graceful.
Choosing a sparse Merkle tree over a Merkle Patricia trie was deliberate. The MPT is what Ethereum uses because it needs prefix-compressed keys for RLP-encoded account addresses. Our keys are already 32-byte hashes of structured state, so path compression buys nothing, and the fixed-depth SMT gives us O(1) proof size and cheaper delta computation.
Every market has its own OrderBook. Bids and asks are two BTreeMap<Price, PriceLevel>, and each level holds a VecDeque<Order> in arrival order. Best bid is bids.keys().next_back(), best ask is asks.keys().next(). Both are O(log n).
The matching loop is price-time priority. A taker walks matchable levels through an iterator that yields references directly into the book:
let matchable_iter: Box<dyn Iterator<Item = (u64, &VecDeque<Order>)>> =
match order.side {
OrderSide::Bid => Box::new(book.matchable_asks_ref(order.price)),
OrderSide::Ask => Box::new(book.matchable_bids_ref(order.price)),
};The _ref in the method name is doing a lot of work. An earlier version cloned matching levels into a Vec<(Price, Vec<Order>)> so the code could iterate without holding the book borrow. That single allocation was almost a third of the wall-clock cost of a match on the 98%-cancel workload. Switching to borrowing iteration was the largest single-commit throughput win we've landed.
Four order types. GTC rests. Post-Only rejects if it would cross. IOC matches what it can and cancels the rest. FOK either matches in full or reverts every state change it touched. FOK atomicity is why the engine matches through a DeltaBuffer, which is a copy-on-write overlay that buffers balance mutations, metadata mutations, and order-book operations. On success we call delta.commit(engine) and every buffered write applies. On rejection we call delta.rollback() and drop the buffer. Measured cost of a full FOK rollback is 841 ns.
Nonce validation happens against a fixed-size ring buffer of the last twenty accepted nonces per user. Twenty is not arbitrary. It matches the maximum number of in-flight orders we let a market maker have before requiring an acknowledgement, which lets makers pipeline order submission without waiting on network round-trips. Storing the window as a fixed [u64; 20] instead of the original BTreeSet<u64> cut per-order metadata clone cost from 200 ns to 58.2 ns.
Everything up to this point describes a single-market matcher. In production the engine is sharded: each market gets its own MatchingEngine instance behind its own tokio mutex, and the batch dispatcher hands orders out across shards in parallel. Because user state (balances, nonces, credit) is shared across markets, this needs a protocol.
The protocol runs three phases per batch. In phase one we take a write lock on UserState, apply deposits and withdrawals directly, snapshot balances and metadata, then validate every incoming order against the snapshot while tracking in-batch reservations so a user can't double-spend across markets. Order IDs get allocated here. Nonces do not get accepted here and balances do not get locked here. That's the shard's job. Then we release the lock.
Phase two acquires the per-market shard locks concurrently through join_all. Each shard receives the snapshot as its DeltaBuffer base and processes the orders routed to it. Nonces are accepted, balances are locked, matches happen. The shards run in parallel and don't coordinate.
Phase three takes the write lock on UserState again and folds each shard's final balance, metadata, and fee deltas back into the shared state.
The subtle bug in this design was in the fold. The original code did current.nonce_window = final_m.nonce_window.clone(). Whichever shard the loop applied last silently overwrote nonces that earlier shards had accepted, and replay protection became order-dependent. The fix was to union the windows using the same eviction policy as accept(), keeping the newest NONCE_WINDOW_SIZE nonces across all shards. Three property tests now cover the distinct, duplicate, and overflow cases.
Axum on tokio, with a batch dispatcher sitting between the HTTP handlers and the shard engines. The dispatcher opens a window on the first incoming order and closes it at whichever comes first: 500 μs elapsed or 256 orders accumulated. Both bounds are configurable via VELA_BATCH_WINDOW_US and VELA_BATCH_MAX_SIZE. Whatever the window closes on gets shipped to MarketShards::dispatch_sharded_batch as one logical unit. One mutex acquisition, one commit, one WAL flush.
Every order request gets a oneshot channel receiver back. If a shard engine wedges (panic, deadlock, GC pause on the state RwLock), the receiver is wrapped in tokio::time::timeout with a 500 ms default and the client sees a 504 rather than a hung request. The timeout is VELA_DISPATCH_TIMEOUT_MS.
WebSocket is the primary transport for live traders. Four channel types: orderbook:{market}, trades:{market}, markets, and per-user authenticated account:{address}. Every envelope carries a per-channel sequence number so a client can detect a gap after reconnection and re-request from the last known seq. The public order-book broadcast task copies raw (price, quantity) tuples out from under the engine mutex, releases the lock, and does the JSON serialization outside. Earlier versions held the lock across serde_json::json! for every subscribed market on every tick, which pinned the mutex across hundreds of allocations per iteration and stalled every incoming order for the duration.
Observability is a hand-rolled /metrics endpoint emitting Prometheus text format. Nothing exotic: orders per second, batch dispatch latency, WebSocket client count, feed drops, order-channel send failures, last snapshot timestamp. A scrape-side aggregator can compute percentiles from the drainable batch-size histogram.
Every dispatch batch produces a CommitBatch: a snapshot of balances and metadata at commit time, the list of requests, and a keccak256 root over the fill IDs in the batch. The committer writes this to a durable log and hands the state root off to the anchor task, which posts it on Ethereum every ten minutes.
The committer is also where forced-inclusion requests get injected. A user who thinks the operator is censoring their orders can call /force-include (production will verify an L1 Merkle proof that the request was posted to a delayed inbox on Ethereum; the current beta gates it on an admin token as a placeholder). Force-included requests get prepended to the next batch after a configurable timeout, bypassing the normal dispatch path.
Vela runs an optimistic-ZK settlement model, not full ZK per batch. Generating a STARK or SNARK proof over a matching-engine state transition function at 2.5M ops per second is four to six orders of magnitude too slow with today's provers. So we do what Arbitrum does: assume batches are valid, post the root, and let anyone challenge inside a fixed window.
The zkvm crate ships with a PlaceholderProver that returns a deterministic tag rather than a real proof. The trait interface, the batch-proof storage, and the on-chain anchoring are wired up so that swapping in a real prover the day one is fast enough is a matter of replacing one implementation.
The prover is now selectable at boot via VELA_PROVER=placeholder|sp1. The SP1 path uses an HTTP-backed Sp1Prover that speaks to any SP1-compatible proving service (Succinct's Prover Network or a self-hosted sp1-server) via VELA_SP1_PROVER_URL. When no URL is set, it falls back to a deterministic sp1-mock pseudo-proof (a domain-tagged keccak of the public inputs) so downstream verification code has something concrete to exercise in CI without needing a live prover network. A matching Sp1Verifier re-derives the mock proof from public inputs for the mock branch and defers to a verify endpoint for the real branch. The remaining critical-path work is refactoring the matcher STF into a no_std core that compiles as an SP1 zkVM guest ELF; that's the multi-week piece.
The threshold-encrypted order book uses real BLS12-381 threshold ElGamal via the blst FFI, with Lagrange share reconstruction on the group side. Recent hardening: at key generation we now derive per-node public key shares pk_i = share_i · G and expose a verify_pk_shares_reconstruct_group() function that Lagrange-combines any t of them in the exponent and checks against the group pub key. Rotated or swapped share configs no longer pass silently. Full Chaum-Pedersen NIZK proofs of per-share correctness on the decrypt path (so a bad share can be attributed to a specific node cryptographically) are the audit-blocked follow-up.
Traditional DEXs make you escrow every quote. To quote 100 ETH of bids across a market you need 100 ETH in the contract, locked and unusable for anything else. Serious market makers won't operate under that constraint and it's the biggest reason professional flow lives on CEXs. Vela lets a maker quote beyond their deposit up to a per-user credit ratio (default 5×, tunable per address). Quoted notional lives in UserMetadata.total_quoted_notional and is enforced on every post-order.
What happens when a maker breaches their ratio is more interesting than the ratio itself. Usually a breach happens because a fill drained available collateral, or because the credit ratio was revised down. Instead of rejecting the incoming order and leaving the book stale, the engine walks the maker's open orders in arrival order and cancels the oldest ones until they're back inside their limit. The tightest quotes stay live. Only the deepest, oldest quotes get pruned.
Every taker fill gets scored on the hot path in the range [0.0, 1.0]. The score combines three signals: an order-flow-imbalance accumulator maintained in a fixed-size ring buffer, the depth the taker walks into the book, and the taker's size relative to the maker's quoted liquidity at that level. The scorer uses interior mutability (RefCell<ToxicityScorer>) so it can update per fill without needing a &mut self from the matching loop.
Fills above the threshold trigger a temporary credit penalty on the affected maker. The same maker keeps their spread but their effective credit ratio drops for the penalty duration. The score also broadcasts on an authenticated ws://.../feed/toxicity channel so makers can react to a run of toxic fills before the penalty even lands. Hot-path overhead stays under 50 ns at p50 because the OFI ring is allocated once at shard construction and updated in place.
The most interesting thing on the roadmap, and the thing that's in the committee crate today as a working prototype, is a threshold-encrypted order book. Orders arrive as ciphertexts encrypted to a committee public key. A threshold of committee nodes runs partial decryption. The plaintext order is only revealed once t-of-n shares are combined. Every decryption produces a public proof that gets committed alongside the batch.
This buys frontrunning resistance at the sequencing layer. The operator can't peek at pending orders, can't reorder them by observed content, and can't sandwich a large taker. The threshold ElGamal itself is real BLS12-381 via blst; the per-share HMAC that the committee nodes use for share-submission auth is the layer that still needs to move to per-share BLS signatures + Chaum-Pedersen correctness proofs before slashing is meaningful. The pipeline (ciphertext arrival, pending queue, partial-decrypt request, threshold combine, proof recording, per-node pk-share Lagrange consistency check) is fully wired.
Called out in the matching section but worth expanding here because it's the reason FOK works. Every mutation an order might perform (balance debit, balance lock, metadata update, order insert, order remove, partial fill) buffers into the delta. If the order matches in full, commit() replays the buffered writes into the engine. If it rejects, rollback() discards the buffer. Partially-applied FOK state corruption is not representable.
Autonomous agents are becoming a real fraction of trading intent. The interesting design question is not whether an exchange should let LLM-driven callers place orders, it's what a venue needs to expose so agent flow can be underpriced when it's clean and overpriced (or blocked) when it's adverse. Vela added an agentic surface with that framing.
MCP server. A JSON-RPC 2.0 endpoint at POST /mcp speaks the Model Context Protocol so any MCP-capable client (Claude Desktop, agent runtimes) discovers eight tools out of the box: list_markets, book_snapshot, toxicity_score, points, portfolio, place_order, cancel_order, place_twap. The signed-order tools reuse the master-or-agent verification path so an agent using its delegated session key can operate without exposing the master signature to the model.
Capability tokens on session keys. Session keys were the biggest single UX unlock (no more MetaMask popup per order); capability tokens are the risk layer that makes them safe to hand to an agent. An AgentDelegation now carries a CapabilityScope: allowed markets, allowed sides, allowed order types, max notional per hour, max notional per day. The scope is hashed into the delegation-signing message so a compromised agent can't widen its own permissions post-hoc. Rate enforcement runs as a two-phase check-and-record with atomic rollback on rejection so caps stay tight under concurrent order submission.
ERC-8004 reputation attestations. Every taker fill Vela processes is already public evidence. On top of that we emit portable reputation attestations: an operator-ECDSA-signed (address, dimension, score_bps, expires_at) tuple where score_bps is a weighted average of clean-flow (1 − average toxicity), volume, and activity. Attestations expire in 24 h so a downgrade in behavior actually costs the address something. Relying parties (Vela's own RFQ and credit-line paths, plus external ERC-8004-aware venues) verify the signature offline against the operator's well-known pubkey.
Agent-flow toxicity tier. The existing per-fill toxicity score got aggregated into a green / amber / red tier per address (thresholds default 0.3 and 0.6, notional-weighted rolling 30-day average, taker-side only). Green flow is unrestricted. Amber flow gets an extra 1 ms deterministic delay stacked on the IEX speed bump, so it still trades but doesn't win the quote-cancel race. Red flow is blocked at order intake until an operator clears it. Human retail almost never hits amber; badly-tuned bots do.
Verifiable-intent order type. A POST /orders/from-intent endpoint accepts signed natural-language intents (“buy 0.5 BTC-USDC at market”, “sell 100 SOL-USDC limit 145.25 post-only”) and runs them through a hand-rolled deterministic parser. The intent goes through a prompt-injection firewall first (regex + heuristics for override-instruction patterns, chat-template tokens, zero-width joiners, RTL overrides, oversize base64 blobs); a Block verdict short-circuits before the parser sees the text. On accept, the response is a signed receipt binding the raw intent bytes plus the parsed order plus a keccak256 intent_hash, so an auditor can re-derive the parse offline and prove what the agent meant vs. what the exchange saw.
Reasoning-trace audit log. Any signed order can carry an optional keccak256 hash of the caller's model reasoning trace plus an agent identifier string. Vela stores the hash on the order record, emits a structured tracing event on target=reasoning_trace for operator log aggregators, and exposes POST /agents/reasoning/attest which returns an operator-signed EIP-191 receipt binding (address, hash, agent_id, order_id, ts). The trace itself stays in the caller's own S3 or private log; Vela commits only the hash, so compliance auditors can prove the trace existed at submission time without Vela storing it.
Copy-trading strategy contracts. A strategy owner publishes a signed identity; followers subscribe with an allocation_bps and optional per-trade notional cap. Every mirrored order remains a normal signed order attributed to the follower. Owner cannot pull follower funds, cannot skim, cannot silently swap the strategy identity. mirror_quantity() scales follower size by allocation_bps with the notional cap applied per fill.
Reputation-collateralized credit lines. Agents with a non-expired reputation attestation can open a short-duration credit line sized by score_bps × $10/bp (default). The line auto-expires after 5 min; a background sweep every 10 s emits a warn-level event if the address is still drawn when it expires so ops can force-flatten. The idea is to underprice the collateral-fragmentation cost that keeps well-behaved agent flow on centralized venues.
Backtest attestation. A strategy owner can submit claimed backtest metrics (total return, drawdown, trade count, Sharpe) plus a window and a replay-hash. Vela counts the actual trades in the tape over that window, runs a deterministic sanity gate (drawdown ≤ 100%, trade_count ≤ tape, |Sharpe| ≤ 20, window well-formed), and returns an operator-signed receipt covering the whole tuple. The guarantee is narrow but real: the claimed metrics were derived from Vela's actual tape, not a fabricated dataset. Deterministic re-execution inside the fraud-proof harness is the follow-up.
Machine-readable stream schema. Every WebSocket message shape (order-book snapshot, trade, markets, account envelope, drop-copy fill, bare admin) is published as JSON Schema draft 2020-12 at GET /agent-stream/schema.json. Agent runtimes with structured-output support (OpenAI, Anthropic, Google) constrain their generated messages against it; agents with a JSON Schema validator reject malformed inbound messages instead of accepting them and crashing later.
Leveraged spot is the missing rung between cash spot and perps. Vela added a two-asset borrow-lend pool (USDC + ETH seeded, wBTC next) with the machinery to grow.
Interest-rate model. Kink curve, similar in shape to Compound V2 and Aave. Below the kink (default 80% utilization) the borrow rate is base + slope1 × u/u_kink. Above it, base + slope1 + slope2 × (u − u_kink)/(1 − u_kink). Defaults: 0 / 400 bps / 20 000 bps. Supply rate = borrow rate × utilization × (1 − reserve factor). Reserve factor default 10%.
Accrual. Index-based, lazy: each supply/borrow position stores a scaled principal; the market maintains borrow_index and supply_index (both starting at 1e18 ray) that tick up on any interaction. No per-block cron. Current-balance queries just multiply the stored principal by the current index. Interest accrued on the outstanding borrow book gets split between the reserve pool and the supply index in one pass.
Liquidations. Health factor = borrowing power / total borrow value, in bps. Under 10 000 = liquidatable. Public liquidator endpoint with a 50% close-factor cap: repay in one asset, seize in another at price × (1 + liquidation_bonus_bps), default 500 bps. Every borrow and every withdraw is HF-gated with atomic rollback if the post-op HF drops below 10 000.
Portfolio margin. A separate SPAN-style scenario-sweep engine runs on top of the borrow-lend positions. Twelve shocks per risk factor (BTC as primary at −30% / −20% / −10% / −5% / −2% / 0 / +2% / +5% / +10% / +20% / +30% plus a −50% extreme), scaled by each asset's correlation to BTC (env-configurable, defaults 90% for ETH, 75% for SOL, 0 for anything unmodeled). Account is solvent iff every scenario nets to ≥ maintenance requirement (default 5% of gross notional); opening a new position requires passing the initial threshold (default 10%). positions_from_borrow_lend() surfaces supplies as long positions and borrows as shorts so the sweep already sees today's balances. A /portfolio-margin/preview/:address endpoint runs the sweep against a hypothetical extra position so a caller can check whether a proposed order would push them under water.
The bet: an MM quoting BTC/USDC + ETH/USDC pays for margin twice under isolated rules even though the drawdowns are 90% correlated. Under portfolio margin, the hedged book's capital requirement drops toward the residual scenario risk, and the correlations show up in the math instead of being ignored.
Perps are where volume and revenue on DEXs actually live. Vela shipped the ledger side first in its own perp crate so the matching-engine integration and the ledger accounting can move in parallel.
Position math. Position carries a signed size (positive = long, negative = short), a volume-weighted entry_price, a funding_index_snapshot, and cumulative realized_pnl_micro_usdc. apply_fill(pos, fill_size, fill_price) handles four cases in one pass: open, add-to (VWAP the entry), close (realize P&L on the closed portion), and flip-and-reopen (realize the close, then reopen the residual at the fill price). All arithmetic on i128 so sign handling stays legible.
Funding. Rate = (mark − index)/index + interest, in bps per hour, clamped ±max_funding_bps_per_hour (default 100 bps/hour). accrue_funding() advances the market's cumulative funding_index by rate × elapsed_ms scaled into price space. settle_funding() subtracts (delta × size)/SIZE_SCALE from a position's realized P&L and updates its snapshot. Longs pay when premium is positive; shorts receive.
Margin. Initial margin bps = 10 000 / max_leverage. Maintenance = initial × maint_ratio_bps (default 5000 = 50%). margin_report() returns notional, initial requirement, maintenance requirement, unrealized P&L, equity, and pass/fail booleans. liquidation_price_micro_usdc() computes the price at which equity first hits maintenance — useful as a UI signal even before a real liquidator ships.
Service layer. api::perp_service::PerpRegistry seeds 8 majors at their configured leverages (BTC/ETH 50×, SOL/LINK 20×, HYPE/SUI/DOGE/ARB 10×). Endpoints: GET /perp/markets (accrues on read, shows funding rate + OI + margin bps), GET /perp/account/:address (per-position margin report with settled funding), POST /perp/positions/open (signed manual fill for dev/testing until the matcher bridge lands), POST /perp/admin/mark (signed mark bump until Pyth is wired).
What's not yet done: the matching-engine bridge (route perp orders through the existing sharded matcher with a margin gate on dispatch), Pyth pull-based index prices with Chainlink secondary and a 10 s staleness guard, insurance fund + ADL, the ZK circuit extension for the perp STF. That's the multi-quarter piece.
Serious flow does not walk in through a browser wallet. What follows is the collection of features that add up to “can a real desk connect.”
FIX 4.4 gateway. Every institutional OMS speaks FIX. WebSocket JSON forces custom adapters and gets a hard “no” from compliance-gated funds. Vela added a hand-rolled fix crate (SOH-delimited tag=value codec, BodyLength + checksum, typed builders for Logon / Heartbeat / TestRequest / ResendRequest / Logout / Reject / NewOrderSingle / OrderCancelRequest / OrderCancelReplace / ExecutionReport / OrderCancelReject) plus an api::fix_gateway tokio TCP listener that spins up when VELA_FIX_BIND is set. One task per connection, streaming parser draining complete frames from a rolling buffer, SessionState stamps outbound with monotonic MsgSeqNum and enforces monotonicity inbound. Logon → Heartbeat loop → NewOrderSingle → ExecutionReport(New/New) is the session envelope today; the bridge into AppState.order_tx with FIX→PostOrderBody translation is the next commit. Rolling FIX ourselves instead of wrapping quickfix-rs eliminates the C++ dependency that's painful on Fly.io and in distroless containers.
Sub-accounts + master API keys. One master wallet spawns N logically isolated child accounts, each with its own balance, positions, API key, and permissions (trade-only, read-only, withdraw). Non-negotiable for any prop desk running multi-strategy books; every child derives deterministically from the master signature so there's nothing new to register.
Volume-tiered maker rebates + formal MM program. A published fee schedule tied to 30-day rolling volume, evaluated at fill time so a maker whose volume just crossed a threshold sees the new rebate on the very next fill. MMs are P&L-driven; without a rebate ladder they won't quote tight.
WebSocket drop-copy. A dropcopy:0x<address> channel mirrors every fill on the account, delivered to the risk / back-office system on a separate connection from the trading connection. Institutions mandate segregation-of-duties: trading and risk systems consume fills independently.
MM credit vaults. LPs deposit USDC into a vault run by a chosen operator; PnL and fees stream on-chain. Vela's existing 5× MM credit system is the substrate — operators get leverage, LPs get the yield. Turns an internal MM feature into a retail product no competitor can copy without also having the credit system.
Permissionless market listing. Anyone posts a USDC bond plus oracle spec plus tick/lot params; the market goes live after a delay if unchallenged. HIP-1-style auction. The bond and the “unvetted” tag mitigate scam-token brand risk.
RFQ / block-trade venue. Off-book quote request for trades above the min-notional floor (default 250k USDC). Whitelisted MMs post signed quotes; requester accepts one; the trade settles atomically through the same delta-buffer path as CLOB matches. Book-improvement is enforced at accept time so an MM cabal can't collude to fill worse than the requester could have gotten in the book. As of Tier 3 the maker slot also admits any address with a non-expired reputation attestation ≥ 6000 bps, tagged with a MakerProvenance so the requester can see whether a quote came from a human desk or an agent maker.
Session keys / agent wallets. ECDSA subkeys delegated by the master via signed enrollment, scoped and rate-limited (see Section 11). The single biggest UX win: no MetaMask popup per order. Master compromises still let the attacker revoke or rotate every agent; agent compromises are bounded by the CapabilityScope.
The path from “works” to sub-microsecond p50 was mostly a sequence of surgical changes, each measured before and after. The high-signal ones:
| Change | Before | After |
|---|---|---|
| Zero-allocation matchable-level iteration | Vec<(Price, Vec<Order>)> per match | Borrowing iterator, no alloc |
| NonceWindow: BTreeSet → fixed [u64; 20] | user_metadata_clone: 200 ns | 58.2 ns (−71%) |
| Sharded matching (per-market lock) | single engine mutex | N shards, concurrent execution |
| ahash + mimalloc | stdlib HashMap + system malloc | 2-3× faster hashing on small keys |
| WAL fsync amortization | 1 fsync per WAL entry (3-5 per order) | 1 fsync per dispatch batch |
| WS lock-hold reduction | engine lock across JSON serialization | lock only for raw tuple copy |
| Release profile: fat LTO, panic=abort | default codegen-units, panic=unwind | smaller binary, tighter hot loops |
Two of these are worth expanding on because they explain most of the throughput gap.
The first version of the matcher pulled matching levels out of the book into an owned Vec<(Price, Vec<Order>)>. One allocation for the outer vec, one clone per level for the inner. On the 98%-cancel workload almost every taker call matched at most one level, but the allocation happened every time regardless. Adding matchable_asks_ref and matchable_bids_ref, which yield (Price, &VecDeque<Order>) pairs by reference into the underlying BTreeMap, moved the whole match inside the borrow region and dropped that allocation to zero.
A secondary bug hidden by the old code came out in the same commit. A resting order that had been partially filled before the current dispatch was being compared against its original quantity when checking full consumption. Any partial-fill-before-dispatch scenario would fail to remove the resting order after a fill that consumed the remaining portion. The switch to borrowing iteration made the code path shorter, and shorter code made the bug legible enough to notice.
The single-mutex matcher scales linearly with active markets. Sixteen markets doing 100k operations per second each all contend on the same lock. Sharding gives each market its own lock and lets the sixteen shards make progress concurrently, with the three-phase protocol described earlier keeping the shared user state coherent. The 2.5M ops per second figure in Tier 1 is achievable specifically because the ten-market benchmark can dispatch across ten shards without lock contention.
Where the placeholders still sit, and what the honest gap is:
sk_i · R can be attributed to a specific node and slashed. External audit (Trail of Bits, Least Authority, or Sigma Prime) is the gating dependency.tee crate still ships a PlaceholderAttester. Wiring AMD SEV-SNP attestation into the request path is well-scoped work with the interfaces already in place; timing is a function of the mainnet-audit calendar.no_std matcher-core refactor so the STF compiles as an SP1 zkVM guest ELF. That's the multi-week critical path per the roadmap; the surrounding wiring is done so the swap-in is local.perp crate owns position accounting and funding accrual. The matcher extension (route perp orders through the sharded matcher, plug a margin gate into dispatch), Pyth pull-based index prices with a Chainlink secondary and a 10 s staleness guard, insurance fund + ADL, and the ZK circuit extension for the perp STF are all follow-ups.ExecutionReport(New/New) so a FIX conformance client can validate the round-trip. Translating FIX fields into PostOrderRequest and pushing them through AppState.order_tx is the next commit.VelaSettlement.sol has not been through an external audit and shouldn't hold real funds until it is.ahash plus mimalloc combination that's worth 5-15% on the HashMap-heavy hot path. The M3 is also faster than the M2 Pro on single-core integer work, which accounts for some but not most of the gap.initiateEmergencyExit on the settlement contract, wait seven days, then call executeEmergencyExit to withdraw deposited funds directly. No operator signature required. Deposited funds are never in operator custody./force-include endpoint. In production it verifies an L1 Merkle proof that the request was submitted to a delayed inbox on Ethereum. Once submitted, the request is guaranteed to be included in the next batch after a configurable timeout, or the operator can be slashed via fraud proof.address(this) so signatures can't be replayed across contracts, and consumes the nonce so they can't be replayed on the same contract.ZkProver trait is where we swap in the real implementation.OrderBook::max_orders. On the current Fly VM (2 vCPU, 2 GB), the engine holds 10k orders per market across all sixteen markets while keeping the p50 numbers reported here. Serialized snapshots are around 1 MB at that load. Scaling further means either a larger VM, sharding markets across processes, or offloading historical order data while keeping only the live book in memory.perp crate's ledger — position accounting, funding-index accrual, margin math, and a service-layer with signed manual-fill endpoints for dev testing. The matching-engine bridge, Pyth wiring, insurance fund, and ZK circuit extension are the multi-quarter follow-up. Publishing the ledger side first lets us wire real integration tests against margin and funding before the matcher extension ships.Vela is being built at Monolith Investments LP. Code is on GitHub, docs at monolithsystematicllc.mintlify.app, and the live beta at vela.monolithsystematic.com (Sepolia only; do not deposit real funds).
Reach me at asomu@ucsd.edu with questions, corrections, or if you want to trade against it.