All packages
polymarket_orderbook_substreams

polymarket_orderbook_substreams

PaulieB14
v1.0.0/50 downloads/Repository

Package ref

polymarket-orderbook-substreams@v1.0.0

Run package

Start block
57000000, the module's first block

This module reads a store, which starts empty at the module's first block: it runs from there, so its output is from then, not from the chain head. Any later block would first need the store rebuilt up to it.

README

Polymarket

Polymarket Orderbook Substreams

Real-time orderbook analytics for Polymarket prediction markets on Polygon

Substreams Package CLOB v1+v2 Polygon License PostgreSQL Clickhouse


Polymarket orderbook flow on Polygon: every fill, match, fee and resolution from CLOB v1, CLOB v2 and Protocol V2 (ExchangeV3). Includes per-market, per-trader and global aggregates, with schemas for PostgreSQL and ClickHouse.

What's new in v1.0.0 (breaking)

#ChangeWhy
1ExchangeV3 0xe3333700cA9d93003F00f0F71f8515005F6c00Aa (Protocol V2) is indexedIts fills were missing (705 in a 500-block sample)
2Single pass: map_events replaces the 8 per-exchange mapsOne block decode instead of eight
3blockFilter (ethcommon:index_events) plus lazy block decode (&BlockLazyView) and constant address bytesBlocks with no Polymarket logs are skipped; no Hex::decode per log
4FeeCharged (v1 and v2/V3) → fees table. Resolutions (CTF ConditionResolution, Protocol V2 ConditionResolved) → resolutions tableNew data
5OrdersMatched → orders_matched table. v1 matches are now decodedThe v0.x v1 ABI had no indexed fields, so v1 OrdersMatched never matched
6Aggregates are cumulative (add stores) and written with upsert_rowv0.x wrote each fill's own values with update_row onto rows that were never created
7block_timestamp on every event tableTime-range queries
8Correct side and price: v1 side comes from makerAssetId == 0; price = collateral ÷ shares, fee-awarev0.x used a digit-parity heuristic and maker ÷ taker amounts (inverted for sells)
9ClickHouse schema with PARTITION BY toYYYYMM(block_timestamp) / ORDER BY (token_id, block_timestamp, id), and ReplacingMergeTree for aggregatesQuery performance
10Deterministic ordering (BTreeMap) and unit testsReproducible output

Migrating from v0.x

  • Renamed or removed modules:
    • map_ctf_exchange_order_filled, map_neg_risk_exchange_order_filled, map_ctf_exchange_orders_matched, map_neg_risk_exchange_orders_matched, and the four *_v2_* maps are replaced by map_events (OrderbookEvents: fills, matches, fees, resolutions).
    • map_all_order_fills is kept, with the same OrderFilledEvents type, now fed from map_events.
    • store_markets, store_traders and store_global_stats are replaced by store_totals, store_last_price, store_first_trade and store_last_trade.
    • map_market_orderbooks, map_trader_accounts, map_global_orderbook_stats and db_out keep their names.
  • Semantics:
    • order_fills.id is now tx_hash-log_index.
    • market_orderbooks.id is the token id, and condition_id is dropped.
    • Market and global counts are per match (the taker order's fill). Trader stats are per order owner (maker), so the exchange address is no longer counted as a trader.
    • New OrderFilledEvent fields: exchange, log_index, shares, collateral.
  • SQL: the schema changed. Recreate the tables (substreams-sink-sql setup) and resync from the start block.

Quick start

substreams run polymarket-orderbook-substreams@v1.0.0 map_events -e polygon.streamingfast.io:443 -s -1000 -t 0
substreams-sink-sql setup "psql://user:pass@localhost:5432/ob?sslmode=disable" polymarket-orderbook-substreams@v1.0.0
substreams-sink-sql run   "psql://user:pass@localhost:5432/ob?sslmode=disable" polymarket-orderbook-substreams@v1.0.0
# ClickHouse: apply clickhouse-schema.sql, then use a clickhouse:// DSN

Modules

Block ─(blockFilter, lazy)─ map_events ─┬─ map_all_order_fills
                                        ├─ store_totals ─────┬─ map_market_orderbooks ─┐
                                        ├─ store_last_price ─┘                         │
                                        ├─ store_first/last_trade ─ map_trader_accounts ┤
                                        ├─ map_global_orderbook_stats ──────────────────┤
                                        └───────────────────────────────────────────── db_out

Semantics (from Polymarket Trading.sol)

  • maker is the owner of the order that was filled, and side is that order's side. The taker order's own fill has the exchange as taker.
  • v1: the owner receives taking − fee. v2/V3: the fee is collateral, so a BUY pays makerAmountFilled + fee and a SELL receives takerAmountFilled − fee. shares / collateral apply this.
  • Amounts are raw 6-decimal base units.

Contract addresses

From https://docs.polymarket.com/resources/contracts. The v1 exchanges come from github.com/Polymarket/ctf-exchange. All of them are in src/decode.rs.

ContractAddress
CTF Exchange v1 / Neg Risk v10x4bfb41d5b3570defd03c39a9a4d8de6bd8b8982e / 0xC5d563A36AE78145C45a50134d48A1215220f80a
CTF Exchange v2 / Neg Risk v20xE111180000d2663C0091e4f400237545B87B996B / 0xe2222d279d744050d28e00520010520000310F59
ExchangeV3 (Protocol V2)0xe3333700cA9d93003F00f0F71f8515005F6c00Aa
Conditional Tokens0x4D97DCd97eC945f40cF65F87097ACe5EA0476045
BinaryModule / NegRiskModule0x1000008dD9001B968442c1000017eaE6E0dA00Ba / 0x200000900045e3B6259600682756002200028933

Build

substreams build   # CLI >= 1.25, buffa codegen with lazy_views=true (buf.gen.yaml)
cargo test --lib

Not included

  • Unique-trader and active-market counts (not cheaply countable with stores). These fields stay 0.
  • Order-book depth, spreads, volatility and similar fields in the proto are not computed. Fills don't carry resting-order state.

License

MIT

Modules

Execution graph

12 modules
map

map_events

Single pass (lazy block decode) over the five Polymarket exchanges: OrderFilled, OrdersMatched, FeeCharged; plus CTF ConditionResolution and Protocol V2 ConditionResolved. Replaces the eight per-exchange maps of v0.x.

from #57000000
store

store_first_trade

from #57000000

Store value

int64

Update policy

min

store

store_last_price

{token_id} -> price of the latest match

from #57000000

Store value

string

Update policy

set

store

store_last_trade

from #57000000

Store value

int64

Update policy

max

store

store_totals

Counters: m:{token}:trades|buys|sells|volume|fees, t:{owner}:..., g:...|fee_revenue

from #57000000

Store value

bigint

Update policy

add

Show 2 dependencies
map

ethcommon:all_events

`all_events` gives you all the events in a block (from successful transactions), with basic block hash/number/timestamp and transaction hash

Output

sf.substreams.ethereum.v1.Events

blockIndex

ethcommon:index_events

`index_events` sets the following keys on the block: * Event signatures evt_sig:0x0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef (signature in hex, prefixed by 0x) * Event address evt_addr:0x0123456789abcdef0123456789abcdef01234567 (address in hex, prefixed by 0x)

Output

sf.substreams.index.v1.Keys