Overview

High-performance Substreams modules for extracting, processing, and persisting orderbook events from Polymarket's CTF Exchange and Neg Risk Exchange contracts on Polygon. Built with foundational stores for efficient parallel execution and includes ready-to-use SQL and Clickhouse sinks.

Key Features

Quick Start

Installation

# Install Substreams CLI
brew install streamingfast/tap/substreams

# Authenticate
substreams auth

Run Streaming Modules

# Stream all order fills from both exchanges
substreams run https://spkg.io/PaulieB14/polymarket-orderbook-substreams-v0.2.0.spkg \
  map_all_order_fills \
  -e polygon.substreams.pinax.network:443 \
  -s 57000000 -t +1000

# Stream market analytics
substreams run https://spkg.io/PaulieB14/polymarket-orderbook-substreams-v0.2.0.spkg \
  map_market_orderbooks \
  -e polygon.substreams.pinax.network:443 \
  -s 57000000 -t +1000

# Stream global platform stats
substreams run https://spkg.io/PaulieB14/polymarket-orderbook-substreams-v0.2.0.spkg \
  map_global_orderbook_stats \
  -e polygon.substreams.pinax.network:443 \
  -s 57000000 -t +1000

Architecture

                              Polygon Blockchain
                                     │
                                     ▼
                            ┌─────────────────┐
                            │ Firehose Blocks │
                            └─────────────────┘
                                     │
                    ┌────────────────┴────────────────┐
                    ▼                                 ▼
        ┌───────────────────┐             ┌───────────────────┐
        │   CTF Exchange    │             │  Neg Risk Exchange│
        │ Order Fill Events │             │  Order Fill Events│
        └───────────────────┘             └───────────────────┘
                    │                                 │
                    └────────────┬────────────────────┘
                                 ▼
                    ┌────────────────────────┐
                    │   map_all_order_fills  │
                    │  (Combined Event Stream)│
                    └────────────────────────┘
                                 │
            ┌────────────────────┼────────────────────┐
            ▼                    ▼                    ▼
   ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐
   │  store_markets  │  │  store_traders  │  │store_global_stats│
   │ (Market Stats)  │  │ (Trader Stats)  │  │ (Platform Stats) │
   └─────────────────┘  └─────────────────┘  └─────────────────┘
            │                    │                    │
            ▼                    ▼                    ▼
   ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐
   │map_market_      │  │map_trader_      │  │map_global_      │
   │orderbooks       │  │accounts         │  │orderbook_stats  │
   └─────────────────┘  └─────────────────┘  └─────────────────┘
            │                    │                    │
            └────────────────────┼────────────────────┘
                                 ▼
                    ┌────────────────────────┐
                    │        db_out          │
                    │   (SQL/Clickhouse)     │
                    └────────────────────────┘
                                 │
                    ┌────────────┴────────────┐
                    ▼                         ▼
           ┌─────────────────┐      ┌─────────────────┐
           │   PostgreSQL    │      │   Clickhouse    │
           └─────────────────┘      └─────────────────┘

Modules

Layer 1: Event Extraction

Layer 1.5: Combined Events

Layer 2: Foundational Stores

Layer 3: Analytics Outputs

Layer 4: Database Sinks

SQL Sink (PostgreSQL)

Setup

# Create database
createdb polymarket_orderbook

# Apply schema
psql -d polymarket_orderbook -f schema.sql

# Setup sink
substreams-sink-sql setup \
  "psql://user:pass@localhost:5432/polymarket_orderbook?sslmode=disable" \
  https://spkg.io/PaulieB14/polymarket-orderbook-substreams-v0.2.0.spkg

# Run sink
substreams-sink-sql run \
  "psql://user:pass@localhost:5432/polymarket_orderbook?sslmode=disable" \
  https://spkg.io/PaulieB14/polymarket-orderbook-substreams-v0.2.0.spkg \
  -e polygon.substreams.pinax.network:443

Example Queries

-- Top markets by volume
SELECT * FROM top_markets_by_volume;

-- Top traders by volume
SELECT * FROM top_traders_by_volume;

-- Recent large trades (> 1000 USDC)
SELECT * FROM recent_large_trades;

-- Market activity over time
SELECT
  DATE(created_at) as date,
  COUNT(*) as trades,
  SUM(taker_amount_filled::numeric / 1e18) as volume
FROM order_fills
GROUP BY DATE(created_at)
ORDER BY date DESC;

Clickhouse Sink

Setup

# Create database
clickhouse-client -q "CREATE DATABASE polymarket_orderbook"

# Apply schema
clickhouse-client -d polymarket_orderbook < clickhouse-schema.sql

# Setup sink
substreams-sink-sql setup \
  "clickhouse://default:@localhost:9000/polymarket_orderbook" \
  https://spkg.io/PaulieB14/polymarket-orderbook-substreams-v0.2.0.spkg

# Run sink
substreams-sink-sql run \
  "clickhouse://default:@localhost:9000/polymarket_orderbook" \
  https://spkg.io/PaulieB14/polymarket-orderbook-substreams-v0.2.0.spkg \
  -e polygon.substreams.pinax.network:443

Example Queries

-- Top markets by 24h volume
SELECT
    market_id,
    sum(total_volume) as volume_24h,
    sum(trades_count) as trades_24h
FROM hourly_volume
WHERE hour >= now() - INTERVAL 24 HOUR
GROUP BY market_id
ORDER BY volume_24h DESC
LIMIT 10;

-- Trader leaderboard
SELECT
    id,
    total_volume,
    trades_quantity,
    total_fees
FROM trader_analytics FINAL
WHERE is_active = 1
ORDER BY total_volume DESC
LIMIT 100;

-- Hourly volume trend
SELECT
    hour,
    sum(total_volume) as volume,
    sum(trades_count) as trades
FROM hourly_volume
WHERE hour >= now() - INTERVAL 7 DAY
GROUP BY hour
ORDER BY hour;

Data Schema

OrderFilledEvent

MarketOrderbook

Contract Addresses

Using as a Dependency

Import this package to build higher-level analytics:

# substreams.yaml
imports:
  polymarket: https://spkg.io/PaulieB14/polymarket-orderbook-substreams-v0.2.0.spkg

modules:
  - name: my_analytics_module
    kind: map
    inputs:
      - map: polymarket:map_all_order_fills
    output:
      type: proto:my.custom.Analytics

Build from Source

# Clone repository
git clone https://github.com/PaulieB14/polymarket-orderbook-substreams
cd polymarket-orderbook-substreams

# Build
substreams build

# Run locally
substreams run substreams.yaml map_all_order_fills \
  -e polygon.substreams.pinax.network:443 \
  -s 57000000 -t +100

Performance

Related Projects

• Polymarket P&L Subgraph - Track profit & loss
• Substreams Documentation - Learn more about Substreams
• Substreams Sink SQL - SQL sink documentation

License

MIT License - see LICENSE for details.