v4-substreams — the full Uniswap v4 subgraph, ported to Substreams

A complete port of subgraphs/v4-subgraph (The Graph / AssemblyScript) to Substreams. Complete means every one of the 19 entities in that subgraph's schema.graphql is produced — including the ones the sibling sibling partial-port package deliberately skipped because the backend does not query them.

Locating that package: the sibling partial-port package (it has moved twice during development; as of 2026-08-30 it lives at subgraphs/ponder + subgraph (v4 substream)/, NOT subgraphs/substreams — check before linking).

There are two output modules, over one emitter:

• graph_out emits EntityChanges against the v4-subgraph's own schema.graphql (copied here verbatim), so the package deploys as a Substreams-powered subgraph and every existing GraphQL query keeps working unchanged.
• db_out emits DatabaseChanges against schema.sql, so substreams sink postgres streams the same nineteen entities straight into Postgres. That is the path for the eight chains The Graph cannot host (see below): the package indexes them fine, only subgraph hosting is missing.

Both run src/modules/emit.rs, which is the only place any entity is assembled — so the two outputs cannot disagree about what a row contains, and their manifest input lists are identical by requirement (make guards).

	this package	sibling substreams/
Goal	full schema parity with the v4-subgraph	the 86 fields the backend actually reads
Output	graph_out → EntityChanges → graph-node, db_out → DatabaseChanges → Postgres	db_out → DatabaseChanges → Postgres
Entities	all 19	13 tables, a different shape
Chains	all 25 in networks.json	3

They are independent; this one does not replace or depend on that one.

---

Entity coverage

All nineteen, from schema.graphql:

Entity	Source
PoolManager	store_counts + store_totals
Bundle	store_eth_price
Token	map_tokens → store_tokens, store_totals, store_derived_eth
Pool	store_pools + store_pool_state + store_pool_fee + store_liq_* + TVL stores
Tick	store_tick_meta (price0/price1) + store_ticks (gross/net)
Transaction	map_events
Swap	map_event_rows
ModifyLiquidity	map_event_rows
UniswapDayData	map_intervals → six interval stores
PoolDayData, PoolHourData	same
TokenDayData, TokenHourData	same
Position	store_position_meta + store_position_owner
Subscribe, Unsubscribe, Transfer	map_events
EulerSwapHook	store_euler_hooks (the only entity that is ever deleted)
ArrakisHook	store_arrakis_hooks

Both eth_call sites the subgraph makes are ported too: ERC20 metadata (map_tokens) and AggregatorHook.pseudoTotalValueLocked (map_hook_tvl).

---

The one structural difference, and what it costs

A subgraph handler reads its own prior writes, event by event. Substreams forbids that — the module graph must be a DAG, and a module may not read the store it writes. That is not an assumption; it was measured:

$ substreams info substreams.yaml    # with a store listing itself as an input
Error: read manifest "substreams.yaml": modules graph has a cycle

So every read-modify-write chain in the subgraph becomes a chain of modules. Three consequences, each handled rather than hand-waved:

1. Within-block ordering — solved, not approximated. Where a handler needs state as of a specific log (the pool price a ModifyLiquidity was valued at, the token prices a Swap was valued at), the module takes the upstream store twice, once as get and once as mode: deltas. The first delta for a key carries its pre-block value in old_value; replaying the block's own events over that seed reproduces the exact value the subgraph would have read at any ordinal. See src/blockstate.rs.

2. Pool liquidity — solved with two stores. handleSwap assigns pool.liquidity (the log carries the absolute post-swap value, including tick crossings) while handleModifyLiquidity adds to it. One store has one update policy, so the two operations are split and recombined at read time:

liquidity(now) = anchor.liquidity + (cum(now) − anchor.cum_at_anchor)

store_liq::liquidity_now is the only place that formula exists. See src/modules/store_liq.rs.

3. Snapshot columns are end-of-block — a real divergence. Columns the subgraph assigns (close, liquidity, sqrtPrice, tvlUSD, token0Price) are written from end-of-block state rather than after each log. open, high and low are not affected — those are fed the price at each event's own ordinal — and every additive column (volume, fees, txCount, liquidityGross) is exact. The visible effect is confined to a snapshot column read at a block that contained more than one event for that pool.

derivedETH is computed by a bounded re-walk. findNativePerToken reads other tokens' stored derivedETH, which is self-referential. Rather than lag a block or restrict to one hop, store_derived_eth re-walks the whitelist graph recursively from the analytic base cases (native/wrapped = 1, stablecoin = 1/ethPriceUSD), memoised, capped at MAX_DEPTH = 3. The subgraph's stored value is a fixed point of exactly this recursion, so for any token reachable within the cap the two agree; beyond it a token gets no price rather than a wrong one.

---

Upstream behaviour reproduced deliberately

This is a port, not a rewrite. Where the subgraph does something surprising, the surprise is preserved and marked at the site:

• TokenDayData.untrackedVolumeUSD receives the tracked figure (swap.ts:337). It looks like a bug; it is the shipped behaviour.
• bundle.ethPriceUSD prefers the static price in handleInitialize but not in handleSwap (poolManager.ts:184 vs swap.ts:266). On Tempo — the only chain with a static price — the value can drift from 1 after the first non-hook swap.
• pool.feeTier tracks the last swap's actual fee, not the 0x800000 dynamic-fee sentinel. Ponder freezes the sentinel instead, so the two indexers disagree on dynamic-fee pools by construction.
• An unresolved decimals() read drops the pool entirely (poolManager.ts:79). Reproduced, but the reason is recorded in a flag and logged rather than the pool just being absent.

Fields that are structurally zero

These are non-null in the schema, initialised to zero by the subgraph, and never written again. They are emitted as zero for schema parity; giving them real semantics here would make this package disagree with the one it replaces.

Token.poolCount · Pool.liquidityProviderCount · Pool.observationIndex · Pool.collectedFeesToken0/1/USD · Pool.totalValueLockedUSDUntracked · Token.totalValueLockedUSDUntracked · PoolManager.totalValueLockedUSDUntracked · PoolManager.totalValueLockedETHUntracked · UniswapDayData.volumeUSDUntracked

The collected-fee columns are zero for a deeper reason: v4 settles fees silently inside modifyLiquidity and the amount appears in no log. Recovering it needs Call.return_data from an EXTENDED Firehose block — out of scope here, and analysed in docs/substreams-feasibility.md §5(a).

One field that is better than upstream

Transaction.gasUsed is hardcoded to zero in the subgraph, with the comment "needs to be moved to transaction receipt". Firehose carries the real value in-band, so this package emits it.

---

Per-chain configuration

The subgraph selects config with a 25-branch if/else on dataSource.network() and rewrites subgraph.yaml before every deploy. Substreams has a native equivalent, so the whole table lives in substreams.yaml under networks: and the binary is chain-agnostic — --network base is the entire deploy difference.

That block is generated, not typed: 25 chains × ~10 fields, including 16-entry whitelists and 32-byte pool hashes, is a guaranteed transcription error, and a wrong anchor-pool id does not fail — it produces ethPriceUSD = 0 and zeroes every USD figure on that chain. scripts/generate-networks.mjs executes the subgraph's own chains.ts (transpiled, with graph-ts stubbed) and reads the config out of it.

make networks    # regenerate after an upstream change
make guards      # fail if substreams.yaml has drifted from the subgraph

---

Build and deploy

make check       # guards + tests + wasm build — the gate
make pack        # → tickwise-uniswap-v4-full-v0.1.0.spkg

Then deploy as a subgraph:

graph deploy <slug> subgraph.yaml --node <studio-url> --ipfs <ipfs-url>

Or stream it into Postgres instead

make sink-setup           # createdb + `substreams sink postgres setup`
make sink-test            # 50k blocks, flushing every 100, to see rows land
make sink NETWORK=base    # the real thing
make sink-status          # answered from the database, not the sink's logs

sink postgres setup applies schema.sql — which substreams pack embeds in the .spkg from the manifest's sink: block — and adds its own cursors and substreams_history tables (the latter is how a reorg is undone). Those two are the sink's; do not define them in schema.sql.

Three things about the DDL are load-bearing, and tests/schema_sql.rs checks the first of them against schema.graphql column by column:

• Names are mechanical snake_case — PoolDayData → pool_day_data, totalValueLockedUSD → total_value_locked_usd — because db_out derives them at emission time rather than from a hand-written mapping.
• BigInt/BigDecimal are NUMERIC, never BIGINT: liquidity is uint128 and sqrtPriceX96 is uint160, so BIGINT overflows on ordinary mainnet rows.
• No array columns. The sink resolves each column against a fixed set of scan types and rejects anything else with "unsupported column type", so Token.whitelistPools is a ;-joined TEXT column rather than TEXT[].

One sink-side caveat worth knowing before a long backfill: its loader keeps one pending operation per primary key for an entire flush batch and refuses to upsert a key already scheduled for deletion in that batch. EulerSwapHook is the only entity that is ever deleted, so this can only be reached by uninstalling and redeploying the same EulerSwap tuple within one batch; --batch-block-flush-interval 1 makes every block its own batch and closes the window.

Which chains can actually be deployed

A substreams-powered subgraph needs BOTH a Substreams provider (to run the package) and a subgraph provider (to host the GraphQL). A chain can have one and not the other, and Studio reports either as the same bare error — "Specified network is not supported" — which does not tell you which is missing.

17 of 25 of our chains have both. The other 8 do not:

our network	registry id	substreams	subgraphs	blocked by
arc-mainnet	arc	0	1	no Substreams provider
blast-mainnet	blast-mainnet	1	0	no subgraph host
ink	ink	1	0	no subgraph host
megaeth-mainnet	megaeth	1	0	no subgraph host
monad	monad	1	0	no subgraph host
robinhood-mainnet	robinhood	2	0	no subgraph host
worldchain-mainnet	worldchain	1	0	no subgraph host
zora-mainnet	zora	2	0	no subgraph host

make deploy refuses these up front with that explanation rather than letting Studio return the opaque error. For a blocked chain the package is still fine — what is missing is hosting, so self-host graph-node against a Substreams endpoint for that chain.

Note also that our network names come from the v4-subgraph's dataSource.network() and 7 of 25 differ from The Graph's registry id (soneium-mainnet → soneium, unichain-sepolia → unichain-testnet, …). Substreams accepts ours as aliases; graph-node needs the canonical id, so prepare-deploy.sh writes that into subgraph.yaml automatically. The mapping is generated from The Graph's own registry — node scripts/generate-graph-networks.mjs refreshes it.

The .spkg is chain-specific — pack per chain. A packed package has one default network and that is what graph-node uses; it does not pass --network through, and source.package.params patches graph_out rather than the parameterised map_events, so it cannot carry chain config either. Deploying a mainnet package for another chain fails silently: no log matches, and the subgraph indexes nothing while looking healthy.

make deploy SLUG=<studio-slug> NETWORK=base

There is nothing to edit by hand. scripts/prepare-deploy.sh packs for the chosen chain and then DERIVES subgraph.yaml's network: and .spkg filename from it, so the two manifests cannot disagree — and graph build runs only after that rewrite, because it stages whatever package the manifest points at. The .spkg filename carries the network so the wrong artifact cannot be deployed by accident.

Guards

Both catch failures that compile, pack, and run without error:

Guard	Catches
scripts/generate-networks.mjs --check	per-chain config drifting from the subgraph
scripts/check-module-arity.mjs	manifest inputs vs Rust parameters, by count / order / kind

The second one exists because Substreams binds inputs to parameters positionally. Swapping two inputs of the same shape does not error — the module decodes the wrong bytes and silently produces nothing. The sibling package lost an entire table to exactly this, and its guard shipped broken twice (first matching one handler per file, then checking counts but not kinds), so this one checks kinds and fails if it cannot find a handler it expected.

Tests

cargo test    # 23 tests

The tick math is not trusted, it is tested: tests/tickmath.rs checks all 20 magic constants and 216 amount cases — in-range, out-of-range both sides, both signs of liquidity delta — against a fixture produced by running the reference TypeScript implementation. This is the fourth transcription of that ladder in the repo; a single mistyped digit would not crash, it would silently return slightly wrong amounts for one narrow tick range.

---

Coverage audit

The port was audited against the subgraph across five dimensions — entity/field coverage, handler side-effects, utility/math algorithms, per-chain config, and event/ABI coverage — with every claimed gap adversarially re-verified against the code before being accepted. Result: 19/19 entities and 197/197 non-derived fields emitted, each exactly once and with a setter matching its schema type; all 10 registered event signatures decoded; 28 of 29 utility functions ported algorithm-for-algorithm; all 25 chains configured with zero address mismatches.

Six real defects were found and fixed:

Defect	Effect if shipped
logIndex read from Firehose's transaction-relative Log.index instead of block_index	Wrong logIndex on 5 entities, wrong <txHash>-<logIndex> ids — and at BASE detail level index is unpopulated, so two v4 logs in one transaction collide on one id and overwrite each other
UniswapDayData row created only from swaps	handleModifyLiquidity also calls updateUniswapDayData, so any day with liquidity activity but no swaps was missing entirely from the table
HookSwap bypassed the stable-stable-hook guard in the interval path	A hook swap naming a pool with a different hook moved that pool's and both tokens' day/hour txCount and OHLC, where the subgraph ignores it
Transaction rows emitted for every matched log	loadTransaction has exactly five call sites upstream; pool creations and hook deploys produced rows the subgraph never had
Token row emitted for the resolved side of an abandoned pool	handleInitialize saves both tokens only after BOTH decimals checks pass, so a pool dropped on one side leaves no token at all
ERC20 metadata: no isNullEthValue guard, whitespace stripped from bytes32, bytes32 fallback taken on an empty (not reverted) string call	A U+0001 control character as a token symbol; "Sai Stablecoin v1.0" renamed to "SaiStablecoinv1.0"; "" replaced by a fabricated value

Two further divergences were closed rather than documented: per-data-source start blocks are now enforced as a log filter (23 of 25 chains have a non-uniform spread — mainnet's EulerSwapFactory is +987,833 blocks, unichain's +18.4M), and the PoolManager/Bundle singletons are no longer emitted before the chain's first pool exists.

Findings deliberately NOT changed, because they are faithful or immaterial: Transaction.gasUsed (documented improvement over upstream's hardcoded zero), the derivedETH depth cap, the cross-block EulerSwapHook uninstall, and intermediate BigDecimal rounding granularity (~34th significant digit).

Verified against the deployed subgraph

Checked against the live Uniswap v4 subgraph on Ethereum mainnet — subgraph DiYPVdygkfjDWhbxGSqAQxwBKmfKnkWQojqeM2rkLb3G, deployment QmZsgJLiLQKpb8hxTmQ5LWyrFVvfWzVaL4WK8dfFBn7EeK.

Schema: this package is a strict superset. Introspecting the deployment returns 17 entity types; this port emits 19. Nothing the deployment serves is missing here. The extras are EulerSwapHook, ArrakisHook (upstream added hook tracking in 2025) and Pool.isExternalLiquidity (added 2026-03-05) — that deployment simply predates them.

Provenance is exact. The local v4-subgraph checkout is upstream Uniswap/v4-subgraph at origin/main (0 ahead, 0 behind, HEAD 0c13ab2), this package's schema.graphql is byte-identical to it, all 8 ABIs are identical, and the mainnet start block matches (21688329, the PoolManager deploy block) so there is no history gap.

Claims read out of the source, now confirmed against production data at block 22,000,000 — every field this port hardcodes to 0 really is 0 upstream: PoolManager.totalValueLocked{USD,ETH}Untracked, Pool.collectedFeesToken0/1/USD, Pool.liquidityProviderCount, Pool.observationIndex, Pool.totalValueLockedUSDUntracked, Token.poolCount, Token.totalValueLockedUSDUntracked, UniswapDayData.volumeUSDUntracked, and Transaction.gasUsed. PoolManager.owner is the zero address. Native derivedETH is exactly 1. Stored decimals top out at exactly 34 significant digits, which is what GRAPH_PRECISION pins.

And it caught a real bug. Production logIndex values are 112, 111, 162, 253 — block-relative. This port originally read Firehose's transaction-relative Log.index, which would have made every Swap, ModifyLiquidity, Transfer, Subscribe and Unsubscribe id disagree with production.

The snapshot is committed at tests/fixtures/mainnet_subgraph_golden.json and asserted by tests/golden_parity.rs. It was captured with a time-travel query pinned to a fixed block, so it is reproducible: scripts/capture-golden.sh (needs a gateway API key, which is never stored) returns the same numbers whenever it runs.

Operational note observed while capturing: that deployment currently fails chain-head queries across every indexer (no attestation: indexing_error) while historical queries succeed and _meta reports hasIndexingErrors: false.

Tests

cargo test — 82 tests, in four files:

File	What it pins
src/** unit tests	config parsing, decimal semantics, swap economics, ERC20 metadata resolution
tests/tickmath.rs	the 20-constant ladder and 216 amount cases, against a fixture from the reference implementation
tests/subgraph_parity.rs	45 assertions lifted from the subgraph's own matchstick suite, re-asserted against this port — exact expected values, e.g. tick 16080 → 4.992414224852787072040281992144524. None are #[ignore]d: every portable upstream assertion holds here.
tests/golden_parity.rs	id schemes and constant fields against the live mainnet deployment

Third pass: the completeness critique

A third audit ran with one instruction — find what the first two MISSED, and do not re-run them. It took five angles the earlier passes had not: entity ROW existence and staleness, Substreams store semantics (ordinal ordering, key-prefix collisions, delta assumptions), reachable panics and overflow, manifest/deploy parity, and an honest map of the untested surface. It confirmed 9 findings, all now fixed:

Finding	Class	Why it mattered
decimals() narrowed to u64 before the range check	data-loss	A word ≥ 2^64 aliased to 0, admitting a pool the subgraph rejects AND storing the token at wei scale permanently — store_tokens is write-once
.spkg default network is what graph-node uses	data-loss	Deploying the mainnet package for another chain matches no logs and indexes nothing, silently. Now packed per chain, filename included
store_pool_fee had no stable-stable-hook guard	wrong-value	A HookSwap naming an unrelated pool overwrote that pool's feeTier — the earlier HookSwap fix reached map_intervals but not here
store_eth_price / store_derived_eth wrote end-of-block values at per-event ordinals	wrong-value	Silently collapsed the per-ordinal replay everything downstream depends on, so the 2nd..Nth swap in a block was valued at prices that had not happened yet. Fixed with get_at
Walker memo keyed by token, not (token, depth)	divergent	A depth-truncated zero could be served to a shallower walk, making derivedETH depend on which pool was walked first
Token OHLC read pre-event, close read post-event	divergent	Token candles shifted one event back, and close could exceed high — a value the subgraph cannot emit
Bundle gated on poolCount > 0	divergent	The subgraph saves the Bundle before the decimals bail-out and the PoolManager after, so they do not appear at the same moment
symbol/name truncated to 64/128 chars	cosmetic	The subgraph stores them verbatim; spam tokens routinely exceed both

It also returned clean on the things most likely to bite: no key-prefix collisions across the flat keyspace (every near-miss — tvol:/tvolusd:, poolday/poolhour, pos:/posowner: — separates on the delimiter byte), the single delete_prefix is safe because its key components are fixed-width addresses, every deltas consumer is fed by a producer that writes in sorted ordinal order, and no staleness class exists — every store that moves a Pool or Token field is driven by an event that puts that pool in touched_pools.

Running the sink in the cloud

The Postgres path is not a deployment in The Graph's sense — nothing is uploaded. It is a long-lived process that streams from a remote Substreams endpoint into your database, so it runs wherever you can run a container.

Files: Dockerfile (self-contained: compiles the wasm and packs inside the image), Dockerfile.slim (ships a pre-built .spkg — use this on a platform with a slow or memory-capped builder), docker-compose.yml (local parity), railway.toml, scripts/entrypoint.sh, scripts/sink-health.sh.

Railway

1. Pack for the chain on a machine with the toolchain, and un-ignore it so the builder can see it:

   ./scripts/pack.sh mainnet
   git add -f tickwise-uniswap-v4-full-mainnet.spkg
   

2. New Railway project → add a Postgres service → copy its connection string.
3. Add a service from this repo. In its settings set Root Directory to v4-substreams (this package is a subdirectory; without it Railway builds the repo root).
4. Variables:

   name	value
   SUBSTREAMS_API_KEY	from thegraph.market — must start server/web/worker/mobile/hosted
   SUBSTREAMS_SINK_DSN	psql://user:pass@host:5432/railway?sslmode=require (note psql://, not postgres://)
   SPKG (build arg)	tickwise-uniswap-v4-full-mainnet.spkg

5. Add a Volume mounted at /app/spool.
6. Deploy. Keep replicas at 1.

Four things that will bite otherwise

One writer, always. Replicas share the sink's single cursor row and will fight over it. numReplicas = 1 is not a starting point to scale from.

Mount the spool volume. Segments land on disk before the database load and survive a restart. On an ephemeral filesystem a restart re-streams — and re-pays for — every block that was spooled but not yet committed.

Railway's healthcheck cannot see this service. It probes HTTP; this is a stream consumer with no HTTP surface. The failure that matters is a container that stays up while the cursor stops advancing, which no HTTP probe would catch anyway. Point an external monitor at scripts/sink-health.sh, which reads cursor progress from the database and exits non-zero when it stalls.

The image is chain-specific. A .spkg has one default network and that is what map_events filters against; a mainnet image pointed at Base matches nothing and indexes silently. That is why the network is a build arg and the filename carries it.

Cost shape

Two meters run at once: the container (cheap, one small process) and the Substreams provider, billed per processed block. Backfilling mainnet from 21,688,329 is ~4.2M blocks and dominates the bill; following head afterwards is minor. Test with a bounded run before committing to a full backfill.

Not done

• No end-to-end parity run — but the harness for it now exists. make parity (or node scripts/parity-check.mjs --block N) streams graph_out, folds its EntityChanges into entity state, and diffs that state against the live subgraph at the same block, across Bundle / PoolManager / Pool / Token / Tick. It has never been executed, because it needs a credential this repo does not have:

  Variable	What it is	Note
  SUBSTREAMS_API_KEY	StreamingFast / thegraph.market, from substreams auth	must start with server/web/worker/mobile/hosted
  GRAPH_API_KEY	Subgraph Studio / gateway key	used only to query the subgraph

  These are different credentials and it is easy to assume otherwise: a gateway key fed to StreamingFast is rejected outright with "not a valid API key". Verified the hard way.

  The static half is now done — the schema is a verified superset, the subgraph's own test assertions all hold here, and a golden fixture from production is committed. What is still missing is the dynamic half: actually running it. That is the only thing that can confirm the AGGREGATES (volume, TVL, fees, day/hour rows), and no amount of code reading substitutes for it.

• Never streamed against live data. The DAG validates, the package packs, the guards pass, 87 tests pass, and the schema is confirmed against production — but no block has been indexed by this package. make run needs substreams auth first.

• The Postgres sink has never been run either. schema.sql is applied and verified against a real Postgres 18 (psql -f schema.sql, then dropped), and db_out emits the same rows as graph_out by construction — but no DatabaseChanges message has ever reached a sink, so the value-level conversions (empty-string-to-NULL handling, NUMERIC parsing of BigDecimal's string form) are unexercised. make sink-test is the smallest run that would exercise them; it needs substreams auth.

• Collected and uncollected fees — see above; needs EXTENDED Firehose.

• arc-mainnet is not a known Firehose network, which substreams pack warns about. The config is generated and correct; the chain simply has no public Firehose endpoint, so that network entry cannot be streamed today.