substreams_preflight_demo_xh

XuechunHHH

v0.1.0/21 downloads/Repository

Package ref

substreams-preflight-demo-xh@v0.1.0

Download .spkg

Run package

CLI

Run graph_out from the command line.

substreams run substreams-preflight-demo-xh@v0.1.0 graph_out -e mainnet --start-block -1

Authenticate by running substreams auth or directly on thegraph.market (see docs).

README

substreams_preflight_demo_xh

Ethereum mainnet USDC transfers as typed records and canonical The Graph EntityChanges.

Overview

This package decodes ERC-20 Transfer events emitted by the USDC contract from block 18,000,000 onward. It emits raw token amounts as decimal strings and performs no RPC enrichment.

Modules

Module	Kind	Output	Description
`map_usdc_transfers`	map	`demo.usdc.v1.UsdcTransfers`	Decodes typed USDC transfer records.
`graph_out`	map	`sf.substreams.sink.entity.v1.EntityChanges`	Creates `UsdcTransfer` entities with ID `{tx_hash}-{log_index}`.

Prerequisites

Substreams CLI
Rust with the wasm32-unknown-unknown target
buf and protoc
The Graph Market authentication for live runs

Quick Start

From the repository root:

npm run preflight -- \
  --intent demo/usdc-transfers-graph-out/substreams.intent.json \
  --project demo/usdc-transfers-graph-out

cd demo/usdc-transfers-graph-out
substreams build
substreams auth
. ./.substreams.env
substreams run ./substreams.yaml graph_out \
  --start-block 18000000 \
  --stop-block 18000101 \
  --output jsonl

The stop block is exclusive, so this command processes blocks 18,000,000 through 18,000,100.

Modules

Execution graph

2 modules

mapMain

graph_out

Emits canonical UsdcTransfer EntityChanges keyed by transaction hash and log index.

from #18000000

Output

sf.substreams.sink.entity.v1.EntityChanges

Inputs

map_usdc_transfers

map

map_usdc_transfers

Decodes USDC ERC-20 Transfer events into typed records without RPC calls.

from #18000000

Output

demo.usdc.v1.UsdcTransfers

Inputs

sf.ethereum.type.v2.Block

Protobuf

demo.usdc.v1

UsdcTransfersmain

demo.usdc.v1.UsdcTransfers

UsdcTransfer[]transfers = 1

UsdcTransfer

demo.usdc.v1.UsdcTransfer

stringfrom = 1

stringto = 2

stringamount = 3

stringtx_hash = 4

uint64log_index = 5

uint64block_number = 6

Show 27 dependencies

schema

ClickhouseIndexField

schema.ClickhouseIndexField

ClickhouseIndexField defines a secondary index for improved query performance. Secondary indexes help ClickHouse skip irrelevant data blocks during query execution.

stringname = 1

The name of the index (must be unique within the table)

stringfield_name = 2

The field name on which to create the index

IndexTypetype = 3

The type of index to create (e.g., minmax, bloom_filter, set)

uint32granularity = 4

The granularity parameter that controls index density (higher = less memory, lower precision)

Functionfunction = 5

Optional function to apply to the field value before indexing

ClickhouseOrderByField

schema.ClickhouseOrderByField

ClickhouseOrderByField defines a field used in the ClickHouse table ORDER BY clause. ORDER BY determines the physical storage order of data for optimal compression and query performance.

stringname = 1

The name of the field to include in the ORDER BY clause

booldescending = 2

Whether to sort this field in descending order (default is ascending)

Functionfunction = 3

Optional function to apply to the field value before ordering (e.g., toDate for timestamps)

ClickhousePartitionByField

schema.ClickhousePartitionByField

ClickhousePartitionByField defines a field used for ClickHouse table partitioning. Partitioning splits data into separate parts based on the specified field values.

stringname = 1

The name of the field to use for partitioning

Functionfunction = 2

Optional function to apply to the field value before partitioning (e.g., toYYYYMM for dates)

ClickhouseTableOptions

schema.ClickhouseTableOptions

ClickhouseTableOptions defines ClickHouse-specific table configuration options. These options are used to configure ClickHouse table engines and optimize query performance.

ClickhouseOrderByField[]order_by_fields = 1

Fields to use for the ORDER BY clause in ClickHouse table definition. ORDER BY determines the physical ordering of data on disk for better compression and query performance.

ClickhousePartitionByField[]partition_fields = 2

Fields to use for table partitioning in ClickHouse. Partitioning helps with query performance and data management by splitting data into logical parts.

ClickhouseIndexField[]index_fields = 4

Secondary indexes to create for improved query performance on specific columns. These are data skipping indexes that help ClickHouse skip irrelevant data blocks during queries.

Column

schema.Column

Column defines metadata for mapping a protobuf field to a database column. This message is used with the (field) extension on protobuf fields.

stringname = 1

Custom column name. If not specified, the field name is used.

stringforeign_key = 2

Foreign key reference to another table's column in the format "table.column". Used to establish referential integrity constraints in relational databases.

boolunique = 3

Whether this column should have a unique constraint. Ensures no duplicate values exist for this column.

boolprimary_key = 4

Whether this column is part of the primary key. Primary key columns are automatically non-null and unique.

StringConvertionconvertTo = 5

Specifies how to convert string fields to specific numeric types. Only applies to string fields that contain numeric data. Leave unset for no conversion (store as string).

boolinline = 6

Decimal128

schema.Decimal128

Decimal128 represents a 128-bit decimal type for string conversion. Used in StringConvertion to specify that a string field should be converted to Decimal128.

int32scale = 1

The number of digits after the decimal point for this decimal type.

Decimal256

schema.Decimal256

Decimal256 represents a 256-bit decimal type for string conversion. Used in StringConvertion to specify that a string field should be converted to Decimal256.

int32scale = 1

The number of digits after the decimal point for this decimal type.

Int128

schema.Int128

Int128 represents a 128-bit signed integer type for string conversion. Used in StringConvertion to specify that a string field should be converted to Int128.

No fields.

Int256

schema.Int256

Int256 represents a 256-bit signed integer type for string conversion. Used in StringConvertion to specify that a string field should be converted to Int256.

No fields.

StringConvertion

schema.StringConvertion

StringConvertion defines how string fields should be converted to specific numeric types. This is used when protobuf string fields contain numeric data that should be stored as native numeric types in the database for better performance and storage efficiency. Usage: Set this in Column.convertTo field to specify the target numeric type.

Int128int128 = 2oneof convertion

Convert string to 128-bit signed integer

UInt128uint128 = 3oneof convertion

Convert string to 128-bit unsigned integer

Int256int256 = 4oneof convertion

Convert string to 256-bit signed integer

UInt256uint256 = 6oneof convertion

Convert string to 256-bit unsigned integer

Decimal128decimal128 = 50oneof convertion

Convert string to 128-bit decimal with specified scale

Decimal256decimal256 = 51oneof convertion

Convert string to 256-bit decimal with specified scale

Table

schema.Table

Table defines metadata for mapping a protobuf message to a database table. This message is used with the (table) extension on protobuf messages.

stringname = 1

The name of the database table. If not specified, the message name is used.

stringchild_of = 2

Optional field to specify a parent table for establishing table relationships. Used for creating foreign key relationships in relational databases.

ClickhouseTableOptionsclickhouse_table_options = 200

ClickHouse-specific table options for configuring table engine parameters. Only applicable when using ClickHouse as the target database.

UInt128

schema.UInt128

UInt128 represents a 128-bit unsigned integer type for string conversion. Used in StringConvertion to specify that a string field should be converted to UInt128.

No fields.

UInt256

schema.UInt256

UInt256 represents a 256-bit unsigned integer type for string conversion. Used in StringConvertion to specify that a string field should be converted to UInt256.

No fields.

Functionenum

schema.Function

unset = 0

No function applied (use raw field value)

toYYYYMM = 1

Convert date/timestamp to YYYYMM format (useful for monthly partitioning)

toYYYYDD = 2

Convert date/timestamp to YYYYDD format (useful for daily partitioning)

toYear = 3

Extract year from date/timestamp (useful for yearly partitioning)

toMonth = 4

Extract month from date/timestamp (useful for monthly grouping)

toDate = 5

Convert timestamp to date (removes time component)

toStartOfMonth = 6

Convert date to first day of the month (useful for monthly aggregations)

IndexTypeenum

schema.IndexType

minmax = 0

MinMax index stores minimum and maximum values for each granule (fastest, least memory)

set = 1

Set index stores unique values for each granule (good for low cardinality fields)

ngrambf_v1 = 2

N-gram Bloom filter index for substring searches (good for text search)

tokenbf_v1 = 3

Token Bloom filter index for full-word searches (good for tokenized text)

bloom_filter = 4

Bloom filter index for equality checks (good for high cardinality fields)

sf.firehose.v2

BlockMetadata

sf.firehose.v2.BlockMetadata

uint64num = 1

Num is the block number of this response's block.

stringid = 2

ID is the block ID of this response's block. The ID actual representation is chain specific.

Antelope & Ethereum uses hex.
NEAR & Solana uses base58.

Refer to the chain documentation for more details.

uint64parent_num = 3

ParentNum is the block number of the parent of this response's block

stringparent_id = 4

ParentID is the block ID of the parent of this response's block. If this response is the genesis block, this field is empty.

The ID actual representation is chain specific.

Antelope & Ethereum uses hex.
NEAR & Solana uses base58.

Refer to the chain documentation for more details.

uint64lib_num = 5

LibNum is the block number of the last irreversible block (a.k.a last finalized block) at the time of this response' block. It determines the finality of the block.

Timestamptime = 6

Time is the time at which the block was produced.

InfoRequest

sf.firehose.v2.InfoRequest

No fields.

InfoResponse

sf.firehose.v2.InfoResponse

stringchain_name = 1

Canonical chain name from https://thegraph.com/docs/en/developing/supported-networks/ (ex: matic, mainnet ...)

string[]chain_name_aliases = 2

Alternate names for the chain.

uint64first_streamable_block_num = 3

First block that is served by this endpoint. This should usually be the genesis block, but some providers may have truncated history.

stringfirst_streamable_block_id = 4

InfoResponse.BlockIdEncodingblock_id_encoding = 5

This informs the client on how to decode the block_id field inside the "Clock" message as well as the first_streamable_block_id above.

string[]block_features = 10

features describes the blocks. Popular values for EVM chains include base, extended or hybrid.

Request

sf.firehose.v2.Request

int64start_block_num = 1

Controls where the stream of blocks will start.

The stream will start inclusively at the requested block num.

When not provided, starts at first streamable block of the chain. Not all chain starts at the same block number, so you might get an higher block than requested when using default value of 0.

Can be negative, will be resolved relative to the chain head block, assuming a chain at head block #100, then using -50 as the value will start at block #50. If it resolves before first streamable block of chain, we assume start of chain.

If start_cursor is given, this value is ignored and the stream instead starts immediately after the Block pointed by the opaque start_cursor value.

stringcursor = 2

Controls where the stream of blocks will start which will be immediately after the Block pointed by this opaque cursor.

Obtain this value from a previously received Response.cursor.

This value takes precedence over start_block_num.

uint64stop_block_num = 3

When non-zero, controls where the stream of blocks will stop.

The stream will close after that block has passed so the boundary is inclusive.

boolfinal_blocks_only = 4

With final_block_only, you only receive blocks with STEP_FINAL Default behavior will send blocks as STEP_NEW, with occasional STEP_UNDO

Any[]transforms = 10

Response

sf.firehose.v2.Response

Anyblock = 1

Chain specific block payload, ex:

sf.eosio.type.v1.Block
sf.ethereum.type.v1.Block
sf.near.type.v1.Block

ForkStepstep = 6

stringcursor = 10

BlockMetadatametadata = 12

Metadata about the block, added in some Firehose version, so consumer should be ready to handle the absence of this field.

SingleBlockRequest

sf.firehose.v2.SingleBlockRequest

SingleBlockRequest.BlockNumberblock_number = 3oneof reference

SingleBlockRequest.BlockHashAndNumberblock_hash_and_number = 4oneof reference

SingleBlockRequest.Cursorcursor = 5oneof reference

Any[]transforms = 6

BlockHashAndNumber

sf.firehose.v2.SingleBlockRequest.BlockHashAndNumber

Get the current block with specific hash and number

uint64num = 1

stringhash = 2

BlockNumber

sf.firehose.v2.SingleBlockRequest.BlockNumber

Get the current known canonical version of a block at with this number

uint64num = 1

Cursor

sf.firehose.v2.SingleBlockRequest.Cursor

Get the block that generated a specific cursor

stringcursor = 1

SingleBlockResponse

sf.firehose.v2.SingleBlockResponse

Anyblock = 1

BlockMetadatametadata = 2

Metadata about the block, added in some Firehose version, so consumer should be ready to handle the absence of this field.

ForkStepenum

sf.firehose.v2.ForkStep

STEP_UNSET = 0

STEP_NEW = 1

Incoming block

STEP_UNDO = 2

A reorg caused this specific block to be excluded from the chain

STEP_FINAL = 3

Block is now final and can be committed (finality is chain specific, see chain documentation for more details)

BlockIdEncodingenum

sf.firehose.v2.InfoResponse.BlockIdEncoding

BLOCK_ID_ENCODING_UNSET = 0

BLOCK_ID_ENCODING_HEX = 1

BLOCK_ID_ENCODING_0X_HEX = 2

BLOCK_ID_ENCODING_BASE58 = 3

BLOCK_ID_ENCODING_BASE64 = 4

BLOCK_ID_ENCODING_BASE64URL = 5