Indexing
How Do I Fix Indexer Timeouts?
BlockScout utilizes two separate indexers in order to index the network history as well as keep up with new incoming blocks. Due to this process, a node may become overloaded and not respond to BlockScout’s RPC requests within the designated timeout period.
The indexer umbrella application can be tweaked to meet your node’s size and responsiveness. Outlined below are the locations and adjustments that can be made to each fetcher.
Fetchers are available at https://github.com/blockscout/blockscout/tree/master/apps/indexer/lib/indexer/fetcher
| Fetcher | Description | Batch size | Concurrency |
|---|---|---|---|
| Catchup Block Fetcher | This fetcher indexes blocks, transactions and receipts starting from the tip of the chain working backward to the Genesis block. | 10 INDEXER_CATCHUP_BLOCKS_BATCH_SIZE | 10 INDEXER_CATCHUP_BLOCKS_CONCURRENCY |
| Transaction Receipts Fetcher | Transaction receipts fetcher. | 250 INDEXER_RECEIPTS_BATCH_SIZE | 10 INDEXER_RECEIPTS_CONCURRENCY |
| Internal Transaction Fetcher | This fetcher indexes internal transactions. For the real-time fetcher this process is done synchronously and for the catchup fetcher, this process is done asynchronously. | 10 INDEXER_INTERNAL_TRANSACTIONS_BATCH_SIZE | 4 INDEXER_INTERNAL_TRANSACTIONS_CONCURRENCY |
| Coin Balance Fetcher | This fetcher indexes each coin balance at the block height the address was interacted with. | 100 INDEXER_COIN_BALANCES_BATCH_SIZE | 4 INDEXER_COIN_BALANCES_CONCURRENCY |
| Uncle Block Fetcher | This fetcher indexes non-consensus blocks, transactions, and receipts. | 10 | 10 |
| Token Balance Fetcher | This fetcher indexes token balances. Note you may experience some token balances that cannot be fetched due to a malformed smart contract or other functions that do not allow a balance to be fetched. | 100 INDEXER_TOKEN_BALANCES_BATCH_SIZE | 10 INDEXER_TOKEN_BALANCES_CONCURRENCY |
| Token Fetcher | This indexer fetches the metadata for a token contract | 1 | 10 |
| Block Reward Fetcher | Block reward fetcher. | 10 INDEXER_BLOCK_REWARD_BATCH_SIZE | 10 INDEXER_BLOCK_REWARD_CONCURRENCY |
| Token Instance Realtime Fetcher | Token instance realtime fetcher. | 1 INDEXER_TOKEN_INSTANCE_REALTIME_BATCH_SIZE | 10 INDEXER_TOKEN_INSTANCE_REALTIME_CONCURRENCY |
| Token Instance Retry Fetcher | Token instance retry fetcher. | 10 INDEXER_TOKEN_INSTANCE_RETRY_CONCURRENCY | 10 INDEXER_TOKEN_INSTANCE_RETRY_BATCH_SIZE |
| Token Instance Sanitize Fetcher | Token instance sanitize fetcher. | 10 INDEXER_TOKEN_INSTANCE_SANITIZE_BATCH_SIZE | 10 INDEXER_TOKEN_INSTANCE_SANITIZE_CONCURRENCY |
Understanding Errors and Timeouts
application=indexer fetcher=coin_balance count=500 error_count=500 [error] failed to fetch: :timeout
In the error provided above, we can tell that the indexer application failed to fetch 500 coin balances due to a timeout error. Usually, when an indexer starts to receive timeouts, many more timeouts from other fetchers will occur as well.
Resolving Timeout Issues
The best action to take when fetchers start to receive timeouts is to lower the batch size and concurrency of a few fetchers which will put less pressure on the node. The two indexers that cause the most strain to the node are the block fetcher and the internal transactions fetcher. It would be advised to cut these fetchers values in half and restart the application. A restart of the node may also be required.Other Actions
BlockScout comes equipped with functionality to automatically stop sending requests forn seconds if timeouts are detected. You may adjust these settings by setting a different value for wait_per_timeout.
https://github.com/poanetwork/blockscout/blob/1475e3bfd002d9397efd0f0cc29c20f39a70d023/apps/ethereum_jsonrpc/config/config.exs#L3-L9
Content moved from https://forum.poa.network/t/faq-how-do-i-fix-indexer-timeouts/1829