diff --git a/docs/develop/blockchain/sharding-lifecycle.mdx b/docs/develop/blockchain/sharding-lifecycle.mdx index edde6b18a7..b91afc5e1f 100644 --- a/docs/develop/blockchain/sharding-lifecycle.mdx +++ b/docs/develop/blockchain/sharding-lifecycle.mdx @@ -14,6 +14,75 @@ ISP underpins the TON Blockchain's design, treating each account as part of its Each shardchain, or more precisely, each shardchain block, is identified by a combination of `workchain_id` and a binary prefix `s` of the account_id. +## Algorithm for deciding whether to split or merge + +Validators decide whether to split or merge shards in the following way: +1. For each block, block size, gas consumption and lt delta are calculated. +2. Using these values, blocks can be considered overloaded or underloaded. +3. Each shard keeps underload and overload history. If enough recent blocks were underloaded or overloaded, `want_merge` or `want_split` flag is set. +4. Validators merge or split shards using these flags. + +### 1. Assessment of the current state of the block + +Each block has the following parameters. They are used to determine overload and underload. +1. *Block size estimation* - not an actual block size, but an estimation calculated during collation. +2. *Gas consumption* - total gas consumed in all transactions (excluding ticktock and mint/recover special transactions). +3. *Lt delta* - difference between start and end lt of the block. + +### 2. Block limits and classification + +Block limits are loaded from the [configuration parameters 22 and 23](/develop/howto/blockchain-configs#param-22-and-23). +Each of the three parameters has three limits: underload, soft, hard: +1. *Block size*: `128/256/512 KiB`. +2. *Gas consumption*: `2M/10M/20M` in basechain, `200K/1M/2.5M` in masterchain. +3. *Lt delta*: `1000/5000/10000`. +Also, there is a medium limit, which is equal to `(soft + hard) / 2`. + +We classify the three parameters (size, gas, and lt delta) into categories: +- `0` - underload limit is not reached. +- `1` - underload limit is exceeded. +- `2` - soft limit is exceeded. +- `3` - medium limit is exceeded. +- `4` - hard limit is exceeded. + +Block classification is max(`Classification of size`, `Classification of gas`, `Classification of lt delta`). For example: if classification of size is 2, classification of gas is 3, classification of lt delta is 1, then the final block classification is 3. + +- When classification of the block is 0 (underload), the block is inclined to merge with its sibling. +- When classification of the block is 2 (soft limit reached), collator stops processing internal messages. The block is inclined to split. +- When classification of the block is 3 (medium limit reached), collator stops processing external messages. + +### 3. Determination of overload or underload + +After classifying the block, collator checks overload and underload conditions. +Size of the outbound message queue and status of dispatch queue processing is also taken into consideration. +- If the block class is ≥ `2` (soft) and message queue size ≤ `SPLIT_MAX_QUEUE_SIZE = 100000` then the block is overloaded. +- If limit for total processed messages from dispatch queue was reached and message queue size ≤ `SPLIT_MAX_QUEUE_SIZE = 100000` then the block is overloaded. +- If the block class is `0` (underload) and message queue size ≤ `MERGE_MAX_QUEUE_SIZE = 2047` then the block is underloaded. +- If message queue size is ≥ `FORCE_SPLIT_QUEUE_SIZE = 4096` and ≤ `SPLIT_MAX_QUEUE_SIZE = 100000` then the block is overloaded. + +### 4. Deciding whether to split or merge + +Each block keeps underload and overload history - it is a 64-bit mask of the underload/overload status of the last 64 blocks. +It is used to decide whether to split or merge. + +Underload and overload history has a weight, which is calculated as follows: +`one_bits(mask & 0xffff) * 3 + one_bits(mask & 0xffff0000) * 2 + one_bits(mask & 0xffff00000000) - (3 + 2 + 1) * 16 * 2 / 3` +(here `one_bits` is the number of `1`-bits in a mask, and the lower bits correspond to the most recent blocks). + +When underload or overload history has a non-negative weight, the flag `want_merge` or `want_split` is set. + +### 5. Final decision + +Validators decide to split or merge shards using `want_split` and `want_merge` flags and [workchain configuration parameters](/develop/howto/blockchain-configs#param-12). + +- If the shard has depth < `min_split` then it will split. +- If the shard has depth > `max_split` then it will merge. +- Shards with depth `min_split` cannot merge, shards with depth `max_split` cannot split. +- If the block has `want_split` flag, the shard will split. +- If the block and its sibling have `want_merge` flag, the shards will merge. + +Shards split and merge in `split_merge_delay = 100` seconds after the decision is made. + ## Messages and Instant Hypercube Routing (Instant Hypercube Routing) In the infinite sharding paradigm, each account (or smart-contract) is treated as if it were itself in a separate shardchain.