From d7da55012c75a8ac297abdc4d360a061d194dd5b Mon Sep 17 00:00:00 2001 From: aigerimu <89766357+aigerimu@users.noreply.github.com> Date: Tue, 9 Dec 2025 19:00:10 +0700 Subject: [PATCH 1/4] auto-serialization --- .../tolk/features/auto-serialization.mdx | 262 ++++++++---------- 1 file changed, 120 insertions(+), 142 deletions(-) diff --git a/languages/tolk/features/auto-serialization.mdx b/languages/tolk/features/auto-serialization.mdx index 88a7f1018..af1992cce 100644 --- a/languages/tolk/features/auto-serialization.mdx +++ b/languages/tolk/features/auto-serialization.mdx @@ -4,9 +4,7 @@ title: "Automatic serialization" import { Aside } from '/snippets/aside.jsx'; -All data in TON (messages, storage, etc.) is represented with **cells**. -Tolk type system is designed to express cell contents, -enabling auto-serialization via `fromCell` and `toCell`: +All data in TON, such as messages, storage, etc, is represented as [cells](/foundations/serialization/cells). The Tolk type system is designed to express cell contents, enabling auto-serialization through `fromCell` and `toCell`: ```tolk struct Point { @@ -24,12 +22,9 @@ fun demo() { } ``` -## How each type is serialized +## Type serialization -To dig into binary data, follow [Overall: serialization](/languages/tolk/types/overall-serialization). - -A `struct` can provide its "serialization prefix". -32-bit ones are typically called **opcodes** and used for messages (incoming and outgoing): +A `struct` can define [a serialization](/languages/tolk/types/overall-serialization) prefix. 32-bit prefixes are commonly used as opcodes for incoming and outgoing messages: ```tolk struct (0x7362d09c) TransferNotification { @@ -37,22 +32,16 @@ struct (0x7362d09c) TransferNotification { // ... } ``` +A prefix is not required to be 32 bits: `0x000F` is a 16-bit prefix, and `0b010` is a 3-bit one. -But prefixes are not restricted to be 32-bit: `0x000F` is a 16-bit prefix, `0b010` is 3-bit (binary). - -[Serialization of structures](/languages/tolk/types/overall-serialization#structures) is also on the "overall" page. - -## Controlling cell references. Typed cells +## Cell references and its types -Fields of a struct are serialized one by one. -The compiler does not reorder fields, create implicit references, etc. -When data should be stored in a ref, it's done explicitly. -A developer controls exactly when each ref is loaded. +Fields of a struct are serialized one by one. The compiler does not reorder fields or insert implicit references. When data should be stored in a ref, it's done explicitly. A developer controls exactly when each ref is loaded. -**There are two types of references — typed and untyped**: +There are two types of references – typed and untyped: -- `cell` — untyped ref — just "some cell", "arbitrary content" -- `Cell` — typed ref — a cell, which internal structure is known +- `Cell – typed reference; the cell has a known internal structure; +- `cell` – untyped reference; the cell content is not described. ```tolk struct NftStorage { @@ -70,13 +59,15 @@ struct RoyaltyParams { A call `NftCollectionStorage.fromCell()` is processed as follows: -1. read `address` -1. read `uint64` -1. read two refs without unpacking them: only their pointers are loaded +1. read `address`; +1. read `uint64`; +1. read two refs without unpacking them – only their pointers are loaded. -## `Cell` must be loaded to get `T` +### `Cell` -Let's look at the `royalty` above: +`Cell` must be loaded to get `T`. + +Consider the `royalty` field: ```tolk struct NftStorage { @@ -85,7 +76,7 @@ struct NftStorage { } ``` -Since it is a cell, `storage.royalty.xxx` is NOT valid: +Since it's a cell, `storage.royalty.numerator` is not valid: ```ansi // error: `Cell` has no field `numerator` @@ -93,18 +84,18 @@ storage.royalty.numerator; ^^^^^^^^^ ``` -To access `numerator` and other fields, manually **load that ref**: +To access `numerator` and other fields, load the reference: ```tolk val royalty = storage.royalty.load(); // Cell to T // or, alternatively val royalty = RoyaltyParams.fromCell(storage.royalty); -// ok +// works royalty.numerator; ``` -And conversely, when composing an instance, **assign a cell, not an object**: +When composing an instance, assign a cell, not an object: ```tolk val storage: NftStorage = { @@ -115,18 +106,18 @@ val storage: NftStorage = { } ``` -The following snippet summarizes the behavior: +Summary: ```tolk pCell = point.toCell(); // `Point` to `Cell` point = pCell.load(); // `Cell` to `Point` ``` -Note that `Cell
` or even `Cell` is also okay, `T` is not restricted to structures. +`Cell
` or `Cell` is supported, `T` is not restricted to structures. ## Custom serializers for custom types -Using type aliases, it is possible to override serialization behavior when it cannot be expressed using existing types: +Type aliases allow overriding serialization behavior when the required encoding cannot be represented using existing types: ```tolk type MyString = slice @@ -140,17 +131,11 @@ fun MyString.unpackFromSlice(mutate s: slice) { } ``` - - -See [Serialization of type aliases](/languages/tolk/types/overall-serialization#type-aliases) for examples. +This mechanism applies only to [type aliases](/languages/tolk/types/overall-serialization#type-aliases). It does not work for structures, enums, etc. -## What if input is corrupted +## Behavior with corrupted input -How will `Point.fromCell(c)` work if `c` is less than 16 bits? +`Point.fromCell(c)` throws an exception if the cell does not contain the required data. The function expects at least 16 bits. ```tolk struct Point { @@ -163,18 +148,19 @@ fun demo() { } ``` -The answer: **an exception is thrown**. In multiple cases, actually: - -- input is too small — not enough bits or refs, unless `lazy fromCell` -- input is too big — contains extra data (can be turned off) -- `address` has incorrect format -- `enum` has an invalid value -- a struct prefix does not match -- etc. +Typical failure cases include: +- not enough bits or refs, unless `lazy fromCell` is used; +- extra data after the expected fields; can be enabled; +- `address` has an invalid format; +- `enum` has an invalid value; +- a mismatched struct prefix; +- other format inconsistencies. -An exception code is typically 9 ("cell underflow") or 5 ("out of range"). +Common exception codes: +- [9 – cell underflow](/tvm/exit-codes#9%3A-cell-underflow); +- [5 – out of range](/tvm/exit-codes#5:-integer-out-of-expected-range). -Some aspects of this behavior can be controlled. For example, if "input is too big" is okay, use an option: +Some behaviors are configurable. For example, to allow extra data: ```tolk MyMsg.fromSlice(s, { @@ -182,9 +168,9 @@ MyMsg.fromSlice(s, { }) ``` -## UnpackOptions and PackOptions +## Cell packing and unpacking options -Behavior of `fromCell` and `toCell` can be controlled by options: +Behavior of `fromCell` and `toCell` can be controlled by an option object: ```tolk MyMsg.fromCell(c, { @@ -192,7 +178,7 @@ MyMsg.fromCell(c, { }) ``` -For deserialization (`fromCell` and similar), there are two options: +For deserialization, there are two options: ```tolk MyMsg.fromCell(c, { @@ -207,7 +193,7 @@ MyMsg.fromCell(c, { }) ``` -For serialization (`toCell` and similar), there is one option: +For serialization, there is one option: ```tolk obj.toCell({ @@ -219,59 +205,59 @@ obj.toCell({ }); ``` -## Not only `fromCell`, but `fromSlice` and more +### Functions -This API is also designed to integrate with low-level features. -Each of these functions can be controlled by `UnpackOptions`. +Functions such as `fromCell()`, `fromSlice()`, etc., are designed to integrate with low-level features. -1. `T.fromCell(c)` — parse a cell: "c.beginParse() + fromSlice": +For deserialization, each of the functions can be controlled by `UnpackOptions`: -```tolk -var storage = NftStorage.fromCell(contract.getData()); -``` +1. `T.fromCell(c)` parses a cell using `c.beginParse() + fromSlice`: -2. `T.fromSlice(s)` — parse a slice (a slice is **not mutated**): + ```tolk + var storage = NftStorage.fromCell(contract.getData()); + ``` -```tolk -var msg = CounterIncrement.fromSlice(s); -``` +1. `T.fromSlice(s)` parses a slice; the slice is not mutated: -3. `slice.loadAny()` — **mutate** the slice: + ```tolk + var msg = CounterIncrement.fromSlice(s); + ``` -```tolk -var storage = s.loadAny(); -var nextNum = s.loadAny(); // also ok -``` +1. `slice.loadAny()` mutates the slice: -Note: `options.assertEndAfterReading` is ignored by this function because it is intended to read data from the middle. + ```tolk + var storage = s.loadAny(); + var nextNum = s.loadAny(); // also ok + ``` -4. `slice.skipAny()` — like `skipBits()` and similar: + `options.assertEndAfterReading` is ignored because the function reads from the middle of the slice. -```tolk -s.skipAny(); // skips 16 bits -``` +1. `slice.skipAny()`, such as `skipBits()` and similar: -**Same for serialization.** -Each of these functions can be controlled by `PackOptions`. + ```tolk + s.skipAny(); // skips 16 bits + ``` -1. `T.toCell()` — works as "beginCell() + serialize + endCell()": +For serialization, each of the functions can be controlled by `PackOptions`. -```tolk -contract.setData(storage.toCell()); -``` +1. `T.toCell()` works as `beginCell() + serialize + endCell()`: -2. `builder.storeAny(v)` — like `storeUint()` and similar: + ```tolk + contract.setData(storage.toCell()); + ``` -```tolk -var b = beginCell() - .storeUint(32) - .storeAny(msgBody) // T=MyMsg here - .endCell(); -``` +1. `builder.storeAny(v)`, such as `storeUint()` and similar: -## Special type: RemainingBitsAndRefs + ```tolk + var b = beginCell() + .storeUint(32) + .storeAny(msgBody) // T=MyMsg here + .endCell(); + ``` -It's a built-in type to get "all the rest" slice tail on reading. Example: +## `RemainingBitsAndRefs` + +`RemainingBitsAndRefs` is a built-in type to get remaining part of a slice when reading. Example: ```tolk struct JettonMessage { @@ -280,24 +266,15 @@ struct JettonMessage { } ``` -After `JettonMessage.fromCell`, forwardPayload contains **everything left after reading the fields above**. -Essentially, it's an alias to a slice which is handled specially by the compiler: +After `JettonMessage.fromCell`, `forwardPayload` contains everything left in the slice after reading the preceding fields. It's an alias for a slice that the compiler treats specially: ```tolk type RemainingBitsAndRefs = slice ``` -## What if data exceeds 1023 bits - -Tolk compiler warns if a serializable struct potentially exceeds 1023 bits. -A developer should **take one of the following actions**: - -1. to suppress the error; it means "okay, I understand" -1. or reorganize a struct by splitting into multiple cells +## What if data exceeds 1023 bits? -Why "potentially exceeds"? Because for many types, their size can vary. For example, `int8?` is either one or nine bits, `coins` is 4..124 bits, etc. - -So, given a struct: +The Tolk compiler issues a warning if a serializable struct may exceed 1023 bits. This can happen because many types have variable size. For example, `int8?` can be 1 or 9 bits, `coins` can range from 4 to 124 bits, etc. Consider a struct: ```tolk struct MoneyInfo { @@ -307,51 +284,54 @@ struct MoneyInfo { } ``` -And trying to serialize it, the compiler prints an error: +Serializing this struct produces a compiler warning: ```ansi wrap struct `MoneyInfo` can exceed 1023 bits in serialization (estimated size: 808..1048 bits) ... (and some instructions) ``` -Actually, two choices are available: +Consider one of the following actions: -1. if `coins` values are expected to be relatively small, and this struct will 100% fit in reality; then, suppress the error using an annotation: +1. Suppress the error. -```tolk -@overflow1023_policy("suppress") -struct MoneyInfo { - ... -} -``` + If `coins` values are expected to be relatively small and the struct will fit in practice, suppress the error using an annotation: -2. or `coins` are expected to be billions of billions, so data really can exceed; in this case, extract some fields into a separate cell; for example, store 800 bits as a ref or extract the other two fields: + ```tolk + @overflow1023_policy("suppress") + struct MoneyInfo { + ... + } + ``` -```tolk -// extract the first field -struct MoneyInfo { - fixed: Cell - wallet1: coins - wallet2: coins -} +1. Reorganize the struct by splitting into multiple cells. -// or extract the other two fields -struct WalletsBalances { - wallet1: coins - wallet2: coins -} -struct MoneyInfo { - fixed: bits800 - balances: Cell -} -``` + If `coins` values are expected to be relatively large and the data may exceed 1023 bits, extract some fields into a separate cell. For example, store 800 bits as a ref or extract the other two fields: -A general guideline: leave frequently used fields directly and place less-frequent fields into a nested ref. -Overall, the compiler reports potential overflow, and it is the developer's responsibility to resolve it. + ```tolk + // extract the first field + struct MoneyInfo { + fixed: Cell + wallet1: coins + wallet2: coins + } -## What if serialization is unavailable + // or extract the other two fields + struct WalletsBalances { + wallet1: coins + wallet2: coins + } + struct MoneyInfo { + fixed: bits800 + balances: Cell + } + ``` + +Frequently used fields should remain in the struct; less-frequent fields can be moved to a nested ref. + +## What if serialization is unavailable? -A common mistake: using `int` (it cannot be serialized; use `int32`, `uint64`, etc.; see [numeric types](/languages/tolk/types/numbers)). +A common mistake is using `int`. [It cannot be serialized](/languages/tolk/types/numbers#stack-layout-and-serialization); instead, use `int32`, `uint64`, etc. ```tolk struct Storage { @@ -364,7 +344,7 @@ fun errDemo() { } ``` -The compiler reports a reasonable error: +The compiler reports: ```ansi auto-serialization via fromSlice() is not available for type `Storage` @@ -375,7 +355,7 @@ hint: replace `int` with `int32` / `uint64` / `coins` / etc. ## Integration with message sending -Auto-serialization is integrated natively with message sending to other contracts: +Auto-serialization is integrated when [messages sending](/languages/tolk/features/message-sending) to other contracts: ```tolk val reply = createMessage({ @@ -387,12 +367,10 @@ val reply = createMessage({ reply.send(SEND_MODE_REGULAR); ``` -See: [sending messages](/languages/tolk/features/message-sending). +## `lazy` for deserialization -## Not "fromCell" but "lazy fromCell" - -Tolk provides a special keyword `lazy` combined with auto-deserialization. -The compiler loads only the fields requested, rather than the entire struct. +Tolk provides a special keyword `lazy` for use with auto-deserialization. +The compiler loads only the requested fields, rather than the entire struct. ```tolk struct Storage { @@ -410,4 +388,4 @@ get fun publicKey() { } ``` -See: [lazy loading](/languages/tolk/features/lazy-loading). + From 02730eaa0121063527204daf32af31ebea461e1f Mon Sep 17 00:00:00 2001 From: aigerimu <89766357+aigerimu@users.noreply.github.com> Date: Tue, 9 Dec 2025 19:02:23 +0700 Subject: [PATCH 2/4] formatting & link --- .../tolk/features/auto-serialization.mdx | 107 +++++++++--------- 1 file changed, 54 insertions(+), 53 deletions(-) diff --git a/languages/tolk/features/auto-serialization.mdx b/languages/tolk/features/auto-serialization.mdx index af1992cce..46a880556 100644 --- a/languages/tolk/features/auto-serialization.mdx +++ b/languages/tolk/features/auto-serialization.mdx @@ -32,6 +32,7 @@ struct (0x7362d09c) TransferNotification { // ... } ``` + A prefix is not required to be 32 bits: `0x000F` is a 16-bit prefix, and `0b010` is a 3-bit one. ## Cell references and its types @@ -149,6 +150,7 @@ fun demo() { ``` Typical failure cases include: + - not enough bits or refs, unless `lazy fromCell` is used; - extra data after the expected fields; can be enabled; - `address` has an invalid format; @@ -157,6 +159,7 @@ Typical failure cases include: - other format inconsistencies. Common exception codes: + - [9 – cell underflow](/tvm/exit-codes#9%3A-cell-underflow); - [5 – out of range](/tvm/exit-codes#5:-integer-out-of-expected-range). @@ -213,47 +216,47 @@ For deserialization, each of the functions can be controlled by `UnpackOptions`: 1. `T.fromCell(c)` parses a cell using `c.beginParse() + fromSlice`: - ```tolk - var storage = NftStorage.fromCell(contract.getData()); - ``` + ```tolk + var storage = NftStorage.fromCell(contract.getData()); + ``` 1. `T.fromSlice(s)` parses a slice; the slice is not mutated: - ```tolk - var msg = CounterIncrement.fromSlice(s); - ``` + ```tolk + var msg = CounterIncrement.fromSlice(s); + ``` 1. `slice.loadAny()` mutates the slice: - ```tolk - var storage = s.loadAny(); - var nextNum = s.loadAny(); // also ok - ``` + ```tolk + var storage = s.loadAny(); + var nextNum = s.loadAny(); // also ok + ``` - `options.assertEndAfterReading` is ignored because the function reads from the middle of the slice. + `options.assertEndAfterReading` is ignored because the function reads from the middle of the slice. 1. `slice.skipAny()`, such as `skipBits()` and similar: - ```tolk - s.skipAny(); // skips 16 bits - ``` + ```tolk + s.skipAny(); // skips 16 bits + ``` For serialization, each of the functions can be controlled by `PackOptions`. 1. `T.toCell()` works as `beginCell() + serialize + endCell()`: - ```tolk - contract.setData(storage.toCell()); - ``` + ```tolk + contract.setData(storage.toCell()); + ``` 1. `builder.storeAny(v)`, such as `storeUint()` and similar: - ```tolk - var b = beginCell() - .storeUint(32) - .storeAny(msgBody) // T=MyMsg here - .endCell(); - ``` + ```tolk + var b = beginCell() + .storeUint(32) + .storeAny(msgBody) // T=MyMsg here + .endCell(); + ``` ## `RemainingBitsAndRefs` @@ -295,37 +298,37 @@ Consider one of the following actions: 1. Suppress the error. - If `coins` values are expected to be relatively small and the struct will fit in practice, suppress the error using an annotation: + If `coins` values are expected to be relatively small and the struct will fit in practice, suppress the error using an annotation: - ```tolk - @overflow1023_policy("suppress") - struct MoneyInfo { - ... - } - ``` + ```tolk + @overflow1023_policy("suppress") + struct MoneyInfo { + ... + } + ``` 1. Reorganize the struct by splitting into multiple cells. - If `coins` values are expected to be relatively large and the data may exceed 1023 bits, extract some fields into a separate cell. For example, store 800 bits as a ref or extract the other two fields: - - ```tolk - // extract the first field - struct MoneyInfo { - fixed: Cell - wallet1: coins - wallet2: coins - } - - // or extract the other two fields - struct WalletsBalances { - wallet1: coins - wallet2: coins - } - struct MoneyInfo { - fixed: bits800 - balances: Cell - } - ``` + If `coins` values are expected to be relatively large and the data may exceed 1023 bits, extract some fields into a separate cell. For example, store 800 bits as a ref or extract the other two fields: + + ```tolk + // extract the first field + struct MoneyInfo { + fixed: Cell + wallet1: coins + wallet2: coins + } + + // or extract the other two fields + struct WalletsBalances { + wallet1: coins + wallet2: coins + } + struct MoneyInfo { + fixed: bits800 + balances: Cell + } + ``` Frequently used fields should remain in the struct; less-frequent fields can be moved to a nested ref. @@ -369,7 +372,7 @@ reply.send(SEND_MODE_REGULAR); ## `lazy` for deserialization -Tolk provides a special keyword `lazy` for use with auto-deserialization. +Tolk provides a special keyword [`lazy`](/languages/tolk/features/lazy-loading) for use with auto-deserialization. The compiler loads only the requested fields, rather than the entire struct. ```tolk @@ -387,5 +390,3 @@ get fun publicKey() { return st.publicKey } ``` - - From 2e25aefe8ad788ead918a5e085ce9efcdddf3a1a Mon Sep 17 00:00:00 2001 From: aigerimu <89766357+aigerimu@users.noreply.github.com> Date: Tue, 9 Dec 2025 19:08:16 +0700 Subject: [PATCH 3/4] aside --- languages/tolk/features/auto-serialization.mdx | 2 -- 1 file changed, 2 deletions(-) diff --git a/languages/tolk/features/auto-serialization.mdx b/languages/tolk/features/auto-serialization.mdx index 46a880556..6db82af7e 100644 --- a/languages/tolk/features/auto-serialization.mdx +++ b/languages/tolk/features/auto-serialization.mdx @@ -2,8 +2,6 @@ title: "Automatic serialization" --- -import { Aside } from '/snippets/aside.jsx'; - All data in TON, such as messages, storage, etc, is represented as [cells](/foundations/serialization/cells). The Tolk type system is designed to express cell contents, enabling auto-serialization through `fromCell` and `toCell`: ```tolk From 62322c287acc5cc0d2c1149e1a5be97860a1974b Mon Sep 17 00:00:00 2001 From: aigerimu <89766357+aigerimu@users.noreply.github.com> Date: Tue, 9 Dec 2025 19:26:57 +0700 Subject: [PATCH 4/4] ai reviews --- languages/tolk/features/auto-serialization.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/languages/tolk/features/auto-serialization.mdx b/languages/tolk/features/auto-serialization.mdx index 6db82af7e..3cb461827 100644 --- a/languages/tolk/features/auto-serialization.mdx +++ b/languages/tolk/features/auto-serialization.mdx @@ -2,7 +2,7 @@ title: "Automatic serialization" --- -All data in TON, such as messages, storage, etc, is represented as [cells](/foundations/serialization/cells). The Tolk type system is designed to express cell contents, enabling auto-serialization through `fromCell` and `toCell`: +All data in TON, such as messages, storage, etc., is represented as [cells](/foundations/serialization/cells). The Tolk type system is designed to express cell contents, enabling auto-serialization through `fromCell` and `toCell`: ```tolk struct Point { @@ -39,7 +39,7 @@ Fields of a struct are serialized one by one. The compiler does not reorder fiel There are two types of references – typed and untyped: -- `Cell – typed reference; the cell has a known internal structure; +- `Cell` – typed reference; the cell has a known internal structure; - `cell` – untyped reference; the cell content is not described. ```tolk @@ -159,7 +159,7 @@ Typical failure cases include: Common exception codes: - [9 – cell underflow](/tvm/exit-codes#9%3A-cell-underflow); -- [5 – out of range](/tvm/exit-codes#5:-integer-out-of-expected-range). +- [5 – out of range](/tvm/exit-codes#5%3A-integer-out-of-expected-range). Some behaviors are configurable. For example, to allow extra data: