Short ABIs for Calldata Optimization

layer 2

Intermediate

Ori Pomerantz

Abril 1, 2022

14 minute read minute read

Introduction

In this article, you learn about optimistic rollups, the cost of transactions on them, and how that different cost structure requires us to optimize for different things than on the Ethereum Mainnet. You also learn how to implement this optimization.

Full disclosure

I'm a full time Optimism(opens in a new tab) employee, so examples in this article will run on Optimism. However, the technique explained here should work just as well for other rollups.

Terminology

When discussing rollups, the term 'layer 1' (L1) is used for Mainnet, the production Ethereum network. The term 'layer 2' (L2) is used for the rollup or any other system that relies on L1 for security but does most of its processing off-chain.

How can we further reduce the cost of L2 transactions?

Optimistic rollups have to preserve a record of every historical transaction so that anybody will be able to go through them and verify that the current state is correct. The cheapest way to get data into the Ethereum Mainnet is to write it as calldata. This solution was chosen by both Optimism(opens in a new tab) and Arbitrum(opens in a new tab).

Cost of L2 transactions

The cost of L2 transactions is composed of two components:

L2 processing, which is usually extremely cheap
L1 storage, which is tied to Mainnet gas costs

As I'm writing this, on Optimism the cost of L2 gas is 0.001 Gwei. The cost of L1 gas, on the other hand, is approximately 40 gwei. You can see the current prices here(opens in a new tab).

A byte of calldata costs either 4 gas (if it is zero) or 16 gas (if it is any other value). One of the most expensive operations on the EVM is writing to storage. The maximum cost of writing a 32-byte word to storage on L2 is 22100 gas. Currently, this is 22.1 gwei. So if we can save a single zero byte of calldata, we'll be able to write about 200 bytes to storage and still come out ahead.

The ABI

The vast majority of transactions access a contract from an externally-owned account. Most contracts are written in Solidity and interpret their data field per the application binary interface (ABI)(opens in a new tab).

However, the ABI was designed for L1, where a byte of calldata costs approximately the same as four arithmetic operations, not L2 where a byte of calldata costs more than a thousand arithmetic operations. The calldata is divided like this:

Section	Length	Bytes	Wasted bytes	Wasted gas	Necessary bytes	Necessary gas
Function selector	4	0-3	3	48	1	16
Zeroes	12	4-15	12	48	0	0
Destination address	20	16-35	0	0	20	320
Amount	32	36-67	17	64	15	240
Total	68			160		576

Explanation:

Function selector: The contract has less than 256 functions, so we can distinguish them with a single byte. These bytes are typically non-zero and therefore cost sixteen gas(opens in a new tab).
Zeroes: These bytes are always zero because a twenty-byte address does not require a thirty-two-byte word to hold it. Bytes that hold zero cost four gas (see the yellow paper(opens in a new tab), Appendix G, p. 27, the value for G_txdatazero).
Amount: If we assume that in this contract decimals is eighteen (the normal value) and the maximum amount of tokens we transfer will be 10¹⁸, we get a maximum amount of 10³⁶. 256¹⁵ > 10³⁶, so fifteen bytes are enough.

A waste of 160 gas on L1 is normally negligible. A transaction costs at least 21,000 gas(opens in a new tab), so an extra 0.8% doesn't matter. However, on L2, things are different. Almost the entire cost of the transaction is writing it to L1. In addition to the transaction calldata, there are 109 bytes of transaction header (destination address, signature, etc.). The total cost is therefore 109*16+576+160=2480, and we are wasting about 6.5% of that.

Reducing costs when you don't control the destination

Assuming that you do not have control over the destination contract, you can still use a solution similar to this one(opens in a new tab). Let's go over the relevant files.

Token.sol

This is the destination contract(opens in a new tab). It is a standard ERC-20 contract, with one additional feature. This faucet function lets any user get some token to use. It would make a production ERC-20 contract useless, but it makes life easier when an ERC-20 exists only to facilitate testing.

1    /**
2     * @dev Gives the caller 1000 tokens to play with
3     */
4    function faucet() external {
5        _mint(msg.sender, 1000);
6    }   // function faucet
 Kopyahin

CalldataInterpreter.sol

This is the contract that transactions are supposed to call with shorter calldata(opens in a new tab). Let's go over it line by line.

1//SPDX-License-Identifier: Unlicense
2pragma solidity ^0.8.0;
3
4
5import { OrisUselessToken } from "./Token.sol";
 Kopyahin

We need the token function to know how to call it.

1contract CalldataInterpreter {
2
3    OrisUselessToken public immutable token;
 Kopyahin

The address of the token for which we are a proxy.

1
2    /**
3     * @dev Specify the token address
4     * @param tokenAddr_ ERC-20 contract address
5     */
6    constructor(
7        address tokenAddr_
8    )  {
9        token = OrisUselessToken(tokenAddr_);
10    }   // constructor
Ipakita lahat
 Kopyahin

The token address is the only parameter we need to specify.

1    function calldataVal(uint startByte, uint length)
2        private pure returns (uint) {
 Kopyahin

Read a value from the calldata.

1        uint _retVal;
2
3        require(length < 0x21,
4            "calldataVal length limit is 32 bytes");
5
6        require(length + startByte <= msg.data.length,
7            "calldataVal trying to read beyond calldatasize");
 Kopyahin

We are going to load a single 32-byte (256-bit) word to memory and remove the bytes that aren't part of the field we want. This algorithm doesn't work for values longer than 32 bytes, and of course we can't read past the end of the calldata. On L1 it might be necessary to skip these tests to save on gas, but on L2 gas is extremely cheap, which enables whatever sanity checks we can think of.

1        assembly {
2            _retVal := calldataload(startByte)
3        }
 Kopyahin

We could have copied the data from the call to fallback() (see below), but it is easier to use Yul(opens in a new tab), the assembly language of the EVM.

Here we use the CALLDATALOAD opcode(opens in a new tab) to read bytes startByte to startByte+31 into the stack. In general, the syntax of an opcode in Yul is <opcode name>(<first stack value, if any>,<second stack value, if any>...).

1
2        _retVal = _retVal >> (256-length*8);
 Kopyahin

Only the most significant length bytes are part of the field, so we right-shift(opens in a new tab) to get rid of the other values. This has the added advantage of moving the value to the right of the field, so it is the value itself rather than the value times 256^something.

1
2        return _retVal;
3    }
4
5
6    fallback() external {
 Kopyahin

When a call to a Solidity contract does not match any of the function signatures, it calls the fallback() function(opens in a new tab) (assuming there is one). In the case of CalldataInterpreter, any call gets here because there are no other external or public functions.

1        uint _func;
2
3        _func = calldataVal(0, 1);
 Kopyahin

Read the first byte of the calldata, which tells us the function. There are two reasons why a function would not be available here:

Functions that are pure or view don't change the state and don't cost gas (when called off-chain). It makes no sense to try to reduce their gas cost.
Functions that rely on msg.sender(opens in a new tab). The value of msg.sender is going to be CalldataInterpreter's address, not the caller.

Unfortunately, looking at the ERC-20 specifications(opens in a new tab), this leaves only one function, transfer. This leaves us with only two functions: transfer (because we can call transferFrom) and faucet (because we can transfer the tokens back to whoever called us).

1
2        // Call the state changing methods of token using
3        // information from the calldata
4
5        // faucet
6        if (_func == 1) {
 Kopyahin

A call to faucet(), which doesn't have parameters.

1            token.faucet();
2            token.transfer(msg.sender,
3                token.balanceOf(address(this)));
4        }
 Kopyahin

After we call token.faucet() we get tokens. However, as the proxy contract, we do not need tokens. The EOA (externally owned account) or contract that called us does. So we transfer all of our tokens to whoever called us.

1        // transfer (assume we have an allowance for it)
2        if (_func == 2) {
 Kopyahin

Transferring tokens requires two parameters: the destination address and the amount.

1            token.transferFrom(
2                msg.sender,
 Kopyahin

We only allow callers to transfer tokens they own

1                address(uint160(calldataVal(1, 20))),
 Kopyahin

The destination address starts at byte #1 (byte #0 is the function). As an address, it is 20-bytes long.

1                calldataVal(21, 2)
 Kopyahin

For this particular contract we assume that the maximum number of tokens anybody would want to transfer fits in two bytes (less than 65536).

1            );
2        }
 Kopyahin

Overall, a transfer takes 35 bytes of calldata:

Section	Length	Bytes
Function selector	1	0
Destination address	32	1-32
Amount	2	33-34

1    }   // fallback
2
3}       // contract CalldataInterpreter
 Kopyahin

test.js

This JavaScript unit test(opens in a new tab) shows us how to use this mechanism (and how to verify it works correctly). I am going to assume you understand chai(opens in a new tab) and ethers(opens in a new tab) and only explain the parts that specifically apply to the contract.

1const { expect } = require("chai");
2
3describe("CalldataInterpreter", function () {
4  it("Should let us use tokens", async function () {
5    const Token = await ethers.getContractFactory("OrisUselessToken")
6    const token = await Token.deploy()
7    await token.deployed()
8    console.log("Token addr:", token.address)
9
10    const Cdi = await ethers.getContractFactory("CalldataInterpreter")
11    const cdi = await Cdi.deploy(token.address)
12    await cdi.deployed()
13    console.log("CalldataInterpreter addr:", cdi.address)
14
15    const signer = await ethers.getSigner()
Ipakita lahat
 Kopyahin

We start by deploying both contracts.

1    // Get tokens to play with
2    const faucetTx = {

We can't use the high-level functions we'd normally use (such as token.faucet()) to create transactions, because we do not follow the ABI. Instead, we have to build the transaction ourselves and then send it.

1      to: cdi.address,
2      data: "0x01"

There are two parameters we need to provide for the transaction:

to, the destination address. This is the calldata interpreter contract.
data, the calldata to send. In the case of a faucet call, the data is a single byte, 0x01.

1
2    }
3    await (await signer.sendTransaction(faucetTx)).wait()

We call the signer's sendTransaction method(opens in a new tab) because we already specified the destination (faucetTx.to) and we need the transaction to be signed.

1// Check the faucet provides the tokens correctly
2expect(await token.balanceOf(signer.address)).to.equal(1000)

Here we verify the balance. There is no need to save gas on view functions, so we just run them normally.

1// Give the CDI an allowance (approvals cannot be proxied)
2const approveTX = await token.approve(cdi.address, 10000)
3await approveTX.wait()
4expect(await token.allowance(signer.address, cdi.address)).to.equal(10000)

Give the calldata interpreter an allowance to be able to do transfers.

1// Transfer tokens
2const destAddr = "0xf5a6ead936fb47f342bb63e676479bddf26ebe1d"
3const transferTx = {
4  to: cdi.address,
5  data: "0x02" + destAddr.slice(2, 42) + "0100",
6}

Create a transfer transaction. The first byte is "0x02", followed by the destination address, and finally the amount (0x0100, which is 256 in decimal).

1    await (await signer.sendTransaction(transferTx)).wait()
2
3    // Check that we have 256 tokens less
4    expect (await token.balanceOf(signer.address)).to.equal(1000-256)
5
6    // And that our destination got them
7    expect (await token.balanceOf(destAddr)).to.equal(256)
8  })    // it
9})      // describe
Ipakita lahat

Reducing the cost when you do control the destination contract

If you do have control over the destination contract you can create functions that bypass the msg.sender checks because they trust the calldata interpreter. You can see an example of how this works here, in the control-contract branch(opens in a new tab).

If the contract were responding only to external transactions, we could get by with having just one contract. However, that would break composability. It is much better to have a contract that responds to normal ERC-20 calls, and another contract that responds to transactions with short call data.

Token.sol

In this example we can modify Token.sol. This lets us have a number of functions that only the proxy may call. Here are the new parts:

1    // The only address allowed to specify the CalldataInterpreter address
2    address owner;
3
4    // The CalldataInterpreter address
5    address proxy = address(0);
 Kopyahin

The ERC-20 contract needs to know the identity of the authorized proxy. However, we cannot set this variable in the constructor, because we don't know the value yet. This contract is instantiated first because the proxy expects the token's address in its constructor.

1    /**
2     * @dev Calls the ERC20 constructor.
3     */
4    constructor(
5    ) ERC20("Oris useless token-2", "OUT-2") {
6        owner = msg.sender;
7    }
 Kopyahin

The address of the creator (called owner) is stored here because that is the only address allowed to set the proxy.

1    /**
2     * @dev set the address for the proxy (the CalldataInterpreter).
3     * Can only be called once by the owner
4     */
5    function setProxy(address _proxy) external {
6        require(msg.sender == owner, "Can only be called by owner");
7        require(proxy == address(0), "Proxy is already set");
8
9        proxy = _proxy;
10    }    // function setProxy
Ipakita lahat
 Kopyahin

The proxy has privileged access, because it can bypass security checks. To make sure we can trust the proxy we only let owner call this function, and only once. Once proxy has a real value (not zero), that value cannot change, so even if the owner decides to become rogue, or the mnemonic for it is revealed, we are still safe.

1    /**
2     * @dev Some functions may only be called by the proxy.
3     */
4    modifier onlyProxy {
 Kopyahin

This is a modifier function(opens in a new tab), it modifies the way other functions work.

1      require(msg.sender == proxy);
 Kopyahin

First, verify we got called by the proxy and nobody else. If not, revert.

1      _;
2    }
 Kopyahin

If so, run the function which we modify.

1   /* Functions that allow the proxy to actually proxy for accounts */
2
3    function transferProxy(address from, address to, uint256 amount)
4        public virtual onlyProxy() returns (bool)
5    {
6        _transfer(from, to, amount);
7        return true;
8    }
9
10    function approveProxy(address from, address spender, uint256 amount)
11        public virtual onlyProxy() returns (bool)
12    {
13        _approve(from, spender, amount);
14        return true;
15    }
16
17    function transferFromProxy(
18        address spender,
19        address from,
20        address to,
21        uint256 amount
22    ) public virtual onlyProxy() returns (bool)
23    {
24        _spendAllowance(from, spender, amount);
25        _transfer(from, to, amount);
26        return true;
27    }
Ipakita lahat
 Kopyahin

These are three operations that normally require the message to come directly from the entity transferring tokens or approving an allowance. Here we have a proxy version these operations which:

Is modified by onlyProxy() so nobody else is allowed to control them.
Gets the address that would normally be msg.sender as an extra parameter.

CalldataInterpreter.sol

The calldata interpreter is nearly identical to the one above, except that the proxied functions receive a msg.sender parameter and there is no need for an allowance for transfer.

1        // transfer (no need for allowance)
2        if (_func == 2) {
3            token.transferProxy(
4                msg.sender,
5                address(uint160(calldataVal(1, 20))),
6                calldataVal(21, 2)
7            );
8        }
9
10        // approve
11        if (_func == 3) {
12            token.approveProxy(
13                msg.sender,
14                address(uint160(calldataVal(1, 20))),
15                calldataVal(21, 2)
16            );
17        }
18
19        // transferFrom
20        if (_func == 4) {
21            token.transferFromProxy(
22                msg.sender,
23                address(uint160(calldataVal( 1, 20))),
24                address(uint160(calldataVal(21, 20))),
25                calldataVal(41, 2)
26            );
27        }
Ipakita lahat
 Kopyahin

Test.js

There are a few changes between the previous testing code and this one.

1const Cdi = await ethers.getContractFactory("CalldataInterpreter")
2const cdi = await Cdi.deploy(token.address)
3await cdi.deployed()
4await token.setProxy(cdi.address)
 Kopyahin

We need to tell the ERC-20 contract which proxy to trust

1console.log("CalldataInterpreter addr:", cdi.address)
2
3// Need two signers to verify allowances
4const signers = await ethers.getSigners()
5const signer = signers[0]
6const poorSigner = signers[1]
 Kopyahin

To check approve() and transferFrom() we need a second signer. We call it poorSigner because it does not get any of our tokens (it does need to have ETH, of course).

1// Transfer tokens
2const destAddr = "0xf5a6ead936fb47f342bb63e676479bddf26ebe1d"
3const transferTx = {
4  to: cdi.address,
5  data: "0x02" + destAddr.slice(2, 42) + "0100",
6}
7await (await signer.sendTransaction(transferTx)).wait()
 Kopyahin

Because the ERC-20 contract trusts the proxy (cdi), we don't need an allowance to relay transfers.

1// approval and transferFrom
2const approveTx = {
3  to: cdi.address,
4  data: "0x03" + poorSigner.address.slice(2, 42) + "00FF",
5}
6await (await signer.sendTransaction(approveTx)).wait()
7
8const destAddr2 = "0xE1165C689C0c3e9642cA7606F5287e708d846206"
9
10const transferFromTx = {
11  to: cdi.address,
12  data: "0x04" + signer.address.slice(2, 42) + destAddr2.slice(2, 42) + "00FF",
13}
14await (await poorSigner.sendTransaction(transferFromTx)).wait()
15
16// Check the approve / transferFrom combo was done correctly
17expect(await token.balanceOf(destAddr2)).to.equal(255)
Ipakita lahat
 Kopyahin

Test the two new functions. Note that transferFromTx requires two address parameters: the giver of the allowance and the receiver.

Conclusion

Both Optimism(opens in a new tab) and Arbitrum(opens in a new tab) are looking for ways to reduce the size of the calldata written to L1 and therefore the cost of transactions. However, as infrastructure providers looking for generic solutions, our abilities are limited. As the dapp developer, you have application-specific knowledge, which lets you optimize your calldata much better than we could in a generic solution. Hopefully, this article helps you find the ideal solution for your needs.

Huling pag-edit: @Shiva-Sai-ssb(opens in a new tab), Hunyo 30, 2024

Short ABIs for Calldata Optimization

Introduction

Full disclosure

Terminology

How can we further reduce the cost of L2 transactions?

Cost of L2 transactions

The ABI

Reducing costs when you don't control the destination

Token.sol

CalldataInterpreter.sol

test.js

Reducing the cost when you do control the destination contract

Token.sol

CalldataInterpreter.sol

Test.js

Conclusion

Nakatulong ba ang tutorial na ito?