OneSwap Series 13 — delegatecall: neither fish nor fowl
For programmers, these four operations are commonplace:
- Create a symbolic link (Linux) or shortcut (Windows) for the program, and click this symbolic link or shortcut to start the program
- Use a dynamic link library to reduce the size of executable files
- Regularly upgrade the operating system and applications, to enhance functions, and apply security patches
- Source or exec another script in a script, and use environment variables to pass information to it
These operations seem unrelated to each other, but in Solidity, to achieve similar effects, we need the same mechanism: delegatecall.
The delegatecall in Solidity is so special that there is no direct corresponding concept to it in other programming languages. It is difficult to understand, even for programmers. In this article, we use the above four operations as an analogy to understand it.
Four mechanisms for calling contracts
In the previous article, we introduced the low-level function call mechanism of Solidity, that is, applying the call function directly to the contract address, which corresponds to the call instruction at the low level of the EVM. Its meaning will not be repeated here.
EVM also supports three other instructions for inter-contract calls: staticcall, delegatecall, and callcode.
Staticcall is very similar to call, except that the persistent state storage of the called contract will be set to read-only and cannot be modified. Solidity uses it to call external functions of view or pure attribute.
delegatecall is very special. When Contract A calls Contract B, the persistent state storage that Contract B reads and writes is actually in Contract A. In other words, Contract A authorizes its storage to Contract B and asks it to modify it.
Callcode has long been deprecated. Similar as it is to delegatecall, callcode does not retain msg.sender and msg.value. It is an EVM instruction designed with a bug. The delegatecall instruction was introduced into EVM to fix this bug.
Shortcut for smart contracts
Suppose that on a PC shared by multiple users, user Alice installs a puzzle game on Disk D, and user Bob also wants to play the game. Then he can:
- Copy this program to his own directory, which will take up some hard disk space
- Or create a desktop shortcut that points to the executable file of this game, so that it hardly takes up hard disk space
Similarly, on Ethereum, User A deploys a smart contract, and User B wants to deploy an identical contract. Then he can:
- Download the bytecode of the contract deployed by User A, and then deploy another copy by himself
- Or deploy a contract with a function similar to a shortcut, pointing to the contract deployed by User A
Ethereum's EIP-1167 stipulates how to write such a shortcut-like contract. It applies some tricks when writing contracts using EVM bytecodes, and minimizes overhead in terms of both the number of bytes and gas consumption. For the sake of clarity, we will not introduce its bytecode one by one, but explain its operation in a clearer way.
let ptr := mload(0x40)
let size := calldatasize()
calldatacopy(ptr, 0, size)
let result := delegatecall(gas(), impl, ptr, size, 0, 0)
size := returndatasize()
returndatacopy(ptr, 0, size)
switch result
case 0 { revert(ptr, size) }
default { return(ptr, size) }
This code accomplishes the same task as EIP-1167, but it is more readable. The variable ptr points to the starting position of the allocatable free memory, which is stored at address 0x40 by Solidity. The variable size is the length of calldata of this contract, calldatacopy copies calldata of this contract to free memory. delegatecall uses the data in the length of size starting from ptr in memory (the data is the calldata that has just been copied) to call the impl contract. The first parameter of delegatecall is gas(), which means that the remaining gas of the current contract is handed over to the called contract. The latter returndatacopy copies the return value of the called contract to free memory. The last switch determines whether the current contract is ended with return or revert according to whether the impl contract is successfully called (the bool value returned by delegatecall is True or False). The end with “return” indicates that the current contract has been successfully executed, and the end with “revert” indicates an execution failure. Whether it is successful or not, the returndata copied to the memory just now is returned to the caller of the current contract. From the whole process we can see:
- The calldata used to call this contract is exactly what is used by this contract to call impl
- If Impl returns successfully, this contract will also return successfully; if impl is reverted, this contract will also be reverted
- returndata that impl returns to this contract, will be returned by this contract to its caller without any change
From the outside, calling this contract has exactly the same effect as calling impl, but consumes slightly more gas because more operations are involved.
The "shortcut contract" optimized by EIP-1167 is as small as only 90 bytes, so its deployment is rather cheap.
Here is another detail: What are the last two parameters of delegatecall for? To copy returndata. If you are sure of the size of returndata before calling, you can fill in these two parameters to complete the copy, without the need of calling returndatacopy specifically. If fill them with zeros, then nothing is copied.
An imperfect dynamic link library
The dynamic link library, that is, the .dll file of Windows and the .so file of Linux, allows many programs to share the same library code, avoiding a swell of bulk resulted from copying a library code in each executable file. Once the executable file is linked to the dynamic link library, the functions in the library share the memory space and permissions of the process. We can give the starting address of an array to a library function for sorting, or the name of a directory to a library function to batch compress the files inside.
On Ethereum, the need to reduce the size of smart contracts is even more urgent because ever since EIP-170 was enabled, the bytecode of a contract cannot exceed 24,576 bytes in length. OneSwapPair.sol only has 1200+ lines. The compiled bytecode's length has reached 17,882 bytes. A slightly more complicated contract will almost certainly exceed that upper limit.
Therefore, Solidity supports Library, which is a separately deployed contract, and its external function is called by other contracts through delegatecall. In this way, other contracts can reduce their size.
But it is a pity that with delegatecall the called library function can only control the storage space of the caller, but not access its memory space, making delegatecall something like an imperfect dynamic link library, of which only the disk can be accessed yet the memory space cannot be shared. Syntactically, although the external function in the Library can also receive memory type parameters, such as variable-length arrays and strings, these parameters are passed by copying calldata, instead of a pointer (reference), because the called function completely works in the memory space of another EVM, like the following function:
function findMin(uint[] memory arr) external returns (uint) {
uint min = 0;
for(uint i = 0; i < arr.length; i++) {
if (i < min) min = i;
}
return min;
}
If it is called as a library function, the entire arr must be copied from calldata, resulting in considerable gas consumption: Although it charges no gas to copy data into calldata, the called function will consume gas when the library function copies data from calldata to memory. Reducing the code size a little bit makes you risk incurring an unaffordable gas fee.
Therefore, the Library in Solidity should only contain internal functions. Similar to a static link library, these functions will be copied into the contract you deploy, which can reduce gas consumption. The Library in the OneSwap project is designed in this way.
When library functions accept functions of storage type, they do take the pass-by-reference approach. Therefore, if the above function declaration is function findMin(uint[] storage arr) external returns (uint), it can do the same job as a dynamic link library function at lower gas overhead.
A proxy supporting dynamic upgrade (proxy)
After the release of the operating system and applications, there will be regular or irregular upgrades, which is nothing new for users. However, once deployed on Ethereum, a smart contract cannot be updated. If you want to fix bugs or enhance functions, you have no choice but to deploy another contract.
To smoothly migrate between the old and new contracts and achieve an effect similar to an upgraded contract, we can extend the "smart contract shortcut" just mentioned. If the called contract address impl is not a hard-coded value in the contract code, but a variable from storage, then the contract pointed to by this "shortcut" can be switched by modifying the storage variable. In this scenario, this "shortcut" contract is customarily called a proxy contract.
When implementing a proxy contract, note that the slots allocated by the storage variable that the proxy contract and the final called contract cannot conflict with each other. That is because the two contracts share the same persistent storage space. To avoid such conflicts, we often divide storage variables into 3 categories, and their slot numbers increase in turn.
- The first category: variables that are only accessed by the proxy contract, for example, the address of the contract that is ultimately called by delegatecall, and who is entitled to modify this address under what circumstances.
- The second category: variables that both the proxy contract and the called contract need to access, mainly variables that need to be initialized in the constructor
- The third category: storage variables that are only accessed by the finally called contract
Variables of the first and second categories need to be declared in both contracts (the proxy contract and the called contract), and the order is exactly the same. Variables of the third category only need to be declared in the called contract.
In OneSwapPair.sol, the first slots of OneSwapPairProxy and OneSwapPair define these variables:
uint internal _unusedVar0;
uint internal _unusedVar1;
uint internal _unusedVar2;
uint internal _unusedVar3;
uint internal _unusedVar4;
uint internal _unusedVar5;
uint internal _unusedVar6;
uint internal _unusedVar7;
uint internal _unusedVar8;
uint internal _unusedVar9;
uint internal _unlocked;
Among them, _unlocked falls in the second category mentioned above. Both the proxy contract (OneSwapPairProxy) and the called contract (OneSwapPair) need to access it, and OneSwapPairProxy is responsible for assigning the initial value to it in the constructor.
The first ten _unusedVar* variables are in the first category mentioned above. Although they are not used in OneSwapPairProxy or OneSwapPair, we still declare them, or in other words, we reserve the slot numbers 0-9 for future use of such variables.
Why doesn't OneSwapPairProxy need the first category of variables? Because it can query which OneSwapPair contract it should call from its creator OneSwapFactory contract:
address impl = IOneSwapFactory(address(_immuFactory)).pairLogic();
In this way, as long as the OneSwapFactory contract updates pairLogic, the OneSwapPairProxy contracts of all trading pairs can query to get the updated impl address.
Immutable Forwarding
As we mentioned earlier, delegatecall does not perfectly implement the function of a dynamic link library. A contract calling another contract with delegatecall is more like to source or exec another script in a script and to ask it to modify the files stored on the disk.
When we run a script, there are three ways to pass information to it to control its behavior: command line parameters, configuration files, and environment variables.
The proxy contract passes the parameters to the called contract as they are, similar to passing information through command line parameters. The second storage variables mentioned above can play a role similar to configuration files. The proxy contract writes values to them, and the called contract reads them. As mentioned in the previous article, the disadvantage of this configuration method is that the gas consumption of storage variables is much larger than that of immutable variables.
The OneSwapPairProxy contract uses immutable variables to store configuration information for different trading pairs. The configuration information must be forwarded to the OneSwapPair contract before it is effective. Since Proxy can only package and forward calldata as it is, it is difficult to pass this information through the parameter list of the function in the OneSwapPair contract. Yet considering the gas consumption, we don't want to pass configuration information through storage variables. So what can we do? How about stimulating the effect of environment variables and passing the configuration information by other means?
There is indeed such a method. As we mentioned before, if there is extra data at the end of calldata that is not needed by the ABI parsing logic, then no error will be thrown (which happens only when the data is insufficient). Therefore, we can append the configuration information to the back of calldata and pass it to the OneSwapPair contract. In the code of OneSwapPairProxy, it is achieved as below:
uint internal immutable _immuFactory;
uint internal immutable _immuMoneyToken;
uint internal immutable _immuStockToken;
uint internal immutable _immuOnes;
uint internal immutable _immuOther;
constructor(address stockToken, address moneyToken, bool isOnlySwap, uint64 stockUnit, uint64 priceMul, uint64 priceDiv, address ones) public {
_immuFactory = uint(msg.sender);
_immuMoneyToken = uint(moneyToken);
_immuStockToken = uint(stockToken);
_immuOnes = uint(ones);
uint temp = 0;
if(isOnlySwap) {
temp = 1;
}
temp = (temp<<64) | stockUnit;
temp = (temp<<64) | priceMul;
temp = (temp<<64) | priceDiv;
_immuOther = temp;
_unlocked = 1;
}
receive() external payable { }
// solhint-disable-next-line no-complex-fallback
fallback() payable external {
uint factory = _immuFactory;
uint moneyToken = _immuMoneyToken;
uint stockToken = _immuStockToken;
uint ones = _immuOnes;
uint other = _immuOther;
address impl = IOneSwapFactory(address(_immuFactory)).pairLogic();
// solhint-disable-next-line no-inline-assembly
assembly {
let ptr := mload(0x40)
let size := calldatasize()
calldatacopy(ptr, 0, size)
let end := add(ptr, size)
// append immutable variables to the end of calldata
mstore(end, factory)
end := add(end, 32)
mstore(end, moneyToken)
end := add(end, 32)
mstore(end, stockToken)
end := add(end, 32)
mstore(end, ones)
end := add(end, 32)
mstore(end, other)
size := add(size, 160)
let result := delegatecall(gas(), impl, ptr, size, 0, 0)
size := returndatasize()
returndatacopy(ptr, 0, size)
switch result
case 0 { revert(ptr, size) }
default { return(ptr, size) }
}
}
The constructor receives several parameters and stores them as immutable variables. The assembly code in the fallback function is similar to the "shortcut contract" just mentioned in terms of the general structure, just with an extra end variable and some mstore statements. These mstore statements save the value of the immutable variable in the memory and append it to the calldata. The original calldata, together with the value of the additional immutable variable, is passed to the OneSwapPair contract during delegatecall.
The OneSwapPair contract uses the fill function to copy the additional information at the end of the calldata to the proxyData in the memory. After that, the factory, money and other functions further retrieve the configuration information we need from the proxyData.
function factory(uint[5] memory proxyData) internal pure returns (address) {
return address(proxyData[INDEX_FACTORY]);
}
function money(uint[5] memory proxyData) internal pure returns (address) {
return address(proxyData[INDEX_MONEY_TOKEN]);
}
function fill(uint[5] memory proxyData, uint expectedCallDataSize) internal pure {
uint size;
// solhint-disable-next-line no-inline-assembly
assembly {
size := calldatasize()
}
require(size == expectedCallDataSize, "INVALID_CALLDATASIZE");
// solhint-disable-next-line no-inline-assembly
assembly {
let offset := sub(size, 160)
calldatacopy(proxyData, offset, 160)
}
}
The fill function also specifically checks whether the length of calldata after the configuration information is added is exactly equal to expectedCallDataSize. If not, the OneSwapPairProxy contract is likely to be called by an abnormal calldata constructed by a hacker, which should be guarded against.
Summary
It is difficult for programmers to understand EVM's delegatecall since there are no corresponding concepts in other programming languages. This article introduces delegatecall through some typical cases to help you get clear about it.