Blockchain Dev: Modifying Tables in EOSIO Blockchains (WAX)

marcosdk (70)in #cryptoprofessor • 3 years ago (edited)

A context

In EOSIO blockchains, such as WAX Blockchain, smart contracts are basically organized into actions and tables. Actions group the set of operations that the smart contract will perform; it's the program code. Tables are the data that will be handled by the smart contract and stored on the Blockchain.

A user, either directly or through a dApp, can interact with a smart contract in two ways:

Make a call for the execution of one or more actions.
Request data stored in tables.

Actions can perform read and write operations on data tables, as we can see in the attached schema.

When an action is called, the code of the associated method will be executed in the smart contract code. The steps can be summarized as follows:

Call to action with input data (if needed)
Executing the action (read/write to tables if applicable)
Storing results in tables (if applicable)
Return final action state (Success or Error)

It is important to note that actions do not return processing results. It only returns the information needed to know if its execution has been successful or has failed. In case of error it will also return the information of that error.
If we need to know the result of a data processing by the action we must instruct you to store that data in a table for further reference.

An important limitation of multi-index tables is that their structure cannot be modified once they are created if they contain data as this data will be lost.

It is currently in development EOSIO v.2.1 RC3 that incorporates solutions to these mentioned problems. With this version, actions can return data, and you can create tables of type Key-Value whose structure can be modified in production.

https://eos.io/news/eosio-2-1

But until this version is deployed we will have to learn how to solve the problems with the material we have.

Understanding the problem

Suppose we want to add a new field to the table structure.

Currently it is not possible to modify the structure of a multi-index table of an EOSIO smart contract if it already contains data. If we tried to do so, we would irretrievably lose all the contents of the table.

Fortunately it is possible to modify the structure of a table that does not contain data, but what good is this solution in production?

We can solve this by using a temporary table to download the main table and, once empty, modify its structure. Next, we can return the data to the original table and destroy the helper table.

That said, it seems simple, but in practice it's not that simple. We must carefully study the structure of our smart contract to prevent data loss or corruption during the procedure.

In addition, many of the operations we are going to perform; create auxiliary table, move data, modify structure, restore data, delete auxiliary table... they're going to require compiling and publishing the smart contract on the blockchain.
And that's not forgetting that the smart contract will be accessible to other users or dApps during the process, since we will need it to be published to be able to execute the actions that will modify the table(s).

First step: Lock the smart contract and create the auxiliary table

As we said before, the smart contract must be published and accessible if we want to call the auxiliary actions that we need to create to mobilize the data between the table that we want to modify and the auxiliary table. During this process it is necessary to prevent other users or dApps from performing operations on the smart contract that may alter some data that will be involved in the operation.

We must locate all actions that involve access to the data and prevent it from being called. To do this, it may be sufficient to change the authority required for the call.

It is common for an action that can be called by a user to require the user's own signature to verify authorization:

require_auth(user);

so by changing this authorization you can temporarily block any attempt to access the action.

require_auth(_self);

In this example we specify that only the smart contract itself will be able to call the function.

If we want to be more cautious and avoid scaring users we can leave the authorization unchanged, but cause an error to display a text that informs the caller that we are performing maintenance, for example, comparing the user's account with the smart contract account:

require_auth(user);
check(user == _self, “ERROR: Smart Contract under maintenance. Please try it in a while”);

Once all the accesses are blocked we can define a table identical to the table whose structure we want to change and a new action that contains the code to read the original table, record to record, and dump its contents into the helper table.

Sample code

auto idxOriginalTable = _originalTable.begin();
while(idxOriginalTable ¡= _originalTable.end()){
    _auxTable.emplace( _self, [&](auto &rec) {
        rec.field1 = idxOriginalTable->field1;
        rec.field_n = idxOriginalTable->field_n;
    });
    // erase actual row
    // index to nex row (or end)
    idxOriginalTable = _originalTable.erase(idxOriginalTable);
}

At the end of the loop, all the information in the original table will be in the helper table and the original table will be empty.

Summary of the first step:

Block accesses
Create auxiliary table
Create action to dump original table contents onto auxiliary table and delete the original.
Build the smart contract and publish it on the blockchain
Call the action created for the data dump

Second step: Modify the structure of the original table and return the data to your site

Going back to the smart contract code the next step will be to modify the structure of the original table, now that it has been left empty and with its data safe in the helper table, with the changes we wanted to make; change some field, add new ones, etc.

At this point we can create the action for retrieving the data from the auxiliary table to the modified table.

Now the most important thing is to make sure that the original data will be placed correctly in the new modified table, that is; we will be very careful with transforming the data types if we have changed the data type of any field. If we have added new fields, they must be empty unless we have some way to complete them with the existing information or by adding a table with the new data to the action.

I will not repeat the code as it is almost identical to that of the previous case except for the adjustment of the field types, or the collection of new data, if necessary.

It would be wise, in this step, not to delete the data from the auxiliary table in case we have not designed the adaptation process correctly. When the entire operation is complete and we remove the helper table from the smart contract code, your data will disappear as well.

It would also be a good time to make all the necessary modifications to the other actions that will make use of the table with its new structure.

Summary of the second step:

Modify the structure of the original table.
Create the action to retrieve the data from the helper table with the necessary adaptations.
Compile the smart contract and publish it on the blockchain.
Call the action created for the data dump.

Last step: Cleaning and Relaunch

If all went well we already have the modified table without losing data. It's time to clean the work material and leave the smart contract operational again. To do this we will delete the auxiliary table and the actions that we have created for the dumps in both directions (and save us RAM). We also need to review the permissions we changed in step one to unlock calls to the actions we had blocked.

Summary of the third step:

Delete auxiliary table and helper actions.
Unlock authentication permissions on actions blocked in step 1.
Compile the smart contract and publish it on the blockchain.

Test

As you can see, modifying a multi-index table on an EOSIO blockchain is no easy task but, with patience and good analysis, it shouldn't be very problematic. This is a tedious job that has required compiling and publishing the smart contract 3 times!

My last recommendation is a call to wisdom and prudence: use the testnet blockchain to test before hammering your smart contract in production.

I hope this is helpful and sorry for my crappy English.

I am Marcos DK, Steem Witness and top21 WAX Block Producer. Please, support me with a vote as your witness.