[Steem on RPi] Sync Failed

in Steemit Dev Group3 years ago

Hi, @etainclub is here.

Finally, I downloaded block data which is 210GB as of August 7, 2021 and decompressed it into the SD card of my RPi.

But when I executed the following command, it occurred an error which says 'Error opening database`.

$ ./run.sh start

Here is the full error message.

3192100ms database.cpp:192 open ] args.data_dir: /steem/witness_node_data_dir/blockchain args.shared_mem_dir: /steem/witness_node_data_dir/blockchain args.shared_file_size: 8589934592
3192101ms chain_plugin.cpp:608 plugin_startup ] Error opening database. If the binary or configuration has changed, replay the blockchain explicitly using --replay-blockchain.
3192101ms chain_plugin.cpp:609 plugin_startup ] If you know what you are doing you can skip this check and force open the database using --force-open.
3192101ms chain_plugin.cpp:610 plugin_startup ] WARNING: THIS MAY CORRUPT YOUR DATABASE. FORCE OPEN AT YOUR OWN RISK.
3192101ms chain_plugin.cpp:611 plugin_startup ] Error: {"code":13,"name":"St13__ios_failure","message":"basic_ios::clear: iostream error","stack":[{"context":{"level":"warn","file":"raw.hpp","line":337,"method":"operator()","hostname":"","timestamp":"2021-08-10T11:53:12"},"format":"${what}: Error unpacking field ${field}","data":{"field":"props","what":"basic_ios::clear: iostream error"}},{"context":{"level":"warn","file":"raw.hpp","line":624,"method":"unpack","hostname":"","timestamp":"2021-08-10T11:53:12"},"format":"error unpacking ${type}","data":{"type":"steem::protocol::witness_set_properties_operation"}},{"context":{"level":"warn","file":"raw.hpp","line":337,"method":"operator()","hostname":"","timestamp":"2021-08-10T11:53:12"},"format":"Error unpacking field ${field}","data":{"field":"operations"}},{"context":{"level":"warn","file":"raw.hpp","line":624,"method":"unpack","hostname":"","timestamp":"2021-08-10T11:53:12"},"format":"error unpacking ${type}","data":{"type":"steem::protocol::signed_transaction"}},{"context":{"level":"warn","file":"raw.hpp","line":337,"method":"operator()","hostname":"","timestamp":"2021-08-10T11:53:12"},"format":"Error unpacking field ${field}","data":{"field":"transactions"}},{"context":{"level":"warn","file":"raw.hpp","line":624,"method":"unpack","hostname":"","timestamp":"2021-08-10T11:53:12"},"format":"error unpacking ${type}","data":{"type":"steem::protocol::signed_block"}},{"context":{"level":"warn","file":"block_log.cpp","line":264,"method":"read_block_helper","hostname":"","timestamp":"2021-08-10T11:53:12"},"format":"rethrow","data":{}},{"context":{"level":"warn","file":"block_log.cpp","line":336,"method":"read_head","hostname":"","timestamp":"2021-08-10T11:53:12"},"format":"rethrow","data":{}},{"context":{"level":"warn","file":"database.cpp","line":192,"method":"open","hostname":"","timestamp":"2021-08-10T11:53:12"},"format":"rethrow","data":{"args.data_dir":"/steem/witness_node_data_dir/blockchain","args.shared_mem_dir":"/steem/witness_node_data_dir/blockchain","args.shared_file_size":"8589934592"}}]}

image.png

I tried the --force-open option like this

$ ./run.sh start --force-open

I am sure this is the right command but the same error occurred.

@ety001 could you guide me what's wrong here?
I followed your post

  • downloaded steem_witness.lz4 (2021-08-07 version)
  • decompressed it into sd card. the blockchain folder has been successfully generated
  • run the ./run.sh start

here is the source of my steem-docker-ex for arm64
https://github.com/EtainLove/steem-docker-ex

The Dockerfile used to build the steemd image for arm64. I think the MIRA is on, right?

Anyone has any idea what's wrong?

#FROM ubuntu:bionic
FROM phusion/baseimage:18.04-1.0.0-arm64

# cd ~/dkr && docker build -t steem .
# docker run -p 0.0.0.0:2001:2001 -v $PWD/data:/steem -d -t steem


RUN apt-get update && \
    apt-get install -y gcc g++ cmake make libbz2-dev libdb++-dev libdb-dev && \
    apt-get install -y libssl-dev openssl libreadline-dev autoconf libtool git gdb liblz4-tool wget jq virtualenv libgflags-dev libsnappy-dev zlib1g-dev libbz2-dev liblz4-dev libzstd-dev && \
    apt-get install -y autotools-dev build-essential libbz2-dev libicu-dev python-dev wget doxygen python3 python3-dev python3-pip libboost-all-dev curl && \
    apt-get install -y python3-jinja2 && \
    apt-get clean -qy

#RUN pip3 install jinja2

###
# Overridable build arguments
# Can override during docker build using '--build-arg' like so:
#
#   docker build --build-arg "ENABLE_MIRA=OFF" --build-arg="STEEM_STATIC_BUILD=OFF" -t steem
#
###

ARG steemd_version=0.23.x-arm64
ENV steemd_version ${steemd_version}

#ARG STEEM_SOURCE="https://github.com/steemit/steem.git"
ARG STEEM_SOURCE="https://github.com/EtainLove/steem.git"
ENV STEEM_SOURCE ${STEEM_SOURCE}

ARG STEEM_STATIC_BUILD=ON
ENV STEEM_STATIC_BUILD ${STEEM_STATIC_BUILD}

ARG ENABLE_MIRA=ON
ENV ENABLE_MIRA ${ENABLE_MIRA}

ARG LOW_MEMORY_MODE=ON
ENV LOW_MEMORY_MODE ${LOW_MEMORY_MODE}

ARG CLEAR_VOTES=ON
ENV CLEAR_VOTES ${CLEAR_VOTES}

ARG SKIP_BY_TX_ID=ON
ENV SKIP_BY_TX_ID ${SKIP_BY_TX_ID}

RUN cd ~ && \
    echo " >>> Cloning tag/branch ${steemd_version} from repo: ${STEEM_SOURCE}" && \
    git clone ${STEEM_SOURCE} steem -b ${steemd_version} && \
    cd steem && \
    git submodule update --init --recursive && \
        cd ~/steem && \
    cmake -DCMAKE_BUILD_TYPE=Release . \
            -DLOW_MEMORY_NODE=${LOW_MEMORY_MODE} \
            -DCLEAR_VOTES=${CLEAR_VOTES} \
            -DSTEEM_STATIC_BUILD=${STEEM_STATIC_BUILD} \
            -DSKIP_BY_TX_ID=${SKIP_BY_TX_ID} \
            -DENABLE_MIRA=${ENABLE_MIRA} && \
    make -j$(nproc) && make install && rm -rf ~/steem
VOLUME /steem
WORKDIR /steem

# P2P (seed) port
EXPOSE 2001
# RPC ports
EXPOSE 5000
EXPOSE 8090
EXPOSE 8091

ARG STEEMD_BIN="/usr/local/bin/steemd"
ENV STEEMD_BIN ${STEEMD_BIN}

RUN echo "This container has been built with the following options:" >> /steem_build.txt && \
    echo "----" >> /steem_build.txt && \
    echo "Git Repository:              ${STEEM_SOURCE}" >> /steem_build.txt && \
    echo "Git version/commit:          ${steemd_version}\n----" >> /steem_build.txt && \
    echo "Default steemd executable:   ${STEEMD_BIN}\n---" >> /steem_build.txt && \
    echo "--- CMake Config Options ---" >> /steem_build.txt && \
    echo "LOW_MEMORY_MODE=${LOW_MEMORY_MODE}\nSTEEM_STATIC_BUILD=${STEEM_STATIC_BUILD}" >> /steem_build.txt && \
    echo "SKIP_BY_TX_ID=${SKIP_BY_TX_ID}\nENABLE_MIRA=${ENABLE_MIRA}\nCLEAR_VOTES=${CLEAR_VOTES}" >> /steem_build.txt && \
    echo "----\nBuilt at: $(date)\n----" >> /steem_build.txt

RUN echo "Please configure me! You need to mount a data directory onto /steem of this container to it to function correctly. (if you're using Steem-in-a-box most of this is handled automatically)"
CMD ["sh", "-c", "${STEEMD_BIN}"]

The weird thing is the error message which says about "timestamp":"2021-08-10T11:55:08.
It is future timestamp since the block data is until 2021-08-07.

I also tried the 'replay-blockchain' but it was stuck due to no space left error.
Is the replay way to go?


cc.
@steemcurator01
@steemitblog

Sort:  

If you only have the block_log without the rocksdb files, you definitely need the --replay-blockchain argument. I guess due to the different system architecture you will need the replay anyway.

the file includes rockdb files, too. I kept failed so I tried a few things such as --replay-blockchain. You think I need to execute replay?

What I know for replay is to only use the block_log file. could you guide me the procedure? I don't have much knowledge on running a node.

I searched a little bit. low device such as RPi cannot run a non-MIRA witness. So I thought that non-replay is the way to go.

You only need to add the arg --replay-blockchain, any old rocksdb files should get removed automatically. Replay is also required for MIRA when any plugin (or maybe the system architecture) has changed.

I guess the replay on that device will take weeks though :) Maybe you can replay on a faster ARM machine with same OS and then just copy the data to the Pi.

That's good idea! I forgot that I have Apple M1 macbook! Thanks. I will try to replay on M1 if possible.

Loading...
Loading...

The weird thing is the error message which says about "timestamp":"2021-08-10T11:55:08.

I think you could ignore these error messages. Those error messages are stupid. I guess these messages show the status of realtime p2p network data. Maybe the p2p network startup will not wait for data synchronization to complete.

완전 암호문이군요^^

저도 이유를 알 수 없는 에러네요 ㅜ.ㅜ

Coin Marketplace

STEEM 0.19
TRX 0.16
JST 0.031
BTC 63342.86
ETH 2731.01
USDT 1.00
SBD 2.64