[PSA] steem.ws + golos.ws = downtime

last month
69 in steem-dev

Just providing a heads up with what's going on surrounding these public API servers.

I've been working on the both of these on and off for the last 3 days since they've both encountered issues of falling behind on blocks, forking off, and now while rebuilding hitting "unlinkable block" errors. With HF19 due tomorrow, and myself still needing to upgrade a number of other servers, I'm taking these both offline.

This morning I woke up after the 3rd replay of node.steem.ws to find

394288ms th_a       fork_database.cpp:43          push_block           ] Pushing block to fork database that failed to link: 00c28867f008d2d091f18d92ae0383db6971e205, 12748903
394288ms th_a       fork_database.cpp:44          push_block           ] Head: 12748898, 00c28862c05ae3eded793d0c6fa34e39fc6b7cd8
394288ms th_a       application.cpp:517           handle_block         ] Error when pushing block:
4080000 unlinkable_block_exception: unlinkable block
block does not link to known chain
    {}
    th_a  fork_database.cpp:64 _push_block

    {}
    th_a  database.cpp:625 _push_block

and

393272ms th_a       database.cpp:2678             _apply_block         ] 13 N5boost12interprocess9bad_allocE: boost::interprocess::bad_alloc
boost::interprocess::bad_alloc:
    {"o":{"voter":"lingfei","author":"papa-pepper","permlink":"steem-pocalype-survival-game-contest-season-3-day-5-round-2-700-steem-prize-pool-so-far-yes-700","weight":6500},"what":"boost::interprocess::bad_alloc"}
    th_a  steem_evaluator.cpp:1543 do_apply

    {"op":["vote",{"voter":"lingfei","author":"papa-pepper","permlink":"steem-pocalype-survival-game-contest-season-3-day-5-round-2-700-steem-prize-pool-so-far-yes-700","weight":6500}]}
    th_a  database.cpp:2879 _apply_transaction

    {"trx":{"ref_block_num":34908,"ref_block_prefix":3997405554,"expiration":"2017-06-12T06:51:17","operations":[["vote",{"voter":"lingfei","author":"papa-pepper","permlink":"steem-pocalype-survival-game-contest-season-3-day-5-round-2-700-steem-prize-pool-so-far-yes-700","weight":6500}]],"extensions":[],"signatures":["2025705a48eede55710aee5342c06ee2d761dc4f436e6525c85c74d5ae77bf56b714ca6b5105db3e7d7f83438f5bad8ffbad1d4190279be79af24d16feb4edd44a"]}}

The plan is to decommission the existing public API's I run, both golos.ws and steem.ws, and setup brand new (and upgraded servers).

Apologies for the downtime thus far, and moving into the future. In the mean time, I'd recommend you point your scripts to one of the other public API's available:

  • wss://steemd.steemit.com
  • wss://seed.bitcoiner.me
  • wss://gtg.steem.house:8090

or use @almost-digital's new fancy tool to help find one:

https://bench.steem.vc/

Authors get paid when people like you upvote their post.
Join our amazing community to comment and reward others.
Sort Order:  trending
58
  ·  last month

I am not a programmer so I cannot understand the code very well. But it is nice of you to mention that you took your servers down so that people can adjust accordingly. Have fun out there and good luck with future posts!

6
  ·  last month

Looks different ,
Thank you so much for this great article! I start reading this knowing that my ignorance in this field could penalize me but it was so great to learn something New

71
  ·  last month

No need to decline payment on stuff like this, IMO. You're doing fantastic work for the network and the community, Jesta, and you deserve to be richly rewarded for your efforts. We all greatly appreciate your efforts. Thank you!

·
69
  ·  last month

Hah, for some reason it didn't feel right to say "Hey, my servers are down, I should be rewarded!".

56
  ·  last month

How long does it take to update servers to hardfork 19?

·
69
  ·  last month

Depends on what kind of server it is, for a witness node it's a couple hours, and for a full API node it can be anywhere from 1-3 days (depending on the configuration and hardware). Replaying the blockchain to a new server is quite the process.

·
·
52
  ·  last month

Thanks for putting in all that hard work!

60
  ·  last month

Is this why Poloniex isn't sweeping their wallet ?

·
69
  ·  last month

No, they are probably running their own server. I'd hope they wouldn't use my node without telling me :)

·
·
60
  ·  last month

hehe

Thanks @jesta :-)

39
  ·  last month

Though I do not understand this programming and stuff but I appreciate your hard work.

53
  ·  last month

thanks for all the great work you do.