SteemWatch Update 2017-11-08steemCreated with Sketch.

in #steemwatch7 years ago

Hi everyone,

As mentioned previously, there were some issue with SteemWatch that were pretty noticeable for people and they were complaining. I tried to address these issues and SteemWatch should be much more stable now. What has been improved follows.

Web Application

The input fields were not being validated properly, which lead to people inserting crazy values, i.e. using a single input field to insert hundreds of usernames separated by commas. This created very long values in the DB that prevented me from creating indexes since there is a hard limit on the string length in MongoDB if you want to create an index. So first of all, I had to fix input validation and then write a script to fix the database. This is all done now.

Steem Client Library Rewritten

There were some issues with go-steem/rpc considering the WebSocket transport. When trying to reconnect, it was not possible to close the transport and exit. In the end I decided to completely rewrite that package.

When I was at it, I also added support for setting multiple RPC endpoint addresses, which are being rotated on reconnect using round-robin. This makes it possible to survive one particular RPC endpoint crashing by passing multiple addresses to the package.

Performance

I actually started digging into SteemWatch because it was lagging far behind the last block produced. This made SteemWatch pretty much useless since the main point is to watch content as it is being published. So I started measuring.

It turned out that SteemWatch was so slow because it was using just a single thread to process blocks, and processing a block meant that the steemd RPC endpoint may be called even multiple times. I am not using a local steemd node, but rather the public nodes that are managed by other people. This makes every roundtrip a bit expensive.

So, the solution to the performance issue was to spawn multiple threads and process blocks concurrently. So instead of a single thread connected to a single RPC endpoint SteemWatch now uses tens of threads connected to various public RPC endpoints.

There is still room for improvements, though, because when a call to any of the RPC endpoints times out, SteemWatch restarts itself. That was the fastest solution, but the best solution would be to keep trying forever I guess.

Other things

I also spent some time updating libraries to the current version. Particularly I started using the current version of Echo and got rid of some ugly hacks considering session management. I am pretty happy for that.

TL;DR

SteemWatch should be ok for now. I am not receiving any warnings any more that it is lagging behind the current block.

Sort:  

Nice project - Keep going :)

Very useful post thanks for sharing with us
your post resteemed!

wow really amazing
kip it up

Just got a notification from Steemwatch that @void has published or updated a story... Simply had to check it out 😂

Steemwatch is a Steemit's essential, can't go without!

Thanks for the update!

Thanks @void - this is one of my favorite apps :)

One question / bug report. Are you aware of an issue where accounts configured in the Events "Author Blacklist" section still get notifications? There are a small handful of names that I have configured there which for some reason I still keep getting notifications for.

Ok, I will take a look, eventually :-)

Since a few hours today, no more new notifications... anything the matter with the Steemwatchbot?

-edit- never mind: I just got a new notification, seems to be working again :)

Hello void!
I really like the Project, but poorly the homepage got an "502 Bad Gateway" nearly everytime. Sometimes it opens and 10 seconds later it's "offline".
I hope you can Fix it, because it's the best Telegram Wallet Watcher - or maybe the only Wallet Watcher for Telegram.
Have a great day ;-)

Hola @void, necesito ayuda de tu parte, soy estudiante desarrollandor junior, puedes darme un medio de comunicación más fluido como Telegram o Discord, igual sí es por correo.! Gracias! 😀😁

Coin Marketplace

STEEM 0.16
TRX 0.15
JST 0.030
BTC 59218.43
ETH 2534.91
USDT 1.00
SBD 2.44