Hivemind database dump available

in Witness Activities2 months ago (edited)

The Great Hivemind Reply Race is over!


It was more like a cross-country road rally than a horse race. Several competitors were spotted fixing their cars at the side of the road; some even changed their engines!

Eventually, after 2 gruelling weeks, the SteemWOW team crossed the finish line in first place 😄

Hivemind DB Download

If you're running an API node, you should really be running a local Hivemind database too. In association with @pennsif.witness, we're proud to offer a mirrored download of the DB to get you up to date:

Download mirror 1 (From @pennsif.witness)
Download mirror 2 (From @steemwow)

The total download is 199Gb, and unpacks to 396Gb. You can use a download accelerator tool such as axel to download from both mirrors at once to speed things up.

Alternatively, you can use this cunning trick to stream the download through lz4 as you grab it, meaning you don't need buffer space to store the unpacked archive:

wget -c -qO- | lz4 -d | tar x

You'll definitely want to run that in a screen session, and it doesn't show progress. I like to use watch du -hcs in the directory it's unpacking to in another session, so I can see it's working by watching the disk usage going up.

As well as the download, you'll need the docker-compose.yml and my-postgres.conf from Section 4 of @ety001's file sharing site.

Please support us with Witness Votes

Hosting costs for an archive this size can be quite significant. We're really happy to be able to support the Steem community, and would love your support in return, in the form of a witness vote.

Every Steemian has 30 witness votes that they can change at any time.

From the Steemit Wallet Witness list, all you need to do is tick both our arrows:

vote tag.png


Nice.. so where the files unzipping to?

They'll unpack to a directory named data. You should make a directory, hivemind, and in there place the data directory, docker-compose.yml, and my-postgres.conf

Hey.. not sure following your instruction on the hivemind
i have unzipped and have the data folder
but i don't see the data folder in the current hivemind directory
the data contains postgres data
shouldn't it be overwriting /var/lib/postgresql ?

Hi, sorry for the delay getting back to you. The data directory won't be there until you unpack the archive.

Hopefully this image makes it clearer:


Note that the network part of the docker config is specific to my setup; @ety001's docker-compose.yml is the best starting point

Thank you.. this is great!


Your post is manually rewarded by the
World of Xpilar Community Curation Trail


Upvoted! Thank you for supporting witness @jswit.

Hello @rexthetech! You are fabulous!

command: !thumbup is powered by witness @justyy and his contributions are:
More commands are coming!

This post has been featured in the latest edition of Steem News...

If you would like to support Steem News and Witness Weekly please consider voting for @pennsif.witness

Coin Marketplace

STEEM 0.20
TRX 0.06
JST 0.025
BTC 22671.21
ETH 1566.35
USDT 1.00
SBD 2.51